Large Scale Testbeds are necessary to perform acceptance, interoperability and scalability tests.
These tests require a different implementation with respect to that of EMI Integration testing testbed, since the main objective is to stress the service under test reproducing conditions as similar as possible to real production environment. Approaching the goal above implementing a stable and monitored testbed infrastructure permanently replicating a production environment is both very difficult to achieve in terms of contributors availability and also very inefficient, being at the same time, expensive to maintain and never fit for any test in particular. Hence the proposal of two larger scale testing scenarios:

1. Pre-Deployment across voluntary sites of software product at Release Candidate stage, with standard deployment modality and service usage in production environment.
   • Pre-Requirements:
     1. Standard deployment material: Documentation, Release Notes, Repository for installation
     2. Generic benchmarks, feedback forms
   • Expected Results:
     1. The aim is to get early feedback on RCx to improve components and fine-tune EMI procedures. Feedback is expected on:
        • Release Notes and Documentation
        • Installation / deployment
        • Production environment usage product functionality
          
2. Large Scale specific product testing, according to software component test plans or for multiple components scalability/performance testing at a larger scale. The testbed definition and implementation is achieved through a "Demand and Supply" approach, meaning:
   • DEMAND: A request of test describing the testing scenario (resources needed, services involved, number of instances, number of users involved etc.etc.).
   • SUPPLY: A community of partners (NGI, EGI, PRACE, EMI internal partners, EMI user community partner or whoever interested) available to participate to specific tests campaigns with X effort, Y resources for Z time to test P1, P2,.., PN product. Declaring availability just for specific products testing will enhance the chance of having motivated contributors, which is a key aspect for the success of the model.

The role of EMI quality assurance WP to implement this model is to provide operational resources (coordination, communication channels, expertise in testbed setup, general purpose utilities (fake certificates or fake CA generation utility)) to match DEMAND and SUPPLY.

Agreement with EMI partners:

• EGI and NGIs The informal and operational agreement with EGI project is summarized by the following EGI statement of intent

Site/Institute	Type of Contribution	Details	Contact
INFN-Catania	HW/sys admin effort	2 machines User Interface	emidio.giorgio@ctNOSPAMPLEASE.infn.it
INFN-CNAF	HW/sys admin effort	2 machines WMS /LB	andrea.cristofori@cnafNOSPAMPLEASE.infn.it
D4Science-II	HW/sys admin effort	Will install some services; can provide around 4-5 machine for the tests DPM, CREAM, Site BDII, WN, UI	andrea.manzi@cernNOSPAMPLEASE.ch
EGEE-SEE-CERT	HW/sys admin effort	EMI Products: CREAM, site-bdii,dpm head,dpm pool,glite wn,glite ui,stand alone torque node and a stand alone mysql node. Number of Boxes: 13 boxes.	Dimosthenes Fioretos dfiore@nocNOSPAMPLEASE.edunet.gr, Institution: National and Kapodestrian University of Athens / GRNET
e-NMR Project	Testing Effort	Requires e-NMR VO to be configured on resources	marco.verlato@pdNOSPAMPLEASE.infn.it

Key Performance Indicators for EMI Large Scale Acceptance Testbed is:

CODE	KPI	Description	Method to Measure	Estimated Targets
KSA2.3	SA2 Distributed Testbed Size	Number of CPUs available for distributed testing through collaborations with external providers (NGIs sites, commercial providers, other projects, etc)	Participating sites monitoring tools	Year 1: 50 CPUs Year 2: 200 CPUs Year 3: 500 CPUs

To match the task KPI of 50 CPU for the first year, we keep track of the testbeds provided per campaign (x instances for y time).

Large Scale testbed success depends on the availability of volunteer contributors. Contributors can be either EMI partners or EMI product users ( EGI, NGIs, potential customers, Virtual Organizations etc.) wishing to provide some HW resources and/or collaborative effort to test EMI products they are interested in. Being aware of the fact that interest or effort/resources contribution is always limited in quantity and time, as a contributor you can declare your availability just for a subset of products for a limited number of campaigns per year. Even 2 weeks and few CPUs per year can be very helpful for EMI Testbed.
To become a EMI Large Scale testbed contributor contact EMI Testbed Staff providing the following information:

1. Your Name, Role and Institution
2. Which EMI products testing activity you may be interested in and why.
3. Number of boxes ready to deploy EMI components you may provide and for how long
4. Number of testing campaign (installation kept for y days during campaign) you may contribute to.
5. System administration effort you may provide during the campaigns or availability to grant access to external administrators + minimal support from local system administrators.

1. Default Pre-Deployment scenario:
   1. Deployment modality: please deploy the service according to your site typical deployment, this will provide us with use cases representative of real production environment. You are also requested to enable emi-testers.emi-eu.eu VO on your instance ( configuration details-> section 3.5 )
   2. Usage / Testing: this scenario will take advantage of service exposition to normal production usage.
   3. Feedback Provision: please fill the following Large Scale Testbed Pre-Deployment Template and mail it back to contact EMI Testbed Staff
2. Demand/Supply scenario*:
   1. Deployment modality: Depending on the type of deployment request, you will be given specific testing case instruction which may sometime involve particular deployment scenarios, even though the fous is always on keeping close to real production needs. You are also requested to enable emi-testers.emi-eu.eu VO on your instance ( configuration details-> section 3.5 )
   2. Usage / Testing: Usage and Testing instruction will be provided in the testbed use case setup definition.
   3. Feedback Provision: Report about the test can be sent to contact EMI Testbed Staff.

WORK IN PROGRESS:

Component Name	Requests for Large Scale Deployment (Y/N)	Flavour	Deployed at Site	N° Instances	Notes
AMGA v.2.1.2
APEL parsers v.0.0.0
APEL publisher v.3.2.7
ARC CE v.1.0.0	Y	with gridftp interface and ARIS	-	-
ARC Clients v.1.0.0
ARC Core v.1.0.0
ARC gridftp server v.1.0.0
ARC InfoSys v.1.0.0
ARGUS v.1.3.0	Y	Site + Top + VO level	-	1 Site level Argus per Site, Target >= 2 Sites
ARGUS-EES v.0.1.0
BLAH v.1.16
CEMon v.1.13
core BDII v.0.0.0
CREAM v.1.13	Y	LFS + Torque	-	Target >= 2 Sites
dCache v.1.9.12	Y	-	-	-
Delegation Java v.2.0.0
DGAS-sensors v.4.0.1
DPM v.1.8.1	Y	-	-	-
emi-sherpa: an example component entry
emi-ui v.0.0.0
emi-wn v.0.0.0	Y	glexec enabled	-	Target >= 2 Sites
FTS v.2.2.6	Y	-	-	-
GFAL/lcg_util v.1.11.18	Y
gLExec v.0.8.3	Y	-	-	Target >= 2 Sites
glite-gsoap/gss v.3.0.1
gLite-MPI v.1.0.0
glite-proxyrenewal v.1.3.17
glite-yaim-core v.5.0.0
gridsite v.1.7.12
Hydra v.1.0.1
L&B v.3.0.5	Y	-	-	-
LCAS v.1.4.0
lcg-info-clients v1.0.0
LCMAPS v.1.5.0
LCMAPS-plugins-c-pep v.1.1.0
LFC v.1.8.1
RAL-SAGA-SD v.1.0.0
site BDII v.1.0.0
SLCS v.0.0.0
StoRM SE v.1.7.0
top BDII v.1.0.0
Trustmanager v.3.0.3
UNICORE AIP v.2.0.0
UNICORE Client v.6.4.0
UNICORE Gateway v.6.4.0
UNICORE HILA v.2.2
UNICORE Registry v.6.4.0
UNICORE sec libs v.2.0.0
UNICORE Services Environment v.6.4.0
UNICORE TSI v.6.4.0
UNICORE WS v.6.4.0
UNICORE XACML PDP v.2.0.0
UNICORE XNJS v.1.4.0
UNICORE XUUDB v.1.3.2
UVOS v.1.4.1
VOMS v.2.0.0	Y	-	INFN_CNAF	1 + Replica	Just the CNAF instance will be used, but we include it here, since we want Large Scale users to test the clients.
VOMS-Admin 2.6.0	N	-	-	-
WMS v.3.3.0	Y	-	-	-

To deploy testers.emi-eu.eu on your service you need:

1. To enable the VO add it in the VOS variable of the yaim site-info.def file of your service (ex. VOS="alice atlas cms dteam infngrid lhcb ops testers.eu-emi.eu")
2. Create in the directory with site-info.def a directory vo.d (if not existing) and save there the following file (including replica): testers.eu-emi.eu
3. Check you have INFN CA installed with command $> rpm -qa |grep INFN .... output should be somethig like "ca_INFN-CA-2006-1.37-1"
4. For pool accounts add in users.conf the following sample: users_testers.conf
5. For groups enabling add in groups.conf the lines in the attached file: groups.conf
6. Add new certificates (master and replica) to /etc/grid-security/vomsdir/ directory:
   • NEW CERTIFICATES FROM Jul 20 2011 NEW !! Install EMI Testing Infrastructure Utility repo and run yum install emi-tb-vomscerts

For the manual configuration of vomses parameters ( ex. .voms/vomses , .glite/vomses on the UI) use the following parameters:

"testers.eu-emi.eu" "emitestbed07.cnaf.infn.it" "15002" "/C=IT/O=INFN/OU=Host/L=CNAF/CN=emitestbed07.cnaf.infn.it" "testers.eu-emi.eu"
"testers.eu-emi.eu" "emitestbed01.cnaf.infn.it" "15002" "/C=IT/O=INFN/OU=Host/L=CNAF/CN=emitestbed01.cnaf.infn.it" "testers.eu-emi.eu"

The second line is for the replica In the near future, the VO will be deployed with EMI releases, and above info and files integrated in yaim profiles/examples and the certificate deployed through an rpm file

For ARC users the VO configuration is described at http://www.nordugrid.org/documents/voms-notes.html . For vomses file use the input as above.

For Unicore users: UNICORE does not support VOMS, so no configuration can be provided. UVOS, UNICORE VO solution, is to be phased out during the course of EMI, so we didn't deploy it in the testbed. It is planned to support the SAML based VOMS later on, but that is not available yet. If you require access to the testbed, you'll have to open a GGUS tkt send me your certificate and I will register it.

• Testers VO subscription: The default VO for testing in EMI is testers.eu-emi.eu. So, for authentication on services supporting voms certificates, users need to subscribe to the VO for accessing the testbed. To enable the VO on Product Teams instances or User Interface follow instructions in section 3.3 in this page.
  

1. General Configuration Issues:
   1. EMI Repository Link
   2. General EMI-1 Release and per product Release Notes documentation.
   3. To let EMI Testbed team test your correct installation/configuration and volunteer users test service functionality, some services need to be configured to interact with each other. In particular:
      1. To publish your services you can use at first the EMI Infrastructure information system bdii for configuration , and then the volunteer site deploying site and core bdii will be configured to see your resource. A gLite information system (BDII) was setup ( certtbpre-bdii.cern.ch working as site and top bdii) for gLite and dCache resources.
         1. Top-level BDII . Contact it from UI by command: ldapsearch -x -H ldap://certtbpre-bdii.cern.ch:2170 -b mds-vo-name=local,o=grid (see also details )
         2. Site-level BDII . Contact it from UI by command: ldapsearch -x -H ldap://certtbpre-bdii.cern.ch:2170 -b mds-vo-name=cert-tbpre-cern,o=grid (see also details )
      2. You are also requested to enable emi-testers.emi-eu.eu VO on your instance ( -> section 3.3 )
      3. Depending on the service you are configuring you would to set your configuration values to point to other sites deployed services. For configuration values please contact EMI Testbed Team.

EMI testbed unit provides coordination effort to match testbed requirements for specific testing needs with infrastructural resources provided on a voluntary basis from a group of providers internal or external to EMI project. The setup of a new large scale testbed infrastructure starts from the accurate definition of testing case, needed resources, testing-period etc. First step in requiring a testbed is then the submission of a testbed request according to the following procedure:

• Testbed Requests: WHO
  • Product Team members needing to test their product under specific conditions
  • EMI Release Manager, Quality Assurance leaders, Technical Area leaders
  • Users Community interested in specific testing campaign of Release Candidate version products
• Testbed Requests: HOW
  • Open GGUS ticket by Mail explaining your testing and configuration needs reporting answer to following template questions. Please cat & paste the Large Scale Request Mail Template providing requested information.
• Testbed Requests: WHEN
  • Request tickets will be evaluated and handled eventually leading to the setup of a specific description paragraph in the "Active Testbed Description" section below.
  • Notice that the testbed setup process can take a few weeks depending on the complexity of testbed request and availability of resource providers. Therefore apply for a testbed as soon as you realize you may need it.

Testbed Applicant: Andrea Ceccanti (Argus PT)
Tracking: GGUS tkt
Status: Waiting for Argus Team to define policies to be tested
Logbook: ArgusTestBed0012011

Testbed Applicant: Shiraz Memon (Emir PT)
Tracking: GGUS tkt
Status: Waiting for Services to be Ready for Deployment
Logbook: EmirTestBed0012012

Testbed Applicant: EGI-EMI
Tracking: SAVANNAH EMI-SA2 TASK
Status: Waiting for Services to be Ready for Deployment
Logbook: NagiosServerEMITestbed0022012

Testbed applicant: Marco Verlato (WeNMR grid manager)
Tracking: https://ggus.eu/ws/ticket_info.php?ticket=89884
Status: Waiting for reply. The application segfaults.

Testbed x, request description, implementation, contributors, Notes, period....etc.etc.

Testbed x, request description, implementation, contributors, Notes, period....etc.etc.