Service Certification Checklist

This page provides a checklist for every service on what to check in any case when certifying a patch for a specific service. These checks include basic regression tests. When a patch is certified it must include a comment describing the outcome of the basic tests. Note that the tests can be launched standalone (usually on a UI). The test results must then be attached or pasted to the patch.

Some quick links to testing resources: available tests

General proxy and server/service test scenarios

Information System tests

AMGA

AMGA server

• AMGA server tests
• Meta-script to launch the tests:

  AMGA-certtest.sh

AMGA API

• AMGA API tests

Computing Element (CE)

• The tests have to be done using the org.glite.voms-test VO (>15 char, DNS style VO)
• Job submission to the CE via the WMS, e.g. you can use the WN tests.
• Direct job submission with globus-job-run.

Data Management Tools (DM)

• The tests have to be done using the org.glite.voms-test VO (>15 char, DNS style VO). When using this VO the cross-se tests fails on Castor; comment the line CASTORHOST in the DM-certconfig file and repeat the test using the dteam VO and enabling only DM_CROSS_SE tests and using CASTORHOST. The lcg-sd command fails on dCache due to bug #43002.
• Using the meta-script

  DM-certtest.sh

  lcg_utils, gfal, and cross-se tests will be automatically run. Instructions and meta-script are in org.glite.testsuites.ctb/DM/
• Regression tests listed in the bug lists: lcg_util-standard.txt, lcg_util-classicSE.txt. Look for the regression tests here. Use your personal certificate and the VO org.glite.voms-test to run the regression tests and be sure to have more than 2GB of space available on your HD.

File Transfer Service (FTS)

• To configure FTS/FTA a painless installation recipe is provided here.
• The tests have to be done using the org.glite.voms-test VO (>15 char, DNS style VO)
• Using the meta-script

  FTS-certtest.sh

  FTS-basic, FTS-services, FTS-channels and FTS-submission (info here) will be run.
• Regression tests: use the fts.txt testlist file to run available regression tests.

Grid File Access Library (GFAL)

• The tests have to be done using the org.glite.voms-test VO (>15 char, DNS style VO)
• Use the meta-script

  GFAL-certtest.sh

Logging and Bookkeeping service (LB)

• Use the meta-script

  LB-certtest.sh

  4 sets of tests (lb-test-server-remote.sh, lb-test-logger-remote.sh, lb-test-normal-event-delivery-remote.sh and lb-test-job-registration.sh) will be run. lb test scripts

LCG File Catalog (LFC)

• The tests have to be done using the org.glite.voms-test VO (>15 char, DNS style VO)
• Use the meta-script

  LFC-certtest.sh

• LFC tests. The tests defined in test-cli.lst, test-api-python.lst and test-sequence.lst will be automatically run by the meta-script.

Site Central Authorization Service (SCAS)

Storage Element (SE)

• The tests have to be done using the org.glite.voms-test VO (>15 char, DNS style VO)

dCache

• Use the DM tests specifying, in the DM-certconfig file the tests LCG_UTILS and GFAL and the SE to test as both FIRSTSE and SECONDSE.
• S2 SRM tests, in particular the s2-srmv2.2-MoU script that launches 25 basic tests. The 25 tests must succeed on the ctb machines provided by Desy.

Disk Pool Manager (DPM)

• Use the DM tests specifying, in the DM-certconfig file the tests LCG_UTILS and GFAL and the SE to test as both FIRSTSE and SECONDSE.
• S2 SRM tests, in particular the s2-srmv2.2-MoU script that launches 25 basic tests.

User Interface (UI)

• The tests have to be done using the org.glite.voms-test VO (>15 char, DNS style VO)
• UI tests
• Tests must be repeated, when necessary, using the TAR UI (TarballCreation)

VOBOX

• VOBOX tests

Virtual Organization Membership Service (VOMS)

• VOMS tests. Use the voms-core and voms-admin tests.
• Proxy renewal test using the new VOMS
• In case of simultaneous update of client and server test all 4 possibilities: prod client - new server, new client - prod server, new client - new server

Worker Node (WN)

• The tests have to be done using the org.glite.voms-test VO (>15 char, DNS style VO)
• UI tests on WN to be launched on some UI. Alternatively the SAM CE tests offer a similar test coverage.
• Tests must be repeated, when necessary, using the TAR UI (TarballCreation)
• For glexec enabled WNs: glexec tests.

Workload Management System (WMS)

• The tests have to be done using the org.glite.voms-test VO (>15 char, DNS style VO)

WMProxy (WMS CLI and API)

• WMS CLI tests
• WMSProxy Python API

WMS Service

• WMS CLI tests
• Basic jobsubmission.
• Basic matchmaking.

YAIM

In order to certify YAIM the following tests have to be taken into account:

• Deployment tests: For each relevant node type affected by a YAIM upgrade (see the affected metapackages list in the corresponding patch), check the following deployment scenarios:
  • Clean installations
  • Upgrades from production or pps installations

It's also important to read the information about the metapackages to be restarted or reconfigured and run the suitable commands after an upgrade. This is not needed for a clean installation. After the configuration has finished, it's important to check whether the relevant daemons and processes are running. For this, you can check the service reference cards.

• Basic tests:
  • Check that the different command options are working properly, at least the most important ones like: -c, -v, -r, -d, -a
  • Check that all the variables to configure a service are actually required by YAIM (-v option). To know which are the necessary variables you can check the YAIM configuration variable wiki.
  • It's also important to check that new variables are properly documented in the YAIM configuration variable wiki, otherwise, this should be reported to the developer.
  • Depending on how critical an update is (i.e. it modifies critical configuration issues like grid-map file generation, user configuration, security issues like fetch-crl or lsc files, etc...), it's important to run more or less tests in the affected node types. If it's necessary to check thoroughly the new configuration, follow the guidelines in this wiki page for tests affecting the different node types.

• Specific tests:
  • UI/TAR UI: check that the grid environment can be sourced without any problem in other shells than bash. Specially zsh and (t)csh are also used and some users have reported problems in the past.
  • TAR UI/ TAR WN: for yaim core and client the UI and WN tarball must be tested: look TarballCreation for information on how to create a tarbal.
  • lcg CE: please check that the gridmap-file contains the list of DNs and VOMS FQANS. Test job submission for both simple and voms proxies.
  • cron jobs: If some cron job has been configured, it's interesting to check the node type some hours or days after the configuration to check that the cron jobs are running as expected. Specially fetch-crl or edg-mkgridmap. In the case of the lcg CE, it's also important to check that lcg-ce-mkgridmap cron job is running fine.
  • VO/VOMS: It's better to run your tests against one known VO and VOMS server from which you are able to get special roles like sgm or prd. You can use our VOMS server in the testbed for this purpose where we have test users. See https://lxbra2309.cern.ch:8443/vomses for the list of VOs and users supported by our VOMS server.
  • VO variables: Remember to test the definition of VO related variables both under site-info.def and under vo.d directory. It's also interesting to run tests for a DNS like VO name. You may find bugs in any of the mentioned scenarios so it's important to test both, including a DNS like VO name. Our VOMS server contains several VOs, one of them with a DNS like VO name.

• Bug fixes:
  • YAIM releases are normally associated with a list of bugs where the new features and the bug fixes introduced by a new release are described. Most of the certification job is based on verifying the bugs. Reading the bug and understanding the problem is very important to certify a YAIM release. Reproduce the problem to check whether it has been fixed and report on the steps carried out to verify the bug and the result of the test in the associated patch.
  • Regression tests should be written for each bug. They will be run in future releases to check whether a bug has been reintroduced or not. Please check the README file in the CVS regression test location to understand the framework to write regression tests.

-- AndreasUnterkircher - 06 Feb 2008