How to certify YAIM

In order to certify YAIM the following tests have to be taken into account:

• Deployment tests: For each relevant node type affected by a YAIM upgrade (see the affected metapackages list in the corresponding patch), check the following deployment scenarios:
  • Clean installations
  • Upgrades from production or pps installations

It's also important to read the information about the metapackages to be restarted or reconfigured and run the suitable commands after an upgrade. This is not needed for a clean installation. After the configuration has finished, it's important to check whether the relevant daemons and processes are running. For this, you can check the service reference cards.

• Basic tests:
  • Check that the different command options are working properly, at least the most important ones like: -c, -v, -r, -d, -a
  • Check that all the variables to configure a service are actually required by YAIM (-v option). To know which are the necessary variables you can check the YAIM configuration variable wiki.
  • It's also important to check that new variables are properly documented in the YAIM configuration variable wiki, otherwise, this should be reported to the developer.
  • Depending on how critical an update is (i.e. it modifies critical configuration issues like grid-map file generation, user configuration, security issues like fetch-crl or lsc files, etc...), it's important to run more or less tests in the affected node types. If it's necessary to check thoroughly the new configuration, follow the guidelines in this wiki page for tests affecting the different node types.

• Specific tests:
  • UI/TAR UI: check that the grid environment can be sourced without any problem in other shells than bash. Specially zsh and (t)csh are also used and some users have reported problems in the past.
  • TAR UI/ TAR WN: for yaim core and client the UI and WN tarball must be tested: look TarballCreation for information on how to create a tarbal.
  • lcg CE: please check that the gridmap-file contains the list of DNs and VOMS FQANS. Test job submission for both simple and voms proxies.
  • cron jobs: If some cron job has been configured, it's interesting to check the node type some hours or days after the configuration to check that the cron jobs are running as expected. Specially fetch-crl or edg-mkgridmap. In the case of the lcg CE, it's also important to check that lcg-ce-mkgridmap cron job is running fine.
  • VO/VOMS: It's better to run your tests against one known VO and VOMS server from which you are able to get special roles like sgm or prd. You can use our VOMS server in the testbed for this purpose where we have test users. See https://lxbra2309.cern.ch:8443/vomses for the list of VOs and users supported by our VOMS server.
  • VO variables: Remember to test the definition of VO related variables both under site-info.def and under vo.d directory. It's also interesting to run tests for a DNS like VO name. You may find bugs in any of the mentioned scenarios so it's important to test both, including a DNS like VO name. Our VOMS server contains several VOs, one of them with a DNS like VO name.

• Bug fixes:
  • YAIM releases are normally associated with a list of bugs where the new features and the bug fixes introduced by a new release are described. Most of the certification job is based on verifying the bugs. Reading the bug and understanding the problem is very important to certify a YAIM release. Reproduce the problem to check whether it has been fixed and report on the steps carried out to verify the bug and the result of the test in the associated patch.
  • Regression tests should be written for each bug. They will be run in future releases to check whether a bug has been reintroduced or not. Please check the README file in the CVS regression test location to understand the framework to write regression tests.

-- GianniPucciani - 19 Jan 2009