Pilot Testing

This document describes how the PanDA Pilot is being tested before a release.

Main Pilot 2 GitHub repository

The main Pilot 2 GitHub repository can be found here. Stable developments are merged with the Next branch, and at the end of a development cycle, this branch is merged with the Master branch.

Personal developments

A development cycle should begin with personal developments in the developer's personal GitHub repository. That repository should be refreshed/synced with the main repository Next branch to facilitate later merging. For info regarding GitHub usage, see the "Information for PanDA Pilot developers" document. Before any pull requests are done, the developer should make sure that the new code works as expected by relevant unit testing, flake8 validation and/or by running the pilot against a job that will test the new functionality. Standard test jobs are described below. It is recommended to run both unit tests and flake8 prior to making a pull request, as these automatically trigger flake8 using the Travis CI and will fail the pull request if it does not pass the tests.

Test scripts

Up to date test scripts are kept by the different developers (each have their own favourite test scripts) so contact them for the latest recommendations. It is common to use existing scripts as templates for new test job scripts by cloning job parameters from current jobs. Typically, the following parameters should be updated in the new test job script:

• job.AtlasRelease (e.g. 'Atlas-20.7.7')
• job.homepackage (e.g. 'AtlasDerivation/20.7.7.2')
• job.transformation (e.g. 'Reco_tf.py')
• job.prodDBlock (e.g. 'mc15_13TeV:mc15_13TeV.411258.aMcAtNloPy8EvtGen_MEN30NLO_A14N23LO_ttbar_noShWe_LEPrb_smt.merge.AOD.e7418_a766_a821_r7676_tid18581201_00')
• job.cmtConfig (e.g. 'x86_64-slc6-gcc49-opt')
• job.coreCount (only set this for AthenaMP, e.g. '8')
• input file lists with file names, guid, file sizes, checksums, scopes, datasets

Common test job scripts include Reco (using global and permanently available replicas) and Sim. These scripts are typically executed with (e.g.) 'python testReco.py queuename cloud'.

Pilot versions

A developer may put a development version in a www-reachable area where it can be downloaded by a wrapper. A bespoke pilot request should be filled in and Peter.Love(at)cern.ch should be informed. He will then start a special pilot factory that will send that pilot version to the queue specified in the bespoke request page.

Official pilot versions

The official bleeding edge pilot version is called pilot2-dev2.tar.gz and is normally used for prodsourcelabel=ptest test jobs (maintained by paul.nilsson(at)cern.ch). The more stable development version is called pilot2-dev.tar.gz and is used for HammerCloud testing. Version history is kept in a Google Doc.

HammerCloud testing

When a development version is deemed stable, it is deployed for HammerCloud testing. HammerCloud generates test jobs on both user and production queues using prodsourcelabel=rc_test2, which are picked up by wrappers that use the stable development version of the pilot. It is recommended to test a new pilot version for at least one day before a release. (The existing HammerCloud templates are currently (July 2019) being updated with more queues).

Current HammerCloud tests can be followed from the PanDA Monitor.

Event service testing

Currently (as of July 2019), HammerCloud does not offer event service testing which instead has to be done manually.

• Pilot2Test

-- PaulNilsson - 2019-07-26