Trigger partition testbed

The trigger partition testbed is a testing utility for testing the Atlas HLT in a partition. It is primarily intended for nightly tests, whose results can be found at http://atlas-project-trigger-release-validation.web.cern.ch/atlas-project-trigger-release-validation/www/parttesttbed/

The overview there shows all the testcases run during the last run, together with results from several of the most important programs. An orange background indicate there were one or more warnings in the logfiles, and red indicates there were errors.

Reconstructing testcases

For each testcase, the name of the testcase links to a directory with all the information from running that particular testcase. It contains all the logs generated by the partition (not just the ones linked to in the overview) and all of the material needed to reproduce the testcase. Below is an overview of the most important of these

generatePartition.tar.gz contains the folder from which the testcase is run, which in turn contains the partition XML file, the python options file used to generate it, as well as several post processing and utility scripts used by various testcases. The content of this directory reflect that of the directory just after the generation of the OKS database, eg. any specified pregenerate commands and the partition maker have already ran.

cmdlog contains a transcript of all the commands executed in the generatePartition directory for this testcase. These are useful when understanding what a testcase does, and for repeating it by hand.

testcase.out and testcase.err are the logfiles produced by the above commands, and contain at the minimum the output of the runner for this particular testcase.

Data generation testcases

Beyond the files mentioned above, output data comparing testcases also produce tar.gz files with the data files produced. reference.tar.gz contains the data file generated using athenaHLT, which is used as a reference. testData.tar.gz contains all the output files produced by the partition. This is one file per stream tag in use in the testcase.

Structure of a testcase

Testcases are defined in python files located at python/tests. All of the files in this directory are automatically imported on starting the tester, and add one or more testcases to the testcaselist located in the dict testcaseList.testcases. The name of the testcase is determined by the key in this map, and the actual contents is a dict describing the actions necessary for the testcase

A testcase has the following configurable properties:

  • template
  • template_arguments
  • HLTPath
  • runner_test
  • runner_testarguments
  • pregenerate
  • preprocess
  • postprocess

Template and template arguments

The template and template_arguments properties configure the generation of the python file used to generate the partition. Template is the name of the template (located under scripts/htltemplates) that contains the testcases specific options such as the configuration for the hlt implementation. The template_arguments option is passed to this template, and is meant primarily to allow testcases that differ very little in their general configuration (such as the memory measurement testcases) to use the same testcase templates.

HLTPath

The HLTPath option can be set either to None, or to an HLT path descriptor. This is used to setup an environment for using a specific HLT version. The HLT path descriptor contains four fields, each containing a root path for an HLT directory (eg. the section of the path to the HLT without the InstallArea/*/data part). These four parts are: - workarea - patch - release - nightly The first three correspond to their hlt-implementation counterparts. Nightly is used for running with nightly builds and is inserted in the path between patch and workarea, and used as the final bit of workarea when passed to the python testcase generation.

runner test and arguments

The nightly tester uses the runner application to run tests on the partitions it generates. The runner can execute a variety of testcases, and they each take different arguments. The runner_test option specifies which runner test should be used to run this particular testcase, and the runner_testarguments is a string that is passed to the runner with the -A option, as arguments for this testcase.

pregenerate, pre and postprocessing

Some testcases need extra actions to be taken at various steps of the partition generation and running process. For this purpose, there are three points to inject extra commands. When the tester processes a testcase, it executes the steps in the following order: pregenerate - generate oks database (partition maker) - preprocess - run partition (runner) - postprocess. The options pregenerate, preprocess and postprocess specify the command to be executed during that step. They can be any bash command, but for more complex actions it might be preferable to write a (small) script and use the option value to call this. In case any of the above options is empty the respective step is simply skipped.

Writing testcase templates

All templates used by the trigger partition testbed are stringtemplate3 templates (for documentation of this see: https://theantlrguy.atlassian.net/wiki/display/ST/StringTemplate+3+Documentation). The python configuration file uses scripts/hlttemplate/hltopt.st as it's root template. This file contains the basic options such as input file, log directory and options that are the same for all test cases. This file then includes a testcase specific template.

This template can contain it's own options, and take arguments form the python testcase configuration (see above) provided through the tc_args object. For example, if the python testcase configuration's template_arguments has an entry output_dir, then this is available as tc_args.output_dir in the template file. Furthermore, if the HLTPath entry in the python testcase configuration is not None, including HLTPath will make 3 variables available (workarea, patcharea and release) that can be used to configure the HLT paths in hlt implementations that need them.

Writing testcase pre- and post-processing commands

Pre and post-processing commands should be bash shell commands. When executing these commands, this software appends to the end one extra argument, the path to the log area for this specific testcase. The pre- and or post-processing scripts can create and modify files in these directories, and if they end in one of .out, .err, .csv, .root or .tar.gz, they will be included in the log directory displayed on the website.


Instructions for setup

Intended use case

The trigger partition testbed is intended as a tool to do nightly testing on a single machine. The results are produced as a website, which is published to a configurable directory. It is possible to run the testbed manually by calling the runnightlytest.sh script with the scripts directory being the current working directory. Test runs with only a single testcase can be achieved by adding --testtype to the tdaq_python line in this script.

Required software

The testbed software requires trunk versions of the following software to be in the PATH environment variable:

The current setup.sh script included with the software will correctly pick up this software if it is under $HOME/installed

Setup

When setting up the testbed in a specific location a number of steps need to be taken. At a minimum, the path settings specifying the various working directories (see also description below) need to be edited to refer to locations with the required properties.

The testbed installation on pcatr21 uses an extra script that initializes the TDAQ environment. This script is called by the following cronjob at 1:00 am each night: 0 1 * * * pcatr21.cern.ch /afs/cern.ch/user/t/trparttb/docron.sh

Configurable settings

When setting up a new installation of the testbed software, the following parameters are the most important. They specify the working directories used by the various functions of the testbed, and the name of the partitions generated by it.

testareaRoot: This variable specifies the directory used to create the partition configuration files and related activities, it will end up containing the generatePartition folders for each of the testcases. For correct functioning this should be a publicly readable folder that is available on all machines used by a partition running on the host (on the tdaq test cluster, this means that it should be accessible for other nodes in the cluster, as by default a shared initial partition is used).

prepublisharea: This variable specifies the location used for building up the website prior to publishing it. It should be read-write for the user running the testbed software, but is otherwise unconstrained.

publishingPath: This is the path where the final website is published. Should be write and delete access for the user running the testbed software.

logsarea: This path is used as initial storage for the log files of the partition. It should be available on all partition nodes.

output_dir: This path is used for output files from partitions that produce them. This path should be large enough to deal with O(100) MB of data files.

partitionName: This variable specifies the partition name. In order to ensure correct operation, this name should be unique to the installation of the testbed. Failure to do so can cause issues with other partitions with the same name already running, resulting in failed testcases.

The next group of settings control the HLT settings used by the partition.

HLTBase: This is the leading path to both the AtlasHLT and AtlasP1HLT locations. It is combined with the version number (see below) to generate paths to both an AtlasHLT and AtlasP1HLT installation. This version controls the Atlas*HLT location used by testcases that fall back on the default. Testcase can however override this, for this see the individual testcases.

HLTBaseVersion: This is the 4 element version number of AtlasP1HLT to use. The testbed automatically generates the corresponding AtlasHLT version number from this. This number is used in conjunction with HLTBase to generate paths to the Atlas*HLT packages to use.

workarea: Base path (without InstallArea/ segment) to an offline workarea to use in creation and running of the partition. This directory should be accessible for all nodes in the cluster used to run the partition (see also above)

nightly: Base path to a nightly version to use in the partitions.

offlineVersion: Branch/version string to be used for setting up an offline environment with asetup, eg: AtlasP1HLT,20.2.1.7. For correct functioning of testcases that rely on generated database configurations, this should match the version specified with the HLTBase, HLTBaseVersion and nightly settings above.

The rest of the settings control various aspects of some of the testcases and the partition environment they run in.

tdaq_repository: This is a path to an installed directory containing tdaq software that should be used instead of the default when running a partition. This directory should be accessible for all nodes running the partition.

coralAuthFile: This should point to a file used as an authentication file by the coral proxies in the HLTfromDBNewCoralProxy.

testdata: This file is used as datafile by the partition and for generating reference data. It should be accesible by all nodes in the partition.

num_events: The number of events to use for the output comparing testcases.

excludeWarnings: This contains a list of regular expressions that, when they match a warning line in a log file, cause that line to not be included in the warning counts displayed on the website. This can be used to mask warnings that occur often but are known to be harmless.

-- DavidSebastiaanVenhoek - 2015-08-18

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2015-08-20 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback