eticsRegression application

eticsRegression.py is a python application which performs regression tests for the Etics client and serves as a job monitoring tool for Etics.

It looks for an already installed Etics client under $ETICS_HOME, otherwise, if the variable is not set, it downloads and installs a local copy of the Etics client and reads build configurations to be run from a configuration file.

Then it submits the builds to the etics server and polls their status, performing a sleep for a configurable amount of seconds between each submit and poll operation. Upon job completion it retrieves the build report from which it extracts relevant parameters (such as total number of modules, number of successful modules and build duration) to be compared with the expected values written in the configuration file. The build reports are saved under the directory 'reports'.

A final report (index.html) is produced where each job is marked as either successful or failure according to the comparison between the actual values of the parameters and the expected ones. The report is sent via email to a specified contact. The headers of the report are clickable. Clicking on a header column, the rows of the report are sorted alphabetically by that column, either in increasing or decreasing order.

The application is endowed with a feature (triggered by the -f option) which enables to stop the application after that the jobs have been submitted and subsequently restore it from the same point where it was interrupted without any new submissions taking place. Already submitted jobs are reconstructed from a previously generated file (jobs.txt) containing a representation of each submitted job. When the application is restarted it polls the status of the previously submitted jobs.

Another option is -q , (--query) which allows to query for submitted jobs status after having called eticsRegressions without the -q option. The -q option is harmless and can be invoked any number of times. A report is sent via email to a specified contact. The report will contain the current status of each job as final result and the duration will be the time elapsed since job creation.

The option -n (--nocompare) allows to change the format of the final report. If this option is specified the expected values are not saved in the report and the comparison with the obtained values is not done. The report will contain the current status of each job as final result.

The option -r (--reduced) generates a reduced report without the columns "Project" and "ProjectConfig" appearing in the report table.

The module ReportDelivery.py contains a minimal API that allows to create locally the final report index.html (as if option -q is specified) and to send the report via email.

Configuration file

The configuration file is named eticsRegressionConfig.py and is looked for in the same directory as eticsRegression.py.

It contains a list of labels, each one containing a structure with all the different parameters which describe one or more job instances: project, prjconfig,module,configuration,platforms,expsuccess,exptotal,expduration,checkoutopts,submitopts,requirements. The parameters expsuccess,exptotal,expduration represent the expected values to which the actual job values will be compared.

Example of a configuration file:

#!/usr/bin/python

confs = {
    "org.etics1": {
                'PROJECT': 'org.etics',
                'PRJCONFIG': 'etics-dev',
                'CONFIGS': {
                    'org.etics.nmi.scripts': 'etics-nmi-scripts_R_1_6_8_1'
                },
                'PLATFORMS': [
                    'sl5_x86_64_gcc412EPEL'
                ],
                "EXPSUCCESS" : "1",
                "EXPTOTAL" : "1",
                "EXPDURATION" : "00:00:12",
                "CHECKOUTOPTS": "--project-config etics-dev",
                "SUBMITOPTS": "",
                "REQUIREMENTS": ""

    },
    "org.etics2": {
                'PROJECT': 'org.etics',
                'PRJCONFIG': 'etics_R_3_6_8_1',
                'CONFIGS': {
                    'org.etics.build-system.client-py': 'etics-build-system-client-py_R_1_6_3_1'
                },
                'PLATFORMS': [
                    'sl5_x86_64_gcc412EPEL'
                ],
                "EXPSUCCESS" : "8",
                "EXPTOTAL" : "8",
                "EXPDURATION" : "00:04:15",
                "CHECKOUTOPTS": "--project-config etics_R_3_6_8_1",
                "SUBMITOPTS": "",
                "REQUIREMENTS": ""
    }
}

Invocation

The application can be invoked in the following way:

python eticsRegression.py [options]

where the possible options are the following:

-c, --certificate cert         Specify certificate file

-k, --key key                   Specify private key

-f, --filename filename     Specify a file containing already submitted jobs (default specified value should be jobs.txt)

-q, --query filename        Query submitted jobs status (default specified value should be jobs.txt). Implies -n

-n, --nocompare              Don't perform comparison between expected values and obtained actual values

-r, --reduced                    Reduced number of columns in the report (implies -n) 

Note: with the current version of the etics client (1.6.2-1), the certificate and the private key files have to be specified in the configuration file ${HOME}/.etics.conf or ${ETICS_HOME}/etc/etics.conf

-- FabioCapannini - 03-Feb-2012

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r6 - 2012-05-04 - FabioCapanniniExternal
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback