Difference: TestFileDB (14 vs. 15)

Revision 152018-01-15 - BenjaminCouturier

Line: 1 to 1
 
META TOPICPARENT name="TestDrivenDevelopment"

TestFileDB, a database of test files for use in automated software tests

Line: 16 to 16
 

Where is it?

TestFileDB lives under PRConfig, the DBASE package designed to hold options files and tests for the LHCb Performance and Regressions testing system.

Changed:
<
<
>
>
 
Line: 41 to 41
 

- How can I browse what's there?

Either:

Changed:
<
<
  1. Read through the python module in SVN: https://svnweb.cern.ch/trac/lhcb/browser/DBASE/trunk/PRConfig/python/PRConfig/TestFileDB.py
>
>
  1. Read through the python module in SVN: https://gitlab.cern.ch/lhcb-datapkg/PRConfig/blob/master/PRConfig/python/PRConfig/TestFileDB.py
 
  1. Browse through the entries in TestFileDB as shown above
  2. Browse through using the browsing script $PRCONFIGROOT/scripts/BrowseTestFiles.py The inbuilt browser works like browsing a directory structure where files are separated into "directories" based on their metadata.
  3. Browse manually with the low-level helper commands. This is the basic part that is wrapped in the BrowseTestFiles.py script. %SYNTAX{ syntax="python"}%
Line: 61 to 61
 TestFileDB.test_file_db["2012_raw_default"].run(configurable=Moore()) %ENDSYNTAX%
  • This procedure will add the test files through IOHelper, and configure the configurable Moore with the database tags stored in the test file metadata, and simulation True/False flag.
Changed:
<
<
>
>
 

- How can I use it in my project?

Line: 80 to 80
 

- Obtaining a local copy for editing

This obviously depends where (in which project) you're planning on adding the tests. The example below is for Moore, replace with your version.%SYNTAX{ syntax="sh"}%

Changed:
<
<
SetupProject Moore v21r0 --build-env getpack PRConfig head
>
>
lb-dev Moore/latest git lb-clone-pkg PRConfig
 make
Deleted:
<
<
SetupProject Moore v21r0 #next line not always required, depends on the intricacies of cmt cd PRConfig/cmt source setup.csh
  %ENDSYNTAX%

Edit, commit with release notes.

Line: 98 to 94
 
  • You can access the files without needing a grid certificate.
  • The files listed are PFNs.
  • You have added as much information as possible to the metadata of the files.
Changed:
<
<
>
>
  Remember that this is a full python module, so any valid python to create the entries in the database is allowed, even something procedural.
Line: 112 to 108
 

- Automatic scripts for adding to the database:

If you already have a python file which adds test files through IOHelper or EventSelector there is a script which will copy this to the TestFileDB and ask you for the metadata.

Changed:
<
<
>
>
  To use this:
  1. Prepare local copy as above.
Line: 123 to 119
 
    • ./AddToTestFilesDB.py --help
    • ./AddToTestFilesDB.py someoptsfile_or_a_PFN
Changed:
<
<
You can then check the files are readable for you with the script TestReadable.py: https://svnweb.cern.ch/trac/lhcb/browser/DBASE/trunk/PRConfig/scripts/TestReadable.py
>
>
You can then check the files are readable for you with the script TestReadable.py: https://gitlab.cern.ch/lhcb-datapkg/PRConfig/blob/master/scripts/TestReadable.py
 

- Automatic scripts for adding files to CERN-SWTEST/EOS:

ACTION RESTRICTED TO RELEASE MANAGERS

For files actually used in our automated nightly tests, especially centrally produced files we need to make sure they are not tied to the validity of a specific user account or production obsoletion cycle. So, please upload your files to the CERN-SWTEST element using the script provided:

Changed:
<
<
>
>
  To use this:
  1. Prepare local copy as above.
Line: 152 to 148
 
    • ../scripts/MigrateToSWTEST.py --help
    • ../scripts/MigrateToSWTEST.py
Changed:
<
<
You can then check the files are readable for you with the script TestReadable.py: https://svnweb.cern.ch/trac/lhcb/browser/DBASE/trunk/PRConfig/scripts/TestReadable.py
>
>
You can then check the files are readable for you with the script TestReadable.py: https://gitlab.cern.ch/lhcb-datapkg/PRConfig/blob/master/scripts/TestReadable.py
 

Copying files directly to CERN-SWTEST

You can copy the files directly to CERN-SWTEST, by using the following commands: %SYNTAX{ syntax="sh"}%

Changed:
<
<
lb-run LHCbDirac bash
>
>
lb-run -c best LHCbDirac bash
 lhcb-proxy-init dirac-dms-add-file /lhcb/swtest//myfile myfile CERN-SWTEST %ENDSYNTAX%
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback