Software guide to ValidationTools

Complete: 5
Detailed Review status

Validation Tools


Simulation Validation Tools were made to validate a new CMSSW release. The tools do this by taking an analyzer, running it in a new release and then taking the resultant histograms and compares them to an old release.

The development of validation tools has been freezed. It was mostly used for simulating over all workflow. It developing and maintaining has stopped since 04/03/08.




The design is based on that the program has to follow the next conditions

  • Client-server multithread program.
  • Non-blocking for job submission and dataset monitoring.
  • Capable of running hundred of small jobs reliable for 24 hs.
  • Lightweight for minimal overhead in the execution of small jobs.
  • It should be possible run any tipe of analyzer.
  • Minimal dependencies for portability.
  • Mimicking CMS.ProdAgent and CRABServer for future implementations and integrations.


ValidationTools is based on three main components:

  • Subscriber: defines a subscription method to associate cfg file with datasets.
  • DBSMonitor: it is a multithread program that polls information about those dataset that are subscribed and submit a job to the JobMonitor in case the dataset logical file names exist.
  • JobMonitor: it is a multithread program for submitting and controlling jobs.


All the clients are located in the ValidationTools/bin directory. A brief description can be found of each of them can be found next.

  • dbsmonitor: client for interacting with DBSMonitor component.
  • dropbox: tool for creating and cleaning the drop box area.
  • jobmonitor: client for interacting with JobMonitor component.
  • rssparser: tool for requesting to DBS the dataset last updated LFNs.
  • subscriber: tool for create or adding subscriptions to ValidationTools.

Doxygen code documentation

Directions For Running Simulation Validation Tools


Here you will find instructions on how to use them. This instructions are based on a propose directory structure that can be created at lxplus cluster at cern. This simplify the our task for explaining the installation process. The given structure is NOT mandatory and you can change at any time.

Checkout CMSSW working area.

  • Create a central work directory and inside another for the releases.
      ~> mkdir work
      ~> mkdir public_html/validation
      ~> cd work
      work> mkdir CMSSW
      work> cd CMSSW
      CMSSW> scramv1 project CMSSW CMSSW_X_X_X
      CMSSW> cd CMSSW_X_X_X/src/
      src> project CMSSW
      src> cvs co -r CMSSW_X_X_X Configuration/PyReleaseValidation/data/cmsDriver_commands.txt
      CMSSW> cd ~/work

Note: the information about the different relval primary dataset are contain in cmsDriver_commands.txt.

Checkout and initial configuration

  • Check out Validation Tools
      work> cvs co -r V00-03-02 UserCode/ksmith/ValidationTools
      work> mv UserCode/ksmith/ValidationTools .
      work> rm -rf UserCode
  • Run the init.csh file from ValidationTools directory!!!
      work> cd ValidationTools
      ValidationTools> source init.csh
  • Compile the code. Any problem in the compilation is very likely to be related to the location of your ROOT library. Edit the variable ROOTSYS in the file with the right location. By default a location is given lxplus ar CERN.
      ValidationTools> make
  • Change Configuration File to Proper Settings
      ValidationTools> vi src/python/ (any text editor)

      ## set work areas for different cmssw distributions 
      variables['WorkAreas'] = '/afs/"userid"/work/CMSSW'
      ## dropbox and web location
      variables['DropBox'] = '/afs/"userid"/private/DropBox'
      variables['WebLocation'] = '/afs/"userid"/public_html/validation'
  • Create the "DropBox"
      ValidationTools> dropbox --create

Testing connection with DBS

  • This test polls DBS to locate LFNs for RelValTTbar primary dataset for a given release by default at CERN site.

      ValidationTools > rssparser --dataset=RelValTTbar --release=CMSSW_1_8_0 --lastupdate

      Channel title   : RelValTTbar | CMSSW_1_8_0
              lanuage : en-us
              pubdate : (2007, 9, 3, 17, 13, 20, 0, 246, 0)

      Looking for logical file names...

      replace PoolSource.fileNames = {

Subscribe packages to each primary dataset

  • This step is automatic in case of the relval as it is shown below. It subscribes EDMtoMEConverter to all the relval primary dataset.
      ValidationTools> subscriber --relval=CMSSW_X_X_X --print

Note: It is possible to subscribe any other package beyond the relval samples.

Running the main two components

  • Start DBS and Job Monitors
      ValidationTools> python src/python/ &
     [1] #jobA
      ValidationTools> python src/python/ &
     [1] #jobB
  • Testing DBSMonitor and JobMonitor
      ValidationTools > dbsmonitor --status

      There are not listenings in DBSMonitor

      ValidationTools > jobmonitor --status

      There are not monitoring jobs in JobMonitor

Running a validation

* Use dbsmonitor client to declare the release to be listen to (publication stage is off)

      ValidationTools> dbsmonitor --release=CMSSW_X_X_X --trigger=+HLT --nopostprocess

* Check the status of the listenings

      ValidationTools> dbsmonitor --status

      Status for all the listenings in DBSMonitor
      Release              Dataset                                  Site                 Trigger              Status                        
      CMSSW_X_X_X          RelValSinglePiMinusPt1000                LastUpdate           NoUpdate                      
      CMSSW_X_X_X          RelValTTbarLowLumiPileup                 LastUpdate           NoUpdate                      
      CMSSW_X_X_X          RelValSingleElectronE1000                LastUpdate           NoUpdate                      
      CMSSW_X_X_X          RelValMinbias                            LastUpdate           NoUpdate                      

Killing DBS and Job Monitor

  • Before you leave the lxplus node, kill both DBS and Job monitors
      ValidationTools> kill #jobA
      ValidationTools> kill #jobB

Review status

Reviewer/Editor and Date (copy from screen) Comments
Main.VictorBazterra - 03 Apr 2008 Last updates before killing the project.
Main.VictorBazterra - 15 Mar 2008 New instructions for installing ValidationTools.
Main.VictorBazterra - 04 Feb 2008 Adding some information about design.
Main.KennethSmith - 31 Jan 2008 Wiki creation.

Responsible: Main.VictorBazterra
Last reviewed by: Main.VictorBazterra - 04 Feb 2008

Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r5 - 2008-04-03 - VictorBazterra
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback