Error Stream Processing

Error stream events are saved as *.dat files; they have not gone through any processing. There is now a script available to streamline the processing of error stream events. It converts the *.dat files to *.root files, then runs HLT and RECO on them. Finally, it puts the RECO events through the Exotica Hotline, which picks out unusual events with new physics signatures. The goal of this page is to explain the usage of this script.

Preliminary Instructions

cd ~/scratch0
cmsrel CMSSW_4_x_x
cd CMSSW_4_x_x/src
cp /afs/* .

This will copy into your directory the 4 files necessary in order to process error stream events. They are:

  • ErrorStreamProc.csh : the actual processing script
  • DriverInfo.txt : information for setting up CMSSW online releases while offline
  • HLTInfo.txt : information on which CMSSW release to use for HLT for each run
  • HLTFinder.csh : a script to gather the HLT CMSSW release information

Before running ErrorStreamProc.csh:

  • Check that your temporary space (/tmp/<user>/) contains no directories named ErrorRun_* .
  • Check that you have adequate free space in scratch: some online releases take ~250 MB to set up.
  • Check that the parameter Exotica_Version in ErrorStreamProc.csh is up to date
  • Make sure that DriverInfo.txt is updated:
  • Make sure that HLTInfo.txt is updated:
    • If the runs that you wish to process are within the range of runs listed in HLTInfo.txt, then you are fine. If not, then update HLTInfo.txt using HLTFinder.csh, which works by finding non-error stream files from the same run and determining which HLT release was used to process them.
    • Check to see if non-error stream events exist for the run(s) in questions by hunting for them in the directory L1Accept:
       rfdir /castor/ 
       rfdir /castor/ 
      If not, you will have to deduce the HLT release from runs before and after your desired runs.
    • To find the HLT release for a single run: set first_run and last_run in HLTfinder.csh both equal to the run in question
       source HLTfinder.csh 
    • To find the HLT release for a range of runs:
      • "Brute force" method: edit first_run and last_run in HLTFinder.csh to cover the whole range of runs for which the HLT release is unknown.
      • Efficient method (recommended): set first_run and last_run in HLTFinder.csh both equal to the same run: one in the middle of the range of runs to check. As the HLT release of more runs becomes known, continue to check midway between known runs, attempting to find the "borders" between different HLT releases.


It is necessary that DriverInfo.txt and HLTInfo.txt be in the directory in which you run the script.

The parameters that you may wish to edit in ErrorStreamProc.csh are at the beginning of the script. If you do not wish to make any changes, the script should run smoothly with the default values.

MAINDIR and USER are set based on the directory in which the script is run. If this is not what you wish, hardwrite these variables as desired. In this case, be sure to copy DriverInfo.txt and HLTInfo.txt into the directory you use as MAINDIR.

The script ErrorStreamProc.csh takes 5 parameters as input.

  • Input 1: 0 to process files, 1 to list runs
  • Input 2: 0 to process files, 1 to list files within runs
  • Input 3: 0 to use on multiple runs and 1 to use on a single run
  • Input 4: first run number (or only run, if using on a single run)
  • Input 5: last run number (irrelevant if using on a single run), or 0 to process all runs after the first run

(for all of the following, "a" and "b" each represent 6-digit run numbers)

To find a list of all runs in the error directory:

source ErrorStreamProc.csh 1 0 0 0 0

or all runs in the error directory between a and b (inclusive):

source ErrorStreamProc.csh 1 0 0 a b

To find a list of error stream files in run a:

source ErrorStreamProc.csh 0 1 1 a 0
or between runs a and b (inclusive):
source ErrorStreamProc.csh 0 1 0 a b

To run the error stream analyzer on all files in run a:

source ErrorStreamProc.csh 0 0 1 a 0
or on all files in all error stream runs between a and b (inclusive):
source ErrorStreamProc.csh 0 0 0 a b

To put the standard output into a text file, use the following formula:

( <cmd> /dev/tty ) > & <outputname>.txt

For anything involving processing rather than simply listing files/runs, the user will be prompted to enter the desired CMSSW release and global tag to use for RECO. Enter "&" as part of a response in order to invalidate that response. (I am currently using CMSSW_4_2_1 and GR_P_V22A::All). I suggest recording which release and global tag are used, as this will not be recorded in the output text file.

After the script has run, the RECO files and the output of the Exotica Hotline will be in MAINDIR.

You may view any of these ROOT files in an event display:

cmsShow <file_name>

Additional Information

Edited versions of ErrorStreamProc.csh

Two slightly modified versions of ErrorStreamProc.csh are available.

  • ErrorStreamProc_filter.csh: implement HBHENoiseFilter during RECO
  • ErrorStreamProc_hadd.csh: runs HLT multiple files with one input file at a time, then uses hadd to combine the output files before running RECO. Created to fix segmentation violation that was occurring in HLT from using multiple input files, even from the same run.

ErrorStreamProc_filterhadd.csh, which implements both, is coming soon.

Results thus far

I have processed the 45 error stream runs between run 173216 and run 176698 (inclusive).

No events have yet passed Exotica.

Very few runs gave errors:

  • 8 gave segmentation violations: "%MSG-e FatalSystemSignal: TSGFromL2Muon:hltL3TrajSeedIOHit" ; all disappeared when using hadd script (runs 173389, 173406, 175976, 176023, 176169, 176207, 176470, 176698)

  • 3 gave "Incompatible L1 and HLT menus", but this was apparently due to a change in menus (runs 173216-173218)

  • 3 gave "ProductNotFound" errors in RECO, returned RECO files with blank events (runs 174764, 174771, 175132)

My results (RECO files, Exotica files, STDOUT files) are in /castor/

Some pictures of typical events follow.

Known Errors

  • The lumisection information reads LS 1 for every event, even those whose timestamps prove otherwise.

  • The timestamp often reads Jan 1, 1970. This is always the case for the first event of each run, but sometimes true for other events as well.

  • RECO_173663.png:
Topic attachments
I Attachment History Action Size Date Who Comment
JPEGjpg RECO_160359.jpg r1 manage 212.9 K 2012-03-20 - 04:17 SylviaLewin  
PNGtiff RECO_173566_326.tiff r1 manage 172.3 K 2012-03-20 - 04:04 SylviaLewin Checking first box
PNGpng RECO_173663.png r1 manage 189.8 K 2012-03-20 - 04:21 SylviaLewin  
PNGtiff RECO_173663.tiff r1 manage 204.2 K 2012-03-20 - 04:04 SylviaLewin  
Edit | Attach | Watch | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r11 - 2012-03-20 - SylviaLewin
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback