This pages contains instructions for running the upgrade Global Trigger emulator. This is currently under development.

Overview:

Setting up the CMSSW Area:

Setting up and configuring your CMSSW area should be done following the instructions found on other TWikis:

L1Trigger Emulator Development Instructions (This gives the latest and greatest development release)

Development Release

Please use CMSS_8_0_9 as your base release

Currently the most up to data CMSSW development release for uGT emulation and unpacking can be found on gitHub branch::

l1t-integration-CMSSW_8_0_9 (Caution: This release is under active development)

This branch must be merged in the standard fashion. (see L1Trigger Emulator Development Instructions )

IMPORTANT: Use of this release depend on the use of an external library (utm). Currently, the version of the utm library that comes with CMSSW_8_0_9 may not be up to date. You may need to install your own updated version. Instructions for installing this external library can be found on the following TWiki:

UTM Library Instructions

XML Configuration of L1 Menu

The L1 Menu is configured via an XML file. This file (currently) resides in the following area:

L1Trigger /L1TGlobal/data/Luminosity/startup/

If a new L1 Menu is created via the L1MenuEditor, the XML should be placed in this area. In addition, the configuration (see below) should point to the new file. Look for a line such as:

process.TriggerMenu.L1TriggerMenuFile = cms.string('L1Menu_Collisions2016_v1a.xml') (put in your file name)

This is part of more detailed configurations.

Configuration Scripts

There are a number of different configuration scripts that are being used to create test vector files for hardware testing. These continue to be developed as the code changes, etc. The instructions will be updated as changes occur. Don't assume what worked last week will work this week!

Basic Modules:

• uGT Emulation is controlled through the L1TGlobalProducer. In most scripts below it is configured with simGtStage2Digis_cfi.
• L1 Menu is controlled by loading the configuration 'L1Trigger.L1TGlobal.TriggerMenu_cff'
• Test Vectors for hardware testing are created by the EDAnalyzer "l1t::GtRecordDump". In most scripts it is called dumpGTRecord
• Input to the uGT is done through different modules as described below.

Recipe for Generating Test Vectors from MC Generator Objects

It is possible to run the uGT emulation directly on Monte Carlo generator-level objects. This is convenient for making tests of the emulator without having to rely on emulation of the rest of the L1 system. The recipe detailed below generates a test vector to be used for comparison with the uGT firmware. TTbar samples are typically used as input to generate the test vectors.

cmsrel CMSSW_11_0_1
cd CMSSW_11_0_1/src
cmsenv
git-cms-init
git-cms-addpkg L1Trigger/L1TGlobal

A typical workflow is to make a test vector using a candidate L1 menu. The candidate menus are usually first made available as XML files. To use the XML version of the menu, it should be put in directory L1Trigger/L1TGlobal/data/Luminosity/startup. Create the directory if it doesn't exist. Below is an example using an xml menu from the GlobalTriggerAvailableMenus twiki page.

mkdir -p L1Trigger/L1TGlobal/data/Luminosity/startup
cp L1Menu_Collisions2018_v2_1_0-d2.xml L1Trigger/L1TGlobal/data/Luminosity/startup
cp L1Trigger/L1TGlobal/test/runGlobalFakeInputProducer.py .
scram b -j7

Edit runGlobalFakeInputProducer.py so it looks for the new L1 menu. Also make sure the Global Tag is appropriate for the release and update the input filelist if necessary. The number of events contained in the filelist should be at least 3564 (One orbit). Within the configuration of this script you can specify the output name of the test vector file. Currently, this is called "TestVector_XXX.txt" where XXX corresponds to the job number specified in the cmsRun command.

voms-proxy-init -voms cms -valid 168:0 # authenticate grid certificate
cmsRun runGlobalFakeInputProducer.py >& run_NewMenu.log &

Parameters:

• nevents = total number of events to run over (this should be set to 3564. One orbit.)
• job = job ID value if running multiple jobs (simple case job=0)
• njobs = total number of jobs (simple case njobs=1)
• rootout = boolean flag for dumping out emulated events to a root file
• dump = boolean for dumping out full python script file (large). Set equal to false for most cases.

This script will use an EDProducer "l1t:GenToInputProducer" that will use generator-level objects to populate the input to the uGT.

Other Controls within the Script:

Module: mcL1GTinput: Within this script one can set the maximum number of candidates that are reported to the uGT and Et thresholds for making it into the list of candidates. Module: dumpGTRecord This control the dumping of the test vector file and also can dump IO information for each event to the log file.

Open Issues with Emulation Package Development:

1. PR #370 is still not complete. Contains changes for new etSums.
2. PR 414 to improve handling of the saturation of Et values.
3. Add new TOWERCOUNT sum to emulator and menu parsing.
4. Unpackers intial instantiation of the DIGI's based on AMC/board number. Could be improved for robustness.
5. Need to check whether 020 changes for Prescales and Vetos have finally gone in. (Unsure of PR number...orginially it was in a separate branch dedicated to O2O issues. Mike M. knows about this.)
6. Need to change mapping on L1 Object index in the L1GlobalObjectMap that communicates information from the uGT to the HLT. Darren had a working first pass at this, but it probably needs more testing and needs to be pushed into the integration branch. Coordination of this with Mike and Vladimir is extremely improtant since it touches HLT code. Darren had some code in <a data-saferedirecturl="https://www.google.com/url?hl=en&q=https://github.com/puigh/cmssw/tree/mydevel-L1TGlobal-MultiBxIndex&source=gmail&ust=1471889795200000&usg=AFQjCNGEgIxt-S5IxU29HvfD66itbIs3MA" href="https://github.com/puigh/cmssw/tree/mydevel-L1TGlobal-MultiBxIndex" target="_blank">https://github.com/puigh/cmssw/tree/mydevel-L1TGlobal-MultiBxIndex</a>

Description of Code Packages and Modules

L1Trigger /L1TGlobal Package:

This is the primary package for the L1TGlobal Emulator. It has a dependency on the UTM Library, which is an external library that handles the creation and parsing of the the L1 Menu xml file. This package uses that library for parsing the xml formated menu. There are corresponding header files for these classes in the /interface area.

src/AlgorithmEvaluation.cc: Class for handling the evaluation of algorithms (triggers) in the menu. These are for the most part, simple boolean operations on Conditions, which handle the more complex calculations and selection cuts.
src/CaloCondition.cc: Class for performing the condition evaluation involving calorimeter objects (other than sums), this includes jets, EG, Taus. This class performs the selections on the lists of these calorimeters objects (eg. Et Threshold, isolation, etc.)
src/CaloTemplate.cc: This is a template class that contains the description of cuts, objects, etc. that are used for a Caloritmer Condition. It inherits from the GlobalCondition class. It contains methods for copying and printing the condition.
src/ConditionEvaluation.cc: Generic class that holds simple methods for condition evaluation and printing.
src/CorrCondition.cc: Class that performs all corrleation conditions between two L1 objects. This includes delta Eta, delta Phi, delta R and invariant mass.
src/CorrelationTemplate.cc: This is a template class that contains the description of cuts, objects, etc. that are used for a CorrelationCondition. It inherits from the GlobalCondition class. It contains methods for copying and printing the condition.
src/EnergySumCondition.cc: Class for performing the condition evaluation involving energy sums including minbias bits. This class performs the selections on the lists of these objects (eg. Threshold)
src/EnergySumTemplate.cc: This is a template class that contains the description of cuts, objects, etc. that are used for a Energy Sum Condition. It inherits from the GlobalCondition class. It contains methods for copying and printing the condition.
src/ExternalCondition.cc: A class that contains the code for making a selection on external conditions. These are very simple selections (check a specific bit in the external condition input).
src/ExternalTemplate.cc: This is a template class that contains the description of cuts, objects, etc. that are used for a ExternalCondition. It inherits from the GlobalCondition class. It contains methods for copying and printing the condition.
src/GlobalAlgorithm.cc: A class for the algorithms (triggers). Has some utility methods to print results, etc. Makes use of GlobalLogicParser for boolean evaluation of the algorithm's conditions.
src/GlobalBoard.cc: Main driver class for the emulator. Carries out evaluation of the conditions, algorithms (triggers), prescaling, vetoing, and construction of the emulator output.
src/GlobalCondition.cc: Generic parent class for all conditions.
src/GlobalDefinitions.cc: Class that defines some global quantities (types, etc) for emulation.
src/GlobalParamsHelper.cc: Helper class for accessing the setup parameters for the emulation.
src/GlobalScales.cc: A class for holding all the information regarding scales and LUTs as defined in the menu xml file. These are filled by the TriggerMenuParser.cc in the plugins area. These scales/LUTs can be made available through out the code package but they are primarily used in the correlation conditions, which are the only type of conditions that rely on LUTs. Other types of conditions cut on hardware quantities and do not rely on the scale definitions.
src/L1TGlobalUtil.cc A Utility for code for dumping and accessing the results of the emulator. This is used by some of the plugin modules.
src/L1GlobalUtilHelper.cc A helper for the Utility class.
src/MuCondition.cc Class to evaluate conditions involving muons. It applies all the selection cuts (Et, isolation, quality, etc.)
src/MuonTemplate.cc Template class for holding the requirements for a muon condition.
src/TriggerMenu.cc: Holds all the information about the Trigger L1 Menu (conditions, algorithms).

plugin/L1TGlobalProducer.cc/.h: Producer for the uGT emulation. Instantiates GlobalBoard class and runs the emulation including the parsing of the L1 menu.
plugin/StableParameterTrivialProducer.cc: Module to put configuration parameters into the EventSetup to configure the emulator.
plugin/TriggerMenuParser.cc/.h: Performs the parsing of the L1 Menu. Creates all the conditions, algorithms, scales and LUTs.
plugin/L1TUtmTriggerMenuESProducer.cc: Puts the L1 Menu in the EventSetup from an xml file on disk. Note: the menu may already be loaded into the EventSetup via a O2O process. This is a means to overide.
plugin/L1TUtmTriggerMenuDumper.cc: Module to dump library from EventSetup. (Was used mostly for preliminary testing...not particularly a good dump of the menu.)
plugin/L1TExtCondProducer.cc: Module to produce fake external conditions for testing purposes.
plugin/L1TExtCondLegacyToStage2.cc: Module to convert external conditions from lagacy system (pre 2016 data) into new external format for testing.
plugin/GtRecordDump.cc: Module to perform a detailed dump of the inputs and outputs to the uGT emulation. Can be configured via python config to turn certain pieces on and off. It also is used to provide test vector files for testing of uGT firmware.
plugin/GenToInputProducer.cc: Module to produce fake input to the uGT. This module uses generator level quantities to produce the input. There is no attempt to model resolutions, eff. etc. So this should NOT be used for any rate or efficiency calculation. It is simply used to produce non-trivial inputs to the emulator to use for testing of firmware.
plugin/L1TGlobalAnalyzer.cc: Module to make simple plots of inputs and outputs of the emulator.
plugin/...other files... : A few other older modules exist in this area that were used for testing but have not been used in a long time. They may no longer be functional.

python/GlobalParmeters_cff.py: Config for setting upt the input parameters to the emulator.
python/TriggerMenu_cff.py: Config for using a disk file (xml) for the L1 Menu configuration.
python/simGtExtFakeProd_cfi.py: Config for creating fake external condition bits as input to the emulator.
python/simGtStage2Digis_cfi.py: Config for setting up the uGT emulator.
python/l1GtExtCondLegacyToStage2_cfi.py: Config to convert legacy external conditions (pre 2016 data) to external conditions for the uGT.

test/runGlobalFakeInputProducer.py: A config that can be used to produce the test vector files for comparison to firmware. This uses many of the modules defined in the plugin area and many of the config files used in the python area. This is a good file to look at to understand how to configure the emulator and auxilary packages.
test/l1tGlobalEmulation_FromDigi.py: Config to run emulation from existing digis (This config needs updating and testing.)
test/l1tGlobalEmulation_FromRAW.py: Config to run emulation from existing digis (This config needs updating and testing.)

CondFormats /L1TObjects Package:

Area for the definition of the configuration classes of the emulator.

src/L1TGlobalParameters.cc: Defines the configuration parameters for the EventSetup.
src/L1TGlobalPrescalesVetos.cc: Defines the prescales and vetos for the EventSetup.
src/L1Utm*.cc: Series of classes that define the L1 Menu (in Utm format) for the EventSetup. Any changes in these classes must be mirrrored by changes in the external Utm library. Great care in needed here.

DataFormats /L1TGlobal Package:

Area for the definition of the output classes of the emulator. Note: Inputs are defined by similar areas of the other L1 subsystems. There are corresponding header files in the /interface area for these classes.

src/GlobalAlgBlk.cc: Class that defines the uGT Digi that holds the algorithm results. Also has methods for accessing the data.
src/GlobalExtBlk.cc: Class that defines the uGT Digi that holds the external input conditions.
src/GlobalLogicParser.cc: Class for evaluating the boolean logic for conditions. (?? Place here because used by HLT??)
src/GlobalObject.cc: Used for sending the trigger information to the HLT. Holds the objects that fire particular L1 triggers.
src/GlobalObjectMap.cc: Used for sending trigger information to the HLT.
src/GlobalObjectMapRecord.cc: Used for sending trigger information to the HLT.

/ EventFilter/ L1TRawToDigi/ src/implementations_stage2/

Area for the packing and unpacking of RAW data. The uGT follows the structure setup by the Calorimeter. Note: The inputs to the uGT are packed and unpacked with the same code that is used for the Calorimeter and Muon subsystems. Only the uGT output has its own packers/unpackers.

GTSetup.cc: Setup for the uGT unpackers and packers.
GTTokens.cc/h: Part of the setup.
GTCollection.cc/h: Defining the collections of objects that are part of the data stream.
GlobalAlgBlkPacker.cc: Packs the GlobalAlgBlk Digi into RAW data record.
GlobalAlgBlkUnpacker.cc: Fill the GlobalAlgBlk Digi from the RAW data.
GlobalExtBlkPacker.cc: Packs the GlobalExtBlk Digi into RAW data record.
GlobalExtBlkUnpacker.cc: Fill the GlobalExtBlk Digi from the RAW data.

Creating Digis from Streamer or RAW Data (Needs updating below this point)

To create Digis from Streamer Files captured with miniDAQ:

cmsRun EventFilter /L1TRawToDigi/utils/unpackData-CaloStage2.py inputFiles=file:run261151_ls0002_streamA_StorageManager.dat streamer=True skipEvents=0 maxEvents=-1 edm=True

where you substitute the streamer file name. This will produce a file "l1tCalo_2016_EDM.root" which contains both the RAW data and the Digis of the Calorimeter Layer 2 output, the uGT input, and the uGT output.

To create Digis from RAW data files:

The same script can be used, except set "streamer = false" and make the inputFiles point to an EDM root file.

Running the Emulator on RAW or Streamer Files

Running the emulator on the Digis created above and performing simple comparisons of hardware and emulator can be accomplished with the following:

cmsRun L1Trigger /L1TGlobal/test/l1tGlobalEmulation_FromRAW.py inputFiles=file:streamerFile.dat streamer=True skipEvents=0 maxEvents=-1 newXML=True

Comments:

1. This script accepts L1 Menus from XML files and can accept with the old grammar or new grammar style XML
2. The L1 Menu XML file must be located in L1Trigger /L1TGlobal/data/Luminosity/startup/
3. The file name must be set in the python config.
4. The config will execute an EDAnalyzer (L1TGlobalAnalyzer) that will create a root file with histograms that show distributions of uGT input and output and compare hardware to emulation.
5. The config will execute an EDAnalyzer (GtRecordDump) that will perform a text dump of the uGT inputs and outputs. The output can either be set to the hardware quantities or to the emulated trigger bits. Various types of dumps can be turned on or off in the config file.
6. The python config also sets up the emulator, which allows for the selection of prescales and masks through a comma separated file (csv) located in the same area as the menu XML files.
7. The same script can be used, except set "streamer = false" and make the inputFiles point to an EDM root file.

Running the Emulator on Digi Files

Running the emulator on the Digis created above and performing simple comparisons of hardware and emulator can be accomplished with the following:

cmsRun L1Trigger /L1TGlobal/test/l1tGlobalEmulation_FromDigi.py inputFiles=file:l1tCalo_2016_EDM.root skipEvents=0 maxEvents=-1 newXML=True

Comments:

1. This script accepts L1 Menus from XML files and can accept with the old grammar or new grammar style XML
2. The L1 Menu XML file must be located in L1Trigger /L1TGlobal/data/Luminosity/startup/
3. The file name must be set in the python config.
4. The config will execute an EDAnalyzer (L1TGlobalAnalyzer) that will create a root file with histograms that show distributions of uGT input and output and compare hardware to emulation.
5. The config will execute an EDAnalyzer (GtRecordDump) that will perform a text dump of the uGT inputs and outputs. The output can either be set to the hardware quantities or to the emulated trigger bits. Various types of dumps can be turned on or off in the config file.
6. The python config also sets up the emulator, which allows for the selection of prescales and masks through a comma separated file (csv) located in the same area as the menu XML files.

Running Emulator on Output of Stage 2 Calorimeter Objects

Programs for Dumping Test Vector Files:

There are some auxilary programs to dump the contents of the test vector files. (Currently Broken)

Basic Usage:

readTestVector -i myTestVectors.txt -m myL1Menu.xml (Note: must give full path name to XML file)

This will give a summary of the triggers that fired in this test vector file.

Advanced Usage:

readTestVector -i myTestVectors.txt -m myL1Menu.xml -d 1

This gives a detailed dump (input and output) of each event. This can be a lot of information so sending the output into a text file is a good idea.

More:

More features, such as creating histograms of quantities, can be used. For more details

readTestVector --help

Running over Slice Test Data

Basic Requirements:

This requires using the l1t-devel branch of CMSSW_7_5_0_pre4 (or later) for the latest L1 Menu aglorithms and xsd: (See Seting Up CMSSW Area at top of page)

Currently Slice Test data is captured at P5 as a set of text files that represent the RX (receiving) and TX (transmitting) buffers on the MP7 boards. An example of these files can be found on this TWiki:

  https://twiki.cern.ch/twiki/bin/viewauth/CMS/L1CaloUpgradeJan2015

See, for example, ABCD_660.tgz, which is an attachment to the TWiki above.

cmsRun Command w/ configuration:

A configuration file to run over these text files, pack them into a FEDRawDataCollection, unpack them into BXVectors and emulate the uGT is the following:

cmsRun $CMSSW_BASE/src/L1Trigger/L1TGlobal/test/unpackBuffers-CaloStage2_uGTEmulation.py dmOffset=zz gtOffset=xx gtLatency=yy dump=False debug=False maxEvents=150

This must be executed from the directory holding the RX/TX data directories. For example, you should run in "Data/A1" if you unzip/untar the tgz file given above.. The parameters, zz, xx, and yy depend on the capturing of the RX and TX buffers. For example, in the files in ABCD_660.tgz, dmOffset=60, gtOffset=15 gtLatency=47. However, for the late May 2015 Slice testing dmOffset=28, gtOffset=58, gtLatency=44. These parameters can be determined as follows: "gtOffset" is the Frame Number for the first valid piece of data in the RX buffer. "gtOffset" + "gtLatency" is the Frame number for the first piece of valid data in the TX buffer. Note: in this script we are only concerned with the output of the DeMux so the "dmOffset" points to the first valid data in the DeMux TX buffer (i.e. we are running this script with dmLatency=0). Note: we should probably fix this so it is more general.

The CMSSW job given emulates the uGT starting from both the DeMux output data collections and the uGT input collections (calorimeter only at the momemt). It also analyzes the data with an EDMAnalyzer module (L1TGlobalAnalyzer.cc). This module makes plots of DeMux output, uGT input, uGT output, uGT output emulated with DeMux output, uGT output emulated with uGT input, and comparisons of DeMux output versus uGT input. These plots are saved in a ROOT file.

-- BrianWiner - 30 Sep 2014 (Updated Apr 8, 2015)