HLTNtupleProductionSTEAM < Sandbox

-- XuyangGao - 2016-01-12 How to produce HLT Ntuples

Contents:

L1 Skim
- Setup
- Generate the cmssw config file
Setup recipes
Local test
MultiCRAB3 configuration
Links

L1 Skim

Setup

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_25
cd CMSSW_8_0_25/src
cmsenv
rehash

# Standard packages
git cms-addpkg L1Trigger/L1TGlobal
git clone https://github.com/cms-data/L1Trigger-L1TGlobal.git L1Trigger/L1TGlobal/data

# PU filter
git remote add mia https://github.com/mtosi/cmssw.git
git fetch mia
git checkout mia/PUfilter80x -- PhysicsTools/Utilities

# Load the L1 menu, prescales and masks
cp /afs/cern.ch/user/n/ndaci/public/STEAM/L1Menus/L1Menu_Collisions2016_v9_m2.xml L1Trigger/L1TGlobal/data/Luminosity/startup/
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/UGT_BASE_RS_FINOR_MASK_v91.xml L1Trigger/L1TGlobal/data/Luminosity/startup/
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/UGT_BASE_RS_PRESCALES_v214.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# Overwrite one of the plugins
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/L1TGlobalPrescalesVetosESProducer.cc L1Trigger/L1TGlobal/plugins/

# Compile
scram b -j 8

Generate the cmssw config file

cmsDriver.py test -s L1REPACK:uGT --filein=/store/mc/RunIISummer16DR80/WJetsToLNu_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/GEN-SIM-RAW/FlatPU28to62HcalNZSRAW_80X_mcRun2_asymptotic_2016_TrancheIV_v6-v1/120000/441FBC1A-5BAF-E611-AFC0-0025904C6376.root --no_exec --data -n 100 --conditions=auto:run2_mc --era=Run2_2016 --customise=L1Trigger/Configuration/customiseUtils.L1TGlobalDigisSummary --customise=L1Trigger/Configuration/customiseUtils.L1TAddInfoOutput --customise_commands "process.load('L1Trigger.L1TGlobal.hackConditions_cff')\n process.L1TGlobalPrescalesVetos.PrescaleXMLFile = cms.string('UGT_BASE_RS_PRESCALES_v214.xml')\n process.L1TGlobalPrescalesVetos.FinOrMaskXMLFile = cms.string('UGT_BASE_RS_FINOR_MASK_v91.xml')\n process.l1tGlobalSummary.AlgInputTag = cms.InputTag('simGtStage2Digis')\n process.l1tGlobalSummary.DumpTrigResults = cms.bool(False)\n process.simGtStage2Digis.AlgorithmTriggersUnmasked = cms.bool(False)\n process.simGtStage2Digis.AlgorithmTriggersUnprescaled = cms.bool(False)\n process.simGtStage2Digis.PrescaleSet = cms.uint32(4)\n"

Warning: the parameter PrescaleSet sets the prescale column index to be used. See also section HLTNtupleProductionSTEAM#Add_custom_lines_for_L1_prescale

Setup recipes

WARNING: In case you need to develop new trigger algorithms, please refer to this twiki section from the official HLT SW twiki (maintained by STORM). In particular the CMSSW release, and any git command, should be used when setting up the CMSSW area.

Setup: 2016 recipe with menu v4.2 for MC

Menu: /frozen/2016/25ns15e33/v4.2/HLT

CMSSW area

Show the recipe

Hide the recipe


# Install cmssw area
cmsrel CMSSW_8_0_25
cd CMSSW_8_0_25/src
cmsenv
rehash

# Standard packages
git cms-addpkg HLTrigger/Configuration
git cms-addpkg GeneratorInterface/GenFilters
git cms-addpkg SimGeneral/MixingModule
git cms-addpkg L1Trigger/L1TGlobal

# External packages
git clone https://github.com/cms-data/L1Trigger-L1TGlobal.git L1Trigger/L1TGlobal/data
git clone git@github.com:cms-steam/RemovePileUpDominatedEvents.git RemovePileUpDominatedEvent

git cms-addpkg HLTrigger/HLTanalyzers
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# Add fix to the MixingModule
git cms-merge-topic -u ndaci:PR_steam_fix2_80X

# Load the L1 menu, prescales and masks
cp /afs/cern.ch/user/n/ndaci/public/STEAM/L1Menus/L1Menu_Collisions2016_v9_m2.xml L1Trigger/L1TGlobal/data/Luminosity/startup/
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/UGT_BASE_RS_FINOR_MASK_v91.xml L1Trigger/L1TGlobal/data/Luminosity/startup/
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/UGT_BASE_RS_PRESCALES_v214.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# Overwrite one of the plugins
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/L1TGlobalPrescalesVetosESProducer.cc L1Trigger/L1TGlobal/plugins/

# Compile
scram b -j 8

Create CMSSW config files: standard menu

1) Create CMSSW config files using the hltGetConfiguration command:

# Create a working directory
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file: Running L1 emulator uGT only + HLT SW #

hltGetConfiguration /frozen/2016/25ns15e33/v4.2/HLT --full --offline --prescale 1.25e34 --no-output --process TEST --l1-emulator 'uGT' --l1Xml L1Menu_Collisions2016_v9_m2.xml --mc --globaltag 80X_mcRun2_asymptotic_v14 --input /store/mc/RunIISummer16DR80/WJetsToLNu_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/GEN-SIM-RAW/FlatPU28to62HcalNZSRAW_80X_mcRun2_asymptotic_2016_TrancheIV_v6-v1/120000/441FBC1A-5BAF-E611-AFC0-0025904C6376.root --max-events 100 > hlt_L1v9m2_frozen_v4p2_MC_L1uGT_1p25e34.py

2) Edit the config files:

add the PU and GEN filters (see corresponding twiki sections) only in MC

comment out the following line:

 process.DQMOutput = cms.EndPath( process.dqmOutput )

add ntuplizer lines (see corresponding twiki section)
if you want to apply L1 PS, add custom lines at the end of the config: see corresponding twiki section

3) Run the config files:

voms-proxy-init --voms cms --valid 168:00
cmsRun hlt_L1v9m2_frozen_v4p2_MC_L1uGT_1p25e34.py

Setup: 2016 recipe with online menu v4.2 [special recipe for high PU data]

Menu: /online/collisions/2016/25ns15e33/v4.2/HLT

CMSSW area

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_19_patch2
cd CMSSW_8_0_19_patch2/src
cmsenv
rehash

git cms-addpkg L1Trigger/L1TGlobal
git cms-addpkg HLTrigger/Configuration
cp /afs/cern.ch/user/n/ndaci/public/STEAM/HighPU/customizeHLTforHighPU.py HLTrigger/Configuration/python/
git cms-checkdeps -A -a

### Please make sure that the above compiles.

# HLTNtuples tools
git cms-addpkg HLTrigger/HLTanalyzers
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# L1 menu 
cp /afs/cern.ch/user/n/ndaci/public/STEAM/L1Menus/L1Menu_Collisions2016_v9_m2.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# L1 prescales

### save the default file and copy a *single* file depending on your use case.
mv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv.save

### get 1.15e34 prescales csv file [Note: there are also other PS csv files for other PS columns in the same afs directory]
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v9_1p15e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv

# Compile
scram b -j 4
cmsenv
rehash

Create CMSSW config files: standard menu

1) Create CMSSW config files using the hltGetConfiguration command:

# Create a working directory
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file #

## Running only the HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v4.2/HLT --full --offline --unprescale --no-output --process TEST --l1Xml L1Menu_Collisions2016_v9_m2.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016G/HLTPhysics0/RAW/v1/000/279/024/00000/0436CCD4-9A65-E611-82FA-FA163EAAD2D1.root --max-events 100 > hlt_online_v4p2_Data.py

## Running L1 emulator + HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v4.2/HLT --full --offline --unprescale --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v9_m2.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016E/ParkingZeroBias0/RAW/v2/000/277/069/00000/02123456-084E-E611-A1A9-02163E012A56.root --max-events 100 > hlt_L1v9_online_v4p2_Data.py

2) Edit the config files:

add the PU filters as a fake trigger (see corresponding twiki section) only in MC

comment out the following line:

 process.DQMOutput = cms.EndPath( process.dqmOutput )

add ntuplizer lines (see corresponding twiki section)

**Important step for high PU data**: add the following lines at the end of the cmssw config file:

from HLTrigger.Configuration.customizeHLTforHighPU import customizeHLTforHighPU
process = customizeHLTforHighPU(process)

3) Run the config files:

voms-proxy-init --voms cms --valid 168:00

cmsRun hlt_GRun_Data.py
### cmsRun hlt_GRun_MC.py

Setup: 2016 recipe with online menu v4.2

New menu to be tested: /online/collisions/2016/25ns15e33/v4.2/HLT

CMSSW area

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_19_patch2
cd CMSSW_8_0_19_patch2/src
cmsenv
rehash

git cms-addpkg L1Trigger/L1TGlobal
git cms-addpkg HLTrigger/Configuration

### Please make sure that the above compiles.

# HLTNtuples tools
git cms-addpkg HLTrigger/HLTanalyzers
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# L1 menu 
cp /afs/cern.ch/user/n/ndaci/public/STEAM/L1Menus/L1Menu_Collisions2016_v9_m2.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# L1 prescales
### save the default file and copy a *single* file depending on your use case.
mv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv.save
### get 1.15e34 prescales csv file
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v9_1p15e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv

# Compile
scram b -j 4
cmsenv
rehash

Create CMSSW config files: standard menu

1) Create CMSSW config files using the hltGetConfiguration command:

# Create a working directory
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file #

## Running only the HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v4.2/HLT --full --offline --unprescale --no-output --process TEST --l1Xml L1Menu_Collisions2016_v9_m2.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016G/HLTPhysics0/RAW/v1/000/279/024/00000/0436CCD4-9A65-E611-82FA-FA163EAAD2D1.root --max-events 100 > hlt_online_v4p2_Data.py

## Running L1 emulator + HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v4.2/HLT --full --offline --unprescale --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v9_m2.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016E/ParkingZeroBias0/RAW/v2/000/277/069/00000/02123456-084E-E611-A1A9-02163E012A56.root --max-events 100 > hlt_L1v9_online_v4p2_Data.py

2) Edit the config files:

add the PU filters as a fake trigger (see corresponding twiki section) only in MC

comment out the following line:

 process.DQMOutput = cms.EndPath( process.dqmOutput )

add ntuplizer lines (see corresponding twiki section)

3) Run the config files:

voms-proxy-init --voms cms --valid 168:00

cmsRun hlt_GRun_Data.py
### cmsRun hlt_GRun_MC.py

Setup: 2016 recipe with online menu v4.1

New menu to be tested: /online/collisions/2016/25ns15e33/v4.1/HLT

CMSSW area

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_19_patch2
cd CMSSW_8_0_19_patch2/src
cmsenv
rehash

git cms-addpkg L1Trigger/L1TGlobal
git cms-addpkg HLTrigger/Configuration

### Please make sure that the above compiles.

# HLTNtuples tools
git cms-addpkg HLTrigger/HLTanalyzers
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# L1 menu 
cp /afs/cern.ch/user/t/tmatsush/public/L1MenuDev/L1Menu_Collisions2016_v6r8.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# L1 prescales: save the default file and copy a *single* file depending on your use case.
mv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv.save
## 1e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.15e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1p15e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.3e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1p3e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.45e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1p45e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 

# Compile
scram b -j 4
cmsenv
rehash

Create CMSSW config files: standard menu

1) Create CMSSW config files using the hltGetConfiguration command:

# Create a working directory
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file #

## Running only the HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v4.1/HLT --full --offline --unprescale --no-output --process TEST --l1Xml L1Menu_Collisions2016_v6r8.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016G/HLTPhysics/RAW/v1/000/279/024/00000/CA41036D-B865-E611-B7DC-02163E01342D.root --max-events 100 > hlt_online_v4p1_Data.py

## Running L1 emulator + HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v4.1/HLT --full --offline --unprescale --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v6r8.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016E/ParkingZeroBias0/RAW/v2/000/277/069/00000/02123456-084E-E611-A1A9-02163E012A56.root --max-events 100 > hlt_L1v6r8_online_v4p1_Data.py

2) Edit the config files:

add the PU filters as a fake trigger (see corresponding twiki section) only in MC

comment out the following line:

 process.DQMOutput = cms.EndPath( process.dqmOutput )

add ntuplizer lines (see corresponding twiki section)

3) Run the config files:

voms-proxy-init --voms cms --valid 168:00

cmsRun hlt_GRun_Data.py
### cmsRun hlt_GRun_MC.py

Setup: 2016 recipe with online menu v4.0

New menu to be tested: /online/collisions/2016/25ns15e33/v4.0/HLT

CMSSW area

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_19_patch1
cd CMSSW_8_0_19_patch1/src
cmsenv
rehash

git cms-addpkg L1Trigger/L1TGlobal
git cms-addpkg HLTrigger/Configuration
git cms-merge-topic -u cms-tsg-storm:80XHLTAfterMD2Train
git cms-checkdeps -A -a

### Please make sure that the above compiles.

# HLTNtuples tools
git cms-addpkg HLTrigger/HLTanalyzers
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# L1 menu 
cp /afs/cern.ch/user/t/tmatsush/public/L1MenuDev/L1Menu_Collisions2016_v6r5.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# L1 prescales: save the default file and copy a *single* file depending on your use case.
mv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv.save
## 1e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.15e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1p15e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.3e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1p3e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.45e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1p45e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 

# Compile
scram b -j 4
cmsenv
rehash

Create CMSSW config files: standard menu

1) Create CMSSW config files using the hltGetConfiguration command:

# Create a working directory
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file #

## Running only the HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v4.0/HLT --full --offline --unprescale --no-output --process TEST --l1Xml L1Menu_Collisions2016_v6r5.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016G/HLTPhysics/RAW/v1/000/279/024/00000/CA41036D-B865-E611-B7DC-02163E01342D.root --max-events 100 > hlt_online_v4_Data.py

## Running L1 emulator + HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v4.0/HLT --full --offline --unprescale --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v6r5.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016E/ParkingZeroBias0/RAW/v2/000/277/069/00000/02123456-084E-E611-A1A9-02163E012A56.root --max-events 100 > hlt_L1v6r5_online_v4_Data.py

2) Edit the config files:

add the PU filters as a fake trigger (see corresponding twiki section) only in MC

comment out the following line:

 process.DQMOutput = cms.EndPath( process.dqmOutput )

add ntuplizer lines (see corresponding twiki section)

3) Run the config files:

voms-proxy-init --voms cms --valid 168:00

cmsRun hlt_GRun_Data.py
### cmsRun hlt_GRun_MC.py

Create CMSSW config files: user menu

In case you want to use a user HLT menu, you need to build the hlt cmssw config file in a different way. Note that paths in your user configuration should use seeds that exist in the L1 Menu v5. As a general rule, you should always develop your user configuration out of the most recent HLT/V* "master table".

hltConfigFromDB --cff --configName /dev/CMSSW_8_0_0/GRun --nopaths --services -PrescaleService,-EvFDaqDirector,-FastMonitoringService > setup_cff.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --data --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v6r5.xml --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2015D/ZeroBias1/RAW/v1/000/256/843/00000/FE8AD1BB-D05E-E511-B3A7-02163E01276B.root > hlt_user_data.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --mc   --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v6r5.xml --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 --input /store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00002/3E809666-0EAE-E511-A165-002590DD7C9E.root > hlt_user_mc.py

# Edit the config file and add the following line just after 'process = cms.Process( "TEST" )':
process.load("setup_cff")
# also, follow all other necessary editing steps as described in the standard case above (add PU removal, comment DQM module, add ntuplizer)

Setup: 2016 recipe with online menu v3.1

New menu to be tested: /online/collisions/2016/25ns15e33/v3.1/HLT
L1 PS Proposal from L1 DPG in this google doc.
HLT SW setup based on this version of the recipe from STORM.

CMSSW area

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_14
cd CMSSW_8_0_14/src
cmsenv
rehash

git cms-addpkg L1Trigger/L1TGlobal
git cms-addpkg HLTrigger/Configuration
git cms-merge-topic -u cms-tsg-storm:80XHLTJuly29thTrain
git cms-checkdeps -A -a

### Please make sure that the above compiles.

# HLTNtuples tools
git cms-addpkg HLTrigger/HLTanalyzers
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# L1 menu 
cp /afs/cern.ch/user/t/tmatsush/public/L1MenuDev/L1Menu_Collisions2016_v6.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# L1 prescales: save the default file and copy a *single* file depending on your use case.
mv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv.save
## 1e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.15e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1p15e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.3e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1p3e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.45e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v6_1p45e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 

# Compile
scram b -j 4
cmsenv
rehash

Create CMSSW config files: standard menu

1) Create CMSSW config files using the hltGetConfiguration command:

# Create a working directory
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file #

## Running only the HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v3.1/HLT --full --offline --unprescale --no-output --process TEST --l1Xml L1Menu_Collisions2016_v6.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016E/HLTPhysics0/RAW/v2/000/277/069/00000/00574338-064E-E611-89E5-02163E013542.root --max-events 100 > hlt_online3p1_Data.py

## Running L1 emulator + HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v3.1/HLT --full --offline --unprescale --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v6.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016E/ParkingZeroBias0/RAW/v2/000/277/069/00000/02123456-084E-E611-A1A9-02163E012A56.root --max-events 100 > hlt_L1v6_online3p1_Data.py

2) Edit the config files:

add the PU filters as a fake trigger (see corresponding twiki section) only in MC

comment out the following line:

 process.DQMOutput = cms.EndPath( process.dqmOutput )

add ntuplizer lines (see corresponding twiki section)

3) Run the config files:

voms-proxy-init --voms cms --valid 168:00

cmsRun hlt_GRun_Data.py
### cmsRun hlt_GRun_MC.py

Create CMSSW config files: user menu

hltConfigFromDB --cff --configName /dev/CMSSW_8_0_0/GRun --nopaths --services -PrescaleService,-EvFDaqDirector,-FastMonitoringService > setup_cff.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --data --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v6.xml --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2015D/ZeroBias1/RAW/v1/000/256/843/00000/FE8AD1BB-D05E-E511-B3A7-02163E01276B.root > hlt_user_data.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --mc   --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v6.xml --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 --input /store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00002/3E809666-0EAE-E511-A165-002590DD7C9E.root > hlt_user_mc.py

# Edit the config file and add the following line just after 'process = cms.Process( "TEST" )':
process.load("setup_cff")
# also, follow all other necessary editing steps as described in the standard case above (add PU removal, comment DQM module, add ntuplizer)

Setup: 2016 recipe with online menu v3.0

New menu to be tested: /online/collisions/2016/25ns15e33/v3.0/HLT
L1 PS Proposal from L1 DPG in this google doc.
HLT SW setup based on this version of the recipe from STORM.

CMSSW area

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_14
cd CMSSW_8_0_14/src
cmsenv
rehash

git cms-addpkg L1Trigger/L1TGlobal
git cms-addpkg HLTrigger/Configuration
git cms-merge-topic -u cms-tsg-storm:80XHLTJuly6thTrain
git cms-checkdeps -A -a

### Please make sure that the above compiles.

# HLTNtuples tools
git cms-addpkg HLTrigger/HLTanalyzers
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# L1 menu 
cp /afs/cern.ch/user/t/tmatsush/public/L1MenuDev/L1Menu_Collisions2016_v5.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# L1 prescales: save the default file and copy a *single* file depending on your use case.
mv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv.save
## 1e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v5_1e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.15e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v5_1p15e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.3e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v5_1p3e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
## 1.45e34
cp /afs/cern.ch/user/n/ndaci/public/STEAM/Prescales/l1prescales_L1v5_1p45e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 

# Compile
scram b -j 4
cmsenv
rehash

Create CMSSW config files: standard menu

1) Create CMSSW config files using the hltGetConfiguration command:

# Create a working directory
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file #

## Running only the HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v3.0/HLT --full --offline --unprescale --no-output --process TEST --l1Xml L1Menu_Collisions2016_v5.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016B/HLTPhysics/RAW/v2/000/274/241/00000/02CCC876-ED26-E611-BEAD-02163E013827.root --max-events 100 > hlt_online3p0_Data.py

## Running L1 emulator + HLT SW
hltGetConfiguration /online/collisions/2016/25ns15e33/v3.0/HLT --full --offline --unprescale --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v5.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016B/ZeroBias1/RAW/v1/000/271/884/00000/BEA68E0A-3A0D-E611-97C1-02163E013710.root --max-events 100 > hlt_L1v5_online3p0_Data.py

2) Edit the config files:

add the PU filters as a fake trigger (see corresponding twiki section) only in MC

comment out the following line:

 process.DQMOutput = cms.EndPath( process.dqmOutput )

add ntuplizer lines (see corresponding twiki section)

3) Run the config files:

voms-proxy-init --voms cms --valid 168:00

cmsRun hlt_GRun_Data.py
### cmsRun hlt_GRun_MC.py

Create CMSSW config files: user menu

hltConfigFromDB --cff --configName /dev/CMSSW_8_0_0/GRun --nopaths --services -PrescaleService,-EvFDaqDirector,-FastMonitoringService > setup_cff.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --data --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v2.xml --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2015D/ZeroBias1/RAW/v1/000/256/843/00000/FE8AD1BB-D05E-E511-B3A7-02163E01276B.root > hlt_user_data.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --mc   --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v2.xml --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 --input /store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00002/3E809666-0EAE-E511-A165-002590DD7C9E.root > hlt_user_mc.py

# Edit the config file and add the following line just after 'process = cms.Process( "TEST" )':
process.load("setup_cff")
# also, follow all other necessary editing steps as described in the standard case above (add PU removal, comment DQM module, add ntuplizer)

Setup: 2016 recipe with online menu v2.2

New menu to be tested: /online/collisions/2016/25ns10e33/v2.2/HLT
L1 PS csv files correspond to L1 DPG 4th July proposal: 7e33, 8e33, 1e34, 1.15e34, 1.3e34
Anything that is useful only to process MC has been commented out in this recipe. We will focus on processing data for now.
HLT SW setup based on this version of the recipe from STORM.

CMSSW area

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_13_patch1
cd CMSSW_8_0_13_patch1/src
cmsenv
rehash

git cms-addpkg L1Trigger/L1TGlobal
git cms-addpkg HLTrigger/Configuration
git cms-merge-topic -u cms-tsg-storm:80XHLTJune15thTrain
git cms-checkdeps -A -a

### Please make sure that the above compiles.

# HLTNtuples tools
git cms-addpkg HLTrigger/HLTanalyzers
### git cms-addpkg GeneratorInterface/GenFilters   
### git cms-addpkg SimGeneral/MixingModule 

### git cms-merge-topic -u ndaci:b_steam_fix_804

### git clone git@github.com:cms-steam/RemovePileUpDominatedEvents.git RemovePileUpDominatedEvents 
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# L1 menu 
cp /afs/cern.ch/user/t/tmatsush/public/L1MenuDev/L1Menu_Collisions2016_v4.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# L1 prescales: only if you want to apply L1 prescales !!! Copy only a single csv file depending on the PS column you wish to apply !!!
### to save the default csv file
mv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv.save
### for the 7e33 PS column
cp /afs/cern.ch/user/c/clint/Work/public/forNadir/Jul2016/l1Prescales_July2016_7e33.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
### for the 8e33 PS column
cp /afs/cern.ch/user/c/clint/Work/public/forNadir/Jul2016/l1Prescales_July2016_8e33.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
### for the 1e34 PS column
cp /afs/cern.ch/user/c/clint/Work/public/forNadir/Jul2016/l1Prescales_July2016_1e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
### for the 1.15e34 PS column
cp /afs/cern.ch/user/c/clint/Work/public/forNadir/Jul2016/l1Prescales_July2016_1p15e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 
### for the 1.3e34 PS column
cp /afs/cern.ch/user/c/clint/Work/public/forNadir/Jul2016/l1Prescales_July2016_1p3e34.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 

# Compile
scram b -j 4
cmsenv
rehash

Create CMSSW config files: standard menu

1) Create CMSSW config files using the hltGetConfiguration command:

# Create a working directory
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file from the GRun/V115 #

## Running only the HLT SW
hltGetConfiguration  /online/collisions/2016/25ns10e33/v2.2/HLT --full --offline --unprescale --no-output --process TEST --l1Xml L1Menu_Collisions2016_v4.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016B/HLTPhysics/RAW/v2/000/274/241/00000/02CCC876-ED26-E611-BEAD-02163E013827.root --max-events 100 > hlt_online_Data.py
### hltGetConfiguration /online/collisions/2016/25ns10e33/v2.2/HLT --full --offline --unprescale --no-output --process TEST --l1Xml L1Menu_Collisions2016_v4.xml --mc   --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 --input /store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00002/3E809666-0EAE-E511-A165-002590DD7C9E.root --max-events 100 > hlt_online_MC.py

## Running L1 emulator + HLT SW
hltGetConfiguration  /online/collisions/2016/25ns10e33/v2.2/HLT --full --offline --unprescale --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v4.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2016B/ZeroBias1/RAW/v1/000/271/884/00000/BEA68E0A-3A0D-E611-97C1-02163E013710.root --max-events 100 > hlt_online_Data.py
### hltGetConfiguration /online/collisions/2016/25ns10e33/v2.2/HLT --full --offline --unprescale --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v4.xml --mc   --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 --input /store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00002/3E809666-0EAE-E511-A165-002590DD7C9E.root --max-events 100 > hlt_online_MC.py

2) Edit the config files:

add the PU filters as a fake trigger (see corresponding twiki section) only in MC

comment out the following line:

 process.DQMOutput = cms.EndPath( process.dqmOutput )

add ntuplizer lines (see corresponding twiki section)

3) Run the config files:

voms-proxy-init --voms cms --valid 168:00

cmsRun hlt_GRun_Data.py
### cmsRun hlt_GRun_MC.py

Create CMSSW config files: user menu

hltConfigFromDB --cff --configName /dev/CMSSW_8_0_0/GRun --nopaths --services -PrescaleService,-EvFDaqDirector,-FastMonitoringService > setup_cff.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --data --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v2.xml --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2015D/ZeroBias1/RAW/v1/000/256/843/00000/FE8AD1BB-D05E-E511-B3A7-02163E01276B.root > hlt_user_data.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --mc   --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_v2.xml --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 --input /store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00002/3E809666-0EAE-E511-A165-002590DD7C9E.root > hlt_user_mc.py

# Edit the config file and add the following line just after 'process = cms.Process( "TEST" )':
process.load("setup_cff")
# also, follow all other necessary editing steps as described in the standard case above (add PU removal, comment DQM module, add ntuplizer)

Setup: 2016 recipe with online menu v1.0

Warning: anything that is useful only to process MC has been commented out in this recipe. We will focus on processing data for now.
The recipe works to process Data; it should work with MC, but as stated above, we comment out MC utilities here for now.
HLTInfo has been updated to read the Level-1 Stage-2 Menu correctly and create 2 branches per L1 algorithm: 1 for the decision, 1 for the prescale. L1 prescales not properly read yet.
HLT SW setup based on this version of the recipe from STORM.

CMSSW area

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_6
cd CMSSW_8_0_6/src
cmsenv
rehash

# L1+HLT
git cms-addpkg HLTrigger/Configuration
git cms-addpkg L1Trigger/L1TGlobal
git cms-merge-topic -u cms-tsg-storm:towardsFirstHltMenuFor2016_80X
git cms-merge-topic -u 14141
git cms-merge-topic -u 14275

# Dependencies
git cms-checkdeps -A -a

### Please make sure that the above compiles.

# HLTNtuples tools
git cms-addpkg HLTrigger/HLTanalyzers
### git cms-addpkg GeneratorInterface/GenFilters   
### git cms-addpkg SimGeneral/MixingModule 

### git cms-merge-topic -u ndaci:b_steam_fix_804

### git clone git@github.com:cms-steam/RemovePileUpDominatedEvents.git RemovePileUpDominatedEvents 
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# Overwrite L1 menu:
cp /afs/cern.ch/user/t/tmatsush/public/L1MenuDev/L1Menu_Collisions2016_dev_v5.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# L1 prescales: WARNING: launch the command below only if you want to apply L1 prescales
mv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv.save
cp /afs/cern.ch/user/c/clint/Work/public/prescale_L1TGlobal_v6-1e34.csv  L1Trigger/L1TGlobal/data/Luminosity/startup/prescale_L1TGlobal.csv 

# Compile
scram b -j 4
cmsenv
rehash

Create CMSSW config files: standard menu

1) Create CMSSW config files using the hltGetConfiguration command:

# Create a working directory
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file from the GRun #

## GRun table corresponding to online menu v1.0 V1: GRun/V72
hltGetConfiguration  /dev/CMSSW_8_0_0/GRun/V72 --full --offline --prescale 1e34 --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_dev_v5.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2015D/ZeroBias1/RAW/v1/000/256/843/00000/FE8AD1BB-D05E-E511-B3A7-02163E01276B.root --max-events 100 > hlt_GRunV72_Data.py
### hltGetConfiguration /dev/CMSSW_8_0_0/GRun/V72 --full --offline --prescale 1e34 --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_dev_v5.xml --mc   --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 --input /store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00002/3E809666-0EAE-E511-A165-002590DD7C9E.root --max-events 100 > hlt_GRunV72_MC.py

2) Edit the config files:

add the PU filter as a fake trigger (see corresponding twiki section) only in MC

comment out the following line:

 process.DQMOutput = cms.EndPath( process.dqmOutput )

add ntuplizer lines (see corresponding twiki section)

3) Run the config files:

voms-proxy-init --voms cms --valid 168:00

cmsRun hlt_GRunV72_Data.py
### cmsRun hlt_GRunV72_MC.py

Create CMSSW config files: user menu

hltConfigFromDB --cff --configName /dev/CMSSW_8_0_0/GRun --nopaths --services -PrescaleService,-EvFDaqDirector,-FastMonitoringService > setup_cff.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --data --unprescale --process TEST --l1-emulator 'Full' --globaltag auto:run2_hlt_GRun --input /store/data/Run2015D/ZeroBias1/RAW/v1/000/256/843/00000/FE8AD1BB-D05E-E511-B3A7-02163E01276B.root > hlt_user_data.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --mc   --unprescale --process TEST --l1-emulator 'Full' --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 --input /store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00002/3E809666-0EAE-E511-A165-002590DD7C9E.root > hlt_user_mc.py

# Edit the config file and add the following line just after 'process = cms.Process( "TEST" )':
process.load("setup_cff")
# also, follow all other necessary editing steps as described in the standard case above (add PU removal, comment DQM module, add ntuplizer)

Setup: 2016 recipe with L1 Menu v4

This recipe has been tested with the L1 Menus v4.
Warning: L1 prescales are defined in an additional csv file. Avoid launching the "cp /afs/...csv" command if you do not want to apply L1 prescales.
The recipe works to process Data and to process MC without using the L1-based pile-up filters (the GEN PU filters are fine).
Need to migrate from Stage1 to Stage2 L1 objects and menus. Needed to: fill L1 branches; identify prescale column; make use of the L1 pile-up filter.
HLT SW setup based on this version of the recipe from STORM.

Show the recipe

Hide the recipe

cmsrel CMSSW_8_0_4
cd CMSSW_8_0_4/src
cmsenv
rehash

# L1 Emulator + HLT Software
git cms-addpkg HLTrigger/Configuration
git cms-merge-topic -u 13704
git cms-merge-topic -u 13767
git cms-merge-topic -u 13906
git cms-merge-topic -u 14074
git cms-merge-topic -u cms-tsg-storm:HLTinApril80X
git cms-checkdeps -A -a

### Please make sure that the above compiles.

# HLTNtuples tools
git cms-addpkg HLTrigger/HLTanalyzers
git cms-addpkg GeneratorInterface/GenFilters   
git cms-addpkg SimGeneral/MixingModule 

git cms-merge-topic -u ndaci:b_steam_fix_804

git clone git@github.com:cms-steam/RemovePileUpDominatedEvents.git RemovePileUpDominatedEvents 
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

# L1 menu
cp /afs/cern.ch/user/t/tmatsush/public/L1MenuDev/L1Menu_Collisions2016_dev_v4.xml L1Trigger/L1TGlobal/data/Luminosity/startup/

# L1 prescales: WARNING: launch the command below only if you want to apply L1 prescales!!!
cp /afs/cern.ch/user/c/clint/Work/public/prescale_L1TGlobal.csv L1Trigger/L1TGlobal/data/Luminosity/startup/

# Compile
scram b -j 4
cmsenv
rehash

# Prepare to run
cd $CMSSW_BASE/src
mkdir MyTests
cd MyTests

# Get HLT config file from the GRun #

## Latest GRun table with L1 Menu v4: GRun/V56
hltGetConfiguration /dev/CMSSW_8_0_0/GRun/V56 --full --offline --prescale 1e34 --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_dev_v4.xml --data --globaltag 80X_dataRun2_HLT_v12 --input /store/data/Run2015D/ZeroBias1/RAW/v1/000/256/843/00000/FE8AD1BB-D05E-E511-B3A7-02163E01276B.root --max-events 100 > hlt_L1v4_GRunV56_Data.py
hltGetConfiguration /dev/CMSSW_8_0_0/GRun/V56 --full --offline --prescale 1e34 --no-output --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_dev_v4.xml --mc   --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 --input /store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00002/3E809666-0EAE-E511-A165-002590DD7C9E.root --max-events 100 > hlt_L1v4_GRunV56_MC.py

# Edit HLT config file
## - add the PU filter as a fake trigger (see corresponding twiki section)
## - add ntuplizer lines (see corresponding twiki section)
## - remove/comment "process.DQMOutput" module 

# Run
voms-proxy-init --voms cms --valid 168:00

cmsRun hlt_L1v4_GRun_Data.py
cmsRun hlt_L1v4_GRun_MC.py

In case you want to use a user HLT menu, you need to build the hlt cmssw config file in a different way. Note that, in order to develop paths using seeds from L1 menu v4, it is probably better to base your user configuration on the last menu that was using L1 menu v4: HLT/V129 (corresponding GRun table: GRun/V56).

Show the recipe

Hide the recipe

hltConfigFromDB --cff --configName /dev/CMSSW_8_0_0/GRun/V56 --nopaths --services -PrescaleService,-EvFDaqDirector,-FastMonitoringService > setup_cff.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --data --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_dev_v4.xml --globaltag 80X_dataRun2_HLT_v12 > hlt_user_data.py
hltGetConfiguration /users/{username}/{config name}/V{number} --full --offline --mc   --unprescale --process TEST --l1-emulator 'Full' --l1Xml L1Menu_Collisions2016_dev_v4.xml --globaltag 80X_mcRun2_asymptotic_ForTSGStudies_fromRunIIFall15DR76_v1 > hlt_user_mc.py

# Edit the config file and add the following line just after 'process = cms.Process( "TEST" )':
process.load("setup_cff")
# also, follow all other necessary editing steps as described in the standard case above

Setup: 2015 recipe (CMSSW_76X)

Install CMSSW, update the HLTanalyzers code, add the pile-up filter and DQM online codes:

please check SWGuideGlobalHLT for the latest CMSSW release to install

Show the recipe

Hide the recipe

setenv SCRAM_ARCH slc6_amd64_gcc493
cmsrel CMSSW_7_6_3
cd CMSSW_7_6_3/src
cmsenv

git cms-init
git cms-addpkg GeneratorInterface/GenFilters     #only in MC Ntuple production
git cms-addpkg SimGeneral/MixingModule     #only in MC Ntuple production
git cherry-pick 5ac8c6c     #only in MC Ntuple production

git clone git@github.com:cms-steam/RemovePileUpDominatedEvents.git RemovePileUpDominatedEvents     #only in MC Ntuple production

#git cms-merge-topic muell149:HLTObjectMonitor_OfflineTesting #waiting for fixing
git cms-addpkg HLTrigger/HLTanalyzers
git cms-addpkg HLTrigger/Configuration
git cms-addpkg L1TriggerConfig/L1GtConfigProducers
git clone git@github.com:cms-steam/HLTrigger temp
cp -r temp/* HLTrigger/
rm -fr temp/

download the L1 XML menu, it misses in the release:

cp /afs/cern.ch/user/t/tmatsush/public/L1Menu/L1Menu_Collisions2015_25nsStage1_v5/xml/L1Menu_Collisions2015_25nsStage1_v5_L1T_Scales_20141121.xml L1TriggerConfig/L1GtConfigProducers/data/Luminosity/startup/L1Menu_Collisions2015_25nsStage1_v5_L1T_Scales_20141121.xml

override L1 prescales (they depend on the menu!) - but only in MC production:

cp /afs/cern.ch/work/g/georgia/public/L1prescales/Stage1-v5_prescales/7e33/* L1TriggerConfig/L1GtConfigProducers/python/.

scram build -j 4

download the HLT configuration:

hltGetConfiguration \
/frozen/2015/25ns14e33/v4.4/HLT/V3 \
--prescale 7e33 \
--globaltag 76X_mcRun2_asymptotic_v12 \
--l1-emulator 'stage1,gt' \
--l1Xml L1Menu_Collisions2015_25nsStage1_v5_L1T_Scales_20141121.xml \
--input root://xrootd.ba.infn.it//store/mc/RunIIFall15DR76/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/GEN-SIM-RAW/25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/00000/000AB15C-AAAB-E511-9988-0025904A891A.root \
--max-events 100 \
--full --offline --mc --process TEST --no-output \
> hlt.py

please check that all options correspond exactly to what you need!
In particular, please check:

menu (eg. /frozen/2015/25ns14e33/v4.4/HLT/V1)
global-tag (eg. PHY1474_STV4, you can find the correct GT for you dataset here: SWGuideFrontierConditions )
prescale column (usually 5e33 or 7e33 or 1.4e34)
l1-emulator ('stage1,gt' for 25ns 2015 menus)
l1Xml menu (eg. L1Menu _Collisions2015_50nsGct_v4_L1T_Scales_20141121.xml)
Input file for test: must be one root file corresponding to the dataset you will use in the Production. Look for it for example here: DAS example

Add the pile-up filter as a fake trigger

WARNING: an external text file is needed for the GEN PU filter to work:

 /afs/cern.ch/user/s/sdonato/AFSwork/public/genJetPtHatPU/0.txt

. Make sure you work from lxplus, or on a workspace where AFS is mounted. For crab tasks, this file needs to be added as an external file, see the multicrab section on this twiki.

This is specific to the GEN and PU filters (so to use in MC Ntuple production only). To add the fake paths needed for the rate estimate, edit hlt.py and add the following lines just after the "process.source" module:

Show the snippet

Hide the snippet

### STEAM GEN Filters ###
# L1-based PU filter 
# Do *not* include the two lines below if you are using L1 Stage2 system (e.g. in the 80X recipe)
#process.RemovePileUpDominatedEvents = cms.EDFilter("RemovePileUpDominatedEvents") 
#process.HLT_RemovePileUpDominatedEvents_v1 = cms.Path( process.HLTBeginSequence + process.RemovePileUpDominatedEvents + process.HLTEndSequence ) 

# GEN-based PU filter
process.RemovePileUpDominatedEventsGenV2 = cms.EDFilter("RemovePileUpDominatedEventsGenV2")
process.HLT_RemovePileUpDominatedEventsGenV2_v1 = cms.Path( process.HLTBeginSequence + process.RemovePileUpDominatedEventsGenV2 + process.HLTEndSequence )

###and add the particle filter as a fake trigger:
process.genParticlesForFilter = cms.EDProducer("GenParticleProducer",
    saveBarCodes = cms.untracked.bool(True),
    src = cms.InputTag("generatorSmeared"),
    abortOnUnknownPDGCode = cms.untracked.bool(False)
)
process.bctoefilter = cms.EDFilter("BCToEFilter",
                           filterAlgoPSet = cms.PSet(eTThreshold = cms.double(10),
                                                     genParSource = cms.InputTag("genParticlesForFilter")
                                                     )
)
process.emenrichingfilter = cms.EDFilter("EMEnrichingFilter",
                                 filterAlgoPSet = cms.PSet(isoGenParETMin=cms.double(20.),
                                                           isoGenParConeSize=cms.double(0.1),
                                                           clusterThreshold=cms.double(20.),
                                                           isoConeSize=cms.double(0.2),
                                                           hOverEMax=cms.double(0.5),
                                                           tkIsoMax=cms.double(5.),
                                                           caloIsoMax=cms.double(10.),
                                                           requireTrackMatch=cms.bool(False),
                                                           genParSource = cms.InputTag("genParticlesForFilter")
                                                           )
)
process.genmuons = cms.EDFilter("CandViewSelector",
    src = cms.InputTag("genParticles"),
    cut = cms.string("(abs(pdgId())==13) && (pt > 5.0) && (abs(vz())<400) && ((vx()*vx() + vy()*vy() )<(400*400)) && (abs(eta())<2.5)")
)
process.mufilter = cms.EDFilter("CandViewCountFilter",  src = cms.InputTag("genmuons"), minNumber = cms.uint32(1))

process.HLT_BCToEFilter_v1=cms.Path(process.genParticlesForFilter + process.bctoefilter)
process.HLT_EmFilter_v1=cms.Path(process.genParticlesForFilter + process.emenrichingfilter)
process.HLT_EmGlobalFilter_v1=cms.Path(process.genParticlesForFilter + ~process.bctoefilter + process.emenrichingfilter)
process.HLT_MuFilter_v1=cms.Path(process.genmuons + process.mufilter)

process.load("SimGeneral.MixingModule.mixNoPU_cfi")
import SimGeneral.MixingModule.trackingTruthProducer_cfi
process.mergedtruthNoSimHits = process.trackingParticles.clone(
                                                               simHitCollections = cms.PSet(
                                                               muon = cms.VInputTag(),
                                                               tracker = cms.VInputTag(),
                                                               pixel = cms.VInputTag()
                                                               )
)

process.mix.digitizers = cms.PSet( mergedtruth = process.mergedtruthNoSimHits )
process.mix.mixObjects = cms.PSet()
del process.simCastorDigis
del process.simEcalUnsuppressedDigis
del process.simHcalUnsuppressedDigis
del process.simSiPixelDigis
del process.simSiStripDigis 
process.TPmuons = cms.EDFilter("CandViewSelector",
    src = cms.InputTag("genParticles"),
    cut = cms.string("(abs(pdgId())==13) && (pt > 5.0) && (abs(vz())<400) && ((vx()*vx() + vy()*vy() )<(400*400)) && (abs(eta())<2.5)")
)
process.muTPfilter = cms.EDFilter("CandViewCountFilter", src = cms.InputTag("TPmuons"), minNumber = cms.uint32(1))

process.HLT_MuFilterTP_v1=cms.Path(process.mix + process.TPmuons + process.muTPfilter)

### END STEAM GEN Filters ###

Add the Ntuplizer lines

To add the Ntuplizer, please edit the hlt.py config file and add the following lines just after 'process' is defined:

process = cms.Process( "TEST" ) ### this line is already there

process.load("HLTrigger.HLTanalyzers.HLTBitAnalyser_cfi")                                                                                
process.hltbitanalysis.HLTProcessName = cms.string(process.name_() )                                                                     
process.hltbitanalysis.hltresults = cms.InputTag( 'TriggerResults','',process.name_())                                                   
process.hltbitanalysis.l1results = cms.InputTag('hltGtStage2Digis', '', process.name_())                                   
process.hltbitanalysis.RunParameters.HistogramFile = cms.untracked.string('hltbits.root')                           
### process.hltbitanalysis.RunParameters.isData = cms.untracked.bool(True) ### uncomment this one only to process Data !!!
process.HLTBitAnalysisEndpath = cms.EndPath( process.hltbitanalysis )

Then you need to add **ONLY** "process.HLTBitAnalysisEndpath" to the end of the "Scheduler":

process.HLTSchedule = cms.Schedule( *(process.HLTriggerFirstPath, [...] , process.HLTBitAnalysisEndpath ))

Add custom lines for L1 prescales

Add at the end of your cmssw config file the following lines:

process.load('L1Trigger.L1TGlobal.hackConditions_cff')
process.L1TGlobalPrescalesVetos.PrescaleXMLFile = cms.string('UGT_BASE_RS_PRESCALES_v214.xml') 
process.L1TGlobalPrescalesVetos.FinOrMaskXMLFile = cms.string('UGT_BASE_RS_FINOR_MASK_v91.xml')
process.simGtStage2Digis.AlgorithmTriggersUnmasked = cms.bool(False)
process.simGtStage2Digis.AlgorithmTriggersUnprescaled = cms.bool(False)
process.simGtStage2Digis.PrescaleSet = cms.uint32(4)

Where the parameter PrescaleSet sets the prescale column index to be used. WARNING: this index starts at 1 (ie the tightest prescale column corresponds to number "1").

PS Column	1.6e34	1.45e34	1.35e34	1.25e34	1.15e34	1.05e34	9.5e33	8.5e33	7.5e33	5e33	3.5e33	2e33	1e33	7e32	5e32	3.5e32	2e32
PrescaleSet	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17

Local test

Finally you can run a local test before to submit the jobs on crab.

voms-proxy-init --voms cms --valid 168:00
cmsRun hlt.py >& log &

Check the l1 and hlt summary reports from the log file:

cat log | grep  " L1_" > prescalesL1
cat log | grep HLT-Report | grep HLT_ > prescalesHLT

Please check carefully the logs and in particular both L1 and HLT prescales.

Also check the content of the hltbits.root file : e.g. check that all gen variables (in MC production ) are filled correctly , and that you get some entries in the L1_ and HLT_ histograms .

In case all is fine you can proceed with the submission with CRAB.

MultiCRAB3 configuration

Create "multicrab.py" file that contains the following lines (to be adapted).

2016 data samples (HLTPhysics) with 80X

Show the snippet

Hide the snippet

##########################
# #
# HLTPhysics Run2016G #
# #
##########################


if __name__ == '__main__':
from CRABAPI.RawCommand import crabCommand

def submit(config):
res = crabCommand('submit', config = config)

from CRABClient.UserUtilities import config
config = config()

dataset = {
'HLTPhysics0' : '/HLTPhysics0/Run2016G-v1/RAW',
'HLTPhysics1' : '/HLTPhysics1/Run2016G-v1/RAW'
}

nfiles = -1 
filesPerJob = 10
mask = '/afs/cern.ch/user/j/jjbuch/ntuples_HLTPhysics_2016G_menu3p1p6_unprescale/CMSSW_8_0_19/src/MyTests/json4rate_DCSONLY.txt'

name = 'HLTPhysics_ntuples'
config.General.workArea = 'crab_'+name
config.General.transferLogs = True
config.JobType.pluginName = 'Analysis'
config.JobType.psetName = 'hlt_online3p1_Data.py'
config.Data.inputDBS = 'global'
config.Data.splitting = 'LumiBased'
config.Data.publication = False
config.JobType.outputFiles = ['hltbits.root'] #,'DQMIO.root']
config.Site.storageSite = 'T2_CH_CERN'
config.Data.ignoreLocality = True #Required for CAF submission
config.Site.whitelist = ['T3_CH_CERN_CAF'] #Required for CAF submission
config.Site.ignoreGlobalBlacklist = True #Required for CAF submission

listOfSamples = ['HLTPhysics0','HLTPhysics1']
for sample in listOfSamples:
config.General.requestName = sample
config.Data.inputDataset = dataset[sample]
config.Data.unitsPerJob = filesPerJob
config.Data.totalUnits = nfiles
config.Data.outputDatasetTag = sample
config.Data.lumiMask = mask
config.Data.outLFNDirBase = '/store/group/dpg_trigger/comm_trigger/TriggerStudiesGroup/STEAM/Run2016G/HLTPhysics_2016G_menu3p1p6_279694/' + name # or '/store/group/<subdir>'
submit(config)

76X MC samples (flat PU)

The following samples correspond to the 76X MC production which come with flat PU 10to50 . The list of MC samples dedicated for L1 and HLT Rates studies with 76x can be found here: MCsamples_76x

Show the snippet

Hide the snippet

####################
# #
#Summer 15 #
# #
####################

dataset = {
 'QCD15': '/QCD_Pt_15to30_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCD30': '/QCD_Pt_30to50_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCD50': '/QCD_Pt_50to80_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCD80': '/QCD_Pt_80to120_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCD120': '/QCD_Pt_120to170_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW', 
 'QCD170': '/QCD_Pt_170to300_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCD300': '/QCD_Pt_300to470_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCD470': '/QCD_Pt_470to600_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',

# 'QCD600': '/QCD_Pt_600to800_TuneCUETP8M1_13TeV_pythia8/RunIISpring15Digi74-AVE_20_BX_25ns_tsg_MCRUN2_74_V7-v1/GEN-SIM-RAW',
# 'QCD800': '/QCD_Pt_800to1000_TuneCUETP8M1_13TeV_pythia8/RunIISpring15Digi74-AVE_20_BX_25ns_tsg_MCRUN2_74_V7-v1/GEN-SIM-RAW',
# 'QCD1000': '/QCD_Pt_1000to1400_TuneCUETP8M1_13TeV_pythia8/RunIISpring15Digi74-AVE_20_BX_25ns_tsg_MCRUN2_74_V7-v1/GEN-SIM-RAW',
# 'QCD1400': '/QCD_Pt_1400to1800_TuneCUETP8M1_13TeV_pythia8/RunIISpring15Digi74-AVE_20_BX_25ns_tsg_MCRUN2_74_V7-v1/GEN-SIM-RAW',
# 'QCD1800': '/QCD_Pt_1800to2400_TuneCUETP8M1_13TeV_pythia8/RunIISpring15Digi74-AVE_20_BX_25ns_tsg_MCRUN2_74_V7-v1/GEN-SIM-RAW',
# 'QCD2400': '/QCD_Pt_2400to3200_TuneCUETP8M1_13TeV_pythia8/RunIISpring15Digi74-AVE_20_BX_25ns_tsg_MCRUN2_74_V7-v1/GEN-SIM-RAW',
# 'QCD3200': '/QCD_Pt_3200toInf_TuneCUETP8M1_13TeV_pythia8/RunIISpring15Digi74-AVE_20_BX_25ns_tsg_MCRUN2_74_V7-v1/GEN-SIM-RAW',


 'QCDEM15': '/QCD_Pt-15to20_EMEnriched_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDEM20': '/QCD_Pt-20to30_EMEnriched_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDEM30': '/QCD_Pt-30to50_EMEnriched_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDEM50': '/QCD_Pt-50to80_EMEnriched_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDEM80': '/QCD_Pt-80to120_EMEnriched_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDEM120': '/QCD_Pt-120to170_EMEnriched_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',

'QCDMu15': '/QCD_Pt-15to20_MuEnrichedPt5_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDMu20': '/QCD_Pt-20to30_MuEnrichedPt5_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDMu30': '/QCD_Pt-30to50_MuEnrichedPt5_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDMu50': '/QCD_Pt-50to80_MuEnrichedPt5_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDMu80': '/QCD_Pt-80to120_MuEnrichedPt5_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'QCDMu120': '/QCD_Pt-120to170_MuEnrichedPt5_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',

'DYToLL': '/DYToLL_M_1_TuneCUETP8M1_13TeV_pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW',
 'WJets': '/WJetsToLNu_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15DR76-25nsFlat10to50NzshcalRaw_76X_mcRun2_asymptotic_v12-v1/GEN-SIM-RAW'
}

nfiles = {
 'QCD15': -1,
 'QCD30': -1,
 'QCD50': -1,
 'QCD80': -1,
 'QCD120': -1,
 'QCD170': -1,
 'QCD300': -1,
 'QCD470': -1,
 'QCD600': -1,
 'QCD800': -1,
 'QCD1000': -1,
 'QCD1400': -1,
 'QCD1800': -1,
 'QCD2400': -1,
 'QCD3200': -1,

'QCDEM15': -1,
 'QCDEM20': -1,
 'QCDEM30': -1,
 'QCDEM50': -1,
 'QCDEM80': -1,
 'QCDEM120': -1,

'QCDMu15': -1,
 'QCDMu20': -1,
 'QCDMu30': -1,
 'QCDMu50': -1,
 'QCDMu80': -1,
 'QCDMu120': -1,
 'QCDMu120': -1,
 
 'DYToLL': -1,
 'WJets': -1

}

filesPerJob = {
 'QCD15': 5,
 'QCD30': 30,
 'QCD50': 20,
 'QCD80': 10,
 'QCD120': 10,
 'QCD170': 5,
 'QCD300': 1,
 'QCD470': 1,
 'QCD600': 7,
 'QCD800': 7,
 'QCD1000': 6,
 'QCD1400': 5,
 'QCD1800': 3,
 'QCD2400': 3,
 'QCD3200': 3,

 'QCDEM15': 10,
 'QCDEM20': 1,
 'QCDEM30': 10,
 'QCDEM50': 1,
 'QCDEM80': 10,
 'QCDEM120': 5,

 'QCDMu15': 10,
 'QCDMu20': 20,
 'QCDMu30': 20,
 'QCDMu50': 10,
 'QCDMu80': 5,
 'QCDMu120': 1,

 'DYToLL': 1,
 'WJets': 11


 
}

if __name__ == '__main__':
 from CRABAPI.RawCommand import crabCommand

 def submit(config):
  res = crabCommand('submit', config = config)

 from CRABClient.UserUtilities import config
 config = config()
 name = 'HLTRates_2e33_25ns_V4p4_V1'
 config.General.workArea = 'crab_'+name
 config.General.transferLogs = True
# config.General.transferOutputs = True
 config.JobType.pluginName = 'Analysis'
 config.JobType.psetName = 'hlt.py'
 config.Data.inputDBS = 'global'
 config.Data.splitting = 'FileBased'
# config.Data.publication = True
 config.Data.publication = False
 config.JobType.outputFiles = ['hltbits.root'] #,'DQMIO.root']
 config.Site.storageSite = 'T2_CH_CERN'
 
# listOfSamples = ['QCDEM15','QCDEM20','QCDEM30','QCDEM50','QCDEM80','QCDEM120','QCDMu15','QCDMu20','QCDMu30','QCDMu50','QCDMu80','QCDMu120','QCD15','QCD30','QCD50','QCD80','QCD120','QCD170','QCD300','QCD470','QCD600','QCD800','QCD1000','QCD1400','QCD1800','QCD2400','QCD3200','DYToLL','WJets']
 listOfSamples = ['QCDEM15','QCDEM20','QCDEM30','QCDEM50','QCDEM80','QCDEM120','QCDMu15','QCDMu20','QCDMu30','QCDMu50','QCDMu80','QCDMu120','QCD15','QCD30','QCD50','QCD80','QCD120','QCD170','QCD300','QCD470','DYToLL','WJets']
 listOfSamples.reverse()
 for sample in listOfSamples:
  config.General.requestName = sample
  config.Data.inputDataset = dataset[sample]
  config.Data.unitsPerJob = filesPerJob[sample]
  config.Data.totalUnits = nfiles[sample]
  config.Data.outputDatasetTag = sample
  config.Data.outLFNDirBase = '/store/group/dpg_trigger/comm_trigger/TriggerStudiesGroup/STEAM/Spring15/' + name # or '/store/group/<subdir>'
  submit(config)

Launch Multicrab

cmsenv
source /cvmfs/cms.cern.ch/crab3/crab.sh       #or: source /cvmfs/cms.cern.ch/crab3/crab.csh
voms-proxy-init --voms cms --valid 168:00
python multicrab.py >& logMultiCRAB &

Note: there is an issue with crab running on 8.0.x currently. If you follow the standard procedure and do cmsenv then source crab, it breaks cmsRun....but you CAN submit crab commands. If you then do cmsenv after sourcing, you can use cmsRun again, but not crab . CRAB HN suggest to use one shell for cmsRun commands and one for crab commands , until they provide a proper fix.

Links

https://twiki.cern.ch/twiki/bin/viewauth/CMS/TmdRecipes</pre>

Topic revision: r151 - 2017-02-10 - NadirDaci

Account
- Log In

- Cern Search
- TWiki Search
- Google Search
Sandbox All webs

Edit
Attach

Copyright &© 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback