Difference: TreeMaker (1 vs. 3)

Revision 32019-07-30 - MatthewLawrenceJoyce

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"

TreeMaker

Line: 15 to 14
 chmod +x setup.sh ./setup.sh -c CMSSW_10_2_12 -a https -j 1 cd CMSSW_10_2_12/src/
Added:
>
>
voms-proxy-init --voms cms
 export SSL_CERT_DIR='/etc/pki/tls/certs:/etc/grid-security/certificates'

Usage

Line: 51 to 53
 

Add file list to the desired era directory /TreeMaker/Production/python/<name_of_era>

Perhaps there is a better way but I did this manually in the following way. Create a new file dict_<whatever_name_you_prefer>.py and manually enter the information in the following format in the /TreeMaker/Production/python directory:

Deleted:
<
<
 
Added:
>
>
 # format for dict entries:
Changed:
<
<
# data: [True, ['sample'] , []] # MC: [False, ['sample'] , []] # MC w/ extended sample: [False, ['sample','sample_ext'] , []] # MC w/ negative weights (amcatnlo): [False, ['sample'] , [neff]] # MC w/ negative weights (amcatnlo) + extended sample: [False, ['sample','sample_ext'] , [neff, neff_ext]]
>
>
# data: [0, ['sample'] , []] # MC: [1, ['sample'] , []] # MC w/ wrong PU: [2, ['sample'] , []] # MC w/ extended sample: [1, ['sample','sample_ext'] , []] # MC w/ negative weights (amcatnlo): [1, ['sample'] , [neff]] # MC w/ negative weights (amcatnlo) + extended sample: [1, ['sample','sample_ext'] , [neff, neff_ext]]
  Example: If we wanted to include QCD DoubleEMEnriched samples we could look up the the dataset on das and create the file dict_QCD_DoubleEMEnriched.py which we must edit to read

Revision 22019-07-29 - MatthewLawrenceJoyce

Line: 1 to 1
 
META TOPICPARENT name="TWiki.WebPreferences"
Added:
>
>

TreeMaker

 This document is intended as a guide to making TreeMaker ntuples via job submission to Condor. It is assumed that you already have a Grid certificate and access to LPC.

Line: 14 to 17
 cd CMSSW_10_2_12/src/ export SSL_CERT_DIR='/etc/pki/tls/certs:/etc/grid-security/certificates'
Changed:
<
<

Very basic description of structure

The script that runs everything as far as submitting to Condor is submitJobs.py in the /TreeMaker/Production/test/condorSub directory. This script actually uses submitJobsTM.py to submit the jobs. You have to give it a dictionary file that tells it which scenario and what dataset(s) you want to process. Below is an example of one of these dictionary files, dict_Summer16v3_gjets.py

>
>

Usage

 
Added:
>
>
The script that runs everything as far as submitting to Condor is submitJobs.py in the /TreeMaker/Production/test/condorSub directory. You have to give it a dictionary file that tells it which scenario and what dataset(s) you want to process. Below is an example of one of these dictionary files, dict_Summer16v3_gjets.py
 
flist = {
    "scenario": "Summer16v3",
Line: 44 to 45
 
python submitJobs.py -d Summer16v3_gjets.py -o root://cmseos.fnal.gov//store/group/lpcsusyphotons/TreeMakerTest -p -s
Changed:
<
<

Adding new samples

>
>

Adding new samples

  If you want to add samples that are not already included, you can do this in two steps:
Changed:
<
<

Add file list to the desired era directory /TreeMaker/Production/python/<name_of_era>

>
>

Add file list to the desired era directory /TreeMaker/Production/python/<name_of_era>

  Perhaps there is a better way but I did this manually in the following way. Create a new file dict_<whatever_name_you_prefer>.py and manually enter the information in the following format in the /TreeMaker/Production/python directory:
Added:
>
>
 
# format for dict entries:
    #                                                data: [True,  ['sample'] , []]
Line: 59 to 61
  # MC w/ negative weights (amcatnlo) + extended sample: [False, ['sample','sample_ext'] , [neff, neff_ext]]

Example:

Changed:
<
<
If we wanted to include QCD DoubleEMEnriched samples we could look up the the dataset on das and create the file dict_QCD_DoubleEMEnriched.py which contains
>
>
If we wanted to include QCD DoubleEMEnriched samples we could look up the the dataset on das and create the file dict_QCD_DoubleEMEnriched.py which we must edit to read
 
flist = [
[1, ['/QCD_Pt-40toInf_DoubleEMEnriched_MGG-80toInf_TuneCUETP8M1_13TeV_Pythia8/RunIISummer16MiniAODv3-PUMoriond17_94X_mcRun2_asymptotic_v3-v2/MINIAODSIM'] , []],
Line: 70 to 73
 
python get_py.py -d QCD_DoubleEMEnriched_dict.py -o Summer16v3 -p
Changed:
<
<
Create new dictionary file for submitJobs.py to read. You'll need to be in the /TreeMaker/Production/test/condor/Sub directory for this. The script that makes the new dictionary file is makeDicts.py. If we wanted to run over the QCD_Pt samples, which now will include the DoubleEMEnriched samples, we would use
>
>
This adds the file list for those samples to the Summer16v3 era.

Update dictionary for submission to Condor

Now that the file list for the samples we want have been added to the desired era directory we need to update the dictionary that will be read by submitJobs.py. We'll need to be in the /TreeMaker/Production/test/condorSub directory for this. The script that makes/overwrites the new dictionary file is makeDicts.py. If we wanted to run over the QCD_Pt samples, which now will include the DoubleEMEnriched samples, we would use
 
python makeDicts.py -c qcd_pt -e Summer16v3 -l Summer16v3/ -S Summer16v3

The -S specifies that this is a background MC. We would use -s if this were a signal MC.

Added:
>
>
The -c qcd_pt option is specifying what "category" we want. In this case we are asking for filelists which contain the pattern "QCD_Pt" in the base name. Below is the current list of arguments and the pattern(s) associated with each.
 "diboson" : ('WGJets','WW','WZ','ZG','ZZ'),
    "dyjets"  : ('DYJetsToLL',),
    "gjets"   : ('GJets',),
    "qcd"     : ('QCD_HT',),
    "qcd_pt"  : ('QCD_Pt',),
    "signal"  : ('SMS','RPV','StealthSYY','StealthSHH'),
    "singlet" : ('ST_s-channel','ST_t-channel','ST_tW','tZq'),
    "ttbar"   : ('TTJets','TT_Tune','TTTo2L2Nu','TTToSemiLeptonic','TTToHadronic'),
    "tth"     : ('TTZTo','TTWJets','TTGJets','TTGamma','TTHH','TTTT','TTTW','TTWH','TTWW','TTWZ','TTZH','TTZZ','ttHJet'),
    "wjets"   : ('WJetsToLNu',),
    "zjets"   : ('ZJetsToNuNu',)

Now that we have an updated dictionary we can submit to Condor with the following command.

python submitJobs.py -d Summer16v3_qcd_pt.py -o root://cmseos.fnal.gov//store/group/lpcsusyphotons/TreeMakerTest -p -s
 
Changed:
<
<

Setting up for submitting TreeMaker jobs to Condor

>
>
The -o argument specifies where to write the .root files.
  -- MatthewLawrenceJoyce - 2019-07-25

Revision 12019-07-26 - MatthewLawrenceJoyce

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="TWiki.WebPreferences"
This document is intended as a guide to making TreeMaker ntuples via job submission to Condor. It is assumed that you already have a Grid certificate and access to LPC.

Getting your area configured on LPC

Start by logging into LPC and navigating to where you would like to place your CMSSW directory. If you already have a version of CMSSW that you would like to use, then you can just navigate to the directory just above it. This one was done with CMSSW_10_2_12.

wget https://raw.githubusercontent.com/TreeMaker/TreeMaker/Run2_2017/setup.sh
chmod +x setup.sh
./setup.sh -c CMSSW_10_2_12 -a https -j 1
cd CMSSW_10_2_12/src/
export SSL_CERT_DIR='/etc/pki/tls/certs:/etc/grid-security/certificates'

Very basic description of structure

The script that runs everything as far as submitting to Condor is submitJobs.py in the /TreeMaker/Production/test/condorSub directory. This script actually uses submitJobsTM.py to submit the jobs. You have to give it a dictionary file that tells it which scenario and what dataset(s) you want to process. Below is an example of one of these dictionary files, dict_Summer16v3_gjets.py

flist = {
    "scenario": "Summer16v3",
    "samples": [
        ['Summer16v3.GJets_HT-100To200_TuneCUETP8M1_13TeV-madgraphMLM-pythia8'],
        ['Summer16v3.GJets_HT-100To200_TuneCUETP8M1_13TeV-madgraphMLM-pythia8_ext1'],
        ['Summer16v3.GJets_HT-200To400_TuneCUETP8M1_13TeV-madgraphMLM-pythia8'],
        ['Summer16v3.GJets_HT-200To400_TuneCUETP8M1_13TeV-madgraphMLM-pythia8_ext1'],
        ['Summer16v3.GJets_HT-400To600_TuneCUETP8M1_13TeV-madgraphMLM-pythia8'],
        ['Summer16v3.GJets_HT-400To600_TuneCUETP8M1_13TeV-madgraphMLM-pythia8_ext1'],
        ['Summer16v3.GJets_HT-600ToInf_TuneCUETP8M1_13TeV-madgraphMLM-pythia8'],
        ['Summer16v3.GJets_HT-600ToInf_TuneCUETP8M1_13TeV-madgraphMLM-pythia8_ext1'],
        ['Summer16v3.GJets_DR-0p4_HT-100To200_TuneCUETP8M1_13TeV-madgraphMLM-pythia8'],
        ['Summer16v3.GJets_DR-0p4_HT-200To400_TuneCUETP8M1_13TeV-madgraphMLM-pythia8'],
        ['Summer16v3.GJets_DR-0p4_HT-400To600_TuneCUETP8M1_13TeV-madgraphMLM-pythia8'],
        ['Summer16v3.GJets_DR-0p4_HT-600ToInf_TuneCUETP8M1_13TeV-madgraphMLM-pythia8'],
    ]
}

This tells the script where to look to get the list of file paths for the root files you want it to process. The file paths are stored in yet another dictionary file located in /TreeMaker/Production/python/<name_of_scenario>/<name_of_sample>_cff.py. The line Summer16v3.GJets_HT-100To200_TuneCUETP8M1_13TeV-madgraphMLM-pythia8 is basically specifying that the list of file paths is found at /TreeMaker/Production/python/Summer16v3/GJets_HT-100To200_TuneCUETP8M1_13TeV-madgraphMLM-pythia8_cff.py.

To run this over the Summer16v3_gjets samples I used

python submitJobs.py -d Summer16v3_gjets.py -o root://cmseos.fnal.gov//store/group/lpcsusyphotons/TreeMakerTest -p -s

Adding new samples

If you want to add samples that are not already included, you can do this in two steps:

Add file list to the desired era directory /TreeMaker/Production/python/<name_of_era>

Perhaps there is a better way but I did this manually in the following way. Create a new file dict_<whatever_name_you_prefer>.py and manually enter the information in the following format in the /TreeMaker/Production/python directory:

# format for dict entries:
    #                                                data: [True,  ['sample'] , []]
    #                                                  MC: [False, ['sample'] , []]
    #                               MC w/ extended sample: [False, ['sample','sample_ext'] , []]
    #                   MC w/ negative weights (amcatnlo): [False, ['sample'] , [neff]]
    # MC w/ negative weights (amcatnlo) + extended sample: [False, ['sample','sample_ext'] , [neff, neff_ext]]

Example: If we wanted to include QCD DoubleEMEnriched samples we could look up the the dataset on das and create the file dict_QCD_DoubleEMEnriched.py which contains

flist = [
[1, ['/QCD_Pt-40toInf_DoubleEMEnriched_MGG-80toInf_TuneCUETP8M1_13TeV_Pythia8/RunIISummer16MiniAODv3-PUMoriond17_94X_mcRun2_asymptotic_v3-v2/MINIAODSIM'] , []],
[1, ['/QCD_Pt-30to40_DoubleEMEnriched_MGG-80toInf_TuneCUETP8M1_13TeV_Pythia8/RunIISummer16MiniAODv3-PUMoriond17_94X_mcRun2_asymptotic_v3-v2/MINIAODSIM'], []]
]

Then we would type the command

python get_py.py -d QCD_DoubleEMEnriched_dict.py -o Summer16v3 -p

Create new dictionary file for submitJobs.py to read. You'll need to be in the /TreeMaker/Production/test/condor/Sub directory for this. The script that makes the new dictionary file is makeDicts.py. If we wanted to run over the QCD_Pt samples, which now will include the DoubleEMEnriched samples, we would use

python makeDicts.py -c qcd_pt -e Summer16v3 -l Summer16v3/ -S Summer16v3

The -S specifies that this is a background MC. We would use -s if this were a signal MC.

Setting up for submitting TreeMaker jobs to Condor

-- MatthewLawrenceJoyce - 2019-07-25

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback