Andrew Wisecarver's Homepage

Quick Note and Quick Links

Oh, hello. It looks like you've arrived at my homepage. Grab a tea and get cozy.

Disclaimer: The info/commands on this Twiki may not be up to date/work properly, so use the information on this page at your own discretion.

Twiki's:

• CSC Validation: https://twiki.cern.ch/twiki/bin/viewauth/CMS/CSCValidationWebsite
• miniCSC Project: https://twiki.cern.ch/twiki/bin/viewauth/CMS/CSCprotoAt904

• CMSSW Workbook: https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBook
• ROOT Forum: https://root-forum.cern.ch/
• CMS Hypernews: https://hypernews.cern.ch/HyperNews/CMS/cindex
• CMS Task Monitoring Dashboard: http://dashb-cms-job.cern.ch/dashboard/templates/task-analysis/

Event Generation

MadGraph5 _aMC@NLO+Pythia8

genProductions repo: https://github.com/cms-sw/genproductions

If questions, consult:

• Generator HN: https://hypernews.cern.ch/HyperNews/CMS/get/generators.html
• Computing Tools HN: https://hypernews.cern.ch/HyperNews/CMS/get/computing-tools.html

Helpful pages:

• Generator FAQ: https://twiki.cern.ch/twiki/bin/view/CMS/GeneratorFAQ
• GEN Tutorial Twiki: https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookGeneration
• CRAB Twiki: https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCrab

Datacards & Gridpacks

Datacard templates are on the genproductions repository (see link above). gridpack_generation.sh is the main shell script to make gridpacks.

W+jets, NLO FxFx:
https://github.com/cms-sw/genproductions/tree/354d938586a1d54e8220f0b84629a81fa72e9b51/bin/MadGraph5_aMCatNLO/cards/production/13TeV/WJToLNu_NLO

Select LHAPDF number, scale settings.

A) If using lxplus use one of the following:

• ./gridpack_generation.sh <name_of_process_card_without_proc_card.dat> <folder_containing_cards_relative_to_current_location> <queue>
• ./submit_gridpack_generation.sh 30000 30000 <queue> <name of process card without _proc_card.dat> relative/path/to/cards/<name of process card without _proc_card.dat> <queue>

which should submit as a batch job, setting up its own CMSSW environment, downloading the appropriate tarballs, etc.

B) If using cmsConnect (need an account first), go to /local-scratch/<username>

• git clone git@githubNOSPAMPLEASE.com:cms-sw/genproductions.git
• voms-proxy-init -voms cms -valid 192:00
• cd /local-scratch/awisecar/genproductions/bin/MadGraph5_aMCatNLO
• nohup ./submit_cmsconnect_gridpack_generation.sh <base name of datacards> relative/path/to/cards/<base name of datacards> 4 "10 Gb" > mysubmit_<base name of datacards>.debug 2>&1 &

On cmsConnect, generation is split up into the CODEGEN step and the INTEGRATE step, so check the two respective logfiles.

LHE and GEN CMSSW Modules

The initial step is to take a .cff file and run cmsDriver.py on it to get a full .cfg file, on which you cmsRun. The attached file has both LHE and Hadronization modules in it, although they can be done separately, in which case you run the Hadronizer on an LHE .root file.

Example cff file for LHE+GEN step:
WJToLNu_2j_5f_NLO_FxFx_as_0108_cff.py.txt

Example cmsDriver command:
cmsDriver.py Configuration/GenProduction/python/WJToLNu_2j_5f_NLO_FxFx_as_0108_cff.py --mc --conditions auto:mc --step LHE,GEN --datatier GEN-SIM,LHE --eventcontent RAWSIM,LHE -n 5000 --no_exec --fileout file:WJToLNu_2j_5f_NLO_FxFx_as_0108_GEN.root --python_filename WJToLNu _2j_5f_NLO_FxFx_as_0108_cfg.py

The "Configuration/GenProduction/python" part of the path is referring to the relative path within the genproductions repository (Configuration/GenProduction just used for historical reasons I think, I don't think this part is important), although for using cmsDriver I think you just need to be in a CMSSW area.

Make sure your JetMatching:nJetMax in the GEN part is set equal to the maximum number of outgoing partons that you generate on the LHE level (if your samples are jet-binned, and you will merge them later, then this number is the highest jet multiplicity).

Also, JetMatching:qCut represents the merging scale. There is much more than can be said on this.

Running CRAB for LHE-only Production using Madgraph5 Gridpacks

CRAB config files have a config.JobType.inputFiles option which allows you to specify input files that are loaded into a "sandbox" that is then available in whatever environment that job is actually run. The caveat, however, is that there is a file size limit, and some gridpacks (e.g. W+2jets) exceed this limit. A possible solution is to have those gridpacks put on cvmfs (e.g. /cvmfs/cms.cern.ch/phys_generator/gridpacks/2017/13TeV/madgraph/V5_2.4.2/wellnuXj_5f_NLO_FXFX_as/) so that they're able to be seen (this may involve setting the config.Site.storageSite and/or config.Site.whitelist parameters to 'T2_CH_CERN').

config.Data.splitting must be set to 'EventBased' if doing MC production (?)

Note: some fuddling with config.JobType.eventsPerLumi and config.Data.unitsPerJob parameters may have to be done so that it runs.

• WJToLNu_2j_5f_NLO_FxFx_as_0108_LHE_cfg_CRAB.py.txt: LHE-only cfg file used in CRAB event production
• crabSubmitLHE_WJToLNu_MG5_NLO_FxFx_2j_as_0108.py.txt: Corresponding CRAB cfg file for submission

Note that the config.Data.outLFNDirBase parameter directs the output location (here on EOS), where getUsernameFromSiteDB() grabs your lxplus username automatically, if you include from UserUtilities import getUsernameFromSiteDB at the top of your CRAB config file.

Once you've got your CRAB cfg and CMSSW cfg files in the same directory, do

• source /cvmfs/cms.cern.ch/crab3/crab.sh
• voms-proxy-init --voms cms
• crab submit -c crabSubmitLHE_WJToLNu_MG5_NLO_FxFx_2j_as_0108.py

can also do:

• crab checkwrite --site=T2_CH_CERN to check your permissions on this site
• crab status and crab report (or just check the CMS Dashboard, link above)
• crab kill (should only have to be done once)
• crab submit --dryrun <CRABcfg.py> can tell you how long the jobs will run for?
• crab resubmit --dir <projects_directory>/crab_<name_of_project> --maxjobruntime=2800: resubmit command resubmits failed jobs

Wall clock exit code 50664:

Obtaining GEN-level information from LHE-level with Pythia 8

If you've done the previous step, now you have ROOT files with LHE-only information, and you need to hadronize and apply other phenomenological corrections to this parton-level information to bring it to particle-level if you want to compare to data/have something physical.

The two scripts attached here are a hadronizer fragment cfg file that hadronizes LHE information to GEN level using Pythia 8, and a python script to loop over different LHE file indices, alpha-s values, doing an lxbatch submission for each of them. The STDOUT is useful, because the post-matching, final cross section Pythia calculates is used at analysis-level. Note: the hadronization parameters in the GEN step cfg file are inherited from the corresponding cff file that we previously ran cmsDriver.py on.

• runGENBatchNLO.py.txt: .py file to do the batch job submission with lxbatch protocol
• WJToLNu_2j_5f_NLO_FxFx_as_0108_GEN_cfg.py.txt: GEN-step cfg file

• readPythiaXsec.py.txt: Extract post-matching xsec from Pythia output

Misc.

Information about weights in the LHEEventProduct: https://twiki.cern.ch/twiki/bin/viewauth/CMS/LHEReaderCMSSW

Sherpa+MCgrid+FastNLO

FastNLO: http://fastnlo.hepforge.org/
MCgrid manual: https://www.hepforge.org/archive/mcgrid/manual-2.0.0.pdf
Sherpa Manual: https://sherpa.hepforge.org/doc/SHERPA-MC-2.2.4.html

Installation

Use gcc 4.8.1:

• export SCRAM_ARCH=slc6_amd64_gcc481
• source /afs/cern.ch/sw/lcg/contrib/gcc/4.8.1/x86_64-slc6-gcc48-opt/setup.sh or
• source /cvmfs/cms.cern.ch/slc6_amd64_gcc481/external/gcc/4.8.1/etc/profile.d/init.sh

Install swig.

Set environment to grab python 2.7 for Rivet's executables (lxplus default is 2.6):

• PYTHONDIR=/afs/cern.ch/sw/lcg/external/Python/2.7.4/x86_64-slc6-gcc48-opt
• PATH=$PYTHONDIR/bin:$PATH
• LD_LIBRARY_PATH=$PYTHONDIR/lib

LaTeX:

• PATH=/afs/cern.ch/sw/XML/texlive/latest/bin/x86_64-linux:$PATH

LHAPDF:

• export LHAPDF_DATA_PATH=/cvmfs/cms.cern.ch/slc6_amd64_gcc481/external/lhapdf/6.2.1/share/LHAPDF

Rivet and MCgrid

Need MCgrid-enabled Rivet analysis that mimics the physics process and selection cuts you want to study. Can book and fill FastNLO grids using MCgrid classes within this Rivet .cc file. Sherpa runcards call Rivet analysis, Rivet analysis calls the FNLO “.str” cards which specify the parton combinations.

MCgrid::fastnloGridArch arch_fnlo(30,20,"Lagrange","Lagrange", "sqrtlog10", "loglog025")

mcgrid/scripts/identifySubprocs.py -- identify relevant partonic subprocesses from Sherpa output

• identifySubprocs.py −−beamtype=[pp/ppbar/pbarp] Process.db

• createFastNLOSteering.py −−beamtype=[pp/ppbar/pbarp] subproc.config

Commands:

• make plugin-fastnlo
• make install
• Sherpa -f Run.dat INIT_ONLY=1 (to initialize ME generators built in Sherpa)
• ./makelibs (if using AMEGIC)
• nohup Sherpa -f Run.dat -e 100k > WJets2017-FixedOrder-Warmup.out 2>&1 &
• nohup Sherpa -f Run.dat -e 5M > WJets2017-FixedOrder-FillGrids.out 2>&1 &

Notes:

• The first run, or phasespace fill run, determines the relative statistics of each partonic channel in the process such that their statistical samples may be combined correctly, and also establishes the boundaries of the x, Q2 phase space for each of the interpolation grids. Grid phasespace parameters after warmup run stored in “phasespace” folder; delete this folder to get FNLO to make another one.
• The second run actually populates the grids with the Monte Carlo weights.
• It should be noted however that the phase space information provided from the first run must be used by all subsequent parallel runs to ensure the correct combination of the final grids. Therefore the phase space run may not be parallelised.

./fnlo-tk-cppread [PDF] [#scalecombs] [ascode] [norm]

Sherpa Runcard Parameters

Process block:
For a process like

Process 93 93 -> 90 91 93{2}
Order (*,2)

the wildcard indicates the number QCD couplings and the 2 indicates the number of EW couplings (i.e. W+jets)

Scale:

• Default scale setting (needed when multijet merging is used):
  SCALES METS{FSF*MU_F2}{RSF*MU_R2}{QSF*MU_Q2}
• Possible scale setting (renorm. and fact. equal), here for 2 QCD partons:
  SCALES FASTJET[A:antikt,PT:10,R:0.4,M:0]{Abs2(p[2]+p[3])+PPerp2(p[4])+PPerp2(p[5])}

Sherpa's ordering of p[i]'s:

• p[2]+p[3] - outgoing leptons
• p[4]+p[5]+... - jets ordered in pt (when jets have been creating using a clustering algorithm)

Notes:

• If in (process) module the generator isnt specified for the specific final state parton multiplicity, it goes to the default to where it tries each of the matrix element generators in the (me) module in order.
• The partonic contribution is not all there is to a cross section calculation for a multi-jet merged event generation, Sudakov rejection weights also contribute. Only after showering do you get rejection weights, and only after this can you compute the total cross section of a multlijet merged sample.
• If multiple runs, then do a wighted average of all of the output cross sections by weighting the events to get an average cross section.
• EVENTS means the number of events produced (after the phase space integration is done)
• ERROR sets an error target; there are some Vegas steps it does first and then it does the integration to try and reach the error target. If ERROR 0.99, we are allowing for 99% error, meaning that it should stop after the Vegas optimization.
  • Integration_Error key can be specified for each partonic process in the (process) part of run card.

FastNLO

• fnlo-tk-cppread d03-x01-y01.tab NNPDF31_nnlo_as_0118
• fnlo-tk-yodaout d03-x01-y01.tab NNPDF31_nnlo_as_0118
• comparison-plot or rivet-mkhtml *.yoda

To grab python packages, cd into CMSSW_7_1_30/src, do a cmsenv, and then source your FNLO setup script again

CSC Validation

SCRAM guide: https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideScram

CSC Validation works as an offline DQM system, running over RAW and FEVT (Full Event) files that contain CSC-specific information, in order to make occupancy, efficiency, timing, etc. plots: https://cms-conddb.cern.ch/eosweb/csc/

Datasets are organized in Express and Prompt. Express sets are available about an hour after run, and are used by DQM people to do quick evaluations. Prompt sets are called ‘prompt’ due to the algorithm that takes further information about the calibration of the detector components, using this to better reconstruct data. This dataset is available about 48 hours after running. The Prompt sets are organized in different physics triggers e.g. DoubleMuon, SingleMuon, ZeroBias. SingleMuon is used by the Data Certification people.

Remember to scram b to compile any changes! Main codes:

• AutoValidation /run_cscval.py
• AutoValidation /generateCron.py
• src/CSCValidation.cc
• Templates/makeGraphs.C
• Templates/makePlots.C
• Templates/makePlots_emtf.C
• Templates/cfg_yesDigis_RAW_template & Templates/cfg_yesDigis_FEVT_template
• Templates/secondStep_yesDigis_template

Running CSC Validation:

• Create tmpfile using voms-proxy-init -hours 500
• Copy to your home directory
• Do voms-proxy-init --voms cms
• Do source /cvmfs/cms.cern.ch/crab3/crab.sh
• ./generateCron.py /absolute/path/to/proxy.file
• acrontab < autoval.acron
• acrontab -l to list cron contents
• acrontab -e to edit cron contents
• acrontab -r to clear cron contents (and stop the jobs)

— or —

• ./run_cscval.py <dataset> <globaltag> (the "first step" - process_dataset+run_validation)

then

• ./run_cscval.py -ro <dataset> <globaltag> (the "second step" - process_output)
• specify run number using "-rn" tag -- ./run_cscval.py -rn <run number> <dataset> <globaltag>

E.g. ./run_cscval.py /Cosmics/Commissioning2017-v1/RAW auto:run2_data

Data is stored here, at least for a short amount of time: /eos/cms/store/data, so if you're trying to run over an old dataset, and there don't appear to be any files, that's probably why. It will take more time for these runs to appear on DAS, however, and only then can you actually run the code.

miniCSC at b904

miniCSC codes: https://github.com/marctost/miniCSC904

Motivation: CSCs contain mixture of 40% Ar, 50% CO2, 10% CF4, but CF4 is not environmentally friendly. However, it provides nice "anti-aging" properties, leading to a long healthy life for our cathode strip chamber.

Idea: Reduce fraction of CF4 and see if/in what way the chamber dies. Monitor several quantities:

• (Pre)trigger rates
• Gas gain/"cluster charge"
• Interstrip resistance
• "Dark" current

Notes:

• Dispatcher: http://emu42fastprod01.cern.ch:9399/urn:xdaq-application:lid=12/Dispatcher
  • Configuration: miniME_on_MEX2.duck
• STEP test result path may depend upon the configuration, but one should look here:
  • /local/data/cscme42/current/Tests_results, browsable at: http://emume42.cern.ch/data/current/Tests_results/
  • /data/cscme42/current/Tests_results, browsable at: http://emume42.cern.ch/data2/current/Tests_results/
• If STEP tests are halted manually, DQM analyzer doesn't run automatically, but one can edit and run ~/cscme42/bin/rerunAnalysis.sh to produce plots, except for "pulsed" tests

Slides:

• 9-CSC_101_KB_20160531.pdf: Karoly's test stand presentation from CSC 101, 2016
• 14-agingCSC101_010516.pdf: Older CSC 101 presentation that outlines GIF-CSC program
• GIF180417.pdf: Recent CMS week presentation of GIF-CSC progress/plans

CSC Detector On-Call

General --

• Call list:
  • General issues -- Misha, Jian
  • XDAQ, configuration -- Karoly
  • DCS -- Evaldas
• RC meeting every morning: https://indico.cern.ch/event/402363/
• Any time the CSC system is interfered with, need to document in an ELOG
• Phone: +41 75 411 1972 (161972 from CERN phone), PIN: 8724

• Firmware & Configuration check every day through YP (both endcaps simultaneously), do after beams are dumped; take local run afterwards to check health of system
• Taking a local run:
  • Ask Shift Leader and DCS Shifter to be given local control of DAQ and DCS
  • Create new "Local/local" configuration on RCMS, then Initialize, Configure, etc.
  • All dependencies (EMTF, TCDS, FED, local DAQ, local DQM) on EMUPage One should respond and become Configured (yellow) and then Enabled/Running (green) once the local run is started
  • Can choose to Halt or Stop the local run; if Stopped, then must Destroy

DCS --

• If power cycling LV and peripheral crates, system has to reconfigure (yellow bar on DCS must appear and finish) before any further use of YP, RCMS, etc.
• Only difference between STANDBY and ON states for CSCs is that HV is lowered by ~600V in all chambers in the former state (?)

DQM --

• Chambers that are turned off: ME-2/1/3, ME-4/2/21
• Anode Hit Occupancy per Wire Group
• SCA Active Strips Occupancy
• Cathode Comparator Hit Occupancy per Half Strip
• L1A Differences

YP --

• YP is how we load firmware onto the individual chambers' EPROMs, check firmware version, etc.
• "Check Crate Controllers" on YP allows us to check if we're communicating with peripheral crates

RCMS Configurations --

• /DQM/DQM
• /DQM/DQM_Display
• /Commissioning/startPCrate-minus
• /Commissioning/startPCrate-plus
• /tools/PC-TStore
• /DAQ/DAQ_writeN_buildN should be local DAQ configuration when beams are on (?)
• /Local/local or /Global/GLOBAL_TCDS (depending if in local or global DAQ)

Lagniappe

Papers I Like:

• Schwartz's Intro to Collider Physics: https://arxiv.org/abs/1709.04533
• Intro to Event Generation: https://arxiv.org/abs/1304.6677

Useful Commands:

• nohup
• grep

If having troubling importing ROOT into python:

• Probably just need to find the correct environment
• Homebrew might have installed it using python3 instead of python2
• Try putting . /usr/local/bin/thisroot.sh in your .bash_profile
• Can run "2to3" on a script to convert it from python2 to python3

-- AndrewLloydWisecarver - 2018-06-24

• 1711.01408.pdf: alpha-s motivation

• PhysRevLett.75.3226.pdf: alpha-s motivation

• 1304.0322.pdf: nice paper on W boson charge asymmetry at LHC and PDF extraction

• CSC-proto-aging.pdf: miniCSC motivation