Qualification Task for Wei Ding

This is the summary of qulification task for Wei Ding : MG5_aMC technical validation, and there is the JIRA link.

1. Introduction

Background: There are many modes of operation of MadGraph5_aMC@NLO MC generator - LO, NLO, LO-merged, NLO-merged, MadSpin, gridpacks, multicore, loop induced, external models, and more. Some of these are included in the standard MC validation, however several are not and in general these need more scrutiny. This task will address this issue by developing a thorough MG5_aMC validation procedure.

Task description: The task will involve developing a comprehensive validation procedure for the MadGraph5_aMC@NLO MC generator, for a wide range of operating modes. Specifically: LO, NLO, LO-merged, NLO-merged, LHAPDF, MadSpin, gridpacks, multicore, loop induced, external models, and any others that are deemed necessary during the task.

  • For all these running modes jobOptions would need to be prepared and run. The current validation includes ttbar and Z->mumu production, new jobOptions need to be written and technically validated for all modes and could be extended to other processes - e.g. diboson.
  • Validate the various modes in the latest MG5_aMC version against the most recently ATLAS-validated version, e.g. 19.2.5.28.3 MG5_aMC_v2.6.0 vs 19.2.5.29 MG5_aMC_v2.3.3.
  • Where issues are found interaction in the ATLAS MG5_aMC interface interaction with the ATLAS MadGraphControl experts and likely would be required to fix the problems on the ATLAS side
  • If there are issues in the MG5_aMC code we can interact with the authors e.g. Stefano Frixione, who has an ACE scheme with ATLAS.
  • The final outcome should be one fully validated MG5_aMC version and a well defined procedure to validate future versions for all modes of operation.

There is glance link.

There are mainly three steps involved in validation:

  • Test running modes locally.
  • MC generation on grid.
  • JEM (Job Execution Monitor) validation.

2. Running modes

JobOptions Process LO/NLO LHAPDF MadSpin Multileg merging gridpack MCORE
MC15.429200.MadGraphPythia8EvtGen_VALID_ttbar.py ttbar LO No No No No No
MC15.429201.MadGraphPythia8EvtGen_VALID_ttbar_LHAPDFTest.py ttbar LO DONE No No No No
MC15.429202.MadGraphPythia8EvtGen_VALID_ttbar_MadSpinTest.py ttbar LO DONE DONE No No No
MC15.429203.MadGraphPythia8EvtGen_VALID_ttbar_gridpackTest.py ttbar LO DONE DONE No DONE No
MC15.429204.MadGraphPythia8EvtGen_VALID_ttbar_MCOREtest.py ttbar LO No No No No DONE
MC15.429207.MadGraphPythia8EvtGen_VALID_ttbar_CKKWLMerge_Np01.py ttbar LO DONE No DONE No No
MC15.429208.MadGraphPythia8EvtGen_VALID_Zmumu.py Zmumu LO No No No No No
MC15.429209.MadGraphPythia8EvtGen_VALID_Zmumu_LHEInputTest.py Zmumu LO No No No No No
MC15.429213.MadGraphPythia8EvtGen_VALID_Zmumu_CKKWLMerge_Np012.py Zmumu LO DONE No No No No
MC15.429214.MadGraphPythia8EvtGen_VALID_Zmumu_CKKWLMerge_Np012_LHE.py Zmumu LO DONE No DONE No No
MC15.429220.aMcAtNloPythia8EvtGen_VALID_ttbar_Incl.py ttbar NLO No No No No No
MC15.429221.aMcAtNloHerwigppEvtGen_VALID_ttbar.py ttbar NLO No No No No No
MC15.429222.aMcAtNloPythia8EvtGen_VALID_ttbar_MadSpinTest.py ttbar NLO DONE DONE No No No
MC15.429223.aMcAtNloPythia8EvtGen_VALID_ttbar_gridpackTest.py ttbar NLO DONE DONE No DONE No

3. Test running modes locally

How to run

First, download the samples on svn.

One should first setup a recent Rel19 release (For the current recommended production releases see here). It is always best practice to start from a setup that you know worked. At the moment, the most recent release 19 caches are a good starting place if you aren't sure.
Also check Release_history to be sure that a given release is suitable for your needs

cd <workdir>
setupATLAS
asetup 19.2.5.28.3,MCProd,here #MG5_aMC_v2.6.0

For LO ttbar gridpack and NLO ttbar gridpack, These two jobOptions can be used to generate both the gridpack and the events. The jobOptions decide which of the two steps to perform based on whether there is an inputGenConfFile argument passed in the Generate_trf.py command or not.

These commands explain how to generate the gridpack and then how to use it to generate events.

To generate the gridpack execute the following command:

 
mkdir run_makeGridpack
cd run_makeGridpack
Generate_tf.py --ecmEnergy=13000. --maxEvents=-1 --runNumber=999999 --firstEvent=1 --randomSeed=123456 --outputEVNTFile=EVNT.root --jobConfig=MC15.999999.aMcAtNlo_ttbar_gridpackTest.py

At the end of the the gridpack generation you will see:

 
"RuntimeError: Gridpack sucessfully created, exiting the transform. IGNORE ERRORS if running gridpack generation!"

This is just because it's hard to exit the Generate_tf.py quietly without running the shower, please ignore!


Then, to generate events using a previously generated gridpack execute the following command:

          
cd ../
mkdir run_generateFromGridpack
cd run_generateFromGridpack
Generate_tf.py --ecmEnergy=13000. --maxEvents=-1 --runNumber=999999 --firstEvent=1 --randomSeed=123456 --outputEVNTFile=EVNT.root --jobConfig=MC15.999999.aMcAtNlo_ttbar_gridpackTest.py --inputGenConfFile=run_01_gridpack.tar.gz 

This will now fail because we are not running the shower.

IMPORTANT FOR OFFICIAL PRODUCTION: If you are using gridpacks as inputs you need to add the following line in your top jobOptions to tell the production system where to check for the integration grid files - evgenConfig.inputconfcheck=""

For LO Zmumu LHE and LO Zmumu CKKWL LHE, one should look for corresponding LHE input files here, and then randomly download one file using rucio, add an inputGeneratorFile argument in the Generate_trf.py command and run:

Generate_tf.py --ecmEnergy=13000. --maxEvents=-1 --runNumber=999999 --firstEvent=1 --randomSeed=123456 --outputEVNTFile=EVNT.root --jobConfig=MC15.MadGraphPythia8EvtGen_Zmumu_LHEInputTest.999999..py ----inputGeneratorFile=TXT.*.tar.gz.1

For other samples, just execute the following command:

 
Generate_tf.py --ecmEnergy=13000. --maxEvents=-1 --runNumber=999999 --firstEvent=1 --randomSeed=123456 --outputEVNTFile=EVNT.root --jobConfig=MC15.999999.MadGraphPythia8EvtGen_sampledescription.py

One more, if there are too many samples you need to test, you can just use a python script to make it like that.

Summary

13 in totally 14 samples succeeded. Sample NLO ttbar Herwig failed in both release:

  • Aborted in EvtGen
  • There are log file for MG5_aMC_v2.6.0 and log file for MG5_aMC_v2.3.3.

For sample LO ttbar gridpack and NLO ttbar gridpack, I wrote a script for automatic gridpack generation, just execute the command:

python MadgraphTarCreator.py --joboption path/to/jobotion

Sample Zmumu LHEInput and Zmumu Merge LHEInput need new LHEFs.

4. MC generation officially

Before officially generation, one should make a spreadsheet, and fill your spreadsheet by looking into the log file where you locally ran, then you can submit the generation job officially (permission required).

There are mc generation requests for MG5_aMC_v2.3.3: request 1, request 2, request 3.

There are mc generation requests for MG5_aMC_v2.6.0: request 1, request 2, request 3.

5. JEM validation MG5_aMC_v2.3.3 vs MG5_aMC_v2.6.0

How to run

When mc generation job finished, you can get the panda ID or EVNT file name on grid from the generation request. Then one can just submit the validation job here, also one will need a acount from NicolasLang, by executing following on jem:

Click *File generation* -> *Creat Request*,enter your dataset by name or ID, then click *Add file* HepMCAnalyzer -> *Submit*
Once file generation finished, click *Validation*->*Creat Request*, enter the reference file and validated file you generated and *Submit*

Summary

In JEM validation plots:

Reference file Blue line Release 19.2.5.29 MG5_aMC_v2.3.3
Reference file Red line Release 19.2.5.29 MG5_aMC_v2.6.0
And there are summary:

DSID Description Validation link Decision
429200 LO ttbar Validation task GOOD
429201 LO ttbar LHAPDF Validation task BAD - eta is different
429202 LO ttbar MadSpin Validation task BAD - eta is different
429203 LO ttbar gridpack Validation task GOOD
429204 LO ttbar MCORE Validation task GOOD
429207 vs 429210 LO ttbar CKKW-L Validation task GOOD with PDGFix
429208 LO Zmumu Validation task GOOD
429209 LO Zmumu LHE input TODO  
429213 vs 429212 LO Zmumu CKKW-L Validation task GOOD with PDGFix
429214 LO Zmumu CKKW-L input LHE TODO  
429220 vs 429230 NLO ttbar Validation task GOOD with fixed shower starting scale
429221 NLO ttbar Herwig TODO  
429222 vs 429231 NLO ttbar MadSpin Validation task GOOD with fixed shower starting scale
429223 NLO ttbar gridpack Validation task GOOD, but should look different due to SSS gridpack

LO ttbar LHAPDF

Some differences observed in e.g. jet1 eta, lepton1 eta, Bottom-hadron eta.

We still didn't work out where this comes from and need to reproduce the issue with standalone MG5_aMC then it needs to be followed up with the MG5_aMC authors.

LO ttbar MadSpin

Some differences also observed in e.g. jet1 eta, lepton1 eta, Bottom-hadron eta.

We still didn't work out where this comes from and need to reproduce the issue with standalone MG5_aMC then it needs to be followed up with the MG5_aMC authors.

6. JEM Validation of different mode in MG5_aMC_v2.6.0

There are summary:

DSID Description Validation link Decision
429200 vs 429201 LO ttbar simple vs LHAPDF Validation task GOOD - note that 2.3.3 vs 2.6.0 in this comparison was different.
429200 vs 429204 LO ttbar simple vs MCORE Validation task GOOD
429200 vs 429203 LO ttbar simple vs gridpack Validation task GOOD but should include MadSpin differences?
429201 vs 429202 LO ttbar LHAPDF vs MadSpin Validation task Can see expected MadSpin differences, but some follow-up required on e.g. w-boson lineshape.
429202 vs 429203 LO ttbar MadSpin vs gridpack Validation task BAD, should be the same but includes MadSpin differences
429230 vs 429231 NLO ttbar simple vs MadSpin Validation task Can see expected MadSpin differences, but some follow-up required on e.g. w-boson lineshape.
429230 vs 429323 NLO ttbar simple vs gridpack Validation task Can see expected MadSpin differences, but some follow-up required on e.g. w-boson lineshape.
429230 vs 429323 NLO ttbar MadSpin vs gridpack Validation task GOOD

LO ttbar gridpack and NLO ttbar gridpack

We finally checked that MadSpin ran successfully in NLO ttbar gridpack but it did not run successfully in LO ttbar gridpack.

LO ttbar MadSpin and NLO ttbar MadSpin

Can see expected MadSpin differences, but some follow-up required on e.g. w-boson lineshape.

LO ttbar LHAPDF (blue) vs LO ttbar Madspin

NLO ttbar (blue) vs NLO ttbar Madspin

Major updates:

Reference : MadGraph5_aMC@NLO for ATLAS,MadGraphControlValidation

-- WeiDing - 2018-09-18

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2018-09-18 - WeiDing
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback