RIVETTopTutorial

This twiki page serves as a short tutorial on how to run and setup a RIVET routine. The information provided on this twiki page was compiled using different twiki's such as the previous RivetTTbarMCValidation (by Andrea Knue), RivetForAtlas, RivetMCValidation ]. A great tutorial on RIVET can be found here (5th of May 2017).

Get the code

A RIVET framework that contains most of the information that was collected from the various sources and ATLAS twiki pages was compiled and serves as a quick starter for you to run RIVET within athena.

Login into lxplus and checkout

git clone ssh://git@gitlab.cern.ch:7999/sthenkel/RIVETframework.git

In case you have not done so already, go to the git repository webpage and create an ssh key by following the instructions on the respective gitlab page. Once you checked out the package successfully, you will find README's in most of the (sub-)directories. To setup athena and the inherent RIVET version, do

setupATLAS
asetup 21.6.3,AthGeneration,here
source ${LCG_RELEASE_BASE}/LCG_88/MCGenerators/rivet/${RIVETVER}/${LCG_PLATFORM}/rivetenv.sh

Now, navigate to

cd routines/TopRoutines/MC_TTbar_TestSel

Comment: In general a RIVET routine has the following structure with the break down of files:

• ANALYSIS_NAME.cc: This is the C++ file that runs the analysis over input files
• ANALYSIS_NAME.info: This file provides basic info about the analysis
• ANALYSIS_NAME.yoda: This file supplies the reference histograms (typically results from data) of an analysis to compare with Rivet results. Generally this file is also necessary for the ANALYSIS_NAME.cc file to use as reference in building the Rivet histograms, defining bin edges, number of bins, etc.
• RivetMC_NAME.so: This file will be built (or rebuilt) by Rivet when you compile the analysis
• ANALYSIS_NAME.plot: This optional file typically supplies plot cosmetics for the output, such as axis labels and ranges.
• A *.py file: A python steering file is necessary to run the Rivet analysis through ATHENA.

Compile a RIVET routine

The next step is to compile the test routine using:

rivet-buildplugin RivetMC_MC_TTbar_TestSel.so MC_TTbar_TestSel.cc 

The compilation creates or re-creates the RivetMC_NAME.so file in the current directory, building the analysis into the Rivet analysis plugin library. To run on the RIVET routine, navigate back to the main folder and enter the /run/local directory.

NB: You can choose between different running options, either run a certain RIVET release (that you set with the quickSetup.sh execution in the parent directory or $RIVETBASE) locally, on lxbatch or on the grid. The way it is recommended is to run locally and a small test sample for debugging purposes and then to run on the full EVNT statistics (or smaller fractions) on the grid. To get started and run locally, you first need to set a softlink to the compiled routine (<routine>.so) you want to run on.

source quickGridSetup.sh

and download a test file for top quark pair production using:

rucio download --nrandom 1 mc15_13TeV.410501.PowhegPythia8EvtGen_A14_ttbar_hdamp258p75_nonallhad.evgen.EVNT.e5458

Run the Rivet Analysis Locally

In order to run the analysis, set the environment variable to the directory where the *.so file is located, and the data path to the directory where the reference data *.yoda file is located (you must do this for each new login).

source prepareRunning.sh MC_TTbar_TestSel

Open the RunRivet.py and change the InputFolder directory to the folder you downloaded above:

/afs/<path>/mc15_13TeV.410501.PowhegPythia8EvtGen_A14_ttbar_hdamp258p75_nonallhad.evgen.EVNT.e5458/

and choose the analysis you want to run over:

rivet.Analyses += [ 'MC_TTbar_TruthSel' ]

if it takes too long to download the file. Just leave the directory that is already in the RunRivet.py. It will run over a test file in a public area.

Run the analysis with:

athena.py RunRivet.py

[NB: It is also possible to run multiple routines at the same time. However, a slightly different setup is needed. To be able to do this the all *.so files and *.yoda files (and potential steerings) need to be located in one folder and the global variables set in prepareRunning.sh need to be adjusted correspondingly. Additional analyses can then simply be added to the rivet.Analysis instance above. Check with

rivet --show-analysis <Analysis 1> <Analysis 2>

if the analyses are recongnized.]

This will create a file of the format Output_test.yoda, containing the output histogram information from Rivet.

5. Make First Plots with Rivet

The results from a Rivet analysis are stored in the Output_test.yoda file, from which plots can be built using the mkhtml command or the mkplot command. Change to /MakePlots and execute:

For example, to use the mkhtml command:

rivet-mkhtml ../Output_test.yoda:'Title=TestPlot':'LineWidth=0.06':'LineStyle=solid'

This will create a folder called "plots" with the plots in them (and an html file, for your convenience). If your directory includes an ANALYSIS_NAME.yoda file and the RIVET_DATA_PATH is defined as above, the output yoda file will plot histograms compared to the contents of ANALYSIS_NAME.yoda.

6. Build Your Analysis

To build your own analysis, the easiest way is to check out a validated analysis (or several) and then edit them to your analysis specifications. The following example analyses can be useful. A common selection of routines also used for MC comparisons and tuning efforts can be checked out at:

svn co svn+ssh://svn.cern.ch/reps/atlasphys-top/Physics/Top/Software/MCvalidation/Rivet/Rivet2.X/trunk/routines [[TopRoutines][TopRoutines]] 

Available Validated Analyses as Examples

• MC_TTbar_TruthSel: makes a simple ttbar selection and reconstructs the W-bosons and the top-quarks, available for Semileptonic,Dileptonic and Allhadronic channel
• MC_SingleTop_Truth:
• MC_ttH_TruthSel: makes a simple ttH (H -> bb) selection for the lepton+jets channel
• MC_Zt_Truth:
• MC_fragmentation:
• ATLAS_nosel_MCval
• ATLAS_2014_I1304289
• ATLAS_2015_I1404878_Parton

7. Run Your Rivet Analysis on the Grid

After you can successfully test your Rivet analysis locally, you may want to run over a larger number of events on the grid. To do this, change into the directory /run/grid/grid. When you start the grid submission, all files in the directory are transformed into a tar ball and send to the grid. Hence, it is imported that all the files that you need to run the job are send alongside. Make sure that you copy all the files (including the shared library in the /run/grid/grid folder)

cp $RIVETBASE/MC_TTbar_TruthSel/* .

Cross check, if the .yoda file and the .so file are included in the prepare_gridsubmission.py under filesToInclude list.

The script takes the datasets and corresponding Xsec's into account (predefined for you in this tutorial).

This script takes the JO_rivet_withXsec.py defined in JOs/ and replaces everything corresponding to what is given defined in the dictionary in the preparation script (prepare_gridsubmission.py).

It also writes out a download script to download the output for you on a later stage.

Additionally, the preparation script will result in a couple of adjusted JOs for each sample. A submission script with the name Final_RIVETtut.sh. This simple bash script contains the pre-defined pathena command

8. Validate Your Rivet Analysis

To check how well your AnalysisTop /ROOT Analysis matches your Rivet analysis, you can do closure tests by running over the same MC files with both analyses. However, Rivet runs over EVNT files, not NTUP_COMMON DAODs such as in your analysis. Therefore it's necessary to use an EVNT file for Rivet and the exactly corresponding NTUP_COMMON DAOD for the AnalysisTop /ROOT analysis. These corresponding files are stored on EOS:

8 TeV Files

For 8 TeV analyses, the EVENT and DAOD files are at:

/eos/atlas/atlascerngroupdisk/phys-top/topreco/MCValidation/SamplesForRivetValidation/

This directory holds both the EVNT and the DAOD files for comparison of Rivet with your AnalysisTop analysis. For example, EVNT.01529377._000001.pool.root.1 should contain the same events as NTUP_COMMON_250-ev_110404_000001.pool.root.

13 TeV Files

At 13 TeV, the event and run numbers are not conserved between the EVNT file and the final DAOD in the ATLAS production system, so we prepared an EVNT file and a DAOD file with events in exactly the same order, so that you can check event by event if the cuts have the same effect and the objects are the same. For different channels we have different validation files:

• lepton+jets resolved (mc15b):
  • go to /eos/atlas/atlascerngroupdisk/phys-top/topreco/MCValidation/SamplesForRivetValidation_13TeV/LeptonPlusJets_mc15b and compare the rivet output of the EVNT.pool.root file with the numbers you get when you are running your actual analysis code over list_Derivation_ordered.txt
• dilepton resolved (mc15b):
• allhadronic boosted (mc15c):

Major updates:
-- Main.AndreaKnue - 21 Oct 2014

Andrea Knue

%REVIEW% Never reviewed