-- MonicaDobre - 2014-11-05

DerivationFrameworkRTT

Introduction

Currently, there are around 60 derivations defined and over 100 are expected for Run2. In order to assess the impact of the changes in the DerivationFramework software on each of them, a Run Time Tester (RTT) setup was developed in a separate package, DerivationFrameworkRTT, residing in the AtlasOffline area under PhysicsAnalysis /DerivationFramework. The purpose of the RTT test is to check if a particular derivation is produced in a particular Data or Monte Carlo stream, if the merging is successful and to provide information about the memory and CPU consumption for producing a particular derivation.

The Setup of the RTT Test

Currently, the test checks the production of over 55 derivations in four streams: Data Egamma, Data Muons, Data Jet Tau Etmiss and Monte Carlo. The list of the derivations is below. The files used as input are 8 GeV xAOD files with 5000 events each and can be found in /eos/atlas/user/m/mdobre/forRTTx, where x stands for the name of each stream. The DAOD files produced are then used as input for the merging command. In case a particular derivation is not produced, the merging command takes as input one of the xAOD files used to produce the respective DAOD file. The merging is tested running the AODMerge_tf.py command on 5 events from the DAOD file produced.

The following formats are tested:

  • EGAM1, EGAM2, EGAM3, EGAM4, EGAM5
  • EXOT0, EXOT1, EXOT2, EXOT3, EXOT4, EXOT5, EXOT6, EXOT7, EXOT8, EXOT9, EXOT10, EXOT11, EXOT12, EXOT13
  • FTAG1, FTAG2, FTAG3, FTAG4
  • HIGG1D1, HIGG2D1, HIGG2D2, HIGG2D3, HIGG2D4, HIGG2D5, HIGG3D1, HIGG4D1, HIGG4D2, HIGG5D1, HIGG5D2, HIGG8D1, HIGG8D2
  • IDTR1
  • JETM1, JETM2, JETM3, JETM4, JETM5, JETM6, JETM7
  • STDM1, STDM2, STDM3, STDM4, STDM5
  • SUSY1, SUSY2, SUSY3, SUSY4, SUSY5
  • TAUP1, TAUP3
  • TOPQ1, TOPQ2
The files used as input for the derivations come from the following sets:
  • Monte Carlo: mc14_8TeV.117050.PowhegPythia_P2011C_ttbar.merge.AOD.e1727_s1933_s1911_r5591_r5625
  • DataEgamma: data12_8TeV.00203336.physics_Egamma.merge.AOD.r5723_p1751
  • DataMuons: data12_8TeV.00204025.physics_Muons.merge.AOD.r5724_p1751
  • DataJetTauMet: data12_8TeV.00203934.physics_JetTauEtmiss.merge.AOD.r5723_p1751

Where Are the Results?

The RTT test runs every day in the 19.1.X.Y-VAL, AtlasDerivation nightly. In order to check the outcome of a particular nightly, visit the ATLAS RTT results web page, https://atlas-rtt.cern.ch/, select the DerivationFrameworkRTT package and then press the button called "Show me results". Select the release of interest and search for the desired derivation. Keep in mind that, once a release is built (the building starts around 6 a.m.), it takes several hours for the RTT test to run. The files of interest are the DAOD, log.AODtoDAOD, log.AODMerge and the text files containing "Summary" in the name.

The first thing to check is the existence of the DAOD file, which is usually listed last on the right side column and which is typically saved on /eos. If it doesn't exist, check the log.AODtoDAOD file for errors. In case the DAOD file was successfully produced, check the files having "Summary" in the name for information about CPU and memory consumption, skimming and size fractions from the input xAOD file, warnings and errors. Please contact the train coordinators in case of issues with merging or large resource usage. With the exception of the size information, which refers to the size of the derivation on disk, all the other information in the Summary files is taken from the log.AODtoDAOD.

Caveats

The files used for testing the derivation are 8 GeV samples. Eventually, the 13/14 GeV samples will be added to or they will replace the current samples. The group contacts will be notified by this change either in a train coordination meeting or by e-mail.

The test is conceived to be as general as possible and it is only a technical one. It will not replace physics validation, so check the content of the preferred derivation to make sure that everything needed is in.

A successful merging doesn't always imply that the merging is successful for a particular derivation. In case of success, always look in the log.AODMerge file produced if the proper input file was used, i.e., DAOD_{derivation name}.{stream name}.pool.root. If the derivation was not produced, the merging test is not relevant.

Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r3 - 2014-12-04 - MonicaDobre
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback