-- SteffenHenkelmann - 2017-05-06

You are interested in an analysis and you want to request one for multiple Monte Carlo samples? This is the right place to find all the information that is required to do exactly this - Placing a Monte Carlo request.

You want to request derivations from existing Monte Carlo samples? Please follow the information provided here.

Notification:

Please note that DSIDs are not allocated to you until after you have developed your MC generation procedure, produced validation plots, presented your plots in a subgroup meeting, and filled out a complete JIRA request (including all the previous information). Please use dummy DSIDs until they are allocated to you.

This twiki page was prepared compiling information that was partially or fully provided on other twiki pages and are linked where needed. The request procedures are mainly inspired by the ATLASProductionGroup and ExoticsMCRequestHowTo twiki's.

General Information

ATLAS Monte Carlo production system

The ATLAS MC production system is a highly automated system managing the production of tasks on remote computers. The production procedures aim to reduce the number of syntax and run-time errors during MC production. The Monte Carlo production chain is split into production steps:

  • Event generation (evgen),
  • Full Geant-4 simulation followed by digitization (simul), or fast AFII simulation,
  • Reconstruction (recon). This last step produces xAODs for analysis.
Each step uses a job option transform. Typically, the requester only cares about the evgen step, and just accepts the default production for simul and recon. If this is not the case, such as requesting re-digitiation for example, the requester should be very specific about which production tags are required and that existing evgen files (by specifying the tags) can be used. This information is specified in the request spreadsheet as described below.

xAODs are automatically saved, but saving different formats, for example ESDs, need to be specifically requested elsewhere (ask Takuya and Danilo). In addition, specific deviations of xAODs are requested elsewhere.

Placing a MC sample request

In general, a Monte Carlo request is broken down into the following phases to allow for an efficient workflow:

  1. Request preparation (constitutes the most important phase)
  2. JIRA ticket phase
  3. Official MC request compilation
  4. Top group convener handshake
  5. Official MC request
  6. PMG approval
  7. Official production

Request preparation

The preparation of the MC request constitutes the most important phase of a Monte Carlo request and is purely the duty of the analyzer(s), referred to as the requester in the following. The top Physics/MC production contacts (TopMC contacts) can provide guidance and assistance during that phase and can be contacted through this mailing list atlas-phys-top-mc-contacts. The PMG recommended procedure is outlined here.

Make your case as a requester

Try to answer the question: Why is the sample needed?

  • Check if samples suitable for your analysis already exist ( MC15 | MC16)
  • What type of Monte Carlo request suits your case?
    • Is it an extension of a Monte Carlo that already exists? [EVNT exists but no simulation]
    • Is it an alternative simulation model [AFII exists but FS does not, or vise versa]
    • Is it a systematic variation of an existing sample?
    • Is it a new sample?
  • What is the total number of statistics that you intend to request?
    • Justify the number of statistics that you intend to request by providing a valuable and thoughtful measure (where applicable)
    • Needs to be justified independent of the type of the Monte Carlo request
  • What type of simulation do you want to request?
    • Do you really need expensive FullSim samples? Why? Might AFII be sufficient? Do you need a detector simulated sample at all?
You are always encouraged to contact the Top MC contacts in case of questions or doubt via e-mail (as stated above). Depending on the outcome of these questions the requester starts the JO preparation and validation process underlining the case.

As a first starting point it is always good to start from something that already exists and works.

The requester has to

  • Prepare the JobOption (s) file (JO) and if needed the LHE files (for non-OTF ["On the fly production"])
    • In case of Sherpa samples, grid-packs need to be provided centrally (please contact Top MC contacts
Job Option Preparation

The job options should be named by the designation:

MC15.<DSID>.<GENERATOR>_<TUNE>_<PROCESS>.py

As described here.

Running job with prepared Job Options

Please, check the latest caches for AtlasProduction and MCProd before starting a test job here and check that a certain release is not blacklisted. Depending on the type of the MC request, identical caches to the existing samples could be used (check with TopMC contact on a case by case basis).

Validation Step

The requirements for a validation step are outlined here.

Sample requests are required to include, at a minimum:

  • Validation plots. The exact set of plots will depend on the physics process and generator, but you are expected to demonstrate that your job runs, and produces sensible results. For some samples, validation plots may not be appropriate, and in these cases you are expected to provide a justification for the absence of plots.
  • Log files from test jobs, or ideally the validation runs. This will demonstrate that the jobs work, and will help if technical problems appear later.

The Checklist

  • Check if samples suitable for your analysis already exist ( MC15 | MC16)
  • Check MC request type ()
example link

FAQ

More information available here.

  • Q. How long does it take to get my sample processed?
    • A : One week to one month depending on grid occupancy etc.
  • Q : How to interpret production tags?
    • A : The new naming convention consists in adding to the sample name a number of tags, one for each production step. The most common tags are:
      • eXXX: evgen configuration
      • sXXX: simulation configuration
      • dXXX: digitization configuration
      • rXXX: reconstruction configuration
      • aXXX: atlfast configuration (both simulation and digit/recon)
      • tXXX: tag production configuration
      • bXXX: bytestream production configuration
  • Q : What are the production tags used for?
    • A : Each of the tags corresponds to a configuration (athena release, geometry tag, JOs set, ...) in a way that the configuration is unique.
  • Q. How do I find the cross-sections in AMI?
    • A : Go to dataset selection page ("simple search") in AMI. Search for an EVNT dataset like mc15_13TeV.410000.PowhegPythiaEvtGen_P2012_ttbar_hdamp172p5_nonallhad.evgen.EVNT.e4398. Click on 'details' in the first column of the returned table: On the right side you will see the cross-section info supplied by AMI as well as (possibily) a text field with details on the LO and NLO cross-sections and filtering efficiency.
  • Q. How to deal with private production?
    • A : Although for very initial studies (e.g. to determine parameters in the model you wish to study) it is ok to run private simulation however you want to, in order to show results outside the ATLAS collaboration (publication , conference etc) there are strict rules on how private simulation is done. All event generation must be done in the production system - private event generation might be allowed under exceptional circumstances. Detector simulation can be done privately, but official production job transforms must be used. This private production should not be undertaken until a sample of 10k detector level simulated events is made and compared to a 10k sample from the production system. Once validated further private production is allowed. Note that private production means private CPU and DISK space. If you'll consider to produce samples privately for the official analysis (it means analysis for CONF notes, papers etc, which will be public), please contact us (atlas-phys-mcprod-team@cernNOSPAMPLEASE.ch).
Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2017-05-06 - SteffenHenkelmann
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback