ATLASJobs

Introduction

In this twiki we summarize the behavior of some "typical" ATLAS jobs. In this twiki ComputingDocuments you can find the latest computing documents (twiki only for ATLAS people)

Summary

Dashboard

ATLAS
CPU times per event [10 HS06 core] 01.08.2014 01.08.2015 data average
MC SIM 270
FAST SIM 62
DATA RECO 8.2
SIM RECO 41.43
ANALYSIS 10.05
Data IN per event [kB]
MC SIM 856.89
MC RECO 11771.04
DATA RECO 2639.33
ANALYSIS 1333.43
Data OUT per event [kB]
MC SIM 690.1
MC RECO 698.73
DATA RECO 1505
ANALYSIS 21.83

MC Simulation Production

Simulation-Flow.png

  • Values for a typical job (MC15 13TeV ttbar)

CPU times per event [10 HS06 core]
MC SIM FULL 450
MC FAST SIM 55
Digi+reco (pile) 100
MC evgen 0.1 - 10000
Data IN per event [kB]
MC SIM 200
Digi+reco (pile) 16000
Data OUT per event [kB]
MC SIM FULL 1000
MC FAST SIM 1000
Digi+reco (pile) 500
MC evgen 25

Detailed Overview

Dashboard

CPU

The CPU-distribution over 1 year:


hepcpec_CPU_jobtype_1year.png1year_CPU_jobtype_alljob.png

These pie-charts are taken from the dashboard, showing the CPU utilisation by JobType in absolute and relative numbers. For the purpose of making the CPU time comparable, the numbers of the left Plot have been normalized with the HEPSPEC06 benchmark and are in hours.

Breaking this down to the events, it leads to:

1year_jobtypes_avgCPU.png

Simulation: 270,30 [s/evt]

Reconstruction: 40.02 [s/evt]

Analysis: 10,05 [s/evt]

Simulation

It is not very significant to look only at the average, these categories are split further, starting with the MC simulation (the following numbers do not take into account the power of the CPU):

MC15_13TeV (Full Sim): 249 [s/evt]

MC15_13TeV (Fast Sim): 62 [s/evt]

MC15_13TeV (MC Simulation): 48 [s/evt]

Analysis

Analysis can be split further by input datatype in AOD and DAOD, resulting in:

AOD: average CPU time spent on one good event 7.8139 s/evt (0.000915651 hepspecH/evt)

DAOD: average CPU time spent on one good event 0.0619 s/evt (0.000397506 hepspecH/evt)

Data In- and Output

The dashboard provides the size of the input and output (processed data and produced data):

processed_1year_jobtype.pngproduced_1year_jobtype.png

and number of processed events, which leads to:

Simulation: 856.89 [kB/evt] processed data, 690.10 [kB/evt] produced data

Reconstruction: 10030.66 [kB/evt] processed data, 937.03 [kB/evt] produced data

Analysis: 1333.43 [kB/evt] processed data, 21.83 [kB/evt] produced data

  • AOD: 516kB/evt processed data... 4,07kB/evt produced data
  • DAOD: 250kB/evt processed data... 2.11kB/evt produced data

Open Questions

Why is Data IN MC sim so big?

How can "data OUT" MC SIM be so much bigger than "data IN" MC RECO? shouldnt it be the same?

Difference between CPU[s] and CPU[HEPSPEC06H] seems wrong (see: CPU Analysis AOD vs DAOD - factor of 2.3 and factor of 0.04, where 7-10 is expected), that is why often the HEPSPEC information is not used - although it should be better for comparison.

What are the Bandwidth requirement [bytes/sec] for jobs?

What is the memory consumption [bytes] of a typical (MC,RECO,ANALY) job?

What are Job efficiencies [%] and sensitivity to latency on efficiency (for remote data access?

Are there jobs that have more than 1 concurrent data access?

Major updates:
-- AleDiGGi - 2015-10-02

Fill the page:
-- Gerhard Rzehorz - 2015-10-06

Responsible: AleDiGGi
Last reviewed by: Never reviewed

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng Screenshot-3.png r1 manage 32.7 K 2015-10-06 - 11:13 UnknownUser  
PNGpng Simulation-Flow.png r2 r1 manage 60.9 K 2015-11-02 - 14:43 JoseEnriqueGarcia  
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r8 - 2015-11-04 - JoseEnriqueGarcia
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Atlas All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback