ATLASJobs

Introduction

In this twiki we summarize the behavior of some "typical" ATLAS jobs. In this twiki ComputingDocuments you can find the latest computing documents (twiki only for ATLAS people)

Summary

Dashboard

MC Simulation Production

• Values for a typical job (MC15 13TeV ttbar)

Detailed Overview

Dashboard

CPU

The CPU-distribution over 1 year:

These pie-charts are taken from the dashboard, showing the CPU utilisation by JobType in absolute and relative numbers. For the purpose of making the CPU time comparable, the numbers of the left Plot have been normalized with the HEPSPEC06 benchmark and are in hours.

Breaking this down to the events, it leads to:

Simulation: 270,30 [s/evt]

Reconstruction: 40.02 [s/evt]

Analysis: 10,05 [s/evt]

Simulation

It is not very significant to look only at the average, these categories are split further, starting with the MC simulation (the following numbers do not take into account the power of the CPU):

MC15_13TeV (Full Sim): 249 [s/evt]

MC15_13TeV (Fast Sim): 62 [s/evt]

MC15_13TeV (MC Simulation): 48 [s/evt]

Analysis

Analysis can be split further by input datatype in AOD and DAOD, resulting in:

AOD: average CPU time spent on one good event 7.8139 s/evt (0.000915651 hepspecH/evt)

DAOD: average CPU time spent on one good event 0.0619 s/evt (0.000397506 hepspecH/evt)

Data In- and Output

The dashboard provides the size of the input and output (processed data and produced data):

and number of processed events, which leads to:

Simulation: 856.89 [kB/evt] processed data, 690.10 [kB/evt] produced data

Reconstruction: 10030.66 [kB/evt] processed data, 937.03 [kB/evt] produced data

Analysis: 1333.43 [kB/evt] processed data, 21.83 [kB/evt] produced data

• AOD: 516kB/evt processed data... 4,07kB/evt produced data
• DAOD: 250kB/evt processed data... 2.11kB/evt produced data

Open Questions

Why is Data IN MC sim so big?

How can "data OUT" MC SIM be so much bigger than "data IN" MC RECO? shouldnt it be the same?

Difference between CPU[s] and CPU[HEPSPEC06H] seems wrong (see: CPU Analysis AOD vs DAOD - factor of 2.3 and factor of 0.04, where 7-10 is expected), that is why often the HEPSPEC information is not used - although it should be better for comparison.

What are the Bandwidth requirement [bytes/sec] for jobs?

What is the memory consumption [bytes] of a typical (MC,RECO,ANALY) job?

What are Job efficiencies [%] and sensitivity to latency on efficiency (for remote data access?

Are there jobs that have more than 1 concurrent data access?

Major updates:
-- AleDiGGi - 2015-10-02

Fill the page:
-- Gerhard Rzehorz - 2015-10-06

Responsible: AleDiGGi
Last reviewed by: Never reviewed