T2s daily tests

This page contains an example of how CPT can be used for executing daily tests on T2.

Configuration

For these tests the /QCD_Pt80/Summer09-MC_31X_V3_AODSIM-v1/AODSIM has been used, with some cross-check with the /InclusiveBB_Pt30/Summer09-MC_31X_V3_7TeV-v1/GEN-SIM-RECO dataset (the choice was dependent on the data ditribution at the chosen sites).

Three config files has been used:

a "drop *" has been added as usual

Two different TFileAdaptor have been used:

The following sites were involved in the tests:

Some thoughts on test configurations here

CSCS

  • CSCS-PAT1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg:
    CSCS-PAT1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg

  • CSCS-PAT1_RHAUTO_CHAUTO_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg:
    CSCS-PAT1_RHAUTO_CHAUTO_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg

  • CSCS-JRA1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg:
    CSCS-JRA1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg

  • CSCS-JRA1_RHAUTO_CHAUTO_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg:
    CSCS-JRA1_RHAUTO_CHAUTO_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg

KNU

Not big differences during the tests.

  • KNU-PAT1-20100202-Overview.jpg:
    KNU-PAT1-20100202-Overview.jpg

  • KNU-PAT1_RHAUTO_CHAUTO_CACHE20-20100209-Overview.jpg:
    KNU-PAT1_RHAUTO_CHAUTO_CACHE20-20100209-Overview.jpg

  • KNU-JRA1-20100210-Overview.jpg:
    KNU-JRA1-20100210-Overview.jpg

  • KNU-JRA1_RHAUTO_CHAUTO_CACHE20-20100208-Overview.jpg:
    KNU-JRA1_RHAUTO_CHAUTO_CACHE20-20100208-Overview.jpg

Nebraska

Nothing to report

  • Nebraska-PAT1-20100209-Overview.jpg:
    Nebraska-PAT1-20100209-Overview.jpg

  • Nebraska-PAT1_RHAUTO_CHAUTO_CACHE20-20100208-Overview.jpg:
    Nebraska-PAT1_RHAUTO_CHAUTO_CACHE20-20100208-Overview.jpg

  • Nebraska-JRA1-20100209-Overview.jpg:
    Nebraska-JRA1-20100209-Overview.jpg

  • Nebraska-JRA1_RHAUTO_CHAUTO_CACHE20-20100208-Overview.jpg:
    Nebraska-JRA1_RHAUTO_CHAUTO_CACHE20-20100208-Overview.jpg

Pisa

No big differences among the various days, some tails on 2010-02-09

A further check has been done comparing:

  • QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1
  • InclusiveBB_Pt30+Summer09.MC_31X_V3_7TeV.v1-RECO

Where the second dataset has been chosen among the ones available at the site. Despite the fact that the latter is actually reading less data (and so consumes less time), both of them traces well the site performances

  • Pisa-PAT1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg:
    Pisa-PAT1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg

  • Pisa-PAT1_RHAUTO_CHAUTO_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg:
    Pisa-PAT1_RHAUTO_CHAUTO_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg

  • Pisa-JRA1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg:
    Pisa-JRA1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg

  • Pisa-JRA1_RHAUTO_CHAPP_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg:
    Pisa-JRA1_RHAUTO_CHAPP_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg

  • Pisa-PAT1-QCDvsBB-Overview.jpg:
    Pisa-PAT1-QCDvsBB-Overview.jpg
  Pisa PAT1 InclusiveBB_Pt30+Summer09.MC_31X_V3_7TeV.v1+GEN.SIM.RECO 20100210 Pisa PAT1 InclusiveBB_Pt30+Summer09.MC_31X_V3_7TeV.v1+GEN.SIM.RECO 20100211 Pisa PAT1 QCD_Pt80+Summer09.MC_31X_V3_AODSIM.v1+AODSIM 20100209 Pisa PAT1 QCD_Pt80+Summer09.MC_31X_V3_AODSIM.v1+AODSIM 20100210
Success 100.0% (20 / 20) 100.0% (18 / 18) 100.0% (20 / 20) 100.0% (20 / 20)
WrapperTime 14306.00 +- 5801.31 18001.06 +- 7256.93 28498.50 +- 10493.56 26284.05 +- 6260.62
ExeTime 14291.55 +- 5800.85 17976.33 +- 7259.04 28449.60 +- 10462.39 26266.80 +- 6260.18
UserTime 6774.19 +- 2720.80 6548.44 +- 2720.40 10202.66 +- 2284.11 10597.58 +- 2530.22
CpuPercentage 51.25 +- 9.84 37.67 +- 11.66 39.40 +- 10.45 41.45 +- 6.61
User_ReadkBEvt 58.82 +- 22.72 57.42 +- 23.54 82.96 +- 16.54 82.96 +- 16.54
AvgEventTime 0.34 +- 0.06 0.49 +- 0.17 0.62 +- 0.16 0.58 +- 0.07
MaxEventTime 28.34 +- 6.78 47.37 +- 18.13 92.72 +- 58.51 37.00 +- 6.29
TotalJobTime 14234.55 +- 5797.35 17845.92 +- 7248.02 28280.38 +- 10373.32 26193.36 +- 6257.50
MinEventTime 0.06 +- 0.01 0.08 +- 0.04 0.07 +- 0.01 0.07 +- 0.01
StageoutTime -1.00 +- 0.00 -1.00 +- 0.00 -1.00 +- 0.00 -1.00 +- 0.00
SysTime 245.36 +- 112.87 230.76 +- 116.38 291.83 +- 84.45 325.23 +- 113.34
OPEN
dcap-open-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
file-open-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
dcap-open-total-msecs 3921.39 +- 2491.14 85951.52 +- 45445.08 7367.40 +- 3727.94 68606.06 +- 37331.19
file-open-total-msecs 4.66 +- 5.46 18.63 +- 25.43 0.32 +- 0.07 7.23 +- 6.29
dcap-open-num-operations 8.90 +- 3.36 8.67 +- 3.45 7.50 +- 1.53 7.50 +- 1.53
file-open-num-operations 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00
dcap-open-num-successful-operations 8.90 +- 3.36 8.67 +- 3.45 7.50 +- 1.53 7.50 +- 1.53
file-open-num-successful-operations 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00
READ
tstoragefile-read-actual-total-megabytes 2872.31 +- 1109.49 2803.94 +- 1149.35 4050.62 +- 807.55 4050.62 +- 807.55
tstoragefile-read-total-megabytes 2872.31 +- 1109.49 2803.94 +- 1149.35 4050.62 +- 807.55 4050.62 +- 807.55
dcap-read-total-megabytes 2872.31 +- 1109.49 2803.94 +- 1149.35 4050.62 +- 807.55 4050.62 +- 807.55
file-read-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
tstoragefile-read-actual-total-msecs 7204671.40 +- 3107516.74 10918806.67 +- 5400748.93 17659862.50 +- 9226430.64 15119031.50 +- 4464316.84
tstoragefile-read-total-msecs 7206201.85 +- 3108075.12 10920223.89 +- 5400861.63 17661945.50 +- 9226610.27 15121314.00 +- 4464506.23
dcap-read-total-msecs 7202954.20 +- 3106908.14 10917176.11 +- 5400572.87 17657543.00 +- 9226274.16 15116447.00 +- 4464022.17
file-read-total-msecs 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
tstoragefile-read-actual-num-operations 458492.80 +- 177113.33 447539.22 +- 183452.31 610992.30 +- 121892.33 610992.30 +- 121892.33
tstoragefile-read-actual-num-successful-operations 458492.80 +- 177113.33 447539.22 +- 183452.31 610992.30 +- 121892.33 610992.30 +- 121892.33
tstoragefile-read-num-operations 458492.80 +- 177113.33 447539.22 +- 183452.31 610992.30 +- 121892.33 610992.30 +- 121892.33
dcap-read-num-operations 458492.80 +- 177113.33 447539.22 +- 183452.31 610992.30 +- 121892.33 610992.30 +- 121892.33
file-read-num-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
tstoragefile-read-num-successful-operations 458492.80 +- 177113.33 447539.22 +- 183452.31 610992.30 +- 121892.33 610992.30 +- 121892.33
dcap-read-num-successful-operations 458492.80 +- 177113.33 447539.22 +- 183452.31 610992.30 +- 121892.33 610992.30 +- 121892.33
file-read-num-successful-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
READV
dcap-readv-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
file-readv-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
dcap-readv-total-msecs 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
file-readv-total-msecs 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
dcap-readv-num-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
file-readv-num-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
dcap-readv-num-successful-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
file-readv-num-successful-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
SEEK
tstoragefile-seek-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
tstoragefile-seek-total-msecs 6010.07 +- 2962.02 5248.03 +- 3147.93 7635.39 +- 2858.19 8130.07 +- 2981.60
tstoragefile-seek-num-operations 459525.10 +- 177499.62 448547.56 +- 183852.38 612200.90 +- 122127.21 612200.90 +- 122127.21
tstoragefile-seek-num-successful-operations 459525.10 +- 177499.62 448547.56 +- 183852.38 612200.90 +- 122127.21 612200.90 +- 122127.21

Vienna

Vienna seems to have some instability in the file reading:

  • Big failures on (1) dataset
  • Reading time varies quite a lot during the testing days

  • Vienna-PAT1-Overview.jpg:
    Vienna-PAT1-Overview.jpg

  • Vienna-PAT1_RHAUTO_CHAUTO_CACHE20-Overview.jpg:
    Vienna-PAT1_RHAUTO_CHAUTO_CACHE20-Overview.jpg

  • Vienna-PAT1-InclusiveBB_Pt30Summer09.MC_31X_V3_7TeV.v1GEN.SIM.RECO-Overview.jpg:
    Vienna-PAT1-InclusiveBB_Pt30Summer09.MC_31X_V3_7TeV.v1GEN.SIM.RECO-Overview.jpg

  • Vienna-JRA1-Overview.jpg:
    Vienna-JRA1-Overview.jpg

  • Vienna-JRA1_RHAUTO_CHAUTO_CACHE20-Overview.jpg:
    Vienna-JRA1_RHAUTO_CHAUTO_CACHE20-Overview.jpg

  • Vienna-JRA1-InclusiveBB_Pt30Summer09.MC_31X_V3_7TeV.v1GEN.SIM.RECO-Overview.jpg:
    Vienna-JRA1-InclusiveBB_Pt30Summer09.MC_31X_V3_7TeV.v1GEN.SIM.RECO-Overview.jpg

Wisconsin

Nothing to report

  • Wisconsin-PAT1-20100202-Overview.jpg:
    Wisconsin-PAT1-20100202-Overview.jpg

  • Wisconsin-PAT1_RHAUTO_CHAUTO_CACHE20-20100209-Overview.jpg:
    Wisconsin-PAT1_RHAUTO_CHAUTO_CACHE20-20100209-Overview.jpg

  • Wisconsin-JRA1-20100209-Overview.jpg:
    Wisconsin-JRA1-20100209-Overview.jpg

  • Wisconsin-JRA1_RHAUTO_CHAPP_CACHE20-20100211-Overview.jpg:
    Wisconsin-JRA1_RHAUTO_CHAPP_CACHE20-20100211-Overview.jpg

Some Thoughts

There seem not to exist big differences between running on a AOD or a RECO sample (at a first glance). Both of them seem to be sensitive to performance degradations. In this case, an AOD sample would be lighter to distribute amont T2s.

Both JRA and PAT seem sensitive to performance degratation; anyway, they greatly differ in read data quantity (PAT>JRA) and time/event (PAT:~3h, JRA~30 min). The choice between these two is the choice between a short and light job (faster turnaround, less load on the site but more prone to perturbations) and a longer and heavier one (slow turnaround, more load on site but more stable). My proposal is to use both of them at least at the beginning.

Also, I would stick on Std CMSSW settings, as using the AUTO ones (mostly LazyDownload) will boost too much performances and does not put a measurable load on the data reading part (usually the bottleneck)

-- LeonardoSala - 11-Feb-2010

Topic attachments
I Attachment History Action Size Date Who Comment
JPEGjpg CSCS-JRA1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg r1 manage 54.5 K 2010-02-11 - 16:02 UnknownUser  
JPEGjpg CSCS-JRA1_RHAUTO_CHAUTO_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg r1 manage 59.6 K 2010-02-11 - 16:02 UnknownUser  
JPEGjpg CSCS-PAT1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg r1 manage 73.7 K 2010-02-11 - 16:02 UnknownUser  
JPEGjpg CSCS-PAT1_RHAUTO_CHAUTO_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg r1 manage 64.1 K 2010-02-11 - 16:02 UnknownUser  
JPEGjpg KNU-JRA1-20100210-Overview.jpg r1 manage 92.4 K 2010-02-11 - 17:43 UnknownUser  
JPEGjpg KNU-JRA1_RHAUTO_CHAUTO_CACHE20-20100208-Overview.jpg r1 manage 97.2 K 2010-02-11 - 17:43 UnknownUser  
JPEGjpg KNU-PAT1-20100202-Overview.jpg r1 manage 93.8 K 2010-02-11 - 17:42 UnknownUser  
JPEGjpg KNU-PAT1_RHAUTO_CHAUTO_CACHE20-20100209-Overview.jpg r1 manage 91.9 K 2010-02-11 - 17:43 UnknownUser  
JPEGjpg Nebraska-JRA1-20100209-Overview.jpg r1 manage 83.1 K 2010-02-11 - 17:48 UnknownUser  
JPEGjpg Nebraska-JRA1_RHAUTO_CHAUTO_CACHE20-20100208-Overview.jpg r1 manage 96.5 K 2010-02-11 - 17:48 UnknownUser  
JPEGjpg Nebraska-PAT1-20100209-Overview.jpg r1 manage 89.4 K 2010-02-11 - 17:48 UnknownUser  
JPEGjpg Nebraska-PAT1_RHAUTO_CHAUTO_CACHE20-20100208-Overview.jpg r1 manage 91.4 K 2010-02-11 - 17:48 UnknownUser  
JPEGjpg Pisa-JRA1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg r1 manage 69.6 K 2010-02-11 - 17:22 UnknownUser  
JPEGjpg Pisa-JRA1_RHAUTO_CHAPP_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg r1 manage 70.3 K 2010-02-11 - 17:22 UnknownUser  
JPEGjpg Pisa-PAT1-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg r1 manage 73.2 K 2010-02-11 - 17:22 UnknownUser  
JPEGjpg Pisa-PAT1-QCDvsBB-Overview.jpg r1 manage 115.7 K 2010-02-11 - 17:23 UnknownUser  
JPEGjpg Pisa-PAT1_RHAUTO_CHAUTO_CACHE20-QCD_Pt80Summer09.MC_31X_V3_AODSIM.v1AODSIM-Overview.jpg r1 manage 76.0 K 2010-02-11 - 17:22 UnknownUser  
JPEGjpg Vienna-JRA1-InclusiveBB_Pt30Summer09.MC_31X_V3_7TeV.v1GEN.SIM.RECO-Overview.jpg r1 manage 72.4 K 2010-02-11 - 15:39 UnknownUser  
JPEGjpg Vienna-JRA1-Overview.jpg r1 manage 75.9 K 2010-02-11 - 15:38 UnknownUser  
JPEGjpg Vienna-JRA1_RHAUTO_CHAUTO_CACHE20-Overview.jpg r1 manage 78.6 K 2010-02-11 - 15:38 UnknownUser  
JPEGjpg Vienna-PAT1-InclusiveBB_Pt30Summer09.MC_31X_V3_7TeV.v1GEN.SIM.RECO-Overview.jpg r1 manage 93.2 K 2010-02-11 - 15:38 UnknownUser  
JPEGjpg Vienna-PAT1-Overview.jpg r1 manage 86.3 K 2010-02-11 - 15:37 UnknownUser  
JPEGjpg Vienna-PAT1_RHAUTO_CHAUTO_CACHE20-Overview.jpg r1 manage 87.9 K 2010-02-11 - 15:38 UnknownUser  
JPEGjpg Wisconsin-JRA1-20100209-Overview.jpg r1 manage 91.2 K 2010-02-11 - 17:52 UnknownUser  
JPEGjpg Wisconsin-JRA1_RHAUTO_CHAPP_CACHE20-20100211-Overview.jpg r1 manage 100.8 K 2010-02-11 - 17:53 UnknownUser  
JPEGjpg Wisconsin-PAT1-20100202-Overview.jpg r1 manage 100.2 K 2010-02-11 - 17:52 UnknownUser  
JPEGjpg Wisconsin-PAT1_RHAUTO_CHAUTO_CACHE20-20100209-Overview.jpg r1 manage 96.0 K 2010-02-11 - 17:52 UnknownUser  
Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2010-02-12 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback