Table Of Contents

Introduction

Most of the information about IO in CMSSW can be found here:

https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideStorageStats

https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideTFileAdaptor

Three config files has been used:

The CMSSW version used is 3_3_2. Furthermore, a patch from Brian Bockelman has bee tested on a dCache system at T3_CH_PSI:

 --- IOPool/Input/src/RootTree.cc        20 Aug 2009 15:31:44 -0000      1.50
+++ IOPool/Input/src/RootTree.cc        30 Oct 2009 17:53:30 -0000
@@ -138,13 +138,14 @@
        assert(treeCache_->GetOwner() == tree_);
        if (theEntryNumber >= 0 && treeCache_->IsLearning()) {
        treeCache_->SetLearnEntries(1);
-       treeCache_->SetEntryRange(0, tree_->GetEntries());
+/*     treeCache_->SetEntryRange(0, tree_->GetEntries());
          for (BranchMap::const_iterator i = branches_->begin(), e =  
branches_->end(); i != e; ++i) {
          if (i->second.productBranch_) {
            treeCache_->AddBranch(i->second.productBranch_, kTRUE);
          }
        }
          treeCache_->StopLearningPhase();
+*/
        }
      }
      entryNumber_ = theEntryNumber;

Single Job tests

First, tests has been performed on the UI at T3_CH_PSI, with the system unloaded. In this case, Ganglia has been used for estimating the IO throughput. Two files from the /InclusiveMu5_Pt50/Summer09-MC_31X_V3-v1/GEN-SIM-RECO dataset has been used.

  CMSSW_3_3_2 CMSSW_3_3_2 patched
  Real User Sys Eff Real User Sys Eff
JETAna    
no settings 7m30.431s 2m59.981s 0m9.113s 0.4 7m30.501s 3m0.272s 0m9.038s 0.4
noTFileAdaptor, cache=20MB 3m42.478s 3m2.204s 0m15.186s   6m58.138s 3m5.155s 0m10.252s  
noTFileAdaptor, cache=50MB 4m7.407s 3m1.326s 0m23.371s   5m48.293s 3m3.903s 0m7.271s  
readHint=auto-detect, cacheHint=auto-detect 7m10.672s 3m0.080s 0m12.572s 0.4 7m38.046s 3m1.466s 0m13.468s  
readHint=auto-detect, cacheHint=auto-detect, tempdir="" 3m32.395s 3m1.107s 0m11.507s   3m34.705s 3m1.595s 0m11.936s  
readHint=auto-detect, cacheHint=auto-detect, cache=20 MB 7m37.211s 3m1.154s 0m13.760s   7m18.442s 3m1.469s 0m12.841s  
readHint=auto-detect, cacheHint=auto-detect, cache=20 MB, tempdir="" 3m35.453s 3m2.977s 0m13.094s   3m32.673s 3m2.044s 0m11.903s  
readHint=default, cacheHint=auto-detect, cache=20 MB 7m6.389s 3m1.734s 0m14.087s   7m34.119s 3m1.321s 0m13.528s  
readHint=auto-detect, cacheHint=default, cache=20 MB 4m56.850s 3m1.298s 0m14.095s   8m36.212s 3m1.024s 0m9.682s  
cacheHint=app-only, readHint=auto-detect, cache=20MB 5m0.193s 3m0.995s 0m14.243s   8m46.261s 3m1.170s 0m9.396s  
cacheHint=app-only, readHint=unbuff, cache=20MB 15m26.007s 3m1.220s 0m13.509s   10m2.487s 2m58.836s 0m6.489s  
cacheHint=app-only, readHint=buff, cache=20MB 4m46.000s 3m1.128s 0m14.253s   8m34.662s 3m0.755s 0m9.673s  
cacheHint=lazy-download, readHint=auto-detect, cache=20MB 7m6.389s 3m1.734s 0m14.087s   7m26.062s 3m1.188s 0m12.943s  
cacheHint=storage-only, readHint=auto-detect, cache=20MB CRASH
cacheHint=storage-only, read-hint=auto-detect 7m35.328s 3m1.836s 0m9.628s   7m32.917s 3m0.424s 0m9.015s  
read-hint Auto + cache 20 MB 5m1.811s 3m2.803s 0m15.023s

PAT    
no settings, 1k events 5m47.227s 3m55.577s 0m5.063s   5m45.627s 3m56.006s 0m5.333s  
readHint=auto-detect, cacheHint=auto-detect, tempdir="" 4m14.474s 3m56.034s 0m5.346s   4m20.935s 3m55.869s 0m6.766s

  • tempDir defaults to ".", in this case the nfs home dir hosted by FS06. If tempDir="", point to $TMPDIR, if unset to "/tmp". How to unset? In this case, both patched and unpatched CMSSW performs very well, and the former has an higher IO throughput. without cache, both seems to behave the same

JetAna_UI-2ReadFile_Network05_commented.png

  • Disabling the TFileAdaptor give a good boost on IO and CPU efficiency, but the patch seems not to profit of it:

JetAna_UI-2ReadFile_Network04_commented.png

  • The cacheHint=auto-detect option causes big write access to the SE, while selecting application-only (the default) restores good performances

JetAna_UI-1ReadFile_Network-Commented.png

JetAna_UI-2ReadFile_Network02_commented.png

* When using cacheHint=auto-detect, it actually selects the lazy-download option. This creates write accesses to the pool hosting the data and also to another one (shouldn't it write on a /tmp by default?). In the case represented, the files were only on FS02, cache=20MB:

JetAna_FS02-2ReadFile_Network02.png

JetAna_FS06-2ReadFile_Network02.png

When using cache=0 (it means that the file is entirely read from the temporary copy, and not in a virtual buffer fashion as in the case cache!=0) the behaviour seems not to change at all.

  • Also with pat-tuplizer, good performances with lazy-download. Patched version holds a better IO throughput:

Pat_UI-1ReadFile-1k_Network_commented.png

  • When using the patch and no TFileAdaptor sometimes comes out a readv error (e.g. "Invalid vector size 2148")

Scale test at T3_CH_CSCS

This test has been performed on a subsample of /InclusiveMu5_Pt50/Summer09-MC_31X_V3_AODSIM-v1/AODSIM. This different tier has been chosen as it was better distribuited on PSI SE.

JetAna

  CMSSW332 CMSSW332Patched RH-APP_CH-AUTO_CACHE-20_CMSSW332 RH-APP_CH-AUTO_CACHE-20_CMSSW332Patched RH-AUTO_CH-AUTO_CACHE-20_CMSSW332 RH-AUTO_CH-AUTO_CACHE-20_CMSSW332Patched RH-AUTO_CH-AUTO_CACHE-20_TEMPDIR-VOID__CMSSW332 noTAdaptor_CACHE-20_CMSSW332 noTAdaptor_CACHE-50_CMSSW332
Success 100.0% (23 / 23) 100.0% (23 / 23) 100.0% (23 / 23) 100.0% (23 / 23) 100.0% (23 / 23) 100.0% (23 / 23) 100.0% (23 / 23) 100.0% (23 / 23)
CrabCpuPercentage 40.78 +- 2.34 41.61 +- 2.78 76.35 +- 1.05 38.39 +- 1.95 92.35 +- 1.58 91.26 +- 2.29 90.57 +- 1.38 78.22 +- 3.50 53.65 +- 3.52
CrabStageoutTime 9.13 +- 0.85 8.87 +- 0.54 9.04 +- 0.55 0.83 +- 0.38 8.78 +- 0.41 9.13 +- 0.68 9.48 +- 0.83 8.91 +- 0.58 9.61 +- 1.01
CrabSysCpuTime 35.34 +- 5.75 28.93 +- 3.94 26.01 +- 3.39 35.02 +- 5.78 20.71 +- 3.17 22.10 +- 3.24 20.47 +- 3.10 39.27 +- 5.78 78.99 +- 11.65
CrabUserCpuTime 511.41 +- 80.45 487.73 +- 80.15 532.81 +- 79.71 500.81 +- 82.76 534.29 +- 82.05 519.56 +- 88.36 538.62 +- 77.48 514.63 +- 74.81 544.35 +- 80.21
CrabWrapperTime 1351.22 +- 208.98 1247.43 +- 184.14 748.17 +- 107.09 1389.87 +- 226.15 618.78 +- 91.61 608.78 +- 95.43 635.04 +- 88.55 722.78 +- 99.08 1183.26 +- 199.88
ExeTime 1331.39 +- 209.06 1228.65 +- 184.17 728.04 +- 106.75 1378.83 +- 225.94 598.57 +- 91.80 589.52 +- 95.38 614.91 +- 88.50 703.48 +- 99.02 1162.30 +- 199.52
StageoutTime 9.13 +- 0.85 8.87 +- 0.54 9.04 +- 0.55 0.83 +- 0.38 8.78 +- 0.41 9.13 +- 0.68 9.48 +- 0.83 8.91 +- 0.58 9.61 +- 1.01
WrapperTime 1351.22 +- 208.98 1247.43 +- 184.14 748.17 +- 107.09 1389.87 +- 226.15 618.78 +- 91.61 608.78 +- 95.43 635.04 +- 88.55 722.78 +- 99.08 1183.26 +- 199.88
dcap-read-num-operations 170894.39 +- 24603.25 170652.91 +- 25129.53 427.04 +- 61.67 163022.70 +- 23037.68 20.13 +- 2.89 22.61 +- 3.98 20.13 +- 2.89 // //
dcap-read-total-megabytes 385.21 +- 55.39 384.67 +- 56.58 10.37 +- 1.94 367.68 +- 51.97 2430.21 +- 348.32 2747.42 +- 484.00 2430.21 +- 348.32 // //
dcap-read-total-msecs 795096.57 +- 133282.01 707618.23 +- 111842.05 10625.79 +- 2902.85 784073.61 +- 125549.49 44899.63 +- 11772.03 47126.50 +- 11758.52 53139.03 +- 10623.08 // //
dcap-readv-num-operations -1.00 +- -1.00 0.00 +- 0.00 151.22 +- 21.64 3.09 +- 0.58 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 // //
dcap-readv-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 3128.02 +- 448.96 22.51 +- 9.91 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 // //
dcap-readv-total-msecs 0.00 +- 0.00 0.00 +- 0.00 163365.41 +- 25149.60 67310.67 +- 31408.92 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 // //
local-cache-prefetch-num-operations // // // // 3856.00 +- 557.62 10144.43 +- 3704.62 3856.00 +- 557.62 // //
local-cache-prefetch-total-megabytes // // // // 2609.40 +- 375.62 22.52 +- 9.91 2609.40 +- 375.62 // //
local-cache-prefetch-total-msecs // // // // 7802.68 +- 3982.79 34099.33 +- 8369.02 8725.72 +- 4561.12 // //
local-cache-read-num-operations // // // // 170894.39 +- 24603.25 170894.39 +- 24603.25 170894.39 +- 24603.25 // //
local-cache-read-total-megabytes // // // // 385.21 +- 55.39 385.21 +- 55.39 385.21 +- 55.39 // //
local-cache-read-total-msecs // // // // 38613.80 +- 10553.65 14437.22 +- 6609.57 45719.73 +- 9329.95 // //
tstoragefile-read-async-num-operations // // 3.09 +- 0.58 3.09 +- 0.58 3856.00 +- 557.62 10144.43 +- 3704.62 3856.00 +- 557.62 // //
tstoragefile-read-async-total-megabytes // // 0.00 +- 0.00 0.00 +- 0.00 2609.39 +- 375.62 22.51 +- 9.91 2609.39 +- 375.62 // //
tstoragefile-read-async-total-msecs // // 0.00 +- 0.00 0.00 +- 0.00 7810.49 +- 3983.00 34117.85 +- 8372.46 8733.48 +- 4561.49 // //
tstoragefile-read-num-operations 170894.39 +- 24603.25 170652.91 +- 25129.53 170894.39 +- 24603.25 170894.39 +- 24603.25 170894.39 +- 24603.25 170894.39 +- 24603.25 170894.39 +- 24603.25 // //
tstoragefile-read-prefetch-to-cache-num-operations // // // // 170617.65 +- 24563.44 170617.65 +- 24563.44 170617.65 +- 24563.44 // //
tstoragefile-read-prefetch-to-cache-total-megabytes // // // // 374.84 +- 53.87 17.53 +- 7.46 374.84 +- 53.87 // //
tstoragefile-read-prefetch-to-cache-total-msecs // // // // 8776.58 +- 4023.82 34172.76 +- 8377.69 9704.27 +- 4610.10 // //
tstoragefile-read-total-megabytes 385.21 +- 55.39 384.67 +- 56.58 385.21 +- 55.39 385.21 +- 55.39 385.21 +- 55.39 385.21 +- 55.39 385.21 +- 55.39 // //
tstoragefile-read-total-msecs 795783.13 +- 133375.42 708280.73 +- 111927.78 174650.22 +- 27255.28 853072.39 +- 147166.16 49871.74 +- 12253.59 50219.03 +- 12002.76 57917.24 +- 11193.58 // //
tstoragefile-read-via-cache-num-operations // // 170617.65 +- 24563.44 170617.65 +- 24563.44 // // // // // // //
tstoragefile-read-via-cache-total-megabytes // // 374.84 +- 53.87 17.53 +- 7.46 // // // // //
tstoragefile-read-via-cache-total-msecs // // 2223.93 +- 383.64 67413.14 +- 31443.13 // // // // //
tstoragefile-seek-num-operations 170894.39 +- 24603.25 170652.91 +- 25129.53 511979.39 +- 73708.53 349383.74 +- 50878.79 511979.39 +- 73708.53 349383.74 +- 50878.79 511979.39 +- 73708.53 // //
tstoragefile-seek-total-msecs 741.25 +- 113.14 719.27 +- 114.64 1840.65 +- 274.06 1283.46 +- 203.84 1701.37 +- 257.38 1193.79 +- 193.43 1703.31 +- 246.17 // //

CrabCpuPercentage

the baseline is given by CMSSW run with default parameters (no options given): job efficiency is ~35%.

  • Disabling the TAdaptor gives good performances with cache=20, which are anyway degraded with an hingher cache (50 MB). This is seen both for 5k that for 15k events/job

  • Using cacheHint=application-only and readHint=auto-detect gives performances as good as to pure ROOT reading (as expected), but not in the case of the patch

  • With 15k events/job, cacheHint=application-only and readHint=auto-detect, standard CMSSW remains ~75% efficiency, while patched CMSSW below 40%

  • User time is similar for any of the 15k events/job runs

T3Test_JRA1_CrabCpuUserTime.png

  • nevertheless, the cpu efficiencies differ very much, giving the best results for RH=auto and CH=auto. In this case, patched and unpatched CMSSW seems to perform the same, and twice faster than standard CMSSW settings. NoTAdaptor and CH=application-only performs nicely under standard CMSSW but not in the patched one.

T3Test_JRA1_CrabCpuPercentage.png T3Test_JRA1_ExeTime.png

Read performances

  • The differences stated above can be seen also in reading performances, with the same conclusions

T3Test_JRA1_tstoragefile-read-total-msecs.png

  • For local file reading (lazy-download), patched version seems to perform better

T3Test_JRA1_local-cache-read-total-msecs.png

  • Patched version requests less data to be pre-fetched to cache. This seems to produce longer reading times... why?

T3Test_JRA1_tstoragefile-read-prefetch-to-cache-total-megabytes.png

T3Test_JRA1_tstoragefile-read-prefetch-to-cache-total-msecs.png

T3Test_JRA1_tstoragefile-read-async-total-msecs.png

  • Patched CMSSW seems not to profit from dCache vectorial read, as all the read operations goes through single read operations. the behaviour in this case is very similar to standard CMSSW with no TFileAdaptor options. Patched CMSSW reads almost all with single access, with few readv instances.

T3Test_JRA1_dcap-read-total-megabytes.png

T3Test_JRA1_dcap-readv-total-megabytes.png

PAT

  CMSSW332 CMSSW332Patched RH-AUTO_CH-AUTO_CACHE-20_CMSSW332 RH-AUTO_CH-AUTO_CACHE-20_CMSSW332Patched
Success 100.0% (23 / 23) 100.0% (23 / 23) 100.0% (23 / 23) 100.0% (23 / 23)
CrabCpuPercentage 68.87 +- 1.90 68.87 +- 1.92 98.13 +- 0.34 97.65 +- 0.48
CrabStageoutTime 12.96 +- 1.12 12.74 +- 1.15 13.35 +- 1.13 12.91 +- 0.88
CrabSysCpuTime 57.20 +- 10.80 66.54 +- 10.69 25.18 +- 3.46 26.41 +- 4.06
CrabUserCpuTime 2622.39 +- 437.40 2750.28 +- 421.44 2863.10 +- 469.18 2746.64 +- 414.44
CrabWrapperTime 3879.52 +- 612.83 4077.26 +- 594.06 2952.13 +- 477.13 2849.87 +- 421.97
ExeTime 3855.61 +- 611.67 4053.00 +- 593.05 2927.35 +- 476.07 2825.74 +- 421.60
StageoutTime 12.96 +- 1.12 12.74 +- 1.15 13.35 +- 1.13 12.91 +- 0.88
WrapperTime 3879.52 +- 612.83 4077.26 +- 594.06 2952.13 +- 477.13 2849.87 +- 421.97
dcap-prefetch-num-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
dcap-prefetch-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
dcap-prefetch-total-msecs 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
dcap-read-num-operations 185029.57 +- 26720.63 185029.57 +- 26720.63 20.13 +- 2.89 20.87 +- 3.23
dcap-read-total-megabytes 1213.76 +- 175.11 1213.76 +- 175.11 2430.21 +- 348.32 2524.81 +- 391.64
dcap-read-total-msecs 1183419.22 +- 191421.09 1247202.35 +- 182207.66 36425.34 +- 7135.57 40665.97 +- 8468.58
dcap-readv-num-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
file-open-num-operations 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00
file-open-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
file-open-total-msecs 0.72 +- 2.41 0.21 +- 0.01 0.18 +- 0.02 0.19 +- 0.02
local-cache-prefetch-num-operations // // 29512.17 +- 4286.25 8702.17 +- 4426.91
local-cache-prefetch-total-megabytes // // 3013.55 +- 432.43 135.19 +- 75.39
local-cache-prefetch-total-msecs // // 11730.26 +- 3836.58 24442.73 +- 7068.52
local-cache-read-num-operations // // 185029.57 +- 26720.63 185029.57 +- 26720.63
local-cache-read-total-megabytes // // 1213.76 +- 175.11 1213.76 +- 175.11
local-cache-read-total-msecs // // 27169.19 +- 5671.56 18615.62 +- 5042.21
local-cache-readv-num-operations // // 0.00 +- 0.00 0.00 +- 0.00
tstoragefile-read-async-num-operations // // 29512.17 +- 4286.25 8702.17 +- 4426.91
tstoragefile-read-async-total-megabytes // // 3013.54 +- 432.43 135.18 +- 75.39
tstoragefile-read-async-total-msecs // // 11788.79 +- 3841.16 24460.63 +- 7066.46
tstoragefile-read-num-operations 185029.57 +- 26720.63 185029.57 +- 26720.63 185029.57 +- 26720.63 185029.57 +- 26720.63
tstoragefile-read-prefetch-to-cache-num-operations // // 170200.83 +- 24579.66 170200.83 +- 24579.66
tstoragefile-read-prefetch-to-cache-total-megabytes // // 1141.94 +- 164.74 128.13 +- 74.46
tstoragefile-read-prefetch-to-cache-total-msecs // // 12828.27 +- 3920.26 24521.10 +- 7059.64
tstoragefile-read-total-megabytes 1213.76 +- 175.11 1213.76 +- 175.11 1213.76 +- 175.11 1213.76 +- 175.11
tstoragefile-read-total-msecs 1184175.52 +- 191523.51 1247987.78 +- 182311.15 42570.95 +- 7780.44 44815.07 +- 8833.40

No major differences seen wrt JetAna:

T3Test_PAT1_CrabCpuPercentage.png T3Test_PAT1_CrabCpuUserTime.png T3Test_PAT1_ExeTime.png T3Test_PAT1_tstoragefile-read-total-msecs.png

Preliminary conclusions

For dCache, the auto-detect option for RH and CH seems to select the best options, namely lazy-download and read-ahead-buffered. The patched CMSSW do not brings performance improvements (at least in these tests), but actually seems to cause less data to be downloaded by the lazy-download option

Leaving the caching to ROOT (app-only) give also nice results. The TFileAdaptor disabling seems not suggested as not always works (e.g. with PAT, to be checked).

The default behaviour of tempDir can trigger problems, eg when local running on a NFS area, maybe the default value should point to TMPDIR

Next steps

Try JetAna and the Pat-tuplizer on some T2, with different storage technologies

-- LeonardoSala - 11-Nov-2009

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng JetAna_FS02-2ReadFile_Network02.png r1 manage 109.9 K 2009-11-12 - 22:04 UnknownUser  
PNGpng JetAna_FS06-2ReadFile_Network02.png r1 manage 119.4 K 2009-11-12 - 22:04 UnknownUser  
PNGpng JetAna_UI-1ReadFile_Network-Commented.png r2 r1 manage 129.0 K 2009-11-11 - 18:21 UnknownUser  
PNGpng JetAna_UI-2ReadFile_Network02_commented.png r1 manage 77.2 K 2009-11-11 - 18:35 UnknownUser  
PNGpng JetAna_UI-2ReadFile_Network04_commented.png r1 manage 61.6 K 2009-11-12 - 23:07 UnknownUser  
PNGpng JetAna_UI-2ReadFile_Network05_commented.png r2 r1 manage 85.7 K 2009-11-13 - 17:39 UnknownUser  
PNGpng Pat_UI-1ReadFile-1k_Network_commented.png r1 manage 59.4 K 2009-11-16 - 11:11 UnknownUser  
PNGpng T3Test_JRA1_CrabCpuPercentage.png r2 r1 manage 20.0 K 2009-11-16 - 12:32 UnknownUser  
PNGpng T3Test_JRA1_CrabCpuUserTime.png r2 r1 manage 15.4 K 2009-11-16 - 12:33 UnknownUser  
PNGpng T3Test_JRA1_ExeTime.png r1 manage 18.7 K 2009-11-16 - 12:33 UnknownUser  
PNGpng T3Test_JRA1_dcap-read-total-megabytes.png r2 r1 manage 26.9 K 2009-11-16 - 15:31 UnknownUser  
PNGpng T3Test_JRA1_dcap-read-total-msecs.png r1 manage 15.0 K 2009-11-13 - 15:00 UnknownUser  
PNGpng T3Test_JRA1_dcap-readv-total-megabytes.png r2 r1 manage 24.5 K 2009-11-16 - 15:32 UnknownUser  
PNGpng T3Test_JRA1_dcap-readv-total-msecs.png r1 manage 16.4 K 2009-11-13 - 15:03 UnknownUser  
PNGpng T3Test_JRA1_local-cache-read-total-msecs.png r1 manage 13.0 K 2009-11-16 - 12:42 UnknownUser  
PNGpng T3Test_JRA1_tstoragefile-read-async-total-msecs.png r1 manage 14.0 K 2009-11-16 - 12:48 UnknownUser  
PNGpng T3Test_JRA1_tstoragefile-read-prefetch-to-cache-total-megabytes.png r1 manage 11.3 K 2009-11-16 - 12:47 UnknownUser  
PNGpng T3Test_JRA1_tstoragefile-read-prefetch-to-cache-total-msecs.png r1 manage 14.8 K 2009-11-16 - 12:48 UnknownUser  
PNGpng T3Test_JRA1_tstoragefile-read-total-msecs.png r2 r1 manage 17.6 K 2009-11-16 - 12:40 UnknownUser  
PNGpng T3Test_JRA1_tstoragefile-read-via-cache-total-megabytes.png r1 manage 14.0 K 2009-11-13 - 14:47 UnknownUser  
PNGpng T3Test_JRA1_tstoragefile-read-via-cache-total-msecs.png r1 manage 14.7 K 2009-11-13 - 15:04 UnknownUser  
PNGpng T3Test_PAT1_CrabCpuPercentage.png r1 manage 13.9 K 2009-11-16 - 15:38 UnknownUser  
PNGpng T3Test_PAT1_CrabCpuUserTime.png r1 manage 14.7 K 2009-11-16 - 15:39 UnknownUser  
PNGpng T3Test_PAT1_ExeTime.png r1 manage 15.1 K 2009-11-16 - 15:39 UnknownUser  
PNGpng T3Test_PAT1_tstoragefile-read-total-msecs.png r1 manage 15.5 K 2009-11-16 - 15:39 UnknownUser  
Edit | Attach | Watch | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r11 - 2009-11-29 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback