Introduction

Here are described a set of utilities and a procedure to measure T2s performances. This method exploits the information contained in the FrameworkJobReport.

Three config files have been selected for this proposal:

Three config files has been used:

the number of events/jobs is selected to have UserTime>10m and <30m. Furthermore, in order not to suffer from StageOut problems, a drop * were introduced and result files were retrieved on the UI

Some tests were also been performed using different TAdaptor configurations. Most of the information about IO in CMSSW can be found here:

Description of the Test Suite

Some scripts have been developed in order to automatize the gathering and the analysis of data.

Composition of the testing suite:

  • a Python script for quickly submitting jobs to various sites, with necessary templates and cfg files
  • some simple bash script for quickly managing the CRAB jobs
  • a Python script with creates a root files with histograms of all the quantities reported by the job, plus a table with the interesting statistics
  • a Python script to quickly plot the resulting file
  • a Python script which allows a more flexible plotting, for further inspections
  • src and cfgs from the respective authors

The scripts are made in order to quickly evaluate differences in performance between site. This test suite can be also useful for a medium-scale evaluation of CMSSW patches.

In CRAB, CpuPercentage = (UserTime+SysTime)/WrapperTime

Difference between Wrapper and ExeTime (thanks to D.Spiga for clarifications):

Comparison between config files ToBeChecked

  • Similar Cpu performances (on a unloaded site), PAT reads ~3 times more data than JRA.

  • Seek operations similar

  • PAT jobs are more efficient. This can be as JRA jobs must wait for the data to be found in order to start computations

  KNU-JRA1 CMSSW_3_3_2 20091210 KNU-PAT1 CMSSW_3_3_2 20091212
Success 100.0% (20 / 20) 100.0% (20 / 20)
CrabCpuPercentage 44.50 +- 9.14 79.20 +- 3.30
CrabUserCpuTime 572.95 +- 46.67 2398.57 +- 136.59
User_ReadkBEvt 28.07 +- 0.19 89.96 +- 0.26
CrabSysCpuTime 48.91 +- 4.54 73.72 +- 6.67
CrabWrapperTime 1425.30 +- 168.45 3117.40 +- 129.42
User_Wall-Read_CpuPercentage 44.95 +- 9.13 83.11 +- 3.17
READ
tstoragefile-read-actual-total-megabytes 274.09 +- 1.88 878.52 +- 2.59
dcap-read-total-megabytes 274.09 +- 1.88 878.52 +- 2.59
tfile-read-total-megabytes 0.00 +- 0.00 0.00 +- 0.00
tstoragefile-read-total-megabytes 274.09 +- 1.88 878.52 +- 2.59
file-read-total-megabytes // 0.00 +- 0.00
tstoragefile-read-actual-total-msecs 798478.00 +- 185625.72 526871.40 +- 107766.07
dcap-read-total-msecs 797869.05 +- 185624.39 526218.80 +- 107767.87
tstoragefile-read-total-msecs 799500.00 +- 185329.84 527400.00 +- 107823.19
file-read-total-msecs // 0.00 +- 0.00
READV
dcap-readv-total-megabytes 0.00 +- 0.00 0.00 +- 0.00
file-readv-total-megabytes // 0.00 +- 0.00
dcap-readv-total-msecs 0.00 +- 0.00 0.00 +- 0.00
file-readv-total-msecs // 0.00 +- 0.00
SEEK
tstoragefile-seek-num-successful-operations 122585.60 +- 418.64 134977.15 +- 311.18
tstoragefile-seek-total-megabytes 0.00 +- 0.00 0.00 +- 0.00
tstoragefile-seek-total-msecs 1454.59 +- 71.23 1834.81 +- 52.98

Tests

Standard tests

Standard test performed on 2009-12-03

Standard test: December daily summary

Auto-detect settings

Standard test performed on 2009-12-03

Various tests

Testing Brian's Patch

Testing 2009-12-16 Brian's Patch

Testing 2010-01-07 Brian's Patch

Testing 2010-01-20 Brian's Patch

Final test of Brian's Patch

Comparison between TFileAdaptor settings

Daily tests

RECO vs AOD

Single job tests

Daily tests 2

Testing CMSSW_3_6_0pre5io

Testing Brian's second round of patches

Testing Lassi LD patch for CMSSW_3_7_0

Testing IFCA site, Jan 2011

Warning

tests with JRA and CMSSW cannot find the correct AK5 collection (ak5 instead of antikt5) on 3_1_X samples, so those tests cannot be fully trusted

Open questions

  • Why sometimes dcap/gsidcap-read-total/msecs < > tstoragefile-read-total-msecs ?

-- LeonardoSala - 03-Dec-2009

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng CSCSkim_Tight-20091203-CrabCpuPercentage.png r1 manage 9.2 K 2009-12-03 - 18:05 UnknownUser  
PNGpng CSCSkim_Tight-20091203-CrabUserCpuTime.png r1 manage 8.6 K 2009-12-03 - 18:06 UnknownUser  
PNGpng CSCSkim_Tight-20091203-Error.png r1 manage 5.1 K 2009-12-03 - 18:07 UnknownUser  
PNGpng CSCSkim_Tight-20091203-read-total-msecs.png r1 manage 10.1 K 2009-12-03 - 18:08 UnknownUser  
PNGpng CSCSkim_Tight-20091203-tstoragefile-read-total-msecs.png r1 manage 9.5 K 2009-12-03 - 18:07 UnknownUser  
PNGpng JRA1-20091203-CrabCpuPercentage.png r1 manage 8.5 K 2009-12-03 - 18:16 UnknownUser  
PNGpng JRA1-20091203-CrabUserCpuTime.png r1 manage 8.2 K 2009-12-03 - 18:17 UnknownUser  
PNGpng JRA1-20091203-Error.png r1 manage 6.1 K 2009-12-03 - 18:17 UnknownUser  
PNGpng JRA1-20091203-read-total-msecs.png r1 manage 8.9 K 2009-12-03 - 18:17 UnknownUser  
PNGpng JRA1-20091203-tstoragefile-read-total-msecs.png r1 manage 9.8 K 2009-12-03 - 18:18 UnknownUser  
PNGpng PAT1-20091203-CrabCpuPercentage.png r1 manage 9.1 K 2009-12-03 - 18:11 UnknownUser  
PNGpng PAT1-20091203-CrabUserCpuTime.png r1 manage 7.3 K 2009-12-03 - 18:11 UnknownUser  
PNGpng PAT1-20091203-CrabWrapperTime.png r1 manage 7.6 K 2009-12-03 - 18:12 UnknownUser  
PNGpng PAT1-20091203-Error.png r1 manage 6.1 K 2009-12-03 - 18:12 UnknownUser  
PNGpng PAT1-20091203-read-total-msecs.png r1 manage 10.4 K 2009-12-03 - 18:13 UnknownUser  
PNGpng PAT1-20091203-tstoragefile-read-total-msecs.png r1 manage 9.6 K 2009-12-03 - 18:13 UnknownUser  
Edit | Attach | Watch | Print version | History: r25 < r24 < r23 < r22 < r21 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r25 - 2011-01-18 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback