T0-->T1 Data Transfer tests in preparation for Run2
Motivation
Test T0->T1 transfers at the rates expected in Run2, taking into account the relevant changes in Tier-0 and Tier-1 processing, namely:
- Tier-0 processing runs against EOS
- Prompt reconstruction shared with Tier-1s
- Disk/tape separation at Tier-1s
- RECO is a temporary disk-only data format
Assumptions for the test
- Event sizes/rates for 2015 are taken from the presentation at the Fall 2014 O&C week - 3.0 MB/evt RECO, 0.63 MB/evt AOD
- RAW is produced on EOS/T2_CH_CERN and exported in three copies: one custodial copy on tape at T0_CH_CERN_MSS; one custodial copy on tape distributed across T1_*_MSS; one disk copy distributed across T1_*_Disk
- PromptRECO is produced 50% on EOS/T2_CH_CERN and 50% at T1s; the T2_CH_CERN fraction is exported in one copy distributed across T1_*_Disk
- PromptAOD is produced 50% on EOS/T2_CH_CERN and 50% at T1s; the T2_CH_CERN fraction is exported in two copies: one custodial copy distributed across T1_*_MSS; one disk copy distributed across T1_*_Disk
- The dataset transfers to the Tier-1s are distributed in proportion to their 2015 pledges (either by tape, disk or CPU pledge share - it doesn't matter, since the relative shares don't differ by more than a few percent)
- The rates are calculated assuming a worst-case scenario, with no rerouting in PhEDEx, i.e. each RAW & AOD file is exported from T2_CH_CERN to T1 twice (once to Disk and once to MSS)
- With these assumptions, the rates are calculated here:
Procedure for the test
- Set the LoadTest rates for T2_CH_CERN-->T0_CH_CERN_MSS, T2_CH_CERN-->T1_*_Disk, T2_CH_CERN-->T1_*_MSS according to the targets in the table
- Increasing gradually in steps, e.g. 25%-50%-75%-100%
Timeline for the test
- Nov 11th-17th
- Repeat Nov 18th-24th in case of issues
- 2 Round Feb 16th-19th (Increasing in 2 steps 50%-100%)
- 3 Round April 2015
- 4 Round July 7th - 2015
- 5 Round October 16th - 2015
- 6 Round January 25th - 2016
Progress
March 2017
January 2016
October 2015
July 2015
April 2015
Feb 2015
* FNAL_Disk: Outgoing transfers and incoming transfers(Staging Test) in the production instance affected the test.
Nov 2014
Values: Actual Rate/Target Rate
Opened tickets
--
NicoloMagini - 2014-11-10