T0-->T1 Data Transfer tests in preparation for Run2

Motivation

Test T0->T1 transfers at the rates expected in Run2, taking into account the relevant changes in Tier-0 and Tier-1 processing, namely:

Event sizes/rates for 2015 are taken from the presentation at the Fall 2014 O&C week - 3.0 MB/evt RECO, 0.63 MB/evt AOD
- https://indico.cern.ch/event/285674/session/7/contribution/41/material/slides/0.pdf
- TODO - double-check with the significantly lower numbers in the resource request document from February 2014 - 0.8 MB/evt RECO, 0.37 MB/evt AOD https://cms-docdb.cern.ch/cgi-bin/DocDB/ShowDocument?docid=12170
RAW is produced on EOS/T2_CH_CERN and exported in three copies: one custodial copy on tape at T0_CH_CERN_MSS; one custodial copy on tape distributed across T1_*_MSS; one disk copy distributed across T1_*_Disk
PromptRECO is produced 50% on EOS/T2_CH_CERN and 50% at T1s; the T2_CH_CERN fraction is exported in one copy distributed across T1_*_Disk
PromptAOD is produced 50% on EOS/T2_CH_CERN and 50% at T1s; the T2_CH_CERN fraction is exported in two copies: one custodial copy distributed across T1_*_MSS; one disk copy distributed across T1_*_Disk
The dataset transfers to the Tier-1s are distributed in proportion to their 2015 pledges (either by tape, disk or CPU pledge share - it doesn't matter, since the relative shares don't differ by more than a few percent)
The rates are calculated assuming a worst-case scenario, with no rerouting in PhEDEx, i.e. each RAW & AOD file is exported from T2_CH_CERN to T1 twice (once to Disk and once to MSS)

Set the LoadTest rates for T2_CH_CERN-->T0_CH_CERN_MSS, T2_CH_CERN-->T1_*_Disk, T2_CH_CERN-->T1_*_MSS according to the targets in the table
- Increasing gradually in steps, e.g. 25%-50%-75%-100%

Site	16Feb	17Feb	18Feb
JINR_MSS	-	-
KIT_Disk	60/60	117/124	105/124
CNAF_Disk	86/90	147/180	174/180
PIC_MSS	14/15	29/30	29/30
FNAL_MSS	288/350	266/350	353/350
FNAL_Disk	218/550	65/550	40/550
RAL_MSS	22/24	46/47	46/47
KIT_MSS	29/30	56/59	51/59
PIC_Disk	35/36	68/71	69/71
CNAF_MSS	38/38	73/76	73/76
RAL_Disk	53/56	103/111	89/111
JINR_Disk	47/67	53/133	91/133
Transfer Quality	Plot	Plot	Plot

* FNAL_Disk: Outgoing transfers and incoming transfers(Staging Test) in the production instance affected the test.

Values: Actual Rate/Target Rate

Site	11Nov	12Nov	13Nov	14Nov	15Nov	16Nov
FNAL_MSS	59/60	114/120	155/180	31/235	252/235	71/235
FNAL_Disk	157/140	262/280	386/420	451/556	471/556	496/556
CNAF_MSS	19/19	37/38	55/57	72/76	75/76	74/76
CNAF_Disk	51/45	86/90	127/135	172/180	172/180	174/180
RAL_MSS	12/12	23/24	35/36	46/47	46/47	46/47
RAL_Disk	31/28	54/56	80/84	106/111	107/111	107/111
IN2P3_MSS	11/11	21/22	32/33	37/44	43/44	43/44
IN2P3_Disk	29/26	45/52	75/78	86/104	101/104	100/104
KIT_MSS	13/15	32/30	9/45	60/59	51/59	64/59
KIT_Disk	31/30	58/60	48/90	140/124	120/124	119/124
PIC_MSS	8/8	16/16	22/24	29/30	30/30	29/30
PIC_Disk	21/18	35/36	52/54	69/71	68/71	70/71
JINR_MSS	-	-	-	-	-	-
JINR_Disk	38/34	65/68	97/102	120/133	122/133	117/133
Transfer Quality	Plot	Plot	Plot	Plot	Plot	Plot

Site	Target Rate	18Nov	19Nov	20Nov	21Nov	22Nov	23Nov
FNAL_MSS	235 MB/s	245	212	99	5	123	268
FNAL_Disk	556 MB/s	305	187	214	257	110	278
CNAF_MSS	76 MB/s	73	74	74	74	73	75
CNAF_Disk	180 MB/s	158	164	175	182	184	172
RAL_MSS	47 MB/s	47	46	47	46	46	46
RAL_Disk	111 MB/s	106	108	107	107	108	106
IN2P3_MSS	44 MB/s	47	43	42	43	43	43
IN2P3_Disk	104 MB/s	97	105	100	82	104	106
KIT_MSS	59 MB/s	1	49	40	48	1	7
KIT_Disk	124 MB/s	96	59	18	68	6	20
PIC_MSS	30 MB/s	22	5	25	15	35	30
PIC_Disk	71 MB/s	34	5	50	34	73	70
JINR_MSS	40 MB/s	-	-	-	-	-	-
JINR_Disk	133 MB/s	108	100	113	109	120	110
Transfer Quality		Plot	Plot	Plot	Plot	Plot	Plot

Opened tickets

-- NicoloMagini - 2014-11-10

Topic attachments
I	Attachment	History	Action	Size	Date	Who
png	FromFNALProd.png	r2 r1	manage	41.5 K	2015-02-19 - 23:53	JorgeAlbertoDiazCruz
png	T0FNALDebug.png	r1	manage	43.6 K	2015-02-19 - 23:54	JorgeAlbertoDiazCruz
png	ToFNALProd.png	r1	manage	45.2 K	2015-02-19 - 23:54	JorgeAlbertoDiazCruz