(under construction) List of runs which finished with needcheck (so outputs were produced but job had errors) we found 1 980 601 succesfully done recon jobs (project data12*, with an f-tag) + 14581 recon jobs (same criteria) finished with a NEEDCHECK trfacronym. (end of august)

Bulk processing (14581 jobs)

# jobs affected	TASKNAME	Status for reprocessing
1	data12_8TeV.00201006.physics_JetTauEtmiss.merge.RAW.f437.recon.task	checking emails, it seems this was caused by ESD getting too big, and the jobs got splited manually by T0. lb 84 had ~24k events (3 times more than those around). I quickly checked via dq2 the size of the ESD : dq2-ls -f data12_8TeV.00201006.physics_JetTauEtmiss.recon.ESD.f437* grep b0084._SFO and it's bigger than 12GB per SFO, so I suppose we'll need to split also the jobs in the grid.
1	data12_8TeV.00201190.physics_CosmicCalo.merge.RAW.f435.recon.task
1	data12_8TeV.00201190.physics_CosmicCalo.merge.RAW.f437.recon.task
1	data12_8TeV.00201494.physics_JetTauEtmiss.merge.RAW.f436.recon.task
1	data12_8TeV.00201494.physics_JetTauEtmiss.merge.RAW.f437.recon.task	#93997. LAr FEB readout problem (at least for the f437). This event will be flagged as having data corruption issue. I suppose that on the grid we'll lose the entire lb ( 5477 events) ?
1	data12_8TeV.00202609.express_express.merge.RAW.f441.recon.task	see muon comment below for this run
14	data12_8TeV.00202609.physics_Background.merge.RAW.f441.recon.task	see muon comment below for this run
17	data12_8TeV.00202609.physics_CosmicCalo.merge.RAW.f441.recon.task	see muon comment below for this run
2	data12_8TeV.00202609.physics_Egamma.merge.RAW.f441.recon.task	see muon comment below for this run
4	data12_8TeV.00202609.physics_JetTauEtmiss.merge.RAW.f441.recon.task	see muon comment below for this run
3	data12_8TeV.00202609.physics_MinBias.merge.RAW.f441.recon.task	see muon comment below for this run
5	data12_8TeV.00202609.physics_Muons.merge.RAW.f441.recon.task	#94183. data corruption problem for RPC . I am not sure this covers all the streams here shown (presumably so?).
9	data12_8TeV.00202660.express_express.merge.RAW.f442.recon.task	see muon comment below for this run
3	data12_8TeV.00202660.physics_Egamma.merge.RAW.f442.recon.task	see muon comment below for this run
12	data12_8TeV.00202660.physics_JetTauEtmiss.merge.RAW.f442.recon.task	see muon comment below for this run
127	data12_8TeV.00202660.physics_Muons.merge.RAW.f442.recon.task	#93818. we got a fix for the monitoring tag afterwords, so we shouldn't see this happening again at least for the muon stream.

17 30 data12_8TeV.00202668.express_express.merge.RAW.f443.recon.task 18 40 data12_8TeV.00202668.physics_Egamma.merge.RAW.f443.recon.task 19 62 data12_8TeV.00202668.physics_JetTauEtmiss.merge.RAW.f443.recon.task 20 1002 data12_8TeV.00202668.physics_Muons.merge.RAW.f443.recon.task

From savannah ( #94338) = # 93818. Should also be fixed (at least for the muon stream).

21 15 data12_8TeV.00202712.express_express.merge.RAW.f443.recon.task 22 240 data12_8TeV.00202712.physics_Bphysics.merge.RAW.f443.recon.task 23 38 data12_8TeV.00202712.physics_Egamma.merge.RAW.f443.recon.task 24 57 data12_8TeV.00202712.physics_JetTauEtmiss.merge.RAW.f443.recon.task 25 879 data12_8TeV.00202712.physics_Muons.merge.RAW.f443.recon.task

For the muon jobs, Same problem as above (these runs happened the same day). So also fixed a priori.

26 7 data12_8TeV.00202740.express_express.merge.RAW.f443.recon.task 27 48 data12_8TeV.00202740.physics_Bphysics.merge.RAW.f443.recon.task 28 6 data12_8TeV.00202740.physics_Egamma.merge.RAW.f443.recon.task 29 7 data12_8TeV.00202740.physics_JetTauEtmiss.merge.RAW.f443.recon.task 30 186 data12_8TeV.00202740.physics_Muons.merge.RAW.f443.recon.task

Same problem as above.

31 35 data12_8TeV.00202798.express_express.merge.RAW.f443.recon.task 32 445 data12_8TeV.00202798.physics_Bphysics.merge.RAW.f443.recon.task 33 59 data12_8TeV.00202798.physics_Egamma.merge.RAW.f443.recon.task 34 105 data12_8TeV.00202798.physics_JetTauEtmiss.merge.RAW.f443.recon.task 35 1734 data12_8TeV.00202798.physics_Muons.merge.RAW.f443.recon.task

Same problem as in the other ones.

36 1 data12_8TeV.00203636.physics_Egamma.merge.RAW.f446.recon.task #94802. Fixed in the release.

37 1 data12_8TeV.00204240.physics_JetTauEtmiss.merge.RAW.f447.recon.task #95111. LAr data corruption. I think there's no solution. So I suppose we'll loose this lb ( lb1417) in the grid. This has 4266 events.

38 1 data12_8TeV.00204564.physics_Muons.merge.RAW.f448.recon.task From emails exchanged this was large ESD again and jobs got splitted.

39 1657 data12_8TeV.00206248.physics_Muons.merge.RAW.f471.recon.task 40 7713 data12_8TeV.00206253.physics_Muons.merge.RAW.f471.recon.task

This was the MuonAlignment run where there was one unneeded output, and I guess we won't be re-processing this (Jamie/Guillaume). Anyway if we are, I think we just need to have a clone of f471 without the un-needed output.

41 1 data12_8TeV.00206962.physics_Egamma.merge.RAW.f456.recon.task DQMDisplay, bug 96091, fixed

42 1 data12_8TeV.00206971.physics_JetTauEtmiss.merge.RAW.f456.recon.task LAr data corruption, #96164

43 1 data12_8TeV.00206971.physics_MinBias.merge.RAW.f456.recon.task 44 1 data12_8TeV.00206971.physics_Muons.merge.RAW.f456.recon.task same as above ? Called files (need to see log)

45 1 data12_8TeV.00207219.physics_CosmicCalo.merge.RAW.f464.recon.task 46 1 data12_8TeV.00207664.physics_Egamma.merge.RAW.f468.recon.task 47 1 data12_8TeV.00207664.physics_Muons.merge.RAW.f468.recon.task 48 3 data12_8TeV.00208870.physics_CosmicCalo.merge.RAW.f472.recon.task 49 1 data12_muoncomm.00202277.physics_CosmicMuons.merge.RAW.f439.recon.task

I didn't look for the remaining runs (need to see log)

Prompt processing (214 jobs)

-- JoaoFirminoDaCosta - 04-Oct-2012

• x-tag_NEEDCHECK.list: extensive list of the 214 x-tag jobs ending with needcheck