31st August (Monday)

• Data Processing:
  • Drop of MC jobs at T1 sites on the week-end, solved this morning and restarting now. DataReconstruction at T1s.

• T0

• T1
  • RAL: Problem with pilot submission to ARC-CEs, working on a fix
  • CNAF OUTAGE downtime declared Friday evening for LHCb storage

27th August (Thursday)

• Data Processing:
  • p-Ne collisions reconstruction started

• T0
  • Nothing to report

• T1
  • RAL 'at-risk' yesterday and today due to castor update.

24th August (Monday)

• Data Processing:
  • validation productions of 25ns data will be redone this week (ReCo + Stripping), with proper refitting of PV
  • current validation (Reco15b and Stripping 23a) quasi finished (>99% data processed, 75% merged)

• T0
  • Nothing to report

• T1
  • Nothing to report

20th August (Thursday)

• Data Processing: second run of validation for 25ns data started with new calibration, ~90% already processed. New stripping production will start later today.

• T0
  • Nothing to report

• T1
  • Nothing to report
  • CVMFS problem at NIKHEF-ELPROD (GGUS:115767) due to an upgrade of their WNs from CentOS 6.6 to CentOS 6.7. Promptly solved.
  • PBS issues at PIC (GGUS:115748) due to a high load in few WNs, pbs server had problems to refresh information, and denied some connections. Promptly solved.

17th August (Monday)

• Data Processing: validation of 25ns data started. ~70% already processed.

• T0
  • Nothing to report

• T1
  • Nothing to report

13th August (Thursday)

• Data Processing: data "stripping" productions ongoing almost complete. Ready to process new data. User and MC jobs

• T0
  • Nothing to report

• T1
  • GRIDKA: Still we have problems getting TURLs from SURLs.

10th August (Monday)

• Data Processing: data "stripping" productions ongoing almost complete. Ready to process new data. User and MC jobs

• T0
  • Nothing to report

• T1
  • RAL: Network issues over the weekend
  • GRIDKA: Problems over the weekend getting TURLs from SURLs. Overloaded SRM? Difficult to track down as the jobs retry/go to failover and complete.

6th August (Thursday)

• Data Processing: data "stripping" productions ongoing. User and MC jobs

• T0
  • Discussion with LSF team about wall and cpu time queue lengths, ongoing (GGUS:115027). Some progress and bugs found on both sides. Deployment of fixes happening in progress.

• T1
  • RAL: All network problems appear to be solved.

3rd August (Monday)

• Data Processing: data "stripping" productions ongoing. User and MC jobs

• T0
  • Discussion with LSF team about wall and cpu time queue lengths, ongoing (GGUS:115027). Some progress and bugs found on both sides. Deployment of fixes happening in progress.

• T1
  • RAL: LAN network instabilities continue, CVMFS and data access problems over the weekend.

-- JoelClosier - 2016-01-07