28 Jan (Thursday)

• Data Processing
  • Mostly MC and User jobs

• Site Issues
  • T0:
    • Issue with one of the CEs losing jobs. Now fixed. (GGUS: 119136)
  • T1:
    • CNAF: Problems with checksums with uploading files (GGUS: 119127)
    • PIC: Some issues with getting TURLS from SURLS due to a badly behaving pool node (GGUS: 119068)

• Miscellaneous
  • Stripping 24 was started this afternoon

25 Jan (Monday)

• Data Processing
  • Mostly MC and User jobs

• Site Issues
  • T0: NTR
  • T1:
    • CNAF : Minor issue with some nodes not having Machine Job Features installed

• Miscellaneous
  • Stripping 24 to reprocess 2015 data hopefully starting tomorrow
  • Pre-staging of turbo data almost complete

21 Jan (Thursday)

• Data Processing
  • Mostly MC and User jobs

• Site Issues
  • T0: NTR
  • T1:
    • CNAF: Problem staging some files (GGUS:118984)

• Miscellaneous
  • About to start stripping 24 to reprocess 2015 data.
  • Pre-staging of turbo data for re-stripping.

18 Jan (Monday)

• Data Processing
  • Mostly MC and User jobs

• Site Issues
  • T0: NTR
  • T1:
    • Strange transfer failure patterns in SARA in the middle of last week, likely due to srm issues. Being followed up internally.
    • Problems transferring to RRCKI-ARCHIVE (http://lblogbook.cern.ch/Operations/22905) - likely server misconfiguration. Admins being contacted internally.

• Miscellaneous
  • One problematic user running multi-core jobs on the grid - being contacted
  • About to start stripping 24 to reprocess 2015 data. Applications being prepared and start with a small validation before turning on everything. Staging of some of the data already done in December (copied from tape to buffer).

14 Jan (Thursday)

• Data Processing
  • User jobs, since Monday, at a good rate (aprox. 5K jobs/hour) without major errors.
  • MC running (more than 15K jobs/hour) with no significant issues.

• Issues
  • T0: NTR
  • T1:
    • No main issue to be reported

11 Jan (Monday)

• Data Processing
  • Monte Carlo and User analysis at T0/1/2D sites during weekend

• Issues
  • T1:
    • IN2P3: Files could not be opened, because SRM returned a wrong end point (GGUS:118655): Ticket closed on Friday(8th.) as solved

07 Jan (Thursday)

• Data Processing
  • Monte Carlo and user analysis at T0/1/2D sites
  • Pre-staging the data for re-stripping is almost finished.

• Issues
  • T1:
    • RRCKI: Staging problem
    • IN2P3: Files can not be opened, because SRM returned a wrong end point (GGUS:118655)
    • CNAF: Transfer failure (GGUS:118665)

04 Jan (Monday)

• Data Processing
  • Monte Carlo running, with intention of continuing to run this during YETS (including on HLT farm); user analysis at T0/1/2D sites
  • Data prestaging for Stripping 24 Run II data is almost finished.

• T0:
  • CREAM CEs are unavailable (GGUS:118588).
  • 50 Scheduled pilots at each CE. Some of them submitted day ago.

• T1:
  • Staging problem at PIC,Gridka and RRCKI during holidays; Already fixed.
  • RAL: Disk server unavailable

-- JoelClosier - 2017-01-10