December 2012 Reports

To the main

03th December 2012 (Monday)

  • Reprocessing until last stop finished. New DataBases for the last step (from 30th of November) will be ready around Thursday this week.
  • Prompt reconstruction: CERN + 5 Tier2 sites
  • MC productions at T2s and T1s (until reprocessing will restart)
  • T0:
    • Is planned the upgrade of the LFC tu EMI.
  • T1:
    • RAL: Some problem in accessing data. A disk server is down and need a fsck before to put it again in production (not before tomorrow). [Tiju announces that it was put back in production this morning]
    • CNAF: Installed the new disk storage (many thanks to CNAF people)
    • GRIDKA: transfer failure to/from several sites. No clue from the site. [Pavel adds that experts are working on the problem: they increased the debug level and do see some transfers failing (but not all of them)]

04th December 2012 (Tuesday)

  • Prompt reconstruction: CERN + 5 Tier2 sites
  • MC productions at T2s and T1s (until reprocessing will restart)
  • Had some problem because a partition on a vobox got full. Hot-fixed. Plan to reshuffle a bit the distribution of databases among the voboxes.
  • T0:
  • T1:
    • RAL: Some problem in the early morning in with FTS transfer from CERN. It seemed to be a corruption in FTS database. It has been fixed quickly.
    • IN2P3: Lots of FTS transfer failure during the night (also between IN2P3/IN2P3 and IN2P3/IN2P3-T2). Problem disappeared in the morning.

05th December 2012 (Wednesday)

  • Normal operation activities. Waiting for the new databases to start the last reprocessing step.
  • T0:
  • T1:
    • NTR

06th December 2012 (Wednesday)

  • Normal operation activities. Waiting for the new databases to start the last reprocessing step.
  • Still some problem with agents responsible of submitting pilot to the sites. Investigation is ongoing.
  • T0:
  • T1:
    • NL-T1: Bunch of failed FTS transfers just before lunch.

10th December 2012 (Monday)

  • Started reprocessing activities, which means there will should be significant staging at the T1s.
  • T0:
  • T1:
    • NL-T1: We expect the downtime of SARA to be finished today, but we would like a notification in case of delay.
    • PIC had an ARCHIVAL deletion of 930 files by error, due to a bug after enstore update.

11th December 2012 (Tuesday)

  • Reprocessing running smoothly.
  • T0:
  • T1:
    • NL-T1: Dowtime SARA finished
    • PIC files lost from archive in the migration with no other replicas. Not a big issue, files were old and supposed to be deleted anyway.
    • RAL has another T2 attached now: LCG.Krakow.pl
    • Heavy packet loss (~50%) under ping -f, even between local voboxes at CERN; to be discussed with CERN networking team, possible reason why some of our services are hanging quite often.
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r6 - 2012-12-11 - DanielaRemenska
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback