October 2012 Reports

31st October 2012 (Wednesday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 4 attached T2s * MC productions at T2s if resources available

* New GGUS (or RT) tickets

* T0: * CERN: * 4 files with bad checksum on EOS (GGUS:87702) Ongoing.

* T1:

30th October 2012 (Tuesday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 4 attached T2s * MC productions at T2s if resources available

* New GGUS (or RT) tickets

* T0: * CERN: * redundant pilots (GGUS:87448). Still ongoing. * 4 files with bad checksum on EOS (GGUS:87702) Ongoing.

* T1:

29th October 2012 (Monday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 3 attached T2s * MC productions at T2s if resources available

* New GGUS (or RT) tickets

* T0: * CERN: * redundant pilots (GGUS:87448). Still discussions going on, not yet understood. * 4 files with bad checksum on EOS (GGUS:87702) Ongoing.

* T1: * GRIDKA: expired proxy in FTS system (GGUS:87367) can be closed.

19th October 2012 (Thursday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* Nothing new to report

18th October 2012 (Thursday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* T0: * CERN: pilots aborted (GGUS:87447) and redundant pilots, asking to be deleted (GGUS:87448). Still discussions going on this, not yet understood.

* T1: * GRIDKA: Staging efficiency not high enough for current reprocessing activities (GGUS:80794 not urgent but opened since April), data access problems for jobs reading from tape cache (GGUS:87318). Reprocessing going slow due to that.

17th October 2012 (Wednesday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* T0: * CERN: pilots aborted (GGUS:87447) and redundant pilots, asking to be deleted (GGUS:87448)

* T1: * GRIDKA: Staging efficiency not high enough for current reprocessing activities (GGUS:80794 not urgent but opened since April), data access problems for jobs reading from tape cache (GGUS:87318). Reprocessing going slow due to that. * SARA: running out of Tape. Banned for writing GGUS:87486)

16th October 2012 (Tuesday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 ) ongoing

* T1: * GRIDKA: Staging efficiency not high enough for current reprocessing activities (GGUS:80794 not urgent but opened since April), data access problems for jobs reading from tape cache (GGUS:87318), Staging failures, can be closed: GGUS:87061

15th October 2012 (Monday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 ) ongoing

* T1: * GRIDKA: Staging efficiency not high enough for current reprocessing activities (GGUS:80794), data access problems for jobs reading from tape cache (GGUS:87318) * IN2P3: "buffer" disk space not migrated fast enough to tape storage (GGUS:87293), fixed, FTS transfers failing because of "expired proxy" (GGUS:87321) also fixed and closed. * CNAF: "buffer" disk space increasing, transfer rate to tape storage increased * SARA: transfers failing due to authz issues (GGUS:87361), seems to be solved already.

12th October 2012 (Friday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 )

* T1: * GRIDKA: Staging efficiency not high enough for current reprocessing activities (GGUS:80794), data access problems for jobs reading from tape cache (GGUS:87318) * IN2P3: "buffer" disk space not migrated fast enough to tape storage (GGUS:87293), risked to run full on disk storage, mitigated by moving user space into disk space and increasing of transfer rate, FTS transfers failing because of "expired proxy" (GGUS:87321) * CNAF: "buffer" disk space increasing, transfer rate to tape storage increased

11th October 2012 (Thursday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 )

* T1: * GRIDKA: Staging efficiency not high enough for current reprocessing activities (GGUS:80794) * CNAF: slowness of the file system causing FTS failures and jobs not being able to download their input data, fixed now

10th October 2012 (Wednesday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 )

* T1: * GRIDKA: Input data resolution problem (GGUS:86720) has gone down, staging errors (GGUS:87061) have gone down, staging efficiency not high enough (GGUS:80794)

9th October 2012 (Tuesday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 )

* T1: * GRIDKA: Input data resolution problem (GGUS:86720) and staging problems (GGUS:80794)

8th October 2012 (Monday)

* Reprocessing at T1s and "attached" T2 sites * User analysis at T0/1 s * Prompt reconstruction at CERN + 2 attached T2s * MC productions if resources available

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 )

* T1: * SARA: files missing from SARA's storage (GGUS:86635) * GRIDKA: Input data resolution problem (GGUS:86720) and staging problems (GGUS:80794)

5th October 2012 (Friday)

* Running user analysis, prompt reconstruction and stripping at T0 and T1s * Simulation at T2s * Reprocessing at T1s and selected T2s

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 )

* T1: * SARA: files missing from SARA's storage (GGUS:86635) * NIKHEF: CVMFS problem (GGUS:86722); fixed, except two nodes * GRIDKA: Input data resolution problem (GGUS:86720)

4th October 2012 (Thursday)

* Running user analysis, prompt reconstruction and stripping at T0 and T1s * Simulation at T2s * Reprocessing at T1s and selected T2s

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 )

* T1: * SARA: files missing from SARA's storage (GGUS:86635) * NIKHEF: CVMFS problem (GGUS:86722) * GRIDKA: Input data resolution problem (GGUS:86720)

3d October 2012 (Wednesday)

* Running user analysis, prompt reconstruction and stripping at T0 and T1s * Simulation at T2s * Reprocessing at T1s and selected T2s

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 ) * CERN : problem of protocol supported by EOS (GGUS:86226) only gsiftp is supported which prevent access to the DATA from our framework. Fixed after few attempts.

* T1: * RAL: problem with staging; Fixed * SARA: files missing from SARA's storage (GGUS:86635)

LHCb stop replication of ConditionDB and read only LFC.

2nd October 2012 (Tuesday)

* Running user analysis, prompt reconstruction and stripping at T0 and T1s * Simulation at T2s * Reprocessing at T1s and selected T2s

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 ) * CERN : problem of protocol supported by EOS (GGUS:86226) only gsiftp is supported which prevent access to the DATA from our framework..

* T1: * RAL: problem with staging; under investigation

1st October 2012 (Monday)

* Running user analysis, prompt reconstruction and stripping at T0 and T1s * Simulation at T2s * Reprocessing at T1s and selected T2s

* New GGUS (or RT) tickets

* T0: * CERN : cleaning TMPDIR on lxbatch (GGUS:86039 ) * CERN : problem of protocol supported by EOS (GGUS:86226) only gsiftp is supported which prevent access to the DATA from our framework..