Difference: WLCGOpsMinutes130919 (25 vs. 26)

Revision 262013-09-23 - AndreaSciaba

Line: 1 to 1
META TOPICPARENT name="WLCGOpsCoordination"

WLCG Operations Coordination Minutes - 19 September 2013

Line: 9 to 9


  • Local: Alessandra Forti (chair), Andrea Sciabà (secretary),
  • Remote:
  • Local: Andrea Sciabà (secretary), Ken Bloom, Stefan Roiser, Oliver Keeble, Simone Campana, Nicolò Magini, Andrea Valassi, Felix Lee, Maarten Litmaath, Maite Barroso Lopez, Alessandro Di Girolamo, Jan Iven
  • Remote: Alessandra Forti (chair), Alessandro Cavalli, Frederique Chollet, Helge Meinhard, Christoph Wissing, Pavel Weber, Massimo Sgaravatto, Daniele Bonacorsi, Isidro Gonzalez Caballero, Alessandra Doria, Gareth Smith, Di Qing, Michael Ernst


Maarten and Maite introduce the proposal to start the WMS decommissioning task force, already foreseen since long, due to CERN's plans to decommission the WMS service as soon as practical. The residual usage comes from SAM, some small fraction of the LHC experiment jobs and smaller VOs (ILC, GEANT4). The task force will start by analysing the logs to exactly determine who are the users, and then to discuss alternatives with them. A twiki will soon be created, and volunteers are very welcome.

About the ops tests, Maarten explains that WCLG will stop looking at them from January, but presumably they will still be run by EGI for their monitoring.

The task force is approved and its coordinators will be Maarten and Maite.


Middleware news and baseline versions

  • minor change for BDII
  • new baseline version for dCache is 2.2.17, introducing SHA-2 support
  • new baseline version for StoRM is 1.11.2, in EMI-3. It fixes some known issues (also related to FTS 3) and supports SHA-2. Sites should upgrade

Maarten adds that earlier this week a critical bug was discovered in the top BDII. Sites have been notified by EGI. A patch will be released next week (not this week, because an easy workaround is available); still, upgrading to the current latest version is advisable.


Tier-1 Grid services

Storage deployment

Site Status Recent changes Planned changes
Line: 90 to 102
  • arcproxy: due to the problems introduced by the new java version distributed with EMI-3 needed by e.g. voms-proxy-info, ATLAS has decided to replace its pilot calls with arcproxy ones. The latest problem (https://ggus.eu/ws/ticket_info.php?ticket=97230) is not easily solvable since java doesn't like the virtual memory being limited. Limiting the memory to keep out some memory hungry jobs is current practice at many sites. The minimum value sites can set the limit is not predictable as it looks to be connected to the amount of memory+swap. arcproxy is written in C++ and is considered more suitable for the WNs activity.
  • webDAV: for DPM, as reported in the baseline versions which have been updated, the required version for ATLAS is DPM 1.8.7 .

Maarten clarifies that the problem does not originate from the new Java version (EMI does not distribute Java) but from the latest voms client, which are written in Java instead of C++.



  • Operations and Production activities
Line: 113 to 127
      • Status of the WLCG Operations Coordinations TF?
    • SAM tests: condor_g mode, progress?
Maarten replies that work on a new SAM job submission probe based on Condor-G will start in November.


  • WLCG services
Line: 148 to 164
    • soon: switch the VOBOX services of a small site to the use of a SHA-2 proxy
      • no problems foreseen
Alessandra F. asks if experiments tested the full job submission chain since last meeting. Maarten answers that, once the VO central services and the site services are known to work with SHA-2, there shouldn't be too much to worry about, although running the whole chain would be good as final check. ALICE will do it, but they are concerned mostly about the interaction between the VOBOX and some ALICE central services.

Michael adds that in USATLAS they will definitely do vertical tests, and even if BNL is not yet at the latest version of dCache, there will be no problem meeting the December 1st deadline. About CMS, Andrea S. reminds that the full chain has already been tested with SHA-2 user proxies and glexec; using a SHA-2 proxy for the pilot is still to be done.


Machine/Job Features

  • TF started effectively in September. Two meetings so far used for requirements gathering of the experiments.
    • Mainly interested in information on number of cores, time left (wall/cpu), machine power (HepSpec06)
Line: 159 to 179
  • Ongoing good progress in the ALICE deployment campaign, 14 sites left (5 done). Out of those 8 stated they will deploy in Sept.


See the slides.

All sites are invited to install (or upgrade to) version 3.3.1. A broadcast will be sent.



  • IT-PES deployed new FTS3 version on fts3.cern.ch (already running at RAL) fixing several bugs including e.g. checksumming.
Line: 166 to 189
  • CMS: testing new FTS3 server at IN2P3 in Debug, increasing Debug load on CERN FTS3 server.
  • LHCb: using FTS3 instance at CERN after the bugfix with the timestamps in transfer status output.

Tracking tools

Maarten announces that the end of the year deadline for the decomissioning of Savannah will not be met due to other more urgent duties, but it was not a hard deadline anyway. Many complex problems still need migrating. Andrea V. adds that PH-SFT has very little manpower left on Savannah, it's already on best effort.


  • Motivation: the exhaustion of the IPv4 address space is starting to create problems to some sites (in particular CERN) and WLCG needs to have a strategy to become IPv6-ready on a timescale that fits with the needs of the sites and the experiments.
  • An IPv6 validation and deployment task force for IPv6 is being formed, to work in collaboration with the HEPiX IPv6 working group on these aspects:
Line: 190 to 216


The next meetings will be on October 3rd and 24th.

Action list

  1. Tracking tools TF members who own savannah projects to list them and submit them to the savannah and jira developers if they wish to migrate them to jira. AndreaV and MariaD to report on their experience from the migration of their own savannah trackers.
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback