Reducing WLCG dependencies on BDII


The Information System TF is working towards stopping WLCG dependencies in the BDII. This twiki presents the different proposals that the TF is evaluating. Please, note that this is all being discussed and still needs to be approved by the WLCG MB.


WLCG has no dependencies on storage information published in the BDII. Each LHC experiment has its own recipe to collect and maintain storage endpoints and storage capacity information. For this reason, WLCG is investigating the possibility of stopping storage information being published in the BDII for dedicated WLCG resources.

A proof of concept has been done with dCache at pic which has a dedicated service for three LHC VOs: ATLAS, CMS and LHCb. The recipe described in the next section has been succesfully implemented and it is recommended to sites with dedicated WLCG storage.

Recipe to stop publishing Storage Services in the BDII

This recipe is for sites with dedicated WLCG storage services.

  • Remove from GOCDB the tag EGI for the storage service and set it to Local(x) (this may take some time to propagate to the SAM OPS servers)
  • Remove SE Endpoints from Site-BDII and wait until this is propagated to top BDIIs (this may take several hours or days depending on how long it is configured the top BDII cache)
  • Stop BDII service running on the hostname where the storage service is running.

These steps will imply:

  • The SAM OPS tests will not be executed on that service.
  • The storage service won’t be published anymore in the BDII.
  • The storage service will still be tested by ETF since storage services are taken from experiment VOfeeds that do not rely on the BDII to get the list of services.


All LHC VOs rely on the BDII to discover CE queues and related information. However, the amount of information that is actually used by the VOs is very little compared to what the BDII publishes. Moreover, OSG is moving out of the BDII. For this reason, there is a general agreement that the static information needed by LHC VOs could be moved to GOCDB and OIM instead. This would allow WLCG to rely only on GOCDB/OIM to get information from EGI and OSG resources. This would also allow sites not belonging to EGI or OSG and not running a BDII to provide the set of needed attributes in an easy way, avoiding to run a service like the BDII.

The following set of attributes are considered to be sufficient to cover experiment use cases:

Attribute generic name GLUE 1 GLUE 2 Example
CE endpoint Included in GlueCEUniqueID GLUE2EndpointID
Queue name Included in GlueCEUniqueID GLUE2ComputingShareMappingQueue grid_2nh_atlas
CE type GlueCEImplementationName GLUE2EndpointImplementationName HTCondorCE
CE LRMS type GlueCEInfoLRMSType GLUE2ManagerProductName Condor
Max CPU time GlueCEPolicyMaxCPUTime GLUE2ComputingShareMaxCPUTime 2880
Max Wallclock time GlueCEPolicyMaxWallClockTime GLUE2ComputingShareMaxWallTime 2880

The IS TF will investigate how to move this information from the BDII to GOCDB/OIM taking the following actions:

  • The GLUE2 name of the attributes will be used
  • The GLUE Enumerated types values will be used for the types
  • Identify a set of volunteer sites to define the new attributes in GOCDB and OIM
  • Using the web UI, use extension properties to define key/value pairs matching the list of attributes
    • GOCDB: a writeable API has been requested and is under discussion
  • Ask AGIS to consume this information from GOCDB/OIM instead of the BDII

After this exercise is carried out, IS TF will be in a position to evaluate whether it's a feasible choice and plan for a wide deployment if necessary.


WLCG monitoring relies on the BDII to be able to get the queue names for the CEs that need to be tested according to the experiments VOfeeds. There is a proposal to improve the current VOfeed so that it directly includes the queue name on it. However, with the new CRIC, it seems reasonable to get this information directly from CRIC, as shown in the picture below.


The IS TF proposal would be the following:

  • Freeze any developments to provide the VOfeed V2 containing queue information.
  • Wait for CRIC to be ready and then implement an experiment topology API that would allow applications and other services to get WLCG and experiment topology related information in a uniform way.
  • In the meantime, WLCG Monitoring keeps on relying on the BDII to get the queue names until CRIC is ready.

-- MariaALANDESPRADILLO - 2016-04-28

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng cric.png r1 manage 186.0 K 2016-04-29 - 12:29 MariaALANDESPRADILLO  
Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2016-05-31 - MariaALANDESPRADILLO
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EGEE All webs login

This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright &© by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Ask a support question or Send feedback