VO tags Validation

Introduction

On 07.03.2016, GOCDB released a new feature to be able to define WLCG tags associated to WLCG resources. See Release notes for more details. VO tags are in fact used by WLCG to be able to identify which resources declared in GOCDB can actually be used by LHC VOs.

This is a very useful mechanism to be able to query WLCG resources per VO, per Tier, etc. However, the tags were originally defined per site and not per resource. In order to correct this, sites have been asked to look at the tags and doublecheck whether they are correct for each production resource, cleaning the wrong tags accordingly. To verify that the current tags are correct, a validation effort is on going comparing the tags in GOCDB for each resource with the resources declared in each experiment VOfeed. It could also be that the VOfeed is wrong, see the next table to understand the scenarios we are trying to identify.

Resource in GOCDB is tagged Resource exists in VOfeed  
NO NO -
NO YES Ticket opened to the site: we need to clarify with the site whether the tag is missing or whether the VO is wrongly defining the resource in the VOfeed
YES NO Ticket opened to the site: we need to clarify with the site if the resource is wrongly tagged or whether the VO has forgotten to include the resource in its VOfeed
YES YES All OK

Note for sites: If the site doesn't have a certain VO tag and that VO tag is needed in a resource, the GOCDB admin needs to add the tag at site level first.

This task is carried out by Aleksandr Berezhnoi.

Experiment VOfeeds

GOCDB

The following method can be used to retrive all sites that support a specified site-scope tag. i.e.

https://goc.egi.eu/gocdbpi/public/?method=get_site&scope=cms,wlcg

This will return a list of sites. This list can be iterated using the get_service method to get the site's services. i.e:

https://goc.egi.eu/gocdbpi/public/?method=get_service&sitename=FI_HIP_T2

This can then be compared to the experiment VOfeed information for the site.

Monitoring of sites that need to fix their scope tags

Site name Status Verify GGUS ticket Comments
FZK-LCG2 waiting for reply   121344  
egee.irb.hr assigned   121345  
UNIBE-LHEP waiting for reply   121352  
IEPSAS-Kosice waiting for reply   121353  
PSNC assigned   121354  
MY-UPM-BIRUNI-01 in progress   121355  
T3-TH-CHULA in progress   121357  
UKI-SCOTGRID-ECDF waiting for reply   121360  
UA-BITP assigned   121362  
JINR-LCG2 closed   121366  
IL-TAU-HEP assigned   121369  
NIKHEF-ELPROD in progress   121370  
DESY-HH assigned   121378  
UKI-LT2-Brunel waiting for reply   121388  
RO-13-ISS closed   121390  
FMPhI-UNIBA closed   121392  
INFN-CATANIA closed   121395  
IFCA-LCG2 assigned   121402  
NZ-UOA in progress   121403  
ZA-CHPC assigned   121404  
BY-NCPHEP assigned   121406  
INFN-BOLOGNA-T3 assigned   121408  
UA-ISMA in progress   121410  
WEIZMANN-LCG2 assigned   121416  
INFN-TRIESTE in progress   121419  
UKI-LT2-RHUL waiting for reply   121421  
RU-SPbSU in progress   121422  
INFN-NAPOLI-ATLAS assigned   121432  
ICN-UNAM assigned   121433  
RO-11-NIPNE in progress   121439  
TECHNION-HEP assigned   121446  
SE-SNIC-T2 closed   121449  
UKI-SOUTHGRID-BHAM-HEP in progress   121450  
ICM in progress   121452  
GRIF closed   121454  
CERN-PROD in progress   121456  
INFN-CAGLIARI assigned   121460  
Hephy-Vienna solved No 121358 - Remove 'alice' tag in GOCDB for the following services not present in ALICE VOfeed:
hephyse.oeaw.ac.at(SRM).
INFN-T1 solved No 121377 - Remove 'lhcb' tag in GOCDB for the following services not present in LHCB VOfeed:
storm-fe.cr.cnaf.infn.it(SRM).
- Add 'lhcb' tag in GOCDB for the following services present in LHCB VOfeed:
ce01-lhcb-t2.cr.cnaf.infn.it(CE),
ce02-lhcb-t2.cr.cnaf.infn.it(CE).
UKI-SOUTHGRID-BRIS-HEP solved No 121386 - Remove 'atlas' tag in GOCDB for the following services not present in ATLAS VOfeed:
lcgce01.phy.bris.ac.uk(CE).
INFN-FRASCATI solved No 121396 - Add 'lhcb' tag in GOCDB for the following services present in LHCB VOfeed:
atlasce1.lnf.infn.it(CE),
atlasce3.lnf.infn.it(CE).
TRIGRID-INFN-CATANIA should be opened     - Remove 'alice' tag in GOCDB for the following services not present in ALICE VOfeed:
cream-ce-01.ct.trigrid.it(CE).
INFN-CNAF-LHCB should be opened     - Remove 'lhcb' tag in GOCDB for the following services not present in LHCB VOfeed:
ce01-lhcb-t2.cr.cnaf.infn.it(CE),
ce02-lhcb-t2.cr.cnaf.infn.it(CE).

How to renew the information in the table with tickets

After several minutes you should obtain a new file twiki.txt with a new twiki formatted table.

renew.sh runs tree python files:

  • gocbd_info.py (extracts all host from COGDB and saves the data to the shelve file gocdb_info.db)
  • vofeed_vs_gocDB.py (compares the information from COGDB and VOfeed saves the results to the shelve file gocdbvofeed.db)
  • tickets_submission.py (compares the gocdbvofeed.db with the initial data file gocdbvofeed_initial.db, monitors and creates tickets, creates new twiki table)

Now the scripts does not create new tickets. To begin a new campaign of tickets submission your should:

  • in the file tickets_submission.py replace the string "out=ggus_monitor_gocdb_vofeed.ggus_monitor(site,template,'','False','prod')" to "out=ggus_monitor_gocdb_vofeed.ggus_monitor(site,template,'','True','prod')"
  • mv gocdbvofeed_initial.db gocdbvofeed_initial.db.old
  • cp gocdbvofeed.db gocdbvofeed_initial.db
  • ./renew.sh

The file patterns.py contains the patterns used for parsing the COGDB and VOfeed xml files and host names, which are should be removed from the final table for some reasons.

-- MariaALANDESPRADILLO - 2016-04-28

Edit | Attach | Watch | Print version | History: r45 < r44 < r43 < r42 < r41 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r45 - 2016-06-01 - MariaALANDESPRADILLO
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EGEE All webs login

This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright & by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Ask a support question or Send feedback