ATPVOFeeds < Main

Main Web>LCGGridDeployment>LCGProductionServices>ATP>ATPVOFeeds (2016-09-19, MarceloSoares) (raw view)
---+ ATP - VO TOPOLOGY FEEDS

%TOC%

---++ INTRODUCTION

The [[LCG.ATP][ATP]] VO feeds are XML files used by the LHC VOs to define which are the services that they support (from the list of services defined by [[https://goc.gridops.org][GOCDB]] and [[https://twiki.grid.iu.edu/bin/view/MonitoringInformation/RSV][RSV]]). At the same time, these feeds are used to define groups of services from the VO perspective. The groups will be used in the future to query the DB and extract the list of services that correspond to each of them. Groups can be Clouds or Tiers, for instance, or what a VO considers as a Site (a group of services mapped to a specific name).


---++ EXAMPLE OF XML FEED

Check this file as an example of a VO feed: [[%ATTACHURL%/atp_vo_feed_example.xml][atp_vo_feed_example.xml]]


---++ XML ELEMENTS

The different elements defining the XML schema are:
   * <verbatim><root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="atp_vo_feed_validator.xsd">
- Marks the beginning of the XML file and the XSD file that defines the XML schema</verbatim>
   * <verbatim><title>ATLAS VO topology for SAM/Nagios</title>
- Just a title for the VO feed.</verbatim>
   * <verbatim><description>Groups of services defined by ATLAS VO to be used by the SAM/Nagios monitoring infrastructure</description>
- A description of the VO feed.</verbatim>
   * <verbatim><feed_responsible name="Wile Coyote" dn="/DC=ch/DC=cern/OU=Org Units/OU=Users/CN=wco/CN=497/CN=Wile E Coyote"/>
- Defines who is the person responsible for the content. The DN (certificate Distinguished Name) is an optional
attribute.</verbatim>
   * <verbatim><last_update>2010-03-20T15:25:10Z</last_update>
- The  complete date plus hours and minutes in W3C format when the information was updated for the last time.</verbatim>
   * <verbatim><vo>cms</vo>
- The VO name that defines these groups of services.</verbatim>
   * <verbatim><atp_site name="Australia-ATLAS" infrast="EGEE">
- Defines a real site, as it appears defined in GOCDB for services in the EGEE infrastructure or RSV for 
the ones defined in the OSG infrastructure. And for each of these sites we will define the services that 
we want to include in one or more VO groups.</verbatim>
   * <verbatim><service hostname="agh3.atlas.unimelb.edu.au" flavour="SRM" endpoint="srm://agh3.atlas.unimelb.edu.au:8446/srm/managerv2">
- A service is identified by a hostname and a flavour. A flavour is any service type defined in GOCDB or RSV</verbatim>
   * <verbatim><spacetoken base_path="SFN=/dpm/atlas.unimelb.edu.au/home/atlas/atlasdatadisk/" name="DATADISK"/>
- For SRM services, you can optionally define one or more spacetokens, with attributes 'base_path' and 'name'.</verbatim>
   * <verbatim><group name="CA" type="ATLAS_Cloud"/>
- Finally, for each service defined, we specify to which groups it belongs. A group is defined by two 
attributes: 'name' and 'type'. Under one group 'type' we can define many group 'name's in a parent-child 
hierarchy. This can be helpful to query the ATP and retrieve all the groups (services and sites) defined 
under one single 'type' attribute. The groups can also be defined outside the <service> element, but 
inside the <atp_site> element. This means that all the services defined for that site belong to those groups.</verbatim>


---++ XML SCHEMA DEFINITION (XSD) FILE

You can use the XSD file [[%ATTACHURL%/atp_vo_feed_schema.xsd][atp_vo_feed_schema.xsd]] to validate the syntax of your XML VO feed before being consumed by the ATP.


---++ VO FEEDS USING LATEST XSD FILE

   * These are the new VO feeds supporting the XSD file:
    | *VO Name* | *URL* | *XSD COMPLIANT* |
    |ALICE| [[http://pmendez.web.cern.ch/pmendez/alice/Alice-Voboxes-For-Nagios.xml][http://pmendez.web.cern.ch/pmendez/alice/Alice-Voboxes-For-Nagios.xml]] | <img src="https://twiki.cern.ch/twiki/pub/TWiki/SmiliesPlugin/yes.gif" alt="no" title="no" border="0" /> |
    |ATLAS|[[http://atlas-agis-api.cern.ch/request/atp/xml/][http://atlas-agis-api.cern.ch/request/atp/xml/]]| <img src="https://twiki.cern.ch/twiki/pub/TWiki/SmiliesPlugin/yes.gif" alt="no" title="no" border="0" /> |
    |CMS|[[http://dashb-cms-job/dashboard/request.py/cmssitemap][http://dashb-cms-job/dashboard/request.py/cmssitemap]]| <img src="https://twiki.cern.ch/twiki/pub/TWiki/SmiliesPlugin/yes.gif" alt="no" title="no" border="0" /> |
    |LHCb| [[http://lhcb-portal-dirac.cern.ch/topology/lhcb_topology.xml][http://lhcb-portal-dirac.cern.ch/topology/lhcb_topology.xml]] | <img src="https://twiki.cern.ch/twiki/pub/TWiki/SmiliesPlugin/yes.gif" alt="no" title="no" border="0" /> |

---++ VO GROUP NAMING CONVENTIONS

VOs need to ensure that VO feed information should be in a format acceptable to all. The following conventions have been agreed upon.:

   * The group_type names are desired to be in following format for Site, Tier, Federation and Cloud groupings:
 <verbatim><VOname>_Site
 <Voname>_Tier
 <VOname>_Federation 
 <VOname>_Cloud
 </verbatim>
   * The following groupings are obligatory in each group.
 <verbatim> <group name="Tier-x" type="<VOname>_Tier"/>
 <group name="<VO SITENAME>" type="<VOname>_Site"/> 
 </verbatim>

           where _VOname_ refers to the name of the VO and _Tier-x_  refers to Tier-0, Tier-1,Tier-2, Tier-3 etc.

   * For the <VOname>_Tier group type, the group names should be *Tier-0*, *Tier-1*, *Tier-2*, *Tier-3* 

          The groupname for Tier group type is fixed as it is required for generation of Tier-wise reports for all the VOs.

An example of CMS VO grouping is given below:

These 2 fields are obligatory in each group.:
<verbatim><group name="Tier-0" type="CMS_Tier"/>
<group name="<VO SITENAME>" type="CMS_Site"/>
</verbatim>
       
These 3 fields are optional, but should keep the naming convention:
 <verbatim> <group name="<VO CLOUD NAME>" type="CMS_Cloud"/>
 <group name="<VO GROUP NAME>" type="CMS_Group"/>
 <group name="<VO FEDERATION NAME>" type="CMS_Federation"/>
</verbatim>
For the other groups, VOs can put whatever they want, e.g.:
    <verbatim> <group name="T0_CH_CERN" type="All Groups"/>
  <group name="T0_CH_CERN" type="Tier1s + Tier0"/>
  <group name="T0_CH_CERN" type="Tier2s + Tier1s + Tier0"/> 
</verbatim>

---++ HOW TO VALIDATE AN XML FEED

Using an SLC5 machine, you can use the xmlint command, like in this example:
<verbatim>xmllint --noout --schema ./atp_vo_feed_schema.xsd lhcb_topology_example.xml</verbatim>

Another way using python could be running the following command:
<verbatim>python validate_xml.py</verbatim>
For that you need to have the xsd file (attached to this twiki page) and your xml file in the same directory and specify the name of your xml file inside the validate_xml.py file.
The validate_xml.py file is also attached to this twiki page (but auto-renamed to .txt for security reasons)
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
xml	atp_vo_feed_example.xml	r1	manage	4.8 K	2011-03-03 - 06:45	UnknownUser
xsd	atp_vo_feed_schema.xsd	r1	manage	2.4 K	2011-03-05 - 17:02	UnknownUser	VO feed xsd
txt	validate_xml.py.txt	r1	manage	0.4 K	2010-09-01 - 12:25	DavidCollados
xml	vo_topology_feed_example.xml	r1	manage	2.5 K	2010-01-21 - 19:16	UnknownUser
Topic revision: r17 - 2016-09-19 - MarceloSoares
Main
Webs
Public webs
Welcome Guest
- Cern Search
- TWiki Search
- Google Search
Main All webs