VO Feed Documentation

VO feed refers to a set of XML documents following a common schema and describing experiment's topologies (services, sites, etc.). It is coordinated by the Information Systems Task Force.

Elements and Attributes

The following is a complete list of elements and attributes in the latest version of the VO feed schema.

Every feed must have a header with the following elements:

<root>
   <title>Feed title </title>
   <description>
          Feed description
    </description>
    <feed_responsible> DN of the responsible person </feed_responsible>
    <last_update> Date and time of the last update, e.g. 2016-11-30T13:24:23Z</last_update>
    <vo> Virtual Organization </vo>
....
</root>

The header may contain an optional version element, identifying the version of the experiments VO feed implementation.

It then contains a set of site and service elements organized as follows:

  • atp_site element that describes an official (GOCDB/OIM) site name and has the following attributes:
    • name - GOCDB/OIM site name (mandatory)
    • longitude - longitude for a site - degrees minutes seconds format (optional)
    • latitude - latitude for a site - degrees minutes seconds format (optional)
    • contact - e-mail contact for a site (applies to all the underlying services); (optional)
    • infrast - indicates the underlying infrastructure of the site (EGI/OSG) (optional)
  • service - sub-element of atp_site that describes a particular site's service with the following attributes
    • flavour - service type of given service (mandatory)
    • hostname - hostname of given service (mandatory)
    • endpoint - takes the form of [protocol://]hostname[:port]/[path]. For storage elements, specifying endpoints is highly recommended (since not all sites are using standard ports). (optional)
    • production_status - flags the service as production, can be true/false, if missing defaults to true (optional)
    • contact - e-mail contact where notifications can be sent (optional)

An example of atp_site and service elements:

<atp_site infrast="OSG" name=? AGLT2" longitude=?3.4? latitude=?54.3?>
   <service endpoint="srm://head01.aglt2.org:8443/srm/managerv2?SFN=" flavour=?SRMv2? hostname="head01.aglt2.org?>
   <service endpoint=?https://head01.aglt2.org" flavour=?HTTPS? hostname="head01.aglt2.org?>
   <service endpoint=?root://head01.aglt2.org" flavour=?XROOTD?  hostname="head01.aglt2.org?>
   <service endpoint="gate03.aglt2.org:9619" flavour=?HTCONDOR-CE" hostname="gate03.aglt2.org?>
</atp_site>

In order to be able to group services into experiment's sites, tiers and clouds, the grouping mechanism is supported as a sub-element of atp_site, i.e.

  • group - any number of group elements can be specified with the following attributes:
    • name - name of the group (mandatory)
    • type - type of the group (mandatory)
There are two types of groups that are mandatory for each service, i.e. those that indicate experiment site name and experiment tier, they're specified with VO prefix, e.g. type=ATLAS_Site and type=ATLAS_Tier, e.g.

<atp_site name="CSCS-LCG2">
...
    <group name="Tier-2" type="CMS_Tier"/>                  <--- mandatory
    <group name="T2_CH_CSCS" type="CMS_Site"/>    <--- mandatory
    <group name="T2_CH_CSCS" type="AllGroups"/>    <--- optional
    <group name="T2_CH_CSCS" type="Tier2s"/>          <--- optional, etc.
    <group name="T2_CH_CSCS" type="Tier3s + Tier2s"/>
    <group name="T2_CH_CSCS" type="Tier2s + Tier1s + Tier0"/>
    <group name="T2_CH_CSCS" type="Tier3s + Tier2s + Tier1s"/>
</atp_site

There are two optional sub-elements of service available for CE and SE service types; ce_resource listing existing queues for given CE; and se_resource listing existing paths and spacetokens for a given SE:

  • ce_resource - specifies different queues supported by the CE (optional, but will become mandatory), with the following attributes:
    • fqan - limits the supported queues to selected FQDN (optional)
    • queue - batch queue that can be used to submit jobs (mandatory)
    • batch_system - batch system that is deployed (mandatory for CREAM-CEs)
    • etf_default - (under discussion) - if multiple queues are listed, identifies the queue to be preferred by ETF (http://etf.cern.ch/docs/) - can be true or false (mandatory if more than one ce_resource per service specified)
  • se_resource - specifies different paths and space tokens supported by the SE (optional), with the following attributes:
    • path - specifies path supported by the SE (mandatory)
    • token - specifies space token supported by the SE (optional)
    • id - specified a unique identifier for path, token tuple (mandatory)

An example follows:

<service endpoint="agcream1.atlas.unimelb.edu.au:8443" flavour="CREAM-CE" hostname="agcream1.atlas.unimelb.edu.au">
    <ce_resource batch_system="pbs" queue="atlas" etf_default=true/>
    <ce_resource batch_system="pbs" queue="atlas-test" etf_default=false/>
    <ce_resource batch_system="pbs" queue="atlas-test2" etf_default=false/>
</service>

<service endpoint="root://xrootd.aglt2.org:1094" flavour="XROOTD" hostname="xrootd.aglt2.org">
    <se_resource id="AGLT2_CALIBDISK" path="//pnfs/aglt2.org/atlascalibdisk/rucio/" token="ATLASCALIBDISK"/>
    <se_resource id="AGLT2_DATADISK" path="//pnfs/aglt2.org/atlasdatadisk/rucio/" token="ATLASDATADISK"/>
    <se_resource id="AGLT2_LOCALGROUPDISK" path="//pnfs/aglt2.org/atlaslocalgroupdisk/rucio/" token="ATLASLOCALGROUPDISK"/>
    <se_resource id="AGLT2_PERF-MUONS" path="//pnfs/aglt2.org/atlasgroupdisk/perf-muons/rucio/" token="ATLASGROUPDISK"/>
    <se_resource id="AGLT2_PHYS-HIGGS" path="//pnfs/aglt2.org/atlasgroupdisk/phys-higgs/rucio/" token="ATLASGROUPDISK"/>
    <se_resource id="AGLT2_PHYS-SM" path="//pnfs/aglt2.org/atlasgroupdisk/phys-sm/rucio/" token="ATLASGROUPDISK"/>
    <se_resource id="AGLT2_SCRATCHDISK" path="//pnfs/aglt2.org/atlasscratchdisk/rucio/" token="ATLASSCRATCHDISK"/>
    <se_resource id="AGLT2_USERDISK" path="//pnfs/aglt2.org/atlasuserdisk/rucio/" token="ATLASUSERDISK"/>
</service>

For service elements of flavour XROOTD specifying the endpoint is mandatory.

Validation

Sample VO feed can be validated against the latest schema in two steps using the files attached to this twiki, e.g.

$ xsltproc validateFeed-1.xslt <vofeed.xml>  > sitemap_transformed
$ xmllint --noout --schema feed_schema2-1.xsd sitemap_transformed

Implementation

The following feeds are available (some might be only accessible from CERN):

VO Maintained Deps URL
ATLAS AGIS AGIS (GOCDB/OIM, etc.) http://atlas-agis-api.cern.ch/request/atp/xml/
ATLAS AGIS AGIS (GOCDB/OIM, etc.) QA: http://atlas-agis-api.cern.ch/request/atp2/xml/
CMS SSB (IT/MM) SiteDB, PhEDEx, Glide-in Factories http://dashb-cms-vo-feed.cern.ch/dashboard/request.py/cmssitemapbdii
CMS SSB (IT/MM) SiteDB, PhEDEx, Glide-in Factories QA: http://cmssst.web.cern.ch/cmssst/vofeed/dev/vofeed.xml
LHCb DIRAC DIRAC http://lhcb-portal-dirac.cern.ch/topology/lhcb_topology.xml
Alice SSB (IT/MM) ALICE LDAP http://wlcg-sam-alice.cern.ch/dashboard/request.py/alicesitemap
DUNE DUNE CRIC OSG Pilot Factories https://dune-cric.cern.ch/api/dune/vofeed/list/

-- MariaALANDESPRADILLO - 2016-11-21

Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatxsd feed_schema2-1.xsd r1 manage 3.2 K 2016-11-30 - 16:55 MarianBabik  
Unknown file formatxslt validateFeed-1.xslt r1 manage 4.3 K 2016-11-30 - 16:55 MarianBabik  
Edit | Attach | Watch | Print version | History: r13 < r12 < r11 < r10 < r9 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r13 - 2020-09-03 - AndrewMcNab
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EGEE All webs login

This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright & by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Ask a support question or Send feedback