Storage Description

Storage and storage access varies across the sites in the CMS computing grid. Production workflows, user analysis jobs, and central data transfers need to know where data are and how to best access them. During Run 1 and 2 the storage description of PhEDEx, the Trivial File Catalogue (TFC), was contained in SITECONF/PhEDEx/storage.xml and referenced in site-local-config.xml for data reading and stage out. The TFC provided powerfull path translation that is now rarely used but makes reading and maintaining the TFC strenuous. The switch to Rucio provides an opportunity for a revised storage description.

storage.json

The new storage description will be in JSON format. One file should allow to contain all storage description, of a site, or just a sub-site. At most sites, the CMS Logical File Name (LFN) space is at a fixed prefix and thus no complex path translation between logical and physical file path is required. Multiple access protocols, for reading, writing, on-site, and off-site need to be distinguished. Rucio Storage Element (RSE) needs to be known for centrally managed data transfers as well as File Transfer Service (FTS) servers to be used.

The JSON structure is thus a list of site (or sub-site) storage entities ech of which provides an /store LFN space. Storage inside an entity will be treated by CMS as a unit. It can span across one or more grid Storage Elements (SE). The entity is defined by site name (potentially sub-site name) and volume name. Each entity is described by a list of supported protocols, storage type, RSE, and FTS server(s). For each supported protocol the name space translation is specified as either a simple path prefix or more complex "TFC rule". Predefined protocol names are used for centrally managed data transfers ("SRMv2", "XRootD", and "WebDAV" currently).

The storage description will be maintained in the SITECONF GitLab (potentially moved to CRIC once the description is stable) and provided to compute servers and worker nodes via CVMFS (as the TFC is currently distributed).

Example:

[
   {  "site": "Tx_CC_Sitename",
      "volume": "DiskStorageSystem",
      "protocols": [
         {  "protocol": "posix",
            "access": "site-ro",
            "prefix": "file://mount-point/cms-directory"
         },
         {  "protocol": "xrd",
            "access": "site-ro",
            "prefix": "root://onsite-redirector.domain/"
         },
         {  "protocol": "XRootD",
            "access": "global-rw",
            "prefix": "root://site-endpoint.domain/",
            "comment": "xrootd data transfer and stage-out"
         },
          {  "protocol": "SRMv2",
            "access": "global-rw",
            "prefix": "gsiftp://srm-endpoint.domain/"
         }
      ],
      "type": "DISK",
      "rse": "Tx_CC_Sitename_Disk",
      "fts": [ "https://cmsfts3.some.domain:8443" ]
   },

   {  "site": "Tx_CC_Sitename",
      "volume": "TapeLibrary",
      "protocols": [
          {  "protocol": "direct",
             "access": "virtual",
             "rules": [
                {  "lfn": ".*/LoadTest07_Sitename_(.*)_.*_.*",
                   "pfn": "/dir/subdir/store/LoadTest07/Sitename_Tape_$1"
                },
                {  "lfn": ".*/LoadTest07_LoadTest07_Sitename_(.*)_.*_.*",
                    "pfn": "/dir/subdir/store/LoadTest07/Sitename_Tape_$1"
                }
          },
          {  "protocol": "SRMv2",
            "access": "global-rw",
            "rules": [
               {  "lfn": "/+(.*)",
                  "pfn": "srm://srm-tape.domain:8443/srm/managerv2\\?SFN=/$1",
                  "chain": "direct"
               }
            ]
         }
      ],
      "type": "TAPE",
      "rse": "Tx_CC_Sitename_Tape",
      "fts": [ "https://cmsfts3.some.domain:8443" ]
   }
]

• site : CMS sitename of the form Tn_CC_Name
• volume : Name of the storage entity at the site (name is chosen by site and should not contain whitespaces)
• protocol : Name of the access protocol (name is chosen by site; predefined/reserved names for centrally managed transfers are currently"SRMv2", "XRootD", and "WebDAV")
• access : Type of access the protocol provides (must be either "virtual", "site-ro", "site-rw", "global-ro", or "global-rw")
• prefix : A protocol, host, and/or path to be prefixed to the logical filename to translate into the physical filename location
• rules : List of lfn/pfn/chain dictionaries to specify a more complex name space translation similar to the old PhEDEx TFC
• lfn : Regular expression to select the logical filename portion to which the path translation applies
• pfn : Physical filename translation of the logical filename matching the regular expression in lfn
• chain : Name of protocol in the same storage entity whose path translation should be applied before the one specified in this entry
• type : Type of storage (must be either DISK or TAPE, currently)
• rse : Name of the Rucio storage element associated with the storage entity
• fts : Ordered list of FTS3 servers, can be omitted (or empty) and a default server will be used

site-local-config.xml

Data access and stage-out specifications in site-local-config.xml will then point to an ordered list of one or more storage entities (and protocol) in the new storage.json file. The ordered list will allow multiple data read/stage-out methods similar to the current second catalog/fallback stage-out entries. (TFC entries for fallback stage-out need no longer be duplicated in site-local-config.xml.) Reading of data by jobs would proceed (but with different parsing) as currently. Similarly stage-out. Old CMSSW versions will not know about the new storage description. To keep things compatible, the old site-local-config.xml entries and storage.xml files would be kept. In site-local-config.xml a new tag data-access and an additional stage-out tag would use the new storage.json. For example

<data-access>
   <catalog volume="DiskStorageSystem"/>
   <catalog>
      <site>T1_CC_MyTier1</site>
      <volume>CountyXrootdFederation</volume>
      <protocol>national-xrootd</protocol>
   </catalog>
</data-access>

<stage-out>
   <method volume="DiskStorageSystem" protocol="SRMv2" command="gfal2" option="-v"/>
   <method site="Tx_CC_Neighbour" volume="DiskStorageSystem"/>
</stage-out>