genCatalog is now obsolete... Use the genXMLCatalog command instead

Using the genCatalog script

bookkeeping web interface

• Physical file names (PFN): these are targeted to a site and a given file access protocol. Supported protocols depend on the version of Gaudi in use:
  • Versions of Gaudi prior to v18r6: castor: (CERN), rfio: (CERN, PIC, CNAF), dcache: (GridKa, RAL)
  • As from v18r6: in addition to the above, Gaudi can access files through the gfal (Grid File Access Library) plugin of ROOT and the following protocols are supported:
    • gfal:srm: accessing files at a given site through its SRM URL (SURL)
    • gfal:lfn: accessing files at any site through its Logical File Name (LFN)
    • gfal:uid: accessing files at any site through its Global Unique IDentifier (GUID)
    • gfal:gsidcap: on sites using the secure dCache protocol only
• Logical File Names (LFN): these are not targeted to a given site and allow the jobs to run on the Grid. The application has to resolve it to a PFN by means of a File Catalog (FC). The FC can either be the LHCb full catalog (LFC) or a local restricted catalog in XML language (XML catalog). Querying the bookkeeping databse with the web interface, one can either get directly LFNs in jobOptions using the gfal:lfn: protocol (uses the LFC) or the LFC: protocol that needs an XML catalog. Both have different use cases:
  • gfal:lfn: protocol : this is the most straight-forward protocol to use. However it requires the job to run with valid Grid credentials (a.k.a. Grid proxy). This means the user has to have a Grid certificate and provide a valid proxy to the job (if not running itself on the Grid).
  • LFN: protocol : probably the easiest for small tests on a local batch system (e.g. LXBATCH). It needs to be complemented by an XML file catalog (or new jobOptions have to be built with PFNs) using the genCatalog script. Gaudi has to be instructed to use that XML catalog by adding one option that is reminded by genCatalog when it is run.

The genCatalog script allows translating a list of files given by their LFN in jobOptions into one of the above protocols. This can be acheieved in two ways:

• A new jobOption file is created that contains the new options.
• An XML file catalog is created that allows Gaudi to translate the LFN in the jobOptions file to the appropriate PFN. One can also require that the parent files of the files used are included in the catalog (see the -d --depth option). This is necessary for navigating to ancestor files or using Event Tag Collections.

Here is the output of the command

genCatalog --help

 Options:
    -o , --options=
       Job option file to be processed for DATAFILE declarations
    -C , --component=
       Name of a component property to be checked in order to access
       input file declarations (default: EventSelector.Input)

    -p , --pool= 
       Generate POOL file catalog and specify the name. Must be used with -s, -P options
    -c , --Card= 
       Generate the Gaudi card and specify the name. Must be used with -s, -P options

    -g , --gaudi=
       - If an options file was specified, generate GAUDI file catalog
         using the information from the specified options files and the
         information in the LFC database
       - If no options are present assume, that a gaudi catalog exists
         and use this file as an input to produce a POOL file catalog
         slice for a given computing site

    -s, --site=
       Name of the current Site to generate the POOL catalog slice.
       Must be used with -p, -P options

    -d, --depth=
       Depth of input sources contained in file catalog

    -P, --Protocol=
       To generate the PFN with the specified protocol
       Must be used with -s, -p options
    -N, --nostrict Disable strict processing

    -h, --help     Print this help


    -u, --url=

    -i, --info   Print all the informations about sites, storage elemts
                 and protocols. 

    Examples:
    
    $> genCatalog -o Brunel.opts -g myCatalog.xml -C EventSelector.Input -C SpilloverSelector.Input
       Will produce a complete catalog slice according to the LFC database
 
    $> genCatalog -g myCatalog.xml -p poolCatalog.xml -s CERN -P rfio
       Will produce a POOL file catalog slice according using the LFC 
       database slice in myCatalog.xml and will project all files, which
       are availible at CERN.

    $> genCatalog -o Brunel.opts -p poolCatalog.xml -s CERN -P rfio
       Will produce a POOL file catalog slice according to the LFC database
       for CERN only; 

    $> genCatalog -o Brunel.opts -c gaudicard.opts -s CERN -P rfio
       Will produce a Gaudi card according to the LFC database
       for CERN only;