genCatalog is now obsolete... Use the genXMLCatalog command instead
Using the genCatalog script
The LHCb
bookkeeping web interface
allows to query the LHCb datasets is several ways:
- Physical file names (PFN): these are targeted to a site and a given file access protocol. Supported protocols depend on the version of Gaudi in use:
- Versions of Gaudi prior to v18r6: castor: (CERN), rfio: (CERN, PIC, CNAF), dcache: (GridKa, RAL)
- As from v18r6: in addition to the above, Gaudi can access files through the gfal (Grid File Access Library) plugin of ROOT and the following protocols are supported:
- gfal:srm: accessing files at a given site through its SRM URL (SURL)
- gfal:lfn: accessing files at any site through its Logical File Name (LFN)
- gfal:uid: accessing files at any site through its Global Unique IDentifier (GUID)
- gfal:gsidcap: on sites using the secure dCache protocol only
- Logical File Names (LFN): these are not targeted to a given site and allow the jobs to run on the Grid. The application has to resolve it to a PFN by means of a File Catalog (FC). The FC can either be the LHCb full catalog (LFC) or a local restricted catalog in XML language (XML catalog). Querying the bookkeeping databse with the web interface, one can either get directly LFNs in jobOptions using the gfal:lfn: protocol (uses the LFC) or the LFC: protocol that needs an XML catalog. Both have different use cases:
- gfal:lfn: protocol : this is the most straight-forward protocol to use. However it requires the job to run with valid Grid credentials (a.k.a. Grid proxy). This means the user has to have a Grid certificate and provide a valid proxy to the job (if not running itself on the Grid).
- LFN: protocol : probably the easiest for small tests on a local batch system (e.g. LXBATCH). It needs to be complemented by an XML file catalog (or new jobOptions have to be built with PFNs) using the genCatalog script. Gaudi has to be instructed to use that XML catalog by adding one option that is reminded by genCatalog when it is run.
The genCatalog script allows translating a list of files given by their LFN in jobOptions into one of the above protocols. This can be acheieved in two ways:
- A new jobOption file is created that contains the new options.
- An XML file catalog is created that allows Gaudi to translate the LFN in the jobOptions file to the appropriate PFN. One can also require that the parent files of the files used are included in the catalog (see the -d --depth option). This is necessary for navigating to ancestor files or using Event Tag Collections.
Here is the output of the command
genCatalog --help
Options:
-o , --options=
Job option file to be processed for DATAFILE declarations
-C , --component=
Name of a component property to be checked in order to access
input file declarations (default: EventSelector.Input)
-p , --pool=
Generate POOL file catalog and specify the name. Must be used with -s, -P options
-c , --Card=
Generate the Gaudi card and specify the name. Must be used with -s, -P options
-g , --gaudi=
- If an options file was specified, generate GAUDI file catalog
using the information from the specified options files and the
information in the LFC database
- If no options are present assume, that a gaudi catalog exists
and use this file as an input to produce a POOL file catalog
slice for a given computing site
-s, --site=
Name of the current Site to generate the POOL catalog slice.
Must be used with -p, -P options
-d, --depth=
Depth of input sources contained in file catalog
-P, --Protocol=
To generate the PFN with the specified protocol
Must be used with -s, -p options
-N, --nostrict Disable strict processing
-h, --help Print this help
-u, --url=
-i, --info Print all the informations about sites, storage elemts
and protocols.
Examples:
$> genCatalog -o Brunel.opts -g myCatalog.xml -C EventSelector.Input -C SpilloverSelector.Input
Will produce a complete catalog slice according to the LFC database
$> genCatalog -g myCatalog.xml -p poolCatalog.xml -s CERN -P rfio
Will produce a POOL file catalog slice according using the LFC
database slice in myCatalog.xml and will project all files, which
are availible at CERN.
$> genCatalog -o Brunel.opts -p poolCatalog.xml -s CERN -P rfio
Will produce a POOL file catalog slice according to the LFC database
for CERN only;
$> genCatalog -o Brunel.opts -c gaudicard.opts -s CERN -P rfio
Will produce a Gaudi card according to the LFC database
for CERN only;
-- Main.phicharp - 31 Jul 2006