TWiki>
LHCb Web>LHCbComputing>LHCbDiracCLI>LHCbDiracBKCLI (2020-09-30, MichaelWilkinson)
EditAttachPDF
LHCb Web>LHCbComputing>LHCbDiracCLI>LHCbDiracBKCLI (2020-09-30, MichaelWilkinson) EditAttachPDFLHCb Bookkeeping commands
The LHCb Bookkeeping allows many queries from the command line interface. Many of them use the DMS option switches. Note that the output of many commands can be piped into others as explained in the DMS twiki.Before executing the commands in this tutorial
If you want to work with LHCbDirac in bash, instead open a sub-shell with> lb-dirac bash --norcYou will then need to do
> lhcb-proxy-initOnce these things are done, the following commands should work within the bash sub-shell you just opened.
Information on files
dirac-bookkeeping-get-files
Get a list of files for a BK query. The list of files can be saved in a file or piped to an other DMS command.Syntax
$ dirac-bookkeeping-get-files --help Retrieve files of a BK query Usage: dirac-bookkeeping-get-files [option|cfgfile] ... ProdID Type General options: -o: --option= : Option=value to add -s: --section= : Set base section for relative parsed options -c: --cert= : Use server certificate to connect to Core Services -d --debug : Set debug mode (-dd is extra debug) -h --help : Shows this help Options: -P: --Productions= : Production ID to search (comma separated list) -f: --FileType= : File type (comma separated list, to be used with --Production) [All] - --ExceptFileType= : Exclude the (list of) file types when all are requested -B: --BKQuery= : Bookkeeping query path -r: --Runs= : Run or range of runs (r1:r2) - --DQFlags= : DQ flag used in query - --StartDate= : Start date for the BK query - --EndDate= : End date for the BK query - --Visibility= : Set visibility (Yes, No, All) [Yes] - --Output= : Specify a file that will contain the list of files - --MaxFiles= : Print only <MaxFiles> lines on stdout (20 if output, else All)
Example
$ dirac-bookkeeping-get-files --Prod 24847 --FileType FULL.DST --Output myTest.txt
Using BKQuery: {'Visible': 'Yes', 'FileType': 'FULL.DST', 'Production': [24847]}
110 files found (showing only first 20 files):
FileName Size GUID Replica Visible
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000001_1.full.dst 4815735657 8E8F17D3-D7D6-E211-B39C-003048F17956 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000002_1.full.dst 4777687666 90301242-D3D8-E211-AC15-002590352F86 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000003_1.full.dst 4882358770 6093D440-C6D8-E211-AC36-0025901D5CEE Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000004_1.full.dst 4862398870 E80B1DFB-2ADA-E211-A6E6-003048F1DB84 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000005_1.full.dst 4812022999 783B2687-2BDA-E211-83D2-BCAEC5579F00 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000006_1.full.dst 4819311669 A4687FD0-D1D8-E211-ADDF-0026B93B4115 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000007_1.full.dst 4882275421 98FDF292-C2D8-E211-BD17-7845C4FC3608 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000009_1.full.dst 4825949667 BAE21F92-C1D8-E211-A192-E0CB4E4408DD Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000010_1.full.dst 4280255461 622237D4-29DA-E211-A063-003048F00B44 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000011_1.full.dst 4917138693 F8F79D7D-D2D8-E211-A551-0025903534FE Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000012_1.full.dst 4853093706 D44CF54B-CCD8-E211-97BC-003048C45C8A Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000013_1.full.dst 5026741560 16091F6A-CCD8-E211-A48F-001B21ADD058 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000014_1.full.dst 2706044397 D6B1059F-CFD8-E211-A717-90E2BA034BAC Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000015_1.full.dst 4834844058 48D003D7-D1D8-E211-9327-0025903531D0 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000016_1.full.dst 4870141544 F27E00C4-CBD8-E211-9928-1803730B049D Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000017_1.full.dst 4883526421 2ADA99CC-2BDA-E211-9E92-00259094140E Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000018_1.full.dst 4912126905 64A7D618-D4D8-E211-BACB-0026B93B3E92 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000019_1.full.dst 4862828212 7A118B85-D3D8-E211-B025-00259035AE22 Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000020_1.full.dst 1718014956 BCB0438E-CCD8-E211-A547-3640B5BA49AC Yes ?
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000021_1.full.dst 4841828540 000DA63E-CED8-E211-BC66-1803730B5105 Yes ?
List of 110 files saved in myTest.txt
dirac-bookkeeping-file-metadata
Retrieves the BK metadata of a (list of) file(s). By default only a summary is given while with--Full option one gets all information
Syntax
$ dirac-bookkeeping-file-metadata --help Retrieve metadata from the Bookkeeping for the given files Usage: dirac-bookkeeping-file-metadata [option|cfgfile] ... LFN|File Arguments: LFN: Logical File Name File: Name of the file with a list of LFNs General options: -o: --option= : Option=value to add -s: --section= : Set base section for relative parsed options -c: --cert= : Use server certificate to connect to Core Services -d --debug : Set debug mode (-dd is extra debug) -h --help : Shows this help Options: - --File= : File containing list of LFNs -l: --LFNs= : List of LFNs (comma separated) - --Terminal : LFNs are entered from stdin (--File /dev/stdin) - --Full : Print out all metadata
Examples
$ dirac-bookkeeping-file-metadata /lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000001_1.full.dst FileName Size GUID Replica DataQuality RunNumber #events /lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000001_1.full.dst 4815735657 8E8F17D3-D7D6-E211-B39C-003048F17956 Yes OK 115327 49797 $ dirac-bookkeeping-file-metadata /lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000001_1.full.dst --Full FileName /lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000001_1.full.dst ADLER32 None CreationDate None DQFlag OK EventStat 49797 EventType 90000000 FileSize 4815735657 FileType FULL.DST FullStat None GUID 8E8F17D3-D7D6-E211-B39C-003048F17956 GotReplica Yes InsertTimeStamp 2013-06-25 11:45:39.300944 InstLuminosity 0 JobId 129552274 Luminosity 4085.68992445 MD5SUM 515cec21800944c0947da0088fd4b2cb RunNumber 115327
dirac-bookkeeping-genXMLCatalog
Allows to get an XML catalog from a list of files or from a Gaudi options file (it can also create a new options file containing PFNs).Syntax
$ dirac-bookkeeping-genXMLCatalog --help Create a pool xml catalog slice for the specified LFNs Usage: dirac-bookkeeping-genXMLCatalog [option|cfgfile] ... General options: -o: --option= : Option=value to add -s: --section= : Set base section for relative parsed options -c: --cert= : Use server certificate to connect to Core Services -d --debug : Set debug mode (-dd is extra debug) -h --help : Shows this help Options: -g: --Sites= : Sites to consider [ALL] (comma separated list) -S: --SEs= : SEs to consider [ALL] (comma separated list) - --File= : File containing list of LFNs -l: --LFNs= : List of LFNs (comma separated) - --Terminal : LFNs are entered from stdin (--File /dev/stdin) - --Options= : List of option files to consider - --NewOptions= : Name of a new options file to be generated with LFNs (default: none) - --Catalog= : Catalog name (default: pool_xml_catalog.xml - --Depth= : Depth for ancestor consideration (default: none) -v --Verbose : Verbose gLogger.always(out - --Ignore : Ignore missing files - --GaudiVerbose : Set Gaudi verbose when parsing option files
Example
Creating an XML catalog
$ cat example.py
from Gaudi.Configuration import *
from GaudiConf import IOHelper
IOHelper().inputFiles([
'LFN:/lhcb/LHCb/Collision11/BHADRONCOMPLETEEVENT.DST/00022761/0001/00022761_00018925_1.bhadroncompleteevent.dst'
], clear=True)
$ dirac-bookkeeping-genXMLCatalog --File=example.py --Catalog /dev/stdout
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<!-- Edited By PoolXMLCatalog.py -->
<!DOCTYPE POOLFILECATALOG SYSTEM "InMemory">
<POOLFILECATALOG>
<File ID="8E7DCD3B-3680-E211-8528-003048F00B24">
<physical>
<pfn filetype="ROOT" name="root://eoslhcb.cern.ch//eos/lhcb/LHCb/Collision11/BHADRONCOMPLETEEVENT.DST/00022761/0001/00022761_00018925_1.bhadroncompleteevent.dst"/>
</physical>
<logical>
<lfn name="/lhcb/LHCb/Collision11/BHADRONCOMPLETEEVENT.DST/00022761/0001/00022761_00018925_1.bhadroncompleteevent.dst"/>
</logical>
</File>
</POOLFILECATALOG>
POOL XML catalog /dev/stdout successfully created
Creating a new options file with PFNs:
[localhost, Patch] ~ $ dirac-bookkeeping-genXMLCatalog --Option example.py --NewOption examplePFN.py New options file examplePFN.py successfully created [localhost, Patch] ~ $ cat examplePFN.py from Gaudi.Configuration import * from GaudiConf import IOHelper IOHelper().inputFiles([ 'root://eoslhcb.cern.ch//eos/lhcb/LHCb/Collision11/BHADRONCOMPLETEEVENT.DST/00022761/0001/00022761_00018925_1.bhadroncompleteevent.dst' ], clear=True)
dirac-bookkeeping-file-path
Allows to get the BK path for a file or list of files. It is quite handy to know what is the origin of that file.Syntax
$ dirac-bookkeeping-file-path --help Return the BK path for the directories of a (list of) files Usage: dirac-bookkeeping-file-path [option|cfgfile] ... LFN|File Arguments: LFN: Logical File Name File: Name of the file with a list of LFNs General options: -o: --option= : Option=value to add -s: --section= : Set base section for relative parsed options -c: --cert= : Use server certificate to connect to Core Services -d --debug : Set debug mode (-dd is extra debug) -h --help : Shows this help Options: - --File= : File containing list of LFNs -l: --LFNs= : List of LFNs (comma separated) - --Terminal : LFNs are entered from stdin (--File /dev/stdin) - --Full : Print out full BK dictionary (default: print out BK path) - --GroupBy= : Return a list of files per <metadata item> - --GroupByPath : Return a list of files per BK path - --GroupByProduction : Return a list of files per production
Examples
$ dirac-bookkeeping-file-path /lhcb/LHCb/Collision12/FULL.DST/00024847/0000/00024847_00000001_1.full.dst
Failed : []
Successful :
/lhcb/LHCb/Collision12/FULL.DST/00024847/0000 : /LHCb/Collision12/Beam4000GeV-VeloClosed-MagDown/Real Data/Reco14/90000000/FULL.DST
[localhost, Patch] ~ $ cat files.txt
/lhcb/LHCb/Collision11/BHADRONCOMPLETEEVENT.DST/00024034/0000/00024034_00002238_1.bhadroncompleteevent.dst
/lhcb/LHCb/Collision11/CHARM.MDST/00024034/0000/00024034_00002240_1.charm.mdst
/lhcb/LHCb/Collision11/CHARMCOMPLETEEVENT.DST/00024034/0000/00024034_00002242_1.charmcompleteevent.dst
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00001915_1.dimuon.dst
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00002244_1.dimuon.dst
/lhcb/LHCb/Collision11/EW.DST/00024034/0000/00024034_00002246_1.ew.dst
/lhcb/LHCb/Collision11/LEPTONIC.MDST/00024034/0000/00024034_00002248_1.leptonic.mdst
/lhcb/LHCb/Collision11/PID.MDST/00024034/0000/00024034_00002250_1.pid.mdst
/lhcb/LHCb/Collision11/RADIATIVE.DST/00024034/0000/00024034_00002252_1.radiative.dst
/lhcb/LHCb/Collision11/SEMILEPTONIC.DST/00024034/0000/00024034_00002254_1.semileptonic.dst
$ dirac-bookkeeping-file-path --File files.txt --GroupByPath
Successful :
/LHCb/Collision11/Beam3500GeV-VeloClosed-MagUp/Real Data/Reco14/Stripping20r1p1/90000000/BHADRONCOMPLETEEVENT.DST :
/lhcb/LHCb/Collision11/BHADRONCOMPLETEEVENT.DST/00024034/0000/00024034_00002238_1.bhadroncompleteevent.dst
/LHCb/Collision11/Beam3500GeV-VeloClosed-MagUp/Real Data/Reco14/Stripping20r1p1/90000000/CHARM.MDST :
/lhcb/LHCb/Collision11/CHARM.MDST/00024034/0000/00024034_00002240_1.charm.mdst
/LHCb/Collision11/Beam3500GeV-VeloClosed-MagUp/Real Data/Reco14/Stripping20r1p1/90000000/CHARMCOMPLETEEVENT.DST :
/lhcb/LHCb/Collision11/CHARMCOMPLETEEVENT.DST/00024034/0000/00024034_00002242_1.charmcompleteevent.dst
/LHCb/Collision11/Beam3500GeV-VeloClosed-MagUp/Real Data/Reco14/Stripping20r1p1/90000000/DIMUON.DST :
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00001915_1.dimuon.dst
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00002244_1.dimuon.dst
/LHCb/Collision11/Beam3500GeV-VeloClosed-MagUp/Real Data/Reco14/Stripping20r1p1/90000000/EW.DST :
/lhcb/LHCb/Collision11/EW.DST/00024034/0000/00024034_00002246_1.ew.dst
/LHCb/Collision11/Beam3500GeV-VeloClosed-MagUp/Real Data/Reco14/Stripping20r1p1/90000000/LEPTONIC.MDST :
/lhcb/LHCb/Collision11/LEPTONIC.MDST/00024034/0000/00024034_00002248_1.leptonic.mdst
/LHCb/Collision11/Beam3500GeV-VeloClosed-MagUp/Real Data/Reco14/Stripping20r1p1/90000000/PID.MDST :
/lhcb/LHCb/Collision11/PID.MDST/00024034/0000/00024034_00002250_1.pid.mdst
/LHCb/Collision11/Beam3500GeV-VeloClosed-MagUp/Real Data/Reco14/Stripping20r1p1/90000000/RADIATIVE.DST :
/lhcb/LHCb/Collision11/RADIATIVE.DST/00024034/0000/00024034_00002252_1.radiative.dst
/LHCb/Collision11/Beam3500GeV-VeloClosed-MagUp/Real Data/Reco14/Stripping20r1p1/90000000/SEMILEPTONIC.DST :
/lhcb/LHCb/Collision11/SEMILEPTONIC.DST/00024034/0000/00024034_00002254_1.semileptonic.dst
$ dirac-bookkeeping-file-path --File files.txt --GroupByProduction
Successful :
Production 24034 :
/lhcb/LHCb/Collision11/BHADRONCOMPLETEEVENT.DST/00024034/0000/00024034_00002238_1.bhadroncompleteevent.dst
/lhcb/LHCb/Collision11/CHARM.MDST/00024034/0000/00024034_00002240_1.charm.mdst
/lhcb/LHCb/Collision11/CHARMCOMPLETEEVENT.DST/00024034/0000/00024034_00002242_1.charmcompleteevent.dst
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00001915_1.dimuon.dst
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00002244_1.dimuon.dst
/lhcb/LHCb/Collision11/EW.DST/00024034/0000/00024034_00002246_1.ew.dst
/lhcb/LHCb/Collision11/LEPTONIC.MDST/00024034/0000/00024034_00002248_1.leptonic.mdst
/lhcb/LHCb/Collision11/PID.MDST/00024034/0000/00024034_00002250_1.pid.mdst
/lhcb/LHCb/Collision11/RADIATIVE.DST/00024034/0000/00024034_00002252_1.radiative.dst
/lhcb/LHCb/Collision11/SEMILEPTONIC.DST/00024034/0000/00024034_00002254_1.semileptonic.dst
$ dirac-bookkeeping-file-path --File files.txt --GroupBy RunNumber
Failed : []
Successful :
RunNumber 100254 :
/lhcb/LHCb/Collision11/BHADRONCOMPLETEEVENT.DST/00024034/0000/00024034_00002238_1.bhadroncompleteevent.dst
/lhcb/LHCb/Collision11/CHARM.MDST/00024034/0000/00024034_00002240_1.charm.mdst
/lhcb/LHCb/Collision11/CHARMCOMPLETEEVENT.DST/00024034/0000/00024034_00002242_1.charmcompleteevent.dst
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00001915_1.dimuon.dst
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00002244_1.dimuon.dst
/lhcb/LHCb/Collision11/EW.DST/00024034/0000/00024034_00002246_1.ew.dst
/lhcb/LHCb/Collision11/LEPTONIC.MDST/00024034/0000/00024034_00002248_1.leptonic.mdst
/lhcb/LHCb/Collision11/PID.MDST/00024034/0000/00024034_00002250_1.pid.mdst
/lhcb/LHCb/Collision11/RADIATIVE.DST/00024034/0000/00024034_00002252_1.radiative.dst
/lhcb/LHCb/Collision11/SEMILEPTONIC.DST/00024034/0000/00024034_00002254_1.semileptonic.dst
Ancestors and descendants
dirac-bookkeeping-get-file-ancestors
Gets a list of ancestors for a given file or list of files. One can define the level up to which to go up as well as also get files without replicas.Syntax
$ dirac-bookkeeping-get-file-ancestors --help Report ancestors for the given LFNs Usage: dirac-bookkeeping-get-file-ancestors [option|cfgfile] ... [LFN|File] [Level] Arguments: LFN: Logical File Name File: Name of the file with a list of LFNs Level: Number of levels to search (default: 1) General options: -o: --option= : Option=value to add -s: --section= : Set base section for relative parsed options -c: --cert= : Use server certificate to connect to Core Services -d --debug : Set debug mode (-dd is extra debug) -h --help : Shows this help Options: - --File= : File containing list of LFNs -l: --LFNs= : List of LFNs (comma separated) - --Terminal : LFNs are entered from stdin (--File /dev/stdin) - --All : Do not restrict to descendants with replicas - --Full : Get full metadata information on ancestors - --Depth= : Number of processing levels (default:1)
Examples
$ dirac-bookkeeping-get-file-ancestors /lhcb/LHCb/Collision11/CHARMTOBESWUM.DST/00022760/0008/00022760_00088328_1.CharmToBeSwum.dst 2
Failed : []
Successful :
/lhcb/LHCb/Collision11/CHARMTOBESWUM.DST/00022760/0008/00022760_00088328_1.CharmToBeSwum.dst :
/lhcb/LHCb/Collision11/FULL.DST/00022754/0006/00022754_00060707_1.full.dst : Replica-Yes
/lhcb/data/2011/RAW/FULL/LHCb/COLLISION11/100254/100254_0000000130.raw : Replica-Yes
$ dirac-bookkeeping-get-file-ancestors /lhcb/LHCb/Collision11/CHARMTOBESWUM.DST/00022760/0008/00022760_00088328_1.CharmToBeSwum.dst
Failed : []
Successful :
/lhcb/LHCb/Collision11/CHARMTOBESWUM.DST/00022760/0008/00022760_00088328_1.CharmToBeSwum.dst :
/lhcb/LHCb/Collision11/FULL.DST/00022754/0006/00022754_00060707_1.full.dst : Replica-Yes
$ dirac-bookkeeping-get-file-ancestors /lhcb/LHCb/Collision11/CHARMTOBESWUM.DST/00022760/0008/00022760_12388328_1.CharmToBeSwum.dst
Failed :
/lhcb/LHCb/Collision11/CHARMTOBESWUM.DST/00022760/0008/00022760_12388328_1.CharmToBeSwum.dst
dirac-bookkeeping-get-file-descendants
Gets a list of descendants for a given file or list of files. One can define the level up to which to go down as well as also get files without replicas. One may also provide a production number for which to look at in processing chain.Syntax
$ dirac-bookkeeping-get-file-descendants --help Report descendants for the given LFNs Usage: dirac-bookkeeping-get-file-descendants [option|cfgfile] ... [LFN|File] [Level] Arguments: LFN: Logical File Name File: Name of the file with a list of LFNs Level: Number of levels to search (default: 1) General options: -o: --option= : Option=value to add -s: --section= : Set base section for relative parsed options -c: --cert= : Use server certificate to connect to Core Services -d --debug : Set debug mode (-dd is extra debug) -h --help : Shows this help Options: - --File= : File containing list of LFNs -l: --LFNs= : List of LFNs (comma separated) - --Terminal : LFNs are entered from stdin (--File /dev/stdin) - --All : Do not restrict to descendants with replicas - --Full : Get full metadata information on ancestors - --Depth= : Number of processing levels (default:1) - --Production= : Restrict to descendants in a given production (at any depth)
Examples
dirac-bookkeeping-get-file-sisters
Gets a list of files that have a common parent (default, therefore looking for sisters) or ancestor at any level (using--Depth, in this case cousins at various levels), by default in the same production as the file itself. One can also ask for a sister/cousin in an other transformation. The option --AllFileTypes allows to get files of a different type (by default one checks only files of the same type).
Syntax
$ dirac-bookkeeping-get-file-sisters --help Report sisters (i.e. descendant of ancestor) for the given LFNs Usage: dirac-bookkeeping-get-file-sisters [option|cfgfile] ... [LFN|File] [Level] General options: -o: --option= : Option=value to add -s: --section= : Set base section for relative parsed options -c: --cert= : Use server certificate to connect to Core Services -d --debug : Set debug mode (-dd is extra debug) -h --help : Shows this help Options: - --File= : File containing list of LFNs -l: --LFNs= : List of LFNs (comma separated) - --Terminal : LFNs are entered from stdin (--File /dev/stdin) - --All : Do not restrict to sisters with replicas - --Full : Get full metadata information on sisters - --Depth= : Number of ancestor levels (default:1), 2 would be cousins, 3 grand-cousins etc... - --Production= : Production to check for sisters (default=same production) - --AllFileTypes : Consider also files with a different type
Examples:
$ dirac-bookkeeping-get-file-sisters /lhcb/LHCb/Collision11/BHADRON.MDST/00024034/0000/00024034_00002236_1.bhadron.mdst
NoSister :
/lhcb/LHCb/Collision11/BHADRON.MDST/00024034/0000/00024034_00002236_1.bhadron.mdst
$ dirac-bookkeeping-get-file-sisters /lhcb/LHCb/Collision11/BHADRON.MDST/00024034/0000/00024034_00002236_1.bhadron.mdst --Depth 2 --AllFileTypes
NoCousin : []
Successful :
/lhcb/LHCb/Collision11/BHADRON.MDST/00024034/0000/00024034_00002236_1.bhadron.mdst :
/lhcb/LHCb/Collision11/BHADRONCOMPLETEEVENT.DST/00024034/0000/00024034_00002238_1.bhadroncompleteevent.dst
/lhcb/LHCb/Collision11/CHARM.MDST/00024034/0000/00024034_00002240_1.charm.mdst
/lhcb/LHCb/Collision11/CHARMCOMPLETEEVENT.DST/00024034/0000/00024034_00002242_1.charmcompleteevent.dst
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00001915_1.dimuon.dst
/lhcb/LHCb/Collision11/DIMUON.DST/00024034/0000/00024034_00002244_1.dimuon.dst
/lhcb/LHCb/Collision11/EW.DST/00024034/0000/00024034_00002246_1.ew.dst
/lhcb/LHCb/Collision11/LEPTONIC.MDST/00024034/0000/00024034_00002248_1.leptonic.mdst
/lhcb/LHCb/Collision11/PID.MDST/00024034/0000/00024034_00002250_1.pid.mdst
/lhcb/LHCb/Collision11/RADIATIVE.DST/00024034/0000/00024034_00002252_1.radiative.dst
/lhcb/LHCb/Collision11/SEMILEPTONIC.DST/00024034/0000/00024034_00002254_1.semileptonic.dst
Statistics
dirac-bookkeeping-get-stats
Returns statistics for a given BK query (number of files, of events, luminosity…)Syntax
$ dirac-bookkeeping-get-stats --help Get statistical information on a dataset Usage: dirac-bookkeeping-get-stats [option|cfgfile] General options: -o: --option= : Option=value to add -s: --section= : Set base section for relative parsed options -c: --cert= : Use server certificate to connect to Core Services -d --debug : Set debug mode (-dd is extra debug) -h --help : Shows this help Options: -P: --Productions= : Production ID to search (comma separated list) -f: --FileType= : File type (comma separated list, to be used with --Production) [All] - --ExceptFileType= : Exclude the (list of) file types when all are requested -B: --BKQuery= : Bookkeeping query path -r: --Runs= : Run or range of runs (r1:r2) - --DQFlags= : DQ flag used in query - --StartDate= : Start date for the BK query - --EndDate= : End date for the BK query - --Visibility= : Set visibility (Yes, No, All) [Yes] - --File= : File containing list of LFNs -l: --LFNs= : List of LFNs (comma separated) - --Terminal : LFNs are entered from stdin (--File /dev/stdin)
Examples
Unique dataset:
$ dirac-bookkeeping-get-stats --BK /LHCb/Collision12//RealData/Reco14/Stripping20r0p1//BHADRON.MDST
For BK query: {'ConfigName': 'LHCb', 'EventType': '90000000', 'FileType': 'BHADRON.MDST', 'ProcessingPass': '/Real Data/Reco14/Stripping20r0p1', 'Visible': 'Yes', 'ConfigVersion': 'Collision12'}
Getting info from filesSummary...
Nb of Files : 4'322
Nb of Events : 43'844'045
Total size : 329.150 GB (7.5 kB per evt)
Luminosity : 1.657 /fb
Size per /pb : 0.2 GB
Files per /fb : 2609.1
$ dirac-bookkeeping-get-stats --BK /LHCb/Collision12//RealData/Reco14//FULL.DST
For BK query: {'ConfigName': 'LHCb', 'EventType': '90000000', 'FileType': 'FULL.DST', 'ProcessingPass': '/Real Data/Reco14', 'Visible': 'Yes', 'ConfigVersion': 'Collision12'}
Getting info from filesSummary...
Nb of Files : 584'518
Nb of Events : 27'814'353'411
Total size : 2.777 PB (99.9 kB per evt)
Luminosity : 2.043 /fb
Size per /pb : 1359.2 GB
Files per /fb : 286044.6
Multiple dataset (several file types). In that case an average luminosity is reported as different datasets may have different luminosities.
$ dirac-bookkeeping-get-stats --BK /LHCb/Collision12//RealData/Reco14/Stripping20r0p1//ALL.MDST,ALL.DST
For BK query: {'ConfigName': 'LHCb', 'EventType': '90000000', 'FileType': ['BHADRON.MDST', 'PID.MDST', 'LEPTONIC.MDST', 'CHARM.MDST', 'RADIATIVE.DST', 'BHADRONCOMPLETEEVENT.DST', 'EW.DST', 'CALIBRATION.DST', 'CHARMCOMPLETEEVENT.DST', 'DIMUON.DST', 'SEMILEPTONIC.DST'], 'ProcessingPass': '/Real Data/Reco14/Stripping20r0p1', 'Visible': 'Yes', 'ConfigVersion': 'Collision12'}
Getting info from filesSummary...
Nb of Files : 54'696
Nb of Events : 1'673'006'191
Total size : 80.648 TB
Avg luminosity : 1.669 /fb
Size per /pb : 47.7 GB
Files per /fb : 32774.4 Topic revision: r12 - 2020-09-30 - MichaelWilkinson
- LHCb Web
- LHCb Web Home
- Changes
- Index
- Search
- LHCb webs
- LHCbComputing
- LHCb FAQs
- LHCbOnline
- LHCbPhysics
- LHCbVELO
- LHCbST
- LHCbOT
- LHCbRICH
- LHCbMuon
- LHCbTrigger
- LHCbDetectorAlignment
- LHCbTechnicalCoordination
- LHCbUpgrade
- ABATBEA
- ACPP
- ADCgroup
- AEGIS
- AfricaMap
- AgileInfrastructure
- ALICE
- AliceEbyE
- AliceSPD
- AliceSSD
- AliceTOF
- AliFemto
- ALPHA
- ArdaGrid
- ASACUSA
- AthenaFCalTBAna
- Atlas
- AtlasLBNL
- AXIALPET
- CAE
- CALICE
- CDS
- CENF
- CERNSearch
- CLIC
- Cloud
- CloudServices
- CMS
- Controls
- CTA
- CvmFS
- DB
- DefaultWeb
- DESgroup
- DPHEP
- DM-LHC
- DSSGroup
- EGEE
- EgeePtf
- ELFms
- EMI
- ETICS
- FIOgroup
- FlukaTeam
- Frontier
- Gaudi
- GeneratorServices
- GuidesInfo
- HardwareLabs
- HCC
- HEPIX
- ILCBDSColl
- ILCTPC
- IMWG
- Inspire
- IPv6
- IT
- ItCommTeam
- ITCoord
- ITdeptTechForum
- ITDRP
- ITGT
- ITSDC
- LAr
- LCG
- LCGAAWorkbook
- Leade
- LHCAccess
- LHCAtHome
- LHCb
- LHCgas
- LHCONE
- LHCOPN
- LinuxSupport
- Main
- Medipix
- Messaging
- MPGD
- NA49
- NA61
- NA62
- NTOF
- Openlab
- PDBService
- Persistency
- PESgroup
- Plugins
- PSAccess
- PSBUpgrade
- R2Eproject
- RCTF
- RD42
- RFCond12
- RFLowLevel
- ROXIE
- Sandbox
- SocialActivities
- SPI
- SRMDev
- SSM
- Student
- SuperComputing
- Support
- SwfCatalogue
- TMVA
- TOTEM
- TWiki
- UNOSAT
- Virtualization
- VOBox
- WITCH
- XTCA
 |
|
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback