Single script method

dirac-lhcb-production-check-files jobID [jobID]

dirac-lhcb-production-check-files 37833
/lhcb/MC/DC06/DST/00003017/0000/00003017_00002344_5.dst 1 rfio://clhcbstager.ads.rl.ac.uk:9002?svcClass=lhcbRawRdst&castorVersion=2&path=/castor/ads.rl.ac.uk/prod/lhcb/lhcb/MC/DC06/DST/00003017/0000/00003017_00002344_5.dst
/lhcb/MC/DC06/DST/00003017/0000/00003017_00002305_5.dst 1 rfio://clhcbstager.ads.rl.ac.uk:9002?svcClass=lhcbRawRdst&castorVersion=2&path=/castor/ads.rl.ac.uk/prod/lhcb/lhcb/MC/DC06/DST/00003017/0000/00003017_00002305_5.dst
/lhcb/MC/DC06/DST/00003017/0000/00003017_00003257_5.dst 1 rfio://clhcbstager.ads.rl.ac.uk:9002?svcClass=lhcbRawRdst&castorVersion=2&path=/castor/ads.rl.ac.uk/prod/lhcb/lhcb/MC/DC06/DST/00003017/0000/00003017_00003257_5.dst
/lhcb/MC/DC06/DST/00003017/0000/00003017_00002280_5.dst 1 rfio://clhcbstager.ads.rl.ac.uk:9002?svcClass=lhcbRawRdst&castorVersion=2&path=/castor/ads.rl.ac.uk/prod/lhcb/lhcb/MC/DC06/DST/00003017/0000/00003017_00002280_5.dst
/lhcb/MC/DC06/DST/00003017/0000/00003017_00001113_5.dst 1 rfio://clhcbstager.ads.rl.ac.uk:9002?svcClass=lhcbRawRdst&castorVersion=2&path=/castor/ads.rl.ac.uk/prod/lhcb/lhcb/MC/DC06/DST/00003017/0000/00003017_00001113_5.dst

The output gives the LFN, the cached state (1=cached/staged, 0=not cached) and the TURL (Transfer URL). If a problem occurs with a file, the list of problematic files will be printed at the end. You can use the -o LogLevel=ALWAYS option to suppress any debugging output.

Step-by-step method

* Get the input file of the job (actually this is the lfn name of the file) e.g. JobId = 37833

dirac-lhcb-production-job-lfn 37833

$ dirac-lhcb-production-job-lfn 37833
2008-09-18 13:49:20 UTC dirac-lhcb-production-job-lfn/DiracAPI  INFO: Files retrieved and extracted in /afs/cern.ch/user/r/rvazquez/InputSandbox37833
OutputLFN: ['/lhcb/MC/DC06/FETC/00003030/0000/00003030_00004539_1.root', '/lhcb/MC/DC06/DST/00003030/0000/00003030_00004539_2.dst', '/lhcb/MC/DC06/SETC/00003030/0000/00003030_00004539_2.root']
InputLFN: ['/lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst']

* Get in which SE is the replica of the file and what is the pfn associated

dirac-dms-lfn-replicas /lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst

dirac-dms-lfn-replicas /lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst
2008-09-18 13:50:17 UTC dirac-dms-lfn-replicas/DiracAPI  INFO: Replica Lookup Time: 0.44 seconds
{'Failed': {},
 'Successful': {'/lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst': {'RAL-tape': 'srm://ralsrmb.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst'}}}

* Check if a TURL can be generated at the site, otherwise the job will not be able to access the dataset

dirac-dms-lfn-accessURL /lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst RAL-tape

 dirac-dms-lfn-accessURL /lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst RAL-tape
2008-09-18 13:51:27 UTC dirac-dms-lfn-accessURL  INFO: ReplicaManager.getReplicaAccessUrl: Attempting to get access urls for 1 replicas.
2008-09-18 13:51:27 UTC dirac-dms-lfn-accessURL  INFO: ReplicaManager.getReplicaAccessUrl: Resolving replicas for supplied LFNs.
2008-09-18 13:51:28 UTC dirac-dms-lfn-accessURL  INFO: ReplicaManager.__getPhysicalFileAccessUrl: Attempting to get access urls for 1 files.
2008-09-18 13:51:41 UTC dirac-dms-lfn-accessURL  INFO: Using lcg_util from: /afs/cern.ch/lhcb/software/DEV/DIRAC/DIRAC3_v3r1/Linux_x86_64_glibc-2.3.4/lib/python2.4/site-packages/lcg_util.pyc
2008-09-18 13:51:41 UTC dirac-dms-lfn-accessURL  INFO: The version of lcg_utils is 1.6.13
2008-09-18 13:51:41 UTC dirac-dms-lfn-accessURL  INFO: Using gfalthr from: /afs/cern.ch/lhcb/software/DEV/DIRAC/DIRAC3_v3r1/Linux_x86_64_glibc-2.3.4/lib/python2.4/site-packages/gfalthr.pyc
2008-09-18 13:51:41 UTC dirac-dms-lfn-accessURL  INFO: The version of gfalthr is 1.10.15
2008-09-18 13:51:41 UTC dirac-dms-lfn-accessURL  INFO: StorageElement.isValid: Determining whether the StorageElement RAL-tape is valid for use.
2008-09-18 13:51:41 UTC dirac-dms-lfn-accessURL  INFO: StorageElement.isLocalSE: Determining whether RAL-tape is a local SE.
2008-09-18 13:51:41 UTC dirac-dms-lfn-accessURL  INFO: StorageElement.getAccessUrl: Generating protocol PFNs for SRM2.
2008-09-18 13:51:41 UTC dirac-dms-lfn-accessURL  INFO: StorageElement.getAccessUrl: Attempting to get access urls for 1 physical files.
{'Failed': {},
 'Successful': {'/lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst': {'SRM2': 'rfio://clhcbstager.ads.rl.ac.uk:9002?svcClass=lhcbRawRdst&castorVersion=2&path=/castor/ads.rl.ac.uk/prod/lhcb/lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst'}}}

* Check the status of the file in the site

dirac-dms-pfn-metadata srm://ralsrmb.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst RAL-tape

dirac-dms-pfn-metadata srm://ralsrmb.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst RAL-tape
2008-09-18 10:13:46 UTC dirac-dms-pfn-metadata  INFO: ReplicaManager.__getPhysicalFileMetadata: Attempting to get metadata for 1 files.
2008-09-18 10:13:46 UTC dirac-dms-pfn-metadata  INFO: Using lcg_util from: /afs/cern.ch/lhcb/software/DEV/DIRAC/DIRAC3_v3r1/Linux_x86_64_glibc-2.3.4/lib/python2.4/site-packages/lcg_util.pyc
2008-09-18 10:13:46 UTC dirac-dms-pfn-metadata  INFO: The version of lcg_utils is 1.6.13
2008-09-18 10:13:46 UTC dirac-dms-pfn-metadata  INFO: Using gfalthr from: /afs/cern.ch/lhcb/software/DEV/DIRAC/DIRAC3_v3r1/Linux_x86_64_glibc-2.3.4/lib/python2.4/site-packages/gfalthr.pyc
2008-09-18 10:13:46 UTC dirac-dms-pfn-metadata  INFO: The version of gfalthr is 1.10.15
2008-09-18 10:13:46 UTC dirac-dms-pfn-metadata  INFO: StorageElement.isValid: Determining whether the StorageElement RAL-tape is valid for use.
2008-09-18 10:13:46 UTC dirac-dms-pfn-metadata  INFO: StorageElement.isLocalSE: Determining whether RAL-tape is a local SE.
2008-09-18 10:13:46 UTC dirac-dms-pfn-metadata  INFO: StorageElement.getFileMetadata: Attempting to get metadata for 1 physical files.
{'Failed': {},
 'Successful': {'srm://ralsrmb.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/lhcb/production/DC06/phys-v4-lumi5/00001897/RDST/0000/00001897_00002575_1.rdst': {'Cached': 1,
                                                                                                                                                                'Directory': False,
                                                                                                                                                                'File': True,
                                                                                                                                                                'Migrated': 1,
                                                                                                                                                                'Permissions': 511,
                                                                                                                                                                'Size': 372667640L}}}

In this case the status of Cached is 1, meaning that everything is fine.

-- RicardoVazquez - 18 Sep 2008

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2008-09-22 - GreigCowan
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback