TWiki> LHCb Web>LHCbComputing>ClosingProcedure (revision 3)EditAttachPDF
-- MarcoCorvo - 2016-10-07

Before closing an official production (Reco, Stripping, Turbo or TurCal) some checks are needed to be performed in advance. A production can show files in different statuses, the most important being:

These files can be checked using the command

dirac-transformation-debug <prodID> --Status <status> --Info <files|jobs|flush>  

Many are the reasons why a file can be in MaxReset or Assigned, but to check if this is not a pathological situation, one can verify with:

dirac-transformation-debug <prodID> --Status <MaxReset|Assigned> --Info files  | dirac-production-check-descendants <prodID> 

This command line provides a summary of files which are the descendants of the ones in MaxReset or Assigned status, if any, and gives also some suggestions on how to cure the situation.

A file can also remain Unused basically for two reasons:

  • the files doesn't have an active replica
  • the run to which the file belongs has not yet finished the online processing (basically HLT2)

To verify in which case the file falls one can try:

dirac-transformation-debug <prodID> --Status Unused --Info flush 

which says if the run is still in the HLT2 processing phase or:

dirac-transformation-debug <prodID> --Status Unused --Info files | dirac-dms-lfn-replicas 

which says if the file has a replica.

Another useful chain of command is:

dirac-transformation-debug <Prod2> --Status Unused --Info files| dirac-bookkeeping-get-file-ancestors --All | dirac-production-check-descendants <Prod1>

where Prod2 is, for example, a Merging production and Prod1 is a Stripping one. In this case it's possible to understand whether the files are still Unused because the ancestors file where in some particular status, e.g. MaxReset.

To perform the final checks on a given production the LHCbDirac command to be used is

dirac-production-check-descendants <prodID>
For Reco, Turbo or Turcal production, the command can complete in few hours, but for Stripping it can take much longer. To solve this problem these are the steps to perform when closing a production with final Merging steps (Stripping or Turbo), given that <prod1> is the Stripping/Turbo and <prod2> the Merging:

dirac-bookkeeping-get-files —Prod <prod1> —Visibility No | dirac-production-check-descendants <prod2>

This would verify if the non-merged files still in the BK have been merged or not. This is not enough as there could be files in the FC but not in the BK which are not picked up by the previous command. To solve this there's another useful command (will be released soon, not yet available with the current LHCbDirac release v8r3p7):

dirac-loop TurboFileTypes.txt 'dirac-dms-list-directory /lhcb/LHCb/Collision16/@arg@/000<prod1> | grep mdst | dirac-production-check-descendants <prod2>' | dirac-bookkeeping-get-file-descendants

this currently works for Turbo and should be adapted for Stripping.

This command can be followed by a final

dirac-dms-check-fc2bkk —Last 

in case some files are not in the BK.

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r3 - 2016-10-19 - MarcoCorvo
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback