Procedure for obtaining, validating and deploying alignment constants

Table of Contents

Glossary

  • CAF: Cern Analysis Farm
  • ES: express stream
  • DCM: database content manager
  • FSR: File summary record
  • SE: storage element

Protocol (first draft)

  1. alignment/calibration/monitoring runs on ES data on CAF, produces new xml files. (see also below.)
  2. xml constants are collected into a single sqlite file, with a corresponding interval of validity. the result is copied to a grid storage element. (see instructions below)
  3. the production teams runs a test production over the ES data.
  4. the monitoring teams looks at the histograms from this production and signs off
  5. the database content manager copies the sqlite file into the official database
  6. tier 1 processing starts

Summary of protocol, to be transformed into a fancy flow diagram:

step process input output where who
1 alignment/calibration/monitoring ES data xml files, monitoring histos CAF sub-detector experts, alignment convenor
2 creating temporary DB for production xml files sqlite file on storage element - alignment convenor
3 test production ES data, sqlite file logfiles, monitoring histos cern tier1 or CAF ? production team
4 sign-off logfiles, monitoring histos green/red light - monitoring team
5 updating official DB for production sqlite file updated DB - database contents manager
6 tier 1 processing full data stream, updated DB dst tier1 production team

Various steps in testing this

  • What was done?
    • running production using an sqlite file on a storage element
  • What has not yet been done, among others?
    • writing an sqlite file with a finite interval of validity.
    • interaction with monitoring team
    • finalizing ES trigger

Step 1. Running the prompt calibration on the CAF

There are two scenarios.

1. Ultimately, the alignment/calibration process should run without too much manual intervention. The easiest is if this is a production task. The idea is that the express stream will be produces by several parallel task, that write DSTs with alignment derivatives in the FSR. There will then be a single 'update' tasks that reads the FSR DSTs and spits a new database layer. This process can eventually be repeated if the monitoring group decides that more iterations are needed.

2. More realistically, certainly in the beginning, is that the alignment constants are produced by non-automated tasks, probably performed by different persons for different subdetectors. The alignment convenor collects database constants and creates a database layer. The production team runs a standard brunel task with the new layer such that the monitoring group can look at the result even before anything is added to the 'official' database.

Step 2. Handing over alignment xml to production team

Creating the sqlite file

The alignment xml is written into a sqlite file as a data base layer with finite interval of validity. Typically, this will be 'valid from run XXX', but I don't quite understand yet what the requirements are.

Escher contains algorithms to write the alignment constants for all subdetectors. The instructions for creating an sqlite layer can be found here. If you run the Escher/options/gaudipariter.py script, the sqlite file will be created automatically.

Copying the sqlite file to a storage element

To use the sqlite file in production, it must be copied to a storage element (SE) using the following DIRAC command:

  SetupProject Dirac
  dirac-dms-add-file <LFN> <local file> <SE>

with for example

 
  <LFN> = /lhcb/user/w/wouter/Alignment/Alignment_20090701.db
  <local file> = Alignment_20090701.db
  <SE> = CERN-USER

If we see scaling problems we can deploy it on multiple SEs, but all jobs (running on the Xpress stream) should run at CERN.

Run boundaries

Currently, thereb is no easy way to find the beginning and end of a run. The dirac-bookkeeping-run-informations script will be adaped to return also the begin and end time of a run. You can run the script with

   SetupProject Dirac
   dirac-bookkeeping-run-informations <run_number>

Open issues

A rather non-exhaustive list ...

  • Does the latest version of copy_file_to_db already allow to set the interval of validity?
  • How do we extract the interval of validity for a run range? Or for the start of a particular run?

-- Main.WouterHulsbergen - 01 September 2009

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r6 - 2009-10-26 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback