Bookkeeping developments

This page is intended for sharing information amongst developer of the Bookkeeping system. These developments are meant at integrating the Bookkeeping system into the DIRAC framework (as part of DIRAC3) as well as implementing the recommendations of the Bookkeeping Working Group.

Schema for Online data

As defined in the BKWG, for online data, the "job" entity from which the file originates will be identified to the DAQ "run". All files of that run will be "output files" of that "job/run". There are however some limitations in the current model that need to be addressed urgently in order to be usable for online data. These limitations are described in this twiki page.

The new schema of the BK database

A new schema for the BK database has been designed, in order to enable the BK to host real data. For more information about the new schema click here.

The Processing Pass

This is a new concept intruduced in the BK. The idea is to group datasets on the basis of the application version and conditions DB tag used to produce them and to produce their ancestors.

How the Processing Pass has been computed for already existing data (DC06 data): offline computation of the Processing Pass.

How it is computed for newly inserted data: online computation of the Processing Pass.

The Simulation Conditions and Data Taking Conditions tables

The simulation conditions and data taking conditions are new tables for simulated and real data respectively. Data with same simulation or data taking conditions can be merged at processing stage. More details about the simulation conditions and data taking conditions tables.

Migration of the database to the new schema

After implementing the new database schema in an integration oracle database, we made a migration of the DC06 data. More details on the procedure of the migration and the tool used for it are available in this link: Migration.

Bookkeeping Client Documentation

This section is the Bookkeeping Client documentation.

Bookkeeping development and production setups

This section describe Bkk Service development and production setups.

Bookkeeping release procedure

This section describe a release procedure of the BKK system.

Meetings

Current status of the activity on the BK

Go to this page to check the current status of the activity on the BK.

The new Bookkeeping system is foreseen to be put in production by the beginning of November 2008: updated Check list of all the last pending issues.

Miigration plan of Bookkeeping(2011)

We are using pass indexes to describe a processing chain. The processing chain can have different steps. The processing pass is a group of the pass indexes which are compatible. The Production System are used to define a activity(processing chain). The data processing can be simulation, reconstruction, reprocessing etc. The users can use the production system the define their requests for data processing. The current processing pass is created and published to the bkk when the production has been created. The processing pass created by the production manager or data quality person. This persons who create a production have to contact with the application persons to know the option files and different attributes. They can make mistake for example: typo, bad option file, forget the file list one stream file. etc. We decided to modify the existing processing passes. We are using steps. Each steps are defined by the application responsible person. They have to provide the option files and the input and output of each steps. The production request people can put together the processing steps.This make simple the production definition. This modification is affected the following systems: Bookkeeping, Production systems and work flow. The main modification has been done on the Bookkeeping system. We are considering the migration plan of bkk system, but also we try to summarize what we need to be done in the other system. You can found the migration plan in the following link Check list.

-- PhilippeCharpentier - 10 Dec 2007

Edit | Attach | Watch | Print version | History: r24 < r23 < r22 < r21 < r20 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r24 - 2011-02-14 - ZoltanMathe1
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback