Difference: BKWGMinutes070330 (1 vs. 2)

Revision 22007-04-18 - PhilippeCharpentierSecondary

Line: 1 to 1
 
META TOPICPARENT name="BKWG"

LHCb Bookkeeping Working Group Meeting

Line: 29 to 29
 

Run mixing

Although it was proposed not to mix runs at a previous meeting, this is felt to be too restrictive, leading to complicated data handling and probably many short files. In fact if "periods" are defined (see above), this should be the entity inside which data mixing is allowed at the level of stripping. For the first pass reconstruction, periods could be defined open-ended at the start of each data-taking campaign and closed if something obviously changes or anyhow at the end of the data-taking campaign.
Changed:
<
<
--+ Queries PhC reported that a fellow in Dublin has designed a BK browser GUI that presents the queries as a tree structure: feicim.
>
>

Queries

PhC reported that a fellow in Dublin has designed a BK browser GUI that presents the queries as a tree structure: see the demo at feicim and the sceen-shot below:
 feicim.png
Added:
>
>
In fact such an approach is very intuitive for users as they usually define their selection in a given order. An "Any" field should appear at each level in order to wildcard it. The WG recommends investigating a single query approach based on this idea. There should be a single script to maintain that could be used in a web browser and as a GUI as well as from Ganga.

Currently the jobs are refered to by the version of the application that is used. In fact what is more relevant is if this version of the application is compatible with a certain processing. Hence the WG recommends to add a processing history to the existing application version history. The actual implementation of it (additional relation table between application versions / Conditions tag and processing history should be further investigated.

Currently the application history is limited by construction to 3, which is already not enough for stripping (there are 5 levels). Hence the WG recommends to make provision for an arbitrary number of processing steps in the history. This is part of the view implementation and doesn't concern the warehous DB, hence should not be too heavy to introduce.

Datasets

PhC mentioned that there is a possibility in the LFC to define datasets as directories containing symbolic links to other directories or to files. This would also be a way to define physics datasets independently of the BKDB.

Ganga also has a notion of datasets. One could have static datasets (list of files, possibly obtained from a query to the BK) or dynamic datasets (known as a query to the BK or an LFC dataset directory).

The WG feels that user's datasets should be defined at the level of Ganga while physics groups datasets should be defined at the level of the LFC. Physics group should use the BK queries in order to build the LFC datasets.

Next meeting

Friday 20 April at 9:30
  -- Main.phicharp - 18 Apr 2007

Revision 12007-04-18 - PhilippeCharpentierSecondary

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="BKWG"

LHCb Bookkeeping Working Group Meeting

Date and Location

30 March 2007
9:30 - 11:00

Attendees

Andrew, Carmine, Markus Niko, Olivier C., Philippe (chair, notes)

Apologies

Michael, Olivier D.

Subjects

Files sorting

Olivier reports that the files are not sorted when querying for a given production but only when querying for a dataset. Carmine understands why and will try an fix it.

Definition of "periods"

At the previous meeting we had proposed that the CONFIGNAME be the partition name and CONFIGVERSION be the DAQ configuration. It is clear also that the activity must be present (physics, calibration...) as part of the Configuration.

It is also felt that there is a need for being able to query datasets corresponding to a given "period" of data taking. This can be done by selecting a date range, but there will probably be official "periods" defined (that could be channel-dependent and certainly processing-dependent) and a common query would be "give me stripped DST for Stripping2009_3 and period 2008_P3". The definition of periods comes only after the data have been entered in the BKDB. Thus the WG proposes to add a new table that would define for each processing "periods" in terms of interval of time. Is this table needed also for MC data?

Run mixing

Although it was proposed not to mix runs at a previous meeting, this is felt to be too restrictive, leading to complicated data handling and probably many short files. In fact if "periods" are defined (see above), this should be the entity inside which data mixing is allowed at the level of stripping. For the first pass reconstruction, periods could be defined open-ended at the start of each data-taking campaign and closed if something obviously changes or anyhow at the end of the data-taking campaign.

--+ Queries PhC reported that a fellow in Dublin has designed a BK browser GUI that presents the queries as a tree structure: feicim. feicim.png

-- Main.phicharp - 18 Apr 2007

META FILEATTACHMENT attachment="feicim.png" attr="" comment="" date="1176904309" name="feicim.png" path="feicim.png" size="32502" stream="feicim.png" user="Main.phicharp" version="1"
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback