Philippe, Marianne, Zoltan and Elisa

Quality Parameters Table

The table should be kept in the new schema. Currently, the attributes of this table are not used. The only check on the file quality is based on the got_replica flag. This means that the BK checks either the file exists or not, but no check on the quality is done. We will investigate on the meaning of the attributes of the table. Possibly new attributes will be added, following the feedback of the collaboration.

RemoveReplica functionality

Marianne briefly explained how it is done. She will provide to Zoltan detailed information.

New database schema

Long and detailed discussion on the new database schema. Main points:
  • Add magnetic field to the data taking period table.
  • Remove Executable from the jobs table
  • Move EventInputStat from the files table to the jobs table. In fact, this is an attribute of the job, not of the output file.
  • Remove RunNumber from jobs table. In fact, this would be meaningful only for raw data. For all the following steps, we are not interested in the run (or job) number because data from many runs can be merged in the same job, so we lose track of the initial run. For raw data we will store a unique run number in the attribute Name of the jobs table.
  • Drop the WNCPU attribute (it is included in the WNModel) and add the WNPower in the jobs table
  • Rename the XmlDDDBVersion as GeometryTag. It makes more sense.
  • Rename the lchbGUID as GUID
  • Data Taking Period table: this table should be used also for simulated data. In this case, it groups files not chronologically consecutive, but jobs which have the same condition wrt geometry, beam energy, magnetic field.
  • Concept of generator: the generator and the generator version should be added for simulated data. Here we have two possibilities: either we add these 2 attributes in the jobs table, or we create a new table with these 2 attributes and put a GeneratorId in the jobs table to connect to it. The same could be done with the ConfigurationName and Version. In general, whenever a group of attributes are common to more jobs, we could extract them from the jobs table and put them in a separate table. The advantage is that we save space (the jobs table has 10 millions rows!), the drawback is that when making queries we have to make the one additional joint and the performance drops. Conclusion: ask to the Oracle experts, they know very well what is best to do in these cases. So, for the time being we keep the in the jobs table, then, after interacting with Oracle experts, we will decide.
  • roottree table for simulated data. Here also we should include the processingPassId, the DAQPeriodId, the GeneratorId. Remove the inputfile0,1,2 and the Program0,1,2.

Next Software Week

Elisa will prepare a draft for the presentation. The main points will be:
  • Current implementation:
  • problems and issues
  • Comments on the problems of double counting (which must be fixed by the software week!) and the java crash that Zoltan has fixed.
  • New activity on the BK:
  • Overview of what has been done so far:
  • Implementation of a new DIRAC service: BkkManager
  • New schema for the database. Main differences from the old schema. New tables and attributes. Ask feedback from users.
  • Data search functionalities: prototype based on the AMGA python API. Explain the functionality and ask users if it fulfills their needs.
  • Next to do:
  • Optimization of the new schema.
  • Migration of data
  • Graphical user interface

New service released by?

-- ElisaLanciotti - 06 Mar 2008

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2008-03-06 - ElisaLanciotti
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback