UPDATEDCMS.PopCon development guide

Complete: 4

CMS.PopCon transfers the conditions objects from a user-defined data source to the off-line database. It is the developer's responsibility to provide the code which handles the data source and to specify the destination for the data. One should instantiate his/her objects, provide the IOV information for such objects and configure the database output module. Once the objects are built, the CMS.PopCon application writes the data to the specified database. Subdetector code must not access the target output database. It only passes the objects to the output module.

Architecture at a glance

Popcon is based on cmsRun infrastructure, so the base CMS.PopCon application class is the EDAnalyzer . Therefore it's possible to use different data sources such as data bases, ROOT files, ASCII files etc. For each conditions object (payload) class one should create a CMS.PopCon application.

The core framework consists of three parametrized classes.

  • CMS.PopCon
  • PopConSourceHandler
  • PopConAnalyzer

To create a cmsRun PopCon application which, for example, has to store MyPedestals, the subdetector developer should implement a derived class of PopConSourceHandler (say named MyPedestalSource lwhere all the online (source handling) code should go.

In this new class the developer needs just to implement

  • the getNewObjects method that loads the calibs,
  • the sourceId method that returns a text identifier of the source itself,
  • and provide a constructor that accept a ParameterSet

One has then just to declare a new module to the framework of the type PopConAnalyzer

The analyzer object holds the source handling object. It also serves some additional functionality such as:

  • Locking mechanism
  • Transfer logging
  • Payload verification (IOV sequence)
  • Application state management
  • Database output service

Inside the source handler class the developer should instantiate his/her objects, associate the IOV information with it and put the objects into the container. In case the information on the contents of the off-line database is required, it can be accessed through the two methods

  • cond::TagInfo const & tagInfo() const;
  • cond::LogDBEntry const & logDBEntry() const;
The content of the two objects is quite self explanatory. Please refer to the this twiki fro more info on the subject.

The writer in CMS.PopCon iterates over the container of user objects and stores it in the user-configured data destination.

Obtaining the sample code

Check out the following packages from CMSSW cvs server

  • release code

Due to dependencies some more code is required to be rebuilt. For instance to run line-commands tools one needs to rebuild

CondCore/Utilities  (from the current release)

After the build, one can verify that popcon does work running

source $LOCALRT/src/CondCore/PopCon/test/runPopConExamples
A real unit tests will be provided soon

How to write the code

To see how to create new conditions objects please refer to SWGuideCondObjectsTutorial

Creating the source handler class

Here you should perform all the data manipulation. Once the object is instantiated, create an IOV (either since or till) information and push it back the vector

std::vector<std::pair<T*, cond::Time_t> > m_to_transfer 
Where T is template parameter specifying the payload class

  • Define the source handler class:
class MyPedestalsHandler : public popcon::PopConSourceHandler<MyPedestals>
        void getNewObjects();
        MyPedestalsHandler(edm::ParameterSet const &);


  • define getNewObjects() method, which instantiates the objects and fills the m_to_transfer vector
void MyPedestalsHandler::getNewObjects()

    //to access the information on the tag inside the offline database:
    cond::TagInfo const &  = tagInfo();

    //to access the information on last successful log entry for this tag:
   cond::LogDBEntry const & = logDBEntry();     

   //to access the lastest payload (Ref is a smart pointer)
    Ref payload = lastPayload();

    //eg. to get the object from somewhere
    MyPedestals * mypedestals = getPedestals();

    //for each payload provide IOV information (say in this case we use since)
    const::Time_t snc = 15;


  • implementation specific parameters such as online DB passwords should be passed through the ParameterSet identified by the keyword Source in the config of the module

How to manage RDBMS type of external source

If some information stored in a relational database is required for the process of building payload objects, it is required to be accessed via Persistency interface. The example of accessing the oracle database can be found in test subdirectory of CondCore/CMS.PopCon package. To simplify the process of establishing the connection to the database, CondCore/DBCommon package can be used.

   cond::DbConnection conn;
    conn.configure( cond::CmsDefaults );
    cond::DbSession session = conn.createSession();
    cond::DbScopedTransaction tr(session);
    tr.start(true); // start read transaction
  coral::ISchema& schema= session.nominalSchema(); //obtain schema handle
  coral::ITable& table = schema.tableHandle("MyTable"); //obtain table handle
  std::auto_ptr<coral::IQuery> query(table.newQuery());
  //do some query and deliver result

  • Additional examples of Persistency usage can be found on the cvs server:
http://coral.cvs.cern.ch/cgi-bin/coral.cgi/coral/Tests/Integration_Basic/src/ e.g. Queries class

  • An example for establishing a connection to the oracle rdbms without DBCommon package (raw CORAL): TestBase class

  • Examples which make use of TestBase

  • Persistency documentation:

Creating the analyzer class

The Analyzer class need to be registered to the framework

typedef  popcon::PopConAnalyzer<MyPedestalsSource> MyPedestalsAnalyzer
//define this as a plug-in

Database Output

The standard PoolDBOutputService stores the Objects along with the IOV information in the user-specified database. For further information please refer to this twiki

Configuration and deployment

Since the CMS.PopCon application is in fact an EDAnalyzer, the deployment is quite simple. One needs to follow CMS Offline WorkBook.

  • add
    <flags EDM_PLUGIN=1>
    to the build file
  • add
    to analyzer cpp file

CMS.PopCon module specific parameters

  • record - the record into which to write. has to be one of those in the toPut VPSet
  • SinceAppendMode - If the tag exists, the output service will append since time, otherwise till time
  • loggingOn - to activate logging in the logDB

    module Test1 = MyPedestalAnalyzer{
        #mandatory parameters
        string record= "MyPedestalsRcd"
        bool SinceAppendMode = true

        #subdetector application specific parameters
        PSet Source = {string MyParameter ="omds"}

For the configuration file language manual please refer to this twiki

CMS.PopCon specific deployment

As CMS.PopCon supports transfer logging, locking mechanism and state management, a special database account is required to store additional information.
Please refer to the this twiki fro more info on the subject.

Users can add more info in the usertext field of the log tables, filling m_userTextLog in the subdetector application, as in this examples:

  std::ostringstream ss; 
  ss << "num=" << m_number <<","
     << "firstSize=3," <<"lastSize=" << size-2; 
  m_userTextLog = ss.str()+";";

If you're using sqlite or an oracle owner account as log DB, the schema will be created on the first use
In the case of separation of owner and writer oracle account, the log DB should be prepared by the schema owner using the following commandline tool:

cmscond_logdb_prepare -c oracle://service/account -u account_owner -p pass
or replace -u -p with -P which identifies the location of authentication.xml.

Inspect log entries

cmscond_logdb_dump -c oracle://devdb10/me -u me -p mypass
or replace -u -p with -P to specify the location of authentication.xml. In case of sqlite db, no identification is required.

CMS.PopCon dashboard

A monitoring tool is provided. Just look to PopConDashBoardNew.

CMS.PopCon support

For any question or information, please contact MicheleDeGruttola, SalvatoreDiGuida and AntonioPierro For support during operation see help/panic page: popcon help during data taking

Review Status


Review Status

Editor/Reviewer and Data Comments
MarcinBogusz - 03 Oct 2007 Page Author
VincenzoInnocente - 05 Feb 2008 update to Version 2
ZhenXie - 07 Feb 2008 update coral example
MicheleDeGruttola - 24 April 2008 update to 20X

Responsible: VincenzoInnocente
Last reviewed by: MicheleDeGruttola

Topic attachments
I Attachment History Action Size Date Who Comment
Texttxt PopConStatus_16_08.txt r1 manage 1.7 K 2007-08-17 - 14:51 MarcinBogusz Presentation of the status for DB meeting
JPEGjpg popcon_classes_v2.jpg r1 manage 87.4 K 2007-08-10 - 17:05 MarcinBogusz  
Edit | Attach | Watch | Print version | History: r43 < r42 < r41 < r40 < r39 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r43 - 2010-06-11 - PeterJones

    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback