PopCon development guide

%COMPLETE3%

popcon_classes_v2.jpg

PopCon transfers the conditions objects from a user-defined data source to the off-line database. It is the developer's responsibility to provide the code which handles the data source and to specify the destination for the data. One should instantiate his/her objects, provide the IOV information for such objects and configure the database output module. Once the objects are built, the PopCon application writes the data to the specified database. Subdetector code must not access the target output database. It only passes the objects to the output module.

Architecture at a glance

PopCon is based on cmsRun infrastructure, so the base PopCon application class is the EDAnalyzer . Therefore it's possible to use different data sources such as data bases, ROOT files, ASCII files etc. For each conditions object (payload) class one should create a PopCon application.

The core framework consists of two parametrized base classes.

  • PopConAnalyzer
  • PopConSourceHandler

To create a cmsRun PopCon application which, for example, is to store MyPedestals the subdetector developer should inherit both classes with Mypedestals parameter. Virtually all the online (source handling) code should go into the class derived from PopConSourceHandler. The analyzer object holds the source handling object. It also serves some additional functionality such as:

  • Locking mechanism
  • Transfer logging
  • Payload verification (IOV sequence)
  • Application state management
  • Database output service

Inside the source handler class the developer should instantiate his/her objects, associate the IOV information with it and put the objects into the container. In case the information on the contents of the off-line database is required, it can be accessed through the OfflineDBInterface object. The analyzer's DB output service iterates over the container of user objects and stores it in the user-configured data destination.

UPDATEDObtaining the sample code

Check out the following packages from CMSSW cvs server

  • Since CMSSW_1_7_X_2007-10-03-0200
Use the release code

  • For releases past 1.7.0_pre5
cvs co -r V00-03-01 CondCore/PopCon

  • For CMSSW releases prior to 1.7.0_pre4 (OBSOLETE CODE)
cvs co -r V00-01-00 CondCore/PopCon
cvs co -r V02-04-00 CondCore/DBOutputService

How to write the code

To see how to create new conditions objects please refer to SWGuideCondObjectsTutorial

Creating the source handler class

Here you should perform all the data manipulation. Once the object is instantiated, create an IOV (since and till) information and push it back the vector

std::vector<std::pair<T*, popcon::IOVPair> >* m_to_transfer 
Where T is template parameter specifying the payload class

  • Define the source handler class:
class MyPedestalsHandler : public popcon::PopConSourceHandler<MyPedestals>
{
    public:
        void getNewObjects();
        ~MyPedestalsHandler();
        MyPedestalsHandler(std::string,std::string,std::string, const edm::Event& evt, const edm::EventSetup& est, std:string);
    private:
        MyPedestals * mypedestals;
};

  • define getNewObjects() method, which instantiates the objects and fills the m_to_transfer vector
void MyPedestalsHandler::getNewObjects()
{
    //to access the information on the tags inside the offline database call:
    std::map<std::string, popcon::PayloadIOV> mp = getOfflineInfo();

    for(std::map<std::string, popcon::PayloadIOV>::iterator it = mp.begin(); it != mp.end();it++)
        cout << "Tag: " << it->first << " , last object valid since " << it->second.last_since << endl;

    //eg. to get the object from es_source
    edm::ESHandle<MyPedestals> pedestals;
    esetup.get<MyPedestalsRecord>().get(pedestals);
    mypedestals = pedestals.product();

    //for each payload provide IOV information
    unsigned int snc = 1;
    unsigned int tll = 1234567;
    popcon::IOVPair iop = {snc,tll};

    m_to_transfer->push_back(std::make_pair(mypedestals,iop));
}

  • The constructor should call the base class constructor:
MyPedestalsHandler::MyPedestalsHandler(std::string name, std::string cstring, std::string cat,const edm::Event& evt, const edm::EventSetup& est, std::string some_specific_param) :
popcon::PopConSourceHandler<MyPedestals>(name,cstring,cat,evt,est)
{
//implementation specific parameters such as online DB passwords should be passed through the constructor
//name - name of the payload object (should be "MyPedestals" in this case)
//cstring - connect string to the offline databse
//cat - connect string to the Pool catalog
}

Accessing online database

If some information stored in a database is required for the process of building payload objects, it can be accessed via Persistency interface. The example of accessing the oracle database can be found in test subdirectory of CondCore/PopCon package. To simplify the process of establishing the connection to the database, CondCore/DBCommon package can be used.

cond::DBSession* session;
cond::RelationalStorageManager* m_coraldb;

session=new cond::DBSession();
//to get the username/passwd from $CORAL_AUTH_PATH/authentication.xml
session->sessionConfiguration().setAuthenticationMethod( cond::XML );
session->sessionConfiguration().setMessageLevel( cond::Error ); //or Debug
session->connectionConfiguration().setConnectionRetrialTimeOut(60);
session->open();

m_coraldb = new cond::RelationalStorageManager("oracle://omds/foobar",session);
m_coraldb->connect(cond::ReadOnly);
m_coraldb->startTransaction(true);
//connection established
coral::ITable& mytable =
m_coraldb->sessionProxy().nominalSchema().tableHandle("A_TABLE");
//and perform the query on A_TABLE table 

There is however going to be a change in the DBCommon interface in CMSSW 1.8 As long as you're planning to release for 1.7 the tutorial code should be fine.

Please refer to CoralIface.cpp file for full example.

  • Additional examples of Persistency usage can be found on the cvs server:
http://coral.cvs.cern.ch/cgi-bin/coral.cgi/coral/Tests/Integration_Basic/src/ e.g. Queries class

  • An example for establishing a connection to the oracle rdbms without DBCommon package (raw CORAL): TestBase class
http://coral.cvs.cern.ch/cgi-bin/coral.cgi/coral/OracleAccess/tests/Common/

  • Examples which make use of TestBase
http://coral.cvs.cern.ch/cgi-bin/coral.cgi/coral/OracleAccess/tests/SimpleQueries/

  • Persistency documentation:
http://pool.cern.ch/coral/

Creating the analyzer class

Since our analyzer will hold MyPedestalsHandler class, one needs to initialize it in initSource method.

class MyPedestalsAnalyzer : public popcon::PopConAnalyzer<MyPedestals>
{
    public:
        MyPedestalsAnalyzer(const edm::ParameterSet&);
    private:
        string some_specific_param;
        void initSource(const edm::Event& evt, const edm::EventSetup& est){
            //pass specific parameters through the constructor
            m_handler_object=new MyPedestalsHandler("MyPedestals",m_offline_connection, m_catalog,evt,est,some_specific_param);
            //m_offline_connection and m_catalog are the members of the base class, the values are passed through edm::ParameterSet
       }
};

  • The Constructor should pass the ParameterSet to the base class so that
    m_offline_connection
    and
    m_catalog
    are initialized.

MyPedestalsAnalyzer::MyPedestalsAnalyzer(const edm::ParameterSet& ps) : PopConAnalyzer<EcalPedestals>(ps,"MyPedestals")
{
    //a place to get some subdetector specific parameters
    some_specific_param= ps.getParameter<std::string>("MyParameter");
}

Database Output

The PopConDBOutputService stores the Objects along with the IOV information in the user-specified database. Basically it is cond::service::PoolDBOutputService with some minor extensions. The configuration is also identical. For further information please refer to this twiki

UPDATEDConfiguration and deployment

Since the PopCon application is in fact an EDAnalyzer, the deployment is quite simple. One needs to follow CMS Offline WorkBook.

  • add
    <flags EDM_PLUGIN=1>
    to the build file
  • add
    DEFINE_FWK_MODULE(MyPedestalsAnalyzer);
    to analyzer cpp file

The data destination is configured identically as PoolDBOutputService :


replace PopConDB.connect = "sqlite_file:DB.db"
replace PopConDB.timetype = "runnumber"

service = PopConDBOutputService {
    using PopConDB
    VPSet toPut = { { string record = "MyPedestalsRecord" string tag = "MyPedestals_test" } }
}

PopCon module specific parameters

  • connect - connect string to the offline DB. Get offline info reads from it. This should usually be identical to CondDBCommon.connect
  • popConDBSchema - connect string to the PopCon specific schema, which stores logging, state etc. information
  • SinceAppendMode - If the tag exists, the output service will append since time, otherwise till time
  • debug - outputs lots of debugging information

    module Test1 = MyPedestalAnalyzer{
        using PopConDB
        #mandatory parameters
        bool SinceAppendMode = true
        bool debug = false

        #subdetector application specific parameters
        string MyParameter ="omds"
        }

For the configuration file language manual please refer to this twiki

UPDATEDPopCon specific deployment

As PopCon supports transfer logging, locking mechanism and state management, a special database account is required to store additional information.
Therefore after creating a now PopCon application (payload specific analyzer) it has to be registered and the state of the off-line database has to be calculated.
Currently the registration is performed manually by PopCon schema administrator. It is however possible to register a new payload type manually using cmscond_popcon_register tool.

$ cmscond_popcon_register popcon_schema_connect_string  payload_name  destination_database_connect_string
Parameters
  • popcon_schema_connect_string
    • form of oracle://db/schema
    • holds the metadata
    • currently CMS_POP_CON on omds
  • payload_name - name of conditions object such as MyPedestals
  • destination_database_connect_string - same as PopConDB.connect

If you're using sqlite as a destination, no logging and state checking is performed and there is no need to register the payloads

So called patches no longer exist ( since V-00-03-00) in CondCore/PopCon/data directory. Use command line registration tool.

Review Status

Editor/Reviewer and Data Comments
MarcinBogusz - 10 Aug 2007 Page Author

%RESPONSIBLE% MarcinBogusz
%REVIEW%

Topic attachments
I Attachment History Action Size Date Who Comment
JPEGjpg popcon_classes_v2.jpg r1 manage 87.4 K 2007-08-10 - 11:13 MarcinBogusz  
Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2010-06-11 - PeterJones
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback