Migration from CMSSW 1_7_7 to 3_1

Contents

• Migration Trace
  • Step 1 Transportation
  • Step 2 Translation
  • Step 3 Testing
• Change Log
  • Step 1 Transportation
  • Step 2 Translation
  • Step 3 Testing
• Synchronization Log
  • Synchronization 1
  • Synchronization 2

Migration Trace (Jun 01 2009 - Aug 21 2009)

Roadmap:

Comments:

As the migration work is progressing, people are still updating the code based on the Totem software @ CMSSW_1_7_7. Version synchronization work (merge the updated code to the new Totem software @ CMSSW_3_1_X) is done at some certain check points. The following list shows synchronization history. For detail information about each synchronization, you can check the Synchronization Log section.

The work is carried out by:

1. Check out the latest code @ CMSSW_1_7_7 from the CVS.
2. Compare the code with migrating version.
3. Merge the code.

Step 1: Transportation

Transport all the modules in the old Totem package (@ CMSSW_1_7_7) to the new framework CMSSW_3_1_0. Make sure that all modules get compiled successfully.

The module transporting trace is decided by the increasing dependencies among all these modules.

NOTE:

1. IORawData module in the original Totem package (@ CMSSW_1_7_7) is obsolete, thus it is not migrated into the new version of Totem package (@ CMSSW_3_1_0).
2. Geometry/Records (@ CMSSW_1_7_7) will not be included into the new version of Totem package (@ CMSSW_3_1_0). Instead A new submodule Geometry/TotemRecords, which contains only RealGeometryRecord.[h,cc] and MisalignedGeometryRecord.[h,cc] about Totem, will take its place.

References:

• Offline Simulation Software - getting started
• Migration to newer version of CMSSW framework
• CMSSW Software Cross-Reference

Step 2: Translation

Transfer the configuration files in CMS-specific configuration language to the equivalent ones in Python Configuration Syntax.

The translation trace follows the transporting trace in step 1.

NOTE:

There are some old format configuration files which have not been translated because they are obsolete. The list for these files are:

• RecoTotemT1T2/T2MakeCluster/test/*.cfg > ==idealOnlyTotemGeometryXML.cfi is missing
• RecoTotemT1T2/T2RecHit/test/*.cfg > ==idealOnlyTotemGeometryXML.cfi is missing
• RecoTotemT1T2/T2RoadProducer/*.cfg > ==idealOnlyTotemGeometryXML.cfi is missing
• TotemBackground/BeanGas/data/RP_sim_*.cfg ==> refer to many obsolete cfi files

References:

• Introduction to CMS Configuration Files
• Description of the cmsRun Configuration File Language
• Description of the cmsRun Python Configuration Synta

Step 3: Testing

Run the test suit (offline/cmssw/examples/) to make sure that the preceding transportation process does not change any functionality of the Totem software. This process is done by running the cmsRun with a series of test configuration files. The resulting root files will be sent to the experts from T1/T2/RP, who are responsible for certain modules for further validation if necessary. This process may require to go back to the Transportation and Translation phases, because errors would be included in these two phases or some modules need to be updated.

The testing trace is shown as follows. The test suit can be found from offline/cmssw/examples/. There are two folders under this folder: production/ and validation/. The configuration files under production/ aim to simulate events ,do the track reconstruction, and produce the results. The configuration files under validation/ aim to validate the results made by the files under production/ and produce the validation summary results.

Target Detector	Production	Validation	Process
T1T2	gunT1T2mu_cfg.py	valgunT1T2mu_cfg.py	OK
T1T2	gunT1T2pi_cfg.py	valgunT1T2pi_cfg.py	OK
RP	RPelastic1535_cfg.py	valRPelastic1535_cfg.py	Fail
RP	RPelastic90_cfg.py	valRPelastic90_cfg.py	Fail
RP	RPinelastic1535_cfg.py	valRPinelastic1535_cfg.py	Fail
RP	RPinelastic2_cfg.py	valRPinelastic2_cfg.py	Fail
RP	RPinelastic90_cfg.py	valRPinelastic90_cfg.py	Fail
RP + T1T2	RPT1T2pythiaSDbeta2_cfg.py	valRPT1T2pythiaSDbeta2_cfg.py	Fail
RP + T1T2	RPT1T2pythiaSDbeta90_cfg.py	valRPT1T2pythiaSDbeta90_cfg.py	Fail

-- ZhengkuiZhang - 07 August 2009

Change Log (Jun 01 2009 - Aug 21 2009)

Roadmap:

NOTE: Because the change log is so long (>1000 lines), it is more advisable to only give a summary on what kinds of changes are made. The detail change log can be downloaded from https://twiki.cern.ch/twiki/pub/TOTEM/CompOfflineCMSSWMigration177to31/change_log.

Step 1: Transportation

Summary of changes

<use name=SimDataFormats/HepMCProduct>  
-->  <use name=SimDataFormats/GeneratorProducts>
#include "SimDataFormats/HepMCProduct/interface/HepMCProduct.h" 
-->  #include "SimDataFormats/GeneratorProducts/interface/HepMCProduct.h"

#include "PluginManager/ModuleDef.h"  
-->  #include "FWCore/PluginManager/interface/ModuleDef.h"

#include "TClass.h"

#include "TMath.h"

#include <map>

<use name=rootgraphics>

#include <math.h>

<use name=hepmc>

<use name=SimDataFormats/GeneratorProducts>

<use name=clhep>

#include "CLHEP/Vector/LorentzVector.h"

<use name=Foundation/PluginManager>

<use name=Framework/SealKernel>

<use name=DataFormats/TotemData>

<use name=RecoTotemRP/RPServiceRecords>

<use name=SimGeneral/HepPDT>

Step 2: Translation

Translation is done by mainly done by running the translation tool from the CMS workbook: Translation Utilities

1. Dump to the screen a translation of your configuration file: python cfg2py.py
2. Put the output file in the appropriate python/subdirectory of your package: python translate.py <subsystem/package/dir/your cfg, cff, or cfi>
3. Find the comments in your old file, and tries to translate them over to the python file: python comments.py
4. Test if your new python file will compile: python <subsystem/package/python/your py file>

Summary of changes

IOMC/Elegent/test

IOMC/Elegent/validation

IOMC/EnergyVertexGenerator

Geometry/TotemRPGeometryBuilder/data/PrintGeometry.cfg

"error.txt", "warning.txt", "info.txt", "debugmessage.txt"  
-->  "errors", "warnings", "infos", "debugs"

Configuration/TotemCommon/python/LoggerMax_cfi.py

include "SimGeneral/MixingModule/data/mix_none.cfi"
-->  process.load("SimGeneral.MixingModule.mixNoPU_cfi")

L1TriggerTotem/T1Trigger/test/triggerGunT1_cfg.py

process.load("Configuration.StandardSequences.MagneticField_38T_cff")

RecoTotemT1T2/T2TrackProducer/test/T2RecoProdExample_cfg.py

RPInelasticReconstruction/test/runGun_transport_primvert_2stations_cfg.py

g4SimHits.Generator.ApplyPtCuts

g4SimHits.Generator.ApplyPCuts

Configuration/TotemOpticsConfiguration/python/*_cfi.py

RecoTotemRP/RPInelasticReconstruction/python/*_cfi.py 
Configuration/TotemCommon/python/g4SimHits_cfi.py

Configuration/TotemOpticsConfiguration/python/OpticsConfigBeta_(1535/05/2/90)_cfi.py
Configuration/TotemCommon/python/g4SimHits_cfi.py
TotemRPValidation/Examples/Data/*_cfg.py
L1TriggerTotem/CoincidenceChip/test/simdigi_cfg.py

Configuration/TotemCommon/data/geometry*.cfi

L1TriggerTotem/CoincidenceChip/test/simdigi_cfg.py
TotemRPValidation/HitDistributions/test/hitsBeta90_cfg.py
TotemRPValidation/Examples/Data/inelastic_*_cfg.py

process.XMLIdealGeometryESSource.geomXMLFiles.append('an xml file name')

Configuration/TotemCommon/data/geometryRP.cfi
TotemRPValidation/Examples/Data/inelastic_2_smeared_sim_cfg.py

ID	2-9
Reason	Make the pythia configuration files of Totem compatible with the ones in CMSSW_3_1_1.
Change	#1 PythiaSource is not registered as a plugin anymore, but Pythia6GeneratorFilter should be used.
	#2 The pythia module should be created as a generator of EDFilter not a source of EDSource any more. In order to use the pythia generator an empty source needs to be created. The random number generator should also include the seed for pythia generator.
	#3 'MSTP(52)=1' is added to the FORTAN code block named pythiaMinBias.
	#4 comEnergy = cms.double(X) is added.
Example-1	SimG4CMS/Forward/python/test/PythiaSD_cfi.py
Example-2	SimG4CMS/Forward/python/test/PythiaDD_cfi.py
Example-3	SimG4CMS/Forward/python/test/PythiaMB_cfi.py
comment	More info about the pythia fortun parameter switches: PYTHIA 6.4 Physics and Manual --> 9.3 The General Switches and Parameters (P 201 ~ P 225)

pair<T1DetId,vector<T1Cluster> >
pair<T1DetId,vector<T1DigiVfat> >
pair<T1DetId,vector<T1DigiWire> >
pair<T2DetId,vector<T2PadDigi> >
pair<T2DetId,vector<T2StripDigi> >
pair<T2DetId,vector<T2Cluster> >

DataFormats/T1Cluster/src/classes_def.xml
DataFormats/T1Cluster/src/classes.h
DataFormats/T1DigiVfat/src/classes_def.xm
DataFormats/T1DigiVfat/src/classes.h
DataFormats/T1DigiWire/src/classes_def.xml
DataFormats/T1DigiWire/src/classes.h
DataFormats/T2Digi/src/classes_def.xml
DataFormats/T2Digi/src/classes.h
DataFormats/T2Cluster/src/classes_def.xml
DataFormats/T2Cluster/src/classes.h
DataFormats/T1DetId/src/classes_def.xml
DataFormats/T1DetId/src/classes.h

TotemHitsT2Gem --> g4SimHitsTotemHitsT2Gem
TotemHitsT1 --> g4SimHitsTotemHitsT1
TotemHitsT2Gem --> g4SimHitsTotemHitsT2Gem
TotemHitsRP --> g4SimHitsTotemHitsRP

SimTotem/T2Digitizer/src/T2DigiProducer.cc
SimTotem/T1Digitizer/plugins/T1DigiProducer.cc
TotemT1T2Validation/T2RecoValidation/src/T2RecoAnalyzer.cc
SimTotem/RPDigiProducer/src/RPDigiProducer.cc

std::vector<edm::HepMCProduct*>
edm::Wrapper<CrossingFramePlaybackInfo >
edm::Wrapper<CrossingFrame<PSimHit> >
edm::Wrapper<CrossingFrame<PCaloHit> >
edm::Wrapper<CrossingFrame<SimTrack> >
edm::Wrapper<CrossingFrame<SimVertex> >
edm::Wrapper<CrossingFrame<edm::HepMCProduct> >
boost::shared_ptr<const edm::Wrapper<PCaloHit> >
boost::shared_ptr<const edm::Wrapper<PSimHit> >
boost::shared_ptr<const edm::Wrapper<SimTrack> >
boost::shared_ptr<const edm::Wrapper<SimVertex> >
boost::shared_ptr<const edm::Wrapper<edm::HepMCProduct> >
boost::shared_ptr<const edm::Wrapper<std::vector<PCaloHit> > >
boost::shared_ptr<const edm::Wrapper<std::vector<PSimHit> > >
boost::shared_ptr<const edm::Wrapper<std::vector<SimTrack> > >
boost::shared_ptr<const edm::Wrapper<std::vector<SimVertex> > >
boost::shared_ptr<const edm::Wrapper<std::vector<edm::HepMCProduct> > >
boost::detail::shared_count
boost::detail::sp_counted_base*
edm::Wrapper<PCaloHit>
edm::Wrapper<PSimHit>
edm::Wrapper<SimTrack>
edm::Wrapper<SimVertex>
edm::Wrapper<std::vector<edm::HepMCProduct> >

ID	2-13
Reason	Dictionary SimDataFormats/GeneratorProducts/src/classes_def.xml seems to be corrupted ("<" instead of "<"), this could be a reason of missing dictionary for HepMC::GenVertex. As a result, validation run, defined in separate configuration file cannot access data saved by event source.
Change	Copy SimDataFormats/GeneratorProducts/ from CMSSW_3_1_1's reposity, save them to Totem software. In src/classes_def.xml replace "&lt;" to "<" and "&gt;" to ">".
Comment	Useful Link: CMSSW/SimDataFormats/CrossingFrame/src/classes_def.xml

 + Add HepMCProductLabel = cms.string('source') 
    into TotemT1T2Validation/T2RecoValidation/src/T2RecoValidation_cfi.py, Line 14
 + Add HepMCProductLabel = cms.string('source') 
    into TotemT1T2Validation/T2RecoValidation/src/T2RecoValidation2_cfi.py, Line 14
 + Add HepMCProductLabel as a string variable 
    into TotemT1T2Validation/T2RecoValidation/interface/T2RecoValidation.h, Line 104
 + Add HepMCProductLabel = iConfig.getParameter<std::string>("HepMCProductLabel") 
    into TotemT1T2Validation/T2RecoValidation/src/T2RecoAnalyzer.cc, Line 44
 + Replace iEvent.getByLabel("source", EvtHandle) --> iEvent.getByLabel(HepMCProductLabel, EvtHandle) 
    in TotemT1T2Validation/T2RecoValidation/src/T2RecoAnalyzer.cc, Line 215

offline/cmssw/examples/validation/valgunT1T2mu_pro_and_val_cfg.py

offline/cmssw/examples/validation/ T1T2pythiaMBbeta2_pro_val_fullchain_cfg.py

Remarks (Mirko Berretti)

The label of HepMCProduct can no longer be 'source' for CMS-maintained generators.

For example, in order to retrieve HepMC particles in your analyzer, the following lines can be utilized (both for Particle Gun or Pythia Generator):

Handle<HepMCProduct> EvtHandle; 
iEvent.getByLabel(HepMCProductLabel, EvtHandle);

where the std::string HepMCProductLabel is initialized with the value "generator".

| Example-1 | TotemT1T2Validation/T2RecoValidation/T2RecoValidation_cfi.py

| Example-2 | Further information: https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideEventGenerationReleaseNotes31X

Step 3: Testing

The following tables show the testing procedure for the example test configuration files under offline/cmssw/examples/production/ (and validation/) @ CMSSW_3_1_1.

production/gunT1T2mu_cfg.py
validation/valgunT1T2mu_cfg.py

[1] cmsRun gunT1T2mu_cfg.py ==> gunT1T2mu.root
[2] copy production/gunT1T2mu.root to validation/
[3] mkdir gunT1T2mu/ under validation/
[4] cmsRun valgunT1T2mu_cfg.py ==> valT1PlotsMu.root, valT2PlotsG4Mu.root, 
    valT2PlotsDigiMu.root, valT2PlotsRecoMu.root, valT2PlotsRecoMu2.root
    under validation/gunT1T2mu/

production/gunT1T2pi_cfg.py
validation/valgunT1T2pi_cfg.py

[1] cmsRun gunT1T2pi_cfg.py ==> gunT1T2pi.root
[2] copy production/gunT1T2pi.root to validation/
[3] mkdir gunT1T2pi/ under validation/
[4] cmsRun valgunT1T2pi_cfg.py ==> valT1PlotsPi.root, valT2PlotsG4Pi.root,
    valT2PlotsDigiPi.root, valT2PlotsRecoPi.root, valT2PlotsRecoPi2.root
    under validation/gunT1T2pi/

-- ZhengkuiZhang - 15 July 2009

Synchronization Log (Jun 01 2009 - Aug 21 2009)

Synchronization 1 (10 July 2009)

CVS Check Out Date: 10 July 2009

Full synchronization trace can be downloaded from: https://twiki.cern.ch/twiki/pub/TOTEM/CompOfflineCMSSWMigration177to31/syn_1st.

Merged file list:

IOMC/Elegent/src/Elegent.cc

TotemDataReadout/Testbeam/interface/RawDataSource.h
TotemDataReadout/Testbeam/interface/RPDataDigiProducer.h
TotemDataReadout/Testbeam/plugins/BuildFile
TotemDataReadout/Testbeam/plugins/RawDataSource.cc
TotemDataReadout/Testbeam/plugins/RPDataDigiProducer.cc

TotemAlignment/RPRates/data/MCScoringPlaneProfiles.cfi
TotemAlignment/RPRates/data/ScoringPlaneProfiles.cfi
TotemAlignment/RPRates/plugins/BuildFile
TotemAlignment/RPRates/plugins/MCScoringPlaneProfiles.cc
TotemAlignment/RPRates/plugins/ScoringPlaneProfiles.cc
TotemAlignment/RPRates/interface/ScoringPlaneDistributions.h (a new file)
TotemAlignment/RPRates/src/ScoringPlaneDistributions.cc (a new file)

L1TriggerTotem/CoincidenceChip/interface/RPCoincidenceAnalyzer.h
L1TriggerTotem/CoincidenceChip/interface/RPCoincidenceProducer.h
L1TriggerTotem/CoincidenceChip/plugins/RPCoincidenceAnalyzer.cc
L1TriggerTotem/CoincidenceChip/plugins/RPCoincidenceProducer.cc

RecoTotemT1T2/PrimaryVertexProducer (a new module)

TotemBackground/PP/data/PPBckgAnalysis.cfi (a new file)
TotemBackground/PP/data/PPBckgSource.cfi
TotemBackground/PP/plugins/BuildFile
TotemBackground/PP/plugins/PPBckgAnalysis.cc (a new file)

Jan Kaspar made further changes and checked in to TOTEM repository's CMSSW_3_1_x branch. The changes Jan made is as follows:

Modifications by Jan Kaspar (14 July 2009)

TotemDataReadout package renamed and restructured

The package has been renamed to TotemRawData (it make more sense and is closer to CMS terminology). Moreover, the contents have been reshuffled into 3 submodules: DataFormats, Readers (alogorithms to read raw data files) and RawToDigi (algorithms to covert raw data to digi).

TotemBackground/BeamGas

has been freed from THnSparse class definition (it is now a part of the official ROOT distribution)

Shared packages under Geometry directory

There 3 packages originated by CMS and modified by TOTEM: CMSCommonData, ForwardCommonData and ForwardSimData. At the version 177, I compared files in these packages between CMS framework and TOTEM SW. Modified/new files have been then merged/copied to the 311 framework version, resulting in TOTEM 311 files.

In CMSCommonData, those were:

• recoConfiguration.xml (copied)
• rotations.xml (copied as rotations2.xml)
• totem_rotations (copied)
• normal/cmsextent (merged)

In ForwardCommonData (all copied unless stated differently)

• castor1.xml
• castor2.xml
• castor_fibres.xml
• cuts-beampipe.xml
• ionpump.xml (copied as ionpump2.xml)
• totem2cuts.xml

In ForwardSimData (all copied)

• TotemProdCuts.xml
• a.xml
• totemProdCuts.xml
• totemsensGem.xml

g4SimHits_cfi.py

3_1_1 framework version merged with 1_7_7 TOTEM version.

-- JanKaspar - 14 July 2009

Visualization module authored by Maciej Besta is added to Totem software @ CMSSW_3_1_1. This module is created for using Frog.

-- ZhengkuiZhang - 07 August 2009