XML To Database Round-Trip Validation of Detector Description

This page describes the validation procedure of taking a set of XML files, confirming they are correct, loading them into a single XML file which does not use DDAlgorithms, testing that file is correct, loading that single file into a local database, then reading that out and comparing the data content to the data content of the original set of XML files.

This process is meant to be used before loading the CMS integration database with payloads for the Geometry to be built from Conditions.

Introduction

As described in other documents (Offline Software Guide, Detector Description), the XML files are the master copy and input mechanism for the Detector Description of CMSSW. Here, we provide a mechanism to test the XML Schema conformance, validity of the DDD in memory model and the agreement of the database with the original set of XML files. We validate that a derived version (database) conforms to the original source (set of XML files).

Tutorials

Running the test out of the box.

In order to run the test you do the following in you private area. In a release area that is being created/validated there is no need to check out the package mentioned (as it should be in the release).

cmsrel YOURVERSION
cd YOURVERSION/src
cmsenv
addpkg GeometryReaders/XMLIdealGeometryESSource
scram b
cd GeometryReaders/XMLIdealGeometryESSource/test/
./runXMLBigFileToDBAndBackValidation.sh

The output consists of more than one file. Each can be examined for specific errors. A successful run looks like this:

START - All messages in this script pertain to geometry data described in Configuration/StandardSequence/python/GeometryIdeal_cff.py
        and xml files in: /home/wdd/CMSSW_installation/slc5_ia32_gcc434/cms/cmssw/CMSSW_3_9_0/src/Geometry/CMSCommonData/python/cmsIdealGeometryXML_cfi.py
Assuming the schema is here:  file:///home/wdd/CMSSW_installation/slc5_ia32_gcc434/cms/cmssw/CMSSW_3_9_0/src/DetectorDescription/Schema/DDLSchema.xsd
There ARE NO XML Schema violations in original XML files.
There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run.
Start to write the single BIG XML file.
Finish the write to the single BIG XML file.
There ARE NO Schema violations in the single BIG XML file.
There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run.
Start to write all geometry objects to the local DB including BIG XML file.
Finish writing all geometry objects to the local DB including BIG XML file.
Start to read the big XML file FROM the DB object
Done with reading the big XML file FROM the DB object
Start reading the XML from the original config file.
End reading the XML from the original config file.
All differences in position are less than tolerance.
ALL DONE!

The key lines are:

• There ARE NO XML Schema violations in original XML files.
• There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run.
• There ARE NO Schema violations in the single BIG XML file.
• There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run.
• All differences in position are less than tolerance.

Failures:

• The standard ones are: WARNING: There ARE ...
• Sometimes you might see a missing file failure. These should be investigated.
• Sometimes you might see an error with an @ sign. Again, this means something else failed.
• Of course there are many others you can see but all of these except the first one in this list, require examining the script and finding the failure.

If there are XML Schema violations, then one might want to debug by running the DOMCount executable by itself and there are instructions for that here: Detector Description Schema Validation.

Here is an example of a serious discrepancy one might see:

WARNING: There are 8 lines with differences greater than tolerance.  Please check tcdf.out for differences.
WARNING: Tolerance can be changed in the file testCompareDumpFiles.py.

There are many different files generated in this test in the current directory and also in a subdirectory that is created called "workarea". In the case above, one would want to look in the file "tcdf.out" for further details. If there are no errors that file will be empty, but when errors occur it can contain useful information.

Lines don't match or are out of synchronization. 
The difference is much bigger than just the numbers, actual parts are missing or added.
This program  does not handle this at this time... use diff first.
[ - /cms:OCMS[0]/cms:CMSE[1]/cmsextent:CMStoZDC[1]]
[ - /cms:OCMS[0]/cms:CMSE[1]/tracker:Tracker[1]]

Running the test for a particular geometry configuration (scenario)

In the previous example you were just running, without knowing it, against Configuration/StandardSequences/python/GeometryIdeal_cff.py and something it includes:

#PART of GeometryIdeal_cff.py
# Ideal geometry, needed for simulation
from Geometry.CMSCommonData.cmsIdealGeometryXML_cfi import *
...

Which refers to another python configuration. To run the validation explicitly for the default mentioned above, one would have done ./runXMLBigFileToDBAndBackValidation.sh GeometryIdeal $CMSSW_RELEASE_BASE/src/Geometry/CMSCommonData/python/cmsIdealGeometry_cfi.py.

Now, let us run it for the extended geometry of CMS.

#assuming you are still in YOURVERSION/src/GeometryReaders/XMLIdealGeometryESSource/test
rm *.out
rm -rf workarea
./runXMLBigFileToDBAndBackValidation.sh GeometryExtended $CMSSW_RELEASE_BASE/src/Geometry/CMSCommonData/python/cmsExtendedGeometryXML_cfi.py

There are several other scenarios one can test in an analogous manner.

Contacts & References

• Detector Description
• Details for Database Geometry
• Contact: Michael Case
• Hypernews forum: https://hypernews.cern.ch/HyperNews/CMS/get/geometry.html

Review status

Reviewer/Editor and Date (copy from screen)	Comments
KatiLassilaPerini - 23 Jan 2007	created template page

Responsible: MichaelCase
Last reviewed by: Most recent reviewer