Difference: SWGuideXMLToDatabaseRTValidation (1 vs. 8)

Revision 82014-08-19 - IannaOsborne

Line: 1 to 1
 
META TOPICPARENT name="SWGuideDetectorDescription"

XML To Database Round-Trip Validation of Detector Description

Line: 24 to 24
 cd YOURVERSION/src cmsenv git cms-addpkg GeometryReaders/XMLIdealGeometryESSource
Deleted:
<
<
git cms-addpkg DetectorDescription/RegressionTest
 scram b -j 6 cd GeometryReaders/XMLIdealGeometryESSource/test/ ./runXMLBigFileToDBAndBackValidation.sh

Revision 72014-08-12 - IannaOsborne

Line: 1 to 1
 
META TOPICPARENT name="SWGuideDetectorDescription"

XML To Database Round-Trip Validation of Detector Description

Line: 23 to 23
 cmsrel YOURVERSION cd YOURVERSION/src cmsenv
Changed:
<
<
addpkg GeometryReaders/XMLIdealGeometryESSource scram b
>
>
git cms-addpkg GeometryReaders/XMLIdealGeometryESSource git cms-addpkg DetectorDescription/RegressionTest scram b -j 6
 cd GeometryReaders/XMLIdealGeometryESSource/test/ ./runXMLBigFileToDBAndBackValidation.sh

Revision 62011-12-16 - IannaOsborne

Line: 1 to 1
 
META TOPICPARENT name="SWGuideDetectorDescription"

XML To Database Round-Trip Validation of Detector Description

Added:
>
>
Responsible: IannaOsborne
 

Revision 52010-12-13 - IannaOsborne

Line: 1 to 1
 
META TOPICPARENT name="SWGuideDetectorDescription"

XML To Database Round-Trip Validation of Detector Description

Line: 117 to 117
 

Contacts & References

Changed:
<
<
  • Contact: Michael Case
>
>
  • Contact: Ianna Osborne
 

Line: 130 to 130
 
<!-- In the following line, be sure to put a blank space AFTER your name; otherwise the Summary doesn't come out right. -->
Changed:
<
<
Responsible: MichaelCase
>
>
Responsible: IannaOsborne
 Last reviewed by: Most recent reviewer

Revision 42010-11-02 - DavidDagenhart

Line: 1 to 1
 
META TOPICPARENT name="SWGuideDetectorDescription"

XML To Database Round-Trip Validation of Detector Description

Line: 31 to 31
 The output consists of more than one file. Each can be examined for specific errors. A successful run looks like this:
Changed:
<
<
START - All messages in this script pertain to geometry data described in Configuration/StandardSequence GeometryIdeal and xml files in: /afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src/Geometry/CMSCommonData/python/cmsIdealGeometryXML_cfi.py Assuming the schema is here: file:///afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src/DetectorDescription/Schema/DDLSchema.xsd No XML Schema violations in original xml files.
>
>
START - All messages in this script pertain to geometry data described in Configuration/StandardSequence/python/GeometryIdeal_cff.py and xml files in: /home/wdd/CMSSW_installation/slc5_ia32_gcc434/cms/cmssw/CMSSW_3_9_0/src/Geometry/CMSCommonData/python/cmsIdealGeometryXML_cfi.py Assuming the schema is here: file:///home/wdd/CMSSW_installation/slc5_ia32_gcc434/cms/cmssw/CMSSW_3_9_0/src/DetectorDescription/Schema/DDLSchema.xsd There ARE NO XML Schema violations in original XML files.
 There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run. Start to write the single BIG XML file. Finish the write to the single BIG XML file.
Changed:
<
<
There ARE NO differences in DOMCount of the single BIG file since the last time it was run
>
>
There ARE NO Schema violations in the single BIG XML file.
 There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run. Start to write all geometry objects to the local DB including BIG XML file. Finish writing all geometry objects to the local DB including BIG XML file. Start to read the big XML file FROM the DB object Done with reading the big XML file FROM the DB object Start reading the XML from the original config file.
Deleted:
<
<
Here I am Top Most LogicalPart =cms:OCMS mat=materials:Air solid=cms:OCMS Polycone_rrz: 0 6.28319 -27100 0 1000 -27000 0 1000 -27000 0 13000 27000 0 13000 27000 0 1000 27100 0 1000 After the GeoHistory in the output file dumpGeoHistoryOnRead you will see x, y, z, r11, r12, r13, r21, r22, r23, r31, r32, r33 finished
 End reading the XML from the original config file.
Deleted:
<
<
109.343u 1.457s 1:50.87 99.9% 0+0k 0+160io 1pf+0w
 All differences in position are less than tolerance. ALL DONE!
Deleted:
<
<
470.012u 38.582s 8:32.12 99.3% 0+0k 0+24io 39pf+0w
 

The key lines are:

Changed:
<
<
  • No XML Schema violations in original xml files.
>
>
  • There ARE NO XML Schema violations in original XML files.
 
  • There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run.
Changed:
<
<
  • There ARE NO differences in DOMCount of the single BIG file since the last time it was run
>
>
  • There ARE NO Schema violations in the single BIG XML file.
 
  • There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run.
  • All differences in position are less than tolerance.
Line: 75 to 65
 
  • Sometimes you might see an error with an @ sign. Again, this means something else failed.
  • Of course there are many others you can see but all of these except the first one in this list, require examining the script and finding the failure.
Changed:
<
<

Running the test for a particular geometry configuration.

>
>
If there are XML Schema violations, then one might want to debug by running the DOMCount executable by itself and there are instructions for that here: Detector Description Schema Validation.

Here is an example of a serious discrepancy one might see:

WARNING: There are 8 lines with differences greater than tolerance.  Please check tcdf.out for differences.
WARNING: Tolerance can be changed in the file testCompareDumpFiles.py.

There are many different files generated in this test in the current directory and also in a subdirectory that is created called "workarea". In the case above, one would want to look in the file "tcdf.out" for further details. If there are no errors that file will be empty, but when errors occur it can contain useful information.

Lines don't match or are out of synchronization. 
The difference is much bigger than just the numbers, actual parts are missing or added.
This program  does not handle this at this time... use diff first.
[ - /cms:OCMS[0]/cms:CMSE[1]/cmsextent:CMStoZDC[1]]
[ - /cms:OCMS[0]/cms:CMSE[1]/tracker:Tracker[1]]

Running the test for a particular geometry configuration (scenario)

  In the previous example you were just running, without knowing it, against Configuration/StandardSequences/python/GeometryIdeal_cff.py and something it includes:
Line: 97 to 112
 ./runXMLBigFileToDBAndBackValidation.sh GeometryExtended $CMSSW_RELEASE_BASE/src/Geometry/CMSCommonData/python/cmsExtendedGeometryXML_cfi.py
Changed:
<
<
Upon completion we might get:

START - All messages in this script pertain to geometry data described in Configuration/StandardSequence GeometryExtended
        and xml files in: /afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src/Geometry
               /CMSCommonData/python/cmsExtendedGeometryXML_cfi.py
Assuming the schema is here:  file:///afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src
                                                        /DetectorDescription/Schema/DDLSchema.xsd
WARNING: There ARE XML Schema violations in original XML files and can be seen in dcorig.out.
There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run.
Start to write the single BIG XML file.
Finish the write to the single BIG XML file.
There ARE NO Schema violations in the single BIG XML file.
There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run.
Start to write all geometry objects to the local DB including BIG XML file.
Finish writing all geometry objects to the local DB including BIG XML file.
Start to read the big XML file FROM the DB object
Done with reading the big XML file FROM the DB object
Start reading the XML from the original config file.
Here I am 
Top Most LogicalPart =cms:OCMS 
 mat=materials:Air
 solid=cms:OCMS   Polycone_rrz: 0 6.28319 -450000 0 1000 -27000 0 1000 -27000 0 13000 27000 0 13000 27000 0 1000 450000 0 1000 
After the GeoHistory in the output file dumpGeoHistoryOnRead you will see x, y, z, r11, r12, r13, r21, r22, r23, r31, r32, r33
finished
End reading the XML from the original config file.
109.999u 1.524s 1:51.53 99.9%   0+0k 0+152io 0pf+0w
All differences in position are less than tolerance.
ALL DONE!
479.959u 38.694s 8:39.52 99.8%   0+0k 0+0io 0pf+0w

As you can see, even with warning regarding XML schema violations the data in the DDD memory is the validated. The WARNING near the top in this case concern niceties which should be fixed but did not affect the DDD in-memory model. In some cases such warnings can lead to serious discrepancies in the data, in which case the message All differences in position are less than tolerance would be replaced by something else. At best a warning indicating the number of lines difference, at worst a CMS Exception is thrown due to major differences in the output. In other words, running DOMCount and DDErrorReport eliminate some troubling errors in the main CMSSW before they start, but also catch some trivial mistakes that do not affect the core CMSSW code.

In this case, the XML Schema violations are due to one file having an incorrect relative path for DOMCount (see Detector Description Schema Validation).

An example of how the error with some actual discrepancies one would see:

WARNING: There are 8 lines with differences greater than tolerance.  Please check tcdf.out for differences.
WARNING: Tolerance can be changed in the file testCompareDumpFiles.py.

And in the case of a CMS Exception one would see in tcdf.out

Lines don't match or are out of synchronization. 
The difference is much bigger than just the numbers, actual parts are missing or added.
This program  does not handle this at this time... use diff first.
[ - /cms:OCMS[0]/cms:CMSE[1]/cmsextent:CMStoZDC[1]]
[ - /cms:OCMS[0]/cms:CMSE[1]/tracker:Tracker[1]]
>
>
There are several other scenarios one can test in an analogous manner.
 

Contacts & References

Revision 32010-11-02 - DavidDagenhart

Line: 1 to 1
 
META TOPICPARENT name="SWGuideDetectorDescription"

XML To Database Round-Trip Validation of Detector Description

Line: 7 to 7
  This page describes the validation procedure of taking a set of XML files, confirming they are correct, loading them into a single XML file which does not use DDAlgorithms, testing that file is correct, loading that single file into a local database, then reading that out and comparing the data content to the data content of the original set of XML files.
Changed:
<
<
This process is meant to be used before loading the CMS integration database with payloads for the Geometry to be built from from Conditions
>
>
This process is meant to be used before loading the CMS integration database with payloads for the Geometry to be built from Conditions.
 

Introduction

Line: 19 to 19
 In order to run the test you do the following in you private area. In a release area that is being created/validated there is no need to check out the package mentioned (as it should be in the release).
Changed:
<
<
scram p CMSSW YOURVERSION
>
>
cmsrel YOURVERSION
 cd YOURVERSION/src cmsenv addpkg GeometryReaders/XMLIdealGeometryESSource
Added:
>
>
scram b
 cd GeometryReaders/XMLIdealGeometryESSource/test/ ./runXMLBigFileToDBAndBackValidation.sh

Revision 22009-10-23 - MichaelCase

Line: 1 to 1
 
META TOPICPARENT name="SWGuideDetectorDescription"
Deleted:
<
<
 

XML To Database Round-Trip Validation of Detector Description

Line: 8 to 7
  This page describes the validation procedure of taking a set of XML files, confirming they are correct, loading them into a single XML file which does not use DDAlgorithms, testing that file is correct, loading that single file into a local database, then reading that out and comparing the data content to the data content of the original set of XML files.
Added:
>
>
This process is meant to be used before loading the CMS integration database with payloads for the Geometry to be built from from Conditions
 

Introduction

Changed:
<
<
As described in other documents, the XML files are the master copy and input mechanism for the Detector Description of CMSSW. Here, we provide a mechanism to test the XML Schema conformance, validity of the DDD in memory model and the agreement of the database with the original set of XML files. We validate that a derived version (database) conforms to the original source (set of XML files).
>
>
As described in other documents (Offline Software Guide, Detector Description), the XML files are the master copy and input mechanism for the Detector Description of CMSSW. Here, we provide a mechanism to test the XML Schema conformance, validity of the DDD in memory model and the agreement of the database with the original set of XML files. We validate that a derived version (database) conforms to the original source (set of XML files).
 

Tutorials

Added:
>
>

Running the test out of the box.

In order to run the test you do the following in you private area. In a release area that is being created/validated there is no need to check out the package mentioned (as it should be in the release).

scram p CMSSW YOURVERSION
cd YOURVERSION/src
cmsenv
addpkg GeometryReaders/XMLIdealGeometryESSource
cd GeometryReaders/XMLIdealGeometryESSource/test/
./runXMLBigFileToDBAndBackValidation.sh

The output consists of more than one file. Each can be examined for specific errors. A successful run looks like this:

START - All messages in this script pertain to geometry data described in Configuration/StandardSequence GeometryIdeal
        and xml files in: /afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src/Geometry/\
CMSCommonData/python/cmsIdealGeometryXML_cfi.py
Assuming the schema is here:  file:///afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/\
src/DetectorDescription/Schema/DDLSchema.xsd
No XML Schema violations in original xml files.
There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run.
Start to write the single BIG XML file.
Finish the write to the single BIG XML file.
There ARE NO differences in DOMCount of the single BIG file since the last time it was run
There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run.
Start to write all geometry objects to the local DB including BIG XML file.
Finish writing all geometry objects to the local DB including BIG XML file.
Start to read the big XML file FROM the DB object
Done with reading the big XML file FROM the DB object
Start reading the XML from the original config file.
Here I am 
Top Most LogicalPart =cms:OCMS 
 mat=materials:Air
 solid=cms:OCMS   Polycone_rrz: 0 6.28319 -27100 0 1000 -27000 0 1000 -27000 0 13000 27000 0 13000 27000 0 1000 27100 0 1000 
After the GeoHistory in the output file dumpGeoHistoryOnRead you will see x, y, z, r11, r12, r13, r21, r22, r23, r31, r32, r33
finished
End reading the XML from the original config file.
109.343u 1.457s 1:50.87 99.9%   0+0k 0+160io 1pf+0w
All differences in position are less than tolerance.
ALL DONE!
470.012u 38.582s 8:32.12 99.3%   0+0k 0+24io 39pf+0w

The key lines are:

  • No XML Schema violations in original xml files.
  • There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run.
  • There ARE NO differences in DOMCount of the single BIG file since the last time it was run
  • There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run.
  • All differences in position are less than tolerance.

Failures:

  • The standard ones are: WARNING: There ARE ...
  • Sometimes you might see a missing file failure. These should be investigated.
  • Sometimes you might see an error with an @ sign. Again, this means something else failed.
  • Of course there are many others you can see but all of these except the first one in this list, require examining the script and finding the failure.

Running the test for a particular geometry configuration.

In the previous example you were just running, without knowing it, against Configuration/StandardSequences/python/GeometryIdeal_cff.py and something it includes:

 
Changed:
<
<
In order to run the test you do the following:
>
>
#PART of GeometryIdeal_cff.py
# Ideal geometry, needed for simulation
from Geometry.CMSCommonData.cmsIdealGeometryXML_cfi import *
...

Which refers to another python configuration. To run the validation explicitly for the default mentioned above, one would have done ./runXMLBigFileToDBAndBackValidation.sh GeometryIdeal $CMSSW_RELEASE_BASE/src/Geometry/CMSCommonData/python/cmsIdealGeometry_cfi.py.

Now, let us run it for the extended geometry of CMS.

#assuming you are still in YOURVERSION/src/GeometryReaders/XMLIdealGeometryESSource/test
rm *.out
rm -rf workarea
./runXMLBigFileToDBAndBackValidation.sh GeometryExtended $CMSSW_RELEASE_BASE/src/Geometry/CMSCommonData/python/cmsExtendedGeometryXML_cfi.py

Upon completion we might get:

 
Added:
>
>
START - All messages in this script pertain to geometry data described in Configuration/StandardSequence GeometryExtended and xml files in: /afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src/Geometry /CMSCommonData/python/cmsExtendedGeometryXML_cfi.py Assuming the schema is here: file:///afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src /DetectorDescription/Schema/DDLSchema.xsd WARNING: There ARE XML Schema violations in original XML files and can be seen in dcorig.out. There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run. Start to write the single BIG XML file. Finish the write to the single BIG XML file. There ARE NO Schema violations in the single BIG XML file. There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run. Start to write all geometry objects to the local DB including BIG XML file. Finish writing all geometry objects to the local DB including BIG XML file. Start to read the big XML file FROM the DB object Done with reading the big XML file FROM the DB object Start reading the XML from the original config file. Here I am Top Most LogicalPart =cms:OCMS mat=materials:Air solid=cms:OCMS Polycone_rrz: 0 6.28319 -450000 0 1000 -27000 0 1000 -27000 0 13000 27000 0 13000 27000 0 1000 450000 0 1000 After the GeoHistory in the output file dumpGeoHistoryOnRead you will see x, y, z, r11, r12, r13, r21, r22, r23, r31, r32, r33 finished End reading the XML from the original config file. 109.999u 1.524s 1:51.53 99.9% 0+0k 0+152io 0pf+0w All differences in position are less than tolerance. ALL DONE! 479.959u 38.694s 8:39.52 99.8% 0+0k 0+0io 0pf+0w

As you can see, even with warning regarding XML schema violations the data in the DDD memory is the validated. The WARNING near the top in this case concern niceties which should be fixed but did not affect the DDD in-memory model. In some cases such warnings can lead to serious discrepancies in the data, in which case the message All differences in position are less than tolerance would be replaced by something else. At best a warning indicating the number of lines difference, at worst a CMS Exception is thrown due to major differences in the output. In other words, running DOMCount and DDErrorReport eliminate some troubling errors in the main CMSSW before they start, but also catch some trivial mistakes that do not affect the core CMSSW code.

 
Changed:
<
<

Documentation

A group of algorithms

  • link to a doc page - short explanation
  • link to another doc page - another short explanation

Algorithms for something else

  • link to a doc page - short explanation
>
>
In this case, the XML Schema violations are due to one file having an incorrect relative path for DOMCount (see Detector Description Schema Validation).
 
Changed:
<
<

Heading 1

>
>
An example of how the error with some actual discrepancies one would see:
 
Changed:
<
<

Subheading 1

>
>
WARNING: There are 8 lines with differences greater than tolerance.  Please check tcdf.out for differences.
WARNING: Tolerance can be changed in the file testCompareDumpFiles.py.
 
Changed:
<
<

Contacts

>
>
And in the case of a CMS Exception one would see in tcdf.out

Lines don't match or are out of synchronization. 
The difference is much bigger than just the numbers, actual parts are missing or added.
This program  does not handle this at this time... use diff first.
[ - /cms:OCMS[0]/cms:CMSE[1]/cmsextent:CMStoZDC[1]]
[ - /cms:OCMS[0]/cms:CMSE[1]/tracker:Tracker[1]]

Contacts & References

 

Revision 12009-10-22 - MichaelCase

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="SWGuideDetectorDescription"

XML To Database Round-Trip Validation of Detector Description

This page describes the validation procedure of taking a set of XML files, confirming they are correct, loading them into a single XML file which does not use DDAlgorithms, testing that file is correct, loading that single file into a local database, then reading that out and comparing the data content to the data content of the original set of XML files.

Introduction

As described in other documents, the XML files are the master copy and input mechanism for the Detector Description of CMSSW. Here, we provide a mechanism to test the XML Schema conformance, validity of the DDD in memory model and the agreement of the database with the original set of XML files. We validate that a derived version (database) conforms to the original source (set of XML files).

Tutorials

In order to run the test you do the following:


---++ Documentation
---+++ A group of algorithms
   * link to a doc page - short explanation
   * link to another doc page - another short explanation
---+++ Algorithms for something else
   * link to a doc page - short explanation

---++ Heading 1

---+++ Subheading 1

---++ Contacts
   * [[DetectorDescription][SWGuideDetectorDescription]]
   * Contact: Michael Case
   * *Hypernews forum*: [[https://hypernews.cern.ch/HyperNews/CMS/get/geometry.html]]

#ReviewStatus
---++!! Review status 

<!-- Add your review status in this table structure with 2 columns delineated by three vertical bars -->

| *Reviewer/Editor and Date (copy from screen)* | *Comments* |
| Main.KatiLassilaPerini - 23 Jan 2007 | created template page  |

<!-- In the following line, be sure to put a blank space AFTER your name; otherwise the Summary doesn't come out right. -->

%RESPONSIBLE% !MichaelCase %BR%
%REVIEW% Most recent reviewer
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback