Difference: SWGuideXMLToDatabaseRTValidation (3 vs. 4)

Revision 42010-11-02 - DavidDagenhart

Line: 1 to 1
 
META TOPICPARENT name="SWGuideDetectorDescription"

XML To Database Round-Trip Validation of Detector Description

Line: 31 to 31
 The output consists of more than one file. Each can be examined for specific errors. A successful run looks like this:
Changed:
<
<
START - All messages in this script pertain to geometry data described in Configuration/StandardSequence GeometryIdeal and xml files in: /afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src/Geometry/CMSCommonData/python/cmsIdealGeometryXML_cfi.py Assuming the schema is here: file:///afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src/DetectorDescription/Schema/DDLSchema.xsd No XML Schema violations in original xml files.
>
>
START - All messages in this script pertain to geometry data described in Configuration/StandardSequence/python/GeometryIdeal_cff.py and xml files in: /home/wdd/CMSSW_installation/slc5_ia32_gcc434/cms/cmssw/CMSSW_3_9_0/src/Geometry/CMSCommonData/python/cmsIdealGeometryXML_cfi.py Assuming the schema is here: file:///home/wdd/CMSSW_installation/slc5_ia32_gcc434/cms/cmssw/CMSSW_3_9_0/src/DetectorDescription/Schema/DDLSchema.xsd There ARE NO XML Schema violations in original XML files.
 There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run. Start to write the single BIG XML file. Finish the write to the single BIG XML file.
Changed:
<
<
There ARE NO differences in DOMCount of the single BIG file since the last time it was run
>
>
There ARE NO Schema violations in the single BIG XML file.
 There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run. Start to write all geometry objects to the local DB including BIG XML file. Finish writing all geometry objects to the local DB including BIG XML file. Start to read the big XML file FROM the DB object Done with reading the big XML file FROM the DB object Start reading the XML from the original config file.
Deleted:
<
<
Here I am Top Most LogicalPart =cms:OCMS mat=materials:Air solid=cms:OCMS Polycone_rrz: 0 6.28319 -27100 0 1000 -27000 0 1000 -27000 0 13000 27000 0 13000 27000 0 1000 27100 0 1000 After the GeoHistory in the output file dumpGeoHistoryOnRead you will see x, y, z, r11, r12, r13, r21, r22, r23, r31, r32, r33 finished
 End reading the XML from the original config file.
Deleted:
<
<
109.343u 1.457s 1:50.87 99.9% 0+0k 0+160io 1pf+0w
 All differences in position are less than tolerance. ALL DONE!
Deleted:
<
<
470.012u 38.582s 8:32.12 99.3% 0+0k 0+24io 39pf+0w
 

The key lines are:

Changed:
<
<
  • No XML Schema violations in original xml files.
>
>
  • There ARE NO XML Schema violations in original XML files.
 
  • There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run.
Changed:
<
<
  • There ARE NO differences in DOMCount of the single BIG file since the last time it was run
>
>
  • There ARE NO Schema violations in the single BIG XML file.
 
  • There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run.
  • All differences in position are less than tolerance.
Line: 75 to 65
 
  • Sometimes you might see an error with an @ sign. Again, this means something else failed.
  • Of course there are many others you can see but all of these except the first one in this list, require examining the script and finding the failure.
Changed:
<
<

Running the test for a particular geometry configuration.

>
>
If there are XML Schema violations, then one might want to debug by running the DOMCount executable by itself and there are instructions for that here: Detector Description Schema Validation.

Here is an example of a serious discrepancy one might see:

WARNING: There are 8 lines with differences greater than tolerance.  Please check tcdf.out for differences.
WARNING: Tolerance can be changed in the file testCompareDumpFiles.py.

There are many different files generated in this test in the current directory and also in a subdirectory that is created called "workarea". In the case above, one would want to look in the file "tcdf.out" for further details. If there are no errors that file will be empty, but when errors occur it can contain useful information.

Lines don't match or are out of synchronization. 
The difference is much bigger than just the numbers, actual parts are missing or added.
This program  does not handle this at this time... use diff first.
[ - /cms:OCMS[0]/cms:CMSE[1]/cmsextent:CMStoZDC[1]]
[ - /cms:OCMS[0]/cms:CMSE[1]/tracker:Tracker[1]]

Running the test for a particular geometry configuration (scenario)

  In the previous example you were just running, without knowing it, against Configuration/StandardSequences/python/GeometryIdeal_cff.py and something it includes:
Line: 97 to 112
 ./runXMLBigFileToDBAndBackValidation.sh GeometryExtended $CMSSW_RELEASE_BASE/src/Geometry/CMSCommonData/python/cmsExtendedGeometryXML_cfi.py
Changed:
<
<
Upon completion we might get:

START - All messages in this script pertain to geometry data described in Configuration/StandardSequence GeometryExtended
        and xml files in: /afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src/Geometry
               /CMSCommonData/python/cmsExtendedGeometryXML_cfi.py
Assuming the schema is here:  file:///afs/cern.ch/cms/sw/ReleaseCandidates/slc4_ia32_gcc432/thu/3.4-thu-01/CMSSW_3_4_X_2009-10-22-0100/src
                                                        /DetectorDescription/Schema/DDLSchema.xsd
WARNING: There ARE XML Schema violations in original XML files and can be seen in dcorig.out.
There ARE NO differences in the DD named objects from the standard xml files since the last ddreport.sh was run.
Start to write the single BIG XML file.
Finish the write to the single BIG XML file.
There ARE NO Schema violations in the single BIG XML file.
There ARE NO differences in the DD named objects from the single BIG xml file since the last ddreport.sh was run.
Start to write all geometry objects to the local DB including BIG XML file.
Finish writing all geometry objects to the local DB including BIG XML file.
Start to read the big XML file FROM the DB object
Done with reading the big XML file FROM the DB object
Start reading the XML from the original config file.
Here I am 
Top Most LogicalPart =cms:OCMS 
 mat=materials:Air
 solid=cms:OCMS   Polycone_rrz: 0 6.28319 -450000 0 1000 -27000 0 1000 -27000 0 13000 27000 0 13000 27000 0 1000 450000 0 1000 
After the GeoHistory in the output file dumpGeoHistoryOnRead you will see x, y, z, r11, r12, r13, r21, r22, r23, r31, r32, r33
finished
End reading the XML from the original config file.
109.999u 1.524s 1:51.53 99.9%   0+0k 0+152io 0pf+0w
All differences in position are less than tolerance.
ALL DONE!
479.959u 38.694s 8:39.52 99.8%   0+0k 0+0io 0pf+0w

As you can see, even with warning regarding XML schema violations the data in the DDD memory is the validated. The WARNING near the top in this case concern niceties which should be fixed but did not affect the DDD in-memory model. In some cases such warnings can lead to serious discrepancies in the data, in which case the message All differences in position are less than tolerance would be replaced by something else. At best a warning indicating the number of lines difference, at worst a CMS Exception is thrown due to major differences in the output. In other words, running DOMCount and DDErrorReport eliminate some troubling errors in the main CMSSW before they start, but also catch some trivial mistakes that do not affect the core CMSSW code.

In this case, the XML Schema violations are due to one file having an incorrect relative path for DOMCount (see Detector Description Schema Validation).

An example of how the error with some actual discrepancies one would see:

WARNING: There are 8 lines with differences greater than tolerance.  Please check tcdf.out for differences.
WARNING: Tolerance can be changed in the file testCompareDumpFiles.py.

And in the case of a CMS Exception one would see in tcdf.out

Lines don't match or are out of synchronization. 
The difference is much bigger than just the numbers, actual parts are missing or added.
This program  does not handle this at this time... use diff first.
[ - /cms:OCMS[0]/cms:CMSE[1]/cmsextent:CMStoZDC[1]]
[ - /cms:OCMS[0]/cms:CMSE[1]/tracker:Tracker[1]]
>
>
There are several other scenarios one can test in an analogous manner.
 

Contacts & References

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback