DataQualityTools

Package with Data Quality algorithms targeting Global Monitoring and beyond
https://svnweb.cern.ch/trac/atlasoff/browser/DataQuality/DataQualityTools/trunk

Get a working area

From CERN lxplus:

  • You can use this setup script: vim DQdevStart
    echo "setting up Athena"
    cd /afs/cern.ch/work/e/estel/athena/18.X.0
    export AtlasSetup=/afs/cern.ch/atlas/software/dist/AtlasSetup
    alias asetup='source $AtlasSetup/scripts/asetup.sh'
    asetup rel_3,here
    source /afs/cern.ch/atlas/software/tools/pyAMI/setup.sh
    voms-proxy-init -voms atlas
  • create the directory /afs/cern.ch/work/e/estel/athena/18.X.0
  • The rel_3 means you are using a nightly, modify it needed
  • Make sure you have a Grid Certificate installed
  • Check out the DataQualityTools package via cmt (not svn!):
    mkdir DataQuality
    cmt co svn+ssh://svn.cern.ch/reps/atlasoff/DataQuality/DataQualityTools DataQuality/DataQualityTools

Every time you start a terminal

  • Configure cmt and compile:
    cd /afs/cern.ch/work/e/estel/athena/18.X.0/DataQuality/DataQualityTools/cmt
    cmt config
    source setup.sh
    gmake

Run

  • Go to your test folder:
    cd  /afs/cern.ch/work/e/estel/athena/18.X.0/
    mkdir testarea; cd testarea
  • Make a subdirecory for every test. You can name it according to data/mc/cosmics. See RecoTrf twiki.
    AMI=q122     (Test tag: real collisions)
    AMI=q121     (Test tag: simulated collisions)
    AMI=q120     (Test tag: real cosmics)
    You should make sure your changes run on the 3 types.
    So, if you plan to run a test on MC, you can do:
    mkdir q121_001; cd q121_001
  • The run command (runs on 10 events):
    # cosmics
    Reco_trf.py AMI=q120  2>&1 | tee log_Reco_trf_q120.txt
    # ttbar MC
    Reco_trf.py AMI=q121  2>&1 | tee log_Reco_trf_q121.txt
    # collision data
    Reco_trf.py AMI=q122  2>&1 | tee log_Reco_trf_q122.txt
  • The main output file with the histograms will be called: Monitor.root
  • The main JO (where you can disable the different tools) are located under: DataQuality/DataQualityTools/share/DataQualityMon_jobOptions.py
  • AMI=XXX is actually a shorcut for a longer command. You might not need all the output requested by that command. To find out the full command use GetCommand.py AMI=q121 . Documented in https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/RecoTrf#GetCommand
  • for instance: Reco_trf.py beamType='collisions' maxEvents='-1' AMITag='q121' outputAODFile='myAOD.pool.root' outputTAGFile='myTAG.root' preExec='rec.Commissioning.set_Value_and_Lock(True);rec.doFloatingPointException.set_Value_and_Lock(True)' autoConfiguration='everything' outputESDFile='myESD.pool.root'  outputHISTFile='myHIST.root' inputRDOFile='/afs/cern.ch/atlas/offline/test/mc11_valid.105200.T1_McAtNlo_Jimmy.digit.RDO.e835_s1310_s1300_d622.RDO.10evts.pool.root'

How to put your changes into the release:

  • create a new tag (document your changes in the ChangeLog before commiting!):
    svn cp blabla/trunk blabla/tags/blabla-XX-YY-ZZ  -m "blabla-XX-YY-ZZ"
    example:
    svn cp svn+ssh://svn.cern.ch/reps/atlasoff/DataQuality/DataQualityTools/trunk svn+ssh://svn.cern.ch/reps/atlasoff/DataQuality/DataQualityTools/tags/DataQualityTools-00-03-17 -m "DataQualityTools-00-03-17"
  • Use TagCollector http://atlastagcollector.in2p3.fr/AMI/
  • otherwise the list to contact is atlas-sw-tagapproveSPMANOTcern.ch

rel 19 migration

How to test your changes (from Alex Koutsman)

  1. asetup 19.0.X.Y-VAL,rel_3,64,here,builds (in your favorite directory)
  2. cmt co DataQuality/DataQualityTools (see above) or /afs/cern.ch/user/k/koutsman/public/forEstel/19.0.X.Y-VAL_rel_3
  3. Run: Reco_tf.py '--conditionsTag=COMCOND-ES1PA-006-05' '--inputBSFile=/afs/cern.ch/work/j/jfrost/public/AMITagtest/data12_8TeV.00213204.express_express.daq.RAW._lb0220._SFO-2._0001.data' '--outputESDFile=myESD.pool.root' '--ignoreErrors=True' '--autoConfiguration=everything' '--maxEvents=5' '--preExec=InDetFlags.useBeamConstraint.set_Value_and_Lock(False);DQMonFlags.doTRTMon.set_Value_and_Lock(False);DQMonFlags.doJetTagMon.set_Value_and_Lock(False);DQMonFlags.doHLTMon.set_Value_and_Lock(False);DQMonFlags.doMuonRawMon=False;DQMonFlags.doMuonSegmentMon=False;DQMonFlags.doMuonTrackMon=False;DQMonFlags.doMuonAlignMon=False;DQMonFlags.doMuonTrkPhysMon=False;DQMonFlags.doMuonPhysicsMon=True;' '--geometryVersion=ATLAS-GEO-20-00-02' '--outputAODFile=myAOD.pool.root' '--outputHISTFile=myHIST.root' >& /tmp/koutsman/19.0.X.Y-VAL_rel4_test.log & (pay attention to DQMonFlags.doMuonTrkPhysMon=False and the log-file location /tmp/koutsman/19.0.X.Y-VAL_rel4_test.log)
  4. tail -f  /tmp/koutsman/19.0.X.Y-VAL_rel4_test.log
  5. exampe log-file is here: /afs/cern.ch/user/k/koutsman/public/forDQ/19.0.X.Y-VAL_rel5_test.log
  6. to compile the package you can use: source cmtCleanMake.sh DataQuality/DataQualityTools/ (a script written by Stephen and in that directory too) but cd cmt and gmake also works

Useful Links

  1. http://acode-browser.usatlas.bnl.gov/lxr/source/atlas/PhysicsAnalysis/PhysicsValidation/PhysValMonitoring/src/PhysValExample.cxx
  2. http://acode-browser.usatlas.bnl.gov/lxr/source/atlas/Tracking/TrkVertexFitter/TrkVxEdmCnv/TrkVxEdmCnv/IVxCandidateXAODVertex.h for the converter from xAOD to old-EDM
  3. http://acode-browser.usatlas.bnl.gov/lxr/source/atlas/Event/xAOD/xAODEgamma/xAODEgamma/versions/Egamma_v1.h
  4. http://acode-browser.usatlas.bnl.gov/lxr/search

Some twikis

  • http://atlastagcollector.in2p3.fr/AMI/servlet/net.hep.atlas.Database.Bookkeeping.AMI.Servlet.Command
  • https://twiki.cern.ch/twiki/bin/viewauth/Atlas/OnlineReconstructionMaintenance#SVN_command
  • https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/RecoTrf
  • https://twiki.cern.ch/twiki/bin/viewauth/Atlas/OnlineReconstruction

Collecting here Algorithms INFO:

Most plots could(should) be adapted in order to be able to apply automatic checks in DQMD. Some need luminosity and/or trigger information in order to provide relevant information

  • DataQuality/DataQualityTools/src/DataQualityFatherMonTool.cxx ---> The father of all tools, all tools inherit from this one

  • DataQuality/DataQualityTools/src/DQTGlobalWZFinderTool.cxx ---> use electrons and muons to reconstruct the Z and W. show Z and W rate. Need luminosity and trigger information to make these plots useful

  • DataQuality/DataQualityTools/src/DQTBackgroundMon.cxx ---> Timing ...

  • DataQuality/DataQualityTools/src/DQTLumiMonTool.cxx ---> Number of vertices and pixel clusters per event. Should use instantaneous luminosity info.

  • DataQuality/DataQualityTools/src/DQTMuonIDTrackTool.cxx --> reconstruct Zmm Jpsi Ypsilon q/p eta separately for ID and MS, quantify agreement.

  • DataQuality/DataQualityTools/src/DQTCaloClusterTool.cxx ---> Calo clusters offline vs L1

  • DataQuality/DataQualityTools/src/DQTDetSynchMonTool.cxx ---> BCID L1ID sincronization within subsystems. B field vs lumiblock. --> should be automatized

  • DataQuality/DataQualityTools/src/DQTMuTrkEff.cxx ---> tag&probe Zmm

  • DataQuality/DataQualityTools/src/DQTDataFlowMonTool.cxx --->

  • DataQuality/DataQualityTools/src/DQTNonCollBkg_ZDC.cxx ---> ?

Tools that have been removed:

  • DataQuality/DataQualityTools/src/DQTElectronQualityTool.cxx --> REMOVED. Electron quality should be monitored by the CP groups via DQMD

  • DataQuality/DataQualityTools/src/DQTTopLeptonJetsFinderTool.cxx ---> reconstruct semileptonic ttbar: 3 jets mass, 1lep1jetmet. REMOVED. Too high level for now.

  • DataQuality/DataQualityTools/src/DQTGlobalTopDilFinderTool.cxx ---> reconstruct dileptonic ttbar. REMOVED. Too high level for now.

  • DataQuality/DataQualityTools/src/DQTRateMonTool.cxx ---> object rate e mu jet vs each other and offline vs online. REMOVED. Does not make sense until we have the trigger information.

  • DataQuality/DataQualityTools/src/DQTGlobalWFinderTool.cxx ---> REMOVED. Obsolete
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r8 - 2014-09-16 - EstelPerez
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback