Active notes for working on CTIDE.

Development sequence

  • Make sparse checkout of Athena packages that I want to change.
  • Make relevant changes. Ensure they compile
  • Make sure I can see my changes in local tests
  • Make MR back into Athena for 21.0 (?) What do I need to do this?

Ultimate goal:

  • Timing information which can be easily turned on or off from config
  • Easy development system for testing stuff

Athena development log

Sparse checkout InDetRecTools, InDetExample and compile. Make my own branch so I can merge my developments back later. My development location is: /afs/

Process based on instructions in ATLAS tutorial. As of Oct 8, best release is 21.0.82.

lsetup git
# Currently using my own fork.
git atlas init-workdir ssh://
cd athena

# Sparse checkout
git atlas addpkg InDetExample
git atlas addpkg SiClusterizationTool

# Get ready to develop my code. Make sure I use an up to date branch!
git fetch upstream
# Template:
# git checkout -b master-my-topic upstream/[parent_branch] --no-track
# My command:
git checkout -b 21.0-add-CTIDE-timing-diagnostics upstream/21.0 --no-track

# To check what releases are available:
asetup --listreleases
# Compile with correct release
mkdir ../build && cd ../build
asetup 21.0.[my_release],Athena

# If I used a sparse checkout I shouldn't need to muck about with the
# package_filters. It should just only compile the packages I checked out.
cmake ../athena/Projects/WorkDir
make -j

# Now test it.
mkdir ../run && cd ../run
source ../build/x86_64-slc6-gcc49-opt/ --AMI q431

Perform tests


import os

ref =   """ --AMITag=r8618 --autoConfiguration=everything --conditionsTag "default:OFLCOND-MC16-SDR-10" --digiSteeringConf="StandardSignalOnlyTruth" --geometryVersion="default:ATLAS-R2-2016-00-01-00" --numberOfCavernBkg=0 --preInclude="" --postExec "all:CfgMgr.MessageSvc().setError+=[\\"HepMcParticleLink\\"]" "HITtoRDO:streamRDO.ItemList+=[\\"SiHitCollection#*\\",]" "RAWtoESD:StreamESD.ItemList+=[\\"InDetSimDataCollection#*\\",\\"SiHitCollection#*\\"]" "ESDtoAOD:fixedAttrib=[s if \\"CONTAINER_SPLITLEVEL = '99'\\" not in s else \\"\\" for s in svcMgr.AthenaPoolCnvSvc.PoolAttributes];svcMgr.AthenaPoolCnvSvc.PoolAttributes=fixedAttrib" "ESDtoDPD:xAOD_PixelPrepDataToxAOD.UseTruthInfo=True" --postInclude "default:PyJobTransforms/" --preExec "all:rec.Commissioning.set_Value_and_Lock(True);from AthenaCommon.BeamFlags import jobproperties;jobproperties.Beam.numberOfCollisions.set_Value_and_Lock(20.0);from LArROD.LArRODFlags import larRODFlags;larRODFlags.NumberOfCollisions.set_Value_and_Lock(20);larRODFlags.nSamples.set_Value_and_Lock(4);larRODFlags.doOFCPileupOptimization.set_Value_and_Lock(True);larRODFlags.firstSample.set_Value_and_Lock(0);larRODFlags.useHighestGainAutoCorr.set_Value_and_Lock(True); from InDetRecExample.InDetJobProperties import InDetFlags; InDetFlags.pixelClusterSplitProb1.set_Value_and_Lock( 0.6 ); InDetFlags.pixelClusterSplitProb2.set_Value_and_Lock( 0.2 ); InDetFlags.doTIDE_RescalePixelCovariances.set_Value_and_Lock(False);InDetFlags.writeRDOs.set_Value_and_Lock(True)" "ESDtoAOD:TriggerFlags.AODEDMSet=\\"AODFULL\\"" "all:from egammaRec.egammaRecFlags import jobproperties;jobproperties.egammaRecFlags.doSuperclusters.set_Value_and_Lock(False)" "HITtoRDO:from Digitization.DigitizationFlags import digitizationFlags;digitizationFlags.doBichselSimulation.set_Value_and_Lock(True)" --runNumber=361027  --inputHITSFile=/afs/ --outputDAOD_IDTIDEFile=OUT.pool.root --maxEvents=2"""

os.system( ref)



from IOVDbSvc.CondDB import conddb;
# Updated NN folder
#conddb.addFolder("","<dbConnection>sqlite://;schema=newpixelNNdb_v2.db;dbname=OFLP200</dbConnection> /PIXEL/PixelClustering/PixelClusNNCalib <tag>PixClusNNCalib-00-00-42</tag>", force=True);
conddb.addFolder("","<dbConnection>COOLOFL_PIXEL/OFLP200</dbConnection> /PIXEL/PixelClustering/PixelClusNNCalib <tag>PixelClusNNCalib-SIM-RUN12-000-04</tag>", force=True);

# conddb.blockFolder("/PIXEL/PixelClustering/PixelCovCorr")
# conddb.addFolder("","<dbConnection>sqlite://;schema=newpixelErrRescale.db;dbname=OFLP200</dbConnection> /PIXEL/PixelClustering/PixelCovCorr <tag>PixelCovCorr-SIM-RUN12-000-01</tag>", force=True);

To run any ATHENA job

Let's say I want to test a specific job options found somewhere in the InDetRec package. How?

With my sparse checkout, ../athena/InnerDetector/InDetExample/InDetRecExample/share/

If I have a local jobOptions which calls the InDetRecExample job options, it can be set up quite simply and I can just run on that. Thus, it suffices to call 

where just contains the lines:

# Last step: load common job options
include ("InDetRecExample/")

Flags for controlling input

Example here for other items which can be read from a provided file:

Example here of instructions Sourav was given once he had a local DB set up: \
 --AMI q221 \
 --preInclude="" \
 --inputRDOFile /eos/user/s/sosen/RDO_CTIDE_timing/user.rjansky.mc16_13TeV.427080.Pythia8EvtGen_A14NNPDF23LO_flatpT_Zprime.RDO_20180110_EXT0/user.rjansky.12944101.EXT0._005040.RDO.root \
 --maxEvents 10 \
 --preExec='from PerfMonComps.PerfMonFlags import jobproperties as pmjp;pmjp.PerfMonFlags.doPostProcessing=True;pmjp.PerfMonFlags.doSemiDetailedMonitoringFullPrint=True' \
 --outputAODFile myAOD.root 2>&1 | tee _log_localtestWithPerfMonSD_q221.txt;

where contents of preInclude file are:

from IOVDbSvc.CondDB import conddb;
# Updated NN folder
conddb.addFolder("","<dbConnection>sqlite://;schema=newpixelNNdb.db;dbname=OFLP200</dbConnection> /PIXEL/PixelClustering/PixelClusNNCalib <tag>PixClusNNCalib-SuperS10122018</tag>", force=True);

Assess test results

Input files

We want to automatically reproduce CTIDE validation plots, which means running on appropriate inputs. For CTIDE performance note, we had:

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2018-10-23 - KatherinePachal
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback