Difference: WhbDatasetTags (1 vs. 23)

Revision 232013-04-09 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 8 to 8
 

whb018

Deleted:
<
<
WORK IN PROGRESS...
 The tag used used 17.0.6.4.6,AtlasPhysics (similar to AMI tag p937). A patch was made to at the WZ truth jet collection.

To recreate the configuration used, download whb018-patch.tar.gz and

pkgco.py TopInputsD3PDMaker-00-01-57

Changed:
<
<
PhysicsAnalysis/D3PDMaker/TopD3PDMaker/TopInputsD3PDMaker
>
>
cd PhysicsAnalysis/D3PDMaker/TopD3PDMaker/TopInputsD3PDMaker # move whb018-patch.tar.gz into this directory
 tar xvfz whb018-patch.tar.gz
Added:
>
>
patch share/PrepareTopInputs.py PrepareTopInputs_py.diff
 patch python/topInputsD3PD.py topInputsD3PD_py.diff
Line: 367 to 367
 
META FILEATTACHMENT attachment="whb002-patch.tar.gz" attr="" comment="whb002 patch files" date="1331804068" name="whb002-patch.tar.gz" path="whb002-patch.tar.gz" size="1243" stream="whb002-patch.tar.gz" tmpFilename="/usr/tmp/CGItemp60672" user="wbell" version="1"
META FILEATTACHMENT attachment="whb003-patch.tar.gz" attr="" comment="whb003 patch files" date="1331804777" name="whb003-patch.tar.gz" path="whb003-patch.tar.gz" size="1267" stream="whb003-patch.tar.gz" tmpFilename="/usr/tmp/CGItemp60688" user="wbell" version="1"
META FILEATTACHMENT attachment="whb007-patch.tar.gz" attr="" comment="whb007 patch files" date="1333443809" name="whb007-patch.tar.gz" path="whb007-patch.tar.gz" size="709" stream="whb007-patch.tar.gz" tmpFilename="/usr/tmp/CGItemp17953" user="wbell" version="1"
Added:
>
>
META FILEATTACHMENT attachment="whb018-patch.tar.gz" attr="" comment="whb018 patch files" date="1365498959" name="whb018-patch.tar.gz" path="whb018-patch.tar.gz" size="1022" user="wbell" version="1"

Revision 222013-02-11 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 
Added:
>
>

whb018

WORK IN PROGRESS...

The tag used used 17.0.6.4.6,AtlasPhysics (similar to AMI tag p937). A patch was made to at the WZ truth jet collection.

To recreate the configuration used, download whb018-patch.tar.gz and

pkgco.py TopInputsD3PDMaker-00-01-57
PhysicsAnalysis/D3PDMaker/TopD3PDMaker/TopInputsD3PDMaker
tar xvfz whb018-patch.tar.gz
patch python/topInputsD3PD.py topInputsD3PD_py.diff 

The D3PDs were produced with,

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb018
for in_ds in $(<input-ds.txt); do 
  out_ds="user.wbell."${in_ds/\/}"_"$tagversion
  out_ds=${out_ds/AOD/NTUP_TOP}
  pathena --trf "Reco_trf.py --ignoreerrors=True preExec='InDetFlags.doV0Finder.set_Value_and_Lock(True);from D3PDMakerConfig.D3PDMakerFlags import D3PDMakerFlags;D3PDMakerFlags.TruthWriteHadrons.set_Value_and_Lock(True)' autoConfiguration=everything maxEvents=-1 inputAODFile=%IN outputNTUP_TOPFile=%OUT.NTUP_TOP.root DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --long --nGBPerJob=3
done
<!-- end SyntaxHighlightingPlugin -->
 

whb017

The tag used 17.2.2.1.2,AtlasPhysics (similar to AMI tag p1032) with an empty TestArea and

Revision 212012-12-14 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 8 to 8
 

whb017

Changed:
<
<
The tag used 17.2.2.1.2,AtlasPhysics (similar to AMI tag p1032) with an empty TestArea and
>
>
The tag used 17.2.2.1.2,AtlasPhysics (similar to AMI tag p1032) with an empty TestArea and
  %SYNTAX{syntax="sh"}% #!/bin/bash
Changed:
<
<
tagversion=whb016
>
>
tagversion=whb017
 dataset_num=0001 for in_ds in $(<input-ds.txt); do out_ds="user.wbell."${in_ds/\/}"_"$tagversion
Line: 21 to 21
 done %ENDSYNTAX%
Added:
>
>
This transformation configuration was checked against whb016, by running jobs locally and found to be the same. In the whb017 case, TruthD3PDMaker/TruthD3PDfromEVGEN_preInclude.py is included by the transform automatically.
 

whb016

The tag used 17.2.2.1.2,AtlasPhysics (similar to AMI tag p1032) with an empty TestArea and

Revision 202012-12-13 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 
Added:
>
>

whb017

The tag used 17.2.2.1.2,AtlasPhysics (similar to AMI tag p1032) with an empty TestArea and

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb016
dataset_num=0001
for in_ds in $(<input-ds.txt); do 
  out_ds="user.wbell."${in_ds/\/}"_"$tagversion
  out_ds=${out_ds/evgen.EVNT/merge.NTUP_TRUTH}
  pathena --trf "Reco_trf.py --ignoreerrors=True preExec='from D3PDMakerConfig.D3PDMakerFlags import D3PDMakerFlags;D3PDMakerFlags.TruthWriteHadrons.set_Value_and_Lock(True)' autoConfiguration=everything maxEvents=-1 inputEVNTFile=%IN outputNTUP_TRUTHFile=%OUT.NTUP_TRUTH.root DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --noBuild --long --nGBPerJob=3 --nFilesPerJob=10
done
<!-- end SyntaxHighlightingPlugin -->
 

whb016

The tag used 17.2.2.1.2,AtlasPhysics (similar to AMI tag p1032) with an empty TestArea and

Line: 21 to 36
 done %ENDSYNTAX%
Deleted:
<
<
WARNING: this configuration causes the neutrinos to be clustered within the particle jets.
 

whb015

The tag used 17.0.5.6,AtlasProduction with an empty test area and the mc11c configuration,

Revision 192012-11-22 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 15 to 15
 tagversion=whb016 dataset_num=0001 for in_ds in $(<input-ds.txt); do
Deleted:
<
<
#out_ds="user.wbell."${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds="user.wbell."${in_ds/\/}"_"$tagversion
Deleted:
<
<
#out_ds=${in_ds/\/}"_"$tagversion
  out_ds=${out_ds/evgen.EVNT/merge.NTUP_TRUTH} pathena --trf "Reco_trf.py --ignoreerrors=True preExec='from D3PDMakerFlags import D3PDMakerFlags;D3PDMakerFlags.TruthWriteHadrons.set_Value_and_Lock(True)' preInclude='TruthD3PDMaker/TruthD3PDfromEVGEN_preInclude.py' autoConfiguration=everything maxEvents=-1 inputEVNTFile=%IN outputNTUP_TRUTHFile=%OUT.NTUP_TRUTH.root DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --noBuild --long --nGBPerJob=3 --nFilesPerJob=10 done %ENDSYNTAX%
Added:
>
>
WARNING: this configuration causes the neutrinos to be clustered within the particle jets.
 

whb015

The tag used 17.0.5.6,AtlasProduction with an empty test area and the mc11c configuration,

Revision 182012-11-19 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 
Added:
>
>

whb016

The tag used 17.2.2.1.2,AtlasPhysics (similar to AMI tag p1032) with an empty TestArea and

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb016
dataset_num=0001
for in_ds in $(<input-ds.txt); do 
  #out_ds="user.wbell."${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds="user.wbell."${in_ds/\/}"_"$tagversion
  #out_ds=${in_ds/\/}"_"$tagversion
  out_ds=${out_ds/evgen.EVNT/merge.NTUP_TRUTH}
  pathena --trf "Reco_trf.py --ignoreerrors=True preExec='from D3PDMakerConfig.D3PDMakerFlags import D3PDMakerFlags;D3PDMakerFlags.TruthWriteHadrons.set_Value_and_Lock(True)' preInclude='TruthD3PDMaker/TruthD3PDfromEVGEN_preInclude.py' autoConfiguration=everything maxEvents=-1 inputEVNTFile=%IN outputNTUP_TRUTHFile=%OUT.NTUP_TRUTH.root DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --noBuild --long --nGBPerJob=3 --nFilesPerJob=10
done
<!-- end SyntaxHighlightingPlugin -->
 

whb015

The tag used 17.0.5.6,AtlasProduction with an empty test area and the mc11c configuration,

Revision 172012-05-30 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 16 to 16
 dataset_num=0001 for in_ds in $(<input-ds.txt); do out_ds=${in_ds/\/}"_"$tagversion
Changed:
<
<
out_ds=${out_ds/simul.HITS/reco.ESD}
>
>
out_ds=${out_ds/simul.HITS/reco.AOD}
  run_number=$(echo $in_ds | tr '.' '\n' | grep -vP "[a-zA-Z]+") # taken from https://twiki.cern.ch/twiki/bin/view/Main/MC11PrivateProduction
Changed:
<
<
pathena --useShortLivedReplicas --noBuild --trf "DigiMReco_trf.py inputHitsFile=%IN outputESDFile=tmpESD.pool.root outputAODFile=%OUT.AOD.pool.root maxEvents=1000 triggerConfigByRun='{180164:\"MCRECO:DB:TRIGGERDBMC:308,134,246\",183003:\"MCRECO:DB:TRIGGERDBMC:308,134,246\",186169:\"MCRECO:DB:TRIGGERDBMC:308,134,246\",189751:\"MCRECO:DB:TRIGGERDBMC:310,139,252\"}' preInclude_h2r='Digitization/ForceUseOfPileUpTools.py,SimulationJobOptions/preInclude.PileUpBunchTrains2011Config8_DigitConfig.py,RunDependentSimData/configLumi_mc11b.py' preExec_h2r='from DigitizationFlags import digitizationFlags;digitizationFlags.overrideMetadata+=[\"SimLayout\",\"PhysicsList\"]' postExec_h2r='ToolSvc.LArAutoCorrTotalToolDefault.NMinBias=0' preExec_r2e='rec.Commissioning.set_Value_and_Lock(True);jobproperties.Beam.energy.set_Value_and_Lock(3500*Units.GeV);muonRecFlags.writeSDOs=True;jobproperties.Beam.numberOfCollisions.set_Value_and_Lock(8.0);jobproperties.Beam.bunchSpacing.set_Value_and_Lock(50)' preExec_e2a='TriggerFlags.AODEDMSet=\"AODSLIM\";rec.Commissioning.set_Value_and_Lock(True);jobproperties.Beam.energy.set_Value_and_Lock(3500*Units.GeV);muonRecFlags.writeSDOs=True;jobproperties.Beam.numberOfCollisions.set_Value_and_Lock(8.0);jobproperties.Beam.bunchSpacing.set_Value_and_Lock(50)' RunNumber=$run_number autoConfiguration=everything DBRelease=%DB:LATEST conditionsTag=OFLCOND-SDR-BS7T-05-24 outputRDOFile=tmpRDO.pool.root geometryVersion=ATLAS-GEO-18-01-01 numberOfLowPtMinBias=39.954664 numberOfHighPtMinBias=0.045336 LowPtMinbiasHitsFile=%LOMBIN DataRunNumber=-1 jobNumber=%RNDM:100 digiSeedOffset1=%RNDM:100 digiSeedOffset2=%RNDM:100 HighPtMinbiasHitsFile=%HIMBIN" --lowMinDS mc11_7TeV.119906.pythia_minbias_low.merge.HITS.e977_s1354_s1369/ --nLowMinPerJob 5 --highMinDS mc11_7TeV.119907.pythia_minbias_high.merge.HITS.e977_s1354_s1369/ --nHighMinPerJob 5 --inDS $in_ds --outDS $out_ds --dbRelease LATEST --nEventsPerJob 1000 --nEventsPerFile 50 --long
>
>
pathena --useShortLivedReplicas --noBuild --trf "DigiMReco_trf.py inputHitsFile=%IN outputESDFile=tmpESD.pool.root outputAODFile=%OUT.AOD.pool.root maxEvents=1000 triggerConfigByRun='{180164:\"MCRECO:DB:TRIGGERDBMC:308,134,246\",183003:\"MCRECO:DB:TRIGGERDBMC:308,134,246\",186169:\"MCRECO:DB:TRIGGERDBMC:308,134,246\",189751:\"MCRECO:DB:TRIGGERDBMC:310,139,252\"}' preInclude_h2r='Digitization/ForceUseOfPileUpTools.py,SimulationJobOptions/preInclude.PileUpBunchTrains2011Config8_DigitConfig.py,RunDependentSimData/configLumi_mc11b.py' preExec_h2r='from DigitizationFlags import digitizationFlags;digitizationFlags.overrideMetadata+=[\"SimLayout\",\"PhysicsList\"]' postExec_h2r='ToolSvc.LArAutoCorrTotalToolDefault.NMinBias=0' preExec_r2e='rec.Commissioning.set_Value_and_Lock(True);jobproperties.Beam.energy.set_Value_and_Lock(3500*Units.GeV);muonRecFlags.writeSDOs=True;jobproperties.Beam.numberOfCollisions.set_Value_and_Lock(8.0);jobproperties.Beam.bunchSpacing.set_Value_and_Lock(50)' preExec_e2a='TriggerFlags.AODEDMSet=\"AODSLIM\";rec.Commissioning.set_Value_and_Lock(True);jobproperties.Beam.energy.set_Value_and_Lock(3500*Units.GeV);muonRecFlags.writeSDOs=True;jobproperties.Beam.numberOfCollisions.set_Value_and_Lock(8.0);jobproperties.Beam.bunchSpacing.set_Value_and_Lock(50)' RunNumber=$run_number autoConfiguration=everything DBRelease=%DB:LATEST conditionsTag=OFLCOND-SDR-BS7T-05-24 outputRDOFile=tmpRDO.pool.root geometryVersion=ATLAS-GEO-18-01-01 numberOfLowPtMinBias=39.954664 numberOfHighPtMinBias=0.045336 LowPtMinbiasHitsFile=%LOMBIN DataRunNumber=-1 jobNumber=%RNDM:100 digiSeedOffset1=%RNDM:100 digiSeedOffset2=%RNDM:100 HighPtMinbiasHitsFile=%HIMBIN" --lowMinDS mc11_7TeV.119906.pythia_minbias_low.merge.HITS.e977_s1354_s1369/ --nLowMinPerJob 5 --highMinDS mc11_7TeV.119907.pythia_minbias_high.merge.HITS.e977_s1354_s1369/ --nHighMinPerJob 5 --inDS $in_ds --outDS $out_ds --dbRelease LATEST --nFilesPerJob 10 --long
 done %ENDSYNTAX%
Changed:
<
<
to digitize and reconstruct the events produced with the whb014 tag. The TestArea was empty. The output data set name is somewhat confusing, since it contains AODs rather than ESDs. It would be better to use out_ds=${out_ds/simul.HITS/reco.AOD}.
>
>
to digitize and reconstruct the events produced with the whb014 tag. The TestArea was empty.
 

whb014

Revision 162012-05-29 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 
Added:
>
>

whb015

The tag used 17.0.5.6,AtlasProduction with an empty test area and the mc11c configuration,

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb015
dataset_num=0001
for in_ds in $(<input-ds.txt); do
  out_ds=${in_ds/\/}"_"$tagversion
  out_ds=${out_ds/simul.HITS/reco.ESD}
  run_number=$(echo $in_ds | tr '.' '\n' | grep -vP "[a-zA-Z]+")
  # taken from https://twiki.cern.ch/twiki/bin/view/Main/MC11PrivateProduction
  pathena --useShortLivedReplicas --noBuild --trf "DigiMReco_trf.py inputHitsFile=%IN outputESDFile=tmpESD.pool.root outputAODFile=%OUT.AOD.pool.root maxEvents=1000 triggerConfigByRun='{180164:\"MCRECO:DB:TRIGGERDBMC:308,134,246\",183003:\"MCRECO:DB:TRIGGERDBMC:308,134,246\",186169:\"MCRECO:DB:TRIGGERDBMC:308,134,246\",189751:\"MCRECO:DB:TRIGGERDBMC:310,139,252\"}' preInclude_h2r='Digitization/ForceUseOfPileUpTools.py,SimulationJobOptions/preInclude.PileUpBunchTrains2011Config8_DigitConfig.py,RunDependentSimData/configLumi_mc11b.py' preExec_h2r='from Digitization.DigitizationFlags import digitizationFlags;digitizationFlags.overrideMetadata+=[\"SimLayout\",\"PhysicsList\"]' postExec_h2r='ToolSvc.LArAutoCorrTotalToolDefault.NMinBias=0' preExec_r2e='rec.Commissioning.set_Value_and_Lock(True);jobproperties.Beam.energy.set_Value_and_Lock(3500*Units.GeV);muonRecFlags.writeSDOs=True;jobproperties.Beam.numberOfCollisions.set_Value_and_Lock(8.0);jobproperties.Beam.bunchSpacing.set_Value_and_Lock(50)' preExec_e2a='TriggerFlags.AODEDMSet=\"AODSLIM\";rec.Commissioning.set_Value_and_Lock(True);jobproperties.Beam.energy.set_Value_and_Lock(3500*Units.GeV);muonRecFlags.writeSDOs=True;jobproperties.Beam.numberOfCollisions.set_Value_and_Lock(8.0);jobproperties.Beam.bunchSpacing.set_Value_and_Lock(50)' RunNumber=$run_number autoConfiguration=everything DBRelease=%DB:LATEST conditionsTag=OFLCOND-SDR-BS7T-05-24 outputRDOFile=tmpRDO.pool.root geometryVersion=ATLAS-GEO-18-01-01 numberOfLowPtMinBias=39.954664 numberOfHighPtMinBias=0.045336 LowPtMinbiasHitsFile=%LOMBIN DataRunNumber=-1 jobNumber=%RNDM:100 digiSeedOffset1=%RNDM:100 digiSeedOffset2=%RNDM:100 HighPtMinbiasHitsFile=%HIMBIN" --lowMinDS mc11_7TeV.119906.pythia_minbias_low.merge.HITS.e977_s1354_s1369/ --nLowMinPerJob 5 --highMinDS mc11_7TeV.119907.pythia_minbias_high.merge.HITS.e977_s1354_s1369/ --nHighMinPerJob 5 --inDS $in_ds --outDS $out_ds --dbRelease LATEST --nEventsPerJob 1000 --nEventsPerFile 50 --long
done
<!-- end SyntaxHighlightingPlugin -->

to digitize and reconstruct the events produced with the whb014 tag. The TestArea was empty. The output data set name is somewhat confusing, since it contains AODs rather than ESDs. It would be better to use out_ds=${out_ds/simul.HITS/reco.AOD}.

 

whb014

This tag was used to produce more Pythia u4u4 evgen, matching the e997 configuration (16.6.8.3,AtlasProduction with MC11JobOpts-00-02-27_v0.tar.gz). The jobs were submitted with an empty

Revision 152012-05-25 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 24 to 24
  pathena --noBuild --trf "Generate_trf.py ecmEnergy=7000 firstEvent=%SKIPEVENTS randomSeed=%RNDM:$rndm jobConfig=$opt evgenJobOpts=MC11JobOpts-00-02-27_v0.tar.gz outputEVNTFile=%OUT.EVNT.pool.root runNumber=$run_number" --nEventsPerJob 5000 --split 200 --outDS $out_ds --long rndm=$(($rndm + 200)) # to make sure event job has a unique seed done
Added:
>
>
%ENDSYNTAX%
 
Changed:
<
<
The Generate_trf.py does not allow the maximum number of events to be configured, instead it internally defaults to 5000 events.
>
>
The Generate_trf.py does not allow the maximum number of events to be configured, instead it internally defaults to 5000 events.
 

whb013

Revision 142012-05-20 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 
Added:
>
>

whb014

This tag was used to produce more Pythia u4u4 evgen, matching the e997 configuration (16.6.8.3,AtlasProduction with MC11JobOpts-00-02-27_v0.tar.gz). The jobs were submitted with an empty TestArea and

%SYNTAX{syntax="sh"}% #!/bin/bash tagversion=whb014 dataset_num=0001 rndm=1034 for opt in $(<generator_options.txt); do out_ds=${opt/MC11./} out_ds=${out_ds/.py/} out_ds="user.wbell.mc11_7TeV."$out_ds".evgen.EVNT."$tagversion run_number=$(echo $out_ds | tr '.' '\n' | grep -vP "[a-zA-Z]+") pathena --noBuild --trf "Generate_trf.py ecmEnergy=7000 firstEvent=%SKIPEVENTS randomSeed=%RNDM:$rndm jobConfig=$opt evgenJobOpts=MC11JobOpts-00-02-27_v0.tar.gz outputEVNTFile=%OUT.EVNT.pool.root runNumber=$run_number" --nEventsPerJob 5000 --split 200 --outDS $out_ds --long rndm=$(($rndm + 200)) # to make sure event job has a unique seed done

The Generate_trf.py does not allow the maximum number of events to be configured, instead it internally defaults to 5000 events.

 

whb013

16.6.7.13,AtlasProduction to match s1310, used for the Pythia u4u4 samples. The TestArea was empty.

Revision 132012-05-18 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 
Added:
>
>

whb013

16.6.7.13,AtlasProduction to match s1310, used for the Pythia u4u4 samples. The TestArea was empty.

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb013
dataset_num=0002
for in_ds in $(<input-ds.txt); do 
  out_ds=${in_ds/\/}"_"$tagversion
  out_ds=${out_ds/evgen.EVNT/simul.HITS}
  run_number=$(echo $in_ds | tr '.' '\n' | grep -vP "[a-zA-Z]+")
  # taken from https://twiki.cern.ch/twiki/bin/view/Main/MC11PrivateProduction
  pathena --noBuild --trf "AtlasG4_trf.py inputEvgenFile=%IN outputHitsFile=%OUT.HITS.pool.root maxEvents=50 skipEvents=%SKIPEVENTS AMITag=s1310 DBRelease=%DB:LATEST preInclude=SimulationJobOptions/preInclude.FrozenShowersFCalOnly.py,SimulationJobOptions/preInclude.BeamPipeKill.py geometryVersion=ATLAS-GEO-18-01-03_VALIDATION conditionsTag=OFLCOND-SDR-BS7T-05-12 randomSeed=%RNDM:100 physicsList=QGSP_BERT RunNumber=$run_number DBContent=FSRelease-0.7.1.2.tar.gz" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --nEventsPerJob 50 --nEventsPerFile 5000 --long
done
<!-- end SyntaxHighlightingPlugin -->
 

whb012

This tag was used to produce more Pythia u4u4 evgen, matching the e972 configuration (16.6.8.2,AtlasProduction with MC11JobOpts-00-01-64_v2.tar.gz). The jobs were submitted with an empty TestArea and

Revision 122012-05-18 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 8 to 8
 

whb012

Changed:
<
<
...info soon...
>
>
This tag was used to produce more Pythia u4u4 evgen, matching the e972 configuration (16.6.8.2,AtlasProduction with MC11JobOpts-00-01-64_v2.tar.gz). The jobs were submitted with an empty TestArea and
<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb012
dataset_num=0001
num_evts=5000
for opt in $(<generator_options.txt); do
  out_ds=${opt/MC11./}
  out_ds=${out_ds/.py/}
  out_ds="user.wbell.mc11_7TeV."$out_ds".evgen.EVNT."$tagversion
  run_number=$(echo $out_ds | tr '.' '\n' | grep -vP "[a-zA-Z]+")
  pathena --noBuild --trf "Generate_trf.py ecmEnergy=7000 firstEvent=%SKIPEVENTS randomSeed=%RNDM:100 jobConfig=$opt evgenJobOpts=MC11JobOpts-00-01-64_v2.tar.gz outputEVNTFile=%OUT.EVNT.pool.root runNumber=$run_number" --nEventsPerJob 5000 --split 200 --outDS $out_ds --long
done
<!-- end SyntaxHighlightingPlugin -->
The value of 100 for the random seed is incremented by one for each sub-job and should therefore be changes for each job configuration file. The Generate_trf.py does not allow the maximum number of events to be configured, instead it internally defaults to 5000 events.
 

whb011

Revision 112012-05-18 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 
Added:
>
>

whb012

...info soon...

 

whb011

This tag used 16.6.7.27,AtlasProduction, the same cache used for the beam-gas validation sample. The simulation was run using,

Revision 102012-05-09 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 
Added:
>
>

whb011

This tag used 16.6.7.27,AtlasProduction, the same cache used for the beam-gas validation sample. The simulation was run using,

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb011
dataset_num=0002
num_evts=5100
for in_ds in $(<input-ds.txt); do 
  out_ds=${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds=${out_ds/evgen.EVNT/simul.HITS}
  run_number=$(echo $in_ds | tr '.' '\n' | grep -vP "[a-zA-Z]+")
  # based on https://twiki.cern.ch/twiki/bin/view/Main/MC11PrivateProduction
  pathena --noBuild --trf "AtlasG4_trf.py inputEvgenFile=%IN outputHitsFile=%OUT.HITS.pool.root maxEvents=50 skipEvents=%SKIPEVENTS DBRelease=%DB:ddo.000001.Atlas.Ideal.DBRelease.v16080101:DBRelease-16.8.1.1.tar.gz preInclude=SimulationJobOptions/preInclude.HaloSim.py geometryVersion=ATLAS-GEO-18-01-03_VALIDATION conditionsTag=OFLCOND-SDR-BS7T-05-24 randomSeed=%RNDM:100 physicsList=QGSP_BERT RunNumber=$run_number DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --nEventsPerJob 50 --nEventsPerFile 5100 --long
done
<!-- end SyntaxHighlightingPlugin -->
 

whb010

Changed:
<
<
...information to follow...
>
>
This tag used 16.6.7.24,AtlasProduction. The event generation jobs were run on a local PBS, using commands of the form
<!-- SyntaxHighlightingPlugin -->
Generate_trf.py ecmEnergy=7000 firstEvent=0 randomSeed=123 maxEvents=5100 jobConfig="./"MC11.013059.RBruceBeamHaloB1.py inputGeneratorFile="./"group.phys-gener.beamhalo.013059.RBruceBeamHaloB1_7TeV.TXT.mc11_v1._00001.tar.gz outputEVNTFile=group.phys-gener.beamhalo.013059.RBruceBeamHaloB1.evgen.EVNT._7TeV.TXT.mc11_v1._00001_whb005.pool.root runNumber=013059 &> run.log
<!-- end SyntaxHighlightingPlugin -->
The output files were then renamed and transferred to the GRID.
 

whb009

Revision 92012-05-07 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 
Added:
>
>

whb010

...information to follow...

 

whb009

This tag used 17.2.2.2,AtlasProduction, the first AtlasProduction cache which included BackgroundD3PDMaker-00-03-09. The tag BackgroundD3PDMaker-00-03-09 is required, since it includes the configuration to write the mcevt_weight. The primary particle position four vector is stored in the event weight vector. Beam background validation D3PDs were produced with Pixel cluster information using,

Revision 82012-05-02 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Added:
>
>
This page describes the configuration used for each of the whb private dataset tags
 
Added:
>
>

whb009

This tag used 17.2.2.2,AtlasProduction, the first AtlasProduction cache which included BackgroundD3PDMaker-00-03-09. The tag BackgroundD3PDMaker-00-03-09 is required, since it includes the configuration to write the mcevt_weight. The primary particle position four vector is stored in the event weight vector. Beam background validation D3PDs were produced with Pixel cluster information using,

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb009
dataset_num=0001
for in_ds in $(<input-ds.txt); do 
  out_ds=${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds=${out_ds/reco.ESD/merge.NTUP_BKGD}
  pathena --trf "Reco_trf.py --ignoreerrors=True preExec='from BackgroundD3PDMaker.BackgroundD3PDMakerFlags import BackgroundD3PDMakerFlags; BackgroundD3PDMakerFlags.doPixelClus=True' autoConfiguration=everything maxEvents=-1 inputESDFile=%IN outputNTUP_BKGDFile=%OUT.NTUP_BKGD.root DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --long --nFilesPerJob 5
done
<!-- end SyntaxHighlightingPlugin -->

The no build option could have been used, since there were no additional tags in the test area. The number of events in the input ESD, produced with whb008, were irregular due to the G4HitFilter. Therefore, the number of files was required rather than specify the number of events in each file. While the number of files per job was required to be five for this job, the optimal number is probably of the order of 20. The CPU requirements and output file size were small.

whb008

This tag used 17.0.6.9,AtlasProduction, recommended as a validated release for digitization and reconstruction. Beam background validation ESDs were produced with inner detector RDO information,

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb008
dataset_num=0001
for in_ds in $(<input-ds.txt); do 
  out_ds=${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds="user.wbell."${out_ds/simul.HITS/reco.ESD}
  run_number=$(echo $in_ds | tr '.' '\n' | grep -vP "[a-zA-Z]+")
  # adapted from https://twiki.cern.ch/twiki/bin/view/Main/MC11PrivateProduction
  pathena --noBuild --trf "DigiMReco_trf.py preExec='from InDetRecExample.InDetJobProperties import InDetFlags; InDetFlags.writeRDOs.set_Value_and_Lock(True)' inputHitsFile=%IN outputESDFile=%OUT.ESD.pool.root maxEvents=1000 RunNumber=$run_number autoConfiguration=everything DBRelease=%DB:LATEST conditionsTag=OFLCOND-SDR-BS7T-05-24 outputRDOFile=tmpRDO.pool.root geometryVersion=ATLAS-GEO-18-01-03 DataRunNumber=-1 jobNumber=%RNDM:100 digiSeedOffset1=%RNDM:100 digiSeedOffset2=%RNDM:100 doAllNoise=False" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --nFilesPerJob 10 --long
done
<!-- end SyntaxHighlightingPlugin -->

Following the G4HitFilter, the number of events in each HITS file is not uniform. Therefore, it is necessary to require a number of files per job, rather than specify the number of events in each file.

 

whb007

The tag was based on 17.0.5.6.1,AtlasPhysics (p834). There were two patches to correct the generator record (whb003) and add additional branches. Additional branches were added for pileup and vertex information.

Revision 72012-04-03 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 6 to 6
 

whb007

Changed:
<
<
More information soon...
>
>
The tag was based on 17.0.5.6.1,AtlasPhysics (p834). There were two patches to correct the generator record (whb003) and add additional branches. Additional branches were added for pileup and vertex information.

To recreate the patch, download whb003-patch.tar.gz and whb007-patch.tar.gz and

pkgco.py TopInputsD3PDMaker-00-01-46
PhysicsAnalysis/D3PDMaker/TopD3PDMaker/TopInputsD3PDMaker
tar xvfz whb007-patch.tar.gz
patch python/topInputsD3PD.py topInputsD3PD_py.diff 
cd $TestArea
pkgco.py TruthD3PDAnalysis-00-00-15
cd PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis
tar xvfz whb003-patch.tar.gz
patch src/TruthParticleFilterTool.cxx TruthParticleFilterTool_cxx.diff
patch src/TruthParticleFilterTool.h TruthParticleFilterTool_h.diff

The D3PDs were produced with,

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb007
dataset_num=0003
for in_ds in $(<input-ds.txt); do 
  out_ds="user.wbell."${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds=${out_ds/digit.RDO/NTUP_TOP}
  out_ds=${out_ds/group.perf-idtracking./}
  pathena --trf "Reco_trf.py --ignoreerrors=True preExec='InDetFlags.doV0Finder.set_Value_and_Lock(True); from D3PDMakerConfig.D3PDMakerFlags import D3PDMakerFlags; D3PDMakerFlags.TruthWriteHadrons.set_Value_and_Lock(True); D3PDMakerFlags.TruthDoPileup.set_Value_and_Lock(True)' autoConfiguration=everything maxEvents=-1 inputAODFile=%IN outputNTUP_TOPFile=%OUT.NTUP_TOP.root DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --long --nFilesPerJob 1
done
<!-- end SyntaxHighlightingPlugin -->
where nFilesPerJob was used to force the transform to produce one file per input file. The reason for this is that D3PD making of these pileup D3PDs is very slow. (There are several processing steps before the TTree write command, such as the truth filter, re-building jet collections etc..)
 

whb006

Changed:
<
<
More information soon...
>
>
The evgen files produced using the whb005 setup were passed through the Geant4 simulation using 17.1.4.2,AtlasProduction and
<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb006
dataset_num=0001_test2
for in_ds in $(<input-ds.txt); do 
  out_ds=${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds=${out_ds/evgen.EVNT/simul.HITS}
  run_number=$(echo $in_ds | tr '.' '\n' | grep -vP "[a-zA-Z]+")
  if [[ $run_number == "013059" ]]; then
    num_evts=23188
  fi
  # taken from https://twiki.cern.ch/twiki/bin/view/Main/MC11PrivateProduction
  pathena --noBuild --trf "AtlasG4_trf.py inputEvgenFile=%IN outputHitsFile=%OUT.HITS.pool.root maxEvents=50 skipEvents=%SKIPEVENTS DBRelease=%DB:ddo.000001.Atlas.Ideal.DBRelease.v16080101:DBRelease-16.8.1.1.tar.gz preInclude=SimulationJobOptions/preInclude.HaloSim.py geometryVersion=ATLAS-GEO-18-01-03_VALIDATION conditionsTag=OFLCOND-SDR-BS7T-05-24 randomSeed=%RNDM:100 physicsList=QGSP_BERT RunNumber=$run_number DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --nEventsPerJob 50 --nEventsPerFile $num_evts --long
done
<!-- end SyntaxHighlightingPlugin -->

It is not forseen to use 17.1.4.2,AtlasProduction for the beam halo simulation, but rather 16.6.7.24,AtlasProduction should be used for beam halo evgen and simulation. 16.6.7.24,AtlasProduction is the recommended setup for mc11c evgen and simulation and contains the changes needed for beam halo production. This cache was not available when the whb006 or whb005 files were produced.

 

whb005

Line: 125 to 170
 
META FILEATTACHMENT attachment="whb002-patch.tar.gz" attr="" comment="whb002 patch files" date="1331804068" name="whb002-patch.tar.gz" path="whb002-patch.tar.gz" size="1243" stream="whb002-patch.tar.gz" tmpFilename="/usr/tmp/CGItemp60672" user="wbell" version="1"
META FILEATTACHMENT attachment="whb003-patch.tar.gz" attr="" comment="whb003 patch files" date="1331804777" name="whb003-patch.tar.gz" path="whb003-patch.tar.gz" size="1267" stream="whb003-patch.tar.gz" tmpFilename="/usr/tmp/CGItemp60688" user="wbell" version="1"
Added:
>
>
META FILEATTACHMENT attachment="whb007-patch.tar.gz" attr="" comment="whb007 patch files" date="1333443809" name="whb007-patch.tar.gz" path="whb007-patch.tar.gz" size="709" stream="whb007-patch.tar.gz" tmpFilename="/usr/tmp/CGItemp17953" user="wbell" version="1"

Revision 62012-04-03 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Added:
>
>

whb007

More information soon...

whb006

More information soon...

 

whb005

The tag was based on 16.6.OLD.Y,rel_4,AtlasProduction (2012-03-14) with BeamHaloGenerator-00-00-31, which includes the implementation of the callGenerator function. The job configuration files found in MC11JobOptions-00-03-80 (MC11.013059.RBruceBeamHaloB1.py MC11.013060.RBruceBeamHaloB2.py and MC11.013061.RBruceBeamGasB1.py) were used to produce evgen from the input data sets,

Revision 52012-04-02 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 23 to 23
 

whb004

Changed:
<
<
...info soon...
>
>
The tag was based on 17.0.5.6.1,AtlasPhysics (p834) with the fix to used for whb003 for PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis to fix the bug, https://savannah.cern.ch/bugs/?91063

To recreate the patch, download whb003-patch.tar.gz and

pkgco.py TruthD3PDAnalysis-00-00-15
cd PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis
tar xvfz whb003-patch.tar.gz
patch src/TruthParticleFilterTool.cxx TruthParticleFilterTool_cxx.diff
patch src/TruthParticleFilterTool.h TruthParticleFilterTool_h.diff

The D3PDs were produced with,

<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb004
dataset_num=0001
for in_ds in $(<input-ds.txt); do 
  out_ds="user.wbell."${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds=${out_ds/AOD/NTUP_TOP}
  pathena --trf "Reco_trf.py --ignoreerrors=True preExec='InDetFlags.doV0Finder.set_Value_and_Lock(True);from D3PDMakerConfig.D3PDMakerFlags import D3PDMakerFlags;D3PDMakerFlags.TruthWriteHadrons.set_Value_and_Lock(True)' autoConfiguration=everything maxEvents=-1 inputAODFile=%IN outputNTUP_TOPFile=%OUT.NTUP_TOP.root DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --noBuild --long --nGBPerJob=3 
done
<!-- end SyntaxHighlightingPlugin -->
 

whb003

Revision 42012-03-19 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Added:
>
>

whb005

The tag was based on 16.6.OLD.Y,rel_4,AtlasProduction (2012-03-14) with BeamHaloGenerator-00-00-31, which includes the implementation of the callGenerator function. The job configuration files found in MC11JobOptions-00-03-80 (MC11.013059.RBruceBeamHaloB1.py MC11.013060.RBruceBeamHaloB2.py and MC11.013061.RBruceBeamGasB1.py) were used to produce evgen from the input data sets,

group.phys-gener.beamhalo.013059.RBruceBeamHaloB1_7TeV.TXT.mc11_v1/
group.phys-gener.beamhalo.013060.RBruceBeamHaloB2_7TeV.TXT.mc11_v1/
group.phys-gener.beamhalo.013061.RBruceBeamGasB1_7TeV.TXT.mc11_v1/
and commands of the form,
<!-- SyntaxHighlightingPlugin -->
Generate_trf.py ecmEnergy=7000 firstEvent=0 randomSeed=123 maxEvents=-1 \
jobConfig="./"$config_file \
inputGeneratorFile="./"$input_file \
outputEVNTFile=$output_file runNumber=$run_number &> run.log
<!-- end SyntaxHighlightingPlugin -->
The jobs finished with errors when the input file eof was reached, but the resulting evgen files were completely correctly. (Production jobs require the number of events to be hardcoded. The input files have to be cut to 5100 events after the filter for this to work.)

whb004

...info soon...

 

whb003

The tag was based on 17.0.5.6.1,AtlasPhysics (p834) with a fix to PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis to fix the bug,

Revision 32012-03-15 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 20 to 20
 patch src/TruthParticleFilterTool.h TruthParticleFilterTool_h.diff
Added:
>
>
D3PDs were then made with
<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb003
dataset_num=0001
for in_ds in $(<input-ds.txt); do 
  out_ds="user.wbell."${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds=${out_ds/AOD/NTUP_TOP}
  pathena --trf "Reco_trf.py --ignoreerrors=True preExec='InDetFlags.doV0Finder.set_Value_and_Lock(True)' autoConfiguration=everything maxEvents=-1 inputAODFile=%IN outputNTUP_TOPFile=%OUT.NTUP_TOP.root DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --noBuild --long --nGBPerJob=3 
done
<!-- end SyntaxHighlightingPlugin -->
 

whb002

Line: 37 to 48
 patch src/TruthParticleFilterTool.h TruthParticleFilterTool_h.diff
Added:
>
>
D3PDs were then made with
<!-- SyntaxHighlightingPlugin -->
#!/bin/bash
tagversion=whb002
dataset_num=0001
for in_ds in $(<input-ds.txt); do 
  out_ds="user.wbell."${in_ds/\/}"_"$tagversion"_"$dataset_num
  out_ds=${out_ds/AOD/NTUP_TOP}
  pathena --trf "Reco_trf.py --ignoreerrors=True preExec='InDetFlags.doV0Finder.set_Value_and_Lock(True)' autoConfiguration=everything maxEvents=-1 inputAODFile=%IN outputNTUP_TOPFile=%OUT.NTUP_TOP.root DBRelease=%DB:LATEST" --inDS $in_ds --outDS $out_ds --dbRelease LATEST --noBuild --long --nGBPerJob=3 
done
<!-- end SyntaxHighlightingPlugin -->
 

whb001

A small number of data sets were run using this tag. The tag was based on 17.0.5.6.1,AtlasPhysics (p834) with a partial fix to PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis to patch a bug,

Revision 22012-03-15 - WilliamBell

Line: 1 to 1
 
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

Line: 11 to 11
  This patch was successful on SHERPA samples too.
Changed:
<
<
To recreate the patch,
>
>
To recreate the patch, download whb003-patch.tar.gz and
 

Changed:
<
<
>
>
pkgco.py TruthD3PDAnalysis-00-00-15 cd PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis tar xvfz whb003-patch.tar.gz patch src/TruthParticleFilterTool.cxx TruthParticleFilterTool_cxx.diff patch src/TruthParticleFilterTool.h TruthParticleFilterTool_h.diff
 
Line: 24 to 28
  The fix was successful on all signal samples except SHERPA.
Changed:
<
<
To recreate the patch,
>
>
To recreate the patch, download whb002-patch.tar.gz and
 

Changed:
<
<
>
>
pkgco.py TruthD3PDAnalysis-00-00-15 cd PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis tar xvfz whb002-patch.tar.gz patch src/TruthParticleFilterTool.cxx TruthParticleFilterTool_cxx.diff patch src/TruthParticleFilterTool.h TruthParticleFilterTool_h.diff
 

whb001

Line: 39 to 47
 

-- WilliamBell - 14-Mar-2012

Added:
>
>
META FILEATTACHMENT attachment="whb002-patch.tar.gz" attr="" comment="whb002 patch files" date="1331804068" name="whb002-patch.tar.gz" path="whb002-patch.tar.gz" size="1243" stream="whb002-patch.tar.gz" tmpFilename="/usr/tmp/CGItemp60672" user="wbell" version="1"
META FILEATTACHMENT attachment="whb003-patch.tar.gz" attr="" comment="whb003 patch files" date="1331804777" name="whb003-patch.tar.gz" path="whb003-patch.tar.gz" size="1267" stream="whb003-patch.tar.gz" tmpFilename="/usr/tmp/CGItemp60688" user="wbell" version="1"

Revision 12012-03-14 - WilliamBell

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WilliamBell"

WhbDatasetTags

whb003

The tag was based on 17.0.5.6.1,AtlasPhysics (p834) with a fix to PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis to fix the bug, https://savannah.cern.ch/bugs/?91063

This patch was successful on SHERPA samples too.

To recreate the patch,


whb002

The tag was based on 17.0.5.6.1,AtlasPhysics (p834) with a fix to PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis to fix the bug, https://savannah.cern.ch/bugs/?91063

The fix was successful on all signal samples except SHERPA.

To recreate the patch,


whb001

A small number of data sets were run using this tag. The tag was based on 17.0.5.6.1,AtlasPhysics (p834) with a partial fix to PhysicsAnalysis/D3PDMaker/TruthD3PDAnalysis to patch a bug, https://savannah.cern.ch/bugs/?91063

These data sets should not be used, instead whb002 and whb003 fully fix the bug.


-- WilliamBell - 14-Mar-2012

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback