Difference: PrivateTCK (1 vs. 17)

Revision 172016-04-28 - MikaVesterinen

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

How to create HLT TCKs

Changed:
<
<

Creating split HLT1 and HLT2 TCKs for 2016

HLT piquets should follow this set of instructions for production TCKs for 2016 datataking

Producing a dataset with at least one accepted event per HLT line

This step is not normally needed, as the existing dataset should be sufficient. Please consult with one of the HLT experts if in doubt.

There is a script in Hlt/HltPiquetScripts which does this. scripts/Moore_Hlt1Hlt2_MakeDataForTCK_Parallel.py This script runs Moore with a configurable number of parallel processes. It keeps track of which lines have fired, and only writes out an accepted event if it is the first time for any of the lines. It is slightly more efficient though, since it doesn't actually require that the line fires -- only that the next to last algorithm has been executed. First step is to make sure that we have suitable input data. The process is most efficient if we start from a mix of raw data from the different streams. E.g,.

<!-- SyntaxHighlightingPlugin -->
nsls /castor/cern.ch/grid/lhcb/data/2015/RAW/TURCAL/LHCb/COLLISION15/164699/ | head -n 5
<!-- end SyntaxHighlightingPlugin -->
Then copy the data to /scratch/hlt/make_tcks on the online system. From the online system do
<!-- SyntaxHighlightingPlugin -->
getpack Hlt/HltPiquetScripts
<!-- end SyntaxHighlightingPlugin -->
You probably want to do this in a screen session:
<!-- SyntaxHighlightingPlugin -->
lb-run Moore v25r1 python Hlt/HltPiquetScripts/scripts/Moore_Hlt1Hlt2_MakeDataForTCK_Parallel.py 2>&1 | tee log
<!-- end SyntaxHighlightingPlugin -->
There will be an .mdf and .log file for each Moore process, and they will be filled in real time.

Creating and testing split HLT TCKs.

Before you begin, you should have an appropriate hex key for the new TCKs. You can e.g. use the latest (use SetupMoore --list-versions) released Moore (v25r2 in this example) with TCKsh to find out:

<!-- SyntaxHighlightingPlugin -->
lb-run Moore v25r2 TCKsh
>>> listConfigurations()
.... ....
MOORE_v25r2
    Calibration_VeloMicroBias
      0x11301715 : b95ffa736bfc93d10bae16e76a3a6c96 : Prescale = 1
    Physics_pp_May2016
      0x11291600 : 8eed26d4c1e458200a7bb0ff69deeaf3 : Hlt1, Physics_pp_May2016, 0x1600
      0x21291600 : c3269de2a6f7e0bd687a82af0879778d : Hlt2, Physics_pp_May2016, 0x1600
<!-- end SyntaxHighlightingPlugin -->
Now if we want to make a new set of TCKs with the 0x1600 L0 TCK, we would use the keys 0x112a1600 and 0x212a1600. (For more details on the TCKsh functions, see this page.)

Now you are ready to create and test the TCKs. First, create the Moore environment. If you need to check out versions of packages that are different from the Moore release, then you will need to do an lb-dev. The following is an example with Moore v25r2.

<!-- SyntaxHighlightingPlugin -->
lb-dev Moore v25r2
cd MooreDev_v25r2
# getpack XYZ if needed
make purge
make -j 8 install
<!-- end SyntaxHighlightingPlugin -->
If you don't need any package updates, then, in the following you could replace all "./run" with "lb-run Moore v25r2"
>
>
This page describes the pre-2016 procedure to make a single HLT1+HLT2 TCK. For datataking in 2016 onwards, we have split HLT1 and HLT2 TCKs. HLT piquets should go to these instructions for producing split TCKs for 2016 datataking
 
Changed:
<
<
The scripts to create and test the Hlt TCKs live in the Hlt/Moore package under tests/options/TCK/ You can see them e.g. with
<!-- SyntaxHighlightingPlugin -->
./run ls '$MOOREROOT/tests/options/TCK'
<!-- end SyntaxHighlightingPlugin -->
CreateTCK1.py  CreateTCK2.py  CreateTCKEntry.py  RedoL0.py  TCKOptions.py  TestTCK1.py  TestTCK2.py 
You will need to edit these scripts, so copy them locally
<!-- SyntaxHighlightingPlugin -->
cp `./run echo '$MOOREROOT'`/tests/options/TCK/* .  ## for some reason I can't just do ./run cp '$MOOREROOT/tests/options/TCK/*' .
<!-- end SyntaxHighlightingPlugin -->
ways check the following items in TCKOptions.py
  • Do you have the correct ThresholdSettings?
  • Do you have the correct L0TCK?
  • Your proposed new TCK numbers are correct
  • The config label for the TCKs are sensible and meaningful

First we need to prepare the input data. The starting point is a small sample of real data events which is selected such that all HLT lines get to their next to last algorithm. It is currently at

 /eos/lhcb/wg/HLT/2016CommissioningDatasets/OneAcceptedPerLine_v25r1_Physics_pp_May2016_190416 
Now, we need to process these data with the correct L0 TCK (0x1600 in this example).

You can run it with

<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py RedoL0.py 2>&1 | tee RedoL0.log
<!-- end SyntaxHighlightingPlugin -->
It should produce a file RedoL0.mdf, which will be the input to the next step.

The next step is to use the output .mdf from RedoL0.py to create the HLT configuration using CreateTCK1.py. You need to make sure that the script points at the correct input data

<!-- SyntaxHighlightingPlugin -->
IOHelper('MDF').inputFiles(['PFN:RedoL0.mdf']) 
<!-- end SyntaxHighlightingPlugin -->
Also pay particular attention to Moore().ThresholdSettings, Moore().ConfigLabel and the L0TCK. Now you should be ready to run
<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py CreateTCK1.py TCKOptions.py 2>&1 | tee CreateTCK1.log
<!-- end SyntaxHighlightingPlugin -->
This should create a directory TCKData containing a file, config.cdb If there is an issue with this, or any subsequent step, it is safest to delete the config.cdb file, fix the problems and start again. Pay attention and report all ERRORs and any WARNINGs that you are unsure of to the corresponding JIRA task. Now you need to map the configuration to a TCK number. You need to get the id from the log file, e.g. like this
<!-- SyntaxHighlightingPlugin -->
ID=`cat CreateTCK1.log | grep HltGenConfig | grep "top id for this config" | cut -d ":" -f 2 | xargs`
<!-- end SyntaxHighlightingPlugin -->
And then feed this ID to the CreateTCKEntry.py script, together with the your proposed TCK number (see above).
<!-- SyntaxHighlightingPlugin -->
./run python CreateTCKEntry.py ${ID} 0x112a1600 2>&1 | tee CreateTCKEntry1.log
<!-- end SyntaxHighlightingPlugin -->
The next step is to test that we can actually run from this TCK. Have a look at TestTCK1.py, and check that the input data is the same as in CreateTCK1.py. The following will run Moore in Split="Hlt1" with your newly
<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py TestTCK1.py TCKOptions.py 2>&1 | tee TestTCK1.log
<!-- end SyntaxHighlightingPlugin -->
This should produce an output .mdf file called TestTCK1.mdf. We can use this file as input to produce the Hlt2 TCK. Take a look at CreateTCK2.py, CreateTCKEntry.py and TestTCK2.py, and make sure that the settings are correct. In particular:
  • CreateTCK2.py and TestTCK2.py should take TestTCK1.mdf as their input data
  • CreateTCK2.py should set the appropriate L0TCK and the ConfigLabel for the TCK should be sensible.
Now you should be able to run the remaining steps in one go
<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py CreateTCK2.py TCKOptions.py 2>&1 | tee CreateTCK2.log
ID=`cat CreateTCK2.log | grep HltGenConfig | grep "top id for this config" | cut -d ":" -f 2 | xargs`
TCK2=0x212F1600
CreateTCKEntry.py ${ID} ${TCK2} 2>&1 | tee CreateTCKEntry2.log
./run gaudirun.py TestTCK2.py TCKOptions.py 2>&1 | tee TestTCK2.log
<!-- end SyntaxHighlightingPlugin -->
Once you are comfortable with all of the above, you can of course wrap all the steps in one script, in case you need to redo the procedure, e.g. to pick up some package updates or other changes.

We could now add the new TCKs to the config.cdb in the TCK/HltTCK package, but first, we should

  • Share and discuss the log files, and warnings, errors etc... with the experts
  • Diffs with previous TCKs should be circulated for review by the trigger WG liaisons

Finally, you will want to copy these TCKs into the config.cdb in the TCK/HltTCK package. Here is an example. Checkout the head of TCK/HltTCK

<!-- SyntaxHighlightingPlugin -->
# copy_TCK.py
from TCKUtils.utils import *
source_cas = ConfigAccessSvc("ConfigAccessSvcSource", File = 'TCKData/config.cdb') ## copy the new entries from here
target_cas = ConfigAccessSvc("ConfigAccessSvcTarget", File = 'TCK/HltTCK/config.cdb', Mode = 'ReadWrite') ## into here

# either of these would work
for source in ("0x*12a1600"):
#for source in (0x112a1600,0x212a1600):
    glob = 'TCK/0x%08x' % source if type(source) == int else "TCK/" + source
    copy(source = source_cas, target = target_cas, glob = glob)
<!-- end SyntaxHighlightingPlugin -->
Run like this (currently need to use the nightly since we need some features in TCKUtils that are newer than Moore v25r2). Or you could getpack Hlt/TCKUtils head and rebuild, and then just use ./run.
<!-- SyntaxHighlightingPlugin -->
lb-run --nightly lhcb-head TUE Moore HEAD python copy_TCK.py
<!-- end SyntaxHighlightingPlugin -->

Creating unsplit HLT TCKS prior to 2016

HLT piquets should go to the instructions for producing split TCKs for 2016 datataking
>
>
 

Creating a private TCK

To keep this tutorial simple and complete a separate config.tar file is used to store the private TCK, but the same steps apply when using config.tar from the TCK/HltTCK package.

Revision 162016-04-28 - RosenMatev

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

How to create HLT TCKs

Line: 9 to 9
 

Producing a dataset with at least one accepted event per HLT line

Changed:
<
<
*This step is not normally needed, as the existing dataset should be sufficient. Please consult with one of the HLT experts if in doubt.* There is a script in Hlt/HltPiquetScripts which does this. scripts/Moore_Hlt1Hlt2_MakeDataForTCK_Parallel.py
>
>
This step is not normally needed, as the existing dataset should be sufficient. Please consult with one of the HLT experts if in doubt.

There is a script in Hlt/HltPiquetScripts which does this. scripts/Moore_Hlt1Hlt2_MakeDataForTCK_Parallel.py

 This script runs Moore with a configurable number of parallel processes. It keeps track of which lines have fired, and only writes out an accepted event if it is the first time for any of the lines. It is slightly more efficient though, since it doesn't actually require that
Line: 23 to 23
 
<!-- SyntaxHighlightingPlugin -->
nsls /castor/cern.ch/grid/lhcb/data/2015/RAW/TURCAL/LHCb/COLLISION15/164699/ | head -n 5
<!-- end SyntaxHighlightingPlugin -->
Changed:
<
<
Then copy the data to /scratch/hlt/make_tcks on the online system.
>
>
Then copy the data to /scratch/hlt/make_tcks on the online system.
 From the online system do
<!-- SyntaxHighlightingPlugin -->
getpack Hlt/HltPiquetScripts
<!-- end SyntaxHighlightingPlugin -->
You probably want to do this in a screen session: %CODE{lang="sh"}%
Changed:
<
<
lb-run Moore v25r1 python Hlt/HltPiquetScripts/scripts/Moore_Hlt1Hlt2_MakeDataForTCKs_Parallel.py 2>&1 | tee log
>
>
lb-run Moore v25r1 python Hlt/HltPiquetScripts/scripts/Moore_Hlt1Hlt2_MakeDataForTCK_Parallel.py 2>&1 | tee log
 %ENDCODE% There will be an .mdf and .log file for each Moore process, and they will be filled in real time.
Line: 248 to 248
 ConfigAccessSvc().File = '/path/to/your/config.tar'
Deleted:
<
<
-- RoelAaij - 04-Jul-2011
 
META FILEATTACHMENT attachment="runMoore_CreateTCK.py.txt" attr="" comment="" date="1414655804" name="runMoore_CreateTCK.py.txt" path="runMoore_CreateTCK.py.txt" size="804" user="sbenson" version="3"
META FILEATTACHMENT attachment="runMoore_CreateDST.py.txt" attr="" comment="Script to generate a private TCK" date="1414655573" name="runMoore_CreateDST.py.txt" path="runMoore_CreateDST.py.txt" size="1918" user="sbenson" version="3"
META FILEATTACHMENT attachment="runMoore_CreateTCK_2.py.txt" attr="" comment="" date="1414655855" name="runMoore_CreateTCK_2.py.txt" path="runMoore_CreateTCK_2.py.txt" size="804" user="sbenson" version="1"

Revision 152016-04-27 - MikaVesterinen

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

How to create HLT TCKs

Line: 50 to 50
 
    1. x21291600 : c3269de2a6f7e0bd687a82af0879778d : Hlt2, Physics_pp_May2016, 0x1600
%ENDCODE% Now if we want to make a new set of TCKs with the 0x1600 L0 TCK, we would use the keys
Changed:
<
<
0x112F1600 and 0x212F1600.
>
>
0x112a1600 and 0x212a1600.
 (For more details on the TCKsh functions, see this page.)

Now you are ready to create and test the TCKs.

Line: 114 to 114
 %ENDCODE% And then feed this ID to the CreateTCKEntry.py script, together with the your proposed TCK number (see above). %CODE{lang="sh"}%
Changed:
<
<
./run python CreateTCKEntry.py ${ID} 0x112F1600 2>&1 | tee CreateTCKEntry1.log
>
>
./run python CreateTCKEntry.py ${ID} 0x112a1600 2>&1 | tee CreateTCKEntry1.log
 %ENDCODE% The next step is to test that we can actually run from this TCK. Have a look at TestTCK1.py, and check that the input data is the same as in CreateTCK1.py.
Line: 143 to 143
 
  • Share and discuss the log files, and warnings, errors etc... with the experts
  • Diffs with previous TCKs should be circulated for review by the trigger WG liaisons
Changed:
<
<
Finally, you will want to copy these TCKs into the config.cdb in the TCK/HltTCK package.
>
>
Finally, you will want to copy these TCKs into the config.cdb in the TCK/HltTCK package.
 Here is an example.
Changed:
<
<
Checkout the head of TCK/HltTCK
>
>
Checkout the head of TCK/HltTCK
 %CODE{lang="python"}% # copy_TCK.py from TCKUtils.utils import *
Line: 153 to 153
 target_cas = ConfigAccessSvc("ConfigAccessSvcTarget", File = 'TCK/HltTCK/config.cdb', Mode = 'ReadWrite') ## into here

# either of these would work

Changed:
<
<
for source in ("0x*12F1600"): #for source in (0x112F1600,0x212F1600):
>
>
for source in ("0x*12a1600"): #for source in (0x112a1600,0x212a1600):
  glob = 'TCK/0x%08x' % source if type(source) == int else "TCK/" + source copy(source = source_cas, target = target_cas, glob = glob) %ENDCODE%

Revision 142016-04-27 - MikaVesterinen

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

How to create HLT TCKs

Line: 66 to 66
 %ENDCODE% If you don't need any package updates, then, in the following you could replace all "./run" with "lb-run Moore v25r2"
Added:
>
>

The scripts to create and test the Hlt TCKs live in the Hlt/Moore package under tests/options/TCK/ You can see them e.g. with

<!-- SyntaxHighlightingPlugin -->
./run ls '$MOOREROOT/tests/options/TCK'
<!-- end SyntaxHighlightingPlugin -->
CreateTCK1.py  CreateTCK2.py  CreateTCKEntry.py  RedoL0.py  TCKOptions.py  TestTCK1.py  TestTCK2.py 
You will need to edit these scripts, so copy them locally
<!-- SyntaxHighlightingPlugin -->
cp `./run echo '$MOOREROOT'`/tests/options/TCK/* .  ## for some reason I can't just do ./run cp '$MOOREROOT/tests/options/TCK/*' .
<!-- end SyntaxHighlightingPlugin -->
ways check the following items in TCKOptions.py
  • Do you have the correct ThresholdSettings?
  • Do you have the correct L0TCK?
  • Your proposed new TCK numbers are correct
  • The config label for the TCKs are sensible and meaningful
 First we need to prepare the input data. The starting point is a small sample of real data events which is selected such that all HLT lines get to their next to last algorithm. It is currently at
 /eos/lhcb/wg/HLT/2016CommissioningDatasets/OneAcceptedPerLine_v25r1_Physics_pp_May2016_190416 
Now, we need to process these data with the correct L0 TCK (0x1600 in this example).
Deleted:
<
<
%CODE{lang="python"}% .py from Gaudi.Configuration import ApplicationMgr from Configurables import L0App app = L0App() app.ReplaceL0Banks = True app.outputFile = "RedoL0.mdf" app.EvtMax = -1 app.TCK = '0x1600' app.DataType = '2015' app.CondDBtag = 'cond-20150828' app.DDDBtag = 'dddb-20150724'

from GaudiConf import IOHelper IOHelper('MDF').inputFiles(['mdf:root://eoslhcb.cern.ch//eos/lhcb/wg/HLT/2016CommissioningDatasets/OneAcceptedPerLine_v25r1_Physics_pp_May2016_190416/Moore%02d.mdf' %i for i in range(0,15)])

from Configurables import EventSelector EventSelector().PrintFreq = 10

from Configurables import HltRoutingBitsFilter rb_filter = HltRoutingBitsFilter( "RoutingBitsFilter" ) rb_filter.RequireMask = [0x0, 0x4, 0x0]

# HLT Bank killer from Configurables import bankKiller hlt_banks = [ 'HltDecReports','HltRoutingBits','HltSelReports','HltVertexReports','HltLumiSummary','HltTrackReports' ] killer = bankKiller( 'RemoveInputHltRawBanks', BankTypes = hlt_banks )

 
Deleted:
<
<
ApplicationMgr().TopAlg = [rb_filter, killer] %ENDCODE%
 You can run it with
<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py RedoL0.py 2>&1 | tee RedoL0.log
<!-- end SyntaxHighlightingPlugin -->
It should produce a file RedoL0.mdf, which will be the input to the next step.
Changed:
<
<
The scripts to create and test the Hlt TCKs live in the Hlt/Moore package under tests/options/TCK/ You can see them e.g. with
<!-- SyntaxHighlightingPlugin -->
./run ls '$MOOREROOT/tests/options/TCK'
<!-- end SyntaxHighlightingPlugin -->
You will need to edit these scripts, so copy them locally
<!-- SyntaxHighlightingPlugin -->
cp `./run echo '$MOOREROOT'`/tests/options/TCK/* .  ## for some reason I can't just do ./run cp '$MOOREROOT/tests/options/TCK/*' .
<!-- end SyntaxHighlightingPlugin -->
The first step is to create the HLT configuration using CreateTCK1.py.
>
>
The next step is to use the output .mdf from RedoL0.py to create the HLT configuration using CreateTCK1.py.
 You need to make sure that the script points at the correct input data
<!-- SyntaxHighlightingPlugin -->
IOHelper('MDF').inputFiles(['PFN:RedoL0.mdf']) 
<!-- end SyntaxHighlightingPlugin -->
Also pay particular attention to Moore().ThresholdSettings, Moore().ConfigLabel and the L0TCK. Now you should be ready to run %CODE{lang="sh"}%
Changed:
<
<
./run gaudirun.py CreateTCK1.py 2>&1 | tee CreateTCK1.log
>
>
./run gaudirun.py CreateTCK1.py TCKOptions.py 2>&1 | tee CreateTCK1.log
 %ENDCODE% This should create a directory TCKData containing a file, config.cdb If there is an issue with this, or any subsequent step, it is safest to delete the config.cdb file, fix the problems and start again.
Line: 137 to 117
 ./run python CreateTCKEntry.py ${ID} 0x112F1600 2>&1 | tee CreateTCKEntry1.log %ENDCODE% The next step is to test that we can actually run from this TCK.
Changed:
<
<
Have a look at TestTCK1.py, and check that the input data is the same as in CreateTCK1.py, and that Moore().InitialTCK = .
>
>
Have a look at TestTCK1.py, and check that the input data is the same as in CreateTCK1.py.
 The following will run Moore in Split="Hlt1" with your newly %CODE{lang="sh"}%
Changed:
<
<
./run gaudirun.py TestTCK1.py 2>&1 | tee TestTCK1.log
>
>
./run gaudirun.py TestTCK1.py TCKOptions.py 2>&1 | tee TestTCK1.log
 %ENDCODE% This should produce an output .mdf file called TestTCK1.mdf. We can use this file as input to produce the Hlt2 TCK. Take a look at CreateTCK2.py, CreateTCKEntry.py and TestTCK2.py, and make sure that the settings are correct. In particular:
  • CreateTCK2.py and TestTCK2.py should take TestTCK1.mdf as their input data
Changed:
<
<
  • CreateTCK2.py should set the appropriate L0TCK and the ConfigLabel for the TCK should be sensible.
>
>
  • CreateTCK2.py should set the appropriate L0TCK and the ConfigLabel for the TCK should be sensible.
 Now you should be able to run the remaining steps in one go %CODE{lang="sh"}%
Changed:
<
<
./run gaudirun.py CreateTCK2.py 2>&1 | tee CreateTCK2.log
>
>
./run gaudirun.py CreateTCK2.py TCKOptions.py 2>&1 | tee CreateTCK2.log
 ID=`cat CreateTCK2.log | grep HltGenConfig | grep "top id for this config" | cut -d ":" -f 2 | xargs` TCK2=0x212F1600 CreateTCKEntry.py ${ID} ${TCK2} 2>&1 | tee CreateTCKEntry2.log
Changed:
<
<
./run gaudirun.py TestTCK2.py 2>&1 | tee TestTCK2.log
>
>
./run gaudirun.py TestTCK2.py TCKOptions.py 2>&1 | tee TestTCK2.log
 %ENDCODE% Once you are comfortable with all of the above, you can of course wrap all the steps in one script, in case you need to redo the procedure, e.g. to pick up some package updates or other changes.

Revision 132016-04-26 - MikaVesterinen

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

How to create HLT TCKs

Line: 164 to 164
 
  • Share and discuss the log files, and warnings, errors etc... with the experts
  • Diffs with previous TCKs should be circulated for review by the trigger WG liaisons
Added:
>
>
Finally, you will want to copy these TCKs into the config.cdb in the TCK/HltTCK package. Here is an example. Checkout the head of TCK/HltTCK
<!-- SyntaxHighlightingPlugin -->
# copy_TCK.py
from TCKUtils.utils import *
source_cas = ConfigAccessSvc("ConfigAccessSvcSource", File = 'TCKData/config.cdb') ## copy the new entries from here
target_cas = ConfigAccessSvc("ConfigAccessSvcTarget", File = 'TCK/HltTCK/config.cdb', Mode = 'ReadWrite') ## into here

# either of these would work
for source in ("0x*12F1600"):
#for source in (0x112F1600,0x212F1600):
    glob = 'TCK/0x%08x' % source if type(source) == int else "TCK/" + source
    copy(source = source_cas, target = target_cas, glob = glob)
<!-- end SyntaxHighlightingPlugin -->
Run like this (currently need to use the nightly since we need some features in TCKUtils that are newer than Moore v25r2). Or you could getpack Hlt/TCKUtils head and rebuild, and then just use ./run.
<!-- SyntaxHighlightingPlugin -->
lb-run --nightly lhcb-head TUE Moore HEAD python copy_TCK.py
<!-- end SyntaxHighlightingPlugin -->
 

Creating unsplit HLT TCKS prior to 2016

HLT piquets should go to the instructions for producing split TCKs for 2016 datataking

Revision 122016-04-26 - JessicaPrisciandaro

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

How to create HLT TCKs

Line: 134 to 134
 %ENDCODE% And then feed this ID to the CreateTCKEntry.py script, together with the your proposed TCK number (see above). %CODE{lang="sh"}%
Changed:
<
<
CreateTCKEntry.py ${ID} 0x112F1600 2>&1 | tee CreateTCKEntry1.log
>
>
./run python CreateTCKEntry.py ${ID} 0x112F1600 2>&1 | tee CreateTCKEntry1.log
 %ENDCODE% The next step is to test that we can actually run from this TCK. Have a look at TestTCK1.py, and check that the input data is the same as in CreateTCK1.py, and that

Revision 112016-04-26 - MikaVesterinen

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

How to create HLT TCKs

Line: 7 to 7
 

Creating split HLT1 and HLT2 TCKs for 2016

HLT piquets should follow this set of instructions for production TCKs for 2016 datataking
Changed:
<
<

Producing a dataset with at least one accepted event per HLT line

>
>

Producing a dataset with at least one accepted event per HLT line

 
Changed:
<
<
This step is not normally needed, as the existing dataset should be sufficient. Please consult with one of the HLT experts if in doubt.
>
>
*This step is not normally needed, as the existing dataset should be sufficient. Please consult with one of the HLT experts if in doubt.*
 There is a script in Hlt/HltPiquetScripts which does this. scripts/Moore_Hlt1Hlt2_MakeDataForTCK_Parallel.py This script runs Moore with a configurable number of parallel processes.
Line: 34 to 34
 %ENDCODE% There will be an .mdf and .log file for each Moore process, and they will be filled in real time.
Changed:
<
<

Creating and testing split HLT TCKs.

>
>

Creating and testing split HLT TCKs.

  Before you begin, you should have an appropriate hex key for the new TCKs. You can e.g. use the latest (use SetupMoore --list-versions) released Moore (v25r2 in this example) with TCKsh to find out:

Revision 102016-04-26 - MikaVesterinen

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"
Changed:
<
<

How to create HLT Trigger Configuration Keys

>
>

How to create HLT TCKs

 

Creating split HLT1 and HLT2 TCKs for 2016

Added:
>
>
HLT piquets should follow this set of instructions for production TCKs for 2016 datataking
 

Producing a dataset with at least one accepted event per HLT line

Line: 33 to 34
 %ENDCODE% There will be an .mdf and .log file for each Moore process, and they will be filled in real time.
Changed:
<
<

Creating and testing the split TCKs from the above dataset

>
>

Creating and testing split HLT TCKs.

 
Added:
>
>
Before you begin, you should have an appropriate hex key for the new TCKs. You can e.g. use the latest (use SetupMoore --list-versions) released Moore (v25r2 in this example) with TCKsh to find out:
<!-- SyntaxHighlightingPlugin -->
lb-run Moore v25r2 TCKsh
>>> listConfigurations()
.... ....
MOORE_v25r2
    Calibration_VeloMicroBias
      0x11301715 : b95ffa736bfc93d10bae16e76a3a6c96 : Prescale = 1
    Physics_pp_May2016
      0x11291600 : 8eed26d4c1e458200a7bb0ff69deeaf3 : Hlt1, Physics_pp_May2016, 0x1600
      0x21291600 : c3269de2a6f7e0bd687a82af0879778d : Hlt2, Physics_pp_May2016, 0x1600
<!-- end SyntaxHighlightingPlugin -->
Now if we want to make a new set of TCKs with the 0x1600 L0 TCK, we would use the keys 0x112F1600 and 0x212F1600. (For more details on the TCKsh functions, see this page.)

Now you are ready to create and test the TCKs.

 First, create the Moore environment. If you need to check out versions of packages that are different from the Moore release, then you will need to do an lb-dev. The following is an example with Moore v25r2.
Deleted:
<
<
If you don't need any package updates, then you could replace all "./run" with "lb-run Moore v25r2"
 %CODE{lang="sh"}% lb-dev Moore v25r2 cd MooreDev_v25r2
Line: 46 to 64
 make purge make -j 8 install %ENDCODE%
Changed:
<
<
The scripts to create and test the Hlt TCKs live in the Hlt/Moore package under tests/options/TCK/
>
>
If you don't need any package updates, then, in the following you could replace all "./run" with "lb-run Moore v25r2"

First we need to prepare the input data. The starting point is a small sample of real data events which is selected such that all HLT lines get to their next to last algorithm. It is currently at

 /eos/lhcb/wg/HLT/2016CommissioningDatasets/OneAcceptedPerLine_v25r1_Physics_pp_May2016_190416 
Now, we need to process these data with the correct L0 TCK (0x1600 in this example).
<!-- SyntaxHighlightingPlugin -->
#RedoL0.py
from Gaudi.Configuration import ApplicationMgr
from Configurables import L0App
app = L0App()
app.ReplaceL0Banks = True
app.outputFile = "RedoL0.mdf"
app.EvtMax = -1
app.TCK = '0x1600'
app.DataType = '2015'
app.CondDBtag = 'cond-20150828'
app.DDDBtag = 'dddb-20150724'

from GaudiConf import IOHelper
IOHelper('MDF').inputFiles(['mdf:root://eoslhcb.cern.ch//eos/lhcb/wg/HLT/2016CommissioningDatasets/OneAcceptedPerLine_v25r1_Physics_pp_May2016_190416/Moore%02d.mdf' %i for i in range(0,15)]) 

from Configurables import EventSelector
EventSelector().PrintFreq = 10

from Configurables import HltRoutingBitsFilter
rb_filter = HltRoutingBitsFilter( "RoutingBitsFilter" )
rb_filter.RequireMask = [0x0, 0x4, 0x0]

# HLT Bank killer                                                                                                                                                                                                                               
from Configurables import bankKiller
hlt_banks = [ 'HltDecReports','HltRoutingBits','HltSelReports','HltVertexReports','HltLumiSummary','HltTrackReports' ]
killer = bankKiller( 'RemoveInputHltRawBanks',  BankTypes = hlt_banks )

ApplicationMgr().TopAlg = [rb_filter, killer]
<!-- end SyntaxHighlightingPlugin -->
You can run it with
<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py RedoL0.py 2>&1 | tee RedoL0.log
<!-- end SyntaxHighlightingPlugin -->
It should produce a file RedoL0.mdf, which will be the input to the next step.

The scripts to create and test the Hlt TCKs live in the Hlt/Moore package under tests/options/TCK/

 You can see them e.g. with %CODE{lang="sh"}% ./run ls '$MOOREROOT/tests/options/TCK'
Line: 55 to 116
 
<!-- SyntaxHighlightingPlugin -->
cp `./run echo '$MOOREROOT'`/tests/options/TCK/* .  ## for some reason I can't just do ./run cp '$MOOREROOT/tests/options/TCK/*' .
<!-- end SyntaxHighlightingPlugin -->
Changed:
<
<
The first step is to create the HLT configuration using CreateTCK1.py.
>
>
The first step is to create the HLT configuration using CreateTCK1.py.
 You need to make sure that the script points at the correct input data
Changed:
<
<
<!-- SyntaxHighlightingPlugin -->
IOHelper('MDF').inputFiles(['mdf:root://eoslhcb.cern.ch//eos/lhcb/wg/HLT/2016CommissioningDatasets/OneAcceptedPerLine_v2\
5r1_Physics_pp_May2016_190416/Moore%02d.mdf' %i for i in range(0,15)]) 
<!-- end SyntaxHighlightingPlugin -->
Also pay particular attention to Moore().ThresholdSettings, Moore().ConfigLabel and the L0TCK.
>
>
<!-- SyntaxHighlightingPlugin -->
IOHelper('MDF').inputFiles(['PFN:RedoL0.mdf']) 
<!-- end SyntaxHighlightingPlugin -->
Also pay particular attention to Moore().ThresholdSettings, Moore().ConfigLabel and the L0TCK.
 Now you should be ready to run
<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py CreateTCK1.py 2>&1 | tee CreateTCK1.log
<!-- end SyntaxHighlightingPlugin -->
Added:
>
>
This should create a directory TCKData containing a file, config.cdb If there is an issue with this, or any subsequent step, it is safest to delete the config.cdb file, fix the problems and start again. Pay attention and report all ERRORs and any WARNINGs that you are unsure of to the corresponding JIRA task. Now you need to map the configuration to a TCK number. You need to get the id from the log file, e.g. like this
<!-- SyntaxHighlightingPlugin -->
ID=`cat CreateTCK1.log | grep HltGenConfig | grep "top id for this config" | cut -d ":" -f 2 | xargs`
<!-- end SyntaxHighlightingPlugin -->
And then feed this ID to the CreateTCKEntry.py script, together with the your proposed TCK number (see above).
<!-- SyntaxHighlightingPlugin -->
CreateTCKEntry.py ${ID} 0x112F1600 2>&1 | tee CreateTCKEntry1.log
<!-- end SyntaxHighlightingPlugin -->
The next step is to test that we can actually run from this TCK. Have a look at TestTCK1.py, and check that the input data is the same as in CreateTCK1.py, and that Moore().InitialTCK = . The following will run Moore in Split="Hlt1" with your newly
<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py TestTCK1.py 2>&1 | tee TestTCK1.log
<!-- end SyntaxHighlightingPlugin -->
This should produce an output .mdf file called TestTCK1.mdf. We can use this file as input to produce the Hlt2 TCK. Take a look at CreateTCK2.py, CreateTCKEntry.py and TestTCK2.py, and make sure that the settings are correct. In particular:
  • CreateTCK2.py and TestTCK2.py should take TestTCK1.mdf as their input data
  • CreateTCK2.py should set the appropriate L0TCK and the ConfigLabel for the TCK should be sensible.
Now you should be able to run the remaining steps in one go
<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py CreateTCK2.py 2>&1 | tee CreateTCK2.log
ID=`cat CreateTCK2.log | grep HltGenConfig | grep "top id for this config" | cut -d ":" -f 2 | xargs`
TCK2=0x212F1600
CreateTCKEntry.py ${ID} ${TCK2} 2>&1 | tee CreateTCKEntry2.log
./run gaudirun.py TestTCK2.py 2>&1 | tee TestTCK2.log
<!-- end SyntaxHighlightingPlugin -->
Once you are comfortable with all of the above, you can of course wrap all the steps in one script, in case you need to redo the procedure, e.g. to pick up some package updates or other changes.

We could now add the new TCKs to the config.cdb in the TCK/HltTCK package, but first, we should

  • Share and discuss the log files, and warnings, errors etc... with the experts
  • Diffs with previous TCKs should be circulated for review by the trigger WG liaisons

Creating unsplit HLT TCKS prior to 2016

HLT piquets should go to the instructions for producing split TCKs for 2016 datataking
 

Creating a private TCK

To keep this tutorial simple and complete a separate config.tar file is used to store the private TCK, but the same steps apply when using config.tar from the TCK/HltTCK package.

Revision 92016-04-26 - MikaVesterinen

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"
Changed:
<
<

Contents

>
>

How to create HLT Trigger Configuration Keys

 
Changed:
<
<

Creating a split TCK

Start dumping instructions here: Creating a split TCK takes a few more steps.
  • run from threshold settings in split=Hlt1, to produce the Hlt1TCK
  • run from this Hlt1TCK in split=Hlt1 to test it, and to produce an .mdf that is need for the next step
  • run over .mdf from previous step, and from threshold settings in split=Hlt2, to produce Hlt2 TCK
  • run from Hlt2TCK to test it..
>
>

Creating split HLT1 and HLT2 TCKs for 2016

 

Producing a dataset with at least one accepted event per HLT line

Added:
>
>
This step is not normally needed, as the existing dataset should be sufficient. Please consult with one of the HLT experts if in doubt.
 There is a script in Hlt/HltPiquetScripts which does this. scripts/Moore_Hlt1Hlt2_MakeDataForTCK_Parallel.py This script runs Moore with a configurable number of parallel processes.
Line: 36 to 33
 %ENDCODE% There will be an .mdf and .log file for each Moore process, and they will be filled in real time.
Added:
>
>

Creating and testing the split TCKs from the above dataset

First, create the Moore environment. If you need to check out versions of packages that are different from the Moore release, then you will need to do an lb-dev. The following is an example with Moore v25r2. If you don't need any package updates, then you could replace all "./run" with "lb-run Moore v25r2"

<!-- SyntaxHighlightingPlugin -->
lb-dev Moore v25r2
cd MooreDev_v25r2
# getpack XYZ if needed
make purge
make -j 8 install
<!-- end SyntaxHighlightingPlugin -->
The scripts to create and test the Hlt TCKs live in the Hlt/Moore package under tests/options/TCK/ You can see them e.g. with
<!-- SyntaxHighlightingPlugin -->
./run ls '$MOOREROOT/tests/options/TCK'
<!-- end SyntaxHighlightingPlugin -->
You will need to edit these scripts, so copy them locally
<!-- SyntaxHighlightingPlugin -->
cp `./run echo '$MOOREROOT'`/tests/options/TCK/* .  ## for some reason I can't just do ./run cp '$MOOREROOT/tests/options/TCK/*' .
<!-- end SyntaxHighlightingPlugin -->
The first step is to create the HLT configuration using CreateTCK1.py. You need to make sure that the script points at the correct input data
<!-- SyntaxHighlightingPlugin -->
IOHelper('MDF').inputFiles(['mdf:root://eoslhcb.cern.ch//eos/lhcb/wg/HLT/2016CommissioningDatasets/OneAcceptedPerLine_v2\
5r1_Physics_pp_May2016_190416/Moore%02d.mdf' %i for i in range(0,15)]) 
<!-- end SyntaxHighlightingPlugin -->
Also pay particular attention to Moore().ThresholdSettings, Moore().ConfigLabel and the L0TCK. Now you should be ready to run
<!-- SyntaxHighlightingPlugin -->
./run gaudirun.py CreateTCK1.py 2>&1 | tee CreateTCK1.log
<!-- end SyntaxHighlightingPlugin -->
 

Creating a private TCK

To keep this tutorial simple and complete a separate config.tar file is used to store the private TCK, but the same steps apply when using config.tar from the TCK/HltTCK package.

Revision 82016-04-21 - SaschaStahl

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

Contents

Added:
>
>

Creating a split TCK

Start dumping instructions here: Creating a split TCK takes a few more steps.
  • run from threshold settings in split=Hlt1, to produce the Hlt1TCK
  • run from this Hlt1TCK in split=Hlt1 to test it, and to produce an .mdf that is need for the next step
  • run over .mdf from previous step, and from threshold settings in split=Hlt2, to produce Hlt2 TCK
  • run from Hlt2TCK to test it..
 

Producing a dataset with at least one accepted event per HLT line

There is a script in Hlt/HltPiquetScripts which does this.

Revision 72016-04-19 - MikaVesterinen

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

Contents

Line: 6 to 6
 

Producing a dataset with at least one accepted event per HLT line

There is a script in Hlt/HltPiquetScripts which does this.

Changed:
<
<
scripts/Moore_Hlt1Hlt2_MakeDataForTCK.py
>
>
scripts/Moore_Hlt1Hlt2_MakeDataForTCK_Parallel.py This script runs Moore with a configurable number of parallel processes. It keeps track of which lines have fired, and only writes out an accepted event if it is the first time for any of the lines. It is slightly more efficient though, since it doesn't actually require that the line fires -- only that the next to last algorithm has been executed. First step is to make sure that we have suitable input data. The process is most efficient if we start from a mix of raw data from the different streams. E.g,.
<!-- SyntaxHighlightingPlugin -->
nsls /castor/cern.ch/grid/lhcb/data/2015/RAW/TURCAL/LHCb/COLLISION15/164699/ | head -n 5
<!-- end SyntaxHighlightingPlugin -->
Then copy the data to /scratch/hlt/make_tcks on the online system. From the online system do
<!-- SyntaxHighlightingPlugin -->
getpack Hlt/HltPiquetScripts
<!-- end SyntaxHighlightingPlugin -->
You probably want to do this in a screen session:
<!-- SyntaxHighlightingPlugin -->
lb-run Moore v25r1 python Hlt/HltPiquetScripts/scripts/Moore_Hlt1Hlt2_MakeDataForTCKs_Parallel.py 2>&1 | tee log
<!-- end SyntaxHighlightingPlugin -->
There will be an .mdf and .log file for each Moore process, and they will be filled in real time.
 

Creating a private TCK

To keep this tutorial simple and complete a separate config.tar file is used to store the private TCK, but the same steps apply when using config.tar from the TCK/HltTCK package.

Revision 62016-04-15 - MikaVesterinen

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

Contents

Added:
>
>

Producing a dataset with at least one accepted event per HLT line

There is a script in Hlt/HltPiquetScripts which does this. scripts/Moore_Hlt1Hlt2_MakeDataForTCK.py

 

Creating a private TCK

To keep this tutorial simple and complete a separate config.tar file is used to store the private TCK, but the same steps apply when using config.tar from the TCK/HltTCK package.

Revision 52015-07-31 - MatthewKenzie

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

Contents

Line: 9 to 9
 The first thing which needs to be done is to ensure that a ThresholdSettings is available which contains all of the required features. This will generally involve getpacking Hlt/HltSettings, Hlt/Hlt{1,2}Lines and sometimes other packages and making the required changes. This is assumed to be completed and the settings are assumed to be named PrivateThresholdSettings.

Setting up the environment

Changed:
<
<
Start by setting up the cmt environment for Moore and creating an empty file in which the TCK(s) will be stored; this needs to be done only once. A template script to use is attached as https://twiki.cern.ch/twiki/pub/LHCb/PrivateTCK/runMoore_CreateTCK_2.py.txt; save it somewhere and modify it to suit the input MC DST you want to use. Make sure that the database tags and datatype are correct.
>
>
Start by setting up the cmt environment for Moore and creating an empty file in which the TCK(s) will be stored; this needs to be done only once. A template script to use is attached as https://twiki.cern.ch/twiki/pub/LHCb/PrivateTCK/CreateTCK.py.txt; save it somewhere and modify it to suit the input MC DST you want to use. Make sure that the database tags and datatype are correct.
 
Changed:
<
<
$> SetupProject --build-env Moore v12r7 $> cd $User_release_area/Moore_v12r7
>
>
$> SetupProject --build-env Moore v24r0 $> cd $User_release_area/Moore_v24r0
 $> mkdir TCKData
Changed:
<
<
$> touch TCKData/config.tar $> mkdir scripts $> cp /path/to/runMoore_CreateTCK.py.txt scripts/runMoore_CreateTCK.py
>
>
$> cp /path/to/CreateTCK.py.txt CreateTCK.py
 

Make sure that the TCKData property in the script points to the directory you just created.

Line: 24 to 22
 

Create the TCK

Logout and log back in (or create a new shell), make sure that the modified threshold settings and other packages which contain the changes you require are properly installed and run the script.
Changed:
<
<
$> SetupProject Moore v12r7 $> cd $User_release_area/Moore_v12r7/scripts $> gaudirun.py runMoore_CreateTCK.py
>
>
$> SetupProject Moore v24r0 $> cd $User_release_area/Moore_v24r0 $> gaudirun.py CreateTCK.py
 
Changed:
<
<
You should now have the TCK dumped in the config.tar file. If something goes wrong, delete the config.tar file, touch it again, fix the problem and run again.
>
>
You should now have the TCK dumped in a config.cdb file in the TCKData directory. If something goes wrong, delete the TCKData directory, make it again, fix the problem and run again.
 

Mapping the configuration to a TCK.

The following steps require the use of TCKsh; you can stay in the same shell.
Line: 44 to 42
 Once you have found a suitable TCK, it is time to map your configuration to it. Below 0x00900032 is used.
$> TCKsh
Changed:
<
<
>>> cas_rw = ConfigAccessSvc( Mode = 'ReadWrite', File='/path/to/config.tar', CompressOnWrite = True )
>
>
>>> cas_rw = ConfigAccessSvc( Mode = 'ReadWrite', File='/path/to/config.cdb' )
 >>> printConfigurations( getConfigurations( cas = cas_rw ) )
starting appMgr @ pid = 720 MOORE_v12r7
Line: 66 to 64
  The following lines insure the available TCKs are read from your private file:
Changed:
<
<
from Configurables import ConfigAccessSvc ConfigAccessSvc().File='config.tar' Moore().TCKData='/project/bfys/raaij/cmtuser/Moore_v12r7/TCKData'
>
>
from Configurables import ConfigCDBAccessSvc ConfigCDBAccessSvc().File = 'config.cdb' Moore().TCKData='/path/to/TCKData'
 

A template script is attached as https://twiki.cern.ch/twiki/pub/LHCb/PrivateTCK/runMoore_CreateDST.py.txt; save it somewhere.

Line: 91 to 89
 
META FILEATTACHMENT attachment="runMoore_CreateTCK.py.txt" attr="" comment="" date="1414655804" name="runMoore_CreateTCK.py.txt" path="runMoore_CreateTCK.py.txt" size="804" user="sbenson" version="3"
META FILEATTACHMENT attachment="runMoore_CreateDST.py.txt" attr="" comment="Script to generate a private TCK" date="1414655573" name="runMoore_CreateDST.py.txt" path="runMoore_CreateDST.py.txt" size="1918" user="sbenson" version="3"
META FILEATTACHMENT attachment="runMoore_CreateTCK_2.py.txt" attr="" comment="" date="1414655855" name="runMoore_CreateTCK_2.py.txt" path="runMoore_CreateTCK_2.py.txt" size="804" user="sbenson" version="1"
Added:
>
>
META FILEATTACHMENT attachment="CreateTCK.py.txt" attr="" comment="" date="1438344251" name="CreateTCK.py.txt" path="CreateTCK.py.txt" size="637" user="mkenzie" version="1"

Revision 42015-05-01 - VladimirGligorov

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

Contents

Line: 44 to 44
 Once you have found a suitable TCK, it is time to map your configuration to it. Below 0x00900032 is used.
$> TCKsh
Changed:
<
<
>>> cas_rw = ConfigTarFileAccessSvc( Mode = 'ReadWrite', File='/path/to/config.tar', CompressOnWrite = True )
>
>
>>> cas_rw = ConfigAccessSvc( Mode = 'ReadWrite', File='/path/to/config.tar', CompressOnWrite = True )
 >>> printConfigurations( getConfigurations( cas = cas_rw ) )
starting appMgr @ pid = 720 MOORE_v12r7
Line: 66 to 66
  The following lines insure the available TCKs are read from your private file:
Changed:
<
<
from Configurables import ConfigTarFileAccessSvc ConfigTarFileAccessSvc().File='config.tar'
>
>
from Configurables import ConfigAccessSvc ConfigAccessSvc().File='config.tar'
 Moore().TCKData='/project/bfys/raaij/cmtuser/Moore_v12r7/TCKData'
Line: 82 to 82
 

Running DaVinci on your DST with private TCK

Once the DST has been created, DaVinci can be run on it, but it needs to be told where to find the private TCK. This is done by adding the following options to the script:
Changed:
<
<
from Configurables import ConfigTarFileAccessSvc ConfigTarFileAccessSvc().File = '/path/to/your/config.tar'
>
>
from Configurables import ConfigAccessSvc ConfigAccessSvc().File = '/path/to/your/config.tar'
 

-- RoelAaij - 04-Jul-2011

Revision 32014-10-30 - SeanBenson

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

Contents

Line: 9 to 9
 The first thing which needs to be done is to ensure that a ThresholdSettings is available which contains all of the required features. This will generally involve getpacking Hlt/HltSettings, Hlt/Hlt{1,2}Lines and sometimes other packages and making the required changes. This is assumed to be completed and the settings are assumed to be named PrivateThresholdSettings.

Setting up the environment

Changed:
<
<
Start by setting up the cmt environment for Moore and creating an empty file in which the TCK(s) will be stored; this needs to be done only once. A template script to use is attached as https://twiki.cern.ch/twiki/pub/LHCb/PrivateTCK/runMoore_CreateTCK.py.txt; save it somewhere and modify it to suit the input MC DST you want to use. Make sure that the database tags and datatype are correct.
>
>
Start by setting up the cmt environment for Moore and creating an empty file in which the TCK(s) will be stored; this needs to be done only once. A template script to use is attached as https://twiki.cern.ch/twiki/pub/LHCb/PrivateTCK/runMoore_CreateTCK_2.py.txt; save it somewhere and modify it to suit the input MC DST you want to use. Make sure that the database tags and datatype are correct.
 
$> SetupProject --build-env Moore v12r7
$> cd $User_release_area/Moore_v12r7
Line: 88 to 88
  -- RoelAaij - 04-Jul-2011
Changed:
<
<
META FILEATTACHMENT attachment="runMoore_CreateTCK.py.txt" attr="" comment="Script to generate a private TCK" date="1309801925" name="runMoore_CreateTCK.py.txt" path="runMoore_CreateTCK.py" size="1013" stream="runMoore_CreateTCK.py" tmpFilename="/usr/tmp/CGItemp19643" user="raaij" version="2"
META FILEATTACHMENT attachment="runMoore_CreateDST.py.txt" attr="" comment="Script to run Moore on MC using a private TCK" date="1309803384" name="runMoore_CreateDST.py.txt" path="runMoore_CreateDST.py" size="1919" stream="runMoore_CreateDST.py" tmpFilename="/usr/tmp/CGItemp15319" user="raaij" version="1"
>
>
META FILEATTACHMENT attachment="runMoore_CreateTCK.py.txt" attr="" comment="" date="1414655804" name="runMoore_CreateTCK.py.txt" path="runMoore_CreateTCK.py.txt" size="804" user="sbenson" version="3"
META FILEATTACHMENT attachment="runMoore_CreateDST.py.txt" attr="" comment="Script to generate a private TCK" date="1414655573" name="runMoore_CreateDST.py.txt" path="runMoore_CreateDST.py.txt" size="1918" user="sbenson" version="3"
META FILEATTACHMENT attachment="runMoore_CreateTCK_2.py.txt" attr="" comment="" date="1414655855" name="runMoore_CreateTCK_2.py.txt" path="runMoore_CreateTCK_2.py.txt" size="804" user="sbenson" version="1"

Revision 22011-07-27 - RoelAaij

Line: 1 to 1
 
META TOPICPARENT name="LHCbTCK"

Contents

Line: 62 to 62
 Instead of creating a new TCK from scratch, it can be easier, safer and faster to create a new TCK by modifying an existing one. This is described on the HLTTCK page.

Creating a DST using a private TCK

Changed:
<
<
Once the private TCK has been generated, Moore needs to be run on the input DST using the private TCK. a template script is attached as https://twiki.cern.ch/twiki/pub/LHCb/PrivateTCK/runMoore_CreateDST.py.txt; save it somewhere.
>
>
Once the private TCK has been generated, Moore needs to be run on the input DST using the private TCK.

The following lines insure the available TCKs are read from your private file:

from Configurables import ConfigTarFileAccessSvc
ConfigTarFileAccessSvc().File='config.tar'
Moore().TCKData='/project/bfys/raaij/cmtuser/Moore_v12r7/TCKData'

A template script is attached as https://twiki.cern.ch/twiki/pub/LHCb/PrivateTCK/runMoore_CreateDST.py.txt; save it somewhere.

 
$> SetupProject Moore v12r7
$> cd $User_release_area/Moore_v12r7/scripts

Revision 12011-07-04 - RoelAaij

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="LHCbTCK"

Contents

Creating a private TCK

To keep this tutorial simple and complete a separate config.tar file is used to store the private TCK, but the same steps apply when using config.tar from the TCK/HltTCK package.

The first thing which needs to be done is to ensure that a ThresholdSettings is available which contains all of the required features. This will generally involve getpacking Hlt/HltSettings, Hlt/Hlt{1,2}Lines and sometimes other packages and making the required changes. This is assumed to be completed and the settings are assumed to be named PrivateThresholdSettings.

Setting up the environment

Start by setting up the cmt environment for Moore and creating an empty file in which the TCK(s) will be stored; this needs to be done only once. A template script to use is attached as https://twiki.cern.ch/twiki/pub/LHCb/PrivateTCK/runMoore_CreateTCK.py.txt; save it somewhere and modify it to suit the input MC DST you want to use. Make sure that the database tags and datatype are correct.
$> SetupProject --build-env Moore v12r7
$> cd $User_release_area/Moore_v12r7
$> mkdir TCKData
$> touch TCKData/config.tar
$> mkdir scripts
$> cp /path/to/runMoore_CreateTCK.py.txt scripts/runMoore_CreateTCK.py

Make sure that the TCKData property in the script points to the directory you just created.

Create the TCK

Logout and log back in (or create a new shell), make sure that the modified threshold settings and other packages which contain the changes you require are properly installed and run the script.
$> SetupProject Moore v12r7
$> cd $User_release_area/Moore_v12r7/scripts
$> gaudirun.py runMoore_CreateTCK.py

You should now have the TCK dumped in the config.tar file. If something goes wrong, delete the config.tar file, touch it again, fix the problem and run again.

Mapping the configuration to a TCK.

The following steps require the use of TCKsh; you can stay in the same shell.

To prevent possible confusion and errors, It is vital that you select a unique and unused TCK for your configuration. To see which TCKs are already in use, start TCKsh and look at the configurations. Search for the latest Moore version and find the highest TCK.

$> TCKsh
>>> listConfigurations()
>>> quit()

Once you have found a suitable TCK, it is time to map your configuration to it. Below 0x00900032 is used.

$> TCKsh
>>> cas_rw = ConfigTarFileAccessSvc( Mode = 'ReadWrite', File='/path/to/config.tar', CompressOnWrite = True )
>>> printConfigurations( getConfigurations( cas = cas_rw ) )
starting appMgr @ pid = 720
MOORE_v12r7
    Physics_May2011
          <NONE> : 24e17751af7447fed73ec35b108218b6 : Test TCK
>>> id = '24e17751af7447fed73ec35b108218b6'
>>> createTCKEntries( { 0x00900032 : id }, cas = cas_rw )
creating mapping TCK: 0x00900032 -> ID: 24e17751af7447fed73ec35b108218b6
>>> quit()

The private TCK 0x00900032 is now created and stored in your own config.tar

Modifying a private TCK.

Instead of creating a new TCK from scratch, it can be easier, safer and faster to create a new TCK by modifying an existing one. This is described on the HLTTCK page.

Creating a DST using a private TCK

Once the private TCK has been generated, Moore needs to be run on the input DST using the private TCK. a template script is attached as https://twiki.cern.ch/twiki/pub/LHCb/PrivateTCK/runMoore_CreateDST.py.txt; save it somewhere.
$> SetupProject Moore v12r7
$> cd $User_release_area/Moore_v12r7/scripts
$> cp runMoore_CreateDST.py.txt runMoore_CreateDST.py
$> gaudirun.py runMoore_CreateDST.py

Running DaVinci on your DST with private TCK

Once the DST has been created, DaVinci can be run on it, but it needs to be told where to find the private TCK. This is done by adding the following options to the script:
from Configurables import ConfigTarFileAccessSvc
ConfigTarFileAccessSvc().File = '/path/to/your/config.tar'

-- RoelAaij - 04-Jul-2011

META FILEATTACHMENT attachment="runMoore_CreateTCK.py.txt" attr="" comment="Script to generate a private TCK" date="1309801925" name="runMoore_CreateTCK.py.txt" path="runMoore_CreateTCK.py" size="1013" stream="runMoore_CreateTCK.py" tmpFilename="/usr/tmp/CGItemp19643" user="raaij" version="2"
META FILEATTACHMENT attachment="runMoore_CreateDST.py.txt" attr="" comment="Script to run Moore on MC using a private TCK" date="1309803384" name="runMoore_CreateDST.py.txt" path="runMoore_CreateDST.py" size="1919" stream="runMoore_CreateDST.py" tmpFilename="/usr/tmp/CGItemp15319" user="raaij" version="1"
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback