Difference: HLTDevelopersChecklist (1 vs. 10)

Revision 102014-03-25 - RobLambert

Line: 1 to 1
 
META TOPICPARENT name="LHCbTrigger"

HLT Developers Checklist

LHCb code development is fully described in SVNUsageGuidelines. You must be familiar with the guidelines laid out there, for which this page is just a summary.

Added:
>
>
 

Shortlist

In this section, the ideal development strategy is presented, summarizing what's in SVNUsageGuidelines, in a bit more ordered fashion.

Line: 45 to 47
 
  1. Present results in the Friday HLT operations meeting.
Changed:
<
<

Other notes

  • make or cmt make?
>
>

Make or CMT make?

 
    • typing make in the top level directory will ensure everything you have getpacked is up-to-date and built. It is slow, though.
Changed:
<
<
    • If you;ve only modified one package, you can type cmt make in the cmt directory of that package instead, which is quicker, but can cause problems if other things have changed.
>
>
  • If you've only modified one package, you can type cmt make in the cmt directory of that package instead, which is quicker, but can cause problems if other things have changed.

Creating your development/testing sandbox

When you start your development, you;re going to want to have some measurable outcome that really tells you if you have made the improvement you though you had made, and not messed anything else up in the process.

Creating this safe space where you know what and where you are changing things, and you know what is going on before and after your test is "sandboxing" and can be used to be really sure you are only changing what you thought.

Writing a QMTest

If you are starting your development, you can write a QMTest which verifies the problem you though was there really is there, in some measurable way that you can really see. Any generic Gaudi job can be wrapped rather straightforwardly into a test within whatever package you are using.

Simplify the problem as much as possible

The first step in correcting a problem is isolating a problem, to identify the cause. This isolation step should result in some repeatable behaviour which you can clearly identify, and maybe solidify into a QMTest.

 
Changed:
<
<

Running Moore

Using Moore QMTests

>
>

Easy ways to simplify things: use GaudiExcise

GaudiExcise can cut out a single algorithm from a complicated sequence and make a sandbox for you to repeatedly run only the algorithm you are changing.

How can I check I didn't change anything else: GaudiRegressionTester

Ideally the release manager and the integration tests in the nightlies will be checking this for you. However, even if the nightly tests do break, you initially probably have no way of knowing if it was you that really broke something. A lot of the time, the nightlies are broken for completely different reasons and failures are nothing to do with you.

Also, the nightly tests might not be useful for your exact use case, and might be too time-consuming to run yourself. You may be working on some branch, or in some other complicated mode.

We are working on a tool called GaudiRegressionTester to make this easy for you.... pretty much you should be running:

  • Collect the benchmark from before your change
<!-- SyntaxHighlightingPlugin -->
SetupProject Moore --no-user-area #choose the version you want to test your change against, --no-user-area means it will just take the released version
gaudiregressiontester.py $MOOREROOT/tests/qmtest/moore.qms/physics.qms.2012 #or some options file of your choosing
<!-- end SyntaxHighlightingPlugin -->
  • This will store a local benchmark.xml (xmlsummary), benchmark.csv (timingtable), and benchmark.log (warnings/errors/fatal)
  • Run the same thing with your changes
<!-- SyntaxHighlightingPlugin -->
SetupProject Moore #choose the version you want to test your change against, --no-user-area means it will just take the released version
#getpack/make whatever you may need to pick up your changes
gaudiregressiontester.py $MOOREROOT/tests/qmtest/moore.qms/physics.qms.2012 #or some options file of your choosing
<!-- end SyntaxHighlightingPlugin -->
  • This will re-run the exact same thing and then tell you the diffs which result from that

Using Moore QMTests

 There are several qmtests that run moore in Hlt/Moore/tests/

cmt TestPackage will run all the tests, as usual.

Line: 61 to 100
  To manually run Moore with the different options available, look into Hlt/Moore/tests/options/. For example to run Hlt1 Only (split scenario) do:
Changed:
<
<
gaudirun.py Moore_Hlt1AndHlt2.pr Moore_Hlt1Only.py | tee mylogfile.log

Using RunMoore

Another option is to use RunMoore.py from the Hlt/Hlt2CommissioningScripts. You need to have a close look at the options to that script. Note that the input file locations are taken from RealDataMB.py and RealDataSignal.py (and the analog MC files). Also don't forget to switch on the L0 emulation with --L0

Example: python ./RunMoore.py -n 100 -df 27 -t Physics_September2012 --L0

>
>
gaudirun.py Moore_Hlt1AndHlt2.py Moore_Hlt1Only.py | tee mylogfile.log

Revision 92014-01-14 - SebastianNeubert

Line: 1 to 1
 
META TOPICPARENT name="LHCbTrigger"

HLT Developers Checklist

Line: 7 to 7
 

Shortlist

In this section, the ideal development strategy is presented, summarizing what's in SVNUsageGuidelines, in a bit more ordered fashion.

Changed:
<
<
Description: Video Example:
Before you start editing:
>
>
Description: Video Example:
Before you start editing:
 
  1. Discuss your proposed development with the release manager.
  2. Make a savannah task and assign to yourself, or take over an existing task for this development. You can be as verbose as you like on the tasks you are assigned to. Treat it like a log book, if you like.
  3. Decide or discuss which version of Moore this development is for, most likely it is for the Head. HEAD modifications can normally be developed either on top of the latest released version or the HEAD which is being built each night in the nightlies. If you're working with the head release, take care that the previous nightlies did actually work, and if not, discuss with the release manager what to do.
  4. Consider talking about your proposed development in the HLT operations meetings (on Friday afternoons)
Changed:
<
<
Setup your local project structure and environment,
>
>
Setup your local project structure and environment,
 
  1. first by cleaning out everything which is already in ~/cmtuser/Moore_HEAD (replace HEAD with your version, or the latest released version)
  2. Setup the appropriate project. For the HEAD in the nightlies this is: SetupProject Moore HEAD --nightly lhcb-head --build-env #replace head with your version...
  3. Checkout ONLY the code you want to modify: getpack Hlt/MyPackage head #replace with your package name. If you don't getpack the head here, you'll have problems later.
  4. make #typed directly in the top directory
  5. Setup a second time! SetupProject Moore HEAD --nightly lhcb-head #i.e. without the Build-env. You must have typed SetupProject twice, once before and once after the make.
Changed:
<
<
Think about TestDrivenDevelopment:
>
>
Think about TestDrivenDevelopment:
 
  1. To fix any problem, you will anyway first need to check you can reproduce it, and then re-run this test to verify when you think you have fixed it. It's best to encapsulate this in the existing testing framework
  2. Ideally you will add a QM-Test, either in Moore or in your package which demonstrates the problem, so that you know when it is fixed and so that the nightly tests will clearly fail before it is fixed. Contact your release manager if you don't know how to do this.
  3. Running your own qmtest can be done simply by going to the cmt directory of your project, and typing something like cmt TestPackage mypackage.mynewtest
Changed:
<
<
The editing process:
>
>
The editing process:
 
  1. Once your local package structure is created, and the first build worked, you will then only need to type SetupProject once (without build-env). If you getpack something else, best to repeat the earlier steps.
  2. | Modify what you want to modify.
Line: 72 to 36
 
  1. Commit often, but only code which compiles and runs. If you've added a test this is easy to verify. Read The Rules here!! Commit messages should really tell us what has changed, and whether it works. Don't use stupid trivial commit messages.
  2. Communicate effectively with your release manager during this process. Easiest way to do this is through savannah and the tag collector.
  3. Keep an eye on the nightlies (of your modifications take more than a day to complete).
Changed:
<
<
Aftercare:
>
>
Aftercare:
 
  1. Update/close savannah tasks
  2. Ensure your changesets appear in the the tag collector
  3. Ensure your documentation is really complete and understandable
  4. Double-check that any failures in the nightlies are not due to your modifications. Probably you need to check with your release manager.
  5. Present results in the Friday HLT operations meeting.
Changed:
<
<
>
>
 

Other notes

  • make or cmt make?
    • typing make in the top level directory will ensure everything you have getpacked is up-to-date and built. It is slow, though.
    • If you;ve only modified one package, you can type cmt make in the cmt directory of that package instead, which is quicker, but can cause problems if other things have changed.
Added:
>
>

Running Moore

Using Moore QMTests

There are several qmtests that run moore in Hlt/Moore/tests/

cmt TestPackage will run all the tests, as usual.

cmt TestPackage moore.physics.deferral will only run the tests suit for the split HLT

To manually run Moore with the different options available, look into Hlt/Moore/tests/options/. For example to run Hlt1 Only (split scenario) do:

gaudirun.py Moore_Hlt1AndHlt2.pr Moore_Hlt1Only.py | tee mylogfile.log

Using RunMoore

Another option is to use RunMoore.py from the Hlt/Hlt2CommissioningScripts. You need to have a close look at the options to that script. Note that the input file locations are taken from RealDataMB.py and RealDataSignal.py (and the analog MC files). Also don't forget to switch on the L0 emulation with --L0

Example: python ./RunMoore.py -n 100 -df 27 -t Physics_September2012 --L0

Revision 82013-09-20 - RobLambert

Line: 1 to 1
 
META TOPICPARENT name="LHCbTrigger"

HLT Developers Checklist

Line: 48 to 48
 
Changed:
<
<
Think about test-driven-development:
>
>
Think about TestDrivenDevelopment:
 
  1. To fix any problem, you will anyway first need to check you can reproduce it, and then re-run this test to verify when you think you have fixed it. It's best to encapsulate this in the existing testing framework
  2. Ideally you will add a QM-Test, either in Moore or in your package which demonstrates the problem, so that you know when it is fixed and so that the nightly tests will clearly fail before it is fixed. Contact your release manager if you don't know how to do this.

Revision 72013-08-12 - RobLambert

Line: 1 to 1
 
META TOPICPARENT name="LHCbTrigger"

HLT Developers Checklist

Line: 28 to 28
 
Changed:
<
<
>
>
 
Line: 43 to 43
 
Changed:
<
<
>
>
 
Line: 56 to 56
 
Changed:
<
<
>
>
 
Line: 75 to 75
 
Changed:
<
<
>
>
 
Line: 90 to 90
 
Changed:
<
<
>
>
 

Revision 62013-06-07 - RobLambert

Line: 1 to 1
 
META TOPICPARENT name="LHCbTrigger"

HLT Developers Checklist

Line: 8 to 8
  In this section, the ideal development strategy is presented, summarizing what's in SVNUsageGuidelines, in a bit more ordered fashion.
Changed:
<
<
  1. Before you start editing:
>
>
Description: Video Example:
Before you start editing:
 
    1. Discuss your proposed development with the release manager.
    2. Make a savannah task and assign to yourself, or take over an existing task for this development. You can be as verbose as you like on the tasks you are assigned to. Treat it like a log book, if you like.
    3. Decide or discuss which version of Moore this development is for, most likely it is for the Head. HEAD modifications can normally be developed either on top of the latest released version or the HEAD which is being built each night in the nightlies. If you're working with the head release, take care that the previous nightlies did actually work, and if not, discuss with the release manager what to do.
    4. Consider talking about your proposed development in the HLT operations meetings (on Friday afternoons)
Changed:
<
<
  1. Setup your local project structure and environment,
>
>
Setup your local project structure and environment,
 
    1. first by cleaning out everything which is already in ~/cmtuser/Moore_HEAD (replace HEAD with your version, or the latest released version)
    2. Setup the appropriate project. For the HEAD in the nightlies this is: SetupProject Moore HEAD --nightly lhcb-head --build-env #replace head with your version...
    3. Checkout ONLY the code you want to modify: getpack Hlt/MyPackage head #replace with your package name. If you don't getpack the head here, you'll have problems later.
    4. make #typed directly in the top directory
    5. Setup a second time! SetupProject Moore HEAD --nightly lhcb-head #i.e. without the Build-env. You must have typed SetupProject twice, once before and once after the make.
Changed:
<
<
  1. Think about test-driven-development:
>
>
Think about test-driven-development:
 
    1. To fix any problem, you will anyway first need to check you can reproduce it, and then re-run this test to verify when you think you have fixed it. It's best to encapsulate this in the existing testing framework
    2. Ideally you will add a QM-Test, either in Moore or in your package which demonstrates the problem, so that you know when it is fixed and so that the nightly tests will clearly fail before it is fixed. Contact your release manager if you don't know how to do this.
Changed:
<
<
    1. Running your own qmtest can be done simply by going to the cmt directory of your project, and typing something like cmt qmtest_run mypackage.mynewtest
  1. The editing process:
>
>
  1. Running your own qmtest can be done simply by going to the cmt directory of your project, and typing something like cmt TestPackage mypackage.mynewtest
The editing process:
 
    1. Once your local package structure is created, and the first build worked, you will then only need to type SetupProject once (without build-env). If you getpack something else, best to repeat the earlier steps.
    2. | Modify what you want to modify.
    3. | Compile your changes (make in the top-level directory, or see below)
Line: 33 to 72
 
    1. Commit often, but only code which compiles and runs. If you've added a test this is easy to verify. Read The Rules here!! Commit messages should really tell us what has changed, and whether it works. Don't use stupid trivial commit messages.
    2. Communicate effectively with your release manager during this process. Easiest way to do this is through savannah and the tag collector.
    3. Keep an eye on the nightlies (of your modifications take more than a day to complete).
Changed:
<
<
  1. Aftercare:
>
>
Aftercare:
 
    1. Update/close savannah tasks
    2. Ensure your changesets appear in the the tag collector
    3. Ensure your documentation is really complete and understandable
    4. Double-check that any failures in the nightlies are not due to your modifications. Probably you need to check with your release manager.
    5. Present results in the Friday HLT operations meeting.
Added:
>
>
 

Other notes

Revision 52013-05-08 - RobLambert

Line: 1 to 1
 
META TOPICPARENT name="LHCbTrigger"

HLT Developers Checklist

Changed:
<
<
This checklist summarizes the steps needed to modify HLT code. Please keep the material here as concise and practical as possible and use links to point to more detailed explanations.
>
>
LHCb code development is fully described in SVNUsageGuidelines. You must be familiar with the guidelines laid out there, for which this page is just a summary.
 
Changed:
<
<
THIS LIST IS STILL UNDER CONSTRUCTION USE WITH CAUTION
>
>

Shortlist

 
Deleted:
<
<
  • Make a savannah task for your work
  • Setting up the Moore development environment
    • Always develop on the head / nightly of the project you are working on
    • For example: SetupProject --build-env --nightly lhcb-head Brunel HEAD
  • Checking out a package
    • getpack head of package if you want to develop
  • Building a package
    • check if the cmt/requirements are up to date. All packages you depend on need to be listed there!
    • from the top level directory of the project do make
    • you can also do make in the subpackage directories
    • If you develop in a package from another project (say working on the tracking in the moore environment) make sure your code also compiles in the original project of the package
  • Updating application environment after adding a package!
    • After you built the package you need to do another SetupProject <project> HEAD in order to update the env variables
  • Debugging your code
    • FILL IN
    • How to use Gaudi Debug Output
  • Testing
  • Check in
    • discuss you changes with the package manager
    • update your package to the repository head and check/resolve conflicts
    • write a comment into doc/release.notes of your package
    • check in
 \ No newline at end of file
Added:
>
>
In this section, the ideal development strategy is presented, summarizing what's in SVNUsageGuidelines, in a bit more ordered fashion.

  1. Before you start editing:
    1. Discuss your proposed development with the release manager.
    2. Make a savannah task and assign to yourself, or take over an existing task for this development. You can be as verbose as you like on the tasks you are assigned to. Treat it like a log book, if you like.
    3. Decide or discuss which version of Moore this development is for, most likely it is for the Head. HEAD modifications can normally be developed either on top of the latest released version or the HEAD which is being built each night in the nightlies. If you're working with the head release, take care that the previous nightlies did actually work, and if not, discuss with the release manager what to do.
    4. Consider talking about your proposed development in the HLT operations meetings (on Friday afternoons)
  2. Setup your local project structure and environment,
    1. first by cleaning out everything which is already in ~/cmtuser/Moore_HEAD (replace HEAD with your version, or the latest released version)
    2. Setup the appropriate project. For the HEAD in the nightlies this is: SetupProject Moore HEAD --nightly lhcb-head --build-env #replace head with your version...
    3. Checkout ONLY the code you want to modify: getpack Hlt/MyPackage head #replace with your package name. If you don't getpack the head here, you'll have problems later.
    4. make #typed directly in the top directory
    5. Setup a second time! SetupProject Moore HEAD --nightly lhcb-head #i.e. without the Build-env. You must have typed SetupProject twice, once before and once after the make.
  3. Think about test-driven-development:
    1. To fix any problem, you will anyway first need to check you can reproduce it, and then re-run this test to verify when you think you have fixed it. It's best to encapsulate this in the existing testing framework
    2. Ideally you will add a QM-Test, either in Moore or in your package which demonstrates the problem, so that you know when it is fixed and so that the nightly tests will clearly fail before it is fixed. Contact your release manager if you don't know how to do this.
    3. Running your own qmtest can be done simply by going to the cmt directory of your project, and typing something like cmt qmtest_run mypackage.mynewtest
  4. The editing process:
    1. Once your local package structure is created, and the first build worked, you will then only need to type SetupProject once (without build-env). If you getpack something else, best to repeat the earlier steps.
    2. | Modify what you want to modify.
    3. | Compile your changes (make in the top-level directory, or see below)
    4. | Test your changes
    5. Repeat 2-4.
    6. Document continuously. In the savannah task, if you have one, and also always in the doc/release.notes file
    7. Commit often, but only code which compiles and runs. If you've added a test this is easy to verify. Read The Rules here!! Commit messages should really tell us what has changed, and whether it works. Don't use stupid trivial commit messages.
    8. Communicate effectively with your release manager during this process. Easiest way to do this is through savannah and the tag collector.
    9. Keep an eye on the nightlies (of your modifications take more than a day to complete).
  5. Aftercare:
    1. Update/close savannah tasks
    2. Ensure your changesets appear in the the tag collector
    3. Ensure your documentation is really complete and understandable
    4. Double-check that any failures in the nightlies are not due to your modifications. Probably you need to check with your release manager.
    5. Present results in the Friday HLT operations meeting.

Other notes

  • make or cmt make?
    • typing make in the top level directory will ensure everything you have getpacked is up-to-date and built. It is slow, though.
    • If you;ve only modified one package, you can type cmt make in the cmt directory of that package instead, which is quicker, but can cause problems if other things have changed.

Revision 42013-03-27 - SebastianNeubert

Line: 1 to 1
 
META TOPICPARENT name="LHCbTrigger"

HLT Developers Checklist

Line: 9 to 9
 
  • Make a savannah task for your work
  • Setting up the Moore development environment
    • Always develop on the head / nightly of the project you are working on
Added:
>
>
    • For example: SetupProject --build-env --nightly lhcb-head Brunel HEAD
 
  • Checking out a package
    • getpack head of package if you want to develop
  • Building a package

Revision 32013-03-08 - SebastianNeubert

Line: 1 to 1
 
META TOPICPARENT name="LHCbTrigger"
Changed:
<
<
This checklist summarizes the steps needed to modify HLT code. Links to more extensive documentation are given.
>
>

HLT Developers Checklist

 
Changed:
<
<
THIS LIST IS STILL UNDER CONSTRCUTION
>
>
This checklist summarizes the steps needed to modify HLT code. Please keep the material here as concise and practical as possible and use links to point to more detailed explanations.
 
Changed:
<
<
  • Setting up the Moore development environment
>
>
THIS LIST IS STILL UNDER CONSTRUCTION USE WITH CAUTION

  • Make a savannah task for your work
  • Setting up the Moore development environment
    • Always develop on the head / nightly of the project you are working on
 
  • Checking out a package
Added:
>
>
    • getpack head of package if you want to develop
 
  • Building a package
    • check if the cmt/requirements are up to date. All packages you depend on need to be listed there!
Changed:
<
<
    • from the cmt directory of your package use
    • cmt br cmt make
>
>
    • from the top level directory of the project do make
    • you can also do make in the subpackage directories
    • If you develop in a package from another project (say working on the tracking in the moore environment) make sure your code also compiles in the original project of the package
  • Updating application environment after adding a package!
    • After you built the package you need to do another SetupProject <project> HEAD in order to update the env variables
 
  • Debugging your code
    • FILL IN
    • How to use Gaudi Debug Output
  • Testing
Added:
>
>
 
    • FILL IN
Changed:
<
<
  • Check in
>
>
 
    • discuss you changes with the package manager
    • update your package to the repository head and check/resolve conflicts
    • write a comment into doc/release.notes of your package

Revision 22013-03-08 - SebastianNeubert

Line: 1 to 1
 
META TOPICPARENT name="LHCbTrigger"
This checklist summarizes the steps needed to modify HLT code. Links to more extensive documentation are given.
Added:
>
>
THIS LIST IS STILL UNDER CONSTRCUTION
 
  • Setting up the Moore development environment
  • Checking out a package
  • Building a package
    • check if the cmt/requirements are up to date. All packages you depend on need to be listed there!
    • from the cmt directory of your package use
    • cmt br cmt make
Added:
>
>
  • Debugging your code
    • FILL IN
    • How to use Gaudi Debug Output
 
  • Testing
    • FILL IN
  • Check in

Revision 12013-03-08 - SebastianNeubert

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="LHCbTrigger"
This checklist summarizes the steps needed to modify HLT code. Links to more extensive documentation are given.

  • Setting up the Moore development environment
  • Checking out a package
  • Building a package
    • check if the cmt/requirements are up to date. All packages you depend on need to be listed there!
    • from the cmt directory of your package use
    • cmt br cmt make
  • Testing
    • FILL IN
  • Check in
    • discuss you changes with the package manager
    • update your package to the repository head and check/resolve conflicts
    • write a comment into doc/release.notes of your package
    • check in
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback