LHCb Core Software Meeting

Date and Location

07-Mar-2012
10:35 - 11:45
CERN (2-R-030)

Attendees

Ben Couturier, Emmanouil Kiagias, Gloria Corti, Illya Shapoval, Joel Closier, Marco Cattaneo, Marco Clemencic (minutes), Thomas Hartmann (Vidyo)

Subjects

News

  • Geant4 Technical Forum (27 March 2012)
  • New Techincal Student: Emmanouil Kiagias
    • will work with Gloria, Ben and Marco Cl. to set up a benchmarking and regression testing infrastructure (database, job submission/execution, web interface...)
  • Niko asked if there is an interest on GPU-based optimizations.

Specific topics

Projects for VIA Fellows (Marco Ca.)

The LBC group produced 9 projects for VIA fellows, but we will not be able to get people for all of them. Thus we have been asked to prioritize the list.

After a brief discussion, we agreed on the following priority list for the Offline computing projects (the position of the placeholders for Online projects have been agreed between Marco Ca. and Niko):

  1. Unifying the monitoring of grid computing services for the LHCb experiment
  2. Online
  3. Technical Optimization of LHCb Software
  4. Online
  5. An improved detector description framework for Gaudi
  6. Development of a User Interface for the automation of release management tasks in the LHCb experiment

During the discussion, Gloria suggested that we produce a publicly accessible web page with the descriptions of projects of work in LHCb Computing, so the interested people can propose themselves for them. Marco Ca. volunteered to prepare the page: CoreSoftwareProjects.

Install Project (Ben)

After that the latest version of LbScript has been put in production on Tuesday, several jobs failed on the grid because of a bug in a piece of new code for a feature which is not even used. Of course the problem was due to lack of testings, which was caused by lack of documentation on the use-cases.

Discussing with Marco Cl., we agreed on a plan:

  • document the use cases to be tested to validate a new version of install_project.py
  • review the behavior of install_project.py, essentially to allow to rollback to a different (older) version
  • the version of install_project.py (and LbScripts) to be used on the Grid should not be defined by a symlink in AFS, but it should be an explicit parameter in the Configuration System
    • this will allow us to validate a new version in the interactive environment, before proposing it for adoption in the Grid environment, which will be subject to validation by the LHCbDirac team
  • install_project.py and LbScripts do not really need to be coupled: it is much better if they have independent life cycles, so we want to split them again

As a side note, during the emergency release, the build of LbScripts failed (the parallel build cannot be used).
Marco Ca. reminded that it's along standing bug in mkproject that failures are not correctly reported.

Gaudi releases (Marco Cl.)

Just after the release of Gaudi v23r1, Chris reported a problem with the version of RooFit in ROOT 5.32/00, which is a show stopper for his users (the only way around being reverting to the old version). The latest version (5.32/01) might solve the problem, looking at the long list of fixes in RooFit, but it is not sure.

The other important external that we should pick up before running reconstruction jobs is the latest version of NeuroBayes, but here the problem is the license (which is being reviewed at CERN... again).

Both new versions require a new release of Gaudi on top of a new LCGCMT. The deadlines will be discussed at the PAC on Friday.

For the special case of RooFit, we can also take an alternative route. It has been asked several times to be able to install ROOT by itself, independently of Gaudi. To do it we need some development in install_project.py and in the whole distribution machinery. This development has a lower priority than the the ones discussed earlier. Moreover, the feature will be a side effect of the new installation system we are planning.
As a temporary work around, we may prepare the tarball for the new LCGCMT (yet to be defined) and install it independently of Gaudi.

Round Table

Marco Cl.

  • Working to fix ProdConf (the production Configurable) after some problems discovered by Federico in the first test.
    • implementing a mini test framework to be able to run production-like jobs without having to submit grid jobs.
  • QMTest doesn't seem to be supported anymore, while the test runner tool used for ProdConf (nose) is well supported and is extremely powerful, flexible and extensible.
    • it seems possible (and not too difficult) to write a nose plugin to replace QMTest; will prepare a project description for a student.

Joel

  • Started testing of Online data transfer
    • everything OK except that the files are not shown on the BKK because of a negative TCK (which shows as something like 0x-1234abc); to be investigated with the Hlt experts

Gloria

  • We still have the problem that our build of Geant 4 depends on Gaudi, only for GaudiPolicy, thus requiring a rebuild every time we have a new Gaudi
    • Marco Cl. suggests to have a copy of GaudiPolicy in Geant4, just checking the order of use in Gauss to ensure that the correct one is picked up
  • ATLAS would like to use our latest version of EvtGen, and asked it to be installed in the LCG Generator Service area. John and Witek are in contact to do so. This version uses Pythia8, Photos++ and Tauola++ so it needs a rebuild when changing their versions. We take care of compatibility and tuning for major productions when we freeze the versions. ATLAS raised the issue that in BaBar they had included a frozen version of Pythia6 in EvtGen to avoid this and suggested to do the same. I think this is dangerous at all of these codes are actively being developed and re-syncronizing them would require even more work. It is part of the validation of an EvtGen release to check and warn about this. Need to discuss this with Witek: if they insist in wanting this then this part should be kept as a separate library so that we can avoid using it.

Marco Ca.

  • In the new Gaudi, a more aggressive compression algorithm is used by default. We should check the effect on the file sizes in Gauss.

Ben

  • We ran out of space on $LHCBTAR/sources (where we keep LCGCMT-like tarballs for install_project.py).
    • This triggered an investigation that showed a lot of inconsistencies in the set of tarballs we have (e.g. broken dependencies chains, tarballs unreferenced in the html files and vice versa,...); a clean up will give us back ~20GB
  • We just obtained a storage element where to keep the distribution tarballs
    • preparing the scripts and set-up to use it

Illya

  • The problem of permissions on some files created by the new SQLite deployment model has been fixed in the code (still to be released, in v6 and v7)
  • A system of squid proxies and of environment variables to find them is being set up (coordinated by Stefan). We will add the support for it before releasing the new version of SQLDDDB.

Thomas

  • Setting up SLS sensors for the nightlies.
  • There has been a problem of memory usage by one of the scripts, which was run on lxplus via ACRON (now moved to lxbuild for testing).
    Marco Ca.: we should not use lxplus for LHCb specific tasks, we should use our pool of lxbuild nodes (increasing it if necessary).


-- MarcoClemencic - 08-Mar-2012

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2012-03-08 - GloriaCorti
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback