LHCbNightliesNewMachine < LHCb

LHCb Web>LHCbComputing>LHCbNightliesOld>LHCbNightliesNewMachine (2013-09-27, MarcoClemencic) (raw view)

---+ How to prepare a new machine for running the nightlies

---++ Overview of existing machines
[[https://twiki.cern.ch/twiki/bin/view/LHCb/LHCbNightliesPrerequirements][LHCbNightliesPrerequirements]]

---++ Scientific Linux release specific information
   * SLC6 
      * link
   * SLC5 

---++ general steps
   * machine accessible... --> ask IT for moving the machine into production... ;)
   * local build directory exists and is accessible (depends on the slot configuration!)
      * standard is =/build/nightlies/=  --> +rw for =lhcbsoft=
   * prepare for use of distcc
      * create =/build/nightlies/.distcc/= directory
   * for some old logs a local directory is used
      * create = /build/nightlies/OldLogs/=

   * add the machine to  =acrontab=
      * build jobs
         * =09 02 * * * buildlhcbXX nice -n 5 ~/bin/nightliesClient.sh x86_64-slc5-icc11-opt > /afs/cern.ch/lhcb/software/nightlies/www/logs/nightliesClient/nightliesClient_`date +\%a`_XXc.txt 2>&1=
      * clean local build dirs
         * =2 0 * * * buildlhcbXX rm -rf /build/nightlies/lhcb-*/`date --date='1 days ago' +\%a`/*=
         * =10 0 * * * buildlhcbXX rm -rf /build/nightlies/lhcb-*/`date +\%a`=
      * kill all remaining jobs from yesterday
         * =3 0 * * * buildlhcbXX ~/bin/killTree.sh nightlies `date --date='1 day ago' +\%a` > /build/nightlies/OldLogs/KillClient.log 2>&1=

---++ SLS sensor
   * A [[https://sls.cern.ch/sls/service.php?id=LHCb_Nightlies_BuildMachines][SLS sensor]] is to be placed for a new machine to monitor its disk usage
   * to monitor the build machine, a script collects the information and pushes it to SLS as xml
      * run the monitor from =acrontab= and let it place the xml onto AFS to be picked up from AFS
         * =*/20 * * * * buildlhcbXX export PYTHONPATH=/afs/cern.ch/lhcb/software/releases/LBSCRIPTS/prod/InstallArea/python:$PYTHONPATH && /afs/cern.ch/user/l/lhcbsoft/PRODUCTION/LHCbNightliesProd/LHCbNightlies/NightliesSensors/AFSSensor.py --aw /afs/cern.ch/lhcb/software/nightlies/www/LHCb_Nightlies_buildlhcbXX.xml --dirdepth 0 --dirroot /build/nightlies --xmlid LHCb_Nightlies_buildlhcb04 >  /build/nightlies/OldLogs/localdisk.out 2>&1=
   * prepare the SLS interface
      * the admin page is available under [[https://apex.cern.ch/pls/htmldb_itcore/f?p=116:LOGIN:888212673570801:::::]]
      * if you not yet an account for SDB ask Victor for example
         * you will most probable get an account, able to edit existing sensors but not create new ones
            * ask Victor, if he can clone an existing sensor for you
            * when available, you can add the path to the sensor =xml= (were you let =acrontab= write to ) to the sensor etc.

---++ local services
   * to organize log files in the local build directory =logrotate= is used
      * normally the general =logrotate= configuration in =/afs/cern.ch/user/l/lhcbsoft/public/PRODUCTION/LHCbNightliesProd/LHCbNightlies/NightliesConfig/logRotateLocal.conf= should be fine
         * the local state file is normally placed at =/build/nightlies/logrotate.local=
      * set up the local =crontab=, i.e. 
         * <verbatim>*/2 * * * * date -Is >>  /build/nightlies/meminfo.txt; cat /proc/meminfo >> /build/nightlies/meminfo.txt; ps aux --forest >> /build/nightlies/meminfo.txt; echo "-----------" >> /build/nightlies/meminfo.txt
55 23 * * * tar cjf /build/nightlies/OldLogs/ProcessesMeminfo.tar.bz2  /build/nightlies/meminfo.txt > /dev/null 2>&1; rm /build/nightlies/meminfo.txt
55 1  * * * /usr/sbin/logrotate -s /build/nightlies/logrotate.local /afs/cern.ch/user/l/lhcbsoft/public/PRODUCTION/LHCbNightliesProd/LHCbNightlies/NightliesConfig/logRotateLocal.conf</verbatim>


-- Main.ThomasHartmann - 12-Oct-2012

Topic revision: r6 - 2013-09-27 - MarcoClemencic

LHCb

Public webs

Welcome Guest

- Cern Search
- TWiki Search
- Google Search
LHCb All webs

Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback