CCNU LHCb farm 2


The computers for LHCb are comput19-23. CVMFS is installed on these machines.

The comput19 is used as interactive jobs, and others are for batch jobs, though this rule is not strict.


LHCb environment

Same as in CCNULHCbFarm
source /cvmfs/



The simplest way to set up root is by running

source /public/home/lhcb/setups/

It will set up the latest version of ROOT from LCG on CVMFS. *Do not copy* it to your directory, because it's keep updating. Instead you can make a soft link to your favorite location, for example:

ln -s  /public/home/lhcb/setups/ ~/

The printout information may break your scp or rsync command if you setup root in your .bashrc. You can setup it this way in your .bashrc

if [[ $- == *i* ]] && [ "$HOSTNAME" = "comput19" ]; then
        source ~/setups/


One can also set up a different version in a similar way. For example, if one needs 6.18.00 with python 2, this command will do it:

source /cvmfs/


aspell is installed on comput19 and comput23


Use this script to setup the hep_ml in LCG_100 (x86_64-centos7-gcc9-opt) environment: source ~lhcb/setups/



Simply do
source /home/lhcb/setups/

See CCNULHCbFarm#Latex for more.

Other handy commands


source /home/lhcb/setups/
source /public/home/lhcb/scripts/scripts_collection/
and you will have a command slides_report avaliable

slurm jobs


An example script for batch job submission can be found in the attachment and this command could be used to submit jobs:


Note that the options are given inside the script.

Useful webpages farm_slurm_commands Slurm Tutorial

Example: run a 60 core job

srun -p high -N 1 -c 60 --mem-per-cpu=1G ./
  • -p high specifies the batch system to use. high is the name of our batch system
  • -N 1 means we want to run the job on one computer (node)
  • -c 60 means we need 60 cpu
  • --mem-per-cpu=1G means we need 1G memory for each cpu
  • ./ is the script I want to run, which uses TProof to run 60 threads for toy MC studies.

I can use the sacct and squeue to check the status of the job in another session.

Example: run Gauss production

Problems and solutions

Auto logout from comput19

comput19 is configured to auto-logout inactive users by setting TMOUT=3600. One can keep some jobs running in screen by setting TMOUT= there. (TMOUT variable is read-only in the login session but could be overwritten inside a screen session). One can also put these lines inside .bashrc to overwrite the variable automatically in any screen sessions:

if [ "$STY" != "" ]; then ## check if this is a screen session
   export TMOUT=

CVMFS Input/output error

Error messages like this:
-bash: /cvmfs/ Input/output error

This is related to the CVMFS system. One can check the information in /cvmfs/ to find the lastest update. Report these information to the administrator.

Usually a reboot of the computer will solve the problem. Other possible causes include: network issue, mis-configuration of the cvmfs...

Direct-access farm2 from hepfarm

We could use sshfs to mount part of the space in farm2 to hepfarm. This is not in place yet. We will only do this if it's considered useful.

  1. mount/unmout need root permission
  2. Need to set up a file to map the users
  3. If we want to set up this, we could use the lhcb account and just mount a subdirectory in its home

User's best practices

Leave your suggestions here...

PID resampling


CcnuTeaching CCNULHCbFarm

-- DongliangZhang - 2020-02-21

Topic attachments
I Attachment History Action Size Date Who Comment
Unix shell scriptsh r1 manage 0.2 K 2020-06-03 - 08:52 DongliangZhang  
Edit | Attach | Watch | Print version | History: r17 < r16 < r15 < r14 < r13 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r17 - 2022-07-19 - DongliangZhang
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback