CCNU LHCb farm 2

Computers

The computers for LHCb are comput19-23. CVMFS is installed on these machines.

The comput19 is used as interactive jobs, and others are for batch jobs, though this rule is not strict.

Softwares

LHCb environment

Same as in CCNULHCbFarm

source /cvmfs/lhcb.cern.ch/etc/login.sh

ROOT

Recommended

The simplest way to set up root is by running

source /public/home/lhcb/setups/setup_latest_root.sh

It will set up the latest version of ROOT from LCG on CVMFS. *Do not copy* it to your directory, because it's keep updating. Instead you can make a soft link to your favorite location, for example:

ln -s  /public/home/lhcb/setups/setup_latest_root.sh ~/

The printout information may break your scp or rsync command if you setup root in your .bashrc. You can setup it this way in your .bashrc

if [[ $- == *i* ]] && [ "$HOSTNAME" = "comput19" ]; then
        source ~/setups/latest_root.sh
fi

Alternative

One can also set up a different version in a similar way. For example, if one needs 6.18.00 with python 2, this command will do it:

source /cvmfs/sft.cern.ch/lcg/releases/LCG_96/ROOT/6.18.00/x86_64-centos7-gcc8-opt/ROOT-env.sh

aspell

aspell is installed on comput19 and comput23

hep_ml

Use this script to setup the hep_ml in LCG_100 (x86_64-centos7-gcc9-opt) environment: source ~lhcb/setups/setup_hep_ml.sh

Geant4

Latex

Simply do

source /home/lhcb/setups/setup_latex.sh

See CCNULHCbFarm#Latex for more.

Other handy commands

slidesshow

source /home/lhcb/setups/setup_latex.sh
source /public/home/lhcb/scripts/scripts_collection/setup.sh

and you will have a command slides_report avaliable

slurm jobs

中文教程

An example script for batch job submission can be found in the attachment and this command could be used to submit jobs:

sbatch test_job.sh

Note that the options are given inside the script.

Useful webpages farm_slurm_commands Slurm Tutorial

Example: run a 60 core job

srun -p high -N 1 -c 60 --mem-per-cpu=1G ./y_simultaneous_fit.py

• -p high specifies the batch system to use. high is the name of our batch system
• -N 1 means we want to run the job on one computer (node)
• -c 60 means we need 60 cpu
• --mem-per-cpu=1G means we need 1G memory for each cpu
• ./y_simultaneous_fit.py is the script I want to run, which uses TProof to run 60 threads for toy MC studies.

I can use the sacct and squeue to check the status of the job in another session.

Example: run Gauss production

Problems and solutions

Auto logout from comput19

comput19 is configured to auto-logout inactive users by setting TMOUT=3600. One can keep some jobs running in screen by setting TMOUT= there. (TMOUT variable is read-only in the login session but could be overwritten inside a screen session). One can also put these lines inside .bashrc to overwrite the variable automatically in any screen sessions:

if [ "$STY" != "" ]; then ## check if this is a screen session
   export TMOUT=
fi

CVMFS Input/output error

Error messages like this:

-bash: /cvmfs/sft.cern.ch/lcg/releases/LCG_101/ROOT/6.24.06/x86_64-centos7-gcc11-opt/ROOT-env.sh: Input/output error

This is related to the CVMFS system. One can check the information in /cvmfs/sft.cern.ch/lcg/lastUpdate to find the lastest update. Report these information to the administrator.

Usually a reboot of the computer will solve the problem. Other possible causes include: network issue, mis-configuration of the cvmfs...

Direct-access farm2 from hepfarm

We could use sshfs to mount part of the space in farm2 to hepfarm. This is not in place yet. We will only do this if it's considered useful.

1. mount/unmout need root permission
2. Need to set up a file to map the users
3. If we want to set up this, we could use the lhcb account and just mount a subdirectory in its home

User's best practices

Leave your suggestions here...

PID resampling

Links

CcnuTeaching CCNULHCbFarm

-- DongliangZhang - 2020-02-21