CCNULHCbFarm2 < Sandbox

---+ CCNU LHCb farm 2

%TOC%

---+ Computers

The computers for LHCb are comput19-23. CVMFS is installed on these machines.

The =comput19= is used as interactive jobs, and others are for batch jobs, though this rule is not strict.

---+ Softwares

---++ LHCb environment
Same as in Sandbox.CCNULHCbFarm
<verbatim>
source /cvmfs/lhcb.cern.ch/etc/login.sh
</verbatim>


---++ ROOT
---+++ Recommended
The simplest way to set up root is by running

=source /public/home/lhcb/setups/setup_latest_root.sh=

It will set up the latest version of ROOT from LCG on CVMFS. **Do not copy** it to your directory, because it's keep updating. Instead you can make a soft link to your favorite location, for example:

=ln -s  /public/home/lhcb/setups/setup_latest_root.sh ~/=

The printout information may break your =scp= or =rsync= command if you setup root in your =.bashrc=. You can setup it this way in your =.bashrc=
<verbatim>
if [[ $- == *i* ]] && [ "$HOSTNAME" = "comput19" ]; then
        source ~/setups/latest_root.sh
fi
</verbatim>

---+++ Alternative
One can also set up a different version in a similar way. For example, if one needs 6.18.00 with python 2, this command will do it:

=source /cvmfs/sft.cern.ch/lcg/releases/LCG_96/ROOT/6.18.00/x86_64-centos7-gcc8-opt/ROOT-env.sh=

---++ aspell

aspell is installed on comput19 and comput23

---++ hep_ml
Use this script to setup the hep_ml in =LCG_100 (x86_64-centos7-gcc9-opt)= environment:
=source ~lhcb/setups/setup_hep_ml.sh=

---++ Geant4

---++ Latex
Simply do
<verbatim>
source /home/lhcb/setups/setup_latex.sh
</verbatim>

See Sandbox.CCNULHCbFarm#Latex for more.

---++ Other handy commands
---+++ slidesshow
<verbatim>
source /home/lhcb/setups/setup_latex.sh
source /public/home/lhcb/scripts/scripts_collection/setup.sh
</verbatim>
and you will have a command =slides_report= avaliable

---+ slurm jobs
[[CCNULHCbSubmitJob][&#x4E2D;&#x6587;&#x6559;&#x7A0B;]]

An example script for batch job submission can be found in the attachment and this command could be used to submit jobs:

<verbatim>
sbatch test_job.sh</verbatim>

Note that the options are given inside the script.

Useful webpages [[https://scicomp.jlab.org/docs/farm_slurm_commands][farm_slurm_commands]] [[https://wikis.nyu.edu/display/NYUHPC/Slurm+Tutorial][Slurm Tutorial]]

---++ Example: run a 60 core job 
<verbatim>
srun -p high -N 1 -c 60 --mem-per-cpu=1G ./y_simultaneous_fit.py
</verbatim>
   * =-p high= specifies the batch system to use. =high= is the name of our batch system
   * =-N 1= means we want to run the job on one computer (node)
   * =-c 60= means we need 60 cpu
   * =--mem-per-cpu=1G= means we need 1G memory for each cpu
   * =./y_simultaneous_fit.py= is the script I want to run, which uses =TProof= to run 60 threads for toy MC studies.

I can use the =sacct= and =squeue= to check the status of the job in another session.

---++ Example: run Gauss production

---+ Problems and solutions

---++ Auto logout from comput19
=comput19= is configured to auto-logout inactive users by setting =TMOUT=3600=. One can keep some jobs running in screen by setting =TMOUT== there. (=TMOUT= variable is read-only in the login session but could be overwritten inside a screen session).
One can also put these lines inside .bashrc to overwrite the variable automatically in any screen sessions:

<verbatim>
if [ "$STY" != "" ]; then ## check if this is a screen session
	export TMOUT=
fi
</verbatim>  

---++ CVMFS Input/output error
Error messages like this:
<verbatim>
-bash: /cvmfs/sft.cern.ch/lcg/releases/LCG_101/ROOT/6.24.06/x86_64-centos7-gcc11-opt/ROOT-env.sh: Input/output error
</verbatim>

This is related to the CVMFS system. One can check the information in =/cvmfs/sft.cern.ch/lcg/lastUpdate= to find the lastest update. Report these information to the administrator.

Usually a reboot of the computer will solve the problem. Other possible causes include: network issue, mis-configuration of the cvmfs... 

---++ Direct-access farm2 from hepfarm

We could use sshfs to mount part of the space in farm2 to hepfarm. This is not in place yet. We will only do this if it's considered useful.

   1 mount/unmout need root permission
   1 Need to set up a file to map the users
   1 If we want to set up this, we could use the lhcb account and just mount a subdirectory in its home

---+ User's best practices
Leave your suggestions here...

---++ PID resampling

---+ Links

Sandbox.CcnuTeaching Sandbox.CCNULHCbFarm


-- Main.DongliangZhang - 2020-02-21
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
sh	test_job.sh	r1	manage	0.2 K	2020-06-03 - 08:52	DongliangZhang
Topic revision: r17 - 2022-07-19 - DongliangZhang
Account
- Log In
- Cern Search
- TWiki Search
- Google Search
Sandbox All webs
Edit
Attach
Copyright &© 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback