LHCb User Enviroment (LbEnv)

Motivations and design

The current tool to set the LHCb User Environment (LbLogin) uses the "SetupProject" tool to look for the various packages to put in the path, python path. SetupProject is now deprecated, and its replacement, lb-run(/lb-dev) is already available but cannot be used for LbLogin/LbScripts. Moreover, LbScripts is showing a number of issues, including incompatibility with Python 3 (Python 2 end of support is scheduled for 2020) and a lot of legacy code that should be decommissioned.

A replacement is therefore necessary and it was decided to redesign the tool making it more lightweight than the current tool (which is rather slow). For that purpose, LbEnv does not scan the file system at every login to find the latest Python, CMake etc. but adds to the path a set of predefined directories containing links to those tools (in the same way the "update-alternatives" tool does).

Since we are using almost only Python for the tools in the LHCb User Environment, we decided, on the same ground of LHCbDIRAC, to decouple the tools from Physics Applications environment, in particular for what concerns the version of Python used (different versions of the Physics Application stack will use different versions of Python that cannot be supported at the same time).

Working only with Python it becomes natural to use standard Python packaging and deployment tools. So we designed the new LHCb User Environment in terms of an ecosystem of Python packages that can be installed with the pip command. Using standard Python tools, gives also access to the wide collection of libraries and tools available in the Python Package Index.

The main packages of the new LHCb User Environment are:

Several other packages can be found in the LHCb Python Package Index (mirrored to the official https://pypi.org), and more are added when needed.

Since the versions of Python on SLC6 and CentOS7 are obsolete (in particular that of SLC6), we decided to also decouple ourselves from the system version of Python on those platforms. To do that we rely on virtualenv to create self-contained Python deployments that include a custom build of Python (taken from LCG builds). Such deployment kits are produced for the two main platforms (x86_64-slc6 and x86_64-centos7) in two flavours: stable (updated only with bug fixes and small additions) and unstable (updated to the latest version of the packages), and installed on CVMFS. To limit the risk of disruption to the users, any update to the stable kit is installed to CVMFS under the alias testing, and it is promoted to stable only after some validation (similar to the dev and -prod_ mechanism used for LbScripts).

It must be noted that LHCbDIRAC will only require the LbEnv package and will not rely on the installation on CVMFS, but it will deploy it's own copy of the specific version that was certified, allowing for more dynamic evolution of the tools, without fear to break productions running on the Grid.

To help users to initialize the environment we provide a few entry points in /cvmfs/lhcb.cern.ch/lib that can be used to initialize the LHCb User Environment when LHCbCERNGroupLogin is not enabled or available. For example (for both bash and tcsh):

source /cvmfs/lhcb.cern.ch/lib/LbEnv
source /cvmfs/lhcb.cern.ch/lib/LbEnv-unstable

For technical reasons we cannot support SLC5, with the new tools, but the entry points still work and delegate to a working version of LbLogin/LbScripts.

Use Cases

Here follow instructions for most common use cases

Regular User

By "regular user" we mean any user that needs to work in the LHCb environment, but not to develop the support tools like lb-run, lb-dev, etc.

Supported OSs (SLC6, CentOS7)

For technical reasons, SLC5 cannot be supported by the new tools, so we only support SLC6 and CentOS7.

For a supported OS with a working /cvmfs mount (e.g. lxplus.cern.ch or lxplus7.cern.ch), you have to disable the default login environment by creating the file ~/.nogrouplogin:
touch ~/.nogrouplogin
and log in again to the machine.

Then set the environment up:

source /cvmfs/lhcb.cern.ch/lib/LbEnv
or, to test the upcoming new features:
source /cvmfs/lhcb.cern.ch/lib/LbEnv-dev

Note: the LbEnv script works for both bash and tcsh.

Local Installation
If running on an SLC6 or CentOS7 machine that does not have a working /cvmfs (e.g. institute local cluster), it's possible to bootstrap a local installation by first installing the environment support tools.

Prerequisites are:

  • working Python 2.7 in the path (system Python on CentOS7 is good enough)
  • a working virtualenv that uses Python 2.7 (available on lxplus7.cern.ch)

To install the tools in /opt/LHCbSoft/env-tools do

cd /tmp
curl -O https://gitlab.cern.ch/lhcb-core/lbenv-deployment/raw/master/basic_install.sh
bash basic_install.sh /opt/LHCbSoft/env-tools
Optionally one can require the dev versions of the packages (instead of the stable prod ones) adding the argument dev to the call to basic_install.sh:
bash basic_install.sh /opt/LHCbSoft/env-tools-dev dev
(note that prod and dev versions can be installed alongside each other).

Then, to set up the environment, invoke:

source /opt/LHCbSoft/env-tools/bin/LbEnv.sh --siteroot /opt/LHCbSoft
where the option --siteroot is used to specify where LHCb Physics Software should be installed with lbinstall.

Note: When bootstrapping a local installation, the environment produced by LbEnv will be basically empty. After the installation of Physics Software packages, you will have to start from a fresh environment and call again LbEnv.sh (or unset LBENV_SOURCED). See lhcb-core/LbEnv#5.

Non Supported OSs (Ubuntu, Windows, MacOS)

For non supported OSs, the best option is to use Docker containers.


  • a working Docker installation that can be used (docker run) by unprivileged users

A working /cvmfs is not mandatory but it's a strongly encouraged option, since CVMFS is available for several platforms.

With a working /cvmfs installation, it's enough to start the LHCb Docker container with
/cvmfs/lhcb.cern.ch/lib/bin/lb-docker-run --lbenv

If you need to be able to request a grid proxy within the container, extend this to:

/cvmfs/lhcb.cern.ch/lib/bin/lb-docker-run --lbenv --volume $HOME/.globus:/root/.globus:ro

If you need to use git commands (or things that hide git commit commands, like lb-dev or git lb-use) then extend to:

/cvmfs/lhcb.cern.ch/lib/bin/lb-docker-run --lbenv --volume $HOME/.globus:/root/.globus:ro --volume $HOME/.gitconfig:/root/.gitconfig:ro
Without CVMFS
If you do not have a working /cvmfs, you can still use the LHCb Software via docker, but it will be slow because the CVMFS cache will be discarded when exiting from the container.

In any case:

curl -O https://gitlab.cern.ch/lhcb-core/LbDocker/raw/master/scripts/lb-docker-run
chmod a+x lb-docker-run
./lb-docker-run --lbenv

Core Developers

With Core Developers we mean developers of the support tools like lb-run, lb-dev, etc. (both main developers of the packages and developers wanting to debug/fix/improve the tools).

Design principles

Python is the language for all LHCb tools and environment script. It was therefore decided to provide them as PyPI packages distributes on LHCb's own server, as described in https://twiki.cern.ch/twiki/bin/view/LHCb/PythonPyPIServer. This approach has the advantage to ease the use of external tools, which can just be installed using pip alongside LHCb's own modules.


Local Installation
Developers may use a local installation of the various Python packages following the instructions for regular users.

Clone Shared CVMFS Installation Locally
The installations of the scripts on CVMFS feature a script that can be used to bootstrap a local installation with the same version of Python and of the packages installed, very useful to debug problems:
$VIRTUAL_ENV/clone_venv /path/to/workspace/my-lbenv
or, if you are not in an LbEnv environment, with something like this:
/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/prod/x86_64-centos7/clone_venv /path/to/workspace/my-lbenv

Development Workflow

This section showcases a way to have a development environment from scratch

Using Visual Studio Code
First of all, you will need an installation of Visual Studio Code on your computer

Then, once you open Visual Studio Code, you will have to install the ssh extension in order to be able to connect to LXPLUS


Once you have that, you can open a new VSCode window through the button on the bottome left-hand corner. ssh-button.png This window will be used during your remote connection.

Then, on the pop-up, you have to click on (or type in the text field) "Remote-SSH: Connect to Host..."


You press on "Add new host" and you type your cern username (e.g. mine is gtsalama, you probably have something similar) @lxplus.cern.ch. So, for me, it is gtsalama@lxplusNOSPAMPLEASE.cern.ch, for you it is something different of that nature


Once you have done all of that, a new VSCode window pops up, asking for your password. You type it in and voila! You have now ssh'd to LXPLUS through VSCode and if you open a terminal inside that window, you will be seeing something similar to this ssh-connected.png

Note: Sometimes, VSCode fails to connect and shows this error


This is normal, just retry.

Now that you have succesfully logged in to LXPLUS, by default, you are using shared installations of some software (e.g. python). In order for you to not interfere with these pieces of software and for these pieces of software to not interfere with you, it is strongly recommended to configure your system so that it disables the use of that software. To do that, the only thing you need to do is to create an empty '.nolbenv' file in your home directory (just type 'touch .nolbenv' and you are good to go). You can also find more information about all this here (https://twiki.cern.ch/twiki/bin/view/LHCb/LHCbGroupLogin). Now, in order for the changes to work, you need log out and back in again. The easiest way to do that is by reloading your window

After that, let's say you want to start working on some project. First thing you need to do is clone it on LXPLUS. It is up to you in which directory you decide to clone your project. You found your project and want to clone it. Using ssh to clone it will not work out of the box though, you have not set your keys up yet, so it needs some extra steps. Instead, you can either use https or krb5 which both work without having to do anything extra.

Now, you have your project cloned, you are connected to LXPLUS and are almost ready to go. We now need to set up the configuration for pip and we need to set up a virtual environement.


If you want to open a folder through vscode and while using ssh, just go to file -> open folder and it gives you the option to select folders from your ssh workstation

Pip configuration:

the pip configuration is not mandatory but it is helpful, so it is recommended that you create $HOME/.config/pip/pip.conf and write this inside this file:

index-url = https://lhcb-repository.web.cern.ch/repository/pypi/simple
extra-index-url = https://pypi.org/simple

format = columns

Virtual Environments:

During normal development you will probably be fine with a python3 development environment. To create this virtual environment, type 'python -m venv venv3' where venv3 stands for "Virtual environment that is using python3"

But there are times where you will need to debug. Let's say there is an error that you want to reproduce, you can't be using a different environment than the one the error occured on as there might be inconsitencies on the findings. So, if the error occured on a centos7 environemnt, you should create a virtual environment that would use the same versions as the centos7 one. You can find these environments at "/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing" already, so, if you want to use a centos7 environment, you should type '/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/testing/x86_64-centos7/clone_venv venv2' and that will clone the centos7 environment into your venv2 environemnt (centos7 has python2 installed and that is why we have the number 2 after venv).

A virtual environment is not always active by default, so you may need to do it manually. To do so, (in bash) type 'source path_to_your_virutal_environment/bin/activate'. You can also deactivate it by calling deactivate in the same manner

ALERT! For virtual environments that are created inside a project, we currently have the convention of naming them either venv, venv2 or venv3. Please try to follow the convention or speak with your colleagues about a potential change/expansion of the convention ALERT!

Now, that everything is done, you can go to your project directory, type 'pip install -e .' and you can now normally work.

Edit | Attach | Watch | Print version | History: r22 < r21 < r20 < r19 < r18 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r22 - 2022-10-05 - DanielJohnson
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback