Making my Ubuntu 18.04/20.04 LTS work for Grid, CMSSW, CRAB, xroot

CVMFS is prerequisite for most things

xroot via anaconda

grid stuff, see also CrabOnUbuntu1204

  • install globus-proxy-utils via synaptic
  • install voms-clients via synaptic
  • install myproxy via synaptic
  • in my .bashrc
    export X509_CERT_DIR=/cvmfs/grid.cern.ch/etc/grid-security/certificates
    X509_VOMS_DIR=/cvmfs/grid.cern.ch/etc/grid-security/vomsdir
    export X509_USER_PROXY=/tmp/x509up_u`id -u`
    export X509_VOMSES=/cvmfs/cms.cern.ch/grid/etc/vomses
    export VOMS_USERCONF=/cvmfs/cms.cern.ch/grid/etc/vomses
    

  • alas myproxy-init fails with some openss dependency error. But after delegation is done from lxplus, things are OK.

CRAB3 client

  • depends on CMSSW see below

CMSSW

  • works via singularity, see below

CRAB w/o Singularity

singularity (i.e. a CC7 container) is needed to run binaries (cmsRun) and to compile CMSSW code. But scram is perl and works also on Ubuntu. So if I copy a CMSSW

all I need is this to make pycurl works, now that I got rid of openssl calls in Crab client,

#install pycurl
pip install pycurl
# map the Ubuntu/Debian location of the cert bundle to the RedHat location so pyucurl is happy
#  ref: https://askubuntu.com/a/386354/603971
sudo ln -s /etc/ssl/certs/ca-certificates.crt /etc/pki/tls/certs/ca-bundle.crt

and then I can run CRAB Client directly from my git clone by sourcing /home/belforte/WORK/CRAB/SETUP-HERE in any new shell. The content of SETUP-HERE is:

#!/bin/bash
source /cvmfs/cms.cern.ch/cmsset_default.sh
cd /home/belforte/STEFANO/WORK/CRAB/CMSSW/CMSSW_10_6_12
cmsenv
cd -

MY_CRABClient=/home/belforte/CRAB/CRABClient
MY_WMCORE=/home/belforte/CRAB/WMCore
MY_CRABServer=/home/belforte/CRAB/CRABServer
MY_DBS=/home/belforte/CRAB/DBS

export PYTHONPATH=${MY_DBS}/Client/src/python:${MY_DBS}/PycurlClient/src/python:$PYTHONPATH
export PYTHONPATH=${MY_WMCORE}/src/python:$PYTHONPATH
export PYTHONPATH=${MY_CRABClient}/src/python:$PYTHONPATH
export PYTHONPATH=${MY_CRABServer}/src/python:$PYTHONPATH

export PATH=${MY_CRABClient}/bin:$PATH
source ${MY_CRABClient}/etc/crab-bash-completion.sh

SINGULARITY

as per http://cms-sw.github.io/singularity.html
  • Ubuntu 18.04: install singularity-container via synaptic
  • Ubuntu 20.04: install golang via synaptic and build singularity from source as per https://sylabs.io/guides/3.7/admin-guide/installation.html#installation-on-linux
  • do not try to bind /afs nor /eos, just do
    export SINGULARITY_CACHEDIR="/tmp/$(whoami)/singularity"
    singularity shell  -B /cvmfs docker://cmssw/cc7:amd64
  • this alias in .bashrc helps
    alias SINGULARITY='singularity -s shell  -B /cvmfs docker://cmssw/cc7:amd64 --login'
    

But actually this also works and if faster:
/cvmfs/cms.cern.ch/common/cmssw-cc7

  • this piece in .bashrc makes everything work
    # if inside singularity, setup for CMSSW
    if ! [ -z "$SINGULARITY_CONTAINER" ]
    then
      # CMS singularity image has scram stuff but apparently not the aliases
      alias cmsenv='eval `scramv1 runtime -sh`'
      alias cmsrel='scramv1 project CMSSW'
      # tweak $HOME definition so that /home/belforte/.local stuff does not get in the way
      # since stuff it containes Ubuntu things
      export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/belforte/myCC7/lib
      export HOME=/home/belforte/STEFANO/WORK      # so that /home/belforte/.local stuff does not get in the way
      cd $HOME
    fi
    
  • of course since $HOME is redined be careful of using commands which point to it or of the shortcut "~" usage in .bashrc etc. Better always used /home/belforte in various scripts
  • it is less confusing if the prompt is different inside the container, for this in .bashrc the lines which set the prompt should be put inside a if block, like this:
    # if sourcing withing a container, do not change current prompt
    if [ -z "$SINGULARITY_CONTAINER" ]
    then
      ..manipulate the prompt
    fi
    

-- StefanoBelforte - 2019-09-08

Edit | Attach | Watch | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r11 - 2020-12-31 - StefanoBelforte
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback