Note on site-launched startd's

For a trusted resource, the site should create a pilot certificate from a CA recognized at CERN and send the DN to the Submission Infrastructure group ( who will enter the certificate DN in the condor_mapfile of the central manager, etc. In special circumstances, the Submission Infrastructure group can provide a CMS pilot certificate.

Creating Pilot and Service Certificates

Every grid job, be it a glideinWMS pilot (glidein) or a user job, needs a grid proxy in order to authenticate at sites. A proxy is created on the frontend from a grid certificate and a key. This page describes how to obtain new pilot certificates from the CERN Certificate Authority.

Creating a New Certificate

On the CERN Certificate Authority website, go to "New Grid Host Certificate" and choose "Request certificate using OpenSSL (for Linux machines)". Create a certificate request with a subject, for example:

You need to be an owner of the machine at CERN in the certificate request name. You should use a certificate-friendly browser like Firefox if you want to not make your life difficult.

You will next be asked to generate a certificate-key pair. You can do this on lxplus:

openssl req -new -subj "/CN=cmspilot01\/" -out newcsr.csr -nodes -sha512 -newkey rsa:2048
or follow the updated instructions on the CERN CA webpage if they have changed. Once the certificate is generated, download the base64 certificate to a file, which we will call host.cert.

Next we will export the certificate from this file to a p12 file:

openssl pkcs12 -export -inkey privkey.pem -in host.cert -out ${dir}.p12

Create cert file and key file from the p12 file and set permissions correctly:

openssl pkcs12 -clcerts -nokeys -in ${dir}.p12 -out ${dir}cert.pem
openssl pkcs12 -nocerts -in ${dir}.p12 -out ${dir}key-enc.pem
openssl rsa -in ${dir}key-enc.pem -out ${dir}key.pem
chmod 400 ${dir}key*
chmod 600 ${dir}cert.pem

Next verify the cert and key have the same hash:

openssl x509 -noout -modulus -in ${dir}cert.pem | openssl md5 
openssl rsa -noout -modulus -in ${dir}key.pem |openssl md5 | uniq
This should always be the case unless you did something really incorrect like mix the files from two different certificates.

Lastly, get some information about the certificate like its Distinguished Name (DN) subject, and its validation period:

openssl x509 -in ${dir}cert.pem -noout -subject -startdate -enddate

Register the Certificate with CMS

For service certificates (i.e. for the frontend), you can skip this step.

"Lasciate ogne speranza, voi ch'intrate" - Dante Alighieri

In order to use CMS resources, every pilot certificate must be registered with the CMS VOMS group as a member of CMS, as well as get special roles like the pilot role. This distinguishes pilot jobs from other types of jobs like user jobs. Follow the procedure to "Add an additional certificate" on the CMS VOMS administration website ( direct link may work, if it does not, use the previous link and scroll down about a page to find the button on the right side) VOMS administrators are Stefano, Andreas Pfeiffer and Tony, if you need speedy approval. Note also that as of Summer 2015 the Global Pool certificate needs also the production role since fair-share at the Tier-1 sites depends on having it.

Collector Authorization

As user condor, authorize the new certificate on the collectors:

glidecondor_addDN -daemon "cms pilot cert DN" "/DC=ch/DC=cern/OU=computers/CN=cmspilot12/" pilot12
Each glidein needs to communicate with the collector, and cannot do so without authorization in the condor_mapfile. Never update the condor_mapfile by hand! Use the script glidecondor_addDN.

Certificate Repository

There isn't one.

Creating a Proxy

You should store the certificate and key files on the glideinWMS frontend. Next you will have to create proxies. The proxies are short-lived versions of the certificate information that can be used more safely on the grid. In the Global Pool, these are found in, and the certificate and key files in the certs directory below that. The fundamental procedure boils down to:

Create the base proxy:

voms-proxy-init -cert ${certdir}/cmspilot${idstr}cert.pem -key ${certdir}/cmspilot${idstr}key.pem -hours 72

Add the appropriate VOMS roles to the base proxy, for example.

export X509_USER_PROXY=${here}/x509_pilot${idstr}_cms.proxy.tmp
voms-proxy-init -cert ${certdir}/cmspilot${idstr}cert.pem -key ${certdir}/cmspilot${idstr}key.pem -voms cms -hours 72 -valid 72:0
See the script for the full details. This script is run as a cron job. The resulting proxy files can be used in the glideinWMS frontend configuration xml file.

List of Certificates in Production

Next expiration date: April 25, 2018

Partial DN Use Expires
cmspilot01/ UCSD pilot certificate Apr 3, 2020
cmspilot02/ Global Pool pilot certificate Apr 3, 2020
cmspilot03/ Development pool pilot certificate Apr 3, 2020
cmspilot04/ Global pool ITB pilot certificate Apr 3, 2020
cmspilot05/ CERN pool pilot certificate Apr 3, 2020
frontend01/ UCSD frontend certificate Apr 3, 2020
frontend02/ Global Pool frontend certificate Apr 3, 2020
frontend03/ Development pool frontend certificate Apr 3, 2020
frontend04/ Global pool ITB frontend certificate Apr 3, 2020
frontend05/ CERN pool frontend certificate Apr 3, 2020
tw/ CRAB3 TaskWorker certificate Apr 23, 2019

Script to simplify certificate creation (July 2016, last rev. March 2018) for all certificates at once.

HOST=`basename $TOP`

if [ $HOST == "vocms080" ] ; then
  DIRS="cmspilot01 cmspilot02 cmspilot03 cmspilot04 cmspilot05 frontend01 frontend02 frontend03 frontend04 frontend05"
elif [ $HOST == "vocms052" ] ; then

for dir in $DIRS ; do
  if [ ! -d $TOP/$dir ] ; then
    mkdir $TOP/$dir
  cd $TOP/$dir
  if [ ! -f host.cert ] ; then
    cp /etc/pki/tls/openssl.cnf .
cat >>openssl.cnf <<END
req_extensions = v3_req

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

    echo "DNS.1 = ${HOST}" >> openssl.cnf

    openssl req -new -subj "/CN=${dir}\/${HOST}" \
      -out newcsr.csr -nodes -sha512 -newkey rsa:2048 \
      -config ${TOP}/${dir}/openssl.cnf
    echo "Go to the following url and request the certificate with the"
    echo "information in newcsr.csr for certificate $dir/${HOST}"
    echo ""
    echo "Download the base-64 certificate and then copy host.cert from"
    echo "your desktop to this area:"
    echo "Enter when prompted:" ${HOST}-$dir
    openssl pkcs12 -export -inkey privkey.pem -in host.cert -out ${dir}.p12
    openssl pkcs12 -clcerts -nokeys -in ${dir}.p12 -out ${dir}cert.pem
    openssl pkcs12 -nocerts -in ${dir}.p12 -out ${dir}key-enc.pem
    openssl rsa -in ${dir}key-enc.pem -out ${dir}key.pem
    chmod 400 ${dir}key*
    chmod 600 ${dir}cert.pem
    openssl x509 -noout -modulus -in ${dir}cert.pem | openssl md5 
    openssl rsa  -noout -modulus -in ${dir}key.pem |openssl md5 | uniq


JamesLetts - 2019-02-27

Edit | Attach | Watch | Print version | History: r33 < r32 < r31 < r30 < r29 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r32 - 2019-02-28 - JamesLetts
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback