Note on site-launched startd's

For a trusted resource, the site should create a pilot certificate from a CA recognized at CERN and send the DN to the Submission Infrastructure group ( who will enter the certificate DN in the condor_mapfile of the central manager, etc. In special circumstances, the Submission Infrastructure group can provide a CMS pilot certificate.

Creating Pilot and Service Certificates

Every grid job, be it a glideinWMS pilot (glidein) or a user job, needs a grid proxy in order to authenticate at sites. A proxy is created on the frontend from a grid certificate and a key. This page describes how to obtain new pilot certificates from the CERN Certificate Authority.

Creating a New Certificate

On the CERN Certificate Authority website, go to "New Grid Host Certificate" and choose "Request certificate using OpenSSL (for Linux machines)". Create a certificate request with a subject, for example:

You need to be an owner of the machine at CERN in the certificate request name. You should use a certificate-friendly browser like Firefox if you want to not make your life difficult.

You will next be asked to generate a certificate-key pair. You can do this on lxplus:

openssl req -new -subj "/CN=cmspilot01\/" -out newcsr.csr -nodes -sha512 -newkey rsa:2048
or follow the updated instructions on the CERN CA webpage if they have changed. Once the certificate is generated, download the base64 certificate to a file, which we will call host.cert.

Next we will export the certificate from this file to a p12 file:

openssl pkcs12 -export -inkey privkey.pem -in host.cert -out ${dir}.p12

Create cert file and key file from the p12 file and set permissions correctly:

openssl pkcs12 -clcerts -nokeys -in ${dir}.p12 -out ${dir}cert.pem
openssl pkcs12 -nocerts -in ${dir}.p12 -out ${dir}key-enc.pem
openssl rsa -in ${dir}key-enc.pem -out ${dir}key.pem
chmod 400 ${dir}key*
chmod 600 ${dir}cert.pem

Next verify the cert and key have the same hash:

openssl x509 -noout -modulus -in ${dir}cert.pem | openssl md5 
openssl rsa -noout -modulus -in ${dir}key.pem |openssl md5 | uniq
This should always be the case unless you did something really incorrect like mix the files from two different certificates.

Lastly, get some information about the certificate like its Distinguished Name (DN) subject, and its validation period:

openssl x509 -in ${dir}cert.pem -noout -subject -startdate -enddate

Register the Certificate with CMS

For service certificates (i.e. for the frontend), you can skip this step.

"Lasciate ogne speranza, voi ch'intrate" - Dante Alighieri

In order to use CMS resources, every pilot certificate must be registered with the CMS VOMS group as a member of CMS, as well as get special roles like the pilot role. This distinguishes pilot jobs from other types of jobs like user jobs. Follow the procedure to "Add an additional certificate" on the CMS VOMS administration website ( direct link may work, if it does not, use the previous link and scroll down about a page to find the button on the right side) VOMS administrators are Stefano, Andreas Pfeiffer and Tony, if you need speedy approval. Note also that as of Summer 2015 the Global Pool certificate needs also the production role since fair-share at the Tier-1 sites depends on having it.

Collector Authorization

As user condor, authorize the new certificate on the collectors:

glidecondor_addDN -daemon "cms pilot cert DN" "/DC=ch/DC=cern/OU=computers/CN=cmspilot12/" pilot12
Each glidein needs to communicate with the collector, and cannot do so without authorization in the condor_mapfile. Never update the condor_mapfile by hand! Use the script glidecondor_addDN.

Certificate Repository

There isn't one.

Creating a Proxy

You should store the certificate and key files on the glideinWMS frontend. Next you will have to create proxies. The proxies are short-lived versions of the certificate information that can be used more safely on the grid. In the Global Pool, these are found in, and the certificate and key files in the certs directory below that. The fundamental procedure boils down to:

Create the base proxy:

voms-proxy-init -cert ${certdir}/cmspilot${idstr}cert.pem -key ${certdir}/cmspilot${idstr}key.pem -hours 72

Add the appropriate VOMS roles to the base proxy, for example.

export X509_USER_PROXY=${here}/x509_pilot${idstr}_cms.proxy.tmp
voms-proxy-init -cert ${certdir}/cmspilot${idstr}cert.pem -key ${certdir}/cmspilot${idstr}key.pem -voms cms -hours 72 -valid 72:0
See the script for the full details. This script is run as a cron job. The resulting proxy files can be used in the glideinWMS frontend configuration xml file.

List of Certificates in Production

Next expiration date: April 25, 2018

Partial DNSorted ascending Use Expires
cmspilot01/ UCSD pilot certificate Apr 25 19:17:56 2018 GMT
cmspilot02/ Global Pool pilot certificate Apr 25 19:20:58 2018 GMT
cmspilot03/ Development pool pilot certificate Jun 27 22:35:42 2018 GMT
cmspilot04/ Global pool ITB pilot certificate Jun 27 22:46:08 2018 GMT
cmspilot05/ Tier0 pool pilot certificate Jun 27 22:48:19 2018 GMT
frontend01/ UCSD frontend certificate Apr 25 19:09:51 2018 GMT
frontend02/ Global Pool frontend certificate Apr 25 19:13:54 2018 GMT
frontend03/ Development pool frontend certificate Jun 27 22:49:50 2018 GMT
frontend04/ Global pool ITB frontend certificate Jun 27 22:51:17 2018 GMT
frontend05/ Tier0 pool frontend certificate Jun 27 22:52:52 2018 GMT
tw/ CRAB3 TaskWorker certificate Jun 30 01:30:44 2018 GMT

There is currently an error in renewing the CRAB3 TaskWorker certificate (May 23, 2017).

Script to simplify certificate creation (July 2016, last rev. March 2018) for all certificates at once.

DIRS="cmspilot01  cmspilot03  cmspilot05  frontend02  frontend04 cmspilot02  cmspilot04  frontend01  frontend03  frontend05 tw"

for dir in $DIRS ; do
  if [ ! -d $TOP/$dir ] ; then
    mkdir $TOP/$dir
  cd $TOP/$dir
  if [ ! -f host.cert ] ; then
    if [ $dir != "tw" ] ; then
      openssl req -new -subj "/CN=${dir}\/" \
        -out newcsr.csr -nodes -sha512 -newkey rsa:2048
      openssl req -new -subj "/CN=${dir}\/" \
        -out newcsr.csr -nodes -sha512 -newkey rsa:2048
    echo "Go to the following url and request the certificate with the"
    echo "information in newcsr.csr for certificate $dir/"
    echo ""
    echo "Download the base-64 certificate and then copy host.cert from"
    echo "your desktop to this area:"
    echo "Enter when prompted:" $dir
    openssl pkcs12 -export -inkey privkey.pem -in host.cert -out ${dir}.p12
    openssl pkcs12 -clcerts -nokeys -in ${dir}.p12 -out ${dir}cert.pem
    openssl pkcs12 -nocerts -in ${dir}.p12 -out ${dir}key-enc.pem
    openssl rsa -in ${dir}key-enc.pem -out ${dir}key.pem
    chmod 400 ${dir}key*
    chmod 600 ${dir}cert.pem
    openssl x509 -noout -modulus -in ${dir}cert.pem | openssl md5 
    openssl rsa  -noout -modulus -in ${dir}key.pem |openssl md5 | uniq


JamesLetts - 2018-03-20

Edit | Attach | Watch | Print version | History: r33 | r31 < r30 < r29 < r28 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r29 - 2018-03-21 - JamesLetts
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback