Tips on using the Grid

Getting ready with your Grid certificate

It is very likely that once you have obtain you grid certificate, you end up with a .p12 file (after you have exported your grid-certificate from your web browser). I copy here the instruction suggested CERN-CA (How to use your certificate with grid-proxy-init ):

openssl pkcs12 -in myGridCert.p12 -clcerts -nokeys -out $HOME/.globus/usercert.pem
openssl pkcs12 -in myGridCert.p12 -nocerts -out $HOME/.globus/userkey.pem
chmod go-rw $HOME/.globus/userkey.pem

Copying files using srmcp (SRM Storage Resource Management)

There are many ways to copy files across the Grid. I describe here an example on using srmcp:

  • get on lxplus (or the UI) and set your grid environment: source /afs/
  • initiate a voms proxy: voms-proxy-init -voms cms
  • issue the command: srmcp options source target

If you type srmcp --help you will get a full description on the available options (help-srmcp.txt).

The options are the key point. We have to use SRM V.2 and tell srmcp to use a script that comes with the distribution (

-srm_protocol_version=2 -use_urlcopy_script=true -urlcopy= ${SRM_PATH}/sbin/

  • Pointing hand Example 1: if you want to copy a file located at CERN ( srm:// ) you will do:

<lxplus231> srmcp -srm_protocol_version=2 -use_urlcopy_script=true -urlcopy=${SRM_PATH}/sbin/ \
srm:// \

  • Pointing hand Example 2: if the file is at FNAL ( srm://\?SFN= ):

<lxplus231> srmcp -srm_protocol_version=2 -use_urlcopy_script=true -urlcopy= ${SRM_PATH}/sbin/ \
srm://\?SFN=/2/rebassoo/JR/simulation-4mu-expoPt-30-0-200-1000-72.root \

  • Info This last command has been tested in our Tier-3

Target From FNAL to CERN

  • Pointing hand Example 3: from local FNAL to CERN Castor

You will first to configure Castor so it is able to receive the files. You need to create a special directory, call it gridfiles in your personal area and give it some special permissions:

<lxplus231> nsmkdir /castor/
<lxplus231> nschmod 775 /castor/

Now, you should be able to copy files across using the srmcp tool as described before:

<cmslpc07> srmcp -srm_protocol_version=2 -use_urlcopy_script=true -urlcopy= ${SRM_PATH}/sbin/ \
file:////uscms_data/d1/rebassoo/LM1SusyTree/job/LM1_SUSY_Tree.root \

Target From CERN to Uniandes

  • Pointing hand Example 4: from local CERN to Uniandes

You will first to configure your user area at the Tier-3 Uniandes so it is able to receive the files. You need to create a directory, for instance call it gridfiles in your personal area and give it some special permissions (you need first to start a grid-proxy):

<lxplus231> rfmkdir /dpm/
<lxplus231> rfchmod 775 /dpm/

  • Now, you should be able to copy files across using the srmcp tool as described before. We first try with a local file in your castor area:

<cmslpc07> srmcp -srm_protocol_version=2 -use_urlcopy_script=true -urlcopy= ${SRM_PATH}/sbin/ \
file:////afs/ \

Things would work silently. You can then list the contents of your remote area:

<lxplus209> srmls --count=10 srm://\?SFN=/dpm/
WARNING: SRM_PATH is defined, which might cause a wrong version of srm client to be executed
  0 /dpm/
      0 /dpm/
      4166 /dpm/
      4166 /dpm/
      0 /dpm/

Note the "--count=10 option": this tells srmls a maximum number of lines to display.

  • Pointing hand Example 5: from your CERN Castor area to Uniandes

The same principle applies here. However this time you will need use the SRM protocol at both ends:

<lxplus209> srmls srm://
WARNING: SRM_PATH is defined, which might cause a wrong version of srm client to be executed
  4166 /castor/

For the moment this does not work ... sorry.

  • Pointing hand Example 6: from your Tier-2 to Uniandes

lxplus230> ./ susy_PatLayer1_LM0.txt 

Copying files: 

srmcp -srm_protocol_version=2 -use_urlcopy_script=true -urlcopy= ${SRM_PATH}/sbin/ \
srm://\?SFN=/pnfs/ \

this procedure seems to be working fine!

Places to look for data

The logical file name is not the complete story, and if you want to perform a srmls command, there is usually some path you have to prepend to the lfn. Here is a list of some of the sites and their particular path where the actual files are located.

srm:// path to data (examples)\?SFN= /dpm/\?SFN= /2/ and others\?SFN= /castor/\?SFN= /pnfs/\?SFN= /pnfs/\?SFN= /dpm/\?SFN= /pnfs/\?SFN= /pnfs/\?SFN= //cms\?SFN= /storage/data/cms

Copying files using LCG commands

The following is a brief description on using the LCG commands in order to replicate / copy files to our Tier-3 in Bogota. This was done with the kind help from Michael Perez our storage expert and suport in Uniandes (thanks). The first thing you need to do is to make sure you have the following environment variables at your UI:

[michael@yali ~]$ echo $LFC_HOST; echo $LCG_GFAL_INFOSYS; echo $LCG_CATALOG_TYPE;

  • Create then a valid grid proxy for this session. We will use the command lcg-cr (have a look to the LCG-2 manual pages or just type lcg-cr to get some information):

[aosorio@yali ~]$ lcg-cr --vo cms -l LFN -d TURL

where LFN is the logical file name that YOU will assign to your file. TURL is the URL to your file using a Transport protocol. In order to get this, you will need to get this information using the command lcg-gt:

  • Pointing hand Example 1: the goal here is to copy files stored at the Aachen Tier-2 to our Tier-3:

lcg-gt \
srm://\?SFN=/pnfs/ gsiftp

gsiftp is the transport protocol to be used. lcg-gt read a SURL (that points to the path to this file on the Grid) and returns something that starts with gsiftp.

  • Now we got what we need:

[aosorio@yali ~]$ lcg-cr --vo cms -l  lfn:/grid/cms/test_lm0_tier3 -d \
> gsi
[BDII] Warning, no GlueVOInfo information found about SE '' (with no tag)

"" is our Storage Element (ignore the warning for the moment). That's all! depending on the size of the file, the system will hang in there until the file has been copied and registered. When it finishes, you can check if the file has been registered:

[aosorio@yali src]$ lcg-lr --vo cms lfn:/grid/cms/SUSY_LM0_sftsht_RECO_V11_1

  • You could also check by using CMSSW edmFileUtil:

[aosorio@yali src]$ edmFileUtil  -d /grid/cms/SUSY_LM0_sftsht_RECO_V11_1

  • Finally, I created the following script (it is sort of case specific) to copy a full data set: replicate files using LCG cmds

CMS Sites Information

A very useful command is lcg-infosites. This command allows to see information on the CE,SE associated to CMS virtual organisation.

  • examples of usage:

<lxplus238> lcg-infosites --vo cms ce se

Using lcg-infosites --vo cms closeSE one can have a list of CE and their nearest SE. I attached in here the current Sept 06 2009 result of such command.

Topic attachments
I Attachment History Action Size Date Who Comment
Texttxt CMS-infosites-closeSE.txt r1 manage 27.5 K 2009-09-06 - 15:28 AndresOsorio Close SE to CE list - Sept 06 2009
Texttxt help-srmcp.txt r1 manage 6.4 K 2008-12-18 - 11:04 AndresOsorio srmcp --help output
Texttxt r1 manage 2.2 K 2009-05-17 - 18:57 AndresOsorio replicate files using LCG cmds
Edit | Attach | Watch | Print version | History: r21 < r20 < r19 < r18 < r17 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r21 - 2010-08-25 - AndresOsorio
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback