Using FIU T3 Computing Resources

Submitting CRAB jobs to FIU Farm from LPC (or outside FIU)

To send your CRAB jobs directly FIU and not anywhere else, one need to "WHITELIST" the FIU-T3. In your crab.cfg file:

jobtype = cmssw
#--- scheduler
scheduler = glite

 ## By ComputingElement

Use of the [CRAB] option

is not necessary.

CRAB Submission from FIU nodes

Follow these step to setup the environment.

Attachment xxx can be used to simplify job creation and submission.

Condor submission

Login into FIU

All access to FIU is via the gatekeeper Once you have setup a user account (see below), you can login via ssh (ssh Once you are in, you can hop to any other node using your ssh-key.

Warning, important Please do not run job on this node and this may severely affect web access and other users.

Accessing CERN CVS

Checking out CMMSW Release

To get access to CERN CVS, here is procedure:

   1. login in to medianoche
   2.  source /osg/apps/cmssoft/cms/fiucmsenv.csh
   3. cmsrel CMSSW 5_2_5
   4. cd CMSSW_5_2_5/src/
   5. cmsenv
   6. cvs login
      . (password = 98passwd # should only need to do this once
   7. addpkg ... 

Checking in User code to CERN CVS

This is still not straight forward. I had to manually change the CVS/Root to
. This allowed me to commit changes to CVS.

If you are inside a CMMSW release, then you may not need to set CVSROOT manually as it will be pciked up from the CVS directory sitting in the working directory. But you need to have valid CERN Kerberose ticket. Execute kinit -l1d your-user-name@CERN.CH to get CERN ticket. The CAPITAL letter are important as they refers to specific kerberose realm. (If you wanto login to a FNAL LPC node which require a Kerberose ticket from FNAL, you need to do kinit -l1d your-user-name@FNAL.GOV).

Storage Space

We use a file storage system very similar to dCache and functions almost the same called hadoop system. Current this file system is visible only to Also the hadoop is now visible to all T3 nodes so your condor jobs should run smoothly. Your large datasets can and should be stored here if your jobs need to access them. Make sure to keep your datasets here if your jobs need to access them. Please do not access large datasets via medianoche which will have many adverse effects.

User Storage Space

You can use /share/store/users/ area under medianoche to keep all you code etc.

Outputting to Storage Element

(I have not tested copying directly to SE yet. Condor jobs should be able to write directly to haddop.)

Datasets Available at FIU

You can have datasets physically available at FIU PheDEx by creating a formal request using this web form. It will be forwarded to approval of Jorge (and other experts). Given there is enough space, your request will be complete in ~days. Currently we have about 40TB of space and will be adding more soon.

The actual physical location of the PheDEx dataset can be accessed via srm node (ssh located in the directory /mnt/hadoop/cms/store/ . For MC datasets /mnt/hadoop/cms/store/mc .

Datasets available at FIU can be found via Data Aggregation Service (DAS). Clicking here will show you all available datasets at FIU.

Requesting a User Account

If you want to submit condor jobs you need to login to a FIU node. For this you need a personal account at FIU. Please contact Jorge or me with your preferred user name.

You'll also need to generate a ssh key-pair to login to FIU nodes. We do not support kerberose access yet. Please forward your public ssh-key with your email.

Below is some instructions on generating a ssh key-pair.

On the local machine, type the BOLD part. The non-bold part is what you might see as output or prompt.

Step 1:
% *ssh-keygen -t dsa*
Generating public/private dsa key pair.
Enter file in which to save the key (~/.ssh/id_dsa): (just type return)
Enter passphrase (empty for no passphrase): (just type return)
Enter same passphrase again: (just type return)
Your identification has been saved in ~/.ssh/id_dsa
Your public key has been saved in ~/.ssh/
The key fingerprint is:
Some really long string
Step 2:
Then, paste the content of the local ~/.ssh/ file into the file ~/.ssh/authorized_keys on the remote host.
RSA instead of DSA
If you want something strong, you could try
% ssh-keygen -t rsa -b 4096
Instead of the names id_dsa and, it will be id_rsa and, etc.
The rest of the steps are identical. 

That's it!


Q: I follow the exact steps, but ssh still ask me for my password!
A: Check your remote .ssh directory. It should have only your own read/write/access permission (octal 700)
% chmod 700 ~/.ssh

System Information Elog

I try to keep some additional and/or redundant information regarding various issues and solutions I came accros in System Info Elog. Using the search option with your keywords may provide some addition info of your problem. If you find a solution, please let us know so we can record it for others.

-- SamanthaHewamanage - 15-Oct-2012

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2012-12-13 - SamanthaHewamanage
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback