Operational Procedures for CMS Sites

Our tools and hardware are constantly changing, while we better understand out needs as we enter the steady state data taking period, so do not be (too much) suprised if you find an old information here, but kindly fix it (that's why this is is a collaborative space) or report it to Facilities Operations.

T2 resource overview

• HS06: https://hepix.caspur.it/benchmarks/doku.php?id=bench:results

Hardware Setup

To provide sufficient resources to CMS, a site should be able to provide about 4000 HS04 of processing resources.

In terms of WAN connectivity a dedicated link of at least 1 Gb/s is required, in order to insure acceptable transfer rates

• Notes: The hardware configuration above is the minimum required to run the CMS application. Sites upgrading or buying new hardware should consider the following.
  • LAN: The CTDR LAN requirement is 1MB/s per core to support a nominal simulation and analysis applications. There is a large variation in the IO needs of some applications and the amount of data delivered from the storage system varies by implementation. The CPU efficiency on average in the cluster may be higher with an improved LAN connection to the worker node. For new installations and upgrades CMS recommends a gigabit port per worker node.
  • Local Disk: The nominal CMS file size is 5-10GB. The applications writes the completed file to local disk space before staging to grid enabled storage. The nominal file size is an average and the tails of the distribution extend in both directions. Additionally, for some applications the CPU efficiency can be improved by staging the input data into local disk space. To maintain the flexibility of handing larger files and input data, sites upgrading systems or with new installations are recommended to have 20GB per core.

CMS tutorials for sites

• CERN, 15 June 2006 | Mumbay, 3 Dec 2006 | Taipei, 31 Oct 2007

Installation

• Start here to setup a new site
• Middleware recommendations:
  • EMI: https://twiki.cern.ch/twiki/bin/view/LCG/WLCGBaselineVersions
  • OSG: use the latest release
• Frontier Squid
• CMS.PhEDEx ( Setting up a brand new site PhEDEx site )
• TrivialFileCatalog and SiteConfigurationFile
• T2AdminGuide (CPU, Storage, name space, all that... a must read)
• StorageConfiguration (permissions, name space, practical tips ...)
• CMSSW
  • From April 2014 on CMS requires software distribution at Grid sites through CVMFS
  • The CMS specific instructions give further details regarding the setup
  • Further questions can be directed to the HN forum hn-cms-softwareDistrib@cernNOSPAMPLEASE.ch or submit a GGUS ticket and select "CMS SW Deployment" for "CMS Support Unit" and "CMS Facilities" in the Type of Problem area
  • Still possibe for local installations but not recommended
    • Instructions for local install by yourself
• UI installation for usage with CRAB 2.3.0 or above
• t1access role is not required/used any longer - just for reference: Configuration of t1access role at Tier-1 sites

Operation topics

• Grid acceptable use policy for CMS users
• DMWM policies
• StoreUser area to be setup at each T2 to support local users
• Mini-Guide to setting up dCache permissions at T2's (from Giacinto Donvito at Bari.infn.it)
  • note that there are two aspects about data access permissions
    • Restrict write access to production data: you MUST do this
    • Restrict/control write access to user's area (/store/user) : this is for your convenience
• Taking as reference the Giacinto's dCache gPlazma1 configuration the Swiss T3 wrote and made available its Ldap to gPlazma2 Python tool, contact them to get further details; again in the dCache context, they created these PostgreSQL views and functions to simplify the dCache understanding and its administration.

Changes / Maintenance ...

• HowToChangeSEName
• What To Do if there is a data loss

FAQ and Common Problems

• Site setup
  • SiteBatchSetup
• Networking and transfers
  • DDTCommonProblems
• SAM test failures

Scratchpad

• GettingRidOfCMS_PATH

-- StefanoBelforte - 29 Jan 2008