Operational Procedures for CMS Sites

Our tools and hardware are constantly changing, while we better understand out needs as we enter the steady state data taking period, so do not be (too much) suprised if you find an old information here, but kindly fix it (that's why this is is a collaborative space) or report it to Facilities Operations.

T2 resource overview

Resource Nominal Recommended Tolerated
CPU 10.9 kHS06 5 kHS06 4 kHS06
Disk 810 TB 400 TB 300 TB
Network 10 Gbps 1 Gbps 1 Gpbs

Hardware Setup

Worker Node Setup CPU Operating System RAM local disk LAN
Minimum Required 64 bit SL6 2.0 GB / core 20 GB /core aggregated 1MB/sec/core

To provide sufficient resources to CMS, a site should be able to provide about 4000 HS04 of processing resources.

Storage Setup Local Protocol Wide Area Protocol Storage Capacity Performance
  Rfio, xrootd, dcap, file* SRM-v2* 400 TB 1MB/s * the total number of batch cores for sequential IO
*CMS Requires the file to be openable with a local protocol supported by Root, and will transfer files to the cluster using SRM

In terms of WAN connectivity a dedicated link of at least 1 Gb/s is required, in order to insure acceptable transfer rates

  • Notes: The hardware configuration above is the minimum required to run the CMS application. Sites upgrading or buying new hardware should consider the following.
    • LAN: The CTDR LAN requirement is 1MB/s per core to support a nominal simulation and analysis applications. There is a large variation in the IO needs of some applications and the amount of data delivered from the storage system varies by implementation. The CPU efficiency on average in the cluster may be higher with an improved LAN connection to the worker node. For new installations and upgrades CMS recommends a gigabit port per worker node.
    • Local Disk: The nominal CMS file size is 5-10GB. The applications writes the completed file to local disk space before staging to grid enabled storage. The nominal file size is an average and the tails of the distribution extend in both directions. Additionally, for some applications the CPU efficiency can be improved by staging the input data into local disk space. To maintain the flexibility of handing larger files and input data, sites upgrading systems or with new installations are recommended to have 20GB per core.

CMS tutorials for sites


Operation topics

  • Grid acceptable use policy for CMS users
  • DMWM policies
  • StoreUser area to be setup at each T2 to support local users
  • Mini-Guide to setting up dCache permissions at T2's (from Giacinto Donvito at Bari.infn.it)
    • note that there are two aspects about data access permissions
      • Restrict write access to production data: you MUST do this
      • Restrict/control write access to user's area (/store/user) : this is for your convenience
  • NEW Taking as reference the Giacinto's dCache gPlazma1 configuration the Swiss T3 wrote and made available its Ldap to gPlazma2 Python tool, contact them to get further details; again in the dCache context, they created these PostgreSQL views and functions to simplify the dCache understanding and its administration.

Changes / Maintenance ...

FAQ and Common Problems


-- StefanoBelforte - 29 Jan 2008

Edit | Attach | Watch | Print version | History: r45 < r44 < r43 < r42 < r41 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r45 - 2016-05-09 - StefanoBelforte
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback