Generic File Monitoring

In order to better capture all data use activities within CMS, we have implemented general-purpose file monitoring within the CMSSW framework, starting in CMSSW_6_0_0.

Each time a file is closed, a single UDP packet is generated and sent to a monitoring destination. The information is roughly equivalent to the combination of the existing Frontier monitoring and the Framework Job Report, but is quite a bit easier to record centrally. This is also equivalent to the summary monitoring collected by Xrootd servers, except it applies to all protocols.

An example of the precise data recorded is copied below.

{
"site_name":"T2_US_Nebraska", 
"fallback": true,
"user_dn":"/DC=com/DC=DigiCert-Grid/O=Open Science Grid/OU=People/CN=Brian Bockelman",
"client_host":"brian-test",
"client_domain":"unl.edu",
"server_host":"cmsstor359",
"server_domain":"fnal.gov",
"unique_id":"0618354C-5680-E211-A78E-00163ED711AD-0",
"file_lfn":"/store/relval/CMSSW_6_1_0-GR_R_61_V6_RelVal_wEl2012C/SingleElectron/RECO/v1/00000/FAC88284-414C-E211-84E7-002618943947.root", 
"file_size":2198266271,
"read_single_sigma":630363,
"read_single_average":161060,
"read_vector_average":547555,
"read_vector_sigma":426840,
"read_vector_count_average":129.667,
"read_vector_count_sigma":181.255,
"read_bytes":4541744,
"read_bytes_at_close":4541744,
"read_single_operations":18,
"read_single_bytes":2899080,
"read_vector_operations":3,
"read_vector_bytes":1642664,
"start_time":1361911890,
"end_time":1361911910
}

Not all fields are available for all protocols. For example, user_dn is only filled if a grid proxy is present; server host is only filled in for Xrootd.

The monitoring currently defaults to OFF and requires the site admin to edit the site-local-config.xml to enable it and set the destination. We would like all sites to make this change. The additional line goes in the source-config stanza:

   <source-config>
     <statistics-destination name="cms-udpmon-collector.cern.ch:9331" />
   </source-config>

A full example of the site-local-config.xml is copied below.

<site-local-config>
<site name="T2_US_Nebraska">
   <event-data>
     <catalog url="trivialcatalog_file://cvmfs/cms.cern.ch/SITECONF/T2_US_Nebraska/PhEDEx/storage.xml?protocol=direct"/>
     <catalog url="trivialcatalog_file://cvmfs/cms.cern.ch/SITECONF/T2_US_Nebraska/PhEDEx/storage.xml?protocol=xrootd"/>
   </event-data>
   <source-config>
     <cache-hint value="application-only"/>
     <read-hint value="auto-detect"/>
     <statistics-destination name="cms-udpmon-collector.cern.ch:9331" />
   </source-config>

   <local-stage-out>
     <command value="hadoop" />
     <catalog url="trivialcatalog_file://cvmfs/cms.cern.ch/SITECONF/T2_US_Nebraska/PhEDEx/storage.xml?protocol=hadoop"/>
     <se-name value="srm.unl.edu" />
   </local-stage-out>
   <fallback-stage-out>
     <se-name value="srm-dcache.rcac.purdue.edu"/>
     <lfn-prefix value="srm://srm-dcache.rcac.purdue.edu:8443/srm/managerv2?SFN=/"/>
     <command value="srmv2"/>
   </fallback-stage-out>
   <calib-data>
       <frontier-connect>
       <load balance="proxies"/>
       <proxy url="http://red-squid1.unl.edu:3128"/>
       <proxy url="http://red-squid2.unl.edu:3128"/>
       <server url="http://cmsfrontier.cern.ch:8000/FrontierInt"/>
       <server url="http://cmsfrontier1.cern.ch:8000/FrontierInt"/>
       <server url="http://cmsfrontier2.cern.ch:8000/FrontierInt"/>
       <server url="http://cmsfrontier3.cern.ch:8000/FrontierInt"/>
     </frontier-connect>
   </calib-data>
</site>
</site-local-config>
Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2013-02-27 - BrianBockelman
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback