Policies concerning Express on EOS at CERN:

  • Express files produced at T0_CH_CERN_Disk EOS and subscribed to T2_CH_CERN EOS by the T0 infrastructure
  • There will be three express streams: Express for pp datataking, ExpressCosmics for interfill and machine downtime and HLTMON that is always taken
  • We assume 4MB/event for Express (FEVT, ALCARECO, DQMIO output), 0.15MB/event for ExpressCosmics (FEVT, ALCARECO, DQMIO output) and 1MB/event for HLTMON (RAW,DQMIO output)
  • The quota for express on T2_CH_CERN EOS ( /store/express ) needs to be sized to hold 30 days of express. The LHC duty cycle is assumed to be 40%
    • Express 50Hx, ExpressCosmics 50Hz, HLTMON 10Hz
      • 30 x 24 x 3600s x ( 0.4 x 50Hz x 4MB + 0.6 x 50Hz x 0.15MB + 10Hz x 1MB ) make about 245TB
  • Automated clean up
    • every day, automated clean up is executed to stay within the allotted quota
    • If the quota for /store/express is used by more than 80%, the automated clean up creates deletion requests and auto-approves them to reduce the volume of express to less than 80% of the EOS quota
    • the atomic unit of deletion is a run, corresponding to one or more blocks of express data
      • the automated clean up is deleting blocks of the oldest runs of express datasets first
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r5 - 2016-06-02 - DirkHufnagel
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback