-- AnaFranco - 28 Jan 2014

Consistency Checks

Introduction

Periodically, we conduct a set of checks at the T1s and T2s, to make sure that the files stored in their storage elements, and registered in the Database are in sync.

Types of Checks

Currently these consist of:

  • StorageConsistencyCheck (SCC) - Check if all files in the storage are registered in the database. The ones that are not are called 'orphans' and are removed from the storage, because without anyone knowing about their presence at the site they are simply useless.
  • BlockDownloadVerify (BDV) - Check that all blocks registered in the DB as present at the site can actually be accessed and that the sizes of the files in them corresponds to what is registered in the DB at file creation.

Storage Consistency Check (SCC)

Check if all files in the storage are registered in the database. The ones that are not are called 'orphans' and are removed from the storage, because without anyone knowing about their presence at the site they are simply useless.

Step 1: Get the storage dump from the site

  • The coordination of the checks is currently done through Savannah Tickets. One ticket should be opened to the site in question, asking the site admins to provide a full storage dump for their site. Example: https://savannah.cern.ch/support/index.php?140931
  • The full storage dump from the site must be gathered in one file, lfn_list.txt:
    • Some sites can send the dump in separated files, for example:
      • data.out
      • generator
      • out hidata.out
      • himc.out
      • lumi.out
      • mc.out relval
      • out results.out
They must be gathered in the file lfn_list.txt.

  • Verify if the lines of the files look like these:

/dpm/brunel.ac.uk/home/cms/store/data/Run2011A/Photon/AOD/08Nov2011-v1/0000/4A4C7E19-250B-E111-BDC3-002590200B64.root

/store/relval/CMSSW_6_1_1-START61_V11/RelValSingleMuPt100/GEN-SIM-DIGI-RAW-HLTDEBUG/v1/00000/62545890-DD76-E211-BFFE-002590494FDC.root

Obs: Sometimes the sites send the dump file in xml format. It must be treated and after that the lines should look like the examples above.

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2014-01-28 - AnaFranco
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback