File Invalidations

Introduction

Occasionally, files are lost or damaged at tier sites, and the only possible action is to invalidate them in the database. Normally, a Savannah ticket is opened by the site, explaining the reason for requesting the invalidation, and listing the files that need invalidation. A Transfers Team member then does the invalidation, and logs this by sending an email to the e-group: cms-file-invalidations@cernNOSPAMPLEASE.ch

Instruction

General Principle: Local invalidation & Global invalidation

Note : for local invalidations only two steps, for global some additional.

  • 1. Take the file(s) and check if it (they) has any replicas available.
  • 2. a) if it does have another replica - local invalidation (TMDB invalidate with -node parameter)
    • This is done with local invalidation, which don't need go further with the left steps.
    • After local invalidation, the file is re-transfered to the site automatically.
  • 2. b) if it doesn't have other replicas, which means it has the only replica at site which is asking to invalidate -- global invalidation.
  • 3. Make invalidation with -invalidate option
  • 4. Make DBS invalidation (with -h option you can get help)
  • 5. Send an email to cms-file-invalidations@cernNOSPAMPLEASE.ch saying which file(s) and why.

TMDB (PhEDEx) invalidation

  • Step1: Set up micro-phedex env
    • For bash shell: source {PhEDEx directory}/etc/profile.d/env.sh
    • For c shell: source {PhEDEx directory}/etc/profile.d/env.csh

  • Step2: Test the configuration of phedex
    • {PhEDEx directory}/Utilities/FileDeleteTMDB -h
    • This also produces the instruction to use FileDeleteTMDB.

  • Step3: Local invalidation
    • {PhEDEx directory}/Utilities/FileDeleteTMDB -db ~/phedex/info/DBParam:Prod/OPSLIUSI -list lfn:[lfn] -node [site name]
    • For T1 tape nodes, PhEDEx will restart the transfer only if file is invalidated on both Buffer and MSS (eg. -node T1_IT_CNAF_Buffer, T1_IT_CNAF_MSS )

  • Step4: Global invalidation
    • {PhEDEx directory}/Utilities/FileDeleteTMDB -db ~/phedex/info/DBParam:Prod/OPSLIUSI -list lfn:[lfn] -invalidate

DBS3 invalidation (Global invalidation only)

  • Step1: Set up CMSSW env

  • Step2: Set up gLite env
    • source /uscmst1/prod/grid/gLite_SL5.sh

  • Step3: Get voms proxy
    • voms-proxy-init -voms cms -cert [your cert] -key [your key]

  • Step4: Setup DBS3 env
    • source /opt/cmsdatatransfers/sw/comp.pre/slc5_amd64_gcc461/cms/dbs3-client/3.1.6a/etc/profile.d/init.sh

  • Step5: DBS3 invalidation command
    • Copy DBS3SetFileStatus.py script from your DBS3 environment: cp /opt/cmsdatatransfers/sw/comp.pre/slc5_amd64_gcc461/cms/dbs3-client/3.1.5a/examples/DBS3SetFileStatus.py
    • ./DBS3SetFileStatus.py --url=https://cmsweb.cern.ch/dbs/prod/global/DBSWriter --status=invalid --recursive=False --files=LFN

DBS3 invalidation for SL6 (Global invalidation only)

  • Step1: Set up CMS Software release environment.
    • cd /afs/cern.ch/user/j/"user_id"/home/cmssw/CMSSW_7_1_4/src
    • cmsenv
    • cd ~

  • Step 2: Setup crab3 to get access to DBS3 client
    • source /cvmfs/cms.cern.ch/crab3/crab.sh

  • Step 4: Grid Proxy
    • voms-proxy-init -voms cms -cert [your cert] -key [your key]

  • Step5: DBS3 invalidation command
    • ./DBS3SetFileStatus.py --url=https://cmsweb.cern.ch/dbs/prod/global/DBSWriter --status=invalid --recursive=False --files=LFN

Debug File Invalidation

  • Note: use Debug part of DBParam file while invalidating debug test files

  • Step2: Invalidate source file using Debug part of DBParam file
    • ./FileDeleteTMDB -db ~/param/DBParam:Debug/Meric -list [file] -invalidate -keepempty dataset

  • Step4: Invalidate injection files using Debug part of DBParam file
    • ./FileDeleteTMDB -db ~/param/DBParam:Debug/Meric -list [file] -invalidate -keepempty dataset

  • Since these are PhEDEx test files, they're not registered to DBS, so no action is required in DBS

  • If you need to invalidate all source&destination loadtest files to inject new ones, you can check this twiki

  • NOTE: Use "-keepempty dataset" option, since you can not delete the entire dataset

Some statistics of invalidations per month, based on the logs in the e-group.


Month
# of files
invalidated
# of inv.
requests
# of files in requests
2013/08 1824 4 1,7,1,227,1588
2013/07 10980 9 42,9186,3,100,6,1626,1,14,2
2013/06 211 4 105,19,43,44
2013/05 401 12 1,1,1,170,1,1,49, 3, 9,4,2,159
2013/04 1230 17 15,1,608,1,14,242,4,1,2,60,250,1,10,1,14,3,3
2013/03 661 19 1,4,1,1,2,3,1,1,3,5,9,10,9,1,2,160,2,444,2
2013/02 342 15 4,1,4,2,1,1,2,1,6,1,309,1,5,4
2013/01 271 17 1,1,1,1,1,30,2,69,4,46,1,1,108,2,1,1,1
2012/12 62 2 1,61
2012/11 162 20 2,1,1,1,5,24,2,5,3,14,26,1,8,5,1,36,4,1,19,3
2012/10 219 27 22,1,21,115,1,1,1,1,4,5,5,2,1,1,3,1,4,1,4,1,2,5,6,7,1,2,1
2012/09 804 21 1,1,1,1,1,2,1,1,2,1,4,3,1,5,5,746,5,2,8,2,11
2012/08 82 11 1,1,1,6,65,1,1,1,1,1,3
2012/07 617 8 602,2,6,2,1,1,1,2
2012/06 557 23 2,1,2,2,1,13,3,1,1,1,6,1,446,1,1,2,1,62,1,2,1,1,5
2012/05 256 16 5,8,77,58,1,1,5,1,55,37,1,1,2,2,2
2012/04 27 6 4,2,2,2,1,16
2012/03 139 7 65,66,1,2,2,2,1

Edit | Attach | Watch | Print version | History: r21 < r20 < r19 < r18 < r17 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r21 - 2018-05-02 - PradeepJasal
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback