Manual File Re-Injection

Introduction

Some times files are invalidated globally in TMDB, but still exists at storage. In that cases, you may want to re-inject the files. Also, files could exist at TMDB but with no replica, even if they have repica(s) at a given site(s). In those cases, you need to invalidate the files globally (with -keepempty block, to maintain existing subscriptions) and then re-inject them into TMDB

NOTE: When injecting to tape (_Buffer & _MSS), you just need to inject into Buffer. Then, if there is a subscription, the MSS replica will be recreated. If there is no subscription, you need to subscribe the files

Step By Step

Step 1 Generate the injection .xml file. The file should contain the XML structure representing the data to be injected.

• See requirements of the XML file at phedex::Web::API::Inject
• Script to generate the XML file from a csv file at: /afs/cern.ch/user/j/jodiazcr/public/csv2xml.py
  • CSV file:

  • dataset_name1,block_name1,file_name1,file_size1,file_cksum1
    dataset_name2,block_name2,file_name2,file_size2,file_cksum2
    ...
    dataset_nameN,block_nameN,file_nameN,file_sizeN,file_cksumN

Step 2 Reactivate the blocks.

• Command:

  • while read line; do sqlplus -S $(~/home/phedex/PHEDEX/Utilities/OracleConnectId -db ~/private/DBParam.Jorge:Prod/OPSJORGE) @/afs/cern.ch/user/j/jodiazcr/private/reactivateblock.sql $line; done < /afs/cern.ch/user/j/jodiazcr/private/blocks_to_reactivate.list

• SQL:

  • set role phedex_opsjorge_prod identified by XXXXXXXXXXX;
    insert into t_dps_block_activate (block,time_request,time_until) values ((select id from t_dps_block where name=':1'), (SYSDATE - TO_DATE('1970-01-01', 'YYYY-MM-DD')) *60*60*24, (SYSDATE - TO_DATE('1970-01-01', 'YYYY-MM-DD')) *60*60*24 + 86400);
    commit;
    quit
    /

Step 3 Then wait ~15 mins for the central BlockActivate agent to activate the block. Check mgmt-blockactivate logs for block activation message.

• Ssh to Phedex Machine

  • ssh phedex@vocms0214

• Check mgmt-blockactivate

  • cd /data/ProdNodes/Prod_Mgmt/logs
    tail -n 30 mgmt-blockactiv

• You can also check if the block is active directly in TMDB:

  • select is_active from t_dps_block_replica where block=6414441

Step 4 Reopen the blocks.

• Commnad:

  • while read line; do sqlplus -S $(~/home/phedex/PHEDEX/Utilities/OracleConnectId -db ~/private/DBParam.Jorge:Prod/OPSJORGE) @/afs/cern.ch/user/j/jodiazcr/private/reopenblock.sql $line; done < /afs/cern.ch/user/j/jodiazcr/private/blocks_to_reopen.list

• SQL:

  • set role phedex_opsjorge_prod identified by xxxxxxxxxxxxxxxx;
    update t_dps_block set is_open='y', time_update=((SYSDATE - TO_DATE('1970-01-01', 'YYYY-MM-DD')) *60*60*24-3600) where name=':1';
    commit;
    quit
    /

Step 5 Use PhEDEx datasvc inject API to inject the files and close the blocks

• Command:

  • source ~/PHEDEX/etc/profile.d/env.sh
    ~/PHEDEX/Utilities/phedex --instance prod inject --datafile filestoinject.xml --node TX_XX_XXXXX_XXX --strict

Step 6 (Optional) You can check the replicas of the files to make sure the files have been injected at the correct sites

-- JorgeAlbertoDiazCruz - 2016-02-03