CheckDpmConsistency < LCG

LCG Web>LCGGridDeployment>LCGSoftware>DataManagementTop>DataManagementDocumentation>CheckDpmConsistency (2007-10-15, LanaAbadie) (raw view)

EditAttachPDF

<!-- ALLOWTOPICCHANGE = DepITGDSCGroup -->

---+ DPM Consistency Check Tool 

%TOC%

---++ What does it do ?

The DPM consistency check tool is an admin tool which can be used for the following purposes 
      * behaviour by default:
      * it allows listing files which are stored in the name server but not physically stored;
      * it allows listing files which are stored physically and not declared in the DPNS DB;
      * it allows listing files whose sizes stored in the name server differ from the physical size.



In the future, when specifying the =--force_deletion= option, you can (TO BE USED ONLY IN CASE OF A CRASH TO KEEP CONSISTENCY):
      * remove the files on the problematic file system from the DPM Name Server database.

---++ Requirements

=DPM-client= is required to run the DPM consistency script.

The commands need to be run as root on the DPM head node.

---++ Installation

   * Download the attachment and untar it.
   * there are two executables check-db-files and check-physical-files in output
   * the sources and makefile are provided for security reasons (some sites don't trust executables)

  
 NB: in the tar, the commands are provided for SL3-32 bits.
---++ Usage

The DPNS_HOST environment variable must point to the host which runs the DPNS (name server) daemon.

---+++ %ICON{"warning"}% Warning
 One must be aware of real activities. It can happen that files are being transferred and not yet properly updated  in the DB (filesize entry) ... so this tool should be used very carefully. The site admin has to bear in mind that it can't provide real time information.

---+++ Checking that all files in DB are physically stored

   * Example 1: you want to check that all the entries in the DPNS DB hosted in lxn1177.cern.ch stored in the file system /data02 which is located on the DPM disk server lxb1532.cern.ch.

So you can use it as follows

<verbatim>
export DPNS_HOST=lxn1177.cern.ch
$ ./check-db-files  --server lxb1534.cern.ch --fs /data02 
</verbatim>

   * Example 2: you want to check that all the entries in the DPNS DB hosted in lxn1177.cern.ch stored in the Pool name Pool_00 which is located on the DPM disk server lxb1532.cern.ch.

So you can use it as follows

<verbatim>
export DPNS_HOST=lxn1177.cern.ch
$ ./check-db-files --server lxb1534.cern.ch --poolname Pool_00 
</verbatim>

A log file is written with all the problems which occur. If the log returns "Entries found in the name server are all stored physically with the right file size" , it means that it is OK.

---+++ Checking that all files on disk are declared in the DPNS DB
* Example 1: you want to check that all the entries in the DPNS DB hosted in lxn1177.cern.ch stored in the file system /data02 which is located on the DPM disk server lxb1532.cern.ch.

So you can use it as follows

<verbatim>
export DPNS_HOST=lxn1177.cern.ch
$ ./check-physical-files  --server lxb1534.cern.ch --fs /data02 
</verbatim>

   * Example 2: you want to check that all the entries in the DPNS DB hosted in lxn1177.cern.ch stored in the Pool name Pool_00 which is located on the DPM disk server lxb1532.cern.ch.

So you can use it as follows

<verbatim>
export DPNS_HOST=lxn1177.cern.ch
$ ./check-physical-files --server lxb1534.cern.ch --poolname Pool_00 
</verbatim>

A log file is written with all the problems which occur. If the log returns "Entries found in the name server are all stored physically with the right file size" , it means that it is OK.
---+++ Removing files on problematic file system from the DPM Name Server database (To be provided in the next version as it requires changes in the DPM server code)

The command below removes all entries in the name server which are no longer stored physically.
Referring to our two previous examples, 
   * Example 1
<verbatim>
export DPNS_HOST=lxn1177.cern.ch
$ ./dpns-cleanup --server lxb1534.cern.ch --fs /data02 --force_deletion
</verbatim>

   * Example 2 
<verbatim>
export DPNS_HOST=lxn1177.cern.ch
$ ./dpns-cleanup --poolname Pool_00 --server lxb1534.cern.ch  --force_deletion
</verbatim>
   
Here also, a log file is produced writing down the name of the files which have been deleted.


---++ Troubleshooting



---++ Help ?

For help, please contact hep-service-dpm@cern.ch (remove the NOSPAM part !).


-- Main.LanaAbadie - 19 Sep 2007

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
tar	DPM-check-consistency.tar	r1	manage	90.0 K	2007-09-19 - 09:13	LanaAbadie	Utilities to check the consistency of the DPNS, sources included, application provided for 32-SL3 by default

Topic revision: r10 - 2007-10-15 - LanaAbadie

LCG Wikis

LCG Service
Coordination

LCG Grid
Deployment

LCG
Apps Area

Public webs

Welcome Guest

- Cern Search
- TWiki Search
- Google Search
LCG All webs

Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback