DPM Consistency Check Tool

What does it do ?

The DPM consistency check tool is an admin tool which can be used for the following purposes

• • behaviour by default:
  • it allows listing files which are stored in the name server but not physically stored;
  • it allows listing files which are stored physically and not declared in the DPNS DB;
  • it allows listing files whose sizes stored in the name server differ from the physical size.

In the future, when specifying the --force_deletion option, you can (TO BE USED ONLY IN CASE OF A CRASH TO KEEP CONSISTENCY):

• • remove the files on the problematic file system from the DPM Name Server database.

Requirements

DPM-client is required to run the DPM consistency script.

The commands need to be run as root on the DPM head node.

Installation

• Download the attachment and untar it.
• there are two executables check-db-files and check-physical-files in output
• the sources and makefile are provided for security reasons (some sites don't trust executables)

NB: in the tar, the commands are provided for SL3-32 bits.

Usage

The DPNS_HOST environment variable must point to the host which runs the DPNS (name server) daemon.

Warning

One must be aware of real activities. It can happen that files are being transferred and not yet properly updated in the DB (filesize entry) ... so this tool should be used very carefully. The site admin has to bear in mind that it can't provide real time information.

Checking that all files in DB are physically stored

• Example 1: you want to check that all the entries in the DPNS DB hosted in lxn1177.cern.ch stored in the file system /data02 which is located on the DPM disk server lxb1532.cern.ch.

So you can use it as follows

export DPNS_HOST=lxn1177.cern.ch
$ ./check-db-files  --server lxb1534.cern.ch --fs /data02 

• Example 2: you want to check that all the entries in the DPNS DB hosted in lxn1177.cern.ch stored in the Pool name Pool_00 which is located on the DPM disk server lxb1532.cern.ch.

So you can use it as follows

export DPNS_HOST=lxn1177.cern.ch
$ ./check-db-files --server lxb1534.cern.ch --poolname Pool_00 

A log file is written with all the problems which occur. If the log returns "Entries found in the name server are all stored physically with the right file size" , it means that it is OK.

Checking that all files on disk are declared in the DPNS DB

* Example 1: you want to check that all the entries in the DPNS DB hosted in lxn1177.cern.ch stored in the file system /data02 which is located on the DPM disk server lxb1532.cern.ch.

So you can use it as follows

export DPNS_HOST=lxn1177.cern.ch
$ ./check-physical-files  --server lxb1534.cern.ch --fs /data02 

• Example 2: you want to check that all the entries in the DPNS DB hosted in lxn1177.cern.ch stored in the Pool name Pool_00 which is located on the DPM disk server lxb1532.cern.ch.

So you can use it as follows

export DPNS_HOST=lxn1177.cern.ch
$ ./check-physical-files --server lxb1534.cern.ch --poolname Pool_00 

A log file is written with all the problems which occur. If the log returns "Entries found in the name server are all stored physically with the right file size" , it means that it is OK.

Removing files on problematic file system from the DPM Name Server database (To be provided in the next version as it requires changes in the DPM server code)

The command below removes all entries in the name server which are no longer stored physically. Referring to our two previous examples,

• Example 1

export DPNS_HOST=lxn1177.cern.ch
$ ./dpns-cleanup --server lxb1534.cern.ch --fs /data02 --force_deletion

• Example 2

export DPNS_HOST=lxn1177.cern.ch
$ ./dpns-cleanup --poolname Pool_00 --server lxb1534.cern.ch  --force_deletion

Here also, a log file is produced writing down the name of the files which have been deleted.

Troubleshooting

Help ?

For help, please contact hep-service-dpm@cernNOSPAMPLEASE.ch (remove the NOSPAM part !).

-- SophieLemaitre - 04 Apr 2007