DPM Flushing Old Requests utility

Introduction

This page describes how to flush old requests (put, get, copy and bring online requests) of the DPM DB. Tables affected are dpm_put_filereq, dpm_get_filereq, dpm_req and dpm_copy_filereq. For the moment, the tool works only for MySQL DB. if you need it for Oracle, please send me a mail. This tool must be used as long as the flushing mechanism is not integrated in the DPM server.

Outline of the tool

The tool consists of a command line dpm-flush-req which takes the following input arguments:
  • dpm_db_name : the DPM DB name (yaim uses by default dpm_db). However there is no default value
  • dpm_db_host : the DPM MySQL DB host name (need to be put even if you run the command locally, i.e. if localhost is passed as dpm_db_host, the db connection will fail
  • dpm_db_pwd : password of the DPM DB
  • dpm_db_user : login od the DPM DB(yaim by default uses dpmmgr)
  • max_request_lifetime : maximum time to keep a request in the tables. Time to be given in seconds. If this argument is omitted, the default value is 2592000

Installation

  • Download and extract the tar file. Type make and if needed edit the makefile to match your environment.
  • the executable and the log files will be located in the output directory.
  • Please look at the MySQL configuration parameters (innodb_buffer_pool_size>=256M). To get its value, connect to the mysql and type show variables like 'innodb_buffer_pool_size'. The default value is 8M (too small). See Any problems or questions section to update this value.

Example of usage

To delete old requests which have been performed two months (5270400 sec) before, type:
./dpm-flush-oldreq --dpm_db_name dpm_db --dpm_db_pwd  --dpm_db_host lxn1177.cernc.h --dpm_db_user dpm_mgr --max_request_lifetime 5270400

Then look at the log file produced dpm-flush-oldreq.log to see if the execution was successful.

Remark

if the number of requests to be flushed is very big around millions of row, it might be a good idea to start this script before migrating to a new DPM version so that it doesn't impact the performance of the DPM. it takes roughly one hour to delete around a few millions of old requests. Of course the execution time will depend on the DPM load.

Any problems or questions?

  • if you get the following error in the log file: the total number of locks exceeds the lock table size, then you need to increase the innodb_buffer_size as follows:
    • edit your /etc/my.cnf (MySQL configurable parameters)
    • add in [mysqld] section : innodb_buffer_pool_size=256M (provided that this value does not exceed 80% of the physical memory of the machine).
    • restart mysql (service mysql/mysqld restart resp for SL3/SL4)
send me a mail

-- LanaAbadie - 15 Oct 2007

Topic attachments
I Attachment History Action Size Date Who Comment
Compressed Zip archivetar dpm-flush.tar r1 manage 230.0 K 2007-10-16 - 22:03 LanaAbadie  
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r3 - 2007-10-16 - LanaAbadie
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback