CASTOR dumps at CERN

by Ignacio Reguero

Dear Flavia and all,
 As just discussed,
 I temporarily put a couple of the scripts that we use at CERN in
http://castor.web.cern.ch/castor/DIST/dumpDiskPool.pl
http://castor.web.cern.ch/castor/DIST/dumpDiskPoolavail
- dumpDiskPool.pl is the one generally used where you get a sorted listing
  of the paths of the files in the disk pool with fileid, size, status,
  diskcopy creation time and file last access time.

- dumpDiskPoolavail is a version developed for a special case where we
  list all diskcopies (so you may have several entries for the same 
path and fileid) with their availability information (based on the
  filesystem+diskserver one).

Both scripts assume that there is a one to one mapping of Castor diskpool 
to service class.

We run the scripts once a day early in the morning on weekdays (not in 
the weekend) to minimize impact on the stager DB.

They both put the dump in a directory in AFS that is accessed through 
http://cern.ch/castor Website.

We have a convention where the URLs are 
http://castor.web.cern.ch/castor/DiskPoolDump/..last
this is pointing to the latest instance of
http://castor.web.cern.ch/castor/DiskPoolDump/atlas..
for each pool so that you can bookmark the reference. Please note that 
older outputs are removed when creating the latest one.
For instance
http://castor.web.cern.ch/castor/DiskPoolDump/cms.t0input.last
or
http://castor.web.cern.ch/castor/DiskPoolDump/atlas.default.last

As you get a sorted list of the paths in the pool with their associated
information, you can exploit them in your own framework or use simple
scripts to do resource consumption queries
For instance
- The total file size of files in atlt3 in GB
cat atlas.atlt3.last | awk 'SUM+=$4 {printf("%f\n",SUM/1.e9)}' |tail -1
905.836689
- The total file size of the files in atlt3 in the hierarchy under
/castor/cern.ch/grid/atlas/atlt3/scratch in GB
cat atlas.atlt3.last | grep /castor/cern.ch/grid/atlas/atlt3/scratch | awk
'SUM+=$4 {printf("%f\n",SUM/1.e9)}' |tail -1
248.180882

-- FlaviaDonno - 08 Jul 2008

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2008-09-12 - FlaviaDonno
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback