Difference: CernLbdDiskManagement (8 vs. 9)

Revision 92010-12-20 - RobLambert

Line: 1 to 1
 
META TOPICPARENT name="CernLbdDiskSpace"

CernLbdDiskManagement

This is the LHCb CERN LBD group page describing the management of locally available disk space

Line: 69 to 69
 There are two types of files which can be on the lhcbt3: user files, and central files. In general there are two spying scripts, check30TB.py and castorQuota.py, each with many options to look in specific directories, etc.
Changed:
<
<

- user files

>
>
  • check30TB.py: poles the castor dump of all files from last night to categorize all files which were on the lhcbt3 then
  • check30TB_slow.py: uses nsls -Rl and stager_qry to find files on lhcbt3, this is up-to-date but much slower
  • castorQuota.py uses nsls -Rl and stager_qry to categorise files from a given user, again this is up-to-date but slow
 
Changed:
<
<
To find out the used quotas of all users:

python  /afs/cern.ch/project/lbcern/scripts/check30TB.py

Then to get the list of files:

stager_qry -M  castor/cern.ch/user/<u>/<uname>

- central files

It is not clear how we can check quickly what is on lhcbt3 from central productions, and which user staged any individual file into the pool is unknown.

python  /afs/cern.ch/project/lbcern/scripts/check30TB.py --data >year< --MC

This will take a very long time, probably.

Then to get the list of files:

stager_qry -M  castor/cern.ch/grid/lhcb/data/<year>/ -M  castor/cern.ch/grid/lhcb/MC

>
>
Each script has a --help flag to tell you about the other options
 

- removal notes:

Line: 99 to 83
  Or remove a big list with:
Added:
>
>
stager_rm -f <file with a list in it> -S lhcbt3

or similarly:

 /afs/cern.ch/project/lbcern/scripts/lhcbt3_rm.py <username or directory or file with a list in it>

Removing a user file potentially deletes the file forever, so it's nice to get the users to do that themselves, but any centrally produced or grid-resident file is by definition backed-up and can be removed arbitrarily.

Added:
>
>

- Files you can't remove

If a user stages a file from a different user's home directory, then they won't have the ability to unstage them, neither will the owner unless they are also on the access list for lhcbt3, so they are stuck. Only Joel can remove them, or the castor support who certainly could do it, but if you ask them will reply with the annoying:

Unfortunately, as service policy, CASTOR Operations does not preform delete operations on files.
 

-- RobLambert - 19-Nov-2010 \ No newline at end of file

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback