Difference: LocalData (1 vs. 3)

Revision 32008-04-14 - NikoNeufeld

Line: 1 to 1
 
META TOPICPARENT name="LHCbOnline"

Writing files using the runcontrol

Line: 38 to 38
 
/castor/cern.ch/lhcb/online/point8/<group name>/

Notes

Changed:
<
<
  • Do NOT use special characters in your file names: blanks, , ^, %, $, &, ? - these might or might not work and we will *NOT check that this works
>
>
  • Do NOT use special characters in your file names: blanks, \, ^, %, $, &, ? - these might or might not work and we will NOT check that this works
 
  • small files (< 1 MB) will not be copied to CASTOR - pack them into archives
  • files which are marked as migrated in CASTOR (a small m in the nsls output) can and will eventually be deleted from the Online storage. We never delete files from CASTOR
  • Our scripts are paranoid - if after a file has been copied to CASTOR, you change the size of the file (rewrite, truncate, etc...) - the new file will not be copied and the original file on CASTOR will never be overwritten. Use a new file-name

Revision 22008-04-14 - NikoNeufeld

Line: 1 to 1
 
META TOPICPARENT name="LHCbOnline"
Changed:
<
<

Writing files locally

>
>

Writing files using the runcontrol

For this you need to be under dynamic allocation. Run-numbers will be allocated automatically from the Rundatabase. Your files are stored locallly under

/daqarea/lhcb/data/2008/RAW
This is mounted on plus02 and plus01. We do not want to mount it on other systems to ease the load on our NFS servers.

These files are regulary (every hour) copied to CASTOR. You will find them there under

/castor/cern.ch/grid/lhcb/data/2008/RAW

From RAW onward the structure is the same

PARTITION/ACTIVITY/RUN

Example

Run number 21444 for RICH1 was taken in "ALICE" activity. On plus02 you find it on

/daqarea/lhcb/data/2008/RAW/RICH1/ALICE/21444/021444_0000002461.raw

and on CASTOR

/castor/cern.ch/grid/lhcb/data/2008/RAW/RICH1/ALICE/21444/021444_0000002461.raw

Notes

  • If you don't find a file on CASTOR, don't panic. Files are only copied when they are not of zero-length and at least 10^6 bytes long.
  • Files are not automatically deleted from the /daqare. So nothing will vanish, before you will get an explicit warning
  • Files are never deleted from CASTOR, unless you explicitly ask for this

Writing files locally (using DiskWR)

  When writing files locally, that is by using DiskWR, the files are not managed centrally by the Run Database. You are yourself responsible for managing these files. We provide an area for such files, which is mounted on all farm-nodes under /data. There is a group-writable subdirectory for every sub-detector.
Line: 9 to 38
 
/castor/cern.ch/lhcb/online/point8/<group name>/

Notes

Changed:
<
<
>
>
  • Do NOT use special characters in your file names: blanks, , ^, %, $, &, ? - these might or might not work and we will *NOT check that this works
 
  • small files (< 1 MB) will not be copied to CASTOR - pack them into archives
  • files which are marked as migrated in CASTOR (a small m in the nsls output) can and will eventually be deleted from the Online storage. We never delete files from CASTOR
  • Our scripts are paranoid - if after a file has been copied to CASTOR, you change the size of the file (rewrite, truncate, etc...) - the new file will not be copied and the original file on CASTOR will never be overwritten. Use a new file-name
Line: 19 to 48
 
Changed:
<
<
-- NikoNeufeld - 03 Jul 2007
>
>
-- NikoNeufeld - 14 Apr 2008
 \ No newline at end of file

Revision 12007-07-03 - NikoNeufeld

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="LHCbOnline"

Writing files locally

When writing files locally, that is by using DiskWR, the files are not managed centrally by the Run Database. You are yourself responsible for managing these files. We provide an area for such files, which is mounted on all farm-nodes under /data. There is a group-writable subdirectory for every sub-detector.

These files are regulary (every hour) copied to CASTOR. You will find them there under

/castor/cern.ch/lhcb/online/point8/<group name>/

Notes

  • small files (< 1 MB) will not be copied to CASTOR - pack them into archives
  • files which are marked as migrated in CASTOR (a small m in the nsls output) can and will eventually be deleted from the Online storage. We never delete files from CASTOR
  • Our scripts are paranoid - if after a file has been copied to CASTOR, you change the size of the file (rewrite, truncate, etc...) - the new file will not be copied and the original file on CASTOR will never be overwritten. Use a new file-name
  • It's ok to create directories on the
    /data/group name>/
    area, they will be recursively copied

-- NikoNeufeld - 03 Jul 2007

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback