CDR - Central Data Recording

Simone Bifani, Emanuele Leonardi and Stefano Venditti will take over the CDR responsability from Venelin Kozhuharov for the P-326 run in year 2007

CDR mailing list: na48-cdr@cernNOSPAMPLEASE.ch

CDR setup: Castor notes

DAQ


daq.jpg

CDR Chain


cdr_chain.jpg

Machines

  • log-in to NA48CDR machine (data transfer & set-up for L3 jobs):
     ssh na48cdr@lxfsrb6102.cern.ch

     ssh na48cdr@lxplus.cern.ch

The na48cdr = lxfsrb6102 alias has been defined.

Directories

Directories have been arranged with the following conventions:

  • NA48CDR machine:
    • config: configuration files
    • execs: script files (csh, perl)
    • logfiles: logs info
    • src: program sources
    • status: tag files

  • LXPLUS machines:
    • bin: program sources
    • log: logs info
    • status: tag files

e.g.:

  • /shift/lxfsrb6102/data02/na48cdr/cdr/src : CDR program sources
  • /shift/lxfsrb6102/data02/na48cdr/l3/logfiles : L3 logfiles
  • /shift/lxfsrb6102/data02/na48cdr/reprocessing/status : Reprocessing tag files
  • /afs/cern.ch/user/n/na48cdr/lsf/bin : Batch monitor program sources

Take a Look at Crontab

  • NA48CDR machine:
:~> crontab -l

#1,8,16,24,35,40,48,51,56 *  *  *  * /shift/lxfsrb6102/data02/na48cdr/cdr_na48/execs/cdr_watchdog >> /shift/lxfsrb6102/data02/na48cdr/cdr_na48/logfiles/logfile_watchdog 2>&1
#9 0,2,4,6,8,10,12,14,16,18,20,22 * * * /shift/lxfsrb6102/data02/na48cdr/marco/check_bad_burstlets.pl >> /shift/lxfsrb6102/data02/na48cdr/cdr_na48/logfiles/log.check_bad_burstlets.pl 2>&1
#1 * * * * /shift/lxfsrb6102/data02/na48cdr/venelin/scripts/chk_status.csh >& /dev/null
#1,31 * * * * /shift/lxfsrb6102/data02/na48cdr/venelin/scripts/stop_tape.csh >> /shift/lxfsrb6102/data02/na48cdr/venelin/logs/stop_tape.log 2>&1
#1,8,16,24,32,40,48,56 *  *  *  * /shift/lxfsrb6102/data02/na48cdr/cdr_na48/execs/cdr_watchdog_linux_cdrserver >> /shift/lxfsrb6102/data02/na48cdr/cdr_na48/logfiles/logfile_watchdog_linux_cdrserver 2>&1
#1,8,16,24,35,40,48,56 * * * * /shift/lxfsrb6102/data03/na48db2004/software/scripts/startw2port.pl >> /shift/lxfsrb6102/data03/na48db2004/log/w2port-watchdog.log
#0 * * * * /shift/lxfsrb6102/data03/na48db2004/software/scripts/rcmysql start >> /shift/lxfsrb6102/data03/na48db2004/log/mysql-start.log
#2,17,32,47 * * * * /shift/lxfsrb6102/data02/na48cdr/l3/scripts_2003/html_info.csh > /shift/lxfsrb6102/data02/na48cdr/l3/scripts_2003/info.html
#0 0 * * * /shift/lxfsrb6102/data03/na48db2004/software/scripts/DB-Snapshot.sh run >> /shift/lxfsrb6102/data03/na48db2004/log/db-backup.log
#0 12 * * 3 /shift/lxfsrb6102/data03/na48db2004/software/scripts/DB-Snapshot.sh calibration goldcmp30 goldraw30 >> /shift/lxfsrb6102/data03/na48db2004/log/db-backup.log

  • LXPLUS machines:
:~> acrontab -l

#0,6,10,20,30,43,50 * * * * lxplus011 /afs/cern.ch/user/n/na48cdr/lsf/bin/watchdog.csh >> /afs/cern.ch/user/n/na48cdr/lsf/log/watchdog.log 2>&1
#1 * * * * lxplus011 /afs/cern.ch/user/n/na48cdr/lsf/bin/logrotate.csh >> /afs/cern.ch/user/n/na48cdr/lsf/log/logrotate.log 2>&1
#4,19,34,49 * * * * lxplus011 /usr/local/bin/rfcp /shift/lxfsrb6102/data02/na48cdr/l3/scripts_2003/info.html /afs/cern.ch/na48/www/private/L3rtr/info.html >& /dev/null
#1,6,11,16,21,26,31,36,41,46,51,56 * * * * lxplus011 /afs/cern.ch/user/n/na48cdr/public/L3Mon/L3Mon.csh >& /dev/null
#3 * * * * lxplus011 /afs/cern.ch/user/n/na48cdr/venelin/scripts/chk_status.csh >> /afs/cern.ch/user/n/na48cdr/venelin/logs/chk_status.log 2>&1

Check Deamons, Programs & Scripts

  • NA48CDR machine:
    • cdr_watchdog: checks all needed deamons, programs and scripts are running on the na48cdr machine
    • CDR_watchdog_linux_CDRserver: copies data files (RAW) from the Online PC-Farm (Meryn) to the Private Disk-Servers (CERN IT)
    • create_burstlet_list_files: looks for all the burstlets that belong to the same burst and when they are available fills a path list file to be used by the L3 filter (00 = burstlets header file, 02-05 = burstlet files)
    • remote_copy_daemon_linux
    • L3_control_pool: looks for a list file and if it is available sends a notification (tag file) to submit an L3 job to process data (GoldRAW)
    • delete_files_rawdata: checks if a burst has been successfully filtered by the L3 and deletes all the processed burstlets
    • check_bad_burstlets.pl
    • chk_status.csh: sends L3 processes info to the Online PC-Farm monitor (control-room)
    • stop_tape.csh
    • startw2port.pl: copies the run status from the DAQ (control-room) to the na48cdr machine (MySQL database)
    • rcmysql: puts processed burst info into the database
    • html_info.csh
    • DB-Snapshot.sh: creates a copy of the database

  • LXPLUS machines:
    • watchdog.csh: checks all needed deamons, programs and scripts are running on lxplus machines
    • logrotate.csh
    • info.html
    • L3Mon.csh
    • chk_status.csh

Tag Files

Tag files report scripts status. These flags are usually placed in /shift/lxfsrb6102/data02/na48cdr/cdr/status (na48cdr machine).

  • CDR_watchdog_linux_CDRserver: writes a tag for each copied burst in /copy_done and /copy2_done to report a successful transfer while in /incomplete_copy_done for a failed copy
  • L3_control_pool: deletes the tags from /copy2_done if the corresponding burst has been processed by L3
  • delete_files_rawdata:

Data Streams

L3 processes RAW data (burstlets files) stored in the Private Disk-Servers producing different streams and choosing in which storage element (Private Disk-Servers, CASTOR) save them:

  • GoldRAW: burst files without reconstruction
  • MainRAW: GoldRAW copy useful for calibration
  • GoldCMP: burst files with reconstruction
  • COmPACT: GoldCMP copy with extended DCH information
  • Calibration: LKr calibration



P326 CDR Group can edit this page to add content.
Topic attachments
I Attachment History Action Size Date Who Comment
JPEGjpg cdr_chain.jpg r1 manage 153.3 K 2007-04-27 - 16:08 SimoneBifani CDR Chain
JPEGjpg daq.jpg r1 manage 205.2 K 2007-05-04 - 15:41 SimoneBifani DAQ
Edit | Attach | Watch | Print version | History: r21 < r20 < r19 < r18 < r17 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r21 - 2007-07-27 - SimoneBifani
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    P326 All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback