DreamDataAnalysis < DREAM

DREAM Web>DreamDataAnalysis (2015-11-02, AlanSill) (raw view)

---+ Dream Data Analysis
%TOC%
---++ Run information

---+++ Logbooks

   * *Logbook July 2011 (1st part)* [[%ATTACHURL%/LOG_JULY_2011_1part.pdf][LOG_JULY_2011_1part.pdf]]
   * *Logbook July 2011 (2nd part)* [[%ATTACHURL%/logbook_July_2011_part2.pdf][logbook_July_2011_part2.pdf]]
   * *Logbook June 2011* https://atlserver.pv.infn.it/franchin/DREAM/LOGBOOK/logbook_TB_June2011.pdf
   * *Logbook October 2010* https://atlserver.pv.infn.it/franchin/DREAM/LOGBOOK/Logbook_Dream_October_2010.pdf
   * *Logbook July 2010* https://atlserver.pv.infn.it/franchin/DREAM/LOGBOOK/Logbook_Dream_July_2010.pdf

---+++ Run numbers

   * 2009 : from 0067 to 1543
   * 2010 Jun: from 1554 to 2875
   * 2010 Oct: from 3048 to 4303
   * 2011 Jun: from 4335 to 4693
   * 2011 Jul: from 4685 to .... run list file [[%ATTACHURL%/RunList.pdf][RunList.pdf]]

---+++ Ntuple Structure:
   * [[https://twiki.cern.ch/twiki/bin/view/DREAM/NtupleStructureJuly2011][July2011]]
   * [[https://twiki.cern.ch/twiki/bin/view/DREAM/NtupleStructureNovember2011][November2011]]

---++ Data Storage

---+++ NFS Mount

     * Paranters in /etc/fstab for NFS mount of dreamix2:
<verbatim>
dreamix2.cern.ch:/nfs/data      /mnt/dreamix2   nfs  rw,noatime,exec,dev,suid,bg,soft,nolock,nfsvers=3,rsize=1048576,wsize=1048576,timeo=20,retrans=10 0 0
</verbatim>

---+++ Setup
<!--   * The temporary storage h/w is =pcdream:/mnt/working= (not used in 2014?)
      * A small HD (460 GB) hosted on =pcdream=
      * The disk is NFS mounted by both =pcdreamdaq= and =pcdream=
      * It is supposed to temporarily host data before backup to =pcdream:/mnt/wd/working_bak=
      * At the end of each run the =~/bin/backupDataFile= (called by =daq=) takes care of copy data and pedestal files to =pcdream:/mnt/wd/working_bak= -->
   * The  storage h/w is =pcdream:/mnt/storage= 
      * It is a 3 TB external disk connected to =pcdreamdaq2=
<!--      * The script =~/bin/mirrorRuns= can be used to verify the status of the backup procedure and to backup files which failed the automatic copy procedures (e.g: due to a crash of the daq script) (??)
      * The data is manually mirrored (~daily) via globus to a storage element at TTU
 <!--     * As the disk is getting full, the ntuple files (~600GB already copied at TTU) are manually moved to =pcdreamdaq:/mnt/storage/ntuple= -->
   * The storage element at TTU 
      * Used to mirror, via GlobusOnline, the =pcdream:/mnt/storage/= disk
      * ~Daily TTU mirror procedure: Note: REQUIRES the TTU endpoint to be activated by Michele, Sehwook or Alan by grid proxy ahead of time to work
         * Login to the dreamtest account on pcdreamdaq2, or to any other account that has ssh keys stored in the Globus dreamdaq identity store
         * Execute: =ssh -t dreamdaq@cli.globusonline.org scp -r -s 0 -v pcdreamdaq2personal:storage/ TTU-Sigmorgh-DREAM:/lustre/hep/osg/dream/2015/=
      * The script =~/bin/ttuls= from the dreamtest directory (on pcdreamdaq2) lists files hosted in the storage element at TTU. The =bin= area is in the path for the dreamtext account, so the command =ttuls= should work without prefix. <br /> e.g.: =~ttuls data= or =ttuls data/datafile_run100*.dat= for a limited range, etc.
      * How to retrieve data by anyone in DREAM from the storage element at TTU: .see GridHelp / GridSetup / GlobusOnline / DataTransfer pages
 <!--  * Second USB disk: =pcdream:/mnt/bak2/working_bak= (not used in 2014?)
      * It stores ntuples as the =pcdream:/mnt/wd/working_bak= disk becomes full -->

---+++ Where are the data files?
   1 In =pcdreamdaq2:~dreamtest/storage/=
   1 In a disk at TTU. Use the =ttuls data= script to lists them ([[https://twiki.cern.ch/twiki/bin/view/DREAM/GlobusOnline][How to copy ntuple on your PC]])

---+++ Where are the pedestal files?
   1 In =pcdream:/mnt/storage/pedestal=
   1 In a disk at TTU. Use the =ttuls pedestal= script to lists them ([[https://twiki.cern.ch/twiki/bin/view/DREAM/GlobusOnline][How to copy ntuple on your PC]])

---+++ Where are the ntuple files?
   1 In =pcdreamdaq2:/mnt/storage/ntuple= <!-- or =pcdreamdaq2:~dreatmtest/storagte/ntuple= -->
   1 In a disk at TTU. Use the =ttuls ntuple= script to lists them ([[https://twiki.cern.ch/twiki/bin/view/DREAM/GlobusOnline][How to copy ntuple on your PC]])

<verbatim>
[pcdream] /home/dreamtest/working/ntuple >              DRS 32 channels
[pcdream] /home/dreamtest/working/ntuple/silvia >     DRS read with 16 channels
</verbatim>

---+++ Example macros

[[https://twiki.cern.ch/twiki/bin/view/DREAM/ExampleMacros][A few example macro to analyze the DREAM ntuples.]]
---+++ Useful information to do analysis:

   * [[https://twiki.cern.ch/twiki/bin/view/DREAM/GridSetup][How to connect to TTU via Grid]]
   * [[https://twiki.cern.ch/twiki/bin/view/DREAM/GridHelp][How to register at the Dream VO]]
   * [[https://twiki.cern.ch/twiki/bin/view/DREAM/NtupleProductionAtCERN][How to produce Ntuple at cern]]
   * [[https://twiki.cern.ch/twiki/bin/view/DREAM/NtupleProductionAtTTU][How to produce Ntuple at TTU]]
   * [[https://twiki.cern.ch/twiki/bin/view/DREAM/GlobusOnline][How to copy ntuple on your PC]]

Main.AlanSill - 2015-11-01

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who
pdf	LOG_JULY_2011_1part.pdf	r1	manage	14266.0 K	2011-11-04 - 01:05	GabriellaGaudio
pdf	RunList.pdf	r1	manage	68.0 K	2011-11-04 - 01:09	GabriellaGaudio
pdf	logbook_July_2011_part2.pdf	r1	manage	8656.3 K	2011-11-04 - 01:07	GabriellaGaudio

Topic revision: r7 - 2015-11-02 - AlanSill

DREAM

Public webs

- Cern Search
- TWiki Search
- Google Search
DREAM All webs

Copyright &© 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback