TbUT < LHCb < TWiki

LHCb Web>WebPreferences>TbUT (2016-10-31, StevenBlusk) (raw view)
<!--* Set ALLOWTOPICVIEW = lhcb-general-->

---+!!  TbUT analysis software

This page provides information about the TbUT package. The software is a part of the  [[https://lbtwiki.cern.ch/bin/view/VELO/Tpx3TestbeamSoftware][Kepler project]]. The main aim of this software is to perform whole processing of the testbeam data using [[http://lhcb-comp.web.cern.ch/lhcb-comp/Frameworks/Gaudi/][Gaudi framework]]. 
%TOC%

---+  Contact persons
Christopher Betancourt [[mailto:christopher.betancourt@cern.ch][Mail]], Iaroslava Bezshyiko [[mailto:iaroslava.bezshyiko@cern.ch][Mail]], Steven Blusk [[mailto:sblusk@syr.edu][Mail]], Adam Dendek  [[mailto:adendek@cern.ch][Mail]]

---+  Installation
   * To install (you need to be in the lhcb computing group on lxplus):
   <pre>
lb-dev Kepler v3r0  
cd ./KeplerDev_v3r0
git clone https://github.com/yabezsh/Tb.git
make -j4 make install
 </pre>
   * Make sure, that in your config file (in Tb/.git), you have a line that has a URL with your username included:
   <pre>
     url = https://sblusk@github.com/yabezsh/Tb.git
 </pre>

---+  Setup


   * After each time you open a new terminal
   <pre>
SetupProject LHCb
cd ~/KeplerDev_v3r0
./run bash 
source /afs/cern.ch/project/eos/installation/lhcb/etc/setup.sh
eosmount ~/eos
cd Tb/
 </pre> 

   * To run over the data from May testbeam: 
 For tests, please, go to Analysys.py and Analysis_onlyPlots.py and change the line
outputPath = "$KEPLERROOT/eos_"+str(sigFile.split('-')[3])+"/lhcb/testbeam/ut/TemporaryData/May2016/DQMTest"
to the folder where you want to have your outputs.


---+  Running the TbUT software
For tests, please, go to Analysys.py and Analysis_onlyPlots.py and change the line
outputPath = "$KEPLERROOT/eos_"+str(sigFile.split('-')[3])+"/lhcb/testbeam/ut/TemporaryData/May2016/DQMTest"
to the folder where you want to have your outputs.

After that you can run (Oct 2016 data example):
<pre>
python -u RunAnalysis.py --board A3_Lower  --PA TTV2_01 --DUTRun 2259 --mode local --mask 1 
</pre>

Here, a May 2016 data example:
<pre>
python -u RunAnalysis.py --board F1 --PA FanUp --DUTRun 1073 --mode local --evts 100000
</pre>

The latter example will produce all plots in "outputPath"/F1/FanUp/output_1073/Plots/
If you want to update plots without reproducing of all data analyze you can add flag "--onlyPlots 1"
If you want to run over all events in run just take away the flag "--evts" (usually 1000000 events; it's better to go over all of them to see nice plots).
You can run it in terminal (use "--mode local") or in batch ("--mode batch").

If you will not specify a run number ("--DUTRun 1073") it will start to process all runs from testbeam data sheet ( [[https://docs.google.com/spreadsheets/d/1xYgpb3AxP4JJ7Dkx1YeNV75q0nNxwPrvBhoMgyBD0N4/edit]] )

Two canvases will be produced, one with Alignment information and one with Performance information. They should be hecked by the user. 
Below is an example for Run 15046, sensor M1, 300 V bias, 0 degree rotation angle

   * *Alignment Plots*

<th width="100%"> <img width="800" alt="Thresholds" src="%ATTACHURLPATH%/AlignmentPlots_M1_s2_vb300.png" /> 

   * *Performance Plots*

<th width="100%"> <img width="800" alt="Thresholds" src="%ATTACHURLPATH%/PerformancePlots_M1_s2_vb300.png" /> 

---+  Known issues
   * The TbUT cannot read eos files. *workaround:  copy analyzed file into your work directory or use eosmount as shown above. 
   * TbUT (run mode) need to be executed *twice* to correct calculate noise and extract clusters. 
   * The tools should inherit from GaudTool class.
   * The unit test have to be implemented
 
---+  Developers section
---++ General overview
Every  TbUT component class is a member of =TbUT= namespace. 
---+++ Algorithms
The TbUT is a Gaudi based application. The application is separated into two sets of Gaudi algorithms. First group is responsible for processing the testbeam data. This algorithms inherit from *GaudiAlgorithm* class.
The names of it contains string =Algorithm= for instance[[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTRawDataReaderAlgorithm.cpp][TbUTRawDataReaderAlgorithm]] . 

*To understand TbUT source code, the algorithm classes should be treated as a starting point !*
 
The second group was designed to prepare monitoring plots. This classes inherit from base called [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTDataMonitorAlgorithm.h][TbUTDataMonitorAlgorithm]] (  see also  [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTDataMonitorAlgorithm.cpp][ implementation ]]). 
According to the file name convention it's names have to contains string  =DataMonitorAlgorithm= in their's file names. E.g.  [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTPedestalSubtractorDataMonitorAlgorithm.cpp][TbUTPedestalSubtractorDataMonitorAlgorithm]] .
 
---+++ Tools
That kind of classes are responsible of performing the data modification. Each of this inherit from virtual interface called  [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTIProcessingEngine.h][IProcessingEngine]], therefore they have to implement =processEvent= method. 

---+++ Tool's Factories
This kind of classes are responsible for dynamic, depend of options, creation of proper version of tools. This classes based on [[https://en.wikipedia.org/wiki/Factory_method_pattern][Factory design pattern]] (see also the [[https://www.youtube.com/watch?v=N8_wwa90tzM&list=PL6A29DA14366FF6C3&index=13][beautiful explanation]] why is is so useful).  See for example [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTRawDataFactory.h][Raw Data Reader Factory]] and it [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTRawDataFactory.cpp][implementation]]

---+++ Other types
   * *Data structures* (inherits from  =GaudiKernel/DataObject= ) are used to be manipulated via the tools and be stored in TES. The TbUT package contains  [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTRawData.h][Raw Data]] and [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTCluster.h][Clusters]] .  
   * *Additional data structures*. Can be distinguished [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTNoise.cpp][noise]] and [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTPedestal.cpp][pedestals]]. Their are used as a auxiliary structures by the tools. 
   * *Additional services*. Used for manipulation of the *additional data structures*. See for instance  [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTNoiseCalculator.h][Noise Calculator]]. Each of this services has own interface ( e.g. Noise calculator are based on [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTINoiseCalculator.h][INoiseCalculator]]).  
 
---+++ TbUT high level architecture

   <th width="40%"> <img width="600" align="middle" alt="TbUT high Level architecture" src="%ATTACHURL%/corr_example.pdf" />

---++ Description of the algorithms

---+++ Raw Data Reader

This algorithm is designed to read raw data from specific input file. Currently TbUT can work with files produced by Alibava and Mamba DAQs. The heart of this algorithm is tool inherit from interface [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTIDataReader.h][IDataReader]]. This tool acts [[https://en.wikipedia.org/wiki/Facade_pattern][fascade]] for specific DAQ reader. 

The execution of Raw Data Reader can be modified using options:

 | *option name* | *brief description* |
      | InputDataType | type of string, choose one of DAQs data format (Mamba or Alibava)  |
      | inputData | type of string, path to the input data. |
      | standalone | type of  boolean, if is true the run is executed with Kepler (haven't been tested yet)  |

---+++ Pedestal Subtractor 
The pedestal subtraction algorithm has two phases. In first, optional one, the pedestal values are calculated.  This phase is also called training. During the second one the determined pedestal values are subtracted from the raw ADC data.
From technical point of view the [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTPedestalSubtractorAlgorithm.cpp][Pedestal Subtractor Algorithm]] calls two types of tools. One of them calculates or retrieves pedestals values( this classes inherit from [[https://svnweb.cern.ch/trac/lhcb/browser/Kepler/trunk/Tb/TbUT/src/TbUTIPedestalFollowing.h][I pedestal following]] ) and the second one remove pedestals values from the raw data. See data flow. 

  <img width="400"  src="%ATTACHURLPATH%/Pedestal_Subtrator.png" alt="Pedestal_Subtrator.png"  /></verbatim>

The execution of Pedestal Subtractor Algorithm can be modified using options:

 | *option name* | *brief description* |
      | FollowingOption | type of string, choose type of execution (training  or run)  |
      | treningEntry | type of integer, number of training entry |
      | ChannelMaskInputLocation | type of string, location of the channel mask |
      | PedestalInputFile | type of string, path to the location where previously calculated pedestals are stored |
      | PedestalOutputFile | type of string, path to the location where calculated pedestal have to be stored  |
      | standalone | type of  boolean, if is true the run is executed with Kepler (haven't been tested yet)  |

---++++ Pedestal following
From the mathematical point of view the calculation of the pedestal can be described as a running average. In every training event then pedestal sum is updated. This update takes into account the previous value of the pedestal sum and the current ADC count. The pedestal sum is calculated for each channel separately. To be more precisely, the pedestal sum, %$p_i(n)$%, for channel %$n$% and event %$i$% can be expressed as follows:

%$P_{i}^{sum}(n+1)=P^{sum}_{i}(n) + \frac{\Delta_{i}(n+1)}{N}$%

Where the %$\Delta_{i}(n+1)$% is a event to pedestal correction. This correction is expressed as:

%$\Delta_{i}(n+1)=ADC_{i}(n+1)-P^{sum}_{i}(n)$%

To increase the suitability of the pedestal the limit for correction is applied. If the condition:
%$ \left| \Delta_{i}(n+1) \right| \leq 15  $%
is not fulfilled the correction value is set to 15.

To determinate the pedestal values the pedestal sum should be normalized, so:
%$p_{i}=\frac{P^{sum}_{i}}{N}$%

---++++ Pedestal removing algorithm
The second phase of the pedestal subtraction algorithm is subtraction determined pedestal values from the raw data. This procedure can be expressed as fallows:

%$ADC_{i}=ADC^{RAW}_{i}-p_{i}$%

where: %$ADC_{i}$% is signal value after pedestal subtraction for event %$i$%, %$ADC^{RAW}_{i}$% is a raw data and the %$p_{i}$% is the pedestal value. Each of this quantities are in the unit of ADC counts. 


---+  Additional scripts
   All scripts are stored in =TbUT/scripts= directory. (Not yet committed)  
   * *Noise monitor* This script takes as an input noise file. The exemplary output: 

 <th width="40%"> <img width="400" alt="Thresholds" src="%ATTACHURLPATH%/Thresholds.png" />

---+  Operations with the git repository

   * To update your local copy with the latest version in the git repository (you will need a git account)
<pre>
git pull
</pre>

   * To make modifications, you will need to be added to the list of users allowed to modify (contact Iaroslava). After you make your changes:
<pre>
git add <list of files that you modified>   (w/o brackets) - designate which files will be sent to git repo (this is for new or modified files)
git commit -m"Add a comment here about your changes"
git push     (this pushes the commits to the repo... will prompt you for your password)
</pre>

* To get a completely fresh working copy, see instructions under Installation, above.


-- Main.AdamMateuszDendek - 2015-08-03
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
png	AlignmentPlots_M1_s2_vb300.png	r1	manage	42.4 K	2016-05-13 - 22:19	UTTestbeamService
png	Landau_example.png	r1	manage	0.9 K	2016-05-10 - 07:37	ChristopherBetancourt1	Example plots from runClusterWithTrackAna.C
png	Pedestal_Subtrator.png	r1	manage	53.7 K	2015-08-26 - 19:21	AdamMateuszDendek	Pedestal Subtractor Data Flow
png	PerformancePlots_M1_s2_vb300.png	r1	manage	60.7 K	2016-05-13 - 22:19	UTTestbeamService
png	TbUT_Architecture.png	r1	manage	103.6 K	2015-08-26 - 18:26	AdamMateuszDendek	TbUT high level architecture
png	Thresholds.png	r1	manage	27.8 K	2015-08-26 - 13:59	AdamMateuszDendek
png	corr_example.png	r1	manage	4.3 K	2016-05-10 - 07:39	ChristopherBetancourt1	Example plots from runClusterWithTrackAna.C
png	profile_example.png	r1	manage	0.9 K	2016-05-10 - 07:40	ChristopherBetancourt1	Example plots from runClusterWithTrackAna.C
png	res_example.png	r1	manage	0.8 K	2016-05-10 - 07:40	ChristopherBetancourt1	Example plots from runClusterWithTrackAna.C
Topic revision: r25 - 2016-10-31 - StevenBlusk
LHCb
Public webs
Welcome Guest
- Cern Search
- TWiki Search
- Google Search
LHCb All webs