Tutorial on How to Analyze Pixel Data

Pixel Reconstruction Flow

Steps (described in detail here: https://twiki.cern.ch/twiki/bin/view/CMS/PixelOfflineSoftware)

What is the Pixel Ntuplizer

How to Find the Pixel Ntuple

  • The latest data and MC ntuples, as of 04/24/2009, are:
    • Data Ntuple
      • Location: /castor/cern.ch/cms/store/group/tracker/pixel/PixelNtuplizer/
      • File: SuperPointing_CRAFT_ALL_V11_NtuplizerV00-01-03_Data_AllTrkAlgos.root
      • made from /Cosmics/Commissioning08_CRAFT_ALL_V11_227_Tosca090216_ReReco_FromSuperPointing_v2/RAW-RECO in DBS
      • Global Tag: CRAFT_ALL_V11
      • TrackRefitter: All algorithms: ctfWithMaterialTracksP5, rsWithMaterialTracksP5, cosmictrackfinderP5
      • CMSSW release: CMSSW_2_2_7
      • Version of Ntuplizer: DPGAnalysis/SiPixelTools V00-01-03
      • produced by: Matthew Searle
    • MC Ntuple with increased thresholds bpix/fpix = 5200/4500
      • Location: /castor/cern.ch/cms/store/group/tracker/pixel/PixelNtuplizer/
      • Files: SuperPointing_COSMMC_22X_V1_newthresholds_NtuplizerV00-01-03_MC_alltrkalgos.root
      • made from /TkAlCosmics4T/Summer08_COSMMC_21X_V1_SuperPointing_v4/RAW-RECO in DBS
      • global Tag: COSMMC_22X_V1
      • TrackRefitter: All algorithms: ctfWithMaterialTracksP5, rsWithMaterialTracksP5, cosmictrackfinderP5
      • Version of Ntuplizer: V00-01-03 DPGAnalysis/SiPixelTools
      • release used for ntupling CMSSW_2_2_7
      • produced by: GavrilGiurgiu

  • To access the pixel ntuple, log into the lxplus machines at CERN:
    • [ggiurgiu@ggiurgiu ~]$ ssh lxplus.cern.ch
  • List all files in the PixelNtuplizer area:
    • [lxcmsf2] nsls /castor/cern.ch/cms/store/group/tracker/pixel/PixelNtuplizer/
      • ...
      • SuperPointing_CRAFT_ALL_V11_NtuplizerV00-01-03_Data_AllTrkAlgos.root
      • ...
  • Copy the pixel ntuple to your scratch (or tmp) area:
    • [lxcmsf2] /localscratch/g/ggiurgiu > rfcp /castor/cern.ch/cms/store/group/tracker/pixel/PixelNtuplize /SuperPointing_CRAFT_ALL_V11_NtuplizerV00-01-03_Data_AllTrkAlgos.root ./
  • The file is 329M so it's not hard to copy
  • It takes 10 minutes to copy it by simple scp from CASTOR at cern to my scratch area at FNAL. (Fancier methods might be used to copy (maybe Phedex ?) but not worth the complication)
  • You can find the latest data pixel ntuple on FNAL machine:
    • cmslpc.fnal.gov:/uscms_data/d2/ggiurgiu/SuperPointing_CRAFT_ALL_V11_NtuplizerV00-01-03_Data_AllTrkAlgos.root
  • You can find the latest MC pixel ntuple in the same place:
    • cmslpc.fnal.gov:/uscms_data/d2/ggiurgiu/SuperPointing_COSMMC_22X_V1_newthresholds_NtuplizerV00-01-03_MC_alltrkalgos.root

How to Run the Pixel Ntuplizer Yourself

  • Get a kerberos ticket and log into a cmslpc.fnal.gov machine:

[ggiurgiu@ggiurgiu ~]$ kinit ... [ggiurgiu@ggiurgiu ~]$ ssh cmslpc.fnal.gov

  • Setup a CMSSW project somewhere in your area:

[ggiurgiu@cmslpc03 ~/work]$ scramv1 project CMSSW CMSSW_2_2_7

[ggiurgiu@cmslpc03 ~/work]$ cd CMSSW_2_2_7/

[ggiurgiu@cmslpc03 CMSSW_2_2_7]$ eval `scramv1 runtime -csh`

  • Get a CERN kerberos ticket. It is needed to check out packages from CVS

[ggiurgiu@cmslpc07 CMSSW_2_2_7]$ kserver_init ...

  • Go the the source area "src":

[ggiurgiu@cmslpc07 CMSSW_2_2_7]$ cd src/

  • Check out the pixel ntuplizer package:

[ggiurgiu@cmslpc07 CMSSW_2_2_7]$ cvs co -r V00-01-03 DPGAnalysis/SiPixelTools

  • Compile and build the package:

[ggiurgiu@cmslpc07 src]$ scramv1 b

  • Go to the "test" area in this package:

[ggiurgiu@cmslpc07 src]$ cd DPGAnalysis/SiPixelTools/test/

  • The configuration file used to run the ntuplizer is:


  • Important things in the configuration file:
    • Global tag: process.GlobalTag.globaltag = "CRAFT_ALL_V11::All" (a global tag is a set of Data Base (DB) conditions)
    • Input file: /store/data/Commissioning08/Cosmics/RAW-RECO/CRAFT_ALL_V9_SuperPointing_225-v3/0006/26B79166-7C00-DE11-AA81-0030486790A0.root
    • Output file: SuperPointing_test_threealgos.root

  • Run the ntuplizer:

[ggiurgiu@cmslpc07 test]$ cmsRun pixelNtuplizer_RealData_cfg38T.py

  • Will see some scary printouts:

2009-04-23 11:57:38 DQMNet[17321]: NOTE: DQM server at localhost:9090 is unavailable. Connection will be established automatically on the background once the server becomes available. Error from the attempt was: Network operation connect() failed (because of System error: Connection refused (<#111,#108>))

  • This is OK, I think... as long as we get to see these printouts:

23-Apr-2009 11:58:59 CDT Initiating request to open file dcap://cmsdca3.fnal.gov:24142/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Commissioning08/Cosmics/RAW-RECO/CRAFT_ALL_V9_SuperPointing_225-v3/0006/26B79166-7C00-DE11-AA81-0030486790A0.root

23-Apr-2009 11:59:28 CDT Successfully opened file dcap://cmsdca3.fnal.gov:24142/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Commissioning08/Cosmics/RAW-RECO/CRAFT_ALL_V9_SuperPointing_225-v3/0006/26B79166-7C00-DE11-AA81-0030486790A0.root


Begin processing the 1st record. Run 68129, Event 453, LumiSection 1 at 23-Apr-2009 12:01:28 CDT

Begin processing the 11th record. Run 68129, Event 5065, LumiSection 1 at 23-Apr-2009 12:01:33 CDT

Begin processing the 21st record. Run 68129, Event 19065, LumiSection 1 at 23-Apr-2009 12:01:34 CDT


Begin processing the 4641st record. Run 68129, Event 10903429, LumiSection 205 at 23-Apr-2009 12:03:25 CDT

23-Apr-2009 12:03:25 CDT Closed file dcap://cmsdca3.fnal.gov:24142/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Commissioning08/Cosmics/RAW-RECO/CRAFT_ALL_V9_SuperPointing_225-v3/0006/26B79166-7C00-DE11-AA81-0030486790A0.root

MessageLogger Summary ...

How to Quickly Look the Pixel Ntuple

  • If you got here, you should have the output ntuple in your working directory:
  • You can look at the ntuple:

[ggiurgiu@cmslpc07 test]$ root -l SuperPointing_test_threealgos.root

root [1] TBrowser t

  • Click on "ROOT Files"
  • Click on file name: "SuperPointing...root"
  • There are three folders: "cosmtfNtuple", "ctfNtuple", "rsNtuple".
  • Each folder corresponds to the tracking algorithm used to reconstruct the cosmic muon tracks:
    • "Cosmic Track Finder = cosmtf"
    • "Combinatorial Track Finder = ctf"
    • "Road Search = rs"

  • Cosmic Track Finder is an algorithm specialized for cosmics. It reconstructs one track per event.
  • Combinatorial Track Finder and Road Search are used to reconstruct collisions as well

  • Inside each of the 3 folders:
    • PixNtuple - filled for each cluster
    • TrackNtuple - filled for each track
      • Click on "PixNtuple" yellow folder
      • Click on "RecHit" yellow folder
      • Click on "residualX" green leaf
      • You get a plot of the X residuals. To zoom in and apply some selection cuts, type this in the Root session:

root [5] PixNtuple->Draw("residualX", "abs(residualX)<0.1 && ladder>0")

      • You get a zoomed in plot of the X residuals for the BPIX detector (units on the horisontal axis are in cm)

How to Write a Root Script to analyze Pixel Ntuple

  • A more programmatic approach is to write a Root script to analyze for example the "ctfNtuple":

[ggiurgiu@cmslpc07 test]$ root -l

root [0] TChain chain("ctfNtuple/PixNtuple")

root [1] chain.Add("*.root") - this command adds all .root files in the current directory into the chain. In this case just SuperPointing_test_threealgos.root

root [2] chain.MakeClass("test")

  • Now, in your working directory, you have a skeleton code (test.h, test.C) that can be run on the pixel ntuple.
  • In the header file "test.h" are all the variable you can look at:
    • Int_t evt_run;
    • Int_t evt_evtnum;
    • ...
    • Float_t Cluster_charge;
    • ...
    • Float_t RecHit_localX;
    • Float_t RecHit_localY;
    • Float_t RecHit_residualX;
    • Float_t RecHit_residualY;
    • ...
    • Float_t track_pt;
    • Float_t track_globalEta;
    • ...

  • Further down in the header file "test.h" is the name of the file that you will analyze:
    • chain->Add("/uscms/home/ggiurgiu/work/CMSSW_2_2_7/src/DPGAnalysis/SiPixelTools/test/SuperPointing_test_threealgos.root/ctfNtuple/PixNtuple");

  • In the source file "test.C" you can write your analysis code:
  • Add this line at the top of the file after the other "include" statements: #include <iostream>
  • Replace: Long64_t nentries = fChain->GetEntriesFast();
  • With: Long64_t nentries = fChain->GetEntries(); cout << "nentries = " << nentries << endl;
  • Compile and run the Root script:

[ggiurgiu@cmslpc07 test]$ root -l

root [0] .L test.C++

... blah, blah, blah, some Root messages...

root [1] test mmm

root [2] mmm.Loop()

  • You just ran over the ntuple, but didn't really do anything yet except for printing out the total number of entries (clusters) in the ntuple.
  • A small example on how to plot X and Y residuals for BPIX and FPIX is located here:/uscms/home/ggiurgiu/work/CMSSW_2_2_7/src/DPGAnalysis/SiPixelTools/test/
    • test.h
    • test.C
  • Copy and run this example in your work area:

[ggiurgiu@cmslpc07 test]$ root -l

root [0] .L test.C++

... blah, blah, blah, some Root messages...

root [1] test abc

root [2] abc.Loop()

  • Another example script:
    • /uscms/home/ggiurgiu/work/CMSSW_2_2_7/src/DPGAnalysis/SiPixelTools/test/test_more_advanced.h
    • /uscms/home/ggiurgiu/work/CMSSW_2_2_7/src/DPGAnalysis/SiPixelTools/test/test_more_advanced.C
  • This script plots the cluster charge and size as well as the cluster track angle alpha between the track and the
local X axis. It also takes a certain cluster in the ntuple (I just took cluster number 1500) and shows its geometrical shape:

[ggiurgiu@cmslpc07 test]$ root -l

root [0] .L test_more_advanced.C++

... blah, blah, blah, some Root messages...

root [1] test_more_advanced xyz

root [2] xyz.Loop()

  • You can modify the above script to do your own analysis.

How to modify the Pixel Ntuplizer

  • The actual code is in: DPGAnalysis/SiPixelTools/plugins/
  • Edit DPGAnalysis/SiPixelTools/plugins/PixelNtuplizer_RealData.cc and .hh files
    • A trivial example: put in some printouts by adding this line at the begining of analyze(...) function
      • cout << "I am adding my stuff here......................................................" << endl;
      • go back to DPGAnalysis/SiPixelTools/
      • compile and build again: scramv1 b
      • go to DPGAnalysis/SiPixelTools/test/
      • now you can run again the ntuplizer which includes your changes:

[ggiurgiu@cmslpc07 test]$ cmsRun pixelNtuplizer_RealData_cfg38T.py

How to Run the Pixel Ntuplizer (or Any Other Module for that Matter) Using CRAB

  • I only did this using CERN machines, lxplus.cern.ch, (doing it from FERMILAB machines, cmslpc.fnal.gov, should be similar, but did not get to test it yet)
  • Log into CERN cluster: ssh lxplus.cern.ch
  • source /afs/cern.ch/cms/LCG/LCG-2/UI/cms_ui_env.csh
  • cd public/
  • scramv1 project CMSSW CMSSW_2_2_7
  • cd CMSSW_2_2_7
  • eval `scramv1 runtime -csh`
  • source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.csh
  • cd src/
  • cvs co -r V00-01-03 DPGAnalysis/SiPixelTools
  • scramv1 b
  • cd DPGAnalysis/SiPixelTools/test/ntuplizer_scripts/
  • cp ../pixelNtuplizer_RealData_cfg38T.py ./
  • mv crab_for_CAF.cfg crab.cfg
  • Edit crab.cfg as follows:
    • datasetpath=/Cosmics/Commissioning08_CRAFT_ALL_V11_227_Tosca090216_ReReco_FromSuperPointing_v2/RAW-RECO
    • pset=pixelNtuplizer_RealData_cfg38T.py
    • total_number_of_events=1000 # after making sure everything runs fine you can run over all events by setting: total_number_of_events=-1 (-1 means all available events in the input files)
    • number_of_jobs = 5 # when running on all events should split this in more jobs (200 ?)
    • user_remote_dir=/user/g/ggiurgiu/ # set the output location in your castor area
    • return_data = 1 # this setting will return your output ntuples in your home area DPGAnalysis/SiPixelTools/test/ntuplizer_scripts/crab_x_xyz_xyz/res
    • return_data = 0 # this setting will put your output ntuples in castor as specified above;
    • if return_data = 0, then also set copy_data=1
  • Save your updates to crab.cfg
  • crab -create
    • You should see:
      • crab. Creating 5 jobs, please wait...
      • crab. Total of 5 jobs created.
  • crab -submit
    • You should see:
      • crab. Total of 5 jobs submitted.

  • Now you can wait and pray smile Or, every few seconds you can check the status of your jobs:
  • crab -status
    • You should see something like this

1 RUN lxb7710

2 RUN lxb8217

3 RUN lxb7708

4 RUN lxb8220

5 DONE lxb7716

  • When all jobs are done you will find the output either in your castor area:
    • nsls /castor/cern.ch/user/g/ggiurgiu/
  • ... or in your home area:
    • DPGAnalysis/SiPixelTools/test/ntuplizer_scripts/crab_x_xyz_xyz/res
  • The output location depends on the setings of "return_data" and "set copy_data" in crab.cfg
  • The log files are here: DPGAnalysis/SiPixelTools/test/ntuplizer_scripts/crab_0_090424_181059/res/

  • You can look at your output directly from castor:
    • root [0] TFile *f = TFile::Open ( "rfio:/castor/cern.ch/user/g/ggiurgiu/SuperPointing_test_threealgos_4.root" )
  • ... or you can copy it from castor to your area:
    • rfcp /castor/cern.ch/user/g/ggiurgiu/my_file.root ./

  • If everything looks good in your output files, you can go ahead and change crab.cfg to run on all events in your sample:
    • total_number_of_events=-1
    • number_of_jobs = 100 or 200 or something like that...

-- GavrilGiurgiu - 23 Apr 2009

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2009-04-24 - GavrilGiurgiu
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback