Latest FCal - related projects

Luminosity Measurement with FCalMon

In order to investigate the drift in the luminosity measurements wrt BCM, we are trying to measure the luminosity with FCal, preferably independently on the actual luminosity information:

Idea is to reproduce the luminosity trend with measuring the energy flow as a function of lumi blocks and investigate the changes throughout the year 2012 on dedicated minbias runs.

Analysis steps:

  1. Define and understand basic observables such as <En> and Σ En
  2. Reproduce the luminosity trend as visible in standard plots
  3. reproduce the luminosity trend with help of energy and not direct luminosity information (to be independent)
  4. Normalize to get a constant or a known shape (i.e. Samir's banana)
  5. Measure that as a function of run number i.e. time to investigate the drift
Studies so far (from 2013, please add more if you have):

  1. First Study of several runs
  2. First Bunch in train added
  3. Full run 204772 study
  4. Issue in Σ En
  5. Issue Solved
  6. Next Steps
  7. More interesting distributions
  8. Negative energy tail subtracted
  9. Negative energy tail subtracted, period B compared
  10. RMS of noise, period B compared
  11. comparison of MB and ZB streams
  12. followup on ZB, Gaussian fits
  13. Dependence on <>
  14. Dependence on |η|, fit of <>
  15. Improved fitting and resolved difference between s
  16. First Calibration derivation
  17. adding C side and high eta calibration
  18. Outliers Study and Solution
  19. Outliers Fix, Merging of runs and RMS systematics

The FCalMonD3PD package

FCalMonD3PD replaces the FCalMon package. The code is available for browsing here or here.

Making the FCalMonD3PD

Before you do anything, check that the athena release is at least as new as the ESD/DESD that you use. To set it up, follow the instructions below:


# 1. Set up athena and check out the code
mkdir 17.2.3.3.1
cd 17.2.3.3.1
export AtlasSetup=/afs/cern.ch/atlas/software/dist/AtlasSetup
alias asetup='source $AtlasSetup/scripts/asetup.sh'
asetup 17.2.3.3.1,here
svn co svn+ssh://svn.cern.ch/reps/atlasgroups/Detectors/LAr/FCal/FCalMonD3PD/trunk FCal/FCalMonD3PD

# 2. Compile it..
cd $TestArea/FCal/FCalMonD3PD/cmt
cmt config
source setup.sh
cmt bro gmake | tee gmake.log
source setup.sh

# 3. Test-run FCalMonD3PD (takes about 5 minutes)
cd ../share/
time athena -c EvtMax=5 DPD_topOptions.py >& athena.log
#change EvtMax=-1 to run over all events locally

# 4. Submit to the grid to make FCalMonD3PDs (look in SubmitGrid.sh for more examples)
# first setup panda
source /afs/cern.ch/atlas/offline/external/GRID/DA/panda-client/latest/etc/panda/panda_setup.sh
# a) MC
pathena --inDS mc12_8TeV.129073.Pythia_AMBT2BCTEQ6L1_minbias_ND.recon.ESD.e1122_s1468_s1470_r3581/ --nGBPerJob=10 --outDS user.${USER}.testing.mc12.08May2012_22h01 DPD_topOptions.py
# b) data
# first get a GRL, for example from the [[http://atlasdqm.web.cern.ch/atlasdqm/grlgen/][Official Automatic Production of GRLs]]
# to get a GRL for a particular period, use the dqu_grl_extract_period.py script on the periodAllYear official GRL
# for example, dqu_grl_extract_period.py -t data12_8TeV -p A8 data12_8TeV.periodAllYear_JetEtMiss.xml
# or (for experts only) from the [[https://atlasdqm.cern.ch/grl/][Good Run List Generator]].
pathena --goodRunListXML data12_8TeV.periodA8_DetStatus-v41-pro13_CoolRunQuery-00-04-08_JetEtMiss.xml --goodRunListDataType DESD_MBIAS --nGBPerJob=10 --outDS user.${USER}.testing.16Apr2012_23h31 DPD_topOptions.py

# 5. get grid results
# a) MC
dq2-get -f '*D3PD*.root' user.${USER}.testing.mc12.08May2012_22h01/
hadd FCalMonD3PD_MC.root user.${USER}.testing.mc12.08May2012_22h01/*D3PD*.root
# b) data
dq2-get -f '*D3PD*.root' user.${USER}.testing.16Apr2012_23h31/
hadd FCalMonD3PD_data.root user.${USER}.testing.16Apr2012_23h31/*D3PD*.root

# 6. make some quick plots
root -l FCalMonD3PD_MC.root FCalMonD3PD_data.root

# draw Energy for the first 1000 events
((TTree*)_file0->Get("caloD3PD"))->Draw("cc_fcal_E>>ha(80,-80E3,80E3)","cc_fcal_eta>0","",1000)
((TTree*)_file0->Get("caloD3PD"))->Draw("cc_fcal_E>>hc(80,-80E3,80E3)","cc_fcal_eta<0","",1000)
TH1F *histA = (TH1F*)gROOT->FindObject("ha")
TH1F *histC = (TH1F*)gROOT->FindObject("hc")
histA->SetLineColor(2)
histA->Draw()
histC->Draw("sames")

Slimming the FCalMonD3PD

Optional: slim the FCalMonD3PDs (remove some branches that are not used), reduces size by 10x

Adds some FCal cell geometry information.


# don't need to compile because we are only slimming, not skimming (definitions below)
# you can compile to see that there are no errors, but compiling takes time and is not necessary here:
# skim: loops over events to keep only those passing certain criteria
# slim: one only removes branches, no looping needed, therefore does not benefit from compiling

# locally
cd $TestArea/FCal/FCalMonD3PD/run
root -b -q -l SlimFCalMonD3PD.C

# submit to grid instead of running locally
# first setup panda
source /afs/cern.ch/atlas/offline/external/GRID/DA/panda-client/latest/etc/panda/panda_setup.sh
# call prun (look in SubmitGrid.sh for more examples)
prun --exec "rm input.txt; echo %IN | grep D3PD | sed 's/,/ /g' > input.txt; root -b -q -l SlimFCalMonD3PD.C" --outputs caloD3PD_slim.root --athenaTag 17.2.3.3.1 --inDS user.${USER}.testing.16Apr2012_23h31 --nGBPerJob 10 --outDS user.${USER}.testing_slim.16Apr2012_23h31 --useShortLivedReplicas

Analysis using the FCalMonD3PD

After you have made the FCalMonD3PDs, you can then do a D3PD analysis either locally, or on the grid.



# 0) Open a new terminal and setup standalone ROOT (compiling RootCore in athena has caused problems)
# Use ROOT v5.30.05, as this is the version packaged with athena 17.2.3.3.1
# This setup is for a node running SLC5; running on SLC6 is not guaranteed to work, and will require a
# a different set of commands for ROOT setup.

source /afs/cern.ch/sw/lcg/external/gcc/4.3.2/x86_64-slc5/setup.sh
cd /afs/cern.ch/sw/lcg/app/releases/ROOT/5.30.05/x86_64-slc5-gcc43-opt/root/
source bin/thisroot.sh
cd -
cd FCal/FCalMonD3PD/run

# 1) checkout RootCore and compile FCalAna

# checkout RootCore
svn co svn+ssh://svn.cern.ch/reps/atlasoff/PhysicsAnalysis/D3PDTools/RootCore/tags/RootCore-00-01-17 RootCore
cd RootCore/
./configure
cd ..
# run this each time a new terminal is opened:
source RootCore/scripts/setup.sh

# use RootCore to get needed packages
$ROOTCOREDIR/scripts/checkout.sh packages.txt
$ROOTCOREDIR/scripts/find_packages.sh
$ROOTCOREDIR/scripts/compile.sh >& compile.log

# something went wrong during compile? clean and recompile
# $ROOTCOREDIR/scripts/clean.sh

# compile FCalAna
sh make.sh

# before running the D3PD analysis a few things are needed:
## i) edit input.txt so that it contains the path/name of the FCalMonD3PD output root file
## ii) for MC we need to do PileupReweighting
# a) go to https://atlas-lumicalc.cern.ch/
# b) at "GRL XML File" box put the location of the GRL XML file used for grid submit
# c) add the --plots modifier to the "Other Options" box
# d) click "Calculate Luminosity" and wait
# e) when it is done, click on "iLumiCalc.exe working directory"
# f) download the ilumicalc_histograms root file, and edit run_FCalAna.C so that "myLumiCalcFilename" to point to this file
# g) in run_FCalAna.C make sure "myConfigFilename" points to the histogram corresponding to the MC Production Campaign
# h) in run_FCalAna.C check that setPileupFiles is not commented out
## iii) for Data we need to do GoodRunList
# a) in run_FCalAna.C point "myGRL" to the GRL
# b) in run_FCalAna.C check that setupGRL is not commented out
# more flexible input/output:
# usage
# ./run_FCalAna.exe inputstring maxevents
# note:
# 1) you have to have inputfile of the syntax: input.inputstring.txt, output root file: histos.inputstring.root will be created
# 2) if no maxevents specified, all entries in the tree will be used
# 3) if no inputstring specified, standard input.txt -> histos.root will be used

# 2a) run FCalAna with run_FCalAna.exe
./run_FCalAna.exe >& log.txt

# then look into histos.root for example histograms

# b) submit to grid instead of running locally the run_FCalAna.exe (point the inDS to the slims if you made them)
# first setup panda
source /afs/cern.ch/atlas/offline/external/GRID/DA/panda-client/latest/etc/panda/panda_setup.sh
# and because we are setting up panda without athena we need to set PATHENA_GRID_SETUP_SH
export PATHENA_GRID_SETUP_SH=/afs/cern.ch/project/gd/LCG-share/current_3.2/etc/profile.d/grid_env.sh

# call prun (look in SubmitGrid.sh for more examples)
prun --exec "rm input.txt; echo %IN | grep D3PD | sed 's/,/ /g' > input.txt; run_FCalAna.exe" --useRootCore --outputs histos.root --athenaTag 17.2.3.3.1 --inDS user.${USER}.testing.16Apr2012_23h31/ --nGBPerJob 10 --outDS user.${USER}.testing.prun.16Apr2012_23h31 --extFile "ilumicalc_histograms_None_201350-201383.root,PileupReweighting/share/mc12a_defaults.prw.root"

# a few things to note
# 1) the inDS has the same name as the outDS from the FCalMonD3PD making (or slimming) stage
# 2) need to include in extFiles the histogram root files used in pileupTool in FCalAna.C
# 3) note that inDS requires a slash but outDS does not

# 3) get the histograms from the grid, and hadd them together
dq2-get -f '*.root' user.${USER}.testing.prun.16Apr2012_23h31/
hadd histos.data12.root user.${USER}.testing.prun.16Apr2012_23h31*/*.root

# 4) make plots from the histogram root files
root -b -q -l draw_overlay.C

The FCalPositionMeasurement package

FCalPositionMeasurement uses the FCalMonD3PD package to calculate the FCal modules position. The code is available for browsing here.

Making the FCalPositionMeasurement package and running the code

Before you do anything, make sure you have set up root. FCalPositionMeasurement needs a set of histograms to perform the position fits. Those histograms are produced by the code FCalHistos.cxx. FCalPositionMeasurement runs from the root file produce by FCalMonD3PD (e.g. FCalMonD3PD _data.root) or the slim root file (caloD3PD_slim.root). Next, follow the instructions below:


cd
mkdir FCalPos
cd FCalPos
svn co svn+ssh://svn.cern.ch/reps/atlasinst/Institutes/Carleton/FCalPositionMeasurement/trunk FCalPositionMeasurement
cd FCalPositionMeasurement
make
./fcal_position.exe -d -m FCal1A FCal1C -r rTrangesLay1.txt -n 200805 -g /afs/cern.ch/work/a/alainb/public/user.alainb.data12_8TeV.periodA2_JetEtMiss_LowMu_Final.120622141405/caloD3PD_small_slim.root

Please consult the README file to understand how to use the numerous arguments. After typing the above commands: the logs are printed in the txt files and the calculations are also in a root file (i.e. FCalPos _caloD3PD_small_slim.root). The results are in the file results_200805/finalShift_200805.txt, while PDF and PNG file summarized the calculation.


ls -lt
root FCalPos_caloD3PD_small_slim.root
cd results_200805
more finalShift_200805.txt

Making monitoring histogram (for Hass)

FCalPositionMeasurement makes a lot of monitoring histograms. To produce them for diagnostics, just type the following:


cd ../FCalPositionMeasurement
./FCalHistos.exe /afs/cern.ch/work/a/alainb/public/user.alainb.data12_8TeV.periodA2_JetEtMiss_LowMu_Final.120622141405/caloD3PD_small_slim.root D3PD_Run200805.root
root D3PD_Run200805.root

Look at all the histograms in root.

The FCalMon package

FCalMon uses CBNT which is going to be phased out in May 2012. FCalMonD3PD will attempt to fill its shoes. FCalMon is an athena package that run over ESD/AOD input and produce diagnostic histograms and a simple n-tuple (optional) that can be used for further analysis, for instance the position measurement. The code is available for browsing here. It is used in an effort to study the inclusive and dijet cross sections in the forward region.

FCalMonTrigger

FCalMonTrigger is able to monitor the FJ triggers by their turn on curves. It makes Per-Jet efficiency and Inclusive efficiency turn on curves. The per jet efficiency has one entry per offline jet in the denominator, and one entry per offline jet matched to the trigger-level ones in the numerator; it is normally plotted as a function of jetEt. The inclusive efficiency has an entry per offline jet. But, if the event is selected, all jets of that event contribute to the numerator, not only the matched ones.

It makes them in two eta bins: from 2.8 to 3.6, and from 3.6 to 4.4. We anticipate that the 2.8-3.6 bin, since it is the boundary region between central and forward triggers, will behave differently.

It makes them in two offline jet types, antiKt4 and antiKt6.

Running and developing the package at the lxplus cluster (including the Canadian PCs)

Here are some instructions/tips for folks that have accounts on the lxplus cluster (including the Canadian PCs at CERN atcanpc001 etc). Below are instructions on how you can set up your area this way. You only need to do this once, and then things should work smoothly every time you log in.

Checking out the code

If you already have checked out the code, you can make sure you are up to date by doing:


cd $TestArea/FCal/FCalMon
svn update

Then compile the code (if needed).

If you want to check out everything from scratch, follow the instructions below:


# 1. Set up a reasonably recent athena release (see below on how to do this smartly on the Canadian PCs)

# 2. Check out the code in the working area
cd $TestArea
svn co svn+ssh://svn.cern.ch/reps/atlasgroups/Detectors/LAr/FCal/FCalMon/trunk FCal/FCalMon
svn co svn+ssh://svn.cern.ch/reps/atlasgroups/Detectors/LAr/FCal/FCalTrigger/trunk FCal/FCalTrigger

# 2b. For FCalMonitoring we need the [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/PileupReweighting][PileupReweighting]] package
cd $TestArea/FCal/FCalMon
svn co svn+ssh://svn.cern.ch/reps/atlasoff/PhysicsAnalysis/AnalysisCommon/PileupReweighting/tags/PileupReweighting-00-00-17
cd PileupReweighting-00-00-17/cmt
make -f Makefile.Standalone clean; make -f Makefile.Standalone

# 3. Compile it..
cd $TestArea/FCal/FCalMon/cmt
cmt config
source setup.sh
cmt bro gmake | tee gmake.log
source setup.sh

# 4a. Test-run FCalMonTrigger!
# This should work out-of-the box when you run on the lxplus or Canadian PCs
# or you need to download a file and edit the jobOption
# such as using dq2-get
# source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.zsh
# dq2-get -f data10_7TeV.00161948.physics_L1Calo.merge.AOD.f286_m588._lb0271-lb0706.1 data10_7TeV.00161948.physics_L1Calo.merge.AOD.f286_m588
cd ../share/
athena FCalMonTriggerMaster.py | tee my.log
# or on the grid
# specifying dataset using GRL
# pathena FCalMonTriggerMaster.py --goodRunListXML MyLBCollection_PeriodF_jetetmiss_7TeV_plus_LUMI.xml --goodRunListDS "*physics_MinBias*" --outDS user.${USER}.FJtrigger.`date +%d%b%Y`
# specifying dataset with wildcard, MC shown here
# pathena FCalMonTriggerMaster.py --inDS mc09_7TeV.1050\*.J\*_pythia_jetjet.merge.AOD.e468_s766_s767_r1303\*/ --nGBPerJob 12 --outDS user.${USER}.MC.`date +%d%b%Y`
# then to combine all the outputs from the grid use, example:
# hadd -T added.root *AANT*.root
# which combines histograms but not Trees (-T option)
# now make some plots
cd ../../FCalTrigger
root -b -q DrawTriggerEff.C

# 4b. Test-run FCalMonitoring!
# This should work out-of-the box when you run on the lxplus or Canadian PCs
# or you need to download a file and edit the jobOption
cd ../share/
athena FCalMonitoring.py | tee my.log
# or if you want to save everything to the log file including the errors (assumes zsh)
# & includes the errors
# ! means ok to overwrite
athena FCalMonitoring.py >&! my.log

# 5. submit grid job
pathena FCalMonitoring.py --inDS mc11_7TeV.105001.pythia_minbias.recon.ESD.e815_s1272_s1274_r2727/ --nGBPerJob 12 --outDS user.${USER}.FCalMon.mc11_7TeV.105001.pythia_minbias

(Obsolete) how to use the output tree

In the run directory there is an obsolete example on how to interface with FCalInfo

Making a version tag


svn copy svn+ssh://svn.cern.ch/reps/atlasgroups/Detectors/LAr/FCal/FCalTrigger/trunk svn+ssh://svn.cern.ch/reps/atlasgroups/Detectors/LAr/FCal/FCalTrigger/tags/FCalTrigger-ZZ-YY-XX
svn copy svn+ssh://svn.cern.ch/reps/atlasgroups/Detectors/LAr/FCal/FCalMon/trunk svn+ssh://svn.cern.ch/reps/atlasgroups/Detectors/LAr/FCal/FCalMon/tags/FCalMon-ZZ-YY-XX

Setting up athena the very first time

If you are using one of the Canadian PCs you should have one or several scratch disk where you can put your files. Perhaps /scartchdisk2/ at atcanpc001. In this folder, there will be a README file saying something like: /scratch/atcanpc001b. This path is accessible from all Canadian PCs. Chose one such folder to be where you will keep all your code. You can check that there's enough space on that disk by:


df -h /scratch/atcancp001b

Create the directory where you will keep the code:


mkdir /scratch/atcancp001b/${USER}/17.0.4.6/

Now, you need to setup this environment every time a new shell is used (including this time): instead of typing the whole string, consider adding these 2 lines to your ~/.zshrc file (assuming that you run the zsh shell, check by echo $SHELL).


export AtlasSetup=/afs/cern.ch/atlas/software/dist/AtlasSetup
alias asetup='source $AtlasSetup/scripts/asetup.sh'

cd /scratch/atcancp001b/${USER}/17.0.4.6/
asetup 17.0.4.6,here

Setting up athena after the first time

Log in and execute


asetup 17.0.4.6,here

You are now ready to go. If you wish to use a later release, repeat the steps above, using the later release instead (perhaps 17.W.X.Y). If you get a warning message that the test area doesn't exist, simply create it:


mkdir $TestArea

Run-II FCalMonD3PD Using Run-I D3PDMaker

The Run-II FCalMonD3PD package

As of August 2015, there is no updated version of the Run-I FCalMonD3PD code that works on Run-II ESD files. Until then, we can make some modifications to the Run-I code and preserve most of the information. The Run-I code is available for browsing here and here.

Making the Run-II FCalMonD3PD

Before you do anything, check that the athena release is at least as new as the ESD/DESD that you use. To set it up, follow the instructions below:

It is important that the Athena release used is at least as new as the ESD/DESD that you wish to run on, usually using the newest version of Athena is recommended. If your version of Athena isn't new enough the Athena log file will usually say, around the end, that there are segmentation faults, bus errors, etc. Just because your major version of Athena is correct (ex. 20.x.y.z), doesn't necessarily mean the code will work. As of the time of writing this, the following code works, but it is important to ensure that your version of Athena is up-to-date.

# 0. After logging into lxplus or wherever your workspace is

# 1. Set up Athena
mkdir 20.6.0.1
cd 20.6.0.1
export AtlasSetup=/afs/cern.ch/atlas/software/dist/AtlasSetup
alias asetup='source $AtlasSetup/scripts/asetup.sh'
# This will setup Athena in your current directory
asetup 20.6.0.1,here
# This will download the code for the Run-I D3PDMaker in your current location
svn co svn+ssh://svn.cern.ch/reps/atlasgroups/Detectors/LAr/FCal/FCalMonD3PD/trunk FCal/FCalMonD3PD

# 2. Compile the code
cd FCal/FCalMonD3PD/cmt
cmt config
source setup.sh
cmt bro gmake | tee gmake.log
source setup.sh

# 3. Test-run FCalMonD3PD on a local (D)ESD file (takes about 5 minutes to run)
cd ../share/
# Open the file titled DPD_topOptions.py in your editor of choice, vim, emacs, nano, etc.
# Near the top of the file there is an uncommented line 'DATAPATH1 = '/afs/... '
# Change this location to the location of a (D)ESD file you have access to, or a directory of (D)ESD files
#
# Supposedly the EvtMax command is being phased out, as of writing this, this is not the case, if this is
# true at the time of using this, then this is the time to adjust this line as well.
#
# Lastly, we must comment out some lines of the file that seem to give Athena issues. The three lines are:
# alg += ClusterD3PDObject (10)
# alg += TowerD3PDObject (10)
# alg += PrimaryVertexD3PDObject (10)
#
# Now, write the file and continue on
time athena -c EvtMax=5 DPD_topOptions.py >& athena.log
# If the words 'failed' or 'aborted' are output, check the .log file
# Otherwise, there is now a .root file 'caloD3PD_from_esd.root' in your run location. This is root-readable

<!--***********************************************************--> <!--Do NOT remove the remaining lines, but add requested info as appropriate--> <!--***********************************************************-->


<!--For significant updates to the topic, consider adding your 'signature' (beneath this editing box)--> Major updates:
-- DagGillberg - 2010-08-18
-- JustinKeung - 2010-09-15

<!--Please add the name of someone who is responsible for this page so that he/she can be contacted if changes are needed.
The creator's name will be added by default, but this can be replaced if appropriate.
Put the name first, without dashes.-->
%RESPONSIBLE% JustinGordonKulp
<!--Once this page has been reviewed, please add the name and the date e.g. StephenHaywood - 31 Oct 2006 -->
%REVIEW% Never reviewed


Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2015-08-07 - JustinGordonKulp
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback