Notes and Instructions

Plan for Analysis

Note: before full-time writing do always ~1.5 days of writing

Month Task Status Notes
December weights Yes / Done  
Fitting with errors No  
Begin Event-Challenge    
January calibration on full samples    
Organizing of 4-blocks/periods Yes / Done  
February-March Systematics   Use CP::SysematicSet , see PATInterfaces/SystematicsUtil.h
Stat. chapter    
April-May Round-up    
W-analysis    
June + Write full-time    

This should bring me to the defense around beginning 2019

HistFitter Recompile

  • You need to make a new compilation of HistFitter if you change root version (or AnalysisBase, for that)
  • Make a new folder in /project
git init
git remote add origin ssh://git@gitlab.cern.ch:7999/gbesjes/HistFitter.git
git pull
git checkout v0.54.0
  • After setting up AnalysisBase version (and thus the corresponding root) do inside the HistFitter folder:
source setup
cd src
  • Change -std=c++11 to -sdt=c++14 inside Makefile
make
  • Update your ~/.bashrc file
  • You're ready to use the newly compiled HistFitter

HistFitter notes (Terry)

  • You can install YAML easily using lcgenv
  • Command for gcc62 is likely lsetup "lcgenv -p LCG_91 x86_64-slc6-gcc62-opt"
  • HistFitter, see the advanced features and flow chart
  • Systematic variations come in 2 types:
    • weight-type --> contained in a branch in the same tree
    • tree-type --> (variable) change affecting the selection, thus reducing to different set of events
  • HistFitter does only binned fits, so no good to replicate our SB-SR fit
  • channel = SR or SB or similar
  • Could maybe use a "RooPdf" instead of a function to fit to the side-band, albeit binned
  • The signal I should be able to directly feed into HistFitter, as then the SR selection is used to determine the correct limit
  • For background I can directly pass it as a properly binned histogram (the SR bin having the correct # of entries), and then apply the syst. variations to it as

  • Use histogram in file:
    • xyz/data/workspacename/hName_Nom_regionname_obs_varname (obs is a fixed term, do not change from "obs", "Nom" is for nominal)
    • For variations can use hists with SystematicNameLow/High instead of "Nom"
    • To make it work need to provide a single bkg with the corresponding name, and the software will find and use the one you stored
    • configMgr.histCalcFile = "location/of/this/file" could be used to supersede the path assumed above
  • Since I will probably have different systematics on singnal and bkg the best way is probably to:
    • Loop over the sample list (sig and bkg for me)
    • Add the systematics individually, providing dummy names for the ones that you actually are taking from histograms in declared file

  • Likely it is though better -PROBABLY BETTER, different approach- to do the following:
    • Pass the central value as a single bin
    • Add systematics as addOverallSyst( ), which in my case is anyway just changing the initial value of the single bin
    • Need to use YAML, or some equally smart way to add al systematics from the RooFit code to HistFitter
  • Fit:
    • As likely as not, do not need the BDT fit-extrapolation for the final value and uncertainty, but only for the choice, so that I could ignore as a source of systematics
    • Need to find out how much the fit to SB-SR is affected by systematics: ideally the correlation is so small I can simply keep the fitting function's values fixed and only add a small systematic based on the uncertainty in the function (need to check this)

  • Terry also sent me codes with almost all I need to get the party started, mainly to read YAML configs in directly.
    • He wrote these codes himself.
    • Instruction: You can reference the Systematics() class in config.py, which read a yaml file and turn things into Syst objects defined in sysutils.py
    • Here you can find Terry's code

Notes meeting 14th October

MC re-running with latest triggers in needs following up
  • Make full list of triggers in the mc we are using right now, to see what will have to be added
  • also check that there are no newer versions of mc
For myself: how does the delayed stream work?
Marcus: function to broaden region of interest in variables
Limit your sidebands on the external side
For BDT check that your # of signal/bkg at a certain signal eff. are compatible,
Get the limit an make rough scan of possible limits
Do a sum-tracks at vertex variable instead of the MET_track

Setting up Derivation for rel.21 WITH GIT

Follow the instructions of my help-package: this one.
Then setup git:
lsetup git
go to the correct folder (source), do git atlas init-workdir <ssh://git@gitlab.cern.ch:7999/mbedogne/athena.git> (link-to-ssh of your athena fork or the original ) cd atlas
git checkout <full-tag> (begins with "release/" usually)
git atlas addpkg

So far using init-workdir makes an extra folder which we didn't figure out how to use (we were deleting it)

Performing a Merge-Request for a change in some Athena software:

Prepare your working directory, for example with the derivation setup mentioned in the previous paragraph.
Now you need to start working with git. Keep in mind these pages for the full reference.
You will need to have forked a version of athena (Marcus claims it doesn't need to be the most recent)
lsetup git
git atlas init-workdir <link-to-ssh of your athena fork or the original (http)> ( ssh://git@gitlab.cern.ch:7999/mbedogne/athena.git )
cd athena
git fetch upstream
git checkout -b <temporary-branch-name> upstream/<ref-branch> --no-track (for example git checkout -b my-new-feature upstream/21.2 --no-track)
git atlas addpkg <pkg-name> (to check-out the package that you want to work on)

Now you can make changes, and test as much as you want.

For compilation do the following:
make a copy of the filter example:
cp Projects/WorkDir/package_filters_example.txt ../package_filters_example.txt
Then move outside into build ( cd ../build )
do
cmake -DATLAS_PACKAGE_FILTER_FILE=../package_filters_example.txt ../athena/Projects/WorkDir
make -j

Never forget to do this as well:
source x*/setup.sh

Feel free to commit changes to this "un-tracked" local branch (git add <files-or-similar> , git commit -m "Committing Message" )

Once you're ready you can upload your changes to your fork of athena:
git push --set-upstream origin <temporary-branch-name>

At this point it is simple to request this change to be merged back into the "real deal" (athena repository):
that part is nicely described here.

Calculating the expected limit from bkg and signal events

Links:
Suggested by Sara
GRL tutorial (very useful page)

go to
/afs/cern.ch/user/m/mbedogne/HISTFITTER/HistFitter
set up a version 6.04 of root, e.g. setting up an athena version of that kind. Then you can do
HistFitter.py -w -f -D "before,after,corrMatrix" analysis/tutorial/MyUserAnalysis.py This calculates the limit etc, first making a re-fit of the bkg, I believe. There is as well a version which does this starting from a TTree with the bin/entries:
analysis/tutorial/MyOneBinExample.py
Looking at these you should be able to calculate your number!!!

I think this should work equally on AOD as on DAOD, but you should test it

Infos are taken from the manual (see your Downloads folder on the laptop)
Have fun!

YAML compilation with new gcc version

As different athena releases require various versions of gcc you need to re-compile yaml accordingly.

  • Copy the basic package to a suitable name (e.g. yaml-cpp_gcc49)
  • setup the version of athena you want (can be done anywhere else as well)
  • go into build
  • cmake ../ -DBUILD_SHARED_LIBS=ON -DCMAKE_CXX_COMPILER=`which g++`
  • make
  • Set the reference to the installation as you prefer (I set a variable in .bashrc)
( irrelevant for this: -DCMAKE_CC_COMPILER=`which gcc`)

Getting the luminosity from data samples

Twiki with full instructions

Go to here and make your histogram file (remember to select Create output plots and ntuples (--plots)).
For now I use the unprescaled trigger:
HLT_mu20_msonly_mu6noL1_msonly_nscan05
See testDERIV pkg for the grl files I keep using.

At this point you have the iLumi file. With this you need to use a full xAOD sample to determine its specific luminosity (I will then use some trigger rate to estimate it on the run)

Go to workspace/LUMIcalc (cern), or /data/atlas/users/mbedog/LumiCalc (Nikhef), setup uisng source setup.sh and look at command.sh for how to run the calculation.

Fill this in before you forget too much of it!!!

Getting Cross section for a MC sample

setupATLAS
lsetup pyami
ami command GetDatasetInfo -logicalDatasetName=dataset | grep crossSection Note that this does not give the reight result for Dai's samples (pre- October 6th), as he messed up with some generator configuration. For these you should take the values he provided, see his presentation of the previous week: https://indico.cern.ch/event/569472/
Cross sections for Dai's samples can be found in a table in the mc section here.

Calculate Cross section for Limit

Dai's list of sample lists three quantities: cross section, filter efficiency and BR To get the correct x-sec to be multiplied by Luminosity, you need to multiply CrossSection*FilterEff. I believe that the BR is already in the CS, while only the produced number of events is kept, thus real x-sec needs to be reduced by this efficiency (filtering eff).

Testing broken ElementLinks

In older versions of the derivation BPHY7 (but not only), the link between the PV linked to Triplets and their InnerDetectorTracks were broken. There is a workaround to try finding the correct tracks anyway:

This is only a temporary solution.

The links are not valid, but the target is properly stored in the derived samples, so one may use the following function to reach the refitted PV.
Assuming you load a secondary vertex pointer as candidate :

const xAOD::VertexContainer* refPV = 0;
RETURN_CHECK("LOAD refitted PV", m_event->retrieve(refPV,"BPHY7RefittedPrimaryVertices"));
const xAOD::Vertex* refittedPV = findRefPV(candidate , refPV);
if(refittedPV) int nTrksPV = refittedPV->nTrackParticles();

The function findRefPV can be found here, starting roughly at line 35.

It fails only about 1.3% of the times, though sometimes it does not return exactly the same refitted vertex as the original link would, as it happens often that several very similar refitted vertices are stored in DAOD_BPHY7.

Here instead are the instructions to test whether in a specific sample the links are valid:

In a bash shell it should be sufficient to run:
source setup.sh
runVALID /Path_to_sample/

Here is a reference to samples which will:
succeed:
user.mbedogne.303562.Pythia8EvtGen_A14NNPDF23LO_PosW_nutau_3mu.DAOD_BPHY7.e4445_s2608_r7772_r7676_ver9_EXT0/
fail
any officially derived data15_13TeV*DAOD_BPHY7*p2757* sample

Added option -tracks to test for missing tracks at the refitted PV in cases where the refitted PV coincides with the original PV

Using screen on lxplus

This should give the right to keep running when you disconnect
Taken from here.

Note: if you use any setupATLAS or asetup or rcSetup, do it after starting the screen session!

If needed you can clear everything inside your shell doing: exec bash

sequence of commands to have screen running correctly on lxplus

screen
k5reauth -x -f pagsh
aklog
bash

Overview τ → 3 μ production cross section at 13 TeV

  • ATLAS 13 TeV estimates
    σ(W)xBR(W → l ν) to be 11.83 0.4 nb (W+) and 8.79 0.3 nb (W-) .
    Using the pdg value BR(W → l ν) = 10.86% 0.09% this can be written as
    σ(W+) = 108.9 nb , σ(W-) = 80.94 nb .
    Please note that the uncertainties are roughly simplified by quadrature sum
  • This theory paper shows the NLLO estimate for the production cross section, for C.M. energies between 7 and 14 TeV, agreeing with the previous values.
  • The values agree with what is written in the run1 paper as well.

To have grid samples copied to CERN

rucio add-rule user.mbedogne... 1 CERN-PROD_SCRATCHDISK

That command will return a rule number and you can check when its copied by
rucio rule-info xxxxxxxxx

Install PRINTER at NIKHEF in UBUNTU

Go to System Settings --> Printers
do Add (+ sign)
go to Network Printer (left menu) --> LPD/LPR Host or Printer
Insert under "Host" < username >@lpd.nikhef.nl/color (or "bw")
press Forward
Select Provide PPD file and choose this one: http://www.nikhef.nl/nikhef/departments/ct/printer/XeroxWorkCentre7556.ppd (download first)
Choose your own names in the following page
DONE!!!!

Easily download packages at Nikhef pcs

In short:

Add to ~/.ssh/config :
Host *svn.cern.ch svn
    User <username>
    GSSAPIDelegateCredentials yes
    GSSAPIAuthentication yes
Restart the terminal

Run:
kinit your-username@CERN.CH

Explanation:

You use kerberos for this.
The program should already be installed (it definitely is on stoomboot).
The configurations for CERN are already in /etc/krb5.conf (if it is missing on your pc you can copy this file and define
the correct path as export KRB5_CONFIG=/path/to/your/config/file in .bashrc)
To make it so that kerberos is used to access svn you need to add the following to .ssh/config
Host *svn.cern.ch svn
    User <username>
    GSSAPIDelegateCredentials yes
    GSSAPIAuthentication yes
For the next step make sure the ssh configurations get refreshed. The easy way is to exit and open another terminal.

At this point you only need to tell kerberos to setup your connection credentials:
kinit your-username@CERN.CH and then type your password.
Note that "CERN.CH" has to be capital letters! That is because of the definition inside /etc/krb5.conf

At this point you can use for example pkgco.py PackageName to checkout your package in a totally painless way.

Using EventLoop send an argument to your algorithm on the Nodes

First of all let me write that this is perfectly normal to do. If it fails here is how it is done correctly:

In your submission script (the one in "/util") you need

job.options()->setBool("YourOptionName", yourValue); 
or equally with setDouble or setInt()
this has to come before
job.algsAdd( yourAlgorithm );

Then in yourAlgorithm::setupJob()
you will read the information into your object with

m_yourVariable = job.options()->castBool("YourOptionName", defaultValue); //defaultValue will be used if the option is not set!

Last thing very important: when you declare m_yourVariable in yourAlgorithm.h , you must avoid adding //!
because this prevents the value being forwarded to the workers.

-- MatteoBedognetti - 2016-07-21

Edit | Attach | Watch | Print version | History: r36 < r35 < r34 < r33 < r32 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r36 - 2018-09-07 - MatteoBedognetti
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback