TopMassLjets8TeV

Documentation:

Code in svn:

Run code on cluster:

Code setup:

Example setup ATLAS releases:

export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
localSetupROOT 5.34.13-i686-slc5-gcc4.3

or
localSetupROOT 5.34.21-x86_64-slc6-gcc48-opt

source /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.13-i686-slc5-gcc4.3/bin/thisroot.sh

echo "Changing to work dir"
cd /afs/ipp-garching.mpg.de/home/g/gcortian/top_8TeV_TRC140028/TMT_8TeV/Munich_ljets/src
echo $PWD
echo "Setting up TopRootCore"
source /afs/ipp-garching.mpg.de/home/g/gcortian/top_8TeV_TRC140028/RootCore/scripts/setup.sh

echo "Not setting up LocalDQ2Client"
echo "to setup DQ2 do: localSetupDQ2Client"
#localSetupDQ2Client

Analysis Top setup



SETUP ROOT FROM CVMFS:

export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
localSetupROOT 5.34.13-i686-slc5-gcc4.3

CREATE DIRECTORIES:

mkdir top_8TeV
cd top_8TeV

DOWNLOAD TRC:

svn co "svn+ssh://$CERN_USER@svn.cern.ch/reps/atlasoff/PhysicsAnalysis/TopPhys/TopRootCoreRelease/tags/TopRootCoreRelease-14-00-26" TopRootCoreRelease

DOWNLOAD TMT_8TEV:

svn co "svn+ssh://$CERN_USER@svn.cern.ch/reps/atlasinst/Institutes/MPI/Top/TMT_8TeV/trunk" TMT_8TeV

EDIT TRC PACKAGE LIST:

cd TopRootCoreRelease/share/
- edit packages.txt and uncomment KLFitter/BAT to enable them

COMPILE PACKAGES:

./build-all.sh
<wait for packages to download and compile>

COPY MODIFIED TRC FILES:

cd TMT_8TeV/MiniNtuples/
./manage_custom_TRC.sh -rc

COMPILE MODIFIED PACKAGES:

cd TopRootCoreRelease/share/
./build-all.sh
<wait for modified packages to recompile>

ADD SETUP SCRIPT: use the following line to setup the installed ROOTCORE version:

source RootCore/scripts/setup.sh

TO LOAD LIBRARIES IN ROOT:

root [0] .x $ROOTCOREDIR/scripts/load_packages.C

TO SETUP DQ2CLIENTS (needed for T3 submission scripts):

export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
localSetupDQ2Client --skipConfirm
export DQ2_LOCAL_SITE_ID=ROAMING

TMVA

* Code installed now in: /afs/ipp-garching.mpg.de/home/k/knand/TMT_8TeV/Munich_ljets (contains tmva and src)

IMPORTANT NOTE:

=========================================================================================

it looks like a special treatment of integer variable was added to the BDT code in TMVA. This was not accompained with a newer version of TMVA but can have yield important changes in the training weights obtained.

see http://sourceforge.net/p/tmva/mailman/message/34300526/ and https://root.cern.ch/root-version-v5-34-00-patch-release-notes

suggest to use setup8TeV_TRC28_Root5.34.21_64bit.sh to generate weights

==========================================================================================

Generate input file to be used by TMVA

First you have to create a input file for TMVA. To do this there is the source file readDataLepJetAllJN.cpp. In the file there is the function

readDataLepJet(TFile* file, TString outputFileName, Int_t NBtag = 2, TString treeName = "Top")

to generate the data file. NBtag sets if there will be the data for one (NBtag = 1), two (NBtag = 2) or all (NBtag = 0) btags in the file.

from Munich_ljets/tmva/

mkdir ../tmva_rootfiles/

root -l
.L readDataLepJetAllJN.cpp++
TFile* inputFile = TFile::Open("/ptmp/mpp/gcortian/TeV8/TRC28/klf6jVF/root//tmt/3dTMTcompact_1725_100_100.fixklfp.root")
readDataLepJet(inputFile, "../tmva_rootfiles/datafile_trc28.root", 0, "Top"); > ./readDataLepJet.log

NOTE: to check input variables and their correlation between data and MC

root -l
.L readDataMCLepJetAllJN.cpp++
 readDataLepJet("../tmva_rootfiles/dataMCfile_trc28.root",0,"Top")

TMVA Training

When the data file is generated you can train the TMVA. To do this there is the file tmvaLepJet.cpp with the function

trainLepJet(TString outputFileName, TString dataFileName, Double_t sigWeight = 1, Double_t backWeight = 1)

The sigWeight and backWeight are there to apply a global weight for the signal or the background tree. Normally should 1 be the right choice.

root -l
.L tmvaLepJet.cpp++
trainLepJet("../tmva_rootfiles/trainingfile_trc28.root","../tmva_rootfiles/datafile_trc28.root"); > ./tmvalogfile_trc28.log

Caution: TMVA automaticlly generates a directory "weights" in your current working directory with the results of the training. Maybe you want to change this directory first to avoid overwritting of old results.

NOTE: to check input variables and their correlation between data and MC

root -l
.L tmvaLepJet.cpp++
trainLepJet("../tmva_rootfiles/dataMCtrainingfile_trc28.root", "../tmva_rootfiles/dataMCfile_trc28.root",1,1,1); >./tmvadataMClogfile_trc28.log

Inspecting the training

root -l
.L /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.13-i686-slc5-gcc4.3/tmva/test/TMVAGui.C
 TMVAGui("../tmva_rootfiles/trainingfile_trc28.root")

TMVA Plots

The following command plots the input variables, their correlations, and the BDT performance on the test and training samples.

root -l Plot_BDT.C

Example TMVA application to data

After training there is a example reader for appling a BDT cut: mvaReaderLepJetAllJN.cpp. In the file there is the function:

mvaReaderLepJet(TFile* file, TString outputFileName, TString treeName, Float_t cut,  Int_t NBtag = 0, TString method = "cut")

file is a sample with raw data (not the output of readDataLepJet!), treeName is for the LepJet normally "Top", cut is the value for the BDT cut, NBtag has the same meaning as in readDataLepJet and method can be "cut" or "weight". "cut" means a normal BDT cut and "weight" is to apply a weight (BDT+1)/2 to the histograms.

To use mvaReaderLepJetAllJN.cpp you have to load first the resulting C++-class in the weights directory.

For example:

root -l
.L mvaReaderLepJetAllJN.cpp++
TFile* inputFile = TFile::Open("/ptmp/mpp/gcortian/TeV8/TRC28/klf6jVF/root//tmt/3dTMTcompact_1725_100_100.fixklfp.root")
mvaReaderLepJet(inputFile, "../tmva_rootfiles/bdtresultfile_trc28.root", "Top", 0, 0); > ./mvaReaderLepJet.log

# to compare with the standard cut

.L readDataLepJetCutBased.cpp++
TFile* inputFile = TFile::Open("/ptmp/mpp/gcortian/TeV8/TRC28/klf6jVF/root//tmt/3dTMTcompact_1725_100_100.fixklfp.root")
readDataLepJetCut(inputFile, "../tmva_rootfiles/cutresultfile_trc28.root", 0,"Top"); > readDataLepJetCutBased.log

Run over grid output

Sample list

This is used to merge several samples (for example ttbar and single top samples) in that way, that you only have one file list for the DSIDs and then produce one output file.

#/afs/ipp-garching.mpg.de/home/g/gcortian/top_8TeV/TMT_8TeV/MiniNtuples/ListMini_TRC28/ # lists are created by the python files in the above directory # scripts need DQ2 setup (atlas proxy-init) # example:

python CreateFileListsMC_truth.py

How to update file with x-sec/processed entries

from MiniNtuples/ListMini/ run:

getEntriesFromMiniSL.C

Then add corresponding info in 8TeV_dsid_entries.txt (this file is read by readklf)

Cut flows

MiniNtuples/ListMini/PrintCutFlowFromMiniSL.C
# example data"
root -l
.L PrintCutFlowFromMiniSL.C
PrintCutFlowFromMiniSL("data12_data_Muons_mu.txt", 12)
PrintCutFlowFromMiniSL("data12_data_Egamma_el.txt", 11)

Yields table

Munich_ljets/src/TeV8_Plot_3dTMT_dataMC_UncBand_4Paper.C

# create dir for figures
mkdir -p fig4paper

# setup TRC: example:
root -l $ROOTCOREBIN/scripts/load_packages.C

.L TeV8_Plot_3dTMT_dataMC_UncBand_4Paper.C++
TeV8_Plot_3dTMT_dataMC_UncBand_4Paper(2,1725, 1000, 1)

In the macro you can change the event selection/cuts for which you want to have the event yield. Note: which uncertainties are to be included for the EvtYields and plots need to be finalized, also according to what is done in the 8 TeV dilepton mtop analysis.

Fits and Templates

Preparing input histogram files for TMT parameterization

Go to folder: Munich_ljets/src

mkdir -p root

# TeV8_Plot_Test3dTMT_KLF.C
    root -l /afs/ipp-garching.mpg.de/home/g/gcortian/top_8TeV/RootCore/scripts/load_packages.C
    # alternatively source setROOT from shell)
    .L TeV8_Plot_Test3dTMT_KLF.C+
    TeV8_Plot_Test3dTMT_KLF(16,4)  # corresponds to 1 btag only, smaller mtop range, window cut on Likelihood value (keep only [-50, -40])
    TeV8_Plot_Test3dTMT_KLF(26,4)  # corresponds to 2 btags only, smaller mtop range, window cut on Likelihood value (keep only [-50, -40])

For the 2 b-tag selection only and the BDT cuts: for example run: Note: everything with seltype within [1000, 1024] applies a BDT based selection. See code for more info. For example: seltypes 1000, 1010, 1020 select events with BDT> 0.0 and 1+2btags, 1btags, 2btags respectively.

# BDT >0.0
    TeV8_Plot_Test3dTMT_KLF(1020,4)   # BDT > 0,  exactly 2 b-tags
    # BDT >-0.10
    TeV8_Plot_Test3dTMT_KLF(1021,4)   # BDT > -0.1, exactly 2 b-tags
    # BDT >-0.05
    TeV8_Plot_Test3dTMT_KLF(1022,4)   # BDT > -0.05 exactly 2 b-tags
    # BDT >+0.05
    TeV8_Plot_Test3dTMT_KLF(1023,4)   # BDT > 0.05  exactly 2-btags
    # BDT >+0.10
    TeV8_Plot_Test3dTMT_KLF(1024,4)  # not used so far

This code create an ntuple containing all template variations for the observables used in the analysis. They will be used in separate and combined fits as described in the following points.

Check if all TestFit....C code is automatically produced in this script! Thats needed in the next step!

Perform separate and combined fits (start+final parameter files)

Now commend the appropriate steps in from above!

# TeV8_Fit_3dTMT_KLF_sep.C

# to run the code corresponding to the previously determined BDT >-0.10 for example:

source setROOT
    .L TeV8_Fit_3dTMT_KLF_sep.C

    # mtop template vs mtop variations
    TeV8_Fit_3dTMT_KLF_sep(1021, 0, 0, 125, 210)
    # mtop templates vs JSF variations
    TeV8_Fit_3dTMT_KLF_sep(1021, 0, 1, 125, 210)
    # mtop templates vs bJSF variations
    TeV8_Fit_3dTMT_KLF_sep(1021, 0, 2, 125, 210)

    TeV8_Fit_3dTMT_KLF_sep(1021, 1, 1, 55, 110)
    TeV8_Fit_3dTMT_KLF_sep(1021, 2, 2, 0.3, 3.0)

# SIMULTANEOUS FITS (for BDT>-0.10 and -0.05) # reco mtop templates # TeV8_Fit_3dTMT_KLF_allFullMatrix.C ---> creates TestFitParameters....C scripts !!! Check if they are there

source setROOT
    .L TeV8_Fit_3dTMT_KLF_allFullMatrix.C
    TeV8_Fit_3dTMT_KLF_allFullMatrix(1201)

    # TeV8_Fit_3dTMT_KLF_all_mW.C
    source setROOT
    .L TeV8_Fit_3dTMT_KLF_all_mW.C
    TeV8_Fit_3dTMT_KLF_all_mW(1201)

    # TeV8_Fit_3dTMT_KLF_allFullMatrix_Rlb.C
    source setROOT
    .L TeV8_Fit_3dTMT_KLF_allFullMatrix_Rlb.C
    TeV8_Fit_3dTMT_KLF_allFullMatrix_Rlb(1201)

Running Pexp for method closure tests

mkdir -p ../peout_BDTpois2m

NEED TO CHANGE FILES TO INCLUDE BUT ALSO SELECTION CUTS FURTHER DOWN !!!

TeV8_Pexp_Syst_3dTMT_KLF_Rlbcalo3d_1plus2btag_sep_poisson.C

# TeV8_Pexp_Fit_3dTMT_KLF_Rlbcalo3d_1and2btag_sep.C
source setROOT
.L TeV8_Pexp_Syst_3dTMT_KLF_Rlbcalo3d_1plus2btag_sep_poisson.C+

To test:

root -l  "run_TeV8_Pexp_Syst_Rlbcalo3d_1plus2btag_poisson.C(1,0,0,1)"
   python submitter4Pexp_Syst_Rlbcalo3d_1plus2btag_8TeV_poisson.py

Plotting the results

BDT>0.0 (Updated prescriptions for JER and ISR/FSR(PowhegPythia based)
             +bug fix in EES uncertianty
    Also using TeV8_Pexp_Syst_3dTMT_KLF_Rlbcalo3d_1and2btag_sep_poisson.C
    25.2.2016
    # 3dTMT
    root -l  "TeV8_Plot_Pexp_Syst_3dTMT_KLF_1and2btag.C(20278,1, 0.02, 1, 0.2, 3, 1, 0, 1,10)"
    # 2dTMT
    root -l  "TeV8_Plot_Pexp_Syst_3dTMT_KLF_1and2btag.C(20278,1, 0.02, 1, 0.2, 3, 1, 0, 0,10)"
    # 1dTMT
    root -l  "TeV8_Plot_Pexp_Syst_3dTMT_KLF_1and2btag.C(20278,1, 0.02, 1, 0.2, 3, 1, 0, -1,10)" 

Items still to do

  • rerun the code with new training for 2 b-tags only
  • add bkg
  • add PDF systematics (done)
  • estimate stat. uncertainty on the systematics (done)
  • produce final plot with uncertainties

Notes

  • always take the mean of the histogram, usually do not fit distribution
  • for rho for error propagation: make list of common events, then rho = sqrt(fraction of common events)
  • always take parabolic error, not error from minuit!

-- AndreaKnue - 2016-03-29

Edit | Attach | Watch | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r11 - 2016-04-29 - AndreaKnue
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback