mjj fit

Running fit using HHbbgg framework

- Make WS: python makeWorkspaceHHbbgg.py -d jsons/VHbb_20190402_full_alessandro_backup_check.json -n 4 --nbins 15 -s jsons/VHbb_20190402_settings_alessandro_backup_check.json -m 0 -o workspace_name.root

- Plot Dataset: python plotDatasets.py -i workspace_name.root -n 4 -s jsons/VHbb_20190402_settings_alessandro_backup_check.json -o VHbb_test

- In order to get functional form from workspace produced by mkaeWorkspaceHHbbgg test: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/checkingFit_WS.C

- Output DC (from HHbbgg framework): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/input_DC

- Output WS (from HHbbgg framework): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/input_workspaces

Output workspaces [100, 135] GeV fit

--> /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_9May_WS_and_DC

Low Vpt

Cat 1) 0.0.35 —> 0 —> 0

Cat 2) 0.35-0.75 —> 0.15 —> 0.016

Cat 3) 0.75-0.90 —> 0.6 —> 0.14

Cat 4) 0.90-1.00 —> 1.90 —> 1.50

High Vpt

Cat 1) 0.0.30 —> 0 —> 0

Cat 2) 0.30-0.65 —> 0.10

Cat 3) 0.65-0.85 —> 0.80

Cat 4) 0.85-1.00 —> 3.90

Bias studies

1) /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies

2) https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/wiki/nonstandard#roomultipdf-conventional-bias-studies

3) Tutorial datacards: http://twiki.ihep.ac.cn/twiki/view/CMS/CombineTutorial#The%20datacards

4) Code for accessing workspaces and creating other fitting form: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/accessingWS.C

==

How to run the bias studies in Combine

1) combine output_DC_forBiasStudy/Mjj_201900402_lowVpt_backup_today_DCnew.txt -M GenerateOnly --setParameters pdf_index=0 -t 100 --expectSignal 1 --saveToys -m 125 --freezeParameters pdf_index -S 0

2) combine output_DC_forBiasStudy/Mjj_201900402_lowVpt_backup_today_DCnew.txt -M FitDiagnostics --setParameters pdf_index=0 --toysFile higgsCombineTest.GenerateOnly.mH125.123456.root -t 100 --rMin -10 --rMax 10 --freezeParameters pdf_index

1) creates toys with a given functional form used for the fit (defined by the pdf_index variable) and for mu=1 (expectSignal=1)

2) runs the fit on the same toys but using other functional forms that will need to be checked (pdf_index=1,2,3,4,…) and check bias

For highVpt workspaces, we need to add an additional command to help the fit convergence (most of the parameters are at boundary):

combine output_DC_forBiasStudy/biasStudy_highVpt_update16May.txt -M FitDiagnostics --setParameters cate3=1 --toysFile higgsCombineTest.GenerateOnly.mH125.123456.root -t 500 --rMin=-10 --rMax=10 --freezeParameters cate3 --cminDefaultMinimizerStrategy 0

How to check bias

root -l fitDiagnostics.root

tree_fit_sb->Draw("(r-1)/rErr>>h(20,-4,4)")

h->Fit("gaus")

Where to find datacards to be used for bias study

* low Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_lowVpt.txt

* high Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_highVpt_update16May.txt

* The workspaces the datacards above refer to are here: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/background_test_lowVpt_v4.root /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/background_test_highVpt_update16May.root

* The code that creates the roomultipdf object dumped in the workspace and in the datacard for the bias study is here: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/producingWS_biasStudy.C

* The code that makes the final bias plot is here: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/plot_biasStudy_highvpt.C /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/plot_biasStudy.C

* REMINDER: pdf_index=0: Cheb, pdf_index=1: Bern, pdf_index=2: Exp, pdf_index=3:PowerLaw

* category0: cate0=0: nominal Cheb, cate0=1: alt Bern, cate0=2: alt exp; cate0=3: alt power law;

* category1: cate1=0: nominal Cheb, cate1=1: alt Bern, cate1=2: alt exp; cate1=3: alt power law;

* category2: cate2=0: nominal Cheb, cate2=1: alt Bern, cate2=2: alt exp; cate2=3: alt power law;

* category3: cate3=0: nominal Cheb, cate3=1: alt Bern, cate3=2: alt exp; cate3=3: alt power law;

Code that runs fit to data

Code here: /t3home/acalandr/VHbb/data_driven/makeWorkspaceHHbbgg.py

Additional implementation that runs more fits for bias study

Optimization F-test nominal fit Cheb

Using F-test

1) low Vpt - cat0: cheb2 ---> cheb1 used (after optimisation on bias study)

- cat1: cheb2

- cat2: cheb3

- cat3: cheb 3

1) high Vpt - cat0: cheb1

- cat1: cheb2

- cat2: cheb2

- cat3: cheb 2

Optimization F-test alternativel fit Bern polynomial

1) low Vpt - cat0: bern2. --> bern1 used (after optimisation on bias study)

- cat1: bern2

- cat2: bern3

- cat3: bern 3

1) high Vpt

- cat0: bern1

- cat1: bern2

- cat2: bern2

- cat3: bern 2

Optmization alternative fit expo for bias study

#low Vpt

if cat==0: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.006,-0.1,0.) #AC slope for cat0 low Vpt

if cat==1: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.008,-0.1,0.) #AC slope for cat1 low Vpt

if cat==2: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.006,-0.1,0.) #AC slope for cat2 low Vpt

if cat==3: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.008,-0.1,0.) #AC slope for cat3 low Vpt

#high Vpt

if cat==0: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.004,-0.1,0.) #AC slope for cat0 high Vpt

if cat==1: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.006,-0.1,0.) #AC slope for cat1 high Vpt

if cat==2: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.002,-0.1,0.) #AC slope for cat2 high Vpt

if cat==3: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.004,-0.1,0.) #AC slope for cat3 high Vpt

---++ Optmization alternative fit power-law for bias study

norm=ROOT.RooRealVar("pow_%scat%d" % (label,cat),"pow_%scat%d" % (label,cat),-100.,100.) #AC: linear term RooGenericPdf

#lowVpt

if cat==0: norm.setVal(-1);

if cat==1: norm.setVal(-1);

if cat==2: norm.setVal(-1);

if cat==3: norm.setVal(-1)

#high Vpt

if cat==0: norm.setVal(-1);

if cat==1: norm.setVal(-1);

if cat==2: norm.setVal(-1);

if cat==3: norm.setVal(-1)

Running fit using HHbbgg framework

—> Combining cheb (nom), i.e. cate0=0 vs bern (alt), i.e. cate0=1

combine /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_lowVpt.txt -M GenerateOnly --setParameters cate0=0 -t 100 --expectSignal 1 --saveToys -m 125 --freezeParameters cate0 -S 0

combine output_DC_forBiasStudy/Mjj_201900402_lowVpt_backup_today_DCnew_new.txt -M FitDiagnostics --setParameters cate0=1 --toysFile higgsCombineTest.GenerateOnly.mH125.123456.root -t 100 --rMin -10 --rMax 10 --freezeParameters cate0

==

Output Datacards for bias study:

/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_lowVpt.txt

/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_highVpt.txt

===

How to check bias:

cate0=0 -> nominal Cheb category 0, cate0=1 -> alternative Bern category 0, cate0=2 -> alternative exp category 0, cate0=3 -> alternative powe law category 0

cate1=0 -> nominal Cheb category 1, cate1=1 -> alternative Bern category 1, cate1=2 -> alternative exp category 1, cate1=3 -> alternative powe law category 1

cate2=0 -> nominal Cheb category 2, cate2=1 -> alternative Bern category 2, cate2=2 -> alternative exp category 2, cate2=3 -> alternative powe law category 2

cate3=0 -> nominal Cheb category 3, cate3=1 -> alternative Bern category 3, cate2=2 -> alternative exp category 3, cate2=3 -> alternative powe law category 3

Calculating significance

combine -M Significance output_9May_WS_and_DC/nom_cheb_lowVpt_v2.txt -t -1 --expectSignal=1

Where to find data cards for final significance estimation:

- low Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_9May_WS_and_DC/nom_cheb_lowVpt_v2.txt

- high VPt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_DC_highVpt_update16May/nom_cheb_highVpt_Update16May.txt

- combine datacard: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/scripts/combined_mjj_2lep.txt

Datacards with background normalization

- Discussion on rate parameters: https://indico.cern.ch/event/577649/contributions/2339440/attachments/1380196/2097805/beyond_simple_datacards.pdf

- low Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_9May_WS_and_DC/nom_cheb_lowVpt_v2_rateParam.txt

- low Vpt (Correlated rate parameters): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_9May_WS_and_DC/nom_cheb_lowVpt_v2_rateParam_correlatedRateParam.txt

- high Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_DC_highVpt_update16May/nom_cheb_highVpt_Update16May_rateParam.txt

- high Vpt (correlated rate parameters): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_DC_highVpt_update16May/nom_cheb_highVpt_Update16May_rateParam_correlatedRateParam.txt

NEW OPTIMIZATION OF DNN BOUNDARIES TO IMPROVE SIGNIFICANCE WITH DATA YIELD FREE FLOATING

/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/newCategorization_DNNcuts

- low Vpt:

- confg0: old DNN categorization

- config1: [0,0.25,0.5,0.75,1]

- ....

- ....

-- AlessandroCalandri - 2018-10-11