CheckLFuncertaintyonVHbb < Sandbox

---++ mjj fit 



---++ Running fit using HHbbgg framework 

- Make WS: python makeWorkspaceHHbbgg.py -d jsons/VHbb_20190402_full_alessandro_backup_check.json -n 4 --nbins 15 -s jsons/VHbb_20190402_settings_alessandro_backup_check.json -m 0 -o workspace_name.root

- Plot Dataset:  python plotDatasets.py -i workspace_name.root -n 4 -s jsons/VHbb_20190402_settings_alessandro_backup_check.json -o VHbb_test

-%RED% In order to get functional form from workspace produced by mkaeWorkspaceHHbbgg test: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/checkingFit_WS.C
%ENDCOLOR%


- Output DC (from HHbbgg framework): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/input_DC

- Output WS (from HHbbgg framework): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/input_workspaces


---++Output workspaces [100, 135] GeV fit

--> /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_9May_WS_and_DC

Low Vpt

Cat 1) 0.0.35  —> 0  —> 0

Cat 2) 0.35-0.75 —> 0.15  —> 0.016

Cat 3) 0.75-0.90 —> 0.6  —> 0.14

Cat 4) 0.90-1.00 —> 1.90 —> 1.50





High Vpt


Cat 1) 0.0.30  —> 0  —> 0

Cat 2) 0.30-0.65 —> 0.10

Cat 3) 0.65-0.85 —> 0.80  

Cat 4) 0.85-1.00 —> 3.90


---++ Bias studies 

1) /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies

2) https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/wiki/nonstandard#roomultipdf-conventional-bias-studies


3) Tutorial datacards: http://twiki.ihep.ac.cn/twiki/view/CMS/CombineTutorial#The%20datacards


4) Code for accessing workspaces and creating other fitting form: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/accessingWS.C



====


---++ How to run the bias studies in Combine


1) combine output_DC_forBiasStudy/Mjj_201900402_lowVpt_backup_today_DCnew.txt -M GenerateOnly --setParameters pdf_index=0 -t 100 --expectSignal 1 --saveToys -m 125 --freezeParameters pdf_index -S 0


2) combine output_DC_forBiasStudy/Mjj_201900402_lowVpt_backup_today_DCnew.txt -M FitDiagnostics --setParameters pdf_index=0 --toysFile higgsCombineTest.GenerateOnly.mH125.123456.root -t 100 --rMin -10 --rMax 10 --freezeParameters pdf_index


1) creates toys with a given functional form used for the fit (defined by the pdf_index variable) and for mu=1 (expectSignal=1)

2) runs the fit on the same toys but using other functional forms that will need to be checked (pdf_index=1,2,3,4,…) and check bias 


For highVpt workspaces, we need to add an additional command to help the fit convergence (most of the parameters are at boundary):


combine output_DC_forBiasStudy/biasStudy_highVpt_update16May.txt -M FitDiagnostics --setParameters cate3=1 --toysFile higgsCombineTest.GenerateOnly.mH125.123456.root -t 500 --rMin=-10 --rMax=10 --freezeParameters cate3 --cminDefaultMinimizerStrategy 0


---++ How to check bias 

root -l fitDiagnostics.root

tree_fit_sb->Draw("(r-1)/rErr>>h(20,-4,4)")

h->Fit("gaus")


---++ Where to find datacards to be used for bias study

*   low Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_lowVpt.txt


*   high Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_highVpt_update16May.txt



*   The workspaces the datacards above refer to are here:
/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/background_test_lowVpt_v4.root 
 /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/background_test_highVpt_update16May.root

*   The code that creates the roomultipdf object dumped in the workspace and in the datacard for the bias study is here:
/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/producingWS_biasStudy.C

*  The code that makes the final bias plot is here:
/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/plot_biasStudy_highvpt.C
/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/plot_biasStudy.C

* REMINDER: pdf_index=0: Cheb, pdf_index=1: Bern, pdf_index=2: Exp, pdf_index=3:PowerLaw

* category0:   cate0=0: nominal Cheb, cate0=1: alt Bern, cate0=2: alt exp; cate0=3: alt power law;

* category1:   cate1=0: nominal Cheb, cate1=1: alt Bern, cate1=2: alt exp; cate1=3: alt power law;

* category2:   cate2=0: nominal Cheb, cate2=1: alt Bern, cate2=2: alt exp; cate2=3: alt power law;

* category3:   cate3=0: nominal Cheb, cate3=1: alt Bern, cate3=2: alt exp; cate3=3: alt power law;



---++ Code that runs fit to data
Backup in mail to alessandro.calandri@cern.ch - May 7 (16h45)

Code here: /t3home/acalandr/VHbb/data_driven/makeWorkspaceHHbbgg.py

---++ Additional implementation that runs more fits for bias study
Backup in mail to alessandro.calandri@cern.ch - May 8 (18h13)


---++ Optimization F-test nominal fit Cheb

Using F-test

1) low Vpt
- cat0: cheb2 ---> cheb1 used (after optimisation on bias study)

- cat1: cheb2

- cat2: cheb3

- cat3: cheb 3 




1) high Vpt
- cat0: cheb1 

- cat1: cheb2

- cat2: cheb2

- cat3: cheb 2 

---++ Optimization F-test alternativel fit Bern polynomial

1) low Vpt
- cat0: bern2. --> bern1 used (after optimisation on bias study)

- cat1: bern2

- cat2: bern3

- cat3: bern 3 




1) high Vpt

- cat0: bern1

- cat1: bern2

- cat2: bern2

- cat3: bern 2 




---++ Optmization alternative fit expo for bias study 


#low Vpt 

   if cat==0:
        norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.006,-0.1,0.)  #AC slope for cat0 low Vpt
 

 if cat==1:
        norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.008,-0.1,0.)  #AC slope for cat1 low Vpt


  if cat==2:
        norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.006,-0.1,0.)  #AC slope for cat2 low Vpt
  

 if cat==3:
        norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.008,-0.1,0.)  #AC slope for cat3 low Vpt 



#high Vpt 

if cat==0:
        norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.004,-0.1,0.)  #AC slope for cat0 high Vpt
    
  if cat==1:
        norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.006,-0.1,0.)  #AC slope for cat1 high Vpt
  

  if cat==2:
        norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.002,-0.1,0.)  #AC slope for cat2 high Vpt



 if cat==3:
        norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.004,-0.1,0.)  #AC slope for cat3 high Vpt 


 ---++ Optmization alternative fit power-law for bias study 

 norm=ROOT.RooRealVar("pow_%scat%d" % (label,cat),"pow_%scat%d" % (label,cat),-100.,100.)  #AC: linear term RooGenericPdf
    
#lowVpt
 
   if cat==0:
        norm.setVal(-1);
    
if cat==1:
        norm.setVal(-1);

    if cat==2: 
        norm.setVal(-1);

if cat==3:
        norm.setVal(-1)

#high Vpt

    if cat==0:
        norm.setVal(-1);
    
if cat==1:
        norm.setVal(-1);

    if cat==2: 
        norm.setVal(-1);
    
if cat==3:
        norm.setVal(-1)




---++ Running fit using HHbbgg framework 


—> Combining cheb (nom), i.e. cate0=0 vs bern (alt), i.e. cate0=1

combine /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_lowVpt.txt -M GenerateOnly --setParameters cate0=0 -t 100 --expectSignal 1 --saveToys -m 125 --freezeParameters cate0 -S 0


combine output_DC_forBiasStudy/Mjj_201900402_lowVpt_backup_today_DCnew_new.txt -M FitDiagnostics  --setParameters cate0=1 --toysFile higgsCombineTest.GenerateOnly.mH125.123456.root  -t 100 --rMin -10 --rMax 10 --freezeParameters cate0


======

Output Datacards for bias study:


/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_lowVpt.txt



/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_highVpt.txt



=======

How to check bias:

cate0=0 -> nominal Cheb category 0, cate0=1 -> alternative Bern category 0, cate0=2 -> alternative exp category 0, cate0=3 -> alternative powe law category 0 

cate1=0 -> nominal Cheb category 1, cate1=1 -> alternative Bern category 1, cate1=2 -> alternative exp category 1, cate1=3 -> alternative powe law category 1 

cate2=0 -> nominal Cheb category 2, cate2=1 -> alternative Bern category 2, cate2=2 -> alternative exp category 2, cate2=3 -> alternative powe law category 2 

cate3=0 -> nominal Cheb category 3, cate3=1 -> alternative Bern category 3, cate2=2 -> alternative exp category 3, cate2=3 -> alternative powe law category 3


---++ Calculating significance 

 combine -M Significance output_9May_WS_and_DC/nom_cheb_lowVpt_v2.txt -t -1 --expectSignal=1


---++ Where to find data cards for final significance estimation:

- low Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_9May_WS_and_DC/nom_cheb_lowVpt_v2.txt

- high VPt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_DC_highVpt_update16May/nom_cheb_highVpt_Update16May.txt

- combine datacard: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/scripts/combined_mjj_2lep.txt

---++ Datacards with background normalization 

- Discussion on rate parameters: https://indico.cern.ch/event/577649/contributions/2339440/attachments/1380196/2097805/beyond_simple_datacards.pdf

- low Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_9May_WS_and_DC/nom_cheb_lowVpt_v2_rateParam.txt 

- low Vpt (Correlated rate parameters): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_9May_WS_and_DC/nom_cheb_lowVpt_v2_rateParam_correlatedRateParam.txt 


- high Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_DC_highVpt_update16May/nom_cheb_highVpt_Update16May_rateParam.txt

- high Vpt (correlated rate parameters): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_DC_highVpt_update16May/nom_cheb_highVpt_Update16May_rateParam_correlatedRateParam.txt

---++ NEW OPTIMIZATION OF DNN BOUNDARIES TO IMPROVE SIGNIFICANCE WITH DATA YIELD FREE FLOATING 


/t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/newCategorization_DNNcuts

- low Vpt:

- confg0: old DNN categorization

- config1: [0,0.25,0.5,0.75,1]

- ....

- ....





-- Main.AlessandroCalandri - 2018-10-11
This topic: Sandbox > WebPreferences > CheckLFuncertaintyonVHbb
Topic revision: r33 - 2019-06-17 - AlessandroCalandri
Copyright &© 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback