---++ mjj fit ---++ Running fit using HHbbgg framework - Make WS: python makeWorkspaceHHbbgg.py -d jsons/VHbb_20190402_full_alessandro_backup_check.json -n 4 --nbins 15 -s jsons/VHbb_20190402_settings_alessandro_backup_check.json -m 0 -o workspace_name.root - Plot Dataset: python plotDatasets.py -i workspace_name.root -n 4 -s jsons/VHbb_20190402_settings_alessandro_backup_check.json -o VHbb_test -%RED% In order to get functional form from workspace produced by mkaeWorkspaceHHbbgg test: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/checkingFit_WS.C %ENDCOLOR% - Output DC (from HHbbgg framework): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/input_DC - Output WS (from HHbbgg framework): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/input_workspaces ---++Output workspaces [100, 135] GeV fit --> /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_9May_WS_and_DC Low Vpt Cat 1) 0.0.35 > 0 > 0 Cat 2) 0.35-0.75 > 0.15 > 0.016 Cat 3) 0.75-0.90 > 0.6 > 0.14 Cat 4) 0.90-1.00 > 1.90 > 1.50 High Vpt Cat 1) 0.0.30 > 0 > 0 Cat 2) 0.30-0.65 > 0.10 Cat 3) 0.65-0.85 > 0.80 Cat 4) 0.85-1.00 > 3.90 ---++ Bias studies 1) /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies 2) https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/wiki/nonstandard#roomultipdf-conventional-bias-studies 3) Tutorial datacards: http://twiki.ihep.ac.cn/twiki/view/CMS/CombineTutorial#The%20datacards 4) Code for accessing workspaces and creating other fitting form: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/accessingWS.C ==== ---++ How to run the bias studies in Combine 1) combine output_DC_forBiasStudy/Mjj_201900402_lowVpt_backup_today_DCnew.txt -M GenerateOnly --setParameters pdf_index=0 -t 100 --expectSignal 1 --saveToys -m 125 --freezeParameters pdf_index -S 0 2) combine output_DC_forBiasStudy/Mjj_201900402_lowVpt_backup_today_DCnew.txt -M FitDiagnostics --setParameters pdf_index=0 --toysFile higgsCombineTest.GenerateOnly.mH125.123456.root -t 100 --rMin -10 --rMax 10 --freezeParameters pdf_index 1) creates toys with a given functional form used for the fit (defined by the pdf_index variable) and for mu=1 (expectSignal=1) 2) runs the fit on the same toys but using other functional forms that will need to be checked (pdf_index=1,2,3,4, ) and check bias For highVpt workspaces, we need to add an additional command to help the fit convergence (most of the parameters are at boundary): combine output_DC_forBiasStudy/biasStudy_highVpt_update16May.txt -M FitDiagnostics --setParameters cate3=1 --toysFile higgsCombineTest.GenerateOnly.mH125.123456.root -t 500 --rMin=-10 --rMax=10 --freezeParameters cate3 --cminDefaultMinimizerStrategy 0 ---++ How to check bias root -l fitDiagnostics.root tree_fit_sb->Draw("(r-1)/rErr>>h(20,-4,4)") h->Fit("gaus") ---++ Where to find datacards to be used for bias study * low Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_lowVpt.txt * high Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_highVpt_update16May.txt * The workspaces the datacards above refer to are here: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/background_test_lowVpt_v4.root /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/background_test_highVpt_update16May.root * The code that creates the roomultipdf object dumped in the workspace and in the datacard for the bias study is here: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/producingWS_biasStudy.C * The code that makes the final bias plot is here: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/plot_biasStudy_highvpt.C /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/plot_biasStudy.C * REMINDER: pdf_index=0: Cheb, pdf_index=1: Bern, pdf_index=2: Exp, pdf_index=3:PowerLaw * category0: cate0=0: nominal Cheb, cate0=1: alt Bern, cate0=2: alt exp; cate0=3: alt power law; * category1: cate1=0: nominal Cheb, cate1=1: alt Bern, cate1=2: alt exp; cate1=3: alt power law; * category2: cate2=0: nominal Cheb, cate2=1: alt Bern, cate2=2: alt exp; cate2=3: alt power law; * category3: cate3=0: nominal Cheb, cate3=1: alt Bern, cate3=2: alt exp; cate3=3: alt power law; ---++ Code that runs fit to data Backup in mail to alessandro.calandri@cern.ch - May 7 (16h45) Code here: /t3home/acalandr/VHbb/data_driven/makeWorkspaceHHbbgg.py ---++ Additional implementation that runs more fits for bias study Backup in mail to alessandro.calandri@cern.ch - May 8 (18h13) ---++ Optimization F-test nominal fit Cheb Using F-test 1) low Vpt - cat0: cheb2 ---> cheb1 used (after optimisation on bias study) - cat1: cheb2 - cat2: cheb3 - cat3: cheb 3 1) high Vpt - cat0: cheb1 - cat1: cheb2 - cat2: cheb2 - cat3: cheb 2 ---++ Optimization F-test alternativel fit Bern polynomial 1) low Vpt - cat0: bern2. --> bern1 used (after optimisation on bias study) - cat1: bern2 - cat2: bern3 - cat3: bern 3 1) high Vpt - cat0: bern1 - cat1: bern2 - cat2: bern2 - cat3: bern 2 ---++ Optmization alternative fit expo for bias study #low Vpt if cat==0: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.006,-0.1,0.) #AC slope for cat0 low Vpt if cat==1: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.008,-0.1,0.) #AC slope for cat1 low Vpt if cat==2: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.006,-0.1,0.) #AC slope for cat2 low Vpt if cat==3: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.008,-0.1,0.) #AC slope for cat3 low Vpt #high Vpt if cat==0: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.004,-0.1,0.) #AC slope for cat0 high Vpt if cat==1: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.006,-0.1,0.) #AC slope for cat1 high Vpt if cat==2: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.002,-0.1,0.) #AC slope for cat2 high Vpt if cat==3: norm=ROOT.RooRealVar("exp_slope_%scat%d" % (label,cat),"exp_slope_%scat%d" % (label,cat),-0.004,-0.1,0.) #AC slope for cat3 high Vpt ---++ Optmization alternative fit power-law for bias study norm=ROOT.RooRealVar("pow_%scat%d" % (label,cat),"pow_%scat%d" % (label,cat),-100.,100.) #AC: linear term RooGenericPdf #lowVpt if cat==0: norm.setVal(-1); if cat==1: norm.setVal(-1); if cat==2: norm.setVal(-1); if cat==3: norm.setVal(-1) #high Vpt if cat==0: norm.setVal(-1); if cat==1: norm.setVal(-1); if cat==2: norm.setVal(-1); if cat==3: norm.setVal(-1) ---++ Running fit using HHbbgg framework > Combining cheb (nom), i.e. cate0=0 vs bern (alt), i.e. cate0=1 combine /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_lowVpt.txt -M GenerateOnly --setParameters cate0=0 -t 100 --expectSignal 1 --saveToys -m 125 --freezeParameters cate0 -S 0 combine output_DC_forBiasStudy/Mjj_201900402_lowVpt_backup_today_DCnew_new.txt -M FitDiagnostics --setParameters cate0=1 --toysFile higgsCombineTest.GenerateOnly.mH125.123456.root -t 100 --rMin -10 --rMax 10 --freezeParameters cate0 ====== Output Datacards for bias study: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_lowVpt.txt /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_DC_forBiasStudy/biasStudy_highVpt.txt ======= How to check bias: cate0=0 -> nominal Cheb category 0, cate0=1 -> alternative Bern category 0, cate0=2 -> alternative exp category 0, cate0=3 -> alternative powe law category 0 cate1=0 -> nominal Cheb category 1, cate1=1 -> alternative Bern category 1, cate1=2 -> alternative exp category 1, cate1=3 -> alternative powe law category 1 cate2=0 -> nominal Cheb category 2, cate2=1 -> alternative Bern category 2, cate2=2 -> alternative exp category 2, cate2=3 -> alternative powe law category 2 cate3=0 -> nominal Cheb category 3, cate3=1 -> alternative Bern category 3, cate2=2 -> alternative exp category 3, cate2=3 -> alternative powe law category 3 ---++ Calculating significance combine -M Significance output_9May_WS_and_DC/nom_cheb_lowVpt_v2.txt -t -1 --expectSignal=1 ---++ Where to find data cards for final significance estimation: - low Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_9May_WS_and_DC/nom_cheb_lowVpt_v2.txt - high VPt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_DC_highVpt_update16May/nom_cheb_highVpt_Update16May.txt - combine datacard: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/scripts/combined_mjj_2lep.txt ---++ Datacards with background normalization - Discussion on rate parameters: https://indico.cern.ch/event/577649/contributions/2339440/attachments/1380196/2097805/beyond_simple_datacards.pdf - low Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_9May_WS_and_DC/nom_cheb_lowVpt_v2_rateParam.txt - low Vpt (Correlated rate parameters): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/biasStudies/CMSSW_8_1_0/src/HiggsAnalysis/CombinedLimit/data/tutorials/bias_studies/output_9May_WS_and_DC/nom_cheb_lowVpt_v2_rateParam_correlatedRateParam.txt - high Vpt: /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_DC_highVpt_update16May/nom_cheb_highVpt_Update16May_rateParam.txt - high Vpt (correlated rate parameters): /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/output_DC_highVpt_update16May/nom_cheb_highVpt_Update16May_rateParam_correlatedRateParam.txt ---++ NEW OPTIMIZATION OF DNN BOUNDARIES TO IMPROVE SIGNIFICANCE WITH DATA YIELD FREE FLOATING /t3home/acalandr/VHbb/data_driven/DD_fit_HHbbgg/HHbbgg_ETH/Limits/macros/newCategorization_DNNcuts - low Vpt: - confg0: old DNN categorization - config1: [0,0.25,0.5,0.75,1] - .... - .... -- Main.AlessandroCalandri - 2018-10-11
This topic: Sandbox
>
WebPreferences
>
CheckLFuncertaintyonVHbb
Topic revision: r33 - 2019-06-17 - AlessandroCalandri
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use
Discourse
or
Send feedback