Hbb comments, Sep 6, presentation ( https
://indico.cern.ch/event/663414/contributions/2711214/attachments/1519476/2373083/ HH_Sep_6_HH4b.pdf)
1) Trigger efficiency measurement : for the b-tagging online requirement can you check the efficiency with respect to your offline selection (on b-tagging)?
B tag variables are close to fully efficient for OR of triggers, slightly less efficient for just the BTag trigger: Slides 15-18 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
2) ak4 dijet mass requirement : have you tried to optimize this selection taking into account the b-jet energy regression ?
Regression doesn't give a definitive improvement: Slide 23 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
Expected yields from datacards, mean/standard deviation of AK4 dijet mass at preselection on slide 7: https://indico.cern.ch/event/668497/contributions/2743135/attachments/1535204/2404751/HH_Oct_4.pdf
3) Have you checked the ttbar normalization in a CR in data? ttbar simulation is not really reliable at such high pt regime . Is it a free parameter in the final fit?
The amount of ttbar doesn't make a big difference in the background estimation: Slides 19-20 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
4) How much is the signal contamination in the CR used for the closure?
Almost none: Slide 13 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
5) Can you show the closure for other variables? It doesn’t look that good for the reduced mass.
Closure region now doesn't include the triAK4jet mass cut to increase stats and is improved - see Slides 12, 21-22 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
6) TF parametrization is quadratic? How the polynomial degree has been decided?
F-test was performed to show that linear is not sufficient and cubic is not needed for the fit to the pass/fail ratio: slide 16-18 https://indico.cern.ch/event/668497/contributions/2743135/attachments/1535204/2404751/HH_Oct_4.pdf
7) How the 750 GeV requirement on the Reduced mass has been optimized for the non -resonant search?
Our analysis is not sensitive below 750 GeV.
8) Please check the overlap of events with the resolved non-resonant analysis
Slide 8 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf, waiting on Martino to look at events
9) Please check the overlap of events with the resolved resonant analysis (refer to AN-16-268v7 and here for the regression)
The overlap with the resolved resonant analysis can be found here: slide 21 https://indico.cern.ch/event/668497/contributions/2743135/attachments/1535204/2404751/HH_Oct_4.pdf
Clemens comments, AN17-182v2 ( http://cms.cern.ch/iCMS/jsp/openfile.jsp?tp=draft&files=AN2017_182_v2.pdf)
Abstract for iCMS title page: end math mode after 1/fb
Please add a changelog after the table of contents, this helps keeping track of changes
The internal links seem to be broken, would be nice to have it fixed in the next iteration
1. Introduction:
107: while it's not that important, and I didn't read the theory part in detail, it's a bit odd you motivate the 2 TeV mass region where your limits basically stop
2. Data and simulated samples:
134: I know that ktilde = 0.1 has been used quite a lot, but one should be careful when eventually writing the paper noting that at this point the decay width is not entirely negligible compared to the experimental resolution
Table 2: low mass cross sections are not listed (while they are in the repository)
Table 4 lists only the mass points that are eventually used, if I understand correctly. In this case, I would suggest a brief comment in the text to avoid confusion
3. Event reconstruction and selection
160-161: Did you try choosing the two AK4 jets that give you the best Higgs mass instead of selecting them based on the b-tag?
Yes, it didn't make much of a difference: Slide 9, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
Fig. 1: Please explain the dip in the high Delta(eta) region to avoid further questions
192-194: Is there another orthogonal sample you could use to study the turn-on? In the muon sample it had shown that the turn-on happened significantly later than in the JetHT sample.
You could actually use a different set of triggers as used in HIG-16-018: HLT_DoubleJetsC100_DoubleBTagCSV_p014_DoublePFJetsC100MaxDeta1p6 and HLT_DoubleJetsC100_DoubleBTagCSV_p026_DoublePFJetsC160 (the latter probably being more generally usable, but maybe the higher pT thresholds render them useless here) Could that serve as a cross check?
No, we have a lepton veto, just like the boosted analysis does: Slide 9, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
205-206: Do you check if the leptons are outside your candidate jets to avoid throwing away leptonic b decays?
No need, since the leptons are isolated and this cut is fully efficient for signal: Slide 9, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
216: If I understand correctly, you use PUPPI also for the AK8 jet 4-vectors, but not for the AK4 ones. Is there any reason not to use it for AK4 as well (would also make the paper writing easier)?
Not implemented in heppy ntuples: Slide 9, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
Fig. 5: Please explain shape of leading jet pT
This shape goes away after we add the mass window cuts on both sides and a tighter AK4 deep CSV cut: Slide 10, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
4. DataMC Comparison
Generally, it would be nice to have ratios below the data-MC comparison plots
Also, is there any scaling applied to the MC or does this really come out like this?
No scaling of data to MC or vice versa: Slide 10, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
5. Background Estimation
Figs. 27-32: Could you add some goodness of fit/measure of agreement to the right-hand side plots? The predictions seem to be quite off, e.g. in Fig. 28.
Also, please make sure the y-axis shows the whole data spectrum.
It would be good to have a table that lists the predicted vs. the observed ones. Could you also show the sideband fits?
We have updated the control region to have more statistics. We also looked at the p-value for the control region: https://indico.cern.ch/event/670946/contributions/2745601/attachments/1536748/2407794/B2G_Oct_6.pdf Slide 7.
7. Systematic uncertainties
Regarding the DeepCSV uncertainties, how do you treat tag and mistag? From the datacard it seems as if you have only one uncertainty
Deep CSV uncertainties are assigned based off of flavour of the jet, Slide 10: https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
M. Blinded limits
Why don't you move them to the results section? They are the central result.
Jacopo's comments, AN17-217v2 ( http ://cms.cern.ch:80/iCMS/jsp/openfile.jsp?tp=draft&files=AN2017_217_v2. pdf)
One of the main concerns is clearly the overlap of the final states probed by this analysis with the fully resolved one. Mostly for the 2+1 category, this could be very relevant, and some strategy to separate the two should be considered, despite the use of a BDT for HIG-17-007 could make the separation complicated.
On the systematics, what is the reason why the PDF and scale systematics is not re-evaluated here for the signal, and is instead taken from the non-resonant non-boosted analysis?
We're using the v0 benchmark samples, so we didn't think there was a point in calculating the PDF and scale systematics (they're very small anyways). Once we have the v1 benchmark samples, we can calculate these systematics for ourselves.
Concerning the implementation of the uncertainties, I don't see however the Jet Mass Resolution (that you quote in L153) nuisance in the datacard.
This is implemented in the H-tagging systematic, just as in the boosted resonant analysis - from AN-16-300-v8, Section 6:
Fig7: closure doesn't seem great. This has been discussed already in the meeting, but the point of this closure test should be to ensure that the procedure is working fine in a region you can control. Please also quantify the data/MC agreement by means of chi-square and/or p-value.
We looked at the p-value for the control region: https://indico.cern.ch/event/670946/contributions/2745601/attachments/1536748/2407794/B2G_Oct_6.pdf
Slide 7.
Sec5: please add also 68/95% coverage of the expected limit on the signal xsec, and include some validation plot showing the impacts of the nuisances on the Expected limit.
Added, and impacts of nuisances can be found Slides 8-10 https://indico.cern.ch/event/670946/contributions/2745601/attachments/1536748/2407794/B2G_Oct_6.pdf.
Hbb comments, Oct 4 presentation (https://indico.cern.ch/event/668497/contributions/2743135/attachments/1535204/2404751/HH_Oct_4.pdf)
-- AliceSady - 2017-09-25
://indico.cern.ch/event/663414/contributions/2711214/attachments/1519476/2373083/ HH_Sep_6_HH4b.pdf)
1) Trigger efficiency measurement : for the b-tagging online requirement can you check the efficiency with respect to your offline selection (on b-tagging)?
B tag variables are close to fully efficient for OR of triggers, slightly less efficient for just the BTag trigger: Slides 15-18 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
2) ak4 dijet mass requirement : have you tried to optimize this selection taking into account the b-jet energy regression ?
Regression doesn't give a definitive improvement: Slide 23 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
Expected yields from datacards, mean/standard deviation of AK4 dijet mass at preselection on slide 7: https://indico.cern.ch/event/668497/contributions/2743135/attachments/1535204/2404751/HH_Oct_4.pdf
3) Have you checked the ttbar normalization in a CR in data? ttbar simulation is not really reliable at such high pt regime . Is it a free parameter in the final fit?
The amount of ttbar doesn't make a big difference in the background estimation: Slides 19-20 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
4) How much is the signal contamination in the CR used for the closure?
Almost none: Slide 13 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
5) Can you show the closure for other variables? It doesn’t look that good for the reduced mass.
Closure region now doesn't include the triAK4jet mass cut to increase stats and is improved - see Slides 12, 21-22 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
6) TF parametrization is quadratic? How the polynomial degree has been decided?
F-test was performed to show that linear is not sufficient and cubic is not needed for the fit to the pass/fail ratio: slide 16-18 https://indico.cern.ch/event/668497/contributions/2743135/attachments/1535204/2404751/HH_Oct_4.pdf
7) How the 750 GeV requirement on the Reduced mass has been optimized for the non -resonant search?
Our analysis is not sensitive below 750 GeV.
8) Please check the overlap of events with the resolved non-resonant analysis
Slide 8 https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf, waiting on Martino to look at events
9) Please check the overlap of events with the resolved resonant analysis (refer to AN-16-268v7 and here for the regression)
The overlap with the resolved resonant analysis can be found here: slide 21 https://indico.cern.ch/event/668497/contributions/2743135/attachments/1535204/2404751/HH_Oct_4.pdf
Clemens comments, AN17-182v2 ( http://cms.cern.ch/iCMS/jsp/openfile.jsp?tp=draft&files=AN2017_182_v2.pdf)
Abstract for iCMS title page: end math mode after 1/fb
Please add a changelog after the table of contents, this helps keeping track of changes
The internal links seem to be broken, would be nice to have it fixed in the next iteration
1. Introduction:
107: while it's not that important, and I didn't read the theory part in detail, it's a bit odd you motivate the 2 TeV mass region where your limits basically stop
2. Data and simulated samples:
134: I know that ktilde = 0.1 has been used quite a lot, but one should be careful when eventually writing the paper noting that at this point the decay width is not entirely negligible compared to the experimental resolution
Table 2: low mass cross sections are not listed (while they are in the repository)
Table 4 lists only the mass points that are eventually used, if I understand correctly. In this case, I would suggest a brief comment in the text to avoid confusion
3. Event reconstruction and selection
160-161: Did you try choosing the two AK4 jets that give you the best Higgs mass instead of selecting them based on the b-tag?
Yes, it didn't make much of a difference: Slide 9, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
Fig. 1: Please explain the dip in the high Delta(eta) region to avoid further questions
192-194: Is there another orthogonal sample you could use to study the turn-on? In the muon sample it had shown that the turn-on happened significantly later than in the JetHT sample.
You could actually use a different set of triggers as used in HIG-16-018: HLT_DoubleJetsC100_DoubleBTagCSV_p014_DoublePFJetsC100MaxDeta1p6 and HLT_DoubleJetsC100_DoubleBTagCSV_p026_DoublePFJetsC160 (the latter probably being more generally usable, but maybe the higher pT thresholds render them useless here) Could that serve as a cross check?
No, we have a lepton veto, just like the boosted analysis does: Slide 9, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
205-206: Do you check if the leptons are outside your candidate jets to avoid throwing away leptonic b decays?
No need, since the leptons are isolated and this cut is fully efficient for signal: Slide 9, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
216: If I understand correctly, you use PUPPI also for the AK8 jet 4-vectors, but not for the AK4 ones. Is there any reason not to use it for AK4 as well (would also make the paper writing easier)?
Not implemented in heppy ntuples: Slide 9, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
Fig. 5: Please explain shape of leading jet pT
This shape goes away after we add the mass window cuts on both sides and a tighter AK4 deep CSV cut: Slide 10, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
4. DataMC Comparison
Generally, it would be nice to have ratios below the data-MC comparison plots
Also, is there any scaling applied to the MC or does this really come out like this?
No scaling of data to MC or vice versa: Slide 10, https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
5. Background Estimation
Figs. 27-32: Could you add some goodness of fit/measure of agreement to the right-hand side plots? The predictions seem to be quite off, e.g. in Fig. 28.
Also, please make sure the y-axis shows the whole data spectrum.
It would be good to have a table that lists the predicted vs. the observed ones. Could you also show the sideband fits?
We have updated the control region to have more statistics. We also looked at the p-value for the control region: https://indico.cern.ch/event/670946/contributions/2745601/attachments/1536748/2407794/B2G_Oct_6.pdf Slide 7.
7. Systematic uncertainties
Regarding the DeepCSV uncertainties, how do you treat tag and mistag? From the datacard it seems as if you have only one uncertainty
Deep CSV uncertainties are assigned based off of flavour of the jet, Slide 10: https://indico.cern.ch/event/666727/contributions/2725251/attachments/1528658/2391357/DIB_Sep_20.pdf
M. Blinded limits
Why don't you move them to the results section? They are the central result.
Jacopo's comments, AN17-217v2 ( http ://cms.cern.ch:80/iCMS/jsp/openfile.jsp?tp=draft&files=AN2017_217_v2. pdf)
One of the main concerns is clearly the overlap of the final states probed by this analysis with the fully resolved one. Mostly for the 2+1 category, this could be very relevant, and some strategy to separate the two should be considered, despite the use of a BDT for HIG-17-007 could make the separation complicated.
On the systematics, what is the reason why the PDF and scale systematics is not re-evaluated here for the signal, and is instead taken from the non-resonant non-boosted analysis?
We're using the v0 benchmark samples, so we didn't think there was a point in calculating the PDF and scale systematics (they're very small anyways). Once we have the v1 benchmark samples, we can calculate these systematics for ourselves.
Concerning the implementation of the uncertainties, I don't see however the Jet Mass Resolution (that you quote in L153) nuisance in the datacard.
This is implemented in the H-tagging systematic, just as in the boosted resonant analysis - from AN-16-300-v8, Section 6:
"Similar to the τ21 scale factor uncertainty above, the jet mass correction is also evaluated using W jets in semileptonic tt+jets events, amounting to 1 ± 0.0094. A jet mass resolution scale factor of 1 ± 0.20 is also obtained using the above measurement. These are propagated to the Higgs-tagged jets along with the uncertainties. "
Alphabet method: what is the rationale for choosing a 2nd order pol for modeling the pass/fail ratio? Is there a physical motivation to it, os is it empirically motivated? If the choice is arbitrary, are you considering to include additional uncertainties, to quantify the impact of choosing a pol2 wrt other trends (e.g. higher-order pols)?
F-test was performed to show that linear is not sufficient and cubic is not needed for the fit to the pass/fail ratio: slide 16-18 https://indico.cern.ch/event/668497/contributions/2743135/attachments/1535204/2404751/HH_Oct_4.pdf
Fig7: closure doesn't seem great. This has been discussed already in the meeting, but the point of this closure test should be to ensure that the procedure is working fine in a region you can control. Please also quantify the data/MC agreement by means of chi-square and/or p-value.
We looked at the p-value for the control region: https://indico.cern.ch/event/670946/contributions/2745601/attachments/1536748/2407794/B2G_Oct_6.pdf Slide 7.
Sec5: please add also 68/95% coverage of the expected limit on the signal xsec, and include some validation plot showing the impacts of the nuisances on the Expected limit.
Added, and impacts of nuisances can be found Slides 8-10 https://indico.cern.ch/event/670946/contributions/2745601/attachments/1536748/2407794/B2G_Oct_6.pdf.
Hbb comments, Oct 4 presentation (https://indico.cern.ch/event/668497/contributions/2743135/attachments/1535204/2404751/HH_Oct_4.pdf)
-- AliceSady - 2017-09-25 
Home 
Plugins
Sandbox for tests
Support
Alice
Atlas
CMS
LHCb
Public Webs
Sandbox Web 
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Account 
Copyright &© 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback