Inclusive ttbar cross section at √s=5.02 TeV using final states with at least one charged lepton;

Questions and answers on TOP-16-023 analysis

Introduction

CADI line

  • Black: Questions.
  • Green: answer by the authors; consensus
  • Blue: answer by the authors, which requires further discussion
  • Red: work in progress.

Comments based on pre-approval (v0 of paper draft):

3rd of February (J. Andrea)

Dear Authors,

even if the pre-approval is already given, I would like to make a few additional comments on this very nice analysis.

- Figure 4, top left plot. I still think there is some sort of trend in that plot, there is at least one bin around 3-3.5 that is statistically incompatible with the data. I would suggest to add data/MC ratio plots (I would do it for all the plots actually). If the uncertainties are covering for the discrepancy, then I would mention it in a way or another : by adding systematic uncertainties in the plot or at the minimum by a sentence in the text. I know that adding systematic to the plots is sometime difficult, and if it was compatible within the statistics it would have been ok as it is. But the 0 btag region is the only region where you have enough statistic to really make a comparison with a good amount of events. So if the plot is not good there, how confident can you be that the signal regions are well modelled ?
The systematic uncertainty associated to the W/QCD backgrounds is generous enough to cover these discrepancies (see answer below for more details). We'll try to produce a figure with syst uncertainties included.

- Section 7. The treatment of systematics, regarding the MC-estimated backgrounds, are significantly different between the dilepton and l+jets analyses. While dilepton use a more conservative approach (30% on the normalization of tW and WV) the lepton+jets analysis consider only the QCD scale uncertainty (I understood nothing more). The difference of approaches needs to be justified. I believe that the dilepton approach is more conservative (even if the 30% are always difficult to justify). In the case of l+jets, one needs anyway to explain why other W+jets modelling uncertainties (PDF, hadronisation, others?) are not considered. Adding a sentence in that direction in the text would be useful.
In the l+jets the W+jets background includes the QCD scale uncertainties (3 uncorrelated shape+normalization), theory cross section uncertainty at NNLO, and all the experimental (b-tag, mistag, jet energy scale, lumi, etc.). In addition bin-by-bin statistical uncertainties are assigned. The W+jets uncertainties are clearly dominated by the QCD scale variations which grow from ~11% in the 2jets-0b category to ~50% in the 2jets-2b category and by the bin-by-bin stat uncertainties which in the signal region can be as large as 50% in some bins. We have no reason to believe there is undercoverage with this prescription.

- section 8.3, line 371 : the sentence is miss-leading to me. It gives the impression the btag, missing ET and QCD normalization are considered as uncorrelated, while they are accounted for only for individual channel (btag and MET are used only for mm, QCD estimation is for l+jets only, right ?). BTW, I suggest you use multijet background instead of QCD : QCD "uncertainty" could refer to the scale.
To do

- I understood that the dilepton analysis is a cut&count analysis, while the l+jets is a profiling method where the postfit systematics should be correlated. How did you deal with that in the BLUE ? I can guess the answer, but that could be useful to explain it in the text.
Indeed, we make the approximation that all systematics are independent of each other. You can find in the new draft of AN-16-320 (l+jets) a plot showing the post-fit correlation matrix, which is mostly very diagonal but with a few exceptions, related to QCD scales in W+jets and JES. In the new draft of AN-16-358 (combination) we address this problem by performing three tests: 1) set W+jets correlation (inter-channel) to 0 instead of 1; 2) set JES correlation (inter-channel) to 0 instead of 1; 3) set emu's and mumu's statistical uncertainties to infinity and their systematic uncertainties to 0. The result of the first two tests show that the treatment of those uncertainties has a negligible effect on the combination, and the result of the third test shows that neglecting the correlations across systematics in l+jets has no visible effect.

I hope it helps, Jeremy

2nd of February (M. Aldaya, L. Skinnari)

Dear Authors,

Thank you very much for this very interesting analysis.

Below we include general and more specific comments on the paper v0 in addition to those already provided at the preapproval and circulated in this HN.

Cheers,

Louise, Maria

General:

- Given that the precision of the analysis is driven by the l+jets result, we would suggest to change the structure of the paper to describe first the l+jets measurement and then the dilepton one
Done.

- There are several TBDs to add numbers/description throughout the text; please provide them
Working on it.

- Please check and fix throughout the paper the inconsistencies in text and figures between l+jets and dileptons
Working on it.

Detailed comments:

- Abstract: propose to modify the first line as: "The top quark pair production is measured for the first time in proton-proton collisions at a centre-of-mass energy sqrt(s) = 5.02 TeV. The analysed data were collected by the CMS experiment at the CERN LHC and correspond to an integrated luminosity of 27.4 pb-1."
Done.

- L7: also cite ATLAS measurements?
To do.

- L11: add "(NLO)" definition here
Done.

- L17-22:

- suggest to change the order in the description of the channels: first describe l+jets and then dileptons. Please change accordingly throughout the document
Done.

- specify that leptons in this analysis refers to muons and electrons
Done in the next paragraph.

- it would be good also to state explicitly here which channels are considered in this analysis, i.e, "l+jets (where l = electron or muon), emu, mumu"
Done.

- L26/27: "former" and "latter" are not clear what they refer to (since there is a sentence between), please rephrase for clarity We have move the sentence in the middle to the end. We hope it is clear now..

- L29: please add the PDF analysis description
Added a sentence on QCD analysis.

- L30-34: references to sections 2, 3, and 9 are missing
Right. They have been added.

- Propose to move Section 4 ("Data and simulation samples") before Section 3 ("Object Reconstruction") to improve the flow of the paper and better connect to Section 5 ("Event Selection")
Done.

- L61-66: is this detailed description really needed, especially since you do not provide a similar description for the electrons? Suggest to remove the paragraph
To do.

- L96-100: "In this process [...] nonzero jet mass": is this description needed? Propose to remove
To do.

- L115:

- "W+jets" --> "W boson production with additional jets (W+jets)"
Done.

- "mll > 50 GeV" --> is this information really necessary? If not, suggest to remove
To do.

- L120: is there no contribution from single top t-channel background in the l+jets analysis?
tW is included as a background, it's at the level of 6% (Table 5/Fig.3).

- Section 5, General: the event selection section has some inconsistencies and is difficult to follow. suggest to:

- rewrite (as already noted above) to describe l+jets first, then dilepton channel
Done.

- remove details about 'loose' vs 'tight' IDs etc.
Done.

- remove table 1
Done.

- move trigger description to earlier in the section
Done.

- L133-140: state explicitly the channels that are considered in the analysis, i.e, l+jets (assuming that "l" has been defined already as proposed in an earlier comment), emu, mumu
This has been done in the Introduction now.

- L148: "Electrons are furthermore [...]" --> remove
Done.

- L152: Remove "electromagnetic calorimeter" (already defined in Section 2)
Done.

- L165: "contribution, events" (comma missing)
Done.

- L166: "106" --> 106 GeV
Done.

- L168: which MET cut is used?
35 GeV. Added now..

- L168-170: the definition of MET should go to the "Object Reconstruction" section. Please also check that the definition follows the Publication Guidelines
To do

- Around L172-176 + later around L325: this was brought up already in the PAG meeting: please clarify the definitions of the number of jets vs b-jets
Events are pre-selected with at least two non b-tagged jets. Extra b-tagged jets are counted in the event. In practice the 2 b-tags category is a 4jets inclusive event with at least two b-tags. We have rephrased it in the paper.

- L177: "Jets are thus are" --> remove second "are"
Done.

- L184: "select a high purity tt->emu sample" --> remove (remnant from PAS)
Done.

- Table 2: MET cut value missing
Fixed.

- L188: tautau also to mumu, right?
Right, added.

- L199: "Z boson mass" ("boson" missing)
Done.

- L200: missing numbers
We have fixed that.

- L218: missing description
We have added it.

- L223-224 and in other parts of the text: please choose a consistent naming convention for c and b quarks/jets/hadrons. Eg, here it is "charm", "beauty"; in L190 "bottom"; in L177-180 "b, c", etc
To do.

- L228-229: "fails the relaxed identification criteria" --> not clear what the "the relaxed" refers to here. Propose to rephrase L227-229: "[...] a control region enhanced in non-prompt leptons, defined through alternative lepton identification criteria". If necessary, then explain in few words what the alternative criteria are.
We have rephrased it.

- L229 and throughout the text: "Sec." --> "Section"
Done.

- L242: "latter" --> "later"
Done.

- L249-252: a bit of jargon here ("changing the SF values" etc), better to specify what the uncertainties actually are?
To do.

- L258: "renormalization and factorization" --> "renormalization (mu_R) and factorization (mu_F)"
Done.

- L278-281: yes, this should be moved later and described together with l+jets
Done.

- Table 3: what is the MET uncertainty? How is it determined?
To do. Enrique et al.?

- L320: "QCD scale choice" --> it is unclear what it is meant in this context. Please rephrase
Rephrased the paragraph

- L344, 347: "cross-check analysis" --> please explain more clearly what the cross-check analysis is
We have rephrased it.

- Table 5: "Z/gamma*" --> please use consistent naming conventions for l+jets and dileptons (Z/gamma* vs. DY). Please also homogenize the information and naming conventions in tables 4 and 5.
To do. Pedro and Enrique?

- left plot in Fig 4 + in fig 6: "Distr" and "Count" is used in the figure and in the caption it is described as "main" and "cross-check", not clear what is what. Please update.
To do. Pedro et al.?

- Table 6: Is the statistical uncertainty for the counting analysis really 1% (compared to 9.5% for the fit method)? Or is this a typo that should say "0.10" instead?
Good catch, thanks. It's 0.1 otherwise wouldn't add to the total 0.19.

- L372: "for a single channel" => do you mean "for the l+jets channel"?
Right. We now write so.

- below L374: Is it possible to separate out the stat vs syst vs lumi uncertainty for the combined result?
Done. And the exact procedure for the splitting is documented in AN-16-358.

- Section 9: please add a few additional sentences as basic introduction for what this is about to describe. Also, propose to change the title of the section to "PDF fit" (also consistent with TOP-14-013 paper)
The very first sentence tells already that the data are subject of the QCD analysis. The "QCD analysis" is the term used in the respective papers by CMS and other collaborations describing a PDF fit, please see SMP-12-021, SMP-14-022, SMP-14-001 which are already published. "PDF fit" is a slang which sneaked unfortunately into the TOP-14-013. Since the QCD analysis is only illustrative and is a small add on to the paper we would like to reduce the amount of details rather than adding yet additional introduction.

- L386: "The ttbar production cross section measurements"
Done.

- L389: please specify which CMS measurement is used and add a reference. It would be useful to add an explanation why these data are used (i.e, to provide further constraints on the valence quark distribution)
Thanks a lot for spotting missing reference. The reference and a sentence on the improved light quark constraint are added.

- L391: "top quark pair reconstruction channels" => "ttbar final states"
Done.

- L396: "background" (typo)
Done.

- L409: missing reference
Done.

- L417-418: this sentence is missing some info, "... used in the QCD analysis as described in the context of ... in Ref [52]" ?
There is nothing missing in the sentence, it describes which PDFs are fitted and is complete.

- L447-450: would it be useful to comment on why the asymmetric iterative procedure from Ref[67] does not yield any significant reduction in the PDF uncertainty? Is this expected? Does it have any implications?
The reason is too few additional points in the analysis with low weight, which apparently does not fix the bias in the asymmetric iterative Hessian procedure. But this is just an assumption. Since for the low number of points with low weight we rather trust MC method for the uncertainty calculation, we would like to only show this result. We removed the technical information on the Hessian method completely from the paper. The additional reason is that the latest version of XFitter where the additional statistical study is possible, is not yet publicly released, so we can not give a proper reference to it.

- L466: "2" --> "two"
Done.

- L468-469: the numbers are different from what is shown in the results section, please make sure they are all up-to-date.
Fixed.

1st of February (R. Gonzalez, M. Aldaya, L. Skinnari)

General comments:

- the paper needs to improve quality:
- unify writing styles and conventions
- unify plotting styles
Working on it.%

- In terms of structure:
- the dilepton channel is presented as main analysis (first) even when it is less precise
- l+jets should be the primary analysis
Done, l+jets is described first now.

- balance it out removing some of the dilepton material
To do.

- Add one line explaining the choice of robust and simple approach for dilepton and sophisticated for l+jets
To do.

- double check that you are quoting the correct beam energy uncertainty everywhere
To do.

- Table1 (paper version 0) to be removed
Done.

- Discuss the journal target as soon as possible
To do.

Slide by slide/specific comments (see here the sides):

30) discrepancies in the 0 b-tags -> show sensitivity by categories

31) # jets vs # b-jets, the 0, 1, 2 b-tag plots mean >=2, 3, 4 jets ? -> Yes (i.e. in addition to two light jets)

33) what is going on with the events around 175GeV in the 2b bin? -> It is likely a fluctuation (stats are low), does not appear in the e+jets channel

38) said that taking electrons that fail iso doesn’t work, instead take fail ID — why does the latter work and former doesn’t? -> fail-iso doesn’t at all describe the data in the endcaps

How were the jj’ pair selected? -> a few metrics, described in notes

45) Systematic table for l+jets should have the same explanation (to avoid discussion along the same lines) as 16-006
This will be less controversial than for 16-006 but something will be added
We have included a sentence commenting on the fact that the procedure may mitigate the impact of certain nuisances, at the cost of correlations, i.e., by pulling other nuisances in the fit.

48) if Iterative was tested, why standard was used? -> iterative offered a slightly better performance but it is more controversial. Since the combination is clearly driven by one of the results and statistically limited, the analyzers decided to sick to the more conservative method

49) Can the mumu channel be removed? -> analyzers prefer to keep it, even when it contibutes with only 5% to the combination. The reason is completness (and self-consistency).

Follow up on this: why the ee was not added? -> the reason is the triger --> add a small statement on the lack of ee channel in the dilepton.

59/60) Impact plots

- why the JES? is pulled in the shape? -> the effect was checked with toys, and the effect of the nuisance is very small, analyzers are not worried

- why the 59 and 60 are different? -> counting does not use the same information, it is less powerful, constrains much less

30th of January (R. Gonzalez)

General comments:

- Some things tbd in the paper such as: splitting of the systematics, description of PDF fit. Will they be done for the approval talk?
To Do.

- Are the numbers in Table 1 final?
Yes. But this table has been removed.

- What happens with the MET cut in dilepton?
It was missing and has been added now.

- I guess that in the paper, when you say "dilepton" in the DY and fakes background estimation(6.1.1 and 6.1.2) you mean same flavor
To Do.

- With respect to the systematic sections: I suggest to keep the general description in Section 7 but the tables referred to by their own sections, the dilepton systematics in the dilepton and the l+jets in the l+jets (within section 8)
To Do.

- Table 2 should also have the use of b-tagging reflected
b-tagging is not used at all in the dilepton channel and in the l+jets channel it is used only to classify the events, not to select/reject them.

- Plots:

I suggest removing the X uncertainty (all plots) and making the data points bigger (Fig 1 and 2).
Done.

Also make sure that the treatment of the uncertainties in the data y axis is correct (and does not go below cero).
We draw Poisson errors in data as suggested for TOP-16-005.

Also, you need to unify style (Figs 1, 2 and 3 are not in the same style)
To do (Enrique and Pedro).

- Figure 4: shall we add the observed point?
To do (Pedro et al).

- Figure 5: maybe in the little square we could add both l+jets and dilepton as well? (just a suggestion to discuss) .
To do (Pedro et al).

Line-by-line comments:

l.4 remove (\sqrt{s}) is not needed
Done.

l.6 you can also remove (pp) or proton-proton, no need to explain the use of the acronym if you use it later, just use either proton-proton or pp everywhere and it should be fine
We have removed proton-proton and use pp everywhere.

l.10-12 this parenthesis is a bit long. I suggest to promote it to normal sentence.
Done.

l.17 in the SM
Done.

l.21 "only" one W bosons decay leptonically
Done.

l.25-26 maybe move this sentence before "The analysis uses the full..." so former and later are even clearer
Done.

l.89 HLT was not described before
We have removed "HLT".

l.108 MC was used already in line 103
Fixed.

l.133-140 is redundant. Consider for example either removing or adding more detail to the intro when this is discussed and removing it as well
Done.

Equation above line 289: there seems to be some space in the equation that is preventing the line numbers to show, use \linenomath to avoid this.
Fixed.

Also, the equation number is not showing.
Added. Although I'm not sure it is needed...

Table 5: add uncertainties to the numbers
The table includes now the stat uncertainty for all processes and stat+syst for the QCD estimate

l.468 71.8~pb
Fixed.

Edit | Attach | Watch | Print version | History: r12 < r11 < r10 < r9 < r8 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r12 - 2017-02-10 - KaterinaLipka
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback