order---+ Review of TOP -21-014

## Documentation

(Analysts: Please do not use .png format for plots on the twiki-- they do not show up on Safari browsers.)

Color code for answers to reviewer questions:

• Green -- we agree, changes to analysis/documentation implemented.
• Lime -- we agree, but the item hasn't been done yet. (Open item.)
• Red -- we disagree, changes to analysis/documentation is not implemented.
• Teal -- we agree, but we don't think any change to analysis/documentation is needed.
• Blue-- authors/ARC/conveners need to discuss. (Open item.)

## Explicit green-lights from experts

Category Name Status
Conveners   DONE
PPD   N/A
GEN   DONE.
TRIG   DONE.
EGM   DONE
MUO DONE
TAU   N/A
JME   DONE
BTV   DONE
TRK   N/A
STAT
Combine   Done

Every analysis is supposed to have a HEPData entry, with a BiBTeX entry in the paper and a reference to it in the introduction in the form of a sentence like: "Tabulated results are provided in the HEPData record for this analysis~\cite{hepdata}.” I think this sentence belongs at the very end of the introduction. More information can be found by searching for “HEPData” in https://twiki.cern.ch/twiki/bin/viewauth/CMS/Internal/Publications

The sentence has been added to the Introduction although the hep data reference still needs to be identified...The end of Section 6 or the Summary seems like a better place for the sentence, if it must go in the paper.

Whenever you use an operator like =, >, < that only has an argument on the right side, it is a unary operator and the spacing is changed so that the operator and the value on the right are right next to each other. The proper LaTeX way is: {>} 900\GeV. You can search https://twiki.cern.ch/twiki/bin/viewauth/CMS/Internal/PubGuidelines for “unary” to see more. This shows up in the last line of the abstract, L76, Table 2 caption, Table 2 first column, L240, Figure 3 x-axis.

Changed in the text (except for L7 where Tom had requested {\approx}\,6). It's not clear that the Guidelines apply to table entries and figures, where the latter don't use latex formatting. For Table 2 we feel that column one looks better with the extra spaces (also with "750 -- 900").

L3: I don’t think “LHC energies” is correct because it is really dependent on both the energy and the colliding species. I would suggest removing the word “energies” or changing the sentence to “The production of \ttbar from proton-proton (pp) collisions at the LHC comes from gluon fusion process about 90% of the time, with \qqbar annihilation making up the rest.”

Done.

L73-75: This is a new sentence which says that AK4 jets are “removed” if they are within DeltaR<0.8 of an AK8 jet. What does this mean? Does this mean the AK4 jet is removed from the AK8 jet (I hope not)? Does this mean the AK4 jet is discarded from the event (it cannot be considered as a b-tag for the W-tag events or as any of the jets in a resolved event)? Please make it clear to me (and ideally in the paper).

Changed to: "Any AK4 jet with $\Delta R = \sqrt{\smash[b]{(\Delta\eta)^2+(\Delta\phi)^2}} <0.8$ from the closest AK8 jet, where $\phi$ is the azimuthal angle, is discarded from the event."

L128-131: I still think there is some ambiguity here. What do you do if an event has more than one t tag or more than one W tag? Do you discard the event? Do you try multiple combinations? Do you give up and look for AK4 jets? I think you need a sentence here explaining.

Added: "Events with more than one \PQt or \PW tag are discarded. "

Table 1 caption: Change “MC simulations” to “MC simulation” or “MC simulation contributions”

The plural form seems more appropriate to us since there are multiple rows - one for each simulation.

Table 1: Change “124” to “120”

Done. Assuming this is a significant digits issue.

L240-2: I think removing the equations defining rpos are a good idea. I think that the main reason to fit for A_C directly is because that is the parameter we are interested in and the returned uncertainties will be more accurate than trying to propagate them. My proposed sentence here would be “Instead, we fit for r_neg and A_C^fid directly to ensure that the uncertainty on A_C^fid is correctly estimated.”

Done.

Table 2: I think this looks pretty good. I think that the last column should be “Theory” rather than “A_C (theory)”. In the first column, there is a problem with a space before “GeV”. I think you can writ this as “\GeVns” rather than “\GeV” to fix this. I suggest a bit more horizontal space between the “Total” column and the “Theory” column. You can add more horizontal space by just adding a new (blank) column between them. I would generally reduce the amount of vertical space, especially between the 750-900 and >900 lines.

Changed to "Theory" and "\GeVns" and added the extra space before the theort column. We see extra spaces in column one (750 -- 900, > 900) as an allowed esthetic improvement over the main body spacing rules.

L255: Remove “also”

Done.

L258: You define \sigma in the Figure 4 caption as it is used in the figure. I don’t think you need to define \sigma in the text as it is not used in the text anywhere. I suggest removing “$(\sigma)$”

Leave in place as this was added at Tom's request.

Figure 3: The y-axis label is too close to the y-axis values.

Done

Figure 4: I realize that the x-axis is currently in %. I would suggest either changing the label to “Impact on A_c (%)” or dividing the x-axis values by 100. In any case, I think the font size of “Impact on A_C” should be increased so it is as large as the values.

Done

Figure 4 caption: You write “dominant” but you include contributions that can’t even be seen. Are there any systematic uncertainties you don’t include? If not, then please remove “dominant” (and also from L258).

Dominant removed. The only ones that are not degligigle are the MC stats and that is mentioned as not being included

Figure 4 caption: I think that “inclusive” is referring to the M_ttbar>750 GeV. If so, then it is redundant. If not, then you will confuse this reader at least. I suggest removing “inclusive”. In fact, it may be useful to replace “inclusive” with “full phase space” as that piece of information is less prominent (a reader has to distinguish between A_C and A_C^fid).

"inclusive " replaced with "full phase space".

L280: Change “The result” to “This result”%

"The" seems stronger to us. ("This" has the implication that there might have been other results.)

72 Is Francesca's question about AK4 and AK8 jets really answered with this sentence? Could you write: "The same PF candidates are used to build large-radius (AK8) jets using a distance parameter of 0.8 ..." This makes it clear that the particles in AK4 and AK8 jets are not exclusive to one or the other.

Done

73 "The AK4 jets" don't start a sentence with an acronym. See Guidelines.

Done

118, 119 You have two consecutive sentences that start "All samples are", which is repetitious. You could rewrite the second sentence as: "The NNPDF 3.0 (2016) and NNPDF 3.1 (2017 and 2018) parton distribution functions (PDFs) [47] are used for all samples and include ..."

Done

128 I think it is useful to mention that you have already qualitatively discussed these three topologies. So write: "separated into the three topologies discussed earlier based on"

Done

141 Is it useful to say "determined from simulation for each topology" ? That is probably described in Ref. [21], but it would show the reader that the top quark mass used is not just one number and depends on the event topology.

"procedure described in Ref. [21]." this is how we directly refer to a reference.

Done

158 I think you have misunderstood the integrated luminosity uncertainty. The CMS detector twiki suggests writing: "while the overall uncertainty for the 2016--2018 period is 1.6\%" So this is not an additional uncertainty. It is the total uncertainty in the summed integrated luminosity for the three years. Unfortunately, the twiki page doesn't give a reference for where this number comes from. But other CMS publications don't have a reference for the 1.6% number either. So I would just use the suggested wording exactly.

Is this what you mean: "Additionally, uncertainties in the integrated luminosity vary per year: 2.5, 2.3, and 1.2\% for 2018~\cite{CMS-PAS-LUM-18-002}, 2017~\cite{CMS-PAS-LUM-17-004}, and 2016~\cite{CMS-LUM-17-003}, respectively, and include both correlated and uncorrelated components across the three years, while the overall uncertainty for the 2016--2018 period is 1.6\%.

167 Would it be clearer to write: "The uncertainty associated with the possible misidentification of the sign of the lepton electric charge is negligible." ?

Done

229 If you don't hyphenate "higher p_T" in the previous line, you shouldn't hyphenate "highest p_T" here. Greg Landsberg believes you should never hyphenate this type of phrase.

Done

236 I worry that a referee is going to ask questions about our fitting for r_neg and A_c^fid the way it is written now. How about writing something like: "One way to measure A_C^fid, the top quark charge asymmetry in the fiducial phase space, is to fit for r_pos and r_neg, the signal strengths that scale the contribution of events with Delta |y| > 0 and < 0, respectively, and then use Eq. (1) to determine A_C^fid. Instead, we fit for r_neg and A_C^fid directly, which insures the proper handling of the correlations between different sources of uncertainty."

Changed to: Combinations of subsets of these channels are also possible and allow us to obtain results for the two mass regions separately. In all cases, the unfolding performs a multi-dimensional maximum likelihood fit of the simulation to the observed data and returns two measured parameters. One way to measure \ACfid, the top quark charge asymmetry in the fiducial phase space, is to fit for \rpos and \rneg, the signal strengths that scale the contribution of events with $\deltay>0$ and < 0, respectively, and then use Eq.~\ref{eq:AC} to determine \ACfid. Instead, we fit for \rneg and \ACfid directly, which insures the proper handling of the correlations between different sources of uncertainty.

Table 2 caption: "compared with the theoretical prediction from MC", since you have "theory" in the table heading, I think it is good to have a similar word in the figure caption description. "are also shown. All values are in percent." It's always good to make this clear in the caption.

Done

Table 2: Capitalize "Measured".

Done

Fig. 3: Increase the size of the numbers on the x-axis labels. They are way too small and you have lots of room. I wouldn't mind if you increased the size of the x- and y-axis labels also. They are bordering on too small, and again, you have lots of space. Make them easy to read. Write "Measured A_C^fid" and "Predicted A_C^fid" in the left legend. This is your "money plot" that will be shown at conferences. You want to make everything very clear and easy to read.

Done

Fig. 4 caption: "uncertainties in the" is the correct wording.

Done

### Tom Ferguson

5 Put a space after the \approx sign. The Guidelines say: "The ${\approx}$ macro will put in the correct spacing". But sometimes LaTeX removes this space to make the line length what it wants. However, it's clear from the Guidelines that we do want a space here. So in this case, you should force a space by putting in a "\,".

The Guidelines seem clear although they don't clarify what the "correct spacing" is. Could check with Guidelines author.

12 "The SM value of A_c is expected to be about 1% for LHC" reads better

Done.

70 "(called AK4 jets)." Putting "AK4" next to the 0.4 is confusing, "pileup per particle indentification algorithm" no need to capitalize the name. You also never use PUPPI in the paper again, so no need to define the acronym.

The source reference has both forms ("pileup.." and Pileup..) but as long as CMS is consistent.

103 "satisfy either the condition" makes what you mean clearer.

Done.

127 "have no t or W tag" reads better.

Done.

135 "on the smallest value of the X^2 variable" it's not the smallest variable, it's the smallest value of a variable.

Done.

166 Give the standard references for the JEC and JER.

Done.

Table 1 caption: We consider "leading" to be jargon. Write: "lepton and the highest p_T jet in", "both the statistical and systematic"

Done.

239 "next-to-NLO (NNLO)" you have this term enough times to warrant its own acronym.

Done.

254 "at NNLO"

Done.

Fig. 3 caption: "including NNLO QCD"

Done.

Fig. 4 caption: "The +-1 standard deviation (sigma)" where "sigma" should be the Greek letter sigma, "uncertainties in the"

Done. $\sigma$

275 "leptonically decaying" no hyphen, see Guidelines.

Done.

### Kevin Stenson

KS1) Section 2, third paragraph: explanation of the top tagging method is well described in the quoted paper(Z’ search) . Maybe a sentence could be added saying “Additional information on the tagging could be found in... “

Action: authors agree - but careful about W tagger.

Added: "Specialized techniques use AK8 jets and jet substructure information~\cite{CMS-PAS-JME-18-002}, including soft-drop clustering''~\cite{Larkoski:2014wba} and N-subjettiness''~\cite{Thaler:2010tr}, to identify the hadronic decay of boosted top quarks, following the techniques detailed in~\cite{Sirunyan_2019}. Two exclusive categories are considered: hadronically decaying top quarks ($\ttag$) in which the three partons are merged into a single AK8 jet, and hadronically decaying \PW bosons ($\Wtag$) in which the two partons from the \PW boson are merged into a single AK8 jet, but the bottom quark is reconstructed as a separate AK4 jet."

KS2) Table 1, caption: the sentence explaining the difference between muons and electrons should be added in the paper, not be just in the caption. Can be e.g. added in the first place the table is mentioned. Authors: agree

Done

KS3) Line 182, top pt reweighting: first time that it was mentioned? Authors: this was added in the part of the corrections. Kevin: Agreed.

No change

KS4) Line 248: Mention that the efficiency is taken from MC Authors: the efficiency and acceptance are joined together, and part of the former includes also the SFs, so it’s corrected by data. It might be better to avoid this . Kevin: Agreed.

No change

### Francesca Cavallari

FC1) Not clear the flow on how to cluster the jets of AK8 and AK4, in particular whether there is some separation. Authors: all particles are clustered in AK4 and AK8 simultaneously. Action: authors will add a sentence clarifying what you do to disambiguate cases when an ak4 and an ak8 are in the same top quark.

Added and reworded to define Delta R first: "The large-radius (AK8) jets are built using a distance parameter of 0.8 and the Pileup Per Particle Identification (PUPPI) algorithm~\cite{Bertolini:2014bba,CMS:2020ebo}. AK4 jets are removed if $\Delta R = \sqrt{\smash[b]{(\Delta\eta)^2+(\Delta\phi)^2}} <0.8$ from the closest AK8 jet, where $\phi$ is the azimuthal angle. The total jet \ptvec is given by the sum of the \ptvec of its constituents. If a lepton is found within $\Delta R < 0.4$ of an AK4 jet or $< 0.8$ of an AK8 jet, its four-momentum is subtracted from that jet~\cite{Sirunyan_2019}."

FC2) Line136 “The difference between the... and the true top quark mass” You say that you use the top quark mass for the hypothesis: what do you use for the top quark mass as central value? Authors: we removed true because it is loaded. Tom/Francesca: does not fully represent the average, does represent the monte carlo truth. Action: better to add a sentence, like “reconstructed top quark mass determined for each category in simulation with the procedure in Ref.[21]”

DONE

FC3) Charge mis-id: there is no uncertainty on the charge asymmetry. Authors : the mis-id is very small, since it integrates over the rapidity. Action: add a sentence in the reconstruction that the charge is measured as... , and in the systematics say that it is negligible.

Added after the systematics of reco, HLT and ID: "The uncertainty associated with the sign of the electric charge of the leptons is negligible.

We could not find any top paper that describes the charge measurement or takes the charge mis-id into account. For instance, TOP-12-038, the single top quark charge asymmetry, found the mis-id to be negligible at 7-8TeV and charge and mis-id was not even mentioned in the 13TeV paper. The asymmetry in the single top is measured more precisely than ours. We prefer to just mention it in the systematics and not try to dwell into the measurement of the charge any further in Section 2.

### Brian Winer

BW1) Lines 53-54: there is no reference about the sum of the uncertainties mentioned here - is there a way to add it? And if not, add a description. Authors: there is no reference anywhere, this is what is usually done by CMS. Action: check whether there is a new reference.

Checked recent papers and everybody seems to be doing the same with no reference

BW2) Lines 188-189: the categories are first mentioned here, and they are actually explained afterwards, around lines 211-213. Propose to add a couple of words to avoid the reader to scroll backwards and then forward. Authors: we can add ”that will be described below” or something in this spirit. Action: add ”that will be described below” or something in this spirit.

Done

BW3) Lines 228-229: the fit does not return rneg and rpos, but then it says it returns rneg and AC Authors: this is a new approach that rewrites the likelihood to include in the fit the error taking into account correlations: Action: rewrite to clarify better what is done.

Changed to say we fit for rneg and Ac directly and actually removed the equations as they were adding unnecessary complexity and they can be worked out by anybody (they are just arithmetic from the Ac definition)

"Combinations of subsets of these channels are also possible and allow us to obtain results for the two mass regions separately. In all cases, the unfolding performs a multi-dimensional maximum likelihood fit of the simulation to the observed data and returns two measured parameters: \rneg, the signal strength that scales the contribution of events with $\deltay<0$, and \ACfid, the top quark charge asymmetry measured in the fiducial phase space. By fitting for \ACfid directly, we insure the proper handling of correlations between different sources of uncertainty."

The corresponding description for the full phase space has been modified accordingly: "The unfolded charge asymmetry at parton level \AC is obtained after correcting for the product of the acceptance determined at generator level times the event selection efficiency ($\acc$). Specifically, the number of unfolded signal events in each channel is divided by the corresponding $\acc$ to correct from the fiducial phase space of that channel to the full phase space, which is common to all 12 channels. The uncertainty in the acceptance arising from theoretical sources in the \ttbar generation is several orders of magnitude smaller than the dominant systematic uncertainty and therefore neglected."

BW4) 243-244 How about moving this summary statement later after you have directed the reader to the table with results? You say that AC is consistent, but you don’t show the numbers. The sense is that the reader look at the number and see these are consistent. Authors: this was not clear - it’s fine to invert the order of the sentences. Action: invert the order of the sentences.

Changed to “Table~\ref{tab:results} and \figrefs{fig:fiducial} (left) summarize the \ACfid…

### Tom Ferguson

Comments are in the introduction, will be followed up during the reading of that part. Authors: can we go back to AC with capital C everywhere? It is consistent with all previous results. Consensus: yes!

Done

Comments on tables and figures Figure 1

- Remove l+jets from all figures, there is no other physics channel.

Done

- try and increase the font size on the x axis.

Done

- remove the / 1 in the jets and /2 from DeltaY

Done

- Data/MC separate better the number in the bottom two plots

Done

- Not necessarily all points need to be in the bottom panel if they are off, but authors’ preference is to keep them if possible.

Done

- Check the Events/45 GeV number, as well as the Events/0.1, doesn’t look right.

Done

Table 1 - too many blank lines. Proposal: delete one blank line between the Total and the Data

Done

- remember to put the comment on the leptons in the text.

Done

- remove “the” in the systematic components.

Done

- Comment about the electrons and the years: should we explain the order? Authors would prefer not.

No change

- Spell out DY → Drell-Yan? We were trying to be consistent - Action: keep it this way.

No change

Figure 2

- Whatever is done for Fig 1 and applies here should be repeated.

Done

- Here you put pre- and post-fit which is a bit jargon. Add it in the caption : before (prefit) and after(post-fit) in the caption; is already there.

No change

- DeltaY / 2 : should go here as well; better to remove it from Figure 1 as well, since it’s not useful.

No change

Table 2 - Add small blank lines so they don’t run into each other (small vertical space) Use a smaller font for all +- separate uncertainty.

Done

- Ac - Theory: spread apart a bit value and uncertainty.

Done

- All % can be removed? Proposal: add a multicolumn that includes “AC(%)” and below”measured stat syst.... theory”. If it doesn’t work, remove it and put it in the caption,

Done

Figure 3

- Dashes are different between 750-900 and 750 - 900

Done

- Predicted shows a band -> is this standard? Results is in the table, does it matter?

No change

- Move measured and predicted bottom, so they look the same. - Why keep a fixed range? A. it was a CWR comment Action: re-adapt it, by adding space on top so the legend looks better, i.e. like the current left plot.

Done

- Drop the caps in the legend data points

Not sure what this refers to

- Subscript c here seems capital, should be lowercase? A. It is harder to see if lowercase. Also it was centrally decided to keep it everywhere capitalised for consistency.

No change

- Do we want % in the figure? Consensus is that it is better this way.

No change

- Lumi and energies are small, should be increased a bit

Done

Figure 4:

- Lumi and energies are small and bolded

Done

- Number on the left should be removed

Done - W+jets has spaces around the ( W + jets), (mu+jets) and (e+jets) don’t have them. Add spaces.

Done

- impact on AC , C is very off.

Done

- grey bars go beyond the box, fix if possible.

Removed

Title No change

No change

Abstract No change - It is not clear which number is given in the abstract. Should we assume it’s full phase space? Yes because it’s the only one specified so far. - We usually say whether it adds statistical and systematic components. Should we do it? Consensus is that it’s obvious and no action is needed.

No change

Introduction - Lines 1-3: qqbar annihilation appears and it is a bit strange. Maybe one wants to mention that it is when qqbar they involved in the collision. Suggestion: one could mention that there are two production mechanisms gg and qqbar, and say the first is dominating. Action: the authors will make a different proposal and explain it in a more straightforward say.

- Line 10: “broader” -> what is the purpose of this description? Authors: this is setting up the next sentence and equation 1. Action: none.

- Lines 5-10 Suggestion: when talk about the high mass region specify that those would be sensitive to AFB, so that it does not have a prominent role that might be misleading. At the beginning start talking about gluon fusion, then add qq, and say we are still sensitive to qq annihilation. Action: Authors will propose how to clarify that all asymmetry should come from that.

Changed to: "The vast majority of top quarks produced at hadron colliders are from \ttbar pairs that originate from a $\cPg\ttbar$ vertex via the strong interaction, where g is a gluon~\cite{Czakon:2013goa,Catani:2019hip}. At the LHC energies, about 90\% of the \ttbar production originates from gluon fusion, while the rest is from \qqbar annihilation. At leading order, the standard model (SM) predicts that \ttbar production from \qqbar annihilation is forward-backward symmetric. However, higher-order SM effects result in a small (${\approx}6.6\%$) positive forward-backward asymmetry \AFB, such that the top quark (antiquark) is preferentially emitted in the direction of the incoming quark (antiquark)~\cite{Czakon_2015}. There is no asymmetry in the gluon fusion \ttbar production that dominates at the LHC, but because valence quarks carry, on average, larger momentum than antiquarks (from the sea), the rapidity distribution of top quarks at the LHC is expected to be broader than that of top antiquarks~\cite{Czakon_2018,Czakon_2016}. The \ttbar charge asymmetry is defined as $$\AC = \frac{N(\deltay>0) - N(\deltay<0)}{N(\deltay>0) + N(\deltay<0)}, \label{eq:AC}$$ where $\deltay = |y_\cPqt|-|y_\cPaqt|$ is the difference between the absolute value of the top quark and antiquark rapidities and $N$ is the number of events. The value of \AC is expected to be about 1\% in the SM for LHC center-of-mass energies~\cite{Czakon_2018}."

- Line 29 uniquely restrict: No other measurements have restricted this phase space? Authors: This is their wording. None of the individual paper does it the same way.

No change

- Line 31 this paper → this Letter

Done

- Line 31 there was some controversy on “first measurement” : it’s the first at 13 TeV, there was a point on whether the “and” includes thot.

No change

- Line 39 selections → selection requirements

Done

- Line 41 multijet backgrounds → QCD multijet backgrounds

Done

Summary - Line 262 In the rest you used “highly lorentz-boosted jets” here highly boosted top quarks. Consensus that it’s fine.

No change

- Line 261 Past tense in 261 and you go back . Action: change to present.

Done

- Line 269 QCD : no acronyms in the summary usually → Consensus that it’s fine like this.

No change

- Line 274 very high → highly

Done

- L278 remove space before the “.”

Done

- Line 279 Conclusions should have a stronger statement on the importance of this measurement, e.g. that this shows that top quarks can be measured precisely in highly boosted topologies... Remove the “opening a new era” part, since it was already opened with the measurement. - Suggestion to swap second and third paragraph. Leave the last sentence.

Done

### Kevin Stenson

#### Type B

Title: “single-lepton channel” could refer to either a single quark or the ttbar pair. Perhaps “Measurement of the ttbar charge asymmetry in events with highly Lorentz-boosted top quarks and a single lepton at 13 TeV” or “Measurement of the ttbar charge asymmetry in highly Lorentz-boosted top quarks at 13 TeV with each event containing one lepton and multiple jets”

Changed to "Measurement of the ttbar charge asymmetry in events with highly Lorentz-boosted top quarks and a single lepton at 13 TeV"

Abstract: I think the last sentence should be removed. The word differential is never mentioned again, the only distributions I see are simply positive and negative Delta Y and it is stretching things to suggest two bins constitutes a differential measurement. Finally, there is nothing mentioned about it in the summary. If you want to include it, then you definitely need to specify the x and y axes of the differential distribution.

Changed to "The result is also presented for two invariant mass ranges..."

L3-L9 and L29-L35: Is there some reason to spend so much time discussing the results for qqbar annihilation and A_FB measurements at the Tevatron. If the idea is that measuring A_FB and A_c probes the same new physics models, then this needs to be made clear. Is it true that new physics should produce deviations in both A_FB and A_c? If not, then I do not see the point of spending so much text on A_FB and the Tevatron. One can simply remove all of L29-L35 and shorten L3-L9 by not mentioning qqbar annihilation or A_FB.

Shortened L3-L9 by removing the sentence "A fundamental difference between \ttbar production in the Tevatron proton-antiproton collisions and the LHC proton-proton ($\Pp\Pp$) collisions is that the former is dominated by \qqbar annihilation and the latter by gluon fusion". Also removed L 29-35 and merged the text into one paragraph

Section 2: I note that Ref 28 is mentioned for some of the jet requirements. I wonder whether the following sentences from that publication also apply to this analysis. If so, I suggest adding them. The PF candidates are clustered into jets using the FASTJET software package [39]. Charged hadrons that are not associated with the PV in the event are excluded from the jet clustering procedure via charged hadron subtraction (CHS) [36].

Ref 28 is the published ttbar resonance search. The Ac measurement is based on that paper but we added 2017 and 2018 data. In L 51 Ref 28 is used to reference the "dedicated jet and lepton selections". Moved the reference after that to make it clear. It refers to the description in paragraph 2 of Section 4 of Ref 28.

L82: Why do you need Ref 28 for this sentence?

This sentence refers again to our specialized treatment of leptons closed to jets, namely, that the 4-momentum of the lepton is removed from the jet. The same paragraph we mentioned before.

Section 2: I also note that in Ref 28, there is much more text devoted to explaining the t tagging algorithm and the inputs. Is is correct that the t tagging in this paper is the same as in Ref 28? If so, perhaps you can explicitly direct the reader to Ref 28 for more information. In Ref 28, there is nearly two full pages (Section 3) which gets reduced to about 3/4 of a page in this paper (last two paragraphs of Section 2).

We use the same top tagger but the W tagger is a new addition. We were asked to add the primary references for CMS top and W tagging rather than Ref. 28. We could also add Ref 28 if you prefer. Maybe after "Specialized techniques?"

L95,96: I am surprised to see AK4 and AK8 jets all the way out to |eta|<2.5. Have other analyses used jets out this far? I thought most of the time only jets with |eta|<2.4 were used. I guess for 2017-18 with the new pixel detector extending coverage, this might be OK but I am not sure about 2016 data

This is a typo, our |eta| goes up to 2.4

L113,L114,L117: Need to clarify exactly what is meant by “all AK4 jets” and “nearest AK4 jets” and “one of the AK4 jets". Is this all AK4 jets in the event? All AK4 jets associated with the primary vertex? I see the “nearest AK4 jets” has a pT and eta requirement but what about the “all AK4 jets”? You may want to define this AK4 jets collection around L77-79.

For the 2D cut it is the closest Ak4 jet to the lepton (Ak4 are all only from the same vertex). b-tag can be any AK4 in the event (irrespective of distance to lepton). Changed to: To reduce the background from QCD multijet events, we apply a two-dimensional (2D) selection that requires leptons to satisfy the condition $\DRmin > 0.4$ or $\ptrellj > 25\GeV$, where \DRmin is the angular separation between the lepton and the closest AK4 jet, and \ptrellj is the transverse momentum of the lepton with respect to the axis of the nearest AK4 jet~\cite{Sirunyan_2019}.

L115-116: Why the difference between muon and electron channels here? Why not ptmiss+pTe > 150 GeV? You may need to add this information to the paper. You could add another sentence like “The larger value of the e+jets \ptmiss requirement efficiently reduces the larger QCD multijet background in this channel and obviates the need for a separate requirement on ptmiss+ptl.” This is the information I got from Ref 28. However, now when I look at the final result, I see that the electron channel has a negligible QCD multijet, while the muon channel has a substantial QCD multijet background. Is this cut responsible for the much lower e+jets yield compared to the mu+jets?

Added the sentence and yes, this cut is responsible for the negligible QCD background in the electron channel. The background in the muon channels ends up being larger but it can be modelled, which was not the case in the electron channel.

Section 4: What happens if a single AK8 jet passes both the t and W tagging requirements? Or are these mutually exclusive?

Line 89 states that they are "two exclusive categories"

What happens in events with two AK8 jets, one that passes the t tag and one that passes the W tag? Please make sure the paper has information that the reader can use to answer these questions. These questions occurred to me while reading L130-134. Most of the questions are answered by the text in L142-145. I suggest moving L142-145 to the beginning of Section 4 (perhaps after the first sentence of Section 4).

Done. Small rewording to make it clear what the final signal candidate sample selection is.

Section 4: I think the reader would appreciate knowing what fraction of the events (either total or signal) are in each topology. This could be inserted around L147. L146-147: This sentence indicates that 70% of the boosted topology candidates have the correct jet assignments. To be clear, the 70% only applies to events where there is a t tagged AK8 jet? This does not seem to be very impressive. This just means that you have selected the correct AK8 jet to go with a top quark and the correct AK4 jet to go with the lepton to make the other top quark. I think it would be better to give the fractions for each of the 3 topologies (or the average value for all events). Specifying just the value for one of the three topologies makes the reader question what the other efficiencies are.

The % depends on the mass but is mostly resolved. The main reason for having the boosted topology is to improve the correct assignments. It is debatable that ~70-80% is not impressive, it is actually worse for the resolved, one item in our list of possible improvements for the next iteration. We can remove the sentence about the 70% but prefer not to give numbers. We could say that the majority of the events is still resolved if needed, but we do not want to get into quantitative details.

L146: Is this chi^2 variable the same as in L139? It needs to be made clear. In L139 the chi^2 variable is just mentioned as a way of arbitrating but then it is used in L146. Maybe it would be better to have a standalone sentence describing the variable. L138-140 could be rewritten as:

We did not get your suggestion, but changed to the following: For each event, one $\ttbar$ hypothesis is selected as the one with the smallest \chisq variable that minimizes the difference between the reconstructed $\tl$ and $\thad$ masses and the true top quark mass determined from simulation~\cite{Sirunyan_2019}. Because background processes typically result in large values of \chisq, only events with \chisq$<30$ are retained. Finally, our signal candidate sample is defined as those events with \ttbar invariant mass greater than $750\GeV$.

Figure 1: It seems like a semi-log plot of the invariant mass would be more informative.

DONE

Table 1: I am surprised that the e+jets is only 1/4 the size of the mu+jets. Why is that? Is it the ptmiss cut? I think the reader is owed an explanation.

It is that but also, and principally, the higher lepton pT cut (80 vs 55GEV) and the leading jet pT (185 vs 150 GeV). Added to the caption: The higher pt thresholds on the lepton and leading jet in the electron channel result in significantly reduced signal acceptance compared to the muon channel.

Figure 2: For the top plots, the “pre-fit” and “post-fit” text interferes with the mass range above. Suggest moving this text down for all 4 plots. Figure 2: For the top plot mass range, the first “GeV” is unnecessary and should be removed. Figure 2: You should change “ele+jets” to “e+jets” to match the rest of the paper. Figure 2: The ratio seems to carry more information than the actual distribution. I think it would be helpful to make the ratio plot slightly larger vertically and to zoom in the y-scale to something like 0.7-1.3.

Done

Figure 3: There seems to be a lot of white space. I suggest changing the y-axis range to be -0.05 to +0.05. Also, you should make the data points larger and the error bars thicker to enhance visibility. It is not like there is any clutter.

Done

L280-282: I do not understand the logic of this sentence. Why is it that having more reactions come from valence quarks means the measurement is especially sensitive to BSM physics?

The majority of the BSM models that predict a larger than SM Ac are qqbar initiated and the PDF for valence quarks increases (compared to gluons) at larger x. This is a restatement of what we said in L17.

L282-284: I do not think this sentence is necessary or helpful. It actually dilutes the message of the current measurement being useful.

We could go either way. Right now the measurement is very statistically limited and this comment tries to be forward-looking to the next iteration.

#### Type A

Abstract: You are allowed to change “CERN LHC” to “LHC” if you like. I am pretty sure everyone reading a top quark paper will know where the LHC is.

Done.

L2-3: I do not like “in the form of”. Suggest a first sentence of “The vast majority of top quarks produced in hadron colliders are from \ttbar pairs that originate…”

Changed to "In hadron collisions, top quarks are produced dominantly in pairs..".

L6: Remove “the” in “that the \ttbar”

Done.

L60: Remove “, respectively”. First, there is nothing in the sentence to pair the terms with. If you go further pack and try to pair the terms with the previous sentences, the ordering does not match. So, just remove the term “,respectively” and assume the reader is smart enough to match the terms or else change the end of the sentence to “and “resolved”, for the high, intermediate, and low \pt regions, respectively.”

Done. (Latter suggestion.).

L122: Change “All the samples” to “All samples” or “All of the samples”

Changed to "All samples".

L128: I do not like starting a sentence or section title with a symbol. How about “Reconstruction of \ttbar events”?

Currently "of the top quark pair events" but the suggestion is better...A top quark pair could be tt.

L131: “considered as candidates”

Done.

L165: Change “reweighed” to “reweighted”

Done.

L180: I think you need something like “sets” before “[53]”. There needs to be a noun of some sort.

Done although "PDFs" (and not "PDF sets") appears earlier in the sentence.

L181: Change “in [58]” to “in Ref. [58]”

Done.

L188-191: I assume that this comes about from the difference of the top quark pT distribution between data and MC. If so, you need to explain the problem. I also think you need to specify whether the central value is obtained with or without the correction.

Not clear. The authors should address this.

Table 1: Please use normal size font for the table. There are tricks to reduce spacing in tables if necessary. For example, the spacing between columns can be greatly reduced.

The font size has been increased.

Table 1: I would suggest the top row have centered labels rather than right justified.

Done.

Figure 2 caption: Can remove “As can be observed,”

Done...but not the only "as can be observed".

L244-5: Writing “which together summarize” means that one needs both the table and figure in order to summarize the values. In fact, they have the same information. Therefore, just write “which summarize”.

Done.

Table 2: Please use normal size font for the table. There are tricks to reduce spacing in tables if necessary. For example, the spacing between columns can be greatly reduced.

Done.

Table 2: I think one can reduce the amount of vertical space in this table.

Done. And the font size has been increased

L248-9: I think we need to mention that this comes from MC. Perhaps add “, both obtained from MC simulation” to the end of the sentence.

Both refers to acceptance and efficiency? One would think that the efficiency also comes from simulation (and alpha epsilon may be determined in one go). Leave for the authors.?

L268: Change “and result” to “and can result”. After all, you do not specifically require the leptons be non-isolated and you do include events where every parton corresponds to its own AK4 jet.

Done.

L322: Fix the title so ttbar is correctly typeset

Done.Used CMS \ttbar alias.

L328-9: Fix the title

Done... $\Pp\PAp\rightarrow\ttbar$.

L339: Remove the end page number

Done.

Ref 13 and Ref 19 are duplicates. Consolidate.

Done: Kept the first..The reference appears twice in the list of BSM models/particles.

L418: Fix the last author (should be Emanuele Re, abbreviated as E. Re)

Done.

L446: Remove the “no. 9” (remove the “number” entry in bibTex)

Done.

### Tom Ferguson

Title "channel in proton-proton collisions at" Without this, the "13 TeV" reads strangely. Not clear what it refers to.

LE: Currently has "pp collisions". Leave for the FR.

Abst. "top quark pair (tt) events" this is our standard wording to define "tt".

The \ttbar symbol should be familiar to everyone reading the paper. If it must be defined in the standard way, then it would be necessary to write out "top quark-antiquark pair events" in the Abstract and Summary by the Guidelines (fewer than 2x appearances). And \qqbar would also need to be explained in the Introduction.

"top quark charge asymmetry of" reads better

Done.

How about writing it as: "(6.9 ^+ 6.5 _- 6.9) x 10^-3" this is much easier to parse. I don't mind one zero after the decimal point, but two is hard to read and then you have to individually look at each of the uncertainties and make sure they have two but not three zeroes after the decimal point. Writing it in scientific notation makes everything much clearer. It's not called scientific notation for nothing.

Somewhat done. The authors prefer (0.69 ^+0.65 _-0.69)% for consistency with previous publications..

"in quantum chromodynamic perturbation" we define "QCD" the first time it is used. See Guidelines. Note: no "s" in "chromodynamic" since it is an adjective here.

Done. The Guidelines discourage the defining of symbols in the Abstract where the symbol is not used two or more times (in the Abstract).

"Differential distributions of the charge asymmetry for two tt invariant mass ranges, 750--900 and > 900 GeV," this is our standard way of writing ranges and of only putting the units on the second value in a list. Use an en dash for "750--900"

Done.

2 "primarily as top quark pairs (tt) that" be consistent with the abstract. Also, "tt pairs" could be misinterpreted as "tt tt".

Changed to "top quark-antiquark pairs (\ttbar)" but \ttbar (without definition) seems more natural here (and in the Abstract and Summary).

4 "production from proton-antiproton collisions at the Tevatron and proton-proton collisions at the LHC is" reads better.

Done.

6 "predicts that tt production from qq annihilation is" reads better

Done. \ttbar. Note that the \qqbar symbol is not defin.

7 Put a small space after the "\approx" sign.

This would seem to be contrary to the Guidelines.

14 "top quark and antiquark" you don't have to repeat "top".

Done.

15 "rapidities, (comma) and" for clarity.

A comma seems to introduce an unnecessary pause.

16 "SM at the LHC" just for clarity. Should you give the center-of-mass energy or is the asymmetry fairly independent of this?

Add "at LHC center-of-mass energies" but the authors should comment.

30 The phrase "by the SM at the time" seems to imply that the predictions changed. I don't think you mean this. I would delete "at the time" not needed.

Agree but this paragraph is now gone?!

37 I don't think you need the second "A_c =" it's clear without it. Again, what about writing these values as "(5 + 7 (stat) - 6 (syst)) x 10^-3" ? This is easier to read and understand.

Dropped the second A_c but the extraction of the 10^-3 seems less appealing in this in-line form?

41 "This paper presents the first measurement of the tt charge asymmetry from pp collisions at sqrt(s) = 13 TeV" reads better

Changed to "This paper presents the first measurement of the \ttbar charge asymmetry that used pp collision data at...The authors should clarify what makes the measurement a "first" one. Is it the AND of 13 TeV and high-mass optimization, or OR?

44 "with one W boson decaying leptonically"

Done.

Done.

49 "allow us" since "selections" is plural.

Done.

64 "calorimeter, (comma) and"

Done...This does appear on the detector description he twiki page without the serial comma.

79 "Large-radius (hyphen) jets (AK8) are built using a distance parameter of 0.8 and the pileup per particle identification (PUPPI)"

Done but left as "Pileup per Particle Identification".

81 "or < 0.8 of" you don't need the second "Delta R"

Done.

87 "including soft-drop" hyphen, delete "the"

Done.

88 "and "N-subjettiness" delete "the"

Done.

89 "in which the jets coming from the fragmentation of the three partons from the top quark decay are" for clarity

Leave in the post-CWR form. The sentence is very long and one would like to believe that the "three partons" are understood at this point.

91 "in which the jets due to the hadronization of the two partons from the W boson decay are", "but the fragmenting bottom quark is"

See previous comment. Note that the resulting sentence would have both "hadronization" and "fragmenting" in it, and also be very long.

94 "DEEPJET [34]" delete the second "algorithm"

Removed the second "algorithm" by writing "(DEEPJET [44])".

95 "applied to each AK4 jet j with" "j" should be in italics. "The t and W tagging" you don't need two "tagging"s.

Done. ($j$)

102 "single-electron" hyphen

Done.

108 "two jets j_1 and j_2 with" you need to define the symbols

Done: "two jets, $j_1$ and $j_2$, with".

116 "from W + jets" delete "the"

The definite article seems appropriate here.

117 "jets must be" reads better

Done.

118 "samples of the"

"samples for the" reads better to me (LS).

133 "or t_h" you don't need the second "the", "a t nor W tag" you don't need the second "a" or the first "tag".

The repeated "the" , "tag", and "a" don't take much space and "the t_l or t_h" seem too terse.

134 "and t_h"

See previous comment.

135 "b-tagged jet" hyphen, see Guidelines.

Done.

137 "to equal the" why use "match" when you mean "equal"

Changed to "constraining...to the W boson mass"?

142 "into the three topologies mentioned earlier" should reference your earlier discussion.

This sentence is no longer present.

147 "comparison of several distributions between the data and the MC predictions" I think "MC" is better than "SM".

"MC" is better than "SM" in this situation (a MC has a specific order whereas SM could mean all orders). But the suggestion is not clear on what is being compared and if the "distributions are within a histogram or the Delta|y|, M_tt,.. distributions. Write as "Figure 1 shows comparison between data and MC simulation for kinematic distributions based on events in the candidate sample.

148 "in the candidate sample. (period) I don't like "our". Also, the current sentence is too long and clumsy. Break it up. "The boosted nature of the top quarks becomes evident in these events in which the M_tt range extends to multi-TeV values, two and three AK4 jets from the collimated top quark decay products are reconstructed, and the leptons are closer to the nearest jet axis than the jet size."

Done.

152 "between the distributions from data and the MC predictions is"

Changed to "between data and prediction" but feel that "distributions" idea has been made and a simple judgment is appropriate for the last sentence.

154 "normalization and shape" you don't need the second "the"

The second "the" stresses that "shape" is a distinct concept from "normalization" (although uncertainties often fall in both categories).

160 "[46], (comma) respectively,

Done.

161 "across the three years."

Done.

162 "There is an additional 1.6% systematic uncertainty in the combined" reads better.

163 "normalization and shape of the MC distributions"

Done.

168 "high-level" hyphen

Done

169 "(Reco), (comma) and"

171 "A constant systematic uncertainty in the efficiency" "flat" is considered jargon.

172 "QCD background is used, which"

Not sure what the full suggestion is but the use of ", which" is questionable. Change to ", and".

175 "in t and W tagging" you don't need both "tagging"s

Done.

178 "incorrectly identify (mistag)" "mistag" is considered jargon and must be defined.

Done.

Fig. 1 caption: "Comparison between the distributions from data (points) and the MC predictions (colored histograms) for signal candidate events in the", "(described in Section 6): Delta |y|", "The vertical bars on the points show the statistical uncertainty in the data. The shaded bands represent the total uncertainty in the MC predictions (described in Section 5). The lower panels give the ratio of the data to the sum of the MC predictions."

Changed the first sentence to "Comparison between data and MC simulation for.. The legend explain the symbols, so it does not seem necessary to repeat the information in the caption.

179 "MC simulations"

Done.

180 "The uncertainty from the choice of PDF is estimated by taking the difference between using versions 3.0 and 3.1 of the NNPDF sets, according"

Done.

181 "in Ref. [58]." See Guidelines on how you refer to references.

Done.

186 "Uncertainties related to the modeling of the initial-"

Done.

187 "(ISR and FSR) in the parton shower are determined by varying" so you don't have "taken into account" in two consecutive sentences.

Done.

191 "without the correction to the simulated samples used to make the MC top quark p_T distribution agree with data [59]." you need to say what the "correction" is.

The sentence explains the correction?!

198 "to the finite MC" I don't like "limited". It's unlimited - you could make as large a sample as you want, but it would still be finite.

If we don't want to admit to MC sample size limitations in a public pager, maybe just "due to the MC sample size"?

219 "The total likelihood is given" it's not a "result"

Done.

222 "advantage that the background contributions are" shorter

Done.

Table 1 caption: "The signal event yields in data and the MC predictions after the likelihood fit for each" Delete "used in the analysis" obvious. "2016, 2017, and 2018, and two tt invariant mass ranges). Put the years in numerical order. Delete "for events that pass the signal selection" not needed if you say "signal" at the beginning. "The uncertainties in the MC predictions include both the statistical and systematic components."

The authors feel strongly about the backward ordering of the years. Agree that the selection qualification is no longer necessary. The last suggestion does improve on the text.

232 "Taking various combinations of the channels gives separate results for the two invariant mass ranges."

Not sure that this improves on the text. But maybe I am missing the significance of "combinations of various channels.". Statistics?.

234 "to the observed"

Done.

240 Delete "and" we don't add this in a bullet list.

Done.

242 "given tt invariant mass region."

Done. "given \ttbar invariant mass region".

Fig. 2 caption: "The signal event yields for Delta|y| < 0 and > 0 from data (points) and the MC predictions (colored histograms) before (left) and after (right) the likelihood fits for each of the analysis channels. The upper and lower plots show the yields for 750 < M_tt < 900 GeV and M_tt > 900 GeV, respectively. The vertical bars on the points represent the statistical uncertainties in the data, and the shaded bands give the combined MC statistical and systematic uncertainties. The lower panels display the ratio of the data yields to the sum of the MC predictions." Delete the last sentence. You already say this in the text, and this is not something you put in a figure caption.

Mostly agree but keep the "comparison" lead to parallel the Fig. 1 caption.

243 Delete "found to be" not needed

Done.

245 "summarize the A_c^fid and A_c values for the complete signal sample and for two tt invariant mass regions, along with their"

"together" is not appropriate and the last phrase of the sentence seems to be a run-on. But "full signal" or "combined signal" sounds better than "complete signal".

246 "uncertainties. The theoretical predictions, which include next-to-NLO QCD and NLO EW corrections from Ref. [4] and are obtained by settting the observed quantities to their expected values ("Asimov data"), are also given, along with their uncertainties."

Done. Maybe should define NNLO? % ENDCOLOR%

Table 2 caption: "charge asymmetry values", "fiducial phase space A_c^fid (top)", "full phase space A_c (bottom) are shown for the total sample and the two M_tt invariant mass ranges, along with the corresponding SM predictions A_c (theory) and their uncertainties. The statistical (Stat) and systematic (Syst) uncertanties in the data, the MC statistical uncertainty (MC stat), and the total uncertainty in the measured values (Total) are also shown."

Don't follow the first sentence in the suggestion?

249 "determined at generator" so you don't have two "measured"s in the sentence.

Done.

252 Delete "In this case," not needed.

Done...Change the last phrase to ", which is common to all 12 Channels".

Eq. (4) Put a comma after the equation.

Done.

253 "where N_gen is the number of generator events in each Delta |y| region, and alpha epsilon^pos and alpha epsilon^neg are the corresponding acceptance times efficiency values. This formula allows us", "strength" (singular)

Done.

255 Delete "and its uncertainty" not needed and doesn't read well. All values come with their uncertainty, you don't have to say this.

Done.

256 "than the dominant systematic" the uncertainty in the acceptance is also a systematic uncertainty, so you must delineate what you mean.

Done.

Fig. 3 caption: "Measured A_c^fid (left) and A_c (right) values (points) for the complete signal sample and for two M_tt ranges, combining the muon and electron samples. The inner tick marks on the points indicate the statistical uncertainty in the data, and the outer tick marks the total uncertainty. The theoretical predictions, including next-to-NLO QCD and NLO EW corrections from Ref. [4], are shown by the bands, with the height of a band representing the uncertainty in the prediction." Delete the last sentence. This should be in the text but not in the figure caption.

"complete" and "combining flavor channels" mean the same thing? Agree that observations about histograms should generally go in the main body and not the caption.

Fig. 4 caption: "The +/- 1 standard deviation (sigma) impacts of the dominant nuisance parameters", "uncertainties in the A_c" you haven't used "inclusive" previously, don't start now. "The MC statistical" never start a sentence with an acronym. See Guidelines.

Done.

261 "Figure 4 shows the +/- 1 standard deviation impacts of the dominant systematic uncertainties in the A_c measurement for the complete signal sample."

Done but used "full signal sample".

270 "fit and then extrapolated from the fiducial to the full phase space."

The seems a bit detailed for the Summary.

271 "events with tt invariant masses satisfying M_tt > 750 GeV, (comma) corrected to the full phase space, (comma) is 0.0069 +0.0065 -0.0069, where the uncertainty includes both the statistical and systematic components."

Done.

276 "top quark" no hyphen. But you just said the first half of this sentence on line 264. There is no reason to repeat it. So write: "This is the first measurement to use a binned maximum likelihood unfolding technique to measure A_c directly at the parton level and in the full phase space."

Done.

278 "It is also the result that"

Assuming you meant "It is also the first result that" but this is re-raising the "first measurement" question..

279 "for the hadronically and leptonically decaying top quarks" delete the second "the" and the hyphen, and make "quarks" plural.

Done.

280 "at both the trigger and offline stages."

Done.

284 "for the upcoming LHC run and the future HL-LHC." We don't use terms like "Run 3" without defining them first. See Guidelines.

%Blue% Agree that "Run 3" is a problem but any reference to time (upcoming, future) will eventually not be accurate. Use "current" for Run3. Maybe the last sentence could be deleted?

322 The second "t" in "tt" should be in roman font.

Done.

328 Only the first word of the title should be capitalized. It should say ppbar --> ttbar" using overlines.

Done.

335 The ttbar should be in roman font.

Done..

339 Delete "-242"

Done.

References 16 and 20 are identical. Same for 13 and 19.

Also Kevin Stinson's comment. Done.

349, 358 Should be "CDF Collaboration"

Done.

352 Should be "D0 Collaboration"

No longer referenced?.

364 Should be "Collaborations"

Done. (Use author field and drop the collaboration field to get the plural form.)

389 Only capitalize the first word in the title.

Done..

402 "drop" lower case.

Done.

432 Put a space between "13 TeV"

Done.

439 "41.9 fb^-1"

Done.

446 Delete ", no. 9,"

Done..

136 Shouldn't you say why the b tagging information is not used? Reads strangely without a reason.

The presence of a b-tag in each event is required to suppress W background (line 117). The information of which jet is or is not b-tagged is not used in the event reconstruction. This is the same as all previous Z' to ttbar boosted paper and we never entered into detail of why or why not and prefer to do the same here

146 Why give the percentage for the boosted topology and not the others? Also, why not give the percentage of signal candidate events in each topology?

The percentage depends on the mass very strongly. We added this because it was requested but given all the comments received we have now removed it as we do not want to go into the detail that would be required to give a meaningful quantitative answer.

147 You need to define "Other" before Fig. 1 is shown. Don't use "Others".

Added to the caption of Fig. 1 and 2: Other'' corresponds to the combined contribution of ST, DY and QCD multijet.

Fig. 1: Put the "Data" at the top of the legend list. That is our standard position. We typically read the legend down in one column and then down in the second column. So the order should be:

Data Other tt MC tot. unc. W + jets

did put Data on top

put the order as it appears

Notice that you should show the shaded band in the legend. You should stack the histograms in the order they are given in the legend from top to bottom. So they should be stacked with tt on top, then W + jets, then Other, not Others.

Increase the font size of "ell + jets" and move it below the legends in some of the white space. Put a space on each side of the + sign to match the figure caption. Same for "W + jets".

did increase font size left the position unchanged as I think it looks better there added space between the +

You have too much white space in many of the plots. You could make the legend just one column in these plots to reduce this amount of white space.

Prefer to leave it as it is as this does not work well for all plots

The top left figure needs the "ell + jets" label.

Done

The y-axis label should give the bin width: e.g., "Events / 250 GeV" with a space on each side of the "/". See Guidelines.

Done

Put a space on each side of the "/" in "Data / MC"

Done

The x-axis numerals and labels should have a much bigger font size. The font size of the top left figure should match the other figures.

Done

Decrease the y-axis range on the Data / MC plots so that the points and errors bars are more visible. Make the plot larger vertically so that the y-axis numbers don't overlap with each other.

Done

205 "The index j runs over the total number of reconstructed signal events N_reco." You need to define the symbol.

Actually, it is the number of bins not the number of events. That is why they are both set to 2. Changed to The index $i$ runs over the number of bins at generator level (\Ngen), and the index $j$ runs over the number of bins at reconstruction level (\Nreco).

208 Delete "In this analysis," not needed. What other analysis could you be describing?!

The description is meant to be generic and we prefer to leave it to make it clear to the reader that we choose to use just two bins, which is our choice and a novel method which then allows us to fit for Ac directly. .

218 Move this sentence up to line 200 where "channels" is first used.

Here again the description is meant to be generic and we prefer to leave it as is and only define the channels of our analysis at the end of the generic description of the method. .

Table 1: The numbers are too small to read. Increase their size until there is only a few spaces between each number and the table fills the page.

Done

I don't like your notation of "mu_2018" hard to read and very clunky. Rewrite as "mu (2018)" etc. changed

I would also put them in the order 2016, 2017, and 2018, not the reverse. We have consistenlty used the muon 2018-2017-2016, electron 2018-2017-2016 order because it corresponds to the importance of the channels in the result. We prefer to leave the order as it is as it is used consistenly throughtout the paper

I would put a horizontal line before "Total" and leave a blank line between "Total" and "Data". done

234 I think you need to define the signal strengths. They are unitless numbers that are usually a measured quantity divided by a predicted quantity. Is this the case here? What is the predicted quantity? Be clear.

it was defined later but moved it here: returns two signal strengths, \rpos and \rneg, that scale the contribution of the events with $\deltay>0$ and $<0$, respectively.

Fig. 2: Use most of the comments suggested for Fig. 1. "Pre-fit" and "Post-fit".

done Use "e" instead of "ele". done

Put the years in numerical order. We prefer to leave the order as it is

The x-axis labels are incredibly hard to read and messy. Why not use the symbols - and +, or "A" and "B" to designate the two regions, and explain what they mean in the caption. We prefer to leave this as we have reached this after several iterations. although crowded it is easy to understand

Table 2: The numbers are different sizes and the smaller ones are too small. Make all the numbers have the same font size by increasing the smaller ones. You have room to make the table wider and lots of blank space between some of the columns. Use this. Capitalize "Stat" and "Syst". Put a space after every + and - sign. Put a space before the +/- signs for the A_c (theory) values. Write "A_c^fid in the fiducial phase space". Delete the parentheses around "750 - 900" Think about writing all the numbers in scientific notation. I don't think this would take up much more space since you would be eliminating "0.00" and replacing it with "x 10^-3" You can remove one of the blank lines after each row of nmbers and after the A_c labels, and at least 2 blank lines between the fiducial and full phase space sections. The table takes up too much space. You are trying for PLB, which does have a page limit. done except the scientific notation, left it as it is for now

Fig. 3: Decrease the y-axis range. You have too much white space. Increase the font size of the legend. You have plenty of space for it. Use "750--900" for the x-axis labels. Remove "GeV" from the labels and put it after the "M_tt" in parentheses, like a normal unit. Put a space after the ">" signs in the x-axis labels.

done

261 This is the first mention of Fig. 4, so it cannot be shown on the previous page. Move it down below this line.

We prefer not to introduce forced locations as the publisher will take care or formatting in any case

Fig. 4: "Integrated luminosity" first word capitalized, "Electron reco" and "Muon reco" second word lower case.

done

### Francesca Cavallari

line 45 either a muon or electron → either a muon or an electron

line 49 Dedicated jet and lepton selections at the trigger and offline levels allows → Dedicated jet and lepton selections at the trigger and offline levels allow (remove the s)

I think that the discussion from line 49 to 60 could be moved in section 2 between lines 86 and 87. We think it is better to leave it in the introduction as it explains what is special about our high-boost topology in general terms but does not get into the specific object identification we do at CMS, which is what we then describe in Section 2.

I don’t understand how you make sure that there is no double counting of the objects in the AK4 and AK8 jets. Can you clarify exactly how the jet clustering procedure flow works for this analysis (what algorithm you run first and what you use to run the second one?)

We use the standard CMS jet reconstruction based on PF which ensures no overlap.

line 138-140 Finally, one tt hypothesis is selected for each event as the one with the smallest chi2 variable that minimizes the difference between the reconstructed tl and th masses and the true top quark mass determined from simulation [28]. I don’t understand what is meant by true top mass determined from simulation and also eq 6.1 and its explanation in ref [28] is not very clear. I don’t understand if it is the event by event mass from reconstructed and MC truth matched objects in simulated signal events that you are using in the chi2, or it is the average mass and resolution for reconstructed and MC truth matched objects in simulated signal events. But then how do you compute this quantity for the background processes and on data? can you please clarify?

Offline objects matched to MC truth information in ttbar MC is used to determine the "true" top mass. The, of course, in data and all our MC samples, we compare the reconstructed mass with that "true" value (which is a Gaussian with a mean and a sigma) to calculate the chisq. Youi can find all details in the appendix of our AN

143-144 please write boosted in quotes or write explicitly the boosted event category, same for the other two categories.

Figure 1why do you show Delta y here? show it later.

We show here representative distributions for the entire sample and show Deltay because it is our main variable. Later we show it in each of the individual channels, but this is the place where we show it for the entire sample

185 for ttbar; for the ttbar process

188 Finally, an uncertainty in the

189 correction to the top quark pT in simulated tt samples, which depends on the generator-level

190 top quark transverse momentum, is evaluated as a one-sided variation computed from the dif191 ference between the top quark pT distribution with and without the correction [59]. I think this is not explained well. of course the top pt depends on the generator level top pt… and the explanation of how you compute this correction is not clear enough.

We added further explanation in the previous section and maybe this is clearer now

196 say to which physics quantity you apply the simultaneous binned maximum likelihood fit to data, perhaps anticipate the lines 228 and following, where you present the figure 2 at least the part of the figures before the fit renormalization, at the beginning of the section.

This is a general introduction to the method and all the details are given later. It is not so trivial as to state the quantity here as we do it to the Ac and the neg side of the Deltay after rewriting the likelihood. The reader really needs to follow closely to understand what is done

227 I think that since the ST sample is mentioned rarely in the paper and it is not a well known acronym it would be better for the reader to spell it out everywhere.

We prefer to leave it as is, we also do not spell out DY and eventually they are combined in any case

I think that you should introduce before somewhere in the chapter about the object reconstruction a sentence or two about the charge measurement for the muons and electrons, and its associated charge misidentification error. The error should also be included in the systematics. These are not typical pt of the leptons, so are the errors on the charge misidentification well known from the MUO and EGM POGs or did you recompute them especially for this analysis ? Normally for the charge asymmetry measurements in other papers (W boson for example) there is a dilution factor due to the charge misidentification which allows to compute the true charge asymmetry, why is this not done in this case?

Nothing special was done as is standard in the top group for other measurements that use the sign of the lepton (see PhysRevD.100.072002 for example)

fig 2 caption text: As can be observed, these uncertainties are reduced significantly after the likelihood fit, and the agreement between data and simulation is improved. this text should be removed from the caption, the caption should only say what is shown in the figure, any comment should be in the text. Just delete this comment from here because it is already in the body of the paper.

OK

I think that in the conclusions a sentence about the interpretation of the results could be added.

Not sure what you mean by interpretation

### Brian Lee Winer

Type A: Line Comment

130 I think you could drop “leg of the” and the sentence would read perfectly fine.

Leave in place to reinforce "leg" concept.

130,132, 135 In the first two lines you use hyphens “t-tagged jet”, “W-tagged jet”, but in the last line there is no hyphen “b tagged jet”. I hate the “Hyphen Wars” that get fought over this point, but this jumped out at me. If the CMS Style Guide says to do it this way, so be it.

The Guidelines do give "b-tagged" jet as an example (of correct hyphenation). Line 135 was using the \btagged newcommand defined by the authors, and this does not fit here. The journal will have its own style guide.. %ENDCOLOR

Type B: Line Comment

59-60 In the text preceding this you describe the high Pt, low Pt, and then intermediate range. The sentence on these lines uses the words “ ‘boosted’, ‘semi-resolved’, and ‘resolved’, respectively.” While I think it is clear, the order of the description does not line up with the order of the descriptive words.

We have been asked to change the order back and forth 3 times. We agree the names do not match with the description, but others thought that the names were self-evident and that the order of non-boost was more important. Left for now to decide by the pib com,

106-108 I understand you are trying to be brief but combining the muon and electron selection via parenthetical statements seems cumbersome. Why not just split out the selection into different sentences?

Indeed, this is not uncommon for a letter to save space and also to make it easy to compare the cuts in the two channels. Left it as is.

138 I think the start of this sentence could be reworded to make it read more smoothly. What about, “Finally, the tt hypothesis selected is the one with the…” or “Finally, we select the tt hypothesis with the…”

161-162 Seems like this statement regarding an added uncertainty deserves a reference or further explanation as to why you are adding it.

Do we have a proper reference to add

Figure 1 Any consideration to show Mtt on a log scale?

DONE

196-197 When I read this first sentence, I felt left in the dark. I had to sit there and think about what all the bins and categories were. (I’m not even sure I came up with the right answer.) I think it would be helpful to the reader to be a bit more explicit about what all the bins are. (Note: You helped me out on the next page on lines 219 – 220, but it was a little frustrating reading to that point and not really knowing.)

This is a general introduction to the method and all the details are given later. It is not so trivial to start explaining what is particular to our analysis earlier without failing to explain the general idea frst.

225-231, Table 1 I am trying to reconcile some statements in this paragraph along with Table 1. The second sentence says you take the backgrounds from simulation. But you also point out the fit constrains uncertainties. I assume the fit can move the normalization of the background through nuisance parameters. Is that right? If so, I assume that Table 1 reflects those shifts. Also, maybe in the Table 1 caption it is worth explicitly mentioning in the last sentence that uncertainties are those constrained by the fit (assuming I have it right).

you are correct about what we do, added "and their normalization allowed to change during the likelihood fit" to the text. The table caption says that the values are after the liklihood fit and we do not think more explanation is needed

232-233 First sentence – I am confused why this statement is here. The rest of the paragraph seems to be talking about extracting A_c

Maybe this is because the figure and the table are in between this text and the previous paragraph. The format has changed now and they are together. The previous paragraph talked about the result when combining the 12 channels and this one explains how to get the Ac for sub-channels

Equation 3 Since r_pos and r_neg come from the fit, why not write down an expression for A_c^fid in terms of these quantities?

The fit gives Ac and r_neg directly, that is why r_pos needs to be written in terms of Ac and r_neg. This methods allows us to obtain Ac and its errors from the fit without having to do an additional error propagation (and have to deal with all the correlations between sources

Figure 2 For the top two plots, the words “pre-fit” and “post-fit” get very close to “Mtt”

This has been fixed%

243-244 How about moving this summary statement later after you have directed the reader to the table with results?

We prefer to leave the formatting to the journal and to latex and not instroduce forced breaks%

Equation 4 Similar question to above. I guess you must have a reason for doing it this way.

We do fit for Ac directly, that is why we need to do this

Figure 4 Do we need to tell the reader what we mean by “impact” or do we believe this is well known?

We think this is known by now in a specialized HEP journal like PLB

261 Perhaps a more descriptive statement about Figure 4. Instead of just “ranking” something like “ranked impact parameters”

Changed to "Figure 4 shows the $\pm 68\%$ confidence level ($1 \sigma$) impacts of the nuisance parameters corresponding to the systematic uncertainties for the inclusive $\AC$ measurement for $\mttbar > 750\GeV$ ranked in order of decreasing importance"

General Comment You spend a lot of time showing the data broken down by years, yet you don’t talk about the consistency of A_c between the years. Are they consistent?

• Done

• L90: (2018, 2017 and 2016) → add a comma: (2018, 2017, and 2016)
• Done

• L92 and following: if you want to use pT with the mu and e apex, please make sure to define it in the first instance. Also in line 92 it refers to the pt of the muon at trigger level, later on at reconstruction level. I think it would be better to just use pT and everytime qualify which particle it refers to, but it’s ok if you prefer to keep it like this * I think the text explains which particle and what level it is implemented at

• L102: I don’t think DR min or PTrel are explained here, right? I think a couple of sentences should be needed. Also , make sure pTrel uses the roman font for “T” and “rel”. I also think rel would be better at the apex, or after a comma, e.g. "pT,rel " or "pTrel"
• added explaination and fixed pTrel

• L106 tt should not be italic, please check the latex
• fixed

• L106 and following single top → single top quark
• Done

• L107 : here it would be a good place to define e.g. Z/gamma+Jets = DY or similar, if you intend to use it later, see further comments on figures and tables
• Done

• L109 “Dibosons “ is jargon: "Vector boson pair (Diboson) "
• Done

• L116 and following: I suggest to use pennames for t_h and t_l, thus avoiding to italicize them all
• fixed

• L126-127: I think Semi-resolved contains events with one W-tag and no t-tag, correct? I think this should be clarified, as you do for the Resolved case.
• We defined before that Wtag and toptag are two exclusive categories and think it is not necessary to repeat it.

• L130 “to multi-TeV region” → maybe “to the multi-TeV region”
• Done

• L132: “Good agreement is observed…” → “Good agreement between prediction and data is observed…”
• Done

• L136 “simulated sample” → “simulated samples”
• Done

• L138 I think you mean “integrated” luminosity of data, correct? Or here you want to mention that the pileup profile is reweighted to match the one in data? I feel this should be clearly split in two sentences to avoid confusion.
• This sentence is reworded.

• L142 “All other sources affect both normalization and shape.” → I think pileup does affect also shapes, so this is another reason to split the two sentences above to this end.
• fixed

• L148 “applied in addition” → “is applied” , in addition is redundant and can make people wonder if you mean something specific.
• Done

• L153-154 add space between “Factorization” and the parenthesis “(muF)”, same for “Renormalization(muR)”
• Done

• L158 (ISR & FSR) → (ISR and FSR)
• Done

• L161: I think besides the top pT, cross section uncertainties are also not mentioned here.
• Added at the beginning that we have 30% for backgrounds and 5% for signal

• L163: “... is obtained using a binned maximum likelihood fit on the data to extract the signal strength modifiers through a simultaneous fit to all bins and categories…” → suggest: “is obtained by performing a simultaneous binned maximum likelihood fit to data in all bins and categories…"
• Done

• L166 “to a single nuisance parameter for each process, this is known” → I think not accurate, as BB-lite does consist in wrapping together stat. uncertainties for each bin. I would suggest to rephrase simply as “Statistical uncertainties due to the limited MC sample size are treated separately in each bin with the Barlow-Beeston approach [37].” without going into details of how they are split per each process.
• Done

• L172: one line number is missing between L172 and Eq. 2, probably because of latex issues, please fix.
• fixed

• L190: (2018, 2017 and 2016) → add a comma: (2018, 2017, and 2016)
• Done

• L195: here you should have the same notation for Single Top, QCD and Z+jets you defined in the Section 3. Please make sure to unify it and use a consistent nomenclature! Example of different instances: Z+Jets, DYJets, DY → should all be e.g. “DY”, if you define it in Section 3 QCD, Multijet QCD, multijet QCD, QCD multijet → should all be e.g. “QCD multijet” SingleTop, single top → could be “single top quark”
• fixed

• Table 1 caption: ”(2018, 2017 and 2016)” → add a comma: “(2018, 2017, and 2016)”
• fixed
• Table 1: same comment as line 195, plus please change DATA → Data, TOTAL → Total
• fixed
• Table 1, significant digits: please check the pubcomm guidelines on the suggested number of significant digits:https://twiki.cern.ch/twiki/bin/view/CMS/Internal/PubGuidelines#Number_of_significant_digits_for
• fixed- now has 2 significant digits

• L198: some line numbers are missing between L198 and Eq. 3, probably because of latex issues, please fix.
• fixed

• L198-through equation 3:“Sub-combinations” → suggest: “Combinations of sub-sets of these channels”
• “multidimensional” → ”multi-dimensional”
• “which allows us to obtain from the likelihood fit directly.” → I think it would be better to end with “directly:” rather than a full stop (“directly.”), so to better introduce Eq. 3.
• done
• L203 “Acfid which gives“ → “Acfid is“
• fixed
• L203 “Acfid is the number of the charge asymmetry Acfid ? the second one is not necessary, since here you are giving the symbols in the context of the formula. This unless you want to separate the measured value from the observable, in which case you should change the first one to “Acfid,meas, but I think it would be an unnecessary complication.
• fixed
• L206 “Table. 2” → remove the dot: “Table 2”
• fixed
• L218 “Table. 2” →remove the dot: “Table 2”
• fixed

• L208: some line numbers are missing between L208 and Eq. 4, probably because of latex issues, please fix.
• fixed

• L208-through equation 4:
• “To do this” → suggest “To achieve this”
• “In this case, ” → suggest “In this case, the relation:”
• fixed

• L209 “from the likelihood fit directly the signal strength” → we already had defined the signal strength, so you can leave only AC full - it seems out of place that out of all quantities you decide to redefine just rneg.
• we are not redefining here but stating the r_neg is one of the signal strengths we extract along with Ac so it is a part of our results

• L212-214 maybe this should be in the Systematic section?
• fixed
• L216 “Figure.4” → remove the dot: “Figure 4”
• fixed

### Approval Questions March 11, 2022

• See presentation about some of these questions
ARC-Authors_Meeting_April_6.pdf: ARC-Authors meeting slides April 6, 2022

• 1. Why the lumi nuisance is pulling by 2 sigma?
• if you look at the yields table pre-fit (page 19 of the Approval talk) you will see that all channels have a data/MC ratio of about 0.9-0.95 while 2017 muons has 1.07 and 2017 electrons has 1.0. It is the need to bring the 2017 MC up that causes the close to 2 Sigma up variation in 2017 luminosity. 2016 and 2018 luminosity are pulled a bit towards the lower side (see appendix G1 for the complete list of pulls). We realized that our implemetation of the 2D SF niusance was not as intended. We now allow it to vary separately from the lepton ID/Reco and that solves the pull on the 2017 luminosity. The resulting impact is shown below * Ac_750_full-new.pdf: Impact after separating the 2D SF nuisances

• 2. Please consider to remove luminosity as a nuisance, since we cannot use same data to measure a cross section and also constrain the LHC machine performance, which is what lumi is. The suspicion is that lumi nuisance serves as a wildcard being shifted while in truth its other nuisances that need shifting.
• we did as requested and the impacts can be seen below. Keep in mind it is only the 2017 luminosity that is pulled up, which is needed because of the difference in the data/MC normalization seen in our sample for that year compared to 2016 and 2018. Not allowing the luminosity nuisance to accommodate the need of the different years causes other nuisances to move away from nominal to try to accommodate the data, as can be seen with single top x-sec (the largest background) and some of the data MC statistics.

* Ac_750_full-fixedlumi.pdf: Impacts when fixing luminosity to the nominal value as a test

• 3. p21, on the hdamp systematics - how did you ensure the MC statistics is not an issue? (A toy study was suggested before entering ARC review.) What will happen if the major systematics are symmetrized?
• The effect of hdamp was adressed with an alternative to toys during ARCreview (See point 4. uner Comments from ARC/Authors meeting Feb 15, 2022). Appendix F shows the nominal and up/down variations for each channel for hdamp with the statistical error on each entry. The are some few bins where the differences between the up/down variations are not statistically significant. We ran with all hdamp systematics symmetrized in all channels and the result is that hdamp is even more constrained. We think the main effect we see from hdamp is statistics and are adding an additional nuisance the size of the stat component of hdamp. This is running now.

• 4. For the hdamp systematics, check if the option of re-weighting the central samples still available, which could give better MC statistics power.
• This does not appear to be possible

• 5. Please fix data error bars for the m(tt) distribution on p17.
• Done %

Comments from ARC/Authors meeting Feb 15, 2022

• 1. Add electron trigger vs pT plot for each year
• These have been added to Appendix A

• 2.1 Regarding contribution of tW in ttbar, ask Single top conveners for latest guidelines. Do we need to use alternate samples as variation?
• Communicated with Matthias Komm. The guidance is that only analyses that have Wt as a signal need to include the different diagram removal/diagram subtraction schemes as modelling uncertainty.

• 2.2 Do a fit with tW+ttbar combined to see the result
• We tried this for one channel 2018 muons (750,900) mass and the measured value of the assymmetry did not change. About half of the single top contribution comes from Wt and accounts for 3% of the sample. Given these findings and the recommendation, we will not add a varied sample to the Wt uncertainty.

• 3. boosted ttbar samples- 2017 ntuples already exist- start by adding that. Look into making ntuples for 2016 and 2018
• We have added the boosted samples for all 3 years and the MC statistical error is reduced by 40% and the total error by 10%. More importantly, the MC statistics is no longer the source with the largest impact. AN and paper were updated.

• 4. Make up/down with nominal hdamp plots including MC stat uncertainty
• These have been added to a new Appendix F. It can be seen that 1.- The uncertainties are strongly not symmetric (selectively for some rather than other years/distributions). 2.- For 2016 and 2018, many bins do exhibit a one-sided behavior. 3.- The shape effects are very different across years, even in the same category. 4.-In one case, the two uncertainties even flip (2016, e+jets, > 900 GeV). However, none of these features can be attributed to statistical fluctuations. After discussion with the ARC, the proposal was to rerun the Impacts using Asimov data but treat hdamp across the 3 years as uncorrelated. Both the central value and the total error remain unchanged compared to the nominal result where hdamp is treated as correlated between the years. We conclude that the constrain seen in hdamp does not affect the final result, no further concern about the effect of hdamp on the result remains.

• 5. Add all data with no Mttbar cut for an inclusive measurement? How many events are there for M<750? Plot |DeltaY| for each channel/region to see.
• We measured Ac in the low mass region (Mttbar<750) for 2018 MUONS which is about 45% of the total sample (the electron channel only contributes about ~5% of the events because of the much higher kimelatic cuts. This means that we could obtain a result with statistical uncertianty comparable to M>750GeV. However, compared to the ATLAS preliminary result, the statistical error would be a factor of 4 larger, which renders our result not competitive in this low mass region. The mass region below 750GeV was beyond of the scope of this analysis from the beginning, and we prefer to stick to the baseline result for this publication.

• 6. How to include Wilson coefficients for EFT interpretations
• We looked in to this. Including Wilson coefficients would require defining a MG process and UFO model, generating the Wilson coefficients and then run the mad graph card with Pythia8 and finally using Rivet to create the needed reweighed observables. This is not something that we can complete in the given timescale, specially as we would have to run this privately and we are pressed in time to finish the analysis before key personpower is lost.

For the paper:

• 1. Figure 3: All statistical error should be summed up together but included in the plot. Meaningful names should be used for the systematics.
• We produced a plots with better names and both pre and post fit values for the paper

• In the introduction and in the summary, it should be written that this is the first measurement published with 13 TeV data, and I think also the first one with substructure techniques to reconstruct top quarks - maybe you can cross-check this, it seems the case to me though.
• There is an 8TeV ATLAS paper (PLB 756 (2016) 52-71) that also includes exclusively masses above 750GeV and uses a fat jet for the hadronic top but isolated leptons. There is a preliminary ATLAS result at 13TeV (ATLAS-CONF-2019-026) that includes both resolved and boosted topologies and also uses only isolated leptons. We will stress that this is the first measurement at 13TeV truly optimized for the very high boosted regime not only on the reconstruction of the hadronic top but also on the reconstruction of the leptonic top (no explicit isolation and jet/lepton cleaning starting at trigger level)

• I feel like the information on the table of systematics could be improved, especially the "uncertainty" column for shape uncertainties. It is not possible of course to convey info on how much it affects the result easily in this table, but maybe you can put the typical range of variation in the pre-fit distribution to give an idea of how large the uncertainty is, e.g. jes : 1-4%, electron id 1-2%, pileup < 1% etc.

• Also, one should if possible add the info on per-year correlation. If the information is trivial, e.g. always correlated/ always decorrelated but one or two, it can go in the text as well.
• We state in the text that there is partial correlation between luminosity and mentioned partial correlations as appropriate (some sources of b-tagging). We feel that going into much more detail is beyond the scope of a letter and all the information will be in HEPdata.

comments from Alberto Orso Maria Iorio on AN_v10 (Feb 10, 2022)

• Comment 2) Page 7, L178 and Table 4 : you are studying boosted channels, but you are not using high Mtt samples TT_Mtt700to1000 or TT_Mtt1000toInf. Is this not affecting your MC statistics? Please show the MC statistics contribution to the uncertainty in your final distributions for Delta|y|. Also follow up on this in the systematic section - see also comment 10)
• We added them and we now quote MC stats separately.

• Comment 3) Page 12, L220-221: you are vetoing events with 1 jet in the HEM damaged sector. Two follow up questions: 3.1) Could you please show a jet-phi and MET-phi distribution in 2018 to make sure this is ok post correction - or point to this distribution if it’s already somewhere? In general the HEM issue should enhance QCD jets, so I’d expect if you check at one of the later stages of selection this should fix the problem, but should at least be verified. 3.2) If there’s still some effect, have you considered extending the veto to electrons as well?
• We are actually already vetoing on the electrons. The figures below show MET_phi and Jet_phi after applying Veto for events passing our preselection. You can see that there is no "bump" in the -2 to -1 phi region as can be seen in AN2018_298 from B2G-19-004

• Comment 4). Page 13, Eq. 2: is there a reason to prefer the 2D cut to the MiniIso? I think there might be some studies in B2G-17-017, in case, could you point us to them?
• The two isolation cuts were compared in AN-2014-035 Appendix C. Quoting from that AN, the mini-isolation has a higher selection efficiency for boosted ttbar final states, but worse rejection of QCD background than the 2D-cut. We stuck with the 2D because the level of QCD is too high with the mini-isolation cut.

• Comment 5) Your trigger for electrons is expected to lose efficiency for high-pt electrons. Did you check the efficiency of your trigger? It is usually recommended to use photon triggers in this case. In case there is some reason (e.g. from B2G-17-017), please link the appropriate studies.
• We have actually measured the efficiency in data and MC and extracted a SF. If you look at Fig 24, 26 and 28 af Appendix A1 you can see that the efficiency in data is very close to 100% even for high electron pT. This is due to the OR with the high pT single electron trigger and none of the triggers having lepton isolation.

• Comment 6) Page 14: 6.1) Can you please clarify a bit more in the text if you are doing the jet-lepton cleaning for all the PF candidates or just for the ones selected? 6.2) Can you please also specify if you have any AK4/AK8 disambiguation before the top-assignment part?
• The lepton/jet cleaning is done at the preselection level and is done to every lepton (signal and veto). Only AK4 jets that are at DeltaR>0.8 from the AK8 jet are considered for the jet assignments (See Sec 4.2, line 427)

• Comment 7) Page 15: My understanding is that you don’t have any strict requirement on the tagging of the jet inside of the hadronic top (although you don’t explicitly veto it ), is that correct? My guess is that always asking for a second b-jet would kill a lot of signal and not help significantly against the background, but could you please elaborate also in the text a bit about this?
• You are correct about our treatment of the b-tagging requirements. We have not studied the effect of requiring a second b-tag but as our reducible background after one b-tag is very small, we think the gain of asking for a second one would not outweigh the loss of efficiency for the signal

• Comment 8) Page 16: 8) Please clarify the following in the selection: 8.1) Veto electrons / muons are always also “main selection” electrons? 8.2) Jets: in the spirit of comment 6.2) if you don’t have an AK4/AK8 disambiguation at any stage, the leading / second to leading jet can overlap with the AK8, correct? I guess it’s fine, it’s a design choice, but it’s for my understanding of things.
• 8.1) Veto leptons are defined in Section 3.2.1 and 3.2.2. The only difference with signal leptons is that they have a lower pT - this is to reject more dilepton events. 8.2) AK4 and AK8 have to be at DR>0.8 apart to be considered.

• Comment 9) Page 23, Tables 8-9: 9.1) You mention that the errors shown include stat. and MC uncertainty. Could you please clarify the entity of both? I am particularly concerned about the number of ttbar. 9.2) Also you mention you have a 30% uncertainty on WJets, single top, and QCD, but the uncertainty here for single-top is around 10%. Please clarify and eventually modify the label accordingly, if this is not what is shown.
• Table has been remade and a mistake in the script was found. Keep in mind that the numbers in the table do not enter the Combine analysis at all, they are just used for the AN as information on the sample composition.

• Comment 10) 10.1) Page 25, L463: It is not clear how you treat MC Statistical uncertainties, and in the paper you mention you use the standard Barlow-Beeston-lite method, I assume as implemented in combine. Please add it in the note as well - and of course sorry in case I missed it!
• Added as an extra item in the list of systematics considered and also to the unfolding section.

• 10.2) From figures 161-222 (pages 143-178)in appendix H it seems like the MC stat uncertainties are by far the largest component, especially in data, but in Figure 3 of the paper they are absent. Can you please clarify if the latter is just done like this to display the physics component of the uncertainty? This it particularly important in view of my comment 2, because this is a point where the addition of the exclusive samples at Mtt > 700 might really help.
• We followed what ATLAS did in his conference proceeding and excluded the MC stat errors from this plot (even though they are very important) to just show the ranking of systematic errors. We will make sure to make it clear in the paper that the MC stats errors are dominating but not shown in the plot.

• Comment 11) I am not sure anybody has ever considered this, but the tWchannel has interference at NLO with the TT. This means that it might be the same new physics contributions apply to both. While from the theory standpoint I am not sure people have considered it in CMS, one could consider to have the measurement done not only on TT, as is done, but considering the sum of TT and single-top tW as a signal. While this might not be in the scope of this paper, at least I think this background requires a bit of care. Therefore I would suggest to: 11.1) Check in the single top component how much is tW and how much the other parts. I think the tW will be the largest part.
• Wt is 48% of the total single top sample in our phase space.

• 11.3) Comment about the feasibility of repeating the analysis by summing together tW and ttbar.
• We tried this for 2018 muons (750,900) and the measured result for the Ac did not change (the Wt events are 3% of the total signal)

• Comment 12) The part on the unfolding might benefit from a bit clearer definition, my main point of doubt being about how you define the fiducial phase space. My understanding is that the fiducial is taken as the reconstructed phase space. Please confirm the following and improve the description accordingly:
• Yes, the fiducial phase space is defined based on reconstructed quantities, added this to the AN

• 12.1) Page 29, L625, and Eq 7: here you always take the top quark at parton level, correct? You never consider any particle-level quantities.
• Yes, we always correct back to parton level (see line 621).

• 12.2) mu_i is the number of parton level top quarks in the full phase space or in the fiducial phase space? Please clarify.
• Yes, mu is the number of expected reconstructed events in the fiducial phase space, as predicted by MC. Added a clarification.

• 12.3) L679 here Ntruth you mean the number of events in the corresponding bin of reconstructed Delta_y, i.e. fiducial phase space == defined by cuts at detector level?
• Ntruth is defined at reconstructed level but includes information from both generator-level bins. Changed the description in the note and added "gen" to the places where generator information is used to hopefully make it easier to follow.

• 12.4) Page 33, figures 10-11 : what is the variable on the x-axis, Delta|y|? This is made a bit confusing by the “>/<” symbols, which refer to the region I guess.
• The x-axis is actually the individual variations of the different theoretical uncertainties. We have fixed the plots

comments from Helena Malbouisson on AN_v10

• 1 - Did the STAT group review the analysis? Were there any recommendations on the handling of the statistical uncertainty of the A_C?
• We filled the questionnaire but did not hear back. The Combine contact approved the Data cards and the treatment of errors within Combine.

• 2 - Trigger efficiency curves ==> is it OK to cut the muons on 55 GeV when the trigger threshold is 50 GeV? Is 55 GeV already in the efficiency plateau? I did not find the efficiency curves for the single muon trigger in the analysis note. Would you please include it or point it to me in case I have missed it?
• We are using the centrally provided SF and did not include the plot but only the reference to the muon POG tiwki (Ref. 24). You can see the plots for 2018 at
https://indico.cern.ch/event/801419/contributions/3330788/attachments/1801407/2938479/190225_wonjun_SingleMuon_TriggerEff_SF_For2018.pdf The SF is essencially 1 for pT>52GeV and 55GeV is certainty in the plateau.

• 3 - Figure 1: There seems to be a trend in the chi2 distribution for the 2016 data ==> is it acceptable/understood? Also, the uncertainties seem a bit large. Since there are both systematics and statistical uncertainties, it is hard to say from the plot which is the dominant. Would you please specify?
• These plots are pre fit and this looks mostly like a normalization issue. Because they are pre-fit, the dominant uncertainties are the flat scale uncertainties on the MC yields.

• 4 - Figure 2: The quality of the fit is not very good for the leptonic tops mass (chi2/ndf is 124/7). Is it understood why? Does it have any impact on the signal event selection?
• We use the same range for all fits and actually the individual years (shown in the appendix) have better chisq/dof values. In any case, the chisq is just a method to assign the jets to the leptonic or the hadronic side and the outcome of the jet assignment is quite insensitive to the values used in the chisq definition. Please note that we use values for sigma that are quite large compared to the error of the Gaussian fit (See Line 904 in Appendix B).

• 5 - I wonder if the top quark mass uncertainty should be included in the systematics? Is the chi2 variation included in the systematics? I did not see it listed in section 5.
• If by top mass uncertainty you mean using MC generated with different masses, then we do not do that. Refer to our previous response about the effect of the chosen values in the chisq. It would not be meaningful to repeat the analysis with different values of parameters in the Chisq definition as it would not result in different jet assignments.

• 6 - l. 697: what is the uncertainty on A_C^{full}? How did you conclude the uncertainties studied are smaller than the systematics of the A_C^{full} ? Would you please mention it in the text?
• Fig 10 and 11 show, in a vertical scale that corresponds to the error on AC(full), the variations from the different theory error sources. The horizontal axis was mislabeled but each point in the plot represents one variation for PDF, q^2, FSR/ISR and hdamp. The plot has been fixed in the AN. All these variations are insignificant compared to the uncertainty on Ac from all the other sources.

• 7 - Figure 14 and similars: I can't say what are some of the nuisance parameters listed in the plot. It's the same in the paper. Shouldn't the names be more suggestive?
• We remade the plot adding the MC statistical error and using more meaningful names for the uncertanites.

• 8 - Maybe it is not the scope of this analysis, but did you also consider doing the differential measurement of the A_C?
• Actually, we do have one inclusive for Mass > 750 and then two differential bins [750,900] and > 900GeV

comments from Jan and Matteo on AN (Nov 23)

• The impact of MC stat is very large, as indicated by the raking of the BB parameters and the strong constraint on hdamp. Please consider using the boosted samples produced in the context of the energy asymmetry analysis (KIT). These include both the nominal model and the hdamp variations
• We determined that the constrain observed in the hdamp samples was not due to low statistics in the samples. We also determined that the effect of the hdamp constrain in the final results is negligible. The samples were privately produced and it is taking forever to run on them. Also, only partial samples are available and the 2017 ones were deleted.

• Out of curiosity, what would the final uncertainty be if you merge the Mtt bins? I understand that from physics considerations it is advantageous to keep them separate, but due to the MC stat issue described above, it could turn out to be the opposite
• The MC statistics error enters combine for the signal samples only. The main problem comes from the migration in the signal sample between mass bins (750-900) and >900GeV, which we were taking into account in all three results we were presenting (Mttbar > 750, (750-900) and > 900). We have now changed how the combine fit is set up and we are able to get much better results for the MC stats. We fit in one mass bin at the time, corresponding to the 3 results: Mttbar > 750, (750-900) and > 900. Events generated with a mass outside the reconstructed mass bin under consideration are taken into account as underflows or overflows. However, Combine is not forced to try to fit a handful of events that have a reconstructed mass outside the mass bin where we are measuring the Ac.

• Given the very strong constraint on hdamp, and therefore the unreliable estimate of this effect, the impact of the MC stat uncertainty of the hdamp samples should be studies. The easiest way would be to use a toy-template procedure as in TOP-17-001, as documented in the corresponding AN
• We were able to determine that the constraint is not caused by low statistics in the samples and that it does not affect the final result.

• Top pt re-weigthing: for clarity, please mention that you are applying the data/powheg+pythia weights
• DONE

comments from Jan and Matteo (Oct 12th)

• In your selection, can a jet be both t-tagged and W-tagged? Or do you first check for t-tags and subsequently for W-tags?
• The conditions for an AK8 jet to be t-tagged or W-tagged are exclusive of each other and a jet cannot fulfill both, (typo on SD mass fixed) . We have added more information in the selection section to hopefully make this clearer.

• For clarification: for leptons you use an ID not requiring isolation (as it should be). Both are cut based; are there more powerful IDs around that you could use in case the analysis would profit (e.g. the ttbar reconstruction)?
• We had compared with the MVA cut a while ago(AN2015_107_v9 attached) and it was found that the cut based ID was better for optimizing the background rejection in the boosted tuba phase space.

• (303): MET: not too important now, but please don’t forget to change it to “missing transverse momentum” for the paper
• Done

• (315): HEM: is the PU distribution affected by this? Do you have control plots showing that? (This might affect more analyses, so please feel free to just refer to a check here)
• We haven't checked the PU distribution, but the jet eta and phi have been studied. Can be seen in B2G-19-004 (AN2019_298).

• Section 4 in general: having in mind the upcoming review process, it would be good if you could motivate the choices, categories, and strategy etc a bit better right in the beginning; right now, it is hard to follow. Some comments below are related to that
• Keep in mind we re-use the event selection that was published in 10.1007/jhep04(2019)031. We added references to the AN that details all the optimizations in object ID and event selection that were done prior to that publication.

• L 348: How do the AK8 jets and the AK4 selection relate? When do you pick which combinations?
• We have changed the description of the event selection and categorization and hope this is clearer now. Only AK4 jets that are at dR>0.8 from a top or W tag are considered for jet assignment.

• L357: Lepton 2D cuts. You claim that the QCD background is reduced strongly. Do you have control plots (in general a few more control plots would help)
• We include plots only after the event reconstruction that includes the final selection on the chisq and also the categorizaion into the 3 topologies. These are shown starting in Sec 4.2 and also in appendix C. We have added the plots of the two Delta|y| bin also.

• Fig.1: could you include more details in the relevant range (around 30, where you cut)?
• DONE, now the shows the 0 to 30 range, which are the events that pass our final selection .

• All figures: keep in mind that for the paper version of all plots you will need to increase the legend sizes (not needed for AN imho). The rest already looks good in terms of sizes etc. I would also recommend choosing different colours. A great way to do this is using https://colorbrewer2.org/ .
• DONE. Now the colors are ttbar(l+jets) in red, ttbar(others) in darker red, W+jets in green and others in blue

• L398: defining these categories (even if you don’t use them explicitly) in the beginning of Section 4 would help the flow of the whole section
• We have moved them to the end of the event selection (Sec 4.1) before the kinematic reconstruction.

• L401: in the boosted jet category, is it strictly necessary to require 0 W-tagged jets? Maybe you can check (for signal and main backgrounds) how many events have 1 t-tagged jet and one or more W-tagged jet. Maybe it’s totally negligible, but you may gain some events by being inclusive on the number of W-tagged jets in this category.
• top and Wtag conditions are exclusive of each other (softdrop mass covers different ranges). We cannot have both at the same time. We also do not expect more than 1 hadronic top in our sample, and it would be either a boosted (top) or semi-resolved (W), not both. We would not expect to gain anything except some all hadronic candidate which we do not want in our sample.

• Section 5: Also here, starting by stating that you are measuring in the full and fiducial phase space defined by XYZ would improve clarity
• We have now stated this at the beginning of the unfolding section and then added separate subsections for each of the corrections. We have also added a section for how we implement the likelihood unfolding using Combine.

• L442: you say that the priors for the nuisance parameters follow a log-normal distribution. This is correct for normalisation parameters, but in all other cases I believe one should use a normal distribution. Is there any specific reason for this choice
• There was a typo and is now fixed in the AN. We indeed use log-normal for the normalizations only and normal for the other uncertainties.

• L448: in the formula, where do you account for correlations between nuisance parameters (e.g. from years)?
• All correlations across the years and channels are taken into account in the combine data cards. We added information about the Combine implementation in a subsection of the unfolding section to make this clearer.

• Yes, we got the green light on Jan 27

• L455: In the next presentation in the TMP meeting, I think we should have a discussion about visible and full phase space again, and how exactly the extrapolation etc. is done.
• yes, that will be helpful

• Section 5.1: it is not quite clear why this is necessary. In the end you don’t have a choice w.r.t. ‘forward’ versus ‘backward’ in terms of binning, and also the binning is very coarse. This is also reflected in the high and continuous purity and stability, and the fact that you can easily do it without regularisation for the forward-backward unfolding. It is not clear from this section though, what happens to migrations between the mttbar bins, and how they relate. This would be the more important question and should be described.
• You are correct, we were not taking the possible migrations between the mass bins into account. We are doing this now by starting from 8 bins at generator level instead of 4 (we split the mass in two bins). The AN has been updated accordingly.

• Figure 7: it looks to me like purity and stability are almost the same in every bin. I am not saying this is wrong, but it would be great if you could double-check this result
• We have double checked this and it is correct.

• Table 13: when you say “Shape” you mean that the normalisation component is removed? I would advise against doing this in the visible phase space: the uncertainty should not have a normalisation effect in the full phase space, but due to acceptance effect you will have a genuine normalisation effect in your visible phase space
• FIXED (it was a typo), all nuisances parameters (except x-sec) have a shape and normalization components

* In the same table, hdamp is classified as a “Normalisation” uncertainty. Is it a mistake in the table?

• FIXED, yes it was a mistake

• Do I understand correctly that the JES are still not split in this version of the fit? What else is missing?
• The JES uncertainty are based on the different sources - there are 10, so it is taking a while but they should be done soon. The JER correction is ready. Both were implemented for version 11 of the AN.

• Which distributions exactly are used as input to Combine? These should be in the main body of the AN
• the delta|y| is used as an input, added in Sec 4.3

• It looks to me like the MC stat uncertainty is quite large in your fit. This is clear from the impact plots, and most likely is also responsible for the strong constraint on hdamp. Would that be possible to a) See the effect of hdamp, on your fit distributions (nominal vs up vs down), including the MC statistical uncertainty b) Re-bin some of the distribution in order to reduce the effect
• hdamp up/dowm plots are shown below for muons and electrons. We have only two bins. Rebinning is not an option

• 2018 muon Ttbar (negative \Delta|y|):

• 2018 muon Ttbar (positive \Delta|y|):

• 2018 electron Ttbar (negative \Delta|y|):

• 2018 electron Ttbar (positive \Delta|y|):

• Why is the effect of the top pt re-weighting two-sided? With one-sided uncertainties it’s common practice to set the down (or, equivalently, up) variation equal to the nominal. In this way the impact will be one sided, and one side of the uncertainty will be unconstrained
• FIXED, we had symmetricized the set down with respect to the nominal to get an up variation, but have changed to what you suggest.

• L73 Ref to the theory predictions?

• L104 13 TeV I hope ...
• fixed

• Sec 4.3.2. / 4.3.3 - Please explain if/how the measurement regions for the top and W SF differ from the analysis selection and whether or not it is a concern.
• The description of the control regions used for the mistag measurement is available in Appendix C. The selection is the same as for the signal region except for an inverted cut on the leptonic term of the chisq discriminator and a veto on b-tagged jets. We believe that the regions are similar in phase-space as to be relevant and at the same time exclusive of the signal region because of the inverted chisq cut and the btag veto.We have added some text in the systematic section to make it easier to follow. This method has been used before and we are confident that it is of no concern.

• Sec. 4.4 I assume the 2017 Met EE fix is applied?
• yes our analysis selection uses jets with pT>50 GeV and in range [-2.5,2.5] eta. So this is taken care of.

• L338 What motivates the dR>1.2 cut (as opposed to 0.8)?
• this is a typo, now fixed to 0.8

• fixed

• Unfolding: Are weight-based inefficiencies accounted for in TUnfold? Please see AN-19-227 Sec. 10 for a detailed explanation. (Please not that a perfect closure when unfolding the MC to itself is NOT a check if the weight-based inefficiencies related to reco level objects are incorrectly applied at the parton level where the closure is checked; please clarify what is done in Fig. 6 in this respect)
• We are following the procedure used in AN-17-130 and believe it is taking correct care of the weight-based inefficiencies related to reco level objects as we never cut any MC event, the weights are just carried along and might be very small but all events are kept.

• Please compute the condition number of the unfolding matrix (see Statcom twiki).
• done, as recommended by the Statistics Committee, we use the singular values instead of using Condition() method. The conditional number is ~10 for all the responses matrices, which together with the very small tau values, allows us to proceed without regularization. We updated the AN.

• done

• Is Fig. 4d from 2017? The lumi value says otherwise.

• Please remember that the 2017 electron channel does not include Run 2B and therefore has less integrated luminosity. It just happens to be very close to the 2016 luminosity, but not exactly the same. We have added a note in the section to remind the readers of this and hopefully avoid confusion.

• Please indicate in the caption which row is which. In Fig. 4b,d the stability is asymmetrical, but in opposite ways. Why is it so? Is it significant? Is it numerically relevant for the result?
• We have improved the caption. We do not know why the up/down variations are not symmetric in the last bins but we have very few events there and also, given the statistical errors, the impact is likely not significant.

• done

• L480 Numbers should be justified, i.e., based on previous measurements and, maybe, inflated with extrapolation uncertainties to your measurement regions. Alternatively, can you consider measuring the background normalization in-situ by letting the nuisance float?
• We use the same prescription as used in AN-2017-130 and assign a 30% rate uncertainty to all MC-derived backgrounds (all in our case, including the ttbar dilepton and all hadronic). We have added a reference to the note. We are not fitting for the background rates in this analysis.

• L485 Please update to current recommendations with partial correlations.
• done and AN updated

• L496, 503. Afaik the Muon SF are partially correlated, e.g. the statistical component of the SF uncertainty is uncorrelated but other sources are not. Please specify whether this is so for the high pt muon ID. Please provide pointers to the exact SFs you are using.

• We either need to see the impacts for the missing uncertainties or a study that shows they are negligible. Otherwise, we can't know whether the strategy is feasible.
• We show here the impacts for the entire sample, and separately for boosted, semi-resolved and resolved. As you can see, top pT dominates in all, but top tag is important in the boosted one, where you expect it to be, and not in the resolved.

* Impacts for all combined

* Impacts for resolved

* Impacts for boosted

* Impacts for semi-resolved

• L567, Fig. 9 As said during the talk, the constraint on top pt should be understood. For example, if there is a mismodelling of the acceptance in one of the categories, it could easily lead to a bias/constraint because your measurement regions roughly correspond to top-pt. I think a dedicated study of the effect of top-pt reweighting is needed.It looks like your pre-fit top_rew uncertainty is already substantially different between e+jets and mu+jets. This should be explained,I see no reason for that.Looking at the top pt shapes in the backup (thank you for adding these), I do not find the pulls so surprising. Can you show a reweighted top pt spectrum and e.g. also bin the reweighted shapes in the analysis categories?

* We have plotted the top pT for the leptonic and the hadronic top in the muon and the electron channel in 2018 to compare data with the reweighed MC with its error, which is given as symmetrized difference between the MC ttbar pT with and without the ptreweighing correction. As can be seen, the description and the error appears adequate in our signal selection (chisq<30 and Mttbar>900 GeV), however, the correction has a trend at high top pT where it is not sufficient as preferred by the data.

• Hadronic top pT for the 2018 muon signal sample:

• Leptonic Top pT for the 2018 muon signal sample:

• Hadronic Top pT for the 2018 electron signal sample:

• Leptonic Top pT for the 2018 electron signal sample:

* We show below a comparison between the data and MC with and without the top pT reweighing in the 3 regimes: resolved, Boosted and Semiresolved. As you can see, the correction helps in all cases and the uncertianty covers the difference between the corrected MC and the data. Keep in mind that in the combined sample the main contribution comes for the resolved regime, then the boosted and a small contribution from the semi-resolved.

• ttbar pT Boosted:

• ttbar pT Resolved:

• ttbar pT Semi-resolved:

• I suspect hdamp could provide a substantial uncertainty, will be interesting to add it.Once the missing systematics are there, it will be good to show the systematic correlation plot to learn more about the important players.
• It does not seem to be very important as you can see in the file below that shows the main systematics for the ttbar signal. We also include the correlation plots
* Systematics for signal

• hdamp and toppT systematics:

• correlations:

• Fig. 71p How is it possible that toptagUp has zero effect (Does it mean the plot is dominated by dileptonic top)?
• Our candidate sample is dominated with the resolved top sample, we do consider ttbar dilepton and all-hadronic as backgrounds

• Fig. 72p Does the size of the variations make sense with the top tag SF uncertainties? Please comment on that in the main body of the AN.
• We believe the answer to your question is the same as above though, our candidate sample is dominated with the resolved top sample.
• Please comment on ultra-legacy usage for later / paper
• We have been approved to stay with the current samples for this publication, will move to ultra-legacy and extend to lower Mttbar regions later

• before Eq.1 - a short discussion of the BSM models predicting modifications of Ac should be added.

• Done

## Object Review

### MC concerns

comment from Enrique Palencia Cortezón:

• I suggest that for 2016 ttbar and single top, you use the CP5 samples.
• we switched to the CP5 samples for 2016 and updated the AN accordingly

### Trigger concerns

comment from Charis and Nicolò:

• In order to ease our review and the book-keeping of all the analyses reviews, we would ask you to fill the questionnaire in the TOP trigger TWiki, in particular listing the trigger paths you are considering, the scale factors you are applying, and the relevant AN where we can find the details. fill out this questionnaire: https://twiki.cern.ch/twiki/bin/view/CMS/TopTriggerTriggerScaleFactors
• Done

• For the muon triggers/trigger SFs we agree that the strategy is fine since you are following the recommendations and using the appropriate centrally provided SFs.

• For the electron cross trigger SF derivation we are a bit confused by the approach you are following, since you are referring to it as 'Tag-and-Probe'. What we would like to understand is : - Are you indeed implementing a T&P approach? If this is the case we would like to understand which is the tag/probe/passing probe selection and what peak do you reconstruct in eμ events?
• In our method that uses the ttbar e mu channel, the tag is the muon and the probe is the electron. Both leptons pass the tight cuts we use in our candidate sample selection. We do not look at the peak in the sense of the Z to dilepton tag & probe method, but we show with the plots in the appendix that the resulting sample is a very pure ttbar e, mu dilepton sample, which means that the "probe" electron should pass the electron trigger.

• Could you be using the orthogonal dataset approach? We understand that you are using an orthogonal, to your analysis region, set of events (eμ di-lepton events) and a reference trigger path (HLTMu50) to determine the trigger efficiency. Is this the case? If so, what dataset are you using to determine the trigger efficiency, SingleMuon or SingleElectron?
• We use the SingleMuon dataset and the muon trigger to measure the electron trigger efficiency. We changed the description in the AN to orthogonal sample to make it clearer and we added the information that we use the Single Muon dataset.

• Also, taking a look at the trigger efficiency and SF plots we observe that there are many bins with very large uncertainties.
• Is this a result of low stats? Have you tried further optimizing the SF binning?
• For events that fall in the empty bins what is the SF and corresponding uncertainty that you apply?
• Do these large electron trigger SF uncertainties have a large/ high ranking impact in your final fit?
• We received this comment from the eGamma POG, and they suggested to merge bins. This was done and is documented in the appendix

### JET/MET concerns

• You need to fill out the survey on jet/MET use before we can begin the review [1], to save yourself time please check that what you did is consistent with the recommendations for JER [2] and JEC [3] and explain if there is any variation, as that is what I will ask about.
• Filled out the survey and we conform that we have no departures from the recommendations

• L 184 : “pileup-hadrons” -> “pileup candidate” hadrons to be consistent with previous description on L 180 (same comment for L 254)
• made the correction and updated AN

• L 263 : Please explicitly state the JER version used as you did for JEC on L 261
• Summer16_25nsV1*(2016v3), Fall17_17Nov2017_V3_* (2017), Autumn18_V7_* (2018). AN has been updated

### BTV concerns

• Please make sure that you also apply nJets/HT-based corrections to your phase-space (omitting b-tagging requirements) as explained in our TWiki for the discriminator reshaping method [1]. Side note: If you are using UL samples, that effect might be negligible, we have not investigated this with UL. In the case of the Re-reco samples, these corrections have to be applied
• Yes, we are applying the 2D correction (nJets/HT) to all events that pass our events selection and reconstruction as instructed. This has been documented on an Appendix

• When using dedicated tagging SFs for your AK8 jets, remove the AK4 subjets of these jets from the collection of jets that is considered for deriving your AK4 b-tagging SFs.
• Yes, we don't take into account the AK4 subjets for deriving AK4 b-tagging SF's.

• In l. 194, you say you are using the btag shape reweighting method. The description of your systematic uncertainties in l. 370 then does not match your chosen reweighting method. For this method, there are in total 8 uncertainties that are applied to jets of all flavors. The recommended correlation scheme is detailed here [2].
• Yes, we are applying the btag shape reweighting method with all 8 uncertainties (cferr1, cferr2, hf, hfstats1. hfstats2, jes, lf, lfstats1, lfstats2) and taking into account the correlation. The AN has been updated with more details

• As soon as you obtained your results, we would like to have a look at the impact plot as well, to make sure that no unexpected behavior is observed for the b tagging nuisance parameters. Please consider filling in the TOP BTV questionnaire by then as well [3].
• We added an appendix with all the details and filled out the questionnaire in [3]

### EGamma concerns

questions comments from Alessia and Mohsen

*Can you confirm that the ID that you are using is indeed the recommended Fall17v2?

Yes. we are using the recommended Fall17v2 ID

• Can you comment on the procedure that you use to remove the isolation cut from the ID definition?
• Isolation can be applied separately from the other ID cuts and we do not apply any isolation requirement (track or calo-based) offline. We also use a trigger that does not include any isolation requirement. However, we realized that the SF is not centrally provided without isolation and remeasured it ourselves. AN was updated with the study and the SF were shown at the EGamma POG meeting on July 2nd.

• Are you applying the reconstruction scale factors? You only mention the trigger and ID scale factors, but you should also apply the reco ones as specified here : https://twiki.cern.ch/twiki/bin/viewauth/CMS/EgammaIDRecipesRun2
• yes, we are applying the reconstruction scale factor. AN has been updated to make this clear.

• About the electron trigger SFs, I see that you derive them yourselves, but they should be approved by the EGM POG first. In this case, if not already done, you should present the SFs in the Egamma Reco/Comm/HLT meeting and get green light from them.
• We presented our results on June 11th and they were approved. They did comment though that we need to measure the ID SF without isolation ourselves, which we did ane presented at the July 2nd meeting. They have been documented in the AN.

### MUON concerns

• you are using the high Pt ID, but this selection does not use the Particle-Flow algorithm. From this twiki page https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideMuonIdRun2 you can read: "The High-pT selection does not use the Particle-Flow algorithm. Please consider this option ONLY if you do not use the Particle-Flow event description in your analysis. If you do, start from the Loose (or Tight) ID and then consider possible addition (or removal) of further quality cuts." It is also true that your pT range is not that high, so using PF might be fine in your case, but we just would like to know whether you considered this or not.
• We use Particle-Flow for all objects except for the muon. This has been made clear in the object ID section of the AN

• It seems to us that you are not applying any ISO SFs. Since you don't have a standard ISO cut, the official ISO SFs are not suitable for your analysis and you should compute these SFs yourselves, then write this to the MUO POG.
• Our 2D cut is really a topological cut and not an isolation cut. It has been studied with 2016 data using a ttbar dilepton (e, mu) sample and the SF was consistent with 1. See AN-2015-107, Section 5.2. However, we ended up adding a 15% uncertainty as requested (see below).

• Is there a reason why you are not considering reconstruction SFs? Are they negligible? * reconstruction SFs are negligible but we are applying them. The AN has been updated.

• Are you applying any additional uncertainty to cover the phase space extrapolation (Zs-to-ttbar) in the SF computation? You can apply 0.5% per muon on the ISO component, following results in http://cms.cern.ch/iCMS/jsp/openfile.jsp?tp=draft&files=AN2018_210_v4.pdf. * We looked at the note you suggest and the 0.5% for muons is specifically for the extrapolation of the isolation component of the SF. Because we do not apply any isolation cut on our muon, we think this is not applicable. Also, we are applying extra 15% flat uncertainty to the 2D cut (our "isolation") which is already very conservative.

Comments from Federica, Sergio, George, and Clara

• About the SF error bars, they are unphysical and need to be correctly computed. “Error propagation” is quite vague and we want to make sure exactly how you compute them. Our recommendation is to use the TEfficiency class in ROOT https://root.cern.ch/doc/master/classTEfficiency.html, where the correct methods for these cases are implemented.
• I have been using TGraphAsymmErrors. If I look here : https://twiki.cern.ch/twiki/bin/view/CMS/DataMCComparison. TGraphAsymmErrors is a valid option to use. I have also checked by hand the errors and they are almost the same as what I get through TGraphAsymmErrors. Do you still think I need to switch to TEfficiency? Regarding the negative errors- they are what I am getting, from what I can see from the definition in TEfficiency, I dont think the negative error would be solved. How do you think I should handle this?
• follow up comment - Thanks for the checks! It is fine but we think you should modify the errors so they don't go below zero.

• You claim that the SF is negligible as the values are compatible with one but that is not true in all cases, see: dRmin_mu2_jet 2017 (fig 52 c), range 0.3, 0.4 by eye: MC = 0.55 with negligible uncertainty data = 0.70 \pm 0.05 ,SF = 1.27 \pm 0.10 (reported in fig 54 c)
• Not compatible with unity. What we meant is that it is very close. Do you propose using an overall 10% error on this? I feel that might be too much
• follow up comment- Even if you are not going to apply SF, we think an uncertainty should be added any way. You have very large error bars in the first bin so maybe the best strategy is to merge bins, but an extra uncertainty should be applied.
• We ended up applying a flat 15% uncertainty to cover the first bin.

• Is there a reason for the worse data/MC agreement in 2017 (fig. 50)?
• I have double checked this. There are no errors, 2016 and 2018 does appear to be slightly better.

• 4) Our suggestion for the mass plots, for example 51 a) b) was to change the width of the bins so they are more readable. On the other hand, the p_Trel plots which were good in AN-21-069_v4, now have very coarse binning (entries/25GeV, see fig 51 c) d) for example)
• I have fixed this. AN s now updated.

• It looks like data is systematically below MC in 2017 and this does not reflect into the efficiencies and SF (Figure 28 and 29 c-d). Could you check this? Maybe it is due to the fact that the range is very large. It could be useful to plot the efficiencies and SF with a better granularity in dR, maybe also add plots as a function of pT_rel.
• The plots in Figure 28 and 29 are the efficiency of the 2D cut on data and MC plotted separately and not the ration of Data/MC. The pT_rel cuts were not included initially as effectively, the 2D-cut can be seen as a cut on pT_rel, for events with deltaR<0.4, which means no cuts are applied for deltaR<0.4. Also added the pT_rel plots now.

• Fig 25,26 and 27 e and f -> are those identical??
• Yes, added both mu+jets plots by mistake, now fixed.

• Fig 15,26 and 27 a and b -> not optimal binning of the mass plots
• fixed.

• Fig 28: data point for dR>0.4 are not reported right? Maybe you can add a line to the AN saying it explicitly and adding the reason why.
• yes and done

• Fig 29: error bands are unphysical because they go to negative values.
• yes, I calculated the errors through error propagation. Should I just remove them?

• Fig 29: not easy to read, could you plot an horizontal line at 1?
• done

• How are jets defined for the deltaR(mu, jet) measurement.
• at least 2 AK4 jets are selected with pT>50 GeV and |eta|<2.4.

• Figure 44: Uncertainties have been reduced a lot since last version of the note. Can you explain why? These uncertainties are obtained by propagation of stat. only uncertainty, right? In the past there where negative values, and we agreed on correcting this, as it was unphysical but now they are much smaller than expected. As a consequence the SF is not compatible with 1 for the first bin (in all three years) anymore, so looking at this plots now we would indeed recommend to apply the SF. But please clarify the previous points first.
• If you remember, the first and the second bin had negative values. These values have been calculated using following: https://twiki.cern.ch/twiki/bin/view/CMS/DataMCComparison. They do end up having negative values. I modified the uncertainty on the first and the second bin to be truncated before it goes below 0 (I made both the positive and negative errors the same). So the uncertainties for the first two bins are smaller and the middle two bins are the same. The SF itself hasn't changed- do you suggest I show the actual positive uncertainty on it (which does make it compatible with 1). I thought I should treat both +/- uncertainty the same way. I have updated the plots to show the errors as is, in the positive side.

• In the systematic section it is not stated that an uncertainty is applied to cover for the 2D data/MC differences. The 5% value for this uncertainty is not well motivated by plots in section A.3. You also mentioned in the past that in previous versions of the analysis a 5% was applied, can you point us to the studies that motivated that 5%?
• This hasn't been implemented yet, but will be. We are rerunning the fit with some changes. I have updated the AN to include this in the systematics section. The paper was B2G-17-017. Since it was applied as a flat uncertainty, I think it got overlooked in the PAS as it doesn't matter in the fit. But if you look at the twiki for that analysis: https://twiki.cern.ch/twiki/bin/viewauth/CMS/B2G17017Review#Muon_Review_AN1 (last question in the muon section): comment: Lines 484-488: Do you apply the additional systematic uncertainties of 1% (ID) and 0.5% (trigger) recommended by the MUO POG [3]? answer: Yes, we apply the recommend muon POG ID and trigger uncertainties as 1 sigma shape systematics. They are fit to their final values during the MLE process. We will double them in light of the fact that we don't use the High-pt muon ID and the loose trk relative isolation. So that comes to ~3%. I had suggested applying a flat 5% to be conservative but have changed to 15% following your recommentation

## CWR comments to the paper (TOP-21-XXX-paper-vxx)

comments from Jan and Matteo paper draft (23rd Nov)

• Intro: when explaining the difference in tt production between the Tevatron and the LHC, I would mention that qq annihilation is also relevant at the LHC for high Mtt, which is the topology you are looking at. Also, this is one of the reason why you would expect higher sensitivity to Ac in the boosted regime, not only in terms of BSM, but also in QCD. This should be better highlighted.
• We added a sentence "Since the relative contribution of valence quarks increases at high momentum transfer~\cite{PDF4LHC}, we expect that measuring A_C in a highly boosted ttbar sample will lead to a more stringent probe of quantum chromodynamics (QCD) predictions and higher sensitivity to BSM physics processes that might alter the asymmetry~\cite{arXiv:1109.3710}."

• When discussing BSM contributions, add references and optionally even mention some models explicitly, e.g. W/Z’, colour triplets. I see now. you mention some models later, but it’s probably better already here. Then you also avoid switching from a theory paragraph to describing the Tevatron measurements, to BSM theory, to introducing your measurement.
• We moved this up and also added more deferences.

• L. 66: I think soft drop and n-subjettiness can be mentioned explicitly here.
• as we do not later mention these two variable any more, we think that the wordy description we have now is enough and the variables themselves do not need to be defined with a special name.

• L. 69: It is not clear which b-tagger are you using in the resolved regime from the text here. Please mention DeepJet+refs explicitly
• Added E. Bols et al 2020 JINST 15 P12012, let us know if this is the one you had in mind.

• L. 86ff will need references
• Done

• When explaining the categories, please mention that the t and W tags are exclusive, as you do in the AN (unless we missed it in the paper)
• Added this information when we first introduced top and W tag around like 65.

• L. 121ff: please also mention the modelling of statistical MC uncertainties (opt. with BB-lite), in particular since they play a major role.
• Done

• Fig1: All labels, legends etc must be increased in size. The simple rule is: A capital letter in the figure should be at least as big as a small letter in the caption. I would also propose to change the colours, in particular the light green
• We are working with the LE on the figures for the paper

• Eq.2 missing definition of “k”. If it refers to the channel, you can add it in the previous line “For each channel k in our analysis”
• This was changed after we redefined the channels

• L142: I would use the word “priors” rather than “constraints”. I don’t find it wrong from the language point of view, but “constraint” is usually intended as the post-fit constraint
• Changed

• Please fix Eq 4 (and its reference in one of the following lines)
• Fixed

• I am not sure that L173-174 are relevant. Would it make any difference to set them to different values? As long as no prior is associated to them (i.e. they are POI) this shouldn’t be the case
• We are indeed setting them as POI and the value will not not matter for the unblinded results. With the data blined, if we were not to set the values the errors would not make sense.

• For the luminosity, I understand from the AN that you use the correct correlations between the years. It would be good to mention around line 180 that the quoted numbers are not uncorrelated (and maybe provide an estimate of the correlation). You can refer to other Run2 papers in case
• Done

• Table 3: I would put the first row at the end, and explicitly write “combined” instead of > 750 GeV. Otherwise it would give the wrong impression that it refers to an inclusive measurement of Ac in the kinematic region Mtt > 750 GeV
• This has changed because of our re-definition of the channels

Things that are missing in the paper draft

• Some control distribution to appreciate data/MC agreement
• We added mass and Delta|y| plots

• The pre-fit and/or post-fit plots of the distributions used in the fit
• We added both pre and post-fit plots for Delta|y| in each channel to the paper

• The impact of the various systematic uncertainties on the final result
• This was added after unblinding

## Comments to the pre-approval talk (AN-2021-069)

• Please ensure that all of the object reviews are fully satisfied before unblinding
• only stat did not respond but Combine approved the Data cards and error implementation

• Please improve the MC statistics if possible, e.g. explore the boosted samples
• We are running on the 2017 boosted samples and will see how much they help.

• W+jet components are not too small on page 9 but nearly invisible on page 10, while the selections should be the same. Please check/verify.
• The difference between page 9 and 10 is that we applied b-tagging, which significantly reduces the W+jets background.

• Demonstrate/cross check if the asymmetry of the background does not course any issue; at least checked with MC
• Theoretically, we do not expect any asymmetry on the backgrounds. We cannot check this at generator level because we need to define the leptonic and the hadronic top with our reconstructions for the background samples that have no true ttbar decay. We did that and the measured asymmetry for the main backgrounds (W+jets and single top) is zero. The main background is actually ttbar dileptons for which a true asymmetry exists. But we do include ttbar dilepton events that pass our selection and reconstruction as signal, i.e. they are not part of the background which indeed has zero Ac.

Topic attachments
I Attachment History Action Size Date Who Comment
pdf Ac_750_full-approvalVersion.pdf r1 manage 48.5 K 2022-04-05 - 16:13 CeciliaGerber Impacts for M750 shown at approval talk
pdf Ac_750_full-fixedlumi.pdf r1 manage 47.3 K 2022-03-14 - 22:32 CeciliaGerber Impacts when fixing luminosity to the nominal value as a test
pdf Ac_750_full-new.pdf r1 manage 49.2 K 2022-03-18 - 18:15 CeciliaGerber Impact after separating the 2D SF nuisances
pdf Ac_750_full.pdf r3 r2 r1 manage 49.2 K 2022-03-18 - 18:14 CeciliaGerber Impact after separating the 2D SF nuisances
pdf Ac_900_full.pdf r1 manage 44.6 K 2022-02-24 - 18:48 HugoAlbertoBecerrilGonzalez
png Ak4_j1_phi.png r1 manage 22.6 K 2022-02-11 - 22:44 TitasRoy
pdf ARC-Authors_Meeting_April_6.pdf r1 manage 4027.2 K 2022-05-10 - 22:36 CeciliaGerber ARC-Authors meeting slides April 6, 2022
pdf ARC-Authors_Meeting_Feb_15.pdf r1 manage 2329.9 K 2022-05-10 - 22:35 CeciliaGerber ARC-Authors Meeting Slides Feb 15, 2022
png correlations.png r1 manage 1229.3 K 2021-07-11 - 23:06 CeciliaGerber hdamp and toppT systematics
pdf hdamp-study-May10.pdf r2 r1 manage 1409.9 K 2022-05-12 - 22:42 HugoAlbertoBecerrilGonzalez
pptx hdamp-study-May10.pptx r1 manage 2194.2 K 2022-05-12 - 22:40 HugoAlbertoBecerrilGonzalez
pdf hdamp-study-May12.pdf r1 manage 1410.4 K 2022-05-12 - 22:43 HugoAlbertoBecerrilGonzalez
png impacts_all.png r1 manage 244.5 K 2021-07-12 - 19:01 TitasRoy
png impacts_boosted.png r1 manage 242.9 K 2021-07-12 - 19:17 TitasRoy
png impacts_resolved.png r1 manage 244.6 K 2021-07-12 - 19:17 TitasRoy
png impacts_semiresolved.png r1 manage 251.3 K 2021-07-12 - 19:08 TitasRoy
png MET_phi.png r1 manage 22.6 K 2022-02-11 - 22:45 TitasRoy
png pT-had-electron-2018.png r1 manage 65.7 K 2021-07-09 - 01:18 CeciliaGerber Hadronic Top pT for the 2018 electron signal sample
png pT-had-muon-2018.png r1 manage 64.3 K 2021-07-09 - 01:14 CeciliaGerber Hadronic top pT in the 2018 muon signal sample
png pT-lep-electron-2018.png r1 manage 60.9 K 2021-07-09 - 01:19 CeciliaGerber Leptonic Top pT for the 2018 electron signal sample
png pT-lep-muon-2018.png r1 manage 64.4 K 2021-07-09 - 01:16 CeciliaGerber Leptonic Top pT for 2018 muons signal sample
pdf r_1.pdf r1 manage 39.1 K 2022-02-24 - 18:48 HugoAlbertoBecerrilGonzalez
pdf r_2.pdf r1 manage 38.9 K 2022-02-24 - 18:48 HugoAlbertoBecerrilGonzalez
jpg systematics.jpg r1 manage 832.8 K 2021-07-11 - 22:52 CeciliaGerber files
jpeg systemtics2.jpeg r1 manage 263.3 K 2021-07-11 - 23:06 CeciliaGerber hdamp and toppT systematics
png SystVariation_Ttbar_1hdamp_elec-1.png r1 manage 40.9 K 2021-11-03 - 17:59 HugoAlbertoBecerrilGonzalez
pdf SystVariation_Ttbar_1hdamp_elec.pdf r1 manage 14.3 K 2021-10-20 - 19:45 HugoAlbertoBecerrilGonzalez
png SystVariation_Ttbar_1hdamp_muon-1.png r1 manage 42.7 K 2021-11-03 - 17:59 HugoAlbertoBecerrilGonzalez
pdf SystVariation_Ttbar_1hdamp_muon.pdf r1 manage 14.4 K 2021-10-20 - 19:45 HugoAlbertoBecerrilGonzalez
png SystVariation_Ttbar_2hdamp_elec-1.png r1 manage 41.1 K 2021-11-03 - 17:59 HugoAlbertoBecerrilGonzalez
pdf SystVariation_Ttbar_2hdamp_elec.pdf r1 manage 14.3 K 2021-10-20 - 19:45 HugoAlbertoBecerrilGonzalez
png SystVariation_Ttbar_2hdamp_muon-1.png r1 manage 43.0 K 2021-11-03 - 17:59 HugoAlbertoBecerrilGonzalez
pdf SystVariation_Ttbar_2hdamp_muon.pdf r1 manage 14.4 K 2021-10-20 - 19:45 HugoAlbertoBecerrilGonzalez
png ttbarpTBoosted.png r1 manage 140.4 K 2021-07-11 - 23:17 CeciliaGerber ttbar pT
png ttbarpTResolved.png r1 manage 256.0 K 2021-07-11 - 23:17 CeciliaGerber ttbar pT
png ttbarpTSemiresolved.png r1 manage 181.3 K 2021-07-11 - 23:18 CeciliaGerber ttbar pT
Topic revision: r171 - 2022-07-30 - CeciliaGerber

Webs

Welcome Guest

 Cern Search TWiki Search Google Search Main All webs
Copyright &© 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback