Paper QA for TOP-13-001

Documentation

AN 2012/248 documents the analysis as it was in the PAS.

AN 2014-001 documents the current analysis; to save space, several details are not repeated and the reader is referred to AN 2012/248.

Previous documentation

Color code

  • Answers in blue are considered final.
  • Answers in orange require further communication between the questioner and the authors.
  • Answers in red still need work from our side.

Paper v0 (AN v3) comments

19/01/15, by Jeremy Andrea

  • Title : why is it changed with respect to the PAS ? Why to remove "top polarization" (I think that in the framework of the SM, top polarization has to appear in the title) ? Is it because \alpha_l can be = 1 in the presence of new physics ? There are no BSM interpetation of the results, I guess because TopFit miss correlations. However, one could always estimate the compatibility with the SM prediction and even set limits on "any models that would induce a deviation of A_l" ?
    • Top polarization is already an interpretation of the asymmetry measurement. Moreover, it is trivial to go from A_l to P if one assumes that the spin analyzing factor of the lepton, alpha_l, has the SM value. At the time of the PAS it sounded less trivial because we also planned to have a concluding section of BSM interpretation with TopFit where we made no assumption on alpha_l. This section, as you may remember, was taken off the PAS eventually. However, for visibility of the paper it was decided during the paper-talk to keep it in the title. The situation is comparable to the ST t-channel. x-sec. measurement which also sets limits on |Vtb*flv|
  • line 129 : is there any specific comment to make on the jet-energy calibration for jet with |eta| >2.5 ?
    • Can you please elaborate a bit on that? We use standard calibration, which is provided also for forward jets.
  • line 152 : could you define "isotropy" or at least give a reference ?
    • The exact definition is here. In words, the algorithm looks for the directions in r-phi that minimize and maximize the sum of the inner products in 2D between their unit vectors and the vectors of the analysis objects. Isotropy is defined as the relative difference between the maximum and minimum sum of these inner products. The value is 1 for spherical and 0 linear events in r-phi. We added a concise sentence to try to explain that in the new paper draft. We tried hard to figure out the original reference, but nobody in CMS seems to remember where this Physics Tools variable came from. We got advice from Roger Wolf, who introduced this variable into CMS Physics Tools from HERA/H1 experience but does not manage to find a reference either, to just describe this variable in our paper, with no reference. Roger also blessed our description of this variable in the new draft.
  • After line 156 : I think it would be interesting at that point to should the data/MC comparisons for BDT_QCD, and say that you validated the input variables.
    • Good idea. Added.
  • Figure 1 : the cos theta* distribution was not described before the plot is referenced in the text.
    • Thanks for noticing, we moved it after its definition.
  • Line 176 : "well modeled by the MC simulation" : in signal and background enriched regions ?
    • Because of blinding we verified it only in control regions, and in 2j1t for negative values of the BDT
  • Line 193 : you should probably add a reference there on anomalous couplings, and maybe quantify the level of deviation that could be expected. Are you precise enough to probe new physics ?
    • TopFit limits are not in the scope of this analysis anymore, for the reasons discussed at the paper-talk and in several occasions before.
    • Deviations from the SM polarization may be generated through additional particles interacting with the top quark (in loop corrections or with invisible decays). Such a scenario is common in BSM models like SUSY or technicolor models. However, a theoretical calculation linking specific model parameter to the single top t-channel polarization is not available. The corresponding operators and there influence on the spin-analyzing power are however given in the paper "JAAS, A minimal set of top anomalous couplings" which is quoted in the introduction.
  • Line 202 : what is the anti-QCD BDT discriminant ? This is not clear reading the text. Actually the whole 6.1 paragraph is rather obscure to me.
    • Indeed the text was a bit cumbersome. Given that the symbol BDT_QCD had been introduced earlier, now we use it also here. Please let us know if that paragraph is still obscure.
  • Line 221-222 : is there any explanation of why the W+jets MC generator has a problem ? Maybe some more information about the issue might help the reader. More generally, the section 6.2 has to be written with a lot of care (and the procedure has to be justified) to avoid any mis-understanding or mis-judgment of what is done.
    • This sub-section has now been rewritten, also adding some more information to justify our choice.
  • figure 2 : clearly there is a problem in the data/MC agreement? Data points are actually overlapping almost perfectly the W+jets backgrounds. Is it expected ? Could you explain ?
    • At the time of the PAS we had observed that one could fix the residual discrepancy by attributing it entirely to the W + light jets component (which is already known to be poorly modeled by the aforementioned cosTheta issue) and scaling this component up by the data-MC difference. Given that this had almost no effect on the result (again, because W + light jets is an insignificant component of W+jets in our signal region), this is essentially mostly a cosmetic issue. However, given your comment we understand that it is better to do something. Instead of the old ad-hoc scaling we normalize the non-QCD component to the result of the QCD fit.
  • Figure 3 : the data/MC agreement is correct, but not so good neither, especially in the tail where differences seem to reach a 15-20% level. That is probably not a statistical variation because same effect is seen on the muon and the electron channels. Was all the input variables to the BDT checked in this region ? Is there any discrepancies that could explain the BDT shape slight disagreement ?
    • data/MC correlations by comparing input variables separated in intervals for signal BDT (see AN) has been checked. No mismodeling observed apart from an overall slight deviation in MET for both channels.
  • Comparing figure 3 and 4 : I don't understand why the number of entries is larger in the 3j1t region compared to the 3j2t region? Is the selection ==1tag or >=1 tag ?
    • The 3j1t region is defined by ==1 tag. At least part of the difference can be explained by the fact that b-tagged jets need to be in the tracker |eta|<2.4.
  • Lines 289 and 291 : to me, scale variation is also a signal modeling uncertainty (at least in my mind). So I think that line 289 "signal modeling" has to be replaced by something different, and a justification of the recipe as to be provided (why to compare Powheg and CompHep ? LO vs NLO ? There would be some double counting with the scale variation then ?).
    • What about "Generator modeling"? We also added an explanation of what is different in Comphep.
  • line 302 : I think the recipe has changed. See https://twiki.cern.ch/twiki/bin/view/CMS/TopSystematics#pt_top_Reweighting
    • We checked with Andreas Meyer. The updated recipe (which requires to not apply it by default, but only as a syst. variation) has to be intended for cases where ttbar is the signal, while in cases where it is a large background, like ours, it makes sense to apply. If we do not, we would give the impression that we have a poor understanding of the background, while in fact we know that most discrepancies all come from the same cause and can be made disappear by the reweighting. Andreas and Andrea agreed on making it more explicit on the wiki of the recommendation, and the wiki has been updated.
  • Line 316 : is there any reference you could add there ?
    • At the time of the PAS there was no proper reference, in the meantime this EGM paper appeared. Do you think it is appropriate to cite? Anyway, we are not aware of any similar paper from the Muon POG. What are other analyses doing when referring to those scale factor uncertainties? It was suggested by Andreas to defer that to later stages of review, letting the Publication Committee give us advice.
  • Line 329 : also there maybe add a reference to the BTV-POG numbers
    • New
  • Line 334 : one should probably try to justify the uncertainties taken for background normalization (well I know that's not easy...). For ttbar background, given the high level of precision we have from cross-section measurement, I think that 50% is by fat too large. 10% would already be conservative.
    • 50% is applied only to minor processes which are added to the fit components - Z +jets for the W/Z+jets component and the single-top tW and s-channel for the top component. Moved QCD yield uncertainty under a separate bullet as it is a separate component not part of anything else.
  • line 342 : maybe you could had some words about how the break-down is determined.
    • Do you mean something like saying that we change one systematic at the time?
  • Table 3 :
    • What "TUnflod" and "analytic" are referring to ?
    • It means unfolding either using TUnfold or an analytic 2-bin approach. We expect smaller uncertainties with TUnfold (=default). Anyway, in the new draft we decided to remove the "analytic" column from the paper (it stays in the AN) as it is just a cross-check result.
    • I don't understand the differences between the muon and electron channels, especial for theoretical uncertainties like signal and background modeling, scale, matching etc... I naively guess these should be close when comparing muon and electron channels.
    • They appear to be dominated by MC statistics limitation of the systematics samples. We had the same in the PAS, and we performed checks that indicated that this was the explanation.
    • why the ttbar matching uncertainty is so high ? It is usually smaller compared to ttbar scale ?
    • It appears to be dominated by MC statistics limitation of the systematics samples.
    • why to give so much digits ? That makes the reading of the table a bit more difficult (and also giving % might help).
    • Table style has be updated. Quoting now deltaA*10^3 using %3.1f format
    • where are the uncertainties on the background normalization ?
    • We propagate the fit uncertainties from the ML-fit using the signal BDT through the unfolding. Additional uncertainties are taken into account by varying each single background process before merging it into the ML-fit components.
  • Figure 5 : it is difficult to compare the 2 distributions as they have different normalization. Why not to provide a differential cross-section instead ? That should be rather easy, and would make muon vs electron comparisons easier. Also, the theoretical community would probably found more useful a differential cross-section?
    • Thanks for the suggestion. In the updated draft we are providing a differential cross section.
  • Equations 4,5,6,7,8,9 => reduce the number of digits ?
    • Done
  • Equations 4-5, that is still not clear to me why the systematic is so much higher for the electron channel (see my comment above).
    • In some cases it is an artifact of limited MC statistics in the systematics channel, but for example all QCD-related systematics are expected to be much larger in the electron channel, where QCD contamination is much larger.
  • Line 347 : I would add the uncertainty of the pohweg prediction, like scale and PDF (that you already have probably).
  • unfoled combined using scale uncertainty from MCunfoled combined using scale uncertainty from ME-based reweighting
  • PDF uncertainty completely negligible, Q-scale uncertainty when taken from dedicated MC sample also negligible
  • Equation 7-8 : are the numbers really the same ? down to the 5th digit ?
    • Yes. These are pseudo-data based on a MC with that asymmetry value.
  • I think one could add easily a "kind of" BSM interpretation, as an, estimation if the compatibility with the SM, that can be converted into : no excess found and generic limits. If you split P_l^{meas.} = P_l^{SM}+ P_l^{NP} you could eventually constrain "P_l^{NP}" ? Would it make sense (I actually don't know)? That is probably not super-great things but a minimum I would say. At least it is easy to do so doesn't cost a lot.
    • Thanks for the suggestion. Done. The p-value is 50 per cent.

20/01/15, by Oliver Gutsche

  • why do the BDT plots quote 15.3 fb-1 for muons and 16.9 fb-1 for electrons and the unfolded plots 16.9 fb-1 for muons and 18.9 fb-1 for electrons, while we quote results for 20 fb-1.
    • Thanks for pointing that out, you made us realize that we mistakenly uploaded the wrong version of the unfolded plots. So, all plots should say 15.3 for muons and 16.9 for electrons. We just fixed that on svn and you will see the new plots during our paper talk. The new plot is essentially identical in the muon channel, while the uncertainty is slightly better in the electron channel (not because of the extra statistics but because a different working point is used with respect to the obsolete plot). We are quoting 20/fb everywhere in the text because we will add the missing lumi sections in the next draft; our incomplete data statistics was due to some technical issues with CRAB that we solved in the meantime.
  • you use a rather old result as reference for the unfolding procedure ([37]) using 1 fb-1 of 7 TeV data. Is there a newer reference, maybe with 8 TeV data that could be used?
    • The corresponding 8 TeV analysis is, so far, public only as a PAS (the CMS editorial rules prevent from citing a physics PAS from a paper). The corresponding paper is currently under review, we may update this reference if they get published before us. But please notice that the unfolding procedure itself is the very same in the 7 and 8 TeV versions of this analysis. So one could argue that the earliest reference should be cited anyway...

21/01/15, by Andreas Meyer

Main comments:

  • I would propose to also publish the differential cross section measurement in full detail, i.e. combine the two channels to one final result of the cross section and give full table of uncertainties in each bin of the measurement.
    • See answer to the same comment by Jeremy above.

  • I doubt that the uncertainty unfolding bias is justified. Any significant deviations of the data from SM would be visible before unfolding. Assuming absence of this, no additional systematic should be applied. If there was something, it could be applied. Either way, no additional unc.
    • The unfolding bias stems from testing the complete analysis strategy (selection, BDTs, unfolding) by injecting Comhep samples with anomalous couplings as pseudo-data.
    • In this first draft, the response matrix predicted by Powheg was used so that the bias contains also the generator difference Powheg <-> Comphep. To quote only the actual bias w/o differences in the response matrices between the two generators, the Comphep response matrix is used during the bias estimation.

  • The selection of figures was not fully obvious to me. Do we need these two distributions for all possible permutations of channels and categories? smile
    • After further discussion with you and Jeremy we agree on removing the 2j0t control region, as it is not used in the fit and its features are complicated to explain. We also removed 3j1t as it is not used in the fit.

Mostly editorial suggestions:

  • title: why "spin asymmetry" and not polarization or helicity?
    • see above a similar comment by Jeremy

  • line 13: what does "\approx" mean? Are there NLO effects that dilute the polarization?
    • Yes. NLO calculations predict a slightly lower polarization.

  • line 30 and following: mention: reconstruction/boost into top rest frame ?
    • reconstruction of top quark candidate mentioned

  • somehow the discussion of BDT appears twice, once in section 4 and once in 6. Could try to merge discussion, by moving details (and figures) from section 4 to section 6.
    • Section 4 introduces the two BDTs as selection tools, while the BDT_QCD is mentioned in section 6.1 in the context of the fit that is performed to extract the QCD rate. Please let us know if we misunderstood your comment.
  • figures: show figure(s) about BDT_QCD, illustrating a bit more details about the QCD determination ?
    • This was also requested by Jeremy. Done.
  • do we need all costheta*_l distributions for all control regions? (I could imagine only showing costheta*_l once before and once after cuts for the signal region.
    • We removed the 2j0t region. We would prefer to keep the other cosTheta plots, to show that we have a good understanding of this variable in all the regions used in the fit.

  • line 262: "parton-level", give a bit more explanation exactly how the lepton is defined. BTW, "parton-level" seems jargon here, in particular as this is a lepton (not a parton).
    • Changed to "Monte Carlo truth level" and added a paragraph to explain how the relevant objects are defined.

  • line 271: "based on 2nd derivatives" overly telegram style. Use one more sentence to distinguish between the unfolding method and the choice of regularization criterion.
    • done

  • line 282: templates and response matrices (?)
    • done

  • line 297: 1 GeV, actually does not harm to give typical uncertainties
    • added central value and uncertainty

  • table 3: too many significant digits ?
    • table style updated

  • section 10: propose to first discuss the measurement of the cross section (e,mu and combined) normalized or absolute)? Give full table of cross section results (in appendix): would like to have a table of results (values and uncertainties). In case a absolute cross section is measured, would like to see systematics separately for shape (bin-to-bin) and global normalization uncs.
    • Added to AN for normalized differential measurement
    • Is there a consensus on making normalized cross section a major output of this paper? That was not our initial intention. Please let us know what you think.

  • figure 5 cosmetics: propose to show data as bullets with error bars with inner statistical and outer total unc.. For a
    • improved plot style

  • line 348: combination done at cross section level or at asymmetry level ?
    • Now it is at cross section level. (For the PAS it was at asymmetry level.)

type A:

  • abstract:
    • suggest: A measurement is presented ... (prefer passive style, as the paper really does nothing
    • Done
    • line 3: defined -> selected
    • Done
    • line 5: infer -> measured
    • Done

  • line 6: "free quark" really ? I'd prefer "decays before hadronization"
  • Done
  • line 10: "retain memory of its spin" -> "angular distributions of the decay products to follow the spin structure of the spin configurations in production and decay of the top quark" ?
  • Done
  • line 14: "stays" -> "remains"
  • Done
  • line 16: "so far" -> "previously"
  • Done
  • line 17: "attempted" -> "performed"
  • Done
  • line 21: suggest to remove colon (?)
  • Removed
  • line 27: "is in general" -> "can be"
  • Done
  • line 30: "this note documents": note->paper, but again, the paper really does nothing smile
  • Sentence has been rewritten to "An analysis of the top-quark spin asymmetry, measured in $t$-channel single top-quark events with one isolated lepton (muon or electron) in the final state, is documented in this paper."
  • line 30: THE top-quark spin (?)
  • Done (see above)
  • line 31: "in the muon... channels" -> "measured in single top-quark events with one isolated lepton (muon or electron) in the final state."
  • Done (see above)
  • line 87: remove "their latest"
  • Removed
  • line 105: present IN THE DETECTOR
  • Done (see above)
  • line 106: typo "althought"
  • Fixed (see above)
  • line 148: forward reference to description of BDT training
  • Sorry, I am not sure I understand. BDT training is never described in this paper draft, or is it?
  • line 179: how about "Small corrections are applied to the simulation where necessary in order to optimize the descriptions of the detector response and event yields to those observed in the data." or similar.
  • Changed to: "Small corrections are applied to the simulation where necessary in order to account for known differences with respect to data."
  • line 218: add blank before "This"
  • Fixed (see above)
  • line 231: by THE CHOICE of b and c quark masses (?)
  • We would like to keep as it is, as what we found is that even 0 mass gives the same distribution. So, having or not having a mass (as implied by the current phrasing) makes no difference.
  • line 236: "as shown in Figures 3 and 4, respectively"
  • Done
  • line 237: fit described in section 7.
  • Done
  • line 246: prefer "except" instead of "but"
  • Done
  • line 248: of -> described in
  • Done
  • line 262: deployed -> used
  • Done
  • line 262: infer -> measure
  • Done
  • line 293: scales (plural)
  • Done

10/02/15, by Jeremy Andrea, based on AN-v3 this time

  • could you please specify explicitly which dataset you are using : prompt reco or rereco. My understanding is that rereco is used, and it thus kind of "justifies" the unblinding.
    • Yes, we are using rereco data. However, this was already done in the previous analysis
  • line 112 : out of curiosity , why only the muon channel uses the mT distribution for the QCD fit ?
    • Historical reasons: we had a shape discrepancy in the electron channel that we did not understand. Now after our latest reprocessing this discrepancy magically disappeared, but we did not repeat this part of the check. We are using the anti-QCD BDT anyway in our analysis as default, so this part of the note is just about checking with other methods.
  • line 124/125 : could you please define the /thrust/ and /aplanarity/ ?
    • Aplanarity is defined as 1.5*q1 where 0<=q1<=q2<=q3 are the eigenvalues of the momentum tensor sum{p_j[a]*p_j[b]}/sum{p_j**2} normalized to 1. Return values are 0.5 for spherical and 0 for plane and linear events.
    • Thrust is defined here.
    • Both definitions have been added to the AN. Anyway, these two variables are not used in the end.
  • Figure 1 : could you please show the input variable to the BDT for the muon channel as well ?
    • It is actually shown on the next page from the electron channel input variables, but the figure caption is off the page. Fixed in the AN now.
  • line 163 : naively I would thought that the weight of a single variable should be estimated by : 1) excluding the variable under study, 2) retrain and estimate the discriminating power, 3) compare with the training that includes all the variables. Is it was you have done ? It doesn't seem so ?
    • Basically, yes. Only with the addition that once we have selected the best variable, the second best variable is selected by repeating the procedure with the best variable excluded from "all variables", the third best variable is selected by repeating the procedure with two of the best variables removed from selection of "all variables" and so on.
  • Section 3.3 : there are not results for the cross-checks. Will they be added soon ?
    • Yes, they will be included in the next version of th AN
  • Table 6, 7 : are the undertainties on the backgrounds containing only MC statistical ? Or this is the global uncertainty from the fit ? If this is the global uncertainty, that's hard to believe that you have less that 2% of uncertainty on all the backgrounds.
    • The tables were accidentally ignoring the MC statistical uncertainty, thank you for pointing that out. And additionally, the yield table was actually for a different BDT cut than the one shown in the caption. Will be fixed in the new AN version.
  • Figures 17-61 : there is a clear issue with the 2j0t region, enriched in W+jets. The data/MC agreements are relatively bad : first there is a clear normalization problem, second the shapes are badly described. Regarding this issue, I have few comments/questions :
    • did you redone the same plots after correcting the W+jets the way you did it for cos theta* ?
      • A section comparing the MG/sherpa reweighting of W+jets has been added to the AN
    • the mWT discrepancy can be solved by letting float the flavour composition of the W+jets sample : W+b, W+c and W+l. Are you using 3 different templates or a single one with a fixed flavour composition ?
      • Single one with a fixed flavour composition. We had tried in the past to let all of them float in the fit, but then the fit did not converge, because it has no handles to separate them and therefore there are too many degeneracies between the parameters. Please note, however, that the flavor fractions are varied as external systematics.
    • Also, I learned recently that the some SMP analyses are reweighting the pt of the W boson in madgraph (they extracted the weights in a control region enriched in W+jets). Is it something that you tested ?
    • The effect on fit results is shown [[https://indico.cern.ch/event/384131/contribution/0/material/slides/0.pdf][here].
    • More generally, I think you already quantified the effect of a bad description of the W+jets sample : could you remind what it the effect on the BDT and on the total uncertainty ? My understanding is that it was small. Could you confirm ? (sorry to ask again).
    • Yes, you remember correctly, it was discusses at length before the PAS: it is small and the reason is that the discrepancy (or at least the difference between Sherpa and Madgraph) seems to come mostly from W+gluons and W+light quarks, which are dominant in 0-tag but very small in 1-tag. Also to be noted that W+light flavor is not fitted, normalization is taken from lumi.
  • Figure 54, 56, 57 : the variables related to m(HFS) (by the way could you define it ?) clearly show a slop in the data/MC ratio. How bad is this ? Is it well covered by uncertainties ? How does it affect the total uncertainty ?
    • HFS (for Hadron Final State) here is defined just as the vectorial sum of the identified jets in the event (just two in the 2jXt categories). This variable, as well as all other discrepant variables, has not been included in the MVA (we use HT=(p_lepton+p_jet)T) and therefore we didn't investigate it further, as its effect on the analysis is null. We chose to illustrate the unused variables in the AN anyway, as they were part of the enlarged pool of variables that our physics intuition had originally suggested us to include.
  • section 8.2.1 : I have to admit that I don't understand exactly what is presented in this section. In particuler the definition of the variance V^(b) after equation 5. First why the background uncertainty can be factorized ? They should not be the same in the two bins ? I guess the off-diagonal terms are simply b1b2. Also the correlations between the two bins are taken as 100 % ? Why ?
    • The section presents an analytical description of perfoming unregularized 2-bin unfolding with background subtraction & uncertainty propagation. For that, one has to solve: data=response*true+backgrounds. The data comes with Poisson uncertainties (eq. 2). The background (taken from MC) has a normalization uncertainty from the ML-fit. The normalization uncertainty yields a 100% correlation between the two background bins. For simplicity, only 1 background w/o an additional uncertainty from limited-MC is taken into account for demonstration purposes only. When performing the 2-bin analytical unfolding, a straight forward generalization of these equations (accounting for more backgrounds and including the effect of limited-MC statistics) is used.

25/02/15, by Oliver Gutsche, based on AN-v3 this time

  • section 3.1.1.:
    • suggestion to add statement at the beginning that the full set of variables listed will be reduced in an optimization step
      • Done
    • inconsistency between line 128 and 130? lepton pt is in the initial list of variables but then muon pt is not used.
      • Corrected
    • mark final selected properties more clearly to guide the reader
      • Done
  • Yellow in Figure 6 is very hard to see, also who plot is not easily readable.
    • Fixed
  • line 266: have s-hat and H_T been defined previously? Couldn't find the definition.
    • They are defined in the list in the next paragraph, added a sentence saying that
  • table 10: the analytic method cross check is mentioned here. I wonder if we need to mention it in the paper?
    • Added a paragraph at the end of the Unfolding section of the paper, mentioning the method but giving no numerical details.
  • Figure 110: I very much like this figure showing the systematic uncertainties. I suggest to approve a version for later usage. Comments to the figure:
    • We will keep in mind to request its approval as "Supplementary Material"
    • why is the 2-bin analytical method W+jets matching systematic so much bigger than the electron-channel unfolding method?
    • same for top quark mass systematics comparing 2-bin analytical and muon unfolding?
      • In general, multi-bin unfolding with TUnfold allows to extract the asymmetry using a chi2-based fit to account for the bin-by-bin correlations and different uncertainties per bin correctly (see AN "Unfolding/Estimation of the asymmetry" for details). This procedure is less affected by fluctuations through limited MC.
    • why is the t-channel systematics asymmetric between electron and muon unfolding?
      • When applying the ME-reweighting for t-channel Q-scale uncertainty, its asymmetric impact on muon/electron channel is gone. Hence we conclude that this was because of the limited MC statistics of the scale-variated samples.

20/02/15, conditions for unblinding

  • re-produce all the control plots and numbers with the full 8 TeV data statistics for both the electron and muon channels,
    • Done

  • include the Sherpa reshaping and validate the data/MC agreement in the 2j0t region,
    • Done

  • make sure that the large difference of systematics between electron and muon channels are indeed due to lack of MC statistics (signal modelling, t-channel scale, top-quark mass, W+jets matching etc...).
    • We believe that this is demonstrated in the Q-scale case by the success of the Q-reweighting procedure. In the case of other systematics, a proof would only come from more MC...

Status report, 03/03/2015

A presentation has been given with the updates before unblinding.

Unblinding, 13/03/2015

Comments by Jeremy, 13/03/2015

  • Slide 3 : for the 3j1t there is a bit of tension between the data and the MC. Could you please make sure it is well covered by systematics ? Why the Chi2 and KS are exactly 0 ? Is it a normalized chi2 or just the chi2 ? Could such a discrepancy pop-up in the signal region ?

  • Slide 22 : There seem to be a kind of slope in data for both the muon and the electron channel (it was electron only in the past, right), however it is much more pronounced for the electron channel. So I have some questions there :
    • Could you give the statistical uncertainty on A for the 2 channels separately ?
    • What's the level of compatibility between the electron and muon channels ? It seems that the electron channel is more than 2 sigma away from the muon channel. Why ?
    • The electron channel is in fact compatible with 0. I'm nto sure what e could conclude out of this.
    • Could you compare to a different generator ? Or to a theoretical calculation directly (if existing) ? On that plot, it would be nice to see also LO vs NLO vs NNLO (that last one is not existing probably).

  • Slide 23 : I'm not sure what the comment on the right side means.

  • Slide 24 : there might be some additional work to be done on the theory side; like is higher order corrections supposed to cover for the deviation (seem to large to me), which NP can contribute and with such a high amplitude ? etc... etc...

  • Is there any correlation between the cos theta* and the BDT cut ? Could you move the BDT cut by 10-20% and see how the results changes ? If the slope is changing, than my be a sign that the BDT cut biases the cos theta* more that what we thought.

  • Is there enough statistics to do the costheta* plot for top and anti-top separately ?

Follow-up discussion, 04/06/2015

Message by the ARC:

We would like to make a short summary of the discussions we had, and 
also to make a proposal. We would like the authors to comment on that.

     - Main conclusion of the meeting was : there is an issue on the MET 
distribution for the electron channel, that seems to show-up in the QCD 
estimation. Probably, it is a shape effect.

We understood there are mainly three possible solutions :
         A) drop the electron channel from the paper,
         B) drop the met from the BDT training : seems to give coherent 
results, but the electron channel ends up with almost no 
sensitivity,even if  it can maybe be partially recovered using some more 
variables in the BDT,
         C) do further checks on the MET : question is, which ones ?


It is clear that we don't have much time to spend on trying to 
understand the MET, and that people are running out of ideas. Point C 
seems hardly feasible, unless we spend a lot of time on it, time that we 
don't have. Point A is a bit too much of a radical choice, and should be 
followed only if all other possibilities are failing.

There is then a sort of consensus in the ARC that option B is a 
reasonable choice, that can even be further developed with the addition 
of angular variables to the BDT.  Of course we understand that the 
electron channel adds only few, but for completeness, and since the work 
is basically already done, it shouldn’t cost anything to add it.

We would then propose option B  as a basis for work.  Please let us know 
your opinion, if you think it is doable, and in which timescale.


The few other less crucial points discussed at the meeting  :
     - Why error bars are larger compared to the unbliding talk  ? A: W 
norm. uncertainty and generator uncertainty are now treated differently 
(but no details given),
     - check the change of signal BDT for various signal generators,
     - make sure the top-pT is compatible with TOP-14-004,
     - add top/antitop asymmetries to the paper, question : what physics 
can we deduce out of the comparison ?
     - something we forget to mention : add scale and PDF uncertainty to 
the Phoweg SM prediction in the differential cross-section plot. That 
should make prediction-measurement comparisons easier.

Reply by the authors:

Based on the slides from the internal authors' meeting of 17/06/2015: agenda

* QCD issue in the electron channel:

Several alternatives have been explored (simultaneously in the electron and muon channel, although the latter doesn't show comparable problems).
One approach was to apply a cut on a discriminating variable that shows a mismodeling, in order to cut away the mismodeled regions. Another approach was to add more discriminating variables to the anti-QCD BDT in the hope that by improving the selection purity against QCD, any mismodeling in QCD would then become irrelevant.
This work is detailed in the slides titled "Different fits" by Andres, and in p.2-15 of Matthias' slides.

The upshot is that we wish to drop the electron channel from the paper (option A in the ARC message), as none of these strategies seems to lead us to a solution of the problem. Therefore, more work would be needed anyway, and we know already that the electron channel is irrelevant in the combination.

* Why error bars are larger compared to the unblinding talk?

Answer is at p.17 of Matthias' slides.

* check the change of signal BDT for various signal generators:

This is shown in the slides titled "Comparison of signal generator shapes" by Andres.

* make sure the top-pT is compatible with TOP-14-004:

This is shown at p.18 of Matthias' slides.

* add top/antitop asymmetries to the paper, question : what physics can we deduce out of the comparison ?

New physics can also be CP violating. Measuring different (compatible) asymmetries for top/antitop may support (limit) such theories.
New plots will be added in next draft.

* add scale and PDF uncertainty to the Phoweg SM prediction in the differential cross-section plot:

Work in progress concerning PDF uncertainties. We expect them to be small. We should see in ~2 days. Q-scale was already proven to be negligible in our QA twiki, here [1]

Best,
the TOP-13-001 authors

[1] https://twiki.cern.ch/twiki/bin/view/Main/PaperTop13001QA#19_01_15_by_Jeremy_Andrea

Paper v3 comments

03/07/15, by Jeremy Andrea

General comments :

  • one can not see at all error bands for the stack histograms. What's the recommendation of the top (single top) group for that matter? I understand that is always a pain to add them, but at least one could add the normalization uncertainties for prefit plots? Postfit uncertainties should be provided by Theta automatically.
All the current plots in the paper are post-fit. As we fit each systematic separately, theta only provides the fit and MC uncertainties, which is not enough here. We have produced the histograms with all the systematics, but probably will be some work to add them to the plots. Would it be ok to flag the status PUB-Draft on CADI and therefore start Language Editing while we work on that?

  • after reading the paper, I'm not sure anymore how A_\mu is extracted after the unfolding ? Is it take from equation 1, using 2-bin unfolding on one hand, and using the differential cross-section measurement by performing a fit using equation 2 (and then how is performed the fit) ? Sorry to ask this trivial thing, but the paper is a bit confusion on this aspect, so I get lost at some point. I think the paper needs to clarify these points.
This is explained in l.35-37, but it is true that after that we have not given any more details. We unfold, and then we perform a linear fit to the unfolded distribution, whose slope gives the asymmetry A_\mu. See replies to related comments below, to see where we added this information in the new draft.

  • one should comment the slight "tension" between data and the powheg prediction of A_\mu. 5% p-value is a about 2 sigma deviation, so not significant, but still has to be mentioned, I think. Maybe that's enough to mention in the abstract and the conclusion the sigmas rather than the p-value ?
We hesitate to do that because it would sound like we are emphasizing a discrepancy, and because when one quotes standard deviations one has to be very careful, editorially, in how to convey the message that no Gaussian assumption is made in reality. In fact, quoting a p-value is more correct and general, and there are even online tools that allow to convert p-values in "sigmas".

Then 'in text' comments :

  • Title : "Measurement of the top-quark polarization in t-channel single top-quark production mode" ? or "singe top-quark events" ? As it is, the title seems to miss one word?
We would like to keep as it is, unless also other reviewers agree on changing.

  • Abstract :
    • ", which is sensitive to the top-quark polarization[...], singe top-quark production mode [or events]. This analysis is based on..."
We would like to keep as it is, unless also other reviewers agree on changing.
    • "A_\mu" and not "A_l"
ok
    • "which is compatible with the SM prediction of 0.438$\pm$XXX with a p-value of 0.05%". or "which is compatible with the SM prediction of 0.438$\pm$XXX within X $\sigma$".
ok

  • line 9 : "production in the t-channel" ?
We would like to keep as it is, unless also other reviewers agree on changing.
  • line 13 : maybe add a reference to anomalous couplings papers at least ? Is there any theoretical papers that involves the production of a new particle ?
ok
  • line17 : "single top-quark", the hyphen.
ok
  • line 20 : "of a decay product particle X"
ok
  • line 21 : "denote respectively" "top decay-product"
ok
  • line 22 : "that are aligned [...] the direction of the momentum of the quark"
ok, and we also replaced l with mu
  • line 24 : "In this analysis, the muon is ..."
ok
  • line 25-26 : I would explain what is the meaning of the spin analysing power somewhere around here.
we now write:
The spin-analyzing power, i.e. the degree of correlation of its angular distributions with respect to the spin of the top quark, is exactly 1 at LO in the SM.
  • line 26 : "can be modified [...] that would be"
ok
  • line 27 : "of the Wtb..."
ok
  • line 32 : "high purity"
ok
  • line 33 : "top-quark candidate"... "in whose", I'm not sure about that last phrasing. I leave it to the LE.
Changed into "A top quark candidate is reconstructed and the angle between the muon and the recoiling jet is calculated in its rest frame."
  • line 70 : "the modelling of the signal, we ..."
ok
  • line 71 : I would add a sentence there on the CompHep samples used for the bias estimation.
That was true in a previous version of the analysis; then we were requested to use only aMCatNLO, as reflected in the text.
  • starting line 86 : I know nobody is doing it, but I would add the uncertainties on the cross-sections there. It is always unpleasant to me to see numbers without uncertainties.
In fact, although we agree in principle, we hesitate because those uncertainties play no role at all in this analysis. So maybe we could even be more radical and remove the values, just to de-emphasize their relevance?
  • line 94 : is the sentence "This analysis reduces ..." really needed there ? That also seems strange because we don't know the normalization uncertainties.
In fact, this sentence was introduced for the reason discussed above: de-emphasizing the importance of the exact values and their uncertainties. We agree on removing this sentence.
  • line 103 : "one isolated muon and a large missing transver energy (\missET) arising from the neutrino coming from the leptonic decay of the W boson". Or something like this.
ok
  • line 106 : "in association with the ..."
ok
  • line 114 : "an isolation requirement. It is defined as ..."
ok
  • line 120 : remove "offline" ?
ok
  • line 121-123 : I would remove the sentence "The jet momentum... detector acceptance". I'm not sure it is really useful here ?
It seems to be very popular to have in CMS publications, but we agree on removing if you insist. Maybe we see what happens at CWR and LE steps?
  • line 130 : "from W+jets events "?
ok
  • line 139 : you mentioned the pT cut of the jet already earlier in the text.
removed
  • line 147 : sometime you use "QCD", some other times "QCD-multijet". I would use QCD-mlutijet everywhere. After all, ttbar is a QCD process, still you estimated it somewhere else.
Changed into "QCD multi-jet" everywhere (hopefully)
  • from line 146 to 157 : shouldn't this be moved to the section 6.1 ?
We intend section 6.1 as the description of the estimation, not of the rejection, which is a selection step.
  • similarly from line 158 to 174 : shouldn't it be moved to section 6.2/6.3 ?
As above.
  • line 159 : "deployed" => "constructed" ? "The training is ..."
ok
  • line 167 : W-boson mass"
ok
  • line 169 "top-quark candidate ... light-quark system"
ok
  • line 170 : I guess p_lj is the light jet momentum ? At first reading, I tought it was the momentum of the l-jet system (don't ask me why...) and then I get confused. But just in case, p_j would avoid misunderstanding ?
used j' for coherence with other symbols
  • line 174 : "using simulated events."
ok
  • caption Figure 2 : in the "2jets 1tag" category".
ok
  • line 178 : "Corrections are ..."
ok
  • line 183 : add "all these corrections are found to be small", whatever "small" means...
ok
  • after line 184 : "top-quark decay" ... "as :"
ok
  • equation 2 : the lambda is not defined. This is the width, not the cross-section I guess. That is not clear in the text where this formula is used, and how. I suggest to clarify this already here, so people knows what to expect. that would also makes more clear the analysis strategy.
We changed the formula into a differential cross section, and we added a sentence to explain that we extract the asymmetry from the slope.
  • line 187 : "cases" => "events"
ok
  • line 188 : "spin would clearly"
ok
  • line 190 : "Therefore, the single top ..."
We don't think "the" is needed here, we are using a general term
  • line 196 : "for estimating the contamination by QCD"
ok
  • line 199 : maybe you could add a sentence to discuss the other sub-leading backgrounds that you are considering in the analysis ?
Not sure here. They are mentioned in the systematics section and no special treatment is performed here
  • line 203 : "likelihood template fit (ML) "
ok
  • in section 6.2 : I'm wondering if the discussion about the W+jets modelling is not a bit too long, given the limited contribution it has at the end ? Do you think it could be reduced/summarize ? Probably not so much ? There are a lot of details there, I'm not sure all of them a necessary ?
You have a point. We removed the sentence about the W pt reweighting, as it has no impact at all on the result. We are open to suggestions about how to reduce further.
  • line 246 : I would use "top-quark pT" rather than "p^t_T", unless you define it.
ok
  • line 255 : "QCD-multijet background"
ok
  • Table 1 : could you specify briefly what are in the uncertainties? Eventually add a link to the section 9 ?
In fact the uncertainties in this table are only from fit and MC statistics. Added "post-fit" before "uncertainties" in l.266.
  • line 263, 265 : rephrase please "+/- X log-normal". That is not really explicit.
Added "we consider a" before the value.
  • line 270 (and elsewhere) : is " at parton level" more appropriate than "at Monte-Carlo truth level" ?
ok
  • line 270-271 : "level. It accounts for distortions coming from ..."
ok
  • line 272 : "imperfect reconstruction of the top-quark mass", what do you mean by this ? You mean resolution of the objects used to reconstruct the top mass ? Because this is what the unfolding use as input : resolutions of objects ?
It is not "mass" but "candidate". We mean that in a certain fraction of cases we use the wrong objects to form the top quark candidate, and this results in a smearing that needs to be unfolded.
  • line 274 : "In simulation, the parton level definition of ..."
ok
  • line 278 : I would phrase the "t->W..." decay chain we words instead.
Changed into "...the case in which the muon comes from $\tau$ decay"
  • line 280 : "their uncertainties, estimated in Section 7, are used."
ok
  • line 282 : "the unfolding procedure"
We prefer the current text.
  • line 285 : "by CMS, which ..."
ok
  • line 287 : " A closure test is performed by injecting ..."
ok
  • line 288 : I would move the CompHep reference (BTW a reference is needed) to the MC section, and give a bit more details there
ok (and added references)
  • is Equation 3 really needed ? That's rather obvious, no ?
removed
  • line 296 : "by fitting the unfolding distribution according to Eq.2...", but the way, I'm not sure how the fit is performed any more. Could you clarify ? It's a chi2 fit that uses the differential cross-section and the total covariance matrix ?
added explanation at end of sec.8
  • line 306 : "showering, and using the same ...". I would remove the last sentence "The signal model in MC@NLO ..."
ok
  • line 310 : "laving their values"
ok
  • After line 310 :
    • "To do so, a re weighting procedure ..."
    • "An event weight is then defined as :
ok; by the way, thanks to your comment we realized that we had forgotten to update the paper to say (as documented in the AN already) that we use this method only for the signal. In fact the procedure didn't work well enough for ttbar and W+jets, so we used the systematic MC samples in a normal way for ttbar (as the statistics was enough) and took the templates with a looser BDT cut for W+Jets.
  • line 315 : " all processes that contain top-quark" ? Or "TTbar and all single-top-quark samples ..."
Changed into the second proposal.
  • line 317 : I'm not sure anymore to understand how the extrapolation is done
In fact, thanks to your comment we realized that this is a mistake in the draft. We don't do any interpolation anymore, just use the +- 3GeV samples. Is that ok with the ARC?
  • line 329 : "cross section in the analysis and "
ok
  • line 338 : can replace "uncertainty envelope" by something that is less jargon ?
Re-wrote the paragraph as:
The uncertainty due to the choice of the set of parton distribution functions (PDF) is estimated by reweighting the simulated events with each of the 52 eigenvectors of the {\sc ct10} collection~\cite{CT10}, plus the best-fit sets from the {\sc mstw2008CPdeut}~\cite{MSTW2013} and {\sc nnpdf23}~\cite{NNPDF23} collections.
 The unfolding procedure is repeated for each of these 54 cases. For the reweighting of the simulated events, the {\sc lhapdf}~\cite{PDF:LHAPDF} package is used.
  • line 340 : it s 3 times 1% right ? 1% for each source : trigger isolation and ID ? Because it is not clear.
ok
  • line 352 : say somewhere that the b-tag efficiencies (and mistaging rates) are measured in data.
ok
  • line 362 : "has shown a small bias"
ok
  • line 368 : "their combination, showing statistical and total uncertainties, with a comparison to the SM expectations, as predicted by Powheg.
ok
  • Figure 4 vs Figure 5 : why the Q2 uncertainty on Powheg disappear on the Figure 5 ? Summing the t and anti top channels makes no sensitivity on Q2 ? OR is it (again) a MC statistic issue ?
We think it is a statistics issue, indeed, because we have much less top-bar than top due to the large charge asymmetry.
  • line 370 : it seems to contradict line 296, and it is incoherent with line 373. Could you clarify ?
We added a sentence at the end of Section 8. Please let us know if now the text around l.296 is more clear.

Paper v4 comments

10/07/15, by Jeremy Andrea

  • About the tension with SM prediction and sigma. I'm not sure I buy your point. I understand that p-value is probably better define in case the uncertainties are not gaussian. On the other hand, the uncertainties on A_mu are symmetric, and the extraction of A_mu is done, I guess, assuming symmetric uncertainties (this is chi2 fit ?). So quoting number of sigmas is not completely sense less. Said that, I'll not push for it. But what I would really like to see is a comment like "we have some deviation from SM expectation, but that is not statistically significant". Something like that. Just quoting the p-value is not enough : it doesn't give to the reader an easy human-understandable information, and could lead to unmotivated "excitement".
    • number of standard deviations has been added
  • line 193 : to be even more explicit, "In this analysis, Amu is extracted by fitting the unfolded differential cross-section as function of costheta*, according to equation (2)." or something similar. I think you get the idea behind. Also, could you clarify how the fit is done ? Is it a chi2 fit ? Is the full covariance matrix after unfolding used (I really hope so) ? This needs to be explained somewhere.
    • chi2-fit is explicity mentioned now in section 5 & technical details are given in section 8.
    • the chi2-fit utilizing the full covariance matrix was exactly chosen for this reason because it easily accounts for the induced correlations after unfolding.
  • For W+jets, that requires a bit more of thinking. This is not a super-strong issue, so I suggest we leave it as it is for now and see LE opinion.
    • pending for LE
  • About the Figure 5 : I guess the powheg Q2 was estimated using dedicated samples, and not from your reweigting techniques. Since it really seems like satistical fluctuation, probably that's better to remove the Q2 uncertainty (sorry to roll back to you previous plot layout:( ) and mention somewhere that Q2 on the pohweg prediction was found to be small, and thus not shown.
    • Q2 band is calculated from the dedicated powheg samples -> now removed from the plots
    • l384: a sentence has been added that Q-scale & PDF are negligible for the prediction at generator level

Paper v5 comments

28/07/15, by Joanne Cole (as Language Editor)

Abstract

L2:  “analysing” -> “based on”
L3:  Here and elsewhere you have used “center”, but I have also seen “centre” in the main body of the text.  You need to decide whether you are using UK or US English and make sure your spellings are consistent.
L5:  “estimated using a fit to data”
L7:  “… used to extract a top-quark spin asymmetry of”
L8:  Add a comma after the value for A_mu.

All done. About UK vs US spelling, our target journal is JHEP which is likely to prefer UK spelling, so we now ran a spell checker with "British" option.

Main text

L3:  You need to define QCD before you can use it - I noticed a definition later one, which needs to be moved here.

Now is " typical Quantum Chromo-Dynamics (QCD) timescales".

L4:  “through electroweak interactions” - I think this should be in brackets or between dashes or something like that.  Otherwise the sentence implies that other quarks can decay before hadronising, but the top quark is the only one that does so via the EW interaction.
(nb.  “hadronizing” is US English whilst “hadronising” is UK English.  See my note about about deciding which one you want to use)

L4 - L5:  “Furthermore, the parity-violating nature of the electroweak interaction at the Wtb vertex (V-A) means that only left-handed quarks are permitted at this vertex.”
(Or something similar!)

L9:  “the t-channel”
L10-L11:  “… the V-A coupling structure along the direction of the momentum of the spectator quark”
L11-L12:  “On the other hand” is generally only used if you have used “on the one hand” at some point shortly before it.
I would recommend replacing it with something like “Equally”

L16:  “center”

All done.

L19-L20:  You start a couple of sentences with variables.  This is best avoided.  I would recommend re-wording to avoid this.

Now we have:

In the above formula, $P_{t}$ represents the top-quark polarization in production and $\alpha_{X}$ denotes the spin-analyzing power of a decay product particle $X$.

L21:  I really don’t like “the number of a top decay-product”, it sounds odd.  I would recommend something like:
“The variables N(up) and N(down) are defined for each top decay product from the decay chain t -> bW -> b mu nu as the number of instances in which that decay product is aligned or counter-aligned with the direction of the spectator quark momentum, respectively.”

L32:  “The analysis strategy is the following:  After applying an event selection designed to produce a sample of relatively”
L33:  “high purity t-channel”

L36:  “An unfolding technique is applied to extract the patron-level version of this angular distribution.”
L38:  “to the polarisation through equation 1, is calculated.”

L62:  Put commas around the pT miss variable.  “projection onto”

All done.

L66:  See my later comment about reference 11.  You also cannot simply say “at 8 TeV”.

Added " a centre-of-mass energy of"

L69:  “… MC generator POWHEG [13-15], interfaced”
L70: “modelled” spelt this way is UK English.  See my earlier comments.

As written above, we choose UK. So we changed "modelling" into "modeling" instead, few lines below.

L77: “W+jets” and “Z/gamma* + jets” - these are both jargon.    They need to be properly defined here, then these shorthand names can be left unchanged elsewhere in the paper.

Now we have:

The \ttbar, W boson in association with jets (\wjets) and Drell-Yan in association with jets (\zjets) processes are generated with (...)

L78:  “whereas” does not make sense in this context.  I would replace it with “whilst” or “while”

Done.

L84:  It is not obviously what “using orthogonal samples in data” means.  I assume this is a reference to the control samples you mention later.  I would recommend that you make this clearer.

What about:

QCD multi-jet events (i.e., events where the muon does not come from a leptonically decaying W boson) are modeled using statistically independent samples in data, as detailed in Sec.~\ref{sec:qcd}.

L90:  You cannot normalise something to “accuracy”.  I would recommend rephrasing as follows:
“quark pair production is normalised to a full NNLO prediction that including soft gluon resummation to next-to-next-to-leading-log order, as calculated …”

Done.

L96:  “superimposing” - you cannot use this word without explaining what you are going to superimpose it on.  I would recommend something like:
“The effect of pileup is evaluated using a simulated sample of minimum bias events, which taken into account in-time and out-of-time pileup conditions.  These are then superimposed onto the events in the simulated samples described above.  The events are then reweighs in order to reproduce the true pileup distribution that can be inferred from the data.”

Your point is valid, but the replacement that you propose sounds like the simulated sample of minimum bias takes into account IT and OOT pileup conditions, while in reality this is taken into account at a later simulation stage (when minbias samples are overlaid).

So, what about instead:

The effect of pileup is evaluated using a simulated sample of minimum bias events simulated with \PYTHIA, superimposed onto the events in the simulated samples described above, taking into account in-time and out-of-time pileup contributions. The events are then reweighted in order to reproduce the true pileup distribution that can be inferred from the data.

L106:  “preferably” -> “preferentially”

L108-109:  “The event selection follows closely that used in the measurement of the production cross section in the same channel [30].”

L111:  CMS style guidelines suggest that it should be “pseudorapidity”, ie. a single word.  Note that I have seen it as both a single word and two separate words in various places in the paper.  You should make all these consistent.

L113-L115:  This sentence about the muon selection does not make sense.  I would recommend changing it to something like:
“Muon candidates are only accepted if they pass the following requirements:  A transverse momentum, pT, of at least …”
The rest of the sentence will then need to be modified to match.

L115:  “Isolation is defined as the sum of the transverse energies deposited by stable charged hadrons, photons and neutral hadrons in a cone of size … around the muon direction.  In order to pass the isolation requirement, this value must be less than 12% of the transverse momentum of the muon.”

L118:  I would put a full stop after “present”, then start a new sentence.  “The selection requirements for these additional electrons/muons are as follows: …”
At the moment, it does not quite make sense.

L121:  “particle-flow objects” (particle-flow particles sounds a bit awkward!)
L121:  “clustered using the anti-kt algorithm”

All done.

L127:  Does this line require a reference at the end?

We presume that the correct reference would be JME-13-004, which is currently in CWR. We can either wait for them to be on arXiv and refer to them, or we can just remove the entire sentence "Jet energy corrections are derived from simulation, and are confirmed with in situ measurements of the energy balance in dijet and photon+jet events." We commented this sentence out in the new draft, as we feel that it is not so needed in the first place, but let us know if you prefer to resurrect it.

L132-L136:  This is an extremely long sentence.  I would recommend breaking into two sentences.  It would probably be best to break it after “track-based lifetime information [33].”

L142-L143:  “are used for the control samples discussed in Section 6.”

L148:  You don’t apply a threshold to a BDT, you apply a threshold to the “output” or “output discriminant” or “discriminant” produced by the BDT.  This occurs in a number of places and needs to be corrected.  Indeed, in some places it is done correctly, so you need to be consistent.

L150:  “the missing transverse energy …”

L152:  This “pseudorapidity” is correct.

All done.

L156-L158:  It seems odd to show this before you have actually defined the control samples?

The concept of control region with different multiplicity has been introduced already at l.139-143.

However, your comment makes us realize that we have never introduced the control region shown in the left-hand side of figure 1, which is a sub-set of 2j1t. Therefore, we moved l.156-158 (and figure 1) after the description of the other BDT (which is used to define CR and SR), i.e. just before l.172, and at the end of l.143 we now added:

The ``2jets 1tag'' itself is further divided into a control region (CR) and a signal region (SR) depending on the value of a multivariate discriminant described later.

L159:  “has been used” -> “was used”
L160:  “Training was performed” - you just referred to the BDT in the past tense, so this should match.

Done.

L163:  This statement is confusing.  You need to say which control sample you are applying this to, as I assume it is one where more jets than tags are required, but you have not specifically said.

You are right, the current sentence makes only sense in 2j1t or 3j2t. Changed into:

the pseudorapidity of the jet with smallest value of the b-tagging discriminator, \etalj;

Figure 1 caption:  “Distribution of the BDT_QCD discriminant in control regions …”

Done.

You also specify a threshold on the other BDT discriminant, with apparently no explanation about where this comes from prior to referencing the figure in the text.

With the new order (see comment above), now it comes after having introduced the other BDT discriminant.

Figure 2 caption:  “BDT_Wttbar discriminant distribution (left) …”

Done.

It seems a bit odd to include cos theta*mu in this figure, given the left plot is referenced in section 4, whilst the right-hand plot is not referenced until section 5?

You are right, so we now reshuffled figure 2 and 3. Figure 2 now includes the two BDT plots, fig.3 the two cosTheta plots; and the latter is referenced in Section 5.

L179:  Put commas around “where necessary”

Done.

L181:  Need a reference for T&P?

We just added a reference to this document. Is it a legitimate reference? Note that, although it is public, it is a rather "low-status" document (even less than a PAS). Unfortunately we could not find any better reference about 2012 muon efficiencies. Anyway, this can be discussed further with the PubComm at the time of the Final Reading, when issues about references to other CMS documents are usually sorted out.

Or was your remark just about the usage of the term "tag and probe"? If so, as we don't know when this technique was first applied in particle physics (probably several decades ago), we would rather remove this term.

L183:  Remove the semi-colon after [33]

L184:  “with respect to” -> “compared to”

All done.

L??? (the lines without a number between L185 and L186):  This does not completely make sense “theta*X” is not an angular distribution, it is an angle.  I would recommend re-writing so that you define the angle in one sentence, then in the next you explain how its distribution can be expressed via a differential cross section.

What about:

The angle between a top-quark decay product $X$ (${=W,\ell,\nu,b}$) and an arbitrary polarization axis $S$ in the top-quark rest frame, $\theta^{*}_X$, distributes according to the differential cross section:

L186:  As I said before, it is not advisable to start a sentence with a variable.

Added word "Variable" to start the sentence.

L187:  “… tends to be aligned with the direction of the spectator quark momentum, resulting in a high degree of polarisation.”
NB.  “polarization” is US English.

L191:  I don’t think this should be a separate paragraph, I would merge this one with the paragraph above.  New paragraphs tend not to start with “Therefore”, as that implies what is coming is immediately related to what went before.

L192:  Put commas around “Amu”, then:  “… of the muon, where the polarisation axis is defined as pointing along the untagged jet (j’) direction in the top quark rest frame.”

All done.

L194-L195:  It is not clear how a cross section can be unfolded according to equation 2.  I assume you fitted the unfolded differential cross section using equation 2?  Also, cross section should not be hyphenated.  You need to check that it is consistently written as “cross section” throughout the paper.

Your understanding is correct. What about:

In this analysis, a $\chi^{2}$-fit of the unfolded differential cross section to equation~\ref{eq:cosThetaDistr} is used to estimate \Amu.

L198-L199:  “Samples with inverted isolation” -> “Samples in which the isolation requirement on the muon has been inverted”
L199:  “… to extract templates for estimating the contamination by QCD”

L201-202:  Why is “in situ” in italics?  You have used it elsewhere without italicising it.

L205:  “performing fits to the BDT_QCD discriminant distributions for each …”

L206:  “contamination from this process is expected.”

All done.

L211:  You’ve just defined the inverted isolation requirement samples above, I’m not sure you need to do it again.  As it stands, it makes this sentence rather tangled.  I’m not sure why you can’t say something like “using the inverted-isolation QCD-enriched data sample defined above.” (or something similar).

L212:  “It was verified that the BDT_QCD discriminant distributions for QCD events …”

L218:  “After the QCD multi-jet contribution to the signal region has been estimated, the agreement between the expectations and the data is verified in …”

L221-L222:  “… control region; this region is expected to be enriched in W+jets events.”

L223-L226:  The start of this paragraph really doesn’t make sense.  I would recommend:
“A similar disagreement between data and expectations in the cos theta*mu distribution was observed in sqrt s = 7 TeV data in the context of the a different analysis [30].  Investigations using different MC generators and their associated settings showed that SHERPA [36] provided a better description of cos theta*mu in this control region at both centre-of-mass energies.”
NB.  You’ve used “centre” here.

All done.

L235:  This is the first time you’ve talked about the massive versus massless approximation.  It is not clear from what you have written why this is less desirable than taking the massive approach for b/c quarks.

The reason is that, if all quarks are massless, then all quarks are produced with equal rate. This is obviously not a realistic simulation (b quarks are rarer than light quarks because of the mass), and it matters when heavy quarks behave differently from light quarks (as in our case).

We changed "affecting the relative fraction of heavy quarks" into "causing the relative fraction of heavy quarks to be unrealistically large".

L239:  As it stands, this line does not make sense because it is not clear what Wf1f2 is.  I assume this is how you label the weights in your reweighting procedure.  I’m not sure you need to mention it at all, as you do not appear to mention it anywhere else, unless I’ve missed it?

Removed:

${\rm W} f_1 f_2$ (where $f_i$ is the flavour of the parton associated to the $i-$th selected jet)

L241:  “including the cos theta*mu distribution”

L242:  “significantly affected by the treatment of the b and c quark masses.”

All done.

L244:  It is not obvious what you mean by the “ttbar model”

Changed into:

In order to validate the \ttbar\ simulation, this is compared to data in the ...

L245:  “In particular, Figure 3 shows the BDT_W,ttbar discriminant and cos theta*mu”

L248:  “than observed in data [37-39]; we therefore reweight the spectrum of generator-level top quarks in ttbar events such that it reproduces the measured differential cross section with respect to top quark transverse momentum.”

L256:  Put commas around “expect the QCD multi-jet background”.  You could even consider brackets.

Figure 3 caption:  “Control plots” feels like jargon.  You don’t even need to use this phrase, you could simply replace it with “Distributions”

Table 1 caption:  “…simultaneous ML-fit to the shape of the BDT_W,ttbar discriminant in the “2jets 1tag” and “3jets 2 tag” control regions.”

L266:  “… W+jets, Z/gamma*+jets and diboson (WW, WZ, ZZ) production with their relative fractions taken from simulation …”

L264 and L267:  “we consider”  -> “we apply”

L269:  “the number of events exceeding the threshold of 0.45 on the BDT_W,ttbar discriminant …”

L272:  “to measure” -> “to extract”

L276:  I think it should be “parton-level”

Table 2 caption, line 2:  “the threshold requirement on the BDT_W,ttbar discriminant and applying the …”
Table 2 caption:  I think you can remove the last sentence, but if you decide to keep it then it should be something like:  “The QCD multi-jet background contribution is estimated using a data-driver procedure.”

L280:  “ … muon comes from a tau decay by …”

L284:  Add a comma after “Section 7”

L287:  This sentence should be part of the paragraph above, ie. L285-288 should be one paragraph.

L289-290:  “An examination of the pull distribution has shown that the uncertainties have been treated correctly.”
Or something similar.  As it is written now it makes you wonder where the pull distribution is in the paper.

L290:  “A closure test was performed by”

L292:  “This test verified that, with this analysis strategy, it is possible to measure different asymmetries correctly with minimal bias.”

L294:  “The value of Amu is extract using a chi^2-fit of the unfolded cos theta*mu distribution, under the assumption that equation 2 is valid.  The fit takes into account the bin-by-bin correlations that are induced by the unfolding procedure.

L302:  “The measurement presented in this paper will potentially be affected by several sources of systematic uncertainty.  To evaluate the impact of each source, we perform a new background estimation …”

L308-L311:  This doesn’t completely make sense.  I would recommend removing “used for the statistical inferences”, as anyone who has read the paper up to this point knows what the templates were used before:  “the uncertainty associated with the limited amount of simulated events used in the templates is taken into account at all stages of the analysis ie. both in terms of fluctuations in the background templates and in determining the elements of the migration matrix.”

L312-L313:  “This uncertainty is determined by propagating the uncertainty associated with the background normalisation coming from the maximum likelihood fit through the remainder of the analysis, ensuring that the correlations between bins are taken into account.”
I’m not 100% sure I’ve understand what you are saying correctly, but for the other sources you explain what you did to estimate the uncertainty, so you need to do the same for this one.

L314-L317:  Delete the last sentence - this should already have been said when you defined the MC samples earlier.

All done.

L319:  You note that the renormalisation and factorisation scales have the same value Q, but you have not said what Q is?

We asked Benedikt Maier for all the exact details, and this is what we understood.

For the single top samples it is a fixed value equal to the top mass, which is motivated by the fact that the top mass is the only relevant scale in the diagrams. For Wjets and ttbar+jets, the default scale in MadGraph is used. It is a dynamical scale whose actual form is described here. This functional form is also dependent on the multiplicity of outgoing partons (as those two samples involve matching/merging of different multiplicities), their masses and transverse momenta.

Going into this level of detail is not considered relevant by the authors, what matters is that the scale is doubled and halved for the estimation of this systematic uncertainty, and this is true independently of how the central value is defined.

L325:  “… a looser threshold is applied to the BDT_W,ttbar discriminant in the simulated samples in which the scale has been varied.  This is done to enhance the statistics available in these samples.”
NB.  “variated” is not a word!

Done.

L326:  “an extracted shape” - of what?

Added cosTheta.

L327:  “… when applying the nominal BDT_W,ttbar discriminant threshold to those samples instead.”

L329:  “Top-quark mass uncertainty:  “Additional ttbar and signal simulation samples have been produced with the top-quark mass varied by +/- 3 GeV.  These samples are used to determine the uncertainty arising from our knowledge of the top-quark mass.  This is a conservative
estimate as the current world average is 173.3 +/- 0.8 GeV [42].”

All done.

L335:  “An uncertainty of the fraction of W+jets events in which the jets arise from heavy flavours is taken into account by scaling its contribution …”

L337-339:  “The uncertainty associated with the reweighing procedure presented in Section 6.2 is estimated conservatively by comparing the result after reweighting with that determined with no weighting applied.  The difference between the two is then symmetrised.”

L341:  “known to predict a harder pt spectrum compared to that observed in the data [37-39].”

L347:  “… independently, using dedicated samples in which the threshold is either doubled or halved.”

All done.

L354:  “is applied independently to each of the muon trigger, muon …”

What about "is applied independently to the muon trigger, muon identification and muon isolation efficiencies."

L360-L362:  The sentence starting “This variation in jet four-momenta …” is quite difficult to read.  I would advise turning the comma after “is also propagated to ETmiss” into a full stop and then starting a new sentence:  “In addition, the effect on the measurement of ETmiss arising from the 10% uncertainty associated with the unclustered energy deposits …”

L365:  “ b tagging:  The uncertainties on the tagging and mistagging …”

L367:  “A 5% uncertainty is applied to the average expected number of pileup interactions in order to estimate the uncertainty arising from the modeling of pileup.”

L371:  “range used to extract the templates used to estimate this background contribution.”

All done.

L373:  Put a comma between “process” and “which”.  Are you sure that the reader will know what you mean by “minor process”?

These are listed immediately after: "These are dibosons and \zjets\ for the W/Z+jets component and the single-top tW and s-channel for the Top component."

Are you sure that we need a comma between “process” and “which”? Maybe, instead, we should change "which" into "that"? What we intend to say is that the minor processes that are lumped with major processes are varied within their uncertainties.

Table 3:  I assume these numbers are percentages.  In which case, this needs to be said somewhere in the table!

These are not percentages, they are absolute uncertainties on Amu.

Figure 4 caption:  “The normalised differential cross section with respect to cos theta*mu for top (left) …”
Comment also applies to the Figure 5 caption.

L382:  Add “production” to the end of this line.

L387:  The asymmetry, Amu, is extracted from the the unfolded cross section using equation 1.  Using this procedure, we obtain:”

L391-L392:  “As a cross-check, we also perform an analytical 2-bin unfolding, which directly yields the numbers N(up) and N(down) (see equation 1).  This results in a value for Amu of:

L394:  The first measurement of the top-quark spin asymmetry (which is sensitive to the top quark polarisation) in t-channel single-top production …

L395:  “center”

L397:  Add a comma at the end of this line (after “SM expectation”)

All done.

References

The titles of the references are presented using two different styles:  A capital letter at the start of the title only or capitalisation of all words exception for prepositions and conjunctions.  I would perhaps be good to pick a style and stick with out throughout?

We stick to the actual titles as published.

We just re-checked all of them, and fixed some cases where we were not using the style of the actual publication. However, styles are still different across our bibliography.

References 8 and 11 are identical!

Reference 33 includes “ttbar” in the title - you should correct this.

All done.

Paper v8 comments

07/08/15, by Jeremy Andrea

  • Could it be possible to increase the size of the plot as much as possible. Plots that the ones on Fig1 are a bit small (not critical).
    • Done: figures 1,2,3 are now top-bottom instead of left-right; figure 4 is still left-right, and only enlarged a little (because these plots are less crowded than the previous ones, and should be easier to read); figure 5 is larger, which also reflects its status as the most important in the paper.
  • There is, in some place, "QCD" and in some others "QCD-multijet", when referring to the multijet sample. Could you please harmonize ?
    • Done (hoping we didn't forget any)
  • lines 6 + 22 : "top-quark decay product"
    • Done
  • lines 11+17 :" single top-quark"
    • We think it is correct as it is
  • line 25 : "because leptons have the highest spin analysing power"
    • Done
  • line 264 : "generator-level top-quark"
    • We think it is correct as it is
  • line 295 : "top-quark"
    • Done
  • line 396 Top->top
    • Done (top quark)
  • line 409 : add "accounting for the full covariance matrix" or something like that.
    • Added ", taking into account correlations"

Paper v9 comments

08/08/15, by Andrea Castro

  • L5: roman fonts (or pen-names) for "Wtb" (also at L8, L29, L79);
    • Done. Found and corrected also two more instances of that.
  • L5: I suggest a long dash in "V-A" (also at L10)
    • Done.
  • L23: roman font for the particles t, W and b
    • Done.
  • L33: "t" in "t-channel" should be in italics; always add "quark" after "top"
    • Done.
  • L60: define "pileup"
    • Added: ", i.e. additional proton-proton collisions whose signals in the detector sum to the products of the primary interaction that triggered the event,"
  • L66: "dataset" ==> "data set"
    • Done.
  • L89, L91, L187, L243: I guess British English favours the double ell in "modelling" or modelled"
    • Done. Found and corrected also some more instances of that.
  • L112: no need to define MET, already done at L64
    • Done.
  • L120 (and elsewhere): you do not use natural units as usually done in our papers. Is there any reason for that? Consider that the formula for the transverse mass at L176 is written assuming natural units
    • No real reason. Done.
  • L161: "the missing transverse energy"
    • Done.
  • L198: I suggest "The variable …"
    • Done.
  • Caption of Figs. 1, 2, 3: the caption is very minimal, at least please describe the bottom part of each plot, i.e. the ratio data/MC
    • Added: "Predictions are normalised to the results of the fit described in Sec.~\ref{sec:fit}. The bottom panel in both figures shows the ratio between observed and predicted event counts, with a shaded area to indicate the systematic uncertainties affecting the background prediction."
  • Table 2: for consistency write "t" and "s" in "t-channel" and "s-channel" in italics
    • Done.
  • L306: "pseudo-experiments" (with hyphen) since you write "pseudo-data" at L308
    • Done.
  • Eq. 3: suggest to define the symbols "PDF" here instead of at L370
    • We think that it is just simpler to anticipate the paragraph about PDF before this one. Same, of course, in Table 2.
  • L341, L343, L345: "statistics" is used in a jargon way
    • Replaced, respectively, with "amount of events", "number of events" and "number of simulated events".
  • L396: "s" in "s-channel" in italics
    • Done.
  • L401: "… of systematic uncertainties …"
    • Done.
  • Eqs 5, 6, 7, 8 and elsewhere: usually we use the symbol "\times" rather than "\cdot" before expressing the powers of ten
    • Done.
  • L412: so far "sigma" was used to represent a cross section. I suggest to write "… 1.7 standard deviations"
    • Done.
  • Table 3: it seems you have been using British English so far, so write "flavour"
    • Done.
  • L416: replace "conclusions" with "summary" which is more appropriate
    • Done.
  • References: quote only the first page; add blank spaces to the abbreviated journal names; for the PASes, remove "CERN" and "Geneva" and leave only the year
    • Done.

Comments from CWR (paper v10)

Bastian Kargoll for AACHEN3B

Type A Comments

  • Abstract, 1st line: "top-quark polarisation" (hyphen)
    • This contradicts Tom Ferbel's comment (see below). We follow Tom's recommendation.
  • Abstract: Please quote 19.7fb instead of "approximately 20fb"
    • ok
  • l.4: "that can decay (through EW interactions) before" -> "that decays through EW interaction before"
    • ok
  • l.12: Change "Equally, ..." to "However, ..."
    • Ok
  • l.21: add "(here X=\mu)"
    • ok
  • l.25: "because of because"
    • ok
  • l.27: "at leading order (LO)"
    • ok
  • l.34: top-quark candidate (hyphen)
    • This contradicts Tom Ferbel's comment (see below). We follow Tom's recommendation.
  • l.34: yields -> fractions
    • changed to: "signal and background composition in data"
  • l.34f: "Each top quark candidate is reconstructed and the angle between the muon and the recoiling jet is calculated in the top-quark rest frame."
    • Changed to "A top quark candidate is reconstructed and the angle between the muon and the recoiling jet is calculated in the top-quark rest frame."
  • l.36f: "… to obtain this angular distribution on parton level."
    • ok
  • l.81: "anomalous structure" (or are really the values anomalous?)
    • ok
  • l.92: "(with a muon not originating from a ...)"
    • ok
  • l.102: use dots instead of commas within numerals
    • We think it is correct like that. Just to avoid misunderstanding, the first number is thirty-seven thousands etc., not thirty-seven dot something. We use commas every three decimal places in numbers of four or more digits, counting right to left.
  • l.110: GEANT4 instead of GEANT 4
    • ok
  • l.114f: "coming from" -> "in"
    • This contradicts Tom Ferbel's comment (see below). We follow Tom's recommendation.
  • l.148: "our signal simulation" -> "the signal simulation"
    • ok
  • l.160: "top-quark reconstruction" (hyphen)
    • This contradicts Tom Ferbel's comment (see below). We follow Tom's recommendation.
  • l.169f: use present tense
    • ok
  • l.182: “… is shown in the ‘2jets 1 tag’ and ‘3jets 2tag’ categories in Fig.1 …”
    • ok
  • l.185: “… discriminant in the same two control regions.”
    • changed to: To reject multijet events, we use events that pass the threshold $\bdtqcd > -0.15$ in the analysis,.
      Figure~\ref{fig:BDT_sig} shows the distribution of the \bdtsigbg discriminant in the ``2jets 1tag'' and ``3jets 2tags'' categories after requiring the selection on $\bdtqcd$.
  • l.277: "inverting some of the event selection cuts" is a little bit sloppy
    • ok, followed suggestion by Greg Landsberg
  • l.309: “… has shown no sign that the uncertainties have been treated incorrectly.”
    • ok
  • l.411: "the the unfolded..."
    • ok
  • l.420f: “approximately 20fb” à “19.7fb”
    • ok
  • l.422: "standard deviations" is missing
    • ok
  • l.422, Figure captions, Figure axis labels: Use same variable name for unfolded cos everywhere, i.e. either "unfolded cos theta" or "cos theta_unfolded"
    • ok
  • l.424: "muons" instead of "leptons"
    • ok
  • Table 1+2 caption: "maximum likelihood fit" instead of "ML-fit"
    • ok
  • Table 3: 10^-2, not 10^2
    • fixed
  • Table 3 caption: "measuremed"
    • ok
  • Figure 3, 4, 5: Use \mu instead of l as index of theta, as only the muon is used in this analysis
    • ok

Type B Comments

  • Before the tables, it is nowhere mentioned that charge-conjugation is implied. Add this somewhere at the beginning. Also, there should be a statement in the text that the analysis is performed individually for top and antitop, as well as combined. On top, a reason why this is done and what we learn from it would be nice.
    • the separation for top/antitop was often asked by theorists at conferences where the PAS result was presented. Added to introduction: "The asymmetry is measured for top and antitop events separatetly to be sensitive to potential CP-violation as it can occur in new physics models."
  • Equation 1: The middle term gives 1/2, but it is not obvious why the right-hand term does not yield 1 (-1) for 100% polarisation.
    • It is just the way Amu is defined: as the denominator is the sum of forward and backward events, while the numerator is the difference, it cannot be larger (smaller) than 1/2 (-1/2). Maybe confusion comes from the fact that there is a term 1/2 out of the parenthesis in one term but not in the other.
  • Also, the POWHEG value given in l.415 is 43.8%. Could you add a statement somewhere on the origin of this difference?
    • this is just NLO
  • l.24: The definition of "spectator quark" was not clear to all readers in our group. In a 4FS, this could also be the 2nd b-quark from the gluon splitting. A Feynman graph would help.
    • Good suggestion, we (re)introduced Feynman diagrams in the Introduction.
  • What is the motivation of using the spectator quark as spin axis? As this is a very forward jet, how precise can the direction of this jet be measured? What is the impact of this on the result? Wouldn't it be possible to use the beam direction as axis?
    • All this is discussed in reference [1], and in other papers like this and this. The beam direction axis in, in fact, one of the alternatives that we have considered in some internal check in an earlier stage of this analysis, finding that the result is equivalent.
  • l.97ff: Why do you use theory values for the cross-sections? Why not use measured xsec values?
    • We re-measure the relevant ones in situ with our fit, so what we report is only the starting point on top of which we apply scale factors. The pre-fit uncertainties are not the theory ones, but way larger. Furthermore, one can not just simply use the measured values, because they are estimated with the same detector, the same MC samples and on the same dataset. Hence, there are correlations of uncertainties that are difficult to account for.
  • Section 4: Add a paragraph describing MET and the applied corrections.
    • ok
  • Is there a cut on MET? It is not mentioned anywhere.
    • There is no cut on MET.
  • l.139-141: Is there are reference?
    • We couldn't find any.
  • l.167: Is this short explanation of the isotropy clear? One can understand what is done, but it is not obvious from the description.
    • made more clear
  • What is the cut value on BDT QCD? The text states that a cut is applied, but the value itself is never given.
    • Added a sentence. The cut is > -0.15
  • Move l.192-198 after l.150. The corrections are applied before the categorization and the BDTs, so it should be described before.
    • ok
  • Section 5: Why is Section 5 located between "Event Selection" and "BG Model Studies"? We recommend to either move it to the beginning and include it in the description of the theory, or (even better) move it after Section 7. We do not see that any information of Sec. 5 is needed in Section 6 or 7, but it sets ground for the upcoming unfolding procedure.
    • The problem is that section 6, and in particular section 6.2, talk among other things about how this observable (by far the most important for this study) is modeled for the background. The reference to Figure 3 should therefore come before section 6.
  • Figure 3/Unfolding: We understand that the unfolded distribution is the most useful way to present the results to theorists. For experimentalists, it seems more natural to compare the data from the detector "as is" with a predicted background, as is done in Fig. 3 (upper plot). Would it be possible to quantify the data/MC agreement (i.e. the SM) using this plot, e.g. giving a chi2 value, fitting a line to the ratio, etc.? What impact has the unfolding on the data/MC agreement?
    • the data/MC shape agreement is rather good (p-value=~95%). You can see by eye that the average deviation per bin is of the order 1. The asymmetry can exhibit a larger deviation nonetheless since it makes the assumption that the distribution follows a straight line after unfolding. The functional form of the data/MC ratio at reconstruction level is however not easy to parametrized in order to calculate a comparable measure of compatibility between data/MC. (Alternatively, using a KS-test may result in some confusion since it is not defined for binned distributions)
  • l.274: Motivate usage of “3jet 2tag” for fit, and give a reason why the other categories are not fitted. We believe that the 3jet2tag category is important to constrain ttbar, but it should be clearly stated.
    • added: "The inclusion of the \ttbar-dominated ``3jets 2tags'' region in the fit provides an additional constraint on the \ttbar background. This also reduces correlations of the estimated \ttbar yield with other contributions."
  • l.283: Can you motivate the usage of a log-normal constraint here? (We are interested, but maybe it does not have to go into the text.)
    • It is generally recommended in order to avoid negative values (as cross sections are positive-defined). But in practice we found that using log-normals or Gaussians gives the same results in this case. (Furthermore, truncated Gaussians (>0) have also a shifted mean value)
  • l.293: Why is q′ direction an approximation for spin-axis? Isn't the chosen spin-axis arbitrary? We did not understand this part.
    • As above, this is discussed in reference [1], and in other papers like this and this.
  • l.295ff: This paragraph caused some confusion, as it was not clear to everyone why it is needed at all. To our understanding, it describes the variable in simulation, and thus describes what the unfolded data distribution is compared against. Maybe add a sentence explaining why this is described here.
    • this paragraph describes the definition of cosTheta at parton level. It is necessary here because there can be some ambiguity on how to define it (e.g treatment of tau decays).
  • Table 2: The numbers in the ttbar line do not add up. There are 2 events more in "Top+Antitop" than in the sum of top and antitop. Same holds for some other lines.
    • Because there are separate fits for the 'top', 'antitop' and 'top and antitop' cases, so this is from the uncertainty of the fit
  • Section 8: The section on "Unfolding" is quite short, given the complexity of what is done here. We are fine with this, especially as the method is described in some references. Nonetheless, it is even more important to have the short text structured efficiently. It might be possible to re-order the first 4 paragraphs of this section. The second paragraph could go first, the first and fourth paragraph belong together.
    • The intentions of the paragraphs are the following: 1. motivation, 2. definition of parton level (target of unfolding), 3. preparation of data prior to unfolding, 4. technical description of unfolding. We are unsure if a reordering would benefit the understanding.
  • Forward references: There are several places where references are given to later parts of the paper. This makes it more difficult to read the paper. See l.183, Captions of Fig. 1+2+3.
    • There is an unavoidable circular dependence: We do not want to show e.g. the BDT QCD distribution w/o QCD estimation (Sec 6) and w/o applying the fitted background scale factors (Sec 7). However after the description of the BDT training (Sec 4) it makes sense to show its distribution already early.
  • l.367: To our knowledge, application of the pT(top) reweighting is in general not recommended. What is the reason it is applied here?
    • The recommendation is to not apply it when ttbar is the signal; see the "clarification 30 Jan.2015" in this wiki. Here ttbar is an important background, and if we don't reweight it, several plots (e.g., BDT inputs) in the ttbar-enriched control regions would appear discrepant and therefore send the erroneous message that we don't understand this background, while instead we know the reason for the discrepancy, as demonstrated by the fact that those discrepancies disappear when reweighting is applied.
  • l.411: This should read "Equation 2", not "Equation 1". It should be stated here how A_mu is extracted from this distribution. This is mentioned in l.208, but that is far away in the text. It might get easier if Sec. 5 is moved (see comment above).
    • changed to eq 2
  • Figure 4-6: While there is no harm in showing 3 almost identical theory predictions, it might be nice to mention in the text why this is done (e.g. experimental resolution is by far worse than theory prediction). Do you think it might give insight to overlay the fitted line on the graph?
    • We prefer not to add much more extra text, or additional elements to these figures. We may reconsider if you suggest some nice formulation. We would like to avoid sending a negative message like "experimental resolution is worse than the spread of theory predictions", but rather a positive one, like "all SM Monte Carlo generators give very similar predictions despite their differences".

Tom Ferbel

  • 1. CMS PubComm recommends considering "top quark" as a single idea/word, which requires no hyphens in any standard usage.
    • ok
  • 2. In CMS usage, "t-channel" should always have a hyphen and use \ital{t} format.
    • ok
  • 3. Do not use "QCD multi-jet," especially since what you mean in "multijet data." (No hyphen, and no QCD, which corresponds to a theory!)
    • ok
  • 4. You define E-slash as the value of the vector of the momentum imbalance in the event, and say it is the missing energy. This sounds awfully nutty to a bystander! If for some religious purpose, you must use E-slash, then at least call it the missing momentum.
    • E-slashed is defined as the magnitude of pvec-slashed that is the missing transverse momentum vector. Definition of MTW adapted accordingly
  • 5. Check with the PubComm web page for your other transgressions: https://twiki.cern.ch/twiki/bin/viewauth/CMS/Internal/PubGuidelines#Introduction
    • ok
  • 6. Make sure that the figures and tables are in proper order, and do not appear prior to their mention in the text. They also should lie within their numbered sections and not in or after the summary.
    • ok

more comments
  • 7. DY refers to q qbar annihilation into (Z/gamma)* into ell+ell-+jets, which is not handled well, but should be!
    • DY -> Z/gamma* (+jets)
  • 8. Check whether your format for JHEP is correct. Usually, only the first word is capitalized in the titles.
    • ok
  • 9. Fix your figure labels, as some are clearly incorrect. Your normalizations seem crazy. Are these arbitrary? Also, please follow standard labeling, using a capital letter in first word of each expression..
    • done. Normalization was not crazy: There was an exponent on top of the y-axis in some plots. This has been now removed to improve readability.

Extracted from PDF:

  • Title
    • ok
  • AL1: A measurement of ... is presented -> A measurement is presented of ...
    • Rephrased
  • A4
    • t-channel is important here
  • A5
    • ok
  • A6
    • ok
  • A7
    • the suggested change makes the text shorter but harder to understand.
  • A8
    • Rephrased
  • A9
    • Would rather keep as it is
  • L1: "discovered so far" -> in the standard model (SM)
    • Would rather keep as it is
  • L2-3: much shorter than the typical quantum chromodynamics (QCD) timescales. -> much shorter than typical timescales of processes in quantum chromodynamics (QCD).
    • Would rather keep as it is
  • L3
    • ok
  • L5
    • ok
  • L6-7
    • Would rather keep as it is
  • L8
    • Would rather keep as it is
  • L9
    • Would rather keep as it is
  • L10
    • ok
  • L12
    • Changed according to recommendations by Bastian Kargoll and Joanne Cole
  • L14-8
    • Removed paragraph as [7] is unpublished
  • Eq1
    • ok
  • L19
    • ok
  • L20
    • ok
  • L21
    • ok
  • L22
    • ok
  • L23
    • Would rather keep as it is
  • L24
    • ok
  • L25
    • ok
  • L26
    • ok, kept "the highest"
  • L28
    • ok
  • L30
    • ok
  • L31
    • ok
  • L32
    • ok
  • L33
    • ok
  • L34
    • reformulated sentence
  • L36
    • ok
  • L37-8
    • The correction would change the meaning
  • L39
    • ok
  • L47
  • L50
  • L51
  • L51-2
  • L53
    • Sentence is from the official PubDetector page; added comma
  • L57
  • L58
  • L60
    • ok
  • L61
    • If we call "pileup" the phenomenon itself (i.e.), "from" is not needed
  • L66 Why not call the |vector momentum |pTmiss| or pT-slash?
    • pt-slashed now and MTW def. changed accordingly
  • L67
    • ok
  • L70-1
    • ok
  • L73
    • ok
  • L74
    • Would rather keep the 'an'
  • L75
    • ok
  • L76-7
    • Would rather keep as it is
  • L77
    • ok
  • L79
    • ok
  • L80-1
    • ok
  • L82
    • ok
  • L84-L85
    • we like to keep the naming of "W->lnu+Nq/g" as "W+jets" and "qqbar->Z/gamma*->ll+Nq/g" as "Z/gamma*+jets" for simplicity. otherwise "ttbar" would also need to be qqbar/gg->g->ttbar->WWbb->jjjjbblnujjbb/lnulnubb+Nq/g
  • L86
    • ok
  • L87
    • We think the sentence is clearer the way it is
  • L88
    • ok
  • L89
    • added "...simulations" at the end of sentence
  • L92
    • ok
  • L94
    • ok
  • L95
    • ok
  • L96
    • Would rather keep as it is
  • L97
    • ok
  • L98
    • ok
  • L101, also L87, L102
    • DY vs. Z/gamma* usage see above
  • Use script \ell for charged leptons
    • ok
  • L104
    • ok
  • L105
    • ok
  • L108.
    • ok
  • L112
    • ok
  • L113
    • Would rather keep as it is
  • L114
    • Kept as is, MET addressed elsewhere
  • L114-5
    • ok
  • L117 "forward region"?
    • -> detector foward region
  • L118
    • ok
  • L119
    • ok
  • L121
    • ok
  • L123-4
    • following completely the suggested reformulation would change the meaning.
    • removed: "the existence.."
    • added PV description
  • L124
    • ok
  • L125-6
    • ok
  • L127
    • ok
  • L128
    • like to keep it mostly as is, changed "sum" -> "scalar sum"
  • L129
    • The previous part of the sentence already excludes muons
  • L132
    • ok
  • L133
    • ok
  • L138-9
    • ok
  • L143
    • reconstructed and selected -> present
  • L144
    • ok
  • L145
    • ok
  • L146
    • ok
  • L147
    • ok
  • L148
    • ok
  • L151
    • ok
  • L153 and later
    • if quotation marks are removed around "2jets 1tag" then this may get more confusing 'cause it may read as two independent words and not as the name of a specific category
  • L155
    • ok
  • L157
    • ok
  • L158
    • ok
  • L159
    • ok
  • L160
    • ok
  • L161
    • ok
  • L162
    • ok
  • L163-8, 172-81
    • Would rather keep the bullet structure, especially as we have a couple of long definitions included (semicolons needed in that case)
  • L163
    • missing transverse energy now defined in detector section
  • L163-6
    • Fixed "top quark" and removed semicolons, would rather keep the commas
  • L167 Some of us don't remember what these silly symbols mean in logic. Use familiar math physics terms!
    • removed set symbol
  • L169
    • ok
  • L170
    • ok
  • L172-181
    • we like to keep bullets & semicolons
  • L176/177
    • like to keep the full name of variable not only the symbol
  • L179
    • see above
  • L182
    • ok
  • L183
    • Only written out if at beginning of sentence
  • L184
    • ok
  • L186
    • ok
  • L189-90
    • ok
  • L192
    • ok
  • L193
  • L194
    • ok
  • L195
    • ok
  • L196
    • ok
  • L199
    • ok
  • L199+1
    • ok
  • L199+2
    • ok
  • Eq. 2
    • ok
  • L200
    • ok
  • L201
    • only written out if at beginning of sentence
  • L202
    • ok
  • L203
    • ok
  • L204
    • ok
  • L205
    • ok
  • L206
    • ok
  • Figure1
    • ok
  • Figure2
    • ok
  • Page8: Put the next paragraph before this statement
    • ok
  • L208
    • Rephrased the sentence
  • L210-1
    • We prefer to keep its as it is
  • L211 Say something about the asymmetry in Figure3 (b)?
    • we do not observe any uncovered asymmetry by the systematics in the control region
  • L212
    • changed to "Studies of background modeling"
  • L214
    • ok
  • L215
    • ok
  • L216
    • ok
  • L217 "background and signal strengths" -> "background and signal strengths relative to the SM"
    • ok
  • L218
    • ok
  • L219
    • We prefer to keep "the" 'cause it direclty refers to the categories define above
  • L220
    • ok
  • L221
    • ok
  • L222
    • Used some of the suggestions
  • L223
    • ok
  • L224
    • ok
  • L225
    • ok
  • L226
    • Reformulated whole sentence
  • L227
    • ok
  • L229-231
    • ok
  • L233
    • We prefer to keep the word ordering
  • L234
    • ok
  • L236
    • ok
  • L237
    • Reformulated the sentence to remove the questionable word
  • L244
    • Changed to "additional"
  • L245
    • ok
  • L248-9
    • The modifications would change the meaning of the sentence
  • Figure3
    • ok
  • L250
    • kinematic properties -> kinematic distributions
  • L251
    • ok
  • L252
    • ok
  • L253
    • ok
  • L255
    • ok
  • L257
    • ok
  • L258
    • ok
  • L260
    • We prefer to keep the word ordering as it is
  • L261-2
    • Rephrased the sentence
  • L262
    • ok
  • L263
    • ok
  • L264
    • ok
  • L265
    • ok
  • L266
    • break into 2 sentences
  • L267
    • ok
  • L268
    • ok
  • L271
    • ok
  • L273
    • ok
  • L275
    • ok
  • L276
    • ok
  • L277
    • ok, used suggestion by Greg Landsberg
  • L278
    • ok
  • L279
    • ok
  • L280
    • ok
  • L281-6
    • We prefer to keep the bulleted list
  • L283,6
    • Changed to "we assign"
  • L285
    • ok
  • L286
    • ok
  • L289
    • We prefer to keep it as it is
  • Put tables below captions
    • ok
  • Table1, caption
    • Changed to "distribution of"
  • Table2, caption
    • rephrased, kept "data-driven"
  • L290
    • We prefer to keep it as it is
  • L291
    • ok
  • L292
    • ok
  • L293: I don't understand which approximation is meant. You don't appear to refer to it below. Can you clarify this phrase? Is it an issue in pQCD?
    • jet direction = quark direction
  • L295-7
    • ok
  • L298
    • ok
  • L299-300
    • ok
  • L301
    • ok
  • L302-3
    • ok
  • L308
    • ok, shortened the sentence as well
  • L309
    • ok + used suggestion by Bastian Kargoll
  • L310
    • ok
  • L311-2
    • ok
  • L314
    • ok
  • L315
    • ok
  • L316-9
    • Rephrased
  • L320
    • ok
  • L321
    • ok
  • L322-3
    • We prefer to keep the current wording with some modifications
  • L324-5
    • The suggestion would change the meaning
  • L328
    • ok
  • L329
    • ok
  • L332
    • ok
  • L335-6
    • ok
  • L339-341
    • Rephrased in a different way, "best-fit set" -> "central set"
  • L342
    • ok
  • L343
    • ok
  • L344+1-2
    • ok
  • L344+5
    • ok
  • L345-7
    • ok
  • L349
    • ok
  • L350-1
    • ok
  • L351-3
    • ok
  • L349-53: Is this right?
    • yes
  • L354-55
    • ok
  • L356: this is a dumb and illogical, but certainly not conservative..
    • there are no better samples available reflecting the current world average on the top quark mass
  • L358
    • We prefer to keep the current word ordering
  • L358-9
    • ok
  • L361
    • We prefer to keep the current word ordering
  • L361-2
    • ok
  • L362-3
    • ok
  • L363
    • ok
  • L364-5
    • We prefer to keep the current wording
  • L366
    • ok
  • L367
    • We prefer to keep the current word ordering
  • L368
    • ok
  • L369
    • ok
  • L370-2
    • ok, with minor changes
  • L373-5
    • ok
  • L377
    • ok
  • L378
    • Reworded
  • L379-80
    • Used suggestion by Greg Landsberg
  • L381-5
    • ok
  • L382
    • ok
  • L383
    • ok
  • L384
    • ok
  • L385
    • ok
  • L386
    • somewhat reformulated
  • L388
    • ok
  • L389
    • ok, reorder the words
  • L392
    • ok
  • L394
    • ok
  • L396: Other background fractions -> Other backgrounds
    • We prefer to keep the current wording
  • L397: These are the diboson and...
    • We prefer to keep the current wording
  • L398
    • Suggestion would change the meaning of the sentence. The component is not single top, but tW and s-channel are
  • L399
    • Rephrased differently
  • L400
    • ok
  • L401
    • ok
  • L402
    • ok
  • L406: Figures 4 and 5 show, respectively
    • ok
  • L406: single top and single antitop -> single top and antitop quark
    • Used suggestion by Greg Landsberg
  • L407: "showing" -> "with their"
    • ok
  • L407: "with a comparison" -> "and a comparison"
    • ok
  • L408: "the SM expectations, as predicted by" -> "SM expectations for"
    • ok
  • L409-10: arising from changes in Q2 scale and choice of PDF were found to be negligible, and are therefore not shown on the predicted distributions"
    • Used suggestion by Greg Landsberg
  • L409: Use Q or Q2, but not both.
    • See previous
  • Eqs 5-8: Why so many decimal figs? I would round up, and write 0.29+/-0.03(stat)+\-0.10 (syst)
    • ok
  • Summary should follow all the tables and figures
    • ok
  • L419-20: -> A first measurement was presented of...
    • would rather keep "the"
  • Table3:
    • Systematic names: Fix according to definitions given in the text, and use standard format
      • ok
    • QCD -> MJ
      • ok
    • "limited MC" -> "limited MC events"
      • ok
    • How do W+jets p_T reweighting and shape reweighting differ?
      • Shape reweighting reweights the shape of cos(Theta*) to Sherpa distribution. Clarified the label
    • Switch order of caption and results
      • ok
    • Caption: "shift" -> "shifts"
      • ok
  • Figure4:
    • y-axis (also Fig. 5): Use "bin", but note that the distribution is not normalized to unity! Changing the scale by a factor of three, will make it right!
      • ok
    • legend, x-axis - capitalize "unfolded"
      • ok
    • Pythia6, Pythia8 - I don't think that these are single words?
      • ok
    • Caption (also Fig. 5) - "with respect to" -> in
      • Changed to "as a function of"
    • Caption L2: top and antitop quarks
      • Changed to "top quark and antiquark"
    • Caption L2-3: "Error bars" -> "Vertical bars"
      • ok
  • L421: Why does this differ from the standard 19.7
    • Changed to 19.7
  • L422: "difference of 1.7 compared to" -> "disagreement of 1.7 standard deviations with"
    • we do not want such a statement
  • L424: "with respect" -> relative
    • We prefer to keep the current wording
L425: Remove hyphen from top-quark
    • ok
  • L426: Use more sensible units, e.g., 0.26+-0.11
    • ok
  • Figure5: Is there a Chi**2 value for the combined result, or, better yet, a KG value. There seems to be a definite trend in toto.
    • chi2 yields a rather good agreement (p-value=~95%) since it neglects any order the bins; KS-test is not defined for binned distributions. We think that the observed differences are best quoted by the asymmetry
  • L567: t in t-channel to italics
    • ok

Albert De Roeck

General comments

  • Shouldn’t we add for clarity a diagram with the process? This will help the selection discussion.
    • Done.
  • abstract/result: at the end the result is a 1.7 sigma effect. What is actually the theoretical uncertainty on the number (not given in the results section). This difference is not likely a statistical fluctuation as bit the top and anti-top both show this discrepancy, but can of course be a systematic. I assume the systematics have been carefully scrutinised in the group ( I cannot think mimedately on anything more except a possible dilution of the effect by the background that we did not catch one way or the other). In Fig 5 our data looks undeniably flatter than the expectation. Could we discuss this a bit more in the results section? Now we sort of say we have 1.7 sigma and that is it… It leaves the reader wondering…
    • added: "The observed asymmetry in data is smaller than the prediction. Separate results from exclusive top quark or antiquark events are compatible within the uncertainties. No single source of systematic explains this difference."
  • How does the analysis/expectation depend on the chosen top mass? We take a value that seems a little high compared to our own measurements (pushed up I believe by the Tevatron measurements). I assume the top mass effect is not too big but I wonder what you would get in data-TH comparison if it would be 172.5 GeV.
    • For what concerns our experimental result, we are already providing the answer, as we have a systematic from top mass, see table 3. For what concerns the expectation, no dependence on the top mass is expected at all.

Details

  • line 25 “…because of because…”
    • ok
  • section 2: add “and objects” in title of this section.
    • Followed suggestion by Tom Ferbel
  • line 52: correction for zero suppression only mentioned here and not in line 55?
    • line 55 refers to electrons not photons
  • line 77-78: Do we see significant effects from the flavours = 4 to flavours = 5 at the end? In some other analyses in e.g. the Higgs this lead to change of the expectation for the process. We do not seem to come back to that in the paper (unless I missed it)
    • this is captured in the generator modeling uncertainty where powheg (5FS) is compared against aMC@NLO (4FS)
  • line 110 “Geant4” is not written in the standard format as the authors like (see other CMS papers, or their web page)
    • ok
  • do we require the leptons to be connected to the primary vertex? How do we define the primary vertex. Given that we work in a relatively high pile-up environment it is allows important that we specify in our papers how we determine the primary vertex.
    • PV definition has been added
  • line 139: how was this calculated (i.e. which method was used for this analysis for the pile-up mitigation? I believe we should always write that explicitly
    • added CHS ref CMS-PAS-JME-14-001
  • line 156: do we use CR and SR symbols later? I did not see it but may have missed it. If not we do not need to define CR and SR explicitly
    • symbols complete removed now
  • line 283-286: perhaps explain what a log-normal constraint means in 1/2 sentence
    • we believe that this is common knowlegde at least in the field of statistics
  • line 308-312: I suggest we give a bit more more details on this test (i.e. some numbers)?
    • we believe that this may be rather technical: details about the generation of the Comhep samples can be found in the provided references; the observed bias is quoted in the systematic table
  • line 345: how was this weighting (scale dependence without parton-shower changes) tested? I guess this was done with full samples at different scales? Doesn’t it mean that we have these samples then already to do the full job? If these were statistically limited then of course also the test is statistically limited…
    • it was tested if the smoother sample obtained through reweighting is statistical compatible with the official sample. we believe that by using the official samples an unjustified large uncertainty through the limited MC-statistics is introduced
  • References: Note that in refs 23 & 24 you have a different spelling for the first author “Hoeche”…
    • ok

Wolfgang Adam (HEPHY-Vienna)

Type A:

  • General: consistency of hyphenation (e.g., "top-quark" vs. "top quark" in the abstract)
    • Removed hyphens
  • L301” “constributions” -> “contributions” or “distributions” smile
    • ok
  • L64 / 66: it would be better to use a consistent notation for the "miss" quantities: either superscript "miss" or a slash.
    • ok
  • L125: no need to redefine pseudorapidity ; similar L163. Also repeated definitions in the list L172-181
    • ok for the event selection; we like to keep it for the input variable description to make it more clear
  • L262: "right-hand panels" -> bottom panels
    • ok
  • L286: "+-50% log-normal constraint" is colloquial
    • changed to: "constraint of ... using a log-normal prior"
Type B:
  • L23 / L25: muon - shouldn't you mention earlier that you are using the muon channel?
    • we do not believe that it needs to be explicitly mentioned in the theoretical part of the introduction
  • L187 the cut on BDT(W,tt) is mentioned more than once in the paper. OTOH, the cut on BDT(QCD) is never mentioned. Since there are plots related to it in Fig. 1, the cut value should be specified somewhere in the paper.
    • Added a sentence, the cut value is -0.15
  • L223: Did you study to which extent the BDT(QCD) shape of non-QCD template can be affected by the top polarisation itself? Shouldnt this be mentioned in the paper?
    • all BDT input variables had been carefully chosen not to be correlated strongly with cosTheta. On top of it, the analysis strategy has be extensively tested by injecting anomalous couplings samples (with different polarizations) and only a small bias was found (see unfolding section for details)
  • Fig. 3: any comment on the small trend in the ratio?
    • ok
  • LL281-286: the constraints described here are different wrt. the common 20% constraint used in QCD determination fit stated on l. 223. Did you check that this does not introduce biases?
    • in the BDT QCD, those components are grouped together. The QCD fit is then only performed in the QCD enriched region (below threshold). Therefore, no uncovered bias by the 50% uncertainty on QCD is expected here.
  • Section 7: similar comment as “l. 223”. To which extent does the signal BDT(W,tt) discriminant shape depend on polarisation? How large is the bias from using a fixed signal template in this fit.
    • ok
  • Table 2: Is the rightmost column the sum of previous two columns? It seems that tt and signal do not add up even assuming rounding. If it is not a simple sum, it should be mentioned somewhere.
    • It is not a simple sum, because there are separate fits for the 'top', 'antitop' and 'top and antitop' cases, so this is from the uncertainty of the fit. added explanation
  • LL361-366: Given that this paragraph describes the largest systematic uncertainty according to the Table 3, it is much too unclear. The first part of this paragraph seems to contradict the last lines in section 6.2 which state that the costh* distribution is not significantly affected by the tratment of the b and c quark masses. The 50% variation of the heavy flavour fraction is nowhere justified. The second part of this paragraph seems to be a repetition of the previous paragraph (about W pT-reweighting uncertainty).
    • this should be more clear now - first paragraph moved to the end; second part is referring to costheta reweighting not w boson pt
  • Table 3: it would be helpful to have this table reflect better the text of the section 9 what concerns the order as well as the name of the systematic uncertainty contributions.
    • ok
  • L415: what is the uncertainty on the predicted value?
    • as stated in the result section, the uncertainties are too small to be shown on the prediction (<~0.001)

Sung Won Lee (TEXAS-TECH)

type A

  • Abstract L2: approximately 20 fb^-1 ==> better to remove 'approximately' and use 19.7 fb^-1 instead of 20
    • ok
  • Abstract L8: SM ==> standard model
    • ok
  • L25: In this analysis, ==> better to avoid the same format (see L18)
    • Reformulated
  • L25: - L26 because of because ……… because of ….. ==> ???
    • ok
  • L27: LO ==> the leader-oder (LO)
    • Changed to leading order (LO)
  • L36: parton-level version ==> 'version': suggest to use other word. How about “information”?
    • Reformulated the sentence.
  • L68: [8]: defined already in L48
    • ok
  • L75-L76: The active tenses are used too many places with similar pattern : 'we XXXXX' e.g. also see L144, L266, L272, L275, L283, L286, L359, L368, L402, L412, L416 Better to rephrase some of them.
    • ok
  • L79: leading-order ==> LO (see L27)
    • ok
  • L79: p_T ==> transverse momentum p_T (see L121-122)
    • ok
  • L83: PYTHIA ==> which version? Since you use both PYTHIA6 and 8, better to specify it in L83
    • ok
  • Section.3: Good to mention about SHERPA since SHERPA is used in section 6.2
    • ok
  • L114: \MET ==> 1st time to use the MET symbol throughout paper. So, good to define this. i.e. missing transverse energy \MET
    • Defined it at the enf of Section 2
  • L115: W-boson ==> need a hyphen?
    • Hyphen is not needed
  • L125: A transverse … a transverse … BTW, words 'transverse momentum' and 'pseudo rapidity' have been defined in L121 and L122.
    • ok
  • L130: transverse momentum ==> p_T
    • ok
  • L138-139: 'additional' is used twice.
    • ok, used suggestion by Tom Ferbel to reformulate
  • L138: proton-proton: Apparently, this is the first time to say 'proton-proton'. Can you say this a bit early? Somewhere in Sec.1 or L68-69 ?
    • ok
  • L149: data-MC scale factors: need a little more clarification.
    • -> data-simulation efficiency scale factors
  • L156: a control region (CR) and a signal region (SR) ==> a signal region and a control region remove CR and SR; these have not been used throughout the manuscript.
    • legacy -> complete removed now
  • L164, L165, L172: m_\mu\nub ==> why not use b\mu\nu ? (see L113; suggest to use the same format)
    • ok
  • L167: P_i ==> lower p, no? BTW, what’s P_i here?
    • ok
  • L169: was used ==> use present tense.
    • ok
  • L177: transverse energy ==> transverse momentum
    • ok
  • L189: variables ==> observables
    • we think that "variables" fit better here since it is a sort of common term: "BDT input variables"
  • eq.2: no space between d and cos
    • ok
  • L213, L255: better to explain 'Orthogonal'a little bit
    • rephrased
  • L222: performed, ==> remove comma
    • ok
  • L240: expectations ==> predictions (also see L363)
    • ok
  • L247: with a W produced in association with jets from gluon fragmentation, ==> simply W+jets (since it’s defined already in L84)
    • we explicitly referring here to the case W+gg which is a subset of the W+jets process
  • L272-273, also L332: maximum likelihood fit ==> ML-fit (see L221)
    • ok
  • L318 (also L424): charged leptons ==> charged muons
    • ok
  • L344: matrix element ==> matrix element (ME) parton shower ==> parton shower (PS) through the matrix element with scale ==> through the ME with scale
    • ok
  • L345: Dedicated simulated samples ==> ???
    • we are referring here to MC samples that were officially generated
  • L346: parton shower ==> PS
    • ok
  • L357: 173.3 \pm 0.8 GeV ==> suggest to quote exact the PDG value: 173.34 \pm 0.76 GeV
    • there is no extra information for this analysis in doing that
  • L368: p_T^t ==> top p_T
    • Defined p_T^t
  • L373-374: Matrix Element and PartonShower ==> ME and PS
    • ok
  • L395: 'used' are used twice in the same line
    • ok
  • L420: presented ==> performed
    • we are referring here to the paper not the analysis so we like to keep "presented"
  • Fig.1,2,3:
    • As for the legend: 'Njet Mtag' e.g. 2j1t and 3j2t It would be good if you could use a full words; 2jets & 1 tag or something like that to distinguish between 't' in Fig4,5 and 't' in Fig.1,2,3
      • ok
    • DY+VV ==> DY+jets \& diboson
      • ->W/Z/diboson
    • s+tW ==> s-channel + tW
      • ->s-ch./tW
  • Fig.3,4,5: x-axis label: cos\theta_l^* ==> cos\theta_\mu^*
    • ok
  • Table.3:
    • muon trigger efficiency or muon trigger
      • ok
    • W+jets W pT reweighting ==> W+jets p_T^W reweighting
      • ok
    • limited MC ==> MC statistics
      • Used suggestion by Tom Ferbel
    • BTW, isn’t that 10^-2 ?
      • ok
  • References:
    • L505: Technical Report ==> CMS Physics Analysis Summary
      • ok
    • L523: T. Sjstrand ==> T.Sjo ̈strand
      • ok
    • L545 & L583: Can you use same format for Conf. Proc. ?
      • ok
    • L537-540: Hoche or Hoeche?
      • ok, it's Höche
    • L546: remove '[,831(2012)].'
      • ok
    • L551: I think [27] has been published in Comput. Phys. Commun. 182 (2011) 2388. Please check this.
      • ok
    • L591: remove 'no. 2'
      • ok

type B

  • Section.1: We think it would be useful to have a Feyman diagrams to visualize the processes.
    • Done
  • L75-L76: You use aMC@NLO with PYTHIA8. Any particular reason to use PYTHIA8? Why not PYTHIA6?
    • NLO generators cannot be interfaced with Pythia6 because of subtraction terms
  • L241-L243: Do you know the reason why? Good to have some explain here.
    • extensive studies have been performed since this was already observed at 7 TeV but no explanation has found so far. It may be a in-depth difference inside the showers of the generators but we are no experts on this.
  • Section.9: Good to have a sentence summarizing the total systematic uncertainty (with number) at the end of the section
    • these numbers are quoted already in the result section directly after it
  • L369: not clear yet ==> better to say this more qualitatively; BTW, Top pT reweighting is generally recommended?
    • The recommendation is to not apply it when ttbar is the signal; see the "clarification 30 Jan.2015" in this wiki. Here ttbar is an important background, and if we don't reweight it, several plots (e.g., BDT inputs) in the ttbar-enriched control regions would appear discrepant and therefore send the erroneous message that we don't understand this background, while instead we know the reason for the discrepancy, as demonstrated by the fact that those discrepancies disappear when reweighting is applied.
  • Section.10: Good to have more discussions in the result section
    • ok

Nikos Varelas (UIC, CHICAGO)

type A

  • Abstract: first sentence does not read well at all, rephrase, possibly breaking up in two.
    • ok
  • Abstract: SM --> "Standard Model"
    • ok
  • L 25: because of because typo, fix.
    • ok
  • L 35: "A top quark candidate is reconstructed in whose rest frame the angle between the muon and the recoiling jet is calculated" -> rephrase.
    • ok
  • L72: "The tau decays" -> “...showering, in which the tau decays..."
    • ok
  • L77: Clarify what "the results" is referring to here.
    • ->measured results
  • L 158, add comma after “category”, so "category, a top-quark"
    • ok
  • L 178 “transverse W-boson mass” à “transverse mass of W-boson”
    • ok
  • L 196 & 213: orthogonal selections is jargon, say mutually exclusive samples or statistically independent samples.
    • ok
  • L211: "3jets 2tag" -> "3jets 2tags", throughout the paper (to be consistent with 2jets 0 tags in L 238)
    • ok
  • L 411: the the unfolded
    • ok
  • L 425-426 – the format of quoting the values is different from the abstract, and there is no apparent reason for that. It would be best to use consistent format.
    • ok
  • Figures: In most cases, data points have no errors, are they smaller than the markers? Can you use smaller markers?
    • one can still observe the errors in the ratios. we fear that using smaller markers will make the data points harder to see properly
  • Figure 1, 2, 3, the labels of (y) axis are far apart from the axis. Please move closer.
    • removed 10^3 -> numbers are now longer. we like to keep this distance coherent for all plots
  • Figure1&2: Clarify the t+tbar symbol
    • symbol removed from all plots at reco level since there is actually no top for the backgrounds

type B:

  • General: How much does the polarization measured in t-channel depend on the polarization of the tops produced as ttbar pairs? Was a different ttbar polarization considered as a systematic uncertainty?
    • There is no inter-dependence between the two. In our signal, polarization arises from it is EW production. In pair production, being a QCD-mediated process, net polarization is strictly 0 in the SM (although spin correlation is non-0).
  • L136-137: The 5-10% resolution relatives to true momentum is after applying JEC. So this should move after the JEC descriptions.
    • ok
  • L138: L1 offset correction corrects the transverse energy of jet. Add "transverse"
    • ok
  • L139: Add a statement of the L2L3 correction, correcting to the true momentum. Then moved L136-137 afterward.
    • ok
  • L 141: The jets are accepted up to a large pseudorapidity of 4.5, where jet reconstruction performance can deteriorate significantly compared to mid-rapidity. Does the 10% uncertainty on jet momenta mentioned above cover the forward region (down to 40 GeV momentum threshold used)? Is there a reference to jet performance studies that could be cited here?
    • CWR of JME-13-004 has ended. If in time, we may consider to cite it
  • L161: The QCD BDT is trained in 2J1T only and applied for all categories, or different BDT trained for different categories? If BDTs is trained in 3J2T case, which b jet is selected for the training variables containing b?
    • It is trained in 2J1T and applied also to other categories, in which case the jet with the highest CSV value is used, as mentioned in the text now
  • L167: If S is sum over all muon + jets, where is the min and max comes from? Define the direction of n
    • definition improved
  • L 184: "Input variables have been individually validated in the same way” Does this imply that they were validated by looking at a plot, i.e. can agreement or not be quantified statistically? Also, line 189 "All BDT input variables have been shown to be well modelled by the MC simulation”, can this be quantified? What statistical test of agreement was done?
    • at first, we checked the distributions for obvious deviations; then we also quantified the agreement by Chi2 and KS tests. A good agreement has been observed even w/o accounting for systematics
  • L192-198: If these corrections are applied before the BDT training, could you move this paragraph before the BDT description as well, even though the corrections are small? Just to keep the flow clear.
    • ok
  • L 238, mismodeling in 2jet0tag, how was it quantified and how was the agreement after reweighting quantified?
    • we plan to add a 2j0t plot as additional material. extensive studies had been performed to understand the costheta shape. in the end, the best shape description lies somewhere between MG and Sherpa and is thus covered by the assumed systematic.
  • L 255-259: It is not completely clear from the description what is done. You use Madgraph but reweight the cosine distribution to match the prediction from SHERPA? Does the treatment of b and c masses refer to Sherpa? How did you verify that it does not affect the distributions?
    • Yes, we use Madgraph but, individually for each flavor category (from MC truth), we reweight it to match Sherpa. The treatment of b and c masses refer to Sherpa. We checked at generator level that the cosine distribution is independent from the quark masses.
  • L 262, what are the the right-hand panels of figures 1, 2 and 3? Do you refer to what is at the bottom plot on these figures?
    • Yes, corrected the reference
  • L 269: In conclusion, the ttbar modeling provided by MADGRAPH is found to be in reasonable agreement with data. -> is this after the corrections or before? If after, the sentence is misleading.
    • ok
  • L285: Change EW to W/Z+jets, for consistency in Table 1? Use consistent notation with Table 2 in describing what is combined to what.
    • ok
  • L 313: the chisq fit on Eq. 2 is it done on the data with Pt(s) as a free parameter? How is the MC used in the fit? The method needs a bit more explanation.
    • A is fitted to the unfolded costheta distribution as it reads: "the value of A is extracted using a chi2 fit of the unfolded costheta ...".
  • L 419 – “The first measurement…”: earlier a CDF paper is mentioned for this measurement, do you mean to say “first measurement at LHC”?
    • Greg Landsberg commented that Ref. [7] has not been published, and therefore it can’t be cited in this paper. Consequently, he suggests to emphasize even more that this is the first measurement of this quantity (not only at LHC). Anyway, originally we wanted to argue that Tevatron tried to measure that, in that document, but was not able to exclude the unpolarized case with that analysis (which is maybe why it was never published, in the end.)
  • Table2: Why the sum of Top and AntiTop column is not equal to "Top and antitop" column? They are off by 2 events.
    • Because there are separate fits for the 'top', 'antitop' and 'top and antitop' cases, so this is from the uncertainty of the fit

Luigi Benussi (FRASCATI)

The Frascati group doesn't have additional comments to add to the already posted by the other readers.

Ivica Puljak (SPLIT-FESB)

Type A:

  • Abstract: In the first line should be “top-quark polarisation”.
  • L39: Titile should be “CMS Detector”, as the style chose is to capitalise first letter in titles words
    • CHanged style of other headings - only first word capitalised
  • L70: "...simulation used in this study have..." -> "...simulation have…"
    • ok
  • L92: "i.e., events" -> "i.e. events”
    • ok
  • L108: delete “ that can be”
    • ok
  • L125: small “a” in “a transverse momentum”
    • ok
  • L261: change “this is” to “it is”
    • ok
  • L419: delete parentheses
    • ok
Type B:
  • Abstract: Title is “Measurement of top-quark polarisation ...”, while the abstract starts with “A measurement of the top-quark spin asymmetry, …”, and the final results is for the “top-quark spin asymmetry”, and we say twice that the measurement (of spin asymmetry and angular observable) are “sensitive to the top quark polarisation”. Therefore maybe the title should read “Measurement of top-quark spin asymmetry …
    • this was discussed with the ARC at some point as well. The conclusion was that top-quark spin asymmetry would fit better but since it is a new term (not yet well known by the community) polarization should stay since it attracts more attention.
  • Use exact luminosity, 19.7 fb-1, do not approximate to 20 fb-1 (the same in Conclusion)
    • Ok
  • Introduction: maybe to explain a bit what is the spectator quark, whose direction is used to measure which events are aligned and non-aligned with the direction of the spectator quark and maybe put the Feynman diagram of the production process
    • ok
  • L20: consider changing to “and alphaX denotes the degree of the angular correlations of decay products with respect to the spin of the top quark, the so called spin-analysing power."
    • ok
  • L25: consider changing to “In this analysis, the muon is chosen as the top-quark spin analyser because leptons have the highest spin-analysing power and since the muon identification efficiency is very high in the CMS detector.”
    • ok
  • L34,35: A sentence "A top quark candidate..." is not clear (not well phrased). Please, rephrase it.
    • ok
  • L51, 52 and 58: A term "zero-suppression effects" is not explained in the text. It could be either explained or removed altogether since it seems like a bit of a detail
    • ok, removed
  • L68: no need for reference [8] here
    • ok
  • L85: reference to MADGRAPH needed
    • ok
  • L95: reference to SHERPA needed
    • ok
  • L102: is BR(W->lnu) included in sigma(W+jets)?
    • ok
  • L110: We presume that a sentence "All generated events undergo..." is relevant for all generated samples and not only for the effects of pileup. Therefore, the sentence could be started in new line (as separate paragraph).
    • ok
  • L126: more details about quality and ID of isolation muon is needed
    • ok
  • L132: more details about quality, ID and isolation for additional leptons is needed. Do we really require no muons in abs(eta) < 2.5, while isolated muons are restricted to eta of 2.1?
    • yes
  • L135, 137: For better clarity (to stress it further), a sentence "The jet momentum is determined..." could be divided in 2 sentences, maybe as follows "The jet momentum is determined as the vectorial sum of all particle momenta in the jet. It was found from simulation for the jet momentum to be within 5% to 10% of the true momentum over the whole pT spectrum and detector acceptance."
    • ok, sentence broken up
  • L159: maybe describe in short the top-quark candidate reconstruction here and then say that more details are available in the reference 39.
    • ok
  • L285: maybe “EW processes” instead of “W/Z+jets”, as this is used in Table 1.
    • that would refer to a theory and not the process, see also Tom Ferbel's comments on QCD to that
  • Table 2: we usually don’t put uncertainty in data, i.e. number of data events is sufficient
    • indeed
  • Summary should be improved. For example the sentence at L422 could be phrased in positive way (btw. is it the cos(theta*) distribution that shows 1.7 difference wrt SM expectation, or the asymmetry extracted from it, as said on line 414?)
    • ok
  • Figure 1, 2 and 3: A label "t+tbar" on the plots is not explained in the captions (combination of top and antitop selection)
    • removed
  • Figures 4 and 5: no need for “=“ signs in error bar legends
    • ok
  • For the systematic uncertainties, can you comment on the BDT treatment? Is any systematic associated to BDT selection?
    • systematics that affects the BDT are considered in the analysis through modified samples. However, there is not specific systematics connected to the usage of the BDT itself or its chosen working point.

Sijin Qian

  • (1) To be consistent with all other CMS papers, the captions of all Tables should be placed above the Tables instead of underneath.
    • ok
  • (2) Throughout this paper, in all terms of "E/T", there seems a gap between "E/" and "T", it will be looked better if the gap is removed, i.e. "E/ T" --> "E/T"
    • ok
  • (3) Throughout the paper, to be consistent with all other CMS papers, the Equation index should be put into a bracket, and the 1st letter of "equation" should be in the upper case, e.g. L38 should be changed from "to the polarisation through equation 1," --> "to the polarisation through Equation (1)," or "to the polarisation through Eq.(1)," Other places where also need to be changed by the same way are at L208, L314, L411, and L417.
    • ok
  • (4) Throughout the paper (including in Eqs., inside Tables, in Table and Figure captions, in superscripts and subscripts, etc.), to be consistent with elsewhere in this paper,
    • (a) e.g. L101, L216 and L367, etc. for the font of "t" (or "tbar") for "top (or anti-top) quark", some places should be changed from "italic" --> "non-italic" e.g. L19, it should be changed from "Pt(italic) represents the top-" --> "Pt(non-italic) represents the top-" Other places where also need to be changed by the same way are at Eq.(1), in the right-handed side (in subscript),
      • L19, Eq.(2), in the right-handed side (in subscript),
      • L200 (in subscript),
      • L368 (in superscript),
      • Eqs.(5)-(8), in the arguments of the terms Amu(t), etc. (total 6 places),
      • Table 3's header row, in the arguments (total 4 places), and
      • Table 3's caption, in the arguments (total 4 places).
    • ok
    • (b) e.g. L9, L30 and L33, etc. for the fonts of "t-" and "s-" in the "t-channel"s and "s-channel"s, some places should be changed from "non-italic" --> "italic" These places are in the title of this article (the 1st line),
      • L98 (in two places of subscript),
      • L398,
      • L420,
      • Table 3, in the header column, the 5th and 27th rows (total two places), and L567.
    • ok
    • (c) e.g. L155, L232, L303 and L364, etc., for the "Section xx", it should not be abbreviated, e.g. L93 should be extended from "as detailed in Sec. 6.1." --> "as detailed in Section 6.1." other places where also need to be changed by the same way are at
      • L183,
      • L201,
      • Fig.1's caption (the 2nd line),
      • Fig.2's caption (the 3rd line),
      • Fig.3's caption (the 3rd line),
      • L264, and
      • L278.
      • Only written out if at beginning of sentence
  • (5) L197 and L382-383, two variables (JER and JES) should be explained at their 1st appearances in text at these two places, but since they have not been used in whole paper except only once each in Table 3, I'm not sure whether you would like
    • (a) either to add two explanation on L197 and L382 each, i.e.
      • L197: "difference in jet energy resolution compared to data [36]." --> "difference in jet energy resolution (JER) compared to data [36]."
      • L382-383: "on the jet energy scale [36]." --> "on the jet energy scale (JES) [36]."
    • (b) or to keep L197 and L382-383 unchanged, but to spell out the JER and JES in the header column and the 11-12th rows, i.e. "JER JES" --> "Jet energy resolution Jet energy scale"
    • ok
  • Page 0, in Abstract: (6) The 7-8th line, I'm not sure whether the subscript "mu" in the variable "Amu" should be explained, i.e. "is used to extract a top-quark spin asymmetry of Amu = ..." "is used to extract a top-quark (decaying to a muon) spin asymmetry of Amu = ..." or "is used to extract a top-quark (with an isolated muon in the final state) spin asymmetry of Amu = ..."
    • ok
  • Page 1: (7) L25, a rather obvious duplication, i.e. "because of because leptons" --> "because leptons"
    • ok
  • (8) L27, the "LO" should be explained at its 1st appearance in text here, i.e. "is exactly 1 at LO in the SM." --> "is exactly 1 at the Leading Order (LO) in the SM."
    • ok
  • Page 2: (9) L39, to be consistent with other Section titles in this paper, the 2nd word of title should start with the letter with upper case, i.e. "2 CMS detector" --> "2 CMS Detector"
    • Changed other section titles to lowercase instead
  • (10) L40-41, the "superconducting solenoid" is repeated on these lines, the latter one may be shortened from "is a superconducting solenoid of 6m internal diameter, providing a magnetic field of 3.8 T. Within the superconducting solenoid volume are a" --> "is a superconducting solenoid of 6m internal diameter, providing a magnetic field of 3.8 T. Within the solenoid volume are a" similar as many newly published CMS papers (e.g. EXO-12-037 and SMP-14-003, etc.) have expressed.
    • ok
  • (11) L79, the "pT" should be explained at its 1st appearance in text here instead of L121, also the "LO" can be used to shorten from "the leading-order diagrams of the 5FS and 4FS based on the pT of the ..." --> "the LO diagrams of the 5FS and 4FS based on the transverse momentum (pT) of the ..." Correspondingly, L121 can be shortened from "with transverse momentum pT > 24 GeV and ..." --> "with pT > 24 GeV and ..."
    • ok
  • Page 3: (12) L84, L87 and L89, the "DY" and "ME" should be explained at their 1st appearances in text on L84 and L87, i.e. "and Drell-Yan in ... at matrix-element level in ..." --> "and Drell-Yan (DY) in ... at matrix-element (ME) level in ..." Then, L89 can be shortened from "generated by the matrix element and ..." --> "generated by the ME and ..."
    • ok
  • (13) L102-103
    • (a) The "BR" should be explained at its 1st appearance in text here, but since it has never been used afterward in whole paper, I'm not sure whether it can be simply spelled out here;
    • (b) the font of the lepton "l", to be consistent with elsewhere (e.g. the line below L199, etc.) in this paper and other CMS papers, should be changed to a special one, i.e. "sigma(...) * BR(Z/gamma* -> l+l-) = 3,504 pb with a generator-level threshold of ml+l- > 50 GeV)." --> "sigma(...) * Branching-ratio(Z/gamma* -> l+l-(special font as on the line below L199)) = 3,504 pb with a generator-level threshold of ml+l-(special font as the above line) > 50 GeV)."
    • ok, new symbol should make it clear now
  • (14) L125-126, it may be shortened from "A transverse momentum, pT, of at least 26 GeV, a pseudorapidity, eta, with absolute value smaller than 2.1," --> "A pT >= 26 GeV, a |eta| < 2.1,"
    • ok
  • Pages 3-5: (15) L116 and L206, to be consistent with elsewhere in this paper (e.g. L173 and L181), the font of j' (at two places) should be changed from "jet(j'(italic))" --> "jet(j'(non-italic))"
    • ok
  • (16) L129, the "phi" should be explained, and as the numerical values of the angle phi have been implicitly shown here and an angle can be measured in either radians or degrees; therefore, the unit of phi may should be specified; "DeltaR = sqrt(...) = 0.4 around ..." --> "DeltaR = sqrt(...) = 0.4 (where phi is the azimuthal angle in radian) around ..."
    • ok
  • (17) L130, it can be shortened from "this value must be less than 12% of the transverse momentum of the muon." --> "this value must be less than 12% of the muon pT."
    • ok
  • (18) L134, the expression of "kt(non-italic)" here is different from the one in the article title of [31], where it it "kt(italic)" I'm not sure whether they should be consistent or not.
    • ok
  • (19) L168, it'll be looked better if the long dash symbol is changed to a shorter hyphen and two spaces before and after it are removed, i.e. "is a unit vector in the r --- phi plane." --> "is a unit vector in the r-phi plane."
    • changed otherwise
  • (20) L169-170 and L190, to be consistent with the tense in this Section, I'm not sure 4 words of "was" on these lines should be changed to "is" or not.
    • ok
  • (21) L177, the word of "energy" should be corrected from "the transverse energy of the b-tagged jet, pbT;" --> "the transverse momentum of the b-tagged jet, pbT;"
    • ok
  • (22) L178, to be consistent with the font in the paper, the subscript "T" of pT under the square-root should be changed from "pT(italic)mu" --> "pT(non-italic)mu"
    • ok
  • (23) L181, as the variable "HT" has not been used afterward in whole paper, it eventually can be removed, i.e. "of the hadronic final state system, HT = (...)T." --> "of the hadronic final state system (...)T."
    • we like to give the definition here to ensure to the reader that it is the same as in other analyses
  • (24) The line below L199, to be consistent with elsewhere (e.g. L73, L76 and L80, etc.) in this paper about the font of "b", it should be changed from "X (= W, l, nu, b(italic))" --> "X (= W, l, nu, b(non-italic))"
    • ok
  • Pages 6-10: (25) Figs.1-3
    • (a) In the vertical axis label and the legend of each plot, the 1st letter of each line should be in the upper case, i.e.
      • (i) the vertical axis: "data/MC" --> "Data/MC"
      • (ii) the legend (the 1st line and the bottom line): "data total syst." --> "Data Total syst."
ok
    • (b) The distance between the upper and lower plots seems better to be increased a little bit, now the axis label of "BDT..." of upper plot is too closer to the the top line above the lower plot.
    • ok
    • (c) In the caption of each Figure,
      • (i) the "top" and "bottom" indicators on the 1st two lines are accidentally coincide with the names of two quarks, the problem may be solved by changing indicators to the "upper" and "lower".
      • enumerated with subfigures (a,b) now
      • (ii) Each Figure has two plots (thus two bottom panels), therefore, the subject word "panel" in the last sentence on the 3rd line should be plural, and corresponding verb and the object word should be plural too; also the word of "figure" on the 3rd line should be changed to "plot", e.g. the 1st 3 lines of Fig.1's caption (together with the item (4c) above for the full spelling of "Section"), "Figure 1: Distribution of the BDTQCD discriminant in the “2jets 1tag” (top) and “3jets 2tag”(bottom) categories. Predictions are normalised to the results of the fit described in Sec. 7. The bottom panel in both figures shows the ratio between ..." --> "Figure 1: Distribution of the BDTQCD discriminant in the “2jets 1tag” (upper plot) and “3jets 2tag”(lower plot) categories. Predictions are normalised to the results of the fit described in Section 7. The bottom panels in both plots show the ratios between ..." or, the last sentence of the above example would not be changed to the plural case by replacing a word of "both" to "each", i.e. "The bottom panel in both figures shows the ratio between ..." --> "The bottom panel in each plot shows the ratio between ..."
      • ok
      • (iii) The last two sentences of all 3 captions (on the last 3 or 4 lines) are identical. I'm not sure whether the last two sentences of Figs.2 and 3 can be replaced by something like "Other explanations are the same as the Figure 1".
      • better to write it out - otherwise a reader may need to turn pages a lot
  • (26) L227, L246 and L252, to be consistent with the tense in this Section, I'm not sure whether 2 words of "was" and one "were" on these lines should be changed to "is" and "are".
    • ok
  • (27) From L235 to L264, many places can be shortened from "control region(s)" --> "CR(s)" and "signal region(s)" --> "SR(s)" These places are CR:
    • L235,
    • L237-238 (two places),
    • L243,
    • L244,
    • L255,
    • L257,
    • L261, and
    • L263-264.
    • SR: L245,
    • L248-249, and
    • L255.
    • these terms have been removed from the draft
  • (28) L262, to be consistent with elsewhere (e.g. L182, etc.) in this paper, the 1st letter of the word "figures" should be in the upper case, i.e. "panels of figures 1, 2 and 3" --> "panels of Figures 1, 2 and 3" or "panels of Figs.1, 2 and 3" or "panels of Figs.1-3"
    • ok
  • (29) L265 and L268, two "pT"s should be changed or can be used from "a harder top-quark pT(italic) spectrum ... quark transverse momentum." --> "a harder top-quark pT(non-italic) spectrum ... quark pT."
    • ok
  • Page 11, Table 2: (30) In additional to the item (1) above for the position of caption to be moved to above the Table, in the header column and the row above the bottom row, the 2nd word should be in the lower case, i.e. "Total Expected" --> "Total expected"
    • ok
  • Page 13: (31) To be consistent with the font in the header of each paragraph, some terms should be changed from the "plain" --> "bold" These terms and their locations are
    • L358 (for "pT"),
    • L367 (for "ttbar" and "pT")
    • L381 (for "E/ T" --> "E/T" according to the item (2) above)
    • ok
  • (32) L368, to be consistent in this paper, the font of both sub- and super-scripts in "ptT" should be changed (including the item (4a) above) "a harder pt(italic)T(italic) spectrum compared to ..." --> "a harder pt(non-italic)T(non-italic) spectrum compared to ..."
    • ok
  • (33) L382-383, a hyphen should be added, i.e. "according to the eta and pT-dependent uncertainties on ..." --> "according to the eta- and pT-dependent uncertainties on ..."
    • ok
  • Page 14: (34) L413, I'm not sure whether the font of two "p"s in the phrase, i.e. "with a p(non-italic)-value of p(italic)(data|SM) = ..." should be consistent or not.
    • now consistent
  • Page 15, Table 3: (35) In the header column, all rows should start with the upper case letters, i.e. (together with the item (4b) above for the font of t- and s-channels, and the item (2) above for the extra space in E/T)
"statistical ------------------
s(non-italic)-channel fraction
------------------
mistagging
unclustered E/  T
pileup
lepton ID
lepton isolation
trigger efficiency
-------------------
top pT reweighting
unfolding bias
-------------------
generator model
top quark mass
Q2 scale t(non-italic)-channel
-------------------
limited MC
total uncertainty" -->
"Statistical
------------------
s(italic)-channel fraction
------------------
Mistagging
Unclustered E/T
Pileup
Lepton ID
Lepton isolation
Trigger efficiency
-------------------
Top pT reweighting
Unfolding bias
-------------------
Generator model
Top quark mass
Q2 scale t(italic)-channel
-------------------
Limited MC
Total uncertainty"
  • ok
  • Pages 16 and 17, Figs.4 and 5 (36) In the horizontal axis label and the legend (the 4th line) of each plot, the 1st letter should be in the upper case (total 6 places), i.e. "unfolded ..." --> "Unfolded ..."
    • ok
  • (37) In the legend (the bottom line) of each legend, a space may be better to be added after each of 2 symbols "=", i.e. "=stat., total" --> " stat., = total"
    • ok
  • (38) The captions of Figs.4 and 5 are almost identical except a couple of words on the 1st two lines, thus they eventually can be combined with a modified Fig.4's caption, i.e. "Figure 4: The normalised differential cross section with respect to cos(theta*unfolded) for top (left) and antitop (right) compared to ..." --> "Figure 4: The normalised differential cross section with respect to cos(theta*unfolded) for top (upper-left), antitop (upper-right) and ttbar combined (lower) compared to ..."
    • we like to keep it as is; also since sizes are different
  • Pages 18-21, in the References Section, (39) L505, in [11], to be consistent with other PAS Refs. (e.g. [9] and [10], etc.) in this Section, it should be changed from "Technical Report CMS-PAS-LUM-13-001, 2013." --> "CMS Physics Analysis Summary CMS-PAS-LUM-13-001, 2013."
    • ok
  • (40) L513, in [14], to be consistent with other JHEP Refs. in this Section, the document index should be shortened from "JHEP 0711 (2007) 070," --> "JHEP 11 (2007) 070," Other ones which also need to be changed by the similar way are Refs.[15], [17], [23], [30] and [37].
    • ok
  • (41) L523, in [18], the 3rd letter in the author's family name is missing, i.e. "[18] T. Sjstrand et al.," --> "[18] T. Sjo(with two dots on the top of "o")strand et al.,"
    • ok
  • (42) The "year" numbers should be given for Refs.[27], [39] and [46]. If there would be problems to display the year number with the default bib file, it may be fixed by changing from "article" to "unpublished" in the bib file.
    • ok
  • (43) L558, in [30], to be consistent in this Section and this paper, the font of "pp" in the article title should be changed from "cross section in pp(italic)" --> "cross section in pp(non-italic)" Another one which also needs to be changed by the similar way is [33] (for the "ttbar").
    • ok
  • (44) L565, in [33], to be consistent in this Section, a space should be added after the symbol "=" in the article title, i.e. (together with the item (43) above for the font) "=8 TeV in multijet, ttbar(italic) and boosted" --> "= 8 TeV in multijet, ttbar(non-italic) and boosted"
    • ok
  • (45) L570-571, in [35], to be consistent with other CMS papers, the document name should be changed from "Technical Report CMS-DP-2013-009, 2013." --> "CMS Detector Performance Summary CMS-DP-2013-009, 2013."
    • ok
  • (46) L590-591, in [43], (a) a word of "effect" in the article title may should start with an upper case letter (b) to be consistent in this Section, an extra index after the year number should be removed, i.e. "and their effect on Lepton Charge Asymmetry from W Decays”, Eur. Phys. J. C73 (2013), no. 2, 2318," --> "and their Effect on Lepton Charge Asymmetry from W Decays”, Eur. Phys. J. C73 (2013) 2318,"
    • ok
  • (47) L598, in [46], to be consistent in this Section, the 1st letter of "The" in the author part should be removed, i.e. "[46] The ATLAS, CDF, CMS and D0 Collaborations," --> "[46] ATLAS, CDF, CMS and D0 Collaborations,"
    • ok

Greg Landsberg

type A

  • General: the CMS style suggest not hyphenating “top quark” whether it’s a [permanent] compound modifier or not. Please, correct in the title [twice], Abstract, LL1,2,4,6,7, main text, LL9,14,18+1,19,22,25, 28,30,37,70,83,95,97,98-99,115,119,158,164,172,180,199,199+1,199+2,200,201,204,265,281,282,293, 354,355,419,425.
    • ok
  • Title: t-channel ["t" in math, not Roman].
    • ok
  • Abstract: LL2-3: based on a sample of pp collisions at a centre-of-mass energy of 8 TeV corresponding to an integrated luminosity of approximately 20 fb−1.
    • ok
  • L8: add blank spaces before the opening parentheses;
    • ok
  • L5: parity-violating (V−A) nature of … Wtb vertex means;
    • Modified according to Tom Ferbel's suggestion
  • LL6-7: decay products inherit information of the top quark spin;
    • we like to keep "retain" since it underlines that the top quark does not hadronize before decaying and thus would loose spin coherence
  • L12: delete “Equally” [not needed] and start the sentence with “New physics”;
    • Changed according to recommendations by Bastian Kargoll and Joanne Cole
  • Eq. (1), L19: subscript “t” should be typeset in Roman.
    • ok
  • L24: aligned or antialigned with;
    • ok
  • L25: delete “because of”;
    • ok
  • L26: of the purity of muon identification;
    • ok
  • L32: The analysis strategy is as follows. After;
    • ok
  • L33: high-purity t-channel single top quark events;
    • ok
  • L38: through Eq. (1), is calculated.
    • ok
  • L39: The CMS detector [can’t start section heading/sentence with an acronym.
    • ok
  • LL49-50: The particle-flow algorithm [9,10] reconstructs … individual particle in an event with an optimised;
    • ok
  • L67: Data and simulated samples;
    • ok
  • LL68-69: at a centre-of-mass energy s√=8 TeV, corresponding to an integrated luminosity;
    • ok
  • L71: next-to-leading-order (NLO) MC generator;
    • ok
  • LL71-72: interfaced with {\sc pythia}; The τ lepton decays;
    • ok
  • LL75-76: interfaced with {\sc pythia};
    • ok
  • L78: and from a leading-order (LO) {\sc CompHEP } generator [19], interfaced with {\sc pythia};
    • ok
  • L79: between the LO 5FS and 4FS diagrams based on;
    • ok
  • L81: Wtb coupling.
    • ok
  • L84: add a comma before “and”; Drell—Yan [en-dash, not a hyphen] (DY) in;
    • ok
  • L86: τ lepton decays.
    • ok
  • L89: generated from the matrix elements and by;
    • -> generated by ME simulation
  • L92: Multijet QCD events [can’t start a sentence with an acronym];
    • Changed to 'Multijet events'
  • L97: next-to-next-to-leading-order;
    • ok
  • L98: t-channel, s-channel [hyphen, not a minus sign];
    • ok
  • L101: Z/γ∗+jets production cross sections;
    • ok
  • L112: Event selection;
    • ok
  • L115: W boson;
    • ok
  • L116: hard-scattering process;
    • ok
  • L119: from the top quark decay.
    • ok
  • L125: do not capitalize “A”;
    • ok
  • LL128,132: add a comma before “and”;
    • ok
  • L134: particle-flow candidates, clustered with the anti-kT algorithm;
    • ok
  • L135: with a distance parameter of 0.5.
    • ok
  • L144: b tagging algorithm [PubComm recommendation];
    • ok
  • L146: selection [33] is applied on the b tagging discriminant; light-flavour jet mistagging;
    • ok
  • LL147-148: in the signal simulation. The b tagging performance;
    • ok
  • L153: jets (0, 1, or 2).
    • ok
  • L158: from the b jet;
    • ok
  • L159: in Ref. [34].
    • ok
  • L160: b tagging discriminant is used;
    • ok
  • L161: Multijet QCD events;
    • Changed to 'Multijet events'
  • L165: the absolute pseudorapidity;
    • ok
  • L167: subscripts “min”, “max” in Roman;
    • ok
  • L170: separate the signal;
    • ok
  • L173: b tagging discriminant;
    • ok
  • L181: final-state system;
    • ok
  • L183: the QCD multijet shape and normalisation; Section 6.1.
    • Changed to "multijet events shape and normalization"; Only written out if at beginning of sentence
  • L190: overtraining;
    • ok
  • LL194-195: ``tag-and-probe” method [35] from DY data; b tagging and mistagging efficiencies;
    • ok
  • L199: do not capitalize words of the section title, except for the first one;
    • ok
  • L201: in Section 1.
    • Only written out if at beginning of sentence
  • L203: momentum is antialigned;
    • ok
  • Fig. 1 caption, L2: in Section 7. L4: background prediction and error bars indicating statistical uncertainties.
    • ok; "Section" only written out if at beginning of sentence
  • Fig. 2 caption, L1: Distributions of the BDTW,tt¯ discriminant in the;
    • ok
  • L3: Section 7. L5: prediction and error bars indicating statistical uncertainties.
    • ok; "Section" only written out if at beginning of sentence
  • L208: based on Eq. (2) is used;
    • ok
  • L212: Background model studies;
    • ok
  • L215: QCD multijet events;
    • Changed to 'multijet events'
  • L216: b-tagged jet multiplicities;
    • ok
  • L218: Multijet background estimation;
    • Changed to 'Multijet events background estimation'
  • L219: QCD multijet background;
    • Changed to 'multijet events background'
  • L221: maximum likelihood (ML);
  • L222: QCD multijet;
    • Changed to 'multijet events'
  • L226: using the inverted isolation requirement.
    • Reworded
  • L227: QCD multijet;
    • Changed to 'multijet events'
  • L229: uncertainties in the QCD multijet yield;
    • Changed to 'multijet events yield'
  • L233: The W+jets model validation and correction;
    • ok
  • L234: QCD multijet;
    • Changed to 'multijet events'
  • L247: W boson produced;
    • ok
  • L248: add a comma before “but”;
    • ok
  • Fig. 3 caption, L3: Section 7. L5: background prediction and error bars indicating statistical uncertainties.
    • ok; "Section" only written out if at beginning of sentence
  • L260: The tt¯ model validation;
    • Would rather leave out 'the'
  • L264: in Section 7.
    • Only written out if at beginning of sentence
  • LL267-268: to the top quark transverse momentum.
    • Used suggestion by Tom Ferbel
  • L271: Extraction of the signal and background yields;
    • ok
  • LL272-273: simultaneous ML fit;
    • ok
  • LL275,276: QCD multijet;
    • Changed to 'multijet events'
  • L277: inverting the isolation selection, as described earlier;
    • ok
  • L278: in Section 6.1.
    • Only written out if at beginning of sentence
  • L282: Top-quark-like processes; single top quark production;
    • ok
  • L283: from simulation, with a ±20\% log-normal;
    • changed to "...constraint of ... using log-normal prior"
  • L285: add a comma before “and”;
    • ok
  • L286: from simulation, with a ±50% log-normal constraint.
    • changed
  • Table 1-2 captions: move the captions above the tables;
    • ok
  • L299: τ lepton decay;
    • ok
  • L302: add a comma before “for which”;
    • Rephrased, using suggestion by Tom Ferbel
  • L314: that Eq. (2) is valid.
    • ok
  • L320: Systematic uncertainties;
    • ok
  • L321: in this paper is potentially affected;
    • Used suggestion by Tom Ferbel
  • L331: Maximum likelihood fit uncertainty: This;
    • ok
  • L332: from the ML fit;
    • ok
  • L335: interfaced with {\sc pythia} 8;
    • ok
  • L339-340: the names of PDF sets should not be in small caps - just regular font; CT10 collection [42, as well as the best fit sets;
    • ok
  • L342: The uncertainties in the;
    • ok
  • L344+1: For the signal;
    • ok
  • Eq. (3): finish the equation with a comma;
    • ok
  • L361: An uncertainty in the;
    • ok
  • L363: associated with the;
    • ok
  • L367: (top quark pT reweighting);
    • ok
  • L371: and additional systematic uncertainty in this;
    • Used suggestion by Tom Ferbel
  • LL371-372: doubling and negating the effect of reweighting.
    • Used suggestion by Tom Ferbel
  • L373: Matrix element/parton shower matching threshold;
    • ok
  • LL373-374: between matrix element and parton shower jet production in the MLM;
    • changed
  • LL374-375: W+jets processes, independently;
    • ok
  • L376: add a comma before “and”;
    • ok
  • L377: to the muon trigger, identification, and isolation efficiencies.
    • ok
  • L379: sampled by the present selection;
    • ok
  • LL37-380: for the tag-and-probe scale factor extraction.
    • ok
  • L382: η- and pT-dependent uncertainties in the;
    • ok
  • L388: b tagging: The uncertainties in the b tagging and mistagging efficiencies;
    • ok
  • L392: Multijet background yield: A 50\%;
    • Changed to 'Multijet events background yield'
  • LL394-395: Multijet background template: A shape …varying the range of inverted isolation requirement used;
    • ok
  • LL398-399: and the tW and s-channel production fro the top quark component. The uncertainty in the yield;
    • ok
  • L402: uncertainty in the;
    • ok
  • LL406-407: single top quark and antiquark production;
    • ok
  • L409: arising from renormalisation and factorisation scale and PDF variations have been found negligible for the;
    • ok
  • L411: according to Eq. (1);
    • ok
  • Eqs. (5-7): t and t¯ should be typeset in Roman, not math; Eq. (7) should be ended with a comma; add spaces before the opening parentheses.
    • ok
  • L417: (in Eq. (1)). This results;
    • changed
  • Eq. (8): t+t¯ should be typeset in Roman, not math; add spaces before the opening parentheses.
    • ok
  • LL420-421: single top quark production; based on a sample of pp collisions at s√=8 TeV corresponding to an integrated luminosity of approximately 20 fb−1.
    • ok
  • Tabel 3 caption: move the caption above the table; L2: top quark (δAμ(t)), antiquark (δAμ(t¯)), and their combination (δAμ(t+t¯)). [Note that all t’s should be in Roman!]
    • ok
  • Table 3 body, header row: all the t’s should be typeset in Roman
    • ok
  • First column: ML fit uncertainty; Diboson bkg. fraction; DY bkg. fraction; jet energy resolution; jet energy scale; lepton identification; top quark pT reweighting; W+jets W boson pT reweight.; W+jets heavy-flavour fraction; W+jets light-flavour fraction; Renorm./fact. scales, t-channel; Renorm./fact. scales, tt¯; tt¯ matching scale; Renorm./fact. scales, W+jets; W+jets matching scale; MC statistics.
    • ok
  • Fig. 4 caption, LL1-2: top quark (left) and antiquark (right);
    • ok
  • L422: The unfolded cosθ∗ distribution displays a difference of 1.7σ compared to;
    • changed differently according to other suggestions
  • L428: add spaces in front of opening parentheses;
    • ok
  • Fig. 5 caption, LL1-2: with respect to unfolded cosθ∗ for top quark and antiquark combined, compared to; LL2-3: The inner and outer error bars represent;
    • ok
  • Ref. [7]: can’t cite a preliminary result in the paper. Please remove the reference.
    • ok
  • Ref. [14]: JHEP {\bf 11} (2007) 070.
    • ok
  • Ref. [15]: JHEP {\bf 05} (2006) 026.
    • ok
  • Ref. [17]: JHEP {\bf 07} (2014) 079.
    • ok
  • Ref. [20]: delete “[Yad.Fiz.69,1352(2000)]”.
    • ok
  • Ref. [23]: JHEP {\bf 04} (2011) 024.
    • ok
  • Ref. [25]: delete “in Proceedings …p. 831” and replace “2012” with "(2012)”; delete “[,831(2012)]” and the doi reference to a DESY preprint.
    • ok
  • Ref. [27]: long published in Comput. Phys. Commun. {\bf 182} (2011) 2388; please add the journal and the doi references.
    • ok
  • Ref. [30]: JHEP {\bf 12} (2012) 035.
    • ok
  • Ref. [33]: cite also the BTV paper here.
    • ok
  • Ref. [35]: cite also the MUO paper here.
    • ok
  • Ref. [37]: JHEP {\bf 02} (2009) 007.
    • ok
  • Ref. [40]: delete “in Advanced …, p. 288” and replace “2002” with "(2002)”.
    • ok
  • Ref. [43]: delete “, no. 2,”.
    • ok
  • Ref. [45]: delete “in HERA … Part B” and replace “2005” with “(2005)”.
    • ok
type B
  • General: a 4.6\% p-value for the SM fit is somewhat worrisome. I think you should at least include the p-values for other extreme hypotheses in the paper, to [hopefully] demonstrate that the alternatives do not fit the data well either. Moreover, I do not understand why many of the uncertainties in Aμ for the top quark are so much smaller than those for the top antiquark, e.g., JES, W+jets shape reweighting, unfolding bias, generator model, top quark mass, and PDFs. Why on Earth the top quark mass affects the two differently? Please, explain here and in the paper.
    • Basically there are a lot of effective (aWtb, contact-interatctions, DM portals) or concrete BSM models (SUSY, technicolor,...) that come with different polarizations. We let this interpretation to the theorists. A minimal comparison with the case Amu=0 has been added.
    • The differences in systematics can be explained by the limited MC-statistics of those samples. We added "The limited number of simulated events can also influence the estimation of other systematics, and lead to potential larger uncertainties." to the description of the limited MC-statistics
  • LL14-16: as Ref. [7] has not been published, it can’t be cited in this paper. Hence, the paragraph should be deleted and instead you should say here (and in the abstract) that this is the first measurement of top quark polarization in single top quark production.
    • ok
  • LL34-35: the sentence is unintelligible - please explain clearly what reference frame you are using.
    • ok
  • LL62-63: non-leading primary vertex” is jargon - please explain in the paper how the hard-scattering primary vertex is defined and cite our tracking an vertexing paper.
    • ok
  • LL64-66: it’s odd to use “miss” for the missing transverse momentum vector and a slash to denote the missing transverse energy; suggest changing p⃗ missT with the p⃗ T/ (slashed pT vector). Also, on LL65-66 better to say “all reconstructed particle flow candidates in an event."
    • changed
  • L68: delete “full” [this is not really a ``full” sample, as it required certain detector quality]; “… recorded with the CMS detector [8] at the CERN LHC in 2012 …”
    • ok
  • LL70-85,102: give full versions of generators: Pythia 6.4.26 (?), Pythia 8.xx, Powheg 1.0, Tauola, MadGraph, FEWZ, etc. Also, list the PDFs used for each generator here. On L83, say whether Powheg was interfaced with Pythia 6 or Pythia 8. [I think it’s the former.]
    • ok
  • L95: … factorisation and renormalisation scales, etc …
    • ok
  • LL102-103: I believe the W+jets cross section includes the branching fraction per lepton channel, as the ratio is close to the canonical 10. Also, we usually denote the branching fraction as B, not BR. Finally, the commas in 37,509, 3,504 should really be spaces: 37 509, 3 504 [or write this as 37.509 nb, 3.504 nb].
    • ok
  • LL127-128: transverse energy has not been defined. Either define it here or use transverse momentum.
    • ok
  • L133: criteria [xx]; pT>10 (20) GeV in the muon (electron) case; and |η|<2.5. [Here you need to give a reference to “loose” isolation criteria or specify them.
    • rephrased
  • L167: define Smin, Smax here.
    • definition improved
  • L175: define the mass of a jet here.
    • ok
  • L262: In particular, bottom panels of Figs. 1-3 …
    • ok
  • Tables 1, 2: there is no such thing as “antitop”; there is a top antiquark. Thus change the headings of the columns to read: “t”, “t¯", and “t+t¯”. In the first column of Table 1 replace “Top” with “Top quark bkg.” and “EW” with "Electroweak bkg.” [as “EW" has not been defined!]; in the first column if Table 2, say “DY+jets” and “Total expected”.
    • ok, replaced EW with W/Z/diboson
  • LL296-298: “… from the decay chain of a top quark or antiquark and … off the top quark or antiquark via virtual W boson exchange …, and … of the generated top quark and antiquark.”
    • ok
  • LL310-311: “… anomalous Wtb couplings generated with {\sc CompHEP } [19,20] as pseudo-data. The test verified …”
    • ok, changed to present tense
  • L343: "… (set to a common scale equal to the momentum transfer Q in the event) …"
    • ok
  • LL370-371: "… measured differential cross section [xx] and apply …” [cite the appropriate ATLAS/CMS papers].
    • ok
  • L413: I do not understand how you convert the p-value of 4.6 x 10−2 into 1.7 standard deviations. For single-sided limits, as the ones you set here, the translation exceeds 2 standard deviations (95\% CL is basically two standard deviations). Please, check your translation here and on L422.
    • indeed: double-sided: 1.7 sd; single sided 1.999=~2.0 sd -> using singled sided now
  • Table 1: explain what “Unclustered \MET” uncertainty is.
    • ok

Tommaso Dorigo (on behalf of the Statistics Committee)

  • I have only one correction to make, relative to line 422, where you do not specify the units of your "difference of 1.7".
    • ok

Joanne Cole

AACHEN3B

  • Line 12: Change "Equally " to "However "; Both work, but if you were to use "However", I would suggest the following: "However, new physics models could also lead to "
    • Changed to "However, ..."
  • Line 34f: It would flow better if it was actually "A top quark candidate is then reconstructed and the angle "
    • ok
  • Line 81: I don't have a problem with using "anomalous structure", but I think the wording should be the following: "special samples are generated using COMPHEP including a Wtb coupling with anomalous structure."
    • ok

Tom Ferbel

  • Lines 2-3: I agree with Tom that the comment about QCD timescales need to be modified. My suggestion would be "is much shorter than the timescales typical of quantum chromodynamics (QCD)." Worded as it currently is, it reads slightly awkwardly.
    • ok

Having checked all the other Type A comments your responses seem reasonable (although it is possible I will contradict myself below in my own comments on version 11! If I do, please just decide which you prefer wink

Comments on Paper v11

Joanne Cole

  • L9: "t-channel, shown in Fig. 1, the standard model (SM)"
    • ok
  • Figure captions: I've noticed that when you have several distributions within a single figure and you address these individually in the caption you put the label after the text. So for Figure 1 it reads "Feynman diagrams for single top quark production in the t-channel: (2) -> (2) (a) and (2) -> (3) (b)." I have always put the label before the text, so I would have written this as: "Feynman diagrams for single top quark production in the t-channel: (a) (2) -> (2) and (b) (2) -> (3)." Depending on what the CMS guidelines say, I think this is more standard (but maybe I'm the non-standard one)
    • ok
  • L16: A suggestion, but not mandatory: "angular correlations of a decay product, particle X " replace with "angular correlation of one of its decay products, denoted X (where for this analysis X = mu) " I think this reads a little better?
    • ok
  • L17: "so-called" should be hyphenated, I think.
    • ok
  • L29: "violation, which is predicted in some new physics models."
    • ok
  • L34-35: " a differential cross section measurement with respect to this angle at parton level." One of the comments I believe was about on versus at parton level. It should be "at parton level", not "on parton level"
    • ok
  • L84-85: I think your bracketed phrase "(Z/gamma*+jets)" should move to line 85 after "association with jets" - this makes it consistent with the way you've described the W+jets on line 84.
    • ok
  • L105: A question: Can we assume that minimum-bias is something well understood by readers and therefore not define it?
    • let's discuss this in the FR if it is necessary to define it
  • L116: "preferentially produced in the forward region of the detector."
    • ok
  • L128: "where deposits linked to pileup are subtracted within a cone of radius Delta R "
    • ok
  • L158-159: "The effects of all these corrections are found to be small."
    • ok
  • L160: "To classify signal and control regions, different event categories, denoted "Njets, Mtag(s)", are defined,"
    • ok
  • L171: "the jet with the highest value of the b tagging discriminant"
    • ok
  • L195: In implementing some of the comments, the tendency to talk about cutting on the BDT has crept back in. You don't cut on the BDT, you cut on the BDT discriminant. This line is a case in point, as is L197 and L199.
    • ok
  • L218: "a chi^2-fit is performed of"
    • ok
  • L232: "constraint of being" -> "constrained to be"
    • ok
  • L245: "and a number of additional variables."
    • ok
  • L263: "performed in all signal and control regions using the event ratio "
    • ok
  • L264: There should be a comma after "flavour component"
    • ok
  • L278: Has ML been defined at this point - I didn't notice it
    • Yes, defined on L231
  • L295: I wonder if, for clarity, this should read " uncertainties for top quark events, top antiquark events, and"
    • ok
  • L299: "infer" -> "determine"
    • ok
  • L303: "defined by using" -> "based on"
    • ok
  • L311: " is applied. At its core it is the application of a matrix inversion "
    • ok
  • L318: "with the analysis strategy described here"
    • ok
  • L326: "matrix inversion is slightly worse when tested using pseudo-data."
    • ok
  • L373: "defines the uncertainty" -> "taken as the uncertainty"
    • ok
  • L389: " as well as using the central sets from the MSTW2008CPdeut "
    • ok
  • L398: I think it should be "The event reweighting", not "An event reweighting"
    • ok
  • L402: "amount" -> "number"
    • ok
  • L406-407: I would put commas around "within the limited statistical uncertainty of the simulation"
    • ok
  • L419: Move "obtained" after "differential cross sections"
    • ok
  • L420-421: I don't think you need to say "with their statistical and total uncertainties", but up to you.
    • removed
  • L422: "Uncertainties arising from the normalisation "
    • Set to "Uncertainties arising from the renormalisation"
  • L423: "and PDF variations have been found to be negligible "
    • ok
  • L427: replace "that corresponds" with ", which corresponds"
    • ok
  • L429: Saying "less compatible" sounds strange somehow, but I don't have a good alternative off the top of my head frown
    • How about "Furthermore, the compatibility of the combined result with the hypothetical case of $\Amu =0$ is smaller, yielding" ?
  • L433: "as in Eq. (1)" -> "defined in Eq. (1)"
    • ok
  • L437: "at a centre-of-mass energy"
    • ok
  • L439: cos theta is not an angle!
    • ok
  • L441: You need a comma after the value of Amu and before "which"
    • ok

Comments on Paper v13

Joanne Cole

  • L415-416: I would slightly rephrase the last part of this: "The limited number of simulated events can also influence the estimation of other systematics, potentially leading to an overestimation of the associated uncertainties." One could also consider quantifying the statement by saying something like "a slight overestimation", if you don't think the impact will be very large, for example.
    • ok
  • L423: I would recommend replacing "The considered" with "These"
    • ok
  • L444-446: I would recommend a slight rephrase of the last part: "This difference cannot be explained by any single source of systematic uncertainty considered in this analysis." My thinking behind this is that it is just possible someone, somewhere will think of some systematic that none of us have thought of that could explain it, so we should not completely rule that possibility out. I may be being over-cautious, but I don't think there is any harm in that
    • ok

Comments on Paper v14

Andrea Castro


Type A

  • Abstract, second line: I suggest to add "the" before "t-channel";
    • We think "t-channel" refers to something general here and does not need a "the"
  • Abstract, the "p" in "p-value" should be in italics
    • ok
  • L5: "V-A" with a dash, as at L10
    • ok
  • L8: rather than "a tool to examine the structure" I suggest "a probe of the structure"
    • ok
  • L71: "next-to-leading-order" (i.e. one more hyphen); add the powheg version number
    • ok
  • L75: add aMC@NLO version number
    • ok
  • L78: add comphep version number (if available)
    • ok
  • L86: add madgraph version number
    • ok
  • L91: add sherpa version number (if available)
    • ok
  • L105: the subscript "WW", "WZ" and "ZZ" with roman fonts
    • ok
  • L156: "tag-and-probe" with hyphens as at L361
    • ok
  • L174 (and other places): of course itâ?Ts a matter of taste but the upper-case for the "M" in the subscript "Multijet" looks (to me) slightly odd
    • ok
  • L182, 197, 200, figure captions, and many other places: the "W" in the subscript in roman fonts
    • ok
  • L193: the subscript T of "H_T" in roman fonts
    • ok
  • L205, 247: "W" in roman fonts
    • ok
  • L234: "A template" (add the article)
    • ok
  • L278: remove the article "The" at the beginning of the section header (as per guidelines)
    • ok
  • L314: I suggest to add a comma after "[44]"
    • ok
  • L325: "cross-check" with hyphen as at L434
    • CMSGuidelinesforAuthors actually lists the convention as "crosscheck". Changed both to that
  • L417: "of other systematic uncertainties"
    • ok
  • L429, 432: the "p" in "p-value" should be in italics
    • ok
  • L438: replace the semicolon with a comma
    • Used suggestion by Eric W. Vaandering (see below)
  • Ref. 33: use the same style as other PASes: "CMS Physics Analysis Summary CMS-PAS-JME-14-001, 2014"
    • ok
  • Ref. 42: you might add "submited to Eur. Phys. J. C"
    • ok
  • Ref. 45: should be "collaborations" with a lower-case initial
    • ok
Type B

  • Fig. 1: while line 9 ("single top quark production") might be intended as the production of single top quarks or single top antiquarks, in Fig. 1 you are drawing ONLY top quark production. It might be better to specify in the caption that "Similar diagrams are expected for top antiquark production", or something similar
    • ok
  • L30: do you have a reference for these new physics models? are they the same as refs. 3-6?
    • we do not have any reference in particular. the effective model given in 3-6 also predicts CP-violation for certain parameters
  • L80: what is "a closure test of the analysis strategy"? Thatâ?Ts jargon. Define this closure test and that is good to clarify also L377
    • removed "closure"; explaination in L317 is now "bias test". see recommendations from stat. committee on closure test
  • L84, 290: the TW channel is defined properly at L290. Better anticipate its definition here: "â?¦ used to model the W-associated (tW) and s-channel single top quark background events" and at L290 simply write "s-channel and TW single top quark production"
    • ok
  • L103: we should define the symbol \cal B, for example by writing "The W+jets and Z/gamma* production cross sections times branching fraction are calculated â?¦"
    • ok
  • L103: "ell" is undefined. Does it refer to e and mu only or are taus included?
    • Taus are included, amended the text accordingly
  • L127: usually we use the relative isolation I_{rel} defined as the sum of â?¦ divided by the pT of the lepton. Why do you use here a different (although equivalent) definition? We used Irel also in the paper quoted as ref [30]
    • changed
  • L149, 172: I do not know if the two terms are equivalent in English, but ref. 34 uses "discriminator" rather than "discriminant"
    • we'd like the LE to comment
  • L171: "missing transverse energy momenta" ==> "missing transverse momenta"
    • ok
  • L175, 191: no need to redefine MET
    • we just like to write the variables out again
  • L176, 184: in this case I believe itâ?Ts more appropriate to call it "the invariant mass of the top quark candidate", because itâ?Ts built from the daughter particles
    • ok
  • L185, 186: it is "the absolute pseudorapidity"
    • ok
  • L187: "the invariant mass of the b-tagged jet"
    • ok
  • All figs: usually the marker for data in the legend is a simple bullet with no error bar.
    • ok
  • L234: "template" is quite jargonish. Please define it. I suggest something similar to "A normalised distribution (template) for the sum â?¦."
    • ok
  • L262: although I understand what you mean, the symbol m_c is undefined and the symbol m_b was defined with a different meaning at L187. Rather than writing "m_b=m_c=0" can you just say that in words? Something like "â?¦ the approximation of using a null value for the b and c quark masses in the generation of SHERPA samples â?¦"
    • ok
  • Caption of table 1: here "scale factors" are mentioned but itâ?Ts not explained how/where they are used. You should explain it better at L297-298 where you quote tab.1. I assume tab. 2 values are indeed rescaled by these factors. Explain that.
    • We say both in the caption of Table 2 and on L298 that the numbers are scaled to the fit results. Can you suggest a better wording?
  • L293: the diboson background has been defined already as WW+WZ+ZZ at L92. No need to repeat here
    • ok
  • Caption of table 2: "MC statistics" is jargon. Maybe: "The uncertainty reflects the limited size of the MC samples and â?¦". "Data-driven" is jargon, better to replace it with "data-based"
    • ok
  • Table 2: itâ?Ts due to roundings but maybe you want to make the total value exactly equal to the sums (first and third columns differ by one unit)
    • This is not due to rounding, but because there is a different fit performed for each of the three cases. For some processes the difference is also larger than one unit.
  • L363: "p_T^t" is undefined (although clear). You might want to define it at L275: "â?¦ as a function of the transverse momentum of the top quark, p_T^t."
    • ok

Eric W. Vaandering

  • 10: This may read better if “produced” is moved before “top quarks”
    • ok
  • 83: as backgrounds in the …
    • ok
  • 96: top quark masses
    • We think it should be "top quark mass", as it refers to the parameter, not the value. Clarified the sentence.
  • 114: I think if you put the “originating from … W boson” in parentheses instead of commas it’s clearer and then there is no need for “as well as by” since you will not have to bring attention back to the main point.
    • ok
  • 134: candidates and clustered with …
    • ok
  • 144: no comma after 4.5. This is a requirement, not a parenthetical exposition
    • ok
  • 229, 235, 239: I think you should have a hyphen in multijet-events here since it’s used as an adjective
    • ok
  • 232: no comma after “data”
    • ok
  • 234: need an article before “Template for the sum” (a/the)
    • ok
  • 251, 252: This paragraph is in past tense for no apparent reason. Change was to is, provided to provides
    • ok
  • 287: either remove “a” before similar or put an “s” on distribution
    • ok
  • 297: Standard procedure is to spell out the word Table
    • ok
  • 305: no comma after antiquark
    • ok
  • 438: the phrase after the semicolon is not a sentence in its own right, which is the rule for using a semi-colon. May want to just start with “This measurement is…”
    • ok

Francisco Matorras


type A

  • L3 typical timescales
    • ok
  • L9 in electroweak t-channel single top...
    • ok
  • L367 and 371 move W+jets after reweighting
    • ok
  • L394 value of what?
    • ok

type B

  • title, you might add in pp collisions abstract. The sentence An unfolding... sound strange, do you need it? If yes, can you replace the sentence by something like, the differential cross section, corrected for detector effects, in terms of an observable sensitive...
    • let's discuss during FR about pp
    • changed to: A differential cross section measurement, corrected for detector effects, of an angular observable sensitive to the top quark polarisation is performed"
  • Formula (1) L16 and later in the paper, why introducing the X if you are only using muons? I would directly use Amu, alphamu. you can move L22 up to L14 and avoid using the X
    • We'd like to make the point that one can also use b, W or the reconstructed neutrino since the cited Refs. are suggesting it as well
  • L93 it does not sound good that multijets are events in which the muon...
    • do you know of a good definition that we could use here?
  • L152 data-simulation efficiency scale factors, correcting the simulation with scale factors obtained from independent samples?
    • this is specified later but also in the cited Ref.
  • L169-170 Check that sentence, please
    • split into two sentences
  • L190 W boson candidate
    • ok
  • L216-218 you mention a trend without quantitative basis and apparently you never use it again
    • this is just to ease the reader towards the fact that also after unfolding he should expect a trend.
  • L233 and 239 20% and 50% do not seem justified
    • .The 20% prior uncertainty covers any possible MC simulation yield uncertainty of all the MC samples taken together. We performed many variations of the multijet fit to test its stability and the 50% uncertainty covers the variations conservatively.
  • L250 a disagreement similar to what? Is this related to any systematics?
    • it is a deviation with unknown cause in the cosTheta shape as predicted by MadGraph
  • L292 and 294 again this 20% and 50%
    • .As the measured ttbar cross-section yield uncertainty is of the order of 10%, the 20% prior uncertainty covers this while also including possible contributions from the single top backgrounds, which are minor compared to ttbar. For the 50% see answer to L376.
  • L298 what is this "residual fitting uncertainty"?
    • to align to table 1 caption changed to "estimated scale factor uncertainties"
  • L312-313 Such a short explanation for the unfolding seems to confuse rather than clarifying. I guess the reference is enough, otherwise have a better explanation
    • the ref should be enough
  • L319-320 I would replace that sentence with "The small bias observed is conservatively taken as a systematic error".
    • we fear that this draws attention away from the closure test interpretation: different asymmetries are correctly measurable
  • L345 again 50% apparently out of nowhere
    • it reads: "... obtained from the multijet events fit"
  • L363 assuming independence with other systematic error sources, the spectrum...
    • the listed systematic sources are all varied independently
  • L376 why +-50%
    • in other analyses as mentioned in sec 6.2. the heavy flavor component seemed to be underestimated and had to be scaled up by 50%. Since we do not estimate this for our analysis we just take a conservative estimate
  • L392-404 I'm a bit uncomfortable with all this paragraph. You use a complex and unusual method to avoid the MC stats of the samples, but use these samples to validate your method. So your precision is that of the limited size samples, you do not gain anything. And you need such a detail for a single systematic?
    • we gain a smoother spectrum that is statistically compatible
  • L405-409 This paragraph is not clear, at least I do not understand what you mean
    • the shape does not change with increased BDT cut so we loosened it to get a template
  • L410-412 why you rely on the samples here and not for the renormalization scales? aren't the samples of similar size?
    • we do not know of a method to perform a reweighting to mimic the matching scale uncertainties at hand so we had to rely on the samples here (although this is not preferred of course)
  • table 3. I am a bit surprised from the difference in systematics between t and tbar. Is that justified or it is just a result of errors on the errors? should you just avergae both and put the same number? For example it is difficult to understand why the unfolding bias should be much larger for tbar
    • this is due to the limited MC statistics as written in its paragraph
  • figure 5, i see here a much more obvious trend, probably related to the otehr one, but you do not mention anything. It is surely related to your lower asymmetry
    • can you clarify?
  • L431 furthermore->alternatively, comparing with the hypothesis of no asymmetry, we obtain.
    • changed furthermore->alternatively; but we like to keep A=0 instead of 'no asymmetry'

Sijin Qian

In general

(1) This is a part of a previous comment for v10, i.e.

(4) Throughout the paper (including in Eqs., inside Tables, in Table and
Figure captions, in superscripts and subscripts, etc.), to be consistent
with elsewhere in this paper,

  (a) ... (b) ...
  (c) e.g. L155, L232, L303 and L364, etc., for the "Section xx", it
should not be abbreviated, e.g. L93 should be extended from

"as detailed in Sec. 6.1." -->
"as detailed in Section 6.1."

other places where also need to be changed by the same way are at

L183, ...
** Your response:
**
** Only written out if at beginning of sentence

=== My further discussion:
===
=== To be consistent with all other CMS papers, there should be no
=== abbreviation on "Section xxx" throughout the paper everywhere.
===
=== In v14, the places where the change should be made are:
===
=== L95,
=== L165,
=== L195,
=== Fig.2's caption (the 2nd line),
=== Fig.3's caption (the 2nd line),
=== L209,
=== L242,
=== Fig.4's caption (the 3rd line),
=== L271,
=== L286,
=== L311 and
=== L372.


  • ok, changed also Tab. -> Table & Eq. -> Equation; PLEASE change the PubGuidelines accordingly since it proposes the convention as we had it before ( link)
     (2) Throughout the paper (including in the Abstract), to be distinguishable from the proton "p" elsewhere in this paper and to be consistent with all other CMS papers, the "p" in the "p-value" should change the font from "p(non-italic)-value" --> "p(italic)-value" In v14, the places where the change should be made are: In Abstract, the bottom line; L429 (two places, i.e. "p-value" and "p(datajSM)") and L432 (two places, similar as L429). * ok Page 0, in Abstract (3) This is a previous comment for v10, i.e. 
    (6) The 7-8th lines (now the 8-9th lines in v14), I'm not sure whether the subscript "mu" in the variable "Amu" should be explained, i.e. "is used to extract a top-quark spin asymmetry of Amu = ..." --> "is used to extract a top-quark (decaying to a muon) spin asymmetry of Amu = ..." or "is used to extract a top-quark (with an isolated muon in the final state) spin asymmetry of Amu = ..." 
    ** Your response: ** ** ok === My further discussion: === === In v14, it has not been changed yet.

  • We mention that we select events where the top quark decays to a muon on line 5 in the abstract. Let's discuss the best wording during FR
Page 3

(4) L99, there seems an extra letter before NNLO, i.e.

"(aNNLO) predictions [25]" -->
"(NNLO) predictions [25]"

  • aNNLO = approximate NNLO (see the ref); this is not the full NNLO calculation

(5) L103, the branching ratio "B" should be explained at its 1st appearance in text here, i.e. "(sigma(W+jets) * B(W -> lnu) = 37 509 pb and ..." --> "(sigma(W+jets) * B(W -> lnu) = 37 509 pb, where B is branching ratio, and ..."
    • changed differently: "The \wjets\ and \zjets\ production cross sections times branching fraction are ..."
Page 4

(6) This is a part of a previous comment for v10, i.e.

  • (16) L129 (now L130 in v14), the "phi" should be explained, and as the numerical values of the angle phi have been implicitly shown here and an angle 
    can be measured in either radians or degrees; therefore, the unit of phi may should be specified; "DeltaR = sqrt(...) = 0.4 around ..." --> "DeltaR = sqrt(...) = 0.4 (where phi is the azimuthal angle in radians) around ..."

    ** Your response: ** ** ok === My further discussion: === === In v14, thank you for explaining the phi; but the unit of "radians" === has not been added yet, please consider. 
    • ok; however natural units of angles are radians since e.g. sin(x), sin(ix) as a series of x^N, N=0 .. infinity can only evaluate for unitless x
  • (7) This is a previous comment for v10, i.e. 
    (18) L134 (now still L134 in v14), the expression of "kt(non-italic)" here is different from the one in the article title of [31], where it is "kt(italic)" I'm not sure whether they should be consistent or not. 
    ** Your response: ** ** ok === My further discussion: === === In v14, it seems have not been done yet. 
    • Now set both to italic
  • Page 5 (8) L171, the "missing transverse energy momenta" seems strange, would it be able to be shortened by using "EmissT"? 
    • Corrected to "missing transverse momenta." We feel it reads better here without shortening as the we are talking about two kinds of momenta.
  • Pages 6-9, Figs.2-4 (9) This is a part of a previous comment for v10, i.e. 
    • (25) Figs.1-3 (now Figs.2-4 in v14) (a)... (b) ... (c) In the caption of each Figure, (i) ... (ii) Each Figure has two plots (thus two bottom panels), therefore, the subject word "panel" in the last
      sentence on the 3rd line should be plural, and corresponding verb and the object word should be plural too; also the word of "figure" on the 3rd line should
      be changed to "plot", e.g. the 1st 3 lines of Fig.1's caption (together with the item (4c) above for the full spelling of "Section"),
      "Figure 1: Distribution of the BDTQCD discriminant in the â?o2jets 1tagâ? (top) andâ?o3jets 2tagâ?(bottom) categories.
      Predictions are normalised to the results of the fit described in Sec. 7. The bottom panel in both figures shows the ratio between ..." -->
      "Figure 1: Distribution of the BDTQCD discriminant in the â?o2jets 1tagâ? (upper plot) and â?o3jets 2tagâ?(lower plot) categories.
      Predictions are normalised to the results of the fit described in Section 7. The bottom panels in both plots show the ratios between ..."
      or, the last sentence of the above example would not be changed to the plural case by replacing a word of "both" to "each", i.e.
      "The bottom panel in both figures shows the ratio between ..." --> "The bottom panel in each plot shows the ratio between ..."
      ** Your response: ** ** ok === My further discussion: === === In v14, thank you for changing the plot indicators to (a, b). 
  • But for === === "figure --> plot" and === "Sec.7 --> Section 7" === === they have not been done yet, please consider. 
    • ok
  • Page 11 (10) L297, to be consistent with all other CMS papers, the "Table XXX" should not be abbreviated to "Tab.", 
  • especially because that the "Tab" is more popular to have other meanings, i.e. "are presented in Tab. 1, while Tab. 2 shows the number of events" --> "are presented in Table 1, while Table 2 shows the number of events" 
    • ok
  • Page 13 (11) L351, since "JES" has not been used afterward in whole paper, it eventually can be removed, i.e. "scale (JES) [38]." --> "scale [38]." 
    • ok
  • (12) L354, as "JER" has been already explained earlier on L159, here it can be shortened from "in jet energy resolution (JER) relative" --> "in JER relative" 
    • ok
  • Pages 20-21, in the References Section (13) This is a previous comment for v10, i.e. 
    (40) L513, in [14], to be consistent with other JHEP Refs. in this Section, 
    the document index should be shortened from "JHEP 0711 (2007) 070," --> "JHEP 11 (2007) 070," Other ones which also need to be changed by the similar way are Refs.[15], [17], [23], [30] and [37].
    ** Your response: ** ** ok === My further discussion: === === In v14, thank you for having made changes for most of cases, but still === Ref.[20] needs to be shortened. 
    • ok
  • (14) L571-572, in [27], to be consistent in this Section, all references should have only one page index instead of two, i.e. 
  • "Comput. Phys. Commun. 182 (2011) 2388â?"2403," --> "Comput. Phys. Commun. 182 (2011) 2388," 
    • ok
  • (15) L585-586, in [33], to be consistent with other PAS Refs. (e.g. [8]-[10], etc.) in this Section, 
  • it should be changed from "Technical Report CMS-PAS-JME-14-001, CERN, Geneva, 2014." --> "CMS Physics Analysis Summary CMS-PAS-JME-14-001, 2014." 
    • ok
  •  (16) L591, in [36], to be consistent in this Section and this paper, the font of "pp" in the article title should be changed from "in pp(italic) collision events at" --> "in pp(non-italic) collision events at" 
    • ok
  • (17) L594, in [37], the "efficiencies" in the article title should start with an upper case letter, i.e. "Single Muon efficiencies in 2012 Data" --> "Single Muon Efficiencies in 2012 Data"
  •  Another one which also needs to be changed by the opposite way is [43], i.e. to change the "Unfording" to the lower case from "An Unfolding method for ..." --> "An unfolding method for ...". 
    • ok
  • (18) This is a previous comment for v10, i.e. 
    (47) L598, in [46] (now L614, in [45] in v14), to be consistent in this Section, the 1st letter of "The" in the author part should be removed, i.e. "[46] The ATLAS, CDF, CMS and D0 Collaborations," --> "[46] ATLAS, CDF, CMS and D0 Collaborations," 
    ** Your response: ** ** ok === My further discussion: === === In v14, it has not been done yet.

  • ok

Final Reading

comments by Andrea on v15

  • l300: "The results of the three fits, and the post-fit uncertainties for top quark events, top antiquark events, and their combination, are presented in Table 1, while Table 2 shows the number of events exceeding the threshold on the BDTW/t¯t > 0.45 discriminant, normalised to the fit results."
    -> "The results of the three fits, and the post-fit uncertainties for top quark events, top antiquark events, and their combination, are presented in Table 1 as scale factors to be applied to simulation yields, while Table 2 shows the number of events exceeding the threshold on the BDTW/t¯t discriminant > 0.45."
    • ok

comments by Eric

  • l103 there is no need for a thin space in 3504 pb. The convention is 4 digit #’s are written out without spaces, 5 digit or more have spaces as separators.
    • ok

comments by Francisco on v15

  • l153 "...using data-simulation efficiency scale factors that..." -> "...using scale factors that..."
    • ok
  • l236: explain the origin of 20% uncertainty -> quote cross section paper here
    alternatively: uncertainty has been conservately increased but is then estimated by the ML-fit
    • changed to: "...constrained to be within +-20% of the expected yield using a log-normal prior that the fit constrains further". We think that this emphasizes the fact that the actual prior width is unimportant to the QCD estimate
  • l324: use similar wording here as in the systematic section: e.g. "...with a small bias that will be accounted for as a systematic uncertainty."
    • ok

comments by Sijn on v15

  • inside a sentence, "Table" and "Section" are not abbreviated while "Figure"->"Fig." and "Equation"->"Eq." are
    • ok
  • l98: use (NNLO) here because aNNLO is never used later
    • ok

reading of Title

(leave as it is)

reading of Abstract

  • remove "A_mu" just quote the number
    • ok
  • "...which is compatible with the standard model prediction of 0.44 with a p-value of 4.6%." -> "...which is compatible with a p-value of 4.6% with the standard model
    prediction of 0.44."
    • ok

reading of Introduction

  • l2: remove "tau", just "(~4x10^-25s)"
    • ok
  • l10: "...top quarks are highly polarised through the V–A coupling structure along..." -> "...top quarks are highly polarised, as a consequence of the the V–A coupling structure, along ..."
    • ok
  • l11: "...that recoils against the single top quark ..." -> "...which recoils against the top quark ..."
    • ok
  • Figure 1 caption: "...production the t-channel" -> "production in the t-channel"
    • ok
  • l18: adding 2 commas to: "The variables N(") and N(#) are defined for each top quark decay product from the decay chain t -> bW -> bmn as the number..." -> "The variables N(") and N(#) are defined, for each top quark decay product from the decay chain t -> bW -> bmn, as the number..."
    • ok

reading of Summary

  • combine paragraph 2 & 3
    • ok
  • l450: add p-value: e.g. "p-value ..., equivalent to 2.0 SD, ..."
    • ok
  • l446: "The asymmetry is ..." -> "The asymmetry, A_mu, is ..."
    • ok
  • l450: SM -> standard model
    • ok
  • l451 "The observed asymmetry" -> "The asymmetry observed"
    • ok

review of Figures and Tables

  • Figure 1: roman font for particle names
  • ok

  • optional suggestion for all reconstruction figures: "remove 'mu +'"

  • ok

  • need to check if we need dots after Stat. & Syst. if written in the legends

  • dots added

  • Figure 2: -> "BDT_multijet"

  • ok

  • optional: reduce thickness of bar line in ttbar somehow

  • ok

  • Table 2: align ±; remove 'events'

  • ok

  • Table 1 & 2: right-justified names like table 3

  • ok

  • Table 3: remove double lines

  • ok

  • Figure 5,6: check offline how to normalize and what to write for "/ 1 Units"

  • removed "/ 1 Units"; normalization is correct

Comments before 'ready for submission' on v16

Joanne Cole (LE)

  • I have read through the part of the introduction that we did not read during the Final Reading, as requested by Andrea (lines 22 - 38 in v16). I think it looks fine. The only correction I would suggest is in line 34, which should read: "and the angle between the muon and the recoiling jet calculated in the top quark rest" ie. the "is" between "jet" and "calculated" should be removed.
    • ok

Tom Ferbel

  • I would recommend that just "Events per bin" or "Events/bin" appear on the ordinate axes in Figs. 2-4
    • ok, using "Events / bin" now

Andrea Castro

L104: “e” (for electron) in roman font

  • ok
L128: to avoid misunderstandings I suggest to add two commas: “… sum, divided by the pT of the muon, of the …”
  • ok
L192: the “W” should be in roman font
  • ok
L231, 238, 242: remove the hyphen in “multijet-events” for consistency with L230
  • ok
L250: the “W” (in W boson) should be in roman font
  • ok, also removed "(pTW)" since it is never used again
L254: “… is observed in data collected at sqrt(s)=7 TeV ” reads better
  • ok
L307: “j’” and “q’” have been defined already at L116. No need to do it again
  • ok
L592: remove “CERN, Geneva,”
  • ok
Then there is a major thing which I overlooked, that is eq. 3. First of all, it is not appropriate to use an acronym (ME, PDF) in stead of a variable, so you could introduce functions “f_{\rm PDF}” which represent the PDFs and a quantity “M” or “\cal M” to represent the ME. Then, I am puzzled about eqs. 3 and 4 (used together). Is this exactly what you do? because since any cross section goes with the square of the matrix element should not the weight scale as (pdf_1*pdf_2)^2 if one uses eq. 3? Since this is not the case maybe the way eq. 3 is presented is not correct. … or am I missing something?
  • Eq. 4 is correct. This is what we do. We however reformulated the reweighting decription. Eq. 3 is replaced by xsec = int dx1 f(x1,Q) int dx2 f(x2,Q) xsec_hat(x1,x2) where f(xi,Q) is the PDF and xsec_hat is the partonic cross section. Eq. 3 is correct (no approximation; independent of the PS) at LO for the t-channel in 5FS as written now in the text.
  • theoretical remark: The approximation of neglecting the PS may not be obvious since Eq. 3 is correct. In general, PS simulation cannot change the overall cross section. It is only interfaced after the simulation of the hard scattering for soft emissions & hardonization which cannot be calculated analytically. Hence, the dependence on variables like number of jets >1 GeV on the Q scale is neglected by the reweighting.

Comments on v18

Andrea Castro

1) add a comma at the end of eq. 3

  • ok

2) remove the colon at L403

  • ok

3) as a matter of facts, "events reweighting" is a procedure and cannot be set equal to the quantity w. I suggest a rewording of the sentence at L402-403:

"The event reweighting to a different scale Q' is then defined using a factor"

(followed by eq. 4.) with no colon after "factor". Doing so we also introduce the meaning of Q'

  • ok

Comments on the first submitted version (v19) from the reviewer, 17.12

In general the draft assumes at many places that the reader is very familiar with the subject such that useful and sometimes important details of the analysis are discarded from the description. In this context the paper follows a, to my mind, dangerous tendency to present results in the form of the output of a multi-variate analysis. Like this it is very difficult to judge on the experimental issues of the paper. For example not less than 10 experimental variables are used as input to the BDT in Figure 3. For the sake of briefness and being aware that the multi-variate techniques are needed for extracting single-top signals I don't ask you to include all these control distributions into the paper. I propose rather that you cite a technical note or a PhD thesis or any other document at your leisure in which the input distributions and details of the analysis can be looked up. On the bottom of Page 5 you say e.g. that All BDT input variables are found to be well modelled by the MC simulations. This means that these checks have been done (which I have never doubted) and thus the material can be made available. I have quickly scanned through [30] where the BDT analysis was introduced in a similar way but this paper has the merit that at least a few experimental distributions are given.

An alternative would be to present the one or two distributions that have the largest pull on the results (or otherwise said the highest ranking in the decision of the BDT).

  • Add |eta_j'| and top quark mass in 2j1t before BDT cut

The reader is often "left" alone with the distributions making it thus difficult to judge on the pertinence of the shown distributions. Again on Fig. 2 and 3, can one understand the shape of the distributions?

  • L196: Add sentence ""In general, a BDT discriminant is the weighted average over many decision trees where each tree contributes +1 (-1) for signal-like (background-like) events."

Page 5, Fig.4:

  • Maybe the most important distribution apart from the final results in the paper is the Fig. 4, the more since it gives information about an experimental (modulo boosting) observable. You give basically no description of the figure. Why is there this drop towards cos theta*_mu = 1? I had to go through reference [30] to finally end up at arXiv:0905.4754 that confirms that the drop is due to kinematic cuts (also [30] is silent about that but gives at least the reference).
    • At L219 add a sentence: "The obtained distribution differs from the theoretical prediction due to detector effects and kinematic selection, most notably by a lack of selected events near cos theta*_mu = 1
  • Is the Fig. 4 only based on a cut on BDTWttbar > 0.45 or not rather also on a cut on BDTmultijet > -0.15? I suppose to arrive at the result of Fig. 4 you have applied both cuts and this should be added to the figure.
    • Caption of Figure 4, L2 add: "Rejection of multijet events by selecting BDTmultijet > -0.15 is included in both plots.

Page 10, Table 2 and related text: For the sake of completeness and guiding of the reader I would like to ask you to comment on the fact that you have more t than tbar events.

  • L300: Add "The number of top quark events is greater than the number of top antiquark events due to the larger up-quark than down-quark or anti-up-quark density at large values of Bjorken $x$ in the incoming protons."

Page 10: I would like to ask you to include the result including a data MC comparison after background subtraction and before unfolding as an important intermediate step.

mu_2j1t_cos_theta_bgsub.png

Sec. 10: The results shown in Figs. 5 and 6 comprise entries in the bin 0.666 < cos theta*_mu < 1. However as we have seen in Fig. 4 the distribution has a drop towards cos theta*_mu = 1. In this bin you must have thus a pretty large correction from the unfolding. Have you tried to restrict your analysis to cos theta*_mu < 0.666 i.e. to a region in which you have full acceptance. Since you fit the cos theta*_mu distribution the lever arm should be still long enough.

  • Say: "We tried it and the result was compatible with the one presented in the paper, but less sensitive."

Sec. 10+11: I agree that it is too early to start with an interpretation of the observed difference between the data and the SM prediction but it may already make sense to give the SM prediction separately for t, tbar and t + tbar.

  • It is expected from theory that the asymmetries are the same. Numerically we get from MC (PowHeg): A(t)=0.438030003 and A(tbar)=0.438030033 which is within the uncertainty from the limited number of MC events
  • Add on line 38: "SM predictions for top quark and antiquark events are equal, which is also reflected in the simulated events we use."
Edit | Attach | Watch | Print version | History: r164 < r163 < r162 < r161 < r160 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r164 - 2016-02-02 - MatthiasKomm
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback