ATLAS+CMS aTGC combination review

ATLAS comments and replies

Reviewer 1



DONE *In the tables captions, refer to ATLAS for T-1,2,3 and 4 to CMS for T-5 and 6

*Sections 2 The detail levels of the descriptions of Event Selection, Backgrounds and Uncertainties for ATLAS and CMS differ too much I think

==>> You are right, we will provide a shorter version of the ATLAS description and refer the reader to the original publication for all the details.

DONE *L171 "... namely Baysian integration ..."

I don't understand to what this refers. By elimination, I guess be this is about the use of a Chi2 law mentioned around L244, but I don't think that this is Bayesian but rather an approximation, or say an interval extraction procedure definition. I suggest to check with those who should know if this can be qualified as Bayesian.

==>> You are right, Bayesian is irrelevant here. This must have been mistakenly written. We deleted any reference to the techniques here and we leave the explanation of the statistical techniques for the following section.

DONE *Section 3.2 It said that 2D parametrisation are done L192-193 and L203 for (f_i^\gamma, f_i^\Z) while the equation (2) is full 4D parametrisations. I guess that what is meant but not said is that the 2f_4 and 2f_5 are having separate life so to say because I guess they are resp. CP-violating and CP-even. I think that this is confusing and I would suggest to drop the f4 X f5 crossing terms in (2) and tell why.

==>> Limits were set for one dimensional and two dimensional parameter space where all other parameters are fixed to their SM values. The two dimensional limits were derived in $(f_4^\gamma,f_4^Z)$ and $(f_5^\gamma,f_5^Z)$ parameter spaces. We have tried to make it more clear in the text by adding the explanation. The equation expresses the reweighting that is followed by ATLAS.

DONE *L217-219 "In the ATLAS ... the summation was used"

I would suggest to put this in footnote It is really a side point with respect to what you are explaining and I am afraid that it can only distract the reader. Still I think it would be nice that you complete this sentence by telling what the impact of this change on ATLAS limits is.

==>> This does not have any big effects (less than 5%), but we keep the original results in the old publications for which we give references.

DONE *L220-223 "Both types of nuisance parameter modeling were used to compare combination results between AT-LAS and CMS. The final decision was to use log-normal constraints for the final combination"

The more direct reading is I think, that "both" refers to equ 4 for ATLAS and equ 5 for CMS as said L217, but since both equation are said to be log normal constraints on L216 then "The final decision was to use log-normal constraints" is highly confusing. I guess that the sentences make sense if you are talking about on one hand Gaussian constraints and on the other log-normal constraints.

I suggest to make this clear by "Both types of nuisance parameter modelling were used to compare .." ->"Both Gaussian and log normal constraints were used to compare .."

Yet the paragraph don't tell what the result of the comparison was, but only that the log-normal is better in principle (bounded effects). Can you tell if the differences were great? I suspect that they were not since large errors look to be on sub-leading components of the counts.

==>> The paragraph was rephrased, thanks for the suggestion, indeed it was misleading. The difference between the logNormal and the gaussian was on average about 1%.

DONE *L227 "The most likely estimators (MLE)"

I suspect a word game around MLE = Maximum-Likelihood Estimators. Even if it is not a word game , I think that this is not too fortunate. I suggest something like "The point estimates " and in any case to avoid "likely" is this context.

==>> changed to MLE = Maximum-Likelihood Estimators

DONE *L228-231 ". A parametric bootstrap scheme was used where theta_hat_hat are the ..."

I suspect "parametric bootstrap scheme" is not helping the understanding of the reader. I suggest to drop ". A parametric bootstrap scheme was used" end point of the preceding sentence included

==>> We have added a link to support the usage of this terminology.

DONE *Equ (7) it is I think intended for Data and pExp. Still the theta_hat_hat has been defined L231-L232 for the data and this can not be for Eq(7) be valid for pExp. To redefine it L235 does not help. Suggestion follows.

==>> fixed by reshuffling the text as suggested below.

DONE *L237 "Pseudo-experiments with \lambda(\alpha) that is smaller than found on data are considered less likely"

"likely" is unfortunate I think since likely-ness would invoke some distribution. Even so, when one integrates the distribution of the -2ln(lambda)s above -2ln(\lambda_data), those are not less likely just less compatible as one can picture it in the case of a -2ln(\lambda_data)~0 where most of the distribution is above -2ln(\lambda_data).

I think that minimally \lambda(\alpha) is just quantifying compatibility of Data with a given hypothesis.

I suggest something like "Small \lambda(\alpha) signs lower compatibility between Data with the actual hypothesis \alpha."

==>> in a second reading it is misleading, you are right. We followed your suggestion.

DONE *L239 I guess that \theta_0 is generated around \theta_hat_hat. If yes I suggest to add the point. If not, is it correct?

==>> This is correct, \theta_0 is generated around \theta_hat_hat. Added in the text.

DONE *Suggestion for L230-242 I would suggest a re-writing from L230-242 into something like

First for Data one computes the test statistic equ(7) with [Here your sentence L235-236 ending at "together"]

Then [Here procedure of the generation of pExp, your lines 230-232]

The Likelihood for pExp [Here your L238]

[Here your equ (8) with a subscript _{pExp} to L ]

[Here L239-241 ending at "uncertainties"]

As for Data one computes for pExp

[Here equ(7) with subscript _{pExp} to L, \lambda, \theta_hat_hat, \alpha_hat and \theta_hat]

with \theta_hat_hat_{pExp}, \alpha_hat_{pExp} and \theta_hat_{pExp} defined as \theta_hat_hat, \alpha_hat and \theta_hat but for L_{pExp}

Ten thousand [Here your L241-242] as:

p-value (\alpha_test) = \frac{Nber of pEXp with \lambda_pExp(\alpha_test) < \lambda(\alpha_test)} {Total nber of pExp}

This I think could be easier for the reader even if arguably somewhat heavier (due to the _pExp subscripts around)

==>> We followed your suggestion, thank you

DONE *L250 I guess that presenting 2 methods calls for comparison. Would you consider to add some comments here or latter when the results are presented? As Masahiro shown, there are some issues for the 2D case between FC and DlnLog methods. I would like to know your position about this since the T-9 and T-10, which are about 1D results, could be seen as a bit misleading about the agreement between the methods.

==>> This is what the table with the final results shows, the comparison between dLog and FC. Since we don't actually discuss that where we show the final results, we could add a comment as you suggest. We could say something like the following: "We expect nice agreement between dNLL and FC limits in 1D as can be seen from table 9 and for 2D limits we expect larger disagreement because dNLL method results with over coverage for the case where 2 parameters are indistinguishable Since it is not crucial in the physics result, we show only the dNLL 2D limits because they are significantly less time consuming compared to FC."

TYPOS I guess =========

DONE *L69-70 "Calorimeter tagged muons ... with pT > 20 GeV" a verb is missing

DONE *L148 This rate was measure as ... ->This rate was measured as ...

DONE *L200 "...set of ([Here 3 couplings]) ..." I think that f^\Z_5 is missing

DONE *L214-215 "The nuisance parameter constrained with a Gaussian distribution." a verb is missing

DONE *L216 Equations 4 abd 5 ->Equations 4 and 5

DONE *equ (6) and (8) I suspect that the normalisation should be sqrt{2 \pi}^J.No?

==>> In the end we are using the ln(likelihood ratio), so the normalization cancels out.

DONE *equ (4) I think that "J" is missing over the capital PI

DONE *L289 "/ZZ/" ->ZZ

Reviewer 2


As this is an ATLAS/CMS combination, I was wondering whether this is already the envisaged final text (i.e. CMS is reviewing exactly the same document) or is this the ATLAS part to a final document which will be written later? Thanks for the clarification.

==>> Yes, this is the common, final version between ATLAS and CMS. Both sides will implement their comments, we are in contact of course for some of them. We plan is to merge the two versions into a refurbished one.


DONE 1) L60: introduce definition of dR already here (it is given later in line 64).

DONE 2) What triggers have been used for this analysis? I couldn't find it in the original publication either. I am just curious by how much the online and offline pT thresholds differ.

==>> The events used in this analysis were selected with single-lepton triggers with nominal transverse momentum (pT) thresholds of 20 or 22 GeV (depending on the instantaneous luminosity of the LHC) for electrons and 18 GeV for muons.

DONE 3) L63f: If I understand the original paper correctly, calorimeter and track isolation is applied to both electron and muon candidates. Maybe start the sentence with "Muon and electron candidates..." to avoid some misinterpretation.

==>> Info given in the parentheses form the corresponding sentence for electrons.

DONE 4) L69: The sentence starting with "Calorimeter tagged muons..." seems to be incomplete.

DONE 5) L76: Maybe add half a sentence about the charge of the forward electrons which is important for the determination of the Z candidates.

DONE 6) The event selection described in 2.1.1 only refers to the "on-shell" ZZ selection (if I got it correctly). In 2.1 you mention also the offline ZZ selection (I wouldn't call it categories because these are not disjoint selection regions). Could you add a brief statement why the offline selection is not considered for the rest of the paper (I assume that only onshell bosons contribute to aTGCs, right?).

==>> you are right, the sentence is rephrased.

DONE 7) L106: The extrapolation is done in bins of pT of which object?

==>> of the leptons, added.

DONE 8) Tables 1 and 2: Could you please add (at least the statistical) uncertainties? pT(Z) is the pT of which Z boson (the leading one, the "more onshell" one)? Why does pT(Z) only starts at 50GeV in Table 2? Is this an artifact of the lepton pT > 20GeV + dR(l,l) > 0.3 selection?

==>> About the uncertainties, this has been asked a couple of times and by all other EB members, so it will be addressed separately (maybe after discussing also with CMS). The pT refers to leading (highest pT for 4l and decaying to a charged lepton pair for 2l2n) Z boson. We have added this explanation in the text here as a footnote as it was missing from the text and we have seen that we used the term "leading" later on in the text. 50GeV was a natural selection for the pT bin to start, as it is implied from the lepton and event selection. Low pT bins are also not really relevant to us since TGC can manifest themselves in high pT and going lower in the pT spectrum would not provide a lot of useful information.

DONE 9) Section 2.1.4: Concerning the treatment of the correlation of the uncertainties I have a few questions: a) You assume that the ZZ Theory uncertainties are correlated across the pT(Z) bins. Has this been checked? Naively, I would assume that the theory uncertainties not only impact the overall normalisation of the pT(Z) distribution, but also its shape. Therefore, I would actually expect some anti-correlations among the bins.

==>> The impact of the uncertainties to the final result has been studied by assuming the uncertainties fully correlated or uncorrelated (or by varying them by 50% up and down). In general it has been proven that they don't have an impact to the limit setting. At the same time, the assumption for the anti-correlation is not very well understood.

b) I do not see why the systematic uncertainties on the data driven estimates in Z(4l) and Z(2l2v) categories should be correlated. In the first case, the background is probably dominated by fakes while in the later case you have quite some prompt lepton contributions from ttbar, WW etc which happens to pass the Z mass cut. Without having studied the background estimation techniques in detail, I would only expect a rather small correlation.

==>> We wanted to stick to the original publication, this is what was performed at that time. Indeed the uncertainties of some of the background estimation could be considered uncrorrelated. However, as explained above, after checking the systematic uncertainties on the background estimate were also considered correlated, as it was considered to be the best approach.

c) I think the "MC Background Systematics" and the "Systematics" uncertainties should also be treated correlated across bins and Z(4l) and Z(2l2v) categories because there you are varying the same sources of uncertainties (lepton efficiencies, resolutions, etc). As this document is supposed to serve as template for future ATLAS/CMS aTGC combinations, I would like to add that in my opinion a more detailed breakdown of the "Systematics" and "MC Background Systematics" is needed. When looking at Figure 11 in arxiv:1211.6096, it seems that the highest sensitivity comes from the last bin in the Z(2l2v) category, which is also the bin where the dominant background contribution is estimated with MC.

==>> MC background systematic uncertainties are only relevant for 2l2nu. We cannot add a more detailed breakdown, because we cannot release information that was not included in the original publication.

DONE 10) L136: Do you know why CMS is using this combination pT thresholds? Do I understand it correctly that in the end all selected leptons had a pT > 10GeV? In general, the description of the CMS analysis is quite short compared to the description of the ATLAS analyses (ok, if this is only the ATLAS part of the combination).

==>> The pT threshold of 10GeV is correct only for two out of four leptons. One lepton has pT>20GeV and the second 10GeV. The two remaining stay with their original selection criteria which were 7GeV and 5GeV for electron and muon respectively. The highest pT cuts are applied to two of the leptons in order to suppress the background.

DONE 11) Table 5: Why does the first m(ZZ) bin starts at 100GeV?

==>> Because of the event selection, we don't use events with diboson mass much lower than 100GeV. In any case, such events would not bring in information since we are interested in high mass or pT bins.

Could you please add uncertainties?

==>> similar to other comment, will be addressed together with the others.

DONE 12) Table 6: Do you know why there is reweighting uncertainty for ZZ signal? Taking a flat normalisation uncertainty looks like a too simple approach. Do you know whether the uncertainties are available in bins of m(ZZ)?

==>> This uncertainty was the larger of the last bin, simply because of the statistics in the MC sample. To be conservative they used this value for all MC bins (basically the uncertainty that comes from the fitting procedure of the TGC modelling)

DONE 13) L173: You written "bin-to-bin migrations are taken into account". But previously it was written that systematic uncertainties are treated fully correlated. How does this work?

==>> We delete this phrase, it is a bug in the text.

DONE 14) L199f: Where did you get the weight vector from?

==>> not sure I complete follow the question. Do you mean the weight form eq(2)? If yes, it basically comes from the MC generated events, the events in the MC samples which assume TGC carry this additional weight. It is of quadratic form since we have a linear Lagrangian. Please let us know, in case this is not what you asked.

DONE 15) L212: What is alpha and theta? (I guess the vector of parameters of interest and the nuisance parameters?)

==>> correct, added in the text

DONE 16) L213: Maybe it is worthwhile to note that delta_ij is a relative uncertainty.

DONE 17) I have problems understanding Equation 4. In my opinion one should seperate the discussion on the constraints of the nuisance parameters (Gaussian, log-normal etc) from the inter-/extrapolation of the event yields for different values of the nuisance parameters (as one usually only knows the yield for sigma_j = -1,0,+1). I understand that Equation 3 is the linear interpolation and equation 5 corresponds to a polynomial inter-/extrapolation. What bothers me about equation 4 is that for theta_j = 1 I would expect a factor of (1 + delta_ij). But I can't see how to get there.

==>> Equation (4) is the technical implementation of lnN shape for the nuisances. In CMS they use lnN shape implementation with eg (5). Equation (3) is technical implementation of gaussian shape of nuisances. I don't understand what is meant by "inter-/extrapolation" in this question. If this is referring to the way how, for example, effects from two different nuisances are added together then your understanding is not correct. In all three cases (eq 3-5) we add those by doing a factorisation: nuisance1*nuisance2=(1+relative_uncertainty1)*(1+relative_uncertainty2). Where "linear" summation would be: nuisance1+nuisance2=(1+relative_uncertainty1)+(1+relative_uncertainty2). This "linear" summation is not used in either of eq 3-5.

The way how "nuisance shape" (nuisance effect) (gaussian/lnN) is achieved is the following. "Nuisance distribution" for both cases is unit gaussian. So theta is gaussian distributed. The "process normalization" for gaussian nuisance effect is flat: yield=nominal_yield*(1+theta_j*delta_ij). The "process normalization" for lnN nuisance effect is: yield=nominal_yield*(1+delta_ij)^theta_j. And this is exactly what you see in eq 3 for gaussian and eq 5 for lnN. In eq 4 the final result is again lnN nuisance effect, but it is achieved by different different "process normalization": yield=nominal_yield*e^(theta_j*(ln(1+delta_ij^2))^1/2).

DONE 18) L230: If I understand it correctly, you use the parameter point (alpha_test, theta'') to generate the pseudo-data. To account for variations in the central values of the systematic uncertainties, you later add a penalty term to the likelihood in equation 8 by introducing theta_0. I think this will introduce a (probably small) bias. From what I have seen so far, people usually first draw a "new" central value for the systematics (theta_0 in your syntax) according to the constraint (log-normal in your case) and the value of theta''. This value of theta_0 is than used to generate the pseudo-data.

==>> What we do is the following: Toy tossing (pseudo-data generation) was done in the following way: - toss 1 toy by poisson with signal (using nuisance fixed to best fit on data distribution) - toss theta_0 value with Gaussian mean theta_hat Then calculate likelihood for this 1 toy using theta_0 value

DONE 19) Equation 10: Do you have plots for the sampling distribution of t_alpha in order to see how good the approximation by a chi2 distribution is (maybe only for a few example aTGC points)?

==>>There is no plot available, the two methods were checked to agree.

DONE 20) L248: The threshold of t_alpha = 5.99 for the 95% CL limit comes from the assumption that the sampling distribution for t_alpha follows a chi2 distribution with 2 degrees of freedom. I assume that this was used because of the 2D limit plot. But I think that not onyl the number of the free parameters is governing the dof but also your measurement. You do have a binned distribution and could potentially differentiate between the effects of two different couplings (given that their impact on the expected yields in the bins is linearly independent). But as your whole sensitivity is coming from the last bin, I guess that you have an "effective number" of degrees of freedom very close to 1. Could you please show the sampling distributions of t_alpha for the 2D case?

==>> It is true that there is some level of "indistinguishability" between the two parameters, and we do not take this into account when setting 2D limits. We use the most conservative scenario when using threshold of t_alpha = 5.99 for 95%CL limit. You can find t_alpha 2D distributions attached here.

DONE 21) Tables 9 and 10: When comparing the expected and observed limits, it seems that CMS has slightly worse expected limits but better observed limits. When comparing expected and observed event yields, it is not obvious why ATLAS or CMS should show a different performance. I suspect that the better observed limits from CMS arise from the more agressive binning. For future combinations, it would be good to agree on some criteria for the binning beforehand (like at least 3 events expected from SM in the last bin or similar). I am not sure whether one can do anything about this now.

==>> Table 9 shows that the expected CMS limits are consistently worse as expected since CMS is not using the llvv channel. The observed limits are better for CMS due to a deficit in the last bin and for ATLAS worse because of an excess in the last bins for both channels. We agreed in the beginning that we will stay close to what each experiment was doing for its original analysis/publication. This cannot be changed now.

typos and style

DONE 1) L2: from my limited English knowledge: shouldn't it read "... from ZZ production in pp collisions..."

DONE 2) L7: maybe insert "dervied" before "from ZZ production"?

DONE 3) L9: I think that the inverse femto barns shouldn't be italic according to the ATLAS style.

DONE 4) L35ff: In my humble opinion, this paragraph uses quite a bit of slang. I am not sure that "neutral (boson) vertices" and "charged (boson) vertices" are the correct technical terms. In my understanding, vertices represent interactions and do not carry any charge. Instead, bosons refer to some propagators which do carry charge. So I think that things like "neutral vertices" or "boson vertices" are not well-defined. Could this be rephrased?

==>> rephrased to vertices of neutral gauge bosons

DONE 5) L37: "... deviations of the coupling strength from the SM prediction in the case of ..."

DONE 6) L40 beyong -> beyond

DONE 7) L46: "...combination of the measurements of..." I think, you can't combine limits but only measurements.

==>> you have a point, changed to “the combination of the results of the limits on …”

DONE 8) L47: maybe: was -> is ?

DONE 9) L51: "...while the CMS ..."

DONE 10) L61: The first pT should have a subscript.

DONE 11) In Section 2.1.1 (L64) you defined a cone by dR = 0.2. Now you are writing dR < 0.3. Please harmonise.

==>> 2l2nu uses a different cone. DR=0.3 is correct.

DONE 12) L89: "Events with additional leptons ..."

DONE 13) L93: "... of the scalar sum of the transverse momenta of track associated to the jet originates from ..."

DONE 14) L111: "... to extrapolate to ..."

DONE 15) L115: "... was extracted ..."

DONE 16) L148: add "... pass the isolation requirement..."

DONE 17) L148: " This rate was measured ..."

DONE 18) L159: "... measured as 2.2%" ?

==>> changed to "estimated at 2.2%"

DONE 19) Table 5: There is a 0 at the end of the caption.

DONE 20) L162: "Anomalous couplings result in an increase of the cross section at high energies. Therefore..."

DONE 21) L163: "... the mass of the diboson system and the transverse momenta of the bosons are ..."

DONE 22) L164: "... the leading Z transverse ... CMS uses the mass of the diboson system."

DONE 23) L165: "... but flat in the diboson invariant mass. This results in a ..."

DONE 24) L166: "Anomalous coupling ..." could this sentence be rephrased? It sounds very odd to my (German) ears.

DONE 25) L195: ".. fit to the expected event yield in every bin of the observable."

DONE 26) L214: "The nuisance parameters are constrained ..."

DONE 27) L216: abd -> and

DONE 28) Fig 1 needs some styling (same number of digits for all numbers, labels are overalpping in the bottom left, the integral sign reaches into the diagram at the top).

DONE 29) L289: remove the slashes

DONE 30) L289: harmonise the "pp" collisions (sometimes italic, sometimes not)

DONE 31) L290: inverse femto barn shouldn't be italic

Reviewer 3


  • Comments Set 1

*Are code and inputs used in SVN so the results can be reproduced?

==>> We will make sure this is the case.

*While the sensitivity is driven by the 2l2v channel, not present in the 7 TeV CMS analysis, there is still a 20% improvement in the combined limit. This might be due to the different sensitivity optimisation approach used by CMS - would be good to have a sentence on that in the note.

==>> Table 9 shows that the expected CMS limits are consistently worse as expected since CMS is not using the llvv channel. The observed limits are better for CMS due to a deficit in the last bin and for ATLAS worse because of an excess in the last bins for both channels.

*So far we have no "money plot" illustrating the impact of the combination. I suggest to show the 1d limits for


-ATLAS 2l2v

-ATLAS combined

-CMS 4l

-ATLAS & CMS combined.

==>> Money plot provided.

More detailed comments will follow next week as discussed. An example of how comments from the CMS ARC were handled in a past ATLAS/CMS combination (W helicity) can be found here:

So basically only the resulting note changes were reported in CDS.

  • Comments Set 2


*There is no mention of the (SM) signal MC used and how the (SM) signal expectation was normalised (neither for ATLAS nor CMS). Same for the aTGC normalisation - is it just based on SHERPA xsecs?

==>> ATLAS used the SM cross section form Powheg and the gluon fusion contribution from gg2ZZ (5.8%, a percentage that was taken from MCFM) For the TGC we took the parametrization of the cross section of the reweighed Sherpa sample and we scaled this parametrization to the previous cross section (Powheg) Similarly, CMS used Powheg and gg2ZZ for SM and Sherpa normalized to Powheg for TGC.

DONE *The ZZ theory uncertainties in Tables 6 and 3 differ by up to a factor of four - why?

==>> ATLAS uses pT bins while CMS uses mass bins of the diboson system. This yields to different theory uncertainties as explained in line 165 in the text.

DONE *Tables 3-6: The caption should mention the experiment from which the numbers originate.

DONE Note readability would be much better if all tables could be kept horizontal.

DONE *Tables 2-4: How was the release of addtl. information wrt. publication handled by ATLAS?

==>> This has been taken care by the EW conveners.

DONE *CMS provides a yield breakdown per channel (4e,4mu,2e2mu) while ATLAS does not (Tables 5 vs. 1)

==>> ATLAS does not have a breakdown per channel.

DONE *There is some "statistics jargon" that will need to be properly defined, see below

==>> this is taken care wherever it was indicated either by rephrasing or by giving necessary references.

DONE *Table9/10: It would be good to add the results from the original publications here for comparison to understand the impact of changed limit setting procedures.

==>> As we try to keep the text short, we prefer to direct the reader to the original publications. Additional tables most likely will need more details. An easy comparison can be shown in a comparison plot that you have also suggested, which is under dicussion between the two experiments.


DONE l. 6 : "ATLAS an" -> "ATLAS and"

DONE l. 9 : fb -> fb

DONE l. 35 : "Only the self-coupling ...are..." -> "Only the self-couplings ...are..."

DONE l. 38 : "The deviation is" -> "The deviation from SM expectation is"

DONE l. 42 : "from both" -> "by both"

DONE l. 44 : "worlds most" -> "world's most"

DONE l. 49 : "results" -> "anomalous coupling results"

DONE l. 51 : "while CMS analysis" -> "while the CMS analysis"

DONE l. 54 : use consistent nomenclature throughout the paper: 2l2v, 4l (as in the abstract/introduction) vs. llll llvv

DONE l. 56 : "on-shell Z" -> "on-shell Z boson"

DONE l. 59 : "The event is required" -> "Events are required"

DONE l. 62 : Do you include the transition region between the barrel and endcap calorimeters (1.37 < |η|<1.52)? ==>> YES

DONE l. 63 : "less then" -> "less than"

DONE l. 67 : "forward muons spectrometer" -> "forward muon spectrometer"

DONE l. 69 : "Calorimeter tagged muons reconstructed" -> "Calorimeter tagged muons are reconstructed"

DONE l. 82 : What about the ZZ* mass range you mentioned in l. 56?

==>> 84 ZZ* events are observed, however they are not used for the extraction of the limits on the anomalous triple gauge couplings. The comment was added in the text.

DONE l. 89 : "Event with an additional leptons" -> "Events with additional leptons"

DONE l. 90 : "Jets are used to veto background events" -> "In order to further reduce background contributions, events containing at least one jet are vetoed"

DONE l. 93 : "of the track" -> "of their track"

DONE l. 95 : "If after applying these requirements at least one jet remains the event is rejected. The event is required" -> "Events are required"

DONE l. 97 : "pair are expected" -> "pair is expected"

DONE l. 98 : "decaying Z" -> "decaying $Z$ boson"

DONE l. 99 : "Z+jet" -> "Z+jets"

DONE l.105 : mention use of fake factor?

==>> The current description does not mention the fake factor explicitly; as mentioned before, we would like to keep the description brief and refer the reader to the original publication for the full details.

DONE Table1: Make font size = that of Table 2; Table 1: why are uncertainties from paper [1] dropped here?;

DONE Table 1: pT>200: 0.1 should be 0.01

DONE l.107 : Which Z-pT is used from the two?

==>> in each pT bin of the highest pT Z boson in the event.

DONE l.109 : "techniques were used" -> to do what?

==>> sentence rephrased

DONE l.115 : "(see [1])" -> "[1]" Table2: You provide a breakdown of backgrounds that was not available in the original publication: bg->MC-bg, dd-bg. Was this release of addtl. information blessed by ATLAS? Why are uncertainties from paper [1] dropped here?

DONE l.119 : "Tabel 3 and 4" -> "Tables 3 and 4"

DONE l.124 : "combined uncertainty on the extrapolation procedure" -> what was combined here? ==>> combined was replaced by total (statistical and systematic)

DONE l.125 : "is dominated" -> "are dominated"

DONE l.126 : "and from" -> "and by"

DONE l.137 : No mention of triggers here, unlike in the corresponding ATLAS section -> streamline descriptions

==>> At the trigger level, the selected events are required to have either at least two electrons, one with pT > 17GeV and the other with pT > 8GeV, or at least two muons, one with pT > 13GeV (pT > 17 GeV for high instantaneous luminosity data-taking periods) and the other with pT > 8GeV. Mentioning of the triggers is included now.

DONE l.151 : The isolation requirement on all four leptons was reversed???

==>> a bit unclear from the text indeed, CMS will take care of rewriting this part of the text. The way the background is estimated is in few words: 1) determine fake-rate (FR): from Z+jets sample (select Z1+1 loose lepton, where loose lepton has no isolation requirement and has some other cuts loosen as well) 2) apply FR on control region: here control region is Z+2 OSSF loose leptons. So to answer the question, the isolation was not reversed, but just loosen. And this looser ID is applied to 2 leptons.

DONE l.154 : From which bin was the uncertainty value obtained then? Using the one with largest uncertainty?

==>> from the total, using the integral of all bins

DONE l.156 : "uncorrelated across the three decay channels": this is the opposite of what ATLAS does. Can a reason be given for the differing approaches? How does the result change when assuming correlation across the three decay channels - I assume this is negligible?

==>> As far as I know, CMS's reasoning for assuming the three decay channels uncorrelated is based on the uncertainty on the background that is estimated from data-driven methods is dominated by statistics of the control sample. In the end, it is a matter of choice on how the analyses were performed. We tried to follow the approaches that were used in the original publications as much as possible. The effect of different treatment of the systematics is negligible.

DONE Table3: Why does the "Data Driven Background Statistics/Systematics" uncertainties get smaller with increasing pT(Z)?

==>> It is considered a statistical fluctuation.

DONE Table5: ".0" -> "."

DONE l.162 : "Anomalous couplings results with increase of a" -> "Anomalous couplings result in an increase of the"

DONE l.163 : "AT-LAS" -> "ATLAS"

DONE l.164 : "leading Z" -> "the leading $Z$ boson" ; "uses diboson" -> "uses the diboson"

DONE l.165 : "p_T" -> "p_T^Z" ; "this results in signal" -> "which results in a signal"

DONE l.166 : "Anomalous coupling signal model contin-..." -> not a sentence.

DONE l.170 : "statistical definitions" -> "statistical approaches"

DONE l.173 : "parameters," -> "parameters;"

DONE l.179: "Triliniear" -> "Trilinear"

DONE l.184: "and likewise by" -> "while the terms scaled by"

DONE l.185: Explain the form-factor and its ΛFF, give functional form. ==>> Paragraph refurbished.

DONE l.187: As this contradicts EW group policy, do we have to add a statement here about the (lack of) physical meaning?

==>> ATLAS has published limits without employing a form-factor, so we are not sure this contradicts the EW policy. We consider that a statement indicating explicitly the lack of a physical meaning can likely get political. In any case, this paragraph has been rewritten following other comments and now discusses the difference between the two cases (with and without form-factors).

DONE l.192: What about the neutral ZZZ couplings - were these not simulated? ==>> yes, it is added in the text.

DONE l.197: " re-weight a samples" -> " re-weight a sample"

DONE l.200: missing fZ5 in parentheses?

DONE l.202: How did you evaluate the consistency? CMS's approach is only an approximation?

==>> Both experiments are using quadratic dependence of the yield with aTGC, and the difference in how the fit parameters is derived between the two experiments is covered by the uncertainties on the signal modeling (please see Table 6)

DONE l.208: "The reweighting procedure" -> this only pertains to ATLAS - maybe state "reweighting/fit procedures"?; "allows" -> "allows to"

DONE l.215: "The nuisance parameter constrained with Gaussian distribution." -> "Nuisance parameters are constrained to follow a Gaussian distribution."

DONE l.216: "abd" -> "and"

DONE l.217: Why use log normal rather than gaussian constraints? What is the difference between 4 and 5 conceptually?

==>> These are just two different technical implementations of logNormal shape. They both have full summation of the nuisances. We have rephrased the sentence to convey the message in a clearer way. Both Gaussian and log-normal constraints were used to compare the combination results between ATLAS and CMS. The approach of log-normal constraints was preferred as it has the advantage of a lower bound to the nuisance parameters, while Gaussian constraints must be truncated to have the same effect.

DONE l.219: "the summation was used" -> "their sum was used"? What is the impact of this change on the limits wrt. the original paper?

==>> this does not have any big effects, moved to footnote as suggested by Jean-Francois. The impact was less than 5%.

DONE l.220: What does "Both types" refer to?; "AT-LAS" -> "ATLAS"

==>> The paragraph was rephrased according to JF’s suggestion, it refers to Gaussian and log-normal. The difference between the logNormal and the gaussian was about 1%.

DONE eq3-6: define "J", "I" in text

DONE l.225: "constrain" -> "constraint"

DONE l.229: define " a test statistic" ==>> fixed by reshuffling the text as suggested by JF

DONE l.231: define "parametric bootstrap" ==>> reference is given

DONE l.236: α -> αtest

DONE l.237: α -> αtest

DONE l.238: "pseudo-experiment" -> "pseudo-experiments"

DONE l.243: define Npseudo, Nipseudo

DONE l.246: drop "to" before "define" to make this a sentence?

DONE l.253: "in ATLAS measurement" -> "in the ATLAS measurement"

DONE l.258: "average version of the pseudoexperiments" -> unclear, how is the average obtained?

==>> I don't understand what is unclear. It is the average..

DONE l.264: How are the results impacted by these different methods? Why did you move away from the original methods?

==>> CMS concluded (internally) that CLs method is not appropriate for this type of limit setting. We used deltaNLL because it is faster for consistency checks between ATLAS and CMS tools and in addition we used F-C for the final results.

DONE Table 7, l.266: what about fact./renorm. scale uncertainties?

==>> this is included in the QCD (a_s) scale

DONE Table8: "shape, uncorrelated bins" -> "shape/uncorrelated bins" ? Explain "MC systematics", "Reconstruction" in caption.

==>> we prefer not to repeat explanations in the caption, the details are given on the text.

DONE Why does CMS not have the former?

==>> CMS does not have a MC background estimate

DONE l.270-272: Repeat of l.266-268, remove?

DONE l.273: "Expected" -> "The expected"; "both analysis -> "both analyses"

DONE l.275: How would the limits change if you treated the bgs as correlated?

==>> We have seen that variations in the treatment of the systematics of the backgrounds have insignificant impact on the extracted limits, as the backgrounds are in general low. To support that we could say that both collaborations have proven also for their publications that variations of systematics do not play a significant role on the limit extraction.

DONE l.278: "from ZZ analysis"->"from the ZZ analyses"

DONE l.279: "Limits on aTGC" -> "Limits on aTGCs"

DONE l.282: "two" -> "both"

DONE l.284: Why is F-C so much worse in agreement?

==>> This disagreement (of the level of 5%) between ATLAS and CMS was observed only for one of the TGC parameters on the expected limit. After cross-checks it was concluded that this was a statistical fluctuation.

DONE l.286: Which of the two combined results do you consider as "THE" end result? F-C or deltaNLL?

==>> Both of them, as they are shown on tables 9 and 10. deltaNLL was used for the extraction of 2D limits because it is much faster than F-C.

DONE Table9/10: Define "deltaNLL", "F-C" in caption;

==>> we prefer not to repeat the definitions in the caption, as they are already explained in the text.

DONE "other then" -> "other than";

mention ATLAS/CMS results in caption ==>> not sure I understand this one..

DONE Figure 1: (a), (b): swap "x vs y" to "y vs x" ; "vs" -> "vs."

DONE "other then" -> "other than"; "set to SM value" -> "set to their SM value"

DONE l.289: "/ZZ/" -> "ZZ"

DONE l.290: fb -> fb; in Figure 1 you differentiate the two luminosities used by ATLAS and CMS...

==>> corrected elsewhere in the text, as 5fb-1 for ATLAS is not correct.

DONE l.293: "relative the" -> "relative to the"

DONE l.314: [8] is not cited in the text?

DONE l.316: [9] is not cited in the text?

DONE l.317: [10] is not cited in the text? ==>> The are in Section 3.3

CMS comments and replies



ARC comments on PAS_v1

Questions and comments by R.Klanner

Specific comments:

6. The paper is well structured.

7. The language can be well understood, however, still needs quite some improvements. I guess this is reserved for a later step in the reviewing procedure. In addition, Table captions should also tell from which experiment the results come.


8. In my view the physics case should be somewhat expanded, so that non-hep physicists and also graduate students can grasp the high relevance of the aTGC measurements, and why it is important to get the highest sensitivity from the LHC.

Added few more sentences in the introduction.

9. The lumi uncertainty is considered to be fully correlated, however the uncertainty is different by close to a factor 2! I do not understand how technically this difference is taken into account, nor how they can be fully correlated. Similar for the PDF+alpha_S uncertainties, which because of the different analysis approach have very different values. On the other hand it is not obvious that the background can be treated as uncorrelated. The backgrounds are certainly correlated, and it is not obvious to me that this correlation disappears by estimating the background using data driven methods. I assume that there is little change in the results if the BG is treated correlated or uncorrelated. Has this been checked? I am fairly ignorant about these issues and would appreciate knowing the opinion of the statistics experts. Personally I find it quite difficult to understand the Tables 7 and 8.

Considering correlations between uncertainties from different experiments we followed the same strategy as in ATLAS+CMS higgs combination. Luminosity uncertainty contains machine uncertainty + detector uncertainty. Since machine uncertainty is dominant luminosity is treated as 100% correlated. The reason why the values are so different is because CMS analysis was updated once we had a new luminosity uncertainty value ~4%->2%, while ATLAS analysis was using older estimate. In newer ATLAS 7TeV publications the uncertainty is 1.8%. We did check the affect on the limits to using uncorrelated luminosity uncertainty. The difference was very small, ~1%. Theoretical uncertainties (PDF+scale) are correlated because the source of uncertainty is the same in both experiments. The exact value of this uncertainty depends on the selection. It will have different values for different pt(Z) bins, and in M(ZZ) it is basically flat. But the source of uncertainty is the same, the variation of scale and PDF, therefore they are correlated. Uncertainty on the background estimate is uncorrelated. Statistical part of this uncertainty is uncorrelated by definition. Systematical part depends on the background estimation method. Both experiments use fake rate method for data-driven estimate, but the selections are different, as well as "detector response". Due to too big difference in detectors, fake fractions and reconstruction, the uncertainties are uncorrelated.

10. Is it understood how much, and in which direction a form factor changes the CMS results? I assume that it would be a minor effect, however it would be interesting to know. I guess this can be estimated by reweighting and looking at the M_ZZ distribution.

To see the exact behaviour of the limit from form-factor one would have to regenerate aTGC samples with the form-factor, since the form-factor includes the s-hat of the event.

11. Personally to me the reweighting method used by ATLAS for investigating the effects of different aTGCs appears more attractive and efficient than the method chosen by CMS. Do the authors have an opinion on this? In case yes, is it intended to follow the ATLAS approach in the future? In l.194 it is not clear to me what is meant by consistent. Should it be replaced by equivalent? This has quite some different meaning.

The choice of reweighting procedure starts from what we have available in the available aTGC MC. ATLAS is using some sort of "Baur–Rainwater and BHO MC generators" plug-in to perform this reweighting of Sherpa events. This was never used or setup in CMS. I do agree that their reweighting is nice, since you also do not have to define the exact aTGC parameters values for the "parameter grid" in advance. The two approaches are different in the sense that "the input to the quadratic fit" is determined differently, but in the end we both model with quadratic function. And both inputs are affected by statistics in the high mass and pT region. As for the future studies, we should use whatever MC has the best calculation calculation at that moment. And also there are different parametrizations of aTGC available, and different MCs have different parametrizations implemented.

12. Has the ATLAS method (eq. 3) also used by CMS for limit setting? If yes, did the results differ?

Yes, both CMS and ATLAS tried using both gaussian and lnN distributions. The difference was <2%. The use of lnN is preferred from statistical point of view.

13. l.277 requiring a good agreement between the two results to ensure consistency -> meaning is not clear. I assume that good agreement was found. I do not see, how you could require or what you do in case of disagreement. Personally for me there is no good agreement; either there is agreement or not, and limit of what is called agreement should be stated.

We required a good agreement between results from ATLAS and CMS code for the same input. So this is to make sure that the tools are in sync. The sentance that follows specifies how close the two results are: "For the delta log-likelihood method the results are in agreement at the 1% level. For the Feldman-Cousins method an agreement at the level of 5% was achieved."

Questions and comments by Y.Maravin

As far as I know, the methods of combination were approved by the CMS statistics committee - it would be good to add relevant correspondence (perhaps links to the talk that you gave there) to the TWiki as well.

Here is the link to the talk i gave at CMS stat commitee meeting: aTGC_stat_com

Please see below for my comments/questions:

general questions:

line46 -> definition of isolation is a bit confusing. The way the sentence is constructed, it reads as if electrons are required to be isolated only in ECAL, while muons only in tracker. Is this true? Either way, the sentence is a bit complex and would benefit from some re-wording (and defining what clusters are)

Both calorimetric and tracking isolation is applied to electrons and muons. What changes is the cut; usually tracking isolation is at 15% and calo isolation at 30%. I can rephrase it as follows: Calorimetric and tracking isolation is applied to leptons; for calorimetric isolation, the transverse energy inside a cone of DR = 0.2 is required to be less than 30% of the lepton pT. Similarly for tracking isolation, the transverse momentum inside a cone of DR = 0.2 is required to be less than 15% of the lepton pT. No need to explain what a cluster is anymore.

line73 -> you cannot veto background events - you can suppress or reduce background using jets. So I propose that the sentence is modified (plus adding ZZ->ll nu nu is redundant in this section). You should mention the major background here (Z+jets I presume?), e.g. To reduce the major background from Z+jets process, we reject events that have at least one jet candidate with pT > 10 GeV and |\eta| < 4.5. Jets are reconstructed ....

Indeed the way the sentence was phrased was unfortunate. There is no need to mention here the major backgrounds, since this paragraph discusses the selection. We find relevant to emphasize here some of the selections that easily reduce backgrounds.

line93: There are various processes contributing to the background in ZZ->4l channel as well, but perhaps one of them is the major. You mention backgrounds in ZZ->2l 2nu channel, but nothing is said about 4l. What are the major backgrounds in 4l channel? For 2l2nu you should change various -> many.


line125: The invariant mass of the lepton pair -> which lepton pair? There are two in the selection. Perhaps The invariant mass of either lepton pair... ?

Both lepton pairs. Fixed.

line126: pileup corrected energy in tracks ... sounds very cryptic. Perhaps drop the pileup corrected completely? Again a reference to only two leptons. I propose to have a different structure - before you start talking about pairs - say that lepton candidates in this analysis are required to be isolated from other activity. The energy, corrected for pile-up, in DR < 0.3 cone around the lepton candidate is required to be less than 27.5% of the candidate pT.?


chapter 131: How is WZ contribution taken into account? Naively you would get true leptons there as well from W boson decay?

WZ could give 3 leptons, if there is an extra jet passing lepton requirements that can be fake ZZ background. This is included in fake rate estimate from data.

l139: Why third control region? You mentioned only one.


l141: bins of mZZ are not defined.


All tables: caption must be above the body of the table to conform to CMS standards


l157: I do not understand the last sentence of the first paragraph of Chapter 3.

This is about how the continuous atgc signal model was built. Meaning going from having an atgc grid of generated points to model description that is continuous in atgc parameter space. CMS is using polynominal fit on yield as a function of atgc parameter on expected signal yield, while ATLAS is using ME reweighting.

l176-l178: Lambda is not defined anywhere, so perhaps you drop it, and just mention what it is? As the effective lagrangian with anomalous couplings violates partial wave unitarity at high energy, the values of the couplings can be turn-off by imposing a cut-off at a specific scale Lamdba_FF?

Fixed. Added the description of what lambda represents.

l210-212: Not following the sentence - ATLAS used a different approach but then they decided not to? If so, why mention it here at all?

In ATLAS paper they used different method to sum up contributions from different nuisances. Here for the combination we are using the sumation as was done in higgs measurements. We mention this here because we want to document the method used for the combination but also document the differences wrt individual ATLAS and CMS publications.

l217: What is bin-wise Poisson distribution? Poisson distribution in each bin of data? What bin is that?

What is meant is Poisson distribution for every bin in observable.

Section 3.4 - you essentially repeat the information given in the first paragraph again in the second. I would remove the second paragraph altogether. Move the "Other uncertainties are statistical ..." at the end of the paragraph, and add a statement that PDF and QCD scale uncertainty for signal modeling arise from the same source in both analyses, and thus are considered to be fully correlated.


Typos/grammar etc:

the whole text:

$l$ -> $\ell$


flavour -> flavor


change past tense (mostly were) into present (are)

Fixed. Using present, except when referring to something that was used in original publication but not used in this document.

Once you define TGC - use it? You often spell it out.


Run spell-checker on the PAS - there were some spelling errors (I think I caught some - see below)


title: centre -> center



an -> and;


combined taking -> combined, taking


there are no correlations among experiments smile there are correlation among systematic uncertainties, so the sentence needs to be modified to be clear (taking statistical and systematic uncertainties and their correlations into account ?)


use \fbinv instead of $5fb$


$pp$ -> pp


centre -> center


Add at 95\% C.L. at the end of the sentence.



l15: Only the self-couplingS


l24: worlds -> world's


l25: improve significantly -> significantly improve


l36: $\ell^\prime$ is not defined - text should be added


l37: on-shell Z, -> on-shell Z boson,


l40: four leptons and a -> four leptons, and a


l41: The leptons -> The lepton candidates


l41: move the definition of DR from line 46 here


l42: $pT$ -> $p_{\rm T}$ (check the whole PAS for this)


l51: muon pT -> muon candidate pT


l51: Calorimeter tagged muons are reconstructed

Fixed. Reformulated and merged with ATLAS review comments.

l54: form a Z boson candidate.


l55: add a comma before "where"


l60: must have -> must form


l61: Exactly two Z boson candidates


l62: In case of an event having four same flavor lepton candidates, the lepton pairing ...


l64: were -> are (best to keep the same present tense in the PAS)


l67: add a comma after GeV


l73: An event with additional lepton candidates with pT > 10 GeV are rejected to ...


l81: back-to-back


l82: decaying Z boson, called axial-..., is defined as


l83: To further suppress the background from Z+jet production, the axial-ETmiss mist be greater than 75 GeV.


l88: data-driven is a jargon and should not be used. Propose to change: The background estimate for ZZ is measured in data.


l89: same for "cuts" -> "requirements" or "criteria"


l90: background-dominated


l92: keep oxford comma (A, B, and C), i.e., put a comma before "and the expected"


l92: in each Z boson pT bin


l94: techniques to estimate their contributions. The background from ttbar... is estimated in data by defining a control region with one electron and one muon candidates passing the lepton selection.


l100: estimated in data using so-called "template" method described in [1].


l102: [1],[3] -> [1,3].


l102: signal, and the expected (add a comma)


l105: Tables 3 and 4; was -> is;


l110: You already mentioned that the background for 4l channel is derived in data, so no need to mention it here.


l111-l112: systematics on -> systematics in


l114: were considered -> are considered


l117: used -> use


l117: The data was -> Data were


l118: two same flavor, opposite charge pairs


l119: This section summarizes ....


l120: Why is there a need to repeat CMS and 7 TeV and ZZ again? Remove starting from "used in the..."


l122: Events are selected by requiring two pairs of oppositely charged electron or muon candidates. Electrons are required to have |\eta| < 2.5 and pT > 7 GeV, while muons have to be reconstructed in |\eta| < 2.4 and have pT > 5 GeV. At least one of the lepton candidates must have pT > 20 GeV, and another pT > 10 GeV.


l125: is required to be between 60 and 120 GeV. (drop m_ll)


l129: with an invariant mass


l130: was selected -> is denoted as Z1, while the other pair is labeled as Z2.


l132: The major background, where one of the lepton is a misidentified non-isolated lepton, is identified in data using a control region with negligible signal contribution. The control dataset is selected by requiring a Z1 boson candidate and only one lepton probe without imposing any isolation criteria. The misidentification rate is calculated as a ratio of the ...


l137: This rate is measured ... for both muon and electron candidates.


l143: was -> is


l144: The uncertainty in background estimation is derived from the uncertainties in the misidentification rates due to ... -> then say about correlation (uncertainty is considered correlated across... and not correlated for three decay channels in the analysis: 4e, 4mu, and 2e2mu.


l151: analysis -> analyses


l152: were -> are; results with increase -> results in an increase of ...


l153: You need to clarify high energy of what you refer here? high recoil energy of vector bosons?


l153: high values of recoil energy of vector bosons; therefore, the diboson system mass and ...


l154: Z boson transverse...; put a comma before while


l155: put a hyphen between pT and dependent


l155: As theoretical uncertainties on the signal are pT-dependent, but flat in diboson mass, ATLAS used a pT-dependent signal shape uncertainty, while the CMS analysis uses a flat uncertainty.


l158: section 3.2 -> Section 3.2


l165: section 3.3 -> Section 3.3


l180: were -> are (and throughout the PAS, I will stop commenting this)


l193: eq.2. -> Eq 2 (fix the (2) - it overlaps with the formula)


l204: end with :


l205: lowercase here.

Fixed. Also in the above equations.

l207: Gaussian constraints are used in Eq.3 and log-normal in Eqs. 4 and 5.

Fixed. Combined with comment from ATLAS reviewer.

l209: ATLAS analysis uses implementation of event prediction in Eq. 4, while CMS uses Eq. 5.


l218: constraint


l219: This constraining term is identical for X and Y constraint. May be drop constraining?

If I drop "constraining" I am not sure that it will be clear which term.

l222: Eq. 6


l237: put a comma before "so"


l269: use proper punctuation with "thus", "therefore" etc: blah; therefore, blah


l281: Fig. 1


l284: /ZZ/ -> ZZ; $pp$ -> pp; centre -> center


l285: has been presented -> is presented; $fb$ -> $\fbinv$


l288: relative to


Comments by A.Askew

Abstract: Consider breaking the last sentence into two discrete thoughts: one which states that the leptonic states are used, and a second with the information about the 1-d coupling limits.


Line 17: This sentence is a bit overcomplicated. Since the case at hand is for the neutral gauge bosons, I suggest omitting the "deviations of coupling strength from the SM in the case of charged boson vertices"

Since this is an introduction we want to say in general about aTGCs, so here we mention both, neutral and charged.

Line 18: The statement about the deviation is somewhat odd. I suggest clarifying that any deviation can be quantified in terms of anomalous couplings.


Line 36: There's a statement here about two categories, one with both Z on shell and one with one on and one off, but I don't see the ZZ* selection described anywhere, only the ZZ.

"We have decided to erase the information about ZZ*, we think it is redundant after all."

Line 40: Suggest "Events are required to have..."


Line 42: Typo, $p_{T}$ instead of $pT$.


Line 43: "turn-on region" is a bit colloquial, no?


Line 68: The statement that the isolation requirements are changed is unnecessary, simply state the new isolation definition.


Line 73: "Events with an additional lepton with ..."


Line 152: Statement is a bit mangled. I'd guess it should be "Anomalous couplings result in an increase in cross section at high energies."


Line 155: Should probably be split into two sentences for clarity.

Fixed. Sentence is simplifled.

Line 168: suggest: "...allows to probe for the existence of ZZZ and ZZgamma..."


Line 176: "constrains"

Fixed. Reformulated.

Line 201: "...allows one to parametrize..."


Equation 4 has a period at the end for some reason. In equations 6 and 8, shouldn't it be N^{i}_{data} everywhere? There's a couple of just N_{data}.


Line 247: I think the clause "and also used in ATLAS measurement" can be safely removed for clarity.

I would keep this. Since in the document we want to document how the combination was done, and also what are the differences wrt ZZ ATLAS and CMS publication.

Line 284: $ZZ$


Line 288: Last sentence should be broken into two for simplicity.


ARC comments on PAS_v3

Comments by A.Askew

This new version looks pretty much ready. A couple of very minor language things:

Line 47: "Calorimetric (track) isolation requirement is applied..." should probably be "The calorimetric (track) isolation requirement is applied..."


Line 150: "The uncertainty is treated as correlated..."


Line 236: "...where the best estimate from the data is used..."


Figure 2: Is deltaNLL defined somewhere? Same comment for Table 9-10.

It was not. Added to line 252: "This method for extracting the confidence interval is commonly referred to as the delta log-likelihood method (deltaNLL),... "

Comments by R.Klanner


Overall I am happy with the content of the paper. In my view however, still an effort is needed to bring the manuscript in a shape, which is closer to a publication. A LE should go over the manuscript to correct the English. In the specific comments I give a few examples - however not being a native English speaker, I am not sure if these are correct

Is it clear why the theoretical uncertainty (pdf, scale, etc.) is significantly smaller for CMS tha for ATLAS?

This was because of a typo. In the table CMS, by mistake, included only the part of systematic that affects the acceptance, and not the yield. This is fixed now and both uncertainties are ~6%.

How are correlated systematic errors between the experiments treated, when the range of errors is different? (e.g. luminosity, theory (pdf, scale, etc.).

Uncertainties are treated as 100% correlated even if the value of uncertainty is different.


Abstract: taking into account the statistical and systematic uncertainties and their correlations into account. -> taking the statistical and systematic uncertainties and their correlations into account.


Combined one-dimensional limits (not obvious that meaning will be understood; specify that other parameter fixed to SM value) see also comment l.190

Fixed to: Combined limits on the coupling parameters are ... at 95\% CL, where all other parameters are fixed to standard model value.

l.18 from the SM -> from the SM value


l.47 Calorimetric (track) isolation requirement -> A calorimetric (track) isolation requirement

Already fixed (see comment above by Andrew)

l.52 a DeltaR < 0.2 cone-> a DeltaR < 0.2 cone

There is no difference?

l.57 Extended electron candidates are reconstructed in the range 2.5 < |eta| < 3.16, where neither tracking information nor charge mearsurement is possible. -> Electron candidates reconstructed in the range 2.5 < |eta| < 3.16, where neither tracking information nor charge measurement is possible, are called extended electron candidates.


l.78 the transverse momenta of track associated to the jet -> the transverse momenta of the tracks assigned to the jet (or associated with the jet)


l.89 After the full selection all backgrounds for ZZ etc. are significantly suppressed. The main background originates from events -> The main background of the “ZZ etc.” channel originates from events


l.98 their contribution. -> their contributions.


l.117 by uncertainties on extrapolation factors. -> by the uncertainties of the extrapolation factors.

This part is removed from document from ATLAS side.

l.121 use 5.0 fb-1 -> uses 5.0 fb-1


l.130 in Delta(R) < 0.3 cone -> in a Delta(R) = 0.3 cone


l.136 one of the lepton -> one of the leptons


l.148 The uncertainty on background estimated from data is derived from the uncertainties on the misidentification rates due to the limited quantity of data in the control regions. -> The uncertainty of the background, which was estimated from the data, is derived from the statistical uncertainties of the data in the control regions.


Table 5 Observed and predicted ZZ events for CMS. -> Observed and predicted number of ZZ events for CMS.


l.150 Uncertainty is treated -> The uncertainty is treated


l.156 cross section at high values of recoil energy of vector bosons -> meaning of recoil energy not clear to me

l.160 remove space between boson and , (comma)


l.163 The anomalous coupling signal model continuum in anomalous coupling parameters -> I do not understand the meaning of this long sequence of nouns

"The anomalous coupling signal model continuous in anomalous coupling parameters is built ..". The point is that signal model is continuous as a function aTGC parameter.

l.190 Limits are set for one dimensional and two dimensional parameter space where all other parameters are fixed to their SM values -> Limits are set for one dimensional and two dimensional parameter spaces where all other parameters are fixed to their SM values (I find the formulation: set limits for parameter space not very nice. Maybe, but also not so nice: Two types of limits for the parameters f_i^V are set: for the first all but one f_i^V are set to the SM values, and for the second, all but two are set to the SM values)

Fixed to "Two types of limits for the parameters $f_i^V$ are set: for the first (one dimensional limits) all but one $f_i^V$ are set to the SM values, and for the second (two dimensional limits), all but two are set to the SM values."

l.217 analysis uses implementation of event prediction in Eq.4, while CMS uses Eq. 5 -> The ATLAS analysis uses in Eq.4, while the CMS analysis uses Eq. 5 for the predicted number of events


l.243 but for pseudo-experiment -> but for the pseudo-experiments


l.257 overfull box

Fig.1 The letters are to small

Fixed. Increased the size of the figure.

Refs. [6, 8, 11] have to be brought into order


Comments by Y.Maravin

Note has improved, but still some issues need to be addressed. Please see some of my comments below. Most of my comments are very minor and can be easily implemented. I tried not to repeat what Robert and Andrew already mentioned, but if there is an overlap - sorry.

Major comments - uncertainties in Table 1, 2, and 5 for background and signal predictions are missing - I think all the numbers should have uncertainties except for number of data events - those are easy to estimate.

I see your point. We should list the total uncertainty here. In principle we found out that some of there tables contain the info that was not available in the published papers presenting CMS and ATLAS results individually. And to be able to have this extra information in the document we would have to approve these. So instead we will remove the extra information from the tables...

Uncertainties in Table 3 are given with a ridiculous precision - can you please provide reasonable numbers (same for signal prediction unless it is really known at per mil accuracy).

Same comment as above. We will remove the extra information from the tables...

Introduction got terse and less readable. I do not follow a meaning of a sentence about high energy particles (line 19) at all - you mean that high mass particles can influence production mechanisms at low energy via loops? If so, the sentence is disconnected from the rest of the discussion - this is supposed to be an example of how new physics can cause couplings to deviate from the SM predictions - so you need to introduce it accordingly. A statement that "any" deviation can be qualified in terms of couplings is incorrect - how would a coupling can quantify a deviation from my expectation of a good steak for dinner and the reality (macaroni cheese)? Joking aside - second paragraph looks like a set of two disjoint sentences. I think you should improve the flow of the introduction a bit and get a LE to read the text and fix potential language/flow problems.

Modified intro a little. LE from ATLAS will read it.

Can you please respond to the minor comments/answer Robert's physics questions/get a LE to read/comment on the note asap? I think we are close for a green light, but it would be best if we could push a bit to bring the draft to a better shape.

Regards, Yura

Minor/very easy to fix points: I would go through the text and make sure that all the units and definitions are used consistently, for example 7\TeV (note - no spaces). $\Pp\Pp$ for pp, not $pp$ (sometimes it is italic sometimes roman), $\fbinv$ - check the missing space in abstract etc.

Fixed. Everywhere: "pp", "\mbox{fb$^{-1}$}", "X TeV" (with space). Did you mean to use TeV without spaces? In other papers I see TeV is used with space.

Do not capitalize New Physics or Standard Model


You use \ell inconsistently in the draft - sometimes it is $\ell\ell$, sometimes it is $\ell^+\ell^-$ etc.


line 143 - you use $Z_1$ and $Z_2$ while the text defines Z1 and Z2.


Table formatting is also inconsistent in the draft - 5 and 6 are different, for example.


line 161 - modify $p_{\rm T}^{\rm Z}$ - your current definition has a roman p - inconsistent to the rest of the draft.


line 185 - do not use don't doesn't in a text - spell it out as do not or does not.


-- AlexanderOh - 2015-09-25

Edit | Attach | Watch | Print version | History: r16 < r15 < r14 < r13 < r12 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r16 - 2015-11-12 - SenkaDuric
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCPhysics All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback