Review Twiki for SMP-17-004

The twiki is used to prepare the final reading for SMP-17-004: "Observation of electroweak production of same-sign W boson pairs in the two jet and two same-sign lepton final state in proton-proton collisions at 13 TeV"

All comments received during the CWR can be found here

Color code

• BLUE answered
• PINK not sure
• RED to be answered

Table of content

Gobinda Majumder (majumder.gobinda@gmailNOSPAMPLEASE.com) Date of comment 08 Jul 2017

Thanks author for a good result and first observation.

I am having two questions,

L77 : Which lepton is used in the definition of Zeppenfield variable ? Not clear from the text.

We calculate the variable for both lepton and use the maximum value.

L89 : How do you estimate charge mis-identification of muon ? As I understand that DY sample in data can not be used for muon as it can be done for electron case, where energy can be measured independently using the calorimeter.

This was measured with cosmic events. In this dataset we cross checks using the same-sign mass distribution, knowing that this is a biased method to estimate the effect.

Regards,

Gobinda

Robert Harr (robert.francis.harr@cernNOSPAMPLEASE.ch) Date of comment 11 Jul 2017

Dear authors and reviewers,

Congratulations on a fine analysis and well written draft. I enjoyed reading the draft. I have just a few comments for you to consider.

Thanks, Rob Harr

1. You define abbreviations for particle flow (l. 63) and b-quark tagging (l. 83) that are never used. The abbreviations can be omitted. The abbreviation for parton distribution functions is defined on l. 52 and used once on line 121. The abbreviation could be replaced by the term with little loss of space.

fixed

Fig. 3: In the text you write that the blue region covers the region “where the model is not applicable” while in the caption you state that it is the “theoretically unphysical parameter space”. These are two different things, at least to me. If the region is unphysical, then no model should apply there. But a model being inapplicable does not imply that the parameters are unphysical, just outside the region where the model is valid. Which of these is the case here? Please fix the caption or text to clear up this confusion.

fixed the caption.

Fig. 3: It is odd that the observed limits go below the expected for doubly-charged Higgs mass below about 400GeV. Everywhere else, the observed result is slightly short of the expected. Is it understood why this happens below 400GeV? Can a short explanation be squeezed in the text to aid the curious reader?

While the limit is extracted from mll-mjj, you can the trend from the mll distribution. We have a deficit in the bin around 200 GeV and an access at 400 GeV in mll. This translates into the doubly-charged Higgs mass below. We where asked to remove the doubly-charged Higgs distributions from the plot which explains this. You can find the plot in the PAS or the AN. We did not change the text because the deficit and access are about 1 sigma.

Albert De Roeck (albert.de.roeck@cernNOSPAMPLEASE.ch) Date of comment 18 Jul 2017

Dear proponents Thanks for your paper on the observation of electroweak production of same-sign W boson pairs in the two jet and same sign lepton final state.

It is a nice concise paper, in a good shape, and bringing an excellent result. This should go for publication quite fast. I have only a few comments.

General comment - is this the first observation of this channel? (i.e. ATLAS did not yet publish new results for the 2016 run data, as I believe?). Then I would strongly recommend we say that. This is the first observation. We"first" in abstract and leave a discussion for the FR.

- How competitive are our H++ results with the ones from previous analyses [ref 33]? Can these be directly compared. If so it would be useful to comment on that.

the results are much stronger and can be compared directly. Quantitavely, the ssWW analysis give 1.5 times stronger limits but also uses all of the 2016 data. A WZ analysis with the full dataset is in preparation.

Details - line 4: “This prediction”? There was no discussion of a prediction before. Do you mean the observation of the Higgs? Anyway certainly the study of the EWSB mechanism in the channel studied in this paper is of interest in it’s own right and particularly interesting for internal consistency in the light of the Higgs discovery. So perhaps we can say “This discovery…”

changed to "This discovery ..."

- line 85: Does that really result in a significant reduction of the top quark background? I would think the phase space given by the two leptons from two different tops decaying could be quite large. Just by how much does it reduce that background, typically?

That requirement rejects low mass resonant backgrounds together with W+gamma events. Indeed, it reduces the amount on nonprompt background by ~15%. We rephased the sentence.

- line 103: “negligible for muons…” I believe that but can we add a number to that? Is it like 10^-5? How was it determined? (or was it not really looked into in this analysis, but just used from other analyses?)

First, we look at Z->mumu events and no peak is seen in the same-sign dimuon mass. Second, we refer to previous CMS analyses where this was checked in cosmic muons; and third, we confirm these findings in MC. We don't see any wrong-sign dimuon event in our selection MC events

- line 113: Can we give a motivation where this number is coming from? Why e.g. not 30% or 10%? I assume that the precise number does not affect the result too much, and 30% is not unreasonable but it would be good if we can motivate it, even just a reference.

We leave the sentence as is. We had a number of iterations on this point already with the ARC. The precision of the measurement of the fake background is limit by the number of events used for the measurement, i.e. of statistical nature.

- line 140: So do we consider that tau channels as a background, i.e. we subtract the realigning contribution in our signal region with MC? Or what do we do? This would be useful to state explicitly

we are following requests by the theory community. The fidicual cross section is measured as a signal strength mapped into the fiducial region using madgraph. When we quote the cross section, the contribution from taus is not considered.

- line 186: “No evidence…” isn’t that an obvious statement? Full agreement with the SM prediction was seen as reported just above… Somehow the way it is written now it looks like this is disconnected from the WW observation in the SM.

We agree that "no evidence" is obvious. Removed this part.

cheers, Albert

Stephanie Baffioni (baffioni@llrNOSPAMPLEASE.in2p3.fr) Date of comment 19 Jul 2017

Dear authors,

just a comment about electrons and their reference. Maybe you can add a sentence about electron reconstruction and add a reference to the electron paper L65 after the sentence "A particle-flow (PF) ... leptons.", something like :

"Electrons are reconstructed by associating a track reconstructed in the silicon detector with a cluster of energy in the ECAL"

@article{{Khachatryan:2015hwa, author = "Khachatryan, Vardan and others", title = "{Performance of electron reconstruction and selection with the CMS detector in proton-proton collisions at s√=8\TeV}", collaboration = "CMS", journal = "JINST", volume = "10", pages = "P06005", doi = "10.1088/1748-0221/10/06/P06005", year = "2015", eprint = "1502.02701", archivePrefix = "arXiv", primaryClass = "physics.ins-det", reportNumber = "CMS-EGM-13-001, CERN-PH-EP-2015-004", SLACcitation = "%%CITATION = ARXIV:1502.02701;%%", }

done. We also added a reference to the new muon performance pas.

Regards,

Stephanie.

Yuri Gershtein (gershtein@physicsNOSPAMPLEASE.rutgers.edu) Date of comment 19 Jul 2017

Comments from the Rutgers group

Thank you for producing a fantastic result!!

Overall:

There are two weak points in the presentation. The SSWW search itself needs to take a priority, which means the background description and treatment needs to come much earlier. The paper loses clarity by mixing the search goal with the VBS etc new physics aspects . Consider reorg that focuses on observation and then gets into other things.

we respectfully disagree. It is a short letter and the focus is clearly stated. It seems logical to describe the detector, datasets, objects and reconstruction before discussing the background. The BSM discussion gets a very small fraction of the paper.

Abstract: Overemphasis on statistics, underemphasis on physics Please give only one significance number for the observation. Since this is no place to explain what “based on the sm” means, avoid that number. 5.5 is not that different from 5.7 State the cross section measurement value and whether it agrees with SM Please mention that the search is in e/mu states, ie not taus (also in intro etc)

Let me just say that we will rediscuss the abstract in the FR. We should not state a cross section w/o a discribtion of what the fiducial region is. We added electrons and muons (also in the conclusion)

Body

L17: most (all??) new physics scenarios that result in aQGC are in fact new resonances. So that sentence looks a bit awkward. Rephrase to say that you consider general parametrization of new physics and also one particular model.

We study contributions using an EFT approach which explicitly does not consider addional resonances and one model with a resonance.

L21: first experimental result for same-sign W search…..

added

L24: Explain what is meant by sm based expectation

We write "SM prediction".

L25: The VBS sentence seems out of place. Move it to the para opening?

We explain what has been done and relate to THIS paper.

L36: (Background) simulation should not come before the reader is told what the backgrounds are. I suggest moving the e/mu reconstruction as well as a background description before launching into simulation details. (Admittedly, this is a cms-wide disease, not just this paper.) Just say briefly that nonprompt is the biggest background followed by WZ.

The signal topology is explained. Explaining the background before we talk about the simulation would require an extra paragraph which is then in content repeat when we discuss the background estimation in more detail.

L43: “The interference…” sentence is out of place here, since you are addressing it further on.

We explain EW signal, QCD background and the interference, seems logical.

L60: Not clear what “control” means. Simply state that a number of triggers were used with a resulting efficiency of…

The two sentences have been rewritten.

L68 para: why am I reading about Zeppenfeld variables when I am dying to know the story, ie, what the main backgrounds are and how they were reduced and estimated? Don’t I need to be convinced of the observation before all this detail dump?

we need to explain how we got the observation by saying which variable was used to discriminate the events.

L77: also, it would be good to specify whether you mean both leptons or at least one of the leptons here.

we write "Zeppenfeld vairable ... of the lepton", i.e. the variale is defined for each lepton and we use the max value as discriminant.

L80-82: this is a bit confusing. Either explain what selection is, or remove the sentence completely and just use the reference.

we don't follow why this is confusing.

L92: this para needs more details and needs to come earlier. In particular, you cant dismiss the main bkgnd description by referring it to [10]

we had a much longer description of this background in an earlier version of the paper review and it was decided to shorten it.

L99: should that charge mis-id discussion be earlier, (after L91)?

in L91 we start with the background estimation explaining the largest background first

L100: Scale factor is a buzz phrase. Please explain what it means.

scale factor is a factor to scale data-to-simulation.

L103: drop reference to the muons, already explained

done.

L104: What is the point of all the trigger details when even the main triggers are not described in the text? (We have been told that a multiple triggers were used, that is all. That does not call for all these details.)

this is a short letter.

L122: “is calculated in [28]”?

the uncertainty is calculated as part of the analysis and not in the reference. The reference explain the method.

L123: which backgrounds?

for all background. Removed the sentence.

L125: “Two dominant sources of background…” - this should have been told to the reader sooner! Please move it up.

moved up

L144: it would be good to have a reference or, if you determined it yourself, a description of how you did it.

we calculated this number.

L147: if accepting taus makes this number low, why not accept taus?

this is just a convention to quote the number, prefered by the theory community.

L149: “the value” -> “the lost fraction”

they are not really lost, just number accounted in the number.

Table 1: Looks good as far as it goes, but given the lack of information on the main background determination, it is difficult to take a serious critical look. It would have been nice to see the signal expectation in this table.

the signal is quoted

Figure 2: * does the last bin include overflows? If so, say it. If not - are there any? This should be said explicitely too.

the overflow is in the last bin. is overflow slang?

* it would be good to have another figure that shows the Mll distributions in bins of Mjj (you only have four). This way you could actually see whether Mll agrees for high S/B high Mjj bins.

we will add this plot to the supplementary material

L182: “and large mass” (remove dijet)

removed

Emilien Chapon (emilien.chapon@cernNOSPAMPLEASE.ch) Date of comment 19 Jul 2017

Dear authors,

Congratulations for this first observation of an important process in the study of the Standard Model. I have read the paper draft as a member of the CMS Statistics Committee, and I only have one comment related to statistics, though it is important: how was the significance of the signal estimated? You really need to describe this in the paper: what was the statistical procedure followed, what test statistics was chosen, how were systematic uncertainties treated, were asymptotic approximations used, etc. You can also find some recommended references there: https://twiki.cern.ch/twiki/bin/view/CMS/StatisticsReferences.

we used a likelihood ration and added a statement and references in the paper.

I also have a few minor comments on style and language:

abstract: should we specify "The observed significance of the signal"? Now it is a bit unspecific, though the implicit implication is rather clear. (suggestion, authors may decide what they prefer)

added

l77-78: there are too many "and" in this sentence. I think the first one in the list ("and eta_l") should be replaced with a comma.

rephrased

l106: then -> than caption of Fig. 1, 3rd line: then -> than

fixed both

Best,

Émilien

Greg Landsberg (greg.landsberg@cernNOSPAMPLEASE.ch) Date of comment 20 Jul 2017

Dear Authors and the ARC Members,

Congratulations on the first observation of electroweak same-sign W pair production! The analysis appears to be done carefully and the paper is generally well written.

Please, find my detailed comments below, split in the physics and style sections.

PHYSICS COMMENTS:

- L22: swap the order of Refs. [10,11] to follow the order of appearance and the order of listing them in the text [ATLAS followed by CMS].

fixed

- LL37-50: in Run 2 all the samples are generated by the {\sc MadGraph5_aMC@NLO} v 2.3.3 generator, but the generation was done either in LO or NLO mode. This is evident from the fact that you actually used the same reference for the two generators. [Note that your MadGraph referencing is incomplete - please use PubComm guidelines to complete it.] Therefore, please switch to {\sc MadGraph5_aMC@NLO} v 2.3.3 on L37 and {\sc MadGraph5_aMC@NLO} on L48. Somewhere in this paragraph you should mention that simulations include pileup.

fixed

- L49: {\sc mcfm} v7.0 (?) at next-to-leading order (NLO).

we added the version number.

- LL49-50: ... are generated at NLO with {\sc MadGraph5_aMC@NLO}. [Please, check this - I think the triboson samples are actually LO - this should be specified, if so.] The {\sc pythia} 8.205 ... [Also, you seem to forget mentioning single top quark background, which presumably was generated with Powheg.]

fixed

- L57: please, check this statement. The average pileup for triggers that were exposed to full integrated luminosity in 2016 was 27, see https://cms-service-lumi.web.cern.ch/cms-service-lumi/publicplots/pileup_pp_2016.pdf.

fixed

- LL60-61: surely we haven't designed the single-lepton triggers specifically for this analysis. In fact it appears that you haven't installed any specific trigger for this analysis. Thus, the statement about "a suite of triggers ... designed for this analysis" should be dropped.

fixed

- LL77-79: ... the Zeppenfeld variable [25], ηℓ is the pseudorapidity of a lepton, and ηj1 and ηj2 are the pseudorapidities of the leading and subleading jet, respectively. [Note "a lepton" as there are two leptons per event!]

fixed

- LL97-103: it would be logical to move this paragraph right after the one ending on L91, as you continue about the charge-flip rate determination from data in this paragraph. Delete ", while it is negligible for muons" on the last line though, as it's a direct repetition of what you said on LL88-89.

fixed

- LL104-123: have you considered an uncertainty due to the pileup description? Can you quantify it in the paper?

The uncertainty is below 2%

- On LL110-111, it's better to say that the statistical uncertainty due to finite size of each simulated sample was taken into account.

done.

- L111: The uncertainty of 2.5\% [LUM-17-001] in the integrated luminosity determination is considered ...

fixed

- LL118-119: The interference between the EW signal and the QCD-induced SS W boson pair production background is estimated using {\sc Phantom 1.2.8} generator [27] and is treated as a systematic uncertainty of up to 4.5 in the signal yield.

fixed

- LL121-122: what do you do about the PDF uncertainty in the signal cross section? Is this part of the theoretical uncertainty in the SM prediction? If so, you need to clarify this on L145, as you only mention scales there, but not the PDFs.

For the signal strength, and for the limit computations, the PDF uncertainty is part of the SM prediction. Indeed, that component vanishes once we compute the fiducial cross section. Regarding the theoretical cross section, we added the PDF uncertainty

- L129: please, give some details of the statistical analysis performed. Say that you used the likelihood ratio as the test statistic with nuisances represented as Gaussian (shape) or log-normal (normalization) distributions and that you used asymptotic CLs method, all with proper references.

added describtion and references

- LL131-132: what's the rationale of using just the mjj distribution when fitting the WZ control region? Is this due to a low number of events in that region? Please, explain in the paper.

We want to avoid theoretical uncertainties or biases from this background and demine the normalization in each mjj bin from data.

- LL132-133: the sentence is very confusing. Earlier, you said that the normalization uncertainty for the WZ background ranges 20--40\% from the control region study [LL113-114], but here you are saying that you treat it as a free parameter in the fit, while all other backgrounds are constrained in the fit around their central values within the estimated uncertainties. What do you actually do - let it float freely or constrain it. In the former case, why do you need the control WZ region at all, and in the latter, why do you need to mention the WZ background separately from the other ones? Please, clarify in the paper.

As said before, the purpose of the method is to use the control region to contrain the WZ background in bins of mjj. Text was adjusted.

- L141: use {\sc MadGrap5_aMC@NLO} here and specify that the theoretical cross section is estimated at LO.

fixed and added LO.

- L145: drop "QCD" [the renormalization and factorization scales, are not QCD scales; in fact there is only one QCD scale, which is ΛQCD, and which is definitely not what you meant].

dropped

- LL147-148: the discussion of including or not including the τ decays in the fiducial definition is confusing. Since you include them in the data, why not include them in the fiducial volume as well, which reduces migration from huge 21\% to passable 5\%? Otherwise, you are comparing apple top oranges, given that the fraction of sequential τ decays in your sample is as large as 15\%!

we are followed a request by the theory comminuty.

- Figure 2: Use "Nonprompt" in the legends.

done

- Figure 3 caption, last sentence: The blue area corresponds to an unphysical range of parameters where the width of the doubly charged Higgs boson is expected to exceed its mass [32].

we changed the caption to "where the model is not applicable" w/o giving more detail.

STYLE COMMENTS:

Abstract, LL8-9: vector boson interactions [compound modifiers of the noun-noun type generally do not need to be hyphenated];

fixed

P1:

L2: many accelerator- and nonaccelerator-based experiments. [PubComm style.]

fixed

LL4-5: electroweak (EW) symmetry;

fixed

L6: modifications of the Higgs;

fixed

L8: same-sign (SS) W boson;

I don't like the use of "SS"

L9: via the EW interaction.

fixed

L11: The selection of SS dilepton events reduces;

We did not change that.

L14: for EW and quantum;

fixed

L14+1: induced SS W boson;

not fixed

Fig. 1 caption. L2: the EW and QCD-induced W boson;

fixed

L19: via vector boson fusion (VBF) and decay to pairs of SS W;

not fixed

L21: First experimental results of searches for EW SS W boson production were reported;

fixed

LL21-22: based on data collected ... 8 TeV, corresponding to an integrated luminosity of approximately 20 fb−1 [10,11].

fixed

LL24-25: This Letter presents;

fixed

P2:

L37: The leading order (LO);

changed sentence

LL38-39: up to six EW vertices.

fixed

LL40-41: refer to as EW production. We consider only the EW production as the signal;

fixed

L42: while the QCD production;

fixed

L46: two SS W bosons;

not fixed

L52: (PDFs).

acronym is not introduced anymore

L58: in most of the paper you talk about electrons first and muons - second. The natural order of channels therefore should be e+e+νeνejj, e+μ+νeνμjj, μ+μ+νμνμjj - please reorder;

fixed

L61: The logical ``OR" of single- and double-lepton triggers;

we removed this part

L63: reconstruct observable particles in the event.

fixed

L68: identify SS dilepton events; add a comma before "while";

fixed

LL68-69: while reducing the contribution from the top quark;

fixed

L69: Two SS lepton;

not fixed

L70: electrons or muons;

not fixed

L71: |η|<2.5 (2.4) for electrons (muons) are required.

fixed

L82: associated with the;

fixed

L83: delete "(CSV)" - it's never used [and not the right acronym anyway];

fixed

P3:

L87: hadronically decaying τ lepton with;

fixed

LL87-88: The Drell--Yan events can pass the selection if the charge of one of the leptons is measured incorrectly. The charge mismeasurement in dimuon;

fixed

L94: on electrons and muons.

changed order

L101: charge misidentification rates;

ok

L113: The WZ background normalization uncertainty;

added

L121: uncertainty in the signal;

sentence has been rephrased

L129: of SS W boson pairs;

no SS in this paper.

P4:

L133: free parameters of the fit.

added

L134: The observed (expected) signal significance;

added

L135: The best fit signal [superlative compound modifiers are not hyphenated];

fixed

Table 1 body, header line: typeset all "e" in Roman and reorder the columns as e+e+, e+μ+, μ+mu+, e−e−, e−μ−, μ−μ−; in the first column, typeset "Signal + total bkg."

reorded table and fixed particles

L138: two SS leptons;

not changed

L147: at the reconstruction level;

added

L159: the LHC Run 1.

changed

L161: predict SS dilepton events;

not changed

L162: subscript "H" in Roman in three places;

fixed

P5:

Table 2 caption, L1: for higher-order; L2: in the effective field theory Lagrangian. The last column summarizes the LHC Run 1 observed limits obtained by CMS [10].

fixed. We do not add the references to the caption as they are given per result in the table

Table 2 body, header row: Run 1 observed limits [10];

fixed.

LL165,170,175: subscript "H" in Roman;

fixed

L166: add a comma before "while";

fixed

L170: subscript "VBF" in Roman; add a comma before "as shown";

fixed

L176: CMS Collaboration;

fixed

P6:

Fig. 3 caption, L2: subscripts "VBF", "H" in Roman.

fixed

L187: vector boson interactions;

fixed

LL190-207: flush-align the text to both margins;

fixed

References:

Refs. [1,2,25,28,30]: typeset the volume letter separately from the volume number and don't boldface it.

fixed

Refs. [1,2,14,16,20,25,27,30]: give just the first page, not a range.

fixed

Ref. [20]: separate parts of the journal name with spaces.

fixed

That's all. Good luck with the final editorial work and fast submission!

many thx!

Greg

Michael Henry Schmitt (m-schmitt@northwesternNOSPAMPLEASE.edu) Date of comment 20 Jul 2017

Dear Authors and ARC,

This is a very nice analysis and a beautifully written paper - a real pleasure to read. The result turned out picture-perfect and is very convincing to me. Please find my comments and suggestions below.

Regards, Michael Schmitt (Northwestern University)

---

Type-B: Content and analysis

line 111-112: You state that the luminosity is considered for certain background estimates, which is fine. But this uncertainty is also important for the value of the fiducial cross section that you report. The sentence as written seems to exclude or ignore this fact. You might just add "and for the fiducial cross section" to the end of the sentence, or something like that.

added your recommendation.

line 141: Maybe I am missing something basic here, but I don't understand why you are making an acceptance correction for a fiducial cross section. To me, a fiducial cross section is one that has no acceptance correction. Can you clarify please?

We are defining the fiducial cross section for MC generator objects. The MC is used to translate from the measured objects.

line 160-163: Perhaps you should give a reference where these parameters are defined.

The reference for the specific model is given in line 167 [31].

---

Type-A: Grammar, wording, etc.

Title: please tie the "W" to "boson" using a tilde.

fixed

I like the opening paragraph a lot! (Usually I don't...)

thank you.

line 12: Perhaps better to write "W boson pairs" with no "event"

fixed

Figure 1 caption: Perhaps "QCD-induced" should be hyphenated?

fixed

line 41-42: I suggest to rewrite slightly: " We consider only EW production as the signal in this analysis; QCD production is considered as background. " (The "while" seems to indicate a logical contrast, but here the two sentence are completely harmonious.)

connected with "and" now

line 43: I suggest "on a kinematic basis" in place of "in terms of kinematics" which sounds more conversational.

fixed

line 44: drop "found to be" - wordy.

removed

line 47: drop the comma after "annihilation"

removed

line 69: Change "processes" to "contributions." (You can't change a process by imposing selection requirements. A process is given by physics. You can reduce the contributions of a background process to your selected sample.)

fixed

lines 77-78: For readability, put a comma after "variable [25]", delete the "and" after "[25]", put a comma after "lepton", change "pseudorapidity" to "pseudorapidities" and add an "s" to form "jets".

rephrased

line 80: I think "identification of bottom quark decays" conveys a reconstruction of B meson (or Lambda_b baryon) decays. Could you change this to "identifying b quark jets"?

changed

line 83: I think you can drop "CSV" - not used as far as I can tell.

removed

line 106: then -> than

fixed

line 110: I find the construction "is found to be up to" awkward and ugly. Could we try something else, such as "is 3% or smaller" or "is no larger than 3%" or "ranges up to 3%"?

changed

line 110: Please consider changing "process" (which does not have a statistical uncertainty) to "background estimate".

rephrased sentence

line 117: Again, plese don't use "is found to be up to".

fixed

line 120: You might drop "up to"

dropped

line 120: "in the statistical analysis" seems to be a mistake because you are discussing systematic uncertainties. Perhaps you mean "in the fitting procedure" - but then I'm not sure exactly what you mean. Perhaps you can simply delete this phrase.

removed

line 130: Reword: " ... with a fit to the two-dimensional distribution of mjj and mll." (We fit distributions, not variables.)

fixed

line 134: Please remove "estimated". We don't estimate uncertainties, we assign them based on our judgement. It's not as if there is an exact uncertainty but we can't quite calculate it.

rephrased this part

line 134: Please change "was found to be" to "is" (must be in present tense; the "found to be" is wordy)

rephrased this part

line 135: You could delete "best-fit signal strength, defined as the" to make the sentence more concise. I don't think you need or use the term "signal strength" and of course you use the best-fit value - does not need to be stated. If you accept this change then delete the comma after "SM".

rephrased

Table 1 caption: Perhaps add "added in quadrature" after "systematic uncertainties". Otherwise is sounds like you are listing them separately.

added

Table 1 caption: "then" -> "than"

fixed

Table 1 is nice done, by the way.

thx

line 137-8: You can combine these two sentence so that they flow better: "The cross section is ... region, defined by requiring two same-sign leptons ... "

combined

line 150: Change "extensions to" to "extensions of". ("to" would imply that the SM is the destination of the extensions.)

fixed

line 152: This will read better if you interchange "both" and "in".

changed

line 153: In my opinion you should drop "The" before "EW"

left the article

line 155: You can delete "where all the results are" to make the text more concise.

removed

line 158: I think you can drop "observed" - who would think that you are listing the expected limites?

removed

line 158: Change "by CMS" to "by the CMS Collaboration" for a more appropriate tone.

fixed

The plots in Figure 2 are nicely done. However, I recommend that you move "CMS" to the upper left, above the horizontal axis line, and move the legend to the right and expand it a bit.

to be done

Figure 3 caption: drop "theoretically" - not needed when something is unphysical. Or, perhaps you mean "theoretically disfavored" or "inapplicable in this model"?

changed caption.

line 186: I think you should add "consistent with SM predictions" after "is reported".

changed.

line 187: Please consider deleting "stringent" All new bounds are "stringent" until they are superceded, which always seems a bit silly to me.

removed.

Andrew Buccilli (andrew.buccilli@cernNOSPAMPLEASE.ch) Date of comment 20 Jul 2017

Dear authors,

As part of the CMS Institutional Review, the University of Alabama CMS group congratulates you on a strong analysis, well written draft, and useful result. We’ve provided our feedback below for your consideration.

Best regards,

Andrew

Physics == Abstract, L7: Suggest adding the actual measured xsec value, along with a statement that it agrees with the LO calculation.

the cross section is only well defined with the definition of the fidudical region. Adding both to the abstract would be to long.

L37: Are you certain that MADGRAPH 5.2 is used rather than MG5_aMC@NLO in LO mode? You cite Ref. [13] for both.

we clarified in the text.

L49: What is meant when you say that gg -> ZZ is simulated with MCFM? For example, how do you treat nonperturbative effects that would be included in a full event generator?

removed this sentence. The ZZ background is not relevant to this paper.

L84: Consider citing BTV-15-001 in addition to include information about the updates to the CSV tagger for Run 2.

agreed and added

L84: What is the definition of the b-tag veto?

we veto events with a b tagged jet

Line 133: It is stated that the WZ normalization is taken as a free parameter. How does the fitted value compare to the expectation from the MC simulation of this process? More generally, why did you choose to fit the WZ contribution, rather than fixing it from MC simulation, or from a data control region?

we rephrased the sentence to make this more clear. We are concerned that the mjj distribution and acceptance is properly modeled, hence we let the data guide us.

Line 186: In the conclusion, suggest to repeat again the actual measured xsec value, along with a statement that it agrees with LO calculation.

again, the cross section by itself - w/o the fiducial region - is not very useful.

Editorial === Abstract, L4-5: use semicolons in lists in which an item contains a conjunction

not changed

Abstract, last line: "production" -> "production cross section"

not changed. The limit is on the production cross section x branching fraction.

L8: "the analysis" -> "this analysis"

changed

L19: you use a hyphen in VBF but not in VBS

consistent now

L30: "lead tungsten" -> "lead tungstate"

ok

L36-57: Paragraph starting line 36, about the datasets etc used: I think it might make more sense for the reader to describe these datasets, generators, etc. only after you have described the event selection, main physics backgrounds, etc.

we leave the order as is.

L106: "then" -> "than"

fixed

L114: "small number" -> "statistical uncertainty arising from the small number"

fixed

L155: define "CL"

defined

Table 2: put minus signs in math mode

fixed

Fig. 2: consider swapping the placement of "CMS" and the legend and maybe zooming in more

agreed and changed

Fig. 3: consider changing “blue area” to “hashed” or include “upper right”

not changed

References: General: throughout use CMS style: only the first page of an article is listed, not the range; include any journal series letter in the journal name, not in the volume number. References 1, 2, 14, 16, 25, 27, and 30 are affected. Ref. 14: does the article really span two volumes (205-206)?

fixed

Isa Dumanoglu (isa.dumanoglu@cernNOSPAMPLEASE.ch) Date of comment 21 Jul 2017

Commets from University of Cukurova,

Congratulations for the observation of electroweak production of same-sign W boson pairs in the two jet and same-sign lepton final state. It is well written paper. We have a few comments in below:

Please add the section "Introduction " before Line1.

the target journal is PRL which does not have section titles

L.25 in this line proton-proton is writen as pp but later in lines 54 and 179 it is writen as proton-proton could you please use same form?

consistent now

L.52 & 63 no need to use abbreviation for PDF since it is used only once.

PDF is not used anymore

L. 73 use small letter for "anti-k_t"

this is consistent with CMS notation and the others of the paper are fine with it.

L.83 CSV abbreviation for Combined Secondary Vertex is not used anywhere else in the paper, can it be omitted, or the full name used?

CVS is not used anymore

L. 97 move "p_T >" to the next line

connected the two

L.108 7 in 7% is look likes italic, if so could you please correct it?

fixed

L.111 Please give a reference for the int. lumi uncertainty

added

L.151 Could it be written as "charge conjugate (C) - and parity (P)"?

changed

L.155 Is it to use open form of CL(Confidence Level) ?

defined CL now

L162,165,170,175, Figure 3 caption: The 'H' subscripts on 's' should not be italic.

fixed

L170: The subscript 'VBF' should not be italik.

fixed

L.178 Please add section summary/conclusion

no, PRL does not support this

L.185 Replace "Standart Model" with SM.

replaced

L.190 Please add Acknowledgement as a section title.

no, PRL does not support this

Could it be possible to enlarge particles symbols in Figure 1?

we prefer to leave them as they are.

Best regards, Isa Dumanoglu. On the behalf of Cukurova group.

Alessandro Rossi (alessandro.rossi@cernNOSPAMPLEASE.ch) Date of comment 21 Jul 2017

This comments was submitted twice as CMS-SMP-17-004-001-COMMENT-011 and CMS-SMP-17-004-001-COMMENT-012

Dear Authors,

congratulation for the result obtained and for the clearly written draft.

Follows few comments from the CMS group at Perugia University

best regards

Alessandro

A. English/Style/Formatting (including figures)

L25 : sometimes you use pp and sometimes proton-proton scattering. Please be consistent

fixed

L30 : “lead tungsten” -> “lead tungstate”

fixed

L78 :probably “one of the two lepton” is better wrt “the lepton”, please consider also max(z*l1,z*l2)

rephrased

L106: “then” -> “than”

fixed

L111: reference for integrated luminosity uncertainty

added

L114: “dominated by the small number of events” -> “dominated by the low statistics”

rephrased

L130: “two-dimensional fit of the mjj and mll variables” -> “two-dimensional fit of the mjj and mll variable distributions”

fixed

Table1 (caption): “then” -> “than”

fixed

B. Everything else (e.g. strategy, paper structure, emphasis, additions/subtractions, etc).

L131-132 : Why in WZ control region only mjj is fitted? Can you add the explanation on text?

we added an explanation

L160 : Add some reference where these parameters are defined

the GM model reference explains the parameters

Do you consider the Double-Parton-Scattering background? Are there any specific selection criteria against this background?

yes, but they are tinny

Sijin Qian (sijin.qian@cernNOSPAMPLEASE.ch) Date of comment 21 Jul 2017

Dear SMP-17-004 editors:

It's nice to see this interesting paper having gone into the almost final stage. I have roughly read through the SMP-17-004-paper-v2.pdf, and would have some (large and small) questions and comments from a non-expert's point of view. I list them below, please make a note of it if any of them would be sound.

In case that these comments would not be displayed well, a simple text file is attached with the identical content. Thank you and looking forward to hearing from you.

-Sijin

---

begin --------------------------

In general

(1) In the Abstract, in the main text (L10) and in the summary paragraph (L181), at three places the "moderate" pmissT are mentioned. It

may be better to indicate the "moderate" quantitatively at all three (or at

least one) places, e.g.

"moderate missing transverse momentum" --> "moderate (e.g. < xxx GeV) missing transverse momentum"

we want to give these kind of details in the paragraph on event selection. All three sentences will be reviewed in the FR and we are open to a discussion there.

Page 1-2

(2) L4, L9, L14, L38 and L41. The "EW" should be explained at its 1st

appearance in text on L4 instead of L41, i.e.

(a) L4-5: "study of the mechanism of electroweak symmetry breaking ..." --> "study of the mechanism of electroweak (EW) symmetry breaking ..."

Then, (b) L40-41 can be correspondingly shortened from (and the 2nd of two "EW

production"s on L41 may be better to be changed to an alternative)

"refer to as electroweak (EW) production. We only consider EW production as

the signal in the analysis," --> "refer to as the EW production and only consider them as the signal in the analysis,"

(c) Before that, L9, L14 and L38 can be shortened from (at three places)

"electroweak" --> "EW"

this has been fixed.

(3) L10 and L65-66, the "pmissT" should be explained at its 1st appearance

in text on L10 instead of L66, i.e.

(a) L10: "moderate missing transverse momentum," --> "moderate missing transverse momentum (pmissT),"

Then, (b) L65-66 correspondingly can be shortened from

"The missing transverse momentum pmissT is defined as ..." --> "The pmissT is defined as ..."

this has been fixed (also in the conclusion)

(4) L24-25, it may sound more popular if to change from

"This document presents a study of ..." --> "This paper presents a study of ..." or "This Letter presents a study of ..."

this has been fixed.

(5) L32, L71 and L78-79. The "eta" should be explained at its 1st appearance

in text on L32 instead of L71, i.e.

(a) L32: "the pseudorapidity coverage provided by the barrel and endcap detectors." --> "the pseudorapidity (eta) coverage provided by the barrel and endcap

detectors.

Then, (b) L71 correspondingly can be shortened from

"and pseudorapidity |eta| < 2.4 (2.5) for ..." --> "and |eta| < 2.4 (2.5) for ..."

and (c) L78-79 also can be shortened at two places, i.e. (also, the "jet" at

the end of sentence should be plural)

"etal the pseudorapidity of the lepton and etaj1 and etaj2 the pseudorapidity of the first and second jet." -->

"etal of the lepton and etaj1 and etaj2 of the first and second jets."

fixed a and b, but left c because it read funny.

(6) L65 and L83 may be shortened from

(a) L65: "charged hadrons, neutral hadrons," --> "charged and neutral hadrons,"

fixed

(b) L83: (as the "CSV" has not been used afterward in whole paper)

"a multivariate technique (CSV)," --> "a multivariate technique,"

removed

Page 3

(7) L101-102, to be consistent at the beginning of both lines, a hyphen

should be added on L102, i.e.

"charge-misidentification rates and the scale factors are estimated ... charge misidentification rate is found to be between about 0.01% ..." -->

"charge-misidentification rates and the scale factors are estimated ... charge-misidentification rate is found to be between about 0.01% ..."

fixed but w/o hyphen.

(8) L111, the lumi uncertainty should be given a Reference, i.e.

"The uncertainty of 2.5% on the integrated luminosity is considered" --> "The uncertainty of 2.5% [xxx] on the integrated luminosity is considered"

reference added after luminosity determination in new sentence.

Pages 4-6

(9) L131-132, as the word of "only" may cause the possible ambiguities by

referring to the word before or after it, the prevention of problem may be

made by swapping some words, i.e.

"the signal region and in the WZ control region, although in the latter only

the mjj distribution is used." -->

"the signal region and in the WZ control region, although only the mjj distribution is used in the latter region."

fixed

(10) Table 1

(a) In the caption, the 3rd line, a word of "then" seems a typo, i.e. (also, the 1st of three "background"s in this sentence may be removed)

"Background processes contributing to less then 1% of the total background are not listed but included in the total background yield." -->

"The processes contributing to less than 1% of the total background are not listed but included in the total background yield.

fixed

(b) In the header row and the 3rd, 4th, 6th and 7th columns, to be

consistent with the text (e.g. L58, etc.) in this paper, the font of

electron "e" at four places should be changed from

"e+e+(italic) | e(italic)+mu+ | mu-mu- | e-e-(italic) | e(italic)-mu-" --> "e+e+(non-italic) | e(non-italic)+mu+ | mu-mu- | e-e-(non-italic) | e(non-italic)-mu-"

fixed

(c) In the header column and the 2nd row, the 2nd word should be in the

lower case, i.e. (also it may be looked slightly better if two spaces can be added

before and after the symbol "+")

"Signal+Total bkg." --> "Signal + total bkg."

fixed

(11) L155, the "CL" should be explained at its 1st appearance in text here,

i.e. "expected 95% CL limits for ..." --> "expected 95% confidence level (CL) limits for ..."

fixed

(12) L162, L165, L170, and the Fig.3's caption. To be consistent in this

paper, the fonts of the subscripts "H" and "VBF" should be changed, e.g.

L162: (at three places)

"sin(thetaH(italic)), or sH(italic), where s2H(italic) denotes ..." --> "sin(thetaH(non-italic)), or sH(non-italic), where s2H(non-italic) denotes

..."

Other places where the similar changes can be made are

L165 (for "s2H"), L170 (for "sigmaVBF"), and Fig.3's caption (the 2nd line, for "sigmaVBF" and "sH", two places), etc.

fixed

(13) Table 2

(a) In the caption, the 2nd line, as the "EFT" has not been explained in

text, it may should be spelled out here, i.e.

"operators in the EFT Lagrangian." --> "operators in the effective field theory Lagrangian."

fixed

(b) In the right-most column, I presume that the last digits in the square

brackets in each row is the Reference quotations; if so,

(i) it should be explained in the caption of Table, and

(ii) it'll be clearer and looked better if a comma and a space can be

added before the quotation brackets, e.g. (also the quotation brackets are better to be aligned between rows)

"[-38 , 40] [10] [-118 , 120] [10] [-4.6 , 4.6] [29] . . . [-5.2 , 6.4] [10]" -->

"[-38 , 40], [10] [-118 , 120], [10] [-4.6 , 4.6], [29] . . . [-5.2 , 6.4], [10]"

clarified in caption and added the comma

(14) L173 and Fig.3's caption, the color of "blue" is mentioned at two

places, but can not be displayed in black-white. The problem can be solved

by indicating the locations of blue areas, i.e.

added the location

(a) L173: "The blue region shows the ..." --> "The blue region at the upper-right corner of plot shows the ..."

(b) Fig.3's caption: (the 3rd line)

"The blue area covers the ..." --> "The blue area (at the upper-right corner of the right plot) covers the ..."

(15) Fig.2

(a) In the horizontal axis labels, to be consistent in this paper (e.g.

Fig.3 and Table 2's header row, etc.), the units should be put into the

round brackets instead of the square ones), i.e.

"mjj [GeV] mll [GeV]" --> "mjj (GeV) mll (GeV)"

fixed

(b) The location of Fig.2 may should be moved forward to Pages 4 from 5 (before Table 2), since

the citation of Fig.2 is on L127 in Page 3 which is earlier than the citation of Table 2 on L155.

fixed

By the moving, the Fig.2 and its citation will not be separated by a whole

Page 4.

Pages 7-9, in the References Section

(16) L211, in [1], to be consistent in this Section, (a) a space should be added before the volume index, and (b) all references should have only one page index instead of two, i.e.

"Phys. Lett. B716 (2012) 1–29," --> "Phys. Lett. B 716 (2012) 1,"

Other ones which also need to be changed by the similar way are

for (a): [2], [25], [28] and [30]; for (b): [2], [14], [16], [20], [25], [27] and [30].

fixed the references

(17) L225, in [6], to be consistent in this Section and this paper, (a) an extra space between "p p" at the beginning of article title should be removed, (b) the fonts of two electron "e"s should be changed from

"p p -> jje(italic)+mu+nunu and jje(italic)-+mu+nunu" --> "pp -> jje(non-italic)+mu+nunu and jje(non-italic)-+mu+nunu"

fixed

Other ones which also need to be changed by the similar way as (b) are [11] and [29] (for "pp").

fixed

(18) L262, in [20], to be consistent in this Section, the 3rd word in the

article title should be in the lower case, and the spaces should be added

between the words in the journal name, i.e.

"“GEANT4: A Simulation toolkit”, Nucl.Instrum.Meth.A 506" --> "“GEANT4: A simulation toolkit”, Nucl. Instrum. Meth. A 506"

fixed

(19) The "year" number should be given for Ref.[33]. If there would be

problems to display the year number with the default bib file, it may be

fixed by changing from "article" to "unpublished" in the bib file.

switched to unpublished

---

end --------------------------

-- MarkusKlute - 2017-07-25