Seesaw Multilepton Analysis (EXO17006) CWR Discussion.

Thomas Ferbel:

*CMS no longer recommends the use of ETmiss, but suggests pTmiss.

- replaced with proper pTmiss symbol.

* CMS standard branching fraction is written as B not b.

- changed to B.

* Define all symbols in the text before pointing to a figure or table to assure better understanding.

* Avoid unneeded complications, such as "greater than or equal" signs for non-integers.

* We usually use m as symbol for mass (e.g., mT).

* Why do you use opposite sign (OS) rather than define "opposite-charge" as "OC?"

* It's best not to use horizontal error bars if they don't change over the given range of given values. Just give the value in the label for the ordinate, e.g., Events/10GeV.

For more, see the attached file, made using Adobe Acrobat PDF editing tools. Let me know if you can't get any of the yellow or blue boxes to open for your inspection. The red line is for deletion, and yellow highlighting for reminders of previously mentioned problems.

Greg Landsberg:

L3: corresponds to an integrated luminosity of 35.9 fb−−1 of proton-proton collisions at;

- done.

L7: with the expectations;

- done.

L9: and τ leptons.

- done.

P1:

LL4-5: particles coupled to leptons;

done.

L13: add a comma before "or";

- done.

LL20,22,25: subscript "e" in Roman;

- done.

L30: add a comma before "such as";

- done.

L31: Higgs boson production.

- done.

L34: counterclockwise-beam direction.

- done.

LL39-40: calorimeters that cover;

- done.

L41: steel flux-return yoke;

- done.

LL44-45: double-electron trigger;

- done.

P2:

L46: The double-muon trigger;

- done.

LL47-48: cross-trigger combinations that require a pT>23 GeV muon (electron) and pT>8 GeV electron (muon).

- done.

L51: at leading order precision;

- done.

LL51-52: parton distribution functions (PDFs).

- done.

LL52-53: calculated at next-to-leading order (NLO) plus next-to-leading logarithmic accuracy, assuming that the heavy leptons;

- done.

L54: {\sc powheg} 2.0;

- done.

L55: {\sc MadGraph5 _aMC@NLO} whereas;

- done.

L56: Higgs boson production are generated using {\sc powheg} and {\sc JHUGen} 6.2.8 [28-31] generators.

- done.

L58: For systematic uncertainty studies;

- done.

L60: using {\sc MadGraph5 _aMC@NLO}.

- done.

L63: for the differences;

- done.

LL71-72: from vector boson decays, as well;

- done.

L73: or leptons occurring;

- done.

L91: the charged- and neutral-hadron. muon, electron, and photon PF candidates using the anti-kT [new PubComm standard] algorithm [36] with a distance parameters of 0.4.

- done.

LL93-94: contributions from pileup.

- done.

P3:

Table 1 caption, L2: used for signal discrimination. L5: nonoverlapping [PubComm style];

- done.

Table 1 body, header line: Kinematic variable; CR veto;

- done.

L104: within a Z boson mass window;

- done.

L109: e+e− [e in Roman];

- done.

L111: Z boson mass window;

- done.

L115: charged-lepton momenta;

- done.

LL115-116: all particle names in Roman;

- done.

L120: 700--900 GeV [en-dash, not a hyphen];

- done.

P4:

L129: drop "dedicated";

- done.

L131: control regions.

- done.

LL135-136: satisfying the on-Z criteria.

- done.

L139: Higgs boson processes;

- done.

L151: Z in Roman [twice];

- done.

L153: from photon conversions is also estimated;

- done.

LL167-168: events with three leptons and an OSSF1, above-Z lepton pair, and for OSSF1 events with four or more leptons (16 of the total of 48 signal regions).

- done.

L171: due to the integrated luminosity;

- done.

P5:

Fig. 1 caption. LL1-2: with three leptons and an OSSF1, above-Z lepton paps (left); LL2-3: for OSSF1 events with four or more leptons (right). L6: and the dark- and light-gray bands;

- Changed accordingly.

L174: by the differences in the rates for Z+jets and;

- done.

L177: nonconversion backgrounds;

- done.

L183: 1--10\% [en-dash];

- done.

L185: uncertainties in the signal acceptance;

- done.

L187: due to the integrated luminosity measurement;

-done.

L191: 95\% confidence level (CL) upper limits on the sum of cross sections for the production;

- done.

L182: using the CLs method;

- done.

L195: 5--15\% [en-dash];

- done.

L196: choices of renormalization and factorization scales, and from PDFs in the mass range we consider. This uncertainty is shown;

- changed accordingly.

P6:

Fig. 2 caption, L2: subscript "e" in Roman; LL3-4: below 850 GeV. The expected limit is 780 GeV.

- changed.

L197: in the be--bτ plane;

- done.

L198: subscript "e" in Roman;

- done.

L199: to τ leptons increases.

- done.

L205: subscript "e" in Roman [twice];

-done.

L208: remove the section heading;

- done.

P7:

Fig. 3 caption, L3: The branching fraction for the muon channel can be; LL3,4: subscript "e" in Roman;

- done.

References:

Ref. [1]: add the doi reference.

- done.

Refs. [15,16,18,28-31,34,40]: typeset the volume letter separately from the volume number and don't boldface it.

- done.

Ref. [17]: delete "v3" from the arXiv reference.

- done.

Refs. [24,25]: delete "v2" from the arXiv reference.

- done

Ref. [24]: JHEP {\bf 10} (2012) 081.

- done.

Ref. [36]: fix "anti-k(t)" in the title.

- done.

Ref. [37]: Collaboration is spelled twice; also suggest replacing with the MET paper.

- Replaced the reference with MET paper.

Stephanie Baffioni:

Just a comment about electron reference. The sentence L65-68 is not completely correct, as the electron reconstruction is not entirely done within the PF. It could possibly stay as it is, but adding the electron paper reference after "Electrons" L65, or it could be replaced by the sentence from the guidelines https://twiki.cern.ch/twiki/bin/viewauth/CMS/Internal/PubDetector, adding the reference to the electron paper afer "track" :

"The particle-flow event algorithm reconstructs and identifies each individual particle with an optimized combination of information from the various elements of the CMS detector. The energy of electrons is determined from a combination of the electron momentum at the primary interaction vertex as determined by the tracker, the energy of the corresponding ECAL cluster, and the energy sum of all bremsstrahlung photons spatially compatible with originating from the electron track. The energy of muons is obtained from the curvature of the corresponding track."

@article{Khachatryan:2015hwa, author = "Khachatryan, Vardan and others", title = "{Performance of electron reconstruction and selection with the CMS detector in proton-proton collisions at s√=8\TeV}", collaboration = "CMS", journal = "JINST", volume = "10", pages = "P06005", doi = "10.1088/1748-0221/10/06/P06005", year = "2015", eprint = "1502.02701", archivePrefix = "arXiv", primaryClass = "physics.ins-det", reportNumber = "CMS-EGM-13-001, CERN-PH-EP-2015-004", SLACcitation = "%%CITATION = ARXIV:1502.02701;%%", }

- Keeping mind in the length of the paper, we have chosen your first solution and provided the reference to Electrons on L-65, as you suggested.

Luca Pernie :

Abstract: "heavy fermions..." -> "an upper limit of 847 GeV is set on the mass of heavy fermions"

L5 ...doublets. It accounts…

- Changed the sentence.

L44 I would shorten the sentence as "of 23 GeV and 12 GeV respectively."

-

L47 Same comments as L44

L83 "...and less than 7% and 8% with in a cone of size R=0.3 for electrons whose energy deposits are reconstructed in the ECAL barrel (|h| < 1.479) and ECAL endcap (|h| < 1.479), respectively." Could be: "For electrons, reconstructed energy deposits are required to pass a relative isolation within a cone of R=0.3 of 7% if situated in the ECAL barrel (|h|<1.479)" or 8% if found in the ECAL endcap (|h| > 1.479).

Sudarshan Paramesvaran:

Title could be better worded - if you stick with this format we suggest adding a "the" before "...type-III".

Line 21: "the 7 TeV " -> "7 TeV "

- done.

Line 72: Does "nonprompt" need a hyphen?

- cms style guide suggests to use nonprompt without hyphen.

Line 118: "cos" in equation is typeset in math-mode

- corrected.

Line 120: Wrong type of dash for range 700 to 900 GeV (hyphen used, need an em-dash)

- changed.

Line 183: Wrong type of dash for a range (1-10%)

- changed.

Inconsistent referencing of figures in the text, eg. line 197: "Fig. 2" but then "Figure 3"

- The sentence is ending at Fig.2, new sentence is started with figure 3, where details of Figure 3 are mentioned.

Caption to figure 3 and relevant part of text should mention at what level (95, 68, 90% etc) the limits are set. This is only stated for Fig. 2.

- corrected.

Alberto Zucchetta:

Abstract: L2: The data corresponds -> The data correspond

- corrected.

Abstract L4: wrong word breaking for "primarily"

- Latex type setting, word is "primarily" but latex is breaking it down due to format.

Abstract L5: "number of leptons" is quite generic, it's better to specify here that you are looking at final states with 3 or more leptons

- Its specified in the first line of abstract that we are looking for 3 or more leptons final state.

L25 (and in the whole Letter): you should refer to branching ratio with B not b (as suggested by the PubComm)

- changed accordingly.

L5: SM has one Higgs doublet, why it's plural here? If your model has more than one doublet, it's convenient to say it before

L6: a reference is needed after "masses".

L11: since it's not only one, it sounds better to move to plural "final states"

- done.

L20: to which mixing angle are you referring to? It shoud be specified.

- done.

L25: "probe" is a bit miss-reading word, as you don't explicitly target tau final state.

L31: triboson is jargon, add also "boson" after Higgs.

- given example of triboson processes, added boson word after higgs.

L43: add reference to the trigger paper

- done.

L58: This is confusing as here it says MC Z and tt samples used only for systematics, but Line 28 implies it is a major background. It is only at Line 142 that it is clear that Z and tt are essentially data driven.

- We have mentioned them as reducible background and also categorized it with misidentified and fake backgrounds, from which it is clear that we are using some data-driven technique for these backgrounds. It is too early to mention about data driven techniques in the introduction section.

L61 - 62: No description How you simulated pileup events in MC. Suggest to rephrase: "The presence of multiple pp interactions in the same beam crossing (pileup) is incorporated by simulating additional interactions (both in-time and out-of-time with the collision) with a multiplicity that matches the one observed in data".

- rephrased the sentence according to the suggestion.

L71: "prompt" is jargon.

L72: "arising" already used in line 71, maybe change it to "originating"

- done.

L74: "punch through" is jargon.

L88: add description How you define primary vertex

L91: The word, PF "candidate" is a vague expression (not clear what it means for outside readers)

L94: could replace "other interactions ... crossing" with the previously defined "pileup"

- done.

L118: it's better to say explicitly that you use mT instead of LT+MET. The symbol after cos is not defined. Which angle is this? It's probably delta phi, but it could be something else if you don't define it.

- We have mentioned that we use mT as a discriminator for this region on line 126-127. Corrected the expression, COS between the quantities inside the brackets.

L128: Perhaps move the table to after the reference to Table 1.

- done.

L133: Strictly speaking, WZ is a background too.

- Changed the phrasing to make it clear

L144: no need to specify "pp collision" (also L153-154)

- removed.

L197: "Fig. 2." --> "Figure 2."

"end and middle of the sentence should have Fig." APS PRL guidelines.

L200-207: It started with the past tense and then you changed to present. Should be consistent along the whole paragraph.

- corrected for.

References:

many references do not have DOI, ArXiV.

Several times the letter of the journal, e.g. Phys. Lett. "B" is bold face and merged to the volume number.

- corrected.

"technical report" should not appear in PAS references.

- its appearing in the published result https://cds.cern.ch/record/2264382/files/arXiv:1705.04673.pdf

Please check if [41] is the correct reference for the CLs.

- removed reference 41, added recommended references.

Figure1: suggests to add "category" (e.g. 3 leptons, OSSF1, above Z) inside the plot

Table 1: the acronym CR never explained in the text

- Now mentioned on L133, also in the label of the table.

Daniele Del Re:

General comments:

- “Nonprompt” should probably be written “non-prompt” instead

- - cms style guide suggests to use nonprompt without hyphen.

- be more careful with plural and singular forms

- checked.

Specific:

- Abstract, L2: suggest “The data sample consists in pp collisions at √s=13 TeV collected by the CMS experiment at the LHC in 2016, corresponding to an integrated luminosity of 35.9 fb-1.”

- reworded.

- Abstract, L4: suggest “Selection criteria based on the number of leptons and the invariant mass of opposite-sign lepton pairs are used to discriminate the signal against the standard model background.”

- reworded.

- L1: replace “gave”, e.g. : “provided a strong motivation” or “provided strong motivations”

- done.

- L2: reorder sentence, e.g. “Several extensions of the SM have been proposed to address neutrino masses”

- breaks the consistency

- L7: suggest “mass arises from the mediation”

- corrected.

- L12: “This final state arises from the following decays: …“

- reworded.

- L13: clarify the sentence “Decays of […] result in 27 different production and decay processes” and consider splitting it in 2 sentences for better clarity

- splitted the sentence.

- L29: “leptons from heavy quark decays”

- changed.

- L30: suggest “small background contributions also arise from…”

- reworded.

- L58: “In order to study systematic uncertainties, we also use …”

- reworded.

- L60: “using a dedicated software”

- corrected.

- L61: THE mismodeling

- changed the sentence in response to previous comments.

- L69: “[…] and is matched geometrically with […]” (avoid “spatially match with” formulation)

- done.

- L72: “boson decays”

- done.

- L72: “Nonprompt leptons include […]” --> “The latter include leptons inside or near jets, leptons from heavy quark decays, as well as misidentified leptons from hadrons that reach the muon detectors, or from hadronic showers with large electromagnetic energy fractions.”

- reworded.

- LL77-82: too long sentence. At least you can remove "for both... and muons".

- L85: remove ", respectively"

- done.

- L94: you already defined “pileup” so you could use it to shorten this sentence

- done.

- L101: “Events containing an opposite-sign same-flavor (OSSF) lepton pair with a mass below 12 GeV […]” (avoid repeating “with”)

- done.

- L103: “We reject trilepton events containing an OSSF pair with a mass below […]”

- done.

- L104: “This suppresses events from the Z—>ll—>llg background process, where […]”

- Re-worded according to the suggestion.

- L108: “For example, mmm is tagged as OSSF1, while mmee is tagged as OSSF2.”

- done.

- L118: is this way to indicate an angle commonly used in CMS? It would be nicer to replace it with a sentence.

- corrected the symbol.

- L120: “To maximize the sensitivity …”

- reworded.

- L121: “we categorize events in eight bins …” (not only the data as you do it for background and signal events)

- done.

- L121: “We use a bin width of 150 (100) GeV …”

- done.

- L121: “the highest bin includes overflow events.”

done.

- L123: “in each of five event categories: …”

- done.

- L127: I suggest to replace “orthogonal” by “independent” or “mutually exclusive” here

- done.

- L129: “dedicated simulated samples” (also avoid repeating “dedicated” twice in two subsequent sentences)

- the sentences are now changed.

- L131: “with OSSF1 on-Z” —> “tagged as OSSF1 and on-Z”

- done.

- L135: “with OSSF2 and both pairs satisfying on-Z criteria” —> “tagged as OSSF2 with both pairs satisfying on-Z criteria”

- done.

- L140: “from simulation” —> “from simulated samples”

- done.

- L140: “using the theoretical cross sections”

- done.

- L173: “An uncertainty of … is assigned to the final …”

- done.

- L177: “arises due to” --> “arises from”

- done.

Greg Landsberg:

General: I find the choice of PRL as the target journal odd. This is a nice analysis, but it probes a fairly esoteric [and not very fashionable] model of new physics, as evidenced from, e.g., the fact that most of your theoretical references date back to the eighties. Also, experimentally it is hardly different from, e.g. SUSY multi lepton searches, that we have published multiple times. Consequently, in my opinion it doesn't meet the requirements for the broadness and significant advancement of science, necessary for a PRL publication. Therefore, I believe that the paper would be a hard sell for PRL and will very likely delay the publication. I'd suggest submitting this paper to Phys. Rev. D Rapid Comm. instead, which would require basically no changes to the structure of the paper.

- Titlel: I don't really like "Search for evidence ..." clause. This sounds like a crime drama, not a physics paper! We search for a phenomenon; what me may find is an evidence for it (or the lack of thereof). I'd therefore really suggest that shorten the title to rea: "Search for type-III seesaw ..." You should also spell out proton-proton instead of using pp.

- Abstract, L11 and further in the text: please, do not cite the limits with ridiculous precision of 1 GeV, which is totally misleading, given that theory uncertainty in the signal cross section corresponds to something like 20 GeV accuracy in the mass limit. Please, round all the limits to 2 significant digits, namely use 850 GeV here. Ditto for LL194-195,205-206, Fig. 2 caption, and Fig. 3 where all the numbers should be rounded down.

- rounded down the numbers.

- LL2-3: one can't "address neutrino masses"; the only "masses" that can be addressed are crowds of people as in "the President addressed the masses". Here you can only address an issue with neutrino masses, e.g. address "neutrino mass hierarchy".

- reworded the sentence.

- LL12-13,16: all particle names should be typeset in Roman; better yet you should switch to pen names not to run in this issue. That includes H, Z, Σ, and e. Also, remove a stray "-" subscript from Σ0→Hν. There are many other occasions of italics in the text, which need to be taken care of by context search-and-replace; I'll note some [but not all!] of them in detailed style comments.

- done.

- L44: nowhere in the paper (after the abstract) you describe the data sample used. A natural place would be an opening sentence of the paragraph starting on L44. Please, copy the corresponding sentence from the abstract here; also introduce acronym pp here, , which you use later in the paper.

- done.

- L50: the version of Madgraph5_aMC@NLO you used is 2.2.2 or 2.3.2 [5 is part of Madgraph5] - please correct this; also you should typeset the generator name as {\sc MadGraph5 _aMC@NLO} 2.2.2.

- done.

- LL53-56: give the order at which various backgrounds are calculated (NLO, per the acronym I've introduced on LL52-53).

- L59: give the underlying event tune CUETP8M1 with a reference.

- L62: .... same or adjacent bunch crossings (pileup).

- L68: give also the second standard PF reference here: PFT-10-001.

- Table 1: I find the last column incomprehensible. Why is it called CR veto [it[s not that you veto the CR, but merely ensure orthogonality with the corresponding CR, which is part of CR definition, not the SR definition! Why it's not explained in the text at all? Please, explain this properly in the paper. Most logical step would be to remove this column from the paper and simply specify these selections when you define the signal region and mention that CRs are orthogonal to SRs by construction when you talk about CRs. Otherwise, a reader may wonder how come, e.g., the first SR with LT+EmissT being used as a discriminating variable, doesn't have any EmissT requirement.

- We have included the text explaining the control region in table label also on line 133.

- L118: use \cos, not cos, and remove the angle sign. Define the p⃗ missT on L96 and use it here instead of a really awkward E⃗ missT.

- corrected.

- L125: since you allow for more than 4 leptons, what do you do about the OSSF3 events that may arise. Do you have any? Please, explain in the paper. **

- the number of OSSF3 events are completely negligible and our background estimation method do not explicitly estimate 6 lepton event as a separate category. In case of more than 4 leptons we only use first leading 4 leptons to construct our kinematic variables.Thus we do not mention this in paper.

- L186: cite the Run 2 PDF4LHC recommendations here.

- done.

- L192: use just the standard CLs references: remove Ref. [41] and, if asymptotic CLs approximation was used, add a citation to Cowan et al paper [with the Erratum].

- done.

- Figure 2: use ±1,2 s.d., as σ is already used for the cross section. Ditto about theory uncertainties.**

- will be done.

Luca Pernie:

Abstract: remove "and help discriminate the signal against the standard model background"

L11 examining final states

L86: In the sense of PUPPI or PFCHS?

- PFCHS

L96: Etmiss: Shouldn't that be p_T^miss?

- corrected.

L97 I would say this in the previous paragraph already and only keep "Jet energy corrections" out of the highlighted sentence in this paragraph.

L107: "other kinematic quantities": What exactly does this entail?

- the description of the kinematic quantities is given in the coming text, we have reworded the sentence.

L109: OSSF1 and OSSF2 could become N_OSSF=1 and N_OSSF=2

- we prefer it as it is.

L133 How much of the error is due to statistic and systematic effects? How systematics are computed? Depending on your answer it could be something it is worth to mention in the text.L136 Same comment as for L133 **

- Uncertainty was calucalted as sqrt of squared sum of stat and all syst. Syst were MisID syst, Conv syst, Rare unc, and for WZ the ZZ norm unc.For ZZ all unc comes from stat because it is >99% pure ZZ region. For WZ we have around 10% due to stat, 75% due to MisID, 7% Rare and the rest Conv & ZZ.

L150: Does this also include other leptons, because this is not included in their description of relative isolation. **

- It doesn't include other lepton because isolation criteria ensures that they are in their relative cone. The relative cone size is 0.4.

L151 You do not mention the results you get from the matrix method and the error associate with it (you only show the final result in Fig.1). I believe you should mention the numeric results of the method.

- If by numeric result you mean the actual fake rates then that is not possible to discuss here given the very large number of separate bins in which fake rate is measured. The only meaningful output is the final result because that includes the possibilities as single, double, and triple fake fix.

L165 Also here the description of the method sounds a bit too vague. Could you at least define the selection based on lepton pT and photon Et, and the numeric value of the rate and the uncertainty associated. **

we have tried to improve the description. Given pt and eta dependence for electrons and muons quoting a single numeric value is not possible. Thus we have now mentioned a range.

L171, 173,176 Where this numbers are coming from?

- sources are described right after the numbers are mentioned.

Ref 37: You should write Physics Analysis Summary

Ref 37: duplicate "Collaboration"

- we have changed this reference to MET paper now.

Table 1 header: add "(CRs)" after "control regions". I don't think CR is defined in the text.

- Its defined in label as well as in the text now.

Table1 Why you did not divide the category with >= 4 muons depending on the OSSF mass(es) to be below/above the Z?**

- the number of events with 4 or more leptons are rather small and we wish to avoid further break down of these small numbers

Fig 1: It would be nice to have a solid line at 1 in the ratio plot to ease the comparison there and it's really hard to see the solid colors beneath the hatched grey uncertainty bands in the main plot. I can only guess where the actual values are for the combination with dark green.

Fig. 2 If the authors assume leptons are from W's and Z's, how you can justify a scenario like b_mu =1 or b_e = 1?

Fig. 2 In other words, could you specify any scenario in Type III where you can achieve b_mu = 1 (or b_e = 1)? This means those massive fermions don't decay through W's and Z's. Then how you can generalize your limits where your acceptance for leptons are based on leptons from W's and Z’s.

- the branching ratio ( b_mu, b_e ) refers to decay such as Σ+ -> Zl+, thus if b_mu is 1, then it implies Σ+ -> Zmu+ always. The W's and Z's follow SM branching ratio of-course.

Sudarshan Paramesvaran:

Abstract: You don't mention in the abstract that you use kinematic variables as a major discriminating factor, isn't this worth a mention here?

- We shall try to reword the abstract but this statement was removed because of previous ARC comments.

Line 25: Given the definition of b_l earlier in this paragraph, isn't the sum over all b_l = 1 already implied? In other words, aren't you saying there is no additional constraint on the b_l?

- you are right, We just wanted to be explicit.

Line 141: We think you should mention what the dominant uncertainties are on the "rare backgrounds".

- We do mention this on L170, where we discuss systematic uncertainty.

Line 173: In this paragraph its not clear what the numbers are that go into assigning a 30% uncertainty - it might be good to spell these out in a table.

- We were asked to put text based description of uncertainty rather than a table, we shall try to improve the description.

Albert De Roeck:

General - We consider only final states with electrons and muons in this analysis (which is fine). But in the introduction we seem to give the impression we also should look at taus. Are Tau analyses being considered somewhere in complementary analyses in CMS? In some models/paramater has space the triples couple dominantly to taus - Taus are treated supposedly as a background here. I guess we have some efficiency for the tau final states decaying to taus. Is that taken into account in the cross section evaluation/ limit setting?

- We are not aware of any tau analysis directly targeting see saw fermions. Leptonically decaying taus are naturally included in our analysis and the signal cross-section that we consider takes this into account.

Details - line 18: The conditions that we impose here: what phase space is coverage selection does that imply. Are we looking at a particular (small) corner or do we still cover most of what is considered as interesting phase space for this model?**

- we are covering a large chunk of phase space possible by multilepton analysis. It is a major improvement over previous results. This is also a major chunk of the phase space for see-saw type 3, that can be covered at LHC.

- line 53 Are Weyl fermions not supposed to be massles? Maybe add 1/2 sentence explanation what is meant here, that would would be helpful for the reader.**

- There was a mistake. We corrected it.

- line 61 This is very general. How do we take into account the pile-up? What method of corrections was used. For posterity given that Pile-up at the LHC is know to be an experimental challenge and it varies with time (year…) Giving references to the tools used is sufficient.**

- The presence of multiple pp interactions in the same bunch crossing (pileup)is incorporated by simulating additional interactions (both in-time and out-of-time with the collision) with a multiplicity that matches the one observed in data. (This description has been put into the paper with proper reference.)

- line 73: “non-prompt leptons occurring inside or near jets: are these not from heavy flavour decays? What do you have in mind here with *

- We have reworded the sentence and hope that its more clear now.

- line 88: define how is the primary interaction vertex determined **

- the definition with proper references has been added to the paper.

- line 86: how is the pile-up correction done here, also in line 94 **

-Systematic uncertainties due to pileup modeling are taken into account by varying the total inelastic cross section used to calculate the data pileup distributions by ±5%.

- line 118: somewhat unusual notation for the isolation variable

- did you mean some different line , we don't see isolation variable here.

- line 132: Are the WZ, ZZ cross sections at NLO precision (from POWHEG) or higher. *

- Yes they are generated at NLO precision with POWHEG, this is described in the line 55-57.

- line 140: to what precision are these cross sections calculated? Suggest to add in the paper.**

- This is motivated by the latest CMS measurement of ttZ cross-section (~30% uncertainty, and is about 15% lower than SM prediction) which is the

dominant rare SM process. Reference - Measurement of the cross section of top quark pair production in association with a Z boson in pp collisions at 13 TeV, CMS-PAS-TOP-16-009.

- line 162 Is the cut really an “and” of these two conditions? Just checking to make sure.

- yes it is.

- line 163: what is meant with ‘not suitable for modelling conversions’ how should the reader interpreted that?**

- we have reworded the sentence. We hope its more clear now.

- line 165: was a closure test done on MC to show that this method give the right result? **

- We have done closure in data, we cheked the method in ee+γ (eee) events vs μμ+γ(μμe) events.

- line 190 We show only a few example plots in Fig 1 which looks to me very minimal information we give to the reader that we see no statistical excess anywhere, so he just has to believe us here? I opt here for showing more data!**

- We have shown representative plots. We have made that more explicit in the paper by clarifying that the plots with strongest signal are shown and the plots not shown do not have any excesses etc either. This is not an inclusive search paper, but a targeted seesaw paper. We plan to publish a 2016+2017 inclusive paper a la Run-I with detailed tables independent of any signal models.

- line 195: The 5-15% variation due to PDFs and scale indicates for me (unless the largest part comes from the scale uncertainty) that one should be careful just using one PDF set I’d recommend to use the PDF4LHC recommendation that at high mass can lead to a larger and more realistic uncertainty than that derived from using one family. **

- We use these two TWikis from official SUSY xsec calculation, since it is the same underlying SU(2) production-

1) https://twiki.cern.ch/twiki/bin/view/LHCPhysics/SUSYCrossSections13TeVn2x1wino#Envelope_of_CTEQ6_6_and_MSTW2008

2) https://twiki.cern.ch/twiki/bin/view/LHCPhysics/SUSYCrossSections13TeVx1x1wino

They use CTEQ6_6 and MSTW2008 for their calculation and combine them at the end- For each PDF set the scale unc. is XX% of the PDF unc. (see unc. numbers in the tables)

Alberto Zucchetta:

General comments:

Because of the format, there are no sections. An extra effort should be put to improve the linearity of the text: in several places a topic is introduced, and described thoughly only few paragraphs later. Additionally, there are sharp changes that may surprise the unsuspecting reader when moving from one "Section" to another, for instance the CMS detector description, or the simulation paragraph. We suggest to improve the smoothness of the text.**

- we will try to improve the text.

Add some sentences, why this analysis is better than inclusive SUSY searches in the same final state. (How the results compare by just re-interpreting existing SUSY searches?)**

- we shall try to write some comparison.

You derive scale factors for the WZ and ZZ backgrounds in appropriate control regions, but the SF are not compatible with unity: it is 2 sigma-ish away in case of the WZ, but 4 sigma-ish in case of the ZZ background. These processes have been measured very accurately by ATLAS and CMS, and found in good agreement with the SM, so it feels weird to see something that is not compatible. There is also no explanation to motivate this discrepancy.

- the cross-section measurements are typically done in narrowly defined regions ( for eg. strictly on shell z boson requirement ). This is why our measurements are different from those one's.

The description of the event selections for each signal selection is a bit difficult to follow as reader cannot imagine (easily) which final state it mainly targets (no clear motivation is given). Can you add some explanations?

- There are 27 processes and there isnt strict correlation between a signal bin and a particular final state. our broad divisions (OnZ /OffZ, 3l/4l) also arise in multiple processes, addidng a short explanation might not be possible.

There is no information about the signal fit. Are you considering the 8 bins as statistically uncorrelated? From the AN it doen't seem the case. On the other hand, if you are taking into account the bin-by-bin correlations, you should at least write it in the paper. In this case, the search regions would be just 6, and not 48 as you claim.**

We do take into account bin by bin correlations, and this is a simultaneous fit. we shall try to improve our description.

There is no information at all about the search regions not shown in the plots in Fig. 1, so the reader has no clue about what is happening in the other regions. We feel that data needs to be shown. We understand that there are size constraints, but you should add at least the numbers (observed data / estimated background) in each region that you don't show. For instance, you could add this information in Table 1.**

- We are happy to do it, but we think that this will break the linearity of the paper. We will wait for pubcoms opinion on this.

L16: Do you consider also Higgs -> tau tau leptonic decays? It's not mentioned here.

- we include all possible decays off Higgs boson

L17-20: these lines anticipate the simulation of your signal, which should be treated from L50 onwards, once you have described the detector and so on. It sounds weird to see the details of generation in the middle of the introduction.

we do not consider these as details of generation. Rather this a description of the various signal processes to which this analysis sensitive.

L32-35 it's not really necessary once you reference the CMS paper. It also saves space, and it's not really well integrated with the other text

- We have been asked to add these extra details,

L49: if you describe the lepton trigger, there is a crucial information missing: are the leptons isolated?**

- We use isolated dilepton triggers with the requirement that dZ between two trigger legs should be less than 0.2. We have added this detail in brief to paper now.

L58-59: you mention that those two samples are for systematic studies (and systematic is jargon here), but you don't use them anymore in the rest of the paper

- we do mention it line 151 onwards, also have improved the description.

L62-64: It's more appropriate to discuss this in the lepton section. Id and isolation have not been discussed yet.

- we'd like to move according to your response, but we also want to wait for language editor's comments on this.

L73: it's quite confusing: a prompt lepton from W->l nu, but near a jet, is still considered prompt?

- reworded the description.

L91-94: Why do you need a paragraph for jets that you don't use in the analysis? If it's for the MET, you can just add a sentence to the MET paragraph explaining how candidates are clustered.

- Again we'd like to move according to your response, but we also want to wait for language editor's and ARC's comments on this.

L99: No trigger matching applied ?

- we do not apply trigger match to these 3 leptons.

L101: the efficiency quoted here is referred to signal or background? If it's signal, which mass point?

- the efficiency is close to 100% for signal as well as background , this is due to combinatorial possibilities ( any lepton pair can fire the trigger )

L103-105: How much acceptance loss for the signal? For instance, if 1 of 4 leptons is lost, what percentage of events still have the 3 remaining leptons with a mass in the Z mass window?

- Asymmetric internal conversion is a low energy scale problem and our exclusion is at high mass, so we can neglect that loss in efficiency.

L125: It might not be straightforward to understand a OSSF0 topology. Since you provide examples in other cases, you can add one to this one.

- We have provided an example in line 111-113.

L150: number of particles around the lepton -> which particle ?

- PF candidates.

L171: is the 50% uncertainty motivated? The theory uncertainty on the cross section of the backgrounds is much smaller.**

- This is motivated by the latest CMS measurement of ttZ cross-section (~30% uncertainty, and is about 15% lower than SM prediction) which is the dominant rare SM process. Reference - Measurement of the cross section of top quark pair production in association with a Z boson in pp collisions at 13 TeV, CMS-PAS-TOP-16-009.

L196: is the "scale" the factorization/renomalization scales? Is so, it's convenient to explicitly mention it **

- done.

L198-199 : need some explanation why this is the case; is this due to the lower acceptance for the lepton coming from leptonic tau decay?**

- Yes you are right, as we only consider tau leptonic decay channel, we have lower acceptance in this channel. We will try to improve the text.

Table1: no description about the CR-veto in the text. Also, for 3 leptons, OSSF column should be 1 pair, rather than above/below-Z (It should be added as a separate column if you want to say this).

- We have improved the table.

Daniele Del Re:

General comments:

- The introductory part of the paper could benefit from the presence of some example Feynman diagrams to visualize better the production and decay processes, if there is enough space to add pictures here. For readers not familiar with the model more details can be also provided (branching ratio in different channels, other decays not discussed in the paper?, single production? why Sigma^0 Sigma^0 cannot be produced?) **

- considering our target journal addition of more figures would be difficult. We have tried to improve text description. pp->sigma^0sigma^0 is absent at tree level and only arises at higher orders. Hence it is neglected.

- Did you consider to provide also tables with number of data events and background prediction in each analysis category? It would be useful for reinterpretation in other models, since in the end the analysis selection is quite generic. **

- we will try provide data tables as an additional material to the main letter.

- Did you try to reconstruct the invariant mass of the Heavy Fermion particles? Considering you have two of them in each event, you could exploit the kinematics to suppress significantly the background and improve the sensitivity. The reason why this approach was not pursued, in favour of a more generic search, should be discussed in the paper.

- we have explored such an approach briefly earlier, but it was found to be more beneficial to have a generic approach to target all 27 distinct processes.

Specific:

- Abstract L9: “[..] branching fractions to electrons and tau-leptons. [..]” It looks like only electrons and taus branching ratios are relevant, while the point is that the muon branching ratio is related to those two. I would rather write “[…] branching fractions to electrons, muons and taus. […]"

- changed accordingly.

- LL 11-13: It’s not clear here if the leptons can be only electrons or muons as written in line 11 or also taus as in 13. This sentence can be rewritten saying that the interpretation includes electrons, muons or taus while only final states in leptons are explicitly tagged using electrons and muons.

- reworded the sentence.

- L12: provide table with branching ratios in each decay channel for the model considered. Which are the parameter of the model that control these branching ratios?

- See page 9 at https://arxiv.org/pdf/1107.3463.pdf. The only floating parameters are M_sigma and V_alpha, but the last one is only the restriction to the lepton flavor. Since we see, that this is mass dependent, adding a table for one mass is not really helpful. We would add a sentence, that the BR is mass dependent, but we could not add a specific table, again because of short text requirement in PRL.

- L41: consider simplifying to “The two-level trigger system selects a few hundred events per second.” (if you want to be more specific, the main HLT physics streams were targeting a total budget of 1kHz on average over a fill in 2016)

- reworded according to the suggestion.

- L44: you are specifying pT thresholds only for the trigger, while you specify both pT and eta cuts for the offline selection (line 66) ; you could consider specifying the trigger eta cut as well**

- eta cuts are now provided in the text.

- L82: when specifying these isolation cuts, it could be interesting to indicate their efficiency on real leptons and the residual fake rate (this is potentially interesting for other identification or isolation cuts)**

- The muon efficiency is around 95% according to muon pog, for electrons it would be difficult to find efficiency separately since isolation is included in the ID. For the residual rate it would be very difficult to give exact numbers, since residual fake rate is highly dependent on the origin of fakes and varies according to topology.

- Table 1: please define what "CR-veto" means in the caption.

- defined now, also mentioned on L-133.

- L95: this paragraph is a generic definition of MET, but MET is then mentioned only 3 paragraphs after (when describing the L_T + MET variable). It could be clearer to put it right before line 113 and/or to add a sentence in the MET paragraph explaining that it is useful to catch the final state neutrino contribution

- we have moved the definition of missing ET to more appropriate place.

- L101: I suggest to quote efficiency numbers and their error bars, to give a more quantitative statement than “close to 100%”**

- the efficiency is really close to 100% for signal as well as background , this is due to combinatorial possibilities ( any lepton pair can fire the trigger ). Putting numbers like 99. doesn't seem suitable.

- L101: for which trilepton events you get this efficiency? signal?

- yes, signal as well as background.

- L108-109 and 125: not clear how the category OSSF0 is defined. Please, put the definition and an example here.**

- we have now provided an example.

- L112: could you explain why you choose to give this precedence?

- because our main categories to classify 3 lepton events are On-Z, OFF-Z and above Z, you can say that Z mass window acts as a reference for this classification. Thus we choose to give precedence to the pair which is more closer to Z mass window.

- L142-152: was a closure test performed to validate this matrix method? Please describe the result in the paper.

- we have performed closure test for matrix method in both DYand ttbar MC in various regions, these details are listed in our Analysis note.

- L145: please specify what means “universal” here

- by universal we mean that the rates are same across all the signal regions that we define.

- L152: unclear why the average should be taken. Do you refer to the fact that in the signal region you have both backgrounds? Please clarify in the paper.**

- yes that is correct. In the signal region we have both backgrounds. This is mentioned in the introduction from line 27-32. In our background description, the general assumption is that all the mentioned backgrounds are present in all the signal regions. Mentioning particularly here for this particular background, would not go with the flow of paper.

- L159: which dilepton mass is considered here? the closest one to the Z mass as specified at L112 ?

- yes that is right.

- L192: please provide signal efficiencies and background yields for each of the decay channels. Which is the dominant channel in the final results?**

- we will try to provide additional material with the letter, it would be difficult to accommodate since the length restrictions of target journal wouldn't allow it. L3AboveZ is the dominant channel for final results. ( a specific example -:L3AboveZ bin8 is most significant for mass point 820 GeV)

- L199: "limits decrease" means that limits is better. I think you want to say the opposite. Also explain in the paper why limits are worse when branching ratio to taus increases.**

- corrected.

- L207: it could be useful to quote in this summary also the limit obtained in the b_tau=1 scenario as a lower bound

- we have put the lower bound in this scenario, in the summary.

Olaf Behnke:

You write that you use the "CLs method". CLs should be called a criterion or a prescription but not a method because otherwise one gets the false impression that much older basic concepts like p-values were only invented with CLs.

- You are right, we have corrected the sentence.

- Do you use asymptotic formulae from the paper: G. Cowan, K. Cranmer, E. Gross, and O. Vitells, Asymptotic formulae for likelihood-based tests of new physics, EPJC 71 (2011) 1554. ?? In case you do so you should state in the text that you use the asymptotic formulae and cite this paper. If you use the variant based on toy experiments I would also recommend to say this in the text. **

- we have cited this paper now.

- when you run the combine-tool do you use 'log-normal' type uncertainties for any of the nuisance parameters? Or other non-gaussian types of uncertainties? In case you do so this should be stated in the paper so one can reproduce the results.

- We use log normal type uncertainties for systematics where as statistical uncertainties are treated as gamma distribution, we have mentioned this in the paper now.