Guenther

- General: you still use a mixture of "scale factors", "rescale factors", "scaling factor" etc... unifying the notation would be nice, but not a show-stopper

The difference is used intentionally and cleaned up in a comment by Tristan

-L22: full stop missing after "channels" added

-L89: I suggest the following formulation: The cross sections for the W+jets and Z+jets processes are normalized using the predictions for inclusive W and Z production from FEWZ ...... changed

- Line 4 of caption of Table 1: remove "scale factors" after "efficiency". distinction is intended, see comment above

- L205-206: as I said before, it is "impossible" to calculate cross sections for W+2b with FEWZ at NNLO. What you did, as I understand it, is to take the fiducial cross section for W+2b from Madgraph, and then multiply this with a kind of k-factor obtained from the inclusive W cross section. Here I would take the ratio of the FEWZ NNLO incl. W xsec over the incl. W xsec obtained with Madgraph. If you have not done this, then I have a problem....

If you have done this, then you should describe this here.

What we do is effectively as you describe:

For the normalization, we take the FEWZ NNLO inclusive W + jets cross section and apply it to the inclusive W + jets sample (5 flavor - Madgraph 5 + Pythia 6). We then apply the signal region selections ie. exactly 2 CSV Tight tagged jets, exactly one lepton, etc. and in this way have "NNLO applied to a W+bb 5-Flavor sample." This is technique has been noted on L81-83 (in v17), and has been clarified in the next version (v18) at L205-206 to read:

In this analysis, the cross section used was calculated with FEWZ at NNLO
using the five-flavor CTEQ6M PDF set on the inclusive W+Jets sample.

- L206 : .. channels and results from... changed

Anne-Marie

l22 missing "." at end of sentence. added

ref [7] ATLAS and CMS collaborations -> ATLAS collaboration
From ref [10[ on => number is missing, something went wrong with the
formatting.... fixed

Tristan

- In the abstract "W-> munu ... W->enu" has an extra space. For consistency, you can just write it like l.25. changed, now same as l.25 (ie. with space)

- l.16: " the production ... have" -> "the production ... has" changed

- l.22: "channels A full" -> "channels. A full" changed

- l.146: "jet energy scale" -> "JES" (you just defined it in the preceding line) changed

- Concerning the usage of the "(re)scaling factor" terminology, as commented by Guenther below: I think the text is consistent once you replace "scaling" by "rescaling" in lines 178+182+184. agreed, this does help clarify. changed

- l.188: I think you can just write "\sigma" instead of "standard deviations" here, to be consistent with the surrounding text. changed to define (/sigma_JES)

- l.199: In this line, you say the cross-section is called "sigma", whereas in the formula you refer to "sigma_obs" and "sigma_gen". Although it is clear what you mean, this is not consistent. Possible solutions:

took option 1), and clarified the notation in the caption to Table 1

1) Either change the formula (but be aware to avoid confusion between 'standard deviation' and 'cross section', as they are all called sigma) or

2) Change the symbol in l.199 (but then make sure you are still consistent in the rest of the PAS, e.g. the Abstract and the Conclusions)

- Tables 1+2+3: captions of tables should be placed above the table (as opposed to figures). moved

- Table 3: end of sentence missing in the caption. expanded

- l.290: When adding the references to D0 and CDF, the bibliography got broken. Maybe because you wrote "->" instead of "\rightarrow" in the title? Anyway best to copy the LaTeX title directly from e.g. Spires. fixed bibliography (had \rightarrow but wasn't explicitly in math mode, so LaTeX went to automatic math mode and never exited)

/Content, major comments:

you now say in the abstract, and when you define the signal scale factor, that you compare to NNLO (FEWZ).

It is not correct to say that you compare data to an NNLO prediction, simply because there is no NNLO prediction for W+bb production. FEWZ is an inclusive DY prediction, based on light quarks only (if not mistaken). All what you can compare to are NLO predictions for Wbb (MCFM) or matched LO+PS predictions. And this is what you should quote in the abstract.

Also, it doesn't really make sense to define a signal scale factor w.r.t. FEWZ (for the arguments given above).

OK, NNLO was added to the abstract as instructed in the last round of comments, it is now removed. It is the W+jets cross section that is coming from NNLO (MG LO+PS 5F- where the normalization comes from) and the W+bb component is extracted from there.

- l.98-99: There is no description of how the pileup distribution is adjusted to match the data. Either give more details, e.g. copy some usual CMS description, or add a reference.

The standard CMS procedure is used and the standard description is given (see SMP-14-010 s2 last paragraph, or SMP-14-005 s3 last paragraph for example).

Concerning section 4, 3rd paragraph, Tristan already commented on this paragraph a couple of times. It is almost fine with him now, but he has a remaining comment: - l.119-123: The sentence "If an event contains a b jet, from matrix element or parton shower, it falls into the W+bb category" seems inconsistent with "Exactly two b jets are required in the signal region" --> If both sentences concern the signal definition, make this consistent, because currently it is inconsistent --> If the the last sentence concerns the event selection, please drop it, because it was already mentioned before, and it is confusing here

Removed, you're correct this is redundant. L110 already contains the relevant information that the W+bb phase space contains 2 b-TAGGED jets with those kinematic cuts. The purpose of this sentance was to express the idea that the W+bb component of the W+jets sample in the W+bb signal region phase space contains 2 (gen level) b jets, but we agree that the reader already has enough information to figure that out if they like.

- l.123: Why do you require one b jet to define W+bb, but "an even, non-zero, number of charm jets" to define W+cc? Seems inconsistent.

This difference comes from the difference in production mechanism. While W+c can be directly produced via the interaction of a 1st generation quark with a gluon and is detectable by CMS (see figure 1 in http://arxiv.org/pdf/1310.1138v2.pdf), the analogous W+b final state can not and makes a negligible contribution to the signal region. In contrast, W+cc where the cc is produced via gluon splitting has very similar kinematics to the signal region: W+bb. So the lack of distinction between W+b and W+bb at the generator level was chosen so as to be as inclusive as possible while the distinction being made between W+c and W+cc comes from the fact that both final states occur at a nonnegligable rate but arise from different processes.

BTW: It is not clear what are the pT/eta requirements on the charm jets. Please clarify.

The kinematic requirements on charm jets are exactly the same as on b/light jets, ie. pT<25 GeV, |eta|<2.4.

To clarify, we:

a) select exactly 2 central b-tagged jets (pT<25 GeV, |eta|<2.4, CSV Tight, leading jet veto, etc. - ie. select to be in the W+bb signal region phase space)

b) divide the events passing this selection from the W+Jets MC sample into three distinct categories based on the GEN level information (W+bb, W+cc, W+udscg).

- l.128: "...leptons are not considered". Does this concern simulation? This could be made more explicit. Related to this: (how) are such leptons vetoed in data? Not clear here.

This is referring to GEN level leptons, the sentence now reads:

Generated leptons originating in simulation from the decay of b hadrons or tau leptons are not considered.

- Section 5 on Systematics :

Concerning Table 1: if we are not mistaken, this is the first time we see this table, or did we overlook it before?

Table 1 in its present form has been included since v5 and the approval was given on v9.

Anyway, some comments on it: - What is "the W+bb phase space"? This is also referred to as "the signal region phase space" defined by the selections described in Section 4 Paragraph 2.

- What does "signal strength uncertainty" mean? Is this the effect from these systematics on the cross-section measurement? we think this can be made clearer, and maybe mentioned in the caption. Technically the way this is calculated is by removing the given systematic uncertainty from the fit and redoing the fit. Then the number being quoted is the difference in the uncertainty from the full fit vs. the uncertainty on this reduced fit. We shall change the name to "effect on the measured cross section". A line has been added to the capton reading:

The last column indicates the contribution of the given systematic to the overall uncertainty on the measured cross section.

- The "b tag rescale" uncertainty of "14.8%" is a mystery to us. It is not compatible with the numbers "1.01-1.29" mentioned in section 6. All MC samples get rescaled by a multiplicitave factor of 1.148. We put an uncertainty on this rescaling of 100% of the scale factor itself (ie. 1.148 +- 0.148). We were asked to justify why we chose 100%. Looking at the individual channal fits, (mu = 1.17+-0.12 and ele = 1.13 +- 0.11), the highest possible value of the rescaling comes from the muon channel, taking the upper uncertainty (1.17+0.12=1.29). The lowest comes from the electron channel taking the uncertainty down (=1.01 [the reason it isn't 1.13-0.11=1.02 is just rounding]). Therefore the three relevant scales are 1.01(down), 1.15(central), 1.29(up) and it is consistant.

In the table, we now quote : 0.148 (uncertainty) / 1.148 (scale factor) = 12.9%

- The ordering of the caption does not correspond to the ordering of the entries in the table, which was confusing. The ordering has been changed in the caption.

- 'sigma_Id/Iso/Trg' was not defined - Concerning the variables in the Table: except for sigma_JES, we think all variables "sigma_*" are never used in the paper. Not really sure what their purpose in the paper is.

The uncertainties listed as sigma_* are uncertainties affecting both normalization and shape of the MC samples. The two relevant aspects here are a) that they affect different samples differently (for example, different samples see a different change in normalization corresponding to a 1 sigma shift in JES) and b) that the is no good way to quantify numerically differences in shape. Listed under "variation" in the table, these are the uncertainties as input to the fit, but really the important number to quantify is how the given uncertainties actually affect the result of fit, and this is quantified in the last column. A line has been added to the caption to clarify the use of sigmas which reads:

In the "variation" column, the uncertainties which are correlated across all simulated samples and affect both shape and normalization are indicated by $\sigma$ to indicate that an input variation of one standard deviation is set in the fitting procedure.

- section 6 on Results : - l.178: "...imperfect description of the b-tagging efficiency" Concerning this sentence, we have two remarks: -->

See comments below, line changed to:

The fit in the tt-multijet region is used to obtain scaling factors for the muon and electron channels separately to better describe the b-tagging efficiency in the simulation, as presented in Section 4.

(1) You are rescaling for the product of the b-tag SF and the ttbar cross-section. So is it fair to only blame the b-tagging? We think it would be more fair to say you do a data-derived SF for the b-tagged ttbar, which includes both the b-tagging and the predicted cross-section --> (2) In the end, the correction is 1.15±0.14, which is well in agreement with the b-tag SFs. So it might be good to comment on the compatibilty with the b-tag SFs, as the sentence "imperfect description" makes it sound like the b-tag SFs are bad, whereas we think this cannot be significantly concluded from these numbers.

More philosophically, both for JES and b-tag: we still don't really understand why we correct "in situ" the b-tagging and JES, if the POGs provide these numbers centrally (after a lot of dedicated work). It's a bit of a "search approach", but we don't see the benefit, also since the final numbers found in the analysis actually appear to be in agreement with the numbers provided by CMS anyway.

We understand that this comment comes late (since after approval), and that this has been your strategy since the beginning, but we think it would be a useful improvement of the PAS if it would be explicitly described (i) that these corrections do not only correct for the JES and b-tag, but also for the normalization of the ttbar cross-section in these control regions (ii) why the existing POG recommendations might not be optimal for this analysis (iii) that the difference of these corrections with 1 are only 1-2 sigma, so not really significant.

Then we think the used procedure would be better motivated for an outside reader. To be clear, we are not asking for redoing the analysis, only for a few sentences of clarification

This is why we gave this presentation to the BTV POG. To restate the conclusions that we presented then, and which were approved by the BTV POG.

1. The established tt measurement sees good agreement in their phase space (1 CSVM)

a) we see good agreement in this phase space as well with all recommended BTV POG SFs applied [type 1c]

2. As we move isotropically to our phase space disagreements mount.

a) We use lower pT jets than most other analyses

b) We see normalization differences of ~2% per b-tagged jet using the CSVM tagger

c) We see normalization differences of ~5% per b-tagged jet using the CSVT tagger (again, after applying all BTV POG SFs)

These lead us to the conclusion that it is not a question of rescaling the ttbar cross section (after having initially scaled it to the CMS/ATLAS combined measured result), but really a retuning of the b-tag SFs that is necessary (in our phase space only! We agree that the POGs have spent a lot of effort coming up with their recommendations and that they work for most analyses. It is just in our particular case of having two tight tags on low pT jets that we see rescaling as needed). This is also why we use the term rescaling instead of scaling.

We suggest modifying the sentence (see above), but dont think it will be more clear for outside readers if we go into the details of rescaling the MC wrt the given scaling factors from the POG. We have had our proceedure for rescaling blessed by the stastics committee and the need for a rescaling blessed by the BTV POG and in doing so have proven that our procedure does not demand a change the ttbar cross section but just a retuning of the simulation we use.

- l.188: Does the correction of "1.6 standard deviations" only change the shape, or also the normalization? In case it affects the normalization, can this be explicitly mentioned? And if so, is it compatible with the correction "1.15±0.14" in the other control region? Because otherwise it could just be attributed to the modeling of ttbar in this phasespace.

This correction in principle affects both shape and normailzation, but in practice only the normailzation[1]. It was our response to exactly this question that lead us to our 3-step fitting procedure. Because we could not use shapes to distinguish between the two uncertainties here[2] we instead used physics arguments to separate their contributions and found a control region (tt-multijet) which has contributions coming from essentially only b-tagging[3] and a control region (tt-multilepton) with contributions from b-tagging and JES. The physics arguments (jet veto) are not the only evidence we have justifying isolating the two in this way, but the validity of this separation into two control regions is also confirmed when looking at the distrubutions themselves before doing any of our rescaling (still applying the BTV SFs of course). We see "experimentally", ie. looking at the MC, that there is an uncertainty band due to b-tagging only in the tt-multijet region, and an uncertainty band due to b-tagging and JES in the tt-multilepton region.

Footnotes for clarity

[1] see AN Figures 19-21, the shape differences between JES, b-tag and unshifted are essentially nothing

[2] and these were the only two uncertainties that had any real affect on the normalizations of samples

[3] and the tt cross section, which we include in the fit, but also showed in the study linked above not to be the cause of the disagreement

- l.208: "by varying the PDF set and scale" is a bit vague, some more details could be added.

Changed to:

by varying the PDF set using the LHAPDF/PDF4LHC prescription considering PDF sets from CTEQ, MSTW, NNPDF, and HERA as well as varying the choice of scales $\mu_{\mathrm{F}}$, $\mu_{\mathrm{R}}$ simultaneously up and down by a factor of two.

- l.216: "NNLO PDF set" might be confusing, since you use this for an NLO prediction (MCFM is NLO) removed "NNLO"

Same comment for L219-220: do you really use a NNLO PDF when running Pythia?

No, we use NNLO23_lo_as_0130_qed as the PDF but this was deemed too technical of a name so was reduced to "a NNLO PDF set" and refers to the PDF set used by MadGraph. Pythia is run using the CUETP8M1 Tune. Expanding "with" and "and" for clarification, these sentences have been changed to read:

In the five-flavour scheme the PDF set {CTEQ6L} was used and \PYTHIA6 was run using {TuneZ2*}. The two four-flavour samples were produced using a NNLO PDF set interfaced with \PYTHIA (version 6 in one sample, version 8 in the other) in the {CUETP8M1} tune.

- Fig.4: There is no description of the figure, and it would be particularly interesting to describe the error bars of the various productions. The MG5F prediction has a small error bar, and only 1 (because there is no hadronization and DPS correction?) so maybe describe explicitly.

There is a description given on L238-241 and the caption has been expanded, adding the lines:

The blue error bars on the predictions represent the uncertainty in the given sample associated with PDF choice and the black bars represent the total uncertainty. In the case of the \MADGRAPH + \PYTHIA6 (5F) sample, the effects of DPS are already included in the generated sample so the extra DPS factor was not needed and the blue and black error bars overlap perfectly.

Text/editing comments (note: this is a compilation of comments received by the various ARC members)

-l12-13: ... throughout this note, we ... done

- l.16-18: There is no reference to Tevatron W+bb results, I think they should be added. (Few weeks ago I still heard someone discuss the Tevatron W+bb measurements.) Find below the ones I quickly found from references in an Atlas paper(*), please check.

(*) - CDF Collaboration, T. Aaltonen et al, First Measurement of the b-jet Cross Section in Events with a W Boson in pp¯ Collisions at â^s s = 1.96 TeV, Phys. Rev. Lett. 104 (2010) 131801 [arXiv:0909.1505]. - D0 Collaboration, V. Abazov et al, Measurement of the pp¯ â?' W+b+X production cross section at â^s s =1.96 TeV [arXiv:1210.0627].

added

- l.20: I think the sentence "A full description ... found elswhere" is completely misplaced here. I would put it in Section 2.

Moved to last sentence in introduction.

- l.21-22: This sentence is a bit an anti-climax. I would propose instead to write "Whereas the previous analysis used only the muon decay channel, this analysis uses both the muon and electron W decay channels." This is a bit more optimistic way to end the Introduction.

or alternatively:

l20-22: It look to me the text should be slightly re-ordered as "...[10]. The analysis uses both muon and electron W decay channels, whereas the previous analysis used only the muon channel. A full description of the CMS detector can be found elsewhere [11]"went with combination of options

- l.24: Notation of italic "W" is not consistent with other occurences. changed

- l.25+26+... The spelling "24(27)" and "2.1(2.5)" is not good looking, and not consistent with the notation "0.20 (0.15)" in line 135. It would be better to add some space ("~") inbetween. There were some other places where spaces were not okay. But well, I'll not become too pedantic here, and leave that to the LE. changed to have spaces on L 38, 41, 110, (135 already ok), 206

- eq. 1: the [ ] brackets are not really needed here removed

- l.46: as the angle between pT and MET is discussed, I think it would be more consistent to write something like \vec{pT}. changed

- l.51: "kT"->"kt" (as defined in the title of the paper) changed

l54: "criteria that eliminates clusters originating from"

not changed. Jets are required to pass more than one criterion hence the plural criteria. Criteria eliminate, a criterion eliminates.

-l56: "small corrections to jet energy, relative and absolute calibrations" sounds a bit confusing/unclear. Do you mean: "small corrections to the relative and absolute jet energy calibrations... " ? yes, changed

l65 "..data and simulation take into account dependencies.." changed

- l.73: "W or Z boson plus jets" -> "V+jets" (as just before defined in l.69) changed

- l. 98-100: I would swap those two sentences for a more logical ordering switched

l124: "The remainder events falls into.."

changed to:

The remaining events fall into

-l137-138: "... set to fill the difference..." sounds awkward. Why not something like: "The QCD normalization is adjusted in order to describe the number of data events at MT<20 GeV, after subtracting the non-QCD backgrounds obtained from simulation." that sounds much better, changed

l150-151: "the W+bb LO simulated sample in all regions and may be... has no additional jets to veto." changed

l169: on the QCD ... changed

general (it appears very often, please search for all occurrences ) : "rescale" or "scaling factor" or "rescaling factor" sounds awkward. Why not simply "scale factor" ? see comment above. From the box, we scale, then we rescale

l174: "..the change in acceptance found.." changed

-l182 : .. 129, and with the bounds given ... changed

-l185: change ttbar to the standard notation changed

l196-198: the sentence is not very clear. I would suggest something more simple: "The composition of the event sample in the signal region is summarized in Table 2." changed

I will also move part of the caption of Table 2 to the main text. Following the previous sentence: "Events coming from the production of a Higgs boson in association with a vector boson constitute a negligible fraction of the overall event yield, and it has not been considered here." moved

- l.199 "the cross section for the process sigma(W+bb)" is a pleonasm. Also: use mathrm for the definition of the process. Same comment for the top of table 3 changed

- L204 and 206: "In this analysis" appears twice... removed second occurance

- l.205: "FEWZ at NNLO" is confusing and actually not correct, see the comments at the beginning

"FEWZ at NNLO" is coming from https://twiki.cern.ch/twiki/bin/viewauth/CMS/StandardModelCrossSectionsat8TeV where we apply the W+Jets cross section to the W+Jets sample (5F) and then extract out the W+bb component.

l206-207: ".. muon (electron) channels results from the combined effect of the efficiency from lepton identification requirements (80%), and b tag efficiency (40%)." changed

-l216: comma before "as well" added

- l.219: "Pythia6/8" is a bit vague changed to read:

The two four-flavour samples were produced using a NNLO PDF set interfaced with PYTHIA (version 6 in one sample, version 8 in the other) in the CUETP8M1 tune.

l225: "...list, such..." now reads:

The correction factor is obtained for jets computed excluding neutrinos from the particle list, as such jets are closer in kinematics to particle jets at the detector level.

- l.230: Instead of "the data" I would write explicitly "the W+jets data", because that's where DPS was measured added

-l231 : ...of a W boson produced .... added

- l.234: "their value" -> "the value" changed

-l242 : measurementS rather than measurement ? changed

-l242: should we say "well described" ? sure, added

-l252: .. as well as with the NLO QCD prediction obtained with MCFM, once hadronization... changed

l255: "first one" instead of "only one" ?? changed

- l.255: "for the W+bb process" changed

Figures caption: In Fig 1 and Fig 2 i will change: "... right. The highest bin..." I guess this is also true for Fig 3 but it has not been included. So, i will do. changed, added to third caption

Also, caption Fig. 2: ...after fitting the scale factor for the jet energy calibration (or something along these lines).

not changed, the first sentence reads: "...after fitting to find the appropriate jet energy scale."

caption Table 4: ... section and various QCD predictions. changed

Would it be possible to eliminate the big empty space below Figure 4? it is now smaller

(Received 2015 November 9/10 )

Fabio Cossutti

Type A

l. 59: “V+jets” -> associated production of a vector boson and jets (henceforth V+jets) changed

l. 60 “except QCD” -> except for QCD ? changed

Type B

Abstract and Conclusions: the meaning of italics B in the result expression is unclear, either it is explained or the symbol would be better removed "x B" removed

l. 28 ref 13, usually PFT-10-001 is used instead reference changed

l. 48: why a PAS is quoted instead of the b-tagging paper?

BTV-13-001 is listed in CADI as "PAS-only-PUB" and I did not find a better public result to quote. Is there one?

l. 60: this statement is only partially true, as ttbar simulation is corrected from data control sample. It would be better to add a statement specifying this point.

no change: I don’t understand this comment. Is it referring to the fact that TTbar cross section is taken from data? This is pointed out in L.82 (PASv8)

l. 64: ref 29 does not explain Z2*, but Z1, Z2* is derived from that. reference added

l. 73: references fro POWHEG do not seem correct, please follow instructions provided in http://powhegbox.mib.infn.it/ <http://powhegbox.mib.infn.it/>; (3 general references plus the appropriate one for the process).
Are you sure about the PDF used here?

references added, the PDF (CTEQ6M) is correct

l. 78: are you using FEWZ 2, as you write, or 3? In the former case why? The the latter please update the reference

We are using FEWZ 3.1, reference added

l. 107 - 108: this statement is already made at the beginning of the section, it does not need to be repeated again.

The restatement has been removed, and the line about uncertainties has been moved to appear just after the discussion of shapes and normalizations on MC samples.

l. 129: this sentence sounds obscure, could you please try to formulate it in a more explicit way? “which “each region”?

This has been changed to read:

The QCD normalization is set to fill the difference between
the number of of data events at MT<20 GeV
and the number of events coming from simulated backgrounds
in the same region.

l. 159: the b-tagging rescaling is claimed to change by 14.8% in the table, in the text a 100% change is mentioned, could you please clarify?

The scale factor is 14.8% and the uncertainty on this is 100% of the scale factor = 14.8%. This has been changed to read:

The common b-tagging and JES rescaling factor uncertainties
are set to +-100% of the factor itself

l. 200: it is unclear from the text whether you speak of Madgraph5 + pythia8 or just pythia8 standalone, Please rephrase in a slightly more clear way

MG5+P8 has been made more explicit

l. 201: Z2 or Z2*? Z2*, fixed

l. 207 jets which do not include neutrinos -> jets computed excluding neutrinos from the particle list changed

l. 208: why the 4F Madgraph does not include DPS? Is this a purely parton level calculation? Normally Pythia6 should add this

The WBB 4F sample we mention is by definition already "W+BB", directly produced in Madgraph, before Pythia intervenes.

So, while DPS is indeed switched on in Pythia6 (or 8), the only thing Pythia can do in showering is add an additional BB pair to the already existing one. In other words, in the 4F sample Pythia gives us "W+BB from Madgraph" + "additional BB pair from Pythia from MPI". That contribution is there, both in the 5F and 4F samples.

But, this is subdominant with respect to the main DPS contribution that we are trying to address, which is "W production in MG (without Bs)" + "BB pair from Pythia". This particular contribution is simply not possible to have in a sample that directly has the "W+BB" produced in Madgraph.

l. 216: which prescription are you referring to? Each PDF has its own, are you perhaps meaning PDF4LHC? Is the PDF choice the only considered sure of theoretical uncertainty?

added references for PDF4LHC

PDF and scale are considered in the theoretical uncertainty

Konstantinos Theofilatos

abstract: shouldn’t be x BR instead of x B ? please check latex "x B" removed
abstract: with exactly two b-tagged jets with … -> with exactly two b-tagged jets having … changed
abstract: agrees with standard model predictions at next-to-next-to-leading-order changed

L12: In this note, -> Throughout this summary note, [to avoid repetition of In this also in L10] changed
L37: suppress comma in "QCD multijet events, that have" not changed

The statement is: "MT is a natural discriminator against non-W states that have a lepton candidate..." and the phrase "such as QCD multijet events" is injected into this sentence and therefore bounded by commas as:

"MT is a natural discriminator against non-W states, such as QCD multijet events, that have a lepton..."

L39: suppress ad-hoc changed

L78: check is there is an additional latex space next to the W there wasn't one, but now it looks better

L126: do you have an upper limit on the offline isolation to be safe away from the on-line trigger isolation that’s irreversible ? if not why not? There is no upper bound when inverting isolation. The same triggers are required in MC and Data so the (inverted) isolation when calculating QCD has the same bounds in both.

L126-129: inverting the isolation is known to affect the QCD shape. Is there any systematic uncertainty accounting for this ? please describe the effect and discuss it in the text Performing a study using a BCtoE QCD MC sample, we see that in our particular phase space, in the mT variable, inverting isolation does not significantly affect the shape

modified to read:

In the fiducial regions used in this analysis, minimal correlation
is observed between $I^{\text{rel}}$ and $\MT$, validating
the use of an inverted isolation requirement to obtain
the QCD shape.

L136: is there any uncertainty assigned to account for extrapolating between the different jet multiplicities, from tt-multijet control region to the analysis selection ? please explain An uncertainty of 100% of the scaling derived from the tt-multijet fit is included. This uncertainty covers the extrapolation between jet multiplicities

L191: quote the systematic uncertainty on the A x eff and how this was derived

This is 10% as listed in the bottom row of Table 1. It is derived by varying the PDF set and scale and recalculating the number of events in acceptance.

L204: is the only statistical uncertainty quoted for 0.81 +/- 0.07, please clarify on the text

Added to text:

The uncertainty reflects both the statistics of the
\MADGRAPH+\PYTHIA6 sample as well as a comparison with the
\MADGRAPH+\PYTHIA8 sample.

L213: conservatively estimated ? if this is really estimated please explain how, if this was just assigned without any sophisticated procedure to quantify perhaps mention conservatively assigned instead of estimated changed

L233-235: not sure I understand what’s the message you want to convey here. Is it that ATLAS doesn’t have a Wbb measurement ? Also, how two measurements at different sqrt(s) could be in agreement ? It is more that the conclusions of the theory/data comparison of the 7/8 TeV analyses are in agreement.

modified to read:

The previous CMS measurement at 7TeV
also shows agreement with the SM prediction
within the level of one standard deviation.
This measurement is the only one
performed in the W+(2)b jet final state at 8TeV.

(Received 2015 November 5 )

Abstract
- "pb ," -> "pb," changed

Introduction
- l.4: "Higgs" -> "Higgs boson" changed
- l.5: "boson" -> "vector boson" changed

Selection
- l.56: "dependAncies" changed
- l.56: "transverse momentum" -> "transverse momentum and pseudorapidity"

Kept as is since the tight CSV working point only has one bin in eta.

Strategy
- l.100-103: "events with a third jet are rejected .... control region
obtained requiring a third jet in the event".
These two sentences sound contradictory. Maybe reverse the order of
these two statements to make it more logical?

kept the order, but expanded the sentances to read:

The tt-multijet control region is obtained using the same selection criteria as in the signal region, but requiring a third jet in the event with pT > 25 GeV and |η| < 2.4 instead of vetoing events which have one. The tt-multilepton control also region uses similar selection criteria as the signal region, but changing the lepton requirement from vetoing events which contain a second lepton, to requiring two isolated leptons of different flavor, both with pT > 30 GeV and |η| < 2.1.

- In this version, the definition of the W+bb signal (109-111) and the
gen-level b jet (119-121) are at least together in one paragraph,
which makes the signal definition clearer. Moving these two
definitions even closer to each other, for example directly after each
other, would be even clearer. (Just a suggestion.) changed

Systematics
- l.153: "and and" changed

Results
- Table 2: Maybe mention the expected W(lv)H(bb) background yield?
Even though the SM H->bb decay is still hypothetical, it might soon be
a proven background for this measurement. In the second Z+b-jets paper
at 7TeV we explicitly mentioned such a possible ZH background. (Not in
the table, but in the text.) In the Z+bb final state, the yields
ZH~VV/5, so in your case, I expect the WH might possibly be close to
your W+cc background.

At 7 TeV this was measured to be almost exactly 0, but we are now running the sample to check for a contribution. The caption has been modified to include:

Events coming from the production of a Higgs boson in association
with a vector boson constitute a negligible fraction of the
overall event yield.

- l.183-184: "can be " -> is changed
- l.189: "The in this analysis" changed
- l.189: It is stated that the acc*eff is 11-13%, and "largely due to
b tagging requirements". But in line 54, it is written that the b-tag
efficiency is 50%. Now, (50%)^2=25%, which is still more than a factor
two away from the observed inefficiency, so there must be another
important contribution. Can this be clarified? (Preferably directly in
the PAS.)

The 50% btag efficiency we quoted was the number taken from the BTV group . But this number came from studies performed on TTbar events at higher pT than used our phase space and the BTV group also has noted pT-dependence in the efficiency, shown for medium CSV. We then re-estimate our btag efficiency at 40% (per b tag) and have a lepton efficiency (trigger*id*isolation) of 80%. In total this gives: 0.4 x 0.4 x 0.8 = 12.8%.

In the PAS, the 50% CSVT efficiency was changed to 40% and the description of acceptance x efficiency reads:

In this analysis, the product $A\cdot\epsilon$ is 11(13)\% in the
muon (electron) channels and comes from the 40\% efficiency stated
above per b tag, and an 80\% efficiency in the lepton requirements.

Conclusions
l.233: "the only one performed in this phase space" -> "the only one
performed in the W+(2)b-jets final state" ? changed

(Received 2015 October 28 )

Type A

Type A:

-L15: as as V ok

-L19: of the W+bb ok

-L25 : with transverse momentum pT and pseudo-rapidity |eta|. ok

- eq 1: all sub-and superscripts that are not variables should be
in mathrm ok
- eq 2: use \cos ok

- L38 I realize only now that your reference [15] is a bit old. There is a new one from this year that explains the corrections for PU effects developed for 8TeV data. If you are using then, that i guess yes, I will suggest adding after Ref. [15] the following sentence and reference:

"--- Ref. [15]. Ad-hoc corrections to mitigate the effect of the pile-up are also included [15´]."

Where [15´] is: Performance of the CMS missing transverse energy reconstruction in pp data at sqrt(s)=8 TeV. arXiv:1411.0511 .. pub in JINST 10(2015) added

- L51 b-tag -> b-tagging ok

- L60-61 "merging scheme ... matching scale ..." merging and matching are different things, merging is for assembling different orders (i.e. +1,2,3,4 jets and/or NLO of each of these...), matching is for
matrix element vs parton shower. Here I guess it should actually be "matching scheme ... matching scale ..." ?

ok

- L65: Therefore, in this analysis the ... ok

-L87: in the signal region ok

L89 I think "first" always calls for "second"...Also first is repeated again without a "second" L91. Suggest to remove both. ok

- L105 "... the W+jets simulated sample into .." ok

-L113: is less than ok

-L116: as the number line removed

- L113-116: I would shorter the sentence as :
"The QCD normalization is obtained from the number of data events at Mt<20 GeV for each region, after subtraction of all the other background contributions."
line changed to this

- L117 "There are two..." -> "Two ... affect significantly ..."
-L117: affect the shape ok

- L118: "the b-tagging efficiency", or "and jet energy scale", i.e. both "the" or none... ok, both

- L124-126: slightly modified sentence:
"The same is true for the W+bb simulated sample in all the regions; since the LO W+bb process has no additional jets to veto is therefore less sensitive to JES variations." ok

-L128: in the tt-multijet ok

- L136 uncertainties (you should run a spell-checker to catch these) spell checker run

-L138: only the normalization of the respective contributions, e.g., the uncertainty. ok

-L139 : corresponding ok

- L140 distributions (should run a spell-checker...) ok
-L145 : allowing the fit ok

-L151: related to the imperfect ok

- L156: ... on the scaling ok

-L158: As a result, the simulated ok

- caption Fig. 3: W+bb signal region after ok

- Table 2: change the notation, do not use TTbar, Single T, Drell ok

-L182: b hadron within a cone ok

-L193: The multiplicative hadronization correction factor ok

-L199 : that include ok

l.6-7: Twice 'measured'. Proposal: remove the first one, and add a
comma before the other occurrence.

unclear which lines - change made in abstract

1.Introduction
- l.4: drop abbreviation 'SM'. Not used ok
- l.12: "note"? unclear what change is requested
- l.15: "as as" ok
- l.15: The definition of the non-italic 'V' is not consistent with
later italic occurences, e.g. lines 55+62

kept non-italic 'V' changed in later occurances

2.Selection & reconstruction
- There is no description of the CMS detector? It is a bit unorthodox.
It is fine with me, but there should be at least one line, with a
reference, to the description of the CMS detector (I didn't see it).

reference added

- Equations 1+2: 'max' and 'cos' are italic, should be e.g. "\cos{}". ok

3.Samples
- Although this is more a comment for the paper, I think we are not
supposed to say "Monte Carlo" when we mean "simulated" (as the first
refers to the event generation technique, which is usually not the
thing we are actually describing). Also e.g. line 130+174.

good point, "MC" removed
- l.62: italic 'V', see my previous comment in the Introduction.

ok

- l.64+67: "W+jets" inconsistently written with/without spaces. Please
homogenize.

ok
4.Strategy
- l.87: "in signal" -> "in the signal" ok
- l.116: "as number" -> "as the number" line changed
- l.117: "effect" -> â?oaffectâ? line changed

5.Systematics

- l.136: "unseratainties" ok
- l.139: "correpondent" changed (though correspondent can be used as an adjective in this way, at least in non-scientific English)
- l.145: "fit" -> "the fit" ok
- l.147: "two and" -> "two, and" ok

6. Results
- l.158: "variation" -> "the/this variation" removed
- l.162: "uncertainty" -> "the uncertainty" ok
- l.165: "backgound" ok
- l.172: There is a space missing between A and epsilon ok

7. Conclusions
- l.218: "for fiducial" -> "for the fiducial" ok

Type B:

-L73: with the predictions for inclusive W and Z production from FEWZ ok

-L106: you should define here (and not only at the very end of the PAS) what a b-jet at MC level is ok

- L141: log normal sounds very statistics jargon: is there a better way to say this, like "following a log-normal distribution ... and a quadratic distribution...." ok

-L148: you should be more precise here regarding the PDF uncertainty,> ie. do you use the standard prescription?

more info added, yes this is the standard prescription

-Table 1: what is UES, EES, MES ??
Please include in the Table caption the definition of all of them. They have been never mentioned in the text. ok

Also related with Table 1, it is unclear how the rescaling discussed in lines 150-156 is included in the final systematic.
From the table one understand only the 14,8% (coming from POG, right?) is considered. Could you please clarify this in the table caption or in section 5.?? more has been added to the caption, to clarify:

14,8% does not come from POG, that is the uncertainty on our rescaling factor and this is what is included in the final fit. Our procedure from the b tag side is as follows:

a. apply the BTV recommendations for SFs and uncertainties

b. Fit in tt-multijet and see that ON TOP OF the BTV SFs, we additionally need a rescale factor of 1.15+-0.15

c. rescale ALL MC samples (aka all simulation that isn't QCD) by this factor

After c. we no longer explicitly use the uncertainties from the BTV, but take the uncertainty from the fit (i.e. 14.8%) as the uncertainty on the reSF and include this in the fits.

- L160: "standard deviations" here is effectively the sigma_JES introduced L158, right ? I am confused now: if the best fit gives 1.6 sigma, why shift by only 1 sigma ??
-L162: so this means you shift by hand the JES up by 1 sigma? (the fit indicates a higher value than this)

we shift by 1.6 sigma_JES, as the fit indicates - the text has been modified

-L174: when defining the reference sigma_gen, you have to much more specific: which MC exactly?
The 4F or 5F variant, which settings etcetc

added, this information is also present in section 3

- somewhere you should indicate what the values of A and \epsilon are, for the readerâ?Ts interest.
And specify why you would have an acceptance, even with fiducial cuts
( as discussed in the meeting)

added

- L220: you say â?oagreesâ?, but I would be a bit more specific, since the agreement is not perfect
Also, it seems only CMS has results for W+bb at 7 and 8 TeV. A bit of overall discussion will be welcome.

ok

- l.105-109: to me, the definition of what is W+bb signal is still not
clear from the PAS. Are these b's defined at gen level in the full
acceptance? From the text it seems so, but it is not specified. (NB: I
already commented on this in the previous version.)
Furthermore, I think it is a poor choice to define "W+bb" if the b
jets are outside of the acceptance, e.g. at |eta|>2.5, or very soft.
This leads to inconsistencies when correcting for reconstruction
efficiencies, and enhance the sensitivity to the modelling of extra
radiation.
==> So:
1) I think the definition of the signal should be clarified better in
the PAS. This for sure.
2) Furthermore, strictly speaking, I think it would be much nicer to
define the W+bb signal based only on the b jets that are actually in
the acceptance. Since we cannot reconstruct b's outside of the
acceptance, I think it is inconsistent to include them in the signal
definition.
For the paper, I would at least like to have an estimate of how much
this definition affects the signal cross section. But anyway I would
just prefer to see it fixed - it is not much work.
This might sound pedantic, but it can impact the final data/theory
cross-section comparison.

At the generator level, we DO place cuts on acceptance as indicated in PASv5 line 183. This has been moved to earlier in the text (~line 117) so is hopefully clearer.

- l.125: "at LO, W+bb has no additional jets". This is true, but
Nature is not only Leading Order. So I think this sentence should be
rephrased.

changed to:

The same is observed to be true for the $\Wbb$ sample
in all regions and may be due in part to the fact that
$\Wbb$ has no additional jets to veto at LO.

- l.119-120: "equally sensitive to the b-tagging efficiency". This is
true only approximately, as the b-tag efficiency is pT dependent. This
should be clarified in the text if this is actually considered.
In general, I was missing comments on the pT dependence (and
corresponding corrections) of the b-tag efficiencies.

added line to comment on pT dependence at the end of section 2, changed to:

Both control and signal regions show similar
sensitivity to the b-tagging efficiency,
and its adjustment affects all the regions
in a correlated manner.

- l.154-155: "the uncertainty is enlarged to allow the fits to vary
the scaling factor between 1.0 and 1.3" --> This is a circular
argument and does not sound like a good motivation to me.
Furthermore, I think that blowing up the uncertainties unnecessarily
impacts the final uncertainty of this measurement (especially given
how close the central values of the electron an muon samples are), so
it should be better motivated.

changed to:

The measured rescaling factors, $1.17 \pm 0.12$ (muon channel) and
$1.13 \pm 0.11$ (electron channel), are averaged to $1.15 \pm 0.14$, where
the uncertainty allows the following fits to
vary the scaling factor between 1.01 and 1.29;
the bounds given by the fits in the individual channels.

- l.172: Are the efficiencies, used for the 'unfolding', scaled to the
ones measured in data? It is not clear from the PAS.
Furthermore, these SFs should be applied as a function of pT (which is
why the BTV puts a lot of work in doing so). Is this done? It is not
clear from the text. Note that this can also impact directly the
measurement.

Yes, for everything they correspond to the tuned MC based on the data, including all possible pt, eta, phi dependance. It is because of these correlations that we choose to use the signal strength as our measure.

- l.175-186: Is this description of the cross section consistent with
the gen-level definition of W+bb I commented on somewhere in the above
text? I think it would be best to have a consistent definition of
what's a b jet and what's W+bb. Not clear from the text around l.105
if it is really consistent.

this section has been moved, and is hopefully clearer now

- l.210 "The results agree" is vague, some people require 1 sigma,
others are happy with 2.5 sigma. So I would like to explicitly hear
"agree within xx sigma".

Conclusion extended

(Received 2015 October 7 )

From the Meeting / Gunther

Comments

ll.63-69 improve description done

l.71 S10 scheme etc - to explain or to use standard CMS sentence done

l.78 to explain parameters of the jet veto, put in the AN information about performance of the central-jet veto done

l.87 what are the uncertainties of the cross sections used done

l.97 ID should be removed and explained in words done

l.102 add information about amount of the QCD background done

l.106-116 explain better why regions are used in this way, more clear to explain the veto , multijets etc . Add few lines on correlation between regions. done

l.123 to be more precise - acceptance - > efficiency need to explain 50% uncertainty on QCD done

"plus shape" - need to be described in the text, the same is for scale ... done

put all uncertainties in the table, describe better in the text done

l.129 +- 100% explain that why done

l.133 how the shapes vary - how we fit should be better described done

table 1 should include all uncertainties used for the final fit done

figures - clarify the bands uncertainty signal strength - to which cross section? done

l.138 little acceptance issue done

Table 3 - need to describe how combination is done done

l.157 hadronization 0.81 explain why it is small, refer to similar number from Zb done

l.158-159 uncertainty on hadronization corrections - describe done

l.165 DPS 100% uncertainty - describe why done

l.167 explain better done

add conclusions done

Anne-Marie

Comments

l13-15 add also ref to CMS 8 TeV Zbb PAS SMP-14-010 ? done

l19 the former one [8] used only muons. done

l22 is electron trigger not up to eta=2.4 ?? the trigger is up to 2.5

l26+3 add ref for electron paper (JINST 10 (2015) P06005, http://arxiv.org/abs/1502.02701) done

l39-40 cut in two sentences: ...[17]. Those which... done

l48 give b-tag efficiency and mistag for c and light rates for this working point (as given in btag paper ref). Ref[19] should be to published paper. done

l51 ref[20] is not used anywhere here, we don't have MG4 samples anywhere I think... remove. done

l54-55 refs 24 and 25 are for Z+b ... relevant ???? removed

l56 ref 28 seems quite old, 2002 ! Use refs [14-16] (just 16 or all I don't know...) from SMP-14-010 instead ?? updated POWHEG references

l58 Add a paragraph introducing the main background processes to your signal... done

l59 ref [30] is a duplicate of [21] done

l63 line too long. done

l65-66 is not logical: for LO you give PDF version and for NLO Pythia tune :/ Should give both for both... done

l71-72 remove sentence "The S10 scheme..." too detailed. done

l73 re-weighted to the pileup distribution for data -> reweighted to match the pileup distribution of the data. done

l75-77 rephrase along the lines of: "The dominant background arise from the ttbar process. Data and MC are hence first compared in two ttbar-dominated control regions. MC scale factors are extracted and applied to the signal region. Finally a likelihood fit (++add details or ref to method used?++) to the MT distribution is performed in order to extract the W+bb cross section." done

l79-80 replace "falling within the pseudorapidity range" by "and" done

l80 loosely isolated: give isolation value. Jets: give precision on what pT,eta are considered for jets... done

l82 requiring a third jet: it is not clear whether it is then exactly 3 jets or at least 3 jets.... => requiring at least three jets. done

l87 correspondent -> corresponding done

l96-97 pdgId=5 and 4 refer to quarks not hadrons... and is MC jargon... also "the presence...in one of the jets" is highly unclear: parton jet ? hadron jet ? reco jet ?? Is there a dR matching involved ? I think the following would be more accurate (but correct me if I am wrong): "If an event contains a b quark, from matrix element or parton shower, it falls into the W+bbbar category. If an event contains no b quarks but an even, non-zero, number of c quarks, again from matrix element or parton shower, it falls into the W+cc category. The remainder falls into the W+udscg category." done

l101 obtained shapes -> shapes obtained ? since there is a ? I will explain my meaning: I am using 'shapes' as the object of the sentence with 'obtained' as the adjective referring to the procedure described in the preceding sentences. Other options might be 'shapes thus obtained' or 'shapes obtained as such'

l102 give the percentage of such backgrounds for the inverted isolation: it is important to know whether it's small enough or could be source of uncertainties... Same l105 for the non-inverted isolation, give percentage of background, i.e.. if I understand correctly you have 4 numbers in the mT<20 GeV region: isolation data-noniso = QCD-noniso + BKG-noniso and data-iso = QCD-iso + bkg-iso, and you want to set QCD-noniso = QCD-iso .... I'd like to have in the text the fractions BKG-noniso/data-noniso and bkg-iso/data-iso with their stat uncertainties... done

l106-107 needs rephrasing, it sounds weird to have to adjust uncertainties .... What about explaining why b-tagging efficiency and jet energy scale would not be properly corrected for already, or say explicitly that at the same time as extracting the final cross section the fit is sensitive to b-tagging efficiency and jet energy scale. It would be worth mentioning in the text here the answer you made to the stat comm question: these two components are difficult to separate in the signal region but contribute differently to the 2 control regions, hence the strategy you are adopting to help finding stable fit results.... done

l111 move "obtained" at end of line sentence deleted

Section 5 : needs to be expanded !! Add one paragraph per uncertainty source, with exactly what is done and how the variation is calculated. Add a line in the table with all the ones mentioned lines 120-122... "scale" in table can be very confusing, what scale ?? Replace with "QCD scales" ?? done

l127 add uncertainties (stat+syst) to these factors. The final ± 0.15 is also not explained, or is that somehow related to the 14.1% of table 1 ??? done

l130 Add uncertainty on 1.57. done

l136 Make an explicit note here that if previous procedure lead to different scale factors for b-tag and JES than should be in signal region, it would be visible here as they are left floating again ?? But say that it's not the case ?? done

fig 2 caption -> spell out JES done

equation bottom page 5: remove bit between 3rd and 4th "=", I think everyone should be capable of passing Ngen from denominator to numerator done

139 typo in acceptance done

l149+1, l151 cross section*s* done

l153 add Pythia 8 ref (see SMP-14-010) done

l155 NNLO23_.... that's too jargon, is this an NNPDF pdfset ?? add ref., add ref also for CUETP8M1 tune...

l158-159 "a four-flavor" "a five-flavor" replace "a" by "the" to refer to the previously defined samples ??? Or are these new ones Also, it doesn't seem very appropriate to have a variation of matrix element (4 vs 5 flavor) to get an uncertainty on something which concerns rather the parton shower... was it not also pythia 6 vs pythia 8 by any chance ?? done

l168 add ref for LHAPDF. done

l169 add table with theoretical predictions and uncertainties (separating stat, PDF, scales) and refer to it. Figure 4 contains this information, and it is written in the last paragraphs of Section 6.

l173 Add conclusion section !!! done

References:

---

1-9+19+43-44 CMS Collaboration Collaboration done

43: missing journal,doi,arxiv.

All: check to give only first page (example, ref 1 gives 1-29 -> replace by just 1, ref 3 gives only 126 -> ok...) done

7 published, add journal done

15 Physics Letters -> Phys.Lett. in others.... done

37 twice arxiv done

Teresa

Comments

I agree with most of the comments sent by Anne Marie, and i do not repeat them here.

line 50-57: on top of AM comments, make clear you refer here to signal process only. done

line 88-93: i do not understand why this part is not in the MC Samples section 3.

moved

line 112-113: ... and averaged before later use. The average b-tagging scale factor is used to reweigh the MC, from which the jet energy scale is obtained from a fit to the ttbar-multilepton region. done

... i do not if this wording is better, but from the existing one is difficult to understand the two step procedure followed - done

Table 1: as AM says or use explicitly "Re./Frag. scales" done

line 131: what exactly means "adjusted by 1.57 standard deviations"? scale up? down??? done

pag 5, last para w/o line number. It needs reformulation, better wording .. done

Table 2: separate data from the rest. it would be good to include a column with the % of variation. table has been largely expanded

line 137: Nrec is the number of "expected" reconstructed events?? or just the number of reconstructed events?? reconstructed, this has been made more clear in the text

Figure 4: is it understood why the theory is systematically lower? Maybe worth commenting in the conclusion section (missing now) conclusion section added

Tristan

* 1.Introduction *

In the first paragraph, you mention b quarks, b hadrons, and b jets, but you don't define them. On l.44, you mention "b quark jet identification". Also the hyphenation is not always consistent. This is maybe not crucial for a PAS, but anyway some concrete suggestions to homogenize this:

- Define in the first paragraph "b jets" = "jets originating from the hadronization of b quarks" (i.e. generator level).

- Later, use "b-tagged jets" = "jets identified as such" (i.e. reco level).

I call this the Alexander Schmidt convention, which I think is the most consistent one and leads to the least confusion. Maybe also have a look at the Z+b-jets paper (http://arxiv.org/abs/1402.1521) for inspiration. This might seem a bit detailed grammar/spelling comment, but will also help the understandability of the text later on, when these details can be important.

good convention, done

* 2.Selection & reconstruction *

l.21 "with loosely" -> "with a loosely" done

l.32: pT is here defined as a vector, which is a bit unusual, and not consistent with previous appearances of this variable done

* 3.MC Samples *

General comment: You write consistently "five- and four-flavour samples". This is not wrong, but looks strange. I would turn it around. In l.152 it is actually the other way around. done

- l.36+59: "kT" variable not consistent done

- l.59: Period at the end of sentence missing done

- l.60: "V" is not defined. You can probably do that already in the first sentence of the Introduction. done

- l.70-73: The whole paragraph is an unusual description of pileup. I would have a look at other CMS publications, or simply check the CMS guidelines. done

* 4.Strategy *

- l.79: I think you mean "b-jets" -> "b-tagged jets" done

- l.96+97: as mentioned by Guenther, drop "with pdgId=x" done

One more suggestion: to be extra clear, I think you can specify that W+bb is defined by having a W and at least one b hadron without any gen-level cuts on eta/pT. (At least, that is how I understand the text.) This might seem too detailed, but sometimes people define V+b as having a b in the jet in the acceptance... so it is better to make it extra clear if there are any criteria on the generator b when defining your signal. in fact, we do cut around signal acceptance. PAS has been updated to reflect this

- l.101: You write "The obtained shapes are corrected for the presence of all other backgrounds"... I don't understand what this involves. Does this mean you subtract MC backgrounds from data? Or something else? Please clarify in the text. done

- l.106-116: Guenther asked some questions about the fit here, some clarification about the motivation and correlation would indeed be useful. done

* 5.Systematic *

Table 1: the table is very minimalistic. I am especially a bit puzzled by the 14.1% of the b-tagging. Some questions: we have a new, more inclusive table now

1) Where does this number come from? Only b-tagging, or also other contributions?

14.1 actually should have been even higher, 14.8 which is the 100% uncertainty on rescale factor. The uncertainty on the original b Tagging scale factor varies by sample, for example 6-7% for Wbb, T, TT 13-15% for Wcc 40% for W+light see Table 8 in AN

2) Since this is the largest contribution... isn't this actually a bit big? Given that the b-tag uncertainties are usually 3.5%, I would expect an uncertainty of 2*3.5%=7%. Or is it because you are using the Tight WP? (I think you already explained me this some time ago...)

I think the confusion is between the b-tag SF with uncertainties as perscribed by the BTV and the b-tag rescale factor that we fit in Step 1, ttbar multijet region. We find that the b-tag SF should be rescaled by 14.8% and put a 100% uncertainty on this re-scale factor to cover the imprecision of this method - all of the change in normalization from the ttbar-multijet fit is absorbed into this re-scale factor which is a rough estimation.

3) Why do you actually fit the b-tag SFs yourself, instead of taking from BTV? Would that make the uncertainties possibly smaller? (I have the vague impression you also have explained me this already before, sorry if I forgot.) We apply the SFs from the BTV and then see that they do not accurately describe our data in many related ttbar phase spaces. This was the conclusion of the presentation made to the BTV, that further scaling was appropriate.

4) Is there any correlation with the ttbar normalization? In other words: are you possibly double counting the uncertainties, thereby overestimating your uncertainty on the cross section? That would be a pity.

The difference between the b tag SF and ttbar normalization comes in how they are correlated between samples. The b tag SF is correlated across all MC samples while the ttbar xc uncertainty (this is what I assume you mean by “ttbar normalization”) is only tied to the ttbar sample. With the shape of the b tag SF shifted templates being so similar to the shape of the unshifted template, this is hard to disentangle in fitting a region dominated by ttbar and one of the reasons for assigning such a large uncertainty to the rescaling factor.

==> I vaguely remember having asked many of these questions before (sorry for my poor memory)... but maybe some clarification in the text will help. And I just wish the impact on your measurement from uncertainties from b-tagging could be smaller somehow... I think the confusion is between the b-tag SF and b-tag reSF

- l.123: "affects acceptance" -> "affects the acceptance" done

- Table 1: It was not clear to me what "+ shape" means. done

- Table 1 caption: "in fitting" -> "in the fitting" done

* 6.Results *

l.127: You write "b-tagging efficiency scale factors", and I know what you mean.... but the phrasing is confusing, because normally this concerns the scale factor of a single b-tagged jet (whereas in this case it concerns the scale factor of a sample with two b-tagged jets per event). So I would phrase it a bit less ambiguous. done

l.128: I unfortunately still don't understand where the uncertainty in "1.15±0.15" comes from. done

Caption Figs1+2+3: "muon on the left and electron on the right" -> "the muon sample on the left and the electron sample on the right" done

l.133: "normalization" -> "normalizations" done

l.139 "accpetance" done

l.140: "channel, " -> "channel, and " done

l.142: "exactly b-jets" -> "exactly two b-tagged jets" ? done

l.145: "a B-hadrons" done

l.149+151: "section are" -> "sections are" (twice) done

l.153: comma usage not consistent with previous occurrences done

l.157: For the hadronization factor you could maybe refer the reader to the first CMS Z+b-jets measurement (see my e-mail). Although this only concerns a single b in the final state, so not sure it helps... (Anne-Marie will have better suggestions on this topic) added

l.163: "of W produced" -> "of a W produced" done

l.167: "Uncertainty" -> "The uncertainty"? "section" -> "sections"? done

l.168: For reference to be used for the PDF uncertainties, see my forwarded email from the Higgs mailing list. thanks

l.168: As discussed during the meeting, I was told that LO ME should be used with LO PDFs. (But I don't know if this was the opinion of the theorists I was talking to, or a fact.) Anyway, I think it is sufficient to use systematics for the corresponding PDF set alone, without additionally invoking higher/lower-order PDFs. done

l.171: space after Fig done

l.172: "of the order of" -> "of approximately"? done

General physics comment: the predictions are almost all the same, and the measurement does not distinguish between them.... This is a pity, but I would try to make some philosophical comments here, about the consequences of your measurement, in order to give more weight to your study. As Theresa suggested, maybe comment on the comparison with Atlas, or with the 7TeV measurement... but at least try to make the reader aware of the relevance of the measurement, and what we should learn from this. This is certainly worth some effort when you go for a paper.

Currently, it looks like "we are 1 sigma away from the predictions, and that's what it is", but I would really try to squeeze out more on the physics interpretation.

(DPS was a hot topic in this final state, given the observation in W+1b. But I am not sure that quantitatively commenting on DPS is possible, given the still limited sensitivity in this final state.)

And as said before, it would also be nice to have some distributions. E.g. DeltaR (b,b). You already said you would doing it for the paper, just making sure

(And it would be nice to have an interpretation of H±->W±A. But that's more something for Maria's HigExo group.)

* References *

1+2+3+4+5+6+7+8+9+19+43+44: "Collaboration Collaboration"

done

(Received 2015 July 17)

Anne-Marie

Comments

Better describe the new fit procedure

The new fit procedure is driven by the need to fit for both JES and btag

1- btag effect is dominant in ttbar multijet control region: fit m_T in this region by itself, extract btag scale factor and apply everywhere with 100% uncertainty.

2- JES is dominant in ttbar-multilepton control region due to jet veto: again from fit to m_T in this region, extract JES in terms of number of sigmas to be applied per sample event-by-event everywhere, apply with 100% uncertainty.

3a- Fit signal region in e and mu channels separately

3b- Fit signal region simultaneously in e and mu channels.

- why 100% uncertainty: to not overconstrain the fit at each step and allow the next fit to go back to nominal value if necessary. In practice, it never happens: results are stable. Closure test done applying JES and btag scaling in ttbar-multijet and everything comes back at 1.

Action Items:

- discussion on pulls: some of them (QCD) not 0 with width 1. What we were calling "pulls" were actually uncertainty bounds on fitted parameters. We have evaluated the effect of each uncertainty in Figure 22 of the AN.

Mean has to be recalculated as described in AN line 413: Tom will update tables 9-11 with the correct mean.

Tables 4,5 in the AN show normalizations of all samples at all stages in the fitting procedure. We have incorporated a shape uncertainty into the QCD estimation as well, see below

QCD in ttbar CR should be ignored: it is small/negligible. In signal region, Tom will check the shape of the pull distribution, and the meaning of sigma=0.2. This is also valid for the CSV pull: sigma=0.5... Philippe mentionned that sigma<1 means uncertainty over-estimated. AM mentionned correcting to have pull=1, as suggested in this pdf, last page, text quoted below: physics.rockefeller.edu/luc/technical_reports/cdf5776_pulls.pdf

"It may happen that the pull distribution is approximately Gaussian, but its width is not 1. Assuming that this is understood to be an effect of the non-asymptotic nature of the problem and not a programming error (this can always be tested by running pseudo-experiments closer to the asymptotic limit!), one may want to correct the quoted uncertainties by multiplying them by the width of the pull distribution. In other cases the non-asymptotic nature of the problem manifests itself by the appearance of tails in the pull distribution. One must then be careful with the interpretation of the uncertainties. If the percentage of pulls between -1 and +1 is 68.27%, then â?o1-Ïfâ? errors have the usual meaning. However, since the pull distribution is not Gaussian, â?o2-Ïfâ? errors no longer have a coverage of 95.45%, etc. Finally, as illustrated in section 5, one should keep in mind that different pull definitions have different rates of convergence towards the asymptotic limit. Thus it may be that the choice of pull definition itself is the cause of non-Gaussian distortions in the pull distribution."

See AN Tables 15-17 for pull distributions on final fit. The fit was repeated on an Asimov set of toys based on MC+QCD distributions. All uncertainties fitted values are centered around 0 and nearly all have unit width. The ones which do not always have unit width (QCD, reweighted b Tag SF, JES) have uncertainties which were deliberately estimated to be conservative so the width of the pull being less than one is not a concern. The errors could be reweighted and reevaluated as described above but this should have no effect on the central value of the final effect of a given uncertainty, and would only be used in estimating the contributions of uncertainties to the final error. We have chosen instead to estimate the contribution of a parameter to the fit by observing directly the effect of removing the uncertainty on the final error on the fit - see AN Figure 22.

- discussion on QCD: AM concerned about 50% uncertainty not so justified. No impact on results (authors checked varying up to 500%) but would be nice to have something to base the choice of 50% as being reasonable. Shape diff from Giuseppe vs Tom ? Normalisation diff when using mT<20 or 20<mT<60 GeV ? Everybody else had no concern about this so AM will not insist further

QCD is taken using the WI initial normalization, then using the WI and TR shapes as +1 and -1 sigma with the average as the nominal.

- fig9 shows DY+jj control region, excess of MC over data: Tom will check.

Comparing to the Z+Jets AN: 2013/224 http://cms.cern.ch/iCMS/jsp/db_notes/showNoteDetails.jsp?noteID=CMS%20AN-2013/224 Fig. 23, leading lepton pT shows rising MC/Data ratio with pT_Z. We also checked many related DY+(b)jet(s) phase spaces and see agreement with their results. A full set of plots can be found: http://www.hep.wisc.edu/~tperry/wbb/studies/tmperry_2015aug01_Drell.pdf

- fig12: "after fit" uncertainties much smaller than "before fit" fig11 -> expected, combine constrains the uncertainties well.

correct

- fig17-19 LES seems to affect shape a lot for electrons: Tom will check (could it help understand e-mu discrepancy ???)

We had a bug that has been fixed. Electrons were shifted by 1\% then added to MET that was corrected for an electron shift of 6 \% in the calculation of the mT variable. AN Tables 6,7,8 show the updated shifts to be on the order of 1\% for both leptons.

Tristan

Section 6

- Fig.9: I am a bit surprised by the meager agreement between data and simulation in the DY final state in Figure 9. Are the lepton scale factors applied here? Has such a discrepancy been seen before in other CMS Z+jets searches? Please confirm

Lepton SFs for (ID,Iso,Trigger)Efficiency are applied for both leptons, only counting the trigger once. A fuller look at related DY regions agrees with a dedicated study - see comments above

- l.336+349: You determine the SFs you find for both ttbar control regions. Unfortunately, I am not sufficiently expert of ttbar measurements to know by heart if the numbers you find are in agreement with previous CMS measurements. Could you provide some references (or -better- numbers) such that the reader can judge if your scale factors are indeed in agreement with what is expected for your crosschecks?

A presentation with references and many numbers is http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/Tests/tmperry_2015mar31_TT.pdf A summary was presented to the BTV POG https://indico.cern.ch/event/436169/session/1/contribution/25/attachments/1134376/1622924/tmperry_2015july30_TT.pdf A further update including a b Tag on the subleading jet (but not necessarily the first) can be found here http://www.hep.wisc.edu/~tperry/wbb/studies/tmperry_2015aug01_TTbar.pdf

- l.365: You write "two-dimensional simultaneous fit", but from the text I think the fit is actually only one dimensional (i.e. not 2D), it is merely simultaneous. You fit the el+mu samples together ('simultaneous') but you only use the MT observable ('1D'). Can you confirm this?

correct, fixed

- l.369: You find a SF of 1.44, but it is not clear with respect to what. It should be made clear what reference normalization you are referring to. This is also important in comparison with l.435, where I don't find a SF of 1.44, so it seems not consistent with the final number from MCFM. So can you please clarify the normalization?

This has been clarified in the text

- l.369: The uncertainty of your combined electron+muon fit hardly improves wrt the separate uncertainties. This is a pity, but it's probably due to the correlations between both final states... We already discussed this issue during the meeting yesterday, it would be nice if you could comment on this in the AN (and the PAS).

- l.369: Your combined measurement also seems closer to the electron measurement than the muon measurement. This is not intuitive. Please give some explanation on this (in the AN and the PAS).

Measured signal strengths are Muon: 1.2728 +/- 0.242 Electron: 1.3779 +/- 0.300 Combined: 1.3525 +/- 0.225 and the fact that the combined result ends up closer to the electron number than the muon reflect correlation between uncertainties. In the previous iteration with the ARC, the numbers (signal strength and yields) were slightly different than presently. This was due to having used an electron cone size of 0.4 instead of 0.3 and the full set of differences at each step of the fitting procedure are given here http://www.hep.wisc.edu/~tperry/wbb/studies/fit_difference_electronIsolation.txt Also here are the numbers to see how the yields vary between the individual and combined fits, the basis of the signal strength.

- Fig.15: You use the QCD shape from a control region. It would be nice for the reader if you could refer to the corresponding section/plot.

AN Fig. 2 shows the QCD distribution made using the procedure described in Section 3.5

- Fig.16: What do you finally get for the ttbar scale factor in the signal region? I did not see this in the text. It is important to know if the ttbar is described well in the signal region.

Like all MC samples, TTbar sample is scaled up by 14.8% as a result of the fit in the tt-multijet region. It is then scaled by 1.57 standard deviations in the jet energy scale. For the ttbar sample in the signal region 1 standard deviation in the jet energy scale corresponds to a normalization change of 0.993(0.996) in the muon (electron) channels. The normalization here is given in AN Tab. 4,5 column "Step 2". The final fitted normalizations of the TTbar and all other samples are found in these tables as well.

Section 7

- l.405: The sentence is not fully clear. Does this correspond to the full b-tag uncertainties (i.e. 2b-tags) or is this per b-tag? And if this is the full uncertainty, does this correspond to the expectation? From the top of my head, b-tag SF uncs are 3% per b-tag, so 2*3%=6%, but please confirm in the AN.

This is the full uncertainty accounting for both b-Tags, applied according to the BTV POG perscription. 3% per gen level B, 6% for gen level C, and 20% for light so 3*2=6% is expected.

- l.432: The contribution from DPS, there is not really a chance we can measure this in W+bb, right?

Correct, we can not measure this.

General comment: it would enhance much the impact of the measurement if you could provide more theory comparisons for the measured cross section. Maria already showed these plots on her phone, it's great if those will be included.

- l.435: As stated before: I think the measured cross sections, and comparison with theory, are not consistent with the previously quoted SF of 1.44. As it is important to know how far we are from the various predictions (i.e. how far we are from SF=1) it would be helpful to clarify where the different reference values come from.

More comparisons have been added and documented

- Table 9+10+11: It is impossible for the reader to judge why these pulls are okay. For example, all pull widths are smaller than 1 (coincidence?), quite some are significantly smaller than 1, and the means of the pulls are often nonzero. (Although one would need to uncertainties on the pull properties to be really sure about this.) The variables with such values includes important uncertainties like 'QCD', 'CSV' and 'DY'. So I think some more clarification is needed to make sure this is indeed okay.

See commend to AM above, uncertainties have been reassessed and all behave as expected. See AN Figs. 19-22.

* D: ttbar crosscheck

I think your study in Appendix D is nice, and it would be interesting to discuss this in BTV with the experts. I think they can also learn from it. However, it was not clear to me from the text if and how the b-tag scale factors are applied to include differences between data and simulation. I think you should describe the procedure (possibly referring to standard BTV methods) in order to let people judge&appreciate the described discrepancies.

Presentation made to BTV group, followup study to check bias from ISR by checking single b Tag on subleading jet is: http://www.hep.wisc.edu/~tperry/wbb/studies/tmperry_2015aug01_TTbar.pdf

* Comparison to H->bb *

For your reference, the CMS H->bb paper is here:

http://journals.aps.org/prd/pdf/10.1103/PhysRevD.89.012003

The table with corresponding W+bb scale factors are attached. I think you can refer to them in the PAS, and possibly make some comments how they compare with the values you observed.

(Received 2015 March 2)

GENERAL: 1) overall, we think that the PAS requires a MAJOR overhaul in order to become readable and understandable. The current version is considered to be very far from a status of becoming public.

2) for a CMS SMP measurement in the year 2015 it looks like the amount of results (1 integrated cross section compared to 1 theory prediction) is a bit meagre. However, we fully understand the circumstances and limitations, which might prevent going much further than that. Nevertheless, we would like to ask: - If the measured background correction is wrt MadGraph, then why not add the comparison of the measurement with the MadGraph prediction in the results? - There should be many more comparisons with other generators, preferentially in a nice table or graph. E.g. comparisons with MadGraph 5F, MadGraph 4F, aMC@NLO, should be possible. They are also mentioned in the Introduction of the PAS .. Also, MCFM is one of the least preferred comparisons (as actually written by the authors around PAS line 158), so we should really show some of the various other available generators.

Answer: We will add to the PAS/AN the 5f and 4F Madgraph predictions. Currently these predictions are missing the PDF uncertainty; which will be added in time for approval. We will add additional generators (such as aMC@NLO) in the transition from PAS to paper.

In the following, our comments focus now mostly on the documentation which is found in the AN

3) MC samples selected for this study: appear ok. In fact you use the same generators (same versions and same conventions) as in the published 7TeV paper SMP-12-026. Also same MC normalizations (NNLO, etc..), except for ttbar that now uses LHC measured x-section (~ 7% total uncertainty?)

Answer: correct

4) Object definition and Event selection: also ok, also here no changes wrt to 7 TeV but for: Electrons have been added and there is a new veto on single-top (forward jets)

- are the recommendations for b-tagging scale factors applied?

Answer: The selections also differ from 7 TeV in the bTag requirement. At 7 TeV we required CSV Tight and SVmass > 0 while at 8 TeV we only have the CSV Tight requirement. And yes, the Type 1C bTagging scale factors are applied.

5) overall scaling factor for MC pred.: MAJOR issue for us, regarding this analysis: We are not convinced by this applying of a common "cScale" everywhere. The multijet region has a massive difference wrt signal region in that it has a lot more jets !! Something related to jet energy correction would just be enhanced there and not necessarily applicable to the signal region, right ? Having an explicit veto on extra jets also kind of removed purely hadronic top decays for the signal. Have you studied the composition of the ttbar in the signal selection ? Is it dominated still be purely hadronic or by leptonic with leptons outside the "tight" acceptance (eta<2.1, pT>30 ...)

We have two different approaches, when making this analysis. The one, which is described in the PAS with cScale and another one using combined tool (see AN), which allows to modify all shapes and normalisations in correlated/uncorrelated matter as we did for 7 TeV analysis. Using this tool we can see how the different scaling factors change and the major players are the JEC and b-tagging contributions. We demonstrate in the AN that if we go as close as possible to the selections, which were used in top analyses we get exactly the same results, confirming that what we do is consistent with CMS published results (see later in in twiki). As soon as we move closer to our phase space the scaling factors start to change. We explore 2 different ttbar control regions - multijet and multi leptons and observe, within uncertainties, similar results. At the end when we fit signal and control regions together with observe that the fit gives the ttbar scaling factor closer to the one we observe in the multijet region alone. This is why we also use it to demonstrate the ttbar description. Using multi lepton region will not change the final fit result, but "only control region fit" will be slightly different, as shown in the AN.

A few things here: - In the final fit (baseline result) we understand that the C(scale) is obtained from the combined fit to the multi-jet ttbar control region and signal sample, right? It is not fully clear in the text .. or we missed it. There are two (slightly) different numbers when using one or the other. - In reality this common factor looks as clearly obtained just from ttbar. ttbar dominates the control region used (multi-jet top enriched) as well as the signal sample. To start with, just in the signal sample there is almost 3 times more ttbar than signal, and about 2 times more ttbar than the sum of the rest of the background.. roughly speaking). - It is not fully clear how you do the final QCD normalization: It seems that as result of these fits (with a common scale and letting vary the ttbar, Wbb and QCD) you obtain that the initial QCD contribution must be re-scaled by 0.8 (see figures 11-14 with the result of the fit in the various versions). It is also at the end re-scaled up by the common factor, Cscale? Of course, given the 50% "standard" error in this normalisation, anything may be perfectly acceptable.. - is it correct to assume that you have made all possible basic cross-checks using the old selection to verify that the changes are coherent with the cross-section changes from 7 to 8 and the changes in detector efficiency, etc.. ? If yes, then this should be clearly mentioned. - Cross section formula: Introducing Cscale explicitly in the denominator could be misleading.. maybe it is just better to make more explicit that Ntotal and Nback are the results of the fit.

- also, in general: how is the fit done exactly? binned likelihood? which fit ranges? You don't say anything about this

Answer: In the final fit, c(scale) is constrained to the scale factor obtained from the multi-jet ttbar, within its uncertainties.The normalization of the QCD is obtained from the final fit in the signal region. In order to initialize its value to a reasonable approximate number, we perform an initial fit in the mT < 20 GeV sideband, while keeping the Wbb signal and all other backgrounds fixed to their MC prediction.The c_scale factor is not applied to QCD, only to simulated samples.The fit is a binned ML and it has been performed in the range 0 < mT < 200 GeV. The motivation to include the low end tail of the mT distribution is to retain discrimination power wrt QCD.We agree that the choice of c(scale multi-jet) and its application to all of the simulated samples is the most delicate point of the procedure. Indeed we are trying to demonstrate that the observed data/MC disagreement is the consequence of a global scale scale factor (dominated by b-tagging). In this case the application of c_scale to all of the simulated samples is well motivated. More studies are being performed in order to better understand this disagreement. As pointed out above, we obtain the same results, using the combined tool, where each contribution of possible uncertainties JEC,b-tagging, effieicinices are all allowed to wary within their uncertainties in the fit. E.g. we know exactly what is changing when we perform the fit.

6) signal definition and control regions: - PAS L.104: is this the recommended W+bb signal definition following the LHC EWK WG recommendations? Are there no b-hadron cuts required? This could be important when comparing to Atlas.

Answer: At detector level there are currently not Pt requirements on the BHadron. We do check that at least one of the jets is matched to a BHadron at generator level. We have compared this selection definition to several others: splitting the sample by parton content (bbbar, c, ccbar...) and by the presence of bHadrons at generator level (without jet matching). Our tight selection (2 bjets at selection level) makes the contamination of other W+Jets sources (W+c, W+ccbar, W+LightJets) minimal; and there is no significant difference between the different selections.

For the definition of the fiducial volume in which the cross section is defined, we ask for the generator level jet (pt>25GeV) to be matched to a BHadron. We do not ask for a pt cut on said BHadron. We have studied the effect of said BHadron Pt cut, and found that for Jets of 25 GeV the requirement of a 5GeV BHadron Pt Cut is redudant (cross section difference < 0.005). If the ARC requires it we can recompute the acceptance with such a cut and propagate the effect to the cross section.

- Fig. 5: the disagreement data-MC in this TTbar enriched region seems quite large to us. DId you discuss with the ttbar group about this, do they see similar disagreements in related 8 TeV analyses, and did you use possible prescriptions by the ttbar group to handle such issues?

This disagreement also seemed quite large to us at first. This is why we performed the study detailed in Appendix C. The summary is: when we move from our selections to those used in the published TTbar dilepton paper, we see agreement on the percent level in yield and a flat ratio in LeadingJet _Pt, W_TransverseMass, and MET as illustrated in Figure 38. So we agree with the official TTbar measurement when we do an apples-to-apples comparison, it is only when we move to our adopted definition (lower pt thresholds, more btags) that disagreement arises.

- similarly, Fig. 4: did you discuss with the group working on the incl. W+jet analysis about the observed discrepancies? Is it purely a MADGRAPH problem?
We did discuss this with the W+Jets people and see a similar slope in the ratio plot of the LeadingJet _Pt as can be seen in Figure 6 of AN-2014-114 (http://cms.cern.ch/iCMS/jsp/db_notes/noteInfo.jsp?cmsnoteid=CMS%20AN-2014/114). We are also presently in contact with the WW/WZ group who also examine a similar phase space and are cross checking our results against theirs.

- is there any chance to use aMC@NLO instead of MADGRAPH? would there be enough samples available?

Answer: not at this stage, no samples were generated centrally for us; and the private production and reconstruction of a large enough sample will not be possible in time for the PAS. If for paper we will get a sample we can use it to complement the Madgraph results.

7) Uncertainties: - As it is mentioned in the text, still some Theo errors are missing (renor, fact, and PDF). When will these be ready?

Answer: we will use for now the 7 TeV numbers (see SMP-12-026 and the associated review, in particular https://twiki.cern.ch/twiki/bin/view/CMS/WPlusBBPreCWRl and https://twiki.cern.ch/twiki/bin/view/CMS/Wbb7TeV). They will be cross checked for 8 TeV in the timescale of the paper.

- Table 6: we do not understand why JEC and mainly JES is so different between electrons and muons

Answer: We have cross-checked the JEC and JER systematics and indeed we can explain the difference only as a statistical fluctuation. We will include some distribution plots for the JEC(up/down) and JER(up/down) distributions in the AN, and a table with the corresponding cross sections.

- where is the error on the modeling of the signal (W+bb, which is corrected by ~30% by the fit) introduced?

I'm interpreting this question as "which uncertainties contribute most to the measured signal strength (of ~1.3)" From the fit using the Higgs Combine Tool described in AN-Appendix B, most of the overall change in normalization of the signal comes from the shape uncertainties illustrated in AN-Figure 28. Taking a representative fit (4-Flavor Wbb,Multi-Lepton TTbar, Electron Channel - chosen because it has the signal strength closest to the mean of the eight), the bTagging and the JES are both pulled by ~0.95sigma and these are the major contributions to the overall change in normalization. Detailed pulls for every systematic for each fit can be found here http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/Tests/systematicShapes/pulls.txt

8) QCD and other backgrounds: AN l194: why complicate the formula doing 1 + X - 1 ???? Just (d20-m20)/q20 is OK and very clear: you normalise the QCD sample to the data-bkg in the 0-20 GeV bin in mT, cannot be simpler....
ok

At what level in the selection is fig2 made ? You seem to have a quite good agreement, but don't explain why 50% uncertainty :/ You talk about "fit" and "extrapolation to high mT region" but all you said before is the normalisation to the low mT bin: so what is the fit ???

Figure 2 is the template used to make the QCD shape. Plotted is the data and the simulated backgrounds. The backgrounds are then subtracted from the data to give the QCD shape not stacked with the other MC. The "good agreement" is then really just an indication that not very much MC passes this inverted selection and we take our shape almost entirely from data.

Could we have the following distributions:

- purely from the inverted selection, data and bkg, so we can see how much the bkg matters in the QCD shape.

Figure 2 is this distribution

- fig2 in log scale and add ratio plot? so (qcd+bkg)/data...

Since there was confusion about what Figure 2 showed we are not sure about this request

- Is this entire section 3.5 in the AN misplaced maybe ?? Would fit better in section 5?

possibly, we will think about it

- Did you consider W+bc as a background?

W+bc is a very small contribution, and gets absorbed into our definition of the signal

- PAS Fig.1: Why is W+dusg only visible in a peak at MT=80 GeV? And is it a coincidence that this is the region which seems modeled least well? Given the fact that this is the only observable used to estimate the non-b backgrounds, this observation might not be irrelevant.
We're investigating this

9) Kinematical distributions: Maybe Fig 15-on give a hint of what happens. The pt distributions show a deficit at low momenta (QCD? Wc?, Wtau?)

Same as above

================================================================================================================================

Further detailed comments to the AN:

l65-69: PDGIDs of 1,2,3,4,5 refer to partons (quarks), not hadrons. It would be good to specify that you consider all partons before hadronisation. In Pythia jargon, that means status 3 (from matrix element calculation) and 2 (from the parton shower). How do you decide it belongs to one of the jets ? Are you making parton-level jets, matched to genjets/recojets ? Or are you doing a matching of genjets/recojets to the partons ?? Please give more details.

eq.1: you do not explain how you compute E_T^PU in the isolation formula

E_T^PU is the so called deltaBeta correction to the deposits coming from neutral particles and is calculated as a sum of track Pts not coming from the PU vertices inside the isolation cone. This is divided by the factor of 0.5 which corresponds approximately to the ratio of neutral to charged hadron production in the hadronization process of pile-up interactions.

l140-141: any pileup corrections to ETmiss?

Corrections made to the jets are propagated to ETmiss. Since we use ak5PFchs jets, the PU corrections applied are taken into account

l211 you don't mention what you do for the electron ECAL gap in eta :/

Events in which there are electrons with a supercluster inside the ECAL gap are rejected. which is included in the electron definition in section 3.2.

l228 a plot of mT at this stage would be good for the reader to attest that discrimination power.

This is shown in figure 7, it can be moved here.

l237 calls for the distribution in jet multiplicity to be shown here.

OK

Fig3 left: what is the origin of the peak at ~23 for the W+jets sample, but not present for the other processes ??

Fluctutation?

Missing line numbers: can be fixed by removing/adding white lines around an equation, even if the equation is 10 lines after...

We'll fix this in the end.

l266+3 lepton from the W I suppose ?

Yes.

l266+4 B hadron is all pdgids in ranges 500-599 and 5000-5999 ?

Efficiencies: would it make more sense to call this acceptance rather ? If I understand correctly it's purely generator level, and is the ratio of events from a Wbb 4F sample passing the l+2b particle-level selection over total. We normally refer to efficiency for the detector-level selection:given what is in the acceptance (the generator-level selected events), how many actually have reconstructed leptons and jets passing the reco-level selections. Right ?

Fig4: you imply l286 that this is before b-tagging. So just l+2j ? It seems W+cc is half the sample: is that really the case before any b-tagging ??

pT: start binning at 30 GeV to avoid first bin lower...

OK

The QCD has a weird step at 40 GeV : why ?

This is the result of not applying the trigger to QCD. Events are entering (MC) via the SingleMu40 trigger that wouldn't pass our trigger so don't appear in Data. If we apply the trigger, then this problem disappears and the plot in the AN has been updated.

What is the shaded band ? Add in caption. - also, your statement in the text that the MC describes well (!) both shape and yield is simply not true, when looking at Fig. 4

Shaded band in the plot is the statistical uncertainty. The statement has been removed.

l281: to which NNLO prediction to you normalise the signal? NNLO for incl. W production?

Yes.

Fig 5: shaded band ? Caption: "multilepton inverts the jet veto" but doesn't say that l.299 :/

The shaded band is statistical uncertainty. Multilepton TT control region has the same selection as signal region except for the requirement for presence of two oppositely flavored leptons. The statements have been corrected.

- multilepton control region: what happens if you also add the ee + mumu regions cutting out the 60-120 Mll region ? Would it help anywhere to have more stat ?

We can check this, but the level of agreement is apparent with the statistics we have.

l312: the set of MC SF described earlier" what are you referring to here ? An exact section would be nice Or repeat lepton reco and id SF and btagging SF ??

OK

Table 4: add a line with total bkg to simplify the reader's task ?? And really should add at minimum the stat uncertainties for the MC... - similarly for Table 5: put first the sum of all backgrounds, then maybe your line data-bckg, and then maybe a line with the expected signal (using one of the predictions as example)

OK

section 5.2: for multijet ttbar region, veto is removed, but does that mean it includes the signal region ?? Is there an explicit requirement of at least 3j ?

There is an explicit requirement of having 3 or more jets in the event which excludes the signal region.

- section 7: formula: (same for the PAS): something wrong with just efficiency term. This joins the comment about the efficiency calculated in section 4.2, which we think is an acceptance, but then we don't know where is the efficiency....

- section 7.1.6: anything not purely "data" we would call syst: so we would put all the MC-related stat uncertainties in systematics

ok

Table 6: we cannot reproduce or understand how you calculate the combined uncertainties...

Answer: The systematics in the electron and muon channels have been considered either as fully correlated (JEC, JER, pileup, Lumi, b-tag) or uncorrelated (lepton SFs). The central value of the combination is the result of the weighted average of the cross sections for electrons and muons, taking into account both statistic and uncorrelated systematic uncertainties. The uncertainties on the combination coming from fully correlated systematics have been estimated by averaging the corresponding systematics uncertainties obtained on the electron and the muon channels, and applied to the final result. The breakdown of the different contributions to the final error has been evaluated by applying small perturbations to the uncertainties in input, and estimating the effects on the final total error. Given the simple problem at hand (only two results to combine, correlations at 0% or 100%), this approach is totally equivalent to BLUE (this has been verified as an exercise).

l472-478: Is the background-subtraction systematics considered as uncorrelated or correlated ? It is omitted in the list given here.

- section 7.2.3: it is not clear if you apply in the end this 10% theory uncertainty or not.

No, not at the moment.

- section 7.2.4: we suspect that the combination e-mu is not done properly, regarding the systematic uncertainty. Can you explain how you did it exactly.

see answer to Table 6 question above

Fig 7,8,9 no gray region anymore ...

Which grey region? The band in the ratio?

Fig 10 shows a cQCD not explained in section 6.1.1.

Added. It's a scale factor for QCD contribution with the shape extracted from data, as described in the section 3.5.

in fig11,12 cQCD is very different than in fig 9...

These are QCD contributions for TT control region(fig. 9 and 10) and for signal region(fig. 11 and 12). We don't expect these numbers to be the same.

================================================================================================================================

Further detailed comments which arise when reading the current version of the PAS:

In the following, we don't comment at all about text issues, we focus on content. Text edits will only come once we are happy with the actual content.

* Introduction - Some words should be spent on the Atlas W+1b measurement, and why this motivated to do the W+2b measurement: data/simulation tensions in W+1b, possible DPS contributions, possible effects of gluon splitting in Z+bb, etc... - Also, it should be said what this measurement contributes to the existing CMS W+2b-jets analysis at 7TeV, and in which sense it differs (e.g. in terms of analysis strategy). I hope we will learn even more this time, but it was not clear to me. - For an SMP paper, this kind of motivation should be the focus, and not simply as a background of the "discovery of a neutral Higgs boson". I would recommend the authors to have a look at the Introduction of some other CMS SMP papers for inspiration. - I also find it strange that the production of simulated samples is described here, this should go in a dedicated section called "Datasets" or similar. ==> General comment on the Introduction: the physics motivation can be made clearer (although I know there is a good one)

Answer: Yes, we absolutely agree with this statement and will work out an extended introduction for the paper that is expected for the summer and will include much more studies, then just a cross section measurement. For this PAS we want to restrict the introduction to this very short part, since it may look strange if after an extended explanation what should we do to make more use of Wbb we will show only one number.

* Event selection & reconstruction - No description of the CMS detector? That's quite unorthodox. - I did not see any description of the recipe for the application of b-tag SFs. Were they applied?
For the PAS only it was removed according to the CMS rules. Yes the SF's were applied, we have added a line at the end of the b-tagging description

* Signal&control regions - The last paragraph of this section discusses the ttbar regions, but no comments on the (mis)modelling. I would like to read if e.g. the MT is modeled well, as this is an important element in this study.
added

* Background modelling: - It's strange that this measurement relies on other estimates of e.g. ttbar (except for the multi-component fit of the MT variable)

indeed the measurement just uses 2 regions one is ttbar dominated and one where the signal is present, all other regions are used only for checking the data-to-MC description. We will make it more clear in the text

- Fig.1: What does it mean that all background factors ("c_QCD" and "c_scale") are inconsistent with 1? Given the background estimation procedure (and the big size of the backgrounds in this analysis) I am a bit worried about this.

the MC does not perfectly describe the data, the fit scaled different background contributions and provides a signal strength for the signal. We know the JEC and b-tagging efficiencies need to be scaled from their nominal values

- Fig.1: I don't see any uncertainty band? This should certainly be added on the plot, to at least absorb the huge uncertainties due to limited MC statistics...

We are preparing this plot

* B-tagging & non-b backgrounds

- Within BTV, I have had some discussion with some of the authors about the (mis)modelling of the M(SV) observable, which was used to suppress backgrounds but showed a particular problem in the 8TeV analysis. In the current version of the PAS, this observable is not used anymore, so I would like to ask the authors to instead show the observable which is currently used, namely the CSV discriminator. (It's not in the AN.) These variables are partly correlated, and in case there are big differences here, this might require an additional uncertainty related to b-tagging.

w + bb:

http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/wbb/Histograms_wbb_goodJ1_CSV_mu.png

http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/wbb/Histograms_wbb_goodJ1_CSV_ele.png

w + jj:

http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/wjj/Histograms_wjj_goodJ1_CSV_mu.png

http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/wjj/Histograms_wjj_goodJ1_CSV_ele.png

tt multi-jet:

http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/ttjjj/Histograms_ttjjj_goodJ1_CSV_mu.png

http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/ttjjj/Histograms_ttjjj_goodJ1_CSV_ele.png

tt multi-lepton:

http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/ttme/Histograms_ttme_goodJ1_CSV_mu.png

http://www.hep.wisc.edu/~tperry/wbb/latestGreatest/ttme/Histograms_ttme_goodJ1_CSV_ele.png

* Results

- Formula: shouldn't there be any acceptance term?

right, should be added in the description, will do

- Why are the SFs in the plots >1.30, whereas the final cross section measurement ratio data/MCFM~1.2?

Answer: The wbb SF of 1.3 should be compared with the data/MadGraph5FS: the pre-fit normalization of the signal Wbb sample corresponds to the MadGraph 5F cross section, not MCFM. In this case there is good agreement: 0.66/0.51=1.3.

- Table 2: it does not make sense that the combined el+mu result is not better than el and mu separately...
see answer to Table 6 question above.

- Table 3: the JEC uncertainties are larger than the b-tag uncertainties, I find that strange. The description in the PAS of the systematics is very "minimal", it would be nice to clarify.

We are presently doing a study on these two specific uncertainties.

* Theory comparisons

- I am not a big fan the way the DPS and hadronization contributions are presented in the theory comparison: it is unclear. I would simply make a comparison with the MCFM prediction (as this is the number that the program gives), and then add the DPS & hadronization contributions separately for people if they trust them sufficiently that they actually want to use them. (BTW: with the current level of precision they actually hardly matter....)

Answer: yes we provide this as a separated number

- I would propose the authors to get some inspiration from e.g. the CMS Z+b-jets paper in the way to present the results.

we will check

* Conclusions Conclusions are not there?? Could be nice to add some philosophical comments for the reader. E.g. how this measurement compares with the existing CMS/Atlas V+HF results, and what we should learn from the agreement of the measured cross section.

For short PAS we stay without special section "conclusions", will be certainly added to the paper

https://indico.cern.ch/event/370975/session/0/contribution/2/material/slides/0.pdf

Comments/Questions

• Is there a dependence of the results on Wbb FS? Should we associate a systematic difference in on the MC efficiency due to the 5FS vs 4FS difference? With the current statistics it is likely impossible to disentangle a systematic and a statistical component, think whether to use conservatively the whole difference as a systematic uncertainty.

Answer: We see similar results in both flavour schemes, though in both the electron and muon channels, we see a 3% difference in efficiency with 5 flavour samples more efficient, calculated as (effi_5 - effi_4) / effi_4. We can easily add this as a systematic uncertainty if agreed that this is appropriate.

• What does justify the choice of the multijet ttbar control sample wrt the dieletion one? This is an important point in the analysis, and should be explicitly addressed in the document. Please clarify where it is or add it.

Answer:The multijet phase space is more similar to the signal region than is the multilepton phase space because jet multiplicity isn't nearly as well modeled as lepton multiplicity. It is consistent then that the SF calculated in TTbar region fit is more similar to that calculated in the multilepton phase space.

• W+jets background is computed subtracting the b component; is using 5FS for background and 4FS for signal creating some bias/inconsistency in the analysis?

Answer: We see compatible results when fitting against 4 and 5 flavor schemes.
Including the systematic mentioned in Q1 nonetheless compensates for any small bias.

• How are taus considered? If they are subtracted as background it should be explicitly documented. It needs to be specified in Section 5 while discussing the definition of the fiducial cross section.

Answer: At the gen level, taus are rejected. At the reco level, they're only referenced in the (gen level) splitting of w-jets sample into different components (Wl, Wc, Wcc, Wbb, W(tv)). [updated in PAS, AN]

• How is the cut on the lepton-jet distance treated in the definition of the fiducial cross section? Is it used (as it seemed using the discussion)? And which is its effect at generator level? This should be clarifiedin the definition of the fiducial phase space in Section 5.

Answer: Jets within deltaR<0.5 of an isolated lepton are rejected (just the jet, not the entire event). [updated in PAS]

• Systematic uncertainties: average JEC/JER/pileup among channels, no reason to have differences between leptons (unless explained by different background)

ok, removed

• B tagging scaling factor: given the observed discrepancies in the top control region, are they appropriate/correctly used? Is the systematic uncertainty associated to then appropriate?

Answer: The answer to the first question is yes, and this can been seen in Appendix C; when we look at the phase space most similar to that used in the published ttbar-multilepton analysis, we see agreement to ~1%, and it is only when we start changing to our cuts that the disagreement mounts. The level of agreement is affected not only by the b-jet multiplicity, but also by the jet veto and transverse mass requirements. The global SF that we apply can then be thought of as a correction for all of these effects and not just for the b-tags.
As for the appropriateness of the uncertainty on the SFs, this is harder to answer. From the combine fit, we see a pull of about 1 sigma on the b-tagging uncertainty, so the end result of the fit still stays within the uncertainties as quoted.

• check the number of events in electron and muon channels vs signal strength - there is 3% difference in the signal strength, and no difference in the number of Wbb events wrt the expected

Answer: We checked the numbers and the fit procedure and still see no bugs. The cross section is obtained from the data yield subtracted of all backgrounds and divided by (luminosity*efficiency*scale): 1051.8/(19767.0*0.0699*1.142) = 0.667 for electrons; 1215.3/(19783.0*0.0820*1.120) = 0.669 for muons. The same level of agreement is obtained by performing the computation with signal strength and scale applied on top of the Wbb pre-fit sample: (729.8*1.366*1.142)/(19767.0/0.0699/1.142) = 0.721 for electrons; (872*1.328*1.120)/(19783.0/0.0820/1.120) = 0.714 for muons. Eventually the discrepancy in the signal strength between electrons and muons is reabsorbed after applying the efficiency correction.

• leave only post fit plots in the PAS
• there are too many plots in the PAS, please leave only important
• the style of the plots - please use CMS standards
• in the PAS please use 0.67 +- .... etc e.g. 2 significan digits should be enough
• combine table 1 and 2 in one
• section 4.1 belongs to AN, this is not for external consumption please define you base line control region and use it.
• the signal strenth for Wbb is also only for internal use, remove from the PAS if it is not used in the cross section calculation

ok