-- AustinBaty - 2019-08-05

HIN-19-003 preapproval

ToDo: Address all comments bellow on AN + paper, and raised during the preapproval presentation. The answer to the questions (if they require additional studies etc) have to be documented in the AN (body or a dedicated Appendix).

Preaproval presentation

***** Slides with ToDo from the discussion

S9-10: Q2: every bin is averaged for every other kinematics. need to check the model dependence on this. also inside 1 sample, variations because of npdf-s and scale

We have checked the dependence of the efficiency on the spectral shape by reweighting the MC to data and comparing with the unweighted result. This weighting corresponds to around a 20-30% difference in shape peaking at about 10 GeV. The effects of this reweighting are found to be negligible on the efficiency. This is most likely because the efficiency is binned as a function of pt (so only differences in shape that are smaller than the bin size are relevant), and the reco efficiency is not found to have any strong pt-dependence. The pdf and scale variations do not produce a large enough difference in the spectrum to affect the efficiency calculation. Plots of the efficiency difference between the weighted and unweighted sample are added to an appendix in the AN.


-- makes still sense to show the signal MC --> add to this plot. keep the stacked, just add the signal, reco level.


---- also, add horizontal error bars for bin width


-- for the darker blue, what's the charge flipping for electrons? (charge mis-identification question from the AN)

The charge-swapping for electrons is around a 0.5% effect in MC (although it gets larger at forward rapidities). The same-sign distribution is now corrected for contributions from real Z bosons producing same-sign pairs, and an 0.5% uncertainty related to potential difference between the data and MC charge-swapping rate has been added to the analysis. More information has been added in the AN under the section that discusses the same-sign background subtraction. The charge-swapping effect, although non-negligible, is fairly small in relation to the electron reconstruction uncertainties, so this does not significantly affect the conclusions of this analysis.

-S19: what's on the left, with the dip? Is there a fluctuation, or is a failed fit? Is there any reason for that dip? If just a fluctuation, should consider a fit to use for corrections, and not the point.

From the EGamma mini-pog:
The "dip" has much more statistics then very first and last bins, maybe only the very last bin is fluctuating. The error bars on the SFs have only "statistical" component. The systematic one will be added for the final version.

At this point we don't have any reason to believe these points are incorrect.

S23: why the HF uncert is signif different for muon than for electron channel? - sthg with EM contamination? could it be different in ee and mumu?

We are not sure how any EM contamination in the HF could cause such an effect for the different channels. This bin of the analysis has fairly low Z yields, so we believe this difference is partly caused by statistical fluctuations when doing the variation of the centrality table. In the muon channel we see a difference of around 8% between variations, but the statistical error of this bin is on the order of 10%, so this is still not the leading uncertainty in this bin. If anything, we could be slightly double-counting the statistical uncertainty with the current procedure, but I am not aware of a better way of separating which part of this 8% comes from statistical and systematic effects, so we believe the current procedure is reasonable (although perhaps a bit conservative).

** (A,pT) cut:

------ should be customized for muon and electron channel separately; if too small counts, at least cook-up some sanity check that it’s working OK for both

------ add additional material to convince the remaining contamination is negligible (ideally a study on a dedicated EM MC sample, if not the plot from the LbyL AN might be enough)

--------- cf also FSQ-16-012 plots of QED ee

A dedicated study using STARLIGHT MC has been done to optimize the A,pT cut separately for both channels. We found that the A cut was placed in a good position, but the pT cut was a bit too agressive, so this has been relaxed a bit. Additionally, the systematic variations are slightly less arbitrary now; we choose a working point that gives 90% background rejection, with systematic variations to 80% and 98% background rejection. The efficiency loss from these cuts is on the order of 0.3% or less. An appendix with detailed plots has been added to the AN.


-- could the centrality dependence made flat by some shift on EHF? how large a shift would be needed?

A quick study in the muon channel indicates that the distribution can be made reasonably flat by subtracting off around ~40 GeV of pT from the HF sum. This corresponds to a relative reduction of the HF sum by a fairly large amount in peripheral events (15-30% of the total hfSum in the 70-90 centrality bin). This study has been added to an appendix of the AN.

-- can you look at the ZDC distribution (if you look at centrality bias)?Calibrations are available, and data can be looked at.

An appendix showing the result of using the preliminary ZDC calibration has been added to the AN. In general, the trends vs centrality looks flatter in the 30--70% centrality range, but there seems to be strange jumps for the 20-30% and 70-90% centrality bins. My understanding is that the centrality group is still working on fine-tuning the ZDC calibration and also adding ZDC simulation to the reconstruction, so for now we would like to proceed with the HF calibration.

S28: w/ 8TeV data pp data-- might be useful to unfold electrons vs rapidity (which will impact the results on S28). saw from pp studies, that the effects vs y was much more important

We have now looked at the resolution effect for the electron channel. In general, the resolution in Z rapidity is <0.02 while the bin width we are using for the measurement is much larger at 0.3. Checking migration effects on the spectrum indicates that these effects are quite small, and siginificantly smaller than the uncertainties for the electron channel, which are dominated by the electron TnP. Based on this, we believe that unfolding for rapidity is unnecessary. An appendix has been added to the AN detailing these studies.


1) compute the lumi using the crab report json file

This was already done for the result shown in the pre-app presentation

2) why ele eff so low? Check main source of inefficiency

The electron reconstruction efficiency (i.e. matching to a track) has a strong centrality dependence, with the barrel efficiency being 80% and endcap efficiency being only 50% for 0--10% events. (see https://twiki.cern.ch/twiki/pub/CMS/HiHighPt2019/reco_efficiency.pdf) This efficiency gets squared for Z bosons, leading to max efficiencies of <65% in the barrel and <25% in the endcap for central events. There are also additional inefficiencies on top of that due to the trigger (relatively small effect) and the electron ID selections. We are choosing the 'loose' working point which I believed is tuned to keep 80% of the remaining electrons, so there is another factor of 0.64 on the Z efficiency from this selection. There are also the vetoed regions of the detector to consider as well. I believe it is not surprising that the electron efficiency is significantly below the muon one.

3) bin-by-bin unfolding -> not good. do matrix inversion


4) - smearing correction -> assumes pt spectrum in MC. moving to matrix inversion. should also compare data and MC distributions, and reweight MC to data when producing the response matrix.

----- - need to account for SFs and energy scale when filling the response matrix

The SFs and energy scale corrections/energy scale smearing (for electrons) is taken into account when filling the response matrix. We also evaluate the effect of a difference in shape (very small effect) when reweighting the data spectral shape to MC, and the statistical uncertainty of the response matrix. The stat. uncertainty is the most impotant effect here for the unfolding uncertainty.

5) Discrepancy between electron and muon channel: investigate cause, solve. Ideas:

-- centrality dependent tnPs

We will update to the latest TnP once they are final (should be soon). This is expected to help the agreement between channels (muons should go up, electron uncertainties should go up), but I am not sure if this fully explains the discrepancy between channels. We will keep investigating...

Paper v0

  • General comments

    • Many jargons

    • Many typos and weird English

    • Quite some work is needed to fix the paper

    • Many references missing

    • if you say ‘help constrain models’, you have to add those models that are constrained on figures

  • Final message a bit too strong

  • Line 23: what is "hydrodynamic flow coefficiency (v2)"

  • Line 65: What generator for ttbar and W jet?

  • Line 75: What do you mean by "expected shapes"

  • Line 77: what scaling

  • Line 94: probability of what

  • "Two sections of the ECAL still have poor performance" => you mean transition region?

  • For the same sign number 1% is quite big, judging from your plots in AN. Can you quote something smaller?

  • Fig. 1: same comment as for AN→ this is meant as a S/B plot. This makes sense, only if you prove that the MC is trustful

  • CCLE fixes the paper to a better shape, after the technical issues pointed for the AN are fixed

AN v3


  • Please include more kinematic plots of leptons for reference (like comparison between data and MC)

    These have been added.
  • Please provide Z y resolution and show that migration effect is small due to pointing resolution

    An appendix has been added to the AN with these studies. See our response to a similar comment above for a more detailed discussion.
  • A table of MC-based based background yield (together with Z yield) would be useful. Why 10% variation is enough?

    The tables have been added to the AN. The MC-based backgrounds added together only account for 0.4% of the total yield in each channel, while the TnP uncertainties are at least 1.5 (3.0)% in the dimuon (dielectron) chanenls. The 10% number is fairly arbitrary and has been increased to 20%, but this uncertainty will be negligible for this analysis unless the value is taken to be unreasonably large (such as 200%).
  • How do you estimate inefficiency from the A+PT cut? How do you correct for it?

    The A and pT cuts are also applied in MC when calculating the Z efficiency, so these effects are already naturally taken care of by the efficiency correction.
  • Background: have you checked the charge flip? (meaning leptons that have been reconstructed with the wrong charge, so it will end up in the SS while they should be in OS; and vice-versa)

    Charge-flipping for electrons has now been accounted for (see the response to a similar comment above for more details).
  • Z boson reconstruction efficiency could be model dependent based on Z decay kinematics in MC.

    • Check the same map with another MC

      We now reweight the MC pt spectrum to better match the data, so that we observe good agreement between the two. The differences caused by this reweighting procedure were found to be negligible because the efficiency table is binned in both y and pt. We believe this check is sufficient to convince ourselves that the Z kinematics do not strongly affect the calculated efficiencies when compared to the effects of the TnP corrections.
    • What does the selection criteria include? (line 167)

      These are the same criteria enumerated in the tables from previous sections on the electron and muon reconstruction. I have added a statement clarifying this.
    • The efficiency map is kind of low in statistics. Please improve, or include an uncertainty if smoother map won't change result

      The efficiency table has been slightly rebinned, and the statistical uncertainty of the MC efficiency is now propagated as a new syst uncertainty. In general, it is not the leading uncertainty in this analysis, although it can be as large as 1.5% for the high-pt points.
  • Bin-by-bin PT resolution correction should be used only for negligible bin migrations ( https://twiki.cern.ch/twiki/bin/view/CMS/ScrecUnfolding) which is not your case. Please use unfolding (matrix inversion should work).

  • Missing:

    • you are missing an acceptance correction: to correct for the effect of single lepton cuts, to all Z bosons measured in |y|<2.1 (or 2.4 for muons), for easier comparison to theory, other experiments, and other CMS results.

      This is now applied.
    • you are missing uncertainties that can affect the acceptance correction. The main effects supposed to be affecting the Z production, are nPDF effects and isospin effects. In addition, your acceptance will be different depending on the input gen spectra.

      An uncertainty is now evaluated, although it is not significant compared to other sources of uncertainty. See the items below for more details.
  1. have to try different nPDFs and see the difference in acceptance, and

    We have varied the nPDF used (EPPS16) to all the different copies in it's nPDF set, as well as changed it to nCTEQ15 and derived a systematic uncertainty on the acceptance based on these variations. In general the difference between all the EPPS16 copies and the different nPDF type (nCTEQ15) to the nominal result is very small, and the TnP uncertainties are much larger. PDF scale variations were found to be negligible with respect to other effects.
  2. also to cook-up a cocktail that gives you the isospin effects (pythia has p-p only, but PbPb has p-p, p-n and n-n collisions). (this is fact should be what the nominal result is based on)

    We now use a set of pdf weights which reproduces EPPS16nlo+CT14nlo for the nominal result. This weighting also naturally include isospin effects, so no such cocktail is needed.
  3. different shape all together: in the past, the nominal shape was from POWHEG (NLO generator)

    We have compared the values of the acceptance before and after applying a pt-dependent reweighting to the data spectrum. The difference is found to be negligible for this analysis. The study as been added as an appendix to the AN.

L97 Is there a reference (study) to the fact that OfflinePrimaryVerticesRecovery module does nothing in 0-90%

From the run twiki: https://docs.google.com/document/d/1xPfdJrnKJXPAMqdl-YcJGPjvDooFoK-UgYIYo13bodU/edit# there is a link to this presentation (by me)


The appropriate plots are on slide 4. Note that I was not claiming that it does nothing in 0-90%, only that the expected effect is fairly small (actually a decent number of vertices are recovered in the 80-90% range, but we only measure 70-90% here). In any case, I have removed this sentence about the expected result from the AN to avoid confusion; the recovery module is applied so these events will be kept.

L112 remove 2nd 'the'


L119-123: the numbers are available already--> check doc with calibrations (and update)

The numbers were updated for the pre-app presentation, I just forgot to remove this line from the AN. Has been fixed.

Fig. 2--70-90% bin: the en corrections for electrons doesn't seem to be OK (the peak is still shifted). What is going on?

I believe this is just the result of stat. fluctuation in this bin, as there are fairly few counts. All other centrality bins look solid for the energy corrections now (in particular the 50-70% bin which has much more stats, and where occupancy effects should still not be too big).

Fig. 3: need a coarser binning for efficiency (there seems to be large fluctuations)

The efficiency table has been rebinned for the high-pT region. Furthermore, the statistical uncertainty from the MC efficiency table is now properly propagated to the systematic uncertainties in the final result. This negligible for nearly all data points measured here, but is non-negligible for the most forward rapidity region and highest 2 pT data points in the electron channel. In general, we believe that statistical accuracy of the efficiency table is sufficient given the precision of the TnP SFs of this analysis.

L176-177/Fig. 5: this is a very big difference in efficiency before the 2 channels and with very large uncertainties for |eta|>2 for electrons. What is the main source of inefficiency for electrons?

I address the difference between the electron and muon channel efficiency above in the 'general comments' section. Note that although the two bins w/ |eta|>2.1 have very large uncertainties, they are not used in the analysis because we are cutting on the lepton |eta|<2.1.

L179, 237: add a \clearpage before starting new sections


L192: which 'this correction'? Clarify how the correction was calculated

This has been reworked because we now use unfolding.

Sec. 7.3

Do I understand right what is done? Looking in data only (with no MC x-check), you subtract from the dileptons candidates, the SS distribution, and also the ones that fail an (A,pT) cut?

-- if you do this: you have to correct (using MC) for good Zs that you removed in the process, for leptons that had the charge wrongly identified, etc--> you have to show that DAta and MC distributions in these variables match

Yes, you understand correctly. We already at pre-approval correct for the Z's lost to these effects. A comparison of the SS distribution in data and MC is now in the AN concerning the charge-swapping study. Some difference is observed (probably because data has legitimate same-sign background while in MC it is purely from charge-swapping), but this is covered with a modest systematic uncertainty of 0.5% now.

Sec. 7.3.1

L212 and beyond: so this process is an UPC-type of event--> so we expect to be in peripheral bins. That you convinced yourself it's like this, it's good.

-- just so we are clear, in your efficiency calculation, you included this selection, meaning your eff corrections, accounts for good Zs that got killed by this cut aimed only at UPC Zs?

Yes, the efficiency correction includes the cuts on Acoplanarity and pT.

Fig 14->20: this seems like a perfect match all over the board... This is based on the way you do the normalization (L247-249). You get the fraction from MC, then you take all data, and each color corresponds to frac_mc*Data. → so the plots are all normalized to 1 (Data).

The yellow histogram will by definition match the black data because it is the signal fraction of data stacked onto the backgrounds we calculate. This has been fixed to compare to MC instead.

-- but while the idea here is to show that the remaining/contaminating bkgd is small, you can not really use these plots alone to make the point. Because you are missing a set of stack plots, that show that the MC can be trusted (see AN_2014_310_v10, Zmumu in pPb, or even AN2017_248_v8 DrellYan in pPb)

A set of stacked histograms showing the contribution of each background process and the signal MC, compared to data, is now shown. In general the agreement between data and MC is quite reasonable.

Sec. 9.4

L307: how you decided on the 10%?

There is not a strong motivation for 10%, I simply chose a number that seemed fairly 'large.' However, this uncertainty is tiny (~0.01%) and do not believe the final analysis uncertainties will be sensitive to this choice. We have changed this to 20% to be conservative.


  1. how is the choice made?

    This choice has been changed to correspond to variations (around the 90% bkg rejection point) that are the 80 and 98% bkg rejection working points.
  2. how is the uncert. propagated? you take the max of the 2 variations or what?

    The max of the 2 variations is taken as the uncertainty. This is clarified in the AN.
  3. aaah, what is the correlated uncert. relevant here? (note, yes, it removes/add in the same time, but different amount; so the effect on each channel is different)

    Have changed this to be considered uncorrelated now.

Sec. 9.6

L326: why 5% is enough?

L328: … is the existing resolution? (I don’t get the English here)

L328 which ‘both of these corrections’?

L329 difference wrt what?

L329 which correction?

Note: This entire section (9.6) has been reworked because we swapped to an unfolding procedure.

Fig. 23: please clean-up

-- what’s the deal with 20-80%?! please remove.

We were trying to pick a bin which could have a 'maximum' v2 because of initial state geometry. Given that this can be more-or-less inferred from the other centrality results which are all compatible with 0, I removed this bin.

-- having bins (like 50-90) that one can not use to draw any conclusion are not useful. Suggest to combine with 30-50

I disagree that one cannot use the 50-90% bin to draw conclusions. The charged hadron v2 in bins from say, 50 to 80%, maxes out at around 0.2, while we measure -0.01 +/- 0.02. Although this result is not as precise as the inclusive bin, it is still precise enough to have significant comparisons to other particle types.

L377: the ATLAS result is likely corrected for acceptance, while yours, based on what you describe, is not.

The ATLAS cross section shown here is actually not corrected for acceptance. The paper clearly states that this is the yield for a lepton selection of |eta|<2.5 (not Z rapidity of |y|<2.5), so this is a fair comparison (except for the small difference between |eta|<2.4 vs 2.5). It is true that in this figure we have not applied an acceptance correction yet (in order to keep the comparison fair).

Fig. 24:

-- what’s going on with the 40-50% ee yield?

This looked like a stat. fluctuation to me. Note that between freezing and the pre-app we changed an electron cut to be updated w/ the mini-POG suggestions and this point moved up and is slightly more in line with the other points.

-- i would combine 50-70 w/ 70-90: make the point much clear with the combined bin (which should have higher precision)

I am strongly against combining these two bins. Because of NCol scaling, the precision of the 50-90% bin would be driven by the 50-70% result and the 70-90% data will get 'hidden'. Furthermore, this result is still reasonably significant (nearly 2 sigma away from the 0-90% result) and supports the interpretation of a centrality bias, which could be an interesting angle to pursue when writing the paper.

-- the 0-5% mumu result is consistently smaller than the neighboring bin. it should not be. Are you over subtracting smth, under correcting for smth? This has to be understood

This is already understood as part of the muon reconstruction and the muon TnP is being updated to compensate for this (per my discussions with Andre). We will update to those TnP corrections when available.

L381: no ‘enhancement’ is expected on physics grounds. what you see is a discrepancy, which should be investigated and understood.

We did not mean to claim this was physics, this has been reworded.

Figure 25:

-- what happens to boxes on the blue on the right?

This was a bug in the plotting macro and was already fixed in the pre-app presentation

-- fix caption (it’s vs y not pT)


Fig. 26: fix caption (it’s vs pT not y)


Other changes

Based on iteration with the EGamma mini-POG, we removed the L1 matching requirement for electrons because the L1 matching is not well-produced in MC and the L1 trigger seems to be very efficient in the region of study. This causes the central electron channel yields to go down about 3%

An additional TnP SF was added for the electron reconstruction (matching of the Supercluster to the electron's track). The stat uncertainty of this also significantly increases the systematic uncertainty for the electron reco efficiency.

An additional TnP SF was added for the electron HLT trigger efficiency (this is a very tiny correction because the HLT trigger was quite efficient and the single electron trigger gets 2 chances to fire per Z)

Some discrepancy in the Nmb calculation from the GO group was spotted between different MB PDs, so we are now using an new number of ~11e9 (up 15%) MB events. This number seems to be much more consistent with the expectation from the lumi measurement + Glauber cross section. We will update this further once the numbers are final. This fixes the TAA-scaled yields in PbPb being higher than the pp cross section.

Lumi numbers were updated to the final ones.

Some other changes based on feedback from the conveners (AN v4)

3. Even though things are within ~2 sigmas between electron and muon, there seems to be a slight trend in the e/mu difference as a function of centrality. Please double check with object groups to see if there is anything that can be refined to improve this.

With the upgrade to the latest official electron TnP results, I no longer think this is the case, as the agreement is now better. We will still of course endeavor to make sure the objects are calibrated as precisely as possible.

4. AN v4, around line 263: please expand a bit more why 0.5% is a good choice.

A short explanation has been included. This is basically the difference in the data and MC same-sign contributions under the Z peak at 91 GeV .

5. Please include some more material on the quality of unfolding in the AN - for example, some checks with refolding the unfolded distributions.

Some material has been included. In general, these comparisons look very robust because we are using a matrix inversion procedure, so it can be reversed exactly except for some edge effects at very high-pt.

Edit | Attach | Watch | Print version | History: r50 < r49 < r48 < r47 < r46 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r50 - 2019-10-01 - AustinBaty
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback