Z' Statistical tools

Working Twiki on the use of the standard Z' statistical tools for limit setting and significance calculations in CMS. The focus of this Twiki is on the use of these tools for the Z'->ee analysis

Master reference (G. Kukartsev Twiki):

https://twiki.cern.ch/twiki/bin/viewauth/CMS/RooStatsZprime

Getting started on lxplus

The Z' statistical tools require a well-configured recent version of the ROOT package (>=5.27), with ROOSTATS and PYROOT support. There are several ways to achieve this. I found the simplest method was to source a recent version of CMSSW (>= CMSSW_4_1_X) which links to the required version of ROOT. Note that the standard Z' statistical tools Twiki generally refers to code running on the FNAL lpc cluster

Worked example using CMSSW_4_1_4

Set up ROOT version + ROOTSTATS + PYROOT

> cd cms_releases/CMSSW_4_1_4/src

> cmsenv

Check out Exost statistical package

> svn co svn+ssh://[your_cern_login]@svn.cern.ch/reps/exost

> cd exost

Source setup script for CMSSW installation

> source setup/cmssw_setup.[c]sh

> cd workdir

Check out the Z' code

> setenv CVSROOT ':gserver:cmssw.cvs.cern.ch:/cvs/CMSSW'

> kinit

> cvs co -d twobody UserCode/GenaKukartsev/twobody

> cd twobody

Making a workspace

> exost -a workspace -c dielectron_ratio.cfg

Note that when using the ROOT installation linked to CMSSW, you may see the following harmless messages when executing this command:

Show... Hide

> cp myWS.root ws_dielectron_ratio.root

Check configuration of the workspace

> exost -a mcmc -c dielectron_ratio.cfg

(success: the code should compile and execute without crashing)

A quick test of the limit setting code:

> root

> .L twobody.C+

> limit("dielectron","observed",1000, "test", 1, 10000, 100, "")

Interpreting dielectron_ratio.cfg

Input file: must have branch named "mass", "weight" branch is optional

systematics:

mass_kappa = [1.01]; fix; -> 1% error on energy scale

Running the limit setting code:

limit("dielectron","observed",1000, "test", 1, 10000, 100, "")

limit("dielectron","expected",1000, "test", 2, 10000, 100, "")

Simple script to compute observed limit for a given Z' mass:

Show mass.C... Hide

Output files and limit plots

Running the above macro will produce an output root file (.ascii) with:

a) mass; b) limit on xsec ratio

(one per job)

plotting file expects a concatenated list

> ./merge_ascii.py {input file list} [to stdout]

> ./merge_ascii.py -f {output file>}  {input file list}

> ./plots.py -p {output plot name} -d  {input dir}

> ./limit_plots.py see here for definition of input ascii file

{plotter.py is worker file}

Significance calculation

What: "code to calculate local significance and significance corrected for trials factors, for the Z' dilepton search in CMS". Incorporated into the twobody.C macro.

Principle: compute likelihood ratio between S+B and Background-only hypotheses. Find value of M_leplep that maximises likelihood ratio

see:

https://twiki.cern.ch/twiki/bin/view/CMS/RooStatsZprime#Significance_and_trials_factors

Example:

> root

> .L twobody.C+

significance("data", "dielectron", "", 600.0, 2000.0, 10.0, 1, "observed", 200.0, 2200.0, false)

Trials factor: supply, as arguments, min and max M_leplep values (in this case 600 and 2000 GeV). M_leplep step size for the trials factor evaluation is the 5th argument (in this case 10 GeV). Sixth argument is number of pseudo-experiments to generate. Set to 1.0 for observed significance.

Local significance: set min and max M_leplep values to the same value: M_obs.

-- DavidPetyt - 27 Mar 2009