Running GlaNtp

Introduction

We plan to run the GlaNtp package with an example from Alistars' ttH analysis. This will run the analysis all the way to the limits. Then we will make the changes in another page to personalize it for the ZH analysis.

ssh -Y ppepc137.physics.gla.ac.uk

Then open an xterm terminal.

xterm

Prepare the scripts

The scripts and configuration files that we will use are independent of the GlaNtp package and therefore do not come with the package. We will use the latest versions Rick made for running Alister' ttH analysis. I store them in

cd /data/atlas13/abuzatu/forTutorial
ls

As you see, there are several text files called steering files. Also there is the file "genemflat_batch_Complete2_SL5.sh" which we will use to run the job. There is also the file "setup_glantp.sh" to setup the things needed for GlaNtp. There is also "runlist.sh" to contain an example of the command to run (better to copy the command from there and run in the console).

We want to work in "/data/atlas13" as this was decided to be where the final analysis for ZH using the common framework will be.

mkdir /data/atlas13/$USER
cd /data/atlas13/$USER
mkdir batch
cd batch
cp -r /data/atlas13/abuzatu/forTutorial/* .

Create the job file

Before running the actual job, we need to create a job file. We copy the example from "runlist.sh"

./genemflat_batch_Complete2_SL5.sh 20000 400 1.04 tth 120 120 6 00-00-44 /data/atlas07/stdenis/v16-r13/bjet2 adrian.buzatu@glasgow.ac.uk srv001 abuzatu

The meaning of the inputs to the scripts are the following.

• 20000 is the name you choose for your run for the current job. The following time you run with different options you can call it 20001, etc.
• 400 is the number of the subjob, it is ignored completely at the moment, but it is designed if in the future we want to introduce subjobs, for example each Higgs mass to be a subjob
• 1.04 is the luminosity of the analysis, to be used in the calculation of templates and to draw on the plots
• ttH is the name you choose from your analysis
• 120 is the first mass of the Higgs boson
• 120 is the last mass of the Higgs boson (in this case we only run for one mass, that of 120)
• 6 I do not remember what it is, but I think it is not used
• 00-00-44 is the version of the GlaNtp that we are running
• /data/atlas07/stdenis/v16-r13/bjet2 the folder of the input flat ntuples for our analysis (in this case it is ttH signal region, that of two b-tagged jets, the folder has 1.2 GB)
• adrian.buzatu@glasgowNOSPAMPLEASE.ac.uk is your email address
• srv001 is the server for the Neurobayes machines (in the end we may all want to use Neurobayes, at the moment we are running TMVA though)
• abuzatu is your username

You will get the following error.

You must do: source setup_glantp.sh

So we have to setup the correct version of GlaNtp. First check that you have defined what version of GlaNtp you are working on.

echo $GLANTP_TAG

If it is not defined, use the one that is installed

export GLANTP_TAG=00-00-49

Then setup the package.

source setup_glantp.sh $GLANTP_TAG

We can check that our setup succeded Our code is in the folder with this the path (saved also as an environment variable)

echo $GLANTPDIR
ls $GLANTPDIR

Now we can run again our command.

./genemflat_batch_Complete2_SL5.sh 20000 400 1.04 tth 120 120 6 00-00-44 /data/atlas07/stdenis/v16-r13/bjet2 adrian.buzatu@glasgow.ac.uk srv001 abuzatu

Now it will fail again with this error, thanks to a nice verification we have added lately.

The following variables are not equal, so we will abort: GlaNtp-00-00-49 vs GlaNtp-00-00-44

./genemflat_batch_Complete2_SL5.sh 20000 400 1.04 tth 120 120 6 00-00-44 /data/atlas07/stdenis/v16-r13/bjet2 adrian.buzatu@glasgow.ac.uk srv001 abuzatu

So now we modify the command to use the "$GLANTP_TAG" instead of "00-00-44"

./genemflat_batch_Complete2_SL5.sh 20000 400 1.04 tth 120 120 6 $GLANTP_TAG /data/atlas07/stdenis/v16-r13/bjet2 adrian.buzatu@glasgow.ac.uk srv001 abuzatu

Now we have a folder called run20000, which has all the things our job will need.

cd run20000

The key file is the job file, the one that we will be able to execute to run our job.

ls tr20000.sh

We are now ready to run. We can run either interactively or in the batch system.

Running interactively

This is the simplest way and it is recommended for testing. First we make our job file executable, then we execute it.

chmod +x tr20000.job
./tr20000.job

Below we will run interactively one step at a time, so that we can learn. It will take a bit of time, so this is why in the future we will run all the steps in one job in the batch mode, as shown in the chapter below.

emacs -nw tr20000.job
# Flags to limit the scope of the run if desired
Computentps=1
DoTraining=1
UseTMVA=1
DoTemplates=1
Dostackedinput=1
DoStackedPlots=1
DoFit=1

All of these show things to do, except UseTMVA which shows how things are done. In all the steps keep UseTMVA as 1, then set all to zero, except the first step, Computentps. You should have something like this.

# Flags to limit the scope of the run if desired
Computentps=1
DoTraining=0
UseTMVA=1
DoTemplates=0
Dostackedinput=0
DoStackedPlots=0
DoFit=0

Then you run again

./tr20000.job

The output of the Computentps stage is used as input in the DoTraining stage, so now we set off the previous stage and set on the next stage.

# Flags to limit the scope of the run if desired
Computentps=0
DoTraining=1
UseTMVA=1
DoTemplates=0
Dostackedinput=0
DoStackedPlots=0
DoFit=0

Then you run again

./tr20000.job

The output of the training stage is used as weights for the stacked signal and background which are inputs to the neural network. Which is the order? First we do the templates DoTemplates=1 only and set the rest to zero. Once we have the templates we can create the stacked plots for the inputs as well as the fit (limit) if we choose a variable from the inputs. If we want the NN output stack plots we do the stacked plots for the ouput and we can run the limit on that variable.

# Flags to limit the scope of the run if desired
Computentps=0
DoTraining=0
UseTMVA=1
DoTemplates=1
Dostackedinput=1
DoStackedPlots=0
DoFit=0

Then you run again

./tr20000.job

Then we compute the neural network output and create the stack plot of that variable.

# Flags to limit the scope of the run if desired
Computentps=0
DoTraining=0
UseTMVA=1
DoTemplates=0
Dostackedinput=0
DoStackedPlots=1
DoFit=0

Then you run again

./tr20000.job

Then we do the limit calculation using the MCLimit program, also known as Tom Junk's fitter.

# Flags to limit the scope of the run if desired
Computentps=0
DoTraining=0
UseTMVA=1
DoTemplates=0
Dostackedinput=0
DoStackedPlots=1
DoFit=0

Then you run again

./tr20000.job

Running in the batch system

First we have to ssh to a special machine here at glasgow.

ssh ppepbs

Then we go to exactly the same folder where we worked above. The "/data/atlas13/" is also mounted on the pbs machine. The output of the jobs will be saved in our same batch folder as if we were running interactively! It is quite comfortable!

cd /data/atlas13/$USER/batch/run20000
qsub tr20000.job

Typically we run the entire sequence of steps by using the default configuration, but you can still set some steps to zero and submit the job to the queue. In fact, you can still run interactively on this machine, if the step is really quick and you do not want to wait in the queue!

We get the output that the job is appended to the list of jobs

168559.ppend01.physics.gla.ac.uk

We can see the list of jobs with the command

qstat

We can kill a particular job with the command

qdel 167559

It will take about 2 hours 45 minutes for the job to finish. When that happens, you will receive an email.

Reading the output

All your inputs were in the folder tr20000. Also all your outputs will be in the same folder. Since most are plots and text files, it is easier to read them from an internet browser. So let's link our batch folder to our public_html at Glasgow. We link the batch folder and not tr20000 because later we might want to run another run tr20001 and we will not need to do the linking again.

cd
ln -s /data/atlas13/$USER/batch .

Now just point your browser to this path, where USER is your username (abuzatu in my case)

ppewww.physics.gla.ac.uk/~USER/batch/

Navigate to the folder of the desired run, like tr20000.

The first .html file is the one showing the stacked plots of background, signal and data in all the variables we want, namely the file "InputDist_run20000_120.html". Also the systematic uncertainties is shown over the backgrounds. Since last week the MET variable has been fixed, the titles for the x and y axis appear automatically, etc. Typically these variables can be used as inputs to the neural network, which is discussed below.

The actual non-log plots are in the folder "stackedinput/StackInput/tth120/FlatStackInputSteer/" and the log plots are in the folder "stackedinput/StackInput/tth120/FlatStackInputSteerLog/". The name of the output folder "tth120" was contructed automatically from the input of the "genemflat_batch_Complete2_SL5.sh" script where we chose the name of the analysis to be "tth" and the name of the mass point to be "120". The actual run produces .eps files. Then Rick added a script to make .gif from .eps and then make the .html file for that.

The second .html file is the one related with the neural network training "TMVAPerf_120_Run20000Job2.html". You will scroll down and see plots of background vs signal rejection, correlation matrices for background and signal, NN score distributions, training error, and finally the many neural network input distributions. The variables will be listed in alphabetical order, which is also the order they are sent to the neural network. This is thanks to a sort command in our "genemflat_batch_Complete2_SL5.sh" file. The order of variables does not matter in TMVA, but is crucial in Neurobayes, which we will use later on (but was not used in the current run).

The neural network output plot is in the folder "stacked/Plots_120_TMVA_Lum_1.04/". For now Rick has not made a script to make .eps to .gif and make a webpage. But we have here two .eps files, one for log and one without log: "FlatStack_1.eps" and "FlatStack_log_1.eps". You can convert them to .gif yourselves and then read them from the browser.

cd /data/atlas13/$USER/batch/run20000/stacked/Plots_120_TMVA_Lum_1.04
convert FlatStack_1.eps FlatStack_1.gif
convert FlatStack_log_1.eps FlatStack_log_1.gif

A nice thing about these plots is that they also contain a zoom in the signal region (high NN score).

The limit calculation is shown in the file "templates/fit/out_120.log". At the end of the file you see the limits. The first one is the observed limit "Bayesian s95 limit" and the following ones are the expected limits (median, +/- 1/2 sigma).

Done fill pseudoexperiment distn with 1000 entries
Bayesian s95 limit: 63.3896 +- 0
Expected limits: 
120 -2 sigma: 10.0977 NSig= 1.80901 NBkg= nan NData= 1258
120 -1 sigma: 16.0314 NSig= 1.80901 NBkg= nan NData= 1258
120   median: 26.8245 NSig= 1.80901 NBkg= nan NData= 1258
120 +1 sigma: 41.4361 NSig= 1.80901 NBkg= nan NData= 1258
120 +2 sigma: 59.3199 NSig= 1.80901 NBkg= nan NData= 1258

-- AdrianBuzatu - 14-Mar-2012