Decorrelated Adversarial Neural Network

In this project the decorrelation was achieved by introducing a second network which tries to estimate the mass of an event based on the output of the classifier and the of the event. The loss functions of the combined net, which we will call the adversarial net from now on, is given by

where is the loss function for the neural net, that tries to estimate the mass and the one for the classifier. The classifier uses cross entropy as a loss function while the regression uses mse. At first only the classifier is trained (as of now) just for fix 50 epochs, from epoch 50 to 100 then only the regression is trained, after epoch 100 both nets are trained simultaneously.

The decorrelated neural Network can be used by setting the parameter 'massless' to 'adversarial' . The following hyperparameters can then be set:

'massless_importance': This is the term in combined the loss function

tbd: 'nNodes_reg': regression network architecture

Chi Square for the Histograms

In order to get a quantisation of the sculpting a chi-square difference, given by:

i,j runs over the bins of the two histograms and a Poisson distribution is assumed.

The two histograms to be compared are the background without any cuts applied on the DNN variable and the background where some cuts on the DNN are applied.

In the following two extreme cases can be seen:

is equal to 0, implying that the dnn classifies everything as the same class (not straightforward from that plot but from the corresponding DNN score plot)"> is equal to 0, implying that the dnn classifies everything as the same class (not straightforward from that plot but from the corresponding DNN score plot)" />

is equal to 0, implying that the dnn classifies everything as the same class (not straightforward from that plot but from the corresponding DNN score plot)

is high and we can see the sculpting clearly"> is high and we can see the sculpting clearly" />

is high and we can see the sculpting clearly

However we want to balance the two extreme cases, one example for that can be seen in the following figure:

is greater than zero, but still close to zero, that is what we would like to see"> is greater than zero, but still close to zero, that is what we would like to see" />

is greater than zero, but still close to zero, that is what we would like to see

Too make it easier to check the goodness of the chosen Hyperparameters in first order a heatmap is used. The value of each field is given as the difference between the scaled significance and the scaled , both scaled to 0 with 1 standard deviation. A plot for this can be seen in the section "Scanned Hyperparameters"

Scanned Hyperparameters

There has been conducted a grid search with the following parameters, the learning rate was set to be:

for the initial classifier training: 1.

for the initial regression training: 1.

'massless_importance':[0.001,0.01,0.005,0.0005]

'learning_rate' (from epoch 100 to 150): [1.,0.1,0.001,0.0001]

'learning_rate' (from epoch 150 to 200): [1.,0.1,0.001,0.0001]

The result in this grid search can be seen here:

, both normalized to 0 with 1 standard deviation">, both normalized to 0 with 1 standard deviation" />

First Grid search, empty fields mean that the DNN only classified everything as the same class, the value of each gridpoint is given by the difference between the significance and the , both normalized to 0 with 1 standard deviation