%CERTIFY%

CaloCollisionMonitoring

Note

For offline shift, please follow instructions described at Shifters Offline

Introduction

The CaloMonitoring package consists of software tools aimed at monitoring properties of cells, clusters and towers in the ATLAS calorimeters at the full reconstruction level. The goal is to provide input to assessment algorithms that will give fast feed-back on the quality of cells and clusters/towers at all steps of data taking: from cosmics ray signals to single beam and collisions resulting from the operation of the LHC.

The currently available tools are:

  • CaloCellVecMon: providing extensive set of distributions for the properties of fully reconstructed CaloCells per each layer of the calorimeters
  • CaloClusterVecMon: providing set of distributions for the properties of CaloClusters
  • CaloTowerVecMon: providing set of distributions for the properties of CaloTowers

Developers Tasks

The list of action items and possible open topics is maintained on the CaloCollisionMonitoringTasks twiki. Developers names and activities are reported there ( to be updated).

What's new?

Information about CaloMonitoring developments can be obtained by browsing the Status and Presentations links Contributors (or prospective contributors) should have a look at the Task list for Developers

Status and Presentations

The latest developments are announced to the hn-atlas-calo-monitor-discuss@cern.ch: the Calorimeter Monitoring Discussion list that was set up to facilitate a common forum between LAr and Tile developers.

The ongoing efforts are documented in a few venues:

  1. LAr Data Quality meetings
  2. Tile Data Quality and Processing meetings
  3. specific CaloMonitoring developers meetings: to discuss calomonitoring details between LAr and Tile developers (meetings are recent and agenda location is temporary)

Important Mailing Lists

Additional more general sources of info are
  1. hn-atlas-DataQuality@cern.ch: the general atlas data quality list
  2. hn-atlas-data-quality-operations@cern.ch : the data quality operation list (used during the full dress rehearsal exercise, not very much used now)
  3. atlas-dq-automatic-notifications@cernNOSPAMPLEASE.ch
  4. atlas-data-preparation

Important Meetings

DP/DQ combined meetings are held daily. The main DQ issues are discussed in DQ open discussion site. The Calorimeter related Weekly reports take place at Wednesday's DP/DQ meeting. The agenda can be found here

Useful links

The ATLAS DQ discussion space where one can check the Active DQ items and the ATLAS DQ meeting reports and post main issues on the General Discussion site

ATLAS Operation page

SIMBA2/E-groups mailing lists

The ATLAS LAr Data Investigation group is a useful link to check known problems in the LAr.

ATLAS general data quality web display

LAr Monitoring Expert page

The CaloMonitoring Tools

The CaloMonitoring package is a set of tools aimed at filling in histograms that are stored in specific folders. Whatever the configuration that determines what tools to run the highest level folder will always be named CaloMonitoring. Each tool writes histograms in his own folder, possibly producing corresponding sub-folders.

CaloCellVecMon

The CaloCellVecMon tool monitors the properties of Tile and LAr fully reconstructed CaloCells in a varied set of histograms. The various folders and their content are described below.

LAr (EM and HAD ) cell monitoring

The properties of the LAr CaloCells are monitored with histograms that are stored in a set of sub-folders located in CaloMonitoring/!LArCellMon_NoTrigSel

  • Summary (TH1F) nEvtsByTrigger where the bins report events in the following way : Total Events: bin 0, RNDM Trigger: 1, Calo Trigger: 2, MinBias Trigger: 3, MET Trigger: 4, Misc Trigger: 5, Events Selected for Noise Plots: 6
  • 2d_Occupancy this folder contains 2-dimensional distributions in () of the percentage of the total events for which a the energy of a given cell is above a certain threshold. One plot per layer is available for each of the two sides of the detector. Four thresholds are present:
    • |E_{cell}| > 4 times the expected noise in the database ( ).
    • E_{cell} > 5 times the expected noise in the database ( )
    • E_{cell}<- 3 times the expected noise in the database ( )
    • E_{cell} > fixed (high) threshold for all channels in a given layer
    • E_{cell} > fixed (high) threshold for all channels in a given layer only for missing ET triggered events.

Total: 150 plots.

  • 1d_Occupancy (TH1F) This folder contains the 1-dimensional distributions in and in (integrated over ) of the percentage of the total events for which a the energy of a given cell is above a certain threshold (integrated over ), with one threshold: E_{cell }<- 3 times the expected noise in the database ( ). One plot per layer is available for each of the two sides of the detector. Total: 60 plots.
  • Energy (TH1F) This folder contains
    • 1d reconstructed energy distributions of the cells. One plot per layer is available for each of the two sides of the detector.
    • 1d distributions for the ratio of cell energy to the expected noise in the database for the given cell energy. One plot per layer is available for each of the two sides of the detector.

Total: 60 plots

  • EnergyVsTime (TH2F) This folder contains the 2d distributions in () of reconstructed energy vs reconstructed time. One plot per layer is available for each side of the detector. Total : 30 plots.
  • 2d_AvgEnergy (TH2F) This folder contains the 2d distributions in () of the average energy deposited in a cell. Plots are filled for all events and for those in which the absolute value of the cell enregy is larger than four times the expected noise from the database. One plot per layer is available for each of the two sides of the detector. Total: 60 plots.
  • 2d_AvgQuality (TH2F) This folder contains the 2d distributions in () for the average quality for cells with an energy whose absolute value is larger than 4 times their expected noise from the database. One plot per layer is available for each of the two sides of the detector. Total: 30 plots.
  • 2d_TotalEnergy (TH2F) This folder contains the 2d distributions in () for the total energy deposited in a given cell when the cell energy is larger than 5 times the expected noise from the database. The total energy is summed over all the events that enter the plot for a given cell. One plot per layer is available for each of the two sides of the detector. Total: 30 plots.
  • 2d_Noise (TH2F) This folder contains
    • the 2d distributions in () for the RMS of the cell energy in () with thresholds. One plot per layer is available for each of the two sides of the detector. Total: 30 plots.
    • the 2d distributions in () for the ratio (measured cell energy RMS - expected cell database noise)/expected cell database noise i.e. the percentage variation of the measured cell energy RMS with respect to the expected cell database noise. One plot of each type is available per layer for each of the two sides of the detector. Total : 60 plots.
  • 2d_AvgTime (TH2F) This folder contains the 2d distribution in () of the average cell time for cells whose absolute value of the energy is above 4 times the expected database noise. One plot per layer is available for each of the two sides of the detector.
  • 2d_FractionOutOfTime (TH2F) This folder contains the 2d distribution in () of the fraction of times the time for a given cell is away from zero more than 4 time units; this is evaluated only for events where the absolute value of energy is larger than 4 times the expected database noise
  • 2d_PoorQualityFraction (TH2F) This folder contains the 2d distribution in () of the fraction of times the quality of a given cell exceeds 4000; this is evaluated only for events where the absolute value of energy is larger than 4 times the expected database noise. One plot per layer is available for each of the two sides of the detector. Total : 30 plots.
  • KnownBadChannels (TH2F) This folder contains the 2d distribution in () of the position of problematic channel . One plot per layer is available for each of the two sides of the detector. As these are database quantities and are unchanged while data taking, the plot is only filled on the first event of each run. Total:30 plots
  • DatabaseNoise (TH2F) This folder contains the 2d distribution in () of the expected noise in the database . One plot per layer is available for each of the two sides of the detector. As these are database quantities and are unchanged while data taking, the plot is only filled on the first event of each run. Total:30 plots

The content of these files is configurable by job options (see below). The description being reported for each folder corresponds to the default configuration for collisions as it is the one that comprises all the available folders.

One sided threshold are used for cell energies E with three types:

  • no threshold (only filled once at the beginning of the run)
  • medium threshold: ECell>3 times the expected noise for that cell obtained by using the CaloNoiseTool (default-N sigma is JO tunable)
    • If no CaloNoiseTool is available (i.e. not retrieved from StoreGate) fall back to fixed value thresholds that are layer dependent.
      • in EM Barrel: Ecell> 0.150 GeV
      • in EMEC: Ecell_Presampler> 0.340 GeV, Ecell_EMEC1> 0.075 GeV, Ecell_EMEC2> 0.090 GeV, Ecell_EMEC3> 0.090GeV
      • in HEC: E_{cell}_HEC1> 0.600 GeV, Ecell_HEC2> 0.840 GeV, Ecell_HEC3> 1.270 GeV, Ecell_HEC4> 1.800 GeV
      • in FCAL: E_{cell}_FCAL1> 0.540 GeV, Ecell_FCAL2> 1.090 GeV, Ecell_FCAL3> 1.240 GeV
  • high threshold (about one order of magnitude larger than the medium threshold): sensitive to physics and hot cells. Currently set to hard values, cell-dependent CaloNoiseTool threshold as medium thresholds are also possible.
    • in EM Barrel: E_{cell}> 1.0 GeV
    • in EMEC: E_{cel}l> 5.0 GeV
    • in HEC: E_{cell}> 20 GeV for every layer
    • in FCAL: E_{cell}> 60 GeV for every layer

TileCal cell monitoring

For the CalTile monitoring, a group of histograms was designed for cells with the minimum energy 0.3 GeV (can be changed by modifying the variable "energyThreshold" in the option file. The default cell container is "AllCalo" and only cells with "is_tile()==true" are considered. All histograms are registered under the "shift"-type histograms. The following histograms are filled:

  • CellsXE: Energy cells (in GeV)
  • CellsXEta: Profile histogram, the average cell energy (in GeV) as functions of cell Eta;
  • CellsXPhi: Profile histogram, the average cell energy (in GeV) as functions of cell Phi
  • CellsXModuleS1, CellsXModuleS2, CellsXModuleS3, CellsXModuleS4 - Profile histogram for average cell energy (in GeV) as a function of the Tile module number for each samplings (4 in total)
  • CellsXTower: average cell energy (in GeV) as a function the TileCal tower
  • CellsXEtaVSPhi: 2D histogram for TileCal cell distribution in Eta-Phi

There are several histograms with the cell noise information: First of all, the profile histograms such as:

  • CellsNoiseXEta: - Tile cell sigma(DB) from database vs Eta (profile histograms)
  • CellsNoiseXPhi: - Tile cell sigma(DB) from database vs Phi (profile histograms)
Profile histograms with cell energies. Errors on data points indicate RMS (not the mean on cell energies!):
  • CellsRMSXEta: - Tile cell energy with RMS errors for Eta (profile histogram)
  • CellsRMSXPhi: - Tile cell energy with RMS errors for Phi (profile histogram)
Another sets of histograms are after the division of the above tow sets of histograms with a proper error prorogation:
  • CellsRMSXEtadivNoiseDB: - RMS divided by sigma(DB) for Eta (profile histogram)
  • CellsRMSXPhidivNoiseDB: - RMS divided by sigma(DB) for Phi (profile histogram)
For all these histograms, bad cells (from the database) are removed from the consideration.

There is also a set of histograms for cell energies above 4-sigma threshold, i.e. when abs(cell energy) are larger than sigma(DB). Bad cells (from the database) are removed from the consideration.

  • CellsNoiseXEtaVSPhi: - Number of TileCells with the abs(energy) larger than 4 sigma(DB)
  • CellsNoiseXEtaVSPhiAcells: - Number of TileCells with the abs(energy) larger than 4 sigma(DB) for A-type cells (first sampling)
  • CellsNoiseXEtaVSPhiBCcells: - Number of TileCells with the abs(energy) larger than 4 sigma(DB) for BC-type cells (2nd and 3rd sampling)
  • CellsNoiseXEtaVSPhiDcells: - Number of TileCells with the abs(energy) larger than 4 sigma(DB) for D-type cells (2nd and 3rd sampling)

Some of the histograms above are shown at the the offline testing program for FDR data "TileCal" Offline ANL DQM web page. The page contains the description of each "TileCal" histogram.

General Caveat on granularity and definitions

  1. Care is taken in the cells 2 dimensional maps in (eta,phi) to ensure that the information in one (eta,phi) bin corresponds to one and only one channel. This goal is achieved all layers. In some calorimeter layers, the phi granularity changes with eta; as root does not allow for non aligned binning (every vertex must connect to four bin edges), some strips of empty bins in constant phi are unavoidable. In particular: * Strips of empty bins are to be expected in the EMEC, 2nd sampling, and in all layers of the HEC, for |eta| > 2.5. * Empty bins are to be expected in all layers of the FCAL. As the FCAL is built in x-y rather than eta-phi coordinates, no fully compact binning is possible in eta-phi; by using variable sized bins in both eta and phi, however, the number of empty bins in minimized while guaranteeing no more than 1 cell per bin. * The phi granularity of the first sampling of the EM barrel increases by a factor of four for eta between 1.4 and 1.475, from 0.1 to 0.025. If this higher phi granularity were used for the entire plot, almost three out of four bins in the histogram would be empty. To avoid such a large number of empty bins, the lower phi granularity of 0.1 is used throughout, in order to avoid having more than 1 cell per bin, the eta granularity in the region of 1.4 < |eta| < 1.475 is increased by a factor of four, from 0.025/8 to 0.025/32. These bins are filled according to eta = celleta + 0.2551 * mod( |phi| , pi/32 ) - 0.0125. With this arrangement a one to one mapping between cells and bins in the EM barrel in maintained.

  1. To identify bins that are not connected to any cell in the readout, the CellOccupancyVsEtaPhi_[layer]_hiEth plots are useful. Every bin that does not correspond to a CaloCell in the first event is marked in this plot with a -1.

CaloClusterVecMon

The input to the monitoring histogram is the "!CaloCalTopoCluster" container.

The folders being filled are:

  • General : Contains plots of Number of Cells in cluster for cluster in event, number of clusters in each event, number cells in clusters in eta/phi (location of cluster).
  • 1d_Occupancy : 2d distribution vs ( or ) with 3 thresholds for each 1. Energy> 0 GeV 2. Energy > 100 GeV 3. Energy > 250 GeV
  • 2d_Occupancy : Occupancy in () with 3 thresholds for each 1. Energy> 0 GeV 2. Energy > 100 GeV 3. Energy > 250 GeV
  • 1d_AvEnergy : Average energy of clusters as a fuction eta or phi
  • 2d_AvEnergy : Average energy of clusters in eta/phi with 3 thresholds for each 1. Energy> 0 GeV 2. Energy > 100 GeV 3. Energy > 250 GeV. and average energy of maximum energy cluster in event as a fuction of eta/phi
  • LeadCell : Average energy of cell with energy > 0.9 of cluster as a function of eta/phi. Rates of cells with energy > 0.9 of cluster in eta/phi. Ratio of energy of cluster divided by energy of lead cell.
  • TimeEnergy : Energy and Et of clusters in three eta regions a. |eta|<1.4 b. 1.4<|eta|<2.5 c. |eta|>2.5

All cluster monitoring

To verify various corrections to the clusters, first the energy E(cells) of all uncorrected cells which belong to a CaloCalTopoCluster was calculated. Then, we calculate the ratio R=E(CaloCalTopoCluster)/E(sum over all cells in the cluster). The average value for such ratio was calculated using 3 profile plots:
  • TileCalXRatioXE - as a function of E(CaloCalTopoCluster) (in GeV)
  • TileCalXRatioXEta - as a function of Eta
  • ClustersXRatioXPhi - as a function of Phi

TileCal cluster monitoring

There is an intrinsic ambiguity in the definition of "!TileCal" cluster. For the monitoring tasks, we call a cluster which contains more than 50% of its energy measured by the TileCal (the energy was calculated from the cells in the cluster). The energy fraction measured by the TileCal was calculated as RT=E(sum over tile cells in the cluster) / E(CaloCalTopoCluster). Thus, the TileCal clusters have RT>0.5. Only first 4 highest-energy CaloCalTopoClusters are filled in the histograms.

The following histograms are filled:

  • TileCalXclustersXE - Energy of TileCal clusters
  • TileCalXclustersXEta - Profile histogram for the mean energy of the TileCal cluster as a function of Eta
  • TileCalXclustersXPhi - Profile histogram for the mean energy of the TileCal cluster as a function of Phi
  • TileCalXclustersXEtaVSPhi - Eta - Phi distribution for TileCal cluster

In addition, the TileCal energy fraction, RT, was measured as functions various variables.

  • TileCalXRXE - Average TileCal energy fraction as a function of E(CaloCalTopoCluster)
  • TileCalXRXEta - Average TileCal energy fraction as a function of Eta
  • ClustersXRXPhi - Average TileCal energy fraction as a function of Phi

One can look at the offline "TileCal" Offline ANL DQM web page to see how "TileCal" histograms look like for FDR2 data. The page contains the description of each "TileCal" histogram for clusters.

CaloTowerVecMon

The input to the monitoring histograms is the "!CombinedTower" container.

The folders being filled are:

How to Run CaloMonitoring

CaloMonitoring uses information contained in the CaloCell and CaloCluster containers: it can then be run online and offline both directly from bytestream/RDO and from ESD. The output is then used for visualization and data quality tests in order to contribute to the determination of the data quality for calorimeters.

The package code is available in /atlasoff/Calorimeter/CaloMonitoring.

The package is by now fully embedded in the general Atlas Data Quality monitoring. The DataQualitySteering job option now contains a fragment that refers to the global CaloAllMonitoring_jobOptions.py. This job options file :

  1. chooses the job options files that will configure the monitoring of cells and clusters depending on the data taking condition: cosmics, single beam, anything else
  2. chooses whether to run CaloMonitoring from ESD or from bytestream/RDO depending on the value of the monManEnvironment() flag: tier0Raw imples running from raw data, tier0ESD imples running from ESD.

The data quality steering Job option is included in RecExCommon_topOptions.py,the main job options used in reconstruction. In this way it is possible to run CaloMonitoring by running one of the different instances of the reconstruction and switching all the remaining monitoring tasks.

Steering CaloMonitoring with Job Options flags

It is possible to steer CaloCell and CaloCluster plots with flags depending on the type of data taking.

CaloCells monitoring is driven by two job options file: CaloCellVecMonCollision_jobOpt.py (collisions and single beam runnning) and CaloCellVecMonCosmics_jobOpt.py (cosmics running).

CaloCells plots can be changed by

  1. masking channels Channel masking for LAr uses the LArBadChannelTool. For cosmics the current status is that one instance of the LArBadChannelTool is generated and it can be used to mask certain channels in all plots (selective masking is not implemented). There are several variables that are used to determine what if anything is masked, and if so, how it is done. All of these variable are set to false by default, so that nothing is masked. The variables are as follows: MaskBadChannels = TRUE/FALSE will choose whether to mask bad channels or not MaskNoCondChannels = TRUE/FALSE will choose whether to mask channels for which no proper cell conditions are loaded, as specified by the appropriate cell provenance bits. doInverseMasking = TRUE/FALSE will choose whether to have inverse masked histograms. doInverseMasking will not be applied if MaskBadChannels is not TRUE. doMaskingOnline = TRUE/FALSE determines whether masking is implemented for every event in the event loop, allowing gain dependent masking, or, if set to false, only in finalize based the static map of bad channels recorded in the KnownBadChannels histogram.
  2. Using triggers Trigger awareness, using the TriggerDecisionTool, can be globally enabled, disabled by setting useTrigger true/false. Several trigger names are defined which can be used to specify which triggers are included or excluded from different plots. With default setting, if useTrigger is set to true, the 1d DBNoiseNormalized energy plot will be filled for events which pass at least one of the triggers specified in rndmTriggerNames without also passing any of the triggers specifies in caloTriggerNames. By defuault, the same criteria is used for the eta phi AverageEnergy, EnergyRMS, and RMSvsDBnoise plots. metTriggerNames specify which triggers will be used for filling the missing energy triggered etaphi occupancy plot.
  3. Specifying energy thresholds for eta-phi plots Eta-phi occupancy, total energy, average energy, energy rms, rms vs DB noise, quality and time average and fractional occupancy, and 1d eta and phi energy occupancy plots can all take configurable energy thresholds. These thresholds are specified for a threshold type, for which an absolute or noise dependent threshold can be specified, along with the direction of this threshold ( E > thresh, E < thresh, or |E| > thresh ). As eluded to above, triggers to exclude and exclude can also be specified for a threshold type. It also possible to set specific energy thresholds for each calorimeter layer.
  4. Switching plots on or off Most plots can be turned on or off in jobOptions. Eta-Phi plots for which energy thresholds are defined can be switched on or off for each threshold, for instance, the average energy plot for a given energy threshold and trigger configuration can be turned on or off by setting the appropriate value in the array DoEtaPhiAverageEnergy. DataBaseNoise, KnownBadChannels, and 1d normalized and unnormalized energy distributions do not take thresholds, and thus are switched on or off by setting a single variable.
  5. Making other changes to plots of a give threshold type By setting the appropriate element in the DoPercentOccupancy array, one can specify whether all occupancy plots which use the corresponding threshold will show a percentage or absolute occupancy. Similarly, with the MaskEmptyBins array, one can specify that bins for which there is no CaloCell read out in the first event have are set to -1 by setting the appropriate element to TRUE. QualityFactorThresholds and TimeThresholds can also be set for the fraction over Qth and fraction past Tth plots, respectively.

CaloClusters monitoring is driven by CaloCosmicsClusterMon_jobOpt.py and CaloClusterMon_jobOpt.py for cosmics running; collisions running is steered by CaloClusterMonCollisions_jobOpt.py and EMClusterMonCollisions_jobOpt.py

CaloCluster plots are modified by setting the three available thresholds on the energy of the clusters to be used for filling the plots with format xxxEthresh = value (in GeV) where xxx=low,med,hi

How to monitor

Data quality monitoring takes place online and offline. Different tests and visualization schemes are involved in getting a quality statement.

DQ Assessment strategy

The distributions being produced by the various tools are tested to assess the quality of the data. The use of the Data Quality framework (DQMF) allows to define automatic tests aimed at discovering problems in a robust manner. Two ideas underlie the present DQ assessment strategy:

  • set up automatic tests that will perform robust, self consistent checks. Separate tests are use to find "local" and "global " anomalies. Most tests are currently aimed at finding "localized" anomalies. Tests for global effects are planned.
  • develop summary logic that is allows to merge information from various tests performed on different distributions: in this way correlations are used to produce a robust and efficient assessment to single out real problems and assign them a level of severity.

At all levels various distributions are tested and their results properly merged and summarized. The different implementations for cells, clusters and towers are described below. Some common features in the implementation of the ideas above are obtained when describing the algorithms being used.

DQAlgorithms

BinsDiffByStrips

Description
This algorithm relies on our expectation of at least approximate symmetry by looking for cells that are significant outliers within the distribution of bins at the same . The code for the algorithm ism in the dqm_algorithms package ( BinsDiffByStrips.cxx and BinsDiffByStrips.h). Specifically, the algorithm takes all bins from a given strip in eta, uses an iterative technique to estimate the mean and standard deviation of the values of the bins in the strip once outlier bins are removed (as well as their statistical error). It then assigns a quality flag (red, yellow, green, or Undefined) to each bin in the strip based on the size of its relative deviation from the mean value of the strip and its uncertainty.

Specifically

BinDeviation = ( BinValue - StripMean ) / StripVariance .

where BinValue is the value of the property under consideration in the given bin, StripMean is the average value of the property over all the bins of the strip, StripVariance is the standard deviation calculated from the values of the bins of the given strip, Sigma_CellDeviation is the uncertainty on BinDeviation calculated by propagating the errors from BinValue, StripMean and StripVariance. A bin will be labeled

  • red: if BinDeviation > RedThreshold + 5 * Sigma_BinDeviation.
  • yellow: If BinDeviation > GreenThreshold + 5 * Sigma_BinDeviation
  • green : if BinDeviation < GreenThreshold - 5 * Sigma_BinDeviation
  • undefined : if Abs( BinDeviation - GreenThreshold ) < 5 Sigma_BinDeviation

The logic hierarcy for a given histogram assessment is as follows. If there is one or more red bin, the flag for the overall histogram is red, otherwise if there is one or more yellow bin, the histogram is yellow, otherwise, if more than half the cells in the histogram are Green, it's status will be Green or, if not, Undefined.

In addition an algorithm that clusters problematic bins (clustered problematic region) is also implemented. By default the clustering works as follows.

If a bin passes a SeedThreshold (which is default equal to the Red threshold) then it seeds a cluster (these clusters are used for publishing purposes only in the default setup, they define a region of cells that are problematic but since the requirement is that they be seeded by a red cell, the dq result for the histogram would already be red even if no clustering was done). If any neighboring bins to the seed bin pass the threshold to be added to a cluster, GrowthThreshold , they are added. Because of the way histograms are binned in root, we are guaranteed that all bins have exactly eight nearest neighbors (unless they are on an edge of the histogram which is not mapped to another edge, such as the upper and lower eta bounds of an eta-phi histogram). By default, GrowthThreshold is set to be equal to the green threshold, that is, any bin that has a clustered binl as a nearest neighbor that would be published separately as yellow will instead be merged into the cluster. The BinDeviation value assigned to a cluster is simply the sum of the deviations from all of the bins contained within that cluster.

The results of the algorithms are reported for each histogram being tested.

Example results: how to read a BinsDiffByStrips test i.e. a 2d test for CaloCells or CaloTower or CaloCluster plots.
  • Offline.
    • The test name is reported under Assessment details as Algorithm: BinsDiffByStrips.
    • The configuration parameters are:
      • MinBinsAfterSkimming: the
      • PublishBins: 1.0_ means that results will be published
      • ignoreval = 0_ means that the test will ignore bins with value equal to 0.
      • MaxDeviation is assigned to the label that reports the value of GreenThreshold and RedThreshold
      • Results : shows the result of the algorithm.
        • Problematic bins are reported in the format NNN--(eta,phi)[err]=(ax.xxx,by.yyy)[z.zzz] = BinDeviation where NNN is the number of the bin (the highest the value the largest the BinDeviation), Colour is R(red), Y(yellow) and G(green), a and b are positive or negative sign, x.xxx and y.yyy are the eta and phi coordinates respectively, z.zzz is the value of Sigma_CellDeviation.
        • Clustered problematic regions are reported in the format CXXX--00NBins@ Eta=(ax.xxx_to_by.yyy) Phi=(cx.xxx_to_dy.yyy) Center=(ex.xxxfy.yyy). Radius=gr.rrr; ddd.d where CNNN indicates the fact that there is a clustered region (C) and NNN is the number of the region (the highest the value the largest the assigned BinDeviation); x.xxx and y.yyy are eta and phi cordinates, the letters (a,b,c,d,e,f,g) stand in for the Eta=(ax.xxx_to_by.yyy) (Phi=(cx.xxx_to_dy.yyy)) indicates the extension in <\latex>eta (<\latex>phi). The value of ddd.d is the sum of the BinDeviations o fthe bins in the clustered region.
    • *Example for the test of the 2-dimensional (eta,phi) plot for positive clusters in the barrel (see *m_EtaVSPhi0* below). One finds
    1. 000-Y-(eta,phi)[err]= (-0.450,+1.329)[4.661]: 28.41
      • The worst bin is a yellow bin at =-0.450 and Mlatex>\phi = +1.329; the BinDeviation 28.41 and its uncertainty Sigma_BinDeviation is 4.661. The threshold on BinDeviation to go from green to yellow for this bin is 2+5* Sigma_BinDeviation=2+23.305=25.305, so 28.41 makes it yellow.
    2. C000-R-002Bins@ Eta=(+0.650_to_+0.750) Phi=(+0.148_to_+0.148) Center=(+0.662,+0.148) Radius=+0.022: 231.5
      • There is a clustered region of bins; it is classified red and the sum of the deviatins from the mean of its bins is 231.5 standard deviation: extends from+0.650_to_+0.750 , is +0.148 and its center is at (,)=(+0.662,+0.148)) with a radius of 0.02
    3. MaxDeviation-(eta,phi)[err]= is located at [27.99]: 202.9
      • The bin with the maximum Bindeviation from the mean over all eta strips is located at ((,)=+0.650,+0.148) and its BinDeviation is of 202.9. The threshold fro it to become yellow is 2++5* Sigma_BinDeviation=2+139.95=141.95. The threshodl to be RED is 10+139.95=149.95. In fact it is included in the clustered region.
    4. NBins_RED: 2.0
    5. NBins_YELLOW: 1.0
      • There are then 2 red bins and 1 yellow bin

DQ Assessment strategy for CaloCells

CaloCells provide the foundation for all higher level Calorimeter objects: all problems that are apparent in the clusters and towers should be traceable back to the CaloCells. Unlike LArRawChannels, the LAr CaloCells incorporate all HV and other corrections such as the filling of the energy for empty/masked cells with the average of the energies of neighboring cells. In the offline DQ hierarchy, the LAr CaloCells can be found under LAr/[partition]/MisBehaving_Channels/CaloCells/[layer]/. For each of the 30 layers of the LAr calorimeter, 21 distinct plots are presented, although they are not all tested. Of those that are tested, some are given full weight, while others are given reduced or zero weight (limiting the effect of the test on the overall dq assessment). The BinwiseSummary summary maker is used to combine the results from the various eta-phi histograms on a bin by bin basis, with a masking based on the KnownBadChannels histogram, the effects of this masking depend on the weight given to each histogram test, as defined below.

Plots and subregions: (ordered as they apear in the web display, left to right, top to bottom)

The average quality factor for cells with energy above 4Sigma is plotted versus eta and phi. Tested with the BinsDiffByStrips Algorithm, the histogram is given a weight of 0, so its status will never affect the DQ assessment. That is, this is a purely diagnostic plot.

The average time for cells above 4Sigma is tested with BinsDiffByStrips Algorithm and given a weight of 0, as with AverageQuality.

The average cell energy from all random trigger events, and without any energy threshold, is tested with BinsDiffByStrips Algorithm with the configuration parameter AbsDiffGreenThresh = 10, meaning that no cells with an average energy within +/- 10 MeV of the average for the strip will be flagged as problematic. This test is given a weight of 1, meaning that cells that are flagged here as problematic and which do not correspond to cells in the KnowBadChannels will affect the DQ assessment, while those which are flagged as known bad channels will not affect the status for this layer.

The 2d distribution of cell energy and time is plotted for this layer but not tested.

#Energy The 1d energy distribution for all cells in the layer is plotted with logarithmic binning but not tested.

The RMS of the energies from each cell is plotted against eta-phi, but no test is performed.

#4SigmaOccupancy The percentage of events for which each cell's energy is beyond four times it's DB noise ( |E| > 4*sigma ) is tested with the BinsDiffByStrips Algorithm. This is considered the most important and robust test of CaloCell DQ, as cells with high 4Sigma occupancy are guaranteed to seed an excess of clusters, and is given weight=2.0. Masking has no effect on the results from this test. The config parameter AbsDiffGreenThresh=0.0001 prevents cells with variations in occupancy on the order of the occupancy expected from noise from effecting the DQ assessment.

The percentage occupancy beyond some fixed, layer depended energy threshold, is tested with BinsDiffByStrips Algorithm. All bins for which there was no CaloCell read out in the first event have value -1 and are ignored. Bins with zero occupancy are not ignored, allowing the algorithm to have sensitivity to dead channels. This test is given weight=1.25, meaning that the status for cells that are not in KnownBadChannels is propagated unchanged while the status from cells in KnownBadChannels is downgraded ( Yellow -> Green, Red->Yellow ).

#-3SigmaEtaPhiOccupancy The percentage of events for which each cell's energy is less than negative three times it's DB noise ( E < -3*sigma ) is tested with the Bins_GreaterThan_Threshold Algorithm with a threshold of 1.5. If more than 64 cells have an occupancy greater than 1.5%, the test will give a red, if the number of cells is less than 64 but more than 0, the test will return yellow. There is no down weighting or masking, so this test will always effect the DQ assessment. The thresholds for this test are set to be the same as those used for the LArRawChannels noise_acceptance plots (which also show percentage occupancy per cell with E < -3*sigma) to facilitate cross comparisons.

The number of events per cell with E > 1GeV in which a missing energy trigger is passed are tested with the BinsDiffByStrips Algorithm. This plot is given a weight of 1.25, meaning that the status for cells that are not in KnownBadChannels is propagated unchanged while the status from cells in KnownBadChannels is downgraded ( Yellow -> Green, Red->Yellow ).

#-3SigmaEtaOccupancy The percentage of events in each eta bin (integrated over phi) with E < -3*sigma. This plot is tested with Bins_GreaterThan_Threshold, as with the -3Sigma EtaPhi occupancy, but a tighter threshold of -0.5 is used as integration over multiple bins should decrease statistical fluctuations. This test is given full weight.

#-3SigmaPhiOccupancy The percentage of events in each phi bin (integrated over eta) with E < -3*sigma. This plot is tested with Bins_GreaterThan_Threshold, as with the -3Sigma EtaPhi occupancy, but a tighter threshold of -0.5 is used as integration over multiple bins should decrease statistical fluctuations. This test is given full weight.

The energy per cell divided by the noise for that cell from the DataBase is tested with the Simple_gaus_Fit algorithm. As this plot is only filled with randomly triggered events, one expects it to correspond purely to noise and thus have a gaussian distribution with sigma=1. If the fit finds Sigma + 5 * Sigma Error > 1.25, the plot is flagged as Red, while otherwise, if Sigma + 5 * Sigma Error > 1.05, it is flagged as Yellow. As the trigger awareness was found to be unreliable, this plot is currently given weight = 0, so it has no effect on the DQ assessment; once the triggers become more stable, it should be given weight = 1.

#DBNoise A plot of the noise values from the database, as retrieved on the first event of the run, is presented as a reference. As this is a static representation of the known characteristics of the calorimeter, it is of course not tested.

A plot of the percentage deviation of the energy RMS per cell from its expected noise ( 100 * (RMS - DB) / DB ) as recorded for random trigger events is tested with the Bins_GreaterThan_Threshold algorithm and a threshold of 20%. If there are any cells with 20% greater noise than that recorded in the database, this plot gives Yellow, if there are more than 64, Red. As with the DBNoiseNormalizedEnergy, the weight is currently set to 0, and should be reset to weight=1 once trigger awareness is felt to be reliable.

A map of the KnownBadChannels from the database is presented for reference.

The fraction of events with energy beyond 4sigma for which the quality factor is greater than 4000 is tested with the BinsDiffByStrips Algorithm. The result is given weight = 0, so it does not effect the DQ assessment.

The fraction of events with energy beyond 4sigma for which |time| > 8 is tested with the BinsDiffByStrips Algorithm. The result is given weight = 0, so it does not effect the DQ assessment.

#4Sigma The eta-phi average energy for cells beyond 4Sigma is tested alongside the 4Sigma occupancy in the "4Sigma" subregion using the BinsDiffByStrips Algorithm along with the BinwiseSummary summary maker to flag cells with both anomalously high average energy and occupancy (the same cell must be flagged as red in both histograms for the 4Sigma region itself to be red). This region is given full weight: a red status here will lead to a red for the whole layer and masking has no effect. Note that the thresholds given for the test of the occupancy here is tighter than in the standalone test of the 4Sigma occupancy described above, so it is possible for this region to be red even when the stand alone 4Sigma occupancy test is not.

#Sigma A similar procedure is used the the "5Sigma" subregion for testing the 5sigma average and total energies, although this is given a weight of 0.5, meaning that a red from this region will only lead to a yellow for the full layer.

DQ Assessment strategy for CaloClusters and CaloTowers

Clusters and Towers provide an overall summary quality image of the whole calorimetry. Histograms taken from the CaloClusterMon folders and from the CaloTowerMon folders are used to assess the quality of the data. For each main DQ region one super-folder is defined that contains three folders. Each folder contains a certain number of plots. Three super-folders are available CaloMonBAR (DQ assessment for the Barrel, |eta|<1.5), CaloMonECA (DQ Assessment for the EndCapA, eta in (1.5,5] ), CaloMonECC (DQ Assessment for the EndCapC, eta in [-5,-1.5)). Each super-folder contains 4 subfolders

CaloMonStructure.jpg.png

An overall view of all othem is shown in this example for run 185353.

The 17 selected plots are:

  • Hit map of cells with E_cell/E_cluster>0.9&E_cluster>0: two dimensional (eta,phi) distribution of the clusters for which one cell accounts for 90% of the cluster energy(>0).Example in EndcapA.

  • Average number of cells in clusters : two dimensional (eta,phi) distribution of the average number of cells that form a cluster. Example in EndcapA.

  • total energy of cluster with Energy >0.0 GeV : two dimensional (eta,phi) distribution of the total energy for positive energy clusters Example in EndcapA.
  • total energy of cluster with Energy >2.5 GeV : two dimensional (eta,phi) distribution of the total energy for clusters with energy larger than 2.5GeV Example in EndcapA.
  • total energy of cluster with Energy >10.0 GeV : two dimensional (eta,phi) distribution of the total energy for clusters with energy larger than 10.0 GeV Example in EndcapA.

  • avg energy of cluster with Energy >0.0 GeV : two dimensional (eta,phi) distribution of the average energy for positive energy clusters Example in EndcapA.
  • avg energy of cluster with Energy >2.5 GeV : two dimensional (eta,phi) distribution of the average energy for clusters with energy larger than 2.5 GeV Example in EndcapA.
  • avg energy of cluster with Energy >10.0 GeV : two dimensional (eta,phi) distribution of the average energy for clusters with energy larger than 10.o GeV Example in EndcapA.

  • Hit Map of Cluster with Energy >0.0 GeV : two dimensional (eta,phi) distribution of the number of clusters with positive energy Example in EndcapA.
  • Hit Map of Cluster with Energy >0.8 GeV : two dimensional (eta,phi) distribution of the number of clusters with energy larger than 0.8 GeV Example in EndcapA.
  • Hit Map of Cluster with Energy >2.5 GeV : two dimensional (eta,phi) distribution of the number of clusters with energy larger than 2.5 GeV Example in EndcapA.
  • Hit Map of Cluster with Energy >5.0 GeV : two dimensional (eta,phi) distribution of the number of clusters with energy larger than 5.0 GeV Example in EndcapA.
  • Hit Map of Cluster with Energy >10.0 GeV : two dimensional (eta,phi) distribution of the number of clusters with energy larger than 10.0 GeV Example in EndcapA.

  • Hit Map of Cluster with Et >5.0 GeV : two dimensional (eta,phi) distribution of the number of clusters with et larger than 5.0 GeV Example in EndcapA.
  • Hit Map of Cluster with Et >10.0 GeV : two dimensional (eta,phi) distribution of the number of clusters with et larger than 10.0 GeV Example in EndcapA.

  • Eta Energy > 0.0 GeV : one dimensional eta distribution of the number of clusters with positive energy Example in EndcapA.

The 15 2-dimensional eta.phi maps that are all tested with the BinsDiffByStrips Algorithm to find anomalous regions. The two 1d occupancy plots in eta and phi are not tested.

The value for each folder is determined by the worst case summary that takes into account the flags of the four 2-dimensional plots i.e. the worst result of the test of the plots is displayed.

The overall value for each super-folder is the worst case sammary of the four sub-folders i.e. the worst result of the subfolders will be propagated to the DQ region.

DQ Assessent for LAr CaloCells

DQ Assessment for Tile CaloCells

Instructions for DQ assessment of CaloClusters and CombinedTowers (CaloGlobal flags)

The DQ assessment uses the results of the DQ tests performed on the plots produced by the different tools and configured in DQMF (offline) and DQMD (online). The offline and online configurations are currently set up to be the same.

Shifters Offline

Know what runs to check

DQ assessment is performed on the express stream. The runs to perform the assessment are determined by criteria that are agreed upon in the ATLAS DQ group. Currently the requirement is that only runs with stable beam (for collision runs), long enough (>10/ub for collisions, >100k events for cosmics, >15min for single beam, any events for heavy ions runs), and without oddities in their Run Control information need to be checked
  • The list of runs to be checked in mailed out in atlas-dq-automatic-notifications@cernNOSPAMPLEASE.ch. Shifters should go to E-groups page and search for this mailing list and subscribe.
    • Mails with title "DQ runs to consider for DD.MM.YYYY" includes runs needed be checked today
      • (An example of a typical automated e-mail message is here.)
    • Mails with title "Missing DQ signoff reminder" includes runs which haven't been checked in the previous days or reprocessed runs (bulk)
      • (An example of a typical automated e-mail message is here.)
  • Note
    • make an elog for each run separately instead of putting all runs in one elog
    • Don't forget to check the bulk for the runs listed in the automatically sent email titled "Missing DQ signoff reminder", and then reply to the corresponding elog entries whether the spots are gone or not. (Do not create a new entry for the bulk, just reply to the elog for ES1.)

Review the DQ assessment strategy

Before starting the assessment, check the DQ logic, the description of the plots and the DQ assessment algorithms if you have not done it already. This step is mandatory to perform the assessment that follows.

Perform a Single Run Asssessment

Thanks to Michael Tuts who provides a great detailed example of how to do CaloGlobal offline shift.

  • Since 2011, the new password for CaloGlobal shift is "CaloDq".

The shifter is expected to act according to the following instructions:

  • Go to the Atlas Offline DQ page
  • Under the heading Current Data Taking go to DQ Web Displays: Offline monitoring(http://atlasdqm.cern.ch/webdisplay/tier0/) If it does not work (if the page is not loaded or if the page is loaded but subsequent directories are not accessible) send an e-mail to the Data Quality hypernews hn-atlas-DataQuality@cernNOSPAMPLEASE.ch. (Refrain from calling the DQ phone (16 1809) at the moment)
  • Go to the run number you have chosen and click on the express-express link . If such link is not available, the run is automatically "undefined".
  • Go to the Entire Run link as in this example for run 185353
  • Navigate the DQ web display folder to CaloMonitoring and then down to CaloMonShift. The three super-folders mentioned in here Each super-folder corresponds to a DQ flag in the general ATLAS Data Quality assessment. The automatic tests already provide an assessment that is visible in the DQ web display (red,yellow,green). The shifter is supposed to report the results of the automatic assessment amd any additional features that are resulting from vbisual inspection.
  • Perform the DQ Assessment for each super-folder (BAR/ECA/ECC)
    • Go through the plots in the CaloTopoClusters. Use the CaloGlobalRunAssessTemplate.txt to fill in the assessment. Example assessments are reported here.
    • For the CaloTopoClusters subfolder
      • Check the results of the automatic test (described in DQ Assessment strategy) for Hit map of clusters with Energy >0.0 GeV (i.e. the 2d occupancy for positive clusters). Add visual inspection to consider additional spots which the test might have missed or global effects that are not reported by the test (for instance a high population in a whole eta strip). Presently also consider the MaxDeviation result: if its coordinates do not coincide with any of the problematic regions, consider it as a result of the test even If the result of the test is green and report it.
      • Check the results of theautomatic test ( DQ Assessment strategy) in avg energy of cluster with Energy>0 (i.e. the 2d average energy distribution for positive clusters) . Compare the coordinates of the problematic spots (single spots, clustered regions, MaxDeviation) with those found in Hit map of clusters with Energy >0.0 GeV (also visually).
      • Classify the problems as:
      • If a problematic bin is present in 2 consecutive runs add "(persistent)" in parenthesis close to it the second time you find it.
      • Report noisy and hot spots (if any) according to the template above by a) copying the links to the occupancy and average energy plot b) copying the coordinates of the problematic regions
      • Perform additional diagnostic by looking at * 2d (eta,phi) distribution for the dominant cell: compare coordinates of hot and noisy; if anything coincides, add info to the correspo nding region with the words: "present in dominant cell". * 1d phi occupancy: if any spike or depressed region is found in the 2d occupancy plot a scorrelation should be possible with the 1d plot.
      • If at least one problem is found the sub-folder corresponding to the DQ region is to be considered YELLOW and it should be reported as such in the e-log for the expert to see.
    • For the EMTopoClusters folder
      • check if the problems found in the CaloTopCluster are found there too in the same corresponding two plots. Report "found in EMTopo" for those problems that are found in both CaloTopoCluster and EMTopoCluster plots
    • If you are considering the ECA or ECC subfolder, check CombinedTowers occupancy to make sure about the coverage i.e. check if there are entries in the 2d occupancy plots. No additional checks are done on the towers for the moment (instructions to come).
    • Report the automatic flag value for CaloTopoClusters, EMTopoClusters and Combined Towers
    • Report any particular feature that is visually spotted (high occupancy island, high occupancy strip along eta or phi,...)
    • Move on to the next run!

  • 2012 strategy
    • Go to the Point of Entry, you can see list of runs to be checked and the systems missing signoff and tier0 reconstructure status showing if ES1 or Bulk is ready to be checked.
    • Go to the Atlas Offline DQ page
    • Under the heading Current Data Taking go to DQ Web Displays(https://atlasdqm.cern.ch/webdisplay/tier0/)
    • Go to the run number you have chosen and click on the express-express link. (if stream name is express_express***, it means the datafile is not ready yet, and you need wait. )
    • Go to the Entire Run link as in this example for run 185353
    • Navigate the DQ web display folder to CaloMonitoring and then down to CaloMonShift. The three super-folders mentioned in here Each super-folder corresponds to a DQ flag in the general ATLAS Data Quality assessment. The automatic tests already provide an assessment that is visible in the DQ web display (red,yellow,green). The shifter is supposed to report the results of the automatic assessment amd any additional features that are resulting from vbisual inspection.
    • Perform the DQ Assessment for each super-folder (BAR/ECA/ECC)
      • Go to the Defect Database link as in this example for run 185353
        • Go to Upload, open Calo, for each item listed, go through the corresponding plot in CaloTopoClusters. During visual inspection, if you find spots with much higher energy or population than the rest in the same eta bin, report defect is present, and list the coordinate of the spot in comment line. When check bulk, if the defect seen in ESD disappears, report defect is absent. (Password:CaloDq)
        • after checking all plots listed in Upload _Calo, go to Sign_off_a_run, choose CALO or CALO_BULK as system depending on ESD or Bulk you are looking. (Password:CaloDq)
      • go to logbook (https://atlasdqm.cern.ch/dqsignoff/ ), click "Sign-Off Day" link of the run you just assessed, and then enter new comment (make sure choose the right process: ES1 or Bulk) :
        • Subject: CaloGlobal is OK if all partitions(CALB, CALEA, CALECC) are fine, otherwise CaloGlobal: problem in XX (XX is the partition which has hot spots)
        • Systems: CaloGlobal
        • Comment: For each part(CALB, CALEA, CALEC), if no intolerable defects exist, report as OK. Otherwise, list the coordiante of intolerable defects (no need to list tolerable ones) and the corresponding plot link. Below is how we define tolerable and intolerable defect. (X={CALB, CALEA, CALEC})
          • Intolerable:
            • CALO_X_TopoClusterNoise_ET10 at least 1 noisy cell in the plot of number of clusters with cluster transverse energy cut > 10 GeV
            • CALO_X_TopoClusterNoise_ET15 at least 1 noisy cell in the plot of number of clusters with cluster transverse energy cut > 15 GeV
            • CALO_X_TopoClusterNoise_ET20 at least 1 noisy cell in the plot of number of clusters with cluster transverse energy cut > 20 GeV
          • Tolerable:
            • CALO_X_TopoClusterNoise_E5 at least 1 noisy cell in the plot of number of clusters with cluster energy cut > 5 GeV
            • CALO_X_TopoClusterNoise_E10 at least 1 noisy cell in the plot of number of clusters with cluster energy cut > 10 GeV
            • CALO_X_TopoClusterNoise_E15 at least 1 noisy cell in the plot of number of clusters with cluster energy cut > 15 GeV
            • CALO_X_TopoClusterNoise_E20 at least 1 noisy cell in the plot of number of clusters with cluster energy cut > 20 GeV
            • CALO_X_TopoClusterNoise_ET5 at least 1 noisy cell in the plot of number of clusters with cluster transverse energy cut > 5 GeV
            • CALO_X_TopoClusterNoise_AvgE_E0 at least 1 energetic cell in the plot of average cluster energy with cluster energy cut > 0 GeV
            • CALO_X_TopoClusterNoise_LowStat: low statistics
            • CALO_X_TopoClusterNoise_Unknown: inaccessible info
            • CALO_X_TopoClusterNoise_Disabled: calorimeter is turned off
      • If there are still some intolerable defects in Bulk reprocess, please send an email with link of the run's logbook to Diedi (dhu@cernNOSPAMPLEASE.ch), Jun(jun.guo@cernNOSPAMPLEASE.ch) and also write an elog (https://atlasop.cern.ch/elog/ATLAS/ATLAS/): choose Data Quality as message type, offline as DQtype, LArg & Tile as System Affected, and put the logbook link as content.

    • Run signoff
      • has to enter comment on logbook before 4:00pm
    • Period signoff
      • Signoff time is informed by email sending to <hn-atlas-DataQuality@cern.ch>, and shifter of that day is supposed to make sure that there is no intolerable defect in runs of that period before DQ meeting, and then can signoff during the meeting.
      • Here is the guidance
    • Weekly report on Wednesday DQ meeting

Additional Comments For comparisons with cells in (eta,phi) keep the tolerance to 0.1 granularity in both eta and phi this is valid for all comparisons: amongst single bins, amongst clustered bins, between a single bin and a set of clustered binsNotice that hot spots in average energy, but not in occupancy can hardly be a collective pedestal shift as in order not to be seen in the occupancy plots should be a low noise region with a small shift that makes it stay below the level of high occupancy. (visually a sharper peak that its surroundings whose average is moved higher up).


%RESPONSIBLE% KhadeejahALghadeer
%REVIEW% Never reviewed

-- KhadeejahALghadeer - 08-Sep-2012

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2012-09-08 - KhadeejahALghadeer
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback