Using the Online Monitor

Useful Links

• Monet production web viewer - only accessible from within the CERN network.
• Monet develop web viewer - only accessible from within the CERN network.
• Online monitoring presentation from 8th January 2018.
• Guide to developing for the OM.

Useful People

• Antonino Sergi
• Aidan Reynolds
• James Pillow

---

Introduction to the OM

The online monitoring (OM) software is designed to run on the raw data files acquired from the DAQ. The OM passes the data output through a number of raw decoders, produces various plots and histograms through art analysers, and then displays the results on the online monitor, Monet.

The OM software should automatically start whenever the run controls are started. If you find this is not the case, refer to the section below.

The software

The OM software can be accessed from any of the np04 servers, and is installed in two places:

• Production version - /nfs/sw/om/ProtoDUNE_OM_artdaq_v3_02_prod
• Development version - /nfs/sw/om/ProtoDUNE_OM_artdaq_v3_02_dev

DO NOT edit code found in these directories. It is not possible to push updates to the repos from the np04 servers, and so any code modifications you make are liable to be lost if things go wrong. It is also incredibly annoying for the person trying to pull updates from the repository, as they then need to deal with the versioning conflicts you have caused. Any and all code alterations should be done in your own personal version of the branches used for the OM. Once you have pushed your changes, the np04 dev version will need to pull them, and be rebuilt. These branches and repositories are:

Repository	Branch/Version
dunetpc	feature/online_monitoring_artdaq_s65_forkv06_73_00
dune_raw_data	feature/online_monitoring_artdaq_v3_02
lbne_raw_data	feature/online_monitoring_artdaq_v3_02
artdaq_utilities	feature/online_monitoring_artdaq_v3_02
artdaq	feature/pdune_from_v3_02_00a
artdaq_core	v3_02_01

If you want a script that will build you your own version of the OM, then look here.

Monet

Foreword - Some things about Monet are quite annoying. Please appreciate its limitations. Some of this is also liable to change, so if you notice something new on Monet, check back here for an updated brief.

When you first click on the Monet link above, you will be presented with the webpage below. If you just click the "DUNE Online" button, it will take you to Monet.

Once in Monet, you will most likely see a blank page like:

Navigating Monet

To navigate Monet, there are two areas of interest to you.

Top Bar

The first area of interest is the right hand side of the top bar.

This thin green bar at the top of the page contains a number of buttons, and a single entry field. The entry field is used to selected the run you wish to view the plots for, and should auto fill with the most recent run when you open Monet.

There are 3 levels of granularity for selecting histogram data with this box:

1. Run Number
2. Raw data file number
3. Histogram file number

these are specified in the run number entry field as numbers separated by dots, i.e. [run number].[raw data number].[hist number], and finer granularity can always be ommited.

Using this entry field comes with a few caveats:

• Entering a non-existant run number. The page will load without apparent issue, and the entry field will still display the number you entered, however, Monet will automatically load the plots for the most recent run.
• Not all raw data files are processed by the online monitoring, so an invalid raw data file number will return the most recent run.
• Leaving the field blank. If you leave the field blank and click 'Go', Monet will load the most recent run. However, the entry field will remain unfilled and this warning will appear (which you can ignore) - .

As a result of these caveats it is always worth checking the Rendering info box after requesting a specific run, section below.

On the top bar, you will also notice a 'Reference' button. This will toggle reference plots on and off, allowing you to compare the current run to nominal results. This button currently does not work! We expect to have reference plots within a few weeks of the high voltage being turned on.

Side Bar

The second area of interest is the menu on the left hand side of the page.

This menu is organised by various folders, indicated by the icon when closed, and the icon when open. Plots found in these folders are indicated by the icon.

This menu is scrollable, but the scroll bar is only visible when scrolling, so be aware that folders and plots could be hidden past the 'Loading alarms'.

Viewing the Plots

As an example, we will open 'General', located under 'By APA/DSDaS (APA 4)/TPC/'. This will result in a page with two plots.

At the top right of this page, below the green top bar, you will notice some new buttons have appeared:
The important button for now, is 'Rendering Info'. If you click this, a small context window will appear below it:

This window tells you exactly which file Monet is using to display the plots. It is always a good idea to check this whenever you open Monet, or change the run number, to ensure you are looking at the run number that Monet claims you are.

---

Beam Monitoring

In the top bar of Monet, use the current run number with ".0" on the development version (http://np04om.cern.ch:7123/prompt_dq/) to see the beam monitoring plots.

e.g. run number = 4127.0

OM Output

Raw data files

The raw data files produced by the DAQ are saved in /data0 → /data3 on np04-srv-001 → np04-srv-004. The have the form np04_raw_runX_Y_dlZ.root, where 'X' is the run number, 'Y' the subrun number, and 'Z' the event builder number.

Log files

There are two types of log files with the OM. The main log file which stores information for the whole run, and can be used to find log files for each individual instance of the OM as it runs on the separate raw data files.

The whole run log files are located in /log/om/ on np04-srv-003. Do ls -lhtr in this directory to locate the latest log file - it will be the file at the bottom of the list with that command. If you open it up and look to the bottom of the file, you will see lines that look like:

//////////////////// Start process 0 ///////////////////////
1
art -c /nfs/sw/om/fcl/RunOnlineMonitor_1_0.fcl -s /data1/np04_raw_run003473_0001_dl1.root >/scratch/OMoutput/logfile_1_0_20180820_161408
art -c /nfs/sw/om/fcl/RunOnlineMonitor_dev_1_0.fcl -s /data1/np04_raw_run003473_0001_dl1.root >/scratch/OMoutput_dev/logfile_dev_1_0_20180820_161408
art -c /nfs/sw/om/fcl/RunOnlineMonitor_1_0.fcl -s /data1/np04_raw_run003473_0012_dl1.root >/scratch/OMoutput/logfile_1_0_20180820_162040
art -c /nfs/sw/om/fcl/RunOnlineMonitor_dev_1_0.fcl -s /data1/np04_raw_run003473_0012_dl1.root >/scratch/OMoutput_dev/logfile_dev_1_0_20180820_162045

This is informing you which raw data files are being run on, and where the output associated to this data file is located.

If you open up one of the log files, for example /scratch/OMoutput/logfile_1_0_20180820_161408, you will see that it is very long and full mostly of information you don't care about. The important thing to look at in this file is whether or not art finished successfully, which will be located right at the end of the file.

Histogram files

The histogram files will also be stored in /scratch/OMoutput/ alongside the log files. These files will have the form np04_hist_runN_W_X_Y_Z.root, where 'N' is the run number, W is the raw data file number, X is the histogram file number, and 'Y', 'Z' will indicate which process and partition the raw data file was processed on.

---

OM Didn't Auto Start?