JCF, Feb-20-2018: The following instructions are largely cut-and-pasted from the standard DAQInterface command line instructions at ArtdaqBasicOp; however, modifications have been made so that you test using an artdaq v3-based dune-artdaq, rather than the standard artdaq v2-based dune-artdaq

Preliminaries

The following are step-by-step instructions on how to take data on np04-srv-010 using an instance of DAQInterface which you can control directly from the command line, as opposed to DAQInterface as controlled through the GUI-based JCOP framework. Please note: familiarizing yourself with the bash shell's "ctrl-r" command will save lots of typing - e.g., if you executed a command called "foo" earlier in the day which took many hard-to-type-or-remember arguments, and you wish to execute the same command again, rather than retyping the whole thing you can just type "ctrl-r" and then "foo", and the most recent command with the token "foo" will appear.

A step-by-step guide to running the DAQ

  1. log into lxplus (or lxplus7) using your CERN credentials (ssh -X lxplus7.cern.ch -l <username>)
  2. log into np04-srv-010 using the np04daq account (ssh -X np04-srv-010 -l np04daq)
  3. Execute these commands:
      cd /nfs/sw/artdaq/DAQInterface_artdaq_v3/
      source source_me 6701  # Could also be 6700 or 6702, though boot_port6700.txt assigns ports which will collide with JCOP
    
To check whether a DAQInterface instance listening on 6701 is already up, execute:
listdaqinterfaces.sh | grep 6701
and if it isn't, you'll see no output. However, if it is, you'll see something like
DAQInterface instance was launched at 16:47 by np04daq listening on port 6701
If this is the case, assuming someone else isn't actively using that DAQInterface instance, you can kill it via
kill_daqinterface_on_port.sh 6701
Now launch DAQInterface:
DAQInterface &
DAQInterface will be launched on port 6701, assuming that was the port # you passed the source_me file, above. Hit Enter to return to the command line.

Now send DAQInterface the commands shown below. After each transition command, make sure that DAQInterface reports messages like BOOT transition complete, CONFIG transition complete, etc., before you execute the next transition command. Also note that you should replace "your_configname" in the instructions below with the actual configuration name you want. To see the list of available configurations, execute "listconfigs.sh", and the list will appear in the terminal where DAQInterface is running; this list tends to be rather long, though, so checking the e-log or consulting others who have been running the DAQ recently is a good way to figure out which config to use.

setdaqcomps.sh timing cob1_rce01  # You can, of course, choose other RCEs, add SSPs, etc. 
send_transition.sh boot boot_port${DAQINTERFACE_PORT}.txt  # DAQINTERFACE_PORT will be set to the value you passed source_me 
send_transition.sh config artdaq_v3_testing00024
send_transition.sh start
Now let the system run, keeping in mind there's a finite amount of space on the disks; the data will go to /data0, /data1, /data2 and /data3 on np04-srv-001. To send triggers to the DAQ while running (which only makes sense if you're running on hardware, i.e., not component01, component01, etc.), log onto np04-srv-012, and do the following:
source /nfs/sw/timing/pro/software/timing-board-software/tests/env.sh
pdtbutler master DUNE_FMC_MASTER triggen 11  # For a rate of ~6 Hz
If you want half the rate above (i.e., ~3 Hz), bump the argument to triggen by 1, i.e., use 12 instead of 11. A faster rate than 6 Hz appears to harm the RCEs in the sense that they need to be reinitialized by experts if this is attempted.

To stop triggers (necessary before stopping the DAQ, or if you want to change the trigger rate):

pdtbutler master DUNE_FMC_MASTER trigclear
When you want to stop running:
send_transition.sh stop
And when you want to kill the artdaq processes - needed (in artdaq v3_00_02) in order to correctly close the output *.root file, and also something which you'd want to do before logging off, or if you want to use a new configuration:
send_transition.sh terminate
If anything goes wrong during any of these transitions, kill and relaunch DAQInterface as described earlier on this page, and then go through the transitions again (hopefully the error message will be clear enough, otherwise contact John Freeman at jcfree@fnal dot gov). Your first attempt to go through the transitions may fail if there are leftover processes, but DAQInterface will clean those up so that your second attempt should work.

When you're done with your DAQ session, kill DAQInterface, kill_daqinterface_on_port.sh 6701 There are additional hints for recovering when things go wrong that we'll describe as we go.

Monitoring live data

JCF, Feb-28-2018: this will only work if you passed port 6700 to source_me, above, or if you're running the DAQ via JCOP

If you're performing a run on hardware (i.e. timing, cob1_rce01, etc.), to perform online monitoring just log into np04-srv-014 as np04daq, and do the following at any point after the DAQ has reached the configured state:

cd /nfs/sw/om/fcl
./StartOM_lite_artdaq_v3.sh
and then to stop monitoring, do
./StopOM_lite_artdaq_v3.sh

On the other hand, if you're performing a run which includes component01, component02, etc. (i.e., ToySimulators), you can do the following:

cd /nfs/sw/om/ProtoDUNE_OM_artdaq_v3/
source setupONLINEMONITORING
art -c /nfs/sw/control_files/monitoring/SimpleOnlineMonitor_artdaq_v3_multicast.fcl
...and what you'll see is the first 10 ADC values for every fragment coming from component01, which are actually randomly generated values from a gaussian distribution centered at 2048. When you want to stop monitoring, just hit Ctrl-c.

Useful tools for investigating run results

A few simple scripts, available after "source_me" is run, make it simple to investigate the results of past runs. Please note that "past runs" means that the "start" transition went through successfully- if a failure occurred during, say, the "config" transition, these tools won't apply, and it will be necessary to manually look in the directory containing the logfiles (defined in the .settings file) to determine what happened. When a run is started, information about it is saved on disk in /nfs/sw/artdaq/run_records/. This includes both the actual FHiCL documents that were used to initialize the artdaq processes during the "config" transition, the DAQInterface configuration file which was passed as an argument during the "boot" transition, and a file called "metadata.txt" which contains info about the run such as the software versions of dune-artdaq and dune-raw-data used, the configuration selected, the components used, and the location of the logfiles. These scripts take advantage of this information, and are described as follows:

Seeing the configurations and components used in the last N runs

The "show_recent_runs.sh" script takes an integer argument standing for the last N runs for which you want to see a quick overview. E.g.

show_recent_runs.sh 5
will show you something like this:
Run 1000018 (Sep 7 20:04) : np04_teststand_tests00002     : component01 component02 
Run 1000019 (Sep 7 20:07) : np04_teststand_tests00002     : rce00 rce01 
Run 1000020 (Sep 7 20:11) : np04_teststand_tests00002     : rce00 
Run 1000021 (Sep 7 20:13) : np04_teststand_tests00002     : rce01 
Run 1000022 (Sep 7 20:25) : np04_teststand_tests00002     : rce00 

Getting a list of the logfiles containing the output of a specific run

Pass an integer argument standing for the run number you want, and you'll get a list of the logfiles produced during that run. E.g.

show_logfile_for_run.sh 1000020
will output:
/nfs/sw/artdaq/logs/pmt/pmt-33204.1-20170907200957.log
Please note that the logfilenames will sometimes be duplicated on the output of this command.

Changing and adding configurations

If you're reading a configuration's FHiCL documents in from the database, then the instructions for changing the FHiCL parameters via web interface can be found at ArtdaqWebConfigEditorP1 and via basic text editing at TextConfigEditorP1. If you're reading them in from a local file, then you can directly edit the documents in a directory with a name like /nfs/sw/control_files/lbne-daq-config/whatever_the_configuration_name_is .

Further reading

DAQInterface is a general-purpose interface to artdaq-based software, and is not specific to protoDUNE. To learn more about it, you can take a look at the Fermilab DAQInterface page.

-- JohnChristianFreeman - 2018-02-21

Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r5 - 2018-03-01 - JohnChristianFreeman
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CENF All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback