NP04 Shifts

Before your first shift request access to NP04 shift applications - DUNEProtSPEGroups.

Shift Start

  • Be a bit early for your shift.

  • There are power plugs for your laptop hidden under the tables.

  • Each workstation should always be logged in. In case it isn't, login is pduser (password on paper under monitor at shifter station).

  • Review logbook, checking entries from the previous shift. If not already open, link is here.

  • Sign in to logbook

    • Sometimes the logbook does not show the latest entries. Check the date and time of the last entry. If it looks like several hours are passed from the last entry, try to refresh the page a few times, to see whether newest entries are showing up.

    • Problems logging into logbook? Close all firefox windows and start over.

  • Sign in to Run Control by clicking on the key logo on the top right corner of the page and entering your CERN credentials when prompted.

Change_your_user.png


  • If no Run Control window is present, open one by clicking on the link present in the desktop.

  • Make a logbook entry identifying the shifters and current conditions (beam/no beam, Cathode HV on/off) and current run being taken (if any)

  • Make a slack entry on the np04-shifterassistant slack channel identifying the shifters.

During the shift

  • In absence of specific instructions, follow the Run Plan.

  • At the end of a successful BEAM run (i.e. had more than 5 minutes of beam in it and APA3 CE were ok) please fill the spreadsheet here by adding the run number, voltage at which it was taken, and the energy of the beam. This will be INCREDIBLE useful to keep track of the data we're collecting!

  • IMPORTANT DURING BEAM WEEKS!!! While taking a beam run with beam present, look at the Timing page of the online monitor system MONET. Instruction on how to use MONET here. Such plots should always be open in a window with the auto-refresh tab activated, but in case they aren't, use this link to open Monet. In particular, look at the bottom right plot (highlighted in the picture below), showing the time in second after the last spill. In standard conditions, the dots in the plot should be spread across the X axis while we have beam. If they instead accumulates at the end of the X-axis (i.e. at values around X=30 seconds) contact immediately the Timing expert (red number in the Timing section of the Contact list sheets attached to the window behind you) as we are collecting beam data for which the beamline information are not properly recorded (making these data hard to use).

TimingPlot.png


  • Perform this checklist every 2 hours, in the odd hours, posting it in the e-log.

  • Check the cathode HV voltage and current plots every now and then, if possible more frequently than the checklist (see checklist for instructions on opening this plot). Sometimes (once every 4-8 hours) additional current starts to flow in the TPC, raising the cathode current up to the current limit set on the power supply. This is usually shown as a sudden raise of the current in the Heinzeinger current plot, and a series of SMS (one every 5 seconds) to the shifter phone. When this happens:

    • HV expert will already be aware of the situation and be contacting you (on SLACK or the phone) briefly. If you don't hear from her/him, call her/him on the HV expert phone saying that a current stream is happening.

    • In any case, click Inhibit trigger in the Run Control page as soon as the current stream begins (to avoid recording data in off-standard conditions).

    • Once triggers are disabled, go to the SSP panel (if not already open you can reach it from the navigation panel, which in turn can be opened clicking on the sail/parrot icon in the top bar of the main DCS page, as shown below) and click the OFF BIAS button on the left side of the page to lower the SiPM bias.

    • Once these operations are done, the HV expert will start lowering the cathode voltage to stop the current stream, ramping it back up to nominal voltage once the situation is stable. When s/he will be done and will give you the ok, push the ON BIAS button on the SSP panel, click Enable triggers in the Run Control page and resume normal running.

    • Make an e-log entry reporting the current stream event.

Insturction.png


  • If an alarm goes off on the Slow Control panel (i.e. something is flashing red), call the expert of the relative subsystem and/or the Run Coordinator (Numbers here). If the alarm appears during off-working hours and the subsystem does not have an on-call expert (red number in the list), call the Run Coordinator.

  • Keep an eye every now and then on the warning light at the top of the impedance monitor rack (just outside the yellow fence, see left-side picture below). Normally, this light should be off and the number on the display of the rack should be something around 300 (Ohm) and green in color. If the warning light starts to flash, it means there is a short between detector and building grounds. In this case:

    • if the value on the display is around 20 (usually between 19 and 25), call the Run Coordinator to inform him of the situation and proceed with a power cycle of the warning system. To do this, walk down to the rack, open the back door, unplug the 13.5V, 3A WARNING POWER IN black cable (circled in red in the right-side picture below) and plug it back after about 10 seconds. Look now at the display on the rack front: it will show 0 for a few seconds, then it will turn to a different value. If the value is green the problem is solved. Put an entry on the elog reporting the event and the new value. If it is red, inform the Run coordinator and report the event in the elog.

    • if the value on the display is below 10 (usually 5 or 6) it indicates a physical short between the two grounds which won't be fixed by a power cycle. Call the Run Coordinator to inform him of the situation and report the event in the elog.

ImpedanceMonitor.png PowerIn.png


  • During beam time, keep an ear on the bips coming from the beam monitor installed on the right hand corner of the Control Room (typical monitor image below), informing you that beam from SPS is present, and check regurarly that our own beam is good in CESAR. Explanations on what's represented in the monitor and what and how to check on CESAR are here.

Beam_Monitor.jpg


  • Follow instructions here to start a new run or stop an ongoing one.

    • Before starting a new run double check on the np04-shifterassistant slack channel that APAs or SSPs are not already in use by DAQ, CE or PD experts.

    • After some minutes the run started, look at it in the online monitor system MONET. In particular:

      • Look at the Profiled FFT and Profiled RMS by WIB for each APA. If some of the RMS plots are missing (i.e. flat line at 0), check on the Run Control whether one or more WIBs of that APA are not included in the run (i.e. are grey instead of green). If they are, or if the RMS plots shows holes in the data or value distribution is not rather flat, report the issue in the slack channel (reporting the run number as well) and also in the elog with a screenshot of the display.

      • If some of the Profiled FFT plots are missing, report the issue in the slack channel (reporting the run number as well).

      • TIP If the plots look empty, add .1.3 to the run number (i.e. 4018.1.3) and press GO again

Shift end

  • Make a logbook entry identifying the shifters and listing:

    • problems occurred during the shifts (if any)

    • which of the above mentioned problems are solved and which are still pending

    • run currently being taken or last run taken

  • log out from Run Control

  • log out from the logbook

IN CASE OF PROBLEMS

  • Check the troubleshooting page of the subsystem having issues:

    • For problems between trigger board and timing (visible as a run with a correct trigger rate but zero rate for any fragment of any subsystem) check the instructions reported in this e-log entry: https://pdspelog.web.cern.ch/elisa/display/7250 .

    • Problems with CRT? Look here. For basic CRT operations check this page instead.

    • Sometimes restarting the DCS program on the NP04-CR1 computer through the desktop icon fails, giving out an error. When this happens, restart the computer (click on the logoff icon and choose "Restart"). Username is np04dcs, password is on the whiteboard and on a post-it under the screen. If the computer refuses to restart, do an hardware restart pushing the power button on the PC (which is secured under the table).

  • If that doesn't help, reach out for an expert using the np04-shifterassistant slack channel open in one of the control room computers

  • If that doesn't help or no one is replying use the Expert contact list here to reach out to the experts. CALLING RULES:

    • During normal weekdays working hours, call one of the relevant experts present in the list.

    • During weekends or night shifts with beam run, for any problem that prevents you from taking data contact the on-call expert of the relevant subsystem (on-call expert numbers are shown in red). If you are not sure who to call, call the Run Coordinator.

    • During weekends or night shifts with beam run, if there is a minor problem which still allows you to take beam data, send a mail to the expert (if no on call expert is foreseen for that subsystem) and to the run coordinator, to have the problem solved first thing in the morning.

  • If you don't know how to handle the problem and who to contact to get help, contact the Run Coordinator.

  • If you don't understand what the problem is, call the Run Coordinator.

  • If you feel lonely, DO NOT CALL the Run Coordinator! Talk to your co-shifter. He/She is a nice person. I guarantee you.

-- DavidGeoffreySavage - 2018-08-09

Topic attachments
I Attachment History Action Size Date Who Comment
JPEGjpg Beam_Monitor.jpg r1 manage 2803.9 K 2018-09-05 - 16:21 RobertoAcciarri  
PNGpng Change_your_user.png r1 manage 120.4 K 2018-08-31 - 13:14 RobertoAcciarri  
PNGpng ImpedanceMonitor.png r1 manage 1474.0 K 2018-09-17 - 12:35 RobertoAcciarri  
PNGpng Insturction.png r1 manage 997.5 K 2018-10-29 - 12:15 RobertoAcciarri  
PNGpng PowerIn.png r1 manage 884.8 K 2018-09-17 - 12:35 RobertoAcciarri  
PNGpng TimingPlot.png r1 manage 111.7 K 2018-10-29 - 14:14 RobertoAcciarri  
Edit | Attach | Watch | Print version | History: r37 < r36 < r35 < r34 < r33 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r37 - 2018-11-02 - RobertoAcciarri
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CENF All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback