-- ParasNaik - 2017-05-22

RICH detectors' mirror alignment monitoring via Online software:
Information for all shifters, piquets, and experts

Some information may appear to be repeated; eventually some of these pieces of text will be moved to other TWikis.

General Information (please read first)

  • Time scales for the Mirror Alignments to start running (e.g. 2-3 hours after VELO closure) are based on typical HLT line rates; these time scales may be shorter near the start of long ramps (e.g. yearly LHC startup, after technical stops), when there are fewer than 1500 bunches colliding.

  • The mirror alignment is currently optimized only for proton-proton collisions where the L0 data taking rate is saturated.
    • If we are in any other collision state (e.g. Pb-Pb) or other type of running (e.g. SMOG), do not expect a mirror alignment. Our alignment may still run later than expected, or (most likely) not at all.
    • If the L0 rate on LHCb page 1 is less than 800 kHz, do not expect a mirror alignment. Our alignment may still run later than expected, or not at all.
  • These situations are normal, and when they occur are not cause for concern or reporting (i.e. do not call the Alignment piquet). We hope to eventually be able to run the mirror alignment in these situations, but the technology is not yet developed.

  • To call a CERN mobile from outside CERN, dial as if you were calling a Swiss number starting with (+41) 75 411, and then the last 4 digits of the mobile number.

  • The MirrAlign Experts thank you for your careful reading of the following information relevant to you, and your scrutiny of our output.

  • Please now select only the Specific Shifter Information below that applies to your role.

Specific Shifter Information

Data Manager (and Shift Leader)

Data Manager (and Shift Leader): ShifterInstructions Page

for SHIFT LEADER "Alignment" section:

NEW RICH mirror Alignment

  • In the "middle" of each fill (approximately 2-3 hours in) pay attention that new Rich1 and Rich2 mirror alignments should be performed, and ask the Data Manager to check the plots if an update of the alignment was produced [it will appear on the alarm screen, and you will find the plots on the Alignment monitoring ELOG]. More detailed instruction in the Data Manager section, below.

for DATA MANAGER "How To's" section:

NEW RICH mirror Alignment

  • Check the output of RICH mirror alignment jobs
    • About 2-3 hours after the VELO is closed: a Rich1 alignment job and a Rich2 alignment job will check if a new alignment for each of the RICH detectors' mirror systems is needed. The alignment jobs typically need 10 to 15 minutes to complete, each. Please be patient during this time. When the jobs have finished:
      • The output of it will appear at the Alignment monitoring ELOG. If nothing happens 4 hours after the VELO is closed, for one or both alignments, call the Alignment piquet ( Phone 16 14 87 ) UNLESS the reason is one listed in the General Information section of the LHCbRichMirrorAlignShiftInfo TWiki.
        • If the output of the job is that we don't need a new mirror alignment for a Rich detector, you will not have received an alarm, and the message in the eLog for system "RichN Mirrors" is "No RichN mirror alignment update needed for fill nnnn" (where "RichN" can be Rich1 or Rich2). No other action is needed.
        • If one of the RICH detectors needs a new alignment, an alarm will appear on the alarm screen to remind you to look at the plots.
          • Follow these instructions to check the output for the RICH detector that updated (or both RICH detectors, if both were updated):
            • Go to the Alignment monitoring ELOG and search for the latest entry in the logbook with the current fill number and system "RichN mirrors".
            • Read these instructions for more information on how to check the plots in the logbook and assess the quality of the alignment procedure.
              • Ask the Shift Leader to help you, if needed
              • Simple version: If the big-font numbers in the bins of the 2D histograms in the PDF are ALL between -2 and 2, the alignment should be flagged as __Good__. If this isn't the case, then please read the full instructions. If you received any sort of alarm mentioning contacting the alignment piquet, then please read the full instructions.
            • After you did this, reply to the logbook entry, and as you reply you can simply change the Status label to either Good or Bad. The Status will propagate to the original logbook entry.

Data Manager: Instructions

Alarms will be sent to the Shift Leader / Data Manager when the mirror alignment updates.

NOTA BENE: If the mirror alignment doesn't update due to sanity check failure, or if an alignment doesn't update due to lack of HLT1 Line events, etc... the mirror aligners will be sent an email and will investigate. This is not an urgent situation because the mirror alignment provided to HLT2 will not have updated in these cases.

Should you receive an alarm, open the Alignment Monitoring ELOG in a separate tab/window.

First, note if you are looking at a RICH1 or a RICH2 alignment.
Each RICH is aligned separately; you need to check both.
If only RICH2 ran, or RICH1 ran but RICH2 did not run within 20 later, contact the Alignment piquet ( Phone 16 14 87 ), who will investigate and report to the Mirror Aligners.

The mirror alignment should update only occasionally (sometimes more often during ramps), and when it does it should update to reasonable values. This is what you will be tasked to verify.
If there was an update, this will be clear from the subject of the Alignment Monitoring ELOG entry, and a PDF file will be attached.
*Please look at the plots in the PDF file, following the instructions below* (If you are new to this, reference examples are available after the instructions).

Instructions for checking the 2D histograms:

  • These are the key histograms to be checked. From these we determine if the Cherenkov rings are distorted (i.e. the mirrors are misaligned).
  • The big-font numbers in the histogram bins show for each and every mirror in the RICH detector the final mirror tilt compensations (e.g. rotations with respect to the local Y or Z axis in the mirror plane, w.r.t. the existing database conditions) divided by our alignment tolerance then truncated, not rounded.
      • Divided by our alignment tolerance then truncated, not rounded means that if the tolerance for a particular mirror tilt compensation was 0.03 mrad, then you would see:
        • "2" if the compensation were 0.07 mrad (this mirror tilt is beyond the tolerance)
        • "-0" if the compensation was -0.02 mrad (this mirror tilt is within the tolerance)
        • et cetera...
  • Each big-font number is in a histogram bin that represents a not-to-scale relative global X and Y location of a specific RICH mirror. The mirror numbers are also displayed in the boxes, but with a smaller font; do not worry about these.
  • If the mirror alignment did not update, every big-font number will be "0" or "-0" and every box will be green. This is ideal. You should not have received an alarm in this case, unless there was a change in magnet polarity and an update is made by default.
  • If the mirror alignment did update, then some numbers will be "1" or "-2", etc... and the box will not be green. This is normal if an update is needed.
  • If the magnet polarity has not changed since the last fill where a mirror alignment ran and if any big-font number in a box is >= "3" or <= "-3" you will get an alarm saying the variations are too large.
    • If instead the magnet polarity has changed since the last fill then you will get the alarm only if any big-font number in a box is >= "6" or <= "-6".
    • In case of this alarm, you will need to follow the alarm instruction to either call (Phone 16 14 87) or email (lhcb-realtimealignment-trackingsystem@cernNOSPAMPLEASE.ch) the Alignment piquet, depending on the time of day.
      • Note the mirror alignment has a sanity check to avoid updating the DB in this case, and the Alignment piquet will confirm that the DB was not updated.

If you did not have to call or email the Alignment piquet, mark the run as Good in the Alignment Monitoring ELOG.

If you did have to call or email the Alignment piquet, mark it as Bad, and make sure to leave all details in your Alignment Monitoring ELOG reply.

If you have any question or concern about the monitoring plots that is not answered above, email the MirrAlign Experts so that they can address your concern and update the instructions!

Example ELOG entries and plots

When no update was made to the existing DB version for the mirror alignment:

  • In the Alignment monitoring ELOG, there will be no plots for you to look at in this case.
  • The alignment should already be flagged as Good.


Here are example ELOG entries where an update was made to the existing DB version for the mirror alignment:

  • Flag the alignment as Good if it passes the checks above.
  • RICH1
    • R1A1.jpg
  • RICH2
    • R2A1.jpg


Here are example ELOG entries where an update was made to the existing DB version for the mirror alignment, but the updated DB is consistent with the previous DB:

  • This could only happen on change of magnet polarity, when the RICH mirror alignment updates are forced.
  • Flag the alignment as Good if it passes the checks above, if it isn't already flagged as Good.
  • RICH1
    • R1B1.jpg
  • RICH2
    • R2B1.jpg


Here are example ELOG entries where an update was made to the existing DB version for the mirror alignment, that is extremely severe:

If you have any question or concern about the monitoring plots that is not answered above, email the MirrAlign Experts so that they can address it and update the instructions!

Return to TWiki Top

HLT piquet

HLT piquet: Instructions for RICH mirror alignment

Now that the mirror alignment is making automatic decisions, HLT2 no longer needs to suspend its automatic mode on each magnet polarity switch.
We would only recommend that HLT2 runs manually during the start-of-year ramp, until there is enough data for a mirror alignment to be provided, since there may have been changes/shifts in detector components during long shutdowns.

When HLT2 is running in its automatic mode:

  • If there is a run/fill taken over 18 hours ago with a missing mirror alignment version (run marked yellow in the FarmStatus Run database), the HLT piquet should confirm by email (lhcb-realtimealignment-trackingsystem@cernNOSPAMPLEASE.ch) that the Alignment piquet is working with the Mirror Aligners to assign a version within a reasonable time period (at most 36 hours after the missing fill).
    • Timescales may be modified at the HLT piquet's discretion, based on the status of the HLT2 buffer.

When HLT2 runs manually:

  • The mirror aligners should be aware when HLT2 is running manually. The mirror aligners must produce an alignment version within a reasonable time period, depending on the state of the HLT2 buffer (e.g. more time is allowed at start-of-year ramps).

Return to TWiki Top

Alignment piquet

Alignment piquet: Instructions for RICH mirror alignment

  • There are now entries in the Alignment Monitoring ELOG that come from the RICH mirror alignment, for the Data Manager to flag. The same rules for flags apply for other alignments:
    • The Data Manager will look at these entries and flag "Unchecked" alignments as "Good" or "Bad".

  • The Alignment piquet will, as usual, be called in case any of the alignments fail, the alignment didn’t run, the LHCbA partition is in ERROR (for longer than a short period, it may still be configuring).
    • In these situations the piquet should investigate the problem; in case it is a general Alignment partition (or Online farm) problem the Piquet should try to solve it as usual. Do check the standard Alignment piquet known issues and workarounds first.
    • If an alignment v has not yet been provided to HLT2 for a particular set of runs/fills, then perhaps try to run the mirror alignment again to try to identify the problem if it is not obvious from the logs, to see if it was possibly a one-time LHCbA/farm glitch.

  • If however the issue is clearly in the Rich mirror alignment code, or specific to the Rich mirror alignment, send a report to the Mirror Aligners via the RICH ELOG (Mirror Alignment entries only). Further information for quick diagnosis of the Rich mirror alignment code is in the subsection below.

  • Please be aware of the Data Manager instructions above regarding the reading of our histograms, as certain situations could result in a call or email from the Data Manager to you. The following are the reasons why you may have received a call or email, and what your responsibility is in each case:
    • The 2D histograms are outside the range of the sanity check, the mirror alignment did not converge, or our Iterator recieved an unexpected STOP command.
      • Confirm that the DB was not updated and provide the confirmation to the MirrAlign experts via the RICH ELOG. Also let the MirrAlign experts know if there are any detector conditions or problems with the alignment farm that could have caused such a result. You may wish to simply try to run the mirror alignment again in the case of an unexpected STOP command.

  • The Alignment piquet will be contacted by the HLT piquet via email, if for some reason a mirror alignment version is not assigned to any runs or fills after ~18 hours.
    • If the mirror alignment v was not provided to HLT2 for a particular set of runs/fills, but then the alignment works normally on subsequent fills and the existing mirror alignment is validated, the Alignment piquet is free to assign the existing mirror alignment as the mirror alignment v. The HLT piquet should be informed of your manual decision. This could (and should) be done even before you are contacted by the HLT piquet.
    • Otherwise the Alignment piquet should consult with the Mirror Aligners to come to a decision, and together we will inform the HLT piquet. In this case the HLT piquet should be kindly asked to assign the mirror alignment v.
    • If there is an ongoing problem with the Rich Mirror Alignment, and it is not solved or acted upon by the RICH mirror alignment experts and/or Alignment piquet in 36 hours, we should strongly consider asking the HLT piquets to use the current alignment versions to process HLT2 backlog, unless there is good reason not to do so (in which case we must explain this in detail to the HLT piquet, and it should be discussed further at the run meeting).

  • If for some reason the Mirrors HLT line event collection does not stop after 3M (or the typical number) events, feel free to run the alignments manually on just the first 3M events; we benefit from the extra events but not enough to take up more of the HLT farm processing time.

  • If the magnet polarity changed and a sanity check fails while the mirror alignment is in automatic decision mode, we no longer provide the current alignment version to HLT2; instead we provide no version and the mirror aligners must be asked to provide the version (though they should already be aware of the situation and working on it).

  • If there is any strange activity e.g. with the HLT Farm not configuring and mirror alignments not running successfully repeatedly, please note this in the Alignment monitoring ELOG (as you normally would). If you have time email the MirrAlign Experts, just so we are aware of what happened but we are checking to see that the alignments are completing in any case.

  • If the RICH mirror alignment for some reason runs to completion more than once (and two entries show up in the Alignment Monitoring ELOG), and if any alignment except the first RICH1 alignment results in an update of the constants, then the alignment for the current fill is fine but the alignment for the following fill will have the wrong starting version. In this case please email the MirrAlign Experts, so they can correct it in time for the next fill, if possible. The mirror aligners should also be reminded to check the CKresTrend and UpdateTrend text files for multiple entries.

If you have any question or concern about operation of the mirror alignments not answered above, email the MirrAlign Experts so that they can address it and update the instructions!

Alignment piquet: Locating a problem in the RICH mirror alignment software

  • We store the logging messages from hlt02 to the following log files.
    • /group/rich/AlignmentFiles/Logging/Rich1_hlt02.log
    • /group/rich/AlignmentFiles/Logging/Rich2_hlt02.log
  • Note that all hlt02 output goes to these files. To help sift through it run the following on the file [I call it $file] (though it depends on what clutter happens to be running on hlt02 at the time):
    • grep -vE "(LHCb2_HLT02_Hlt2Adder_0|Hlt2SaverSvc|MARK|BusyPub|BusyMon|JobOptionsSvc|environtment)" $file
  • These web pages keep track of the log files, and may help you look for issues inside the log files without having to open up a plus window (and the above grep is already performed):
  • The key thing to look for is a "Traceback" (which for some reason shows up as INFO instead of ERROR), and report it.
  • Early "tmSrv" errors by themselves might be an indication of a HLT farm problem.
  • FATAL and ERROR messages are important to note in the RICH ELOG.
  • You may see some disconcerting WARNINGs as well. Free to comment in the RICH ELOG about this, but this common WARNING you can always ignore:
    • Jul08-162437[WARN] hlt02: INFO: RESET command received at 2017-07-08 14:24:37 UTC

  • An Alignment piquet could also go through the LHCbA ELOG as they might do for any other LHCbA issue.
    • This would be necessary for the mirror alignment, for example, to identify if the worker HLT farm nodes are somehow suddenly being misconfigured.

  • If the RICH mirror alignment appeared to fail directly due to a traceback having to do with the mirror alignment code (and not having to do with say, a farm issue), please send a report to the Mirror Aligners via the RICH ELOG and we will investigate.

If you have any question or concern about operation of the mirror alignments not answered above, email the MirrAlign Experts so that they can address it and update the instructions!

Alignment piquet: "Common" rare issues

  • One rare issue is that the mirror alignment will show Polarity information not found errors (Polarity is "None"). This happens in the rare instance when the RunDB is down. The automatic update will be disabled in this case. However the mirror alignment may not finish running at all, because if the Run DB is down other DBs relevant to the mirror alignment may also be down. If this ever happens, please check with the Online piquet the current DB situation, and when it is fixed, if there is time before the next fill please attempt to run the mirror alignments again if they did not successfully complete the first time.

  • Another rare issue is that the RICH calibrations are not available for some subset of the Runs that we wish to run over. This is usually because the those Runs are too short, and in principle such Runs should be automatically discarded. However if the system is a bit slow, the mirror alignment will try to run over these Runs, and immediately crash (and may repeat up to 9 times if on autopilot). The solution is to identify from the logs which Runs are problematic, and then only select the subset of non-problematic Runs in the current fill, through the run selection in the LHCbA panel (The “ctrl" key allows one to deselect individual runs from the “Choose runs for alignment” panel); then re-run the alignment and it should work. If the next fill has already started though don't bother; the HLT piquets will ask you to follow the usual procedure for a missing alignment.

If you have any question or concern about operation of the mirror alignments not answered above, email the MirrAlign Experts so that they can address it and update the instructions!

Return to TWiki Top

RICH piquet

RICH piquet: Instructions

You can help the MirrAlign Experts by looking at the following information in AlignmentView (there is color/symbol coding to help you):

  • converged =
    • If this is "False", send a report on this alignment via AlignmentView.
  • Number of events =
    • If this is not good, send a report on this alignment via AlignmentView.
    • The number of required events is listed in the Summary, as the number after "requiredEvents".
  • LHCbA config time =
    • If this is more than 20 minutes, send a report on this alignment via AlignmentView.
  • Alignment time =
    • If this is more than 20 minutes, send a report on this alignment via AlignmentView.
  • DB updated =
    • If this is True, send a report on this alignment via AlignmentView (but this is not a bad thing when it happens).
    • If this is None, send a report on this alignment via AlignmentView (also not a bad thing when it happens, but the expert needs to take action).
  • savedir =

Then have a look at the plots in the "(summary plots)" PDF file (note, if there are many iterations you may need to open the PDF file separately as there is a limit to what can be seen on the RICH ELOG (Mirror Alignment entries only) page):

RICH1

  • RICH1 {Primary,Secondary} mirrors Local {Y,Z} compensations, divided by the tolerances (First page of the PDF only)
    • All green is very good and should occur most of the time, close to green means the alignment updated but probably is still OK, unless polarity has changed in which case the alignment is probably OK but some mirrors may be far from green. If many mirrors are far from green (3 or more), send a report on this alignment via AlignmentView.
      • If you can't differentiate the colors too well, for now use the ("moni" plots). Here "0" or "-0" is the same as green. "1" or "-1" is further away, etc...
    • The mirrors are arranged on the plot as they are physically in the detector (but not to scale). The mirror number is also displayed on the 2D histogram, but smaller.

  • RICH1 Cherenkov angle resolution plots (Look for plot of the highest numbered Iteration)
    • The width of this plot is the integrated Cherenkov angle resolution from the data used to align the mirrors only, with additional section criteria.
      • If you do not see a nice Gaussian shape on top of a substantial but not overwhelming background, send a report on this alignment via AlignmentView.
      • If the fit looks awful, send a report on this alignment via AlignmentView.
      • If Gauss Mean > |0.0010|, send a report on this alignment via AlignmentView.
      • If Gauss Sigma is not between 0.00162 and 0.00172, send a report on this alignment via AlignmentView.

  • RICH1 Cherenkov angle resolution per It. plot
    • If the mirror alignment did not update, there is only one data point, and there is nothing else to be concerned about.
    • If the mirror alignment did update, there will be multiple data points; they should generally have an approximate sideways or decreasing trend, but sometimes have an increasing trend.
      • If the trend increases slightly [ total change < 0.020 mrad ], this is not a problem.
      • If the trend increases sharply [ total change >= 0.020 mrad ], send a report on this alignment via AlignmentView.

RICH2

  • RICH2 {Primary,Secondary} mirrors Local {Y,Z} compensations, divided by the tolerances (First page of the PDF)
    • All green is very good and should occur most of the time, close to green means the alignment updated but probably is still OK, unless polarity has changed in which case the alignment is probably OK but some mirrors may be far from green. If many mirrors are far from green (4 or more), send a report on this alignment via AlignmentView.
      • If you can't differentiate the colors too well, for now use the (moni plots). Here "0" or "-0" is the same as green. "1" or "-1" is further away, etc...
    • The mirrors are arranged on the plot as they are physically in the detector (but not to scale). The mirror number is also displayed on the 2D histogram.

  • RICH2 Cherenkov angle resolution plots (Look for plot of the highest numbered Iteration)
    • The width of this plot is the integrated Cherenkov angle resolution from the data used to align the mirrors only, with additional section criteria.
      • If you do not see a nice Gaussian shape on top of a substantial but not overwhelming background, send a report on this alignment via AlignmentView.
      • If the fit looks awful, send a report on this alignment via AlignmentView.
      • If Gauss Mean > |0.0010|, send a report on this alignment via AlignmentView.
      • If Gauss Sigma is not between 0.00063 and 0.00068, send a report on this alignment via AlignmentView.

  • RICH2 Cherenkov angle resolution per It plot
    • If the mirror alignment did not update, there is only one data point, and there is nothing else to be concerned about.
    • If the mirror alignment did update, there will be multiple data points; they should generally have an approximate sideways or decreasing trend, but sometimes have an increasing trend.
      • If the trend increases slightly [ total change < 0.010 mrad ], this is not a problem.
      • If the trend increases sharply [ total change >= 0.010 mrad ], send a report on this alignment via AlignmentView.

If you have any question or concern about the quality of the mirror alignments, send a report on this alignment via AlignmentView.
We will evaluate the situation and respond in due course.

Return to TWiki Top

RICH Mirror Alignment Experts

As an expert, you of course should know all of the above, and know that all the other information you need to know may be found on LHCbRichMirrorAlign and its child pages. However, here's some more tips regarding the analysis of our output.

RICH Mirror Alignment Experts: Additional plots

There are additional 1D plots in AlignSummary.pdf that you can find by clicking on the particular Alignment's name in AlignmentView.

RICH1 Example, alignment updated, see last two plots

  • RICH1 primary mirror tilts w.r.t. to current DB vs mirror number
  • RICH1 secondary mirror tilts w.r.t. to current DB vs mirror number RICH2 Example, alignment updated, see last two plots
  • RICH2 primary tilts w.r.t. to current DB vs mirror number
  • RICH2 secondary tilts w.r.t. to current DB vs mirror number

These facilitate visualizing the mirror compensations relative to each other, and the thresholds which are also drawn on these plots.

NEW There are also now alignment trend plots being automatically generated each time the alignment runs. These can currently be found in the RICH ELOG entry after an alignment completes.

RICH Mirror Alignment Experts: When the mirror alignment is making automated decisions and HLT2 is in automatic mode

In addition to the usual set of tasks for the MirrAlign Experts:

  • The Alignment piquet will inform you when the mirror alignment fails.
    • If it is not an obvious issue with the code (no traceback, and no traceback reported by alignment piquet in the RICH ELOG), the Alignment piquet may have already been able to solve the problem.
    • Otherwise, if the mirror alignment fails to an issue with out code, investigate the cause. If you can solve the issue yourself due so, please use our workflow for dealing with the code.

  • If for any reason certain runs or fills do not have a mirror alignment assigned to them, you will be notified by the Alignment piquet if it is not obvious to them what the alignment version should be.
    • Carefully try to figure out what version number should be assigned. If in absolute doubt then simply use the previous version number.
    • Send the following email, with the v numbers in place of "N" in "vN", and the Fill Number in place of "6xxx" to: Clara Gaspar <Clara.Gaspar@cern.ch>, lhcb-hlt-piquet <lhcb-hlt-piquet@cern.ch>, and the Alignment piquet <lhcb-realtimealignment-trackingsystem@cern.ch>

Dear Clara and HLT piquets,

Please assign 

vN to RICH1
and
vN to RICH2

for the unprocessed Fill 6xxx.

  • If the magnet polarity changed and a sanity check fails, in autoUpdate mode, you will have received a WARNING email and we no longer provide the current alignment version to HLT2; instead we provide no version and the mirror aligners must solve the problem and try to run the mirror alignment again, unless it would be more appropriate to create a new version from the maybe file, or give the existing version.

  • If the RICH mirror alignment for some reason runs to completion more than once (and two entries show up in the Alignment Monitoring ELOG), and if any alignment except the first RICH1 alignment results in an update of the constants, then the alignment for the current fill is fine but the alignment for the following fill will have the wrong starting version. In this case we will be contacted and the responsible expert for the mirror alignment should correct it by renaming the subsequent version(s) to v "_maybe" in time for the next fill, if possible (if not possible, don't worry about it). You should also check the CKresTrend and UpdateTrend text files for multiple entries.

RICH Mirror Alignment Experts: When HLT2 is not in automatic mode

In addition to the usual set of tasks for the MirrAlign Experts:

  • A mirror alignment will be performed
    • If the mirror alignment fails (rare), investigate the cause. If you can solve the issue yourself due so, please use our workflow for dealing with the code.
      • If it is not an obvious issue with the code (no traceback, and no traceback reported by alignment piquet in the RICH ELOG), you can get help from the Alignment Piquet (look up who is Alignment Piquet in the Shift DB, then call Phone 16 14 87 [CERN] (To call a CERN mobile from outside CERN, dial as if you were calling the Swiss number (+41) 75 411, and the last 4 digits of the mobile number.), or email if not critically urgent).
    • If the mirror alignment succeeds, note the v number returned for RICH1 and RICH2 mirror alignments.
      • If an alignment did not update, your email will say something like "The alignment in the DB is still v6." Or you can look in the RICH ELOG or AlignmentView summary.txt file, which should start with something like:
        richDetector            = 1
        DB updated              = False
        resulting DB            = v6
        converged               = True
        
      • If an alignment did update, your email will say something like "Due to a significant change from the alignment in the DB, or a change in magnet polarity, the DB was automatically updated to v7." Or you can look in the RICH ELOG or AlignmentView summary.txt file, which should start with something like:
        richDetector            = 1
        DB updated              = True
        resulting DB            = v7
        converged               = True
        
    • Note also the Fill number from the RICH ELOG entry for the particular alignment, or from AlignmentView
    • Send the following email, with the v numbers in place of "N" in "vN", and the Fill Number in place of "59xx" to: Clara Gaspar <Clara.Gaspar@cern.ch>, lhcb-hlt-piquet <lhcb-hlt-piquet@cern.ch>
      Dear Clara and HLT piquets,
      
      Please assign 
      
      vN to RICH1
      and
      vN to RICH2
      
      for the unprocessed fills preceding and including Fill 59xx.
      

RICH Mirror Alignment Experts: RICH mirror alignment AlignmentView

RICH Mirror Alignment Experts: RICH ELOG (Mirror Alignment entries only)

RICH Mirror Alignment Experts: When the mirror alignment exits immediately due to missing RICH calibrations or for another sanity-check reason

  • The LHCb ELOG may say the RICH1 and/or RICH2 alignments completed, but that no monitoring information was posted in the Alignment Monitoring ELOG. This is likely because the RICH alignment satisfied a condition for immediate exit (e.g. RICH calibrations missing). Unfortunately, in this case LHCbA will assume a crash and instead try multiple times to get it to run. In this case there will be many "WARNING" emails sent by MirrAlign Monitor and also "WARNING" entries posted in the RICH ELOG. The Alignment Piquet should first confirm the reason for immediate exit by going to the RICH LogViewer and select "RICH 1 Align" and/or "RICH 2 Align", and a time period long enough to cover the scope of the issue. They will likely see something like this:
Jun09-015210[WARN] hlt02: WARNING: Missing RICH calibration detected: /group/online/hlt/conditions/LHCb/NoYear/210/210050/rich2calib.xml
Jun09-015210[WARN] hlt02: WARNING: Mirror alignment will be skipped.
Jun09-015210[WARN] hlt02: WARNING: These runs do not yet have RICH calibrations available: ['210050']
Jun09-015210[WARN] hlt02: INFO: RESET command received at 2018-06-08 23:52:10 UTC
In this case they will try to re-run the mirror alignment(s) without the offending run. If all of the runs are causing a problem, or in any other case, the Alignment piquet will send an email to the Mirror Aligners and we should find out if there is another cause for this issue.

Return to TWiki Top

Contact info for RICH Mirror Alignment Experts, RICH Mirror Alignment Software Developers, and RICH experts

On rare occasion, you may wish to contact RICH people outside of the usual flagging of an alignment in the Alignment monitoring ELOG (Data Managers), or the reporting of an issue in the RICH ELOG (Piquets).

Return to TWiki Top


Future:

Start from a list to make a Monet/Presenter solution well suited to the Alignment tasks needs.
   * Need graphical features that can help to judge the quality of the plots, e.g.: vertical lines to set limits to PV left-right distributions for VELO alignment…
   * Alarms (also, is it possible to have one when an alignment task didn’t run?)
Clearly the aim is to concentrate on Monet as it will be used and supported in the future

Desirables/Ideas for improvement
   * Is it possible to display the fraction of collected events for each task somewhere visible to the shifters? 
      * In this case would be easier to understand where one alignment is expected (as well as immediately realize if events are not collected when they should)

Topic attachments
I Attachment History Action Size Date Who Comment
JPEGjpg Mon_Presenter.jpg r1 manage 74.4 K 2017-06-29 - 15:15 ParasNaik  
JPEGjpg Mon_Rich1_1.jpg r1 manage 186.5 K 2017-06-29 - 15:15 ParasNaik  
JPEGjpg Mon_Rich1_2.jpg r1 manage 171.4 K 2017-06-29 - 15:15 ParasNaik  
JPEGjpg Mon_Rich2_1.jpg r1 manage 242.0 K 2017-06-29 - 15:15 ParasNaik  
JPEGjpg Mon_Rich2_2.jpg r1 manage 178.3 K 2017-06-29 - 15:15 ParasNaik  
JPEGjpg R1A1.jpg r2 r1 manage 211.0 K 2017-09-04 - 19:23 ParasNaik  
JPEGjpg R1A2.jpg r2 r1 manage 101.8 K 2017-09-04 - 19:23 ParasNaik  
JPEGjpg R1A3.jpg r2 r1 manage 56.3 K 2017-09-04 - 19:24 ParasNaik  
JPEGjpg R1B1.jpg r1 manage 188.0 K 2017-09-04 - 19:35 ParasNaik  
JPEGjpg R1B2.jpg r1 manage 103.5 K 2017-09-04 - 19:35 ParasNaik  
JPEGjpg R1B3.jpg r1 manage 56.8 K 2017-09-04 - 19:35 ParasNaik  
JPEGjpg R1C1.jpg r1 manage 170.2 K 2017-09-04 - 19:45 ParasNaik  
JPEGjpg R1C2.jpg r1 manage 104.9 K 2017-09-04 - 19:45 ParasNaik  
JPEGjpg R1C3.jpg r1 manage 43.7 K 2017-09-04 - 19:45 ParasNaik  
JPEGjpg R2A1.jpg r2 r1 manage 263.1 K 2017-09-04 - 19:24 ParasNaik  
JPEGjpg R2A2.jpg r2 r1 manage 112.3 K 2017-09-04 - 19:24 ParasNaik  
JPEGjpg R2A3.jpg r3 r2 r1 manage 81.1 K 2017-09-04 - 19:28 ParasNaik  
JPEGjpg R2B1.jpg r1 manage 266.0 K 2017-09-04 - 19:35 ParasNaik  
JPEGjpg R2B2.jpg r1 manage 108.9 K 2017-09-04 - 19:35 ParasNaik  
JPEGjpg R2B3.jpg r1 manage 62.2 K 2017-09-04 - 19:35 ParasNaik  
JPEGjpg R2C1.jpg r1 manage 237.2 K 2017-09-04 - 19:45 ParasNaik  
JPEGjpg R2C2.jpg r1 manage 107.0 K 2017-09-04 - 19:45 ParasNaik  
JPEGjpg R2C3.jpg r1 manage 48.4 K 2017-09-04 - 19:45 ParasNaik  
JPEGjpg Rich1_Primary.jpg r1 manage 40.2 K 2017-05-29 - 16:05 ParasNaik  
JPEGjpg Rich1_Secondary.jpg r1 manage 40.8 K 2017-05-29 - 16:05 ParasNaik  
JPEGjpg Rich2_Primary.jpg r1 manage 61.4 K 2017-05-29 - 16:05 ParasNaik  
JPEGjpg Rich2_Secondary.jpg r1 manage 41.7 K 2017-05-29 - 16:05 ParasNaik  
Edit | Attach | Watch | Print version | History: r68 < r67 < r66 < r65 < r64 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r68 - 2018-08-30 - ParasNaik
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback