NEW Instructions for Alignment piquets (covering VELO, Tracker, RICH, Muon, and Calo calibration)

General Information and useful telephone numbers

  • Piquet assignment: Please self-assign your shifts to the LHCb shift database at: https://lbshiftdb.cern.ch/login.php. If you are not yet registered for these shifts contact the organizer of the real-time alignment activities (currently Giulio). Piquet duty is usually for one week, starting from Wednesday morning after the run meeting. If there is no run meeting on Wednesday then arrange a time for the phone handover. It is the previous alignment piquet's responsibility to make sure the daily report (see below) is delivered on time.
  • Phones: Alignment piquet phone: 16 1487 (from outside CERN: +41 75 411 1487). The PIN is on a sticker along the back of the phone, in case you need to restart the phone for any reason; if the sticker looks frayed please ask someone from the LHCb Secretariat to provide you with a new sticker. See this page for instructions on forwarding the Alignment piquet phone to your CERN phone. Don't forget to reset the forwarding when handing over to the new piquet.
  • Mailing lists: All alignment piquets should be signed up to lhcb-realtimealignment-trackingsystem@cern.ch mailing list. You will also want to sign up for the lhcb-onlinealignmentcalibration mailing list, but you only need to be subscribed to this while you are actively on piquet shift. To subscribe or unsubscribe from a list, go to the CERN e-groups page.
  • Meetings and reports: All piquets should be aware of the current situation of the alignment. They should report at the run meetings if there is any news. The run meetings may be daily or on a reduced schedule; please be aware of the run meetings and attend them.

Alignment Piquet trainings

  • Alignment Piquet training including RICH mirror alignment 27/12/2017 link
  • Alignment Piquet training 2017 07/06/2017 link

Tasks to do every day

  • Every day, inform yourself of what has been happening in the pit recently by consulting the shift logbook: http://lblogbook.cern.ch/Shift
  • Check the status of the alignment at https://lblogbook.cern.ch/Alignment+monitoring/
  • NEW Check the Problem Database for system "TRACK". When data managers post an entry in the logbook from the presenter, it will automatically create a report in the problem database: http://lbproblems.cern.ch/ seen in the second part labelled « Reports »
    • Every morning check the reports created during the last day
      (they appear also in blue in the daily report: https://lbgroups.cern.ch/online/DailyReport/today.html )
    • Give a severity to the entry: if it is not a real problem, set the severity to OK, otherwise minor or severe
    • Close the problem if it is solved or if it was not a real one
  • Write a daily piquet report in the Alignment logbook ( Report&pStatus=Good&pSubject=Piquet Report) before 10.30 am (it will be automatically copied at 10.30am here: https://lbgroups.cern.ch/online/DailyReport/today.html)
  • Attend the run meetings (going personally to the pit [preferable] or by connecting via Vidyo), NEW Tuesday and Friday at 9.30am (though check that the meeting frequency has not increased for some reason). Especially if you cannot even connect, please remember to fill the piquet report, especially in cases of open issues.
  • Check the Alignment histograms in the presenter - In the shift folder: /Shift/TRACK: PV Info, Track properties and PV alignment, Tracker Alignment and Velo alignment.
  • Check Alignment plots in the pdf attached in the logbook entry: see here and here for instructions on the monitoring plots in the logbook pdf for VELO and Tracker, respectively.
  • If you are called to solve a problem and you don't know how to solve it after 5 mins, call another expert
  • Make a note for everything that occurs during the week in the alignment elog-book: https://lblogbook.cern.ch/Alignment+monitoring// (register at same url if first time).
  • Make a note of known problems in the known problems page.
  • Please make a note of known solutions to common problems in the "Known issues and workarounds" section of this page.
  • You should also update this page, or other alignment pages if documentation is missing.

Strategy of the Alignment for 2016-2018

  • The alignment of VELO, Tracker, Muon stations will run automatically as 2015, with automatic updates of the constants for VELO and Tracker while there is no automatic update of the constants for the Muon (the Muon alignment is run as monitoring).
  • RICH, OT t0, LED Calo calibrations are running automatically since 2016.
  • NEW The alignment of the RICH mirrors has been switched to automatic updates of the constants since 10/10/2017 (prior to this, the alignment of RICH mirrors ran automatically, but only as monitoring).
  • Pi0 Calo calibration
    • In 2016: it is executed manually
    • In 2017: First step (production of the DST) is done automatically at the end of each fill while second step is run by hand when the accumulated statistics is enough
    • NEW In 2018, both step 1 and step 2 are automated
  • Concerning the VELO and Tracker alignment
    • At the beginning of every fill a new automatic alignment is evaluated for the VELO and the Tracker, if enough statistics is available.
    • In case the new alignment is significantly different from the old one, the alignment version is updated, a run change is applied automatically and the new alignment is used for the reconstruction of the following runs
    • The data manager has to check a few monitoring plots ( here) to verify the correct convergence of the alignment procedure and the good quality of the new alignment, (see here and here for instructions on the monitoring plots in the logbook pdf for VELO and Tracker, respectively).
    • The data manager has to reply to that entry in the logbook and set the Status label to Good or Bad

A summary table with the procedure for 2016 and the responsibilities for the different sub-tasks can be found here.

NEW in 2017: Fully automatic RICH Mirror Alignment

Since 10/10/2017, the RICH mirror alignment task runs and updates the constants if needed fully automatically (just like the VELO and Tracker alignments).

Please now read the Instructions for the Alignment Piquets regarding the RICH mirror alignment (where there is also some useful information regarding how to view the RICH mirror alignment log files, and evaluate known error messages).

Alarms and elog entries are set up in a similar manner with respect to the Velo and Tracker alignment tasks, but there are a few key differences:

  • Time to collect the event for the RICH mirror alignment is 2-3 hours after the VELO is closed (very different from VELO and Tracker)
  • The partition the task uses is LHCbA, but the code itself is different. We can have and hopefully solve farm/partition related issues (in this way investigate as for Velo and Tracker), but also problems that look related to the RICH or RICH mirror alignment code. In this latter case contact the relevant people as specified in the instructions ( here )
  • The constants produced with the RICH mirror alignments are used in HLT2. This means that in case the version is not provided or there is some issue, it is not as urgent to solve the problem as for HLT1, still it is high priority, so as soon as you become aware of the issue in the morning, investigate what you can investigate (issue with Alignment Partition etc). clarify the picture with the RICH piquet (experts when needed) and when situation is clear remember to assign the version to HLT2, writing an email to the HLT Piquet. Instructions for this action are described here. Please be very careful when doing this action on the FarmStatus panel.

NEW in 2018: pi0 Calo calibration now runs automatically through its second step

A thing that may happen during your shift is that the second step of the pi0 Calo calibration will run automatically. In the LHCbA panel, on the bottom left, you can see a bar with the percentage of the data needed to run this second step. When it arrives to 100% the second step of the pi0 Calo calibration will run automatically, if there is enough time between two fills, and send an email. After it is full, if there are long periods (>3h) without beam in which it should have run and this does not happen let Giulio, Clara, and Jean-François know.

When the second step is finished, an email is sent automatically to the calorimeter piquet (among others) who is responsible for checking the results and updating the reference file to be used for the LED calibration (which is done after each fill). Some instructions for the calo piquet can be found here.

NEW Duties of the alignment piquet

The alignment of the various detector components should be considered as important as the operation of any other subsystem

  • Any alignment problem would affect the quality of our data (like a bad behavior of part of a detector)
  • Any problem should be addressed promptly: Data will not be reprocessed, thus any misalignment affects the DQ!

The piquet can be called by the Data Manager if the alignment is flagged by him as bad or if there is crash/problem of the alignment.

  • Become familiar with the plots of the Presenter, see here.

  • Familiarize yourself with the Alignment plots in the pdf attached in the logbook entry: see here and here for instructions on the monitoring plots in the logbook pdf for VELO and Tracker, respectively. For the RICH mirror alignment see here
    • Such that looking at the plots you could confirm that everything is fine if you get a call from DM.

  • If you get a call from the DM saying that no monitoring information was posted in the Alignment Monitoring ELOG for the Velo or Tracker alignment, but the LHCb ELOG says these alignments ran

  • If you get a call from the DM saying that the new Velo or Tracker alignment is Bad
    • Check the plots in the pdf attached to the logbook entry
    • After having understood the situation, you could change the "TT, IT and OT" or the VELO alignment version to the previous ones (vXX-1, where vXX is the new one).
    • Follow the instructions here.
    • Put an entry in the logbook for any manual intervention.

  • If you get a call or an email from the DM saying that the new RICH mirror alignment looks Bad
    • Check the plots in the pdf attached to the logbook entry
    • Understand the situation (Possible problems could be alignment didn't converge, the Iterator received an unexpected STOP command).
    • Confirm the DB was not updated and provide the confirmation to the MirrAlign experts via the RICH ELOG. Also let the MirrAlign experts know if there are any detector conditions or problems with the alignment farm that could have caused such a result. You may wish to simply try to run the mirror alignment again in the case of an unexpected STOP command.
    • Full instructions here.

  • If you get a call because of the error in the alarm panel ERROR: Velo Alignment converged but constants changes is unreasonably large; please call the alignment piquet (or similar for the tracker)
    • It means that the alignment procedure found a misalignment larger than a conservative threshold. It can either indicate a real big movement or some problems with the alignment procedure. For safety, in this case the alignment is not updated.
    • This should happen really rarely.
    • Here are some information:
    • To exclude possible temporary hiccups the alignment procedure should be resetted and rerun (just go in the alignment panel, check that the correct activity and runs are selected, do a reset and then turn the autopilot on again, full details on haw to manually run the alignment available here). Hopefully the alarm will disappear.
    • In the logbook a new message should have appeared stating Variation too big, see for example here. In that message there are the alignables that moved too much, for example VeloLeft.Tx: SEVERE (0.155321 - 0.175436 = -0.0201151) limits = [0.0015,0.015] where you have the difference between the alignment constant before and after the alignment and the limits (first the limit for having an update and second the limit to trigger the alarm). Compare the variation with the second limit to see if it is just above the threshold.
    • Check the monitoring plots in the attachment to see if the new alignment seems to improve things. Compare also with the plots in the presenter that show the current situation and see if they agree with the distributions "old" in the pdf. If they do, it is probably a real movement and the new alignment should be applied. In this case, follow the instructions described here.

  • If you are called by the DM because some quality plots in the presenter show that the alignment is not optimal any more:
    • Verify that indeed the plots indicate a bad alignment, in that case a new alignment should probably be performed.

  • DM should check after 10/40 minutes that the VELO and Tracker alignment did actually run, respectively. If the alignment did not finish or run and thus did not produce any entry in the logbook for a given fill the DM will call the alignment piquet.

  • DM should check after 6 hours that the RICH mirror alignment (for RICH1 and RICH2) did actually run, respectively. If the alignment did not finish or run and thus did not produce any entry in the logbook for a given fill the DM will call the alignment piquet.
    • In these situations the piquet should investigate the problem; in case it is a general Alignment partition (or Online farm) problem the Piquet should try to solve it as usual.
    • Follow instructions and more information useful for investigation here.

  • If the RICH mirror alignment task did not provide a version of the constants to HLT2, the Alignment Piquet should assign the version ( instructions here ) Procedure to clarify the version/people to contact and how follow the instructions here.

  • If the RICH mirror alignment for some reason runs to completion more than once (and two entries show up in the Alignment Monitoring ELOG), and if any alignment except the first RICH1 alignment results in an update of the constants, then the alignment for the current fill is fine but the alignment for the following fill will have the wrong starting version. In this case please notify the mirror aligners by email (see Instructions), so they can correct it in time for the next fill, if possible. The mirror aligners should also be reminded to check the CKresTrend and UpdateTrend text files for multiple entries.

NEW Keep an eye on the Data Quality issues notified by the Data Quality Piquet.

  • More detailed instructions and help in the "Data Quality (DQ) issues related to Tracking and Alignment" paragraph of this page.

NEW Data Quality (DQ) issues related to Tracking and Alignment

As Alignment Piquet, you will receive the ELOG notifications from the DQ Piquet that concern anomalies/issues related to Tracking and Alignment (these notifications are sent to the lhcb-track_align-dataquality@cernNOSPAMPLEASE.ch mailing list, which has been subscribed with the lhcb-realtimealignment-trackingsystem@cernNOSPAMPLEASE.ch mailing list). Some of the issues reported can be easily addressed, some others require more studies. As Alignment Piquet it would be good if you could:

  • Take a look at the DQ entries and reply if you understand the issue (e.g. already known features or problems, or misunderstanding of the PQ piquet or problem with the instruction for the DQ Piquet...).
  • If you have more information to report on the issue, please do so, you can include other sub-systems to the ELOG entry, if you think it is appropriate, and/or ask explicitly for help from a specific sub-system. If you have a clear understanding of whether the data should be flagged as OK or BAD, please write so. If you have any doubt, please ask for help.
  • When you don't understand the problem or you have any doubt, please report it, lhcb-realtimealignment-trackingsystem@cernNOSPAMPLEASE.ch mailing list, is a good place also for these discussions. Please, report to the next Tracking & Alignment meeting, in case a new problem/feature is observed (not necessary for trivial mistakes or already known issues).
For any plot of the DQ you are looking at the Reference is in RED, the run you want to check is BLUE

Sometimes the plots included in the DQ elog entries are not enough to get a complete view of the run analyzed, so you might want to check other DQ plots. You can do so here:

https://lbdqmweb.cern.ch/prompt_dq/

In the DQ ELOG: https://lblogbook.cern.ch/Data+Quality/ you can also look up if the issue already occurred and investigation and conclusions.

In addition, this page contains the documentation relative to the observed Tracking and Alignment DQ issues.

Additional Information

  • If you are curious and want to see all the pdf also for the fills where an update was not needed you can have a look here and for more information read here
  • More information about the OT t0 calibration can be found here
  • Instructions to check the version of the conditions used in HLT can be found here, but please be very careful any time you are asked to do anything on the FarmStatus panel. Normally nothing should be done on this panel by the Alignment piquet

Most common problems and solutions

Instructions to change manually the alignment version in the panel

(VELO and Tracker; ask mirror aligners if needed for RICH)

If the alignment is bad, it is responsibility of the alignment piquet to go to the previous version of the alignment constants. Similarly, If a new alignment is available but it is not used, it is responsibility of the alignment piquet to use the new version of the alignment constants. Please put an entry with the run numbers in the logbook ( Alignment for any manual intervention and, please, consider this procedure to be used only in very exceptional cases!

  • From the FSM panel LHCb:TOP click on RunInfo
  • Click on "Create/Edit" under "Trigger Configurations"
  • On the bottom left part click on "Curr Vs"
  • A panel with the "Current Versions" will pop-up.

align.png

  • For Velo: Click on Velo/VeloGlobal. Then you can scroll and see the different versions. Please select the latest one (or the latest-1) .
    • Please, remind that VeloModules is not updated! Thus this one doesn't have to be changed.
  • For tracker: Click on TT/TTGlobal, TT/TTModules, IT/ITGlobal, OT/OTGlobal, OT/OTModules. Then you can scroll and see the different versions. Please select the latest one (or the latest-1) .
    • Please, remind that ITModules is not updated! Thus this one doesn't have to be changed.
  • Once you are done click on OK.
  • Please change the run, to allow to use promptly the changed version of the constants.

Instructions to copy manually the xml version

(VELO and Tracker; ask mirror aligners if needed for RICH)

It can happen for several reasons [e.g. for an error in the alarm panel ERROR: Velo Alignment converged but constants changes is unreasonably large; please call the alignment piquet (or similar for the tracker)] that the alignment xml have to be manually copied after evaluating that the new alignment improves the situation.

  • Go to /group/online/AligWork/Velo/runXXXXXX or Tracker/runXXXXXX, where the run number is the one used to evaluate the Alignment in question.
  • The xml versions can be found inside the xml directory:
    • For the Velo the precise path is /group/online/AligWork/Velo/runXXXXXX/xml/Velo/. Since we do not align for modules, you need only to copy the xml file VeloGlobal.xml. It has to be copied under /group/online/alignment/Velo/VeloGlobal/ with the name convention vLASTNUMBER+1.xml (e.g. if the last available version is v84.xml copy it under the name v85.xml)
    • For the Tracker the precise path is /group/online/AligWork/Tracker/runXXXXXX/xml/IT/, and OT and TT. Since we do not align for IT modules, in case of IT you need only to copy the xml file ITGlobal.xml. It has to be copied under /group/online/alignment/IT/ITGlobal/ (or TT/TTGlobal and TT/TTModules or OT/OTGlobal and OT/OTModules) with the name convention vLASTNUMBER+1.xml (e.g. if the last available version is v84.xml copy it under the name v85.xml). Please, make sure that you copy a new version for TT/TTGlobal, TT/TTModules, IT/ITGlobal, OT/OTGlobal, OT/OTModules. Pay extra care also to the fact that the last number in a folder may be different with the one i another folder.
    • Then, follow the instructions above Instructions to change manually the alignment version in the panel to change the version in the panel in order to match the new versions manually copied.

Please put an entry with the run number in the logbook ( Alignment for any manual intervention!

Problems with the convergency plots

(VELO and Tracker)

The alignment is based on the minimisation of a total $\chi^2$ thus this total $\chi^2$ should decrease with the iterations. As a monitoring also the total track $\chi^2$ and vertex $\chi^2$ are shown. Even if usually also those decrease with the iterations it is possible that they do not do so perfectly (e.g. go up before going down or go up with the last point). This is not a problem, especially if the variation is not so big because it is just the consequence that those are partial $\chi^2$ and not the total one which is minimised.

The alignment does not run for a fill

If an automatic alignment does not run, does not converge or crashes while running one can (and should) try to run the alignment manually. Detailed instructions can be found in OnlineAlignmentHowTo.

If the Velo or the Tracker alignment did not finish, they have to be run manually.

  • The DM should check after 10 and 40 minutes that the VELO and Tracker alignment did actually run. If it happens that an alignment does not produce any output the DM should call immediately the alignment piquet.
  • An automatic email should be sent to lhcb-onlinealignmentcalibration@cernNOSPAMPLEASE.ch with the following message "ERROR: Velo Alignment procedure failed; please call the Alignment piquet" or "Not Successfuly Completed. Please call the Alignment Piquet"
  • The expert have run manually the Velo (if it did not run) and the Tracker alignment. If possible also the muon alignment should be run.
  • As this procedure will block the automatic alignment for the Rich and for the Muon. Thus an email to lhcb-onlinealignmentcalibration@cernNOSPAMPLEASE.ch should be sent to inform the Rich mirror expert that it is needed to run the procedure.

Consider that the Velo has to be run as soon as possible at the beginning of the fill. The tracker shows small variations with time, but as the constants are automatically updated, it is needed to be run as soon as possible to maintain stable alignment performance. On the other hand the muon alignment is run only as monitoring, thus it could be evaluated even after the fill, in these cases.

If either RICH mirror alignment did not finish, these also should be run manually. However when another alignment is about to run imminently, the automatic alignments override any ongoing manual RICH alignments. So only try this if no automatic alignment is imminent (i.e. after Muon alignment and before the next fill). If LHCb/you is/are unable to run the mirror alignment for a fill, please follow the contingency plan discussed in the Alignment piquet instructions for the RICH mirror alignments.

The alignment panel goes untaken

During the data taking the panel should be taken by the run control. This is done by Clara running a script so if it goes untaken, she is the one to be called. In particular, if during some tests things goes stuck and it is day time one should give a call to Clara so that she can see what the problem is and, hopefully, avoid it for the future.

What to do if the VELO is open and closed again during a fill

In this case the VELO alignment has to be performed again and it is performed automatically

NEW The monitoring plots show a bad alignment, so a new Velo (Tracker) alignment should be evaluated

To take new data and run again the Velo (Tracker) alignment

  • From the FSM panel LHCb:TOP click on RunInfo
  • Click on "View/Edit" under "Alignment"
  • A panel with the "Alignment Writers" will pop-up.
  • Tick the box "On" for the Velo (Tracker). You should see that new data is gathered.
  • When the correct amount of data is collected the alignment procedure will be rerun automatically
  • Make sure to open the errorLog of LHCbA to check the outcome of the alignment and check the logbook (If you took data in a different run then a new entry corresponding to the same fill will appear).

Alignment Writers

They can be checked in the following way:

  • From the FSM panel LHCb:TOP click on RunInfo
  • Click on "View/Edit" under "Alignment"
  • A panel with the "Alignment Writers" will pop-up.

It is normal that the writers have the box unticked. When they are ticked they collect data and that is usually taken care of by the run control at the beginning of the fill. The possibility to tick them is there in case one wants to take a second round of data during the same fill (but usually one should not mess up with them). N.B. for the Velo and Tracker ticking them again triggers a new Velo alignment. The fact that they were off after the CCE scan is not normal.

Check and change CONDDB and DDDB tags used by Alignment

Instructions to check and change the CONDDB and DDDB tags can be found on this page

If the Autopilot of LHCb_Align is in "Help" and/or LHCb_Align stays in "error"

In this case you should be called by the Shift Leader. This happens for example when one of the alignment tasks has a problem for which cannot configure and/or run. The autopilot in this case keeps trying to configure and run afterwards, but keeps failing until goes to "help"

  • Don't leave the Autopilot "help". Switch it off (or on)
  • check which alignment task caused the problem
  • In order to understand what happened, it is useful to check the logfiles: /clusterlogs/partitions/LHCbA/daq/LHCbA.log (old files in /clusterlogs/partitions/LHCbA/daq/old/). Also trying to re-run the alignment task manually can help in the investigation
  • If the autopilot is not in "help" the next alignment task in the sequence should be run automatically as usual (TO BE CHECKED)

Update reference histograms in the presenter

If the reference histograms in the online presenter need to be updated, the script and options described in this page have to be used. Pay attention that:

  • the reference Saveset needs to be specified for each histogram.
  • to write in the /hist directory, you need to be in the « hstwriter » group
The presenter pages including tracks and PVs properties and alignment plots are the following:
/Shift/TRACK: PV Info
/Shift/TRACK: Track properties and PV alignment 
/Shift/TRACK: Tracker Alignment     
/Shift/TRACK: Velo Alignment

To have a complete list of the histograms included on each page (with the path), check the OnlineHistDB Web Interface

Example:

The command:

python /group/online/presenter/RefEditor/RefEditor.py --copy --task Brunel --histo "Track/TrackVertexMonitor/PV left-right delta x" --run 177105

changes the reference of the plot Track/TrackVertexMonitor/PV left-right delta x", included in the page /Shift/TRACK: Track properties and PV alignment from the current reference to the run 177105.

To have a different reference for the two magnet polarity the option --polarity can be used with argument UP or DOWN, this can be useful for example for the histograms in the page /Shift/TRACK: PV Info.

See the current reference and history:

The flie README.txt that can be found in the directory /hist/Reference/Brunel/ logs all the changes that are make in the reference histograms, since the the RefEditor.py script was made.

Before that, the histograms, in 2016, were taken from run 160398, so the histograms which are not listed in the README.txt file are coming from this run.

Add/update comments and/or add histograms to the presenter pages

For these kind of actions follow the page: https://twiki.cern.ch/twiki/bin/view/LHCb/OnlinePresenter To add/update comments to the presenter pages, the interesting part is the last one: 1. open the presenter and switch to Tools -> Page Editor Online (wait few minutes). 2. Then File -> “Save Page to Database” and then from the window that opens find in the folder “lhcbonr_list” the page you need and add a comment in the Page Comment field and save

Known issues and workarounds

LHCbA

  • It is possible that the job become stuck with the PARTAlign_Master in running and the various nodes (e.g. HLTA05_A) in ready. In this case from the top tree select again start run

  • If only few nodes are in the "Included Nodes and Removed Nodes" in the panel Quick Actions of the window LHCbA _HLT: TOP, try to DEALLOCATE and then ALLOCATE again from the Top panel

VELO/Tracker

  • If the sample collected to run the Tracker alignment span several runs and those runs have a different Velo alignment (i.e. the Velo alignment updated ran and triggered a run change before the Tracker managed to collect enough data) the Tracker alignment will fail. If more than 80% of the data has been collected with the same Velo alignment one can just run again the Tracker alignment selecting only those runs. Alternatively one can collect new data for the tracker alignment and run over those.

  • The folder running should not be present in the /group/online/AligWork directory before the start of the alignment. If it remained there from previous unsuccessful tests it should be removed. To remove it one must log in as online user. Changing the name of the directory also works

  • The Data Manager may call you to tell you that the LHCb ELOG says the VELO and/or Tracker alignments completed, but that no monitoring information was posted in the Alignment Monitoring ELOG (and also perhaps that despite this, the RICH alignments did post monitoring information to the ELOG). This is likely because the VELO and/or Tracker alignments ran fine, but for these alignments there is a monitoring script which controls sending information to the Presenter and the ELOG, and for some reason this script needs to be restarted. You can confirm the alignment finished by looking in the /group/online/AligWork/ directories or grep the LHCbA.log for 'hlt02' and look to see if the alignment converged. To restart the monitoring script, log into hlt02 and run the following commands and then the ELOG entries should appear in the Alignment Monitoring ELOG.
tmKill LHCb_HLT02_MoniOnlineAlig_0

RICH

  • The Data Manager may call you to tell you that the LHCb ELOG says the RICH1 and/or RICH2 alignments completed, but that no monitoring information was posted in the Alignment Monitoring ELOG. This is likely because the RICH alignment satisfied a condition for immediate exit (e.g. RICH calibrations missing). Unfortunately, in this case LHCbA will assume a crash and instead try multiple times to get it to run. First confirm the reason for immediate exit by going to the RICH LogViewer and select "RICH 1 Align" and/or "RICH 2 Align", and a time period long enough to cover the scope of the issue. Then you will likely see something like this:
Jun09-015210[WARN] hlt02: WARNING: Missing RICH calibration detected: /group/online/hlt/conditions/LHCb/NoYear/210/210050/rich2calib.xml
Jun09-015210[WARN] hlt02: WARNING: Mirror alignment will be skipped.
Jun09-015210[WARN] hlt02: WARNING: These runs do not yet have RICH calibrations available: ['210050']
Jun09-015210[WARN] hlt02: INFO: RESET command received at 2018-06-08 23:52:10 UTC
In this case try to re-run the mirror alignment(s) without the offending run. If all of the runs are causing a problem, or in any other case, please send an email to the Mirror Aligners so that they can begin to address the issue.

Presenter

  • An attentive Data Manager may ask about differences between the histograms and the references for the Presenter page /Shift/TRACK: PV Info. In their instructions it should explain that these variations are usually fine because the PV moves around a lot, but if you are asked to have a look, ask them to take a snapshot of the Presenter page and post it to the Alignment Monitoring ELOG. Then you can compare this with the plots you see in the Alignment Piquet Training slides and assess if there is any real issue.

-- FrancescaDordei - 2016-04-21

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng align.png r1 manage 348.7 K 2016-04-21 - 19:04 FrancescaDordei  
Edit | Attach | Watch | Print version | History: r47 < r46 < r45 < r44 < r43 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r47 - 2018-11-14 - ParasNaik
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback