-- AlbertoGasconBravo - 2015-06-04

Prompt Calibration Loop (OLD - TO BE DELETED - REPLACED BY SECTION 4 ABOVE)


UPDATE : 2011.08.10


Shifter work for prompt calibration loop is only for runs with stable beams. Job monitoring and uploads are NoisyStrip, NoiseOccupancy, RawOccupancy and Efficiency only. DeadChip, DeadStrip and ByteStreamErrors are not in operation.

Overview

  • Prompt calibration loop for SCT creates the following conditions data.
    • Conditions data of offline calibration necessary in ATLAS bulk reconstruction at Tier0 (and reprocessing at Tier1s)
      • Noisy strip only (as of today).
    • Conditions data for SCT performance monitoring
  • Prompt calibration loop processes data only for runs with stable beams
  • Shifter work is
    • Monitoring of offline jobs, which run automatically on TMS (Task Management System) and create conditions data
    • Upload of conditions data to COOL manually

List of types for processing

  • Types in the table below are automatically processed in prompt calibration loop.
    • In the automatic processing, each input dataset for a run are required to satisfy the following two criteria.
      • The data was taken with stable beam flags
      • The input dataset contains the number of events more than minimum statistics.
  • Task type is used in Task Lister for job monitoring ( see shifter work below)
  • Number of jobs in each type is defined in the processing for a nominal operation.
  • Processing timing and upload timing are described below.

Type Task type Stream Format Minimum statistics Number of jobs Processing start timing Upload timing Comments
Noisy strip sctns calibration_SCTNoise RAW 10,000 2 (1 for tag c0, 1 for tag c00) After end of run Before bulk reco Used for bulk reconstruction and performance monitoring
Dead strip sctds express_express RAW 200,000 1 After end of run When available Used for performance monitoring
Dead chip sctdc express_express RAW 200,000 1 After end of run When available Used for performance monitoring
Noise occupancy sctno express_express HIST 5,000 2 (1 for tag c0, 1 for tag c00) After bulk reco When available Used for performance monitoring
Raw occupancy sctro express_express HIST 5,000 1 After bulk reco When available Used for performance monitoring
Efficiency scteff express_express HIST 5,000 1 After bulk reco When available Used for performance monitoring
Bytestream error sctbse express_express HIST 5,000 1 After bulk reco When available Used for performance monitoring

Timing in prompt calibration loop and ATLAS bulk reconstruction at Tier0

  • For a given run, fixed 48 hours (as of today) is given to subsystems after the run finishes, as offline calibration period before bulk reconstruction starts at Tier0.
  • Processing for noisy strip, dead chip and dead strip, using RAW
    • Input datasets are RAW and are made available immediately after the run finishes.
    • Offline jobs are automatically launched on the input datasets and creates conditions data.
    • Typical processing time is a few hours (< 6 hours for noisy strip; it might take much time for dead chip and dead stip.).
    • Outputs (conditions data) are copied to SCT server every one hour by a cron job and are then displayed on our dedicated Web page ( see below) to be ready for uploads.
  • Bulk reconstruction
    • Important : Before bulk reconstruction starts, noisy strip has to be uploaded to COOL. No upload timing requirement for dead chip and dead strip.
    • HIST (monitoring histograms) is created.
  • Processing for noise occupancy, raw occupancy, efficiency and bytestream error, using HIST
    • Input dataset is HIST (monitoring histograms) and is made available after the bulk reconstruction finishes.
    • Offline jobs are automatically launched on the input datasets and creates conditions data.
    • Typical processing time is about 10 minutes.
    • Outputs (conditions data) are copied to SCT server every one hour by a cron job and are then displayed on our dedicated Web page ( see below) to be ready for uploads.

Upload to COOL

  • Uploads are manually scheduled in our dedicated Web page ( see below).
  • Actual uploads are done every one hour by a cron job.
  • Deadline for uploads
    • Upload of noisy strip has to be finished within the fixed 36 hour time-window and well before the deadline.
    • There is no upload deadline for other types : dead chip, dead strip, noise occupancy, raw occupancy, efficiency and bytestream error. Upload is done when conditions data is available.

Shifter work

  • Send message to the experts, in case of any questions and issues.
    • Message can be sent to the experts, using a link (phone mark) in the top-left corner on https://pc-sct-www01.cern.ch/24hLoop/index.php.
    • On-call expert will also receive the same message for info.
    • In this page, it takes about 10 seconds to load a list of e-mail addresses (reading on-call expert from OTP). Wait until the list of e-mail addresses appears before sending a message.
  • Uploads of conditions data to COOL
    • Go to our dedicated Web page : https://pc-sct-www01.cern.ch/24hLoop/index.php
    • Login for the shifter is automatic, reading the OTP. Please make sure that you are automatically logged in.
    • Check if Last Cron Update box in the top-right corner is updating or not. We are running cron jobs for automation.
      • If the box is in blue and the "Last Cron Update" is updating, the status is fine.
      • If this box is in red, the cron jobs might be frozen/dead. Send message to the experts.
    • Check a list of runs in time-window of prompt calibration loop
      • A list of runs can be found on ATLAS Run Query plugged into the page and are indicated as (in calib loop) just below run number.
      • Conditions data for the list of runs are created by automatic processing and are uploaded by the shifter.
      • Automatic processing will not run if input datasets don't satisfy stable beam flags and the requirement for minimum statistics ( see the table above). Hence, there will be no conditions for upload.
    • Upload the conditions data when available. Do not upload BytestreamErrors.
      • Select one of tabs (NoisyStrip, NoiseOccupancy, RawOccupancy, DeadChip, Efficiency, ByteStreamErrors or DeadStrip).
      • Repeat procedure below for each tab.
        • Criteria for uploads
          • Noisy strip : Check Test column in the page. If it's green, go ahead for the upload. If not (yellow or red), send message to the experts.
          • Other types : No criteria. Go ahead for the uploads.
        • Upload procedure
          • Select runs by clicking check boxes in upload column.
          • Click Send. A cron job is set up for the uploads every one hour.
          • Check log column some time later (after the cron job does the uploads).
            • If the log is in green, the upload was successful
            • If the log is in red, check the log.
              • During the upload : the log stays in red. No problem here.
              • After the upload : if the upload is failed (seen in the log), send message to the experts.
  • Monitoring of offline calibration jobs
    • Go to Task Lister Web page : https://tzcontzole01.cern.ch/prod1/tasklister/ or https://tzcontzole02.cern.chprod1/tasklister/
    • Monitor jobs for a list of runs in time-window of prompt calibration loop
      • Select sctcalib in Username in the most left column
        • This filters jobs for SCT.
        • if there is no job to run at all, there is no sctcalib in Username.
      • Type in Task Information specifys processing type (e.g. sctns for noisy strip), as shown in the table above.
      • Check if jobs for a list of runs found in ATLAS Run Query (plugged into the upload page) are defined or not.
      • Monitor Status in Task Information
        • RUNNING : job is defined (not running actually. e.g. waiting for input dataset).
        • YELLOW : shown as a yellow band. job is running.
        • FINISHED : job is successfully done.
        • Others states
          • Other color states will appear during transition state (e.g. from RUNNING to YELLOW).
          • Send message to the experts, if jobs are found to be failed.
Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2015-06-04 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback