Vidyo Link


* FNAL - Luis, Dave, SeangChan, Jen & John * CERN - Andrew and Jullian


March 4 -> March 11 Jasper + Xavier
March 11 -> March 18 Xavier
  • Dave M is at a review prep today and will be at the review next week
  • Jen "May take" Mon or Tues off next week due to her Daughter being out of school both of those days.
  • Xavier & Jasper

Shift news

  • Started to run the stuck WF fix, starting to try to understand the outputs
    • for now the operators just run the script and post results and then wait for
  • Possibly using Savannah tickets to assign work to a specific person.
  • we need to get the operators more engaged
  • use the issues page to debug each day. Right now it is only showing issues for MC, we need to get something showing redigi/rereco as well
    • Xavier will add this onto the list of duties
  • Why wasn't CNAF taken down over the weekend? Was assumed it would only work for MC which wasn't running there but now using the SSB proceedure for taking sites in and out it does both MC and Redigi/Rereco so it should have been put in down on the SSB board. John put it in downtime yesterday, and Xavier is taking it out today now that they are back
  • Store results savannah tickets we want to move over to a shifter duty.
    • Luis will document how to do it and hand it over to the rest of the shifters sometimes in the next month, right now it is still in testbed


  • Do we have a concensus on a new time for the workflow team to meet? Yes
  • CNAF in downtime due to a "small fire" since Sunday Morning, it was never properly put in downtime on SSB, I was having issues doing so on Sunday and John Helped me Monday, did anybody EU time try to put it in or out of downtime?
  • Best time would be Thurs at 16:00 Andrew and Xavier can't make that time

Agent issues

Workflow Issues

  • Since DBS3 blocks are reported at the end of the workflow, the status of a dataset is no longer an accurate measure of workflow proggress.
  • For jobs-based workflow progress we need some API-way to access that information.


  • We have been using the trouble-summary page that Vincenzo developed:
    • If a WF has EXT in it's filename it is an extension and should not be force completed
    • If a wf is backfill it shouldn't have custodial sites so those are OK too


  • still trying to keep up/catchup on ACDC's of WF's due to xrootd issues.
  • We have a handful of wf's that had muliple clones made,
    • some of which were announced so those need to be disintangled and one set of outputs invalidated/deleted
    • ones that were not announced can just be recloned, probably quicker
  • Need to re=clone other workflows that were written to the wrong tape family

Site issues related with Workflow Team

  • Site support chat has been moved to Wednesdays at 5PM CERN time
  • Gokhan is now on shift CERN time so we need to start using him more!
  • Thiland wants some production jobs, John will try to commission them
  • CNAF was put down and up in production
  • all the sights that have always had issues will just be ignored from now on, they will be permanantly down in production status

Andrew's questions/Luis & Seangchan's answers wink


  • when you force complete a WF the datasets also need to be invalidated
    • if you use Luis's script is it automatically invalidating the data? no we should be doing a PhEDEx deletion, and doing a PhEDEx deletion will clear all the statuses
    • Luis will look into this and see if the closeout script is doing the right thing
  • the ssb script we activated the alarms, we have reactivated the alarms and now we need to start looking to see if they make sense
  • the alarms - the drain was not used for T1's before so we should turn off the T1's and alarm on them as well if they are being underutilized.
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r5 - 2014-03-11 - JenniferAdelmanMcCarthy
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback