Workflow Team Meeting - Oct 23 4PM CERN time

Vidyo Link

Attending

• FNAL:
• CERN :

Personel

• New US Operator Ian Dyckes

News

• Power outage!
  • We somehow managed to survive the power outage rather unscathed. Agents needed to be restarted but it doesn't appear that they lost their minds.
  • This does bring up the fact that we should at least think about what to do in case of a longer more catastrophic outage. We have agents at FNAL that can run the data, but we are then running blind. Something to think about.
• wmLHE+GEN-SIM and DIGI-RECO 53x (Phys14 MC for next run)
  • First wf ran with 100% CMSSW failure: https://cmslogbook.cern.ch/elog/Workflow+processing/17282
• Possible Urgent data coming late in the week? So far all we have is a rumor that something is coming and we have no idea what!
  • Urgent upgrade, 4 new campains starting
  • Week number 3 of having this in our news notes
• Monitoring scripts: have to point to the global pool. Can we ignore analysis jobs?
  • Production jobs are showing in Dashboard, and being monitored, but the backfill going to global pool still are not and should be watched via the WMAgent
• Ian, new US operator At FNAL for training this week.
• vocms174 and vocms227 will be given back to Ivan on October 31st. Make sure to copy your stuff before this data.
  • vocms049 (already available) is the replacement for vocms174.

Site support

• John is on vacation. Not sure if there is any site news

Sara's notes

Agent Issues

Redeployment plan

• cmsgwms-submit1.fnal.gov - currently not being used to be switched to team backfill running to global pool
• cmsgwms-submit2.fnal.gov, cmssrv219, vocms201, vocms202, vocms216 all being drained
• any word on new SL6 machines for CERN?
• What machine did we finally decide to reshoot to SL6 for restesting?

Workflows

ReDigi

• pdmvserv_EXO-Phys14DR-00009_00001_v0__141017_161834_6747 - CMSSW failure https://cmslogbook.cern.ch/elog/Workflow+processing/17282

miniaod's

• cleared out

Rereco

• nothing... literally

Store Results

• NTR

MonteCarlo

• running smoothly - had to extend a couple workflows but that is it

SL6 testing/backfill

• Monitoring scripts are not grabbing the Production - aka SL6 WF's properly And need to be modified - Luis

RelVal Andrew