Workflow Team Meeting - Nov 11 4PM CERN, 9 FNAL time

Vidyo Link

• https://indico.cern.ch/event/461561/

Attending

• FNAL: Matteo
• US:
• CERN : Alan, Andrew, Julian
• EU:

Personnel

• Jorge - Nov 9-13
• Julian to Colombia Dec 14, and then contract ends, will be working remotely from Colombia until the First of Jan
• Ajit out till Nov
• JR unavailable until Nov 23d
• Dima traveling next week

News - Dima

3 top issues effecting production

• Tables not updating in WMStats: https://cms-logbook.cern.ch/elog/Workflow+processing/22180
  • Can't really figure out whats going on with workflows if we don't have information on the workflows!
  • Mostly solved right now -> main issue, replication between agents and central couch.
• Stuck workflows:
  • https://cms-logbook.cern.ch/elog/Workflow+processing/22138
  • Agents have work to do for these workflows but I'm not seeing jobs show up.
• ReRecos (see below)
• changes in splitting in acdc - https://cms-logbook.cern.ch/elog/Workflow+processing/22195
  • Keep splitting for LHE acdc's.

Site support - Gaston

• No changes in the waiting room.
• No changes in morgue.

Currently in the waiting room: T2_ES_IFCA, T2_IT_Bari, T2_IN_TIFR, T2_TH_CUNSTDA, T2_PK_NCP, T2_EE_Estonia, T2_RU_INR, T2_BR_UERJ.

News & Issues

Workflows

ReDigi

TaskChains

• 1M jobs taskchain, at least 2 > 200K jobs.

StepChain

Rereco

• These are ready to be announced: https://cms-logbook.cern.ch/elog/Workflow+processing/22192
• Are we subscribing the output?
• recovery script needs a patch on WMAgent before working.
• Last report: https://cms-logbook.cern.ch/elog/Workflow+processing/22199
• Need to start returning WF's that are not 100%
  • List of missing lumis: https://github.com/julianbadillo/WmAgentScripts/blob/master/recover/listMissingLumis.py (based on the recovery script)
  • https://twiki.cern.ch/twiki/bin/view/CMSPublic/CompOpsWorkflowTeamLeaderResponsibilities#Finding_Missing_Run_Lumi_informa

Store Results

MonteCarlo

• Big RunIIWinter15GS's and RunIISummer15GS's (at least 6 > 200K jobs)

Agent Issues

Redeployment Plan

• Krista informed me it's time to reboot nodes for kernel upgrades, we can do so as we redeploy. submit1 is being done Wed afternoon.

RelVal Andrew

L3 discussion - Ajit, Jean-Roch, Matteo

Opportunistic Resources

Automatic Assignment And Unified Software

AOB

-- JenniferAdelmanMcCarthy - 2015-11-11