Workflow Team Meeting - March 17 4PM CERN, 10 FNAL time

Vidyo Link

• https://indico.cern.ch/event/493678/

Attending

• FNAL:
• US:
• CERN :

Personnel

• Jorge March 24-25
• Jorge to Columbia April 15-May 2, Talk on April 27
• Welcome to Scarlet and Jesus - Workflow Team Operators
  • Introduce yourselves

News - Dima

3 top issues affecting production

• very quiet

Site support - Gaston

News & Issues

Transfers - Jorge

Workflows

ReDigi

MiniAOD

TaskChains

StepChain

• NA

Rereco

• fabozzi_Commissioning2015-Cosmics-Boff-01Mar2016_763p2_160302_100618_8061
  • file mismatch in PhEDEx and das due to file invalidation, I think the correct course of action is to Kill and clone now that we know what happend, but it is a big workflow so wanted to discuss first, could recovery work here?
    • https://cms-logbook.cern.ch/elog/Workflow+processing/23551
    • https://ggus.eu/index.php?mode=ticket_info&ticket_id=120390
    • https://ggus.eu/index.php?mode=ticket_info&ticket_id=118513

Store Results

• https://ggus.eu/index.php?mode=ticket_info&ticket_id=119273

MonteCarlo

Agent Issues

• • After next Tues, when we redeploy FNAL machines we should reboot

Agent redeployment

ReqestMgr2 Migration

Summary of the scripts to take into account:

Scripts that modify workflows or datasets in the system

1. Abort, clone, reject, assign, announce, close out, force complete, set status, change priority (they use reqMgrClient.py):
abortWorkflows (uses also dbs3Client.py)
abortAndClone (uses also dbs3Client.py and resubmit.py)
assignProdTaskChain.py
assignWorkflow.py
rejectAndClone.py (uses also dbs3Client.py and resubmit.py)
rejectWorkflows.py (uses also dbs3Client.py)
resubmit.py
announceWorkflows.p
closeOutWorkflows.py (uses also dbs3Client.py and phedexClient.py)
closeOutWorkflowsFiltered.py (uses all closeOutWorkflows*)
closeOutWorkflowsManual.py (uses all closeOutWorkflows*)
closeOutWorkflowsWeb.py (uses all closeOutWorkflows*)
forceCompleteWorkflows.py
setCascadeStatus.py
changePriorityWorkflow.py

2. Another operations:
makeACDC.py
makeAllACDC.py (uses makeACDC.py)
changeSplittingWorkflow.py
DBS3SetDatasetStatus.py (uses dbs3Client.py)
extendWorkflow.py (uses dbs3Client.py)
resubmitUnprocessedBlocks.py (uses changeSplittingWorkflow.py and assignWorkflow.py)
createSitesBackfill.py [Used for testing]
deleteInvalidOutput.py (uses also dbs3Client.py and phedexClient.py)
setDatasetStatusDBS3.py


Scripts query * from the system:

duplicateEvents.py (uses also dbs3Client.py)
stuckRequestDiagnosis.py
WorkflowPercentage.py
getDatasetStatus.py (uses also dbs3Client.py)
getInputLocation.py (uses also dbs3Client.py and phedexClient.py)
condor_global_overview.py
condor_overview.py
findWorkflows.py

Clients:

dbs3Client
reqMgrClient
phedexClient

RelVal Andrew

L3 discussion - Ajit, Jean-Roch, Matteo

Opportunistic Resources

Automatic Assignment And Unified Software

AOB

-- JenniferAdelmanMcCarthy - 2016-03-24