Workflow Team Meeting - March 17 4PM CERN, 10 FNAL time

Vidyo Link

Attending

  • FNAL: Jen, Gaston, Scarlet, Jesus, SeangChan
  • US: Allie, Matteo
  • CERN : Paola, Dima, Alan

Personnel

  • Jorge March 24-25
  • Jorge to Columbia April 15-May 2, Talk on April 27
  • Welcome to Scarlet and Jesus - Workflow Team Operators
    • Introduce yourselves
  • Good Friday and Easter Monday of at CERN
  • SeangChan taking 1/2 day next Thurs and Fri off next week

News - Dima

  • So far it looks like the schedule is on! Big Campaign April 1
    • cleanup what we have, finish tests, need to finalize T0 configuration, Alan and John are working on this, challenge is getting the agent
    • JR's "accidental test" of the T0 by using it in overflow worked
    • Testing will likely happen next Tuesday
  • premixing is done, and running tests to see what it does but it is out of question to run for April 1 round
  • multicore - what sites can we use
    • we don't yet have the list of sites for multicore yet. Then we need to have test
    • Jen and Matteo will have to figure this out next week

3 top issues affecting production

  • very quiet

Site support - Gaston

News & Issues

Date Site Into the Waiting Room Out of the Waiting Room Into the morgue Out of the morgue
2016-03-13 00:00:01 T2_RU_INR x      
2016-03-16 00:00:01 T2_FI_HIP x      
2016-03-17 00:00:01 T2_AT_Vienna x      
2016-03-22 00:00:01 T2_IT_Bari x      

Multicore Enabled Sites:

Site MaxCpu MaxMem
T2_UK_SGrid_RALPP 8 24576
T3_US_Omaha 8 16000
T2_CH_CERN_AI 8 16000
T2_CH_CERN_AI 8 16000
T2_UK_London_Brunel 8 20240
FNAL_HEPCLOUD 8 20000
T1_RU_JINR 10 25300
T2_US_Vanderbilt 8 56000
T2_FR_IPHC 8 20240
T1_US_FNAL 8 20000
T2_US_Florida 8 32768
T1_DE_KIT 8 20240
T2_DE_RWTH 8 20240
T2_DE_DESY 8 20000
T1_UK_RAL 8 24576
T0_CH_CERN 8 16000
T2_UK_London_IC 8 20240
T2_US_Nebraska 8 20000
T2_US_Nebraska 4 8000
T2_US_Nebraska 8 20000
T2_US_Nebraska 8 16000
T2_US_Nebraska 32 250000
T2_ES_CIEMAT 8 20240
T1_IT_CNAF 8 20240
T2_US_Purdue 8 24000
T2_PT_NCG_Lisbon 8 20240
T1_ES_PIC 8 20240
T3_US_SDSC 16 49152
T2_IT_Pisa 8 20240
T1_FR_CCIN2P3 8 24576
T3_US_NERSC 8 20000
T2_FR_CCIN2P3 8 24576
T2_CH_CERN_HLT 8 20000
T2_CH_CERN_HLT 8 28000
T2_US_MIT 8 20240
T2_BE_IIHE 8 20240

Transfers - Jorge

Workflows

ReDigi

MiniAOD

TaskChains

StepChain

  • NA

Rereco

Store Results

MonteCarlo

Agent Issues

  • After next Tues, when we redeploy FNAL machines we should reboot
  • Jen needs to poke Dave about redeployting agents

Agent redeployment

ReqestMgr2 Migration

Summary of the scripts to take into account:

Scripts that modify workflows or datasets in the system

1. Abort, clone, reject, assign, announce, close out, force complete, set status, change priority (they use reqMgrClient.py):
abortWorkflows (uses also dbs3Client.py)
abortAndClone (uses also dbs3Client.py and resubmit.py)
assignProdTaskChain.py
assignWorkflow.py
rejectAndClone.py (uses also dbs3Client.py and resubmit.py)
rejectWorkflows.py (uses also dbs3Client.py)
resubmit.py
announceWorkflows.p
closeOutWorkflows.py (uses also dbs3Client.py and phedexClient.py)
closeOutWorkflowsFiltered.py (uses all closeOutWorkflows*)
closeOutWorkflowsManual.py (uses all closeOutWorkflows*)
closeOutWorkflowsWeb.py (uses all closeOutWorkflows*)
forceCompleteWorkflows.py
setCascadeStatus.py
changePriorityWorkflow.py

2. Another operations:
makeACDC.py
makeAllACDC.py (uses makeACDC.py)
changeSplittingWorkflow.py
DBS3SetDatasetStatus.py (uses dbs3Client.py)
extendWorkflow.py (uses dbs3Client.py)
resubmitUnprocessedBlocks.py (uses changeSplittingWorkflow.py and assignWorkflow.py)
createSitesBackfill.py [Used for testing]
deleteInvalidOutput.py (uses also dbs3Client.py and phedexClient.py)
setDatasetStatusDBS3.py


Scripts query * from the system:

duplicateEvents.py (uses also dbs3Client.py)
stuckRequestDiagnosis.py
WorkflowPercentage.py
getDatasetStatus.py (uses also dbs3Client.py)
getInputLocation.py (uses also dbs3Client.py and phedexClient.py)
condor_global_overview.py
condor_overview.py
findWorkflows.py

Clients:

dbs3Client
reqMgrClient
phedexClient

RelVal Andrew

L3 discussion - Ajit, Jean-Roch, Matteo

Opportunistic Resources

Automatic Assignment And Unified Software

AOB

-- JenniferAdelmanMcCarthy - 2016-03-24

Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2016-03-24 - JeanrochVlimant
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    CMSPublic All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback