Workflow Team Meeting - Jan 8 4PM CERN time

Vidyo Link https://indico.cern.ch/event/363272/

Attending

• Jen, Julian, Andrew, Alan, Dima, SeangChan, and the Other Andrew, Dave, Ian

Personel

• EU - Do we have shifters over the Holidays?
• Luis - in Columbia until ??? working remotely
• John (site support) Traveling on Sun 21, in Ecuador till end of Jan. Off until Jan 5, working remotely Jan 5-end of Jan

News

• Andrew and Dave are both stepping down as Redigi co-ordinators but we won't have anybody doing it for the time being
• move to bug tracking system for issues? so we can open/close assign something to discuss
  • we already have elog, hypernews, ggus, github do we need to introduce a new system? Or should we use one of these
• Christmas Production was basically non-existant this year, nobody at the Monday Meeting was sure why
  • Dima - is there anything in the pipeline ? everyone at PPD is still on their way back, no general meeting so no news until next week
  • the only thing that didn't show up was the upgrade sample, we have no news on it. The meeting is 1st or 2nd wk of Feb, we need a drop dead date and agree to force complete and run at a certian %
  • nothing in the queue or waiting for resources so things are quiet
• Proxies - Alan will take over the proxies again, Alan will work with Krista and Diego to get this working properly the ones we have are good till the 12, we have 2 more days before we need to take care of this but let's take care of it the right way. We can't give voms roles to service certs, so maybe a service cert won't work????
  • Tony said if you use a service cert it can be autmatically updated but user certs can't. We could register a voms cert and it could work.
  • the difference between a user cert and a service we don't have to put the pass phrase in the service cert. Do the sites treat them the same? We need to register them, and make sure the sites are registering stage out etc properly in the role. Only CERN and RU are having issues, but if we can get them to make them work. We need to make sure we get things renewed etc in time so we don't end up in trouble. We need to make sure this is done, and then document it. The experation date should send a mail to several people so we can assign somebody the job of renewing it.

EU Operators Meeting Notes

• No EU shifts??? Now that we are back we need to figure out what we are doing with shifting EU time
• Julian - needs to get with Christoph about EU operators

Site support

• Stage out errors at PIC https://cms-logbook.cern.ch/elog/Workflow+processing/18255
  • not sure if it is a site issue or a WF issue
• Submit errors at IN2P3 https://cms-logbook.cern.ch/elog/Workflow+processing/18253
  • site appears to be up but it's not matching???

Agent Issues

• disk thresholds https://cms-logbook.cern.ch/elog/Workflow+processing/18250
  • bumped it up to 75% and cleaned up space to give us time without blowing things up.
• 202 - https://cms-logbook.cern.ch/elog/Workflow+processing/18249
  • maybe add the task to CSP shifters to look at the tab to look at this and report that things are down
  • is there something on condor monitoring that we could have them look at?
  • over the next week we all come up with monitoring and list of things non-experts can look at. We will discuss this at the next meeting then feed that back to people to add to the CSP shift duties.
  • Julian will start the twiki with some of the things out of the operations twiki to add things, bring this up at Monday Meeting

Redeployment plan

• waiting for new agents, hopefully next week or so end of next week???
• we need to start draining and re-deploying agents that are filling up
• Feb deployment will use reqmgr2, so we need to start updating everything then and scripts will need to be changed. MCM needs to be fixed

Workflows

• KIT not mapped properly is not giving 95% to the piolt role, so in the global pool we are not getting fair share and we are having to run all of our jobs through the low-prio agents
• we need to make sure that whoever 's proxy we are using we are working at CERN, we still are not working at HLT machines

ReDigi

• state out issues at PIC
  • have Jorge look at disk at PIC and then work up a deletion request
• Submit errors at IN2P3
• workflows in comlplete : https://cms-logbook.cern.ch/elog/Workflow+processing/18256
  • I was able to clear ~ 1/2 wf's that were there before Christmas but there are still some that need to be addressed

miniaod's

Rereco

• some of the ReReco that is stuck in complete should be closing out - https://cms-logbook.cern.ch/elog/Workflow+processing/18228
• there are a few that could maybe benifit from ACDC, I need to dig further

Store Results

MonteCarlo

• stuck MC: https://cms-logbook.cern.ch/elog/Workflow+processing/18252 & https://cms-logbook.cern.ch/elog/Workflow+processing/18251

SL6 testing/SL5 Decomissioning

RelVal Andrew

AOB