(r5) WorkflowTeamMeeting20130610 < CMSPublic

CMSPublic Web>CompOps>CompOpsWorkflowTeam>WorkflowTeamMeeting>WorkflowTeamMeeting20130610 (revision 5) (raw view)~~EditAttachPDF~~
Link to meeting: 
https://indico.cern.ch/conferenceDisplay.py?confId=254658

---++ Personel
   * Coming off Shift: Xavier
   * Coming On Shift: Joseph
   * Jacob on Vacation until June 16
   * Andrew will be on vacation June 15-22
   * Diego is back in Colombia but is back at work this week.  
      * Diego will be taking a vacation day on Wed
   * Stenick off next week
   * Xavier - how is the shift scheduling doing?  I notice we only have one more more week on the schedule
---++Site Issues:
   * IN2P3 https://ggus.eu/ws/ticket_info.php?ticket=94750 Possible overload, or the change to lazy download. The cause not yet understood.
      * This is effecting the pdmvserv_HIG-Summer12_DR53X_RD workflows so we need to wait until this issue is resolved before starting ACDC's
   * RAL Overload on disk servers for reading. Cause not yet understood.
   * Brunel http://savannah.cern.ch/support/?138044 CVMFS black node
   * RWTH cvmfs black node http://savannah.cern.ch/support/?138046
   * Wisconsin Disks full/glideins not starting http://savannah.cern.ch/support/?138037
   * Sites in and out of the waiting room:

<span style="font-family: 'Courier New', courier, monaco, monospace; color: #7a4707;"><span style="white-space: pre;">Last update: June 10th</span></span>
<table cellspacing="0" frame="VOID" rules="NONE" border="0"> <col width="129"></col><col width="89"></col><col width="71"></col><col width="95"></col> <tbody> <tr> <td width="129" align="CENTER" height="15"> __Sites IN__ </td> <td width="89" align="CENTER"> __This week IN__ </td> <td width="71" align="CENTER"> __# weeks IN__ </td>
 <td width="95" align="CENTER"> __This week OUT__ </td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_IN_TIFR][T2_IN_TIFR]]</span></td>
 <td align="CENTER"><br /></td>
 <td align="CENTER">5</td>
 <td align="CENTER"><br /></td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_MY_UPM_BIRUNI][T2_MY_UPM_BIRUNI]]</span></td>
 <td align="CENTER"><br /></td>
 <td align="CENTER">5</td>
 <td align="CENTER"><br /></td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_PK_NCP][T2_PK_NCP]]</span></td>
 <td align="CENTER"><br /></td>
 <td align="CENTER">5</td>
 <td align="CENTER"><br /></td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_PL_Cracow][T2_PL_Cracow]]</span></td>
 <td align="CENTER"><br /></td>
 <td align="CENTER">5</td>
 <td align="CENTER"><br /></td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_RU_RRC_KI][T2_RU_RRC_KI]]</span></td>
 <td align="CENTER"><br /></td>
 <td align="CENTER">5</td>
 <td align="CENTER"><br /></td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_TH_CUNSTDA][T2_TH_CUNSTDA]]</span></td>
 <td align="CENTER"><br /></td>
 <td align="CENTER">5</td>
 <td align="CENTER"><br /></td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_TR_METU][T2_TR_METU]]</span></td>
 <td align="CENTER"><br /></td>
 <td align="CENTER">3</td>
 <td align="CENTER">X<br /></td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_PL_Warsaw][T2_PL_Warsaw]]</span></td>
 <td align="CENTER"><br /></td>
 <td align="CENTER">4</td>
 <td align="CENTER"><br /></td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_BR_UERJ][T2_BR_UERJ]]</span></td>
 <td align="CENTER"><br /></td>
 <td align="CENTER">4</td>
 <td align="CENTER"><br /></td>
 </tr> <tr> <td align="LEFT" height="16"><span style="font-family: 'Times New Roman';"> [[http://dashb-ssb.cern.ch/dashboard/request.py/siteview?view=default&site=T2_RU_ITEP][T2_RU_ITEP]]</span></td>
 <td align="CENTER">X<br /></td>
 <td align="CENTER">0</td>
 <td align="CENTER"><br /></td>

 </tr> </tbody> </table>

*Comment from Edgar* ITEP is working fine for production so we won't put in drains.

---++ Infrastructure Issues:
   * cmsweb couchdb went down again on the weekend.
   * MC bulk WMAgents working without ACDC creation. Otherwise we are hitting a bug in WMAgent ErrorHandler and TaskArchiver.
      * Short Term: Diego is investigating why those request created the problem.
      * Long Term: Have a threshold on the max number of files in which a merge job will run on top, to prevent huge couch documents. Something reasonable around 50
   * issues with jobs not moving to complete - where do we stand on this?
      * https://cmslogbook.cern.ch/elog/Workflow+processing/8366
      * Fixed by band aid patch
---++ Action Items
   * WMAgent 0.9.7? will be installed this week. - On which machines?
      * Will be installed on vocms235 and this one will be moved to an Oracle production account, we are waiting on Erick to create the tag.
      * Procedure for activating new Agent 
      * Krista has worked on doing upgrades/installs via rpm at FNAL to standardize the machines and will be working with Alison to do the same at CERN.  We need to make sure that we haven't broken anything.  
   * Do we need to convert WMAgentScripts to git? Edgar can you do this and update our notes on updating scripts and how to check things in and out of git?
      * *Comment from Edgar* I would say no since WMAgentScripts is on SVN and it has still support as far as I know.    
   * Procedures for re-commisioning sites after they have been in the Waiting room for a while - John
   * Procedure for comissioning sites - where is it documented - John
   * Recovery workflows - Jen
   * updating missing Lumi scripts - Jen
   * Making new shift schedule - Xavier
Topic revision: r5 - 2013-06-12 - EdgarFajardo
CMSPublic
Create a LeftBar
Public webs
- Cern Search
- TWiki Search
- Google Search
CMSPublic All webs
Copyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback