TWiki> LHCb Web>LHCbComputing>ProductionProcedures>SQLServiceRestartOnVolhcb09 (2008-10-06, RajaNandakumar) EditAttachPDF
-- RajaNandakumar - 06 Oct 2008

Problem seen

The DIRAC3 web interface and the CLI had become unresponsive the from Friday 03 October 2008. For example the elog operations eLog 746

Debug information

  • The CLI was also unresponsive - so it was a problem with the services.
  • The production system alone was affected (volhcb09.cern.ch)
  • Problems seen in WorkloadManagement/JobHistoryAgent

2008-10-04 02:23:52 UTC WorkloadManagement/JobHistoryAgent/JobDB FATAL: Can not connect to DB, exiting...

and also

1205: Lock wait timeout exceeded; try restarting transaction

Solution

  • Restart the mysql server on volhcb09.cern.ch

/opt/dirac/pro/mysql/share/mysql/mysql.server stop  --user=dirac
sleep 10
/opt/dirac/pro/mysql/share/mysql/mysql.server start  --user=dirac

  • Restart all the running agents and services on volhcb09.cern.ch

cd /opt/dirac/startup
../bin/runsvctrl t *

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2008-10-06 - RajaNandakumar
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LHCb All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback