--
RajaNandakumar - 06 Oct 2008
Problem seen
The DIRAC3 web interface and the CLI had become unresponsive the from Friday 03 October 2008. For example the elog
operations eLog 746
Debug information
- The CLI was also unresponsive - so it was a problem with the services.
- The production system alone was affected (volhcb09.cern.ch)
- Problems seen in WorkloadManagement/JobHistoryAgent
2008-10-04 02:23:52 UTC
WorkloadManagement/JobHistoryAgent/JobDB FATAL: Can not connect to DB, exiting...
and also
1205: Lock wait timeout exceeded; try restarting transaction
Solution
- Restart the mysql server on volhcb09.cern.ch
/opt/dirac/pro/mysql/share/mysql/mysql.server stop --user=dirac
sleep 10
/opt/dirac/pro/mysql/share/mysql/mysql.server start --user=dirac
- Restart all the running agents and services on volhcb09.cern.ch
cd /opt/dirac/startup
../bin/runsvctrl t *