Operations Support Document for PDB Level 1
*Please make sure you have read the documents linked in
General system status:
- Check the web page with list of servers and attached storage:
- Connect to the OEM via browser to check system activity (server CPU and DB active sessions)
- Linux command line: sar -u 1 100 to measure CPU load
- Linux command line: sar -b 1 100 to measure CPU load
- Linux command line: �ps �ef|grep pmon� -> must output 2 lines, one for ASM and one for the DB instance
- Connect to the DB instance with �sqlsys_DB� (see below for some useful sqlplus commands)
Relevant log files:
- DB alert log
- Type 'bdump' to go to the directory. The file name is alert*.log
- ASM alert log
- Type 'bdumpasm' to go to the directory. The file name is alert*.log
- Listener log
- �cd $ORACLE_HOME/network/log�. The file is listener*.log
- System�s log: /var/log/messages
- �sudo less /var/log/messages�
Healthcheck of the Oracle clusterware:
- crsstat.sh
- srvctl start service -d dbname can be used to start manually all services for a given db (services which are already up are not touched)
- srvctl start service -d dbname -i instancename -s servicename can be necessary to startup up "stubborn" services
Other clusterware operations:
- srvctl start/stop can be used to start and stop resources managed by the Oracle clusterware. See the syntax for srvctl status above.
Some checks with sqlplus:
- type sqlsys_DB to connect to the db instance with sqlplus and command history
- Check that instances are not close to the maximum number of concurrentprocesses (500 processes per instance):
select inst_id,count(*) from gv$process group by inst_id;
- Check activity on the cluster or drill down on instances close to max processes:
select inst_id,username,osuser,program,last_call_et from gv$session where username is not null;
select * from dba_blockers;
- check for long running ops:
select inst_id,username,time_remaining remaining, elapsed_seconds elapsed from gv$session_longops where time_remaining >0;
- type 'sqlsys_ASM' to connect to the ASM instance with sqlplus and command history
- List disks and their status: @listdisks
- List disk groups and their status: select * from v$asm_diskgroup;
- List ASM rebalancing operations: select * from gv$asm_operation;
Checking the listener on RAC:
- Run
ps -elf | grep tns | grep -v grep
to find out if the listener process is up.
- Run the
crsstat.sh
command to find out what the status of the listener according to the clusterware.
Stopping the listener the listener on RAC:
-
srvctl stop listener -n XXX
(where XXX is name of the node e.g. itrac09)
- If the command above does not work execute:
lsrnctl stop listener_XXX
(where XXX is name of the node e.g. itrac09)
- If none of the commands above does not work find out listener process pids (ps -ef|grep tnslsnr) and kill it with
kill -9
Starting the listener the listener on RAC:
- If the listener is down according to the
crsstat.sh
command execute: srvctl start listener -n XXX
(where XXX is name of the node e.g. itrac09)
- If the listener listener process does not exist and the
crsstat.sh
command claims that the listener is up: lsrnctl start listener_XXX