Appliance reboots unexpectedly

SUMMARY

Identify possible causes for unprompted appliance reboots.

ISSUE

An appliance continues to reboot even though it is not prompted to by the administrator.  

RESOLUTION

1. Determine if the appliance is connected to a UPS.  If so, are there other devices connected to the same UPS having similar issues?
2. If it is determined the UPS is not at fault, connect to the Unitrends appliance via SSH.
3. Show the shutdown history of the appliance:

   [root@LAB ~]# last reboot
   
   reboot   system boot  2.6.32-504.16.2. Sat Jun 18 15:04 - 09:47 (38+18:43)
   reboot   system boot  2.6.32-504.16.2. Mon Feb 15 15:25 - 15:02 (123+22:37)
   reboot   system boot  2.6.32-504.16.2. Wed Feb 10 23:15 - 15:22 (4+16:06)
   reboot   system boot  2.6.32-504.16.2. Thu Jan  7 21:43 - 23:12 (34+01:28)
   reboot   system boot  2.6.32-504.16.2. Thu Dec  3 18:01 - 21:40 (35+03:38)
   

4. Show a list of shutdowns and other pertinent issues on the appliance. If you see any logs that say crash, focus on those times:

   [root@LAB ~]# last |more
   root     pts/0        192.168.103.37   Tue Jun 21 12:44 - 12:48  (00:03)
   reboot   system boot  2.6.32-504.16.2. Sat Jun 18 15:04 - 09:49 (38+18:44)
   root     pts/1        myh-lab-ls09     Sat Jun 18 14:50 - crash  (00:14)
   root     pts/2        10.10.3.54       Mon Jun 13 16:57 - 19:07  (02:09)
   root     pts/1        10.10.3.54       Mon Jun 13 16:13 - 19:06  (02:52) 

5. Check for any logs that correspond to the time of the power down and examine them for events that may have occurred around the time of the crash or reboot:

   [root@LAB ~]# cd /usr/bp/logs.dir/
   [root@LAB logs.dir]# ls -lathr | grep tasker
   
   -rw-rw-r--  1 root   loggers  232K Jun 15 07:26 tasker-5.log
   -rw-rw-r--  1 root   loggers  129K Jun 15 19:26 tasker-6.log
   -rw-rw-r--  1 root   loggers  231K Jun 16 07:27 tasker-7.log
   -rw-rw-r--  1 root   loggers  135K Jun 16 19:27 tasker-8.log
   -rw-rw-r--  1 root   loggers  227K Jun 17 07:27 tasker-9.log

6.  Check the database logs for the day that the reboot happened. Open the log file in an editor, such as vi, and look for the time corresponding to any crashes or unplanned shutdowns:

   [root@LAB logs.dir]# cd /usr/bp/data/pg_log/
   [root@LAB pg_log]# ls -la
   
   total 552
   drwx------  2 postgres postgres   4096 Oct 11  2015 .
   drwx------ 16 postgres postgres   4096 Jun 21 18:10 ..
   -rw-------  1 postgres postgres  21709 Jun 17 23:01 postgresql-Fri.log
   -rw-------  1 postgres postgres  33525 Jun 20 23:01 postgresql-Mon.log
   -rw-------  1 postgres postgres  37607 Jun 18 23:01 postgresql-Sat.log
   -rw-------  1 postgres postgres 372132 Jun 19 20:07 postgresql-Sun.log
   -rw-------  1 postgres postgres  35498 Jun 16 23:01 postgresql-Thu.log
   -rw-------  1 postgres postgres  22933 Jun 21 23:01 postgresql-Tue.log
   -rw-------  1 postgres postgres  16511 Jun 15 08:28 postgresql-Wed.log
   [root@LAB pg_log]# vi postgresql-Sat.log

7. Verify there were no power outages or other issues that were happening at the time of the reboot. If not, this may be a hardware issue.  
8. Check for any memory issues:

   [root@LAB ~]# dmesg |more

CAUSE

This can be caused by conditions such as:

• power failures
• hardware malfunctions
• UPS issues