Portal Home > Knowledgebase > Articles Database > Server down with Kernel Panic
Server down with Kernel Panic
| Posted by techwacky, 07-27-2011, 06:38 AM |
| For the last 2 days i am having this issue.
Yesterday morning the server load went to 798 and every thing was down..
i managed to log in ssh and saw the process monitor
kswap08 was using 61% of cpu on top
so i rebooted the server with init 6
later i waited for 5 mins still not up... waited 15 mins still not up. opened a ticket on dc to reboot and have a look.
they rebooted the server and said the server need a file system check.
so i gave them root pass.
it was up after 1 hr. later after 3 hr server again went down. i opened ticket again whats going on.
they rebooted the server and it was live again.
Until today morning just 2 hr ago it was down and inaccessible able via ssh , ot emails form monitor site resource down down.
opened ticket again " what happened again, server down "
got reply "Server was in kernel panic, rebooted."
now its live again.
i am suspecting it will go down again. and i am kind of mad now.
server config:
P4 3.0GHZ
2GB ram
250gb hdd
100mbps
Some outputs:
What should i do to solve this problem...
|
| Posted by cpservr, 07-27-2011, 07:27 AM |
| could you post your /var/log/messages for kernel error messages?
|
| Posted by techwacky, 07-27-2011, 07:48 AM |
| yes
|
| Posted by cpservr, 07-27-2011, 08:10 AM |
| Jul 26 09:12:48 server kernel: RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
did you use ramdisk for this server? try to crosscheck by typing:
~# dmesg | grep RAMDISK
if you see a message like:
i think you need to configure your ramdisk. and also post your lsmod result here too.
edit: right now i'm still finding some suspicious process on your log messages result.
Last edited by cpservr; 07-27-2011 at 08:13 AM.
|
| Posted by techwacky, 07-27-2011, 08:16 AM |
| i have no idea about ram disk. server was setup by my dc with direct admin installed.
here is the output:
|
| Posted by cpservr, 07-27-2011, 08:22 AM |
| errrr,, i forgot if directadmin using ramdisk or not. but this is some good article http://www.vanemery.com/Linux/Ramdisk/ramdisk.html just try to take a look. i can't found any suspicious result from your log message, and according to your lsmod result they are seems good except ramdisk issue.
|
| Posted by techwacky, 07-27-2011, 09:51 AM |
| i checked ram disk and looks its working...
|
| Posted by HelpingHost, 07-27-2011, 11:26 AM |
| To find the reason of the Kernel Panic you would need to look on the screen when the panic occurs.
It would give you some hints as to what happens.
However, since you seem not to be able to do that since you are not at the machine, I'm guessing (from your description with the high load) that your server is running out of Memory.
Here is what I would do. (I do not know the specifics of DirectAdmin though)
Check the Apache server-status page while it is running when you see the load going up.
Very likely you will see a specific URL/Website being accessed a lot.
If you are using CGI scripts or PHP in CGI mode set RLimitCPU/MEM/NPROC to hopefully prevent them from overloading the server.
If you can identify some kind of URL/Site being hammered, try using a form of throttle like mod_bandwidth.
Last, I found switching to Cloudlinux really prevents these things from happening in the first place.
I switched many Web Servers from Redhat ES 5 or CentOS 5 to Cloudlinux (5 minute process) with overwhelming success and dramatically increased uptime.
|
| Posted by viGeek, 07-27-2011, 12:16 PM |
| Would need some additional information, full dmesg output and additional information from messages.
Looking through (more than likely unrelated), you have some services running that aren't needed for a server.
chkconfig bluetooth off (bluetooth...)
chkconfig gpm off (mouse over console)
chkconfig avahi-daemon off (auto disc: http://avahi.org/)
chkconfig hidd off
chkconfig pcscd off (smart card daemon)
Last edited by viGeek; 07-27-2011 at 12:20 PM.
|
| Posted by techwacky, 07-27-2011, 01:50 PM |
| thanks for identifying
i have stopped those service
now my output is:
|
| Posted by techwacky, 07-27-2011, 03:11 PM |
| full dmesg output
|
| Posted by whmcsguru, 07-27-2011, 07:22 PM |
| See if they can hook a console up to it so that you can check it out if (when) something goes wrong again. There's not a ton we can tell from the output you've given.
|
| Posted by luki, 07-27-2011, 08:38 PM |
| The hints of huge load and kswap showing lots of activity sounds like you're running out of memory, i.e. RAM and swap. Eventually the machine will panic if there is no more memory available and the OOM killer (if enabled) was unsuccessful in making some available.
It could be a hardware problem (bad RAM or drive), but I kind of doubt that.
Simply monitor your RAM usage closely (snmpd, or just write the output of free every minute to a file) and hopefully you can see what the usage was just before the crash.
|
Add to Favourites 
Print this Article
Also Read
