Steps to investigate hacked Linux server

Following are the few steps to investigate whether the Linux server is hacked or not:

Who is on the Server:
$ w

$ netstat -nalp | grep “:22″

$ w && netstat -nalp | grep “:22″
The above commands will say who are all logged into the server.

Who was on the Server
$ last

$ cat /var/log/secure* | grep ssh | grep Accept

$ cat /var/log/secure* | grep ftp | grep Accept

Check what is the Current Network Activity of your server
$ netstat -nalp

$ nmap localhost

$ netstat -nalp && nmap localhost

What Processes are Running:
$ ps -elf

$ ls /proc/*/exe -la

What Files are in the Common Attack Points:
$ ls /tmp -la

$ ls /var/tmp -la

$ ls /dev/shm -la
These are all the common unsecured places where the hacker intrudes into your linux server.

Don’t delete anything or make changes just yet, just catalogue everything. Do not access a file with cat or strings, catalogue the files and save that for later. Once you start deleting things you can no longer further investigate as to how deep they have penetrated. Don’t be fooled into seeing a common Apache compromise and think it ended there. Many times that was just the broken window they used to get in the first time, meanwhile, they are tunnelling deeper trying to get into root access.

What version of Linux is running
$ cat /etc/redhat-release

For non Red-Hat Linux
$ cat /etc/issue

Compare this to the kernel
$ uname -a

and

$ cat /proc/version

Who is the author of the file:
$ ls -la –author

When was the last time the file has been accessed and by who:
$ ls -l –time=access

Before you run off and use the cat command it is good to first check the file type with the file command. Many a time I myself have been fooled seeing a file marked as something.html and finding it was really a binary file.

What kind of file is it(ASCII or Binary):
$ file filename

$ file /path/to/directory/*

You have been trying to be sneaky and not have any obvious virus scan running in the process list so as to not be detected, but that is tedious work and slow.

Update the Locate Database:
$ updatedb &

If this is a web server then the next thing to hunt for is signs of Apache exploits and SQL injection scripts. This nice little script was handed down to me from a co-worker and does a nice job of hunting through the log files rather than the long tedious work of searching manually.

$ egrep -i ‘(chr(|system()|(curl|wget|chmod|gcc|perl)%20′ /path/to/log/files/*

cPanel
$ egrep -i ‘(chr(|system()|(curl|wget|chmod|gcc|perl)%20′ /usr/local/apache/logs/*

$ egrep -i ‘(chr(|system()|(curl|wget|chmod|gcc|perl)%20′ /home/*/statistics/logs/*

Ensim
egrep -i ‘(chr(|system()|(curl|wget|chmod|gcc|perl)%20′/home/virtual/site*/fst/var/log/httpd/*

Plesk
$ egrep -i ‘(chr(|system()|(curl|wget|chmod|gcc|perl)%20′ /home/httpd/vhosts/*/statistics/logs/*

$ egrep -i ‘(chr(|system()|(curl|wget|chmod|gcc|perl)%20′ /var/log/httpd/*

Search for Shell Code:
$ cat /path/to/access/logs/* | grep “/x90/”

From the steps provided above you can make a guess or conclude the hack.

Steps to Investigate Hacked Linux Server