2

I have a server that was running an older version of lighttpd (1.4.19 on a freebsd 6.2-RELEASE (yea, old) machine) and google alerted me that it had found malware embedded on one of my server's pages. It just so happened to be our index page. I promptly removed the malware and started looking at server logs for how it got there. With no trace in any of the logs of the files being edited, I noticed that the index page's owner had been changed to www, which is the lighttpd user. I then concluded that some sort of veunerability must have existed for that software version and promptly upgraded to 1.4.26.

Now the malware is back. I have started some pretty verbose server logging with ftp, lighttpd, and all login attempts to try and see how this script is getting in. Are their any suggestions as to other approaches to take?

Marco Ramos
  • 3,150
jps
  • 123
  • 4

1 Answers1

4

Your website was compromised/defaced and when this happens, it's normally very difficult to re-create all the attackers steps and the best solution is to reinstall the compromised servers. On the other hand, you need to perform some forensics in order to figure out what has possibly happened and preventing it from happening again.

Here's a list of things worth checking:

  • see if there are known vulnerabilities in your webserver and your ftp server versions
  • take a look at every logfiles you can, especially the webserver, the ftp server and the system ones. In the webserver logfiles, check for posts
  • are there any services running that you don't need? Are they accessible from the Internet? Shut them now, check their logs and check for possible known vulnerabilities.
  • run rootkit checkers. They're not infalible but can lead you in the right direction. chkrootkit and especially rkhunter are the tools for the job
  • run nmap from outside of your server and check if there is something listening on any port that shouldn't be.
  • if you've a rrdtool trending application (like Cacti, Munin or Ganglia) take a look the graphs and search for a possible time frame of the attack.

Also, always keep this is mind:

  • shut down all the services you don't need
  • backup everything you need to rebuild your server and test backups on a regular basis
  • follow the least privilege principle
  • have your services updated, especially regarding security updates
  • don't use default credentials

Hope this helps!

Marco Ramos
  • 3,150
  • Good answer. Let me emphasize that you should not restore your entire server from backups from backups if you have them, since they may be compromised as well. You only want to copy configuration files to serve as a guide and data and you want to review them since they may have been modified.

    It's a pain, but it must be done. You are not paranoid when you know they are after you.

     – guilherme Jun 08 '10 at 01:58
  • The website directories should not be writable by the www/httpd daemon in the first place. Start there. – Chris S Jun 08 '10 at 02:21
  • @Chris S. There are two ways to look at this. Only the infected files are writable via the www user (they had www as their owner). I have since changed the owners back to their respective user, so if this happens again, the user is being changed. Otherwise, the problem may be solved. – jps Jun 08 '10 at 02:31
  • @Jud: I'm not following, why are your website files writable by a daemon that shouldn't ever write to them? – Chris S Jun 08 '10 at 04:36
  • To add to this answer, when checking for the timeframe of the attack, look at the timestamp on the files that were modified.

    Once you have that, look through all of your log files, particularly your web server log files, both before that point to see if you can identify the attacker and the attack method and after that point to see if you can identify what they may have done using any PHP shell script they may have uploaded.

    And in answer to Chris S, his webserver files were originally owned by normal users. The attack changed them to be owned by www.

     – Ladadadada Jun 08 '10 at 12:31
  • @Ladadadada, I understand the files were owned by someone else originally. I'm trying to point out that it's bad practice to allow any user to write to the www directory. It should be root writable only (755 or maybe 775 with appropriate owner/group, but never 777, and never owned by www). If they've rooted your machine, that's a different problem... – Chris S Jun 08 '10 at 13:42
  • @Chris S. They are owned by an ftp user with no privileges. And they are all 755. – jps Jun 08 '10 at 14:58
  • @Jud Stephenson: btw, which ftp server/version are you using? – Marco Ramos Jun 08 '10 at 15:04
  • @Marco, pure-ftpd-1.0.21_1 - I just upgraded it to pure-ftpd-1.0.29. I am looking over the server logs right now. Reinstalling everything won't be an option until this weekend. – jps Jun 08 '10 at 15:18
  • @Jud: there are known vulnerabilities regarding pure-ftpd. Maybe that was how they got in. Consider using ProFTPd. Hope this helps. – Marco Ramos Jun 08 '10 at 15:31