You Had a Bad Day

So I just discovered that my dedicated machine, which is generally doing absolutely nothing, was running at a load average of about 1. The top CPU abuser? Some command I didn’t recognize (barbut). I was immediately suspicious. I killed the process, then noticed that it had been running as the cvs user, so I ran a ps to find all commands running as cvs.


Yes, that’s right, my dedicated box was an involuntary participant in a distributed denial of service attack, orchestrated by an IRC bot, also known as barbut (which I found, source and all, in /home/cvs/).

Time for damage control. First, I obliterated the user cvs. Then I installed and ran rkhunter; the “good” news is that no root kits were found. Then I went to change the SSH port — oh, wait, I’d already done that, but never restarted with the new config: shame on me!

One of the unfortunate side effects of using CVS over SSH is that you need accounts with shell access. Apparently I’d created a user with a basic password to allow friends to check code out of my local CVS server; I’m guessing that password just got brute-forced. There doesn’t look to be anything else amiss, so I guess I was somewhat lucky.

Anyone want the source code to an IRC bot?

6 Comments so far

  1. Mr. T on March 26th, 2008

    Passwords are so often the weak link. No matter how good the door if the lock is no good there is a problem. Glad you found it and that nothing worse was done.

  2. Matt on March 26th, 2008

    You should get rid of the dedicated machine! 😉

    Can you check “last” to see where they logged in from? Although, in all probability, it’s another infected machine.

    You can also check your bandwidth usage ( ) — the good news is that on a 10 Mbps, it takes a couple weeks to hit 1TB.

    If you run ssh on port 22 and watch your log files, these days you’ll see thousands of attempts a day. Literally, they just sit there and try common usernames and passwords. I use DenyHosts, which will ban anyone who fails at logging in 5 times in a row. I’m pretty strict because I never fail more than once (on a typo), and because I can get in from multiple IPs easily. You could set the limit at 50.

    Heh, glad you posted this! Writing that just reminded me that I haven’t gotten e-mails on banned IPs in a while… Turns out that DenyHosts wasn’t set up to start at boot, so when the box died a few weeks back, it never restarted…

    Mar 26 14:27:12 oxygen sshd[24986]: Invalid user router from
    Mar 26 14:32:34 oxygen sshd[28543]: Invalid user mambo from
    Mar 26 14:32:35 oxygen sshd[28554]: Invalid user rusticos from
    Mar 26 14:32:36 oxygen sshd[28565]: Invalid user rusticos from
    Mar 26 14:32:37 oxygen sshd[28576]: Invalid user test from
    Mar 26 14:32:38 oxygen sshd[28587]: Invalid user test from
    Mar 26 14:32:39 oxygen sshd[28599]: Invalid user test from
    Mar 26 14:33:05 oxygen sshd[28895]: Invalid user toor from
    Mar 26 14:33:06 oxygen sshd[28906]: Invalid user toor from
    Mar 26 14:33:07 oxygen sshd[28918]: Invalid user toor from
    Mar 26 14:33:08 oxygen sshd[28929]: Invalid user administrator from
    Mar 26 14:33:09 oxygen sshd[28940]: Invalid user dasusr1 from
    Mar 26 14:33:10 oxygen sshd[28951]: Invalid user dasusr from
    Mar 26 14:33:11 oxygen sshd[28956]: Invalid user dasusr1 from
    Mar 26 14:33:12 oxygen sshd[28967]: Invalid user dasusr1 from
    Mar 26 14:33:13 oxygen sshd[28978]: Invalid user dasusr1 from
    Mar 26 14:33:14 oxygen sshd[28989]: Invalid user dasusr1 from

    Of course, that’s not all…
    oxygen ~ # grep /var/log/messages | wc -l

    Anyone want the source code to an IRC bot?

    Source code, not so much. Server / user / password, sure! Years ago when my web-facing desktop got “hacked,” it joined an IRC machine, so after killing it, I joined and just watched.

    Edit: pushing my own comment through moderation.

  3. andrew on March 27th, 2008

    They must not have used the bot much: when I went to reboot the box (accidentally shutdown all ssh connections and the daemon), I looked at my bandwidth usage; there was barely any.

    I’d seen tons of attempted logins in the past, but was never real worried because they were trying accounts that I’d never have….

    We had a similar bot appear on one of the servers at work, and we eventually hacked the source into a daemon that would just log all the activity in the channel. But eventually that got boring. :-p

    I tracked down the IP, and it does appear to be another compromised host. Unfortunately, I couldn’t find an email address to warn them.

    The attacker left a lot of info behind, if it seems interesting, maybe I’ll post some of it.

  4. andrew on March 27th, 2008

    I guess I never mentioned: the initial login attempt occurred on March 11th, about 2 weeks ago. But, like I said, they didn’t use much bandwidth.

  5. Matt on March 27th, 2008

    Unfortunately, I couldn’t find an email address to warn them.

    You could always whois the IP and contact that host’s abuse department… I always have a hard time deciding whether doing things like that is nice and helpful, or if I’m coming across as a jerk who’s getting someone’s machine suspended.

    Who breaches security to launch attacks and then… doesn’t launch attacks?

  6. […] Bandwidth graph of the period during which my machine was compromised: […]

Leave a Reply