httpd Offline Due to Flood
Howdy, all; I’ve taken HTTP offline temporarily while I determine who is hammering the holy heck out of the server. If I bring HTTP back online, 200 processes spawn off in under ten seconds. [Normal load at any one time is 30-50 processes.]
1821 CST: Well, the NOC is involved with me on this one. Having issues running things down …
1910 CST: The NOC has put us under a guard at their network firewall to detect such incoming attacks and limit them before they touch the server. This has brought us to resolution. I’m going to take some time over the break to sift through the logs, domain-by-domain, and see if I can find what was being hammered.
1948 CSR: The user who owns the account responsible reported what he think that he might have done to cause the problem. In digging to his domlog, I quickly determined that he was correct. 20-30 requests a second … yep, that’ll slag this little box.
I won’t out him, but he’s welcome to come and apologize if he wants.
December 21st, 2006 at 7:30 pm
Eep. Hope it gets sorted with a minimum of hair-pulling.
December 21st, 2006 at 7:53 pm
The NOC guys have us on a guard at the firewall to help with the flooding. I will see if I can dig through the domain-level logs and find what the target was. I’d like to harden that target…
December 22nd, 2006 at 1:45 pm
I just can’t believe I wasn’t responsible for once….
December 22nd, 2006 at 3:36 pm
It was me. I apologize to everyone affected. I redirected the WRONG traffic to feedlounge.info, which effectively DOS’ed the box.
It will NOT happen again.