[rocksmyfaceoff.info]

OpenDNS Not Resolving for rocksmyfaceoff.net

Geof F. Morris — Fri, 21 Mar 2008 22:27:18 +0000

I think the title is self-explanatory; nothing is demonstrably wrong with the box. Am investigating.

1729 CDT: The issue is limited to rocksmyfaceoff.net, not www.rocksmyfaceoff.net. Very strange.

1924 CDT: It has apparently re-resolved. :shrug:

True Cause of Slowness Found

Geof F. Morris — Sun, 10 Feb 2008 01:30:10 +0000

Okay, so I thought that we were just getting mailbombed. [We were.] Turns out that Mark is testing out a different piece of forum software, and he’s been building a search index since last night. Makes so much more sense.

Twitter Updates for 2008-02-08

Geof F. Morris — Sat, 09 Feb 2008 05:59:59 +0000

We’re back online. Total downtime ~20 minutes. Better’n Twitter. #
Wuhoh … time to run REPAIR TABLES on The Rumor Forum! Sonofa. #
Okay, everybody back into the pool. And thanks to Smiley for getting the database tables repaired quickly. #

RMFO-Blogs.com Mailbombing Causing Server Slowness

Geof F. Morris — Fri, 08 Feb 2008 23:24:28 +0000

A spammer is performing a botnet-based spamming attack on various non-existent addresses on rmfo-blogs.com right now, and that’s causing significant load on the server. It’s not thing that the server can’t handle, but it is working extra hard, and so other services may be slow. Because of the distributed nature of this mailbombing, there’s really no way to ban specific IP addresses and have that solve the problem for us.

Oddly enough, I’m not sure that there’s ever been a legitimate email address on rmfo-blogs.com.

Outage: Cause Unknown

Geof F. Morris — Fri, 08 Feb 2008 19:33:27 +0000

The database server began acting up about ten minutes ago, so I attempted to shut it down for a reboot. It never wanted to do so successfully, so I took the Web server offline to allow the database server to close, but it never did. I forced a reboot of the machine, but it’s not come back online yet. Will advise.

1345 CST: We’re back online. Sorry about that!

1350 CST: Yes, we know that the tables are hosed on The Rumor Forum. A nice REPAIR TABLES run will work, but it’ll take time. Go play in traffic or something if you’re angry about this.

Twitter Updates for 2008-02-06

Geof F. Morris — Thu, 07 Feb 2008 05:59:59 +0000

WordPress upgrades are complete, save for those users who do it themselves. Those folks are asked to upgrade as soon as is possible! –GFM #

Server Tuning Should Happen Tonight

Geof F. Morris — Mon, 07 Jan 2008 22:13:54 +0000

Sounds like a broken record, I know; thankfully, the nice folks at ConfigServer gave us a 20% discount on the tuning they do for us because we just had it done 10 days ago. [I didn’t even ask for it; they just did it. They rock.]

As a result, I expect that the tuning will happen tonight. In conjunction with the changes already made [we’re now running Apache2 and PHP5], they’re going to upgrade MySQL to 5.0 from 4.1. There really shouldn’t be any issues with that, but if you use a database-based application and have problems after the upgrade, let me know and I’ll help you work them out. [WordPress, SMF, and phpBB are all abstracted enough that I really doubt it’ll matter.]

ConfigServer’s based in the UK, so any downtime would be late Monday evening into early Tuesday morning.

Twitter Updates for 2008-01-04

Geof F. Morris — Sat, 05 Jan 2008 05:59:59 +0000

There won’t be another backup run at 11:30 tonight. Not enough time for it to be completed. #
Account restorations are underway and are approximately 40% done by account count [not by time spent]. Will update as progress is made. #
Some, but not all, of the IP’s have been reallocated. I’ll finish the rest ASAP. #

IP Reallocation Should Be Complete

Geof F. Morris — Fri, 04 Jan 2008 23:43:33 +0000

Everything should be up and running now.

Please let me know if not.

Services may be wonky … report it and I’ll look after it.

Going to nap now.

Account Restorations Nearly Complete

Geof F. Morris — Fri, 04 Jan 2008 17:54:15 +0000

I’ve nearly completed the command line work for restoring all accounts. Because I’m at work, I haven’t completed the IP re-allocation work. I’m going to try to sneak home here at lunch and do that, but it may be later this afternoon before I can perform this. Your data appears to have survived; I just have to get it all back up and online.

Thank you for your continued patience.

Update 1505: All the accounts have been unpacked. Most of the IPs have been re-allocated, but not all. I’ll finish that this evening, and then we ought to be back up and running. I’m going home to see my folks in the morning, rather than tonight, so … I should be able to finish everything.