[rocksmyfaceoff.info]

I’ll keep this up-to-date as things progress today:

1017 CDT: I’ve moved the early morning backups around and have the up-to-1000 backups now in process. As soon as those are complete, I’ll push them onto the Network Attached Storage.

1206 CDT: Probably two-thirds done with the backups.

1246 CDT: Backups are now complete. Beginning the transfer to the NAS.

1319 CDT: User accounts backed up. Now backing up critical system files and MySQL databases [the latter being a redundancy, but a critically important one].

1322 CDT: All files backed up onto the NAS. I was filling out the OS Reload request form earlier, and I saw that there’s an option to leave a disk drive the same. I’m going to have them leave the secondary HD, which is our backup, as it is if they can. That’ll save me time in bringing services back online, because I won’t have to pull data from the NAS box back onto the backup drive so I can then restore from the backups. I’m gonna put in the OS reload request now, and then it’s out of my hands. Please excuse any sounds of nausea coming from my direction. :chuckle:

1333 CDT: Crap. I can’t get RHEL v5 until Tuesday [probably a licensing thing]. I’m not waiting until Tuesday. We’ll got with RHEL 4 and, if we need to do this again some time in the future, we’ll do it again. I’m okay with that.

1350 CDT: Been trading ticket responses with Chris B. at The Planet.

Chris B. - Sunday July 29th, 2007; 1:49 PM CDT
You’re welcome, I will set the secondary drive to preserve, and your OS reload will begin momentarily.

I’m excited!

1459 CDT: The Planet kicked off a tracking ticket for the reload at 1426. I’d expect that the OS reload + cPanel/WHM load will take 90 minutes. That puts us at another hour or so before I can begin restoring accounts from the backups.

1545 CDT: No news from The Planet, but I have finally been able to get to the Twitter signup page. I’ll tweet stuff at rmfoinfo. Of course, I’ll post here, too … but this will allow me to do some fancier stuff. Now, if I can only get the server to do the tweeting for me when it’s back online …

1815 CDT: I had some go-backs on the partition scheme about an hour ago [something I think we would have fixed before, oh, 1400], so we’re still waiting. If you think you’re tired of waiting, I’ll trade ya.

2037 CDT: The OS reinstall is complete, and now cPanel is being reinstalled. This will apparently take about an hour, so hopefully that’ll be back up by 2200, and then I can power through bringing stuff back online.

2205 CDT: I now have access to the server again, and I will now begin testing restorations!

2217 CDT: I have zero access to the secondary drive right now, so I’ve got a second ticket in about that. I’m kinda hosed without that. Even if the drive ends up being unusable, I’ve got a backup on the NAS [that's why I burned a half-hour on it this morning], so I’m not worried. I’ve just gotta have access to that second HD to bring things back online.

2324 CDT: Okay, I have the backups back up and running [and have for a while], but now I’m trying to get the IP allocation set up properly so that, when I bring sites back online, they’ll be in the right spots. I’ve done this before, but it’s been a couple of years …

30 Jul 2007 0001 CDT: Well, I’m still struggling with the IP address issue. I’ve asked the NOC for help because I just can’t figure out WTF is going wrong. That said, I have restored SquarePegAlliance.net as a test to see if everything’s up and running, and that appears to be working. I will start going through and finding all the main IP-based accounts and go ahead and bring them online while working to resolve the other IP problems … which do involve most of the major accounts on the machine.

This entry was posted on Sunday, July 29th, 2007 at 9:18 am and is filed under Announcements. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.