I wanted to wait until now to talk about the “big blackout of ’03” so that anyone was left offline wouldn’t also be left in the dark when it comes to talk about having an effective contingency plan.
From my logs, I know issues such as using crontab to schedule ‘automagic’ backups of your database are popular. And that’s good, but that’s only a first step to making sure you don’t find yourself on your hands and knees feeling about for your archive CD when disaster strikes.
So, with the recent power outage still looming like a dark spot on our memory, I think it is a good time to offer the following axiom “a Failure to Plan is a Plan to Fail …”
As I said, we’ve discussed backups more than once. We even recently went into gory detail on using Linux to get your crashed/infected Windows system back online. But at the end of each of these posts is always a sermonette to plan ahead, to map out contingencies, and to practice, practice, practice.
First, a fun analogy. I used to sing opera … professionally. More than once, something would go wrong. A seam on a costume would give out. Someone onstage would feint because of the heat of the lights, or perhaps trip and break their nose exiting stage left. Or worse, some smarmy brass player in the orchestra pit would spike the water in a prop bottle from which your character was to drink.
The point is, in those productions where we practiced, practiced and then practiced some more, such incidents were mere annoyances we now wax nostalgically about with our friends at parties. In those few productions I involved myself where rehearsal was a dirty word, such occurrences were literal showstoppers we’ve blotted out from our memory as far as the east is from the west.
Put another way, you can CRONTAB all the MYSQL backups you want, but without knowing how to restore the data, the backup is useless. Similarly, if the backup is still sitting there on the same hard disk and/or computer system of your failed website, then you’re only remaining option is going to be feeling you way about a dark room as you seek a place to pray.
That said, I know first hand that disaster recovery is a comprehensive discipline unto itself. One need not go any further than FEMA’s “Emergency Management Guide For Business & Industry.” A document you might want to share with the office manager for your church or charity.
And the since object here is to get you to actually do something about not losing all the data on your church web site … and with copious apologies to real disaster recovery plans and the specialists who produce them … here is a short, very short, not-even-close-to-definitive list of bullet points you need to consider to protect your charity’s website from going black:
- List people involved in the plan (include phone numbers and addresses)
- Identify the potential risks (software, data, hardware, people)
- Amor-plate the endangered data/systems as much as possible
- Keep Up-to-date backups of your data on a readily accessible medium (e.g. CD)
- Validate backups (check logs of process AND check the data)
- Keep archive of backups off-site (but accessible in an emergency)
- Make arrangements on alternate systems (alternate web hosts and servers or any other hardware/systems required to restore your data)
- Take a weekend or three to practice the plan … then practice some more
- Make sure you’re not the only one who knows the plan or has the data
- Secure and document all isp and webhost and domain name and any other pertinent contract information (your account, passwords, billing, etc)
- Secure and document all internal access information (accounts, passwords, mailing lists)
- Document all of the the points above (especially restoration)
- Keep several copies of the documentation handy in several locations (e.g. distribute to all members of your recovery team)
As I said, this is a mere pittance in light of true disaster recovery. That said, it is up to make sure you have at least taken the above precautions … and from there add more points and detail … for as it is written (with apologies to the inspirational text of Romans 10):
How can they restore the data without knowing how?
And how can they know how without having practiced?
And how can they practice without someone teaching them?
And how can someone teach without documentation?
As it is written, “How beautiful are the webmasters who have planned ahead!”