August 22, 2017

Website down Fri Aug 17 through Tue Aug 22, 2017


This weekend I learned the hard way why it is important to have multiple ways to back up your website or blog.

= What happened? =

On Friday evening, I was working on some updates to this site, and something went wrong. All of a sudden I was unable to access the admin interface via login, and something else was throwing the website into a fit, where only the top half of each page was shown, omitting most of the content.

No problem, I thought, this website is hosted at the data center of the largest ISP in Europe, and they do regular backups, so all I need to do is call their 24/7 support hotline, get my site restored to the previous day, and all is fine.

= Support Hotline =

Which is what I did. The support person at the other end told me there is a self-help option: “Just go to the control center, under hosting you will find the option to restore from your backups”, he told me. But when I went to that option, it said: “Sorry, the restore function is not available right now since we’re doing maintenance, please try later.” The support person was unable to tell me when that maintenance session would be over. I tried again on Sunday. The same error message about maintenance. Wow. Called support again, the same person on the line. He said to try again Monday morning. Which I did.

= No Backups? =

When I called and mentioned the story to the new support person on Monday morning, she looked into my file and said: “Well, it seems we sent you an email on 5 November 2015, informing you that you are migrated to a server where there is no backup, because you have too many files.” My thought was: “Woot??”

= Hitchhiker’s Guide to the Galaxy =

I was reminded of the HHGTTG opening sequence, when Arthur Dent learns that Earth will be destroyed immediately to make place for an intergalactic starship superhighway. When he complained he was told that the plans had been on display for a long time on some distant planet in an obscure star system.This was how I felt when I was told that amongst the pile of constant emails from my ISP, most of which are not quite spam, but almost so, was one important one that I should have read instead of filing it away automatically according to some mail program algorithm.

Yes, she told me (on Monday), you only get a daily backup of the previous day. So, this meant, I could restore on Monday to the state of affairs from Sunday, but my problem had started on Friday, so I was doomed!

= There are Backups! = 

I realized I was on my own. So, I checked the file system, and lo and behold!, I remembered that my WordPress site did have a plugin that made some automatic backups, and these were all well preserved. Phew! Saved by the towel (ok, inside joke, but if you know Arthur Dent and Ford Prefect, you know what I mean!)

= Lessons learned =

I was able to restore my site on Tuesday, after various trials and tribulations. I was lucky. But who knows whether I will be so lucky again next time?

This story has taught me some important lesson: Do not just rely on your ISP to make backups for you! They may fail you. Also, do not simply rely on your website installation to make automatic backups. What if the backups are destroyed, since they are on the same server? Therefore: Do also regularly make backups manually and store them in another physical location, so you can go back to some previous stage when necessary.