When your ISP goes dark.

Those who have services with Heart Internet won’t have failed to notice the furore recently with an outage which lasted days (well 22 hours actually), caused considerable downtime for users with websites going dark & VPS’s being unresponsive, & of course a subsequent Twitter mob baying for Heart’s blood. So, what happens when your ISP goes dark?

The Register captured the situation here so I won’t go into the technical’s but what has surprised me is the massively naive comments I’ve seen online via Twitter & Facebook putting sole blame at Heart’s door when frankly, users should take much more responsibility of their websites, especially those running ‘web companies’ from their bedrooms or small businesses doing the same.

Please note I do appreciate people make a living, a livelihood out of blogging, and I take that very seriously. I run my own hosting and website company, I and my clients too (some of them) are also with Heart Internet, and I had no issues. Not one. At the time of outage, I simply pointed my DNS to my failover locations and told my clients of the issue. I still haven’t repointed DNS and I don’t need to. This post isn’t to brag. It’s to just maybe put things into perspective somewhat.

Let’s take a moment to consider how often outages like this happens. Almost never. Individual instances or isolated problems on individual servers are a different matter and are daily life for hosting companies & their consumers. Of course, customer service, (or lack of) plagues companies such as Heart, and you only need to look over service reviews to see how poor hosting companies can be. It’s the same for all of them, 123-Reg, FastHosts, Hearts, none are ideal.

If you’re running a business or mission critical site, and you’re using a hosting provider such as one of the above, and you’re not employing adequate disaster planning then that responsibility lies with you, not the hosting provider. Surely? People surely need to take this on themselves?

Heart Internet have also (which complainers haven’t mentioned) offered a full refund against their SLA, alongside free 10GB backups for life. So that’s pretty reasonable really. I don’t see how they could be expected to answer 1000’s of support emails, Tweets, Facebook rants & telephone call’s whilst also trying to fix the problem.

Again, this post is to address the balance for those who spend just pounds per month, have two or four websites with clients spending just hundreds, and are absolutely slamming Heart with ridiculous demands and claims they are losing them millions, its not for those who have had genuine issues, and have been let down by Heart from a customer service perspective.

Sure they made mistakes, especially around how the event was communicated first blaming a DDoS and then fessing up to power issues at the DC, but the public are the mob and almost none have ever built, taken care of or managed a data centre or hosting company of any kind.

By the way, I have nothing whatsoever to do with Heart Internet!

I wanted to focus this article on what you can do as a bedroom website company or small business though, to ensure you’re protecting yourself as much as possible, and reducing reliance on the hosting company. This should be common sense for website owners or small hosting companies, but looking at the responses online, clearly not.

Load Balancing

If your provider offers such a service, ensure you have load balancing set up to filter traffic to another server in the event of downtime.

Cloud DNS

Use services such as Cloud Flare to help distribute your site to peer networks, enhance performance & their authoritative DNS services and CDN are superb for sites with lots of content/rich media.

Backups

Sounds obvious, but it’s clear in situations like this how many people aren’t making adequate backups. And even if you are, backups are useless unless you have a well-practiced restore plan. Use any control panel (cPanel or Plesk for example) backup tools alongside manual cron’ed FTP jobs to ensure you have all your files.

Use massively scalable services.

Stray away from the more simple point-and-click providers such as Heart or 123-Reg and use a more powerful provider such as Microsoft, Amazon or Digital Ocean. These companies offer absolute bomb-proof scalable services but the rub is you need to understand more about what’s going off under the hood. Those running websites for people paying them money, may want to consider Amazon’s AWS services or Microsoft Azure, more effort up front but much more reliability later down the line.

WordPress,posts and databases

Those running WordPress websites or SQL databases need to ensure they are taking care of backing up the databases too, not just the site files. PHPMyadmin or even straight at the console, a regular scheduled task to pipe the database to a remote location will pay dividends in the long run.

Posts should be backed up, all tables should be backed up. Daily, or however often you update your blog. Host your images highly compressed in a separate high availability location and not on the same server. That way, when you restore your databases, you don’t have to re-link all your images.

Expectations

Manage your expectations. Services go offline all the time – it’s the web. I’m amazed people have this attitude & expectation of 100% uptime just because they are paying for it. Assume for the worst, and plan to fail and you won’t go far wrong. Websites, associated hosting & the systems which tie it all together are massively complicated. Really.

Underpin the expectation-to-fail attitude in your strategies & SLA’s with your clients and you’ll be in a much more manageable situation when the shit hit’s the fan and the clients at your door.

Cost impact management

I’m always surprised at how people measure their financial impact on their business when their site goes down by seemingly pulling a figure out the air, or over egging the cost they pay to hosting companies as a way of justifying their complaint.

When truly asked to assess their actual loss, almost no one can accurately answer.

Try paying half a million a year or more, and maybe then you have a case to argue, but you can’t spend £60 a month and expect bombproof service.

If you can work out a cost per hour that’s realistic, (maybe your sites transactional, or you have Google analytics or AdSense and can work out a true financial cost based on those analytics) and then factor in a running cost plus contingency, and then use that as a running cost measurement, you can start to understand your true financial loss when you have an outage.

Pay for quality

Don’t skimp on hosting, backups or all the boring stuff you can’t see. It’s a bit like paying for fully comprehensive car insurance. It’s expensive, and the aim is you’ll never use it, but when you need it by golly you’ll be glad you paid the extra money. When you set out your website budget, you need to consider this up front. A 10k website with no allowance for maintenance, disaster planning or backups is going to hurt you later down the line.