Everyone loves a good horror story on Halloween; there’s nothing like a good mystery about ghosts and ghouls, or things that go bump in the night to get you into the spirit of the thing. However, the scariest horror stories often aren’t about the supernatural, the gory, or the macabre; they’re the ones that happen in real-life. They’re the ones that blindside you at work when your worst nightmares come true, or the thing that you thought could never happen suddenly does. The IT industry is no stranger to these situations, so I thought I’d share a few of the ‘horror stories’ I’ve seen or heard over the years for Halloween.
The midnight spectre
Anyone who’s ever worked in IT has experienced a high-pressure midnight launch at some stage. Aside from a catastrophic IT outage, they’re probably the tensest situations that IT teams have to face. Imagine the scene then; a major global automotive brand was gearing up to launch a hotly anticipated new website feature that was miles ahead of anything its competitors were doing. Focus groups had revealed huge demand for the feature amongst customers. As a result, the expectations of the business were sky-high and the eyes of the entire executive leadership team were on IT to ensure the midnight launch was a success.
The digital marketing team had flown in from headquarters, along with the hip creative agency from LA. It fell to a band of humble site engineers, developers and sys-admins to ensure everything ran smoothly, and that the launch went off without a hitch. Of course, things didn’t go to plan and disaster struck almost immediately. The site fell over within minutes of the first code deployment and was later found to have been literally DDoS-ing itself because of the way the image files were being served. Unfortunately, the launch had to be abandoned and it wasn’t until the next day that the team was able to identify the various issues by using performance management tools, and to get them fixed. There was a happy ending though – the launch went ahead successfully the next night, resulting in big cheers all round.
The nightmare before Christmas
The festive shopping period is make or break for retailers. Nothing strikes fear into their hearts more than the prospect of a lengthy IT outage during the Black Friday rush, or in the final few days before Christmas. Some of the biggest brands on the high street have fallen foul of their own success, with digital footfall overloading their websites during these crucial trading periods.
It was on just such an occasion that I found myself in the IT department of one of the UK’s luxury goods retailers as it was gearing up for what was expected to be its busiest trading day ever. The company had been preparing its Black Friday deals for months. However, the demand was far greater than anyone had dared hope. Very quickly, the website was struggling under intense demand. It became slower and slower, leaving customers frustrated, before the whole site eventually collapsed. As you’d expect; shopping carts were rapidly abandoned and the check-outs certainly weren’t jingling all the way.
Content from our partners
The ghost licences
A major software vendor was under pressure to increase the take up of its free trial after it introduced a major new feature. However, many of its free trial emails were being filtered out as spam, because they were formatted in HTML. As such, the digital marketing team decided to switch to Plain Text in the expectation that this would increase the delivery rate.
When the new feature went live, a major mailshot went out to all prospects and existing customers and the board sat back and waited for the trials to start rolling out. However, the initiative was met with resounding silence; no trials were taken out and sales didn’t experience any uptick at all. Red faces grew redder still when a customer eventually got in touch to ask why they hadn’t received the free trial licence file. After looking into the issue via a digital performance management solution, IT instantly identified that the switch to Plain Text had broken a line of code that attached the free trial file; marketing had just been firing out blank emails.
Looking back, these stories serve as an entertaining yarn for Halloween, but at the time they were truly terrifying for those involved. What they all underline is the importance of having performance baked-in for any organisation in today’s digital economy. We rely on technology more than ever, but as it becomes increasingly sophisticated, it also grows incredibly complex. That hyper-complexity is making it a nightmare for IT teams to keep everything running seamlessly, unless they can maintain end-to-end visibility across their IT ecosystem. That type of visibility is the only way for IT teams to keep themselves from being the protagonists of their own performance horror story.
