Data centres hold all digital information in the world. If a facility goes down, services go down.
If the recent bomb threat at the Amadeus hub had actually materialised, then European airports and hotels could have been sent into a travel frenzy with bookings and reservations going down across the continent.
Patrik Sallner, CEO of MariaDB said: "A lot of these SaaS have built redundancy and disaster recovery. If there is a failure in a data centre – if a server as a physical problem, if there is a connectivity problem, or if there is an electricity supply problem – all data centres have reserved generators in them. But as we have seen in the past, when the whole data centre is threatened, redundancy and replication needs to be built in the server, in the data centre and across data centres.
CBR has compiled a list of potential scenarios in different industries in the eventuality of a data centre failure.
1. Cybersecurity
On March 10 this year, AVG’s AntiSpam data centre was down due to unplanned maintenance, a week after the company announced it had reached the 200 million users mark.
Thousands of customers worldwide were left powerless, with no access to their email accounts and security services for a few minutes.
AVG said in a statement: "[The] anti-spam portion of the AVG Business CloudCare service could have been disrupted as a result, possibly affecting email security services customers in all regions.
"The exact cause is still being determined but the situation is in the process of being rectified, and normal service is expected to resume within the next 60 minutes. We will continue to monitor developments and regret any inconvenience to those customers affected."
If security fails in the data centre, a host of sensitive information could end up in the hands of hackers and cyber criminals.
2. Telecommunications
There have been several reports of users not able to text or call due to data centre breakdown. For example, in April this year, an electrical problem at one of Three’s data centres in Ireland brought services down to as many as two million costumers.
Even Google is not immune to data centre failures. A power outrage in the early Spring of 2010 brought down the company’s mobile apps services for a few hours.
A Google report explained that the underlying cause of the outage was a power failure in the enterprise primary data centre.
It continued: "While the Google App Engine infrastructure is designed to quickly recover from these sort of failures, this type of rare problem, combined with internal procedural issues extended the time required to restore the service."
3. Internet
With the world ever more connected, if the internet goes down nearly everything stops. That, together with a data centre failure, is a recipe for online mayhem.
Thousands of Australians, costumers of iiNet, the country’s second largest internet provider, were left offline after record temperatures of nearly 45°C put the enterprise’s data centre in Perth on standby on January 6 2015.
Christopher Taylor, an iiNet representative said at the time in a forum post that they had had multiple air conditioners failing on site, causing temperatures to rise rapidly. He added: "Multiple systems have been shut down on site to prevent permanent damage".
Another heat problem brought down Microsoft’s email services for 16 hours, also affecting Skydrive. The incident in March 2013 was caused by a failed software update that rose temperatures up to a breaking point.
4. Justice
Criminal records, police files, court cases, and many more are stored digitally as security departments enhance their IT strategy.
In the UK, almost 100 Crown courts faced operation restrictions in December 2014 after the XHIBIT system went down due to a failure at supplier CGI’s data centre.
In June same year, Marysville Data Centre in Canada cost the local economy $1.6 million leaving several state departments in the dark, the governmental report said at the time.
Critical government computer systems crashed after a power outage took the data centre offline. As a result, courthouses came to a standstill as probation reports and victim impact statements could not be generated delaying the juridical system.
5. Health
Queensland’s hospitals went into meltdown on December 10 2014, when a fault in a data centre took doctors back to pen and paper.
Queensland Health, the organisation in charge of all the state’s hospitals, said an issue in its data centre led services to action "contingency plans" to keep things running.
A public advisory by QH said: "No patients are being turned away from hospitals and surgeries are occurring as normal. Routine downtime procedures have been implemented to ensure patient safety.
"All hospital and health services have experienced some level of interruption to services but all have developed contingency plans to ensure services to patients continue."
Later it was found that a software upgrade of storage controllers in the data centre in the Royal Brisbane and Women’s Hospital, Herston, unleashed the sequence of events.
6. Banking
Not all data centre failures are down to human error or threat. Nature plays a big part, hence companies like Apple building hubs in the Nevada dessert which are believed to be nearly 100% nature proof.
On February 9 2015, a severe thunderstorm shut down Fujitsu’s Tier III Perth Data Centre. The facility went into melt down after two back-to-back failures.
Consequently, Australian Bankwest lost services to all its ATM cash machines, as well as online and phone banking applications for 12 hours.
7. Transportation
Traffic control – ground, air or water – is controlled by systems, and those systems are based in a data centre.
The tourism industry has experienced some serious computer failures in data centres and the bomb threat at Amadeus Data Centre on May 12 2015, brought back the ghosts of January 30 2012.
Amadeus hosts the ALTEA system, used globally by over 100 airlines including British Airways, Qantas, Iberia and Cathay Pacific.
The software controls all online systems used in airports and related websites. In 2012, an airline IT collapse left millions of passengers stranded, with check-ins not being able to be processed, reservations failing and airport counters’ computers down.
The Amadeus data centre failure also affected agency bookings with reservations being lost or misplaced in the system.
Amadeus confirmed at the time that the incident occurred from a network change to a module within the data centre enterprise network in Germany.
8. Energy
In October 2012 New York authorities did not expect Hurricane Sandy to hit so hard – but as we now know, disaster did indeed strike.
Several data centres were flooded and many destroyed causing power outages across Manhattan.
Consolidated Edison, a local electricity supplier, had to temporally shut the electrical grid of lower Manhattan to avoid further damage to their data centre.
The company said that the storm was the largest in their history, with thousands of customers affected.
9. Stock Exchange
Stock Exchanges are the heart of every economy. They provide a secure and fair place for trade and control millions of transitions every day, everywhere.
The Singapore Stock Exchange was faced with a bill of $1 million after it had to shut for three hours on 7th November 2014.
The cause? Lightning and a lot of bad luck.
The voltage fluctuation (which lasted for 0.1 second) led to the malfunction of SGX’s two uninterruptible power supplies (UPSs).
In a statement, SGX said: "The power supply disruption was transient in nature, bringing about a potential mismatch between the primary and secondary market order books.
"As such, a cut-over to the secondary data centre from the primary data centre was not executed. A complete restart of both primary and secondary systems was necessary. This step was essential to ensure continued integrity of the markets."
10. Business Costs
All in all, if a data centre goes down, businesses are affected and organisations are faced with revenue losses. Depending on the database, millions of people could also be affected by losing power, bandwidth or simply by diving into a traffic chaos.
According to the Ponemon Institute, the average cost of a data centre outage to colo providers was $690,204 in 2013. That year, 67 unplanned colo failures cost the industry over $46 million.
The "2013 Cost of Data Centre Outages" study, sponsored by Emerson, concluded that data centre failures in 2013 cost 41% more per minute than in 2010, topping $7,900 and $5,600 respectively.
The most recent paper on data centre outage costs to date said: "Industries with revenue models dependent on the data centre’s availability to deliver IT and networking services to customers – such as telecommunications service providers and ecommerce companies – and those that deal with a large amount of secure data – such as defence contractors and financial institutions – continue to incur the most significant costs associated with downtime. The highest cost of a single event [in 2013] was more than $1.7 million."
The real cost for all businesses – providers, customers and end users – of data centres’ outages has never been calculated. Although, with flights cancelled, businesses closed and no trading in stock exchanges, the bill could easily maybe surpass the trillion Dollar mark.
Also in 2013, Virtual Hosting predicted that if every data centre on Earth failed, it would cost the global economy $69 trillion per hour.
