The largest NHS trust in London has been forced to cancel operations and appointments as it battles IT problems following two data centre outages caused by soaring temperatures in this week’s UK heatwave. Guy’s and St Thomas’ NHS Trust has confirmed some of the systems which went down on Tuesday have yet to be restored.
The outage has been declared a “critical site incident” by the Trust, reports The Guardian, resulting in patients being asked to bring letters or other paperwork about their conditions to any appointment. Doctors and hospital staff have been unable to access medical records and the Trust’s 23,500 members of staff lost access to clinical applications they use to store and share information.
A source has confirmed to the Guardian that the reason for the outages is air-conditioning failing, which keeps the data centres cool during hot temperatures.
Guy’s and St Thomas’ NHS Trust: heatwave triggers data centre outage
A spokesperson for Guy’s and St Thomas’ NHS Foundation Trust told Tech Monitor that the IT problems are ongoing and that teams are working around the clock to fix the problems. “As a result of the extreme temperatures on Tuesday we have experienced significant disruption to our IT systems, which is having an ongoing impact on our services,” they said.
“While the majority of appointments have gone ahead, unfortunately we have had to postpone some operations and appointments and we apologise for any inconvenience this may have caused.
“The Trust has well-established business continuity plans to allow us to continue as much activity as possible and to ensure that patient safety is prioritised at all times.”
Guy and St. Thomas’ Trust joins Google Cloud and Oracle, as both suffered outages at their UK data centres this week due to hot temperatures. However, both of the tech giants had their services back up and running within a day.
The Uptime Institute, a consultancy that advises on digital infrastructure, says that 60% of organisations that experienced lengthy outages polled for its 2022 annual Outage Analysis report ran up bills of at least $100,000 (£83,480).
Climate change is going to hit data centres without investment in cooling technologies
With record-high temperatures in the UK this week, analysts believe that the NHS and the private IT sector will have to invest more in cooling technologies to ensure outages do not happen more frequently during extreme periods of weather.
Sarah Coop, thematic analyst at GlobalData, told Tech Monitor that rising temperatures seen in the UK and across Europe, are not good news for IT systems. “[Data centres] can only operate efficiently between certain temperatures, typically between 15°C and 32°C,” she explains. “Any higher, and there is a severe risk of overheating — meaning, at best, reduced latency and minor outages, and at worst prolonged outages and downtime for customers. As Europe reached highs of 40°C, that is well into the danger zone.”
The Met Office has predicted that extreme heat will continue to plague Europe over the coming decades, with more frequent episodes happening as the Earth warms up. Coop concurs and says that more frequent extreme weather events will mean data centre outages will become more common.
“While costs can be absorbed for one-off heatwaves, a longer-term solution is needed, as data centres continue to grow in importance,” Coop told Tech Monitor. She continues that it is imperative that effective cooling systems are bought as part and parcel of a data centre, or companies will risk disruption to customer’s services.
UK government needs to back investment into NHS data centres
Coop adds that because NHS trusts like Guy’s and St Thomas’ rely on on-premise data centres, rather than cloud services run by providers such as Google Cloud or AWS, they might not have the sophisticated cooling systems in place found in the private sector. She argues more investment in the NHS’ infrastructure is required to ensure it remains running during periods of intense heat.
David Bicknell, principal analyst at thematic research, GlobalData, said that the NHS and UK government will need to “think outside the box” when it comes to ways of keeping data centres cool. This could include data centres going off-premise or being run by private companies such as Microsoft or Google.
Healthcare providers like the NHS have traditionally kept systems on-premises amid fears around the reliability of cloud-based systems and the security of data, but public cloud providers are responding to the needs of the sector by providing dedicated services. Microsoft launched its cloud for healthcare last year.
When it comes to innovative cooling systems, Bicknell references Microsoft’s Project Natick, which was the deployment of a data centre on the sea floor near the Orkney Isles in spring 2018. Microsoft says that the team hypothesised that a sealed container on the ocean floor could provide ways to improve the overall reliability of data centres. This idea is now being picked up by Chinese authorities, showing that the idea has traction with governments.
“There’s four Chinese authorities that are backing underwater data centres at the moment – two provinces and two cities,” Bicknell says.
Read more: AI is no silver bullet for the data centre skills shortage
