Dutch banking giant ING’s data centre was knocked offline this weekend after a fire extinguisher test went wrong.
In what the bank called a “serious technical problem” a planned test of its fire extinguishing system in its data centre managed to knock out servers and the data storage system.
All linked services including card transactions, ATM transactions, Internet banking, communications, and its website were all taken offline.
The data centre issue hit customers of ING Romania and the problem lasted from 1pm to 11pm local time.
Daniel Llano, Head of ING Retail Banking, said on the company’s site: “I activated emergency procedures and recovery plans provided for such situations. But because of the magnitude and complexity of breakdowns, unfortunately the time required to restore activity by the back-up was longer than the tests we perform regularly.”
Llano went on to say that the bank is monitoring developments with its suppliers.
The fire extinguishing system uses a gas called Inergen, which is a mix of nitrogen, argon and carbon dioxide. This system is used because they are not typically harmful to computer equipment, however, the sound caused by the release of the gas could be the cause of the problem.
The high-pressure release of the gas is said to create a 130 decibel sound, according to Motherboard. The sound wave vibrations sent through the air as a result of this is said to cause vibrations in the hard disk drive case, causing misalignment of the read/write heads to the data tracks.
A report by Siemens looked into disruptions to HDDs caused by inert gas extinguishing systems, like the one used by ING, points to the potential cause. However, the study does say that the issue has not yet been fully integrated, due to: “many different designs of storage system chassis and cabinets on the market.”
The report does show though that sounds over 130dB can stop delivery data on most disks and above 140dB would cause permanent damage to most disks.
Llano wrote of untold regret for the inconvenience and said that ING would refund the fees for cash withdrawals from the ATMs of other banks.
