Google has called industry and academia to join hands in the development of a new type of hard disk which will better fit data centers supporting cloud-based storage services.
Google Infrastructure VP Eric Brewer said in a blog: "The rise of cloud-based storage means that most (spinning) hard disks will be deployed primarily as part of large storage services housed in data centers.
"Such services are already the fastest growing market for disks and will be the majority market in the near future.
"For example, for YouTube alone, users upload over 400 hours of video every minute, which at one gigabyte per hour requires more than one petabyte (1M GB) of new storage every day or about 100x the Library of Congress."
The company has also released a white paper suggesting the need to optimise the collection of disks, rather than a single disk in a server.
Google said that YouTube uploads see a 10x increase every five years. It has suggested an overhaul of the design saying that the current generation of modern disks, often called ‘nearline enterprise’ disks, are not optimised to handle the huge inflow of data.
According to the company, the disks were designed for traditional servers and need to be redesigned to meet the need of large scale data centres and services.
Content from our partners
"It’s a fun but apocryphal story that the width of Roman chariots drove the spacing of modern train tracks." Brewer said.
"However, it is true that the modern disk drive owes its dimensions to the 3½" floppy disk used in PCs. It’s very unlikely that’s the optimal design, and now that we’re firmly in the era of cloud-based storage, it’s time to reevaluate broadly the design of modern disk drives."
The ‘data centre disks’ idea highlighted in the white paper suggests changes to disk designs, arguing for three differentiators for HDDs used in data centers.
The first key difference is the collection view, which focuses on aggregate properties of a large collection of disks. Secondly, the whitepaper focuses on tail latency derived from the use of storage for live services, while thirdly it focuses on variations in security requirements that stems from storing others’ data.
Google has suggested five key metrics for disk drives for data centres which include higher input output (I/O) per second IOPS, limited primarily by the HDD seek time, higher total capacity, lower tail latency, security requirements and lower total cost of ownership (TCO).
Brewer added: "This shift has a range of interesting consequences including the counter-intuitive goal of having disks that are actually a little more likely to lose data, as we already have to have that data somewhere else anyway.
"It’s not that we want the disk to lose data, but rather that we can better focus the cost and effort spent trying to avoid data loss for other gains such as capacity or system performance.
"We explore physical changes, such as taller drives and grouping of disks, as well as a range of shorter-term firmware-only changes.
"Our goals include higher capacity and more I/O operations per second, in addition to a better overall total cost of ownership. We hope this is the beginning of both a new chapter for disks and a broad and healthy discussion, including vendors, academia and other customers, about what "data center" disks should be in the era of cloud."
