Those who store their data sets on magnetic tape (certainly on their own premises) are on very thin ice, if you ask Amazon Web Services (AWS).
A press release for “Glacier Deep Archive”, its low cost data storage offering – generally available as of late March – gives a flavour of the mood.
“Maintaining tape infrastructure is difficult and time-consuming; tapes degrade if not properly stored and require multiple copies, frequent validation, and periodic refreshes to maintain data durability. “It is difficult or impossible to do machine learning and other types of analysis directly on data stored on tape…”
Magnetic Tape’s Old Hat, Right?
Tape, a reader not looking carefully would be forgiven for thinking, is a redundant technology being superseded by the cloud.
So what technology does AWS’s Glacier Deep Archive, priced at just $1 per terabyte (TB) per month (with access within 12 hours), exactly use itself for storage?
The company is exceptionally coy about this.
In an emailed response to our questions, AWS declined to give a clear answer.
“S3 Glacier Deep Archive utilizes a combination of common media storage technologies that are purpose-built into systems configurations optimized for cost and use cases where customers’ data is rarely accessed. These configurations let S3 Glacier Deep Archive prioritize and reorder requests to have the optimal read/write patterns on the underlying storage media in the most efficient way.”
So, is tape involved?
“We don’t specify the type of storage media used.”
Industry observers suggest the offering is based on a mix of tape, flash and disc; made available for access via the kind of heavily robotised warehouse infrastructure in which AWS has such extensive experience.
How might that work?
Tape Can be Accessed, Fast
Magnetic tape can now be accessed comparatively rapidly.
The advent of Linear Tape File System (LTFS), a format developed by IBM and adopted with variations by HP, Quantum, and the LTO Consortium, makes the use of LTO (an open-format tape storage technology) magnetic tape both fast and compatible with more modern systems; with an LTFS operating system extension deploying a graphical file manager and directory tree for rapid access to data on an LTO tape cartridge.
As an LTO whitepaper notes: “LTO tapes with LTFS have dual partitions: One partition holds the tape index, and the other holds the content. The tape becomes self-describing and enables easy viewing and access to tape files. Administrators can drag and drop files to and from tapes to browsers or directory trees.”
(LTO technology is currently in its eighth generation. LTO-8 specifications support tape cartridge storage compressed capacity of up to 30 TB, twice that compressed capacity over the previous generation, and tape drive data transfer rates of up to 750MB*per second for over 2.7TB of storage performance an hour per drive).
When a library loads a LTFS tape into the drive and mounts it into the file system, files become visible for those using tools like IBM Spectrum Archive – as if they were on a disk. Because access is not through backup software, cartridges formatted with IBM Spectrum Archive can be exchanged more easily between users working in different operating systems, using different software and in different locations.
In short, with a range of tools [pdf] now available (an area in which IBM has carved out a comfortable niche) to port tape data to and from other storage media, many IT architects aim for what has been dubbed “flape”; a combination of flash and tape.
All About the Flape?
Under this architecture, users place active data on flash to accelerate storage performance and efficiency; move less-active data to tape to lower overall storage costs. As IBM puts it: “Tape actually offers many advantages to cloud service providers (CSPs), thanks in part to the rapid evolution of both tape technologies and complementary SDS capabilities. In fact, today many CSPs are deploying tape in their infrastructures to address challenges of capacity growth, scalability and cost”
AWS’s Deep Archive may well be among magnetic tape’s users.