When IBM makes a serious investment in a technology, you can be pretty sure that it’s not taken the decision lightly. So when the firm announced earlier this month that it plans to invest $1 billion in research and development to design, create and integrate new Flash storage technologies into the enterprise, it’s worth taking note. But with or without IBM’s investment, flash storage arrays have well and truly arrived, and now represent a serious threat to hard disk arrays from incumbent mechanical disk drive players including not just IBM but the likes of EMC, NetApp, HP and HDS. As Sam Cooke sang, a change is gonna come.
Flash memory was invented by Dr. Fujio Masuoka while working for Toshiba in the Eighties. According to Toshiba, the name "flash" was suggested by one of his colleagues, because the erasure process of the memory contents reminded him of the flash of a camera. Since then flash has revolutionized a number of sectors, moving the goalposts in areas including digital photography, digital music and portable high-capacity data storage. Flash memory is big business, with estimates that the total industry was worth $26.8bn in production and sales last year, and that it will grow to $30bn this year, according to IC Insights.
In recent times, flash memory has been flying off the shelves inside smartphones. IC Insights says around 26% of total flash sales are in smartphones, while flash in solid state disks (SSDs) came in second at 13%. Tablets and media players accounted for 12%, with USB sticks and memory cards coming in at 11 and 8% respectively. But the mix is changing, according to the researchers. Memory cards, USB sticks and media players are likely to decline, replaced by smartphones, tablets and cloud services. But SSDs are likely to continue to drive sales of flash memory: their time has finally come.
Mimicking disk
Solid state drives, sometimes known as solid state disks, don’t contain any actual disk, unlike standard mechanical hard drives. Instead they use flash memory and some clever circuitry to act like a traditional hard disk. A bit like one, anyway.
"Our flash arrays are dramatically simpler, uses less power and is ten times faster and no more expensive than traditional disk," says Scott Dietzen, CEO of all-flash array vendor Pure Storage. "People don’t believe us when we say we can give them flash for the price of disk."
SSDs have been around for many years, but despite offering faster data access than mechanical disk have proven too expensive to find much market penetration except for particularly data-intensive applications, or as a cache that sits in front of a storage array. So while Enterprise Strategy Group has said that about 40% of large companies have at least some flash in their enterprise, it still accounts for less than 10% of total storage capacity. But the huge boom in flash-enabled devices in the consumer world, particularly smartphones and tablets, has seen prices come down.
As well as flash becoming more affordable, there’s been a need for something faster than mechanical disk anyway, according to Pure’s Dietzen. "Disk today works slower than tape used to, because networks are 1,000 times faster than the disk," he says. "Moore’s Law has effectively turned disk into tape. Pure can deliver flash at the price of disk, and that’s going to be hugely disruptive."
Virtualisation and read-write intensive applications like Big Data have piled pressure on traditional disk, at times making it the data centre’s main bottleneck. Virtualisation, of desktops or servers, can also put a heavy burden on 15,000 RPM mechanical disks. Flash offers a solution.
Of course, the storage incumbents are not sitting on their hands and simply letting the likes of Pure Storage, Whiptail, Coraid, Violin Memory and others steal their market. EMC, for instance, already has a number of flash products, but these have so far been more of the server-side, flash-as-cache variety.
However in May last year EMC bought all-flash array vendor XtremIO, and it looks like EMC will bring that product to market soon. In February this year NetApp announced a new all-flash array, the EF540, but also announced a brand new flash-architected product family, FlashRay, which it says will be available in early 2014. "Flash changes everything by transforming the speed of business," said Manish Goel, executive vice president, product operations, NetApp. "However, enterprise imperatives for global scale, efficiency, and reliability remain the same."
Talking to CBR this month, Laurence James, product alliances and solutions marketing manager at NetApp said the EF540 is really aimed at dedicated workloads, rather than shared workloads that the FlashRay line will be better suited to. But he also cautions that flash can have its drawbacks. "Flash can have some challenges if you use it just like a hard disk, with heavy reads and writes," he says. "You need to be aware that flash wears out over time, it basically gets ‘poisoned’ by electrons, so we have wear levelling technology to counter that."
On price, James says, "It’s still a technology where you need to look closely at price-performance. Some workloads are suitable for flash but some may not be. Our flash arrays do 300,000 IOPS per second but if you don’t need that, why pay 15-20 times the money per Gigabyte? It’s not always financially viable."
But the price-performance arguments are not damping the spirits of the flash array specialists. Whiptail, for instance, has announced that its Invicta array will scale up to 30 nodes and up to 360TB of all-flash storage. "One clear theme that we’ve heard from our customers is, ‘we want more,’" says Whiptail CEO Dan Crain. "What they are discovering is once they start moving their data and applications to Whiptail, their productivity increases so much that they want to bring all of their business critical apps to flash. It’s like eating potato chips, you can’t stop with just one."
Whiptail said that in initial testing, a 180TB, 15-node Invicta Infinity produced performance numbers of 2.1 million IOPS and 21.8GB/second throughput. "We expect a fully-populated 30 node, 360TB Infinity to exceed 4 million IOPS and 40GB/second throughput in real world use," says Whiptail CTO James Candelaria. "We’re breaking down the data silos, but more importantly, we’re allowing the enterprise to think about flash strategically rather than tactically. You no longer have to worry about hot and cold data when you can have everything available on Whiptail solutions."
The analysts agree that flash has arrived, with IDC for example expecting the market to reach $1.2bn by 2015. However, given the particular wear characteristics of flash, enterprises are advised not to get too distracted by the huge performance gains over traditional disk. "A new class of arrays is unlocking flash’s full potential and delivering capabilities that accelerate the performance, reliability, and efficiency of enterprise data centres," says Jeff Janukowicz, research director, solid state storage and enabling technologies at IDC. But he warns, "For all-flash arrays to gain broader market adoption, it is important to look beyond the performance improvements and deliver must-have reliability, availability, and supportability features."
The likes of big data and other data-intensive applications are making traditional disk look something of a slouch relative to server and network performance. That, combined with the falling cost and increasing dependability of all-flash storage devices is coming together to create something of a perfect storm in the storage industry.
Case study: Anglia Ruskin University
When Anglia Ruskin University needed to provide a new 500-seat Cambridge student IT open access area, within a new building which was built with no provision for additional cooling, it decided that traditional desktops were not viable. "We needed to reduce power consumption and improve our ability to deliver software to those who need them irrespective of location," says Gregor Waddell, assistant director at Anglia Ruskin. ""We wanted a modern and attractive desktop for our student and staff community that could take advantage of recent technology innovations to improve their experience. This all came together at a time that VDI [virtual desktop infrastructure] technology was ‘coming of age’ and offered a credible solution."
With ageing Windows XP desktops (and XP support from Microsoft coming to an end for all enterprises in April 2014) it was time to refresh the estate anyway. But with students using rich multimedia applications it still needed to maintain high quality audio and video, and this required high storage performance – 100,000+ IOPS. At the same time it wanted to reduce power consumption if possible and improve its disaster recovery and resilience.
"We were convinced by the concept and advantages of virtualisation for our server infrastructure and had already pushed almost all of our server estate over to VMWare Sphere based virtualisation," says Waddell. "This naturally led us to consider Desktop Virtualisation and related thin client technologies as a potential way forward."
But to cope with hundreds of thin client users it realised a traditional storage area network would not cut the mustard: it would need to be too large and use too much power and cooling. "After considering several options we chose a 3000 Series Violin flash Memory Array," Waddell says. "Storage performance is key to VDI and our existing traditional spinning disk did not offer good enough performance. The virtual machines needed 80-100 IOPS per desktop in our more demanding environments, of which most were writes."
The Violin flash Memory Array chosen and installed by Anglia Ruskin University is capable of handling 220,000 random write IOPS in 4K blocks, more than 20 times the performance of a comparable SAN disk array. The University says the 500 thin clients boot much faster than PCs, that the whole VDI set-up uses 30-40% less power per user, and that feedback from students has been excellent. Thin clients have since been rolled out to over 1,000 desktops.
