S3 object storage specialist Cloudian has been growing rapidly in recent years, tapping a vein of demand for “super capacity” storage requirements; or capacity-intensive object storage workloads that can reach tens of petabytes.
The company offers a software-defined object storage service that transforms servers into a pool of logical storage resources, using Amazon’s S3 API, which has become the de facto standard for object storage APIs.
Its most recent offering comes via a tie-up with flash storage specialist Seagate (featuring Cloudian software and Seagate hardware) that allows users to store up to a massive 18 petabytes in a single data center rack).
Its focus: private cloud, where more and more data is being stored.
(While in recent years data has moved en masse to the cloud, with a focus emerging on the edge, private cloud environments closer to the edge are becoming crucial to securing faster input and output speeds).
Cloudian CEO Michael Tso, an MIT graduate who was raised in Shanghai and Hong Kong, told Computer Business Review: “I think the thing that is very, very apparent in both of these announcements is that both Seagate and VMware [Cloudian also announced a major partnership with VMware in June] have come to the same conclusion; they also believe that S3 is the standard for the object storage world.”
“Its a bit like NFS: lots of people don’t remember, but NFS ended up creating the entire file storage market. Before that there were a number of other protocols. And the NFS was really only one of them.”
Object Storage: Remind Me…
Object storage doesn’t use a tiered file structure when it stores data or files, instead it keeps all of the information that makes up or is relevant to a file in the one place.
All data is seen as distinct units known as objects. So while some storage systems keep metadata in different file folders, object storage lumps it all together into one storage pool.
This comes with a host of advantages, most notably a greater ability to run analytics on the stored data. Since every piece of data is housed with the relevant metadata, it brings a higher level of classification for the data.
Due to the data all being stored in one storage pool and not spread out across different folders, as it would in a hierarchical file system, it is also quicker to retrieve files.
Cloudian CEO Michael Tso on his S3 Bet
One of the key draws for companies towards Cloudian’s products is the fact that it is fully compatible with Amazon S3, otherwise known as Amazon Simple Storage Service. All of Cloudian’s protocols match perfectly with Amazon’s and that’s no coincidence.
“That was a bet,” Tso happily tells us; one made back in 2011. At that time everyone was using different standards: you had OpenStack protocols, Swift protocols as well as standards pushed out by industry consortiums.
What Tso saw happening was that gateways were being created to help translate each of the protocols into common sets of laws. Tso compares it to a human translator working with different languages: “If you have to translate all these different languages to a single language. It’s never exactly the same.”
So his solution was to pick one and stick with it. “We basically built our product protocol to be completely compatible with the Amazon S3 protocol down to every error code, every single behaviour. Amazon becoming the biggest public cloud provider? That’s secondary! We were betting on S3″.
“It’s all about getting this spiral going,” he adds.
“If you have a good protocol then more developers are going to write their software for that protocol and the more software is available, the more users, and more users attracts more people to write more interesting software.”
When he looked at what Amazon was doing, he saw a rich developer community that was adding to the protocol and APIs at a fast pace, with a multitude of developer blogs outlining their work: “The way they were adding was truly driven by end user demand. And the reason why was because they were actually listening to the users, so they had a very good support system.”
“I spent the early part of my career at Intel and I saw exactly how powerful this kind of spiral or positive feedback is. That’s how you start a tornado. Nobody would say X86 is the most elegant of the CPU protocols, but it got into the [upward] spiral…”
Seagate: New Tech HAMR’ing at the Door
With the Seagate partnership meanwhile, he hopes to tackle a private cloud market in which IDC expects spending to expand 58 percent by 2023.
The two’s “Hyperstore Xtreme” offering represents a significant jump in storage capacity from Cloudian’s previous non-Seagate device, the HyperStore 4000 Series which could only hold up to 840TB.
This is a collaboration that Tso expects to continue. And he thinks Cloudian can play a unique role for Seagate too, owing to some idiosyncrasies of its customer base.
He said: “We’re very excited about the Seagate partnership because Seagate’s got a very aggressive roadmap in terms of denser capacities in devices coming out. They’re using this ‘HAMR’ technology which is basically using a lasor to heat up your hard drive – amazing technology that they’ve spent over a billion dollars to do. They now have a roadmap to 40, 50 TB in a 3.5-inch form factor.”
“For us it’s very critical to have access to the latest technology when it becomes available, because that’s the whole idea of software defined storage. Rather than waiting for six months or a year for these new devices to come out and you have to be quantified by the major storage vendors, you should be able to run the software on whatever the latest devices are coming out, without a tremendous separation in terms of the time lag.”
“And up to this point they’ve only had something like eight large customers around the world: they don’t really do business directly with smaller companies. They sell to the cloud guys; the Dell EMC, IBM and a few others. Their time to market is controlled by eight buyers…”
“Seagate wanted to find a more direct path to market, so they can bring forward their time to money. So rather than waiting nine months to a year for the volume orders, they… need a player who can run in front of the pack with a really solid product that can bring their latest technologies into the marketplace and showcase to everyone else that it works.”
Amid his many years in business, any key lessons he has learned along the way (other than an ability to spot trends and accrue an impressive array of patents?)
Be brutally decisive, is the answer: “In order to create Cloudian we had to close down, or transition a previous startup that we had run for about 10 years in the mobile messaging space. We made large scalable messaging systems for mobile telcos.”
“In order to create this new company. We had to lay off two thirds of that company so we had to do some very very difficult things. I was the CTO and became the CEO; I had to figure out who to lay off and what was the plan. It was extremely challenging. I think the biggest mistake I made during the time was taking too long to make the decision.”
“We were planning to go to IPO in 2009 but after the market collapsed [following the 2009 financial crisis] we kept hoping things would get better and prolonged the pain. Sometimes you have to take drastic action, even if it’s painful.”
See Also: Faulty Firewall Process Eating CPU Causes Major Cloudflare Outage
