Advanced Computing in the Age of AI | Friday, March 29, 2024

Challengers Arise to S3 Storage Dominance 

When it comes to storing data, Amazon’s Simple Storage Service (S3) is a runaway success. With trillions of files under management spanning many exabytes worth of data, S3 is the largest digital storage system in the known universe. But things rarely stay the same for long in the IT business, and S3’s outright dominance is spawning attacks from several angles.

We've reported on Wasabi, the Carbonite spin-off that claims to offer six times the performance of S3 at one-fifth the cost.

Another area where S3 dominance is giving rise to competitive storage regimes comes to us from Qumulo, which has developed a scale-out file system with object-like features.

Yesterday Qumulo (which was founded by former Isilon engineers) launched the second major release of its File Fabric. QF2, as it is called, adds several features that will help it be more like an object system, including support for running on AWS and a new continuous replication feature for on-prem and cloud clusters.

With these features added to its NFS- and SMB-loving file system, Qumulo says it has created a new product category, which it dubs a universal-scale file system.

Ben Gitenstein, Qumulo’s director of product management, says a universal-scale file system offers some of the same benefits found in an object storage system, including elastic scalability and support for cloud deployments, but does so without the drawbacks of object storage, such as the need to rewrite applications to use object protocols, namely Amazon’s S3.

“Our thesis is, what if you made file scale like object and offered the same hardware portability like object and run in the cloud like object, but it’s still file?” Gitenstein says. “That’s our fundamental thesis.”

Practically all on-premises applications were developed to support file systems, namely NFS and SMB (as well as CIFS, which is considered a version of SMB). While a new generation of cloud-native applications read and write data to storage using S3, PI, most existing applications developed to run on Windows, Mac, and Linux still speak SMB or NFS.

“There’s a lot of customers in the world who are actually file customers who are told they have to go buy object, and they’re told they have to go buy object for one of four reasons,” Gitenstein says. The first reason is because the customer needs tremendous scalability. Secondly, they need greater hardware portability. Thirdly, they want to run on the cloud. Lastly, they want to write and manipulate custom metadata in a way that object storage systems are preferable to file systems.

“Our argument, and what we proved with our customers, is that tradeoff is only true because existing file hasn’t innovated,” Gitenstein says. “Customers say, I have outgrown existing file system so I have to either break up my infrastructure in this terrible way, or I have to rewrite everything in object and move to object.  Or I can use object with a file gateway on top, in which case I basically lose all the rich metadata of object and I don’t get the full file semantics because it’s just a gateway, it’s not file at the file level.”

With support for cloud deployments and geographic data replication in QF2, Qumulo thinks that customers no longer have to sacrifice the benefits of traditional file systems  — which at this point include familiarity and stability — to get the benefits associated with object storage. “What’s different about us is we scale to the same breadth and depth as object does,” Gitenstein says. “But at its heart it’s an actual file system. It’s not an object system with a file gateway on top. “

Qumulo doesn’t think AWS customers are going to rip apart their existing EC2 and S3-based apps and move them to Qumulo’s file system. But for some organizations in data intensive industries, such as entertainment, oil and gas exploration, life sciences, who are considering shifting to S3 to get object scalability, the availability of scalable NFS via Qumulo could prove to be a better and more cost-efficient solution.

“We don’t think of ourselves as competing with Amazon,” Gitenstein says. “The customers that we sell to all know and love file. They just want it to scale to the same level as S3 does, and give them the same access to cloud that S3 does.”

Scott Sinclair, senior analyst at Enterprise Strategy Group, told EnterpriseTech he "continue(s) to hear good things about the insights the technology offers. And the new additions only extend the value."

"With the new announcement, Qumulo can now extend its file fabric to Amazon Web Services," said Sinclair. "The result is more deployment flexibility, the same Qumulo technology can be deployed either on premises, in the public cloud, or in a hybrid environment."
About the author: Alex Woodie

Alex Woodie has written about IT as a technology journalist for more than a decade. He brings extensive experience from the IBM midrange marketplace, including topics such as servers, ERP applications, programming, databases, security, high availability, storage, business intelligence, cloud, and mobile enablement. He resides in the San Diego area.

EnterpriseAI