The Inefficiency of Scale-Out Storage

While it seems to solve many storage challenges, the inefficiency of scale-out storage adds cost and complexity to the storage infrastructure. The challenge with scale-out vendors is that they are more concerned with adding storage nodes to the cluster than getting the most out of each node within that cluster. In our on-demand webinar “Showdown Scale-up vs. Scale-out Storage, Which is Best for Your Data Center,” we compare the two common storage architectures. 

Where Did Scale-Out Storage Come From?

Fifteen years ago, scale-out storage came to market when CPU performance, networking bandwidth, media capacity, and performance were too limited. To solve these limitations, scale-out storage vendors needed to “stitch” together resources to meet the data center’s performance and storage capacity demands. In 2020 however, the capabilities of these resources have significantly improved and, in many cases, deliver far more performance than the organization needs. 

Inefficiency of Scale-Out Storage

For example, 100GBE networking cards are not much more expensive than 25GBE cards and every enterprise SSD introduced in the last year is capable of delivering 200K IOPS; some can even provide 400K+ IOPS! The problem is that the core storage software, which was developed years ago for much older media types, has not been re-written to take advantage of these new improvements. As a result, the average starting price and the storage system’s performance capabilities are about the same as three years ago. The inefficient storage software also makes scale-out storage appear as the only answer. 

The Inefficiency of Scale-Out Capacity

One of the scale-out storage vendors’ claims is that the architecture can expand so simply. “Just add a node,” is their battle cry. The reality is that just adding a node is never as simple as it sounds. Logistically, space has to be made in the rack, storage teams need to work with networking to arrange connectivity, and the scale-out storage software has to assimilate the new node into the cluster. These are time consuming processes that may take weeks to coordinate. 

In addition to the logistical challenges that expanding a scale-out storage cluster requires, the architecture cannot effectively utilize high capacity drives. Similar to the obstacles high-density drives bring to RAID (see “Exposing the High Cost of RAID“), they also cause scale-out storage problems. Most clusters need a minimum of three nodes to assemble a quorum, and in most cases, each node in the cluster has to have all its drive bays full. 

High-Density Storage Breaks Scale-Out Storage 

One of the most cost-effective server configurations on the market today is a server with 24 drive bays. Assuming a three-node requirement and using ~16TB drives (SSD or HDD), that means the starting point for most scale-out storage systems is over 1PB. Even with today’s rapidly growing data capacity requirements, most data centers do not need 1PB of storage. 

The result is most scale-out storage configurations don’t take advantage of the density of high capacity drives and will sell customers drives in the 4TB or 8TB range. Using lower capacity drives does bring down the initial capacity but wastes data center floor space and is potentially more expensive per GB. Additionally, most scale-out storage architectures can’t have nodes with different storage capacity levels; which makes matters worse. They have to stand up another cluster to use different sized media, which eliminates much of the selling point of scale-out storage in the first place. 

The Inefficiency of Scale-Out Performance

Another claim of scale-out storage vendors is that as customers add nodes to the storage cluster, performance increases. However, because of the storage software’s inefficiency, it distributes mediocre performance to the new nodes. In most cases, a scale-out storage solution is always limited to a single given node’s performance. The storage software distributes data across all the nodes in the cluster. Delivery of data to an application, however, is almost always through a single node. 

Dissecting Linear Performance

A typical scale-out storage vendor claim is that they can deliver linear scaling and that if three nodes can deliver 150,000 IOPS, then six nodes can deliver 300,000 IOPS, and 12 nodes can deliver 1.2 Million IOPS. None of that is accurate. The truth is that the scale-out storage solution in that example can only provide 50,000 IOPS per node. In a 12 node system, it is delivering 50,000 IOPS 12 times. It is NOT delivering 1.2 Million IOPS. To experience 1.2 Million IOPS, the customer would need twelve workloads simultaneously requiring 50,000 IOPS for a sustained period. 

Inefficiency of Scale-Out Storage

The discussion on IOPS also does not address latency, which is a much greater concern for many customers. When dealing with flash media, and especially with Optane, everything in the infrastructure impacts latency. The scale-out infrastructure, because of its dependence on networking and massive internode communications, tends to have much higher latency levels than a scale-up infrastructure. 

The Inefficiency of Scale-Out per Drive Performance

In addition to modern SSD and HDD density, the performance of flash, on a per drive basis, is also an area of concern for scale-out storage infrastructures. Again, every enterprise SSD introduced this year is capable of delivering more than 200,000 IOPS, we’ve seen several provide more than 400,000, and Optane is providing these kinds of numbers with 100% write IO workloads. The combination of inefficient storage software plus the nodes’ relatively weak processing capabilities means that most scale-out storage systems can’t deliver more than 50,000 IOPS per node. A few claim 100,000 IOPS per node, but none can provide 600,000 per node like the typical StorONE storage controller. 

To learn more about lowering storage costs by extracting maximum per drive performance, read our white paper “Reducing Storage Costs with Maximum Drive Performance.” 

Solving Scale-Out Inefficiency with StorONE

StorONE is a two-node scale-up storage solution, but we get more performance per node than anyone in the industry. Our software turns mid-range Intel servers into high-performance storage controllers. Our scale-up solution, with Intel Optane drives (four), can deliver over 1.2 million IOPS at .5 milliseconds of latency; six flash SSDs can deliver 600,000 IOPS + at sub-millisecond latencies. The two-node cluster can scale to more than 15PBs of capacity and fully leverage high-density SSDs and HDDs. We can also intermix different drive capacities within the existing RAID configuration, with no need to create new RAID groups, or stand-up entirely new clusters to take advantage of the latest technology. 

How? We re-wrote storage software from the ground up. Scale-out storage vendors add the ability to add nodes to an already inefficient storage engine. StorONE, however, uses new algorithms optimized for the capabilities of modern servers, networking, and storage. One of our storage controllers out-performs six storage nodes from most of our scale-out competitors. 

Bring us Your Scale-Out Workloads

Let’s test it. If you think you need scale-out storage because of your workload, bring it to us. Let’s run it on our solution. We can show you that efficient storage is better than scale-out storage. Fill out the “contact us” form to the right, and we can walk you through the architecture. Participate in a Proof of Value with us, and we’ll send you an iPad regardless of the outcome. 

Contact Us

  • This field is for validation purposes and should be left unchanged.

Learn More

In our on-demand webinar “Showdown Scale-Out vs. Scale-Up Storage,” we compare the two technologies and help you decide which is best for your data center. Register now to listen to the discussion and get your questions answered live. 

Posted in
George Crump

George Crump

George has over 25 years of experience in the storage industry, holding executive sales and engineer positions. Before joining StorONE, he was the founder and lead analyst at Storage Switzerland.

What to Read Next

How Do You KNOW All-Flash Is Right For You?

As an IT planner, every so often, you might wonder how a particular workload might perform on an all-flash array instead of the current hard disk-based or hybrid storage system. The problem is how to know if, or how much of a performance difference, that all-flash array might make. Sure, you can look at vendor performance […]
Read More

Simplified Enterprise Storage from Oracle and StorONE

StorONE and Oracle announced a more efficient, future-ready solution architecture and technology which allows companies to better utilize their IT resources, increase efficiency via application and storage tiering. The cloud computing technology has been powering the IT transformation for more than a decade now. We have seen major change in the way applications adapt to […]
Read More

S1 as a Service – A Better Cloud, On-Premises

We are excited to announce the launch of StorONE Storage as a Service (S1aaS), built on the disruptive StorONE Storage Software Platform.  We developed S1aaS in response to our customers who asked us for a way to get the monthly pricing structure of the cloud, the performance and safety of on-site storage and the ease of management and expansion of the StorONE Storage Software Platform. With S1aaS, you get an enterprise-class storage […]
Read More

Learn More About the Hidden Cost of Dedupe

  • This field is for validation purposes and should be left unchanged.