DriveScale is a well-capitalized startup offering scale-out storage. Scale-out storage isn’t new, so what is DriveScale bringing to the party? Flexibility.
DriveScale’s idea is to disaggregate physical disk and server, while avoiding the typical scaling limitations and expense of traditional SAN and NAS products. The disaggregation allows the storage to grow at rack-scale alongside of compute resources. At the same time, DriveScale allows data center operators to replace their compute or storage at will, without requiring the replacement of storage and compute as an intertwined unit.
DriveScale architecture components.
A DriveScale storage solution consists of software, a datapath server, an Ethernet switch, and JBODs.
Software. DriveScale software is the storage management layer. This is licensed based on a per-node and per-disk drive per year scheme. Effectively, this means licensing is based on the number of disaggregated components.
Datapath server. This is the only hardware “magic” in the architecture stack. The datapath server is a 1U box containing four independent adapter cards, each of which translates Ethernet to SAS and back at 20Gbps. And really, the server isn’t that magical. DriveScale is proud of the fact that this server is, in their words, “Dumb as a rock.” All the box does is translate Ethernet to SAS. There’s no RAID, volumes, or LUNs. The box is strictly a datapath.
The datapath server lists for about $6K, and is managed by Redfish as opposed to the slowly aging IPMI.
Ethernet switch. DriveScale expects that most customers will have excess capacity in their ToR switches. DriveScale assumes a ToR oversubscription ratio of 4:1, which, to their mind, leaves a good bit of bandwidth left over inside the ToR to carry storage traffic.
By that logic, a ToR switch with 32x10GbE (320Gbps) server-facing ports and 2x40Gbps (80Gbps) spine-facing ports should have about 240Gbps left over for storage traffic. We know the math isn’t quite so simple, as this oversimplified logic doesn’t take into account server-to-server traffic staying within the rack. Nevertheless, the point is valid that a typical non-blocking ToR switch is, for most operators, going to have some leftover bandwidth to allocate to a storage budget.
Therefore, a separate switch to carry storage traffic is not required. DriveScale doesn’t even insist on QoS. They ask for a non-blocking Ethernet switch with LLDP support that can forward IP traffic inside of jumbo frames. And that’s it.
The Ethernet switch is, interestingly enough, a critical element to the DriveScale solution. DriveScale chose Ethernet as the interconnect between server and disk because it’s cheap and the bandwidth is plentiful. Most of their customers are using 10Gbps connectivity today. 25Gbps is on the rise. DriveScale did not announce 25GbE connectivity for their datapath server, but did point out that switches supporting 25GbE typically also support 10GbE in the same port.
JBODs. The customer will provide these, using a drive vendor of their choice. The intent here is to save money with large-capacity spinning disk, assuming a typical Hadoop deployment scenario. Like the Ethernet switch, the JBOD array is supplied by the customer. DriveScale is not overly choosy about what drives the customer selects.
DriveScale deployment.
A DriveScale deployment is done at the rack level. In other words, servers running distributed computing workloads like Hadoop or Apache Spark live in the same rack with DriveScale datapath servers and JBODs. These are all interconnected at 10Gbps via one or two ToR switches. Servers within the rack access disk within the same rack.
Latency of storage calls between server and disk across the storage fabric is minimal. Although some might fear storage traffic having to traverse “the network” (the horror), DriveScale looks at Ethernet like any other bus. There are pros and cons to Ethernet vs. RDMA, Infiniband, and PCIe, but today’s Ethernet is cheap, mature, and fast. In addition, commodity network switches have very low port-to-port latency, typically measured in the microseconds or nanoseconds.
Beyond the simple capabilities of Ethernet, DriveScale transparently handles multipathing via its own load-balancing algorithms that ensure links are loaded evenly. In a DriveScale setup, there are four controllers — four paths between any server and the adapter. In addition, there are two paths between any adapter and the JBODs.
If you’re wondering just how they do the load-balancing between server and disk…pssst…it’s iSCSI. But you don’t have to concern yourself with that. DriveScale does all of the iSCSI connectivity for you, even working around iSCSI issues they know about in certain versions of the Linux kernel. You’ll never have to worry about maintaining iSCSI, which is a good thing. DriveScale can address individual disks, so the number of iSCSI paths is potentially mind-boggling.
That handles load-balancing traffic between the datapath server and the disk. But DriveScale also load-balances the bandwidth coming out of the storage servers to the compute servers with their own software layer, recommending 2x10GbE links as a minimum.
The view from the hot aisle.
DriveScale is taking the pain out of the painful aspects of scale-out storage. By decoupling server and storage while still allowing for true scale-out, operators have an easier time of both server and storage lifecycle management.
The pricing model and ROI suggested by DriveScale also seems plausible, although YMMV. DriveScale suggests that your ability to independently choose servers and disk should be enough of a financial win to break-even buying their product. But then when it comes to lifecycle management — replacing processors — you’ll save more because you’re buying a box without direct attached storage. That makes it easier to take the old compute out and put the new compute in than it used to be.
I’m always skeptical when hearing, “Our solution pays for itself!” Yet, at the scale we’re talking for large data center environments, it seems possible for DriveScale economics to work out in this way, assuming the refresh cycles are frequent enough.
But why focus overly much on ROI? The flexibility in storage management DriveScale offers makes the product worth some sort of IT investment. No worthwhile technology is ever truly free.
I believe DriveScale is in for the long haul. They are, by their own account, well-funded at present with plenty of runway. The leaders are established technologists with impressive track records of success spanning decades. DriveScale didn’t create a pile of unproven, proprietary technology, instead relying mostly on the most mature, boring standards that exist. They are even building reference architectures for Dell, HPE, Cisco, and SuperMicro.
By and large, DriveScale storage should “just work.” I believe DriveScale should be on your scale-out storage evaluation list.
For more information.
DriveScale presents at Tech Field Day.
