What Are SAI And Switchdev And Why Do We Need Them To Succeed?

Howdy.

In my last post I discussed the need for an open source framework to drive merchant switching silicon. Towards the end of that long post, I mentioned a future post talking about the Switch Abstraction Interface (SAI) and switchdev in depth.

Welcome to that post. There’s been a lot of synergy between both projects, and the only way to do them right is to dedicate a post entirely to them–as always, without the BS.

What Are They And Why Do We Care?

SAI and switchdev are hardware abstraction models for switching silicon (ASICs). They are the open source frameworks that allow ASICs to be represented in software. This means you can use a Broadcom ASIC the same way as one from Mellanox or Cavium XPliant. These frameworks let developers target switching platforms in an agnostic way; as long as you have the necessary ASIC driver, you’re good to go. These two frameworks differ in where they locate the abstraction: SAI in user space, switchdev in kernel space.

SAI

Some History

In the summer of 2014 at an OCP Engineering Workshop at UNH-IOL, Microsoft and Dell both presented work around the idea of a user-space switching application; Microsoft through the Azure Cloud Switch (ACS) presentation and Dell through the switching abstraction.

A few months later, we held the first SAI developers’ meeting at Dell, to a packed room, about the upcoming versions of SAI (0.9.0 and 0.9.1).  Then, during the 2015 OCP Summit, the team demonstrated SAI with a 4-node CLOS setup with various implementations from Dell, Microsoft, Broadcom, and Mellanox.

Over the summer, SAI was officially accepted as an OCP project, followed by an announcement from Microsoft regarding ACS being Debian Jessie + SAI + everything else needed to power Azure (applications like Quagga, and the switch state service based on Redis).

That’s right, it took Microsoft less than a year with about 10 FTEs to build a NOS that supports multiple ASICs. Unheard of at the time.

Just recently, Dell announced its new NOS, OS10 (Debian Jessie + SAI). Expect more NOSes to be announced being SAI-based later this year. (There’s been some talk about HPE’s OpenSwitch making the jump from OpenNSL to SAI. But that might be a going through hell and half of Texas exercise.)

The Grit

As I mentioned before, SAI is a user space abstraction model; this means a user space application is driving the ASIC, bypassing the kernel. This also means that a hardware cache representation (switch state service using ACS parlance) is done in user space as well.  Typically this is a key-value store like Redis (ACS), OVSDB (OpenSwitch), or SysDB (Arista EOS). Using the user space abstraction also requires applications to be either re-written or modified to support communication with the switch state service.

For example (using the OVS model compared with traditional Linux):
ovs-vsctl list-ifaces vs ip link
ovs-vsctl add-br br0 vs brctl addbr br0

One advantage of doing everything in user space is the developer/vendor can take a stock Linux distribution and add their necessary changes/applications with gusto. No Linux kernel expertise required. No need to make sure things are upstreamed. You just need the SAI driver for your ASIC.  Hell, you can move hardware as well (Broadcom to Mellanox) by just changing the SAI driver. (Yes this is a glib statement; I’ve skipped other details like making sure the rest of the hardware, such as fans, PSUs, optics, and such, work in your base OS.)

switchdev

Some History

Over the last decade or so, ASIC vendors have supported Linux (the best way to think about this: Linux is the firmware that boots up a switching application). But eventually, some of them got tired of supporting a userspace SDK application along with a kernel driver.

As such, the first, basic in-kernel switching/offload layer support was introduced in 3.19.  The first switch-device is rocker, an emulated switch chip implemented in qemu. Switchdev was born. In July 2015, after many patches and enhancements, the first real switch-device showed up: mlxsw, a driver for Mellanox’s Switch X-2 chip. Mellanox followed up with support for their 100GB Spectrum chip in November.

The Grit

Switchdev is a in-kernel abstraction model; the switch state is kept inside the kernel (vs. an application).  The best part of this approach is that existing Linux applications just work: no need for new tools or rewrites of existing ones.

What switchdev does require is the ASIC vendors to upstream their drivers (ideally, no one likes managing kernel drivers out of tree) and requires NOS vendors/developers to possibly be on a different kernel than the stock distribution (Debian Jessie is 3.16 vs the latest 4.4 with all the latest switchdev love).

Side By Side

Here’s a brief comparison of the frameworks.

SAI Vs. switchdev: Who Wins?

In the end, both developers and end users win with both projects going forward. This isn’t my way of rounding up everyone around a camp fire singing kumbaya. These are the real winners of having an open source framework to drive merchant silicon regardless of whether the abstraction lives in-kernel or in user space.

As a developer, imagine not having to deal with a cumbersome SDK but a well-known, well-documented abstraction along with your choice of where that abstraction lives.  As an end user, imagine being able to use the NOS you want on the hardware you want without having to wait for a vendor to retool for a different ASIC as long as the drivers are available from the ASIC vendors.

Do you need a special version of Windows to run on Intel vs. AMD CPUs? No, they conform to a well-defined interface and as such, you are free to choose which CPU you want to power your computer.  That is what is in store for end users with open networking and the disaggregation approach that comes with it.

+–+

Carlos