Why Network Engineers Are Sick Of SDN – And What Vendors Can Do About It

2012’s dominant networking buzzword has been SDN: software defined networking. Packet Pushers has talked about SDN quite a bit because it represents a re-thinking of the way that the industry has built networks for the last two decades. SDN as a concept is both technically interesting and creatively inspiring, giving many pause for thought and cogitation along the lines of “what if.”

However, far more network engineers are sick of hearing about SDN than are interested in it. Rather than thinking “what if”, they are thinking “who cares?” The “who cares about SDN” attitude has been related to this Packet Pusher from all aspects of our social media community. Tweets, e-mails, blog comments, IRC chatter, and forum posts have seen a recurring theme of “shut up about SDN already and let’s talk about something that really matters.” In fairness, I empathize. I feel invective thoughts about SDN rising up in my own mind from time to time, and I’m a believer in the SDN idea.

Why are network engineers sick of hearing about SDN?

Engineers have a number of problems running their networks today. From a practical standpoint, these problems are not usually architectural ones. Which isn’t to say that there aren’t architectural challenges today – certainly there are, not the least of which is how to lessen spanning-tree’s impact on data center design. But in a broad sense, the model of fat-tree networks with predictable oversubscription still works in many scenarios, and the majority of issues with this approach can be ameliorated with MLAG.

Instead, I argue that the problem most engineers have is one of efficiency. Managing sprawling network topologies, a problem even in small-to-mid enterprises, is every engineer’s pain in the backside. This is true in at least two significant ways.

  • One is in provisioning, or should I say “orchestration”. Even simple switchport provisioning is a tedious, error-prone process for most networks that can be difficult to delegate. Move up in complexity to something like HTTP server load-balancing with SSL offload, and the tedium and error-proneness only increases.
  • The other pain point is in troubleshooting, where discovering and resolving a failure in an end-to-end path is time-consuming, requiring intimate knowledge of the environment. Security policies, routing protocols, private vs. public transport, disaster recovery, and interaction with third parties make for any number of details that must be well-understood during a network failure to restore service.

If SDN solutions available today were addresses these issues – provisioning/orchestration or reducing the time it takes to resolve a failure – network engineers of the world might be more keen to investigate. Instead, SDN for the most part represents potential that network engineers must leverage by themselves. Network engineers don’t have time for this. Network engineers are insanely busy individuals, usually with too many projects to manage and not enough staff to work with them. There is a dearth of network engineering talent in many job markets, certainly in the United States. Network engineers are leaned upon heavily to execute projects for purposes of cost reduction or business enablement, frequently in compressed timelines that don’t allow for taking any approach other than the trusted, proven one.

Vendors offering APIs to network engineers to help them solve their issues is akin to a an auto manufacturer handing a pile of metal and a welding torch to someone who needs a new car. While there are some shops that will learn how to weld, most would rather keep fixing the old car. Vendors need to deliver a new car, or SDN will remain the corner-case problem-solver it is today, confined to academia and the Googles of the world that can afford to build their own in-house auto-manufacturing plant.

Another adoption challenge SDN faces is that of mixed vendor approaches. There is no standard way to deliver a software defined network, and at this point there is little in the way of standard network abstraction models that are going to get us there. Yes, there is NETCONF. And yes, IRS (recently renamed I2RS) is in the early draft stages. But is the networking industry anywhere near being able to cobble together a vendor-agnostic network and manage it in a predictable, software defined way? Goodness, no. Vendors major on their differentiators; they don’t minor on them. And at this stage of the SDN game, most vendors are telling the market what they want the market to need, using specially crafted marketing messages that play to their own strengths. The fact that the SDN messages have been mixed hasn’t been lost on network engineers, and so the response has been variations on a theme of “meh“.

What’s a vendor to do?

The vendors that will lead the SDN revolution are the ones that deliver an integrated set of tools that make it easy for a network engineer to manage his network. The software defined network of the future won’t merely be vendor operating systems with APIs. Instead, it will be a mix of hardware, software, and controllers that make it possible to eliminate routine tasks through orchestration, deploy forwarding and security policies globally, and pinpoint problems quickly. Will those functions leverage APIs? Very probably, but a network engineer won’t be forced to sort that out.

I’ve heard vendors say repeatedly that they don’t know what a network operator is going to need to do, and so they’d rather offer tools than solutions. Some vendors believe the best approach is to hand off APIs, under the assumption that the network engineers know what they need and (apparently) have time to learn a programming language. The reality is that network engineers don’t have that kind of time, other than perhaps a quick script here or there.

For the majority of networks, it’s incumbent upon the vendor to put together the holistic network management tools that will allow the network engineer to manage the network, and not individual devices. Demonstrate that solution, and more network engineers will be interested. Why? Because that approach might actually solve problems, and not simply give network engineers more work to do.

Ethan Banks
Ethan Banks, CCIE #20655, has been managing networks for higher ed, government, financials and high tech since 1995. Ethan co-hosts the Packet Pushers Podcast, which has seen over 2M downloads and reaches over 10K listeners. With whatever time is left, Ethan writes for fun & profit, studies for certifications, and enjoys science fiction. @ecbanks
Ethan Banks
Ethan Banks
  • Mike Dvorkin

    Awesome writeup. Automation, visibility/monitoring and better fault management and debug. SDN was a chance to reinvent the nms, and this time, solve it well. Instead, we’re keep on adding complexity.

    • http://twitter.com/EricVoit Eric Voit

      Complexity for who is a relevant question. Some Network Engineers will be able to write applications to invoke key interfaces. The majority of Network Engineers are unlikely to attempt that jump. For SDN mainstreaming, look for vendors to fill niches that simplify specific virtualized topologies. Things could then expand/generalize from there. What is an NMS will certainly blur over time.

      • http://twitter.com/cloudtoad Derick Winkworth

        There is no scale of reference for complexity anymore. We need to bring back the “CS” that built “IT.” Network Engineers are in a unique position to effect real change because the network is the nervous system of the IT organism. Vendors should stop trying to convince consumers that they can continue to get by without addressing the inherent complexity of IT.

  • http://twitter.com/MrsYisWhy Mrs. Y.

    Thanks for this. And I would throw firewalls into that complaint as well, because you can’t even leverage a decent API with most. I’m tired of vendor promises of “what could be.” When will they start delivering on solutions to make my life easier NOW.

  • http://twitter.com/shivlu shivlu jain

    Orchestration is the only issue….The same applies when we talk about cloud too…

  • microagent

    Great write up identifying the current challenges and solutions!

  • Sam Stickland

    In the predominantly Cisco-land I inhabit our network ops have a hard time just trying to find software releases that can implement _existing_ functionality without bugs. There’s very much a belief that for something as complex as SDN, any implementation will simply be buggy beyond the point of usability and therefore not worth considering.

    • http://packetpushers.net/author/ecbanks Ethan Banks

      I agree. Cisco has an uphill battle to gain wide market usage of onePK for this reason. Impressive in ambition, onePK does beg the “bug” question.

      • Allen Baylis

        If you listen to David Wards interview ( w/Packetpusher) regarding OnePK, you’ll be left wondering …”how on earth will it ever work” . We’re talking extensive programming project … sounds impossible …

      • OmarSultan

        Ethan:

        First off, great write-up.

        from our perspective, there are a couple of things to bear in mind. First there is a definite split between the SP and enterprise on interest and adoption of SDN technologies. If you think about it, this makes sense as SPs run their IT as a profit center and can invest appropriately and reap the benefits to make SDN worthwhile. The second thing is that the vast majority of customers we talk to, especially on the enterprise side, are looking for incremental capabilities that extend what is already working, not looking to re-invent the wheel. I think a lot of early adoption will be the kinds of things that Mike points out earlier in the comments–automation, monitoring, visibility (Greg’s VLAN provisioning tool). First, it addresses unmet needs, second, it allows folks to cut their teeth and gain confidence to tackle more complicated things.

        As far as “bugs” go, we will continue to support customers that have been playing around with onePK like any other customer. You can happily create config bugs today with the CLI that will take your network down, so in many ways, nothing is really changing.

        Regards,

        Omar (@OmarSultan)
        Cisco

  • http://twitter.com/cloudtoad Derick Winkworth

    Ethan.. you’re rocking it in the last few articles… This is awesome right here.

  • http://twitter.com/metaip Allen Baylis

    Awesome write up !

  • http://twitter.com/sjiveson What Lies Beneath

    I’ve commented on your blog regarding the CLI but another point; when you break your SDN network and your controllers can’t talk to anything. How are you gonna fix it?

    • Wes Felter

      Clearly the network needs to be built in such a way that it can never break that badly. The exact details depend on details such as in-band vs. OOB control, etc.

      • Nikolay Milovanov

        Centralized Controllers and OOB actually are against the initial principles of distributed packet based networks e.g the one defined by Paul Baran.

        Baran’s goal was to create a network able to sustain a nuclear war.

        I guess the current situation is quite different. Everybody is looking for cost, performance and extensibility. In my opinion those are the drivers of SDN. Reliability is somehow a bit out of the focus.

        Isn’t it so?

  • Nimit

    The biggest problem of network is no visibility on distributed systems.
    SDN solves that first problem, when you can see it, you can solve it.
    NE are busy individuals, agreed, because half of their time goes to work out what is running in the network, the design etc.
    Is SDN not helping NE to fix this ?
    Ofcourse network programming bit is complex, but leave that to vendor APIs.

    Are we sick, because of our own insecurity ??
    That one day SDN will eat all our jobs ?

    • http://twitter.com/cloudtoad Derick Winkworth

      SDN is not required to fix that problem, though it is especially suited to do so.

  • samkramos

    Great article, Ethan. Do you think that the Nexus switches are gradually moving towards SDN? The centralized administration of the 2k’s and the capabilities of the 1kv seem to be pointing in that direction. It seems like enterprise networks will still have a long time before anything like this starts seeping in, just curious as to what your thoughts may be.

  • Alternate Opinion

    I believe network engineers have missed the point. Though SDN has been marketed as a Network technology, network virtualization is a computing enabler. The server admin cares about one thing, and that is for host A talks to host B, instantly, regardless where it exist in the network.

    Understandably, Network Admins do not get it, and won’t. So let’s just stop asking to…

  • Will Dennis

    Great article, Ethan… A guy down in the trenches (like me) would initially think, “well, what does this stuff have to do with me?” But in reading/listening a bunch about it (in no small part due to PPP) I have become interested in it, for it seems like just too good of an idea to discard it.

    I think SDN will be a great enabler for the virtualized network, in a scenario like multi-tenant/multi-site cloud, where APIs will dynamically construct and be used to manage the overlay network. Working at a company that has products in the SDN/OpenFlow space, I also see where people are using it to develop monitoring and incident response types of applications for physical (underlay) networks. It may not be what we use in the short term to deploy and manage our current network infrastructure, but I think we’d all do well to keep our eye on it…

    As far as vendors giving us a better way to holistically provision and manage our networks without having to SSH into every box… um, hell yeah. Bet that will take some software and hardware upgrades ($$$) to achieve, though. Not betting on it to be a cross-vendor solution, however, for the reasons you mention above.

  • netstat man

    I enjoyed this article. I am a believer, I drink the SDN coolaid, but like others have stated I don’t think the integration of servers admins, network admins, and programmers will be an easy one.

7ads6x98y