IT is dead. All hail Systems Engineering!

IT vendors live or die based on their ability to manage complexity on behalf of their users. This is true of all vendors: Software companies, hardware companies, cloud services companies, etc. This is true, really, of all types of systems. Cars are driven by idiots everywhere because they piss me off every single day on the way to work. This is a testament to the evolutionary nature of technology. It used to be difficult to operate a car. In the earliest days, there was an incomprehensible set of levers to fiddle with to get the car to move forward.

A Bleak History

When I was a programmer I remember feeling threatened by COTS software. Business folks were keen on COTS software because that meant they didn’t have to manage programmers. They could focus on their core business. I wondered what value I would have if everyone just bought pre-built software. How could those companies really understand what we need? What if it broke? Would you be able to call a programmer to have it fixed or would you have to wait for months? How do we know its really secure? Then they started looking at COTS hardware. They could buy routers from Cisco instead of having multiple teams of programmers maintain and patch unix systems running routed. Putting Cisco in everywhere meant having a consistent implementation of routing protocols across the WAN. It meant the organization could, again, put more time and money into its core mission.  Thus started the consumerization of IT.

The “pros” of this COTS approach are that businesses thrived, innovation (outside of IT infrastructure) happened, and today we enjoy the internet and everything that has come with it. There has been, however, an insidious downside: the dark rise of IT and the slow death of systems engineering in the enterprise. Companies no longer hired systems engineers. They hired folks who specialized in some vendors products: Microsoft, Cisco, Solaris, Red Hat, etc. People made entire careers out of staying abreast of whatever Cisco did next. Had a problem? Ask Cisco. Buy stuff. Cisco events look a lot like Apple events.  Consumers of technology wait on bated breath for Cisco to tell them what will happen next, and thus what they, the consumers, will do next.  Things are further distorted by cult-like certification programs which have created an unhealthy devotion to consumer IT.  Silos litter the IT landscape like impenetrable skyscrapers.

Educational institutes started teaching MIS/IT instead of Computer Science. Many folks in IT have minimal exposure to programming. They might have written a few lines of Java or Perl.

Then Nicholas Carr happened:  First he said IT doesn’t matter, then he said everything is going to the cloud  (utility computing). Nicholas Carr has been massively influential amongst business folk and decision makers. Due to the growing disconnect between IT and Systems Engineering, IT folks could not justify continuing to fund labs, pre-production environments, or even redundancy. In some cases, even maintenance, licensing, and end-of-support upgrades are no longer funded.  Will you laugh or cry if I mention having proper tools to monitor and manage it all?

In this environment IT folks are left with no room to think about long-term strategies that align with the business’s needs. For a number of years now, IT can only make decisions under the immense operational pressures associated with an infrastructure that is not properly funded and does not have the the staff to properly support it. People have come to accept that this normal. Being good at your job is in direct proportion to your ability to keep this all afloat during your 70 hour work week. Terms like “simplicity” and “complexity” have lost all meaning.  With no Systems Engineering practice, what scale of reference do folks have for understanding simplicity and complexity? The mantra has become “keep it simple” instead of “manage complexity,” and keeping it simple means begrudgingly making changes and never considering alternative ways to solve your problems. Rigidity has set in as network infrastructure has become fragile and therefore risky to change.  Changes take forever.

Like vultures, cloud-providers circle the IT husk.

Three Truths for a Brighter Future

The truth is that the “network” is not a car. A car has to move across the ground faster than the person driving it can walk. The car does not have to fly, or go underwater. It has a clearly defined purpose and accepted limitations. Similarly, a router is not an iPad. That might seem like an obvious statement, but in the context of consumerization of technology it isn’t.   Network is hard.  Today’s traffic patterns and applications will not be tomorrow’s.  The network must do many things and tomorrow it may have to do more things we don’t know about and that we didn’t plan for.  If we only think of the network as a collection of boxes that each pass packets, then how can we know what the true purpose of the network is?  Discovering the limitations of the network is difficult, and sometimes painful, when we only think of it this way.

The truth is that vendors can never hide the complexity of the network in the same way the automobile industry allows idiots to drive fast.  Have the vendors ever hidden this complexity well?  No.  In fact, network engineers have absolutely become masters of complexity. A network, by its very nature, is complex. I would argue that the value proposition of COTS network solutions as mere “packet-forwarders” is questionable.  This is due not only to the failure of vendors to hide complexity, but also to the rising convergence of application, compute, and network.  IT-ism in general has led us down a siloed vendor-driven path that has failed to evolve from merely passing packets to what we really need today.

The truth is that modern infrastructure has one purpose: Deliver application flows according to the needs of the flow within a management and policy framework that holistically monitors and controls the infrastructure from host to host. With this in mind, the infrastructure will deliver real value to businesses again instead of the aging, failing burden it has become. This will precipitate businesses investing in staff and infrastructure again. Hopefully. I choose to be optimistic.


I write this sitting in San Diego at CiscoLive after three days of discussion with Omar Sultan, Dave Ward, Frank Brockners, and Richard Pruss. I watched Padmasree’s keynote. I sat in the audience of an “SDN in real-life” panel with Dave Ward (and out of true love, I threw my underwear at him… no really I did).  Cisco’s ONE announcement potentially represents mainstream acceptance of network programmability.  And Cisco has serious people behind it.

You're welcome Dave! You are the first speaker at CiscoLive to ever have underwear thrown at you.


So what does mainstream acceptance of network programmability and SDN really mean for us, the dedicated pushers of packets? It means the network as a whole is a system. It means network engineers must evolve into Network Systems Engineers. You will be writing applications that interact with the network system.  You will be supporting applications that others have written for the network system.   Its time to manage the  inherent complexity of network systems like grown-ups.

I, for one, am super excited.  Underwear-throwing excited.



  1. Gregpclark says

    The panel today did a great job of discussing both the challenges and benefits of SDN. Do you think Dave will have those framed in his office?

  2. Peter Ferro says

    Looks like Cisco takes another step to becoming more like Microsoft / Apple.

    Will you be forced into buying only a specific vendor’s equipment (OS) to ensure the applications you develop remain compatible?

    I’m waiting to see the Cisco R&S App store. 

  3. Art Fewell says

    Dave Ward is a sharp guy, but what I see with Cisco one looks to me like, for SP’s (Dave Wards espertise) who are currently demanding OF, they will probably put some effort behind it. But for enterprise, a SDK that works across OS’s is nice, but a far cry from what is possible. Is it really possible to do more? That is the same question as was it really possible for thin AP’s to wipe out fat AP’s almost overnight. I See open as a big step in the right direction, but a far cry from what is possible. Enterprises could benefit from OF today … and it looks to me that Cisco is going to milk their current platform for as long as they can before giving that benefit. I remember when you said your face melted when you saw the BSN controller, I assume that was because centralized control plane benefits, where the controller can be a PaaS for network services because it can absorb the complexity of distributed computing development. If you want dynamic application interaction, you need centralized control plane, a user waiting on an app to make a network request cant wait for a NB api that has to turn around and poll 50 routers, calculate a consistent view, calculate distributed instruction sets while considering race conditions etc. Which is why if you compare the apps available for even Arista, they pale in comparison to what individual grad students are popping out for OpenFlow already. 

Leave a Reply

Your email address will not be published. Required fields are marked *