No, Not That Kind of Model!

We’re not selling clothes, after all.

Ivan and I have been tossing emails back and forth about Openflow over the last week (too bad y’all didn’t get to listen in!). He’s really been helping me solidify a lot of things about my thinking in the OpenFlow space, so the conversation has been fun and useful –the best kind.

In the midst of it our exchange, I posited that there are four models for the interaction between control planes and data planes. It seemed worth repeating that part of the thread here, and then putting some text behind it to talk about the concept of models in networking a bit more. So, what are the four models of interaction between the control and data planes in a network?

  1. Proactive control plane with a proactive data plane, where you know the destinations and you install every possible destination (or some aggregate thereof) into the data plane (forwarding hardware) as the reachability information changes.
  2. Proactive control plane with a reactive data plane, where you know the destinations at the control plane, and this information changes in near real time with actual topology changes, but you only install forwarding information in the data plane when you need it, like fast cache.
  3. Reactive control plane with a reactive data plane, where you ask someone else about destinations when you receive the first packet (like LISP and reactive MANET protocols).
  4. Topology only control plane with a reactive data plane, where you simply flood to everyone when you don’t know about the destination, because the control plane really doesn’t know about destinations, only the topology (like spanning tree). Here the only reachability information is in the data plane (forwarding table).

Working in the mobile ad hoc (MANET) space, combined with some serious work in understand the EIGRP/RIB interfaces in Cisco’s IOS, RIB interaction in XR, and other stuff, really solidified my understanding of the separation between the control and data planes, and the interface between the two. This is the sort of thing that might seem like such a minor detail in the bigger scheme of things, but it actually has a huge impact on the design of security and policy, understanding where protocols will break, and in a hundred other little things you encounter day-to-day.

In a more general sense, models are a bit of a hobby horse of mine, a form of abstraction that are often useful (so long as they don’t consume your thinking). When I run into a new protocol, or a new problem, I instantly think of some model that helps me think about the way things work. It’s a bit theoretical, but it’s also a good exercise in abstraction. As a result, we’re including an entire chapter on network models in the new book I’m co-authoring, The Art of Network Architecture.

The seven layer cake is fun, but it’s a bit long in the tooth, doesn’t really “fit” the world of TCP/IP, nor really all that useful for designing networks (as opposed to protocols). So we’re going beyond the OSI model in the chapter and covering the 4 layer (DoD) model, my own mental model of how networks work, the proactive/reactive model (above), Places in the Network (PINS, which I think are useful for relating business to network, but not really useful for network design), hierarchy (both horizontal/aggregation and vertical/virtualization), and an iterative/recursive model I’ve picked up from some reading a friend recommended. I’m covering another model of network operation, offline calculation and on-line reaction, in the chapter on SDNs, as well.

What model would you like to talk about, or have me put in the models chapter of a new architecture book? Did I miss one you’d really like me to talk about?

And yes, I’ll get back to the series on prepping for the CCDE soon…

About Russ White

Russ White is a Network Architect who's currently looking for a new challenge. He's scribbled a basket of books, penned a plethora of patents, written a raft of RFCs, taught a trencher of classes, and done a lot of other stuff you either already know about, or don't really care about. If you want letters, well... BSIT/MSIT (Capella University), CCIE #2635, CCDE 2007:001, CCAr. So there.

  • Ahmed

    Ok Russ, dumb question but can a protocol exhibit different behavior in different scenarios?

    I don’t totally understand the difference between OpenFlow and an ideal always-converged routing protocol….then again I don’t think I *truly* understand the difference between control/data plane in all circumstances.

    If we assume those are the only 4 models of control/data plane interaction, can you map all routing protocol behavior to 1 of those 4 models?

    Because it seems like EIGRP transitioning to a feasible successor sounds like control/data plane interaction #2, but the EIGRP query process sounds like control/data plane #3

    • riw777

      Think of routing as the metadata that describes the network, and the data plane as the actual database used to forward information through the network.

      I’ve often struggled to come up with a real life example, but perhaps –a road map verses road signs. The road map shows you where the roads ultimately lead, what path they take to get there, etc. Road signs let you navigate without the map, but they rely on the map, if that makes sense. So imagine driving from Raleigh to Reston without having a map, just road signs –that’s what the packet does when it’s moving through the network. Somewhat unknown to the driver, there’s a crew that looks at road maps, and puts signs at every intersection for the packets to follow. That’s the routing protocol.

      The road crew is the RIB (actually the FIB, but let’s not quibble to much at the moment), the touchpoint between the routing protocol and the road signs. The RIB is a client of the information provided by the routing protocol, and the forwarding plane is a client of the information provided by the RIB. You can think of it as metadata (routing)/data (forwarding), or a client (forwarding)/server (routing), if you want. Two completely different systems separated by a somewhat opaque API.

      To carry on with the road sign example…

      Some road crew someplace could put up a sign that leads around some specific stretch of highway –a detour. That’s local policy influencing the end-to-end path, either through policy based routing, static routes, or some other mechanism.

      The four interactions above can be seen as the four methods the crew uses to put up the road signs. Do they wait ’til a car comes along, ask the driver where he’s going, and then run around trying to find the destination (there’s no map! –or the map is stored in a central building and they have to go consult it before they put up a new sign)? That’s #3. Do they wait ’til a car comes along, ask the driver where he’s going, consult the map, and then race ahead to put up just the road signs needed? That’s #2. Do they put up road signs with every possible destination listed before the first car comes along? That’s #1.

      #4 isn’t technically routing at all (IMHO). In this case, the road crew is so lazy they don’t bother finding a map or making up signs. They just close down roads ’til there’s only one way to reach any given city. When a driver comes along from the edge, they just send him along to the most central location on the map and record where he came from on a piece of paper. When a driver comes along who’s coming from the central location, they consult the piece of paper to find out where to send him. The piece of paper is rather too small to hold information about every destination, so they have to erase one line to write another one on it. In #4, the control plane is really really dumb, so the data plane does all the heavy lifting –it’s even hard to say there is a “control plane” in this model, really, so the definitions tend to slide away on us.

      The way the routing protocol works to build a topology, or a set of best paths to every destination, is completely separate from the way the forwarding engine (the control plane) consumes the information the routing protocol builds. It’s not one process, it’s two; it’s not one database, it’s two.

      What Ivan and I were discussing is precisely which of these models OpenFlow can use –and the answer turns out to be, “all of the above.” I suppose that shouldn’t be such a surprising answer, because OpenFlow is, after all, just another routing system, like EIGRP, OSPF, IS-IS, and BGP.

      • Ahmed

        Even though they’re 2 databases, isn’t the job of the forwarding to enforce the marching orders the routing protocol? i.e. the routing protocol controls the input information to that whole process.

        So going from Routing Protocol –> RIB/FIB—> Forwarding Plane, could you say that the routing protocol thrives on as much information as possible (gossip junkie) and the Forwarding Plane
        can’t be bothered about anything other than how to package (encapsulate) the data to be shipped to an outbound interface? At some point the rubber has to meet the road and data moved out of an interface so what is the name of the thing that puts the frame on the wire? Since #4 is kind of dumb and annoying….why
        would OpenFlow want to regress to that?

        Looking at it from a lowest-layer up perspective, in your model could you consider:

        - A hub a static data plane with absolutely no pretention of a data plane?

        - A switch having enough of a control plane so that it’s smart enough (after a little eavesdropping) to know not to send it to everybody else
        but dumb enough to get duped easily.

        - The additional intelligence introduced by Spanning-Tree creates a little bit more of a control plane, albeit limiting. This is #4 and the beginning of what can be considered a control-plane.
        So, could we say that Spanning Tree was an attempt to introduce control-plane concepts into switching? Spanning-Tree creates some unique addressing (Bridge ID) but it’s all geared to deactivating links
        so it’s still only mucking w/ the topology.

        • riw777

          To answer the last question first –OpenFlow probably won’t ever stoop to the point of just being spanning tree. At least I can’t imagine any reason for an OpenFlow implementation to imitate STP. Being able to do something and actually doing it are completely different things.

          For the military metaphor –the routing protocol isn’t the General, but rather the Scout (or even spy), moving ahead of the army and mapping out the lay of the land. How would I get to this city? Are there alternate routes? Where are the choke points and problems?

          The Scout (routing protocol) brings back information that’s injected into a local routing protocol database (IS-IS LSDB, BGP AdjRIB, EIGRP topology table). They sit before the general and tell him about what they’ve found (try to install routes in the RIB.

          The General decides which Scout’s information to take for specific destinations based on local policy (eBGP is always chosen over EIGRP, static over eBGP, PBR wins over statics), and builds a routing table from this information. The RIB is then used to issue orders to the FIB, which might, again, ignore some specific information (sorry, this link is down, so I will ignore your decision to send traffic towards this destination that direction).

          I hope that makes more sense. :-)

          • Ahmed

            Yes, I think it makes sense. If I can try to summarize:

            The RIB represents a synthesis of multiple input streams.

            The individual scouts have intelligence assets (neighbors) and maintain their own dossiers (topology tables like ‘sh ip bgp’). Even if their suggestions to the General don’t get accepted they keep them in case real-time changes need to occur.

            To maintain secrecy, the General keeps the individual scouts unaware of each other and he has a pecking order of which scout is the most reliable and from that he generates the orders (RIB).

            Then the General gives general guidelines and mission objectives (‘take that fort!!’) to the field commanders who have some license to achieve the mission however they see fit by taking into consideration conditions on the field (interface status etc.) and produce their own marching orders (FIB, CEF, adjacency table) actually do the work

          • riw777

            Yes –you have it. Very good adding the secrecy between the protocols (spies) that are providing information to the general. :-)

          • Ahmed

            So, could redistribution between protocols be seen as intelligence leaks between different scouts/spies (intel agencies) done outside of the mandate of the General (RIB)?

            I don’t know a lot about OpenFlow, but it seems like all the knobs to control traffic already exist and have been proven to be interoperable. If people wanna go the OpenFlow route, why not just build an

            1. build an API for routing protocols and ‘fake’ information using pseudo-neighbors and pseudo-adjacencies from an OpenFlow controller?

            Or

            2. Its not like we have new algorithms…and since link-state protocols compute metrics from the perspective of the other neighbors it wouldnt be too hard for a powerful server to do that on behalf of the network & just insert that information into the RIB & topology tables simultaneously on each node & nothing has really changed because it was a pseudonode that did all the modifications so you can troubleshoot it with all the existing tools and companies can continue to develop their own CLIs and just start introducing that functionality gradually.

            To save on convergence time, you could send those link-state changes simultaneously across all affected nodes in the area just like it would have seen it if it were coming from the real neighbors and if you lose your OOB connection from your controller you have redundancy b/c those same updates should come from the real neighbors eventually.

            and for the layer 2 stuff just use a similar technique to convince the device that the topology information has changed….like introducing Spanning-Tree Input Events.

            Having gone through all this effort of automating all this complex interconnectivity, why go through the process of dismantling that & then introducing error-prone human intervention?

    • riw777

      I’m going to bump back up to the top because the comment space is getting really narrow… :-)

      “If people wanna go the OpenFlow route, why not just uild an API for routing protocols and ‘fake’ information using pseudo-neighbors and pseudo-adjacencies from an OpenFlow controller?”

      You could, but this is more difficult than what it sounds like — it’s actually easier just to open the API between the RIB and the FIB than it is to either calculate just what needs to be injected into the routing protocol where to influence a particular flow in a particular direction.

      Another point going this direction would miss is that OpenFlow is more like policy routing at it’s deepest than like routing as we know it. In traditional routing, you have just the destination address to work with, but with OpenFlow you can also work with the source address, and all the other tuples on the source side of the flow.

      “Its not like we have new algorithms…and since link-state protocols compute metrics from the perspective of the other neighbors it wouldnt be too hard for a powerful server to do that on behalf of the network & just insert that information into the RIB & topology tables simultaneously on each node & nothing has really changed because it was a pseudonode that did all the modifications so you can troubleshoot it with all the existing tools and companies can continue to develop their own CLIs and just start introducing that functionality gradually.”

      This is one way you could deploy OpenFlow, yes… Except some of the tools you’re used to would be run on the controller, rather than the forwarding devices themselves.

      “Having gone through all this effort of automating all this complex interconnectivity, why go through the process of dismantling that & then introducing error-prone human intervention?”

      You’re always going to have human intervention, at least so long as you have policy. The only way to get away from intervention is to develop a scale free infinite bandwidth network, so you won’t ever care about how packets are routed, or anything like that. :-)

  • Ahmed

    I understand routing protocols don’t operate on data plane, but the way they react seems to force one of those 4 interactions.

    Would a switch with Spanning Tree disabled just be a feedback-enabled data plane?

  • Mdukes

    How about an interesting scenario – a hybrid network consisting of a mix of OpenFlow and non-OpenFlow capable nodes. How does the topology get shared and updated in this case?

    If #3 is the suitable candidate, the topology gets a reactive update on how to get from an OF node to the next non-OF node when the first packet is received.
    Do you see a different model other than #3 being applied here?

    As of now, it looks like a hybrid environment of OpenFlow and non-OpenFlow nodes can only exist within the same L2 domain. Given that several devices support policy based routing, would that be the key to share the information between OF and non-OF nodes irrespective of underlying routing protocols?