Uses of MPLS in the Enterprise and Data Center

15 November 2012 by 7 Comments

MPLS in the Enterprise and Data Center has been a topic of some debate from some of the leading independent bloggers out there over the past couple of years. Network virtualization is beginning to be used a bit loosely these days as folks try and define how SDN allows for slicing of Data Center and campus networks. Multi Protocol Label Switching (MPLS) is a core tool for virtualization at scale that is real today, and not promises of SDN x86 hardware bathed in unicorn tears of the future.

Some Important Virtualization Concepts

Virtual Lans (VLAN) – Taking one physical network and carving out separate logical networks into an isolated broadcast domain. It is often safe to think of a network address belonging to a single network like 192.168.1.0/24 (though multiple can exist). That network maintains Layer 2 path isolation throughout the network by having Vlan tags imposed into the Ethernet header. That Vlan ID (VID) is logically isolated by all adjacent switches in the Layer 2 path with the VID defined.

Virtual Route Forwarding (VRF) – Is the ability to have multiple containers of routing tables or Forwarding Information Bases (FIB) inside one Router or Switch. These VRFs operate without knowledge of one another unless for example they are imported or exported into one another. VRFs can be deployed without using MPLS to pass the VRF information over BGP but it can be problematic to operate at large scale since individual IGP mappings are required for each VRF.

Multi Protocol Label Switching (MPLS) – Is a concept whose heritage came from Cisco with the concept of Tag switching that later standardized through the IETF and as it stands today is with BGP,MPLS,VPNs in RFC4364. Simply put, I tend to explain that MPLS is merely an encapsulation that can carry multiple tags inner and outer inside its encapsulation that can carry the destination egress PE (far end of the FEC) and one tag for carrying the VRF information through the MPLS signaled network.

MPLS in the Enterprise

This concept is not new. MPLS/BGP/VPNs has long been a “carrier technology” used for isolating customer traffic A from customer traffic B. As typical hardware in the distribution layers of campus networks such as Cisco 6500, Brocade MLX and Juniper MX switches have had or added MPLS support more have begun adopting the strategy. There are also smaller sized boxes that can act as PE nodes as more vendors understand the importance of path isolation at scale.

mpls policy

Figure 1. Path Isolation funneled to Policy application points.

The core impact of MPLS/VPNs in an enterprise is virtualization at scale. While we once used a VIDs for things that needed to be isolated from one another and sprawled it across the network, we now take lots of VIDs and drop them into containers of VPNs and route them rather than bridge them. To quote my friend Ivan Pepelnjak “The Internet is not make up of Brouters”.

MPLS in the Data Center

MPLS in the Data Center can be used for the same concept as it would be in the campus, path isolation. As the needs of security architects continues to grow, so does the amount of path isolation. We can easily accomplish this in small data centers with Vlans. The problems is every time we extend a “failure domain” e.g. Vlan we also increase the risk associated with unicast flooding, broadcast flooding and all of the other inherent scale problems of Vlan sprawl.

dc mpls

Figure 2. East-West traffic between data centers or failure zones is fine if policy does not need to be applied. Vlans are reasonable.

mpls-north-south

Figure 3. If you need to scale policy application between multiple data centers and apply policy you begin to burn down policy bandwidth in load-balancers, Firewalls and Security monitoring devices if you are inspecting traffic within the same security zone at the front door of each of your data centers or availability/failure zones.

Pros/Cons of MPLS

Pros

  • Faster provisioning of new tenancy or policy domains.
  • No longer needing to really on a less scalable .
  • Overlapping RFC 1918 addresses.
  • I argue much easier operationally to manage post deployment.

Reduced operational complexity as compared to private Vlans, Policy Based Routing (PBR) or Contextual slicing of hardware like Juniper logical systems or Virtual Device Contexts (VDC) in Juniper chassis.

Cons

  • Support from vendors in hardware is often limited.
  • Migration is far from a small undertaking.
  • Different and arguably higher skill set needed for implementation.

Two Men Enter One Man Leaves!

Derick Winkorth and Greg Ferro are getting ready to duke it out over this topic with Ethan Banks playing Switzerland as the referee so tune in!

btDomefight3

Mel Gibson and Greg are both Australian… I’m worried about Ethan’s safety.

So I gotta ask…who owns Bartertown?

 

Filed Under: Blogs, Data Center
About Brent Salisbury

Brent Salisbury is a Network Architect at the University of Kentucky. He is a CCIE #11972. He blogs at networkstatic.net and can be reached at @networkstatic.

  • Joe Cozzupoli

    I’ve done a MPLS VPN and Core network design for a large financial here in Australia a few years back. Its running IS-IS (not very commonly used here at all in Aus) as the core IGP, with multiple VPNs using export-maps to leak routes between these VPNs. Also have many shared services (such as TelePresence, Voice, etc), integration of a dual-hub DMVPN network, and Third-Party services too. Next we will be doing Extranet mVPN (which should be a whole lot of *fun*).

  • http://twitter.com/networkstatic Brent Salisbury

    Hi Joe,

    Finance, Healthcare and anything needing stringent security and regulatory policy is a great example of path isolation needs. I am a DMVPN dummy so hats off to you for knocking that out. Pushing PE’s to remote sites is a nice option also especially if needing isolation their also, beats PBR.

    Only problems I have encountered was LEC to LEC handoffs and PMTUD issues. ATT for a while would offer a max of 1536 or there abouts if I remember correctly which after a few encaps gets to be close to the ceiling.

    My condolences on mVPN and RPF to multiple customers :( Will say MDT is pretty easy peazy for intra-domain. Vendor HW/Firmware interop between of bastardized draft-rosen 6, 7, 8 etc is about enough to pull the fire alarm and go home.

    I banged out the post in about 30 before the death match, so oodles of stuff missing but path isolation at scale is the core message I hope.

    Thanks for sharing, good stuff.

  • ISPking

    Under Pros Cons of MPLS, it says “Reduced operational complexity as compared to private Vlans, Policy
    Based Routing (PBR) or Contextual slicing of hardware like Juniper
    logical systems or Virtual Device Contexts (VDC) in Juniper chassis.” That should say “or Virtual Device Contexts (VDC) in Cisco Nexus chassis.”

    • http://twitter.com/networkstatic Brent Salisbury

      Thanks will post an edit.

  • http://twitter.com/shivlu shivlu jain

    Nice post Brent…Few points I would like to add…When I say MPLS in enterprises, the more is about vrf lite as LDP is not required. The main advantage is to provide the guest access in vrf which make most of the network safe from rogues.

  • http://twitter.com/networkstatic Brent Salisbury

    Hi Shivlu, great point. My beef with VRFs-lite is scale. Im not sure I see much differentiation in care and feeding of VRF-Lite over PBR in many scenarios. To provision new services you are building an overlay IGP to everywhere it needs to be extended to. Lets say a mid size enterprise campus with ~50 PE nodes (L3 points) that may be 150 IGP adjacencies. If you had 4 VRF for something like PCI, VOIP, Data and guest you now have 150*4 = 600 adjacencies to manage. Then the next service that security is convinced needs to be isolated you go build N+1 another 150 adjacencies to provision that service.

    The dynamic nature of MPLS/BGP/VPNs allows for wherever you are to dip your straw into the drink anywhere and catch your service with hardly any provisioning other than FWDing an interface into the VRF.

    Certainly not being presumptuous enough to explain that to you, as I am a big fan of your blog. Only to folks who may read this for some context.

    For anyone who hasn’t before take a look mplsvpn.info for Shivlu’s large archive of writing on MPLS.

    Thanks,
    -Brent

  • Allen Baylis

    Hey Brent ,

    great write up ! Think it maybe approaching complex for enterprise. Would you typically deploy this scenario knowing that is maybe simplified within the next several years. I’m not suggesting that your client would require or consider a forklift but technological advances are changing rapidly and I think its moving away from MPLS.