Show 135 – Change Control Causes Constipation

Ethan & I talk about recent events in the industry and their workday. I have a head cold so apologies for the sniffles / coughs.


How much does your vendor impact your product perspective ? I wouldn’t have considered Zeus previously but this week Greg is considering Riverbed Stingray. Why is that ?

F5 products are priced beyond the reach of most people. It’s worth looking at other products because F5 isn’t “cloud ready” with their pricing. Stingray has the features that you are looking for.

Cisco Nexus 7000 to complex for serious use

Fear of Code Upgrades – more software instances solves routing problems, improves scaling and makes upgrading a lot easier. Greg is having a lot of pain around upgrading the Nexus 7000 in networks and the lack of guarantees that Cisco (and resellers) can provide.

It can takes an eight hour change window to upgrade a single NX7K and there is little certainty that it will actually work. Worse, it’s becoming more and more risky to do so since features like vPC means that devices are interlocked and “fate sharing”.

Cisco Product Launches This Week

Cisco released a broadside volley of products this week. The Cisco C3850, Nexus 6000, ONE Controller and new features for the Nexus 1000V.

We discuss these releases and the relative features and capabilities.

Cisco announcing service modules for NX7K. But virtual versions at same time, aka vNAM in NX1Kv. You can run NX7K as leaf/spine switch. Uses Nexus 5500/Nexus 2200 code base ? Supports Fabric Path.

Amazon and low margins means no sales people

Amazon’s thin margins preclude the use of sales people & fancy executives to sustain revenue. Instead, Amazon develops new products in adjacent markets ( product managers & engineers ) or adds efficiencies to existing products with better engineering or investment.

In short, sales people add no value except to sustain high margins. Amazon products don’t need explanation, customers buy because they know what they want. It’s possible that competitive sales actually help amazon since the help customers learn about the cloud products.

Greg Ferro
Greg Ferro is a Network Engineer/Architect, mostly focussed on Data Centre, Security Infrastructure, and recently Virtualization. He has over 20 years in IT, in wide range of employers working as a freelance consultant including Finance, Service Providers and Online Companies. He is CCIE#6920 and has a few ideas about the world, but not enough to really count. He is a host on the Packet Pushers Podcast, blogger at and on Twitter @etherealmind and Google Plus.
Greg Ferro
Greg Ferro
Greg Ferro
  • Umair Hoodbhoy

    Looks like F5 listened to this episode and did something to get more ‘cloud ready’. Today they announced that they bought LineRate Systems

    • Etherealmind

      Doubtful. Linerate is designed to take advantage of hardware to get it’s performance. They didn’t make a big deal out of their multi-tenancy.

      Either way, Riverbed Stingray is ready today so I’m going with that.

  • Derick Winkworth

    F5 not cloud ready? They have an extensive API and they support 1000 route-tables no?

  • Stefan Mititelu

    Funny you should mention Nexus 7Ks and upgrades. After a few major outages caused by what many folks call “code immaturity”, we actually had to engage the Cisco Optimization Services (extra $$$), for the vendor to actually recommend something that fixes … well … their own product original problems. Buying the product and paying for SMARTnet/support from Cisco does not guarantee support delivery to fix network stability caused by buggy versions of code.

  • Jerold Swan

    Great points on the NAM. I really like having the NAM modules available and have used them in both Cat6ks and ISRs, but paying that price (along with the ridiculous SmartNet price on them) is very hard to justify. I’ve gone back to using SPAN to external appliances for that reason.

  • Jerold Swan

    More on the NAM: I was once involved in writing some customer requirements/wishlist stuff in order to justify the development of some very cool internal network monitoring features in IOS, and was told by the Cisco contact to be careful not to phrase things so as to make it sound like we would be taking revenue away from the NAM product line. :-)

  • Robert Harper

    Our ‘non-disruptive’ code upgrades have caused little to no problems. They do take quite a bit of time. Our pair of 7K Nexii are properly vPc’d and nearly all of our host’s switchports are attached to 2k FEXes. Our FEXes hang off the at least two different 10G modules on a N7K switch (reference “Cisco Nexus 7000/2000 At-a-Glance” (fig 3)). This way the EPLD upgrades do not affect the hosts. I have not been able to detect any packet loss across the data center during OS/EPLD upgrades. In summary, I believe the Nexus 7000s to be a great asset and the complexity and robustness of the switch’s design are part of the appeal for me.

    • Ethan Banks

      To speak to Greg’s rant on this point, IIRC, I believe his concern was that Cisco would not “guarantee” in any meaningful sense that the upgrade process would be non-disruptive. As such, the internal processes of the corporate setting he was in wasn’t comfortable moving ahead with the upgrade.

      • Etherealmind

        As Ethan points out, it was the lack of certainty and perceived quality of Cisco products (especially the reseller) that was the problem. While I had a high level of confidence that the product probably COULD be upgraded smoothly, the reseller did not have the skills or the confidence to deliver. No one from the reseller would vouch for their work, and no one from out company felt they had the skills to correctly administrate the highly complex configuration of NX-OS with vPC & FEX. It’s often beyond the competency of most network engineers.

        Cisco would not abandon the reseller, nor would they guarantee the upgrade process. Cisco refuses to engage AS or PS to come and perform the upgrade.

        Which left us screwed over by the reseller AND by Cisco.

  • Duaneo

    I hope I don’t burst any bubbles here, but we had a team of folks from amazon that smelled a lot like sales people come in to talk to us about putting some of our infrastructure into the cloud. I’m trying to find the meeting invite and presentation to see if I could find any titles in it. So far, no luck.

  • Brett

    Greg, you mentioned in this pod cast about discussing security in a virtualised data center network. I’d very much like to hear you thoughts on the topic as its a pain point for me and has been for some time. It’s hard to have an open discussion about without it turning into vendor wars or religious… and I’ve never found any good articles or papers which don’t try and sell more vendor x to fix the problem.

    Any chance it can be a future topic for a pod cast?

  • pzpacket

    I have to say this was one of my more favorite podcasts. A great reminder of the pitfalls of a proprietary redundancy scheme. It has its use but when availability and predictable change control is king, hours upon hours can be burned preparing for the kind of change you are headed into.

    I’ve been bitten on early version Juniper SRX “ISSU” and EX virtual-chassis in similar situations. They both have matured greatly, but I’ll still take an independent set of each with autonomous protocols to manage code upgrades more easily. This also means NOT clustering a pair of firewalls, for example (especially if they’re already dual-power) just so I can move traffic reliably, with a known impact, when the time comes.

    Several of my demanders (internal customers) are trying to push more consolidation of devices / more clustering, and your present issue reminds me why not to give in… it’ll be us killing a weekend to handle the proprietary issues.