Why Compatibility And Support Don’t Justify $1000 Optics

Howdy.

By now I hope y’all realize I don’t pull any punches and no subject is off-limits. Having said that, I want be upfront and say this subject is very near and dear to my heart and is one of my main areas of work for almost two years. I’ll do my best to be as neutral and objective as possible. If any section hits a nerve, I’ll pretext it as so. Deal?

Economics 101: Supply Chain With Manufacturers, Distributors, And Resellers

We’ve all seen Blow and other drug movies where people complain about getting middle’d and how to take a product from source straight to the people, so we are all experts in economics, right?

Well, for those that haven’t seen those movies, let’s revisit some topics and nomenclature.

When a manufacturer (let’s call them an Original Design Manufacturer, or ODM) makes a widget, the cost associated with the widget accounts for parts, labor, engineering costs, overhead, and profit margin. An ODM will only charge slightly more than cost for each widget, because it’s banking on volume for real profits. However, not everyone can buy a production run of equipment at a time, nor wait weeks for shipments.

The next step in the model is distribution through–wait for it–a distributor. A distributor takes large quantities of product and then sends smaller quantities to a reseller (you know, the people selling things on the street) or to a large customer.

You can rest assured that the middle man will get his cut of the action for providing logistics between the ODM and the reseller; typically 5 – 10% in the network industry.

When the average Joe buys product, it’s typically from the reseller who, surprise surprise, has a cut as well (15 – 25%) to cover its marketing fee since it’s them doing the leg work to find new customers, attend trade shows, and other marketing activities.

Case Study: 10GB SFP+ SR Transceiver

Let’s apply this model to the cost of optic transceivers. There are only so many transceiver companies with the manufacturing, engineering, and quality control to make the world go round: Avago, Finisar, and Lumentum (formerly JDSU). So when a networking company is trying to sell you an optic, it’s typically one of these three.

Let’s dissect the Cisco 10GB SFP+ SR transceiver module, which lists for $1000 (I’m sorry, $995).

Cisco doesn’t make cables or optics, so they go to the people who do. Instead of relying on one ODM, they use two, and another two for their telco line, for price guarantees and supply assurances. So how much does a 10GB SFP+ SR optic cost? It turns out around $85 + some margin, bringing the cost to $95.

However, Cisco doesn’t want some ODM’s name on the part. They want the Cisco logo on the part, along with the Cisco logo on the end cap, some branding on the ESD wrapper, tissue paper, stickers, etc….(ok, ok, no tissue paper or stickers, but you get my point).

This re-branding now brings the part to over $600. Bring in the distributors: $800. And now, the resellers: $1000. Meaning, if Cisco sells this part at MSRP, it makes $500 in profit on a $100 part.

What if we bought this product directly without the OEM rebranding? $95 plus distributor + reseller yields $150 with 2-day free shipping on Amazon.

Mental Gymnastics For The High Price Tag

Why are people paying so much more for a $100 part? Users can’t be paying the extra $500 for a Cisco logo on the optic and end cap, can they?

An OEM justifies its margin with additional compatibility testing to ensure the customer has the best experience and things just work. And then of course, someone has to take ownership of their product now that it has the OEM’s logo on it.

The other thing to mention is this padding allows sales folks to be your “friend” when selling solutions: “Since you are upgrading your Nexus line with the new Nexus line, I’ll give you a deal on the optics. How does a 40% discount, or $600, sound?”

Reality

(Feel free to substitute Cisco below with any other traditional network vendor such as Arista, Brocade, and so on; it pretty much applies universally with those vendors.)

Optic ODMs generally don’t make products special for OEMs like Cisco. Instead they make standardized parts based on MSA and IEEE standards. In fact, ODMs have a portfolio of standards-compliant products that OEMs choose from. Once a vendor chooses a part, the ODM essentially takes a snapshot of that product for the OEM. This a form of quality control for the OEM as the part is now frozen.

During the lifespan of a product, an ODM will continue to improve its products as new advances in manufacturing and engineering occur, such as increased power efficiency, a cleaner signal, and so on. Except these changes typically don’t find their way back into the OEM’s snapshot of the product.

What does this mean for the end users? This means if you buy your parts from an OEM like Cisco, you’ll not get any advancements that have been made over time vs. when you buy directly from an ODM or its distributor. You’re paying extra for an inferior product.

So, what about the compatibility testing you pay for? It’s not really compatibility testing (the ODM already does this), it’s interoperability testing among the switch + NOS + module. From my time doing this, most of the interop issues we see stem from the switch having a poor i2c bus or the NOS not doing the correct thing. But the fact that an OEM is passing on the charge for interop testing to find bugs in their NOS is MIND BOGGLING.

Oh, but we are far from being done. OEMs also add innovative enhancements to standardized parts such as changing the link establishment sequence (aka port knocking), verifying legitimate parts with encryption keys, enhancing link down detection schemes with proprietary/non standard ones, along with performing white-listing of parts in the NOS.

The main thing this innovation encourages is people making counterfeit products (you can buy Cisco-compatible optics that have their EEPROMs copied from a legitimate optic, including all encryption keys and serial number.) You wouldn’t believe how many times I’ve seen a Cisco module with the same serial number in the EEPROM in the wild. None of this would exist if people just stayed with standards-compliant parts.

So why doesn’t Cisco prevent these counterfeit products with black listing (the serial numbers are well known and prevalent)? A better question would be, how much of the Internet will stop functioning if they did?

Some FUD And Some Real Issues For Open Networking

When this subject is broached, there’s always a small contingent of folks that like to spread FUD (heck, in some companies there’s a separate charge number in marketing for it).

The main point they raise is that it’s a hardware issue (switch or module). That’s complete BS. With the amount of work a NOS has to do to set up and configure a module, to say a NOS is error free in this area is just a cop out. A NOS must know how a particular switch is set up and configured (down to the i2c bus) so that it can perform any workaround needed for a lousy switch design. The same goes for modules.

However, the other points they bring up: you can’t just plug in any ol’ module into a box/NOS and expect it to work. Who’s going to manage support and testing in Open Networking? These are valid points as switch ODMs perform compatibility testing against IEEE and MSA standards and NOS vendors take that info and provides additional guidance in the form of a Hardware Compatibility List (HCL).

But in the Open Networking world where users have the power to choose hardware, NOS, and modules along with the ability to purchase directly from vendors or from a system integrator like Dell, a lot of these issues have typically been masked.

What’s Being Done Now

That was the case two years ago. NOS vendors performing limited testing with modules. This doesn’t instill confidence for end users, and therefore reduces adoption of Open Networking.

In the summer of 2014 at an OCP Engineering Workshop at UNH-IOL, the topic of optics interop testing came up by Finisar. After a few months of calls and throwing around test plans with participants from switching, module, and NOS companies, we decided the best way to move forward was to perform dry run testing to make sure our test plans pass muster.

In 2015, we held 2 dry run plugfests at UNH-IOL. (We chose UNH-IOL as the official testing partner for the following reason: they are the most trusted name in neutral, 3rd party testing in the industry with over 25 years of experience.)

On September 21, 2015, we launched the OCP Networking Optics Interop Program complete with open test guides, open test configs, and a publicly available Integrator’s List, a page showing all passing configurations.

So far, the tests have been based on SR (Short Reach) technology but we are actively working on the following:

Wait, BASE-T aren’t valid optic options; what are you trying to pull? You’re right. These interfaces are included for the summer because of what’s going to happen at OCP Summit.  I’ll update this post with the rationale afterwards.  Until then, I’ll leave y’all in suspense.

+–+
Carlos