My First Junos Switch: Detailed Review After Three Days Under The Covers

Background

This post is the story of my first practical look at Junos on Juniper EX-series switches.

One day last December, Skeeve Stevens from eintellego opened a can of worms by offering a deal on Juniper equipment to all network engineers on the AusNOG mailing list. I had been looking for an excuse to try out Juniper switches for a while; my office switch, an HP ProCurve 3400cl (J4905A), was very loud, and the EX2200-C was the only affordable, fully-managed Gigabit fanless switch which I could find from a major networking vendor. So I ordered one and waited patiently. I expected it to take 6 weeks or so to arrive (since apparently all the cool kids ordered one too), so I was pleasantly surprised when it arrived the day after Boxing Day. (For those non-Commonwealth inhabitants reading this page, Boxing Day is the day after Christmas Day; it’s always a holiday here in Australia.) Over the Christmas/New Year break, I spent three days learning and playing with Junos. I don’t claim this as a fair, balanced, or reasonable review – it’s simply a recounting of my experiences.

I wanted to get a good general idea of the Junos platform and set it up as my main office switch, moving the HP to my test lab (which is only turned on when i’m using it). So the basic task I set myself was to learn enough Junos to replicate the functions of my ProCurve as completely as possible. I knew ahead of time that it wasn’t going to be a 100% fit – the 3400cl was a great value switch for its day (and still is, in many ways), and had a lot of features included out of the box (like OSPF and PIM-DM multicast routing). Junos requires an additional license to implement OSPF, and it wasn’t discounted (buying it would nearly have tripled the cost of the switch), so I knew that i’d be eliminating that part of my config. My other OSPF devices are VMs running Debian Linux with Quagga, and Cisco 1700 & 1800 series routers running IOS, so I simply connected them to the same VLANs and phased out the ProCurve as a router.

Hardware

My first few impressions of the hardware were positive:

  • The switch has a solid metal case with a nice heavy feel to it.
  • It was well packed, and came with a very well-packed rack mount kit (perhaps a little too well-packed):
    Juniper EX2200-C rack mount kit packaging
  • It also came with a rather longer-than-expected, but solid, power cord retainer:
    Juniper EX2200-C power cord retainer
  • It sports wall-mounting holes as well as a rack-mount kit (hat tip to Chris Lathem for pointing this out):
    Juniper EX2200-C underside, incl. wall-mounting holes

The biggest turn-off for me on the hardware side was the realisation that Juniper switches use zero-based port numbers. Seriously? Being an old C programmer, I think of this as “proper” from a programming perspective, but try telling an end-user (troubleshooting an issue in a branch office over the phone with tech support) that port 1 is the second port not the first. I can’t think of any reason other than compatibility with “the way they’ve always done it” on their routing platforms, and I consider this self-indulgent behaviour on an equipment manufacturer’s part. Other vendors manage to start unit or chassis numbers at zero without being bloody-minded about starting port numbers at zero (although HP’s Comware switches start their port numbers at the bottom left rather than the top left, which is almost as frustrating).

Software

After my inspection of the hardware, I moved on to plugging it in and inspecting the software. Bootup took a bit longer than I expected (the system uses a 1.2 GHz Marvell ARM CPU – the same as in my QNAP TS-219P), but it looked reassuringly Unixy: Junos uses a FreeBSD-based management plane. After seeing the switch booted through to a login prompt, I logged in as root and had a poke around. Typical Unix commands like ls, df, and ps all had their usual functions, and one runs “cli” to get into the Junos configuration system. I worked for the first day or so from the serial console.

I noticed some disconcerting dates in the initial copyright notice:

“This product includes FreeBSD software developed by the University of California, Berkeley, … 1979, … 1994. “

Does this mean that 1994 is the most recent date they synchronised with the FreeBSD project? I hope this is not true, and that Juniper are actively engaging with the Open Source project upon which their code is based.

“GateD software copyright © 1995, the Regents of the University. All rights reserved. Gate Daemon was originated and developed through release 3.0 by Cornell University and its collaborators. Gated is based on Kirton’s EGP, UC Berkeley’s routing daemon (routed), and DCN’s HELLO routing protocol. Development of Gated has been supported in part by the National Science Foundation. Portions of the GateD software copyright © 1988, Regents of the University of California. All rights reserved. Portions of the GateD software copyright © 1991, D. L. S. Associates.”

It has been a very long time since I’ve seen a Unix distribution old enough to contain a gated implementation, and I sincerely hope that no vestiges of this code remain.

During boot an alarm light appeared on the front panel. I was afraid I had a hardware fault and would have to return my shiny new switch! After some investigation in the relevant manuals, I found the command to view the active alarms (“show chassis alarms”) and found that it was due to the management interface being down. I already had a management VLAN in my network, and I was expecting this switch to host it. I investigated the management LAN (at this point a shout-out to the helpful folks in #juniper is due) and found that it was not VLAN-capable at all. It’s more like a NIC for the switch’s management plane. So it’s a true out-of-band network, and can’t be bridged onto any part of the data plane. I quickly decided not to use it (and disabled the alarm with “set chassis alarm management-ethernet link-down ignore”). I can understand why the management port is provided, but i’d much prefer that it could be bridged (in software, if necessary) onto the relevant VLAN for installations that aren’t large enough to have a dedicated physical out-of-band management network. I also think enabling alarms for the management LAN being disconnected in the factory default configuration violates the principle of least surprise.

Software Updates and Support

When commissioning a new switch, router, server, storage array, etc., one of the first things I do is look for the latest firmware/software to ensure I’m up-to-date with security patches. This proved to be a bit of an issue for Junos. As a newly-arrived customer, I didn’t know what to expect; HP offers all software updates for their switches on their public web and FTP sites; with Cisco it’s a bit more involved, requiring a login to their site, and with routers you need to know the right things to say to TAC if you don’t have a support contract (with switches they’re more generous – could this have anything to do with the existence of GNS3?).

With Juniper, it seems one must have a login on their site that is specifically granted access to register devices. I had already registered a user account on their site in order to download the documentation and view their certification materials, and I was somewhat perplexed and frustrated to find that I couldn’t even register my switch until Juniper had changed my account to have “real customer” status. Their support staff were first class though, and got it sorted out pretty quickly.

Documentation

My reading of documentation began with Juniper’s “Day One” guides, some of which I had downloaded prior to receiving the switch. These guides range from the 66-page “Exploring the Junos CLI” to the massive “Complete Software Guide for Junos OS for EX Series Ethernet Switches, Release 11.1″, weighing in at 4628 pages. Suffice it to say, I only used the latter as a reference. If you’ve used Vyatta at all, you’ll find the former guide a fairly dull read. A quick skim to see the differences between Junos and Vyatta was all that was necessary for me. (Vyatta is a deliberate clone of the applicable parts of Junos’ interface.) “Configuring Junos Basics” was a useful guide to the initial setup of the switch, some parts of which have made it into my Network Engineer’s Rosetta Stone – initial setup wiki page. At 84 pages, it’s pretty digestible in a short CLI session. I used the free PDF format downloads, but most of the Day One guides are available for purchase in print format also. (Curiously, they cost US$0.99 to purchase in Kindle format, even though Kindle can read PDFs just fine.) The Day One guides (and the follow-on series “This Week”) are generally excellent in quality, and formatted for easy comprehension, although I thought that the headings could have been clearer in some cases. (“This Week: Hardening Junos Devices” devices is the one that I made mention of in my notes.)

Configuration

My checklist for features to convert from ProCurve to Juniper looked like this:

  • Initial setup: user logins, hostname, management IP address, default gateway, DNS, NTP, time zone
  • Essentials: VLANs, Rapid Spanning Tree Protocol (RSTP), LACP trunk to VM server, port labels, syslog, SNMP, ssh public keys
  • Hardening: login inactivity timer, login banner, disable unused services (including telnet and web management), management ACLs, eliminate LLDP data leakage on Internet interfaces, Spanning Tree BPDU protection/BPDU filtering
  • Nice-to-have: QoS on VLANs, port mirroring, IGMP querier, DHCP relay

Initial setup was pretty straightforward, especially given the aforementioned Day One guide. Having worked with Vyatta for a few months, I found it easy to adjust to the Junos CLI style. It’s a simple and elegant configuration system which made instant sense to me the first time I used it, and Junos worked very much as I expected. When logged in via ssh, responsiveness of the EX2200-C at the CLI (in terms of character echo delay) is noticeably snappier than the ProCurve 3400cl it replaced (and many other similar devices).

Junos has a number of powerful configuration features. The ones I came to appreciate most were configuration groups (I used this to set up SNMP) and interface ranges (which I used to set up my VLAN assignments). The latter is very powerful: a range’s name can be used in place of an interface name almost anywhere, and it lives in the configuration as a real entity, as opposed to the temporary nature of the equivalent on IOS. (Recent HP Comware versions allow a port list to be permanently associated with a name, but store the configuration in the individual ports rather than the interface range itself.)

Configuration is very slow to commit on the EX2200-C compared with Vyatta on an Intel server (understandable, given the low-end CPU); “commit | display detail” set my mind at rest that it was doing useful things, so I just got used to waiting for my commits to complete. When working in configuration mode, using “run <operational command>” works similarly to IOS’ “do <operational command>” with the added bonus that tab and ? completion works. This was very helpful when I found that “run show interfaces brief” is not very brief, and “run show interfaces terse” is probably more like what I expected of briefness.

There were a number of features in the configuration process that struck me as sub-optimal, or at least unpleasantly surprising:

  • There are separate NTP servers during boot and normal operation, with separate configurations for each. (Presumably under the covers the former are used as arguments to ntpdate during boot whereas the latter are used to configure ntpd.) This strikes me as a rather obscure feature which probably resulted from a large government department insisting on its inclusion in a large RFP. The very strange thing about this is that only one boot NTP server is allowed, whereas multiple NTP servers are allowed during normal operation.
  • The Junos shell interprets the pipe character (vertical bar) in a counter-intuitive fashion: the pipe is a well-established mechanism on Unix/Linux, Windows, IOS and Comware (amongst others) that sends the output of the first command to the input of the second command, which is a filter. Junos seems to use it as a generic way to affect the display of the output of the first command, which would more regularly be done by flags or additional arguments on other operating systems (e.g. “ls -l” on Unix/Linux, “dir /b” in DOS/Windows, “show ip interfaces brief” on IOS). This is another departure from the principle of least surprise, and i’m glad that Vyatta didn’t follow Junos in this respect (thus “show | compare” in Junos is simply “compare” in Vyatta).
  • The Junos best practice (i can’t remember which guide I read this in) is to set the IP address of lo0.0 to 127.0.0.1/32 rather than 127.0.0.1/8, as it is on Linux and Windows. This is more likely an issue when using Junos for routing rather than switching, but I still couldn’t see a reason for it.
  • Entering multi-line banners is annoying: one must use \n within the string instead of just pressing enter and finishing the string on the next line.
  • VLAN setup is a little tricky. I initially tweeted: “#Juniper #EX2200-C impression #9: #Junos makes more sense as a router OS than a switch OS. VLAN setup is almost as painful as #Mikrotik.” But after spending more time setting up VLANs on Mikrotik recently, I can now formally rescind that remark. :-) VLANs are configured in two different ways: under the interface (or interface range) and under the VLAN definition. Juniper recommends using the former for trunk (tagged) interfaces, and the latter for access (untagged only) interfaces. Because I often play around with my VLAN assignments, I ignored this advice and defined them all as if they were trunks.
  • Junos includes a concept of an “edge” port, but I couldn’t find a clear definition of it in the manuals I was working with, and as soon as I tried to apply BPDU filtering using “bpdu-block-on-edge” in interface context, I discovered its inflexibility, and discarded it in favour of “set ethernet-switching-options bpdu-block interface NAME”.

Features

Almost everything I had on my checklist to convert from my 3400cl was done at the end of my three-day lab. The exact mechanisms for each feature (on both the Juniper EX2200-C and the HP 3400cl it replaced) can be found in the config files attached to this post. A few features seemed to be missing or deficient (although this could be due to my poor research skills rather than a feature gap):

  • There is seemingly no equivalent of IOS’ “switchport trunk allowed vlan remove ID”. I couldn’t find a way to set up a VLAN trunk such that all VLANs except a particular list were allowed.
  • There’s also seemingly no equivalent of ProCurve’s “lldp admin-status PORTNUMBER RxOnly” (for allowing receive but not send of LLDP info); Julien Goodwin suggested that ACLs could be used to fix this; I simply chose to disable LLDP entirely on my ISP uplinks.
  • More perplexingly still, there’s seemingly no equivalent of IOS’ “no ip routing”. Seems very strange that there is a Junos kernel option to drop all TCP packets (I can’t think why anyone would want to turn on this option, since it would prevent remote management via ssh), but no option to turn off IP routing. Workarounds suggested by the IRC crowd include:
    • blocking traffic between vlans by policy
    • using multiple routing instances and putting each VLAN in its own instance
    • create only L2 VLANs with no L3 interface (which is the option I chose in this case)
  • It appears there’s no way to enable all interfaces for sFlow – one must add each interface individually. And further, there’s no SNMP write access at all, so I couldn’t just configure authentication and leave the heavy lifting to sFlowTrend.
  • Probably the feature I spent the most time investigating and discussing on IRC (hat tip especially to Kurt Bales) was VLAN QoS priorities. Junos’ QoS capabilities are extensive, but configuring them is not as straightforward (some would say “simplistic”) a matter as ProCurve makes it (vlan X; qos priority N). Because my home office switch is never congested, I decided it wasn’t worth spending the time to learn this part of Junos until later.

Conclusion

Junos is a mostly-logical network OS with an elegant configuration system and excellent freely-available documentation. The quality of the EX2200-C hardware is first rate, and I suspect this only increases in subsequent models in the series. I’m glad I took the time to get familiar with the basics of Junos, and I’ll probably try to be certified to their entry-level standard if time permits. I don’t think Juniper’s offering is unique enough to cause me to choose them ahead of Cisco or HP (the latter’s Comware switches are still my go-to choice), but they’re certainly an excellent choice for quality managed switches, especially for those organisations with an investment in Juniper’s routing platform.

Paul Gear
Paul is a freelance consultant working in Linux system administration, virtualisation, programming, and networking. When not working, he can often be found enjoying the surf near his home on Australia's Sunshine Coast.
  • http://libertysys.com.au/ Paul Gear

    I just realised i didn’t actually include the configuration files. These can be found on my blog: http://libertysys.com.au/blog/three-days-with-junos

  • Glen Turner

    Hi Greg.

    Copyright messages are the year the work was written. So you get Cisco IOS having a date of 1986. The reason you don’t see gated anymore is because development was taken up by a private company with no more BSD releases. It didn’t go away, it just went private. The JUNOS routing software is the most solid of all of the vendors, which is why its a popular choice for being a route reflector.

    The minor alarm light for loss of carrier on the management port is irritating stupidity, even on their larger platforms. The alarm light in general however is excellent, and you can poll it from SNMP which leads to a very easy check for “is the hardware OK”.

    I don’t know about switches, but on the routers the flow configuration is best done through a “sample; accept” firewall rule, and it’s no trouble at all to apply that to all interfaces. Maybe the switches are only running the older sflow feature rather than sampling and IPFIX.

    You’re right about NTP. The main point here is that using “boot-server” gets you the correct timestamp on the boot messages. That’s something classic IOS doesn’t both with, but the JUNOS boot sequence is a lot more complex (and took a lot longer, a cold start on their original M40 router takes about 15 minutes).

    The “edge” port refers to its role in spanning tree. Note that in JUNOS it is soft — if a BPDU arrives then the port becomes non-edge. So to make it stick you need BPDU filtering. I’d suggest using MST for any spanning tree deployment.

    I can sort of understand the lack of receive-only LLDP. The usual case is that ISPs don’t want to talk or hear LLDP from customers. They don’t want to hear because that gives customers a way to have a go at the router’s control plane CPU from unauthenticated packets. IOS suffered from CDP exploits and DoS vectors for years.

    “no ip routing” — JUNOS has strict separation between the forwarding plane and the management plane, and so not configuring an IP address in the forwarding plane is all that is sufficient to be equivalent to “no ip routing”. Use IRB to set a management address in a VLAN in the forwarding plane if you want to mix management and forwarding. It’s very easy to extend that into a routed management network, implemented on VLANs and routing tables distinct from customer traffic. If you need some other sort of “no ip routing” then use the “rib-group” feature.

    The QoS is straight from their big routers. It’s easy enough to configure once you have the DiffServ architecture clear in your mind — which it follows rigorously. :Use “classfiers” to identify traffic, Use “scheduler-maps” to set a queuing behaviour. Apply those to each interface. There are essentially four WRED queues, each with two levels of drop priority. Note that you *must* configure QoS, otherwise your control traffic will blackhole when links congest. Cisco IOS does this under the hood, it needs to be done explicitly for JUNOS.

    Overall the EX switches are an interesting combination of big router software on low end hardware. That sometimes asks a lot of the network engineer — arguably too much for a small deployment from the CLI, and in that role the switch shouldn’t be evaluated away from the JUNOS Space element manager. On a large sites the availability of serious software is a win, as then you get a level of interior misuse and weirdness which means that you want to configure the advanced features, and the “big router” heritage means they’ve been thought out well. It’s also in large deployments that a lot of the operational features of the box start to win (eg, configuration archiving, commit architecture, helper scripting, RADIUS auth, etc). I find the operational features to be a lot better thought out on JUNOS than on any other platform I have used (and I’ve used a lot).

    Glen

  • http://twitter.com/packettalk Tyler Christiansen

    Great post. I’m not 100% convinced the EX switches have a place with strictly L2 switching, despite being a huge Junos fan. Cisco reigns supreme here IMO, but I haven’t had experience with Vyatta or HP. We have some HP ProCurve switches in our network for customer edge, but I absolutely hate them. They’re HP2620s, probably running old code. I’m sure newer switches with newer code releases are better.

    They have an okay place as L3 switches, perhaps for core or even distribution in enterprise, but they can’t touch MX switches for SP work.

    Thanks for the post! Enlightening read.

    • http://libertysys.com.au/ Paul Gear

      P.S. Vyatta are routing only, and i’m pretty sure they concentrate purely on the virtual space nowadays.

  • Chad Myers

    The drop tcp option, specifically no-tcp-reset drop-all-tcp, is to protect the control plan and has no affect on the data plane. It stops the system from sending a tcp reset if someone is probing random tcp ports that don’t have an active service on them. Useful if you don’t set up a loopback filter to simply black hole unwanted traffic destined for the switch itself.

    To enable sflow on all interfaces simply use interface ranges. You can have one interface in multiple interface-ranges. For example, I created two interface-ranges to gather 1G and 10G interfaces together for sflow.

    # This grabbed ports 0-19 for the three nodes in a virtual chassis
    set interfaces interface-range IFR-All_1G_Ports member-range ge-0/0/0 to ge-0/0/19
    set interfaces interface-range IFR-All_1G_Ports member-range ge-1/0/0 to ge-1/0/19
    set interfaces interface-range IFR-All_1G_Ports member-range ge-2/0/0 to ge-2/0/19

    # This grabbed the 10G ports for the three nodes in the virtual chassis
    set interfaces interface-range IFR-All_10G_Ports member “xe-[0-2]/1/[0,2]”

    # Now, enable sflow
    set protocols sflow interfaces IFR-All_10G_Ports
    set protocols sflow interfaces IFR-All_1G_Ports

    That’s it, and it could have been done with one group. I used two because I wanted to be able to do things to the 1G and 10G ports independent of each other.

    Cheers!

    -Chad

  • anonymous

    Pretty much if it isn’t Cisco it started based on GateD. Nokia, Juniper, HP, 3Com, Alcatel and Lucent were all customers. Mostly what you find is that they re-worked everything around the application (CLI, logging, etc) and just used it. Over time I’m sure they each diverged in their own way but expect much of your RIP, OSPF, BGP, and IS-IS code to be largely GateD influenced.

    • Bubba

      JUNOS diverged from gated back in 1997. It was used as a general framework to start and dramatically morphed. Back then gated could not handle millions of routes and other protocol hooks at scale, which JUNOS had to have for the original M40 release. Pretty much all the functions and data structures were completely re-written.

      If you want to piss off the old school JUNOS software developers, tell them JUNOS is nothing but BSD and gated, and **insist** on it! I have, and the reactions are funny, you might even get punched in the face!