Linux Network Interface Configuration With udev

I’ve recently been running down (or is it through) the Linux, systemd, networkd, udevd rabbit hole at full pelt and thought perhaps now was a good time to come up for some air and share what I’ve learned. I could (and have elsewhere) written long essays on why Linux makes an excellent network ‘platform’ in many cases, but others have made that point elsewhere: e.g.:[1]. Ivan has been discussing this for many years in a number of ways. Sure, most network device operating systems are Linux-based, but these things were ‘forked’, mangled, and frankly, crippled ages ago. Market leader Cisco itself has recognised the benefit of a ‘real’ Linux interface (from an administrative point of view at least) with IOS-XE and others.

Arista and Juniper have recognisable Linux underbellies, which bring (unintended or otherwise) advantages where SDN, automation and ‘next generation’ networking features are concerned. Cumulus, Pica8 and many others have followed that lead and taken it in many different directions. Whitebox is an integral part of the story, but for the purposes of this article,  I’ve chosen to ignore it because of its proprietary nature where Broadcom, Cavium, Mellanox and other chipset vendors are concerned. Either way, a raw Linux network interface configuration is required whether we’re routing, switching, proxying, or load balancing.

This shouldn’t be too hard to do and really it’s not. However, in these times of great and rapid change within IT, things are not quite as simple as you might hope. I’m sure they will be eventually, but first the long shadows and technical debt of legacy must be overcome and the night endured before we will see the light of a new day. With Linux, the legacy is the distribution specific (proprietary in some ways) network interface configuration file. The night is systemd and the considerable conflict around it. I won’t dive into that (I don’t think I’m qualified) other than to say, from a purely network point of view, it seems like progress. That’s three related subjects I’ve ducked out of so far – its a complicated world right now.

Systemd is the rapidly evolving mother-ship of Linux initialisation systems (and rather more besides) and one that most popular distributions now use (in recent versions at least) including RHEL, Fedora, CentOS, Debian, Ubuntu, Arch, and CoreOS. This brings us some degree of ‘standardization’ in how we configure networking on these distributions. It’s not complete and it’s a work in progress, but it surely is progress.

Systemd first touches the networking stack in relation to hardware, with the udev device manager, the core library of which was merged into systemd in 2012 (it used to be a separate project). You can do pretty much everything you’d want with a network interface using udev, but it’s jarring if you are used to the legacy configuration methods and it seems rather odd to be configuring IP addressing in a hardware device configuration file, thus systemd-networkd was born. I’ll cover networkd in a following piece, but suffice to say it deals with things in a more logical, better-defined way. Why cover udev at all then? Well, networkd is pretty new and it’ll be a while before it’s widespread so for now, udev is probably the best thing most have at their disposal and it’s still very useful.

So, what can you configure with a network interface using udev, all in one place, with a single configuration file, across multiple distributions?

• Name
• Speed and duplex
• MDIX
• Tx queue length
• MTU and Jumbo Frames
• Hardware offload
• Wake on LAN
• Static IP address(es)
• DHCP Use
• Static Routes

In essence, you can run any command you’d like to use to configure the interface via udev, such as ethtool, ip and dhclient. Better yet, consistent network device naming is fully supported and you don’t need to rely on MAC addresses for identification (despite the numerous examples you’ll find via Google). I’m not aware of any Network Operating Systems (NOSs) using systemd at present, but I’m sure it’s in the pipeline so I’ll continue and anyway, we all need to be silo free and multi-disciplinary right?

I’ve no doubt I’ve missed a few tricks and nuances during my research and testing so please don’t assume this is the best way, just what I’ve surmised as that for want of contrary evidence. Hopefully if this isn’t the best way, someone will put me right.

Do I Have systemd and udevd?

To start, lets ensure our host has systemd (and thus udevd) installed with this command:

systemctl --version

The version number should be v197 or later. If it’s not or if the command is ‘not found’ then you’ll need a more recent distro.

Remove Network Manager

You might not even know it’s in use, but Network Manager can conflict with udev in strange and unusual ways so I prefer to simply remove it. This isn’t strictly required and may not be necessary but if you hit problems, its an extra complication. Be aware that doing so will probably result in the host losing all network connectivity on reboot if udev encounters an error, but hey, we’re just doing this on a vm right? Here’s a few examples of how to do so, depending on your distribution (further suggestions welcome):

Debian-based Distros:

[sudo] apt-get remove network-manager

RHEL-based Distros:

yum remove NetworkManager

Alternatively, you can edit the relevant /etc/sysconfig/network-scripts/ifcfg-* interface file and add the following line to whichever interfaces you plan to configure with udev:

NM_CONTROLLED=no

That Single Configuration File

Once that is done, let’s create or modify this file:

vi /etc/udev/rules.d/70-persistent-net.rules

Next, we populate it accordingly using commands we’re familiar with. Just before we do though, a word on consistent network device naming.

Consistent Network Device Naming

The bane of Linux for many a year, physical network interfaces were typically named and numbered based on the order in which the kernel detected them, which could be change on each boot. Imagine a switch randomly assigning different numbers to its physical interfaces on each boot. Fine if they are all configured the same, far from it if not. This issue was resolved around 2012 with the fairly widespread implementation of the biosdevname module, based on a standard created in 2009. Things were slower back then right?

Anyway, what this means is, often the ethN interface names many are used to have changed to names like em1 and enp0s3. If you switch to using udev from the legacy configuration files and don’t have biosdevname installed, this may come as a surprise. If your host doesn’t currently use consistent naming, you can see what udev will name your interfaces using this command (replacing N with a relevant number):

udevadm test-builtin net_id /sys/class/net/ethN 2> /dev/null

[> is actually >] – no idea why its being mangled.

In case you are wondering why they didn’t stick to the old naming scheme but just implement something more consistent, it’s to do with physical changes. The new scheme is based on embedded and PCI network interface locations, so even if you swap out a faulty network card, the name of the associated interfaces doesn’t change. Additionally, MAC addresses play no part, which works rather well with virtual machines and containers where such addresses may change on each boot.

If you’d like to prevent this sort of naming, you should be able to use some symlinks like so:

#v197 To v208 inclusive:
ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules

#v209 Onwards:
ln -s /dev/null /etc/udev/rules.d/80-net-setup-link.rules

However, I find this doesn’t work too reliably whereas adding this to the kernel boot command line (typically in the GRUB configuration file) works reliably (systemd v199 or later required):

net.ifnames=0

Matching/Identifying Interfaces

As I’ve mentioned, most examples I’ve seen when searching using Google and others seem to rely on matching the interface using its MAC address. This seems to be at odds with the whole point of consistent naming and won’t work well with vm images, Vagrant or containers. Of course, with embedded interfaces on physical servers it’s not a problem because the MAC will never change.

The match is used by udev to identify an interface to be configured, just to reiterate, it’s specified in this file:

/etc/udev/rules.d/70-persistent-net.rules

It’s taken me an age to find an alternative but the PCI bus identifier seems rather more reliable than a MAC address. You can find out what it is by using this command (you’ll probably have to install the pciutils package to make this command available):

lspci

Here’s what the output looks like with a NAT interface on VirtualBox:

00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

We can now use the data with an entry in the file like this:

SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:00:03.0", ....

To actually configure the name of the interface:

SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:00:03.0", NAME=="bingo"

To set the speed and duplex (scroll right):

SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:00:03.0", NAME=="bingo", RUN+="/sbin/ethtool -s '%k' speed 100 duplex full autoneg off"

‘%k’  Is a variable that represents the device name udev has given to the network interface. I know we’ve called it bingo already, but even if our ‘rename’ fails, the ethtool and ip link commands will still succeed.

To modify the MTU too:

SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:00:03.0", NAME=="bingo", RUN+="/sbin/ethtool -s '%k' speed 100 duplex full autoneg off", RUN+="/sbin/ip link set mtu 1400 dev '%k'"

Let’s make use of DHCP too (and log things verbosely with -v):

SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:00:03.0", NAME=="bingo", RUN+="/sbin/ethtool -s '%k' speed 100 duplex full autoneg off", RUN+="/sbin/ip link set mtu 1400 dev '%k'", RUN+="dhclient -v '%k'"

As this is a 10Gb interface, lets up the transmit queue length to 10,000 (no, this isn’t buffer bloat):

SUBSYSTEM=="net", ACTION=="add", KERNELS=="0000:00:03.0", NAME=="bingo", RUN+="/sbin/ethtool -s '%k' speed 100 duplex full autoneg off", RUN+="/sbin/ip link set mtu 1400 txqlen 10000 dev '%k'", RUN+="dhclient -v '%k'"

In case you’re wondering, here’s what it would look like if we did a match based on MAC address:

SUBSYSTEM=="net", ACTION=="add", ATTR{address}="08:00:27:de:b1:0a", NAME=="bingo", RUN+="/sbin/ethtool -s '%k' speed 100 duplex full autoneg off", RUN+="/sbin/ip link set mtu 1400 txqlen 10000 dev '%k'", RUN+="dhclient -v '%k'"

Let’s throw in some static routes:

..., RUN+="/sbin/ip route add 192.168.5.0/24 dev '%k'", RUN+="/sbin/ip -6 route add default via 2001:db8::1"

Testing

You can force a reload of udevd and its functions with this command:

systemctl restart systemd-udevd

Note that if you’ve manually reconfigured an interface handled by udev, this will revert those changes if the settings in question are specified in a valid .rules file. Also note that restarting udev will not bring an interface UP if it has been manually set to DOWN and there is no command to bring it UP in a valid .rules file. A reboot however, will.

I prefer to reboot myself, just to be 100% sure.

You can also ‘test’ changes using this command, but I’ve found this isn’t too helpful (perhaps I’m wrong):

udevadm test /sys/class/net/eth0 | less

Logging

The logging level for udev can be controlled by editing this configuration file: /etc/udev/udev.conf like so:

dev_log="info" #Change to err, debug etc. or a syslog number as required

You can inspect the log entries generated by udev using this command (-b restricts things to boot time entries, remove it if you want everything):

journalctl -b |grep udev

Sneaking in Logical Interfaces

I’ve found you can also use udev to create logical interfaces, not that it’s intended for this. Of course, this ties your logical interface to a physical one in some respects, is not intended behaviour and may thus may stop working unexpectedly. Still, this may be useful in a lab environment. Simply add relevant RUN+= statements to the end of a physical interface’s entry in a valid .rules file, for example:

..., RUN+="ip link add dummy0 type dummy", RUN+="ip link set dummy0 up"

Security

On discovering the ‘trick’ above, I wondered if I could delete files using a RUN+= statement and I could. That being the case, I’d suggest you are cautious when doing any lazy search-engine cut-and-paste ‘work’, or if a webpage requires you to download a .rules file from somewhere.

I did get in touch with networkd’s (very responsive) maintainer Tom Gundersen about this and he kindly pointed out that only a user with root level privileges can create or modify .rules files (directly or via a package manager such as apt-get or yum). That being the case, there is no need for any additional security measures, in my view at least.

This should also serve as a good example of why you should be very careful about which repositories you enable on a Linux host.

Gimme More

One thing I find frustrating with many a blog is an assumption that the reader understands everything that makes up the context and pretext or that refers to other blogs written in a very different style, from a different POV. In the case of this article, I’d like to think I’ve got most things covered. I appreciate that VirtualBox, ethtool and dhclient are not covered. I’ve taken the odd shortcut to keep this to under 2,000 words; I’m happy to write about these should you wish. Let me know in the comments and I’ll expand things accordingly.