
Someone asked so lets walk through the overhead introduced when using IPSec with AES; it’s higher than you might think and I haven’t even factored in ISAKMP.
Encryption really isn’t ‘my bag’ so if anything is wrong, do let me know; hopefully public scrutiny will mean I can truly rely on these figures.
Take a look at my TCP Over IP Bandwidth Overhead article if you’d like to understand the further overheads introduced by TCP/IP itself, plus those of Ethernet.
TL;DR – As before, skip to the end for the summary if you really can’t face the math.
Assumptions
The following assumptions have been made when formulating these calculations;
- No retries, packet losses or other events occur
- One way, one to one host communication data and overhead
- TCP & IP packet headers of 40Bytes (no TCP options)
- Use of a full-duplex communications medium, i.e. the full bandwidth is available both upstream and downstream, at the same time
- Where division of the data into the maximum packet size results in a fraction, a packet for that remaining data must still be transmitted so I always round-up when calculating the number of packets
Units of Measurement
The abbreviations used with the data/traffic values in this article are metric prefixes (aka SI prefixes) indicating decimal multiplication (rather than binary prefixes (based on powers of two) where 1kbit = 1024bits), as follows;
- 1 kB = 1,000 Bytes (8,000 bits)
- 1 MB = 1,000,000 Bytes (8,000,000 bits)
A few things to remember;
- Serial line speeds are typically quoted using binary prefixes, so a 2Mb E1 is actually 2.048Mb using metric prefixes
- Ethernet speeds are typically quoted using metric prefixes, so a 100Mb Ethernet link is exactly that (100,000,000bit)
- Linux commands display file size information using metric prefixes. However, using a –human-readable or -h switch normally results in output using binary prefixes which is rather confusing
- Windows displays file sizes using binary prefixes. To make accurate calculations, view the file properties or use a command prompt to discover the file size in Bytes and apply the metric prefix
- Most other storage systems use binary prefixes
- Hard drive manufacturers typically use metric prefixes which means they appear smaller than specified when capacity is displayed using binary prefixes
Bits & Bytes
- 1 Byte = 8 bits
- File sizes are normally quoted in Bytes
- Linux commands display file size information in Bytes
- Link speeds are quoted in Mb (Megabits) per second, not MegaBytes, thus (ignoring all overheads) it’ll take 8s to move 100MB (MegaByte) over a 100Mb (Megabit) per second Fast Ethernet link
Other Overheads
TCP/IP handshake overheads have been ignored as these are negligible.
ISAKMP overheads have been ignored.
For the sake of simplicity I’ve ignored Ethernet’s preamble, start frame delimeter and interpacket gap when calculating it’s overhead.
Encryption Padding
AES Only operates using a specific size of data called block size. The AES block size is 128 bits or 16 Bytes. If the body of a packet is smaller than or not divisible by this 16 Byte block size it is ‘padded’ so it can be operated on. For instance;
- A 1 Byte packet body will have 15 Bytes of padding added, resulting in a 16 Byte body.
- A 500 Byte packet body will have 12 Bytes of padding added, resulting in a 512 Byte body.
- A 1400 Byte packet body will have 8 Bytes of padding added, resulting in a 1408 Byte body.
Reference Note: AES block size – RFC3602, Section 2.4
Hash/HMAC Padding & Additions
MD5 and SHA-1 HMAC processing also operates using a specific block size; 512 bits or 64 Bytes. Every packet (aka message) also has a 1 bit padding identifier added (even if there is no padding) and a 64 bit or 8 Byte message length added. If the body of a packet, with the extra 65 bits added, is smaller than or not divisible by the 64 Byte block size it is ‘padded’ so it can be operated on. For instance;
- An 8 Byte packet body will have 48 Bytes of padding added, (including the 1 bit padding identifier,) resulting in a 56 Byte body prior to the 8 Byte message length information being added. Total body size will be 64 Bytes.
- A 500 Byte packet body will have 4 Bytes of padding added, (including the 1 bit padding identifier,) resulting in a 504 Byte body prior to the 8 Byte message length information being added. Total body size will be 512 Bytes.
- A 1399 Byte, 7 bit packet body will have no padding added except the the 1 bit padding identifier, resulting in a 1400 Byte body prior to the 8 Byte message length information being added. Total body size will be 1408 Bytes.
IP Sec Headers & Trailers
AH and ESP both add headers to the TCP/IP packet itself, ESP also adds an Initialisation Vector (IV) and a trailer. The size of this additional data depends on the IPsec protocol and mode used, as follows;
- Tunnel Mode: 20 Byte header regardless of protocol used
- Transport Mode: No additional data, headers or trailers
- AH: 24 Byte header
- ESP: 40 Bytes (8 Byte header (SPI and Sequence Number,) 16 Byte IV and 16 Byte trailer)
Reference Note: The Initialisation Vector (IV) is always be the same as the encryption block size – RFC3602, Section 2.1
Overhead Calculations
Now we understand all the possible additions to the packet body and the TCP/IP packet itself, we’ll calculate the overall affect or overhead when encrypting packets with AES and sending them across an IPsec secured network link. We’ll assume SHA-1 hashing, ESP tunnel mode is used and the ESP IV is 16 Bytes.
Transmitting 1 Byte of Data
This might seem unlikely but programs such as Telnet and SSH transmit a packet for every character sent or received during a session.
- Add 15 Bytes for AES padding to reach the 16 Byte AES block size (1 16 Byte block)
- Add 1 bit for the padding identifier
- Add 8 Bytes for the SHA-1 message length information
- Add 39 Bytes, 7 bits padding to reach the 64 Byte SHA-1 block size (1 64 Byte block)
- Add 20 Bytes for the ESP tunnel mode header
- Add 8 Bytes for the ESP header
- Add 16 Bytes for the ESP IV
- Add 16 Byes for the ESP trailer
- Total packet size (minus TCP/IP headers) is now: 124 Bytes – an increase of 12,300%
Transmitting 1000 Bytes of Data
- Add 8 Bytes for AES padding to reach the 16 Byte AES block size (63 16 Byte blocks)
- Add 1 bit for the padding identifier
- Add 8 Bytes for the SHA-1 message length information
- Add 7 Bytes, 7 bits padding to reach the 64 Byte SHA-1 block size (16 64 Byte blocks)
- Add 20 Bytes for the ESP tunnel mode header
- Add 8 Bytes for the ESP header
- Add 16 Bytes for the ESP IV
- Add 16 Byes for the ESP trailer
- Total packet size (minus TCP/IP headers) is now: 1084 Bytes – an increase of 8.4%
Transmitting 1328 Bytes of Data
- Add 0 Bytes for AES padding to reach the 16 Byte AES block size (83 16 Byte blocks)
- Add 0 bit for the padding identifier
- Add 8 Bytes for the SHA-1 message length information
- Add 8 Bytes padding to reach the 64 Byte SHA-1 block size (21 64 Byte blocks)
- Add 20 Bytes for the ESP tunnel mode header
- Add 8 Bytes for the ESP header
- Add 16 Bytes for the ESP IV
- Add 16 Byes for the ESP trailer
- Total packet size (minus TCP/IP headers) is now: 1404 Bytes – an increase of 5.72%
Transmitting 1460 Bytes of Data
- Add 12 Bytes for AES padding to reach the 16 Byte AES block size (92 16 Byte blocks)
- Add 1 bit for the padding identifier
- Add 8 Bytes for the SHA-1 message length information
- Add 55 Bytes, 7 bits padding to reach the 64 Byte SHA-1 block size (24 64 Byte blocks)
- Add 20 Bytes for the ESP tunnel mode header
- Add 8 Bytes for the ESP header
- Add 16 Bytes for the ESP IV
- Add 16 Byes for the ESP trailer
- Total packet size (minus TCP/IP headers) is now: 1596 Bytes – an increase of 9.32%
Summary
So, as demonstrated, for data payloads in excess of the common TCP payload maximum segment size (the MSS) of 1460 Bytes, the IPSec bandwidth overhead using AES is approximately 9.32%. This equates to an ‘efficiency’ of 91.48% (1460/1596) – in other words, that’s how much bandwidth is left for actual data if you’re putting as much data in each packet as possible. Note however that as this packet size is larger than the typical IP (and Ethernet) MTU, it’s very unlikely that you’ll achieve this level of efficiency. As noted in the comments, the ideal MSS appears to be 1328 when using ESP.
Keep in mind that for very small data payloads (common with applications such as Telnet, TN3270 mainframe emulation and SSH) the IPSec bandwidth overhead can as high as 12,300%.
If you add TCP/IP and Ethernet (and VLAN tagging) into the mix (see the calculations from Wikipedia here) then the throughput of a 100Mb link is 100 x 0.92.64 (IPSec+AES efficiency) x 0.9733 (TCP/IP efficiency) x 0.9728 (Ethernet (with tagging) efficiency) which equals 87.71Mbps, a combined efficiency of 87.71%. assuming ideal conditions.
Thanks To
This article would not of been possible without the analysis and information provided by Ray Savarda and Matt Karash in this article: http://www.tisc-insight.com/newsletters/39.html.
Chris Bull for correcting me over the ESP IV size back in 2010.
Next
My head hurts but I’m having quite a bit of fun removing the rose coloured spectacles and taking away everyone’s bandwidth. I’ve already ‘stolen’ 1.2Gb+ of every 10Gb link out there and I really want more. So, next up I’ll calculate the bandwidth overhead of SSL/TLS and see if we can get down to less than 80% efficiency. Think this sounds fanciful? What’s so odd about using a VPN with HTTPS through a trunked firewall interface?
Other articles in this series;
The icon Artwork used in this article is by the GNOME Project and licensed under the Creative Commons Attribution-Share Alike 3.0 United States License.
Actually it’s even lower, 1328 bytes MSS is the max since it needs multiple of 16-byte for AES.
I think 1392 is a better MSS;
No AES padding
+16 Hash/HMAC padding
+20 Tunnel Mode
+40 ESP
=1448, allowing the 40Byte TCP/IP headers to be added without exceeding any ‘standard’ MTU.
seems your math is wrong. 1392 + 16 + 20 + 40 + 40 (outer headers) = 1508 >1500
Yes it is. I’ll never be employed again at this rate! Thanks. 🙂
So, yes you are correct, 1328 is indeed the ideal MSS but because of the 64Byte multiple, not the 16Byte one.
The math:
1328 (16-byte aligned)
+8 (sha-1 length)
+8 (sha-1 padding for 64-byte alignment)
+ 20 bytes (tunnel mode)
+ 40 (esp headers)
+ 40 (outer tcp+ip headers)
=1444 bytes
The next multiple of 64 is 1508 which is > 1500 MTU and will/can break in several networks. But the min configured of MSS these days seem to be 1380 bytes:
http://www.caida.org/~mjl/pubs/measuring-pmtud.pdf
how do you explain this?
There’s many ways to ‘mix’ this, see my response to your other comment for my best attempt at the highest MSS (not MTU).
Regarding the 1380 figure, this relates only to the internet and in my view, quite a small sample. Also, IPsec use is not considered in any way. It also only related to <11% of the sample, not significant in my view.
In my (enterprise and related internet (not SP)) experience where MTU issues are considered to be a potential issue when using IPsec, MSS and DF values are transparently modified where necessary. This is very easy to do on Cisco equipment and no doubt others,
I dont think there’s a problem with the size of the sample (50k+ dests over 5k asns); but more to the type of targets (alexa websites). But my point is, if the end points are lowering the MSS to 1380 already, why not bring it down to 1328 to account for the worst case? e.g. is the 1380 MSS value representative of a typical/common ipsec setup?
Thanks.
FWIW, this brings up another issue. A large part of the overhead described above is the padding needed for the Message Authentication Code (MAC). Padding is usually all 0’s or a repeated constant value. So why send it? Why not just logically add it back in at the far end?
The ESP specification seems to indicate that the default is NOT to send MAC padding (RFC 4303, Section 3.3.2.1, last paragraph). But other observations/analysis indicate that it is sent when the MAC is SHA-1. As far as I can tell, this is because SHA-1 messages do not end with the padding. Instead, SHA-1 padding is the next to last field in the message. The last field is the message length. This makes SHA-1 padding a little more complicated, and seems to have resulted in all the extra padding being sent.
Anyway, this made me curious, so I did a little investigating. And since I invested the time, I though it would be worthwhile mentioning. Please feel free to correct what I got wrong.
Thanks Jim, I’ve made a note of this in the article. Cheers
One note about transport mode:
Since transport mode reuses the IP header from the data packet it can only be used if the VPN enpoints are the same IP as data end point.Transport mode works great for GRE over IPsec because the GRE and IPSec tunnel enpoints can be the same. I have used this for a MPLS-over-GRE-over-IPSec deployment to reduce the MTU overhead by 20B.
Thanks for the info Eric. Cheers
Steven,
Naturally the overhead will vary with the amount of data is being sent and what is included in the overhead (GRE, tunnel key, MPLS, tunnel vs. transport mode).
Checkout the tool here http://www.cisco.com/c/en/us/support/web/redirects/l2l-checker.html
-Jay Young
Sorry wrong link, here is the correct one http://www.cisco.com/c/en/us/support/web/redirects/ipsec-overhead-calc.html
Indeed Jay, packet size is all important and traffic patterns also.
I specifically based the calculations based on IPSec using AES. Obviously other encapsulation/tunnelling methods will have different overheads.
In “Transmitting 1460 Bytes of Data” example there is an arithmetic mistake. (24 64 Byte blocks) is 1536 bytes. 1536 + 20 + 8 +16 +16 = 1596 Bytes. 1576 Bytes is mentioned in the post.
Hey Jay,
Thank you – can’t believe there are still errors after all this time. Corrected, plus added in the MSS 1328 calculations for extra clarity.
Thank you for the quick update. I am also trying to calculate the maximum size of a RTP header. If you are aware of a post which gives this detail, please let me know.
You’re welcome. Regarding RTP, I’m not sure that’s possible based on a quick read of RFC3550, as it’s ‘user’ defined.