Stateless Routing Through an in-line F5 LTM

When using an F5 load balancer there are 2 predominant ways to setup the network topology. While there are many different names for these methods, in this article I will call them “load balancer on a stick” and in-line. Although the article is about the in-line method, we will quickly review both methods for comparison.

Load Balancer on a stick

The diagram below shows us an example of load balancer on a stick in operation –
stick
In the picture above we see that the load balancer and the server are on the same network as the Layer 3 switch (or it could be a firewall). In this setup the F5 must use a feature called SNAT, which stands for Secure NAT. Why did F5 call it this? Because calling it source NAT would just make too much sense. The F5, in addition to destination NAT that it already does, NATs the source address so that the server will return application traffic back to the F5 rather than using the default gateway.

Why is this important? If the server’s default gateway is the upstream layer 3 device, then the response packet’s source IP will be the server’s IP. Once the client receives the response it will not recognize the packet and drop it. If the upstream layer 3 device is a firewall it will not have an established session and will drop the packet as well. Also, the F5 needs to receive the return traffic for additional processing and proper session tracking. The F5 does have the ability to do direct server return but we will not be covering that in this article and I wouldn’t recommend it unless the need outweighs the complexity.

The attributes of load balancer on a stick can be viewed as such –
Advantages

  • Simple to setup
  • Does not require changing an existing network

Disadvantages

  • Source IP is not preserved. For HTTP this can be overcome with the X-Forwarded for header and web server modification. However, most other protocols don’t have any work around.
  • Harder to troubleshoot individual connections in things like a packet capture on the server because the source IP is not preserved.

In-Line Load Balancer

With the in-line method the servers are behind the F5 and the F5 becomes the default gateway for the servers.
inline1
This method preserves the source IP which is one of the best methods for non-HTTP applications and will also ease troubleshooting.

However, now all host traffic passes through the F5 such as database, DNS or any other non-load balanced traffic going to or from the server. Most of the time this is not a huge volume of traffic (except maybe backups) and many of the larger F5s have an FPGA to accelerate the L2-L4 traffic so as not to burden the CPU. If you see a model where the traffic throughput specification is much larger for the L4 traffic than the L7 traffic this is one of the models.

The picture below shows an example of server traffic –
inline2
Neither stick or in-line topology is wrong. If you are debating which method to use, choose the method that is best for your environment. If you choose in-line, the next section will show you how to get non-load balanced traffic through your F5.

Routing Though the in-line F5

Below, we have a diagram of a typical in-line setup where the F5 has a default route to the upstream switch and the servers have a default route to the F5 Self IP on the internal VLAN. A Self IP is an IP assigned to the F5 that is usually not used by load balanced traffic. The upstream L3 device must have a static route to the server IP subnet via the F5 Self IP on the external VLAN.
inline3
You will often hear F5 sales engineers tell you that the LTM is a “default deny” device meaning that the only traffic that passes though the LTM is what you define. To unpack this explanation a bit more the LTM is a not too different than a firewall as it denies any traffic not defined by a policy and statefully tracks connections that are defined by a policy (with a few other exceptions we won’t get into).

So, by default when you put the server behind the F5 they will not be able to pass non-load balanced traffic to or from the servers. To allow this traffic we must create a forwarding virtual server.

About Forwarding Virtual Servers

Forwarding virtual servers allow traffic to connect through the F5 LTM to specific destinations. They have an attribute called a fastL4 profile that defines the settings for layer 2-4 traffic. The default fastL4 profile has three of these settings worth explaining –

  • Connection Idle Timeout of 300 seconds – If an established session does not send a packet within this time the sessions is timed out on the LTM. We wont be changing it but I just wanted to mention it to put everything in context.
  • Reset on Timeout – When a session times out TCP resets are sent to client and server to terminate the connection.
  • Loose Initiation disabled by default – With this settings being disabled by default, this means that TCP session can only be established by proper TCP handshake with the initial packet having the SYN flag set.

Because of these defaults setting they can cause issues for many protocols that don’t have traffic for over 5 minutes such as some SQL protocols, SSH, etc.

Simulating Stateless Behavior on a Forwarding Virtual Server

While stateless behavior is not possible for TCP on a forwarding virtual server we can simulate it with a customized fastL4 profile. We can change two of the settings mentioned in the last section to achieve this –

  • Disable “Reset on Timeout” – By disabling this the LTM will not send a TCP RST to the client and server, keeping their connection active.
  • Enable “Loose Initiation” – By enabling this allows a packet to create an entry in the session table without having to send a packet with the TCP SYN flag set first.

The result is that a connection can timeout on the LTM, but when the client or server resumes an open connection, the LTM will recreate the session table entry.

Creating the Custom fastL4 Profile

To create the fastL4 profile, navigate to the menu shown below –

virtual6

Once you get into the New FastL4 Profile menu, change the settings we mentioned in the previous section –
virtual3

Settings -

  • Name – Give the profile a name that makes sense
  • Reset on Timeout – Check the custom box and uncheck the Enabled box.
  • Loose Initiation – Check the custom box and check the Enabled box.

Creating the Forwarding Virtual Server

To create virtual server navigate to the menu as shown –
virtual2
Once in the Create New Virtual Server menu you will see a selection box below the General properties section. Change the configuration setting to advanced as shown below to reveal more of the settings-
virtual4

Now set the following settings as shown –

virtual5

Settings -

  • Name – Something that makes sense
  • Type – Forwarding (IP)
  • Destination – Select the Network Radio button. Enter 0.0.0.0 in the Address and Mask fields to allow all traffic.
  • Service Port – All Ports
  • Protocol – All Protocols
  • Protocol Profile (Client) – Select the custom FastL4 profile you created earlier
  • SNAT – None (default value)

Once this has been created you should be able to connect to the servers through the LTM.

References -
F5 SOL7595: Overview of IP forwarding virtual servers


web statistics

Eric Flores

Eric Flores

Eric is a New Product Introduction Engineer (NPIE) with F5 Networks (as of 2014) and loves working with any technology that provides a good challenge. The previous 8 years of his life he worked as a Sr Network Engineer for various companies in the defense, broadcasting, payment processing and real estate sectors. Find him on Twitter @nerdoftech. Disclaimer: All views expressed by Eric are the views are his own and not the views of his employer.
Eric Flores
Eric Flores
Eric Flores

Latest posts by Eric Flores (see all)

  • http://etherealmind.com Etherealmind

    Eric

    Great practical networking post. Thanks.

    greg

  • Will

    Eric,

    Thanks, I’ve been waiting to see some great posts on F5 since the demise of ACE. Does LTM also support transparent / bridging?

    /r
    Will

    • What Lies Beneath

      It does.

    • Eric Flores

      I believe that a Forwarding (Layer 2) virtual server will accomplish that but I have never done that type of architecture. If there is not an overwhelming reason to implement that I probably wouldn’t recommend it as it could make troubleshooting more complicated.

  • Nate

    Very helpful. I was researching the F5 documentation, trying to figure out how to do this design. Thx!

  • Dean132

    Just wanted to say thanks for this, been having some issues with an Oracle application dropping out constantly. This article helped me resolve it.

    • Eric Flores

      No problem. It’s funny you mention that because it was Oracle weblogic servers that I first experienced a problem which forced me to look at doing the custom fastL4 profile.

  • Dave

    Thank you Eric, great post.
    I have one questions, Can F5 be configured in such i way , that physical server and virtual IP be same ?

  • Dave

    Thank you Eric, great post.
    I have one questions, Can F5 be configured in such i way , that physical server and virtual IP be same ?

    • What Lies Beneath

      You could do this using Route Domains. It might even be possible without.

  • sonu

    Hi , nice post..I just have one query with regards to f5 . In all f5 blogs , it is mentioned that the BIG-IP system performs load balancing for each TCP connection, so in this case do we really require persistence to be configured. Your inputs would be highly appreciated. Thanks in advance.

    • Eric Flores

      Actually the F5 can do load balancing all the way up to L7, which is why in their marketing literature you hear the statement “full-proxy architecture”. Persistence however would not be used for the type of virtual server mentioned in the article. There are many ways you can do persistence but 2 of the most common are source IP and http cookie insert (which you can see in my other blog post http://packetpushers.net/encrypted-cookie-persistence/)

      • sonu

        Great..Thank you

  • WilliamChen

    Hi Eric:
    I want to know, the session go through forwarding ip V.S. will count on the dashboard “open session”?? and can be show on “tmsh show sys connection”??
    thanks.