12 Tips for Effective IPS Deployment (Goats Optional)

Fresh from a conversation so frustrating that I could have cheerfully punched a goat today, I thought I’d jot down my thoughts on intrusion prevention systems (IPS) and how to effectively deploy it. I would have written a post on how not to deploy it, but that would have meant breaking an NDA.

IPS is a vital security tool in modern networks, especially given the limited value that you get from enterprise firewalls these days*. Unfortunately, they can also be challenging to deploy and manage, and require careful handling to be effective. Here’s how I like to do it:

  1. Know the difference between IPS and IDS. IPS is inline and will block traffic that meets certain criteria. IDS only detects.
  2. Consider whether IPS is the best tool to deploy. Depending on your use case, there may be better security tools for your environment. For example: web application firewalls (WAFs) are often a better option for protecting web services.
  3. Determine what you’re trying to protect. It sounds obvious, but many of my clients identify a generic requirement for IPS without thinking about what it is they want to secure. Depending on your line of business, it may be particular data streams, DMZ services, or an internal network.
  4. Choose your hardware platform carefully. There are a plethora of IPS solutions on the market, from add-on licences for firewalls to full-blown, high throughput appliances. Make sure that the solution you select has sufficient throughput for your environment – you don’t want your IPS platform to become a bottleneck.
  5. Decide where to place your IPS. Most platforms have multiple interfaces and can operate as an IPS and IDS simultaneously. For full prevention, an IPS must be deployed inline with the traffic you wish to analyse. Most platforms will allow the use of VLANs to segment a single interface into multiple virtual interfaces (make up your own mind if this is a good thing). The main point is to install your device in a physical location that allows easy access to the monitoring locations.
  6. If you’re using IDS, make sure that you have sufficient SPAN ports or taps available to connect to. In IPS deployments, consider whether your sensors should fail open or closed. Fail closed is more secure, but obviously has a service impact. You certainly don’t want to DoS your own service through a hardware or software failure on the sensor unless you really have to.
  7. Plan your signature update policy and make sure you understand how to test new signatures as they come out. Don’t just apply new signature updates without testing them, as you may end up generating false positives.
  8. Select a manageable subset of signatures for blocking. This should represent the most dangerous attacks that are also unlikely to be false positives. These signatures should be enabled for blocking only after thorough testing.
  9. Tune the hell out of your sensors and eliminate as many false positives as possible before going into production. Then tune them some more. Get important people to sign off on the tuning. If you’re using IPS, make sure that your blocking signatures have been tuned and will not block legitimate traffic. You may have to write custom signatures for your own environment. Make sure that any custom signatures you write are well documented. The very last thing you want is people being woken up in the middle of the night for false positives – they will seek revenge.
  10. Sort out your management, monitoring and incident response procedures. You need to know that your IPS/IDS alerts are being monitored, tuned and responded to promptly. For this, you need a robust management infrastructure. If you’re implementing IPS for compliance reasons, there will probably be additional requirements around logging and log retention. You may have to consider legal admissibility in this context too, depending on your legal jurisdiction.
  11. Remember that IPS deployments are not “fire and forget.” You need to be watching the IPS alert feed 24/7 and respond meaningfully to alerts. The signature base will require regular updates and tuning. Tuning really is an ongoing process, so do not be tempted to skip it!
  12. If management and maintenance of an IPS platform is too onerous for your organisation, consider taking on a managed service. Choose your managed security service provider (MSSP) carefully, and ensure you’re taking on someone who hires professionals.

A lot of people are nervous about IPS because of its inline nature and the risk of blocking legitimate traffic. However, it does offer a welcome higher level visibility of network traffic, and done right, will give excellent results – potentially saving your network. Stay calm and deploy carefully. Above all, avoid vendor hysteria. IPS isn’t perfect, but until a unicorn brings us an effective neural computing based platform, it remains an effective tool for securing networks.

Oh, and don’t try punching a goat. Not only is it cruel to the goat, they can be pretty cruel back.

* Yes, I’m going to let that statement hang there. I may turn it into another blog post :)

Neil Anderson
Neil is a freelance network security architect and contractor working with a number of clients in Scotland and Europe. He is CCIE #18705 and also holds a CISSP. He can often be found sampling beer in remote locations and ranting about tech to anyone too stupid to run away. If you're very unlucky, he may talk to you in Gaelic. Neil can be occasionally be found on Twitter.
  • Will

    Great post. Just to share a story – IPS cause bad troubleshooting (at least for me).

    I have had issues in the past where IPS signature updates have caused some traffic interruption issues like sending resets, not allowing some data to flow, and other tough to trouble shoot issues (ie when you have an accelerator, a few firewalls of different models, a load balancer, and VPN in traffic path). The security group, in turn, often does not immediately own up to applying signature updates when asked (why is that??) and also have trouble reviewing the logs to determine they are the issue. After finally proving, with captures, the IPS as the culprit, and the signature removed, the server issue is resolved. This can happen three times a year and even more when the appliances are replaced with new models.

    This causes me immediately want to check the IPS every single time there is some odd one off traffic interruption issue before looking at anything else. Which is bad troubleshooting, but I cant help it because of the above.

    • NeilTAnderson

      That’s a great example of why there needs to be testing of IPS sigs before and after updating. It’s also a good reason why blocking signatures should be kept to a small manageable group so that this kind of thing doesn’t happen. If in doubt, install a signature in alert-only mode first and see if there are any false positives before chucking it into block mode.

      As for why security staff won’t own up to causing problems – it puzzles me. I think in general that there’s a certain spikiness there because security types so often have to fight to persuade people to do things properly. When it turns out that the security tool is causing a problem, it can lead to some serious denial, which ultimately contributes to the problem that security has in the first place.

  • Jay Swan

    Good post, especially this: “Determine what you’re trying to protect.” It always amazes me how rarely people ask that question before starting on security projects. They also rarely ask the follow-up question: “Will the process/tool/product I’m considering actually help?”

    I’ve never found a situation where I felt like the risks of IPS mode were outweighed by the (extremely minor) benefits. If your threat intel is so good that you can tune your IPS to accurately block an attack without incurring the risks of false positives or signature update meltdowns, you should be able to protect the asset some other way.

    • NeilTAnderson

      “I’ve never found a situation where I felt like the risks of IPS mode were outweighed by the (extremely minor) benefits.”

      It all depends on your security posture. If you’re a corporation hosting a big website, and are paranoid about accidentally blocking user traffic, then you’d be better going down a WAF + IDS route. On the other hand if you’re a public sector organisation with compliance requirements, IPS is a good option.

      I’ve seen IPS sensors save a network from what would have been catastrophic attacks. I’ve also seen them cause massive problems – it’s about choosing the right tool for the job.