This guest blog post is by Jim Meehan, Solutions Engineer at Kentik. We thank Kentik for being a sponsor.
DDoS is an ever-present scourge. Thankfully, NetFlow offers practical insight into both DDoS and deeper exploits, particularly when you drill down for granular detail. We’ll show how to use raw NetFlow data, enhanced with GeoIP, to examine a DDoS attack that was accompanied by something potentially more damaging.
Investigating An Anomaly
Let’s say we’re seeing symptoms of an attack in our infrastructure. We’ll use NetFlow summary data to quickly scan total traffic in bits per second just to see if anything stands out.
There’s no obvious traffic spike, but that doesn’t mean there aren’t nefarious things going on at deeper levels of the network.
Source Geography Analysis
Combining NetFlow with GeoIP lets us look at traffic by source geography. In this case, the network doesn’t get a lot of traffic from China, so we’ll filter total traffic by China as source.
The graph above shows two obvious spikes that are well above average. Below, we zoom in on the time of the spikes.
Unique Source IP Analysis
The spikes themselves are suspicious, but is this just a large data file transfer? We can find out by looking at traffic in terms of the number of source IPs. At this point, we’ll need to access raw NetFlow record details.
That raw NetFlow detail came in handy, because there is in fact a huge increase in the number of unique source IP addresses sending traffic to particular destination IPs. This tells us that we’re not looking at a large file transfer from a single machine, but a highly distributed set of senders. Do we smell a botnet?
Who’s Getting All That Traffic?
The next step is to to determine which IP or IPs are getting all this traffic from 14,000 or so individual host IPs.
Again, host-level NetFlow details are helpful. We can see that the target is one solitary destination IP address: 10.10.10.1 (real address anonymized to protect the innocent). There’s really only one likely explanation for traffic from thousands of hosts in a suspect country to a single IP that suddenly spikes from nearly nothing to more than 1 Gbps: This is a DDoS attack.
Next Analytical Steps
Now that we know it’s DDoS, it’s worth checking if attack traffic is coming from other countries besides China.
Lo and behold, there is indeed DDoS traffic coming from multiple countries, including the U.S., Japan, Russia, Sweden, China, Taiwan, Brazil, and Estonia. Good to know.
Characterizing The Attack
Next we want to know specifically what type of traffic we’re seeing and what that tells us. We’ll group the traffic coming from all those source host IPs by protocol.
We can see that the UDP traffic is being sent to multiple ports, and it’s obvious that we’re experiencing a DNS redirection/amplification attack occurring on port 53, with a lot of port 0 UDP packet fragments being generated as collateral traffic.
A Deeper Attack Under A DDoS Diversion?
So far we’ve gotten a lot of insight into the details of the DDoS attack from full NetFlow details. But is DDoS the main event, or simply a diversion from other, less obvious threats? We can see a lot of packets being sent to port 4444 (green line in graph).
Port 4444 is the UDP port for the Kerberos service, and is — at least for Windows machines — a well-known target for buffer overflow attacks, often used to insert trojans such as Hlinic and Crackdown.
What’s interesting is that there are potentially two types of attacks going on in parallel: a DDoS attack and a buffer overflow trojan insertion. Many security blogs and publications note that DDoS attacks are often used to obfuscate other exploits. This may very well be an example of that technique.
Better-Informed Attack Mitigation
Characterizing the attacks leads us to mitigation. One way is to take the attack traffic and group it by /24 source network addresses:
We can now take two mitigation steps:
- Ask our upstream ISP to drop this traffic or send it to a scrubbing service or device if we have one
- As an added precaution, drop all traffic from these countries going to port 4444 on our own routers
One of the benefits of being able to dig into full-resolution NetFlow data is that you can get operationally useful insights without needing in-line devices. You also get more freedom to employ a portfolio of mitigation methods.
At Kentik, we’re big believers in the power of network data. Rather than summarize and FIFO raw NetFlow data, we augment raw inbound NetFlow records with BGP, GeoIP, and other datasets, then store that expanded dataset at full resolution for 90 days in our cloud.
Kentik offers a Big Data-based, SaaS network analysis platform that turns NetFlow, BGP, GeoIP and other network data into actionable intelligence for network monitoring, DDoS detection, peering analytics and planning.
If you’re interested in dashboards, alerting, and ad-hoc analysis on this depth of NetFlow data, check us out at www.kentik.com.