Capturing network traffic packets provides insight into what's
happening on your network but you don't want just anyone to do it.
You're in the middle of troubleshooting some weird network or service issue, things just aren't right, the ports are open, the service is listening, but something is still off. Beyond verifying the basics, connectivity, ports, and running services, sometimes you just need to see what's going through the network. Stick around, and we'll talk about a few ways you can snoop on what's running across the wire, or up and down your application stack.
Why capture packets?
Networks and services can be complex. One service reaches out to another to get some bit of information, then that one reaches out to something else, and so on. Everything has to mesh. Name lookups and other dependent services need to function correctly. There's also latency. Wouldn't it be great to figure out where a request is getting bogged down? You can get some of this information in other ways, but actually seeing what's going down the wire can be immensely useful. On top of that, if you're in a secure, or segmented network, you may run into blocked ports. Port scanning can help a lot here.
Simply knowing that traffic can pass isn't always enough. What you'd like to see is exactly what traffic is passing, and what it is saying. Fortunately, there is a way to snoop on that traffic and see what's inside of each and every packet. It's called packet sniffing, or packet capture.
Capturing DHCP packets
First, let's determine a test service and then look at what a packet capture from that service can tell us. We're going to use DHCP as a good test because it's easy to follow, and readily available to me as I write this.
I'll spare you the background on how DHCP works and summarize a bit. When a new host that is configured to use DHCP is brought online, it first asks for an IP via a broadcast (a request sent out that all active devices with an IP on that subnet will see). If there is a DHCP server available, that server responds. How it responds depends on whether it thinks it's allowed to assign an IP, and if it has any IP addresses available in its pool.
You can test this yourself with a basic virtual machine (VM) setup. I have one virtual guest running on my laptop, using libvirt. That guest is configured to get an IP address directly from the NAT network configured by libvirt, which also provides a DHCP server. I ran a packet capture using tcpdump on my host machine while I powered on my guest machine. We'll talk more about how to use the tcpdump command in a moment.
I am displaying the results with Wireshark, another tool we'll talk about later.
The first thing we see in the packet capture is a system with no IP address asking for an IP, and then our DHCP server responding. The response, if you dig a little deeper, contains all the information that the requesting system needs to configure its local IP address.
The Option: (xx) fields are various extra bits of information the DHCP server is configured to provide.
So, you can imagine how useful this tool is when troubleshooting a DHCP issue. You can not only see if your DHCP server responded but what it responded with! If you never saw the initial DHCPREQUEST, it could tell you that your client isn't connected to the proper network, or has a network problem. If you saw the DHCPACK, but some of the info looks wrong, you could try to pinpoint a configuration issue.
Using tcpdump
So how do you perform this wizardry? It's not terribly difficult. All you need to get started is a simple tool called tcpdump. You can get it from yum on a RHEL system. Once installed, you can check out its very extensive man page for more information, but the basics are simple. You need root privileges to gather this sort of information. You need to know what interface to expect traffic on (you can listen on all interfaces, but that gets very chatty), and you'll want to decide how to output the data. In this case, I'm writing to a file. I used the following command for my example:
[nlager@nlager ~]$ sudo tcpdump -i virbr1 -w ./virt-interface1.pcapAfter you've performed whatever task you're trying to troubleshoot, hit Ctrl-C to end the capture. You should get some output telling you how many packets tcpdump captured:
[nlager@nlager ~]$ sudo tcpdump -i virbr1 -w ./virt-interface1.pcap
tcpdump: listening on virbr1, link-type EN10MB (Ethernet), capture size 262144 bytes
^C46 packets captured
54 packets received by filter
0 packets dropped by kernel
[nlager@nlager ~]$And you should have an output file:
nlager@nlager ~]$ ls -l virt-interface1.pcap
-rw-r--r--. 1 tcpdump tcpdump 5293 Mar 5 12:55 virt-interface1.pcapThat file is binary, though, so you can't just cat it and see what's in there. You can read it with tcpdump, but a more helpful tool is something like Wireshark (again, available via yum). You'd install Wireshark, then load in the pcap file. Personally, I run all of my servers without a GUI, so I keep Wireshark on my workstation. After you complete your packet capture, copy it to your workstation and import the pcap file into Wireshark. Browse to File -> Open, and then select your pcap file. Wireshark can also do live packet captures, but tcpdump is much more convenient when you're working with a server because it may not have a GUI. The packet capture includes a lot more information than I showed above. The full window looks like this:
Clicking on an item in the list of packets in the top pane gives you a decoded view of the data in the middle pane. The bottom pane shows you the raw hex of the packet.
Packet captures are also very useful to support engineers when they're trying to help you troubleshoot odd network behavior. A packet capture can tell you how long it took for a remote resource to respond, what it responded with, and whether that data looks sane or not.
Security concerns
Be aware, though, that packet captures can contain anything that the system you are running it on can see on the network, not just what's available inside its own TCP stack. If I am using some unencrypted protocols, like telnet, HTTP, FTP, DNS, or many others, tcpdump can happily snoop on that data and save it to your file. Wireshark can also decode SSL encrypted traffic, and re-assemble it if you can provide it with a certificate. It can even re-construct VoIP calls! If a bad guy gets into one of your systems and finds tcpdump, they'll be thankful that you've left such a valuable tool at their disposal. So, it would be a good idea to remove tcpdump immediately after solving whatever problem you're attempting to troubleshoot.
Certain network-level safeguards can be put in place to help prevent a system from capturing all of the traffic on a given network. So if you're worried about curious users sniffing their co-worker's banking details, fear not, modern network switches disable promiscuous mode to stop such bad actors.
Packet Capture: What is it and What You Need to Know
Packet capture is a vital tool used to keep networks operating safely and efficiently. In the wrong hands, it can also be used to steal sensitive data like usernames and passwords. In this post, we’ll dive into what a packet capture is, how it works, what kind of tools are used, and look at some sample use cases.
What is Packet Capture?
Packet Capture refers to the action of capturing Internet Protocol (IP) packets for review or analysis. The term can also be used to describe the files that packet capture tools output, which are often saved in the .pcap format. Capturing packets is a common troubleshooting technique for network administrators, and is also used to examine network traffic for security threats. Following a data breach or other incident, packet captures provide vital forensic clues that aid investigations. From a threat actor’s perspective, packet captures might be used to steal passwords and other sensitive data. Unlike active reconnaissance techniques like port scanning, capturing packets can be accomplished without leaving any trace behind for investigators.
How Does Packet Capture Work?
There’s more than one way to catch a packet! Packet captures can be done from a piece of networking equipment like a router or switch, from a dedicated piece of hardware called a tap, from an analyst’s laptop or desktop, and even from mobile devices. The approach used depends on the end goal. No matter what approach is used, packet capture works by creating copies of some or all packets passing through a given point in the network.
Capturing packets from your own machine is the easiest way to get started, but there are a few caveats. By default, network interfaces only pay attention to traffic destined for them. For a more complete view of network traffic, you’ll want to put the interface in promiscuous mode or monitor mode. Keep in mind this approach will also capture a limited view of the network; on a wired network, for example, you’ll only see traffic on the local switch port your machine is connected to.
On a router or switch, features known variously as port mirroring, port monitoring, and switched port analyzer (SPAN) allow network admins to duplicate network traffic and send it to a specified port, usually to export packets to a dedicated monitoring solution. Many enterprise-grade switches and routers now have an embedded packet capture function that can be used to quickly troubleshoot right from the device’s CLI or web interface. Other types of networking equipment like firewalls and wireless access points also commonly have packet capture functionality.
If you’re performing a packet capture on an especially large or busy network, a dedicated network tap might be the best option. Taps are the most expensive way to capture packets but introduce no performance penalty since they are dedicated hardware.
How to Read a Packet Capture
In order to understand and analyze a packet capture, you’ll need some fundamental knowledge of basic networking concepts, especially the OSI model. While there may be differences between specific tools, packet captures will always consist of a payload and some headers. The payload consists of the actual data being transferred – this could be bits of a streaming movie, e-mails, ransomware, or anything else traversing a network. Packet headers contain all the critical information that helps network equipment decide what to do with each packet. Most important are the source and destination addresses, but IP packets have a total of 14 headers covering everything from Class of Service to Protocol Type. A professional network analyst will have detailed knowledge of all of these fields, but a general understanding of packet structure is all it takes to get started troubleshooting performance issues or learning more about how networks operate.
| Version | Header Length | DSCP Value | ECN | Total Packet Length |
| Identification | Flags | Fragment Offset | ||
| Time to Live (TTL) | Protocol | Header Checksum | ||
Source IP Address | ||||
Destination IP Address | ||||
Optional | ||||
Apart from source and destination addresses, some of the most important fields from a troubleshooting perspective may include Differentiated Services Code Point (DSCP), Flags, and TTL. DSCP is used to ensure Quality of Service (QoS), and is an important field for real-time traffic like Voice over IP (VoIP). Flags are most often used to control packet fragmentation and can become a problem when a packet that has the Don’t Fragment flag also exceeds the Maximum Transmission Unit (MTU) size of a network link. TTL values are decremented after each hop and can provide important clues about a packet’s path through the network.
For unencrypted types of traffic, packet sniffers can dig deeper than just the headers and inspect the actual payload. This can be incredibly useful for troubleshooting network issues but is also a potential security issue when sensitive data like usernames and passwords are present. Understanding the meaning behind a packet’s payload can require knowledge of the protocol in use.
Packet capture and network analysis applications frequently include tools to filter, visualize, and inspect large amounts of data. These tools allow for analysis that isn’t possible through manual inspection of a packet capture. Capture files can also be fed into an Intrusion Detection System/Protection Systems (IDS/IPS), Security Information and Event Management System (SIEM), or other types of security products to look for signs of an attack or data breach.
Formats, Libraries, and Filters, Oh My!
When it comes to packet captures, there’s a range of related terminology that can get confusing rather quickly. Let’s break down some of the most common and important terms you might hear:
Packet Capture Formats
While packet capture tools like Wireshark can be used to inspect traffic in real-time, it’s more common to save captures to a file for later analysis. These files can be saved in a variety of formats. .pcap files are the most common and are generally compatible with a wide range of network analyzers and other tools. .pcapng builds on the simple .pcap format with new fields and capabilities and is now the default format when saving files in Wireshark. Some commercial tools may also use proprietary formats.
Libraries
Libraries like libpcap, winpcap, and npcap are the real stars of the packet capture show, hooking into an operating system’s networking stack and providing the capability to peer into packets moving between interfaces. Many of these libraries are open-source projects, so you may find them in a wide variety of both commercial and free packet capture tools. In some cases, you may need to install the library separately from the tool.
Filtering
Full packet capture can take quite a bit of space and demand more resources from the capturing device. It’s also overkill in most cases – the most interesting information is typically only a small portion of the total traffic being observed. Packet captures are often filtered to weed out the relevant information. This can be based on everything from the payload to IP address to a combination of factors.
Packet Capture Tools
A large number of different tools are available to capture and analyze the packets traversing your network. These are sometimes known as packet sniffers. Here are some of the most popular:
Wireshark
The quintessential packet tool, Wireshark is the go-to packet capture tool for many network administrators, security analysts, and amateur geeks. With a straightforward GUI and tons of features for sorting, analyzing, and making sense of traffic, Wireshark combines ease of use and powerful capabilities. The Wireshark package also includes a command-line utility called tshark.
tcpdump
Lightweight, versatile, and pre-installed on many UNIX-like operating systems, tcpdump is a CLI junkie’s dream come true when it comes to packet captures. This opensource tool can quickly capture packets for later analysis in tools like Wireshark but has plenty of its own commands and switches to make sense of vast sums of network data.
SolarWinds Network Performance Monitor
This commercial tool has long been a favorite for its ease of use, visualizations, and ability to classify traffic by application. Though the tool only installs on Windows platforms, it can sniff and analyze traffic from any type of device.
ColaSoft Capsa
ColaSoft makes a commercial packet sniffer aimed at enterprise customers, but also offers a pared-down edition aimed at students and those just getting into networking. The tool boasts a variety of monitoring features to aid in real-time troubleshooting and analysis.
Kismet
Kismet is a utility devoted to capturing wireless traffic and detecting wireless networks and devices. Available for Linux, Mac, and Windows platforms, this tool supports a wide range of capture sources including Bluetooth and Zigbee radios. With the right setup, you can capture packets from all of the devices on the network.
Packet Capture and Packet Sniffer Use Cases
While the term Packet Sniffer may conjure up images of hackers covertly tapping into sensitive communications, there are plenty of legitimate uses for a packet sniffer. The following are some typical use cases for packet sniffers:
Asset Discovery/Passive Reconnaissance
Packets by their very nature include source and destination addresses, so a packet capture can be used to discover active endpoints on a given network. With enough data, it’s even possible to fingerprint the endpoints. When done for legitimate business purposes, this is called discovery or inventory. However, the passive nature of a packet capture makes it an excellent way for malicious attackers to gather information for further stages of an attack. Of course, the same technique can be used by red teamers testing an organization’s security
Troubleshooting
When troubleshooting network issues, inspecting the actual network traffic can be the most effective means of narrowing down the root cause of a problem. Packet sniffers allow network administrators and engineers to view the contents of packets traversing the network. This is an essential capability when troubleshooting foundational network protocols such as DHCP, ARP, and DNS. Packet captures do not, however, reveal the contents of encrypted network traffic.
Sniffing packets can help verify that traffic is taking the correct path across the network, and is being treated with the correct precedence. A congested or broken network link is often easy to spot in a packet capture because only one side of a typically two-sided conversation will be present. Connections with a large number of retries or dropped packets are often indicative of an overused link or failing network hardware.
Intrusion Detection
Suspicious network traffic can be saved as packet capture and fed into an IDS, IPS, or SIEM solution for further analysis. Attackers go to great lengths to blend in with normal network traffic, but a careful inspection can uncover covert traffic. Known malicious IP addresses, telltale payloads, and other minute details can all be indicative of an attack. Even something as innocuous as a DNS request, if repeated at a regular interval, could be a sign of a command and control beacon.
Incident Response and Forensics
Packet captures provide a unique opportunity for incident responders. Attackers can take steps to cover their tracks on endpoints, but they can’t unsend packets that have already traversed a network. Whether it’s malware, data exfiltration, or some other type of incident, packet captures can often spot signs of an attack that other security tools miss. As a packet header will always contain both a source and destination address, incident response teams can use packet captures to trace the path of an attacker through the network, or spot signs of data being exfiltrated out of the network.
Packet Capture Advantages and Disadvantages
As already stated, packet captures are a tremendous asset to network administrators and security teams. They aren’t the only option for monitoring network traffic, however, and there may be instances when things like SNMP or NetFlow are better choices. Here’s a look at some of the advantages and disadvantages of using packet captures:
Advantage: Most Complete Look at Network Traffic
Packet capture is by definition a duplicate copy of the actual packets traversing a network or network link. It is, therefore, the most thorough look at network traffic possible. Packet captures contain a great level of detail not available in other monitoring solutions, including complete payload, all IP header fields, and in many cases even information about the capture interface. This can make capturing the only viable solution in cases when lots of detail is required.
Advantage: Can Be Saved for Further Analysis
Packet captures can be saved for further analysis or inspection in industry-standard .pcap and .pcapng formats. This allows, for example, suspicious traffic to be saved by a network engineer and then later reviewed by a security analyst. A wide variety of tools support this format, including security analysis tools. It’s also possible to save a packet capture consisting of several hours’ worth of data and review it at a later point in time.
Advantage: Hardware Agnostic
SNMP and NetFlow both require support at the network hardware level. While both technologies enjoy wide support, they are not universally available. There may also be differences in how each vendor implements them. Packet capture, on the other hand, does not require specialized hardware support and can take place from any device that has access to the network.
Disadvantage: Large File Sizes
Full packet capture can take up large amounts of disk space – in some cases up to 20 times as much space as other options. Even when filtering is applied, a single capture file may take up many gigabytes of storage. This can make packet captures unsuitable for long-term storage. These large file sizes can also result in lengthy wait times when opening a .pcap in a network analysis tool.
Disadvantage: Too Much Information
While packet captures to provide a very complete look at network traffic, they’re often too comprehensive. Relevant information can often get lost in vast sums of data. Analysis tools have features order, sort, and filter capture files, but many use cases might be better served by other options. It’s often possible to troubleshoot a network or spot signs of an attack with just the summarized versions of network traffic available in other monitoring solutions. One common approach is to use a technology like NetFlow to monitor all traffic and turn to a full packet capture as needed.
Disadvantage: Fixed Fields
The most recent iterations of NetFlow allow for customizable records, meaning network admins can choose what information to capture. Since a packet capture is based on the existing structure of an IP packet, there is no room for customization. This may not be an issue, but again depending on the use case there may not be a need to capture all fields of an IP packet.
Conclusion
Packet capture is invaluable from a troubleshooting and security perspective, but should never be the sole tool that a network admin or security engineer relies on. The increased use of encryption for both legitimate and illegitimate purposes limits the effectiveness of tools like Wireshark. Packet captures also do not give incident responders much of an idea of what actions have taken place on a host. Files could have been modified, processes hidden, and new user accounts created without generating a single packet. The Varonis Data Protection platform provides a data-centric view of your organization that can spot threats that go undetected in the network. As always, make sure to employ defense-in-depth and best practices in your network. And when you’re ready to add Varonis to your complement of security tools, schedule a one-on-one demo!
0 Comments