Brief overview of 4 NFATs

I was recently tasked with evaluating the functionality of the freeware version of Netwitness Investigator and other alternative network forensics analysis tools (NFATs) that emulate the functionality of the full version of Investigator, hopefully at a lower cost.

After compiling a list of close to 40 network capture and monitoring tools, I chose four to evaluate a little more closely: 

Netwitness Investigator (freeware version, Windows)

Xplico (open source, Linux)

Solera DeepSee (trial virtual appliance, proprietary OS w/ remote web interface), and

NetworkMiner (free edition, Windows)

I just saw this amazingly helpful list today of freely available forensic tools maintained by Forensic Control, but noticed that there is no network forensics tools category. I'll need to consolidate/condense my list a bit, but I'll see if I can post a table of similar format for network tools tomorrow.

Unfortunately, I didn't have a lot of time to delve deeply (nor do I claim to be anything close to an expert in network forensics), but I wanted to share my thoughts anyway...maybe I'll get a chance to revisit the tools later this week.

Netwitness Investigator (freeware version, Windows)

Netwitness Investigator is a powerful tool for network forensics, but the full version can be very expensive. They provide a great deal of functionality with the free version, but imported pcap files are limited to 1 GB.

Issues loading pcap data: I was not able to analyze a small (870 kb) pcap file – Netwitness gave the error, “not enough data to generate a timeline,” but it may have been some other problem with that particular pcap. DeepSee was also unable to read it (although it gave a different error).

Session reconstruction: Capture data is sorted into categories for easy drill-down, and then subcategorized, giving a summarized view of the data on one screen for quick analysis. The timeline feature shows a distribution of traffic over time and allows you to select a period of time  and “zoom in” for closer analysis. You can also create custom filters – Netwitness features “intellisense” filter action suggestions to assist with query generation.

Overall impression: Netwitness is still my favorite tool in terms of overall functionality and ease of use, despite the 1 GB pcap limit. Data organization and drill down takes some time to get used to due to the volume of data, but it is fairly easy to navigate after some use. The split-screen view of a list of payloads (after you’ve drilled down into a category) allows you to run through a set of data easily; when you have a payload in the preview pane, there is a toolbar at the top that allows you to switch easily between formats of the preview (hex, txt, mail, web, etc):

So for example, if you wanted to view an email, you can either view it with the “mail” reconstruction, or just view the raw text formatting if you want to see all the headers and the network handshake. If the mail has an attachment, there is a button that will give you the options of opening the file or saving it to disk. Furthermore, if there is audio content in the session, there is a button in the toolbar that will reconstruct it and play it back. This toolbar really makes digging through content much faster, and gives you the ability to see one piece of evidence from many different perspectives.

Perhaps my favorite feature, however, is the “file extract” functionality - it makes it very easy to extract and reconstruct files from a pcap. Netwitness comes with a set of predefined and categorized common file extensions to extract, and allows you to easily add your own category based on other extensions. Each category is filtered into its own folder to the destination you specify.

SplitCap and splitpcap.py

Regarding the 1 GB pcap file limit, I took a quick look at some methods for splitting pcap files. I found two tools, splitpcap.py and SplitCap. I’m giving up on splitpcap.py for now…it doesn’t appear to work with current architectures (I don't think it's been updated in awhile). There may be a simple python fix, but I'm not a python whiz, so it will take me some time to look into it.

I played a little with SplitCap though – it is a very powerful tool. It doesn’t really have an option to just split the file into 1 GB segments like splitpcap does, but if you knew what host or what kind of content you were looking for, it would be perfect for filtering out that pcap data for loading into Netwitness. Raw splitting by size of a capture is problematic anyway because the packets are not necessarily in order – they have to be reassembled by analysis tools such as Wireshark, Netminer, Netwitness, etc.

By default, SplitCap will take a pcap file and split it into multiple pcaps by protocol and then by host and port. So for example, for my 2.5 gb capture file, I ended up with 23,462 pcap files. You can shift-select multiple pcaps for import into Netwitness, but that is still kind of a pain. However, if you had some idea what you were looking for, you could easily weed out just that data for import. I won't go through all the available options here, but there is a good short video here that demonstrates some of the main features, like filtering by hostpair and restricting to application layer data.

Xplico (open source, Linux)

Part of my delay in making this post was wanting to revisit Xplico and get it installed on a real machine. I considered leaving it out completely, as I had so many problems with it, but it seems to be a very good tool. There is supposedly an Ubuntu 11.10 package, but the package manager says it has an unsatisfiable dependency on the python3.1-minimal package. I have not yet tried to build it from source – I just ran it off of the SecurityOnion live cd for my evaluation.

Unfortunately, the program stopped working properly several times when I was working with it, forcing me reboot to get it working again (I tried restarting the program and the service to no avail). I was suddenly unable to add a new case or session, although I could continue to analyze the pcap data I had already loaded.

But, I did have a chance to play with it a little bit, and I think it has a lot of potential. There is no limitation on pcap file import size, and I didn't have any issues loading pcap data (aside from the program malfunction I mentioned above).

Session reconstruction: Xplico seems to do a good job of reconstructing sessions from the capture data, although it was unable to categorize some data that was handled properly by other programs. For example, one pcap had several emails that were correctly identified by other tools, but were filed into the “undecoded” section by Xplico.

Session data is prefiltered into categories (Web, Mail, Voip, etc). Filters are based on simple keyword search. However, the Xplico components are modular and open source, so you can add custom functionality or change features as needed.

The initial summary of data gives a nice view of the distribution of network traffic types (http, email, etc.). However, the file extraction is one file at a time – there is no way to get all files of a certain type. I think the modularity sets it apart from the other tools, and make it worth spending some more time investigating...hopefully I'll have some time to reevaluate it soon.

Solera DeepSee (trial virtual appliance, proprietary OS w/ remote web interface)

DeepSee is a commercial product from Solera Networks built to be an enterprise-level network forensics solution. I downloaded a 30-day trial of their virtual appliance to see how it stacked up against the competition. I am certain I barely scratched the surface of the functionality provided by DeepSee, but I can share my first impressions in terms of usability.

There is no limitation on pcap file size, although I did have problems importing a wireless capture I took using Network Miner (this was the same pcap that Netwitness Investigator refused to load, so it could be that something with that particular file was corrupt). DeepSee gave me an error that the pcap “contained no ethernet data.” Thinking maybe it didn't handle wireless captures, I took another wireless capture using Wireshark instead, and it was able to read that with no errors.

Session reconstruction: DeepSee allows you to create filters on capture data as well as in the packet analyzer (using Wireshark syntax). It also provides a number of pre-defined common filters (such as “Email,” “IM Conversations,” etc.) and the ability to create custom summary widgets (although the queries to display cannot be customized; you must use one of their pre-defined queries). Still, I can see the value in creating your own custom summary screen of predefined “widgets” that pertain to the types of data in which you are commonly interested, giving you a good initial breakdown of the content.

The interface for accessing payload data is somewhat inefficient…you can drill down somewhat easily using their pre-defined filters, but those are somewhat limited. There is also no “batch” way to extract all files of a certain extension (or matching a filter/pattern); you have to select each file individually that you want to extract.

However, the timeline feature (showing the distribution of traffic over time and allowing you to jump to a particular time segment) is convenient, and the Artifacts view allows you to very easily see a preview of an individual artifact and quickly take action on it (download, analyze the packets, or view the “reputation” information).

DeepSee has some nice graphing and visualization features that I think some people would find very useful, although they don't top my list of important features. Their timeline feature is also nice, but I like the Netwitness timeline a little better, because you can select any size chunk of time you want to zoom in on and restrict your analysis to that. With DeepSee, you can click on a discrete time segment and see that data, but the interval is fixed.

One thing I do like about DeepSee is their artifact listing…if you click on one of the artifacts, it expands a small pane in the listing that gives you options specific to that piece of data. With one artifact expanded, you have the options to preview or download the file, analyze the packets, explore the root cause, or check its “reputation.” This last option is very interesting…if you see an artifact such as an executable in your listing, you can click on “reputation” and view the “Google safe browse,” ClamAV, and VirusTotal information for the file, and the ISC/SANS information for both hosts. I think the “root cause” information would be interesting to explore in terms of incident response applications, but I don’t have an appropriate data set to exercise it right now.

NetworkMiner (free edition, Windows)

NetworkMiner is another commercial NFAT tool for Windows, although they provide both a free version as well as a paid “Professional” version. Unlike the other tools, the main data display is “host centric,” grouping the data by network hosts rather than on a per-packet basis.

There are no limitations on the size of the pcap files you can import, and I didn't have any issues loading pcap data into the tool.

Session reconstruction: NetworkMiner provides comprehensive session recontruction, sorting the data into individual tabs based on content (Hosts, Files, Images, Credentials, etc.), which makes it easy to drill down into the content you need. Keyword search (either string or byte patterns) functionality is built in, but you have to reload the case files in order to execute a query you have constructed, which can be time consuming for large captures. It also allows you to perform a cleartext search using a loaded dictionary.

All files found in the capture are automatically reconstructed and saved to a folder on the hard drive. The files are sorted into folders by their originating host first, then the protocol. This makes it somewhat difficult to drill down to a particular file, and it is also not as easy to see all files of a certain extension. The program also does not separate assembled files by case, so all the data is saved to the same folder for all captures that are analyzed.

However, the Hosts and Credentials features I found very useful; the “Hosts” tab provides detailed information on every host found in the capture – ip address, operating system, hostname, etc. The “Credentials” tab will show any usernames and their associated passwords (if sent in cleartext), as well as the server logged onto.

Another convenient feature is the portability of NetworkMiner – it can be run directly off of a flash drive. I did not get a chance to give the professional version a test drive, but at only 500 EUR, it is far more affordable than the other commercial tools. As a side note, the NetworkMiner wiki also has a very useful collection of links to publicly available pcap files for testing and analysis.

Although my favorite tool to work with overall was Netwitness Investigator, NetworkMiner has some unique functionality that I am sure I will use again. I am also looking forward to getting Xplico up and running on my machine so I can play with it a little more and explore the expansion possibilities of the individual modules.