I was recently tasked with evaluating the functionality of the freeware
version of Netwitness Investigator and other alternative network forensics
analysis tools (NFATs) that emulate the functionality of the full version of
Investigator, hopefully at a lower cost.
After compiling a list of close to 40 network capture and monitoring tools, I chose four to evaluate a
little more closely:
Netwitness Investigator (freeware version, Windows)
Xplico (open source, Linux)
Solera DeepSee (trial virtual appliance, proprietary OS w/ remote web interface), and
NetworkMiner (free edition, Windows)
I just saw this amazingly helpful list today of freely available forensic tools maintained by Forensic Control, but noticed
that there is no network forensics tools category. I'll need to
consolidate/condense my list a bit, but I'll see if I can post a table of
similar format for network tools tomorrow.
Unfortunately, I didn't have a lot of time to delve deeply (nor do I claim to be anything close to an expert in network forensics), but I wanted
to share my thoughts anyway...maybe I'll get a chance to revisit the tools
later this week.
Netwitness Investigator (freeware version, Windows)
Netwitness Investigator is a powerful tool for network forensics, but
the full version can be very expensive. They provide a great deal of
functionality with the free version, but imported pcap files are limited to 1
GB.
Issues loading pcap data: I was not able to analyze a small (870
kb) pcap file – Netwitness gave the error, “not enough data to generate a
timeline,” but it may have been some other problem with that particular pcap.
DeepSee was also unable to read it (although it gave a different error).
Session reconstruction: Capture data is sorted into categories
for easy drill-down, and then subcategorized, giving a summarized view of the
data on one screen for quick analysis. The timeline feature shows a
distribution of traffic over time and allows you to select a period of
time and “zoom in” for closer analysis.
You can also create custom filters – Netwitness features “intellisense” filter
action suggestions to assist with query generation.
Overall impression: Netwitness is still
my favorite tool in terms of overall functionality and ease of use, despite the
1 GB pcap limit. Data organization and drill down takes some time to get used
to due to the volume of data, but it is fairly easy to navigate after some use.
The split-screen view of a list of payloads (after you’ve drilled down into a
category) allows you to run through a set of data easily; when you have a
payload in the preview pane, there is a toolbar at the top that allows you to
switch easily between formats of the preview (hex, txt, mail, web, etc):
So for example, if you wanted to view an
email, you can either view it with the “mail” reconstruction, or just view the
raw text formatting if you want to see all the headers and the network
handshake. If the mail has an attachment, there is a button that will give you
the options of opening the file or saving it to disk. Furthermore, if there is
audio content in the session, there is a button in the toolbar that will
reconstruct it and play it back. This toolbar really makes digging through
content much faster, and gives you the ability to see one piece of evidence
from many different perspectives.
Perhaps my favorite feature, however, is the
“file extract” functionality - it makes it very easy to extract and reconstruct
files from a pcap. Netwitness comes with a set of predefined and categorized
common file extensions to extract, and allows you to easily add your own
category based on other extensions. Each category is filtered into its own
folder to the destination you specify.
SplitCap and splitpcap.py
Regarding the 1 GB pcap file limit, I took a
quick look at some methods for splitting pcap files. I found two tools, splitpcap.py and SplitCap. I’m giving up on splitpcap.py for now…it doesn’t
appear to work with current architectures (I don't think it's been updated in
awhile). There may be a simple python fix, but I'm not a python whiz, so it
will take me some time to look into it.
I played a little with SplitCap though – it is a very powerful tool. It
doesn’t really have an option to just split the file into 1 GB segments like
splitpcap does, but if you knew what host or what kind of content you were
looking for, it would be perfect for filtering out that pcap data for loading
into Netwitness. Raw splitting by size of a capture is problematic anyway
because the packets are not necessarily in order – they have to be reassembled by
analysis tools such as Wireshark, Netminer, Netwitness, etc.
By default, SplitCap will take a pcap file and split it into multiple
pcaps by protocol and then by host and port. So for example, for my 2.5 gb
capture file, I ended up with 23,462 pcap files. You can shift-select multiple
pcaps for import into Netwitness, but that is still kind of a pain. However, if
you had some idea what you were looking for, you could easily weed out just
that data for import. I won't go through all the available options here, but there is a good short video here that demonstrates some of the main features, like filtering by hostpair and restricting to application layer data.
Xplico (open source, Linux)
Part of my delay in making this post was wanting to revisit Xplico and
get it installed on a real machine. I considered leaving it out completely, as
I had so many problems with it, but it seems to be a very good tool. There is
supposedly an Ubuntu 11.10 package, but the package manager says it has an
unsatisfiable dependency on the python3.1-minimal package. I have not yet tried
to build it from source – I just ran it off of the SecurityOnion live cd for my evaluation.
Unfortunately, the program stopped working properly several times when I
was working with it, forcing me reboot to get it working again (I tried
restarting the program and the service to no avail). I was suddenly unable to
add a new case or session, although I could continue to analyze the pcap data I
had already loaded.
But, I did have a chance to play with it a little bit, and I think it
has a lot of potential. There is no limitation on pcap file import size, and I
didn't have any issues loading pcap data (aside from the program malfunction I
mentioned above).
Session reconstruction: Xplico seems to do a good job of
reconstructing sessions from the capture data, although it was unable to
categorize some data that was handled properly by other programs. For example,
one pcap had several emails that were correctly identified by other tools, but
were filed into the “undecoded” section by Xplico.
Session data is prefiltered into categories (Web, Mail, Voip, etc).
Filters are based on simple keyword search. However, the Xplico components are
modular and open source, so you can add custom functionality or change features
as needed.
The initial summary of data gives a nice view of the distribution of
network traffic types (http, email, etc.). However, the file extraction is one
file at a time – there is no way to get all files of a certain type. I think
the modularity sets it apart from the other tools, and make it worth spending
some more time investigating...hopefully I'll have some time to reevaluate it
soon.
Solera DeepSee (trial virtual appliance, proprietary OS w/ remote web interface)
DeepSee is a commercial product from Solera Networks built
to be an enterprise-level network forensics solution. I downloaded a 30-day
trial of their virtual appliance to see how it stacked up against the
competition. I am certain I barely scratched the surface of the functionality
provided by DeepSee, but I can share my first impressions in terms of
usability.
There is no limitation on pcap file size, although I did
have problems importing a wireless capture I took using Network Miner (this was
the same pcap that Netwitness Investigator refused to load, so it could be that
something with that particular file was corrupt). DeepSee gave me an error
that the pcap “contained no ethernet data.” Thinking maybe it didn't handle
wireless captures, I took another wireless capture using Wireshark instead, and
it was able to read that with no errors.
Session reconstruction: DeepSee allows you to create
filters on capture data as well as in the packet analyzer (using Wireshark
syntax). It also provides a number of pre-defined common filters (such as
“Email,” “IM Conversations,” etc.) and the ability to create custom summary
widgets (although the queries to display cannot be customized; you must use one
of their pre-defined queries). Still, I can see the value in creating your own
custom summary screen of predefined “widgets” that pertain to the types of data
in which you are commonly interested, giving you a good initial breakdown of
the content.
The interface for accessing payload data is somewhat
inefficient…you can drill down somewhat easily using their pre-defined filters,
but those are somewhat limited. There is also no “batch” way to extract all
files of a certain extension (or matching a filter/pattern); you have to select
each file individually that you want to extract.
However, the timeline feature (showing the distribution of
traffic over time and allowing you to jump to a particular time segment) is
convenient, and the Artifacts view allows you to very easily see a preview of
an individual artifact and quickly take action on it (download, analyze the
packets, or view the “reputation” information).
DeepSee has some nice graphing and visualization features
that I think some people would find very useful, although they don't top my
list of important features. Their timeline feature is also nice, but I like the
Netwitness timeline a little better, because you can select any size chunk of
time you want to zoom in on and restrict your analysis to that. With DeepSee,
you can click on a discrete time segment and see that data, but the interval is
fixed.
One thing I do like about DeepSee is their artifact
listing…if you click on one of the artifacts, it expands a small pane in the
listing that gives you options specific to that piece of data. With one
artifact expanded, you have the options to preview or download the file,
analyze the packets, explore the root cause, or check its “reputation.” This
last option is very interesting…if you see an artifact such as an executable in
your listing, you can click on “reputation” and view the “Google safe browse,”
ClamAV, and VirusTotal information for the file, and the ISC/SANS information
for both hosts. I think the “root cause” information would be interesting to
explore in terms of incident response applications, but I don’t have an
appropriate data set to exercise it right now.
NetworkMiner (free edition, Windows)
NetworkMiner is another commercial NFAT tool for Windows,
although they provide both a free version as well as a paid “Professional”
version. Unlike the other tools, the main data display is “host centric,”
grouping the data by network hosts rather than on a per-packet basis.
There are no limitations on the size of the pcap files you
can import, and I didn't have any issues loading pcap data into the tool.
Session reconstruction: NetworkMiner provides
comprehensive session recontruction, sorting the data into individual tabs
based on content (Hosts, Files, Images, Credentials, etc.), which makes it easy
to drill down into the content you need. Keyword search (either string or byte
patterns) functionality is built in, but you have to reload the case files in
order to execute a query you have constructed, which can be time consuming for
large captures. It also allows you to perform a cleartext search using a loaded
dictionary.
All files found in the capture are automatically
reconstructed and saved to a folder on the hard drive. The files are sorted
into folders by their originating host first, then the protocol. This makes it
somewhat difficult to drill down to a particular file, and it is also not as
easy to see all files of a certain extension. The program also does not
separate assembled files by case, so all the data is saved to the same folder for all
captures that are analyzed.
However, the Hosts and Credentials features I found very
useful; the “Hosts” tab provides detailed information on every host found in
the capture – ip address, operating system, hostname, etc. The “Credentials”
tab will show any usernames and their associated passwords (if sent in
cleartext), as well as the server logged onto.
Another convenient feature is the portability of
NetworkMiner – it can be run directly off of a flash drive. I did not get a
chance to give the professional version a test drive, but at only 500 EUR, it
is far more affordable than the other commercial tools. As a side note, the NetworkMiner wiki also has a very useful collection of links to publicly available pcap files for testing and analysis.
Although my favorite
tool to work with overall was Netwitness Investigator, NetworkMiner has some
unique functionality that I am sure I will use again. I am also looking forward to
getting Xplico up and running on my machine so I can play with it a little more
and explore the expansion possibilities of the individual modules.
