VERMONT - VERsatile MONitoring Tool

Vermont is an open-source software toolkit for the creation and processing of network flow data, based on monitored Internet packet data. See the Wiki for full details of all its features.

REQUIREMENTS

VERMONT has been tested on Linux and FreeBSD systems.

For compilation, GNU C/C++ compiler and standard libraries are required, as well as the following Ubuntu/Debian packages (or equivalent packages of other Linux distributions):

• cmake
• libboost-filesystem-dev
• libboost-regex-dev
• libboost-test-dev
• libboost-thread-dev
• libxml2-dev
• libpcap-dev
• libsctp-dev (if not available, disable cmake option SUPPORT_SCTP)

The following packages are optional:

• cmake-curses-gui (ccmake, interactive user interface of cmake)
• libpq-dev (for PostgreSQL support) ==> cmake option SUPPORT_POSTGRESQL
• libmysqlclient-dev (for MySQL support) ==> cmake option SUPPORT_MYSQL
• libgsl-dev (for connection-based sampling with Bloom filters) ==> cmake option USE_GSL
• libczmq-dev (for receiving IPFIX reports over ZMQ) ==> cmake option SUPPORT_ZMQ

For DTLS support, OpenSSL 1.0.0 is required.

BUILDING AND INSTALLATION

This project uses cmake for setting platform- and user-specific compile options. In order to generate the Makefile for actual compilation, you need to call in the root of the source directory:

$ cmake .

In order to change the default compile options, use:

$ cmake -D OPTION1=value1 -D OPTION2=value2 ...

To get a list of the most important options, call:

$ cmake -LH

As a user-friendly alternative, you can use the interactive user interface. Please note that this requires the package cmake-curses-gui, if you are using Ubuntu/Debian.

$ ccmake .

If some libraries are installed in custom directories, use:

$ cmake -D CMAKE_PREFIX_PATH=/custom/directory1:/custom/directory2

After successfully generating the Makefile with cmake, start the compilation with:

$ make

Although not strictly necessary, VERMONT binaries and data files can be copied to the usual install location by running:

$ make install 

BUILDING WITH DTLS-OVER-UDP SUPPORT

VERMONT's DTLS support is based on OpenSSL version 1.0.0. OpenSSL 1.1.0 is not currently supported.

In order to compile VERMONT with DTLS-over-UDP support set the following option:

$ cmake -DSUPPORT_DTLS=YES

If CMake does not find OPENSSL you can explicitly specify the include and library paths:

cmake -DSUPPORT_DTLS=YES -DCMAKE_INCLUDE_PATH=/path/to/openssl/include -DCMAKE_LIBRARY_PATH=/path/to/openssl/lib

BUILDING WITH DTLS-OVER-SCTP SUPPORT

At the time of writing (July 2010), DTLS over SCTP can be used on FreeBSD only! This is due to the fact that FreeBSD is currently the only OS which supports the SCTP-AUTH extension (see RFC 4895) which is required by DTLS.

The current version of OpenSSL (1.0.0a) has no native support for SCTP. You have to download additional patches from

http://sctp.fh-muenster.de/

and apply them to the OpenSSL sourcese before building OpenSSL. Make sure that the patches fit to your local version of OpenSSL. Otherwise, you might need to manually adapt the patch files.

Also, make sure to add the command line argument "sctp" when running OpenSSL's ./config to build SCTP support into OpenSSL.

In order to compile VERMONT with DTLS-over-SCTP support, you need to run cmake with the following three options:

-DSUPPORT_SCTP -DSUPPORT_DTLS -DSUPPORT_DTLS_OVER_SCTP

In addition, you need to indicate the include and library paths to your patched version of OpenSSL as explained for DTLS-over-UDP.

USAGE AND CONFIGURATION

In order to run VERMONT, a configuration file is needed which specifies the modules to be used and their parameters:

$ ./vermont -f <config-file>

Example configuration files can be found in configs/. Documentation of the available modules and their configuration parameters can be found at https://github.com/tumi8/vermont/wiki/Moduleconfiguration

Use Ctrl-C to stop VERMONT. If VERMONT does not exit properly, enter Ctrl-C for a second time.

OPERATION AS COLLECTOR: TUNING SOCKET RECEIVE BUFFER

VERMONT can be used as an IPFIX/PSAMP and NetFlow.v9 collector. As the incoming IPFIX/PSAMP/NetFlow messages usually arrive in bursts, losses may occur due to insufficient buffer space.

As a solution, the socket receive buffer can be increased. To check the current settings, use the following system calls on Linux systems with /proc file system:

$ cat /proc/sys/net/core/rmem_default
$ cat /proc/sys/net/core/rmem_max

In order to configure a new value X (in bytes), call:

$ sysctl -w net/core/rmem_default=X
$ sysctl -w net/core/rmem_max=X

If you want Vermont to use a different buffer size than the default one, you can specify it using the <buffer> directive in the <listener> section.

OPTIMIZED PACKET CAPTURING WITH PCAP

To reduce the number of times packets need to be copied on their way from the network interface card to the user space (i.e., VERMONT), we recommend the utilization of pcap library 1.0.0 or higher.

For earlier versions of pcap, the pcap-mmap patch can be applied to improve the performance: http://public.lanl.gov/cpw/

EFFECTS OF RECEIVE OFFLOAD MECHANISMS

Several mechanisms have been implemented in modern network interface cards, drivers, and kernels to offload common functions from the protocol stack and the application. One particular focus is on TCP segmentation and reassembly.

Receive offload mechanisms aim at reassembling subsequent TCP segments into a single large packet before passing it to the IP/TCP protocol stack and finally to the application. In the Linux kernel, this is done by generic receive offload (GRO) if the network interface card and the driver support NAPI. Latest Intel 10GE controllers (e.g., 82599) support receive side coalescing (RSC) which performs TCP reassembly in hardware.

If any receive offload mechanism is enabled, VERMONT (like any other pcap-based application) does not observe the actually captured TCP packets but the reassembled ones. One consequence is that packet counts of flows will be smaller than the true number of packets.

In order to avoid such distortions, all receive offload mechanisms need to be disabled. In the case of GRO (and the older LRO), this can be done with ethtool. The following call returns a list of the current status of all offload mechanisms for interface :

$ ethtool -k <dev>

If GRO is not shown, you probably need to install a newer version of ethtool. To disable GRO (and LRO), execute:

$ ethtool -K <dev> gro off
$ ethtool -K <dev> lro off

Hardware-based RSC can be deactivated at compile time of the driver as explained here: http://downloadmirror.intel.com/19004/eng/README-2.0.72.4.txt