ddump is a small utility to run multiple network captures on multiple targets
and live-merging the captured streams. This is useful when investigating network
issues in a distributed system.
The main process runs as a command line program and outputs the merged stream on stdout for easy integration.
The worker processes run as a long-running HTTP server, suitable for integration behind a streaming (i.e. non buffering) HTTP reverse proxy. Under the hood, it automatically filters out its own traffic from the captures to avoid capture amplification.
ddump can authenticate using Basic Auth and Mutual TLS, the worker itself
does not support authentication nor authorization. This task is left to a reverse
sudo apt install libpcap-dev
# Build make
Run & test the worker:
# Allow running without root sudo setcap cap_net_raw=eip ddump-worker # Run ./ddump-worker # Query curl http://localhost:8475/capture -vo ./full-capture.pcap curl http://localhost:8475/capture?interface=eth0&filter=tcp%20port%2080 -vo ./ethernet-http-capture.pcap curl http://localhost:8475/capture\?filter\=icmp -sN | tcpdump -vnl -r-
Run & test the CLI:
./ddump \ "http://0.0.0.0:8475/capture?filter=icmp%20and%20host%22.214.171.124" \ "http://0.0.0.0:8475/capture?filter=icmp%20and%20host%126.96.36.199" \ | tcpdump -vnlr-
The command above will start 2 capture streams on a locally running worker:
- Ping to/from 188.8.131.52
- Ping to/from 184.108.40.206
The command then outputs to stdout the merged capture stream in a format suitable
for ingestion by
tcpdump (as demonstrated in the example).
ddump process supports basic Auth as well as mutual TLS
- Basic Auth: To enable Basic Auth, set the “user:[email protected]” on
the target URL. Please note that this will be visible in to any process
/procaccess. In scenarios where security is critical, please use mutual TLS authentication instead.
- Server TLS authentication: By default, when using HTTPS,
ddumpvalidates the server certificate using the system’s CA bundle or the bundle targeted by the
SSL_CERT_DIRstandard environment variables. The CA to use can be customized using the
- Client TLS authentication: To enable mutual TLS authentication, set
--keyparameters to respectively the target PEM encoded client certificate full chain and the key associated to the leaf certificate.
ddump-worker process does not support TLS nor Basic Auth by
itself. Instead, it is strongly advised to run the worker behind a reverse
proxy. Additionally, the proxy could parse and filter the client’s provided
filter for additional security. This is out of the scope of this project.
ddump process does not require any specific privileges. It only
connects to the workers over HTTP(S), merges the streams and outputs the merged
stream to stdout.
ddump-worker process requires
CAP_NET_RAW. This basically
gives full access to all network packets reaching any of the network interfaces
supported by Linux’s packet capture. Please note that this includes packets that
would eventually be dropped by the firewall as the capture hook is before the
filtering. It is strongly advised to use the capabilities and NEVER run as root.
The capabilities can be configured on the binary using:
sudo setcap cap_net_raw=eip ddump-worker
The command line is of the form:
./ddump [ARGS] TARGET_URL_1 TARGET_URL_2 ... TARGET_URL_N
Where target URLs are of the form
PROTOCOL: One of
USER: Optional username, if going through and authenticating reverse proxy
PASSWORD: Optional password, if going through and authenticating reverse proxy
HOST: IP address or domain name of the target capture agent
PORT: Optional TCP port, if the capture agent listens on a non-standard port
FILTER: Optional url-encoded capture filter
And the following flags are supported:
-ca: Path to the trusted CA certificate bundle. (default: Use system CA)
-cert: Path to the client certificate to use for mutual TLS authentication. (default: disable mTLS)
-key: Path to the client key associated to the cert. (default: disable mTLS)
The main output stream is the merged pcap stream. Messages and errors are sent to stderr.
-listen: Address to listen on. It may be an “
<address>:<port>” or “
Start a network capture.
filter: Tcpdump BPF filter. (default: no filter)
Note: There is no “interface” parameter. This is intentional. Indeed, to merge
multiple pcap streams, all streams must have the same type. This implies the use
of a generic link layer. This layer could be one of Linux’s
While SLL2 would have allowed per-interface filters, it is not (yet) supported by the
- Body: Raw pcap stream