Packet Capture in Production with tcpdump: A Runbook

Packet capture (pcap) carries two opposite risks in production: either there’s no evidence at all, or “let’s capture everything” creates new problems on disk, CPU, and privacy fronts. The objective of this runbook is to deliver maximum diagnostic value with minimum risk.

When is pcap really needed?

pcap is highly valuable in cases like:

• TLS handshake stalls / connection resets / strange timeouts
• MTU/PMTUD suspicion (large payloads getting corrupted)
• DNS anomalies (wrong answers, latency, NXDOMAIN waves)
• Retransmission / packet loss / jitter analysis

Principles of safe pcap in production

1. Narrow the scope: filter by host/port/proto
2. Shorten the duration: 30–120 seconds is enough for most triage
3. Use a ring buffer: rotating small files instead of one huge file
4. Tune snaplen: in most cases the full payload isn’t required
5. Own the privacy: pcap can carry sensitive data

Quick start: a safe ring buffer command

Example: capture TCP traffic going to a specific destination for 60 seconds, splitting into 10 MB chunks:

sudo mkdir -p /var/tmp/pcap
sudo tcpdump -i any -nn \
  -s 256 \
  -C 10 -W 12 \
  -w /var/tmp/pcap/incident-%Y%m%d-%H%M%S.pcap \
  'host <hedef-ip> and tcp'

• -s 256: enough header plus a bit of payload for most diagnostics (raise as needed)
• -C 10 -W 12: rotating archive of about 120 MB max
• -i any: handy when you don’t know the right interface (pick a specific interface under high load)

Practical filter recipes

1) TLS/HTTPS handshake (443)

sudo tcpdump -i eth0 -nn -s 256 -w /var/tmp/pcap/tls.pcap \
  'host <hedef-ip> and tcp port 443'

2) DNS issue (53/udp)

sudo tcpdump -i eth0 -nn -s 256 -w /var/tmp/pcap/dns.pcap \
  'host <dns-ip> and udp port 53'

3) PMTUD signal (ICMP)

sudo tcpdump -i any -nn -s 256 -w /var/tmp/pcap/pmtu.pcap \
  'icmp or icmp6'

How do you pick the “right” snaplen?

Rules of thumb:

• MTU/PMTUD, TCP reset, SYN/ACK analysis → -s 128 / -s 256 is enough
• If you need application-layer hints (some protocols) → consider -s 1024
• A full payload only with a clear justification → -s 0 (and definitely shrink the time/scope)

Pcap handover: a frequently made mistake

You collected the pcap but then nobody knows “who has it, where it is, and how long it’s retained.” A simple discipline:

1. Compress the files:

sudo gzip -9 /var/tmp/pcap/*.pcap

2. Take hashes (for evidence integrity):

sha256sum /var/tmp/pcap/*.gz | tee /var/tmp/pcap/SHA256SUMS.txt

3. Restrict access:

sudo chmod 600 /var/tmp/pcap/*

4. Record the following on the ticket:

• Capture time window (UTC preferred)
• Target filter (host/port)
• Snaplen and ring buffer parameters
• File path and hashes

Wrap-up

Done correctly, packet capture dramatically lowers MTTR; done incorrectly, it adds a brand-new layer of risk in production. The essence of this runbook: narrow filter + short duration + ring buffer + privacy discipline. Evidence collection is as much an operational process as it is a technical one.