Why does tcpdump not recognise piped input?

Question

1. I obtained a packet capture file:

   $ sudo tcpdump -w file.pcap
   

2. I can then analyse that file:

   $ tcpdump -r file.pcap
   

   I can even use file redirection to read from standard input:

   $ tcpdump -r - < file.pcap
   

3. However, I cannot use a pipe to read from standard input:

   $ cat file.pcap | tcpdump -r -
   tcpdump: unknown file format
   

   This should work. What's going on?

OS X Yosemite 10.10
tcpdump version 4.3.0 -- Apple version 59
libpcap version 1.5.3 - Apple version 47

---

^{I'm actually trying to sudo tcpdump -w - | tee file.pcap | tcpdump -r -, which gives rise to the same problem—however it appears to arise with any form of piped input.}

Answer 1

This is still an issue with OS X Mojave 10.14.2 (libpcap 1.0.1 Apple 79.200.4, tcpdump 4.9.2 Apple 83.200.2). It's ultimately a bug in Apple's patches to libpcap and tcpdump.

In addition to using upstream tcpdump (which is available in Homebrew), you could also:

• Use tshark (part of Wireshark), which works similarly to tcpdump.

• Write the packets to a temporary file.

• Prefix the stream with 4 null bytes:

  sudo tcpdump -i en0 -w - | cat <(printf "\0\0\0\0") - | tcpdump -nvr -

Cause (and why inserting nulls works)

Apple has patched their tcpdump such that it tries to read files as PCAP-NG format first:

#else /* __APPLE__ */
        pd = pcap_ng_open_offline(RFileName, ebuf);

They also added a -P option to emit packets in pcapng format.

pcap_ng_open_offline ends up in pcap_fopen_offline_internal(..., isng=1). This flow calls pcap_ng_check_header early (see if (isng) in pcap_fopen_offline_internal), and on failure, jumps into the bad: label.

That bad: label attempts to fseeko to seek the file back to the start:

 bad:
    fseeko(fp, offset, SEEK_SET);
    if (p != NULL)
        free(p);
    return (NULL);

But you can't seek a FIFO, so we're still reading from the same spot! fseeko would return an error, but this error is ignored.

On pcap_ng_open_offline returning an error, Apple's patch then calls pcap_open_offline, which follows a similar control flow to normal pcap. However, because the file pointer on the FIFO is still forward by 4 bytes, the next set of headers have the fifo's magics, and so tcpdump gives up.

You can work around this by inserting 4 extra NULL bytes at the start of your pipe, with a little shell-script trickery:

tcpdump -i en0 -w - | cat <(printf "\0\0\0\0") - | tcpdump -nvr -

This means that when fseeko fails, you'll be in the correct spot in the FIFO for the "actual" magic.

However, there are still some limitations:

• tcpdump -P (Apple-specific pcapng flag) still doesn't work.
• These files are no longer valid pcap files. If you wanted to read it with another tool (or even a non-Apple tcpdump), you'll need to tee the output before cat.

This also works with tshark (which can output pcapng, and doesn't work as a source without this work-around):

tshark -F pcapng -w - | cat <(printf "\0\0\0\0") - | tcpdump -nvr -

However, I wouldn't suggest putting this work-around in anything long term. It will break once Apple fixes this bug, or if your users have a non-Apple tcpdump on OSX (such as with Homebrew).

Answer 2

I have the same problem on Mavericks (10.9.5). It's probably not the best answer but I downloaded the open source version of tcpdump (tcpdump-4.6.2) and put it in my path before the system version and it works with pipes. Of course, you lose the extra Apple-specific functionality.