Draft: WBF: Wireshark Binary Format

Dev AMrequested to merge
Dev_AM/wireshark:wbf into master

Add WBF (Wireshark Binary Format) Export Support

Purpose

WBF is a binary export format for packet dissection data designed to solve three critical problems with existing text-based formats (PDML, JSON, EK):

  1. Speed: Text formats are slow due to XML/JSON encoding and string formatting overhead
  2. Size: Text formats produce massive files (50 MB capture → 8 GB PDML)
  3. Completeness: No existing format preserves buffer references (byte-level packet data offsets)

Key Features

Format Design

  • Field Dictionary: Field names stored once, referenced by index (eliminates redundancy)
  • Native Types: 18+ binary types (UINT32, IPv6, ABSOLUTE_TIME, etc.) avoid string conversion
  • LEB128 Encoding: Standard variable-length integers for compact storage
  • Buffer References: Every field includes byte offset/length into original packet data (unique to WBF)
  • Complete Metadata: Preserves colors, flags (hidden/generated/marked/ignored), timestamps

Export Options

  • tshark -r input.pcapng -T wbf -w output.wbf - Full export with all features
  • --wbf-no-buffers - Omit buffer references (30-40% smaller, faster)
  • --wbf-no-columns - Omit UI column data (minimal file size)

Performance

Measured on realistic captures (IPv4/UDP and HTTP/IPv6 traffic):

Metric vs PDML
Export Speed 3.3-4.2x faster
File Size (uncompressed) 4.4-7.1x smaller
File Size (gzip -6) 2.2-4.0x smaller

Performance gains from: no XML/JSON encoding, binary types, field dictionary, efficient VarInt encoding.

Implementation

Core Changes:

  • epan/print.c - WBF export with field dictionary and binary serialization
  • file.c - 1 MB output buffer for optimal I/O
  • tshark.c - Command-line options (-T wbf, --wbf-no-buffers, --wbf-no-columns)

Supporting Tools:

  • tools/wbf_reader.py - Python library + CLI reader
  • tools/WbfReader.cs - C# library for .NET
  • tools/wbf_viewer.py - Qt GUI viewer (Wireshark-style tree navigation)
  • doc/README.wbf-output - Complete format specification
  • doc/man_pages/tshark.adoc - Updated documentation

Backward Compatibility

No changes to existing formats. WBF is purely additive via -T wbf option.

In summary: WBF enables practical export of large captures by being 3-4x faster and 4-7x smaller than PDML while preserving complete dissection data including unique buffer references for byte-level analysis.

Merge request reports