Trace Stream Proxy

Trace Stream Proxy is a TCP proxy and replay tool for protocols that frame every message as:

<N-byte big-endian payload length><payload bytes>

The same framing is used in both directions. The default header length is 8 bytes, and it can be changed with --header-len <bytes> on the proxy, replay, client, and server commands. Header lengths must be between 1 and 16 bytes. Payloads are capped at 16 MiB by default; change that with --max-payload-len <bytes>.

Paper

A short paper is available as Trace Stream Proxy: A Minimal Framed TCP Record-and-Replay Tool.

The main conclusion is that a focused record-and-replay workflow for length-prefixed TCP protocols can stay small and predictable when it treats framing as the core abstraction. Trace Stream Proxy records decoded message payloads instead of raw TCP chunks, bounds header and payload allocation, streams replay files per connection, and restricts recording to one active client so the v1 trace format remains unambiguous.

Build

cargo build --bins --release

Build a docker image

docker build -t ts_proxy .

The container is configured with environment variables. Proxy mode is the default:

docker run --rm -p 9000:9000 \
  -e TS_PROXY_LISTEN=0.0.0.0:9000 \
  -e TS_PROXY_TARGET=host.docker.internal:8000 \
  -e TS_PROXY_HEADER_LEN=8 \
  -e TS_PROXY_MAX_PAYLOAD_LEN=16777216 \
  -e TS_PROXY_FLUSH_RECORDING=false \
  -e TS_PROXY_RECORD=/recordings/session.tsv \
  -v "$PWD:/recordings" \
  ts_proxy

Replay mode uses the same image:

docker run --rm -p 9001:9001 \
  -e TS_PROXY_MODE=replay \
  -e TS_PROXY_LISTEN=0.0.0.0:9001 \
  -e TS_PROXY_RECORD=/recordings/session.tsv \
  -e TS_PROXY_TIMING=original \
  -e TS_PROXY_HEADER_LEN=8 \
  -e TS_PROXY_MAX_PAYLOAD_LEN=16777216 \
  -v "$PWD:/recordings" \
  ts_proxy

Supported container settings:

• TS_PROXY_MODE: proxy or replay; defaults to proxy.
• TS_PROXY_LISTEN: listener address; defaults to 0.0.0.0:9000.
• TS_PROXY_TARGET: target host:port; required in proxy mode.
• TS_PROXY_RECORD: recording path; optional in proxy mode and required in replay mode.
• TS_PROXY_FLUSH_RECORDING: flush after every recorded frame in proxy mode; defaults to false.
• TS_PROXY_TIMING: original or none; used in replay mode.
• TS_PROXY_HEADER_LEN: frame header length in bytes; defaults to 8.
• TS_PROXY_MAX_PAYLOAD_LEN: maximum payload size in bytes; defaults to 16777216.

Run a proxy

cargo run --bin ts_proxy -- proxy --listen 127.0.0.1:9000 --target 127.0.0.1:8000 --record session.tsv --header-len 8 --max-payload-len 16777216

This listens on 127.0.0.1:9000, connects each client to 127.0.0.1:8000, forwards complete framed messages, and records the decoded payloads to session.tsv.

sequenceDiagram
  participant C as Client
  participant P as ts_proxy
  participant S as Server
  participant R as session.tsv
  C->>P: framed c2s
  P->>S: framed c2s
  S->>P: framed s2c
  P->>C: framed s2c
  P->>R: record payloads in HEX format

Loading

This flow records session.tsv as the output of the proxy run, containing <elapsed_us>\t<direction>\t<hex_payload> entries for both c2s and s2c. Recording writes are not flushed after every frame by default. Add --flush-recording when you prefer immediate file visibility over throughput. Because the recording format represents one linear conversation, proxy mode allows only one active client while --record is enabled. Additional clients are closed until the active recorded client disconnects.

The --record argument is optional:

cargo run --bin ts_proxy -- proxy --listen 127.0.0.1:9000 --target 127.0.0.1:8000 --header-len 8 --max-payload-len 16777216

Try the example client and server

Terminal 1:

cargo run --bin simple_server -- --listen 127.0.0.1:8000 --header-len 8 --max-payload-len 16777216

Terminal 2:

cargo run --bin ts_proxy -- proxy --listen 127.0.0.1:9000 --target 127.0.0.1:8000 --record session.tsv --header-len 8 --max-payload-len 16777216

Terminal 3:

cargo run --bin simple_client -- --connect 127.0.0.1:9000 --message message --count 2 --header-len 8 --max-payload-len 16777216

The client should print:

echo: 1_message
echo: 2_message

There is also a script that runs the full record-and-replay flow and then uses nc as the replay client:

./scripts/test_replay_with_nc.sh

To test another header length, set HEADER_LEN:

HEADER_LEN=10 ./scripts/test_replay_with_nc.sh

Replay A Recording

cargo run --bin ts_proxy -- replay --listen 127.0.0.1:9001 --record session.tsv --header-len 8 --max-payload-len 16777216

Replay starts a TCP server on 127.0.0.1:9001. For each c2s frame in the recording, it waits for the client to send the same framed payload. For each s2c frame, it sends the recorded payload back to the client using the same configured length header.

sequenceDiagram
  participant C as Replay Client
  participant P as ts_proxy replay
  participant R as session.tsv
  C->>P: framed c2s
  P->>R: read next recording entry
  Note right of R: session.tsv is replay input
  R-->>P: matching s2c payload
  P->>C: framed s2c

Loading

This flow uses session.tsv as the replay input. The replay server validates that the file can be opened at startup, then each client connection streams the file line by line. That keeps memory usage bounded and lets large recordings start replaying without being fully loaded first. The file drives the expected c2s sequence plus the emitted s2c responses.

By default, replay preserves the original timing between recorded frames:

cargo run --bin ts_proxy -- replay --listen 127.0.0.1:9001 --record session.tsv --timing original --header-len 8 --max-payload-len 16777216

To replay without delays:

cargo run --bin ts_proxy -- replay --listen 127.0.0.1:9001 --record session.tsv --timing none --header-len 8 --max-payload-len 16777216

Recording format

Recordings are tab-separated text files:

<elapsed_us>    <direction>    <hex_payload>

Directions are:

• c2s: client to server
• s2c: server to client

Example:

# ts_proxy recording v1
123     c2s     68656c6c6f
456     s2c     776f726c64

The recorded payload is the message body only. The hex payload must have an even number of digits, so it is always a multiple of 2 characters. The frame header is decoded while recording and regenerated while proxying or replaying.

Frame example

The payload hello with the default 8-byte header is sent as a big-endian length followed by the payload bytes:

00 00 00 00 00 00 00 05 68 65 6c 6c 6f

With a 1-byte header, the same payload is:

05 68 65 6c 6c 6f

With a 10-byte header, the same payload is:

00 00 00 00 00 00 00 00 00 05 68 65 6c 6c 6f

Tests

cargo test

Benchmark

The client_send benchmark measures the framed client send path with a local Tokio duplex stream. It sends 100,000 messages with a 10 KiB payload each while a reader task drains and validates every frame.

Run it with:

cargo bench --bench client_send

Result from this machine:

client_send_100k_x_10k
messages: 100000
payload_len: 10240 bytes
payload_total: 976.56 MiB
elapsed: 0.141 s
throughput: 708382 messages/s
payload_throughput: 6917.80 MiB/s

Machine and toolchain:

OS: macOS 26.4.1 (build 25E253)
Kernel: Darwin 25.4.0 arm64
CPU: Apple M4, 10 physical cores, 10 logical cores
Memory: 16 GiB
Rust: rustc 1.95.0 (59807616e 2026-04-14)
Cargo: cargo 1.95.0 (f2d3ce0bd 2026-03-21)