Skip to content

capture: make request records privacy-safe by default - #356

Open
PhilipJohnBasile wants to merge 3 commits into
youssofal:mainfrom
PhilipJohnBasile:feat/privacy-safe-request-capture
Open

capture: make request records privacy-safe by default#356
PhilipJohnBasile wants to merge 3 commits into
youssofal:mainfrom
PhilipJohnBasile:feat/privacy-safe-request-capture

Conversation

@PhilipJohnBasile

Copy link
Copy Markdown
Contributor

This extracts only the request-capture privacy boundary from #336.

Request captures are still off unless MTPLX_REQUEST_CAPTURE_DIR is set. Once enabled, the default record now replaces prompt and completion token IDs with counts and stable SHA-256 digests. Prompt text, response text, messages, and exception text are also omitted by default, while nested credentials are redacted.

The public completion_token_ids(tokens) helper and its existing one-argument call remain supported. It now returns the completion count and digest by default. Raw completion IDs require their own MTPLX_REQUEST_CAPTURE_INCLUDE_COMPLETION_TOKENS=1 opt-in. That flag is separate from MTPLX_REQUEST_CAPTURE_INCLUDE_PROMPT_TOKENS=1, and both paths enforce independent limits.

Measurement

Command: python scripts/measure_request_capture.py --records 1000

Measured source: fe5b5e59a14e0066f3191951483e3a65b9c55592 on Apple arm64 with Python 3.14.6.

  • Capture disabled: 0.658 ms total, 0.658 microseconds per record
  • Default privacy capture: 595.733 ms total, 595.733 microseconds per record
  • Throughput: 1,678.604 records/s
  • Output: 1,000 files, 1,349,890 bytes total, 1,349.89 bytes per record
  • Payload per record: 256 prompt tokens, 64 completion tokens, 736 prompt characters, 400 response characters

The measurement verifies all 1,000 records completed, raw prompt and completion IDs were absent, prompt/response/message/secret content was absent, and token counts plus digests were present. Validation runs outside the timed section. The committed receipt is request-capture-1000.json.

Validation

  • python -m pytest tests/test_request_capture.py -q: 18 passed
  • python -m compileall -q mtplx/request_capture.py scripts/measure_request_capture.py tests/test_request_capture.py: passed
  • python -m ruff check mtplx/request_capture.py scripts/measure_request_capture.py tests/test_request_capture.py: passed
  • git diff --check origin/main...HEAD: passed
  • U+2014 and U+2013 scan across all four changed files: no matches

Exclusions

This PR does not change server call sites, runtime registries, replay systems, dashboards, API routes, workflow files, model behavior, or scheduling. The measurement covers request-capture persistence overhead only. It does not claim inference throughput or model-quality improvement.

@PhilipJohnBasile
PhilipJohnBasile force-pushed the feat/privacy-safe-request-capture branch from 0149acf to a5b2f9a Compare September 1, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant