fix(client): reject malformed sampled-token evidence

[EazyReal]

Summary

• validate /inference/v1/generate sampled-token evidence before renderer parsing
• require exactly one logprob entry for each completion token, identified as that exact token_id:<id>
• reject missing, malformed, Boolean, non-numeric, non-finite, or ambiguous -9999.0 evidence
• expose MalformedGenerateResponseError so callers can distinguish an unusable provider response from other failures

Why

The generate response contains parallel token_ids and logprobs.content fields. Matching their lengths alone does not prove that a logprob belongs to the sampled token at that position.

The vLLM endpoint labels each entry with token_id:<id>. It also uses -9999.0 both as the default when evidence is absent and as a lower-bound clamp. That value therefore cannot prove the original sampled-token logprob and must fail closed.

Validation belongs before renderer parsing because this is a provider wire-contract invariant. Downstream rollout and training code should not receive incomplete or misbound sampling evidence as a successful generation.

Compatibility

Malformed responses now raise MalformedGenerateResponseError, a ValueError subclass. This intentionally tightens behavior while retaining compatibility with callers that already handle ValueError. The exported subtype lets downstream clients map this provider failure explicitly.

Verification

• uv run pytest tests/test_client.py -q — 20 passed, 2 skipped
• uv run ruff check renderers/client.py renderers/__init__.py tests/test_client.py
• uv run ruff format --check renderers/client.py renderers/__init__.py tests/test_client.py
• hosted checks on Python 3.10–3.13, Ty, and Ruff pass on the identical head

---

Note

Medium Risk
Rollout/training paths that relied on silently accepting bad or placeholder logprobs will now error at generation time, which is intentional but can surface more provider failures.

Overview
Tightens how completion logprobs are read from /inference/v1/generate responses. Instead of coercing missing logprobs to 0.0, the client runs _parse_completion_logprobs before renderer.parse_response.

Validation requires one choice.logprobs.content entry per completion token, each labeled token_id:<id> matching that position’s sampled id, with a finite numeric logprob. It rejects vLLM’s -9999.0 sentinel when it indicates missing sampling evidence. Failures raise the new exported MalformedGenerateResponseError (ValueError subclass).

Tests cover malformed/missing logprobs, count and token-id mismatches, non-finite values, and that invalid responses fail before renderer parsing; valid 0.0 logprobs still pass through.

^{Reviewed by Cursor Bugbot for commit 07ce506. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Reject malformed sampled-token evidence in generate with MalformedGenerateResponseError

• Adds MalformedGenerateResponseError (a ValueError subclass) to renderers/client.py and exports it from the renderers package.
• Introduces _parse_completion_logprobs to strictly validate per-token logprobs: checks structural alignment with token_ids, rejects non-numeric/non-finite values, and rejects the vLLM missing-evidence sentinel (-9999.0).
• generate() now calls _parse_completion_logprobs instead of loosely extracting logprobs, raising MalformedGenerateResponseError on any validation failure while preserving exact zero values.
• Behavioral Change: callers that previously received defaulted/coerced logprob values will now get an exception when the engine response is malformed or missing sampling evidence.

^{Macroscope summarized 07ce506.}