[codex] Support raw image offload in v1 train client

[eligotts]

Design update — inline/offload image storage

This PR now follows the prime-rl multimodal image storage policy:

• offload: current behavior, rewrite base64 data images to file:// run assets and require file-backed image URLs.
• inline: keep data:image/...;base64,... URLs in the message payload and validate them without rewriting.

TrainClient now calls the policy-aware image preparation helper, so prime-rl can be the single source of truth via environment/config propagation.

Validation after latest push: uv run pytest tests/v1/test_train_client_multimodal.py -q passed (5 passed). Commit/push hooks also passed (ruff check, ruff format, generated AGENTS/CLAUDE check, ty).

Design update — dropped the None/cache-only image path

This PR and its companions (prime-rl #2836 / verifiers #1746 / renderers #89) no longer use the "send None for already-cached images" mechanism. Every image carries its raw descriptor ref at every slot (current and prior turns); /inference/v1/generate rematerializes each ref from disk every request.

Why: the None path coupled correctness to deployment (LRU cache present, single replica / DP-affinity, no eviction) and surfaced a miss as a hard vLLM EngineDeadError (qwen3-vl mrope dereferences a None image_grid_thw) that the retry net couldn't catch across the engine→API IPC. Dropping it is deployment-agnostic (a miss is impossible) and non-hacky. vLLM's mm_hash encoder cache still skips the expensive GPU re-encode for free — we only forgo the cheap IPC/CPU-reprocess dedup.

Validated: color-codeword (Qwen3-VL-4B) under DP=2, no affinity / no cache reliance: 0 crashes, 0 data=None, multi-turn accumulation correct, reward ~0.84. Also confirmed under TP.

This repo: with every image carrying its ref, no cache miss can occur — removed the retry subsystem (_generate_with_image_ref_retry, _has_descriptor_only_images, _retryable_mm_error_type, _json_error_type, _RETRYABLE_MM_ERROR_TYPES). Rollouts call renderers.client.generate directly. Obsolete retry tests removed.

---

Original description

Summary

• tighten v1 multimodal graph serialization around strict raw descriptor sidecars
• reject processed multimodal payload keys recursively, including nested pixel_values, image_embeds, and image_features
• update v1 multimodal tests to use strict prime_raw_mm_item envelopes instead of descriptor-only Qwen payloads
• keep raw image offload and retry behavior aligned with the companion Renderers and Prime-RL PRs

Companion PRs

• Prime-RL: Support v1 raw multimodal image offload prime-rl#2836
• Renderers: [codex] Support raw image refs for multimodal rendering renderers#89

Notes

• Draft/WIP: this depends on the renderer generic raw multimodal ref contract in the companion PR.
• v1 multimodal sidecars intentionally carry raw descriptors only, not processed image tensors or image-processor payloads.
• Prime/vLLM materialization happens from raw image refs rather than Verifiers-held processor outputs.

Validation

• Commit hooks: ruff check, ruff format, generated AGENTS/CLAUDE check passed.
• Push hook: ty (ci parity) passed.
• End-to-end hosted-style smoke through Prime-RL with /home/ubuntu/verifiers, /home/ubuntu/renderers, and /home/ubuntu/prime-rl-v1-raw-mm-offload completed inference, env rollouts, train batch creation, trainer step 0, and decoded strict trainer-bound raw image refs.

[!NOTE]

Support raw image offload in v1 train client via prepare_images_inplace

• Adds verifiers/utils/multimodal.py with prepare_images_inplace, which recursively walks message structures and offloads image sources to content-addressed file:// URIs via renderers.mm_store.offload_image_to_run_assets.
• TrainClient overrides prepare_request_body (a new base-class hook in client.py) to run image offload before chat requests are processed; the interception server calls this hook and maps failures to structured rollout errors.
• Multimodal turns are now eligible for bridging in TrainClient.get_response; when the renderer is multimodal, previous turn sidecars are forwarded via PendingTurn.previous_multi_modal_data.
• Multimodal sidecars in graph.py are validated to contain only raw image descriptors (with raw_image_uri) and now include mm_placeholders alongside items and hashes.
• Risk: prepare_images_inplace raises RuntimeError if offload support is unavailable or if any image source is not file://-backed, which will surface as a request failure.

^{Macroscope summarized d9e79d6.}

Update: ingress hardening (2c2824ae)

• prepare_images_inplace now covers the full renderer part treaty: nested image_url dicts, direct-string image_url, direct-string image parts, and typed pydantic parts. Non-string sources raise with the part shape named; renderer-side raw mode hard-requires file:// (no second offload layer).
• Interception server labels prepare_messages failures as InterceptionError instead of misattributing them to the user simulator.
• New shape-coverage test (skips until the renderers pin ships mm_store; passes against the sibling renderers checkout). Suite: 839 passed with pre-existing test_envs/test_opencode_rlm_env failures reproduced on the base branch.

Update: merged main (2b1627d03)

Reconciled with verifiers main (111 commits ahead at the merge-base — mostly unrelated v1 harness/multi-agent, taskset/environment, and trace/data-model churn; none of it touched this branch's actual offload files, verifiers/utils/multimodal.py / clients/renderer_client.py / types.py).

• interception/server.py: main's d21100bea ("resume replaces mid-request user injection") independently moved the user-simulator loop out of the interception server entirely, into Harness.launch/resume, and added graph-atomicity retry coalescing to handle_request/_stream. Took main's structure as authoritative; dropped this branch's now-obsolete session.user/session.opening mid-request injection (and its prepare_messages call sites) since injected/resumed messages now re-enter as ordinary request bodies, already covered by prepare_request_body (kept, rewired to the top of the new single-shot handle_request).
• graph.py: kept finish_reason/usage/multi_modal_data/previous_multi_modal_data() (this branch) alongside main's SkipJsonSchema wrapping convention and commit()'s new (tools) -> assistant_node_id signature. Caught and fixed an auto-merge that silently dropped the FinishReason/Usage imports — surfaced as a pydantic "Trace not fully defined" failure at Trace construction, not a textual conflict.
• trace.py: kept this branch's more accurate multi_modal_data docstring; took main's tuple-based bridge-mutation assertion in the test suite (strictly better — reuses already-computed prior_mm/prior_counts instead of recomputing).
• verifiers/v1/ARCHITECTURE.md: main deleted this file (docs moved to a much shorter docs/v1/architecture.md that doesn't cover this depth) — accepted the deletion; the one load-bearing fact (why multi_modal_data is JSON-excluded) is now a docstring on _NODE_DUMP_EXCLUDE instead of a dedicated doc.

Validation: full suite passing with a local renderers checkout override (uv run --with-editable ../renderers pytest tests/, minus PRIME_API_KEY-gated e2e/env tests) — includes test_prepare_images_inplace_offloads_every_image_part_shape, previously skipped pending a renderers pin with mm_store.

Update: dropped the orphaned prepare_messages hook (d9e79d6c)

Main's harness-segment-resume refactor removed the interception server's mid-request user-simulator injection — the only call sites of this PR's prepare_messages hook. Resumed user turns now re-enter the server as ordinary request bodies, already covered by prepare_request_body at request ingress, so the hook (base Client + TrainClient override) is deleted rather than carried as dead code.

---

Note

Medium Risk
Touches training ingress, trace wire format, and multi-turn renderer bridging; depends on a renderers version with mm_store, and non-file-backed images fail hard at prepare time.

Overview
Adds prepare_images_inplace (verifiers/utils/multimodal.py) to rewrite inline/base64 image parts to content-addressed file:// run assets via renderers.mm_store, covering wire dicts and typed message models. TrainClient and RendererClient run it before rendering; Client.prepare_request_body is the new hook, invoked from the interception server (failures surface as InterceptionError).

v1 multimodal training shifts from processed tensors to raw image descriptors (raw_image_uri): graph serialization validates and rejects pixel_values / embed payloads, mm_placeholders ride per-node and in branch merges, and PendingTurn.previous_multi_modal_data() feeds multimodal bridge turns (multimodal is no longer blocked from bridging). Legacy v0→v1 trace mapping keeps live cumulative multi_modal_data from rollout state.

^{Reviewed by Cursor Bugbot for commit ae516c1. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

New feature adding raw image offload support for multimodal training with 5 unresolved review comments, including a high-severity issue about images potentially being silently dropped and medium-severity backward compatibility concerns. The substantial new functionality and open concerns warrant human review.

^{You can customize Macroscope's approvability policy. Learn more.}

[macroscopeapp]

🟡 Medium v1/graph.py:76

_validate_raw_mm_item now unconditionally rejects processed multimodal payloads containing keys like pixel_values, and deserialize_multi_modal_data runs it on every multi_modal_data field during deserialization. Loading a previously persisted multimodal v1 trace whose sidecars contain pixel_values now raises TypeError instead of round-tripping, breaking backwards compatibility for existing saved rollouts. Consider allowing processed payloads through on the deserialization path (e.g. by skipping the processed-key check in the validator's before path) so old traces can still be loaded.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/graph.py around line 76:

`_validate_raw_mm_item` now unconditionally rejects processed multimodal payloads containing keys like `pixel_values`, and `deserialize_multi_modal_data` runs it on every `multi_modal_data` field during deserialization. Loading a previously persisted multimodal v1 trace whose sidecars contain `pixel_values` now raises `TypeError` instead of round-tripping, breaking backwards compatibility for existing saved rollouts. Consider allowing processed payloads through on the deserialization path (e.g. by skipping the processed-key check in the validator's `before` path) so old traces can still be loaded.

[macroscopeapp]

🟡 Medium utils/multimodal.py:64

prepare_images_inplace skips validation when an image_url part has a missing or None image_url field: lines 65-67 only call _prepare_image_source when source is not None, so the malformed part passes through unchecked. Downstream, ChatDialect.parse_request normalizes it to ImageUrlSource(url=""), forwarding a request with an empty image URL instead of rejecting it. Consider calling _require_file_image_url(value) (or otherwise validating) when source is None so malformed parts are rejected.

Suggested change

	if value.get("type") == "image_url":
	source = value.get("image_url")
	if source is not None:
	_prepare_image_source(source, image_dir=image_dir)
	if value.get("type") == "image_url":
	source = value.get("image_url")
	if source is not None:
	_prepare_image_source(source, image_dir=image_dir)
	else:
	_require_file_image_url(value)

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/utils/multimodal.py around lines 64-67:

`prepare_images_inplace` skips validation when an `image_url` part has a missing or `None` `image_url` field: lines 65-67 only call `_prepare_image_source` when `source is not None`, so the malformed part passes through unchecked. Downstream, `ChatDialect.parse_request` normalizes it to `ImageUrlSource(url="")`, forwarding a request with an empty image URL instead of rejecting it. Consider calling `_require_file_image_url(value)` (or otherwise validating) when `source` is `None` so malformed parts are rejected.

[macroscopeapp]

🟡 Medium utils/multimodal.py:52

_prepare_image_part calls _offload_image_url(url, image_dir) before checking whether url is already a file:// path. When a prompt already contains local file:// image URLs and the installed renderer build lacks offload_image_to_run_assets, _offload_image_url raises RuntimeError even though no offload was needed, causing valid pre-offloaded multimodal prompts to fail. Consider returning early when the source is already a file:// URL before invoking _offload_image_url.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/utils/multimodal.py around line 52:

`_prepare_image_part` calls `_offload_image_url(url, image_dir)` before checking whether `url` is already a `file://` path. When a prompt already contains local `file://` image URLs and the installed renderer build lacks `offload_image_to_run_assets`, `_offload_image_url` raises `RuntimeError` even though no offload was needed, causing valid pre-offloaded multimodal prompts to fail. Consider returning early when the source is already a `file://` URL before invoking `_offload_image_url`.

[macroscopeapp]

🟠 High utils/multimodal.py:27

_part_image_field returns "image" unchanged for parts with type == "image", so _prepare_image_part offloads the source but leaves the part in the non-canonical {"type": "image", "image": ...} shape. Downstream v1 chat parsing only preserves image_url parts, so the image is silently dropped from the traced/training prompt even after prepare_images_inplace runs. Consider normalizing image parts to image_url (or mapping image to the image_url field during offload) so downstream parsers retain them.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/utils/multimodal.py around line 27:

`_part_image_field` returns `"image"` unchanged for parts with `type == "image"`, so `_prepare_image_part` offloads the source but leaves the part in the non-canonical `{"type": "image", "image": ...}` shape. Downstream v1 chat parsing only preserves `image_url` parts, so the image is silently dropped from the traced/training prompt even after `prepare_images_inplace` runs. Consider normalizing `image` parts to `image_url` (or mapping `image` to the `image_url` field during offload) so downstream parsers retain them.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 2b1627d. Configure here.}

[cursor]

Unpopulated MessageNode finish fields

Low Severity

MessageNode gained finish_reason and usage, but _commit_turn never sets them on the assistant node, and nothing reads them. Truncation and dashboards already use ModelCall. The fields stay null on every node while docstrings claim they drive those behaviors.

Additional Locations (1)

• verifiers/v1/graph.py#L631-L645

 

^{Reviewed by Cursor Bugbot for commit 2b1627d. Configure here.}