Skip to content

fix(runtime): eliminate unbounded provider request diagnostics #4082

Description

@M4n5ter
English

Problem

Maka persists three overlapping diagnostic records around each physical provider request:

  1. The complete serialized request body as a hidden provider_request_capture Artifact.
  2. A durable provider_request_captured event carrying request-shape metadata and the Artifact reference.
  3. A durable provider_request_attempt_recorded event carrying the per-message/per-tool segment array again.

A fourth record, model_call_attempt_recorded, is already the canonical authority for the physical call's accounting, outcome, usage, cost, and latency.

Relevant paths:

A repository-wide audit found no production reader of the serialized request body. Its only production path is creation, reference rewriting during conversation copy, and eventual Session purge.

This has three consequences:

  • Unbounded storage: request bodies and per-message segment arrays grow with conversation history. Their cumulative retained size can approach quadratic growth before compaction bounds the provider request.
  • Copy amplification: conversation branch/revision copies these diagnostic Artifacts and rewrites their diagnostic references.
  • Availability coupling: diagnostic Artifact I/O is awaited before dispatch. A full or unavailable Artifact store can prevent an otherwise valid provider request and request Runtime Host drain.

A small nine-call workspace sample made the duplication visible:

Record Average bytes per call
Full request Artifact ~32.3 KiB
provider_request_captured event ~5.8 KiB
provider_request_attempt_recorded event ~6.1 KiB
Canonical model_call_attempt_recorded event ~1.1 KiB

The first three records were about 97% of the retained per-call bytes in that sample. Removing only the JSON Artifact would leave the per-message diagnostic ledger with the same asymptotic growth shape.

Decision

Delete durable full-request capture from normal Runtime operation. Do not preserve it behind a default-off flag or move it into another durable store.

A diagnostic side effect must never become Session replay authority or gate provider dispatch.

The normal path should:

  1. Inspect the prepared request transiently in memory.
  2. Fold it immediately into a bounded context composition using the existing four-kind aggregation and bounded top-tool list.
  3. Dispatch without waiting for diagnostic persistence.
  4. Settle exactly one canonical, bounded model-call fact.
  5. Commit the latest-context projection atomically with the completed main-call attempt, as it does today.

If cold projection rebuild requires request composition, evolve ModelCallAttempt to a versioned bounded shape that can carry the already-folded composition for a completed main request. Do not persist the full body, a per-message array, or per-message hashes. Do not persist speculative fields without a current consumer.

The exact field name is not the architectural decision. The invariant is that the canonical physical-call record has a constant size bound independent of conversation length.

Target ownership model

Data Owner and lifetime Copy behavior
Conversation/Runtime replay facts Durable Session authority Copy semantic history
Read image and Tool Result offload Typed Session context references backed by #4071 Copy references, not payload bytes
Model-call usage/outcome and bounded latest-context composition Canonical physical-call fact plus derived projection Do not create diagnostic payload copies
Complete provider request/response trace Does not exist by default Never copied

If a real product requirement for exact request traces appears later, design a separate explicit ModelCallTraceSink: user-initiated, short-lived, redacted, hard-budgeted, asynchronously best-effort, excluded from Session replay/copy/backup by default, and unable to fail provider dispatch. Do not build that product in this issue.

Kill list

Stop producing and then remove:

  • persistProviderRequestCaptureArtifact
  • createProviderRequestCaptureRecorder from production composition
  • New provider_request_captured events
  • New provider_request_attempt_recorded events with per-message segments
  • New captureId / captureArtifactId joins
  • serializedRequest as a returned durable-capture field
  • Conversation-copy rewriting for capture events and capture Artifact references
  • provider_request_capture as a source for new Artifacts
  • Artifact failure triggering provider-call failure or Runtime Host drain

Legacy decoders may remain temporarily where persisted compatibility is real; legacy writers must stop immediately.

Legacy transition

  1. Stop all new capture Artifact and diagnostic-event production.
  2. Keep old AgentRun event types readable as open legacy input during migration.
  3. After Host Ready, process Sessions in bounded, restartable batches:
    • materialize or repair the latest-context projection where useful;
    • purge legacy provider-request capture Artifacts;
    • purge or tombstone redundant capture/diagnostic events according to AgentRun ledger invariants.
  4. Conversation copy must skip legacy provider-request diagnostic events and Artifacts.
  5. A legacy Session may lose request-composition diagnostics if its projection is unavailable. It must never lose conversation replay, accounting, or the ability to open.
  6. Remove decode-only compatibility after migration evidence shows no retained legacy dependencies.

Cleanup must not enumerate the whole payload population on the Host Ready path.

Acceptance invariants

  • Normal Runtime operation creates no complete provider request Artifact.
  • Provider dispatch performs no diagnostic filesystem or diagnostic-database write.
  • Artifact-store failure cannot fail or drain an otherwise valid model request.
  • Durable request diagnostics have a documented constant upper bound per physical call, independent of message count and request-body size.
  • A stress test with monotonically growing history and a tool-heavy loop demonstrates linear, bounded-per-call storage growth.
  • Context Inspector composition remains available for new completed main requests through the bounded canonical fact/latest-context projection.
  • Conversation copy does not copy legacy request-capture payloads or diagnostic events.
  • Legacy cleanup is bounded, restartable, idempotent, and post-Ready.
  • Existing Sessions remain replayable and accounting records remain valid after cleanup.

Relationships and scope

中文

问题

Maka 当前会围绕每次物理 provider 请求持久化三份相互重叠的诊断记录:

  1. 将完整序列化请求正文保存为隐藏的 provider_request_capture Artifact。
  2. 写入 durable provider_request_captured event,其中包含请求形状元数据和 Artifact 引用。
  3. 再写入 durable provider_request_attempt_recorded event,其中再次保存逐 message/逐 tool 的 segment 数组。

此外,model_call_attempt_recorded 已经是该物理调用的 accounting、结果、usage、cost 和 latency 的 canonical authority。

相关路径:

全仓库审计没有发现完整序列化请求正文的生产读取者。它在生产代码中的用途只有创建、conversation copy 时重写引用,以及最终随 Session 清理。

这会产生三个后果:

  • 无界存储: 请求正文和逐 message segment 数组随对话历史增长;在 compaction 限制 provider 请求之前,累计保留空间可能接近平方增长。
  • 复制放大: Conversation branch/revision 会复制这些诊断 Artifacts 并重写其引用。
  • 可用性耦合: 诊断 Artifact I/O 在 dispatch 前被等待;Artifact store 满或不可用时,会阻止原本有效的 provider 请求并触发 Runtime Host drain。

一个包含九次调用的小型 workspace 样本显示了这种重复:

记录 每次调用平均字节数
完整请求 Artifact ~32.3 KiB
provider_request_captured event ~5.8 KiB
provider_request_attempt_recorded event ~6.1 KiB
Canonical model_call_attempt_recorded event ~1.1 KiB

在该样本中,前三项约占每次调用持久化字节的 97%。仅删除 JSON Artifact,仍会留下具有同类渐近增长形态的逐 message 诊断 ledger。

决策

从正常 Runtime 路径中删除 durable 完整请求 capture。不要通过默认关闭的 feature flag 保留,也不要把它搬进另一个 durable store。

诊断副作用绝不能成为 Session replay authority,也不能阻塞 provider dispatch。

正常路径应当:

  1. 只在内存中临时检查 prepared request。
  2. 立即使用现有的四类聚合和有界 top-tool 列表,将其折叠为有界 context composition。
  3. 不等待诊断持久化,直接 dispatch。
  4. 只结算一条 canonical、有界的 model-call fact。
  5. 与现在一样,将 latest-context projection 和成功完成的 main-call attempt 原子提交。

如果冷启动重建 projection 需要请求 composition,则将 ModelCallAttempt 演进为有版本、有界的结构,只为成功完成的 main 请求携带已经折叠的 composition。不要持久化完整正文、逐 message 数组或逐 message hash。没有当前消费者的推测性字段不要持久化。

具体字段名不是架构决策。真正的 invariant 是:canonical 物理调用记录拥有与对话长度无关的固定大小上限。

目标 ownership 模型

数据 Owner 与生命周期 Copy 行为
Conversation/Runtime replay facts Durable Session authority 复制语义历史
Read 图片和 Tool Result offload #4071 支撑的 typed Session context references 复制引用,不复制 payload 字节
Model-call usage/outcome 和有界 latest-context composition Canonical 物理调用 fact 加 derived projection 不创建诊断 payload 副本
完整 provider request/response trace 默认不存在 永不复制

如果未来出现真实的精确请求 trace 产品需求,再单独设计显式 ModelCallTraceSink:由用户启动、短生命周期、经过脱敏、具备硬预算、异步 best-effort、默认不进入 Session replay/copy/backup,并且绝不能导致 provider dispatch 失败。本 issue 不实现这个产品。

删除清单

停止产生并最终删除:

  • persistProviderRequestCaptureArtifact
  • 生产装配中的 createProviderRequestCaptureRecorder
  • 新的 provider_request_captured events
  • 带逐 message segments 的新 provider_request_attempt_recorded events
  • 新的 captureId / captureArtifactId join
  • serializedRequest 作为 durable capture 返回字段
  • Conversation copy 中对 capture event 和 capture Artifact reference 的重写
  • provider_request_capture 作为新 Artifact 的 source
  • Artifact 失败导致 provider call 失败或 Runtime Host drain

在确有持久化兼容需求时,可以暂时保留 legacy decoder;legacy writer 必须立即停止。

旧数据迁移

  1. 停止产生所有新的 capture Artifacts 和 diagnostic events。
  2. 迁移期间继续把旧 AgentRun event type 作为开放的 legacy input 读取。
  3. Host Ready 后,以有界、可恢复的批次处理 Sessions:
    • 在有价值时物化或修复 latest-context projection;
    • 清理旧 provider-request capture Artifacts;
    • 根据 AgentRun ledger invariant 清理或 tombstone 重复的 capture/diagnostic events。
  4. Conversation copy 必须跳过旧 provider-request diagnostic events 和 Artifacts。
  5. 如果 projection 不可用,旧 Session 可以失去 request-composition diagnostics;但绝不能失去 conversation replay、accounting 或正常打开能力。
  6. 迁移证据表明不存在遗留依赖后,删除仅用于 decode 的兼容逻辑。

清理不能在 Host Ready 路径枚举整个 payload 集合。

验收约束

  • 正常 Runtime 运行不会创建完整 provider request Artifact。
  • Provider dispatch 不执行任何诊断 filesystem 或诊断 database 写入。
  • Artifact-store 失败不能让原本有效的模型请求失败或触发 drain。
  • Durable request diagnostics 每次物理调用都具有明确的固定大小上限,与 message 数量和请求正文大小无关。
  • 使用单调增长历史和工具调用密集循环的压力测试证明存储按调用次数线性增长,且每次调用有界。
  • 对新的成功 main 请求,Context Inspector composition 仍可通过有界 canonical fact/latest-context projection 获取。
  • Conversation copy 不复制旧 request-capture payload 或 diagnostic events。
  • 旧数据清理有界、可恢复、幂等,并且在 Host Ready 之后运行。
  • 清理后已有 Sessions 仍可 replay,accounting records 仍然有效。

关系与范围

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions