Skip to content

fix(compression): materialize nested compression summaries - #10

Merged
boh5 merged 1 commit into
mainfrom
codex/fix-dcp-nested-compression
Jul 31, 2026
Merged

fix(compression): materialize nested compression summaries#10
boh5 merged 1 commit into
mainfrom
codex/fix-dcp-nested-compression

Conversation

@boh5

@boh5 boh5 commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Summary

  • Materialize every referenced child compression summary into the parent summary before committing the parent block.
  • Make CompressionBlock.childBlockRefs the single lineage authority and require exact-once placeholder expansion.
  • Enforce fully materialized summaries and unique lineage at protocol, store, reducer, persistence, and tool boundaries.
  • Remove the previous duplicated summary-level lineage shape and its compatibility paths.

Why

Nested compression previously retained references/placeholders without guaranteeing that the parent summary contained the full child content. That made a later compression block depend on earlier block state instead of being a self-contained compressed representation.

Impact

Parent compression blocks now preserve complete nested context. Compression summaries carrying child refs, unresolved (bN) placeholders, duplicate lineage, or invalid nesting are rejected.

This is an intentional breaking refactor. Old compression state is not migrated and no fallback compatibility path is retained.

Validation

  • bun run test — 8/8 Turborepo tasks passed
  • Independent sub-agent review completed; all findings fixed
  • Independent re-review — 177 targeted tests passed, no remaining findings
  • bun run build
  • git diff --check origin/main...HEAD

Review in cubic

Summary by CodeRabbit

  • Improvements
    • Compression summaries now materialize nested content before being saved.
    • Added validation to prevent unresolved placeholders, duplicate nested references, missing child blocks, and incomplete summaries.
    • Improved token estimation for nested compressed content, avoiding double-counting.
    • Compression guidance now clearly indicates active nested blocks and required summary placement.

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@boh5, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 23 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 06e73da7-de5f-4650-a956-4e9f547b3936

📥 Commits

Reviewing files that changed from the base of the PR and between 80c770b and 5bb74c5.

📒 Files selected for processing (25)
  • packages/agent-core/src/agents/query/hooks/auto-compact.test.ts
  • packages/agent-core/src/agents/query/hooks/hybrid-compression.test.ts
  • packages/agent-core/src/agents/query/hooks/hybrid-compression.ts
  • packages/agent-core/src/compression/constants.ts
  • packages/agent-core/src/compression/dcp-parity.test.ts
  • packages/agent-core/src/compression/dynamic-range.test.ts
  • packages/agent-core/src/compression/dynamic-range.ts
  • packages/agent-core/src/compression/original-range.test.ts
  • packages/agent-core/src/compression/original-range.ts
  • packages/agent-core/src/compression/state.test.ts
  • packages/agent-core/src/compression/state.ts
  • packages/agent-core/src/compression/summary.test.ts
  • packages/agent-core/src/compression/summary.ts
  • packages/agent-core/src/compression/types.ts
  • packages/agent-core/src/compression/validation.ts
  • packages/agent-core/src/store/helpers.test.ts
  • packages/agent-core/src/store/helpers.ts
  • packages/agent-core/src/store/projection.test.ts
  • packages/agent-core/src/store/reduce.ts
  • packages/agent-core/src/store/session-read-projection.test.ts
  • packages/agent-core/src/tools/builtins/compress.test.ts
  • packages/agent-core/src/tools/builtins/compress.ts
  • packages/protocol/src/compression.ts
  • packages/protocol/src/guards.test.ts
  • packages/protocol/src/guards.ts
📝 Walkthrough

Walkthrough

Compression summaries now use templates with runtime child placeholders. Dynamic compression materializes child summaries before commit, token estimation excludes covered content, and storage and protocol validation reject unresolved summaries and duplicate child references.

Changes

Compression Summary Flow

Layer / File(s) Summary
Summary contracts and materialization
packages/agent-core/src/compression/types.ts, packages/agent-core/src/compression/summary.ts, packages/agent-core/src/compression/validation.ts, packages/protocol/src/compression.ts, packages/agent-core/src/compression/summary.test.ts, packages/agent-core/src/compression/dcp-parity.test.ts, packages/agent-core/src/compression/constants.ts
Compression summaries now separate templates from materialized snapshots. Child placeholders are validated and replaced with complete stored child summaries.
Compression execution and state
packages/agent-core/src/compression/dynamic-range.ts, packages/agent-core/src/compression/dynamic-range.test.ts, packages/agent-core/src/compression/state.ts, packages/agent-core/src/compression/state.test.ts, packages/agent-core/src/agents/query/hooks/hybrid-compression.ts, packages/agent-core/src/agents/query/hooks/hybrid-compression.test.ts, packages/agent-core/src/tools/builtins/compress.ts, packages/agent-core/src/tools/builtins/compress.test.ts, packages/agent-core/src/compression/original-range.test.ts, packages/agent-core/src/agents/query/hooks/auto-compact.test.ts
Dynamic compression materializes templates before draft creation and estimates tokens from child summaries plus uncovered messages. Compression state requires child references and rejects duplicates. Compression nudges describe active references and placeholder expansion.
Storage and protocol enforcement
packages/agent-core/src/store/helpers.ts, packages/agent-core/src/store/helpers.test.ts, packages/agent-core/src/store/reduce.ts, packages/agent-core/src/store/projection.test.ts, packages/protocol/src/guards.ts, packages/protocol/src/guards.test.ts
Store and protocol validation require materialized summaries and unique child references. Reduction and projection no longer add top-level childBlockRefs to summaries.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: materializing nested compression summaries.
Description check ✅ Passed The description clearly explains the change, rationale, impact, and validation, but omits the Documentation and security and Related issue sections.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@boh5
boh5 marked this pull request as ready for review July 31, 2026 16:08

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (4)
packages/agent-core/src/compression/types.ts (1)

55-61: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider a nominal marker to separate templates from materialized summaries.

CompressionSummaryTemplate and CompressionSummary are structurally identical. TypeScript therefore accepts a template anywhere a materialized summary is required, so the whole contract depends on runtime checks. A branded field would make the distinction compile-time enforceable.

♻️ Proposed brand
 export interface CompressionSummaryTemplate {
+  readonly __materialized?: never;
   readonly sections: CompressionSummarySections;
 }
 
 export interface CompressionSummary {
+  readonly __materialized: true;
   readonly sections: CompressionSummarySections;
 }

Note that adding a runtime brand field would change the persisted shape, so exact(summary, ["sections"]) in packages/protocol/src/guards.ts would need the same update. A type-only phantom marker avoids that.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/agent-core/src/compression/types.ts` around lines 55 - 61, Add a
type-only nominal brand to CompressionSummaryTemplate and CompressionSummary so
they are no longer structurally interchangeable, without adding any runtime or
persisted field. Update the relevant construction and type-narrowing sites to
satisfy the phantom marker while keeping the serialized shape limited to
sections; do not alter guards unless required by runtime changes.
packages/agent-core/src/compression/summary.test.ts (1)

96-109: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add coverage for the two throwing paths of materialization.

The suite covers the success path only. Two error paths stay untested:

  • summary.ts line 80 throws CompressionSummaryValidationError when the template fails validation.
  • summary.ts line 85 throws Required child block ${ref} does not exist when blocksByRef omits a required ref.

The second path is reachable from prepareDynamicRangeCompression, which converts the throw into a rejection result. A test pins that contract.

💚 Proposed tests
+  test("rejects a required child block that is absent from state", () => {
+    const parentTemplate = template({ sections: sections("Before (b1) after") });
+
+    expect(() => materializeCompressionSummaryTemplate(parentTemplate, ["b1"], {}))
+      .toThrow("Required child block b1 does not exist");
+  });
+
+  test("rejects a template whose placeholder is not required", () => {
+    const parentTemplate = template({ sections: sections("Unexpected (b9)") });
+
+    expect(() => materializeCompressionSummaryTemplate(parentTemplate, [], {}))
+      .toThrow("Placeholder (b9) is not a required child block ref");
+  });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/agent-core/src/compression/summary.test.ts` around lines 96 - 109,
Add tests alongside the existing “compression summary materialization” coverage
for both throwing paths: assert that invalid input to
materializeCompressionSummaryTemplate raises CompressionSummaryValidationError,
and assert that a missing required ref in blocksByRef raises the expected
“Required child block ${ref} does not exist” error. Also cover
prepareDynamicRangeCompression with an omitted required block and verify it
returns the established rejection result rather than propagating the exception.
packages/agent-core/src/compression/summary.ts (1)

90-95: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Flatten child summaries before parent materialization.

renderCompressionSummary(child.summary) inserts ten ## markers into one parent section. This creates nested or inline headings, such as Before ## Current Objective. Each nesting level also repeats the complete child rendering. The token estimate can therefore report savedTokens === 0, while the commit path still accepts the block. Use a header-free child representation and add a depth test that bounds rendered headers and confirms positive savings.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/agent-core/src/compression/summary.ts` around lines 90 - 95, Update
the child-summary substitution in the section materialization flow around
COMPRESSION_SUMMARY_SECTION_NAMES to use a header-free, flattened representation
instead of renderCompressionSummary(child.summary), preventing nested or inline
## headings and repeated rendering at each depth. Add a depth-focused test that
bounds the rendered header count and verifies positive token savings while
preserving successful block commitment.
packages/agent-core/src/store/helpers.ts (1)

844-847: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Optional: use Zod 4's unified error param.

.refine(fn, "Compression child block refs must be unique") uses the plain-string message form. Zod 4 keeps this working but deprecates it in favor of the unified error param.

♻️ Optional modernization
 const CompressionChildBlockRefsSchema = z.array(BlockRefSchema).refine(
   (refs) => new Set(refs).size === refs.length,
-  "Compression child block refs must be unique",
+  { error: "Compression child block refs must be unique" },
 );
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/agent-core/src/store/helpers.ts` around lines 844 - 847, Modernize
CompressionChildBlockRefsSchema by replacing the deprecated plain-string refine
message with Zod 4’s unified error parameter, while preserving the existing
uniqueness validation and message.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/agent-core/src/compression/summary.test.ts`:
- Around line 96-109: Add tests alongside the existing “compression summary
materialization” coverage for both throwing paths: assert that invalid input to
materializeCompressionSummaryTemplate raises CompressionSummaryValidationError,
and assert that a missing required ref in blocksByRef raises the expected
“Required child block ${ref} does not exist” error. Also cover
prepareDynamicRangeCompression with an omitted required block and verify it
returns the established rejection result rather than propagating the exception.

In `@packages/agent-core/src/compression/summary.ts`:
- Around line 90-95: Update the child-summary substitution in the section
materialization flow around COMPRESSION_SUMMARY_SECTION_NAMES to use a
header-free, flattened representation instead of
renderCompressionSummary(child.summary), preventing nested or inline ## headings
and repeated rendering at each depth. Add a depth-focused test that bounds the
rendered header count and verifies positive token savings while preserving
successful block commitment.

In `@packages/agent-core/src/compression/types.ts`:
- Around line 55-61: Add a type-only nominal brand to CompressionSummaryTemplate
and CompressionSummary so they are no longer structurally interchangeable,
without adding any runtime or persisted field. Update the relevant construction
and type-narrowing sites to satisfy the phantom marker while keeping the
serialized shape limited to sections; do not alter guards unless required by
runtime changes.

In `@packages/agent-core/src/store/helpers.ts`:
- Around line 844-847: Modernize CompressionChildBlockRefsSchema by replacing
the deprecated plain-string refine message with Zod 4’s unified error parameter,
while preserving the existing uniqueness validation and message.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d80078cf-5d32-4ac5-b909-845a1dee4cc0

📥 Commits

Reviewing files that changed from the base of the PR and between 3dd05f7 and 80c770b.

📒 Files selected for processing (24)
  • packages/agent-core/src/agents/query/hooks/auto-compact.test.ts
  • packages/agent-core/src/agents/query/hooks/hybrid-compression.test.ts
  • packages/agent-core/src/agents/query/hooks/hybrid-compression.ts
  • packages/agent-core/src/compression/constants.ts
  • packages/agent-core/src/compression/dcp-parity.test.ts
  • packages/agent-core/src/compression/dynamic-range.test.ts
  • packages/agent-core/src/compression/dynamic-range.ts
  • packages/agent-core/src/compression/original-range.test.ts
  • packages/agent-core/src/compression/state.test.ts
  • packages/agent-core/src/compression/state.ts
  • packages/agent-core/src/compression/summary.test.ts
  • packages/agent-core/src/compression/summary.ts
  • packages/agent-core/src/compression/types.ts
  • packages/agent-core/src/compression/validation.ts
  • packages/agent-core/src/store/helpers.test.ts
  • packages/agent-core/src/store/helpers.ts
  • packages/agent-core/src/store/projection.test.ts
  • packages/agent-core/src/store/reduce.ts
  • packages/agent-core/src/store/session-read-projection.test.ts
  • packages/agent-core/src/tools/builtins/compress.test.ts
  • packages/agent-core/src/tools/builtins/compress.ts
  • packages/protocol/src/compression.ts
  • packages/protocol/src/guards.test.ts
  • packages/protocol/src/guards.ts
💤 Files with no reviewable changes (3)
  • packages/agent-core/src/store/session-read-projection.test.ts
  • packages/agent-core/src/agents/query/hooks/auto-compact.test.ts
  • packages/agent-core/src/tools/builtins/compress.test.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 24 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/agent-core/src/store/helpers.ts
Comment thread packages/agent-core/src/compression/types.ts
BREAKING CHANGE: reject compression summaries that carry child refs or unresolved block placeholders.
@boh5
boh5 force-pushed the codex/fix-dcp-nested-compression branch from 80c770b to 5bb74c5 Compare July 31, 2026 16:24

boh5 commented Jul 31, 2026

Copy link
Copy Markdown
Owner Author

Review triage for current head 5bb74c56:

  • Accepted and fixed Cubic's two findings: persisted-lineage integrity and caller-owned mutable arrays. Both threads include code/test evidence and are resolved.
  • Accepted CodeRabbit's request for direct materialization failure coverage; added invalid-template and missing-child tests.
  • Did not adopt summary-heading flattening or a positive-savings commit gate: upstream OpenCode-DCP restores the full prior summary body during range compression, and ArchCode's token delta is telemetry rather than a validity contract. Flattening would also discard ArchCode's structured summary semantics.
  • Did not add nominal string brands or optional Zod/docstring style changes: persisted/wire ids intentionally share a string representation, while the new runtime lineage validator is the actual trust boundary.

Local validation on this head: full bun run test (8/8 Turborepo tasks), bun run build, and git diff --check all pass.

@boh5
boh5 merged commit 6c8078a into main Jul 31, 2026
7 checks passed
@boh5
boh5 deleted the codex/fix-dcp-nested-compression branch July 31, 2026 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant