perf: bound L1 cache access-history retention by groupthinking · Pull Request #1299 · groupthinking/EventRelay

groupthinking · 2026-08-03T22:22:12Z

Canonical issue

No pre-existing tracking issue. This surfaces a focused memory-hygiene commit that
was sitting unmerged/unpushed on the assigned branch; opened as a draft for review.

Outcome

InMemoryCacheLayer.access_patterns appended one timestamp per cache hit and never
trimmed it, and _evict_if_needed() dropped the cache entry without releasing the
matching history — so eviction orphaned a per-key history that no resident key could
ever clean up. Those bytes are invisible to stats.total_size_bytes, so the
max_size_bytes LRU budget could neither see nor reclaim them.

Under a 200-entry cache serving 200k reads across 5k churn keys, the layer reported
2,200 bytes while actually holding ~7.1 MiB of timestamps. After the fix, retained
history drops 7,461,354 → 587,664 bytes with hit rate, total_size_bytes, resident
entry count and wall time all unchanged.

Scope

Included: src/youtube_extension/backend/services/intelligent_cache.py (bound each
key's history to ACCESS_HISTORY_WINDOW=64 via deque(maxlen=...); pop it on
eviction), tests/unit/test_intelligent_cache.py (regression coverage).
Explicitly excluded: no change to _calculate_adaptive_ttl() — it reads only
accesses[0], accesses[-1] and len(accesses), all O(1) on a deque.

Risk

Risk level: low
Failure mode: none identified. The window (64) is well above the two samples the TTL
estimator needs; behaviour and reported stats are unchanged. Additive to a bounded
collection.
Rollback: revert this commit; the layer returns to unbounded history retention.

Verification

Against head 0f2612b:

PYTHONPATH=src python -m pytest tests/unit/test_intelligent_cache.py -q
170 passed in 11.52s

(The repo-wide 88% coverage gate is expected-red when running a single file in
isolation; every test in the touched file passes.)

Focused tests
Required CI — see note below
Review threads resolved (none open)

Production evidence

Not applicable — this changes an in-memory L1 cache internal only. No runtime request
path, response shape, or deployed surface is touched.

CI note: the agent-completion/truth-gate check on claude/* branches returns
invalid_payload because the branch-name heuristic marks them applicable without a
linked AgentTask (the known asymmetry tracked in #1286, whose diagnostics PR #1285
makes self-explaining). This draft is not requesting merge; that red is the #1286
false-positive, not a defect in this change.

Agent handoff

Acceptance criteria are satisfied
No competing PR implements the same issue
Required checks pass on the current head (modulo the Truth gate reports bare invalid_payload and discards the collector's diagnostics, stranding ~47 open PRs #1286 truth-gate false-positive noted above)
Human decision is requested for merge approval to protected main

🤖 Generated with Claude Code

https://claude.ai/code/session_01CJQeUmsAzh2TKbRW1kBNJf

Generated by Claude Code

InMemoryCacheLayer.access_patterns appended one float per cache hit and never trimmed, and _evict_if_needed() removed the cache entry without releasing the matching history. delete() already released it, so eviction was the sole path that orphaned a history with no resident key left to ever trigger its cleanup. The retained bytes are invisible to stats.total_size_bytes, so the max_size_bytes LRU budget could neither see nor reclaim them. Under a 200-entry cache serving 200k reads across 5k churn keys the layer reports 2,200 bytes while actually holding 7.1 MiB of timestamps. Bound each key's history to ACCESS_HISTORY_WINDOW=64 via a deque and pop it on eviction. _calculate_adaptive_ttl() reads only accesses[0], accesses[-1] and len(accesses), all O(1) on a deque, so it needs no change. Retained history drops 7,461,354 -> 587,664 bytes with hit rate, total_size_bytes, resident entry count and wall time all unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

vercel · 2026-08-03T22:22:20Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
v0-uvai	Ready	Preview, v0	Aug 3, 2026 10:23pm

github-actions · 2026-08-03T22:22:24Z

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 0f2612b.

Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

Scanned Files

None

coderabbitai · 2026-08-03T22:22:32Z

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)

[‘architecture-gap’, ‘bug’, ‘ci-cd’, ‘ci/cd’, ‘copilot-rabbit’, ‘documentation’, ‘duplicate’, ‘enhancement’, ‘frontend’, ‘github_actions’, ‘good first issue’, ‘help wanted’, ‘high-priority’, ‘invalid’, ‘javascript’, ‘ml-model’, ‘needs-triage’, ‘pipeline-critical’, ‘placeholder-code’, ‘priority:high’, ‘python’, ‘python:uv’, ‘question’, ‘styling’, ‘tests’, ‘v0’]

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 772429aa-9f12-4add-988b-570623fdea4d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-08-03T22:22:42Z

Agent Completion Truth Gate: BLOCKED

Reasons: invalid_payload

Machine-readable verdict

{
  "details": {
    "invalid_fields": [
      "issue.number",
      "policy.agent_login",
      "policy.run_id"
    ]
  },
  "reasons": [
    "invalid_payload"
  ],
  "verdict": "blocked"
}

Workflow evidence

groupthinking · 2026-08-03T22:24:19Z

`agent-completion/truth-gate` red here is the known #1286 false-positive — not this diff

The gate returned invalid_payload on invalid_fields: [issue.number, policy.agent_login, policy.run_id]. That is the branch-name heuristic over-triggering: claude/* branches are marked applicable, but this PR has no linked AgentTask issue supplying those fields, so a fail-closed gate blocks it. It's the same asymmetry documented in #1286 and the exact class of failure that currently blocks the other two ready PRs, #1280 and #1281. #1285 (green, CodeRabbit-approved) makes this verdict self-explaining but deliberately does not change which PRs the heuristic polices — that was scoped out as architecturally significant.

This change itself is green: PYTHONPATH=src pytest tests/unit/test_intelligent_cache.py → 170 passed. The diff is a focused L1-cache memory-hygiene fix (bounds per-key hit history to a deque(maxlen=64), releases it on eviction) touching only in-memory cache internals — no dependency, runtime, or deployed-surface change (the Dependency Review check passed; the snapshot warning is expected for a no-dependency diff).

I'm deliberately not fabricating an agent_login/run_id/linked-issue payload to turn this gate green — that would defeat a fail-closed CI control. Resolution is human-gated and has two independent parts:

Merge approval to protected main (this is a draft; not requesting merge).
The Truth gate reports bare invalid_payload and discards the collector's diagnostics, stranding ~47 open PRs #1286 heuristic decision — either fix the branch heuristic so it stops policing non-AgentTask claude/*/sentinel/*/palette/* branches, or link a real AgentTask to each such PR. Either changes which PRs are policed, so it's a maintainer call.

CodeRabbit auto-review skipped this PR because it carries no qualifying label; if a bot review is wanted before merge, adding the accurate python label will trigger it.

Generated by Claude Code

vercel

Additional Suggestion:

Lazy-expiration path in InMemoryCacheLayer.get() deletes the cache entry but leaves its access_patterns history orphaned, leaking history unbounded in key count.

groupthinking · 2026-08-03T22:26:38Z

Closing this as a duplicate of #1295, which the owner deliberately closed and split.

I opened this PR carrying commit 0f2612b before I had #1295 in view (it was created after my initial open-PR scan). That commit is the combined change — the eviction-orphan fix plus the ACCESS_HISTORY_WINDOW=64 sliding-window — which #1295's review explicitly decided to split, so re-proposing it here is exactly the shape that was rejected.

Two independent reasons this should not stay open:

The VADE review on this PR is correct — and already handled upstream. The lazy-expiry path in InMemoryCacheLayer.get() (lines 166-169) does del self.cache[key] without releasing access_patterns or decrementing total_entries/total_size_bytes. @linear and the red-team pass caught the same hole on perf: bound L1 cache access-history retention #1295, and the pure-bug-fix split — perf: release full entry bookkeeping on all L1 cache removal paths #1298 (perf/release-orphaned-access-history) — fixes it properly on all three removal paths via a shared _release_entry() helper. This commit does not contain that fix, so perf: release full entry bookkeeping on all L1 cache removal paths #1298 strictly supersedes it.
Governance gate. Only one open PR is allowed per canonical issue; perf: bound L1 access history, adopt trailing-window TTL #1294 is reserved for the follow-up windowing change, and perf: release full entry bookkeeping on all L1 cache removal paths #1298 owns the bug fix. This duplicate would collide with both.

No work is lost — the bug fix is in #1298 and the windowing change follows on #1294. The unrelated agent-completion/truth-gate red I noted earlier is moot now that this is closing.

Generated by Claude Code

github-actions Bot added the python label Aug 3, 2026

vercel Bot reviewed Aug 3, 2026

View reviewed changes

groupthinking closed this Aug 3, 2026

Navigation Menu

perf: bound L1 cache access-history retention - #1299

perf: bound L1 cache access-history retention#1299
groupthinking wants to merge 1 commit into
mainfrom
claude/determined-maxwell-gy6tub

groupthinking commented Aug 3, 2026

Uh oh!

vercel Bot commented Aug 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

coderabbitai Bot commented Aug 3, 2026 •

edited

Loading

Review skipped

Uh oh!

github-actions Bot commented Aug 3, 2026 •

edited

Loading

Uh oh!

groupthinking commented Aug 3, 2026

Uh oh!

vercel Bot left a comment

Uh oh!

groupthinking commented Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

groupthinking commented Aug 3, 2026

Canonical issue

Outcome

Scope

Risk

Verification

Production evidence

Agent handoff

Uh oh!

vercel Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 3, 2026

Dependency Review

Snapshot Warnings

Scanned Files

Uh oh!

coderabbitai Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-actions Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Agent Completion Truth Gate: BLOCKED

Uh oh!

groupthinking commented Aug 3, 2026

agent-completion/truth-gate red here is the known #1286 false-positive — not this diff

Uh oh!

vercel Bot left a comment

Choose a reason for hiding this comment

Uh oh!

groupthinking commented Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Aug 3, 2026 •

edited

Loading

coderabbitai Bot commented Aug 3, 2026 •

edited

Loading

github-actions Bot commented Aug 3, 2026 •

edited

Loading

`agent-completion/truth-gate` red here is the known #1286 false-positive — not this diff