Skip to content

fix: give RedisCacheLayer an event-loop ownership contract - #1368

Merged
groupthinking merged 1 commit into
mainfrom
rediscachelayer-has-no-event-loop-ownership-contract-grv-212-78b9
Aug 4, 2026
Merged

fix: give RedisCacheLayer an event-loop ownership contract#1368
groupthinking merged 1 commit into
mainfrom
rediscachelayer-has-no-event-loop-ownership-contract-grv-212-78b9

Conversation

@groupthinking

Copy link
Copy Markdown
Owner

Canonical issue

Closes #1162

Outcome

RedisCacheLayer and its redis.asyncio.ConnectionPool are now owned by exactly one event loop, enforced at every pool call site, so cross-loop misuse fails loudly (CacheLoopOwnershipError) instead of surfacing as a silent cache miss. This matters because the module builds the IntelligentCacheSystem singleton at import time, outside any loop, so one layer instance is reachable from several loops in a process.

The contract (option 1 from the issue, plus an explicit handoff):

  • Owning loop → allowed. First touch of an existing pool claims ownership.
  • Second live loop → CacheLoopOwnershipError, raised before each method's except Exception so it can't be swallowed into a None/False/0 fallback.
  • Owner loop closed → the pool is unusable by anyone; it is discarded with a WARNING and the next loop may connect() again (this is the per-loop construction story for pytest's fresh-loop-per-test).
  • New disconnect() / IntelligentCacheSystem.shutdown() are the clean handoff.

This supersedes draft #1179 and resolves its three unreviewed-defect findings:

  1. Ownership races — transitions are serialized on a threading.Lock, and connect() claims ownership before building the pool, so two loops connecting simultaneously resolve to one winner instead of both building pools and leaking the loser's. Reconnecting on the owner loop also closes the previous pool rather than abandoning it.
  2. disconnect() lifecycle — state is released only after pool.disconnect() succeeds; teardown failures propagate, so a failed disconnect can be retried instead of silently leaking a live pool.
  3. warm_cache() — re-raises CacheLoopOwnershipError instead of letting gather(return_exceptions=True) fold it into the success count.

Scope

  • Included: the contract and its enforcement in src/youtube_extension/backend/services/intelligent_cache.py (connect/disconnect/get/set/delete/clear/invalidate_by_tags), the new disconnect()/shutdown(), and regression tests.
  • Explicitly excluded: caching semantics, key layout, serialization, TTL handling, the L1 layer, and the tag-write fan-out from perf: issue Redis tag-set writes concurrently on cache set #1152. No signature changes to existing methods.

Risk

  • Risk level: low
  • Failure mode: code that today silently uses the pool from a foreign live loop now raises instead of degrading to a cache miss — intended, but loud. Single-loop use is unchanged apart from an identity check.
  • Rollback: revert the single commit; no config, schema, or dependency changes.

Verification

At head 46ca0a6:

  • Focused tests — pytest tests/unit/test_intelligent_cache.py: 200 passed (183 baseline + 17 new, zero regressions). New coverage: rejection from a real second live loop (background thread) parametrized across all 7 call sites; error not swallowed by the layer, the facade, or warm_cache(); serialized simultaneous first connect() (loser rejected mid-build, winner's pool kept); reconnect closing the previous pool; disconnect failure propagating with state intact then succeeding on retry; closed-owner discard incl. its WARNING; the import-time singleton honouring the contract; shutdown() handoff.
  • Non-vacuity — with _require_pool_loop neutered, 16 tests fail; source restored and re-verified green.
  • Lint — ruff check clean on the source file; the test file's single I001 is pre-existing.
  • Required CI
  • Review threads resolved

One existing test updated: test_tag_write_semaphore_is_replaced_after_its_loop_closes now reconnects on the second loop, as the contract requires; its assertion is unchanged.

Production evidence

Not applicable: backend correctness fix with no user-facing surface, and the failure mode is unobservable on main by construction (every affected method swallows it into a miss-shaped return). The raised error and the closed-owner WARNING are the first observability this failure mode has had.

Agent handoff

  • One canonical issue is linked
  • No competing PR implements the same issue — fix: give RedisCacheLayer an event-loop ownership contract (#1162) #1179 is a stale draft on an orphaned pre-history-rewrite base with unresolved review findings; this PR replaces it
  • Acceptance criteria are satisfied — one contract chosen and documented on the class, enforced at all call sites, regression tests incl. the import-time singleton, suite green
  • Required checks pass on the current head
  • Human decision is requested only where warranted — none needed; the closed-owner discard behaviour follows the option-1 contract plus the recovery story the issue's acceptance criteria require

Agent provenance

A redis.asyncio.ConnectionPool caches connections whose transports are
bound to the loop that opened them, and this module builds an
IntelligentCacheSystem singleton at import time, so one layer instance
is reachable from several loops in one process.

The layer and its pool are now owned by exactly one event loop, enforced
at all pool call sites (connect, disconnect, get, set, delete, clear,
invalidate_by_tags). Use from a second live loop raises
CacheLoopOwnershipError; a pool whose owner loop closed is discarded
loudly and the next loop may reconnect. Ownership transitions are
serialized on a threading.Lock, disconnect() releases state only after
the pool actually closed (failures propagate), and warm_cache()
re-raises the ownership error instead of swallowing it via
return_exceptions.

Generated with [Linear](https://linear.app/myxstack/issue/GRV-212/rediscachelayer-has-no-event-loop-ownership-contract-across-its-six#agent-session-2f0d8551)

Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
@linear-code

linear-code Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

GRV-212

@vercel

vercel Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
v0-uvai Canceled Canceled v0 Aug 4, 2026 6:13am

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • [‘architecture-gap’, ‘bug’, ‘ci-cd’, ‘ci/cd’, ‘copilot-rabbit’, ‘documentation’, ‘duplicate’, ‘enhancement’, ‘frontend’, ‘github_actions’, ‘good first issue’, ‘help wanted’, ‘high-priority’, ‘invalid’, ‘javascript’, ‘ml-model’, ‘needs-triage’, ‘pipeline-critical’, ‘placeholder-code’, ‘priority:high’, ‘python’, ‘python:uv’, ‘question’, ‘styling’, ‘tests’, ‘v0’]

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 63a5942b-72f2-4538-9378-3c1fe7b6cb8f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

🔍 PR Validation

⚠️ Large PR detected (679 lines changed)

@github-actions github-actions Bot added the python label Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 46ca0a6.
Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

Scanned Files

None

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

Agent Completion Truth Gate: BLOCKED

Reasons: invalid_payload

Machine-readable verdict
{
  "details": {
    "collection_errors": [
      "linked_issue_not_agent_task",
      "missing_intent_snapshot",
      "missing_agent_run_id",
      "missing_agent_login"
    ],
    "invalid_fields": [
      "policy.agent_login",
      "policy.run_id"
    ]
  },
  "reasons": [
    "invalid_payload"
  ],
  "verdict": "blocked"
}

Workflow evidence

@groupthinking
groupthinking marked this pull request as ready for review August 4, 2026 06:23
Copilot AI balanced review requested due to automatic review settings August 4, 2026 06:23
@groupthinking
groupthinking merged commit a631194 into main Aug 4, 2026
37 of 42 checks passed
@groupthinking
groupthinking deleted the rediscachelayer-has-no-event-loop-ownership-contract-grv-212-78b9 branch August 4, 2026 06:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds event-loop ownership enforcement and lifecycle handoff for Redis caching.

Changes:

  • Rejects cross-loop pool access with CacheLoopOwnershipError.
  • Adds disconnect() and system-level shutdown().
  • Adds ownership, race, recovery, and propagation tests.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/youtube_extension/backend/services/intelligent_cache.py Implements pool ownership and lifecycle handling.
tests/unit/test_intelligent_cache.py Adds regression coverage for loop ownership.

Comment on lines +538 to +540
old_pool = self.redis_pool
self.redis_pool = None
self._connected = False
Comment on lines +586 to +600
with self._ownership_lock:
owner = self._pool_loop
if owner is not None and owner is not loop:
if not owner.is_closed():
raise CacheLoopOwnershipError(
f"{self.name}: cannot disconnect() from {loop!r}; the "
f"pool is owned by live event loop {owner!r} and must "
"be closed on that loop."
)
# The owner died with its transports; discarding is all that
# is left to do, exactly as on any other post-mortem access.
self._discard_pool_of_dead_owner(owner, loop)
return
pool = self.redis_pool

Comment on lines +543 to +546
if old_pool is not None:
# Reconnecting on the owning loop: close the previous pool
# instead of silently abandoning its connections.
await old_pool.disconnect()
groupthinking pushed a commit that referenced this pull request Aug 5, 2026
The truth gate scores a pull request against the frozen intent snapshot on
its linked issue. That snapshot is only ever written by
snapshot-agent-task-intent, which runs on `issues` events alone -- it never
runs on pull_request_target. So a pull request with no linked issue has no
snapshot, no declared agent_login and no declared run_id, which means
policy.agent_login, policy.run_id and issue.number can never be populated
and the verdict is permanently `invalid_payload` no matter what the author
does.

Arming that unsatisfiable state from a branch-name prefix made the check red
on pull requests that never had a contract to satisfy. It is red on merged
commits too: #1368, the current tip of main, merged with
agent-completion/truth-gate/pr-1368 failing on exactly this. A check that is
red on everything gates nothing and buries real failures, which is the
failure mode agent-completion-enforcement.yml already warns about in its own
comments.

Pull-side provenance now arms the gate only when a linked issue exists to
verify against. With none there is nothing to measure, so the verdict is
not_applicable rather than blocked. This is not an escape hatch: a pull
request that links a dispatched issue is gated exactly as before, and the
requirement to bind a pull request to a focused issue at all is separately
owned by the `Canonical issue and evidence` check, which states a
requirement an author can actually meet.

Consolidates the two competing open implementations of this fix. #1364's
commit is cherry-picked here with authorship intact; #1154 carried the same
intent but had drifted to 117 files and 14k lines of unrelated changes.
Both should close in favour of this.

Full unit suite: 8079 passed, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi
groupthinking added a commit that referenced this pull request Aug 5, 2026
… audit 337 branches (#1377)

* test: stop asserting a CPython-version-specific rmtree detail

test_cleanup_is_total_for_non_oserror_failures failed on Python 3.11.15:
its premise asserted that shutil.rmtree(path, ignore_errors=True) raises
ValueError on a NUL-byte path. That is an implementation detail which has
changed -- ignore_errors now absorbs the non-OSError as well, so the
assertion no longer holds.

The contract under test is unaffected: _cleanup_download_artifacts must
swallow non-OSError failures because it runs from a finally block and
would otherwise replace the in-flight exception. Establish that premise
against the unguarded rmtree call, which still raises, so the test proves
the helper's own defensiveness rather than the stdlib's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi

* fix(ci): stop arming truth gate from bare agent labels on linked issues

agentTaskApplicable() unioned PR labels with linked-issue labels, so a
bare agent-task/mcp-agent label applied by label automation to an issue
that never declared a contract judged any closing PR an agent
completion. The gate then required an Agent Run ID / Agent Login the
issue never declared, yielding a permanent blocked/invalid_payload
verdict no author could satisfy.

Applicability now comes from PR provenance (known agent author, agent
branch prefix, agent label on the PR, lock manifest) or from a genuine
issue-side dispatch: an agent-task/mcp-agent label AND declared Agent
Run ID + Agent Login headings. The generic agent label remains a
PR-side signal only, since the snapshot job and collector never
recognise it issue-side — arming from it would block permanently as
linked_issue_not_agent_task with no snapshot to satisfy.

The collector now emits a mislabelled_agent_task core.notice when a
linked issue carries a contract label without declaring the contract,
keyed on the missing contract itself (not inapplicability) so
Dependabot PRs linked to valid contracts never get a false notice.

Generated with [Linear](https://linear.app/myxstack/issue/GRV-196/agent-completiontruth-gate-is-permanently-unsatisfiable-for-any-pr#agent-session-8e5d62f0)

Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>

* fix(ci): require a linked issue before arming the truth gate

The truth gate scores a pull request against the frozen intent snapshot on
its linked issue. That snapshot is only ever written by
snapshot-agent-task-intent, which runs on `issues` events alone -- it never
runs on pull_request_target. So a pull request with no linked issue has no
snapshot, no declared agent_login and no declared run_id, which means
policy.agent_login, policy.run_id and issue.number can never be populated
and the verdict is permanently `invalid_payload` no matter what the author
does.

Arming that unsatisfiable state from a branch-name prefix made the check red
on pull requests that never had a contract to satisfy. It is red on merged
commits too: #1368, the current tip of main, merged with
agent-completion/truth-gate/pr-1368 failing on exactly this. A check that is
red on everything gates nothing and buries real failures, which is the
failure mode agent-completion-enforcement.yml already warns about in its own
comments.

Pull-side provenance now arms the gate only when a linked issue exists to
verify against. With none there is nothing to measure, so the verdict is
not_applicable rather than blocked. This is not an escape hatch: a pull
request that links a dispatched issue is gated exactly as before, and the
requirement to bind a pull request to a focused issue at all is separately
owned by the `Canonical issue and evidence` check, which states a
requirement an author can actually meet.

Consolidates the two competing open implementations of this fix. #1364's
commit is cherry-picked here with authorship intact; #1154 carried the same
intent but had drifted to 117 files and 14k lines of unrelated changes.
Both should close in favour of this.

Full unit suite: 8079 passed, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi

* chore(maintenance): audit all 337 remote branches and add a prune script

Classifies every remote branch using the one signal that stays honest after
the secret-purge force-push: whether it shares any ancestry with `main`.

    $ git merge-base origin/main origin/<branch>
    (empty)

275 of 337 branches return empty -- they predate the rewrite and no rebase
recovers them. The signals the branch-cleanup harness normally leans on all
mislead here, so they are deliberately not used:

  * `git merge-tree` calls these orphans a CLEAN merge; unrelated trees do
    not textually conflict, they would clobber.
  * A two-dot diff against an empty merge base silently degrades to a
    working-tree diff, which is why a two-line Dependabot bump measures as
    111 files / 15,650 lines.
  * The purge rewrote committer dates, so every branch reads as under 30
    days old and no staleness threshold ever fires.

Running the stock harness on this repo produced 263 REVIEW off those bad
signals. The ancestry test resolves the same set into:

    KEEP-OPEN-PR     28
    REVIEW-SHARED    29   real shared ancestry, no open PR -- not pruned
    CLOSE-MERGED      1   tip is an ancestor of main
    CLOSE-ORPHANED  275

The script prunes only the last two groups (276 branches) and archive-tags
each one first, verifying every tag is on the remote before deleting
anything. It defaults to a dry run.

That dry run caught the audit classifying `main` itself as CLOSE-MERGED --
`git merge-base --is-ancestor origin/main origin/main` is trivially true.
The row is removed, and a protected-ref guard plus a default-branch check
now abort regardless of what the CSV contains.

Not executed here: this session's credentials are scoped to one branch and
tag creation fails with HTTP 403, so the archive tags cannot be written.
Deleting without them would remove the only durable recovery path, so
nothing was deleted. Run the script with tag-write credentials.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi

* docs(maintenance): correct prune-script header counts to match the CSV

The header summary read "CLOSE-MERGED 2 / 277 branches" while the shipped
docs/branch-audit-2026-08-05.csv has 1 CLOSE-MERGED and 276 prunable. The
header was written before the `main` row was dropped from the CSV and was
never updated.

Comment only -- the selection logic already read from the CSV, so the script
was correctly selecting 276 the whole time. Verified: header, CSV tally, and
the script's own dry-run count now all agree at 276.

Reported by the Vercel review bot on #1377.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RedisCacheLayer has no event-loop ownership contract across its six redis_pool call sites

2 participants