ci(release): block publication on agent-server stress regressions

[chrischen-coder]

HUMAN:
I authorized this contribution and will take responsibility for reviewing it and responding to maintainer feedback.

---

AGENT:

Why

The existing stress suite runs on relevant PRs, but release publication does not
depend on it. If a release PR is merged with checks bypassed or against a bad
merge result, create-release.yml immediately creates the tag and GitHub Release
and dispatches PyPI, image, and binary publication.

That gap is high impact because the regressions this suite targets are
process-wide: before #4570, one conversation stuck in close() held a global
lifecycle lock, so create, load, and delete for every unrelated conversation
wedged even while /health remained responsive. A package release can therefore
look healthy and still make the agent server unusable for all active users.

This follows the same principle as production agent harnesses such as
DeepSeek Harness: inject a
controlled failure into the real execution path, assert a deterministic
invariant, and make that invariant a release boundary rather than advisory CI.

Summary

• Add a credential-free stress regression that stalls one real
  EventService.close() and proves unrelated conversation create/load/delete
  operations still finish within a documented five-second CI budget.
• Run the complete agent-server stress suite against the exact release PR merge
  commit before create-release can create any tag or dispatch publication.
• Update the release runbook to document the blocking gate and the actual
  automated release flow.

Issue Number

Fixes #4588

How to Test

Focused reviewer command:

uv run pytest -q -m stress \
  tests/agent_server/stress/test_lifecycle_isolation.py --no-cov

The equivalent local command executed in this macOS workspace was
.venv/bin/python -m pytest with the same arguments. Observed:

1 passed, 40 warnings in 0.19s

Full reviewer/release-gate command:

CI=true uv run python -m pytest -vvs \
  -m stress \
  --durations=10 \
  tests/agent_server/stress

The equivalent full local run used .venv/bin/python -m pytest with the same
arguments. Observed:

14 passed, 17012 warnings in 69.77s

Historical fail-before/pass-after proof:

• Loaded the production packages from 611629e6, the first parent of the
  fix(agent-server): replace global _lifecycle_lock with per-conversation locks #4570 merge and therefore the last tree with the process-wide lifecycle lock.
• Ran the exact new stress test without changing its test or budget code.
• It failed after 5.26 seconds with:

AssertionError: unrelated conversation create/load/delete operations blocked
behind a stuck close; lifecycle work may be globally serialized

Static validation:

uv run pre-commit run --files \
  .github/workflows/create-release.yml \
  .github/workflows/README-RELEASE.md \
  tests/agent_server/stress/budgets.py \
  tests/agent_server/stress/test_lifecycle_isolation.py

All hooks passed, including YAML formatting, Ruff, Pyright, import rules, and
tool registration.

Video/Screenshots

Not applicable. This is a backend lifecycle regression and release workflow;
the direct pass/fail terminal evidence is included above.

Design Doc

Not applicable. The change is test/workflow-only and does not alter a public API
or production runtime behavior.

Type

• Bug fix
• Feature
• Refactor
• Breaking change
• Docs / chore

Notes

• The PR-time stress job remains path-filtered. The new release gate is
  unconditional for merged rel-* PRs and is a hard needs dependency of the
  publication job.
• Tests use the existing in-process FastAPI/ConversationService harness and
  fake LLMs; no paid model credentials or external services are required.
• No agent prompts, tool behavior, model routing, or benchmark-sensitive
  production code changes.

[neubig]

Thanks for the submission and for adding a real-path lifecycle-isolation stress test with the existing credential-free fake LLM harness.

Before this PR is satisfactory, please add concrete fail-before/pass-after evidence for every regression named in acceptance criterion 3 of issue #4588. The current description only documents the lifecycle-lock case (#4570/#4514). Please cover:

• #4570 / #4514: lifecycle lock
• #4481: blocking event search/listing work
• #4417: blocking ConversationInfo composition
• #4473: serialized LLM calls

For each regression, add a short matrix to the PR description or repository documentation identifying:

1. The stress test or scenario that detects it.
2. The pre-fix commit or revert used.
3. The exact command and failure observed on the pre-fix tree.
4. The exact command and passing result on current main.

Running the current stress suite successfully shows that the gate executes, but it does not establish that the suite catches the other historical regressions required by the issue. Once this regression mapping and evidence are included, the acceptance-criteria coverage will be clear.

AI disclosure: This review was generated by an AI agent (OpenHands) on behalf of the requester.