Skip to content

docs(agent): make SUMMARY.md's onward format authoritative over the task prompts - #761

Open
srperens wants to merge 2 commits into
mainfrom
docs/agent-onward-authority
Open

docs(agent): make SUMMARY.md's onward format authoritative over the task prompts#761
srperens wants to merge 2 commits into
mainfrom
docs/agent-onward-authority

Conversation

@srperens

@srperens srperens commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

The Slack message still has no links

The 10:33 notification from the review task, at main@ccb8593:

Agent review — main@ccb8593, 4 items
Needs you: #722 (dispatch macOS CI), #690 (release sign-off), #700 (clamp/reject/rename), #737 (keep-or-close), #749 (non-blocking nit)
Reviewed: #724 Approve, #749 Comment, #722 Comment, #752 Approve
Skipped 16 (drafts/dep bumps); left by cap: #753 #754
Queue: 0 candidates, 16 awaiting reply, 0 untriaged

Nine references, none of them clickable. #722 autolinks only inside this repository; in
Slack it is four characters you have to go search for.

#751 was not wrong, and it did land

ccb8593 has 2b1013b as its parent, so that run read the SUMMARY.md that already
required <https://github.com/Eyevinn/strom/issues/721|#721> and one numbered line per item,
and posted the old shape anyway.

The cause is outside this repo. Both task prompts end with:

Then post one message to the notification channel, rendered from the summary's
needs_human and item lists — five lines at most […] Never put a token, a remote
URL
or a raw log excerpt in the message.

A prompt is the outer instruction. Those two sentences did not drift against the protocol
file — they overrode it, and between them they leave one crammed line of bare refs as the
only legal output. That is exactly what shipped, while the file looked correct the whole time.
README.md's own promise that "the task definitions only need updating […] not when a rule
inside a file changes" is what made it invisible.

What this changes

Nothing about the required format — #751 already specified it. This records where the boundary
is, in the two places a later change reads:

  • SUMMARY.md — "Reporting onward" now states that it owns the format, and that a task
    definition stating a line ceiling or a reference rule is by definition out of date: follow
    the file, and record the conflict in the summary. A run that meets a stale prompt again now
    has an instruction for that case rather than two contradictory ones.
  • README.md — a design note that a prompt must never restate a rule these files own,
    with this as the worked example, and "Changing the protocol" no longer promises that editing
    a file here suffices unconditionally.

The prompts themselves

Deployment configuration, outside this repo. Both have had the five-line cap and the remote-URL
ban removed and now defer to SUMMARY.md for the format; both tasks are updated. Their
reporting sections also gained the payload encoding, which the old five-line shape hid: a
message with one line per item cannot go through a hand-escaped -d '{"text":"…"}' — the first
newline or backtick makes it invalid JSON and Slack answers "ok":false with no other symptom.
They now pipe the rendered message through jq -Rs (with a python3 -c json.dumps fallback)
and post -d @payload.json.

Tests

scripts/agent/test-agent-scripts.sh — 40 passed, 0 failed. Unchanged: this commit touches no
script. Nothing else here is executable; the check that matters is the next 14:00 UTC run's
message.

🤖 Generated with Claude Code

srperens and others added 2 commits September 4, 2026 10:48
…ask prompts

The Slack notification still arrives as one crammed line of bare #721 refs
that nobody can click, two days after #751 required linked, one-line-per-item
output. #751 was not wrong and it did land -- the run at main@ccb8593 read this
SUMMARY.md and posted the old shape anyway.

The cause is that both task prompts end with "rendered from the summary's
needs_human and item lists -- five lines at most" and "Never put a token, a
remote URL or a raw log excerpt in the message". A prompt is the outer
instruction, so those two sentences do not drift against the file; they
override it. Between them they leave a single line of bare refs as the only
legal output, which is exactly what shipped. The protocol file looked correct
the whole time.

So this records where that boundary is, in the two places a later change would
have to read:

- SUMMARY.md's "Reporting onward" now says it owns the format, and that a task
  definition stating a line ceiling or a reference rule is by definition out of
  date -- follow the file and record the conflict in the summary. A run that
  hits the stale prompt again now has an instruction for it.
- README.md gains the design note that a prompt must never restate a rule these
  files own, and "Changing the protocol" no longer promises that editing a file
  here is sufficient without that condition holding.

The prompts themselves are deployment configuration and live outside this repo;
both have had the five-line cap and the URL ban removed, and now defer to this
file for the format.

scripts/agent/test-agent-scripts.sh: 40 passed, 0 failed (unchanged -- this
commit touches no script).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The window between #751 landing (2026-09-03 11:02) and the crammed message that
prompted this branch (2026-09-04 10:33) is two review runs, not the indefinite
stretch "long after" implies. The previous commit's message says "two days" for
the same reason and is off by a day; the count of runs is the number that
matters, since each one was a chance for the stale prompt to win and did.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@srperens srperens left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: Approve — accurately diagnosed and correctly fixed at the doc layer; no code, no radius beyond the two files touched. Filed as a COMMENT-state review, not an APPROVED one: this PR's author is this same account, and GitHub's addPullRequestReview refuses self-approval outright. The marker below still records verdict=Approve because that is the actual conclusion; a maintainer's own approving review is what should gate the merge.

Claims

Claim Verdict Evidence
ccb8593's parent already required linked, one-line-per-item onward output CONFIRMED repos/Eyevinn/strom/commits/ccb8593 parent is 2b1013b; scripts/agent/SUMMARY.md at 2b1013b:131 already reads "Every reference is a full URL, and every item is its own line."
No script logic changed by this diff CONFIRMED scripts/agent/test-agent-scripts.sh — 40 passed, 0 failed, run on this branch; diff touches only README.md and SUMMARY.md
New text doesn't collide with other doc language CONFIRMED grepped scripts/agent/ for "only need updating" and "owns that message" — each string appears exactly once, at the two lines this diff adds/edits

Diagnosis — Root cause matches the body: a task-definition prompt is the outer instruction, so restating a rule a protocol file owns doesn't drift against the file, it silently overrides it — the file can look correct while every run ignores it. The fix is the right layer: it doesn't touch the format rule itself (already correct per the parent-commit check above), it adds the missing statement of precedence to README.md (design note, generalizable to any future file/prompt pair) and to SUMMARY.md (this file owns the onward format; a conflicting prompt is stale, record it and follow the file). That closes the class, not just this one instance — scripts/agent/README.md:82-88 no longer promises unconditionally that editing a file here is sufficient.

RadiusLOCAL: two markdown files inside scripts/agent/, no script, code, or test references the exact wording removed or added (checked above), and nothing outside this doc set consumes README.md/SUMMARY.md programmatically.

Tests & CICheck (Linux), both Build (Linux ...), Check & Build (WASM), API Contract Check green at bfb18ab; expected and uninformative for a docs-only diff — there's no CI target that exercises agent-protocol prose. The PR's own test claim (test-agent-scripts.sh, 40/0) is reproduced above rather than taken on faith.

Design record — Adequate: body states the trigger (a live incident, quoted), the two-file split (behavioral rule vs. file-precedence rule), and explicitly scopes out the deployment-side prompt fix as outside this repo.

Confidence: HIGH

@srperens srperens mentioned this pull request Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant