Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions memories/gh-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -429,6 +429,29 @@
Org-level rulesets (`gh api "orgs/<org>/rulesets"`) can also gate branches in member repos and would still return "nothing required" with the repo queries alone;
add that sweep when the repo belongs to an org.

**One endpoint answers both scopes, and `bypass_actors` is what turns a merge into evidence.**
`repos/<o>/<r>/rules/branches/<branch>` returns the rules actually **in effect** on that branch, org-level rulesets included, so it needs no separate org sweep;
each entry carries `ruleset_source_type` and `ruleset_id`, which is how a repo rule is told from an org one.
Then read the bypass fields off the single-ruleset object before treating a successful merge as evidence about the gates:
`mergeable_state: "clean"` says no required check is missing only when the actor could not have bypassed the rule instead.
```bash
gh api "repos/<o>/<r>/rules/branches/<branch>" \
--jq '.[] | "\(.type) \(.ruleset_source_type) \(.ruleset_id)"'
gh api "repos/<o>/<r>/rulesets/<id>" --jq '{current_user_can_bypass, bypass_actors}'
```
`current_user_can_bypass` is one of `always`, `pull_requests_only`, `never`.
`bypass_actors` can be **absent from the response entirely** rather than an empty array, so test for the key rather than for a length.
(Measured 2026-08-31 against two rulesets, both reporting
`current_user_can_bypass: "never"` and no `bypass_actors` key at all:
`Morrison-Lab/ai-config` ruleset 17712474 and `UCD-SERG/shigella` ruleset 6339629.
The second is worth its own mention because it is how the trap was met:
a reader who tests the value rather than the key sees `None` from
`dict.get`, reports it as `bypass_actors: null`, and has recorded a field
that is not in the response.)
These are plain REST endpoints, so `curl` with `GH_TOKEN` reaches every one of them in a session with no `gh` on `PATH` and no ruleset MCP tool.
Do not read the absence of such a tool as the settings being unreadable, per
[`growth-mindset`](../shared/workflow/growth-mindset.md)'s "A limitation you never tested leaves no error to diagnose" (UCD-SERG/shigella#46, 2026-08-31).

**Required checks are not the only thing a ruleset carries -- Copilot code review is turned on there too.**
A `copilot_code_review` rule schedules Copilot itself, so nothing in the PR requests the review and no per-PR reviewer entry explains where it came from.
Read it off the same endpoint:
Expand Down
2 changes: 1 addition & 1 deletion shared/workflow/challenge-the-assignment.md
Original file line number Diff line number Diff line change
Expand Up @@ -654,7 +654,7 @@ the author had already read the same file twice without finding it.)
- [`metacognitive-monitoring`](metacognitive-monitoring.md) governs a premise
stated as background fact, and the claims you generate yourself.
This governs the instruction, which asserts nothing and so trips none of its
five claim types.
claim types.
Its stripping pass is the nearest authoring-side rule, and the section above
says why it does not reach a load-bearing premise.
- [`derive-dont-enumerate`](derive-dont-enumerate.md) also tells an author to
Expand Down
25 changes: 25 additions & 0 deletions shared/workflow/growth-mindset.cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,28 @@ as a flat denial, and answered by searching the MCP registry and plugin
catalog for a GitHub Discussions server to install.
Neither could have helped -- a local server sits behind the same proxy.
The REST route the refusal named worked on the first attempt.)

## "A limitation you never tested" --- branch-protection settings reported unreadable across several turns, never once queried

(`UCD-SERG/shigella#46`, 2026-08-31: across several turns a session told the
user that GitHub branch-protection and ruleset settings were "not readable
from this session" and that confirming a required-checks list needed a human
to open Settings.
The claim was inferred from the absence of a dedicated MCP tool for rulesets,
never tested, and then restated across several turns --- including in a merged
PR body and in a filed issue, where it became a "needs a human with
branch-protection access" note that outlived the conversation.

`GH_TOKEN` was set in the environment throughout, and plain `curl` against the
REST API answered every part of the question on the first attempt:
`/repos/{owner}/{repo}/rulesets` for the list,
`/repos/{owner}/{repo}/rulesets/{id}` for a ruleset's rules and
`bypass_actors`, `/repos/{owner}/{repo}/rules/branches/{branch}` for the rules
in effect including org-level ones, and
`/repos/{owner}/{repo}/branches/{branch}` for the classic protection summary.
The endpoints are recorded in
[`memories/gh-cli.md`](../../memories/gh-cli.md).

The false claim was load-bearing rather than incidental: it is what put the
"a human has to check this" note into two deliverables, so the cost was not a
mistaken sentence in chat but a premise shipped to later readers.)
61 changes: 61 additions & 0 deletions shared/workflow/growth-mindset.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,67 @@ still the right move, but it is the case where reading it does not help.
- **Don't:** reach for installing something to get past a sandbox boundary;
what you install inherits the boundary.

## A limitation you never tested leaves no error to diagnose

The three sections above all begin with a call that was made and came back
wrong: a tool errored, a bounded probe returned nothing, a request was
refused.
Each remedy reads the artifact that call produced.
None of them reaches the case where no call was made at all.

The shape is a negative claim about your own capability.
"That is not readable from this session."
"Confirming it needs a human with access to the settings page."
Such a claim is inferred from the shape of the tool surface --- no dedicated
tool covers the thing, so the data must be out of reach --- and a tool listing
is a menu rather than a boundary.

Three things make it worse than an ordinary wrong guess.

**It produces no failure to notice.**
Each of the three cases above announces itself with an error, a timeout, or a
status code, and each remedy is to read that output more carefully.
This one leaves nothing red, nothing logged, and no output to re-read.
The cheap test that would refute it is exactly the test the claim tells you
not to bother running, so the claim protects itself.

**A positive claim gets tested by being acted on; this one never is.**
"I can read X" leads to reading X, which either works or does not.
"I cannot read X" ends the inquiry, so nothing downstream ever disagrees with
it, and repeating it across several turns feels like consistency rather than
like an unexamined premise hardening.

**It ships.**
A capability claim is a natural thing to write into a PR body or an issue as
a "needs a human with access" note, where it becomes a premise for whoever
reads it next and outlives the conversation that could have corrected it.
That is [`challenge-the-assignment`](challenge-the-assignment.md)'s
"An issue body is an assignment you author" case with a capability claim in
place of a count, and it puts the sentence under the same bar as any other
factual claim in a deliverable, per
[`fact-check-prose`](../writing/fact-check-prose.md).

The test is one call, and it costs less than the sentence asserting the
limitation.
Before writing that something cannot be read from this session, run the
plainest route: a raw HTTP request against the service's documented API with
whatever credential the environment already holds.
An MCP tool, a CLI subcommand, and a raw request are three routes to one API,
so the absence of the first two is no evidence about the third.

- **Do:** attempt the plainest available route once, and report what came
back, before claiming a thing cannot be read from this session.
- **Do:** hold a capability claim in a PR body, an issue, or a handoff to the
same standard as any other factual claim in a deliverable.
- **Don't:** infer a limitation from the tool listing --- it enumerates what
is convenient, not what is reachable.
- **Don't:** count having repeated the limitation across turns as having
established it; a claim that ends the inquiry can never be contradicted by
it.

See [`growth-mindset.cases.md`](growth-mindset.cases.md), "branch-protection
settings reported unreadable across several turns, never once queried".

## Applies to our own metacognitive tooling, too

The same bias governs the skills, memories, and self-improvement loops in these
Expand Down
18 changes: 13 additions & 5 deletions shared/workflow/metacognitive-monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ check.

- **State** --- is it green, is it pushed, does it exist, is it public.
Re-query, never recall.
- **Capability** --- I cannot read that, no tool covers it, that needs a human
with access.
Attempt the plainest route once and report what came back.
There is nothing to re-query when no call was ever made, which is why this
is not the **State** case:
see [`growth-mindset`](growth-mindset.md)'s "A limitation you never tested
leaves no error to diagnose".
- **Scope** --- all, every, none, only, the whole corpus.
Check the population rather than the sample that came to mind.
- **Cause** --- it failed because, this is flaky, that change broke it.
Expand All @@ -34,8 +41,8 @@ check.

## A premise you were handed is still a claim

All five types above describe assertions **you** generate, so all five trigger
on the act of writing one.
Every type above describes an assertion **you** generate, so each one triggers
on the act of writing it.

- **Do:** restate a load-bearing premise explicitly and name what would
falsify it, before building on it.
Expand Down Expand Up @@ -179,7 +186,7 @@ Each was fixed on its own terms and none of the fixes generalized.)

## An action you recommend is a claim about state

The five types above fire on an assertion, and the section above extends them
The claim types above fire on an assertion, and the section above extends them
to a premise you were handed.

- **Do:** re-query an artifact's state immediately before recommending an
Expand Down Expand Up @@ -1072,8 +1079,9 @@ write the thing that can be wrong.

## Do and don't

- **Do:** classify each assertion as state, scope, cause, inference, or default
before it goes out, and re-measure any that is not from this turn.
- **Do:** classify each assertion as state, capability, scope, cause,
inference, or default before it goes out, and re-measure any that is not
from this turn.
- **Do:** name the falsifying command beside a claim, and run it when it is
cheap.
- **Do:** treat a fluent, undeliberated answer as owing an alternative you can
Expand Down
8 changes: 4 additions & 4 deletions shared/workflow/metacognitive-monitoring.rationale.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,14 +39,14 @@ check.
Claim **type** can, because it is observable in the sentence you are about to
write:

The five are worth keeping as a list rather than collapsing into "check your
They are worth keeping as a list rather than collapsing into "check your
claims", because each names a different *repair*, and the repair is the part
that is easy to skip.

## A premise you were handed is still a claim

All five types above describe assertions **you** generate, so all five trigger
on the act of writing one.
Every type above describes an assertion **you** generate, so each one triggers
on the act of writing it.
A premise supplied by the user triggers on nothing.
It arrives as context rather than as a claim, you did not reason your way to
it, and adopting it feels like listening rather than like asserting.
Expand Down Expand Up @@ -295,7 +295,7 @@ ends up protected while the person does not.

## An action you recommend is a claim about state

The five types above fire on an assertion, and the section above extends them
The claim types above fire on an assertion, and the section above extends them
to a premise you were handed.
Both are things somebody states.
A recommendation states nothing about the world.
Expand Down
Loading