diff --git a/memories/gh-cli.md b/memories/gh-cli.md index 955e48b8d..75ab638fa 100644 --- a/memories/gh-cli.md +++ b/memories/gh-cli.md @@ -429,6 +429,29 @@ Org-level rulesets (`gh api "orgs//rulesets"`) can also gate branches in member repos and would still return "nothing required" with the repo queries alone; add that sweep when the repo belongs to an org. + **One endpoint answers both scopes, and `bypass_actors` is what turns a merge into evidence.** + `repos///rules/branches/` returns the rules actually **in effect** on that branch, org-level rulesets included, so it needs no separate org sweep; + each entry carries `ruleset_source_type` and `ruleset_id`, which is how a repo rule is told from an org one. + Then read the bypass fields off the single-ruleset object before treating a successful merge as evidence about the gates: + `mergeable_state: "clean"` says no required check is missing only when the actor could not have bypassed the rule instead. + ```bash + gh api "repos///rules/branches/" \ + --jq '.[] | "\(.type) \(.ruleset_source_type) \(.ruleset_id)"' + gh api "repos///rulesets/" --jq '{current_user_can_bypass, bypass_actors}' + ``` + `current_user_can_bypass` is one of `always`, `pull_requests_only`, `never`. + `bypass_actors` can be **absent from the response entirely** rather than an empty array, so test for the key rather than for a length. + (Measured 2026-08-31 against two rulesets, both reporting + `current_user_can_bypass: "never"` and no `bypass_actors` key at all: + `Morrison-Lab/ai-config` ruleset 17712474 and `UCD-SERG/shigella` ruleset 6339629. + The second is worth its own mention because it is how the trap was met: + a reader who tests the value rather than the key sees `None` from + `dict.get`, reports it as `bypass_actors: null`, and has recorded a field + that is not in the response.) + These are plain REST endpoints, so `curl` with `GH_TOKEN` reaches every one of them in a session with no `gh` on `PATH` and no ruleset MCP tool. + Do not read the absence of such a tool as the settings being unreadable, per + [`growth-mindset`](../shared/workflow/growth-mindset.md)'s "A limitation you never tested leaves no error to diagnose" (UCD-SERG/shigella#46, 2026-08-31). + **Required checks are not the only thing a ruleset carries -- Copilot code review is turned on there too.** A `copilot_code_review` rule schedules Copilot itself, so nothing in the PR requests the review and no per-PR reviewer entry explains where it came from. Read it off the same endpoint: diff --git a/shared/workflow/challenge-the-assignment.md b/shared/workflow/challenge-the-assignment.md index accd79e37..5b27f0fc2 100644 --- a/shared/workflow/challenge-the-assignment.md +++ b/shared/workflow/challenge-the-assignment.md @@ -654,7 +654,7 @@ the author had already read the same file twice without finding it.) - [`metacognitive-monitoring`](metacognitive-monitoring.md) governs a premise stated as background fact, and the claims you generate yourself. This governs the instruction, which asserts nothing and so trips none of its - five claim types. + claim types. Its stripping pass is the nearest authoring-side rule, and the section above says why it does not reach a load-bearing premise. - [`derive-dont-enumerate`](derive-dont-enumerate.md) also tells an author to diff --git a/shared/workflow/growth-mindset.cases.md b/shared/workflow/growth-mindset.cases.md index 93192c113..db3b63f6c 100644 --- a/shared/workflow/growth-mindset.cases.md +++ b/shared/workflow/growth-mindset.cases.md @@ -43,3 +43,28 @@ as a flat denial, and answered by searching the MCP registry and plugin catalog for a GitHub Discussions server to install. Neither could have helped -- a local server sits behind the same proxy. The REST route the refusal named worked on the first attempt.) + +## "A limitation you never tested" --- branch-protection settings reported unreadable across several turns, never once queried + +(`UCD-SERG/shigella#46`, 2026-08-31: across several turns a session told the +user that GitHub branch-protection and ruleset settings were "not readable +from this session" and that confirming a required-checks list needed a human +to open Settings. +The claim was inferred from the absence of a dedicated MCP tool for rulesets, +never tested, and then restated across several turns --- including in a merged +PR body and in a filed issue, where it became a "needs a human with +branch-protection access" note that outlived the conversation. + +`GH_TOKEN` was set in the environment throughout, and plain `curl` against the +REST API answered every part of the question on the first attempt: +`/repos/{owner}/{repo}/rulesets` for the list, +`/repos/{owner}/{repo}/rulesets/{id}` for a ruleset's rules and +`bypass_actors`, `/repos/{owner}/{repo}/rules/branches/{branch}` for the rules +in effect including org-level ones, and +`/repos/{owner}/{repo}/branches/{branch}` for the classic protection summary. +The endpoints are recorded in +[`memories/gh-cli.md`](../../memories/gh-cli.md). + +The false claim was load-bearing rather than incidental: it is what put the +"a human has to check this" note into two deliverables, so the cost was not a +mistaken sentence in chat but a premise shipped to later readers.) diff --git a/shared/workflow/growth-mindset.md b/shared/workflow/growth-mindset.md index 3af6b0c5f..e5a1ee357 100644 --- a/shared/workflow/growth-mindset.md +++ b/shared/workflow/growth-mindset.md @@ -175,6 +175,67 @@ still the right move, but it is the case where reading it does not help. - **Don't:** reach for installing something to get past a sandbox boundary; what you install inherits the boundary. +## A limitation you never tested leaves no error to diagnose + +The three sections above all begin with a call that was made and came back +wrong: a tool errored, a bounded probe returned nothing, a request was +refused. +Each remedy reads the artifact that call produced. +None of them reaches the case where no call was made at all. + +The shape is a negative claim about your own capability. +"That is not readable from this session." +"Confirming it needs a human with access to the settings page." +Such a claim is inferred from the shape of the tool surface --- no dedicated +tool covers the thing, so the data must be out of reach --- and a tool listing +is a menu rather than a boundary. + +Three things make it worse than an ordinary wrong guess. + +**It produces no failure to notice.** +Each of the three cases above announces itself with an error, a timeout, or a +status code, and each remedy is to read that output more carefully. +This one leaves nothing red, nothing logged, and no output to re-read. +The cheap test that would refute it is exactly the test the claim tells you +not to bother running, so the claim protects itself. + +**A positive claim gets tested by being acted on; this one never is.** +"I can read X" leads to reading X, which either works or does not. +"I cannot read X" ends the inquiry, so nothing downstream ever disagrees with +it, and repeating it across several turns feels like consistency rather than +like an unexamined premise hardening. + +**It ships.** +A capability claim is a natural thing to write into a PR body or an issue as +a "needs a human with access" note, where it becomes a premise for whoever +reads it next and outlives the conversation that could have corrected it. +That is [`challenge-the-assignment`](challenge-the-assignment.md)'s +"An issue body is an assignment you author" case with a capability claim in +place of a count, and it puts the sentence under the same bar as any other +factual claim in a deliverable, per +[`fact-check-prose`](../writing/fact-check-prose.md). + +The test is one call, and it costs less than the sentence asserting the +limitation. +Before writing that something cannot be read from this session, run the +plainest route: a raw HTTP request against the service's documented API with +whatever credential the environment already holds. +An MCP tool, a CLI subcommand, and a raw request are three routes to one API, +so the absence of the first two is no evidence about the third. + +- **Do:** attempt the plainest available route once, and report what came + back, before claiming a thing cannot be read from this session. +- **Do:** hold a capability claim in a PR body, an issue, or a handoff to the + same standard as any other factual claim in a deliverable. +- **Don't:** infer a limitation from the tool listing --- it enumerates what + is convenient, not what is reachable. +- **Don't:** count having repeated the limitation across turns as having + established it; a claim that ends the inquiry can never be contradicted by + it. + +See [`growth-mindset.cases.md`](growth-mindset.cases.md), "branch-protection +settings reported unreadable across several turns, never once queried". + ## Applies to our own metacognitive tooling, too The same bias governs the skills, memories, and self-improvement loops in these diff --git a/shared/workflow/metacognitive-monitoring.md b/shared/workflow/metacognitive-monitoring.md index 22f9a8e21..b285f945c 100644 --- a/shared/workflow/metacognitive-monitoring.md +++ b/shared/workflow/metacognitive-monitoring.md @@ -22,6 +22,13 @@ check. - **State** --- is it green, is it pushed, does it exist, is it public. Re-query, never recall. +- **Capability** --- I cannot read that, no tool covers it, that needs a human + with access. + Attempt the plainest route once and report what came back. + There is nothing to re-query when no call was ever made, which is why this + is not the **State** case: + see [`growth-mindset`](growth-mindset.md)'s "A limitation you never tested + leaves no error to diagnose". - **Scope** --- all, every, none, only, the whole corpus. Check the population rather than the sample that came to mind. - **Cause** --- it failed because, this is flaky, that change broke it. @@ -34,8 +41,8 @@ check. ## A premise you were handed is still a claim -All five types above describe assertions **you** generate, so all five trigger -on the act of writing one. +Every type above describes an assertion **you** generate, so each one triggers +on the act of writing it. - **Do:** restate a load-bearing premise explicitly and name what would falsify it, before building on it. @@ -179,7 +186,7 @@ Each was fixed on its own terms and none of the fixes generalized.) ## An action you recommend is a claim about state -The five types above fire on an assertion, and the section above extends them +The claim types above fire on an assertion, and the section above extends them to a premise you were handed. - **Do:** re-query an artifact's state immediately before recommending an @@ -1072,8 +1079,9 @@ write the thing that can be wrong. ## Do and don't -- **Do:** classify each assertion as state, scope, cause, inference, or default - before it goes out, and re-measure any that is not from this turn. +- **Do:** classify each assertion as state, capability, scope, cause, + inference, or default before it goes out, and re-measure any that is not + from this turn. - **Do:** name the falsifying command beside a claim, and run it when it is cheap. - **Do:** treat a fluent, undeliberated answer as owing an alternative you can diff --git a/shared/workflow/metacognitive-monitoring.rationale.md b/shared/workflow/metacognitive-monitoring.rationale.md index 2b0185d1b..5d783fe6f 100644 --- a/shared/workflow/metacognitive-monitoring.rationale.md +++ b/shared/workflow/metacognitive-monitoring.rationale.md @@ -39,14 +39,14 @@ check. Claim **type** can, because it is observable in the sentence you are about to write: -The five are worth keeping as a list rather than collapsing into "check your +They are worth keeping as a list rather than collapsing into "check your claims", because each names a different *repair*, and the repair is the part that is easy to skip. ## A premise you were handed is still a claim -All five types above describe assertions **you** generate, so all five trigger -on the act of writing one. +Every type above describes an assertion **you** generate, so each one triggers +on the act of writing it. A premise supplied by the user triggers on nothing. It arrives as context rather than as a claim, you did not reason your way to it, and adopting it feels like listening rather than like asserting. @@ -295,7 +295,7 @@ ends up protected while the person does not. ## An action you recommend is a claim about state -The five types above fire on an assertion, and the section above extends them +The claim types above fire on an assertion, and the section above extends them to a premise you were handed. Both are things somebody states. A recommendation states nothing about the world.