Skip to content

feat: Go elides — step 3 of 3, four languages reduce instead of three - #34

Merged
ojassug merged 1 commit into
mainfrom
feat-go-region-elision
Aug 23, 2026
Merged

feat: Go elides — step 3 of 3, four languages reduce instead of three#34
ojassug merged 1 commit into
mainfrom
feat-go-region-elision

Conversation

@ojassug

@ojassug ojassug commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

Step 3 of three, and the one that changes output. §56 fixed the order and measured the precondition; #33 shipped §59 (the drift gate can see Go symbols) and §60 (Go has a validator). This adds the scanner, and Go reduces.

Closes the sole open roadmap item — §7.3 of the status doc. Elision now reduces 4 of 17 probed languages instead of 3.

What it adds

  • scanGoBraceSpans — a Go brace scanner, separate from scanBraceSpans for exactly §60's reason. Go's ` raw string spans lines, takes no escapes, and routinely holds {, } and \, so the TypeScript scanner reads `C:\path\` as an unterminated template literal and every brace after it as string content. A region boundary computed from that is not a function body. No regex-literal state, because Go has none.
  • GO_FUNCTION_HEADER = /^func\b/ — a keyword test where TypeScript needs a shape test, which is most of why Go went first. FUNCTION_HEADER matches anything ending in ), so CONTROL_FLOW_HEADER has to subtract if/for/while back out. ^func needs no subtraction: type Point struct {, Config{, switch v := x.(type) { and every closure form (go func() {, defer func() {, handler := func() {) are excluded by construction.
  • splitGoStatementsGo ends statements at a newline, not at a ;. Semicolon insertion means gofmt-ed source has almost none written down, so splitTypeScriptStatements finds only the } boundaries and calls most bodies indivisible — §50's overshoot, one language along. The boundary is a newline at depth 0, so a multi-line call, a composite literal and a nested block are each one span.
  • 'go' in REGION_ELISION_LANGUAGES, plus a Go arm on isSubstantiveRegion (stripGo).

Measured

Frozen 80-file Go corpus, target 0.3, engine varied and nothing else:

bucket files reduce fallback aggregate
application Go (cli/cli, gin, cobra) 40 32 8 27.46%
stdlib (golang/go src/) 40 25 12 19.42%
_test.go, across both 40 32 7 26.88%
source, across both 40 25 13 14.42%

§56 projected 23–28% from the ceiling, borrowing TypeScript's conversion factor. Application Go landed at the top of that range and above this repo's TypeScript at 21.22% on the same engine — the newest language is the strongest. §56's "test files are the larger prize" is confirmed end-to-end: nearly double the saving and half the fallbacks, because Go's table-driven tests put large literal slices inside function bodies.

Adherence over the 57 reducing files: median 35.8%, 20 in the 25–35% band, 17 in 35–50%, 14 above 50% — the profile TypeScript has after §50.

The 287-file main corpus is 574/574 byte-identical, 0 rows differing across 17 fields. Every Go path is gated on language === 'go', so TypeScript and Python do not move.

The ordering paid for itself twice

Neutering §59's symbol patterns and re-running step 3 reproduces the configuration §56 warned about — the region scanner without the drift gate that can witness it — on real input rather than in simulation:

scanner-first (no §59) shipped
file-route fallbacks 43/80 20/80
rows where drift measured anything 55/80 80/80
median symbolsBefore 2 8
application Go aggregate 14.45% 27.46%

32 files elide with S_k = 0.0000, astMeasured: true, both gates passing and no fallback, on 1–5 symbols that are all type: and import:. accessibility.go loses 78.4% of its tokens that way. With §59 in place the same 32 still pass — correctly, since signature-preserving elision genuinely loses no symbols — but the verdict now rests on a median of 8 real symbols instead of on an import.

And §59 is a precondition for reduction, not a tax on it. Without Go symbols many files have no symbols at all, so §33's measurement gate refuses them outright: fallbacks more than double and application Go reads 14.45%. The safety step and the reduction step turned out to be the same step.

An expectation of §56's that measurement contradicts

§56 could not measure Go's fallback rate and said so. Now: 20 of 80 (25%)18 CONSTRAINT_DIRECTIVE_LOST and 2 SEMANTIC_DRIFT_EXCEEDED.

§56 expected Go's lower comment density to make the constraint gate fire less. It does not. The gate dominates Go's fallbacks exactly as it dominates TypeScript's, at the same rate within noise — 25% against 24% (15 of 62). Density was the wrong variable: what trips it is Go's defensive comment style, and it is §7.5's still-open axis verbatim — // This never happens in practice, // Should never happen, but we, // The user data should always, // Must read more data. Every one is descriptive present tense, which §52 deliberately did not attempt.

So the largest remaining gain on Go is not in the scanner. It is in that gate.

Tests

test/unit/go-region-elision.test.ts — 13 known-answer cases, the set the widen-language skill requires before a scanner's output is used to justify a number: the raw-string form, both comment forms (including a cgo C preamble), a declaration with no body, a nested closure counted once, and braces inside string and rune literals. 9 of 13 fail against the unfixed engine. The 4 that pass both ways assert an absence and are trivially true when the scanner selects nothing, so they are controls on over-selection rather than evidence.

Ten existing tests asserted Go was not elidable and now fail on purpose. They are updated in the same commit, as this repo's contract requires, with the prior assertion recorded in each. language-support.test.ts moves its "an incidental symbol is not support" example from Go to Rust, which is in the position Go used to occupy — a struct yields type:Point through the TypeScript class regex and there is no Rust scanner, so the reasoning is unchanged even though the example had to move.

The "elision reduces TypeScript/JavaScript and Python only" explanation is corrected wherever it is emitted, and Go now reports as supported on trace.languageSupport.

Suite 737 passing, typecheck, lint and build clean.

What this does not establish

  • The Go corpus is alphabetically-first selection — deterministic, and not representative (the harness README's own caveat). 80 files against the main corpus's 287, frozen through a scratch recipe rather than the shipped one, because its roots are a temp-dir clone.
  • A signature broken across lines is silently skipped, and the corpus does not say how often. scanBraceSpans takes the header from the line carrying the {, so a gofmt-wrapped signature presents as ) error. Under-selection costs reduction, never content, which is why it ships characterized rather than fixed.
  • Reversibility was not measured separately for Go. The rehydration path is language-agnostic and covered by token-hashing-reversibility.test.ts; nothing here tested a Go elision round-tripping through a real MCP session.
  • Nothing about Rust, C or Java. Each needs its own three steps, and the header discriminator is the hard part for the C family — int foo(...) cannot be told from a prototype, where func is unambiguous.

DECISIONS §61.

🤖 Generated with Claude Code

Step 3 of three, and the one that changes output. 56 fixed the order and
measured the precondition, 59 gave the drift gate Go symbols, 60 gave Go a
validator. This adds the scanner.

  scanGoBraceSpans   a Go brace scanner. Separate from scanBraceSpans for
                     60's reason: Go's raw string spans lines, takes no
                     escapes, and holds braces and backslashes, so a TS
                     scanner reads `C:\path\` as an unterminated template
                     literal and every brace after it as string content. No
                     regex-literal state, because Go has none.
  GO_FUNCTION_HEADER /^func\b/ — a keyword test where TypeScript needs a
                     shape test, which is most of why Go went first.
                     FUNCTION_HEADER matches anything ending in `)`, so
                     CONTROL_FLOW_HEADER has to subtract if/for/while back
                     out; `^func` needs no subtraction and excludes struct
                     decls, composite literals, type switches and every
                     closure form by construction.
  splitGoStatements  Go ends statements at a newline, not at a `;`.
                     Semicolon insertion means gofmt-ed source has almost
                     none written down, so splitTypeScriptStatements finds
                     only the `}` boundaries and calls most bodies
                     indivisible — 50's overshoot, one language along.

Measured, frozen 80-file Go corpus at target 0.3:

  application Go (cli/cli, gin, cobra)   40 files   32 reduce    27.46%
  stdlib (golang/go src/)                40 files   25 reduce    19.42%
  _test.go, across both                  40 files   32 reduce    26.88%
  source, across both                    40 files   25 reduce    14.42%

56 projected 23-28%; application Go landed at the top of it and above this
repo's TypeScript at 21.22%. 56's "test files are the larger prize" is
confirmed end-to-end: nearly double the saving and half the fallbacks,
because Go's table-driven tests put large literal slices inside bodies.

The 287-file main corpus is 574/574 byte-identical, 0 rows differing across
17 fields — every Go path is gated on the language, so TypeScript and Python
do not move.

THE ORDERING PAID FOR ITSELF TWICE. Neutering 59's symbol patterns and
re-running step 3 reproduces 56's hazard on real input rather than in
simulation:

                                    scanner-first (no 59)     shipped
  file-route fallbacks                    43/80               20/80
  rows where drift measured anything      55/80               80/80
  median symbolsBefore                        2                   8
  application Go aggregate                14.45%             27.46%

32 files elide at S_k = 0.0000 with astMeasured: true, both gates passing and
no fallback, on 1-5 symbols that are all type: and import:. accessibility.go
loses 78.4% of its tokens that way. With 59 in place the same 32 still pass —
correctly, since signature-preserving elision loses no symbols — but on a
median of 8 real symbols rather than on an import.

And 59 turned out to be a precondition for reduction, not a tax on it:
without Go symbols many files have no symbols at all, so 33's measurement
gate refuses them outright. The safety step and the reduction step were the
same step.

Go's fallback rate, which 56 could not measure: 20 of 80 (25%), of which 18
are CONSTRAINT_DIRECTIVE_LOST and 2 are drift. 56 expected Go's lower comment
density to make that gate fire LESS. It does not — it dominates exactly as it
does for TypeScript, at the same rate (24%, 15 of 62). Density was the wrong
variable; Go's defensive comment style is 52's still-open descriptive-present-
tense axis verbatim ("// This never happens in practice", "// Should never
happen, but we"). The largest remaining gain on Go is that gate, not the
scanner.

Ten existing tests asserted Go was not elidable and now fail on purpose;
they are updated in the same commit, as this repo's contract requires, with
the prior assertion recorded in each. language-support.test.ts moves its
"incidental symbol is not support" example from Go to Rust, which is in the
position Go used to occupy. The "elision reduces TypeScript/JavaScript and
Python only" explanation is corrected wherever it is emitted.

test/unit/go-region-elision.test.ts: 13 known-answer cases, 9 of which fail
against the unfixed engine. The 4 that pass both ways assert an absence and
are trivially true when the scanner selects nothing, so they are controls on
over-selection rather than evidence.

DECISIONS 61.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ojassug
ojassug merged commit 7c8fc66 into main Aug 23, 2026
3 checks passed
@ojassug
ojassug deleted the feat-go-region-elision branch August 23, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant