Skip to content

test(docs): bind every constant-backed default to its function signature, not just its constant #442

Description

@bioedca

Grooming status — 2026-08-13

  • Status: unblocked.
  • Open dependencies: none. docs(reference): close the remaining analysis-parameter and drift-guard gaps #220 / docs(reference): close the remaining analysis-parameter and drift-guard gaps #438 landed the mechanism this extends and neither needs
    re-opening.
  • Autonomy: agent-can-do-alone.
  • Chosen design — a hand-written table, cross-checked against the page. Keep the committed
    (module, qualified-name, parameter, constant) table — the _CONSTANT_BACKED_SURFACES shape —
    and add a check that rebuilds the keyword-named documented surface set from the page and asserts
    the table covers it. Same "enumerated but checked" pattern test_the_w0_surface_list_is_exhaustive
    already uses. What that predicate reaches and what it does not is measured in its own bullet below;
    nothing in this issue claims the table is complete for the page.
  • The derived unit is (surface, parameter), not a surface. A backticked token
    NAME(kw=…, kw=…) in the Set via cell of a constant-naming row yields one pair per keyword
    it names; NAME may be dotted (DwellTimeAnalysis.histogram). A token that names no keyword —
    DeepTraceDataset.split(…) — yields no pair and so is nothing the table has to cover. That is a
    real case, not a hypothetical: seven of the 39 tokens name no keyword in any constant-naming
    row's Set via cell (DeepTraceDataset.split, population_anticorrelation_events,
    population_raw_fret_cloud, population_transition_density,
    population_transition_prob_histogram, rank_by_deep_model, transition_prob_histogram), and
    four more are written bare in one such cell while naming keywords in another (fine_tune,
    predict_proba, time_signal_histogram2d, train_classifier). Rows whose Parameter cell
    names no constant are outside the walk entirely, which is why
    population_transition_density(include_stale=True) on the include_stale row does not count.
  • Three dispositions, and the third is a hard failure rather than an escape hatch. Resolve
    NAME under three resolvers — a module-level FunctionDef, a ClassDef field (dataclass), or
    Class.method for a dotted token — then:
    1. the named parameter has an ast.Name default → the pair must be covered by the table;
    2. the parameter exists but its default is not a Name (a literal, or no default at all) →
      allowed, and recorded nowhere;
    3. NAME resolves under none of the three resolvers, or the named parameter does not exist on
      it → the check fails, naming the token.
      There is deliberately no exemption or allowlist collection. All 39 tokens and every keyword
      they name resolve today (measured 2026-08-13), so disposition 3 is green on a clean tree — and an
      escape hatch would let this whole issue be discharged by binding three surfaces and exempting the
      rest, which is the specific hole this grooming pass was asked to close.
      Disposition 2 is where the derivation is blind, so it was measured rather than assumed: 13
      pairs are in that state today (raw_fret_cloud(kde=True), raw_fret_cloud(time_range=None),
      fine_tune(device=None), predict_proba(device=None), population_dwell_times(model='single'),
      train_val_split(stratify=True), k_rmse_elbow(k_min=1), population_state_number's
      states_high=None, alpha_shape(alpha=None), and four more across the two
      population_fret_cloud_* wrappers), and none of their literals equals the value of any
      constant its own row documents. A keyword that had already drifted to a value-preserving literal
      would land in disposition 2 and never be flagged; that set is empty at e70a12c, so the coverage
      assertion blesses no existing defect.
  • Coverage is one-directional: derived ⊆ table. Equality is unsatisfiable, and that is a fact
    about the table rather than a preference: 7 of the 12 entries _CONSTANT_BACKED_SURFACES
    holds today are not page-named at all — weighted_training_set, score_molecules,
    ranker_ranking, ranker_precision_at_k (tether.project.gbranking), next_recommendation
    (tether.project.active), seed_weight, effective_weights (tether.ml.weighting) — because
    the w0 row claims in words that the default holds beyond the surfaces it names ("w0 is the
    same keyword carrying the same default on every public function of the seam beneath them") and
    docs(reference): close the remaining analysis-parameter and drift-guard gaps #438 enumerated that seam exhaustively. The table may hold more than the page names; it may not
    hold less. A stale table entry is still caught, by function_default_name raising when the
    function or parameter is gone, and the w0 seam by test_the_w0_surface_list_is_exhaustive.
  • The module is not derived, and that is why the table stays hand-written. The Parameter
    cell names the module the constant lives in, which is not reliably the module the function
    lives in: the PB_PRIOR_A row says PB_PRIOR_A in tether.fret.photobleach, while one of the
    two functions its Set via cell names, compute_photobleach, lives in
    tether.project.photobleach and imports the constant from the other module. Resolving the module
    from a bare function name instead would resolve by accident: two modules define a module-level
    fine_tune (tether.ml.deep.model and tether.ml.deep._torch_model), and only the first gives
    seed an ast.Name default today, so a name-only lookup picks the right one now and would
    silently pick one rather than fail the day a second Name-defaulted definition appears. Deriving
    the module would trade a loud failure for a quiet mis-resolution. Pure derivation of the whole
    tuple is rejected for the other half of the same reason: a Set via cell is prose, and a regex
    that silently matches nothing on a reworded cell is the false clean ci(agents): report issues whose blockers have all closed, and status:ready issues with no binding marker #326 names as worse than a
    miss. The derivation here can only add a requirement; the table carries the module and
    function_default_name verifies it there.
  • What the keyword predicate does not reach, measured — and no criterion below claims otherwise.
    A pair is derived only where the Set via cell spells the keyword, and that line is prose
    typography rather than risk. Two consequences, both measured at e70a12c. 48 pairs sit on a
    surface a constant-naming row names, carry a constant that same row documents, and are not
    derived because the cell writes the token bare or spells only some of its keywords: the cloud-grid
    row names five constants and its cell reads raw_fret_cloud(time_range=…, kde=…),
    population_raw_fret_cloud(…), so none of signal_bins, time_bins, signal_range,
    hdr_coverages or bw_method is derived although raw_fret_cloud defaults all five to those
    constants; the deep-training row writes train_classifier(…) and fine_tune(…) bare, so
    epochs, batch_size and learning_rate are not derived either. A further 58 pairs are
    documented as a (keyword, constant) pair on a constant-naming row but sit on a public surface the
    table will not carry: point_pbtime and ensemble_pbtime (a, b, beta, mu, in the same
    module as the page-named detect_photobleach), estimate_gamma / trace_gamma against the
    page-named compute_gamma, estimate_leakage_alpha / tail_alpha against
    compute_leakage_alpha, tether.ml.drift.condition_drift.alpha against cross_condition_drift,
    tether.ml.prequential.prequential_uplift.ship_bar_pts against ranker_prequential_uplift, and
    SidecarSupervision.timeout — the field beside the SidecarSupervision(probe_timeout=…) field
    this table does bind, whose own constant is documented on a different row of the page, which
    TIMEOUT_SURFACES does not list, and which that guard's literal sweep does not reach either, since
    the sweep is parametrized over the three modules its five listed surfaces live in and not over
    tether.idealize.supervisor. The two sets overlap in five, so 101 pairs across 40 qualified
    names in 20 modules stay unbound after this lands; four of them (run_vbfret, reidealize,
    Project.idealize, Project.reidealize, all timeout) are pinned by
    tests/test_idealize_timeout_default.py and 97 by nothing.
  • Why the narrow predicate goes first: the wider one is more rows, not more mechanism. Matching
    on the row's own constants instead of on the keywords the cell spells derives 103 pairs — 48 more
    table rows — and matching on any of the 77 constants the 48 rows name derives 120, 65 more rows.
    Both reuse this issue's page walk and these three resolvers unchanged: the walk already extracts a
    row's SCREAMING_CASE constants, since that is how it decides the row is in scope. So the wider
    predicate is a one-line change plus table rows, and ordering the mechanism first costs nothing.
    It is a separate templated issue, not a widening of this one — do not change the predicate here.
  • idealize_molecules.timeout is re-bound here rather than cross-referenced. It is page-named
    and it is already pinned by name in tests/test_idealize_timeout_default.py (refactor(idealize): give the idealization timeout one source of truth #222), which
    covers five sidecar timeout surfaces of which only this one is on the page. It goes in this
    table anyway — one tuple — so the coverage check needs no cross-file dependency and this PR needs
    to open no second file. The two guards then assert the same thing for different reasons, which is
    cheap and is not the double bookkeeping a reviewer should worry about.
  • Review risk: a provider will push on the double bookkeeping between page and table, on the
    enumeration rotting, on whether the derivation is a hole, and on why the predicate is the cell's
    keywords rather than the row's constants — answer the first two with the page-derived coverage
    check and function_default_name's own raise, the third with the fact that there is nothing to
    exempt, and the fourth with the measured bullets above. Second risk: a reviewer may read this as
    an agent-layer capability change frozen by ADR-0064. It is not.
    tests/test_docs_parameter_defaults.py guards docs/reference/parameters.md, a user-facing
    reference page; it is not one of the agent test modules, and none of .agents/, docs/agents/,
    AGENTS.md or CLAUDE.md is touched.
  • Terminal condition: PR-ready handoff.

Acceptance criteria

  • The keyword-named surface set is read off the page, not only enumerated by hand. A check in
    tests/test_docs_parameter_defaults.py walks every six-column body row of
    docs/reference/parameters.md whose Parameter cell names a SCREAMING_CASE constant,
    builds the (surface, parameter) pairs from every backticked NAME(kw=…) token in that
    row's Set via cell, resolves each NAME under the three resolvers, fails outright and
    names the token when it resolves under none of them or when a keyword it spells is not a
    parameter of what it resolved to
    (disposition 3), and asserts that every remaining pair
    whose default is an ast.Name is covered by the committed table. Coverage is
    derived ⊆ table; the table may hold more.
    Two currently-failing demonstrations, both green on main at e70a12c:
    (a) rename apparent_e_histogram to apparent_e_histogram_x inside the Set via cell of
    the DEFAULT_NBINS row, whose cell names that one token — it now resolves under no resolver,
    so the hard failure above must fire; (b) delete the model_gaussian_overlay / n_points
    tuple from the table — the DEFAULT_OVERLAY_POINTS row still names that pair, so coverage
    must fail. Today pytest tests/test_docs_parameter_defaults.py stays fully green for both,
    because the module never reads the Set via column at all: _rows() produces all six cells
    and the checks reference only row[0] (five times), row[1] (four) and row[5] (once) — no
    row[2], row[3] or row[4] anywhere in the file. Pick (b) on a non-w0 entry as written,
    so test_the_w0_surface_list_is_exhaustive is not what turns red.

  • Every page-named module-level function surface binds its signature to the constant by
    name.
    Currently-failing demonstration, verified on main at e70a12c: in
    src/tether/analysis/histogram.py, change transition_sync_histogram2d's
    sync_preframe: int = DEFAULT_SYNC_PREFRAME to sync_preframe: int = 7 and leave
    DEFAULT_SYNC_PREFRAME = 50 alone. Today pytest tests/test_docs_parameter_defaults.py and
    pytest tests/test_analysis_histogram_postsync.py are both fully green while the page still
    prints 50. After this change the docs guard must go red.

  • A page row naming several surfaces for one constant binds each of them, not just the
    first.
    The PB_PRIOR_A row's Set via cell names compute_photobleach(a=…) and
    detect_photobleach(a=…), and the two live in different modules —
    src/tether/project/photobleach.py and src/tether/fret/photobleach.py respectively — so
    the table carries two tuples with different module strings.
    Currently-failing demonstration: mutate only detect_photobleach's a: float = PB_PRIOR_A
    to the literal 1.0, leaving compute_photobleach's alone. 1.0 is the constant's current
    value, so the change is value-preserving and no behavioural test can see it —
    test_priors_match_registered_defaults in tests/test_fret_photobleach.py asserts on the
    constants ((PB_PRIOR_A, …) == (1.0, 1.0, 1.0, 1000.0)), which are untouched.
    pytest tests/test_docs_parameter_defaults.py stays green today and must go red after.

  • A method and a dataclass field are reachable too. function_default_name walks only
    module-level FunctionDef nodes, so two page-named bindings are invisible to it:
    DwellTimeAnalysis.histogram(nbins=…) in src/tether/analysis/dwell.py, and the
    SidecarSupervision(probe_timeout=…) dataclass field in
    src/tether/idealize/supervisor.py. Both are reachable with existing precedent rather than
    new invention — tests/test_idealize_timeout_default.py::_function already walks a
    Class.method body, and a dataclass_field_name() mirror of function_default_name() is the
    same shape as dataclass_field() with the literal_eval replaced by an ast.Name check.
    Neither may be dropped instead of bound.
    Two currently-failing demonstrations, both value-preserving so no behavioural test can see
    them:
    set DwellTimeAnalysis.histogram's default to nbins: int = 51 (exactly one test
    calls it at the default — test_population_missing_state_is_empty_never_raises in
    tests/test_analysis_dwell.py, which asserts centres.size == 0 and density.size == 0 on an
    empty analysis; the two other calls pass nbins=10, and test_defaults in the same file pins
    DEFAULT_DWELL_NBINS == 51, the constant and not the signature), and set
    SidecarSupervision.probe_timeout to float | None = 120.0 (the page's registry reaches
    DEFAULT_PROBE_TIMEOUT through module_constant, never through the field, and no test file
    outside the docs guard names the constant). Both leave
    pytest tests/test_docs_parameter_defaults.py green today and must turn it red after.

  • There is no way to declare a surface out of scope. The derivation introduces exactly one
    module-level collection — the table — and implements exactly the three dispositions above.
    Pinned two ways. Mechanically, on the final head,
    <py> -c "import ast,pathlib;print([ast.unparse(s.targets[0] if isinstance(s,ast.Assign) else s.target) for s in ast.parse(pathlib.Path('tests/test_docs_parameter_defaults.py').read_text(encoding='utf-8')).body if isinstance(s,(ast.Assign,ast.AnnAssign)) and isinstance(s.value,(ast.List,ast.Tuple,ast.Dict,ast.Set))])"
    must print ['_CONSTANT_BACKED_SURFACES'], exactly what it prints today in both lanes — it
    keys on the shape, so an escape collection named _SKIP, _KNOWN_UNBOUND or _PENDING
    fails it too, which a text search for "exempt"/"allowlist" would not. It sees module-level
    list/tuple/dict/set assignments only, so an escape wrapped in frozenset(...) or held inside
    a function body would pass it — which is what the second pin is for: on the final head, each
    demonstration in the four criteria above still turns the guard red, and an escape hatch is only
    worth writing if it makes one of them green.

  • Regression fence — already green today, listed because it constrains how the new checks may
    be written.
    After the change,
    test_a_documented_constant_backed_default_still_names_its_constant still covers all twelve
    entries _CONSTANT_BACKED_SURFACES holds today — the five page-named ones
    (model_gaussian_overlay.n_points, population_model_gaussian_overlay.n_points,
    train_ranker.w0, recompute_label_weights.w0, build_deep_dataset.w0) plus the seven
    further w0 seam surfaces — and test_the_w0_surface_list_is_exhaustive still passes
    unmodified.

  • Nothing outside the guard changes. git diff --name-only origin/main on the final head
    lists tests/test_docs_parameter_defaults.py and nothing else.

  • The new checks are green on an unmutated tree, with no edit to src/. A read-only survey
    on 2026-08-13 resolved all 55 keyword-derived pairs and found every one still spelling a
    constant its own row names, and no disposition-2 literal equal to a documented constant's
    value, so no entry the table will hold is broken today. If a check nevertheless goes red for a
    genuine drift that appeared after that survey, src/ is still not edited here and the
    entry that found it is not deleted: mark that one parametrized case
    pytest.mark.xfail(strict=True) with the new issue number in the reason, raise the separate
    templated issue, and name both in the handoff. A red suite and a deleted entry are both
    invalid terminal states.

  • pre-commit run --all-files is green, and the matrix line for your lane is green — WSL bash
    QT_QPA_PLATFORM=offscreen pytest -m "not large and not sidecar and not deep", native
    PowerShell $env:QT_QPA_PLATFORM='offscreen'; pytest -m "not large and not sidecar and not deep".
    No docs file changes, so mkdocs build --strict is not required; no schema change, so
    scripts/dump_schema.py --check is not required.

  • Ready condition: met; awaiting a fresh exact-scope marker.


Maintenance kind

test

Work requested

tests/test_docs_parameter_defaults.py guards docs/reference/parameters.md against drift, but for
a default whose signature spells a constant rather than a literal, it guards only half the chain.

The registry pins page ↔ constant. Outside the twelve surfaces #438's _CONSTANT_BACKED_SURFACES
binds (see below), almost nothing pins constant ↔ signature: the only other place in the suite is
tests/test_idealize_timeout_default.py, which pins five sidecar timeout surfaces to
DEFAULT_SIDECAR_TIMEOUT by name for #222 — and only one of those five,
idealize_molecules.timeout, is named on this page. Everywhere else a signature can drift off the
constant it is documented through, and the docs-drift guard stays green while the page prints a
number no caller ever gets.

This is by construction, not an oversight in any one entry: function_default() deliberately
rejects an ast.Name default and redirects you to module_constant() (function_default,
"pin it with module_constant() instead"). That refusal is correct — pinning a name as if it were a
value would be a lie — but it leaves the signature unbound and nothing else reads it.

Reproduced on main at e70a12c, on a surface the page names and nothing in the suite exercises:

# src/tether/analysis/histogram.py, transition_sync_histogram2d
#   sync_preframe: int = DEFAULT_SYNC_PREFRAME   ->   sync_preframe: int = 7
# DEFAULT_SYNC_PREFRAME still 50, and the page still prints `50`
pytest tests/test_docs_parameter_defaults.py        -> all green
pytest tests/test_analysis_histogram_postsync.py    -> all green

Nothing notices. No test file outside the docs guard so much as names DEFAULT_SYNC_PREFRAME, and
nothing asserts on the shipped default: of the 24 direct calls in
tests/test_analysis_histogram_postsync.py, 21 pass sync_preframe= explicitly, and the three that
do not (test_length_below_two_skipped, test_validation_errors,
test_mismatched_pair_lengths_raise) assert only n_molecules == 0 / counts.sum() == 0 or raise a
ValueError from argument validation — with DEFAULT_TIME_BINS = 100, a mutated 7 stays inside
the validated range, so even the validation cases are unmoved. The ten
population_transition_sync_histogram2d calls in the same file all pass sync_preframe= explicitly
too. That is exactly the class of change the guard exists to catch.

Why the whole class is invisible, and the lever that closes it

The Set via column is what names function surfaces, and this module never reads it. _rows()
splits all six cells, but every check touches only row[0], row[1] and row[5] — five, four and
one reference respectively, and no row[2], row[3] or row[4] anywhere in the file. Reading
column 4 is what turns "names a function surface" from an editorial judgement into a machine-checked
set, and it is what makes a page-derived cross-check possible at all.

The size of the hole

A read-only survey on 2026-08-13 measured it at e70a12c. git diff --stat e70a12c origin/main
(b078f47) touches neither src/, nor docs/reference/parameters.md, nor either guard module, so
the counts hold at the current head:

  • 94 body rows across the six-column parameter tables — 103 six-column rows including the nine
    | Parameter | Default | … | headers, which is what a _rows()-shaped count returns, since
    _rows() filters separator rows and not headers.
  • 48 of those body rows name a SCREAMING_CASE constant in their Parameter cell.
  • Those 48 rows carry 39 distinct backticked NAME(...) tokens in their Set via cells;
    28 of the 39 name at least one keyword whose default resolves to an ast.Name under the
    three resolvers.
  • Resolving every NAME(kw=…) pair gives 55 bindings whose signature default is a name:
    53 on module-level functions, one method (DwellTimeAnalysis.histogram(nbins=…)) and one
    dataclass field (SidecarSupervision(probe_timeout=…)).
  • 6 of the 55 are already bound — five in _CONSTANT_BACKED_SURFACES, and
    idealize_molecules.timeout in tests/test_idealize_timeout_default.py — leaving 49
    unbound.

So the table goes from 12 entries to 62. Every one of the 55 spells a constant its own row names
— no derived pair reaches for a constant documented only elsewhere — so this is a
guard-completeness job, not a documentation-correctness one. What the keyword predicate leaves
unreached is measured in the grooming block above: 101 further pairs, of which 97 are bound
nowhere. This issue is the mechanism plus the 50 new rows that predicate yields; it is not the
whole class, and no criterion says it is.

What #438 already did, and what it deliberately did not

PR #438 (issue #220) closed this for the surfaces it newly documented — the two n_points overlay
entry points and the w0 seam — with a function_default_name() helper and a
_CONSTANT_BACKED_SURFACES table that binds 12 surfaces today (2 n_points + 10 w0), guarded
by test_a_documented_constant_backed_default_still_names_its_constant and, for the w0
enumeration, by test_the_w0_surface_list_is_exhaustive.

It stopped there on purpose. There are 78 module_constant( registry call sites — 80 textual
hits, the other two being the def and the "pin it with module_constant() instead" error string —
and retrofitting the class is a guard capability change, not the documentation work #220 was scoped
to. Fixing it inside that PR would have been scope breach. The comment above the table says so in
the file: closing the hole everywhere "is a registry-wide change and is tracked separately".

Acceptance criteria

The authoritative list is in the Grooming status block at the top of this issue.

Scope and non-goals

In scope: tests/test_docs_parameter_defaults.py only, extending the mechanism #438 introduced.

Out of scope: any change to src/, to docs/reference/parameters.md — including its prose and its
Set via cells — or to the printed values themselves. Also out of scope: editing
tests/test_idealize_timeout_default.py, which is why idealize_molecules.timeout is re-bound in
this table instead of cross-referenced from it; and widening the derivation predicate beyond the
keywords a Set via cell spells, which the grooming block measures and hands to its own issue.

The page and src/ edits named in the criteria are demonstrations: make them, watch the check,
revert them. If a binding turns out to be already broken — a signature that has silently drifted
off its constant since the 2026-08-13 survey — do not fix the code here: xfail(strict=True) that
one case with the new issue number, report it as its own templated issue since a wrong documented
default is a separate defect with its own review path, and do not delete the entry to get the suite
green.

Size estimate

M — up to 400 added lines

Grooming raised this from S. Fifty new table entries measured at 85 lines in the repo's
100-column style — 43 fit one line, 7 need the wrapped six-line form — plus two new resolvers
(Class.method and dataclass-field), the page-derivation helper and one new check written in this
module's docstring style land near 240 added lines. That is over an S budget of 150 and well inside
M, and an S budget would force either a split or an under-delivered guard.

Risk level

standard — the lane: Codex on the draft, then CodeRabbit

Does this need an architecture decision record?

no

Dependencies / blockers

none — #220 / #438 landed the mechanism this extends, but the work does not require it to be
re-opened.

Execution autonomy — declared in the grooming block

Stated once, as a single bullet in the Grooming status block at the top of this issue, and
deliberately not restated here: two copies of one declaration is the drift the claim gate fails
closed on.

Related work / file overlap

Target area

CI / repository governance

Milestone

none / cross-cutting

Required safety attestations

  • I searched existing issues and this is not a duplicate.
  • I am not reporting a vulnerability publicly; security reports use the private advisory flow.
  • I included no secrets or private, raw, unlicensed, user, or lab data.

Metadata

Metadata

Assignees

No one assigned

    Projects

    Status
    No status

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions