Slot-surgery primitives for capacity birth / column probes (#811/#820) - #997
Open
claude-spd1 wants to merge 15 commits into
Open
Slot-surgery primitives for capacity birth / column probes (#811/#820)#997claude-spd1 wants to merge 15 commits into
claude-spd1 wants to merge 15 commits into
Conversation
Crew-Address: agent/c5xs
Apply component update rules during faithfulness warmup as well as main training, and add a balanced-factor Adam mode that transforms moments under every exact V/U gauge retraction.\n\nCrew-Address: agent/c5xs
…tail pd.component_init: svd_null_tail starts the first min(d_in, d_out) slots at an exact SVD factorization of the frozen weight with CI pinned 1, and every extra slot as an exact null (U row zero, prefix-stable random V column, CI head pinned 0), so every C >= rank(W) starts from the same represented matrix, recon, and L0. Piece 2 of the #811 C-invariance acceptance test (spec: lore 2026-08-01--finding--current-vpd-initialization-and-adam-dynamics-are-not-c-nested). Init tree structure is unchanged (S22-safe); MLP CI fns only, chunkwise refuses. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…train step Stage 1 of the recon-budget controller + demand-triggered birth (task #811, spec: lore 2026-08-02--design--reconstruction-budget-control-and-demand-triggered-capacity-birth). complexity_scale multiplies the COMBINED imp-min + frequency force (one gate pressure — scaling only imp-min would leave frequency as an unrelaxable pruning force at the controller floor); protect_ci pins birth-protected slots to CI 1 in both squashings INSIDE the CI vjp, so masks pin to 1 (S1), neither adversary can touch a protected slot, and the imp-min gradient into its logit is exactly zero. None resolves at trace time to the identity — existing callers unchanged. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stage 2a of task #811: sign-correct reciprocal-coordinate control (violation drives the combined complexity scale DOWN), gain-free multiplicative bracketing (halve the log step on violation/slack sign flips), an explicit c=0 OFF state in place of a tuned positive floor, and the two lifecycle events from the column-generation addendum — FEASIBILITY_BIRTH only from OFF under sustained violation, COLUMN_PROBE on on-budget complexity plateau (accept-iff-improve is the caller's job), CAPACITY_EXHAUSTED reported, never clipped. Pure logic over settled window summaries; trainer integration and birth surgery follow. Spec: lore 2026-08-02--design--reconstruction-budget-control-and-demand-triggered-capacity-birth + 2026-08-02--addendum--capacity-lifecycle-needs-column-generation-not-only-feasibility-birth. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fixes the non-termination Ossicle found in 80087c6: sign-flip step-halving never terminates on one-sided sequences. The search now keeps explicit endpoints — slack raises lo, violation lowers hi, both bisect, only-hi probes OFF (OFF-slack re-enters expanding downward from hi; OFF-violation is the feasibility-birth condition), only-lo expands upward under a hard guard — so all-violation reaches OFF in finite windows and all-slack is bounded. Rejected COLUMN_PROBEs start a cooldown and count toward a terminal NO_IMPROVING_COLUMN so one plateau cannot re-trigger the identical probe forever; probe verdicts arrive via probe_accepted/probe_rejected. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review seam from Ossicle: BIRTH_PROTECTED conflated feasibility protection (timer-expired) with a pending COLUMN_PROBE, so time could silently age a probe back into CONTROL with no verdict. COLUMN_PROBE now enters PROBE_PENDING, which only probe_accepted/probe_rejected exit (asserted); feasibility expiry and accepted probes clear the stale plateau and rejection lineage along with the bracket. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tays OFF through birth protection Two case-blocking bugs from Ossicle's review: (1) a converged bracket holds at feasible lo, whose sampled point is typically slack beyond the deadband — the sign<0 arm reset dwell forever, starving case B's probe; plateau logic now also engages for a converged-at-lo bracket. (2) BIRTH_PROTECTED restored exp(log_c) — the last known-VIOLATING coefficient — while the feasibility newborn settled; c now stays 0 through protection (BIRTH_PROTECTED is feasibility-only after the phase split) and expiry re-enters control one expand-step BELOW the failed value. Regressions for both. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…820) core/slot_surgery.py: find_inactive_slot (exact-null U row; None = the controller's CAPACITY_EXHAUSTED input), birth_direction_from_grad + select_birth_site (GradMax leading singular pair of the per-site represented-matrix recon gradient, power-iterated; the zero-plumbing null-slot probe is documented for callers whose G is too big to materialize), birth_slot (unit-p V column, exact-zero U row — represented matrix unchanged exactly — zeroed CI-head column with bias 1, and the slot's Adam moments reset in BOTH optimizers), snapshot_trial/rollback_trial (bitwise restore via the single shared mutation path), and the StepControls.protected mask builder. Fail closed: MLP CI fns only, exactly one ScaleByAdamState per optimizer chain. Tests: function preservation at birth, first-gradient alignment with the exact SVD, edit locality, bitwise rollback, refusal arms. Task: #820 (parent #811). Spec: lore 2026-08-02--design--reconstruction-budget-control-and-demand-triggered-capacity-birth. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ical Cmax) Ossicle's harness request for acceptance case A: slots at/beyond the given per-site count become exact inactive nulls (U row zero, CI-head column zero with bias 0 — closed, not protected-open — moments cleared in both optimizers; V columns keep their values since null-ness is defined by the U row). Idempotent; goes through the same shared mutation path as birth/rollback. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Blocking review fix (Ossicle, #820): during a COLUMN_PROBE every other parameter, both optimizers' moments, and the persistent adversaries keep training — restoring only the trial slot's slices would accept/reject on contaminated state. TrialSnapshot now retains the full immutable pre-trial TrainState (JAX arrays are immutable, so holding it IS the snapshot); rollback_trial(snapshot) returns it bitwise. Regression contaminates an unrelated slot, a CI hidden layer, both opt states, and the step counter, then asserts every leaf restores. Also dropped the dense-G-is-small claim: at LM scale use the alternating factor-gradient path. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
basedpyright fixes for the PR CI: exhaustive if/elif in _next_log_c (guarded match arms defeat exhaustiveness analysis), no uppercase rebinding in the power iteration, Any-typed duck-typed moment-tree params, protocol-safe casts in tests. Remaining CI failures on this branch are in the inherited experiment commits (run_state.py), not this module. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
basedpyright hygiene so PR #997 can go green: optax-boundary casts in the c-covariant/gauge-balanced transforms and their parity tests (ComponentStacks rides through optax as an opaque pytree — the casts assert that boundary, they change no behavior), GaugeBalancedState signature aligned with TransformUpdateFn, and both suppressions on the fail-closed match arm. Mechanical; candidate cherry-pick for bridge/task-c-insensitive and bridge/task-811-controller. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live-fire bug from the first acceptance run (seed-0 rollback): TrialSnapshot held references into the pre-trial TrainState, but the train step donates its state buffers — the next step deletes them and rollback dereferences dead memory. Immutability alone is not a snapshot under donation. snapshot_trial now copies every array leaf; regression asserts every snapshot leaf is a distinct buffer with equal contents. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…th_slots) The serial one-slot lifecycle has latency proportional to capacity deficit x sites x settle work (live finding from the case-A acceptance arms). The batch primitive opens the top-k singular directions of the demand gradient in ONE event: orthonormal live V columns (asserted — a non-orthogonal batch would double-spend a demand direction), exact-zero paired U rows so the represented matrix is unchanged for the whole batch, selected-only CI/moment edits, and one full-state snapshot covering the batched rollback. Selection policy (k) is deliberately the caller's — a noise floor here would be the next scale-sensitive knob. Per-slot first gradients equal their singular values (test), refusals for rank-deficient batches and duplicate slots. Crew-Address: agent/m6ue Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pure lifecycle surgery primitives for the recon-budget controller's birth/probe events — no run-loop, config, or model coupling. Implements the #820 spec exactly:
find_inactive_slot: exact-null U row;None= the controller's CAPACITY_EXHAUSTED input.birth_direction_from_grad/select_birth_site: GradMax leading singular pair of the per-site represented-matrix recon gradient (power iteration); the zero-plumbing null-slot probe for large targets is documented on the function.birth_slot: function-preserving (represented matrix unchanged EXACTLY — unit-p V column, exact-zero U row), CI-head column zeroed with bias 1, the slot's Adam moments reset in both optimizers.snapshot_trial/rollback_trial: bitwise restore through the single shared mutation path.protected_mask: theStepControls.protectedentry for gate protection.Fail-closed: MLP CI fns only (chunkwise raises), exactly one
ScaleByAdamStateper optimizer chain (muon raises). Tests cover the four spec requirements: VU unchanged at event, first-gradient alignment with the exact SVD, only-selected-slot/moments changed, exact rollback.Experiment branch stacked on
bridge/task-811-controller(base shown vs main includes the controller + svd-init + c-covariant commits); review scope is the tip commit 7cb4e33. Not an adoption PR — consumed by the #811 acceptance harness.Task: #820 (parent #811). Spec: lore
2026-08-02--design--reconstruction-budget-control-and-demand-triggered-capacity-birth+ the column-generation addendum.Crew-Address: agent/m6ue
🤖 Generated with Claude Code