Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
145 changes: 136 additions & 9 deletions RESEARCH_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
**Project:** The Spoon Project ([README.md](README.md) holds the narrative charter; this file holds the protocol)
**Repository:** `VirtualVagus`
**Python package:** `virtual_interoception`
**Status:** Draft research charter v0.1
**Authoring date:** 2026-08-02
**Target machine:** MacBook Pro with 16 GB unified memory
**Status:** Draft research charter v0.2
**Authoring date:** 2026-08-02; amended 2026-08-10 for the two-track structure ([issue #1](https://github.com/BioNanomics/VirtualVagus/issues/1))
**Target machine:** MacBook Pro with 16 GB unified memory (local reference) plus the Purdue Gilbreth GPU cluster (Section 5.2)
**Authoritative workflow:** command-line programs, versioned configurations, automated tests, and Git-tracked gate decisions. No notebook is an authoritative research artifact.

---
Expand All @@ -33,6 +33,15 @@ A standard decoder-only transformer forward pass is a directed acyclic computati

The strongest version of the hypothesis is not merely that bodily words influence later words. It is that the model contains a latent body-state mediator that can be causally inserted, removed, or rerouted even when no bodily words are present.

### 1.2 Two research tracks

Amended 2026-08-10 ([issue #1](https://github.com/BioNanomics/VirtualVagus/issues/1)): the project runs two coordinated tracks.

- **Track A — interoception ladder.** The original program, on the Pythia family (Section 5). Milestones 0–11 and gates G00–G11 are unchanged.
- **Track B — J-space connection.** Tests whether the interoceptive mediator lies inside or outside the verbalization-poised (J) subspace, on the Gemma family, where fitted Jacobian-lens artifacts and validated emotion-vector recipes provide external anchors. Milestones B1–B2, gates GB1–GB2. Method review and evidence floor: [docs/j-space-global-workspace.md](docs/j-space-global-workspace.md).

Track B opens only after gate G02 closes, so the authorized `EXP-003-v2` repeat stays first. Passive J-lens readout is never gate-eligible evidence in either track (see `j_space` in [docs/glossary.md](docs/glossary.md)).

---

## 2. Hypotheses and null hypotheses
Expand Down Expand Up @@ -113,6 +122,9 @@ No result in this plan establishes subjective experience.
- [ ] Large tensors and model files are content-addressed outside ordinary Git history; their manifests and hashes remain in Git.
- [x] CPU float32 is the numerical reference path.
- [x] MPS acceleration is used only after a parity test against the CPU reference.
- [ ] CUDA acceleration is used only after a parity test against the CPU reference; for models exceeding local memory, the CPU float32 reference run executes on cluster CPU nodes.
- [ ] Cluster runs execute inside a versioned Apptainer image whose digest is recorded in the run manifest.
- [ ] Slurm batch scripts are thin wrappers around the same versioned CLI commands used locally; no cluster-only experiment logic.
- [x] Quantized models are not used as primary causal evidence.
- [x] A failed gate can be repeated once for a documented technical or design defect. A second failure forces a pivot or stop decision.

Expand Down Expand Up @@ -172,28 +184,49 @@ Every gate-eligible run must produce a machine-readable `run.json` containing at
}
```

Cluster (Gilbreth) runs must additionally record `gpu`, `cuda`, `driver`, `node_type`,
`slurm_job_id`, and `container_digest`. These fields are null for local runs.

### 4.4 Storage policy

Recommended:

- Git for code, configurations, protocols, compact data, summary results, figures, and decision records.
- `artifacts/` for local activation tensors, excluded from ordinary Git.
- DVC or an equivalent content-addressed store for large reproducible artifacts; its metadata remains in Git.
- Cluster scratch/Depot for fitted lenses, activation dumps, and other large Track B artifacts; only manifests and content hashes return to Git.
- Hugging Face model cache outside the repository, with exact revisions recorded in the manifest.

---

## 5. Recommended model ladder

### Track A — interoception ladder (Pythia)

| Role | Model | Purpose |
|---|---|---|
| Unit-test fixture | TinyStories 1-layer 21M or Pythia-31M | Test hooks, tensor shapes, patching code, and CI without heavy compute |
| Development | Pythia-70M | Rapid end-to-end experiments |
| Primary discovery | Pythia-160M | Main small-model target with enough capacity for structured narrative relations |
| Replication | Pythia-410M | Test whether effects strengthen or reorganize with scale |
| Replication | Pythia-410M | Scale trend; folded into the `EXP-003-v2` behavioral repeat (G02 amendment below) |
| Conditional scale extension | Pythia-1B / Pythia-1.4B | Registered only if the 410M scale trend is promising; runs under Milestone 9 |
| Training-dynamics replication | Selected Pythia checkpoints and seeds | Determine emergence and seed stability |

Pythia is preferred because the suite was designed for interpretability and exposes many training checkpoints. Use models sequentially, not simultaneously, to remain within 16 GB.
Pythia is preferred because the suite was designed for interpretability and exposes many training checkpoints. Use models sequentially, not simultaneously, to remain within 16 GB locally.

### Track B — J-space connection (Gemma)

| Role | Model | Purpose |
|---|---|---|
| Track B behavioral + extraction | Gemma 2 2B | Smallest model with a validated emotion-vector recipe (EmotionScope); behavioral battery partially Mac-runnable |
| Track B lens anchor | Gemma-4-12B | Published fitted J-lenses and preregistered reliability campaign (`solarkyle/jspace`); validates our lens machinery; cluster-only |

No J-space study exists below ~2B parameters, and passive J-lens readouts are already
indistinguishable from shuffled-corpus controls at 2–4B
([docs/j-space-global-workspace.md](docs/j-space-global-workspace.md)). Track B therefore
does not run on the Pythia ladder; a Pythia lens fit may be registered only as a
negative-control curiosity, never as gate evidence. Track B revisions are pinned in
`configs/models/` when Milestone B1 opens.

### 5.1 Memory discipline

Expand All @@ -205,6 +238,13 @@ Pythia is preferred because the suite was designed for interpretability and expo
- [ ] Store analysis copies in float16 only after a float32 reference result exists.
- [ ] Clear model and backend caches between model changes.

### 5.2 Compute environments

| Environment | Role | Constraints |
|---|---|---|
| MacBook Pro, 16 GB unified memory | Development, CI fixtures, CPU float32 reference for Track A models, MPS parity runs | Models used sequentially; Section 5.1 discipline |
| Purdue [Gilbreth](https://www.rcac.purdue.edu/compute/gilbreth) (A100 80GB / H100, Slurm) | Confirmation-scale lens fits and control-lens suites, Gemma-4-12B work, Milestone 9 checkpoint × size × seed sweeps, CPU float32 reference for models exceeding 16 GB | CentOS 7: runs execute inside a versioned Apptainer image; `sbatch` scripts are thin wrappers around the local CLI (Section 4.1) |

---

## 6. Proposed repository
Expand Down Expand Up @@ -515,6 +555,13 @@ Initial preregistered continuation criteria:

**STOP the loop pursuit in this model family when:** Pythia-410M also fails the behavioral prerequisite. Preserve the result as a useful negative finding or move to a different model family under a new charter.

**Amendment (2026-08-10, [issue #1](https://github.com/BioNanomics/VirtualVagus/issues/1)).**
The authorized `EXP-003-v2` repeat additionally runs Pythia-410M under the same
preregistered thresholds, converting the repeat into a three-point scale trend
(70M/160M/410M). The 410M thresholds must be committed before its held-out split is
scored. A Pythia-1B/1.4B scale extension is registered only if the 410M trend is
promising, as a new experiment version under Milestone 9.

---

## Milestone 3 — Representation mapping without causal claims
Expand Down Expand Up @@ -762,14 +809,15 @@ Determine whether the phenomenon is stable, scale-dependent, seed-specific, or d
### Checklist

- [ ] Freeze the discovery protocol before replication.
- [ ] Repeat behavioral and strongest causal endpoints on Pythia-70M, 160M, and 410M.
- [ ] Repeat behavioral and strongest causal endpoints on Pythia-70M, 160M, and 410M; extend to Pythia-1B/1.4B only under the registered conditional scale extension (Milestone 2 amendment).
- [ ] Repeat the primary endpoint across multiple Pythia random seeds where available.
- [ ] Select training checkpoints before running them, for example initialization, early, middle, late, and final checkpoints.
- [ ] Track when behavioral coupling first appears.
- [ ] Track when separable `mobilization`/`body_state` and `action_appraisal` representations appear.
- [ ] Track when causal mediation appears.
- [ ] Compare layer locations using relative depth, not only absolute layer number.
- [ ] Test one unrelated small model family only after the Pythia replication is complete.
- [ ] Test one unrelated small model family only after the Pythia replication is complete. Track B (Milestones B1–B2) designates Gemma as that family; behavioral results from GB1 satisfy this item.
- [ ] Run checkpoint × size × seed sweeps as Slurm array jobs on Gilbreth (Section 5.2), one manifest per run.
- [ ] Separate failure to replicate from failure due to different tokenization or weak task competence.

### Gate G09 — Does the phenomenon generalize?
Expand Down Expand Up @@ -821,6 +869,67 @@ Determine whether sparse autoencoders expose interpretable features correspondin

---

## Milestone B1 — Track B behavioral prerequisite and lens-machinery validation

### Objective

Establish the behavioral phenomenon on the Track B family and validate the Jacobian-lens machinery against published artifacts before any J-space claim about interoception. Begins only after gate G02 closes.

### Checklist

- [ ] Pin Gemma 2 2B and Gemma-4-12B model and tokenizer revisions in `configs/models/`.
- [ ] Re-validate the stimulus corpus under the Gemma tokenizer: continuation tokenization normalization, banned-word coverage, and token-length balance.
- [ ] Register G02-equivalent thresholds for Gemma 2 2B in `configs/gates/` before scoring the held-out split.
- [ ] Run the behavioral battery on Gemma 2 2B (CUDA parity-gated or cluster CPU reference per Section 4.1).
- [ ] Reproduce one published `solarkyle/jspace` Gemma-4-12B result with our lens-application code against their fitted lens, within a registered tolerance.
- [ ] Fit a pilot J-lens (10–25 prompts × 128 tokens) and a token-shuffled-corpus control lens on Gemma 2 2B; record both as content-addressed artifacts with manifests.
- [ ] Record where the shuffled control matches the pilot lens; passive readout scores are recorded as diagnostics only.

### Gate GB1 — Does Track B have a behavioral phenomenon and trustworthy machinery?

**CONTINUE only when all are true:**

- [ ] Gemma 2 2B passes the registered G02-equivalent criteria on the held-out split, including the keyword-free subset.
- [ ] The published-artifact reproduction matches within the registered tolerance.
- [ ] The pilot lens is distinguishable from the shuffled-corpus control on at least one registered causal (not passive) diagnostic.

**PIVOT when:** Gemma 2 2B fails behaviorally but a registered Gemma-4-12B check passes; continue Track B at 12B only.

**STOP Track B when:** both Gemma models fail the behavioral prerequisite, or the machinery cannot reproduce published artifacts after one authorized repeat. Track A is unaffected.

---

## Milestone B2 — J-space decomposition of the interoceptive mediator

### Objective

Test whether the `mobilization`/`body_state` mediator lies inside or outside the verbalization-poised (J) subspace, and whether its influence on appraisal is J-mediated, using the automatic-versus-flexible design. Begins after GB1; does not depend on Track A gates because Track B extracts its own directions on the Gemma family.

### Checklist

- [ ] Extract candidate `mobilization`/`body_state` directions on Gemma 2 2B with an EmotionScope-style recipe (conditioned mean difference, neutral-PC removal), with embedding-only and bag-of-words baselines.
- [ ] Fit a confirmation-scale lens (500–1000 prompts × 128 tokens) plus shuffled-corpus, disjoint-corpus, and rotated-dictionary control lenses on Gilbreth.
- [ ] Sparse-decompose the candidate direction into J and non-J components using both the sparse-cone (gradient pursuit, registered $k \le 25$) and strict local-orthogonal decompositions.
- [ ] Run the norm-controlled intervention matrix: full direction, J component, non-J remainder, sign reversal, norm-matched random, shuffled-lens, and rotated-J controls; report equal-coefficient and equal-norm scalings separately.
- [ ] Run the automatic-versus-flexible battery: identical body-state evidence under (a) natural continuation, (b) explicit report of the feeling, (c) an arbitrary context-specified rule applied to the feeling.
- [ ] Track J-space entry of injected non-J components across layers and repeat interventions under downstream J-clamping.
- [ ] Escalate the strongest registered result to Gemma-4-12B using the published lens.

### Gate GB2 — Is the interoceptive mediator inside or outside the J subspace?

Either answer is a result. **CONTINUE only when all are true:**

- [ ] J and non-J intervention effects are distinguishable from norm-matched random, shuffled-lens, and rotated-J controls.
- [ ] Any off-J (bypass) claim survives downstream J-clamping; any J-mediation claim shows clamp sensitivity.
- [ ] The automatic-versus-flexible contrast was registered before held-out scoring, and at least one predicted dissociation is observed or cleanly falsified.
- [ ] No conclusion rests on passive J-lens readout.

**PIVOT when:** decomposition components are individually inert but the full direction is causal. Report a nonlinear or distributed interaction, not a J-space localization.

**STOP the J-space connection when:** interventions cannot be distinguished from matched geometric controls at either model scale.

---

## Milestone 11 — Clean-room reproduction and release

### Objective
Expand Down Expand Up @@ -920,7 +1029,7 @@ These controls are mandatory before a loop claim.
|---|---|---|---|
| EXP-001 | End-to-end tooling smoke test | TinyStories 21M or Pythia-31M | G01 |
| EXP-002 | Behavioral threat/body factorial | Pythia-70M | G02 |
| EXP-003 | Held-out behavioral replication | Pythia-160M | G02 |
| EXP-003 | Held-out behavioral replication (v2 adds the 410M scale point) | Pythia-160M + Pythia-410M | G02 |
| EXP-004 | Layerwise latent-variable probes | Pythia-160M | G03 |
| EXP-005 | Lexical and causal-attribution falsification | Pythia-160M | G03 |
| EXP-006 | Event-to-body activation patching | Pythia-160M | G04 |
Expand All @@ -934,6 +1043,12 @@ These controls are mandatory before a loop claim.
| EXP-014 | Training-checkpoint emergence | Pythia-160M checkpoints | G09 |
| EXP-015 | SAE factorization | Pythia-160M | G10 |
| EXP-016 | Clean-room release reproduction | Primary model | G11 |
| EXP-017 | Conditional scale extension (only if the 410M trend is promising) | Pythia-1B / Pythia-1.4B | G09 |
| EXP-B01 | Gemma tokenizer validation and behavioral battery | Gemma 2 2B | GB1 |
| EXP-B02 | Published-lens machinery reproduction | Gemma-4-12B | GB1 |
| EXP-B03 | Pilot lens fit with shuffled-corpus control | Gemma 2 2B | GB1 |
| EXP-B04 | Mediator J-decomposition and intervention matrix | Gemma 2 2B | GB2 |
| EXP-B05 | Automatic-versus-flexible interoception battery | Gemma 2 2B → Gemma-4-12B | GB2 |

---

Expand Down Expand Up @@ -1045,7 +1160,7 @@ Effects disappear under body-channel, causal-attribution, or keyword-free contro

### Outcome E — Small-model boundary

The effect is absent in 70M–410M Pythia models despite adequate task competence. This establishes a scale/model boundary and motivates a separately chartered larger-model study.
The effect is absent in 70M–410M Pythia models despite adequate task competence. This establishes a scale/model boundary; Track B (Milestones B1–B2) is the chartered larger-model continuation.

---

Expand Down Expand Up @@ -1075,6 +1190,18 @@ The effect is absent in 70M–410M Pythia models despite adequate task competenc
8. uv project and lockfile documentation:
https://docs.astral.sh/uv/

9. Anthropic Jacobian-lens reference implementation:
https://github.com/anthropics/jacobian-lens

10. `solarkyle/jspace` — Gemma-4-12B reliability campaign with published fitted lenses and traces:
https://github.com/solarkyle/jspace

11. `amaljithkuttamath/jlens-replication` — shuffled-corpus and truncation controls at 2–4B scale:
https://github.com/amaljithkuttamath/jlens-replication

12. Full J-space method review, evidence floor, and repository survey:
[docs/j-space-global-workspace.md](docs/j-space-global-workspace.md)

---

## Final decision rule
Expand Down
Binary file not shown.
23 changes: 23 additions & 0 deletions docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,29 @@ recurrent wire inside a single forward pass. See [architecture-note.md](architec

---

### J-space (`j_space`) — Verbalization-poised subspace

The set of sparse nonnegative combinations of Jacobian-lens dictionary directions at a
layer — a union of low-dimensional cones, not a linear subspace. In this repository
`j_space` means only: the region of activation space poised, on average across contexts,
to influence future verbalization, as defined operationally by a fitted lens and a
registered sparsity $k$.

Requirements for use:

- A J-space claim is causal or it is not made. Passive J-lens readout is never
gate-eligible evidence: token-shuffled-corpus control lenses match it at 2–4B scale.
- Any off-J ("bypass") claim requires survival under downstream J-clamping.
- Every fitted lens is a content-addressed artifact with a manifest, and every J-space
experiment fits or reuses shuffled-corpus and rotated-dictionary control lenses.

"Global workspace" is **not** accepted vocabulary in this repository. It is the external
literature's hypothesis about broadcast geometry, not a validated construct here. See
[claim-boundaries.md](claim-boundaries.md) and
[j-space-global-workspace.md](j-space-global-workspace.md).

---

## 3. Evidence terms

### Efferent / afferent
Expand Down
Loading
Loading