The connect phase writes edge evidence in the form "Concept A ... Concept B ...". In a live store that is 2,665 of 3,259 edges — 82%.
refine consumes edge evidence as its source material, so the model echoes those labels straight into definitions. Observed verbatim:
Concept A emphasizes the capacity of the memory concept to support diverse endeavors, while Concept B explores the nature of inquiry through diachronic analysis of word densities.
Six rows in the live store were carrying it before repair, one of them named Concept A describes the agent's capacity to retain… — a memory whose label is a prompt placeholder.
Why this is worth fixing at the source
#52 blocks the contamination at the write, and that check should stay. But it is catching something generated upstream, so the cost lands as a high rejection rate: in one 30-row dream run, 6 of 8 rejections were placeholder leaks. Every rejection is a refinement thrown away, so consolidation does progressively less useful work while reporting success.
It also propagates. A contaminated definition yields a contaminated name, and that name then re-contaminates any repair attempt that feeds the name back to the model as ground truth — the row becomes permanently unrepairable until the name is withheld.
Options
- Neutralise the labels in
refine before the evidence reaches the model (cheapest, local).
- Have
connect persist evidence that names the actual concepts rather than Concept A/Concept B (better, but changes stored evidence going forward and does nothing for the 2,665 existing edges).
- Both, with a backfill for existing edge evidence.
Reproduce
select count(*) from edges where evidence like '%Concept A%' or evidence like '%Concept B%';
Related: #52 (the gate that currently catches this).
The
connectphase writes edge evidence in the form"Concept A ... Concept B ...". In a live store that is 2,665 of 3,259 edges — 82%.refineconsumes edge evidence as its source material, so the model echoes those labels straight into definitions. Observed verbatim:Six rows in the live store were carrying it before repair, one of them named
Concept A describes the agent's capacity to retain…— a memory whose label is a prompt placeholder.Why this is worth fixing at the source
#52 blocks the contamination at the write, and that check should stay. But it is catching something generated upstream, so the cost lands as a high rejection rate: in one 30-row dream run, 6 of 8 rejections were placeholder leaks. Every rejection is a refinement thrown away, so consolidation does progressively less useful work while reporting success.
It also propagates. A contaminated definition yields a contaminated name, and that name then re-contaminates any repair attempt that feeds the name back to the model as ground truth — the row becomes permanently unrepairable until the name is withheld.
Options
refinebefore the evidence reaches the model (cheapest, local).connectpersist evidence that names the actual concepts rather thanConcept A/Concept B(better, but changes stored evidence going forward and does nothing for the 2,665 existing edges).Reproduce
Related: #52 (the gate that currently catches this).