Release: merge architecture2 into main (23 commits — OSS /v1 + OpenAPI + runtime refactors)#31
Merged
Release: merge architecture2 into main (23 commits — OSS /v1 + OpenAPI + runtime refactors)#31
Conversation
chore: split local side-by-side compose cleanup
…coverage (#8) • add createCoreRuntime, createApp, and structured startup checks under src/app/ • remove setEntityRepository and thread entityRepo explicitly through the search pipeline • slim server.ts to boot from the runtime container while preserving existing exports • add focused runtime-container and deployment/config validation coverage • keep endpoint behavior and route behavior unchanged
• return a canonical search-scope contract from the API instead of relying on implicit behavior • keep scope semantics explicit at the route/service boundary • add focused tests to lock the contract and preserve current behavior
…#10) • return canonical search scope metadata from the API • make retrieval observability explicit while keeping it optional when trace payloads are absent • route config mutation through the runtime adapter instead of hidden direct wiring • preserve current search behavior while tightening the API contract and coverage
• finish the Phase 3 config-seam cleanup needed before honest Phase 4 work • thread runtime config through the search, ingest, route, and lineage paths that previously relied on singleton reads • make API scope/observability behavior explicit without changing the intended fast-path behavior • add focused tests and guardrails so the narrowed seam stays honest during later Phase 4 changes
Extract shared test helpers and deduplicate search pipeline test setup: - Extract bindEphemeral + BootedApp into src/__tests__/test-helpers.ts, used by memory-route-config-seam.test.ts and composed-boot-parity.test.ts - Extract twoLowSimilarityResults + createVectorRepo helpers in search-pipeline-runtime-config.test.ts to eliminate repeated 4-test setup boilerplate - Extract resolveSearchPreamble in routes/memories.ts to deduplicate the parse-scope-limit pattern between /search and /search/fast Remaining 234 lines (0.7%) are structural patterns that can't be meaningfully extracted: camelCase/snake_case config snapshots, idiomatic Express route handler preambles, port interface shapes, and test mock config objects. fallow exits 0 (no threshold configured).
Splits memory-ingest.ts (475 → 215 lines) into three focused modules with explicit seams, completing Phase 4 of the core rearchitecture. Extraction (Steps 4A/4B): - ingest-post-write.ts: backdate, link generation, and composite generation behind runPostWriteProcessors(). Composites are caller-gated via compositesEnabled — only performIngest passes true. - ingest-fact-pipeline.ts: unifies ingestFact, quickIngestFact, and workspaceIngestFact behind processFactThroughPipeline() with FactPipelineOptions. Workspace AUDN asymmetry (no lineage, no deferred AUDN) preserved verbatim. Honesty fixes: - Removed dishonest resolveClaims option that was documented but never consulted in the full AUDN path. - Corrected overstated config-threading claims in runtime-container.ts JSDoc — now explicit about where threading is honest (search/ingest) vs where the singleton still leaks (embedding, LLM, extraction). - Ratcheted singleton gate from 34 → 33. Behavioral fences (codex review): - 6 workspace AUDN tests covering ADD/UPDATE/SUPERSEDE/NOOP routing plus negative assertions that the workspace path bypasses canonical storage, full AUDN resolution, and entropy gating. - 6 post-write tests covering the compositesEnabled boundary, backdate ordering, and composite threshold gating. 933/933 tests pass. tsc clean. fallow clean. No behavior change. Public API (memory-service.ts) untouched.
…aps (#15) Addresses the codex architecture review's two High findings with bounded, honest progress — unifying the search/expand boundary behind MemoryScope and documenting the schema scoping gaps as explicitly deferred. Scope contract (Finding 1 — partially addressed): - Expands MemoryScope workspace variant with required agentId for visibility enforcement, adds ScopedSearchOptions to collapse divergent search parameter lists into one options bag. - Introduces scopedSearch() and scopedExpand() on MemoryService that dispatch on scope.kind — user scope routes to performSearch/expandMemories, workspace scope routes to performWorkspaceSearch/expandMemoriesInWorkspace. - Search routes (/search, /search/fast) and /expand now call the scoped methods instead of branching inline. Old paired methods marked @deprecated. - Honestly scoped: list/get/delete routes stay branched (no agent_id in their request contracts). Ingest stays branched (needs write-context fields MemoryScope doesn't carry). The deeper repo/pipeline split is Phase 5. Schema honesty (Finding 2 — deferred, not resolved): - docs/design/scope-access-contract.md documents the full scope model: what MemoryScope is, where it flows, which tables support workspace scoping and which don't, why claims and foresight are intentionally user-scoped (AUDN contradiction resolution is cross-workspace), and the Phase 5 prerequisites for goal/version expansion. - SCOPE_TODO comments added to all 6 unscoped tables in schema.sql (memory_atomic_facts, memory_foresight, memory_claims, memory_claim_versions, entities, lessons). 938/938 tests pass (6 new dispatch tests). tsc clean. Fallow clean. No behavior change to any existing path.
Replaces the 40+ method MemoryRepository facade with 8 focused store interfaces (MemoryStore, EpisodeStore, SearchStore, SemanticLinkStore, RepresentationStore, ClaimStore, EntityStore, LessonStore), routes workspace ingest through canonical lineage for the first time, and closes a live workspace visibility bypass on the public query routes. Executed in 10 planned steps with codex review at every gate. Access control (breaking): - Workspace GET /memories/list, GET /memories/:id, and DELETE /memories/:id now require agent_id when workspace_id is present — missing returns 400, not a visibility-unsafe fallback. Legacy *InWorkspace facade methods deleted. - scopedExpand, scopedGet, scopedDelete, scopedList all enforce agent-level visibility via the same SQL clause search uses (buildVisibilityClauseForSearch). agent_only memories hidden from non-owning agents; restricted requires an explicit grant. - Malformed agent_id validated as UUID → 400. Invisible workspace delete → 404, not 500. Store layer: - ClaimStore, EntityStore, LessonStore narrowed via Pick<> to the exact method surfaces their consumers call (no full-repo escape hatch). - 5 Pg*Store classes wrap the existing split repository functions. - CoreStores bundle wired into runtime-container alongside existing repos; MemoryServiceDeps now carries only stores.* (deprecated deps.repo/deps.claims/deps.entities/deps.lessons fields removed). - pool: pg.Pool isolated behind CoreStores.pool — all previous deps.repo.getPool() callers (search, link generation, deferred-audn, reconciliation) now source from deps.stores.pool. - All 10 downstream search helpers (iterative-retrieval, agentic-retrieval, 4 query-expansion variants, keyword-expansion, deferred-audn, consolidation-service, subject-aware-ranking) migrated to SearchStore/MemoryStore/EntityStore params. All 8 as any bridges deleted — the search pipeline is now fully compiler-enforced. Schema: - memory_atomic_facts and memory_foresight gain workspace_id + agent_id columns via idempotent ALTER TABLE ... ADD COLUMN IF NOT EXISTS, plus partial indexes on workspace_id. - StoreAtomicFactInput/StoreForesightInput and the AtomicFactRow/ForesightRow types carry the new fields. MemoryRepository marked @deprecated. Workspace canonical lineage (behavioral change): - processWorkspaceFact now routes through storeCanonicalFact and resolveAndExecuteAudn instead of the inline storeWorkspaceMemory path. Workspace memories gain: CMO creation, claim/version lineage, atomic fact decomposition, foresight projections, entity resolution, and linking. - AudnFactContext carries optional workspace; storeProjection, supersedeCanonicalFact, and updateCanonicalFact thread it through — fixing the scope-loss that would have occurred on delete-and-reinsert replace paths. - The two AUDN side branches (CLARIFY clarification memory, opinion-confidence-collapse) also thread workspace scope. - storeWorkspaceMemory deleted. Intentional Phase 5 debt, pinned by test: Claims, claim versions, entities, and entity relations remain user-scoped. Cross-workspace ingests for the same user share entity/claim state. A behavioral regression test runs storeCanonicalFact twice with two workspace contexts against a stateful fake EntityStore and asserts the second call reuses entities from the first, plus that resolveEntity is called with no workspace/agent fields — the exact Phase 6 flip point. Fences added: - 3 visibility regression tests (expand-visibility.test.ts) - 4 route-level 400/404 tests - 3 new 400-fence tests for missing agent_id - 2 runtime-container store-construction tests - 3 scoped dispatch tests - 2 AUDN workspace-scope fences (CLARIFY + opinion-collapse) with strict call-count assertions - 3 behavioral workspace-pipeline tests - 3 cross-workspace coupling fences (stateful fake, Phase 6 flip point) 953/953 tests pass. npx tsc --noEmit clean. fallow --no-cache clean. Config singleton gate unchanged. Public API shape: only scoped methods remain on MemoryService.
…ers (#17) Phases 1A–1B built the composition root (createCoreRuntime, createApp) but kept it internal. Phase 6 promotes the seams to the public surface, documents them, and pins behavior with a contract test so HTTP and in-process ingest cannot silently diverge. Public API src/index.ts exports createCoreRuntime + types, createApp, checkEmbeddingDimensions, bindEphemeral. Existing exports preserved — current research paths keep working. Seam promotion src/__tests__/test-helpers.ts moved to src/app/bind-ephemeral.ts so research harnesses can import the canonical HTTP-boot helper without reaching into a test dir. Two importers updated. Contract test (src/app/__tests__/research-consumption-seams.test.ts) - In-process seam: runtime.services.memory.ingest/search end-to-end - HTTP seam: POST /memories/ingest + /memories/search via bindEphemeral - Parity, both directions: in-process write → HTTP search, and HTTP write → in-process search. Two-directional parity catches regressions where HTTP input parsing or response shaping drifts from the in-process contract — one direction alone would miss that. Docs - docs/consuming-core.md — HTTP / in-process / docker seams, copy-pastable examples using the actual package name (@atomicmemory/atomicmemory-engine), stability boundary for deep-path imports, what belongs in research. - docs/README.md + CLAUDE.md — pointers to the new doc. Out of scope (per plan) - No research-repo migration - No narrowing of deep-path exports in package.json (Risk 4 — avoid breaking research in the same step that gives it a new seam) - No config split or retrieval decomposition Verification - 957/957 tests pass - tsc --noEmit clean - fallow --no-cache 0 above threshold
…18) Phase 7 Steps 3a–3c + doc followup. Completes the Phase 1A-descoped config split, deprecates PUT /memories/config for production, freezes provider/model selection as startup-only. No parity regression — gated by the v1→v2 parity audit (companion: atomicmemory-research#7). Changes 3a — partition runtime config. SUPPORTED_RUNTIME_CONFIG_FIELDS (39 stable) + INTERNAL_POLICY_CONFIG_FIELDS (66 experimental) in src/config.ts, with Pick<> slices as documentation types. New fence test rejects stray/duplicate keys — any new RuntimeConfig field must be tagged. Re-exported from root. 3b — deprecate PUT /memories/config for production. New runtimeConfigMutationEnabled flag, parsed from CORE_RUNTIME_CONFIG_MUTATION_ENABLED (default false). Production returns 410 Gone; .env.test opts in for dev/CI. Flag read from a memoized startup snapshot through configRouteAdapter — no per-request env reads, no NODE_ENV branching. 3c — freeze provider/model. embedding_provider, embedding_model, llm_provider, llm_model now rejected with 400 + rejected array. Rationale: the provider caches are fixed at first use — v1 accepted these fields but mid-flight mutation never took effect. Bug fix, not a feature loss. Followup. docs/api-reference.md § rewritten to match the new contract. Success-response note fixed (it claimed provider/model is applied in-memory; they're now rejected earlier in the same handler). Out of scope Step 3d (multi-PR leaf-config threading), Item 4 (retrieval polish), Item 2 (main cutover — deferred). Test plan - npx tsc --noEmit clean - npm test 963/963 (+6 new: 4 partition, 1 410, 1 400) - fallow --no-cache 0 above threshold
…n audit) (#21) Five leaves, two patterns Explicit parameter threading (callers already had deps.config in scope): - consensus-extraction.ts, write-security.ts, cost-telemetry.ts Module-local init (hot-path APIs with 15+ callers — ripple avoidance): - embedding.ts via initEmbedding(config), llm.ts via initLlm(config) - runtime-container.ts calls both inits inside createCoreRuntime Deep-path init requirement Dropping config from embedding.ts/llm.ts is a contract change for consumers who deep-import /services/embedding or /services/llm (still public per package.json#exports; narrowing is out of scope). Mitigations: - src/index.ts re-exports initEmbedding, initLlm, EmbeddingConfig, LLMConfig from the root. - docs/consuming-core.md adds a Deep-path init requirement section. Test-file cleanups write-security.test.ts, embedding-cache.test.ts, llm-providers.test.ts stopped mutating the config singleton; each builds its own config object / calls init*(testConfig) explicitly. Test plan - npx tsc --noEmit clean - npm test 963/963 pass - fallow --no-cache 0 above threshold, maintainability 91.0 Out of scope Remaining audit drops below 28 (infrastructure files); deep-path export narrowing; Item 4 (separate PR).
…-search.ts so it becomes pure orchestration. 374 → 248 lines (-34%)(#22) Helpers moved - buildInjection → retrieval-format.ts (alongside existing formatSimpleInjection / formatTieredInjection; new InjectionBuildResult interface) - applyFlatPackagingPolicy → composite-dedup.ts (thin wrapper around deduplicateCompositeMembersForFlatQuery already there) - recordSearchSideEffects → new retrieval-side-effects.ts (touchMemory + audit emission) - recordConsensusLessons → lesson-service.ts (static import replaces previous dynamic-import round-trip; signature narrowed to take LessonStore instead of full MemoryServiceDeps) - finalizePackagingTrace → packaging-observability.ts (bundles packaging/assembly summary + tiered-packaging event + trace setters; absorbs deduplicateCompositeMembersHard dependency) Stability performSearch, performFastSearch, performWorkspaceSearch keep their existing signatures. All plan regression canaries pass: smoke.test.ts, memory-search-runtime-config.test.ts, search-pipeline-runtime-config.test.ts, contradiction-safe.test.ts, temporal-correctness.test.ts. Test plan - npx tsc --noEmit clean - npm test 963/963 pass (no new tests — pure refactor) - fallow --no-cache 0 above threshold, maintainability 90.9 - memory-search.ts ≤ 250 lines (248, hits plan target)
Makes atomicmemory-core consumable from public npm under its new name, adds a /v1 API prefix, and scrubs internal codenames + private links.⚠️ Breaking - All endpoints moved under `/v1` (e.g. `POST /v1/memories/ingest`). `/health` stays unversioned. - npm package renamed `@atomicmemory/atomicmemory-engine` → `@atomicmemory/atomicmemory-core`. - Tarball now ships compiled `dist/` (built via `prepublishOnly`); raw `src/*.ts` was unimportable for published consumers. Other - `SuperMem` codename + `supermem` DB name scrubbed from src, tests, docker-compose, .env.example. - Private `atomicmemory-research` URLs unlinked from public docs. - `release.yml` publishes to npm on `v*` tag push (NPM_TOKEN secret). - CI: `CORE_RUNTIME_CONFIG_MUTATION_ENABLED=true` so the `PUT /v1/memories/config` parity test stops returning 410. 966/966 tests pass · tsc clean · fallow clean · `npm publish --dry-run` ok.
* OpenAPI Phase 1: Zod + validate middleware scaffolding
Scaffolds the pieces every subsequent OpenAPI phase will build on,
without changing any runtime route behavior.
Adds:
- zod ^4.3.6 and @asteasolutions/zod-to-openapi ^8.5.0 deps
- src/schemas/zod-setup.ts: calls extendZodWithOpenApi(z) once at
module load so every schema file using `.openapi({...})` shares the
same extended Zod namespace
- src/schemas/common.ts: shared primitives consumed by Phase 2's
route schemas — MemoryVisibility, WorkspaceContext (snake_case wire
form matching parseOptionalWorkspaceContext, memories.ts:601),
AgentScope (matching parseOptionalAgentScope, memories.ts:617),
RetrievalMode, NonEmptyString, IsoTimestamp
- src/schemas/errors.ts: Zod schemas for the three HTTP error
envelopes — ErrorBasic ({ error }) used by every route's 400/500,
ErrorConfig400 ({ error, detail, rejected[] }) and ErrorConfig410
({ error, detail }) for PUT /v1/memories/config's special paths
(memories.ts:269-282)
- src/middleware/validate.ts: validateBody / validateQuery /
validateParams factories. formatZodIssues flattens Zod's structured
issue list into a single descriptive string so the 400 response
stays byte-identical to today's `{ error: string }` envelope
(route-errors.ts:19). assertResponse helper for non-prod-only
response-shape drift detection in Phase 2 handlers.
Nothing imports these yet — Phase 2 wires validateBody into route
handlers and Phase 4 feeds the schemas into OpenApiGeneratorV31.
36 unit tests / 995 full suite tests pass. tsc clean. fallow exit 0.
* Fix Phase 1 shared schemas to match current parser contracts
Codex review caught two behavior-preservation regressions in the
Phase 1 scaffolding:
1. NonEmptyString trimmed input and rejected whitespace-only strings.
Current requireBodyString (memories.ts:571) only checks truthy +
typeof string — no trim, whitespace-only passes. Phase 2 would have
silently mutated existing inputs and 400'd existing whitespace-only
inputs.
Fix: use plain z.string().min(1) with no transform. Tests updated
to pin the preserved behavior (whitespace-only is accepted; input
is not trimmed).
2. WorkspaceContextSchema modeled a required nested-object request
shape with strict visibility enum. Current
parseOptionalWorkspaceContext (memories.ts:601-615) reads
workspace_id / agent_id as top-level body fields, treats either
missing as "no workspace context" (falls back to user scope, no
400), and silently drops invalid visibility values to undefined.
Fix: split into three request-level field schemas
(WorkspaceIdField, AgentIdField, VisibilityField) with
.catch(undefined) on visibility, plus a separate
WorkspaceContextOutputSchema for the post-transform camelCase
internal shape. Phase 2 route schemas compose the request-level
fields and emit the output shape when both ids are present.
Tests updated to lock in the preserved contract:
- NonEmptyString accepts ' ', does not trim
- VisibilityField silently drops 'something_invalid' to undefined
- WorkspaceIdField / AgentIdField are optional at the request level
* Fix Phase 1 schemas to silently coerce invalid inputs (match parser)
Second codex pass flagged three more behavior-preservation gaps.
Current parsers never 400 on malformed workspace / agent_scope / as_of
inputs — they silently return undefined so the route falls back to
user scope or treats the field as absent. Phase 2 composition using
the stricter Phase 1 schemas would have started 400'ing requests that
pass today.
Fixes:
- WorkspaceIdField / AgentIdField: add `.catch(undefined)` so empty
strings, non-strings, null, and object inputs coerce to undefined
instead of failing validation. Matches optionalBodyString's
typeof-string-else-undefined path (memories.ts:601-615).
- AgentScopeSchema: add `.optional().catch(undefined)` so invalid
shapes like `agent_scope: 42` or `{}` become undefined instead of
400 — matches parseOptionalAgentScope (memories.ts:617-623).
- IsoTimestamp: preprocess `''` and `null` to `undefined` before the
string+date validation, so `as_of: ''` stays "absent" rather than
regressing to 400. Matches parseOptionalIsoTimestamp explicit null-
and-empty sentinel handling (memories.ts:641-647).
Tests updated to pin the coerce-to-undefined behavior for every
previously-tolerated input shape.
1009/1009 tests pass; fallow exit 0.
* OpenAPI Phase 2: memories route Zod schemas + handler refactor
Refactors all 24 /v1/memories/* handlers to delegate request parsing
to Zod schemas via validateBody / validateQuery / validateParams.
Deletes the inline parseIngestBody / parseSearchBody /
parseOptionalWorkspaceContext / parseOptionalAgentScope /
parseOptionalIsoTimestamp / parseTokenBudget / parseRetrievalMode /
requireBodyString / requireQueryString / requireUuidParam /
optionalBodyString / optionalQueryString / optionalUuidQuery / parseLimit /
parseUserIdAndLimit helpers (previously 515-647 in memories.ts).
Every request contract is preserved byte-for-byte:
- 400 envelope is { error: string } (formatZodIssues flattens the
ZodError issue list into the same style as the prior InputError
messages).
- parseOptionalWorkspaceContext's silent-drop-partial contract is
preserved via WorkspaceIdField / AgentIdField / VisibilityField.
- parseOptionalAgentScope's silent-drop-invalid contract is
preserved via AgentScopeSchema.optional().catch(undefined).
- parseOptionalIsoTimestamp's '' / null = absent contract preserved
via IsoTimestamp's preprocess step.
- retrieval_mode / token_budget / conversation-length / limit-clamp
math matches the original parsers exactly.
- PUT /config's 410 + rejected[] handler logic stays in-handler
(those are business rules, not input validation).
Middleware fix: Express 5 exposes req.query / req.params as
getter-only properties on Request.prototype. The validators now use
Object.defineProperty to shadow the prototype accessor with an own
property, so handlers see the parsed + transformed values after
validateQuery / validateParams. req.body is writable normally.
1009/1009 tests pass. tsc clean. fallow exit 0. memories.ts shrank
768 → 667 LOC (inline helpers removed).
* Fix Phase 2 wire-contract regressions in required-field messages + filters
Codex review caught two byte-level wire-contract regressions on the
Phase 2 refactor.
1. Required-field error messages leaked generic Zod text. Plain
z.string().min(1) for user_id / conversation / source_site /
query / pattern / memory_ids produced "Invalid input: expected
string, received undefined" on missing/wrong-type inputs, while
the old parsers emitted "${label} (string) is required" /
"memory_ids (string[]) is required". Clients matching on the
exact message would have broken.
Fix: add requiredStringBody(label) and requiredStringArrayBody(
label) helpers that build schemas whose single refine() produces
the exact prior-parser message for EVERY failure mode (missing,
null, wrong type, empty string). Applied to every required body
field across IngestBody, SearchBody, ExpandBody, ConsolidateBody,
DecayBody, ResetSourceBody, LessonReportBody.
2. source_site / namespace_scope empty-string collapsed to undefined.
The Phase 2 OptionalBodyString transform dropped '' to undefined;
optionalBodyString() preserved it verbatim. POST /search with
source_site: '' or namespace_scope: '' no longer reached the
service with the same payload.
Fix: redefine OptionalBodyString as `z.string().optional().catch(
undefined)` — no length transform. Empty string passes through
unchanged; non-strings and null coerce to undefined.
Regression tests in src/schemas/__tests__/memories.test.ts pin both:
- Exact "${label} (string) is required" / "(string[]) is
required" messages across missing / non-string / empty-string /
non-array inputs.
- source_site: '' and namespace_scope: '' round-trip verbatim.
- Over-length conversation error message matches verbatim.
1023/1023 tests pass; tsc clean; fallow exit 0.
* OpenAPI Phase 3: agents route Zod schemas + handler refactor
Refactors all 5 /v1/agents/* handlers to delegate request parsing to
Zod schemas via validateBody / validateQuery / validateParams.
Deletes the inline requireString / requireTrustLevel /
requireResolution helpers at agents.ts:86-109.
Preserved wire contracts:
- "agent_id is required" / "user_id is required" (from requireString)
- "trust_level must be a number between 0.0 and 1.0" (wrong type)
- "trust_level must be between 0.0 and 1.0" (out of range)
— distinct messages preserved via superRefine
- 'resolution must be "resolved_new", "resolved_existing", or
"resolved_both"'
- requireString's array-to-first-element quirk that handles
Express duplicate-query params: `?user_id=a&user_id=b` →
`['a','b']` → takes 'a'. Preserved via requiredStringOrArray helper.
- Empty-array input → "${field} is required" (not generic).
Regression tests in src/schemas/__tests__/agents.test.ts lock in all
error-message contracts + the array-first-element behavior.
Cleanup: with Phase 2+3 done, nothing throws InputError anywhere —
all validation is middleware-emitted 400. Deleted the now-dead
InputError class and simplified handleRouteError to always return 500
(which is what it already did for non-InputError errors). Stack logs
now consistently label caught errors as 500.
1037/1037 tests pass; tsc clean; fallow exit 0.
* Fix stale InputError comment references
Codex noted documentation drift: two docstrings still mentioned the
now-removed InputError type. No runtime impact.
- src/middleware/validate.ts: reword the intro block to describe the
400 envelope without the stale type reference.
- src/schemas/errors.ts: split the canonical-envelope docstring into
a per-status explanation (400 from validate middleware, 500 from
handleRouteError) and drop the InputError mention from the
per-schema comment.
The historical code quote in src/schemas/common.ts:108 intentionally
shows the pre-Zod parser body verbatim and stays as-is.
1037/1037 tests pass; tsc clean; fallow exit 0.
* OpenAPI Phase 4: spec generation + npm publish plumbing
Wires every /v1/memories/* and /v1/agents/* route into an
OpenAPIRegistry and generates `openapi.json` + `openapi.yaml` at
repo root. Spec ships as an artifact of the core npm package.
New files:
- src/schemas/openapi.ts: registerPath() entries for all 29
operations (24 memory + 5 agents) with per-route response
inventory. 200 + 400 + 500 default; GET/DELETE /memories/{id}
also 404; PUT /memories/config adds 410 (runtime mutation
disabled) + richer 400 (startup-only fields with `rejected[]`).
- scripts/generate-openapi.ts: walks the registry, emits
deterministic YAML + JSON (sorted keys, fixed info block).
- scripts/smoke-openapi-export.mjs: consumer-resolution smoke that
uses `createRequire` (Node 22 + "type":"module" + plain ESM JSON
import fails with ERR_IMPORT_ATTRIBUTE_MISSING). Asserts 29+
operations reachable via
`require('@atomicmemory/atomicmemory-core/openapi.json')`.
package.json changes (the load-bearing bits):
- exports: add `./openapi.json` and `./openapi.yaml` entries so
Node ESM resolution accepts `require('…/openapi.json')`. `files[]`
alone is not enough without explicit export entries.
- files: add `openapi.json` and `openapi.yaml` alongside `dist`.
- scripts: `generate:openapi`, `check:openapi` (regenerate + git
diff --exit-code), and updated `prepublishOnly` to regenerate
before build.
CI: .github/workflows/ci.yml gains a `check:openapi` step that runs
`npm run generate:openapi && git diff --exit-code`. Blocks PRs that
change Zod schemas without regenerating the spec.
Zod refactor required for generator compatibility: zod-to-openapi's
traversal fails on ZodCatch nodes even when `.openapi()` metadata is
attached to the wrapper. Replaced every `.optional().catch(undefined)`
chain with `z.unknown().transform(v => ...)` + `.openapi()` type
tags. Runtime semantics are byte-identical — the transform performs
the same "coerce non-matching input to undefined" operation `.catch()`
did. 1037/1037 tests still pass.
Packed tarball verified to include both openapi.{json,yaml}. Smoke
check reports `openapi 3.1.0 / AtomicMemory HTTP API v1.0.0 /
27 paths / 29 operations` after install from tarball.
* Fix OpenAPI spec + smoke script regressions from codex review
Three codex findings on Phase 4:
1. UUID path params emitted invalid JSON Schema `pattern` strings
containing the JS regex-literal syntax (including the trailing
`/i` flag). Consumers parsing the pattern would reject or
mis-validate. Fix: annotate `id` in UuidIdParamSchema with
`.openapi({ type: 'string', format: 'uuid' })` so zod-to-openapi
emits standard `format: "uuid"` instead of stringifying the
regex. Applies to GET /memories/{id}, DELETE /memories/{id},
GET /memories/{id}/audit.
2. PUT /v1/memories/config documented only the rich startup-only-
fields 400 envelope, but `validateBody(ConfigBodySchema)` can
emit the basic `{ error }` envelope too. Fix: widen the 400
response to `oneOf: [ErrorBasic, ErrorConfig400]` so both shapes
are part of the published contract.
3. scripts/smoke-openapi-export.mjs used createRequire(import.meta.url),
which anchors resolution at the script's own location. Running
the script against a packed tarball in a scratch dir still self-
resolved back to the repo's own openapi.json — a false-positive
smoke for publish/export regressions. Fix: anchor createRequire
at `cwd/package.json` so the scratch project's node_modules is
traversed. Verified locally: passes when the tarball is installed,
throws "Cannot find module" when it isn't.
1037/1037 tests pass; tsc clean; fallow exit 0; spec regenerated.
* Remove accidentally committed tarball + gitignore *.tgz
The 827-line hand-authored endpoint reference at `docs/api-reference.md` has been a drift risk ever since routes acquired Zod-generated OpenAPI. atomicmemory-docs already renders the full reference from the spec shipped in `@atomicmemory/atomicmemory-core`. Changes: - docs/api-reference.md shrinks to a ~30-line pointer page that links to the rendered reference and describes the Zod-schemas → openapi.yaml → rendered-docs flow, including how to regen and how downstream consumers resolve the spec. - docs/README.md updates the file summary and "Contributing docs" section to point at src/schemas/ instead of the deleted prose. - docs/consuming-core.md swaps the "See docs/api-reference.md" pointer for a link to docs.atomicmemory.ai. - CONTRIBUTING.md's "Adding a new route" checklist replaces "Update docs/api-reference.md" with the Zod-schema + generate- openapi workflow that CI's check:openapi step enforces. No runtime / code changes. tsc clean, 1037 tests pass, fallow exit 0, openapi.yaml + .json regenerated cleanly (check:openapi passes). This closes Phase 6 of the OpenAPI integration plan.
`npm ci --omit=dev` on `node:22-slim` fails during Docker build with:
npm error ERESOLVE could not resolve
... peerOptional zod@"^3.23.8" from openai@4.104.0
... Conflicting peer dependency: zod@3.25.76 (root pins zod@^4.3.6)
Host-side installs succeed because developer machines have npm configured
with softer peer-dep resolution (or use pnpm, which ignores strict peer
conflicts by default). `npm ci` on the slim Node image is strict, so
every fresh `docker compose up --build` breaks until upstream openai
publishes a zod@4-compatible release.
`--legacy-peer-deps` is the documented escape hatch for this exact case
and matches the resolution behaviour the host already uses. Drop it once
openai supports zod@4.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fast-forward
maintoarchitecture2— 23 commits of shipped work that has been the active development line since the branch opened.architecture2is a strict superset ofmain(0 main-only commits), so this is a clean release.What lands
OSS release +
/v1migrationaa79395OSS release prep: public package +/v1+ npm publish wiring (v1.0.0: architecture2 rearchitecture + OSS release prep #24)348245dDockerfile: pass--legacy-peer-depstonpm ci(Dockerfile: pass --legacy-peer-deps to npm ci #30)OpenAPI generation pipeline (6 phases)
dfdc18fPhase 1: Zod + validate middleware scaffolding (OpenAPI Phase 1: Zod + validate middleware scaffolding #25)5f0b480Phase 2: memories route Zod schemas + handler refactor (OpenAPI Phase 2: memories route Zod schemas + handler refactor #26)a99dfb9Phase 3: agents route Zod schemas + handler refactor (OpenAPI Phase 3: agents route Zod schemas + handler refactor #27)c103224Phase 4: spec generation + npm publish plumbing (OpenAPI Phase 4: spec generation + npm publish plumbing #28)2939eefPhase 6: retire hand-written api-reference.md (OpenAPI Phase 6: retire hand-written api-reference.md #29)Runtime + architecture refactors (Phases 1A–7)
e329c45Phase 1A: runtime container composition root + integration coverage (Phase 1A: introduce runtime container composition root + integration coverage #8)c2c2c42Phase 2A: canonical search scope contract (Phase 2A: add canonical search scope contract #9)0196f74Phase 2B: retrieval observability contract explicit and optional (Phase 2B: make retrieval observability contract explicit and optional #10)874e4b2Phase 3: runtime-config seam cleanup to Phase 4 boundary (Phase 3: runtime-config seam cleanup to Phase 4 boundary #11)c479409Phase 4: decompose ingest pipeline into focused processors (Phase 4: split ingest into extraction, decision, write, and post-write processors #13)28cb232Post-Phase 4: unify search scope contract + document schema gaps (Post-Phase 4 cleanup: scope contract + schema honesty #15)d5ff9aaPhase 5: narrow repository access behind domain-facing stores (Phase 5: narrow repository access behind domain-facing stores #16)f65db73Phase 6: publish stable consumption seams for research and SDK consumers (Phase 6: publish stable consumption seams for research and SDK consumers #17)f987dcePhase 7 Steps 3a–3c: config split +PUT /memories/configdeprecation (Phase 7 Steps 3a-3c: config split + PUT /memories/config deprecation #18)2517774Phase 7 Step 3d: thread config through 5 leaf modules (Phase 7 Step 3d: thread config through 5 leaf modules (33→28 singleton audit) #21)9a25437Phase 7 Item 4: retrieval polish —memory-search.ts374 → 248 lines (Phase 7 Item 4: extract 4 helpers from memory-search.ts #22)Housekeeping
b36e0a3Reduce fallow duplication 367 → 234 lines (chore: fallow duplication cleanup post Phase 3 #12)3e8256bDocument scopedExpand visibility gap7988640Local compose side-by-side (chore: split local side-by-side compose cleanup #7) + CI port assertion fixTest plan
architecture2is running and serving/v1/*— exercised end-to-end by the memory-lab ingest/search flownpm testgreen onarchitecture2HEADnpx tsc --noEmitgreenfallow --no-cacheclean