Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .changeset/metadata-changed-event-payload-retired.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
"@objectstack/spec": minor
---

feat(spec): retire `MetadataChangedEventPayloadSchema` — the `metadata:changed` payload nothing ever emitted or consumed (#14180, ADR-0049)

<!-- adr-0087: registered metadata-changed-event-payload-retired -->

**BREAKING** export removal, landing after the v17.0.0 cut (the lockstep
launch-window convention ships it as `minor`; the prescription is registered
under protocol major 18 — `RETIRED_DEFS_BY_MAJOR[18]` + the D3 semantic entry
`metadata-changed-event-payload-retired` — where `os migrate meta` users will
look).

`kernel/cluster.zod.ts` declared a "canonical payload for the
`metadata:changed` event" and said every metadata persistence layer MUST emit
it after any successful write and every reader MUST subscribe and compare its
`version` before invalidating. Nothing ever did either: zero runtime emitters,
zero subscribers, zero imports outside `packages/spec` (its own unit test, the
isomorphic alias pin and the generated artifacts), in objectstack and in
objectui at the pinned sha. It could not have been honoured as declared — the
`version` field is `z.bigint()`, which the standard JSON serializer refuses, so
the payload could not cross any pubsub transport without a codec no driver
ships. The three cluster channels that do run (`metadata.changed`,
`metadata.mutated`, `datasource.mutated`) all carry an address-only signal
whose receiver re-reads its own store — the opposite of the declared
version-compare receipt — so the one plausible future consumer was decided
against (2026-09-01 ruling), and the triage ruling (2026-09-02) chose removal
over "make a consumer".

FROM → TO:

- `MetadataChangedEventPayloadSchema` / `MetadataChangedEventPayload` →
*(removed)* — no replacement type is declared. Subscribe to one of the lanes
documented in `content/docs/kernel/cluster.mdx` §6.2 instead:
`metadata.changed` (`ClusterMetadataChangedPayload`, `@objectstack/metadata`),
`metadata.mutated` (`ClusterMetadataMutationPayload`,
`@objectstack/metadata-protocol`) or `datasource.mutated`
(`ClusterDatasourceMutationPayload`, `@objectstack/service-datasource`).
- `MetadataChangeOperationSchema` / `MetadataChangeOperation` → *(removed)* —
the orphan value schema of the payload's `operation` field; it had no other
consumer.

One-line fix: delete the import — every one of the four names is TS2305 after
upgrade, and no runtime path ever produced or read a value of these types. A
host that used the retired type for a transport of its own keeps a local type.

The retirement kit:

- **whole-def deletion** (route 3 — not an authorable surface: no metadata-type
binding, stack collection or manifest embed ever carried it, and nothing
parsed it outside its own unit test, so there is no authored document to
rewrite and nobody who could receive a parse-time tombstone):
`kernel/MetadataChangedEventPayload` and `kernel/MetadataChangeOperation` in
`RETIRED_DEFS_BY_MAJOR[18]` plus the D3 semantic entry. The payload def was
never in `json-schema.manifest/` (the JSON Schema build skips `bigint`); the
enum was, so the manifest deletion gate adjudicates it against the entry.
- **retirement pins** in `kernel/cluster.test.ts`: runtime namespace probes
assert both names are absent from `kernel/cluster.zod` and from the
`@objectstack/spec/kernel` entry, with `ClusterCapabilityConfigSchema` as
the positive control; the two isomorphic alias pins left with the schemas.
- **docs**: `content/docs/kernel/cluster.mdx` no longer describes a planned
version-stamped payload or a version-compare reader contract — the shipped
address-only lanes and their re-read receipt are the contract; the
`ClusterMetadataChangedPayload` doc-comment in `@objectstack/metadata` no
longer claims to "align with" the retired schema (it never did).
- zero in-tree consumers, so no in-repo source changes ride along; runtime
behaviour is unchanged.
106 changes: 51 additions & 55 deletions content/docs/kernel/cluster.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -296,31 +296,31 @@ they declare scope and strategy, and the runtime does the rest.

Metadata is the most cache-hot data in the system. Every read path —
ObjectQL planning, REST routing, UI rendering, permission checks —
touches it. Making it correct across a cluster requires two things the
current protocol lacks: a **monotonic version per item** and a
**well-defined invalidation event**.
touches it. Making it correct across a cluster needs a well-defined
invalidation signal per state owner — the three lanes of §6.2 — and, in this
protocol, deliberately **no** per-item version on the wire (§6.1).

### 6.1 Monotonic version

A `version` column is already present on every persisted metadata record in
`system/metadata-persistence.zod.ts` (`version: z.number()`, used today for
optimistic concurrency). This ADR proposes formalising it as a monotonic
per-item version and **widening it to `bigint`** so caches can compare
freshness across long-running clusters without overflow.

> **Status: planned.** The column is still `version: number` in the schema;
> the widening and the cache-comparison contract below describe the target
> design, not current runtime behaviour.

Once wired, caches store `{value, version}`. On any incoming change
notification the cache compares the incoming version with the stored one:

- Incoming version `>` stored → invalidate (apply new value or evict)
- Incoming version `≤` stored → ignore (out-of-order notification, already
superseded)

This eliminates a whole class of bugs where a slow-arriving "old"
invalidation evicts a "newer" value the node has already learned about.
optimistic concurrency). Earlier revisions of this page proposed formalising
it as a monotonic per-item version, widening it to `bigint`, and having every
cache compare it on receipt.

> **Status: not declared.** The column is `version: number` in the schema,
> and nothing in the protocol carries a version across nodes. The
> version-compare invalidation this section once described as the target
> design — caches storing `{value, version}` and ignoring a notification whose
> version is not strictly greater than the cached one — went with the retired
> `MetadataChangedEventPayloadSchema` (§6.2, lane 1; ADR-0049
> enforce-or-remove, ADR-0087 entry `metadata-changed-event-payload-retired`).
> The shipped lanes are address-only signals whose receiver re-reads its own
> store, and a re-ordered or duplicated delivery converges to the same state as
> a single one — which is what removes the "old invalidation evicts a newer
> value" class of bug without a version to compare. A version-stamped protocol
> returns only via the ENFORCE route of ADR-0049: the emitter and the reader
> first, the declaration with them.

### 6.2 The cluster-invalidation family

Expand Down Expand Up @@ -394,30 +394,23 @@ the pre-write set (#5109). The registry entry is **deleted, never pre-filled**
from the payload — the peer re-reads the shared store, which is the source of
truth.

At the payload's top level there is still **no** `version` / `name` /
`tenantId` / `operation` field and **no** version comparison; the item's name
is carried only inside the replayed `event`.

**Target spec (planned).** The richer, version-stamped payload below is
defined as `MetadataChangedEventPayloadSchema` in `kernel/cluster.zod.ts`
but is **not yet wired** into the runtime:

```ts
// MetadataChangedEventPayloadSchema — target shape, not yet emitted
{
type: 'object' | 'view' | 'flow' | …,
name: '<machine-name>',
tenantId?: '<tenant>',
version: <new-version>,
operation: 'create' | 'update' | 'delete' | 'publish',
}
```

When wired with `scope: 'cluster'` + `deliverySemantics: 'at-least-once'`
and a `partitionKey` of `` `${type}:${name}` ``, the partition key would
guarantee that two rapid updates to the same item are applied to every
node's cache in order, and the at-least-once guarantee would let a briefly
partitioned node catch up on reconnect.
At the payload's top level there is **no** `version` / `name` / `tenantId`
/ `operation` field and **no** version comparison — by design, not by
omission; the item's name is carried only inside the replayed `event`.

**No richer payload is declared.** The spec once carried a version-stamped
target shape for this lane — `MetadataChangedEventPayloadSchema` in
`kernel/cluster.zod.ts` (`type` / `name` / `tenantId` / `version: bigint` /
`operation` / `correlationId`), whose docblock said every persistence layer
MUST emit it and every reader MUST compare `version` on receipt. Nothing ever
emitted or subscribed to it, and its `bigint` version could not have crossed a
JSON transport without a codec no driver ships, so it was retired under
ADR-0049 enforce-or-remove (ADR-0087 entry
`metadata-changed-event-payload-retired`, `RETIRED_DEFS_BY_MAJOR[18]`,
together with the `MetadataChangeOperationSchema` enum that typed its
`operation` field). The shape above **is** the lane-1 contract, and its
receipt semantics are the family's: drop the cache entry, re-read the shared
store, no version comparison.

#### Lane 2 — `metadata.mutated` (the ObjectQL object registry)

Expand Down Expand Up @@ -485,16 +478,18 @@ all; the admin service publishes on the write doors it already owns.

### 6.3 Reader contract

> **Status: planned.** Today readers are invalidated by the `metadata.changed`
> PubSub fan-out described in §6.2, which replays watch events verbatim
> without version comparison. The contract below is the target design that
> goes with the version-stamped payload.

All metadata readers (registry, loader, query engine) should:
The reader contract is the one §6.2 states for the family, and it is what
ships — there is no version-stamped variant behind it (the payload that
carried one was retired, §6.1). A metadata reader that keeps its own
long-lived cache:

1. Subscribe to the metadata change channel on startup.
2. Compare incoming `version` with cached `version` before evicting.
3. Treat missing `version` as `0` (legacy compatibility).
1. Subscribes to the lane whose state it mirrors — `metadata.changed` for a
mirror of the metadata service's own caches; lanes 2 and 3 already keep
ObjectQL's object and driver registries converged (§7.3).
2. On receipt drops its own `originNode` messages, then evicts — never
pre-fills from the payload.
3. Re-reads the shared store on the next access; a duplicate or re-ordered
delivery converges to the same state as a single one.

## 7. Plugin author guidance

Expand Down Expand Up @@ -572,8 +567,9 @@ cluster.pubsub.subscribe('metadata.changed', (msg) => {
> already keep those registries converged across replicas, and a second
> subscription would be redundant.

A higher-level cache factory (with automatic version comparison) is part of
the Phase 4 / §6 target design but is **not yet implemented**. No
There is no higher-level cache factory and no version comparison to
automate — the version-stamped payload was retired (§6.1) and the lanes are
address-only, so the subscribe-and-evict shape above is the whole contract. No
`if (cluster)` branches are needed either way — the `memory` driver makes
the subscribe call a no-op-equivalent local fan-out on a single node.

Expand Down
10 changes: 5 additions & 5 deletions content/docs/references/index.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Protocol Reference
description: Every schema published by @objectstack/spec — 1593 schemas across 14 protocol modules
description: Every schema published by @objectstack/spec — 1592 schemas across 14 protocol modules
---

{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
Expand All @@ -26,14 +26,14 @@ counts are sums of the rows they head. Regenerate with
| [Data Protocol](/docs/references/data) | 29 | 166 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. |
| [Identity Protocol](/docs/references/identity) | 5 | 27 | Users and accounts, organizations, positions, SCIM provisioning. |
| [Integration Protocol](/docs/references/integration) | 1 | 27 | The single connector protocol (ADR-0097) — catalog descriptors and provider-bound instances. |
| [Kernel Protocol](/docs/references/kernel) | 30 | 163 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. |
| [Kernel Protocol](/docs/references/kernel) | 30 | 162 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. |
| [QA Protocol](/docs/references/qa) | 1 | 8 | Declarative test suites — scenarios, steps, actions and assertions. |
| [Security Protocol](/docs/references/security) | 5 | 29 | Permission sets, row-level security, sharing rules, tenancy posture. |
| [Shared Protocol](/docs/references/shared) | 8 | 26 | Primitives used across every protocol — identifiers, HTTP, expressions, error maps, enums. |
| [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. |
| [System Protocol](/docs/references/system) | 36 | 291 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. |
| [UI Protocol](/docs/references/ui) | 16 | 153 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. |
| **Total** | **200** | **1593** | 14 protocol modules |
| **Total** | **200** | **1592** | 14 protocol modules |

---

Expand Down Expand Up @@ -217,14 +217,14 @@ The single connector protocol (ADR-0097) — catalog descriptors and provider-bo

## Kernel Protocol

**Source:** `packages/spec/src/kernel/` · **Import:** `@objectstack/spec/kernel` · **30 pages, 163 schemas**
**Source:** `packages/spec/src/kernel/` · **Import:** `@objectstack/spec/kernel` · **30 pages, 162 schemas**

Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry.

| File | Schemas |
| :--- | :--- |
| [`cli-extension.zod.ts`](/docs/references/kernel/cli-extension) | `OclifPluginConfig` |
| [`cluster.zod.ts`](/docs/references/kernel/cluster) | `ClusterCapabilityConfig`, `ClusterDriver`, `ClusterTenantIsolation`, `EventClusterOptions`, `EventDeliverySemantics`, `EventScope`, `MetadataChangeOperation`, `ServiceClusterAnnotations`, `ServiceClusterScope`, `ServiceLeaderStrategy` |
| [`cluster.zod.ts`](/docs/references/kernel/cluster) | `ClusterCapabilityConfig`, `ClusterDriver`, `ClusterTenantIsolation`, `EventClusterOptions`, `EventDeliverySemantics`, `EventScope`, `ServiceClusterAnnotations`, `ServiceClusterScope`, `ServiceLeaderStrategy` |
| [`context.zod.ts`](/docs/references/kernel/context) | `KernelContext`, `RuntimeMode`, `TenantRuntimeContext` |
| [`dependency-resolution.zod.ts`](/docs/references/kernel/dependency-resolution) | `DependencyResolutionResult`, `DependencyStatusEnum`, `RequiredAction`, `ResolvedDependency` |
| [`events/bus.zod.ts`](/docs/references/kernel/events-bus) | `EventBusConfig` |
Expand Down
18 changes: 2 additions & 16 deletions content/docs/references/kernel/cluster.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ subsystems depend on.
## TypeScript Usage

```typescript
import { ClusterCapabilityConfigSchema, ClusterDriverSchema, ClusterTenantIsolationSchema, EventClusterOptionsSchema, EventDeliverySemanticsSchema, EventScopeSchema, MetadataChangeOperationSchema, ServiceClusterAnnotationsSchema, ServiceClusterScopeSchema, ServiceLeaderStrategySchema } from '@objectstack/spec/kernel';
import type { ClusterCapabilityConfig, ClusterDriver, ClusterTenantIsolation, EventClusterOptions, EventDeliverySemantics, EventScope, MetadataChangeOperation, ServiceClusterAnnotations, ServiceClusterScope, ServiceLeaderStrategy } from '@objectstack/spec/kernel';
import { ClusterCapabilityConfigSchema, ClusterDriverSchema, ClusterTenantIsolationSchema, EventClusterOptionsSchema, EventDeliverySemanticsSchema, EventScopeSchema, ServiceClusterAnnotationsSchema, ServiceClusterScopeSchema, ServiceLeaderStrategySchema } from '@objectstack/spec/kernel';
import type { ClusterCapabilityConfig, ClusterDriver, ClusterTenantIsolation, EventClusterOptions, EventDeliverySemantics, EventScope, ServiceClusterAnnotations, ServiceClusterScope, ServiceLeaderStrategy } from '@objectstack/spec/kernel';

// Validate data
const result = ClusterCapabilityConfigSchema.parse(data);
Expand Down Expand Up @@ -118,20 +118,6 @@ Where the event must be delivered: local process, whole cluster, or tenant-scope
* `tenant`


---

## MetadataChangeOperation

Persistence operation that triggered the change.

### Allowed Values

* `create`
* `update`
* `delete`
* `publish`


---

## ServiceClusterAnnotations
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ directory rather than per file.
| `cloud/` | 83 |
| `identity/` | 32 |
| `integration/` | 10 |
| `kernel/` | 261 |
| `kernel/` | 260 |
| `qa/` | 6 |
| `shared/` | 20 |
| `system/` | 370 |
12 changes: 10 additions & 2 deletions packages/metadata/src/metadata-manager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -230,8 +230,16 @@ function generateEventUuid(): string {
* Payload format for cluster-wide metadata change broadcasts.
*
* Published on channel `metadata.changed` by any node mutating metadata
* and consumed by peers to invalidate their local caches. Aligns with
* `MetadataChangedEventPayload` in `cluster-semantics.mdx` §5.
* and consumed by peers to invalidate their local caches
* (`content/docs/kernel/cluster.mdx` §6.2, lane 1). This shape IS the whole
* contract: the origin node for loopback suppression, the metadata type, and
* the local watch event replayed verbatim — an address-only signal. There is
* no `version`, `name`, `tenantId` or `operation` at the top level and no
* version comparison on receipt; a peer drops its cache entry and re-reads
* the shared store. The spec once declared a richer, version-stamped
* `metadata:changed` payload that this shape was said to align with; it never
* did, and that schema was retired (ADR-0087 entry
* `metadata-changed-event-payload-retired`).
*/
export interface ClusterMetadataChangedPayload {
/** Origin nodeId — used for loopback suppression. */
Expand Down
4 changes: 0 additions & 4 deletions packages/spec/api-surface/kernel.json
Original file line number Diff line number Diff line change
Expand Up @@ -163,12 +163,8 @@
"MetadataBulkResultSchema (const)",
"MetadataCategory (type)",
"MetadataCategoryEnum (const)",
"MetadataChangeOperation (type)",
"MetadataChangeOperationSchema (const)",
"MetadataChangeType (type)",
"MetadataChangeTypeSchema (const)",
"MetadataChangedEventPayload (type)",
"MetadataChangedEventPayloadSchema (const)",
"MetadataDependency (type)",
"MetadataDependencySchema (const)",
"MetadataDiffItem (type)",
Expand Down
2 changes: 0 additions & 2 deletions packages/spec/declaration-map/kernel.json
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,6 @@
"MetadataBulkResult": "kernel/MetadataBulkResult",
"MetadataBulkResultSchema": "kernel/MetadataBulkResult",
"MetadataCategoryEnum": "kernel/MetadataCategoryEnum",
"MetadataChangeOperation": "kernel/MetadataChangeOperation",
"MetadataChangeOperationSchema": "kernel/MetadataChangeOperation",
"MetadataChangeType": "kernel/MetadataChangeType",
"MetadataChangeTypeSchema": "kernel/MetadataChangeType",
"MetadataDependency": "kernel/MetadataDependency",
Expand Down
4 changes: 0 additions & 4 deletions packages/spec/export-origins/kernel.json
Original file line number Diff line number Diff line change
Expand Up @@ -163,12 +163,8 @@
"MetadataBulkResultSchema": "src/kernel/metadata-plugin.zod.ts#MetadataBulkResultSchema (const)",
"MetadataCategory": "src/kernel/package-artifact.zod.ts#MetadataCategory (type)",
"MetadataCategoryEnum": "src/kernel/package-artifact.zod.ts#MetadataCategoryEnum (const)",
"MetadataChangeOperation": "src/kernel/cluster.zod.ts#MetadataChangeOperation (type)",
"MetadataChangeOperationSchema": "src/kernel/cluster.zod.ts#MetadataChangeOperationSchema (const)",
"MetadataChangeType": "src/kernel/package-upgrade.zod.ts#MetadataChangeType (type)",
"MetadataChangeTypeSchema": "src/kernel/package-upgrade.zod.ts#MetadataChangeTypeSchema (const)",
"MetadataChangedEventPayload": "src/kernel/cluster.zod.ts#MetadataChangedEventPayload (type)",
"MetadataChangedEventPayloadSchema": "src/kernel/cluster.zod.ts#MetadataChangedEventPayloadSchema (const)",
"MetadataDependency": "src/kernel/metadata-plugin.zod.ts#MetadataDependency (type)",
"MetadataDependencySchema": "src/kernel/metadata-plugin.zod.ts#MetadataDependencySchema (const)",
"MetadataDiffItem": "src/kernel/package-upgrade.zod.ts#MetadataDiffItem (type)",
Expand Down
Loading
Loading