Let a deployment choose the engine collection layout: one collection per root type, or one shared collection

[ddeboer]

Let a deployment choose how root types map onto engine collections: one collection per root type (today, ADR 9) or one shared collection for all of them. The GraphQL surface – the per-type root fields, …Where inputs, …Facets types, introspection – is identical under both; the choice is storage.

Why a second layout

ADR 9’s reasons for per-type collections are operational, not about search quality, and both are narrower than they read:

• Isolation is a matrix. The writer is per dataset (write / flush(dataset, outcome) / reset, ADR 6), with one commit() per run. Blue/green isolates per collection – one alias swap each; In-place isolates per dataset – the outcome-gated sweep and the source stamp – regardless of how many collections there are. A shared collection keeps dataset isolation entirely and gives up only per-type alias independence, which an In-place deployment does not use.
• Per-type collection settings are unused. The adapter exposes three (defaultLocale, defaultSortingField, synonymSets). LOL, with nine root types, sets none of them per type; every type shares LOCALES = ['nl', 'en', 'und'], and Move defaultLocale into the SearchSchema so image deployments can set it #690 moves defaultLocale into the SearchSchema – one value per schema, explicitly not per type. Everything that does vary (weights, sortable, facetable, label sources) is per field, and per-field configuration is untouched by layout.

What a shared collection gains:

• One document per IRI. A node that is both person and place (limburg/lol#164) is one document with _type: [Person, Place], counted in both type facets, instead of two documents in two collections.
• Type-specific facets self-scope. Typesense counts only documents that carry the field, so birthPlace counts persons and material counts works with no extra logic.
• Native cross-type ranking and one total – the storage Object/global search across multiple types (layer a SearchResultItem interface over the concrete per-type API) #530’s search root field needs; with per-type collections it is a multi_search merge whose _text_match scores are not comparable across schemas.
• One alias, one rebuild, one lock.

Proposal

A single option on the search pipeline / engine adapter configuration:

layout: 'per-type' | 'union'   // default 'per-type' – today’s behaviour, unchanged

Under 'union':

• Writer: one collection; every document carries a multi-valued _type stamp (the searchType the projection already mints, ADR 9). The per-dataset flush / sweep are unchanged.
• Query compiler: each per-type root field prepends _type:=‹Type› to filter_by. Nothing else in the compiled query changes.
• Label resolution (ADR 8): a same-collection lookup by IRI filtered by _type; label-source types may stay in their own collections if a deployment prefers – the choice is per reference already.
• Collection definition: the union of the types’ field definitions; defaultSortingField is omitted (a field present on every type is required for it, and no deployment sets one).
• Validation: one cross-type rule – the same field name must have the same engine type and semantics across all member types. This is Reject a declared field name that collides with a physical companion field #766’s companion-collision rule widened to the type set; a profile-derived schema satisfies it almost by construction, since schema.org property names mean the same on every class.
• Blue/green: one alias for everything, so one swap; a failed type holds up the run (mitigated by last-known-good per-source caching).

Under 'per-type' nothing changes. ADR 9 already places the fan-out in engine-agnostic pipeline glue over N single-collection writers, so the number of physical collections is an adapter concern; this makes it a declared one.

Deployment guidance

• LOL: small, profile-derived, In-place, nine types with identical settings – 'union'.
• Dataset Register: label-source collections, blue/green, independent rebuilds – 'per-type'.

One doctrinal point to state in the Stack docs: “one AP per projection … multi-AP search = parallel projections, not a wider SHACL” (layers/platform.md) is the per-type stance. A union collection is a second collection derived from the per-type projections, not a wider shape, and should be described that way.

Relation to other issues

• Object/global search across multiple types (layer a SearchResultItem interface over the concrete per-type API) #530 – the cross-type search root field; this is its storage option (a).
• Reject a declared field name that collides with a physical companion field #766 – field-name collisions; the validation rule above generalises it.
• Move defaultLocale into the SearchSchema so image deployments can set it #690 – defaultLocale in the SearchSchema; consistent with settings being schema-wide.
• limburg/lol#164 – the double-typed GeoNames node this resolves.

Context

From the review of the NDE generic API specification against LDE and LOL, where a mixed-type curated collection needs cross-type search and the cost of the shared layout turned out to be smaller than ADR 9’s framing suggests.