Temporal coverage: packed timestamp|range dual-type time word (design transfer from zagg#410)

[espg]

🤖 from Claude (filed at espg's direction, 2026-08-08 — design transfer from englacial/zagg#410)

What this is

mortie grows from spatial indexing to a coverage library: spatial (HEALPix/morton, existing) plus temporal. The temporal cell type was designed on zagg#410 — see the superseding design comment and the T-MOC research that ruled out the hierarchical-cell alternative — and lands here because:

1. Dependency topology: zagg writes the field, moczarr reads it, and mortie is the dependency both already share.
2. Kernel synergy: a mortie spatial MOC is already, internally, a sorted list of disjoint u64 ranges (the depth-29 linearization in src_rust/src/moc.rs); a temporal coverage is the same object over a 1-D axis, and the range-walk machinery (cf. the Batch MOC set operations: mocs_and / mocs_intersect and the 1xN broadcast family #173 mocs_intersect predicate) generalizes directly.
3. It fits mortie's constraints exactly: pure bit manipulation, zero new dependencies, numpy-only surface.

zagg#410 keeps the consumer side (the per-centroid companion array, its fold law, the config knob, normative spec text + fixtures). This issue is where the type itself gets planned and built.

The type, as decided on zagg#410

One u64, a tagged union of an exact timestamp and a conservative time range:

variant	layout (bit 0 = LSB)
timestamp	bits 63..1 = nanoseconds since epoch (2^63 ns ≈ 292 yr); bit 0 = 0
range	bits 63..32 = start, 32 bits at 2^31 ns (≈2.15 s); bits 31..1 = end, 31 bits at 2^32 ns (≈4.29 s); bit 0 = 1

• Bit split 32/31 — decided (espg, 2026-08-08, on #410): spatial overview folds merge same-pass centroids constantly, so short-span ranges are the bulk of the range population, and their stored width is pure quantization slop — worst case q_start + q_end ≈ 6.4 s under 32/31 vs ≈ 9.7 s under the 33/30 alternative (typical: ~3.2 s vs ~4.8 s). The tighter end bound wins over the finer sort granularity.
• Outward rounding: start floored, end ceiled to their grids — a range is always a conservative superset of its members (the same honesty contract common_ancestor gives the spatial location companion).
• Sort: discriminator in the LSB, shared field high (the zuniq trick). A timestamp's top 32 bits are ns >> 31 — the same 2^31 ns units as a range's start field — so plain unsigned sort orders timestamps and ranges chronologically by start, interleaved, no comparator.
• Merge: min on start codes, max on end codes, timestamps cast outward on first merge. A semilattice join on a fixed epoch-anchored lattice: exactly associative, commutative, idempotent — bit-identical under any fold tree, with error bounded at one quantum per end regardless of fold depth.
• Epoch/timescale — decided: nanoseconds since 1850-01-01T00:00:00 on a continuous, leap-free, GPS-aligned scale (equals GPS time + one fixed constant after 1980; proleptic before). Verified endpoints: CMIP historical starts 1850 (the binding constraint), NCEP/NCAR R1 starts 1948; ceiling ≈ 2142 covers standard ScenarioMIP to 2100. Documented exclusions: CMIP extended-to-2300 scenarios, pre-1850 reanalyses. Leap seconds appear only at the UTC boundary (datetime64/pandas import-export), via the static leap table.
• Rejected alternatives (recorded on #410 with the kill arguments): hierarchical T-MOC-style cell (unbounded merge coarsening, measured), floating-point absolute time (precision inverted — worst on newest data), unit-scale fields and calendar tuples (sort/path-independence/radix-waste failures). Held in reserve: uniform start + float duration (finer everywhere, but ULP-level merge path-dependence).

Open questions

1. Base quantum: 1 ns vs 2 ns. 2 ns doubles the span to 585 yr (1850→2434), covering CMIP extended scenarios, at zero physical cost for ICESat-2 (float64 delta_time carries only ~50 ns at mission magnitudes). Lean: plain ns, exclusions documented; flip only if 2300-scenario output is declared in scope. Must be settled before the epoch constant freezes.
2. Module and naming. A new mortie/temporal.py (mirrored by src_rust/src/temporal.rs)? Names for the type's operations (time_encode/time_merge/…? a TimeWord framing?) — should read as one family with the morton word ops.
3. Phase-1 scope. Scalar encode/decode/merge + vectorized numpy batch forms (encode from datetime64[ns], merge two arrays, reduce an array), UTC↔internal conversion helpers with the leap table. Interval-set coverage algebra (union/intersect over sorted arrays of these) is explicitly deferred — espg parked it pending the scalar type; when it lands it should share the range-walk kernels with the spatial set ops.
4. Conformance surface. Which invariants are normative and fixture-pinned: sort-order property, semilattice laws (associativity/commutativity/idempotence), outward-conservatism, round-trip bounds, the epoch/scale constants.

Sequencing

Blocked on nothing in mortie; gated only on the tail of the zagg#410 discussion (the zagg-side fold law and spec text develop in parallel there). The batch entry points, when they come, land in mortie/batch.py per #170.

Refs: englacial/zagg#410 (design record), englacial/zagg#414 (the compression-factor change the leaf-precision story depends on), #173 (the range-walk kernels this shares), #170 (batch.py).

[espg]

🤖 from Claude

Implementation plan

Scope is phase-1 as scoped in the issue body: the scalar type, its merge law, vectorized array ops, and the timescale boundary. Interval-set coverage algebra stays out (parked). Plan assumes the 1 ns base quantum (the standing lean from zagg#410); that dial must be frozen before phase 1's golden fixtures are captured — it is the one blocking decision, called out in the questions below.

One PR on claude/175-temporal-word, one commit per phase.

Phase 1 — Rust kernel: src_rust/src/temporal.rs (new module, registered in lib.rs)

The layout constants codify the zagg#410 decisions in one place, with the design comment linked from the module docs:

• FLAG (bit 0), the 32/31 split, Q_START = 2^31 ns (≈2.15 s), Q_END = 2^32 ns (≈4.29 s), and the epoch statement (ns since 1850-01-01T00:00:00, continuous GPS-aligned scale).
• encode_timestamp(ns) -> u64, encode_range(start_ns, end_ns) -> u64 (outward rounding: start floored to Q_START, end ceiled to Q_END), decode(word) -> (start_ns, end_ns, is_range), and merge(a, b) -> u64 (min on start codes, max on end codes, timestamps cast outward on first merge).
• cargo property tests as the normative core: semilattice laws (associativity, commutativity, idempotence) over random word pairs/triples; fold-tree independence (random fold trees over the same word set produce the identical u64); outward conservatism (merged range contains every input); the width bound (stored span ≤ real span + Q_START + Q_END); sort monotonicity (ascending u64 order ⇒ non-decreasing decoded starts, timestamps and ranges interleaved); golden u64 fixtures for pinned dates (epoch itself, a known ICESat-2-era instant, a range straddling a quantum boundary).

Phase 2 — pyfunctions + mortie/temporal.py (new)

Array-in/array-out numpy surface over rust_time_* pyfunctions, rayon under allow_threads per the house pattern:

• encode_times(ns_u64) -> u64[N], encode_ranges(start_ns, end_ns) -> u64[N], merge_times(a, b) -> u64[N] (elementwise — zagg's companion fold), reduce_times(words) -> u64 (whole-array reduce), time_bounds(words) -> (start_ns, end_ns), is_range(words) -> bool[N].
• pytest: per-element parity with the Rust scalar ops, the property suite mirrored at array level, empty arrays, dtype/shape validation, fail-fast error style matching the existing modules.

Phase 3 — the timescale boundary

• from_datetime64 / to_datetime64: UTC boundary, using a static in-module leap-second table (data, not a dependency; currently ends at +18 s / 2017-01-01).
• from_gps_ns / to_gps_ns: one documented constant, pure offset arithmetic — the ICESat-2 ingest path (delta_time + atlas_sdp_gps_epoch) composes from this without a mortie-side ATLAS special case.
• Roundtrip tests: datetime64 → word → datetime64 exact for timestamps; GPS conversion exact; behavior at a leap-second boundary pinned by test; pre-1972 proleptic behavior documented in the docstring rather than special-cased.

Phase 4 — docs

• docs/api/temporal.md + mkdocs nav entry, following the per-module pattern; numpydoc on every public function (the numpydoc lint gate is hard); module docstring carries the design rationale link to zagg#410 rather than restating it.

Files touched

src_rust/src/temporal.rs (new), src_rust/src/lib.rs (registration), mortie/temporal.py (new), mortie/__init__.py (exports), mortie/tests/test_temporal.py (new), docs/api/temporal.md (new), mkdocs.yml. No new dependencies; no changes to existing modules. Expected size is #164-shaped: roughly 1,500–2,000 lines across ~8 files, comfortably under the ~1,000-line per-module aim on both new modules.

Acceptance criteria

• Layout constants exactly match the #410 decision record (32/31, LSB flag, outward rounding, 1850 epoch), pinned by golden fixtures.
• The property suite above passes and is treated as normative — it is the conformance surface question (4) of the issue body, answered as: sort monotonicity, semilattice laws, fold-tree independence, conservatism, and the width bound are all fixture-backed.
• Full green gate: maturin develop --release, cargo test / cargo fmt --check / cargo clippy, flake8 mortie --select=E9,F63,F7,F82, pytest -v, numpydoc lint mortie/*.py.

Open questions (one-pass answerable)

1. Freeze the base quantum: 1 ns (lean) or 2 ns? Blocks the golden fixtures; everything else in the plan is quantum-agnostic.
2. Naming: the encode_times / merge_times / reduce_times family in mortie/temporal.py — acceptable, or prefer the x2y house style (time2word / word2time)? Lean the former; the verb family reads better for a type with two variants.
3. Home for the array ops: lean temporal.py (arrays-in/arrays-out is this type's primary API, as in convert.py), reserving batch.py for ragged many-coverage forms when the interval-set algebra eventually lands — with cross-links either way. Confirm, since Resurrect mortie/batch.py as the consolidated home for the bulk operators, with cross-linked scalar/plural docstrings #170 could be read as claiming every plural.
4. Example notebook (binder-ready per repo convention): in this PR or a follow-up issue? Lean follow-up to keep the PR reviewable.

Refs: zagg#410 (design record and decisions), zagg#414 (leaf-precision prerequisite on the consumer side), #173 (range-walk kernels the deferred set algebra will share), #170 (batch.py convention).

[espg]

🤖 from Claude

Decision: base quantum frozen at 1 ns (espg, 2026-08-08) — plan question 1 closed

The golden fixtures in phase 1 of the plan can now be captured. Layout constants as decided: 1 ns base, 32/31 split (start 2^31 ns ≈ 2.15 s, end 2^32 ns ≈ 4.29 s), LSB flag, epoch 1850-01-01 on the continuous GPS-aligned scale.

Why 1 ns — recorded so this is never re-derived from memory

The candidate argument "ICESat-2 digitizes at 1 ns, so the data decides" was checked against the instrument documents and does not hold — but the conclusion survives on different grounds. ATLAS timing is five different numbers depending on the layer, and none of them is 1 ns:

layer	resolution	source
clock quantization (TDC fine time on the 100 MHz USO ruler clock, 10 ns coarse)	~180 ps avg	ATL02 ATBD
event separability (per-channel dead time)	~3 ns	ATL02 ATBD (one-shot pulse length, "minimum time between events on each channel")
single-photon TOF precision (laser pulse ~1.3–1.5 ns FWHM)	~1–1.5 ns RMS	ATL03 ATBD
absolute tie to GPS	via 1 PPS + calibrated USO scale factor	ATL02 ATBD
what the records store (delta_time, float64 s since SDP epoch)	~4–60 ns ULP, growing over mission life	ATL03 data dictionary

The hardware is finer than 1 ns in relative time-of-flight; the physics (pulse width) and the records (float64 absolute epoch time) are coarser — and the record layer is the only one a temporal companion ever ingests. TOF keeps its picosecond chain upstream in ATL02 for ranging; the float64 truncation affects only the absolute event epoch. So ICESat-2 does not discriminate between 1 ns and 2 ns — both sit below the ~4–60 ns the records deliver.

What decides it is the range-slop coupling. The base quantum multiplies every field in the word: a 2 ns base doubles the 32/31 range grids to 4.29 s / 8.59 s, raising short-range slop to ~6.4 s typical / ~12.9 s worst — worse than the 33/30 layout rejected on zagg#410, and spending exactly what the 32/31 decision was made to protect (tight bounds on the abundant single-track spatial-merge ranges). The span 2 ns would buy (CMIP extended-to-2300 scenarios) stays a documented exclusion, as already recorded.

With this, the plan's open questions reduce to (2) naming, (3) array-op home, (4) notebook timing — none blocking phase 1.

[espg]

🤖 from Claude

Two amendments (espg, 2026-08-08) — flag position and naming. Plan questions 2 closed; layout finalized for the golden fixtures.

1. The flag moves from bit 0 to bit 31 — sort tiebreaks become fully semantic

The LSB-flag layout in the issue body and prior comments had a wrinkle: within a tied start quantum, a timestamp's low bits (fine ns-offset) compared against a range's low bits (end code) — deterministic, but semantically void. Relocating the discriminator between the start field and the low field fixes it at zero cost:

[start: 32 bits, 2^31 ns units][flag: 1 bit — 0 = range, 1 = timestamp][low: 31 bits]

• timestamp: low = fine ns-offset within the start quantum (t_ns & (2^31−1)). The word is literally t_ns with the flag bit spliced in at position 31 — still monotone in t_ns.
• range: low = end code (31 bits at 2^32 ns), exactly as before.

Unsigned word order is now order by conservative encoded start with every tiebreak meaningful: (1) start quantum, chronological for all pairs; (2) within a tied quantum, ranges before timestamps — correct, since a range's floored start is ≤ any timestamp inside that quantum ("starts no later"); (3) within type, timestamps exact to the ns, ranges shorter-first by end. Nothing else changes: 1 ns base, 32/31 quanta, 1850 epoch, outward rounding, and the min/max semilattice merge are all untouched. This supersedes every earlier "flag at bit 0 / LSB" statement here and on zagg#410; phase 1's golden fixtures capture this layout.

2. Naming: toc — temporal order coverage

espg's call, and it reads as one family with the morton word ops. Module mortie/toc.py, kernel src_rust/src/toc.rs (supersedes the plan's temporal.py/temporal.rs). The value is a toc word, parallel to "morton word". Concrete spellings in the house x2y style, adjustable in review:

op	name
encode instants (ns / datetime64)	time2toc
encode ranges	span2toc
decode to bounds	toc2time
elementwise merge (the fold)	toc_merge
whole-array reduce	toc_reduce
variant predicate	toc_is_range

Phase-3 conversion spellings (gps2toc / GPS + UTC boundary helpers) settle in review. One naming note for the docs: "toc" pleasantly echoes both tick/tock and T-MOC — the module docstring already disclaims IVOA T-MOC conformance, which keeps the echo from misleading.

Plan question 4, answered while we're here

All of the listed invariants are fixture-pinned — the real content of the question is where the normative line sits, and it is: everything observable in a stored word is normative (the bit layout including flag position and polarity, the epoch/quantum constants, merge results, sort order — since words persist on disk in shards that another reader must interpret identically), while everything about how they are computed is not (parallelization, chunking, error text). Fixtures live in mortie's suite; when the companion lands in zagg, its spec references a conformance subset of the same fixtures rather than duplicating them.

Plan question 3 (interval-set algebra timing) is under active discussion and will be recorded separately.

[espg]

🤖 from Claude

Plan question 3 closed (espg, 2026-08-09) — predicates in, set algebra out

In scope, added to phase 2: the elementwise window predicates —

op	name	shape
word intersects query window	toc_overlaps(words, q_start_ns, q_end_ns)	bool[N]
word contained in query window	toc_contains(words, q_start_ns, q_end_ns)	bool[N]

Both are per-word interval tests against one query window over decode — no coverage object, no normalization semantics — and their consumer shape is already known ("which centroids intersect window W", the first real query against a toc column). Semantics note for implementation: the tests run on the word's conservative encoded bounds (a timestamp is its exact instant; a range is its outward-rounded envelope), so toc_overlaps can over-report near window edges by up to one quantum and never under-reports, and toc_contains correspondingly never over-reports — state this in the docstrings.

Out of scope, split to #177: the interval-set coverage algebra (normalize / union / intersect / minus over arrays of toc words), deferred until the first consumer call site exists. #177 records the three design questions that make deferral the right call — the two-grid adjacency rule, the lossy-timestamp-absorption fork (revisit-set vs store-envelope consumers pull canonical form in opposite directions), and the non-conservativity of difference over conservative covers — plus the #173 call-site-audit precedent.

Where the decision ledger stands (spelled out rather than numbered, since the issue body and the plan numbered their questions differently): decided — 1 ns base quantum, 32/31 split, flag at bit 31, 1850 epoch on the continuous GPS-aligned scale, naming toc (mortie/toc.py / src_rust/src/toc.rs), predicates in / set algebra to #177, and the normative line (everything observable in a stored word). Still open, neither blocking phase 1: whether the array ops live in toc.py (the standing lean — they are the type's primary API, with batch.py reserved for ragged many-coverage forms when #177 activates) or in batch.py per #170's plural convention; and the example notebook, in-PR or follow-up (lean follow-up).

[espg]

🤖 from Claude

Array-op home resolved; label → implement (espg, 2026-08-09)

espg ruled: mirror the moc pattern. Flat-array elementwise ops (time2toc, span2toc, toc2time, toc_merge, toc_reduce, toc_is_range, toc_overlaps, toc_contains) live in mortie/toc.py — the same relationship moc.py has to its ops over one cover, since an array of toc words is this type's scalar surface. mortie/batch.py receives the ragged many-cover plurals when #177 activates, cross-linked both ways per #170. The notebook lands as a follow-up, not in the PR. Every design question is now closed; implementation starts on claude/175-toc-word.