demo: 07_minimal on the clean moczarr/mortie Moc/Toc API (issue #328) - #492
demo: 07_minimal on the clean moczarr/mortie Moc/Toc API (issue #328)#492espg wants to merge 1 commit into
Conversation
|
🤖 from Claude CI is red for a pre-existing reason, unrelated to this PR (which touches one notebook and no
Cause: from mortie.toc import Q_END_NS, Q_START_NSand mortie 0.9.10 retired the Fix is one line — the constants are exported at top level: from mortie import Q_END_NS, Q_START_NSVerified against mortie 0.9.10:
|
|
this isn't matching the minimal example / terse notebook |
Refs #328.
Rewrites
demo/07_minimal.ipynbagainst the clean reader API that landed today: mortie 0.9.10'sMoc/Toc(espg/mortie#197, #199) and moczarr 0.5.0'scoverage_moc/coverage_toc/candidate_leaves(..., when=)(espg/moczarr#48).Landed as
demo/07_minimal_clean.ipynb, not in place.demo/07_minimal.ipynbis untracked in the working tree anddemo/07_minimal-Copy1.ipynb(a byte-identical Jupyter duplicate) was written at 08:39 this morning, minutes before this run started — the notebook looked live in a Jupyter session, and a notebook open in a kernel owns its file. Reconcile and rename at your convenience; the old file is untouched.What the coverage step collapses to
The old
coverage()helper hand-rolledload_root_coverage+root_coverage_and+candidate_leaves+ a manual shard-id set intersection. It is now four lines:with
q = moc(aoi)built once above it.MocandTocride the__morton_moc__()/__toc_words__()protocols straight into moczarr — no adapter, no words-shuffling. The 3-D view also lost ~25 lines of hand-rolled morton bit decoding tomortie.mort2geo+mortie.clip2order, and the 1850-epoch time arithmetic tomortie.to_datetime64.Two regions, not one store plus a parked constant
The GEDI half was verified rather than assumed, and what is actually on
sliderule-publicchanged the design:california(default)…/englacial/zagg/demo/atl03_tdigest_o9.zarr19/h_tdigest_signalsercsliderule-public/zagg-demo/serc_tdigest_strata.zarr19/h_tdigest_signal+…/serc_gedi_flux.zarr18/rx_fluxserc_gedi_flux.zarrandserc_tdigest_strata.zarrboth exist, both cover the same four o9 shards (4331422233,4331422234,4331422411,4331422412), and all four leaves are present in both — so the paired two-sensor view is real today, in Maryland, behind credentials. They are a different geography from the California store, so each region carries its own polygon and the notebook says that plainly rather than implying a CA pair that does not exist.REGIONis the whole switch.Phases
coverage_moc/moc()/contains/candidate_leaves, generalized over the region's sensors so one and two sensors read the same.Nonearm handled explicitly —coverage_tocreturnsToc | None;Noneis printed as "publishes no temporal coverage,when=prunes nothing" with the §10 unlisted-is-unknown reason, never anAttributeError.open_region()probes reachability and, on failure, prints what is wrong and how to fix it, then raises a one-lineRuntimeErrorinstead of an obstore stack.mort2geo/clip2order/to_datetime64; export usessubtree=so it fetches only the tile on screen.How tested
Metadata-only against the live stores (NO EGRESS: coverage sidecars, manifests,
zarr.json— kilobytes; no payload arrays, no shard loops):california, live and anonymous:Moc(326 cells, orders 5-9, finest 12.734 km), 2,726 o9 shards, Yosemite box → 4 shards,coverage_toc→None.serc, live and signed: both storesMoc(4 cells, order 9), SERC box → 4 shards each, intersection 4,coverage_toc→Noneon both.sercwithout credentials: prints the explanation, raisesRuntimeError: region 'serc' unreachable from this kernel. Confirms the negative that shapes the design — unsignedListObjectsV2onsliderule-publicisAccessDenied, while the same call onus-west-2.opendata.source.coopsucceeds.load(),local_xy()andexport()were run to completion offline against moczarr's in-tree fixtures (tests/data/strata_hive, located+untimed;tests/data/spec/temporal, located+timed), exercising both companion-channel branches and producing a real(2, 2, 16)tensor. That covers every line of the read path except the live bulk read itself.ruff check/ruff format --checkclean on the notebook;nbformat.validatepasses.The notebook ships unexecuted, deliberately. Its remaining cells need either a bulk read (egress from a laptop) or cryocloud credentials, and fabricating outputs for them would be worse than leaving them empty. Executing and saving is an operator run.
Questions for review
§4 binder contract vs the paired view — flagged, not resolved. §4 requires notebooks to be binder-runnable on anonymously-reachable data.
californiasatisfies it end to end.serccannot:sliderule-publicrefuses unsigned reads, so the only existing paired store set is cryocloud-only. The notebook handles this by defaulting to the compliant region, documenting the split in a table up front, and failing the other region with an explanation rather than a traceback — but that is a mitigation, not compliance. The clean fix is a California GEDI store onsource.coop(the ~$50 fleet run), which would make one region both paired and binder-clean and letsercbe deleted. Want me to file that as its own issue?The export cell needs
moczarr[zagg], so "no zagg" is not quite true.moczarr.hhdc.read_tensorsimports zagg's t-digest algebra rather than vendoring it (moczarr's ownzaggextra, espg/moczarr#19), so the last cell raisesImportError: HHDC rasterization needs zagg's t-digest algebraon a plainpip install moczarr. The install cell now says: base line for everything through the 3-D view,%pip install "moczarr[zagg]"for the final cell only. Alternative is dropping the export section to keep the notebook strictly reader-only — say the word.containsvsintersects. The ruled snippet assertscover.contains(q), which is what shipped. It is False for a polygon that pokes over the coverage edge even though the overlap is real and readable, so an edge-of-California AOI raises rather than returning the partial roster. The prose points atintersectsfor that case. Fine as the headline, or should the demo model the partial-overlap path instead?API friction worth passing upstream (no action needed here):
candidate_leavestakesstore=but no**store_kwargs, unlikeread_manifest/load_root_coverage/open_leaf. An anonymous store therefore cannot use the bare ruled spelling — it needs an explicitopen_object_storefirst.open_leaf(root, shard, store=st)shares the handle only for the manifest GET; the leaf store is always a fresh open, soanonymous=Truehas to be passed again alongsidestore=. Easy to get wrong silently on a public bucket (the first draft of this notebook did).mortie.clip2order(clip_order, words)takes its parameter before its data, the reverse of every neighbouring kernel.coverage_tocisNoneon all three verified stores and theTocbranch of the demo has never run against real data. Worth a fixture-backed producer before the temporal story is demoed as working.