feat(zarr-indexing): public part lowering, projection-carrying part views, result_into, asyncio example

packages/zarr-indexing/changes/292.bugfix.md

@@ -0,0 +1,14 @@
+Resolving a partition's view on its own (`part.view.result()`) now hands the
+reader the same `ReadContext` the parent's partitioned `result()` passes: the
+paired `ChunkProjection` now rides along instead of arriving as `projection=None`.
+A custom reader keyed on `projection.chunk_coords` — a decoded-chunk cache —
+now behaves identically on both paths, which the dask example's
+task-per-partition pattern relies on. `with_reader` and repartitioning keep
+the pairing; a further `.lazy` selection describes a different read and drops
+it. So does a further partitioning of a part, whose cells are counted in the
+part's own box and would name the wrong chunk of the source — such a read
+pairs with no projection, as it did before, rather than with a misleading one.
+
+Reusing a plan (`view.result(parts=view.parts())`) on an unpartitioned view
+now reads through the same context as the plain call, instead of through a
+freshly synthesized whole-base projection.

packages/zarr-indexing/changes/292.doc.md

@@ -0,0 +1,7 @@
+Added an asyncio integration example (`examples/lazy_indexing_asyncio/`),
+peer to the Dask example: `parts()` driven by `asyncio.gather` against
+`zarr.AsyncArray`, using `Partition.source_selection` for per-part fetches, a
+decoded-chunk cache keyed on each cell's `chunk_domain` origin and placed with
+`chunk_local_selection`, and an ascending-cover fallback for queries, new
+axes, and negative-step slices that `AsyncArray.getitem` cannot take directly. The
+integrations guide gained a matching "Consumer-owned I/O" section.

packages/zarr-indexing/changes/292.feature.1.md

@@ -0,0 +1,13 @@
+`IndexTransform.decompose()` — the total counterpart of
+`as_basic_selection`: every transform factors into an ascending basic cover
+plus an in-memory residual, `(cover, residual)`, such that resolving the
+residual against `source[cover]` reads exactly the transform's cells (and the
+cover, read as the transform it denotes, composes with the residual back to
+the original). Queries decompose into their bounding interval plus a
+block-local gather, so a consumer can keep even query parts on its own I/O
+path. The AsyncArray adapter also uses this factorization for NumPy selectors
+that Zarr rejects directly. `decompose_unit_step()` is the variant whose cover is contiguous and
+ascending, the factorization `UnitStepReader` reads through; both live where
+the readers now source their own decomposition, so the planned request and the
+executed read cannot drift apart. The one refusal is a negative output
+coordinate, which no cover slice can spell: `NoBasicSelectionError`.

packages/zarr-indexing/changes/292.feature.2.md

@@ -0,0 +1,6 @@
+`as_basic_selection` refusals now raise `NoBasicSelectionError`, a dedicated
+`ValueError` subclass exported at the package root. A consumer can catch it to
+fetch the transform's basic cover and apply its residual without silently
+absorbing a genuine defect in the lowering, which bare `except ValueError`
+would do. Existing catch sites keep working: the subclass is caught by
+`except ValueError` unchanged.

packages/zarr-indexing/changes/292.feature.3.md

@@ -0,0 +1,25 @@
+Box selections and partitions now lower to backend-native basic selections,
+for consumers that plan reads here but fetch through their own I/O layer (an
+async store, an HTTP range endpoint):
+
+- `IndexTransform.as_basic_selection()` converts a box transform to a tuple of
+  integers and slices such that `source[selection]` reads exactly the
+  transform's cells, at exactly its domain shape. A collapsed
+  single-coordinate gather keeps its singleton axis through a length-1 slice;
+  queries, broadcasts, and transposed or repeated axes raise
+  `NoBasicSelectionError` (a `ValueError` subclass)
+  instead of guessing a slab.
+- `Partition.source_selection` is that lowering of a part's global read, so an
+  async consumer can hand compatible selectors directly to its backend.
+  Backends narrower than NumPy normalize the remaining selectors at their
+  integration boundary.
+- `Partition.chunk_local_selection` is the same read relative to
+  `projection.chunk_domain`'s origin, for decoded-chunk caches. The domain
+  names its cell in the source's own coordinates even for a view partitioning
+  a window of it, so the cached cell is the same read either way;
+  `base_coords` counts cells of the partitioned base, so it keys such a cache
+  only together with the grid that produced it.
+
+`ChainedIndexingStateMachine` gained an invariant that runs both documented
+assembly loops literally under plain NumPy semantics, with no reader involved.
+Backend-specific acceptance is covered separately by the asyncio example.

packages/zarr-indexing/changes/292.feature.4.md

@@ -0,0 +1,13 @@
+Added `LazyArray.result_into(out, *, parts=None)`: the non-allocating form of
+`result()`. The caller's writable buffer is validated against the view's shape
+and dtype, filled in place — every cell written exactly once — and returned. A
+view into a larger array qualifies, so a part's block can land directly in its
+final slot, and a `numpy.ma` masked buffer keeps a masked source's mask.
+`result()` itself is unchanged and always allocates.
+
+Validation rejects what `result()`'s own allocation made impossible: a plain
+buffer for a masked source, which would silently present the values beneath
+the mask as data, and a buffer sharing memory with the wrapped array, where
+each part would overwrite cells the parts after it still have to read. It also
+rejects internally overlapping strided buffers, whose logical cells cannot
+hold distinct result values.

packages/zarr-indexing/changes/292.feature.5.md

@@ -0,0 +1,21 @@
+`ChainedIndexingStateMachine` gained two rules and a reader, each covering a
+state it could not previously reach:
+
+- `descend_into_a_part` continues the chain from one part's view and boxes it
+  again. A part's view is documented as resolvable on its own, and it bases
+  its boxes on its own window rather than on the source, so following one
+  reaches the part-of-a-part — where a projection's cell coordinates and its
+  view's transform are counted from different origins.
+- `fabricates_an_axis` draws a basic selection carrying `None`
+  (`newaxis_selections`, also exported), which adds a domain axis no source
+  axis backs.
+- `ProjectionReader` reads each part by fetching the cell `chunk_domain` names
+  and gathering it with `chunk_transform`, the way a decoded-chunk cache does.
+  It joins `basic_reader` as a reader every machine draws from, so a
+  `ReadContext` whose projection describes a different read than its transform
+  is caught by the same NumPy model as everything else — no reader that
+  ignores the projection can see that.
+
+Subclasses inherit all three NumPy-semantic checks. A partitioning declared as explicit per-axis
+sizes describes the source's extents, so it is skipped where a descent has
+narrowed the base it would have to sum to.

packages/zarr-indexing/changes/292.feature.md

@@ -0,0 +1,6 @@
+`BasicSelection` — the public alias for the lowering's output vocabulary,
+`tuple[int | slice | None, ...]`, exported at the package root. The name is
+NumPy's basic-indexing contract. It is wider than
+`zarr.AsyncArray.getitem`'s same-named selection type because NumPy also
+accepts `None` and negative-step slices; the asyncio example narrows or
+normalizes those forms at the backend boundary.

packages/zarr-indexing/docs/examples/lazy_indexing_asyncio.md

@@ -0,0 +1,13 @@
+--8<-- "lazy_indexing_asyncio/README.md"
+
+`LazyArray` owns the indexing-derived plan — which grid cells a view touches,
+what to request from each, and where each block lands — while the consumer's
+event loop owns concurrency and I/O. The projection pair travels with each
+partition, so a cache keyed on each cell's `chunk_domain` and sliced with
+`chunk_local_selection` needs no coordinate arithmetic of its own.
+
+## Source Code
+
+```python
+--8<-- "lazy_indexing_asyncio/lazy_indexing_asyncio.py"
+```

packages/zarr-indexing/docs/guide/integrations.md

@@ -70,6 +70,35 @@ view = LazyArray(source).with_reader(unit_step_reader)
 A strided selection then over-reads its cover by the stride factor, which the
 partitioning above bounds by one part.
 
+### Consumer-owned I/O: parts as backend requests
+
+Both regimes above still read *through* the wrapper. A consumer with its own
+I/O layer — an async store, an HTTP endpoint, a connection pool — can instead
+use the wrapper purely as a planner: every box-shaped part lowers to a
+backend-native basic selection with
+[`Partition.source_selection`][zarr_indexing.lazy_array.Partition], and its
+paired `out_selection` places whatever comes back:
+
+```python
+--8<-- "snippets/integrations.py:consumer-owned-io"
+```
+
+[`Partition.chunk_local_selection`][zarr_indexing.lazy_array.Partition] is the
+same read relative to the part's grid cell, for consumers caching decoded
+chunks. `projection.chunk_domain` locates that cell in the source, so it is
+what to fetch and a sound cache key; `base_coords` counts cells of whatever
+base the view partitions, so it keys a cache only alongside the grid that
+produced it. A query part (an `oindex`/`vindex` gather) has no slab spelling
+and raises `NoBasicSelectionError` (a `ValueError` subclass), so the AsyncArray
+adapter stays on its own I/O path with `part.view.transform.decompose()`. That
+factors the transform into an ascending basic cover to fetch plus a residual
+to resolve in memory. The same path handles NumPy's newaxis and negative-step
+slices, which Zarr's narrower basic-selection dialect rejects even though
+`source_selection` can spell them. The
+[asyncio example](../examples/lazy_indexing_asyncio.md) drives all three
+loops — gather-per-part, decoded-chunk cache, and the query fallback — with
+`asyncio.gather` over `zarr.AsyncArray`.
+
 ## napari-like consumer
 
 This is a **napari-like consumer**, not a napari integration. It models the

packages/zarr-indexing/docs/snippets/integrations.py

@@ -165,3 +165,19 @@ def materialize(view: LazyArray) -> Any:
     for key in slab_source.keys
 )
 # --8<-- [end:dense-box-repartition]
+
+
+# --8<-- [start:consumer-owned-io]
+recorder = RecordingArray(np.arange(100).reshape(10, 10), chunks=(4, 4))
+planner = LazyArray(recorder)
+view = planner.lazy[1:9:2, 4:]
+
+consumer_io = np.arange(100).reshape(10, 10)  # stands in for the consumer's I/O layer
+out = np.empty(view.shape, dtype=view.dtype)
+for part in view.parts():
+    # Each box part lowers to a backend-native basic selection; nothing
+    # reads through the wrapper or its reader.
+    out[part.out_selection] = consumer_io[part.source_selection]
+assert (out == consumer_io[1:9:2, 4:]).all()
+assert recorder.keys == []  # the planner's own source was never read
+# --8<-- [end:consumer-owned-io]

packages/zarr-indexing/examples/lazy_indexing_asyncio/README.md

@@ -0,0 +1,48 @@
+# Lazy Indexing with asyncio
+
+This example demonstrates using `zarr_indexing.LazyArray` as a chunk *planner*
+while an async I/O layer — here `zarr.AsyncArray` — performs every read. The
+package deliberately contains no scheduler; `parts()` exposes the partition
+structure and this example shows `asyncio.gather` driving it.
+
+The example shows how to:
+
+- Lower each compatible box-shaped partition to a backend-native request with
+  `part.source_selection` — a tuple of integers and slices in the wrapped
+  array's own coordinates — and fetch all partitions concurrently, assembling
+  each block with `out[part.out_selection] = await source.getitem(part.source_selection)`
+- Normalize the two NumPy basic selectors that `zarr.AsyncArray.getitem` does
+  not accept: a newaxis (`None`) and a negative-step slice. Those parts fetch
+  their ascending cover through Zarr and apply the residual transform in
+  memory.
+- Build a decoded-chunk cache keyed on each touched grid cell's global origin:
+  fetch the cell (`projection.chunk_domain`) once, then serve every overlapping
+  view from the cache with `part.chunk_local_selection`
+- Fall back for query partitions (`oindex`/`vindex`/mask selections), whose
+  gathers have no single-slab spelling: `source_selection` raises
+  `NoBasicSelectionError`, so the adapter fetches their ascending cover and
+  applies the residual gather in memory
+
+The async side only needs one method — `async def getitem(selection)` accepting
+ascending basic slices — so the same adapter drives an HTTP range endpoint or
+any other async source with Zarr's selection dialect. A backend with a wider
+dialect can take the direct `source_selection` path for more parts.
+
+## Running the Example
+
+The script declares its dependencies inline
+([PEP 723](https://peps.python.org/pep-0723/)), so the easiest way to run it is
+with [uv](https://docs.astral.sh/uv/), which installs them automatically:
+
+```bash
+cd packages/zarr-indexing
+uv run --with-editable . examples/lazy_indexing_asyncio/lazy_indexing_asyncio.py
+```
+
+Alternatively, run it with plain Python, in which case you must first install
+`zarr`, `zarr-indexing`, `numpy`, and `pytest` yourself:
+
+```bash
+cd packages/zarr-indexing
+python examples/lazy_indexing_asyncio/lazy_indexing_asyncio.py
+```