Issue 1 sparse zero write lifecycle

Cargo.toml

@@ -11,9 +11,11 @@ b-table = { path = "../b-table", features = ["stream"] }
 destream = "0.10"
 freqfs = "0.12"
 ha-ndarray = "0.5"
+futures = "0.3"
 rand = "0.8"
 safecast = "0.2"
 smallvec = "1"
 
 [dev-dependencies]
 tokio = { version = "1", features = ["macros", "rt-multi-thread", "fs"] }
+tbon = "0.9"

README.md

@@ -8,3 +8,10 @@ A filesystem-backed `Tensor` data structure featuring support for dense and spar
 - `fensor` does not attempt to repair, recover, or auto-heal corrupted metadata or tensor data.
 - If metadata or data is malformed, inconsistent, or unreadable, operations must return a structured error with a clear message.
 - Recovery workflows (restore/rebuild/migration) are external operational concerns, not `fensor` runtime behavior.
+
+## Serializing a tensor
+
+`Tensor<FE, T>` has two, independent wire-format surfaces:
+
+- **Schema-only (`Tensor: ToStream`/`FromStream`/`IntoStream`)**: encodes just the `TensorSchema` (dtype/shape/layout). Only a base (identity-view) tensor can be encoded this way; encoding a transformed (sliced/transposed/reshaped) view is rejected, since views are metadata-only and never persisted. Decoding always builds a fresh, empty base tensor at the given directory via `Tensor::create` — no element data is carried.
+- **View + data streaming (`Tensor::view_encoder` / `TensorViewDecoder`)**: `tensor.view_encoder()` returns a `TensorViewEncoder<'_, FE, T>` that streams the tensor's *current* view — identity or transformed, dense or sparse — directly to the wire via `destream`'s `ToStream`/`IntoStream` contract. The encoder lazily reads from the tensor's filesystem-backed storage and emits values one at a time, with no full in-memory buffering; only non-default (nonzero) values are transmitted, reducing network traffic for sparse-heavy or mostly-empty tensors. On the receiving end, `TensorViewDecoder<FE, T>` implements `destream`'s `FromStream` and writes each arriving value directly to a fresh, independent, identity base tensor's filesystem storage as it arrives off the wire — also with no full in-memory buffering. There is no trailer or checksum on this wire format: a successful transfer is signaled by natural exhaustion of the pairs sequence, and end-to-end transfer completeness/integrity is left to the transport/caller layer rather than re-implemented here. A read failure on the sending side propagates as a bounded error-code sentinel — a closed classification of which failure shape occurred, deliberately excluding free-text detail (filesystem paths, coordinates, or other sender-machine specifics) since this wire format is meant to cross a machine boundary. On that sentinel, a malformed/truncated stream, or any other decode-time failure after the destination storage is created, the directory is truncated and deleted before a fail-closed error is returned. Call `.into_inner()` on the decoder to extract the reconstructed tensor. Like the design it replaces, this produces a fresh, independent identity base tensor with no link back to the source storage; it remains a distinct, additive wire surface alongside the existing schema-only `Tensor: ToStream/FromStream/IntoStream` contract, which is completely unrelated and still only carries dtype/shape/layout metadata without element data.

ROADMAP.md

@@ -79,14 +79,15 @@ Transactional orchestration belongs to `tc-collection`, which composes `fensor`
    - Gate: positive and negative tests cover arbitrary permutation + slicing combinations.
 
 2. **Sparse zero-write lifecycle contract.**
-   - Specify behavior for writing zero into sparse coordinates: row removal, tombstoning, or retained zero rows.
+   - Specify behavior for writing zero into sparse coordinates as row removal.
    - Ensure behavior is deterministic and documented; no silent policy drift.
    - Gate: tests assert index/block state transitions for nonzero->zero and zero->nonzero updates.
 
 3. **Persistence and rehydration semantics.**
    - Validate metadata and block persistence across real `freqfs` reload boundaries.
    - Corruption and malformed metadata must fail closed with structured errors.
    - Gate: integration tests cover create->write->reload->read and corruption->error flows.
+   - Implemented: `Tensor::view_encoder()` and `TensorViewDecoder` provide a genuinely streaming serialization for a tensor's current view (identity or transformed, dense or sparse), with no full in-memory buffering on either side. The encoder lazily reads values from filesystem storage and emits only non-default payloads to the wire; the decoder writes arriving values directly to fresh independent base tensor storage. There is no trailer or checksum: success is natural exhaustion of the pairs sequence, and end-to-end transfer completeness/integrity is a transport/caller concern rather than something this wire format re-verifies itself. A sender-side read failure propagates as a bounded error-code sentinel (a closed classification of the failure shape, with no free-text/sender-internal detail such as filesystem paths); that sentinel, a malformed/truncated stream, or any other decode-time error triggers fail-closed behaviour with clear error propagation. Reconstruction always produces a fresh, independent identity base tensor with no link back to the source storage. This is a distinct, additive wire surface alongside the existing schema-only `Tensor: ToStream/FromStream` contract, not a replacement for it; it does not by itself satisfy the "Phase 5: Conversion and materialization" `Dense <-> Sparse` conversion exit criteria below, since it always preserves the source's layout category.
 
 4. **Base/view write-through semantics.**
    - Define which transformed tensors are writable and which are read-only.
@@ -155,6 +156,7 @@ Transactional orchestration belongs to `tc-collection`, which composes `fensor`
 - **Adaptive block sizing.** Evaluate configurable or data-driven block sizing after baseline persistence semantics are stable.
 - **Typed tensor families.** Expand beyond `f32` once core lifecycle and sparse-index behavior are validated.
 - **Cross-host sharding hooks.** Keep routing/sharding orchestration in client libraries while exposing reusable shard-local primitives in `fensor`.
+- **Encode-side enumeration optimization.** Encode-side view streaming currently walks every logical coordinate and filters defaults at the codec layer. A future optimization could use sparse-index row enumeration and filesystem directory listing to skip known-empty storage regions entirely, reducing I/O cost for extremely sparse large tensors — but this requires solving how to invert arbitrary view transforms (some non-closed-form) back to base storage coordinates, which is out of scope for now.
 
 ## ha-ndarray execution dependency acknowledgement
 

src/error.rs

@@ -15,6 +15,7 @@ pub enum Error {
         hint: String,
     },
     Unsupported(String),
+    DataMismatch(String),
 }
 
 pub type Result<T> = std::result::Result<T, Error>;
@@ -40,7 +41,8 @@ impl fmt::Display for Error {
             | Self::InvalidCoord(cause)
             | Self::InvalidLayout(cause)
             | Self::SparseIndex(cause)
-            | Self::Unsupported(cause) => f.write_str(cause),
+            | Self::Unsupported(cause)
+            | Self::DataMismatch(cause) => f.write_str(cause),
             Self::UnsupportedSparseIterationOrder {
                 requested_order,
                 base_order,