Add graph-native firehose ingest seam

[aaltshuler]

Summary

• add a strict hidden graph-native NDJSON boundary for mixed node and edge declarations
• authorize once at graph scope, pin the accepted catalog and graph witness, and lazily enroll internal Lance MemWAL lanes
• reuse the existing resident driver, recovery healer, node-before-edge ordering, and fold publication path instead of adding another coordinator
• preserve request ordering, stop later enrollment after AckUnknown, and return only redacted logical outcomes
• keep the slice private: no SDK, CLI, HTTP, OpenAPI, storage-format, or recovery-format change

Semantics

Rows are processed in firehose order and are not one atomic graph transaction. Edges may depend on earlier settled nodes; this slice does not buffer a later node as a forward reference.

Validation

• RUST_MIN_STACK=16777216 cargo test -p omnigraph-engine --features failpoints --test memwal_stream --locked -- --nocapture — 120 passed, 1 ignored
• graph-native scenarios — 5 passed
• graph parser unit tests — 6 passed
• stream driver unit tests — 11 passed
• forbidden API tests — 24 passed
• rustfmt check
• git diff check
• scripts/check-agents-md.sh

---

Greptile Summary

The PR adds graph-native NDJSON streaming ingest across the engine, server, client, CLI, and OpenAPI surfaces while reusing the resident MemWAL fold path.

• Adds mixed node/edge parsing, graph-scoped authorization, lazy lane enrollment, and ordered logical outcomes.
• Adds HTTP, remote-client, CLI, documentation, and integration-test coverage for the ingest route.
• Extends resident-driver handoff and graph-dependency settlement behavior.

Confidence Score: 4/5

The PR is not yet safe to merge because an unrelated empty-owner cleanup failure can still abort graph handoff before the required lane is attempted.

The current driver records the unrelated cleanup error more accurately, but cleanup_failed still returns from the finite round before processing the required productive candidate, so the previously reported handoff failure remains reachable.

Files Needing Attention: crates/omnigraph/src/db/omnigraph/stream_driver.rs

Important Files Changed

Filename	Overview
crates/omnigraph/src/db/omnigraph/stream_driver.rs	Adds synchronous graph-route lane settlement and richer handoff accounting, but the previously reported round-wide early return after unrelated cleanup failure remains.
crates/omnigraph/src/db/omnigraph/stream_graph_ingest.rs	Implements graph-native mixed node/edge ingest orchestration over the existing streaming lane machinery.
crates/omnigraph/src/db/omnigraph/stream_ndjson.rs	Adds strict NDJSON declaration parsing and graph-logical outcome serialization.
crates/omnigraph-server/src/handlers.rs	Adds the graph-scoped streaming ingest handler and connects authorization, request handling, and engine dispatch.
crates/omnigraph-cli/src/client.rs	Adds remote-client support for streaming NDJSON requests and ordered response handling.
crates/omnigraph/tests/memwal_stream.rs	Expands integration coverage for graph-native ingest ordering, lane handoff, recovery, and failure behavior.

Sequence Diagram

sequenceDiagram
  participant Client
  participant Server
  participant Engine
  participant Driver
  participant MemWAL
  participant Manifest
  Client->>Server: POST graph stream ingest (NDJSON)
  Server->>Engine: Authorized graph-scoped request
  loop Rows in request order
    Engine->>Driver: Settle dependency or resident handoff
    Driver->>Manifest: Heal and validate authority
    Engine->>MemWAL: Lazily enroll and append row
    Driver->>Manifest: Fold settled generation
    Engine-->>Server: Logical row outcome
  end
  Server-->>Client: Ordered redacted outcomes

Loading

_{Reviews (3): Last reviewed commit: "fix(streaming): close graph ingest drive..." | Re-trigger Greptile}

Context used (3)

• Knowledge Base — OmniGraph core storage: manifest, table store, and Lance access
• Knowledge Base — OmniGraph write path
• Knowledge Base — Server Request Handling

[devin-ai-integration]

Devin Review found 2 potential issues.

View 1 additional finding in Devin Review.