Skip to content

research: validate a provenance-preserving context graph #1427

Description

@Teingi

Feature description

Research and validate a provenance-preserving context graph for relationship retrieval and reasoning.

The graph should be a rebuildable projection over exact Sources and Artifact Revisions, never a second authority for facts or a mandatory database dependency.

Problem and proposed solution

Keyword, vector, hybrid, and reranked retrieval are effective for local similarity, but some questions depend on explicit relationships:

  • which decision depends on a constraint that later changed;
  • which Handoff, Task Outcome, Experience, and Skill share the same evidence;
  • which components, people, incidents, tickets, or documents are connected across time;
  • what downstream context may be affected by revising or retiring one exact asset.

Legacy PowerMem graph issues such as #1002 and #1004 show why a new PowerContext design must start from scope isolation, exact provenance, idempotency, and migration rather than reuse the old graph schema.

The research should define and prototype:

  • versioned entity and relation identities with exact Source/Artifact evidence refs;
  • temporal validity, contradictory claims, supersession, and uncertainty;
  • scope and resource authorization before traversal;
  • deterministic relations already present in PowerContext lineage before model-extracted relations;
  • bounded extraction, entity resolution, deduplication, and abstention for model-derived edges;
  • projection rebuild, drift detection, deletion/erasure handling, and backend portability;
  • a provider-neutral graph query port that can run on a simple relational projection before requiring a graph database;
  • hybrid retrieval that returns exact authoritative citations and explains which relation path contributed.

Start with one discriminative use case, such as decision-impact analysis or cross-artifact evidence tracing, and compare it against the current hybrid/rerank baseline.

Acceptance criteria

  • Every returned entity, edge, and reasoning path is backed by exact evidence or explicitly marked derived/uncertain.
  • Cross-scope and unauthorized entities cannot become search seeds, traversal nodes, counts, or error details.
  • Replaying the same evidence is idempotent, and changing extraction configuration produces a distinguishable projection version.
  • Contradictory or superseded relations remain representable; the graph does not silently choose one as current truth.
  • Removing or erasing evidence invalidates/rebuilds affected projections without redirecting citations to another Revision.
  • A bounded evaluation reports task success, citation/path correctness, leakage tests, latency, storage, extraction cost, freshness, and abstention against the current baseline.
  • No public graph API or default dependency ships until the research shows a measurable benefit and an RFC accepts the contract.
  • The first implementation can use SQLite/OceanBase tables or another replaceable backend; a graph database is not mandatory.

Alternatives considered

  • Reintroduce the legacy PowerMem graph store: its identities and isolation model predate current PowerContext Sources, Artifact Revisions, and authorization boundaries.
  • Store model-extracted graph facts as authority: model output may be incomplete or wrong and must remain a projection over evidence.
  • Adopt a graph database first: selects infrastructure before proving a product use case or retrieval gain.
  • Infer relations at query time only: avoids storage but makes results expensive, non-reproducible, and difficult to audit.

Additional context

Related: #1263 for evaluation infrastructure, #1356 for explainability, #1395 for authorization, and the existing Source/Artifact lineage model.

  • I am willing to contribute code, docs, or design feedback.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestevaluationEvaluation, experiments, and retrieval quality

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions