Skip to content

[P0] Publish the AgentMesh Reliability Model and chaos benchmark #138

Description

@0YHR0

Problem

AgentMesh has stronger failure semantics than typical Agent frameworks, but they remain mostly
architecture claims and unit/integration tests. Users need reproducible evidence that the system
converges after realistic failures.

Proposal

Publish AgentMesh Reliability Model v1 and a runnable agentmesh-chaos benchmark. Start with a
small deterministic workload and failure matrix, then report correctness and recovery metrics.

Required initial injections:

  • duplicate Outbox/Redis delivery;
  • worker death before and after business commit;
  • lease expiry and fenced stale completion;
  • Redis interruption and recovery;
  • checkpoint/business-state disagreement;
  • MCP write succeeds but response is lost;
  • approval/reconciliation process restart.

Acceptance criteria

  • The reliability model documents invariants, crash windows, authoritative state, and convergence.
  • Benchmark runs from one command in CI or a dedicated Compose profile.
  • Every scenario asserts lost tasks, duplicate irreversible effects, stuck nonterminal state, and
    invariant violations—not only process exit status.
  • Results include recovery latency, throughput, outbox lag, duplicate suppression, and unresolved
    outcome count in machine-readable JSON plus a human report.
  • A checked-in baseline is tied to commit, environment, and scenario versions.
  • CI runs a bounded smoke matrix; a larger repeatable profile is documented.
  • Existing rollout proposal [Proposal] Add first-class rollout groups for isolated, concurrent, verifiable agent execution #26 reuses the same isolation/evaluator/failure evidence where relevant.

Non-goals

  • Marketing “production ready” from one laptop result.
  • PostgreSQL HA certification in the first increment.
  • Model-quality benchmarking or RL training.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:reliabilityDurability, convergence, chaos, and benchmarksenhancementNew feature or requestpriority:P0Must land before the next product milestone

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions