Skip to content

[DESIGN]: Advisor evaluation and promotion gate (offline replay + shadow A/B proving a change beats the static baseline before it acts) #154

@ELares

Description

@ELares

Filed from the IronCache pre-implementation coverage audit (2026-06-13): no existing issue adequately owned this.

Why this is needed

The research states as a hard rule that any advisor change must beat the tuned W-TinyLFU+SIEVE baseline on replayed traces before it changes anything. That promotion gate (offline replay + shadow-mode A/B against the live static baseline, with a quantified acceptance margin and no-regression sign-off) is owned by no issue and is the continuous-evaluation contract that turns the one-time headroom study and the offline oracle into a gating pipeline, making the AI-Driven 'no regression below baseline' target enforceable. #90 is a one-time go/no-go headroom study; #91 does runtime ROLLBACK after a regression is observed live, not pre-promotion validation; #93 builds an offline oracle TOOL but defines no acceptance/promotion protocol. Distinct from the advisor audit-trail gap (that records what happened; this decides what is allowed to happen).

Context

Relates to / partially overlaps #91. Part of the vision EPIC #1.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:aiArea: aiarea:evictionArea: evictionarea:testingArea: testingdesignDesign specification / decision record to be vettedwave:3Readiness wave 3: clustering, AI advisor, tiering, advanced

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions