feat: Release Intelligence — tracking contracts gate release comparisons

.github/workflows/ci.yml

@@ -2,7 +2,7 @@ name: CI
 
 on:
   push:
-    branches: [main, "chore/**"]
+    branches: [main, "chore/**", "feat/**"]
   pull_request:
 
 jobs:
@@ -25,8 +25,10 @@ jobs:
           pip install -r requirements.txt
       - name: Ruff
         run: ruff check app tests
-      - name: Pytest
+      - name: Unit + integration tests
         run: pytest -q
+      - name: Golden scenarios (G1–G6)
+        run: pytest -q tests/test_instrumentation.py tests/test_release_compare.py -k "g1 or g2 or g3 or g4 or g5 or g6"
 
   web:
     name: Web (lint + typecheck + build)
@@ -51,3 +53,29 @@ jobs:
         run: npm run build
         env:
           GITHUB_PAGES: "false"
+
+  release-state:
+    name: Release state (snapshot + secret scan)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install API deps
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r apps/api/requirements.txt
+      - name: Regenerate snapshot and require byte-identical output
+        run: python scripts/generate_release_snapshot.py && git diff --exit-code -- apps/web/public/data/release-intelligence.json apps/web/lib/release-intelligence.json
+      - name: Snapshot payload sanity
+        run: python -c "import json;d=json.load(open('apps/web/public/data/release-intelligence.json'));assert d['baseline']=='v2.2.0-healthy';assert len(d['releases'])==3;assert 'instrumentation' in d and 'journey_diffs' in d"
+      - name: Secret scan (current tree)
+        shell: bash
+        run: |
+          ! grep -rInE 'AKIA[0-9A-Z]{16}|ghp_[A-Za-z0-9]{36}|gho_[A-Za-z0-9]{36}|github_pat_[A-Za-z0-9_]{22,}|sk-[A-Za-z0-9_-]{20,}|-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----' \
+            --exclude-dir=node_modules --exclude-dir=.venv --exclude-dir=out --exclude-dir=.next .
+      - name: PII scan on fixtures and docs
+        shell: bash
+        run: |
+          ! grep -rInE '[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}' data/seed/releases docs/RELEASE_INTELLIGENCE_METHOD.md

README.md

@@ -86,6 +86,41 @@ Métricas de vaidade (“quantos se cadastraram?”) escondem o caminho.
 
 ---
 
+## Release Intelligence
+
+Depois de uma release, o funil “melhora”. Mas foi o produto ou a instrumentação?
+
+O lab agora valida **contratos de tracking** antes de comparar releases:
+
+- `data/contracts/events.yml` — contrato versionado por evento (owner, props obrigatórias, ordem, cardinalidade, versão)
+- Validador determinístico: missing props, order violations, duplicate `insert_id`, cardinality, unknown events e event drift
+- Comparação **raw vs trusted** por usuário único com veredito explícito
+- Intervalos de confiança de Wilson; linguagem estritamente observacional
+
+Golden scenario (fixtures sintéticas determinísticas):
+
+```text
+v2.3.0-buggy  raw onboarding_completed  +24.3%  →  trusted −0.9%   ⇒ artefato de instrumentação
+v2.3.0-fixed  activation_completed      +9.1%   →  trusted +9.1%   ⇒ melhora real
+```
+
+Demo: [`/releases`](https://barujafe1.github.io/BehaviorGraph/releases/) · Método: [docs/RELEASE_INTELLIGENCE_METHOD.md](./docs/RELEASE_INTELLIGENCE_METHOD.md)
+
+<table>
+  <tr>
+    <td width="50%">
+      <img src="./assets/screenshots/09-release-verdict-buggy.png" alt="Buggy release verdict" />
+      <br /><sub><strong>v2.3.0-buggy</strong> — raw +24.3% desaparece no trusted (−0.9%): artefato de instrumentação</sub>
+    </td>
+    <td width="50%">
+      <img src="./assets/screenshots/10-release-verdict-fixed.png" alt="Fixed release verdict" />
+      <br /><sub><strong>v2.3.0-fixed</strong> — +9.1% sobrevive ao filtro trusted: melhora real</sub>
+    </td>
+  </tr>
+</table>
+
+---
+
 ## Escopo e limites
 
 - **É:** estúdio lab de behavioral analytics.
@@ -131,6 +166,41 @@ Conversion vanity metrics (“how many signed up?”) hide the path.
 
 ---
 
+## Release Intelligence
+
+After a release, the funnel “improves”. But was it the product — or the instrumentation?
+
+The lab now validates **tracking contracts** before comparing releases:
+
+- `data/contracts/events.yml` — versioned per-event contract (owner, required props, sequence, cardinality, introduced-in)
+- Deterministic validator: missing props, order violations, duplicate `insert_id`, cardinality, unknown events and event drift
+- **Raw vs trusted** comparison on unique users with an explicit verdict
+- Wilson confidence intervals; strictly observational language
+
+Golden scenario (deterministic synthetic fixtures):
+
+```text
+v2.3.0-buggy  raw onboarding_completed  +24.3%  →  trusted −0.9%   ⇒ instrumentation artifact
+v2.3.0-fixed  activation_completed      +9.1%   →  trusted +9.1%   ⇒ real improvement
+```
+
+Demo: [`/releases`](https://barujafe1.github.io/BehaviorGraph/releases/) · Method: [docs/RELEASE_INTELLIGENCE_METHOD.md](./docs/RELEASE_INTELLIGENCE_METHOD.md)
+
+<table>
+  <tr>
+    <td width="50%">
+      <img src="./assets/screenshots/09-release-verdict-buggy.png" alt="Buggy release verdict" />
+      <br /><sub><strong>v2.3.0-buggy</strong> — raw +24.3% vanishes under trusted (−0.9%): instrumentation artifact</sub>
+    </td>
+    <td width="50%">
+      <img src="./assets/screenshots/10-release-verdict-fixed.png" alt="Fixed release verdict" />
+      <br /><sub><strong>v2.3.0-fixed</strong> — +9.1% survives the trusted filter: real improvement</sub>
+    </td>
+  </tr>
+</table>
+
+---
+
 ## Scope and limits
 
 - **Is:** behavioral analytics studio lab.
@@ -303,8 +373,8 @@ More: [docs/TESTING.md](./docs/TESTING.md) · Deploy: [docs/DEPLOYMENT.md](./doc
 ## Roadmap
 
 - **MVP (done):** taxonomy, nested funnel, cohorts, graph, friction, memo, static demo, CI
-- **Phase 2 (in progress):** release intelligence — tracking contracts, instrumentation validator, raw vs trusted funnel comparison per release
-- **Phase 3:** minimal SDK, near-real-time ingest, experiment tags
+- **Phase 2 (done):** release intelligence — tracking contracts, instrumentation validator, raw vs trusted funnel comparison per release
+- **Phase 3 (planned):** minimal SDK, near-real-time ingest, experiment tags
 
 Non-goals: Mixpanel clone, generic metrics dashboard, production tracking.
 

apps/api/app/api/releases.py

@@ -0,0 +1,63 @@
+from dataclasses import asdict
+
+from fastapi import APIRouter, HTTPException
+
+from app.models.release import (
+    InstrumentationReportModel,
+    JourneyDiffModel,
+    ReleaseComparisonModel,
+    ReleasesOverviewModel,
+)
+from app.services.instrumentation import validate_release
+from app.services.journey_diff import diff_journeys
+from app.services.release_compare import build_overview, compare_release
+from app.services.release_fixture import RELEASE_PROFILES
+
+router = APIRouter(tags=["releases"])
+
+
+def _ensure_known_release(release: str) -> None:
+    if release not in RELEASE_PROFILES:
+        raise HTTPException(
+            status_code=404,
+            detail=(
+                f"unknown release '{release}'. Known releases: "
+                + ", ".join(sorted(RELEASE_PROFILES))
+            ),
+        )
+
+
+@router.get("/releases/overview", response_model=ReleasesOverviewModel)
+def releases_overview() -> ReleasesOverviewModel:
+    return ReleasesOverviewModel.model_validate(build_overview())
+
+
+@router.get("/releases/diff", response_model=JourneyDiffModel)
+def journey_diff(
+    baseline: str = "v2.2.0-healthy",
+    target: str = "v2.3.0-buggy",
+    min_support: int = 10,
+) -> JourneyDiffModel:
+    _ensure_known_release(baseline)
+    _ensure_known_release(target)
+    return JourneyDiffModel.model_validate(
+        diff_journeys(baseline, target, min_support=min_support)
+    )
+
+
+@router.get("/releases/{release}/instrumentation", response_model=InstrumentationReportModel)
+def release_instrumentation(release: str) -> InstrumentationReportModel:
+    _ensure_known_release(release)
+    report = validate_release(release)
+    payload = asdict(report)
+    payload["violations"] = [asdict(v) for v in report.violations]
+    payload["unknown_events"] = list(report.unknown_events)
+    return InstrumentationReportModel.model_validate(payload)
+
+
+@router.get("/releases/{release}/comparison", response_model=ReleaseComparisonModel)
+def release_comparison(release: str) -> ReleaseComparisonModel:
+    _ensure_known_release(release)
+    comparison = compare_release(release)
+    payload = asdict(comparison)
+    return ReleaseComparisonModel.model_validate(payload)

apps/api/app/main.py

@@ -1,7 +1,17 @@
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
 
-from app.api import cohorts, demo, events, friction, funnel, health, journeys, segments
+from app.api import (
+    cohorts,
+    demo,
+    events,
+    friction,
+    funnel,
+    health,
+    journeys,
+    releases,
+    segments,
+)
 
 app = FastAPI(
     title="BehaviorGraph API",
@@ -31,3 +41,4 @@
 app.include_router(journeys.router, prefix="/api")
 app.include_router(segments.router, prefix="/api")
 app.include_router(friction.router, prefix="/api")
+app.include_router(releases.router, prefix="/api")

apps/api/app/models/release.py

@@ -0,0 +1,87 @@
+"""Pydantic response models for release intelligence endpoints."""
+
+from __future__ import annotations
+
+from pydantic import BaseModel, Field
+
+
+class ViolationModel(BaseModel):
+    event_name: str
+    code: str
+    release: str
+    user_count: int = Field(ge=0)
+    evidence_count: int = Field(ge=0)
+    severity: str
+
+
+class InstrumentationReportModel(BaseModel):
+    release: str
+    total_events: int = Field(ge=0)
+    trusted_events: int = Field(ge=0)
+    violations: list[ViolationModel]
+    unknown_events: list[str]
+    status: str
+
+
+class ReleaseFunnelStep(BaseModel):
+    step: str
+    users: int = Field(ge=0)
+    conversion_from_start: float = Field(ge=0, le=1)
+    unique_user_basis: bool
+
+
+class VerdictModel(BaseModel):
+    classification: str
+    raw_looks_better: bool
+    survives_trusted_filter: bool
+    gain_step: str | None
+    raw_gain_pct: float
+    trusted_gain_pct: float
+
+
+class ReleaseComparisonModel(BaseModel):
+    release: str
+    total_events: int = Field(ge=0)
+    trusted_events: int = Field(ge=0)
+    raw_funnel: list[ReleaseFunnelStep]
+    trusted_funnel: list[ReleaseFunnelStep]
+    activation_raw_rate: float = Field(ge=0, le=1)
+    activation_trusted_rate: float = Field(ge=0, le=1)
+    verdict: VerdictModel
+
+
+class OverviewReleaseModel(BaseModel):
+    release: str
+    total_events: int = Field(ge=0)
+    trusted_events: int = Field(ge=0)
+    activation_raw_rate: float = Field(ge=0, le=1)
+    activation_trusted_rate: float = Field(ge=0, le=1)
+    activation_trusted_ci: tuple[float, float]
+    instrumentation_status: str
+    verdict: VerdictModel
+    raw_funnel: list[ReleaseFunnelStep]
+    trusted_funnel: list[ReleaseFunnelStep]
+
+
+class ReleasesOverviewModel(BaseModel):
+    method: str
+    baseline: str
+    observational_notice: str
+    releases: list[OverviewReleaseModel]
+
+
+class JourneyEdgeDiff(BaseModel):
+    source: str
+    target: str
+    target_weight: int | None = Field(default=None, ge=0)
+    baseline_weight: int | None = Field(default=None, ge=0)
+
+
+class JourneyDiffModel(BaseModel):
+    baseline: str
+    target: str
+    min_support: int = Field(ge=1)
+    baseline_edges: list[list[str]]
+    added: list[JourneyEdgeDiff]
+    removed: list[JourneyEdgeDiff]
+    notice: str