Skip to content

feat: add annotation audit mode and a local CI tool #384

Description

@vamgan

Summary

Add an annotation-quality audit mode and local CI tool that tells developers whether their Askable context is useful, safe, stable, and within budget.

Proposed UX

<AskableInspector audit={{
  sensitiveKeys: ['email', 'token', 'secret'],
  maxElementTokens: 200,
  requireStableIds: true,
}} />
npx askable audit http://localhost:3000   --journey ./askable.journey.ts   --format json   --fail-on error

Diagnostics

  • malformed metadata
  • unstable or non-unique selectors/IDs
  • suspicious sensitive keys and unredacted packet fields
  • empty or flattened/noisy text
  • excessive token cost
  • missing accessible labels
  • important interactive regions without annotations
  • chart/table regions that should use curated data-askable-text

Acceptance criteria

  • Inspector highlights offending elements and recommends concrete fixes
  • CLI emits stable machine-readable diagnostics with severity and selector
  • CI thresholds can fail builds
  • Reports context coverage and token distribution across a journey
  • Runs entirely locally without uploading page content
  • Supports at least React/Vite and Next.js examples initially
  • Integrates privacy-policy findings from bug(mcp): WebContextPacket serialized to agent without verifying redaction was applied #274 rather than implementing competing rules

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions