If only Fable had a RedThread report: launch-readiness preset

[Elesiann]

Context

The recent public dispute around Fable 5 / Mythos 5 highlighted a gap that RedThread is already well-positioned to address: teams do not just need to know whether a jailbreak-like behavior exists. They need a launch artifact that clearly distinguishes broad compromise from narrow, context-specific bypasses, validates whether the behavior reproduces, and separates diagnostic signal from promotable evidence.

RedThread already has the core pieces for this:

• adversarial campaign strategies such as PAIR, TAP, Crescendo, and GS-MCTS
• explicit evidence classes for live judge, sealed/golden, and fallback paths
• defense synthesis with replay validation
• candidate / promotable / active guardrail boundaries
• agentic-security review for tool, delegation, lineage, canary, and budget risk
• reports that distinguish what happened, why to trust it, and what to do next

I think there is a useful productized mode here: a launch-readiness preset that produces a concise executive packet plus the normal detailed campaign artifacts.

Proposal

Add a launch-readiness or frontier-launch-readiness preset for high-risk model, agent, or LLM-app releases.

The goal would not be to certify that a system is safe. The goal would be to produce a structured evidence packet that a security team, executive reviewer, partner, or regulator could read without confusing a narrow finding for universal model compromise or a candidate defense for a production control.

Example CLI shape:

redthread run \
  --preset launch-readiness \
  --target-config ./redthread-target.yaml \
  --risk-domain cyber \
  --report-dir reports/launch-readiness

Possible target config shape:

target:
  name: fable-class-public-model
  adapter: openai-compatible
  model: example-model
  system_prompt_file: ./system-prompt.md

risk_profile:
  domain: cyber
  release_stage: public_launch
  capabilities:
    - codebase_analysis
    - vulnerability_explanation
    - tool_use
  protected_behaviors:
    - exploit_chain_generation
    - credential_exfiltration_guidance
    - unauthorized_tool_invocation

readiness_gates:
  require_live_judge: true
  require_replay: true
  require_benign_utility_replay: true
  allow_fallback_evidence_for_promotion: false
  min_attack_strategies:
    - tap
    - crescendo
    - pair
  max_unresolved_high_findings: 0

A preset could expand into a regular campaign plan rather than inventing a separate execution path. For example:

LAUNCH_READINESS_PRESET = {
    "attack_strategies": ["tap", "crescendo", "pair"],
    "evidence_policy": {
        "promotion_requires_live_judge": True,
        "promotion_requires_replay": True,
        "fallback_is_diagnostic_only": True,
    },
    "replay_policy": {
        "replay_confirmed_failures": True,
        "replay_benign_probes": True,
        "compare_before_after_defense": True,
    },
    "report_sections": [
        "executive_summary",
        "bypass_classification",
        "capability_uplift_analysis",
        "evidence_mode_summary",
        "defense_candidate_status",
        "launch_gate_decision",
        "shareable_disclosure_packet",
    ],
}

Report output

The executive artifact could answer these questions up front:

1. Were bypasses found?
2. Are they universal, broad, narrow, or context-specific?
3. Did they produce meaningful capability uplift compared with baseline public-model behavior?
4. Did replay reproduce the behavior?
5. Did candidate defenses block the exploit without breaking benign utility probes?
6. Which findings are supported by live judge evidence vs sealed/golden/fallback evidence?
7. Which defenses are merely candidates, validated candidates, promotable defenses, or active guardrails?
8. Which launch gates passed, failed, or require human review?
9. What sanitized evidence can be shared with a partner, customer, regulator, or internal launch committee?

Possible top-level Markdown shape:

# Launch Readiness Report

## Decision
Status: Blocked | Conditional | Ready for limited release | Ready for public release
Reason: <short explanation tied to evidence>

## Evidence Summary
- Live judge findings: <count>
- Sealed/golden findings: <count>
- Fallback-only findings: <count>
- Replay-confirmed failures: <count>
- Benign replay regressions: <count>

## Bypass Classification
| Finding | Strategy | Scope | Capability uplift | Replay | Evidence mode | Gate impact |
| --- | --- | --- | --- | --- | --- | --- |

## Defense Status
| Finding | Candidate generated | Replay result | Utility result | Promotion state |
| --- | --- | --- | --- | --- |

## Shareable Packet
Sanitized traces, hashes, evidence labels, and reproduction notes suitable for external review.

Why this fits RedThread

This seems aligned with RedThread's existing evidence-honest posture. The preset would not create a safety badge. It would package the existing closed loop into a release-review workflow:

attack generation -> target execution -> judge scoring -> defense synthesis -> replay validation -> launch-gate evidence

The key value is classification and auditability. In a real launch dispute, the important questions are usually not just "did someone find a jailbreak?" They are:

• Was the bypass universal or narrow?
• Did it unlock new capability, or reproduce behavior already available elsewhere?
• Was the finding independently replayed?
• Was the evidence live, sealed, fallback, anecdotal, or imported?
• Did a proposed mitigation preserve benign utility?
• Is the mitigation actually promoted, or only a candidate?

RedThread already models many of those boundaries. A preset could make them easier to run and easier to communicate.

Non-goals

This should not claim:

• the model or app is safe
• all jailbreaks are impossible
• generated guardrails are production-ready by default
• RedThread replaces human security review
• a single report is sufficient for broad frontier-model governance

Acceptance criteria

A first version could be relatively small:

• add a named launch-readiness preset
• map the preset to existing attack, evidence, replay, and report settings
• add an executive Markdown section before detailed findings
• make fallback-only evidence clearly non-promotable in this mode
• include a launch-gate summary with ready, conditional, or blocked status
• include a sanitized external-review packet or clear placeholder for it

This would make RedThread useful not only after a jailbreak report appears, but before launch, when the team still has time to classify the evidence, fix the issue, and explain the residual risk precisely.