Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,37 @@ flightdeck pricing show --provider PROVIDER --version VERSION

Both flags are required. If the table does not exist, exits 1 with an error message.

### `flightdeck pricing check`

Check the age of **`flightdeck-bundled-*`** pricing tables in the ledger. Prints one line
per bundled snapshot with its anchor date and approximate age. Non-bundled tables are
ignored.

```bash
flightdeck pricing check [--max-age-days N] [--fail]
```

| Option | Default | Description |
|--------|---------|-------------|
| `--max-age-days` | `90` | Threshold in days. Tables older than this print `STALE` to stderr (and count toward `--fail`). Tables at or under the limit print `OK`. |
| `--fail` | off | Exit 1 if any bundled table exceeds `--max-age-days`. Useful as a CI gate. |

**Example output:**
```
OK flightdeck-bundled-2026-05 (~11 days old; max 90)
```

If no `flightdeck-bundled-*` tables are in the ledger (e.g. after `flightdeck init --no-bundled-pricing`),
exits 0 and prints `No flightdeck-bundled-* pricing tables in the ledger.`

Use in CI to surface stale bundled snapshots before they silently affect cost estimates:
```bash
flightdeck pricing check --max-age-days 90 --fail
```

See [pricing-catalog.md](pricing-catalog.md) for the bundled snapshot lifecycle and when
to replace with `flightdeck pricing import`.

---

## `flightdeck policy`
Expand Down
4 changes: 3 additions & 1 deletion docs/operations-and-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -521,7 +521,7 @@ endpoints (`GET /v1/releases`, `GET /v1/promoted`, `GET /v1/actions`) and intern

## SQLite storage schema

The operations layer reads and writes seven tables (via `src/flightdeck/storage.py`):
The operations layer reads and writes eight tables (via `src/flightdeck/storage.py`):

| Table | Purpose |
|-------|---------|
Expand All @@ -532,6 +532,7 @@ The operations layer reads and writes seven tables (via `src/flightdeck/storage.
| `active_policy` | Single-row table holding the active `Policy` JSON |
| `promoted_releases` | Current promoted pointer per `(agent_id, environment)` |
| `release_actions` | Append-only audit ledger; `audit_seq` is monotonically increasing |
| `promotion_requests` | Pending / completed / cancelled approval requests (added in migration v4); used when `promotion_requires_approval: true` |

`Storage.migrate()` runs forward-only numbered migrations. `flightdeck doctor` verifies
that migrations are applied through `LATEST_SCHEMA_MIGRATION_VERSION` and that
Expand Down Expand Up @@ -593,6 +594,7 @@ Migrations are numbered and forward-only; they are never reversed.
| 1 | Initial schema (all base tables via `CREATE TABLE IF NOT EXISTS`) |
| 2 | `CREATE INDEX … ON run_events(release_id, timestamp)` — speeds up diff/query |
| 3 | `ALTER TABLE release_actions ADD COLUMN audit_seq INTEGER`; backfill existing rows; add unique index |
| 4 | `CREATE TABLE IF NOT EXISTS promotion_requests` — adds the approval request/confirm workflow (columns: `request_id`, `status`, `release_id`, `agent_id`, `environment`, `window`, `reason`, `actor`, `baseline_release_id`, `policy_result_json`, `created_at`, `resolved_at`, `completed_action_id`) |

New migrations must increment `LATEST_SCHEMA_MIGRATION_VERSION` in `storage.py` and add a
corresponding check in `test_schemas.py` (or `test_doctor.py`).
Expand Down
2 changes: 1 addition & 1 deletion docs/pricing-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ your own YAML (and optionally **`--replace`** with **`--reason`**).

Bundled table YAML in the wheel includes **comment links** to each provider’s official list-pricing page so you can spot-check rates between FlightDeck releases.

**Staleness guardrails:** list prices change often. Run **`flightdeck pricing check`** to see whether any **`flightdeck-bundled-*`** table in the ledger is older than **`--max-age-days`** (default **90**); pass **`--fail`** for CI. **`flightdeck release diff`** and **`POST /v1/diff`** add **`pricing.warnings`** when baseline or candidate **`pricing_version`** is a stale bundled snapshot so economics do not look authoritative after the snapshot has aged out.
**Staleness guardrails:** list prices change often. Run **`flightdeck pricing check`** to see whether any **`flightdeck-bundled-*`** table in the ledger is older than **`--max-age-days`** (default **90**); pass **`--fail`** for CI. **`flightdeck release diff`** and **`POST /v1/diff`** add entries to **`pricing.warnings`** when baseline or candidate **`pricing_version`** is a stale bundled snapshot so economics do not look authoritative after the snapshot has aged out. See [cli.md § flightdeck pricing check](cli.md#flightdeck-pricing-check) for the full option reference.

**Maintainer cadence:** the bundled snapshot is **updated on each minor release** when vendor public list pricing changes materially (see **[ROADMAP.md](../ROADMAP.md)**). Operators in production should still treat **`flightdeck pricing import`** as the source of truth.

Expand Down
94 changes: 94 additions & 0 deletions docs/sdk-integrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,100 @@ batch processor. Set **`OTEL_EXPORTER_OTLP_ENDPOINT`** (for example
FlightDeck does not auto-instrument **`httpx`** or the Python SDK; create spans in your app or
attach upstream auto-instrumentation if you need request-level traces.

## Module reference

Each submodule under `flightdeck.integrations` has a single responsibility: map
third-party SDK output into a `RunEvent`. Import only the submodule you need.

### `flightdeck.integrations.common` (no extras required)

Available as `from flightdeck.integrations import make_run_end_event, temporal_labels`.

#### `make_run_end_event(**kwargs) -> RunEvent`

Convenience constructor for a `type=run_end` `RunEvent`. All named parameters map
directly to fields on the v1 wire shape:

| Parameter | Required | Description |
|-----------|----------|-------------|
| `agent_id` | yes | Stable agent ID |
| `release_id` | yes | Release ID from `flightdeck release register` |
| `run_id` | yes | Unique identifier; duplicates are skipped at ingest |
| `tenant_id` | yes | Tenant scoping dimension |
| `task_id` | yes | Task type dimension |
| `environment` | yes | Deployment environment |
| `provider` | yes | LLM provider (e.g. `"openai"`) |
| `model` | yes | Model name (e.g. `"gpt-4o"`) |
| `input_tokens` | yes | Prompt token count |
| `output_tokens` | yes | Completion token count |
| `cached_input_tokens` | no | Cached-prompt token count (default `0`) |
| `latency_ms` | no | End-to-end latency in milliseconds |
| `success` | no | Whether the run succeeded (default `True`) |
| `error_type` | no | Optional error class string |
| `trace_id`, `session_id`, `span_id` | no | Tracing identifiers (stored in `request.*`) |
| `labels` | no | Arbitrary string labels dict |
| `timestamp` | no | Event timestamp (defaults to `datetime.now(UTC)`) |
| `workspace_id` | no | Workspace identifier (default `"ws_local"`) |

#### `temporal_labels(*, workflow_id, workflow_run_id=None) -> dict[str, str]`

Returns a `labels` dict with `temporal.workflow_id` (and optionally `temporal.run_id`)
for tagging run events emitted from Temporal workflows. Pass the result as the `labels=`
argument to `make_run_end_event`.

### `flightdeck.integrations.openai_chat` (no extra needed; `openai` extra for the SDK itself)

#### `run_event_from_openai_chat_completion(response, *, agent_id, release_id, run_id, tenant_id, task_id, environment, **kwargs) -> RunEvent`

Constructs a `RunEvent` from an `openai.types.chat.ChatCompletion` response object.
Extracts `model`, `input_tokens`, `output_tokens`, and `cached_input_tokens` from
`response.usage`. Extra `kwargs` are passed to `make_run_end_event` (e.g. `latency_ms`,
`trace_id`). See `examples/integration/adoption/openai_chat/emit_run.py`.

### `flightdeck.integrations.anthropic_messages` (no extra needed; `anthropic` extra for the SDK itself)

#### `run_event_from_anthropic_message(message, *, agent_id, release_id, run_id, tenant_id, task_id, environment, **kwargs) -> RunEvent`

Constructs a `RunEvent` from an `anthropic.types.Message` object. Extracts `model`,
`input_tokens`, `output_tokens`, and `cache_read_input_tokens` from `message.usage`.
See `examples/integration/adoption/anthropic_messages/emit_run.py`.

### `flightdeck.integrations.openai_agents` (`integrations-openai-agents` extra)

#### `run_event_from_openai_agents_result(result, *, agent_id, release_id, run_id, tenant_id, task_id, environment, **kwargs) -> RunEvent`

Constructs a `RunEvent` from an OpenAI Agents SDK `RunResult` (or compatible object).
Aggregates token usage across all items in `result.raw_responses`. See
`examples/integration/adoption/openai_agents/emit_run.py`.

### `flightdeck.integrations.langchain_callback` (`integrations-langchain` extra)

#### `FlightDeckLangChainCallbackHandler`

A `BaseCallbackHandler` subclass. Pass an instance to LangChain chains or agents as
`callbacks=[handler]`. On `on_llm_end`, extracts token usage from the LLM result and
appends a `RunEvent` to `handler.events` (a list). After the chain completes, call
`client.ingest_run_events(handler.events)`. Constructor parameters:

| Parameter | Description |
|-----------|-------------|
| `agent_id` | Stable agent ID |
| `release_id` | Release ID |
| `run_id` | Unique run identifier (used for all events this handler captures) |
| `tenant_id`, `task_id`, `environment` | Standard scoping dimensions |

See `examples/integration/adoption/langchain/emit_run.py`.

### `flightdeck.integrations.crewai_bridge` (no extra; install `crewai` in your app env)

#### `run_event_from_crew_token_totals(input_tokens, output_tokens, *, model, provider, agent_id, release_id, run_id, tenant_id, task_id, environment, **kwargs) -> RunEvent`

Constructs a `RunEvent` from manually collected CrewAI token totals (no direct dependency
on CrewAI's internal classes). Collect totals from your crew's result callbacks and pass
them here. See `examples/integration/adoption/crewai/emit_totals.py`.

---

## Trust boundaries

Anyone who can reach **`POST /v1/events`** can append ledger rows. Keep **`flightdeck serve`**
Expand Down
Loading
Loading