diff --git a/CHANGELOG.md b/CHANGELOG.md index 477c7f4f..646bf8b7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ ## Unreleased +### Added (CLI) +- **Copilot input/cache tokens are read per request from `~/.copilot/session-store.db`.** Previously, codeburn relied on `session.shutdown` rollups from the Copilot CLI and GitHub Copilot desktop app. Those rollups are written only after a clean shutdown, stamp all usage on the shutdown day, and reset their counters at in-session compaction — so a crash could lose an entire session's input/cache usage, and even cleanly-closed long sessions were silently truncated. On one machine with long history, reading the per-request rows recovered about 35% of actual Copilot spend. Covered sessions now use per-request tokens with their real timestamps, counted exactly once against existing rollups and never added as extra calls or turns. Pre-store CLI sessions continue using the unchanged rollup path, and a locked or unreadable store defers only its own re-read instead of prematurely sealing daily history. Copilot reasoning tokens are also no longer double-billed: they are a subset of output already priced through the per-turn calls. This triggers a one-time re-parse, with the daily cache bumped from v17 to v19 to re-derive finalized days. (#946) + ## 0.9.20 - 2026-08-10 ### Added diff --git a/docs/providers/copilot.md b/docs/providers/copilot.md index 478687e2..c3b05338 100644 --- a/docs/providers/copilot.md +++ b/docs/providers/copilot.md @@ -8,15 +8,17 @@ GitHub Copilot Chat (CLI, VS Code core chat sessions, VS Code extension transcri ## Where it reads from -Three JSONL locations plus an optional OpenTelemetry SQLite source (see below). OTel is -preferred when present; chatSessions are only discovered when no OTel source is found. -Other discovered sources are walked on every run; results merge and dedupe. +Three JSONL locations plus two optional SQLite sources (see the OTel and session-store +sections). OTel is preferred when present; chatSessions are only discovered when no OTel +source is found. Other discovered sources are walked on every run; results merge and +dedupe. 1. **Legacy CLI sessions:** `~/.copilot/session-state/` 2. **VS Code core chat sessions:** `~/Library/Application Support/Code/User/workspaceStorage//chatSessions/*.jsonl` plus `~/Library/Application Support/Code/User/globalStorage/emptyWindowChatSessions/*.jsonl` and equivalents on Windows / Linux 3. **VS Code transcripts:** `~/Library/Application Support/Code/User/workspaceStorage//GitHub.copilot-chat/transcripts/` and equivalents on Windows / Linux 4. **OTel SQLite store:** VS Code Copilot Chat's `agent-traces.db` (see the OTel section). Preferred when present because it carries full input / output / cache token counts; legacy JSONL sources only record output tokens. -5. **JetBrains IDE sessions:** `~/.config/github-copilot////copilot-*-nitrite.db` (see the JetBrains section). Covers IntelliJ IDEA, PyCharm, RubyMine, etc. +5. **CLI session store:** `~/.copilot/session-store.db` (see the session-store section). One `assistant_usage_events` row per API request — the authoritative input/cache source for CLI and GitHub desktop-app sessions. +6. **JetBrains IDE sessions:** `~/.config/github-copilot////copilot-*-nitrite.db` (see the JetBrains section). Covers IntelliJ IDEA, PyCharm, RubyMine, etc. ## Storage format @@ -45,6 +47,52 @@ instead of trying to dedupe across stores. before the upgrade cannot be recovered, so monotonicity starts from the upgrade point, not retroactively. +## Session store (CLI / GitHub desktop app) + +The Copilot CLI and the GitHub Copilot desktop app both write +`~/.copilot/session-store.db` (SQLite/WAL) unconditionally. Its +`assistant_usage_events` table records one row per API request as it happens — +where the `session.shutdown` rollup in `events.jsonl` is written only on clean +shutdown (a crash loses the leg's input/cache accounting), lumps each leg into +one per-model total, and resets its counters at in-session compaction. Rows are +therefore authoritative for input / cache-read / cache-write / reasoning +tokens, with real per-request timestamps; per-turn output stays owned by the +`events.jsonl` `assistant.message` calls. `input_tokens` is cache-INCLUSIVE +(input + cache_read + cache_write), the same convention as the rollups; the +parser emits the uncached remainder. Override the path with +`CODEBURN_COPILOT_SESSION_STORE_DB` (deliberately NOT in the env fingerprint — +see the #927 ruling in `src/session-cache.ts`). + +- **Rollup reconciliation happens at serve time, per (session, model), in + `parseProviderSources`** — never in the parser. Both representations always + parse and cache; wherever store rows exist for a pair, the rollup calls are + dropped and each rollup leg's usage beyond the rows in its own interval + serves once as a synthesized residual call at that leg's timestamp. Sessions + with no rows (pre-store CLI builds) keep the rollup path unchanged. +- **Behavioral weight.** Rollups and residuals are aggregate accounting: tokens + and cost count, but never api-call / model-call / turn weight. A store row + pairs with its per-turn call by timestamp adjacency (2-minute window, + computed over the full serve set); only unpaired rows — crash-recovered, + store-only requests — count as calls. +- **Failure semantics.** True absence (ENOENT, no sqlite driver, `no such + table/column` from pre-store CLI builds) reads as absent — no source, rollups + rule. Every other failure (locked, EACCES, corrupt, mid-replace) emits the + source anyway: its parse defers on the busy shape, previously cached rows + keep serving, and the pass reports incomplete hydration so the daily + backfill holds its watermark. +- **Durable cache.** Rides the same `durableSources` union as OTel, with one + scoped difference: the store declares `retainWhilePresent`, so its rows are + never evicted while the DB exists (crash-only rows have no rollup to fall + back to); every other copilot source keeps the standard durable schedule, + aging out at 90 days whether or not the file remains. A deleted store's rows + serve as orphans until the 90-day age-out. Reconciliation reads only cached + contents, so deleting or resetting the store never changes served totals. +- **Billing metadata.** Each row's `total_nano_aiu` and `request_multiplier` + are captured onto the cached call when the store's schema has them (older + stores parse identically without). Nothing prices or displays them yet — + billing-grade cost is upstream #890. +- **Requires Node 22+** (`node:sqlite`), same as the OTel source. + ## JetBrains IDEs (IntelliJ, PyCharm, …) The JetBrains Copilot plugin does **not** write to any of the VS Code or CLI @@ -160,11 +208,11 @@ surfaces, add a reader with a captured fixture.) ## Caching -None for the JSONL sources. The OTel source uses a durable cache (see above). +None for the JSONL sources. The OTel and session-store sources use the durable cache (see above). ## Deduplication -Legacy JSONL and transcript sessions dedupe per `messageId`. Core chat sessions dedupe per `copilot-chatsession::`, and are not discovered when an OTel source is present. JetBrains `.db` turns dedupe per `copilot:jb::` (a per-conversation index, plus reply-content dedup within each conversation). These sources otherwise touch disjoint locations from the VS Code / CLI sources. +Legacy JSONL and transcript sessions dedupe per `messageId`. Core chat sessions dedupe per `copilot-chatsession::`, and are not discovered when an OTel source is present. Session-store rows dedupe per `copilot-store:::` (the hash covers `created_at`, token counts, and model, so a same-path DB reset reusing AUTOINCREMENT ids cannot alias a different request onto a cached key); shutdown rollups per `copilot::shutdown::`, with serve-time residuals synthesized (never cached) under `copilot::shutdown-residual::`. JetBrains `.db` turns dedupe per `copilot:jb::` (a per-conversation index, plus reply-content dedup within each conversation). These sources otherwise touch disjoint locations from the VS Code / CLI sources. If a workspace hash contains at least one `chatSessions/*.jsonl` file, the provider skips that hash's legacy `GitHub.copilot-chat/transcripts/` directory. The core chat session journal is the modern token-bearing source for the same conversations, so reading both would inflate call counts. diff --git a/src/audit-report.ts b/src/audit-report.ts index a133dcf4..4d3a1bbf 100644 --- a/src/audit-report.ts +++ b/src/audit-report.ts @@ -1,3 +1,4 @@ +import { isBehavioralCall } from './behavioral-weight.js' import { getModelCosts, type ModelCosts } from './models.js' import { getProvider } from './providers/index.js' import { formatCost, formatTokens } from './format.js' @@ -98,7 +99,9 @@ export async function aggregateAudit(projects: ProjectSummary[]): Promise !call.supplementaryAccounting) +} + +/** Number of behavioral turns (weightless accounting containers excluded). */ +export function behavioralTurnCount(turns: readonly WeightedTurn[]): number { + let n = 0 + for (const turn of turns) if (isBehavioralTurn(turn)) n++ + return n +} diff --git a/src/compare-stats.ts b/src/compare-stats.ts index 0068f86a..5faabf73 100644 --- a/src/compare-stats.ts +++ b/src/compare-stats.ts @@ -1,11 +1,20 @@ import { readdir, readFile } from 'fs/promises' import { join } from 'path' -import type { ProjectSummary } from './types.js' +import type { ClassifiedTurn, ProjectSummary } from './types.js' +import { isBehavioralCall } from './behavioral-weight.js' import { getShortModelName } from './models.js' const PLANNING_TOOLS = new Set(['TaskCreate', 'TaskUpdate', 'TodoWrite', 'EnterPlanMode', 'ExitPlanMode']) +/// The turn's primary model: the model of its FIRST BEHAVIORAL call. Undefined +/// when the turn has none — accounting-only turns carry no behavioral evidence, +/// so they are excluded from efficiency comparisons (their spend still appears +/// in every spend report). +function primaryTurnModel(turn: ClassifiedTurn): string | undefined { + return turn.assistantCalls.find(isBehavioralCall)?.model +} + export type ModelStats = { model: string calls: number @@ -39,9 +48,8 @@ export function aggregateModelStats(projects: ProjectSummary[]): ModelStats[] { for (const project of projects) { for (const session of project.sessions) { for (const turn of session.turns) { - if (turn.assistantCalls.length === 0) continue - const primaryModel = turn.assistantCalls[0]!.model - if (primaryModel === '') continue + const primaryModel = primaryTurnModel(turn) + if (primaryModel === undefined || primaryModel === '') continue const ms = ensure(primaryModel) ms.totalTurns++ @@ -57,7 +65,7 @@ export function aggregateModelStats(projects: ProjectSummary[]): ModelStats[] { for (const call of turn.assistantCalls) { if (call.model === '') continue const cs = call.model === primaryModel ? ms : ensure(call.model) - cs.calls++ + if (isBehavioralCall(call)) cs.calls++ cs.cost += call.costUSD cs.outputTokens += call.usage.outputTokens cs.inputTokens += call.usage.inputTokens @@ -224,8 +232,7 @@ export function computeCategoryComparison(projects: ProjectSummary[], modelA: st for (const project of projects) { for (const session of project.sessions) { for (const turn of session.turns) { - if (turn.assistantCalls.length === 0) continue - const primary = turn.assistantCalls[0]!.model + const primary = primaryTurnModel(turn) if (primary !== modelA && primary !== modelB) continue const acc = ensure(primary === modelA ? mapA : mapB, turn.category) @@ -272,17 +279,17 @@ export function computeWorkingStyle(projects: ProjectSummary[], modelA: string, for (const project of projects) { for (const session of project.sessions) { for (const turn of session.turns) { - if (turn.assistantCalls.length === 0) continue - const primary = turn.assistantCalls[0]!.model + const primary = primaryTurnModel(turn) if (primary !== modelA && primary !== modelB) continue const s = primary === modelA ? sA : sB s.totalTurns++ - const turnTools = turn.assistantCalls.flatMap(c => c.tools) - if (turnTools.some(t => PLANNING_TOOLS.has(t)) || turn.assistantCalls.some(c => c.hasPlanMode)) { + const behavioralCalls = turn.assistantCalls.filter(isBehavioralCall) + const turnTools = behavioralCalls.flatMap(c => c.tools) + if (turnTools.some(t => PLANNING_TOOLS.has(t)) || behavioralCalls.some(c => c.hasPlanMode)) { s.planModeUses++ } - for (const call of turn.assistantCalls) { + for (const call of behavioralCalls) { s.totalToolCalls += call.tools.length if (call.hasAgentSpawn) s.agentSpawns++ if (call.speed === 'fast') s.fastModeCalls++ diff --git a/src/daily-cache.ts b/src/daily-cache.ts index 7abb445c..aa6cad4c 100644 --- a/src/daily-cache.ts +++ b/src/daily-cache.ts @@ -5,7 +5,24 @@ import { homedir } from 'os' import { join } from 'path' import type { DateRange, ProjectSummary } from './types.js' -// Bumped to 17: copilot CLI sessions were misclassified as VS Code transcripts +// Bumped to 19, deliberately skipping 18: an earlier head of this same change +// (#946) was pushed publicly claiming v18 under DIFFERENT accounting — no +// supplementary-call weighting, whole-session rollup suppression — and +// adoptOlderDailyCaches/isMigratableCache would take those days forward as +// finalized without re-deriving them. A distinct version is the only thing +// that stops one version number meaning two accountings. +// +// v18 (never released): copilot input/cache tokens for sessions covered by the CLI's +// session-store.db moved from one shutdown-rollup lump (stamped at session +// end) to per-request DB rows with real timestamps, supplementary accounting +// calls (rollups, covered rows) stopped counting as api/model calls, and +// reasoning tokens left the copilot cost recompute (they are inside the +// output the per-turn calls already bill). Per-day attribution, call counts +// and costs all move, so days finalized at v17 would disagree with the live +// parse. Re-derivation rides the v14 carry-forward semantics; sourceless +// days carry forward as-is. +// +// v17: copilot CLI sessions were misclassified as VS Code transcripts // (#944), so days finalized at v16 or earlier carry output-only copilot costs — // the session.shutdown rollup's input/cache tokens were dropped. Raising // MIN_SUPPORTED_VERSION forces the one-time re-derivation under the @@ -73,8 +90,8 @@ import type { DateRange, ProjectSummary } from './types.js' // that older binaries skipped. v8 added local-model savings to the daily // rollup; the `savingsConfigHash` field is invalidated separately when the // user changes their `localModelSavings` mapping. -export const DAILY_CACHE_VERSION = 17 -const MIN_SUPPORTED_VERSION = 17 +export const DAILY_CACHE_VERSION = 19 +const MIN_SUPPORTED_VERSION = 19 // Version-suffixed so different binaries each own a distinct file and never // clobber an incompatible schema. Bumping the version mints a fresh filename; // adoptOlderDailyCaches then unions days out of every previous file (including diff --git a/src/dashboard.tsx b/src/dashboard.tsx index b66b41cf..b1f25302 100644 --- a/src/dashboard.tsx +++ b/src/dashboard.tsx @@ -14,6 +14,7 @@ import { scanAndDetect, type WasteFinding, type WasteAction, type OptimizeResult import { aggregateFileChurn, buildCoachingNotes, computePricingCoverage, medianTimeToFirstEditMs, scanUserCorrections, worstOneShotCategory, type ReworkedFile } from './workflow-insights.js' import { estimateContextBudget, type ContextBudget } from './context-budget.js' import { dateKey } from './day-aggregator.js' +import { behavioralCallCount } from './behavioral-weight.js' import { CompareView } from './compare.js' import { getPlanUsages, type PlanUsage } from './plan-usage.js' import { planDisplayName } from './plans.js' @@ -431,7 +432,7 @@ export function getDailyActivityRows(projects: ProjectSummary[]): DailyActivityR if (!turn.timestamp) continue const day = dateKey(turn.timestamp) dailyCosts[day] = (dailyCosts[day] ?? 0) + turn.assistantCalls.reduce((s, c) => s + c.costUSD, 0) - dailyCalls[day] = (dailyCalls[day] ?? 0) + turn.assistantCalls.length + dailyCalls[day] = (dailyCalls[day] ?? 0) + behavioralCallCount(turn.assistantCalls) } } } diff --git a/src/day-aggregator.ts b/src/day-aggregator.ts index 03695832..bab24617 100644 --- a/src/day-aggregator.ts +++ b/src/day-aggregator.ts @@ -1,6 +1,7 @@ import type { DailyEntry, ProjectDayStats, ProviderDaySlice } from './daily-cache.js' import type { PeriodData } from './menubar-json.js' import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js' +import { isBehavioralCall, isBehavioralTurn } from './behavioral-weight.js' function emptyEntry(date: string): DailyEntry { return { @@ -114,8 +115,14 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: const turnDate = dateKeyFn(turn.timestamp || turn.assistantCalls[0]!.timestamp) const turnDay = ensure(turnDate) - const editTurns = turn.hasEdits ? 1 : 0 - const oneShotTurns = turn.hasEdits && turn.retries === 0 ? 1 : 0 + // A turn whose calls are all supplementary accounting (copilot rollup + // / paired store rows) is not a behavioral exchange: its cost still + // lands in the category, but it must add no turn/edit weight here or + // sealed daily history diverges from the live session summaries + // (which apply the same rule in buildSessionSummary). + const behavioralTurn = isBehavioralTurn(turn) + const editTurns = behavioralTurn && turn.hasEdits ? 1 : 0 + const oneShotTurns = behavioralTurn && turn.hasEdits && turn.retries === 0 ? 1 : 0 const turnCost = turn.assistantCalls.reduce((s, c) => s + c.costUSD, 0) const turnSavings = turn.assistantCalls.reduce((s, c) => s + (c.savingsUSD ?? 0), 0) @@ -123,7 +130,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: turnDay.oneShotTurns += oneShotTurns const cat = turnDay.categories[turn.category] ?? { turns: 0, cost: 0, savingsUSD: 0, editTurns: 0, oneShotTurns: 0 } - cat.turns += 1 + if (behavioralTurn) cat.turns += 1 cat.cost += turnCost cat.savingsUSD += turnSavings cat.editTurns += editTurns @@ -158,7 +165,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: turnSlice.editTurns! += ownsTurn ? editTurns : 0 turnSlice.oneShotTurns! += ownsTurn ? oneShotTurns : 0 const sliceCat = turnSlice.categories![turn.category] ?? { turns: 0, cost: 0, savingsUSD: 0, editTurns: 0, oneShotTurns: 0 } - sliceCat.turns += ownsTurn ? 1 : 0 + sliceCat.turns += ownsTurn && behavioralTurn ? 1 : 0 sliceCat.cost += totals.cost sliceCat.savingsUSD += totals.savingsUSD sliceCat.editTurns += ownsTurn ? editTurns : 0 @@ -168,6 +175,11 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: for (const call of turn.assistantCalls) { const callSavings = call.savingsUSD ?? 0 + // Same weight rule as buildSessionSummary: a supplementary + // accounting call contributes cost/tokens but is not a distinct + // request, so it must not increment any `calls` counter the daily + // cache seals (day, project, model, provider slice). + const callWeight = isBehavioralCall(call) ? 1 : 0 // Call-derived values bucket under the call's OWN day (see the // two-rule comment above). An unparseable call timestamp falls back // to the turn's anchor day rather than producing a garbage date key. @@ -176,7 +188,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: callDay.cost += call.costUSD callDay.savingsUSD += callSavings - callDay.calls += 1 + callDay.calls += callWeight callDay.inputTokens += call.usage.inputTokens callDay.outputTokens += call.usage.outputTokens callDay.cacheReadTokens += call.usage.cacheReadInputTokens @@ -184,7 +196,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: const dayProject = ensureProject(callDay, session.project, project.projectPath) dayProject.cost += call.costUSD - dayProject.calls += 1 + dayProject.calls += callWeight dayProject.savingsUSD += callSavings const model = callDay.models[call.model] ?? { @@ -192,7 +204,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0, } - model.calls += 1 + model.calls += callWeight model.cost += call.costUSD model.savingsUSD += callSavings model.inputTokens += call.usage.inputTokens @@ -202,7 +214,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: callDay.models[call.model] = model const slice = ensureSlice(callDay, call.provider) - slice.calls += 1 + slice.calls += callWeight slice.cost += call.costUSD slice.savingsUSD += callSavings slice.inputTokens! += call.usage.inputTokens @@ -212,7 +224,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: const sliceProject = ensureProject(slice, session.project, project.projectPath) sliceProject.cost += call.costUSD - sliceProject.calls += 1 + sliceProject.calls += callWeight sliceProject.savingsUSD += callSavings const sliceModel = slice.models![call.model] ?? { @@ -220,7 +232,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn: inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0, } - sliceModel.calls += 1 + sliceModel.calls += callWeight sliceModel.cost += call.costUSD sliceModel.savingsUSD += callSavings sliceModel.inputTokens += call.usage.inputTokens diff --git a/src/export.ts b/src/export.ts index 08cd795b..733dff72 100644 --- a/src/export.ts +++ b/src/export.ts @@ -4,6 +4,7 @@ import { dirname, join, resolve } from 'path' import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js' import { getCurrency, convertCost, roundForActiveCurrency } from './currency.js' import { dateKey } from './day-aggregator.js' +import { behavioralTurnCount, isBehavioralCall } from './behavioral-weight.js' import { aggregateModelEfficiency } from './model-efficiency.js' function escCsv(s: string): string { @@ -14,7 +15,7 @@ function escCsv(s: string): string { return sanitized } -type Row = Record +type Row = Record function rowsToCsv(rows: Row[]): string { if (rows.length === 0) return '' @@ -59,7 +60,10 @@ function buildDailyRows(projects: ProjectSummary[], period: string): Row[] { for (const call of turn.assistantCalls) { daily[day].cost += call.costUSD daily[day].savings += call.savingsUSD ?? 0 - daily[day].calls++ + // Same weight rule as aggregateProjectsIntoDays: a supplementary + // accounting call carries cost/tokens but is not a distinct request, + // so daily.csv call counts must reconcile with summary.csv. + if (isBehavioralCall(call)) daily[day].calls++ daily[day].input += call.usage.inputTokens daily[day].output += call.usage.outputTokens daily[day].cacheRead += call.usage.cacheReadInputTokens @@ -104,6 +108,12 @@ function buildRecordRows(projects: ProjectSummary[]): Row[] { cacheReadTokens: Math.max(call.usage.cacheReadInputTokens, call.usage.cachedInputTokens), cost: roundForActiveCurrency(convertCost(call.costUSD)), savings: roundForActiveCurrency(convertCost(call.savingsUSD ?? 0)), + // Records are the raw serve ledger and keep every supplementary + // accounting row; the marker is how a "one row per API call" + // consumer tells them apart. Key present on every row (undefined + // when false) so rowsToCsv, which reads headers off the first row, + // always emits the column; JSON drops the undefined ones. + supplementary: call.supplementaryAccounting ? true : undefined, }) } } @@ -270,7 +280,7 @@ function buildSessionRows(projects: ProjectSummary[]): Row[] { [`Cost (${code})`]: roundForActiveCurrency(convertCost(s.totalCostUSD)), [`Saved (${code})`]: roundForActiveCurrency(convertCost(s.totalSavingsUSD)), 'API Calls': s.apiCalls, - Turns: s.turns.length, + Turns: behavioralTurnCount(s.turns), subagentType: s.agentType?.trim() || undefined, model: models.size === 1 ? [...models][0] : undefined, }) @@ -320,7 +330,9 @@ function buildReadme(periods: PeriodExport[]): string { ' daily.csv Day-by-day breakdown, Period column distinguishes the window.', ' activity.csv Time spent per task category (Coding, Debugging, Exploration, etc.).', ' models.csv Spend per model with token totals and cache usage.', - ' records.csv One row per API call, including optional subagentType and model.', + ' records.csv One row per served call, with optional subagentType and model.', + ' supplementary marks accounting-only rows: recovered tokens/cost that', + ' are not distinct requests.', ' projects.csv Spend per project folder for the selected detail period.', ' sessions.csv One row per session for the selected detail period.', ' tools.csv Tool invocations and share for the selected detail period.', diff --git a/src/format.ts b/src/format.ts index 46aac14f..ed2c3ca2 100644 --- a/src/format.ts +++ b/src/format.ts @@ -1,5 +1,6 @@ import chalk from 'chalk' import type { ProjectSummary } from './types.js' +import { behavioralCallCount } from './behavioral-weight.js' // Re-exported from currency.ts so existing imports from './format.js' keep working. // The currency-aware version applies exchange rate and symbol automatically. @@ -82,7 +83,9 @@ export function renderStatusBar(projects: ProjectSummary[], totals?: StatusBarTo if (!bucketTs) continue const day = localDateString(new Date(bucketTs)) const turnCost = turn.assistantCalls.reduce((s, c) => s + c.costUSD, 0) - const turnCalls = turn.assistantCalls.length + // Cost keeps every call; the calls figure counts only behavioral ones, + // so a supplementary-only turn still spends but adds no requests. + const turnCalls = behavioralCallCount(turn.assistantCalls) if (day === today) { todayCost += turnCost; todayCalls += turnCalls } if (day >= monthStart) { monthCost += turnCost; monthCalls += turnCalls } } diff --git a/src/main.ts b/src/main.ts index d201920b..8dacf5d3 100644 --- a/src/main.ts +++ b/src/main.ts @@ -10,6 +10,7 @@ import { convertCost, formatCost } from './currency.js' import { renderStatusBar } from './format.js' import { toDateString } from './daily-cache.js' import { dateKey } from './day-aggregator.js' +import { isBehavioralCall, isBehavioralTurn } from './behavioral-weight.js' import { CATEGORY_LABELS, type DateRange, type ProjectSummary, type TaskCategory } from './types.js' import { aggregateModelEfficiency } from './model-efficiency.js' import { buildPeriodData, buildMenubarPayloadForRange, buildDurablePeriod, type DurablePeriod } from './usage-aggregator.js' @@ -515,10 +516,16 @@ function buildJsonReport(projects: ProjectSummary[], period: string, periodKey: if (!ts) { continue } const day = dateKey(ts) if (!dailyMap[day]) { dailyMap[day] = { cost: 0, savings: 0, calls: 0, turns: 0, editTurns: 0, oneShotTurns: 0 } } - dailyMap[day].turns += 1 - if (turn.hasEdits) { - dailyMap[day].editTurns += 1 - if (turn.retries === 0) dailyMap[day].oneShotTurns += 1 + // Turn weight follows day-aggregator.ts exactly: a turn whose calls are + // all supplementary accounting (copilot rollup / paired store rows) is + // not a behavioral exchange, so it adds cost below but no turn/edit + // weight here — otherwise this fallback disagrees with durable.days. + if (isBehavioralTurn(turn)) { + dailyMap[day].turns += 1 + if (turn.hasEdits) { + dailyMap[day].editTurns += 1 + if (turn.retries === 0) dailyMap[day].oneShotTurns += 1 + } } for (const call of turn.assistantCalls) { // Cost/savings/calls bucket under each call's OWN day — the same @@ -531,7 +538,7 @@ function buildJsonReport(projects: ProjectSummary[], period: string, periodKey: if (!dailyMap[callDay]) { dailyMap[callDay] = { cost: 0, savings: 0, calls: 0, turns: 0, editTurns: 0, oneShotTurns: 0 } } dailyMap[callDay].cost += call.costUSD dailyMap[callDay].savings += call.savingsUSD ?? 0 - dailyMap[callDay].calls += 1 + dailyMap[callDay].calls += isBehavioralCall(call) ? 1 : 0 } } } @@ -1180,8 +1187,11 @@ program today: { cost: Math.round(todayData.cost * rate * 100) / 100, savings: Math.round(todayData.savingsUSD * rate * 100) / 100, calls: todayData.calls }, month: { cost: Math.round(monthData.cost * rate * 100) / 100, savings: Math.round(monthData.savingsUSD * rate * 100) / 100, calls: monthData.calls }, } - const savingsCallsToday = todayProjects.reduce((s, p) => s + p.sessions.reduce((s2, sess) => s2 + sess.turns.reduce((s3, turn) => s3 + turn.assistantCalls.reduce((s4, c) => s4 + (c.savingsUSD && c.savingsUSD > 0 ? 1 : 0), 0), 0), 0), 0) - const savingsCallsMonth = monthProjects.reduce((s, p) => s + p.sessions.reduce((s2, sess) => s2 + sess.turns.reduce((s3, turn) => s3 + turn.assistantCalls.reduce((s4, c) => s4 + (c.savingsUSD && c.savingsUSD > 0 ? 1 : 0), 0), 0), 0), 0) + // Savings DOLLARS keep every call, but these are request COUNTS: a + // supplementary accounting call (copilot rollup / paired store row) can + // carry configured model-savings too and must not count as a request. + const savingsCallsToday = todayProjects.reduce((s, p) => s + p.sessions.reduce((s2, sess) => s2 + sess.turns.reduce((s3, turn) => s3 + turn.assistantCalls.reduce((s4, c) => s4 + (c.savingsUSD && c.savingsUSD > 0 && isBehavioralCall(c) ? 1 : 0), 0), 0), 0), 0) + const savingsCallsMonth = monthProjects.reduce((s, p) => s + p.sessions.reduce((s2, sess) => s2 + sess.turns.reduce((s3, turn) => s3 + turn.assistantCalls.reduce((s4, c) => s4 + (c.savingsUSD && c.savingsUSD > 0 && isBehavioralCall(c) ? 1 : 0), 0), 0), 0), 0) if (todayData.savingsUSD > 0 || monthData.savingsUSD > 0) { payload.localModelSavings = { today: payload.today.savings, diff --git a/src/models-report.ts b/src/models-report.ts index aaeb2235..720e2994 100644 --- a/src/models-report.ts +++ b/src/models-report.ts @@ -1,6 +1,7 @@ import chalk from 'chalk' import stripAnsi from 'strip-ansi' +import { isBehavioralCall } from './behavioral-weight.js' import { codexCredits } from './codex-credits.js' import { formatCost, formatTokens } from './format.js' import { getProvider } from './providers/index.js' @@ -119,7 +120,11 @@ export async function aggregateModels(projects: ProjectSummary[], opts: Aggregat buckets.set(key, bucket) } bucket.inputTokens += call.usage.inputTokens - bucket.outputTokens += call.usage.outputTokens + call.usage.reasoningTokens + // Copilot reasoning tokens are already INSIDE outputTokens (same rule as cachedCallToApiCall + // in parser.ts), so folding them in would display phantom output for its store rows/rollups. + bucket.outputTokens += provider === 'copilot' + ? call.usage.outputTokens + : call.usage.outputTokens + call.usage.reasoningTokens bucket.cacheWriteTokens += call.usage.cacheCreationInputTokens // cacheReadInputTokens (Anthropic vocab) and cachedInputTokens (OpenAI vocab) // are two names for the same thing. Providers populate one or set both to the @@ -131,7 +136,9 @@ export async function aggregateModels(projects: ProjectSummary[], opts: Aggregat if (!bucket.savingsBaselineModel && call.savingsBaselineModel) { bucket.savingsBaselineModel = call.savingsBaselineModel } - bucket.calls += 1 + // Supplementary accounting calls keep their tokens and cost above but are not + // distinct requests, so they add no call weight (see behavioral-weight.ts). + if (isBehavioralCall(call)) bucket.calls += 1 const modelKey = `${provider} ${model}` let perCat = perModelCategoryCost.get(modelKey) diff --git a/src/parser.ts b/src/parser.ts index 712295b0..e8b76d3e 100644 --- a/src/parser.ts +++ b/src/parser.ts @@ -25,9 +25,11 @@ import { loadCache, reconcileFile, saveCache, + sourcePathStatCandidates, } from './session-cache.js' import { acquireCacheRefreshLock, type RefreshLockHandle } from './cache-refresh-lock.js' import { dateKey } from './day-aggregator.js' +import { isBehavioralCall, isBehavioralTurn } from './behavioral-weight.js' import type { ParsedProviderCall, SessionSource } from './providers/types.js' import type { ApiUsageIteration, @@ -1676,14 +1678,21 @@ function buildSessionSummary( for (const turn of turns) { const turnCost = turn.assistantCalls.reduce((s, c) => s + c.costUSD, 0) const turnSavings = turn.assistantCalls.reduce((s, c) => s + (c.savingsUSD ?? 0), 0) + // A turn whose calls are all supplementary accounting (copilot rollup / + // paired store rows) is not a behavioral exchange: its cost still lands in + // the category so breakdowns keep summing to the totals, but it adds no + // turn/edit/retry weight. Sessions normally never hold such turns (they + // are folded into behavioral turns upstream); this covers the + // accounting-only container of a session with no behavioral turns at all. + const behavioralTurn = isBehavioralTurn(turn) if (!categoryBreakdown[turn.category]) { categoryBreakdown[turn.category] = { turns: 0, costUSD: 0, savingsUSD: 0, retries: 0, editTurns: 0, oneShotTurns: 0 } } - categoryBreakdown[turn.category].turns++ + if (behavioralTurn) categoryBreakdown[turn.category].turns++ categoryBreakdown[turn.category].costUSD += turnCost categoryBreakdown[turn.category].savingsUSD += turnSavings - if (turn.hasEdits) { + if (behavioralTurn && turn.hasEdits) { categoryBreakdown[turn.category].editTurns++ categoryBreakdown[turn.category].retries += turn.retries if (turn.retries === 0) categoryBreakdown[turn.category].oneShotTurns++ @@ -1694,10 +1703,10 @@ function buildSessionSummary( if (!skillBreakdown[skillKey]) { skillBreakdown[skillKey] = { turns: 0, costUSD: 0, savingsUSD: 0, editTurns: 0, oneShotTurns: 0 } } - skillBreakdown[skillKey].turns++ + if (behavioralTurn) skillBreakdown[skillKey].turns++ skillBreakdown[skillKey].costUSD += turnCost skillBreakdown[skillKey].savingsUSD += turnSavings - if (turn.hasEdits) { + if (behavioralTurn && turn.hasEdits) { skillBreakdown[skillKey].editTurns++ if (turn.retries === 0) skillBreakdown[skillKey].oneShotTurns++ } @@ -1714,7 +1723,9 @@ function buildSessionSummary( totalReasoning += call.usage.reasoningTokens totalCacheRead += call.usage.cacheReadInputTokens totalCacheWrite += call.usage.cacheCreationInputTokens - apiCalls++ + // Supplementary accounting calls contribute tokens/cost above but are + // not distinct requests: no api-call or per-model call weight. + if (isBehavioralCall(call)) apiCalls++ const modelKey = call.provider === 'devin' ? call.model : getShortModelName(call.model) if (!modelBreakdown[modelKey]) { @@ -1726,7 +1737,7 @@ function buildSessionSummary( tokens: { inputTokens: 0, outputTokens: 0, cacheCreationInputTokens: 0, cacheReadInputTokens: 0, cachedInputTokens: 0, reasoningTokens: 0, webSearchRequests: 0 }, } } - modelBreakdown[modelKey].calls++ + if (isBehavioralCall(call)) modelBreakdown[modelKey].calls++ modelBreakdown[modelKey].costUSD += call.costUSD modelBreakdown[modelKey].savingsUSD += callSavings modelBreakdown[modelKey].estimatedCostUSD = (modelBreakdown[modelKey].estimatedCostUSD ?? 0) + callEstimated @@ -2404,6 +2415,8 @@ function providerCallToCachedCall(call: ParsedProviderCall): CachedCall { ...(call.locAdded ? { locAdded: call.locAdded } : {}), ...(call.locRemoved ? { locRemoved: call.locRemoved } : {}), ...(call.editFailed ? { editFailed: call.editFailed } : {}), + ...(call.nanoAiu != null ? { nanoAiu: call.nanoAiu } : {}), + ...(call.requestMultiplier != null ? { requestMultiplier: call.requestMultiplier } : {}), activeDurationMs: call.activeDurationMs, activeGeneratedTokens: call.activeGeneratedTokens, toolWaitMs: call.toolWaitMs, @@ -2524,7 +2537,14 @@ function providerCallsToCachedTurns(calls: ParsedProviderCall[]): CachedTurn[] { function cachedCallToApiCall(call: CachedCall): ParsedApiCall { const u = call.usage - const outputForCost = call.provider === 'claude' + // Claude thinking and Copilot reasoning tokens are already INSIDE + // outputTokens (Copilot's own per-request token_details_json prices + // input/cache/output and nothing else, and its reasoning counts are a + // subset of the output count), so adding them here would bill them twice — + // for copilot literally so: its session-store/shutdown supplementary calls + // carry reasoningTokens with outputTokens 0 while the per-turn call bills + // the full output. Other providers report reasoning separately from output. + const outputForCost = call.provider === 'claude' || call.provider === 'copilot' ? u.outputTokens : u.outputTokens + u.reasoningTokens const costUSD = calculateCost( @@ -2584,6 +2604,111 @@ function cachedTurnToClassified(turn: CachedTurn, resolvedBranch?: string): Clas return classifyTurn(parsed) } +// Copilot behavioral-weight assignment + turn folding, applied per session at +// serve time just before summarization. A shutdown rollup (or its synthesized +// residual) is aggregate accounting, never a request, so it is always +// supplementary. A store row is one real request, but when the request's +// per-turn call exists in the cache its row is supplementary too — only the +// unpaired rows (store-only requests: a crash or pruned session-state lost +// their per-turn calls) carry behavioral weight. Which rows are paired was +// decided upstream over the FULL serve set (timestamp-adjacency matching in +// parseProviderSources' reconciliation sweep) and arrives as a key set, so a +// date-range slice that separates a row from its per-turn call cannot +// double-count the request across adjacent day queries. +// A turn made only of supplementary calls folds into the nearest behavioral +// turn — but only within a 30-minute window (deliberately WIDER than the +// 2-minute pairing window: folding only moves turn structure, and same-half- +// hour cost stays on its own day), so a rollup stamped days after the last +// activity keeps its own (weightless) turn and its cost stays on its own +// day. A session with no behavioral turn at all keeps its supplementary +// turns as-is — separate weightless containers, each on its own day. +const FOLD_WINDOW_MS = 30 * 60 * 1000 +// Local calendar day of an epoch ms value, matching day-aggregator's local +// bucketing (not UTC): the fold must not move a turn across the boundary the +// daily rollup buckets on. +function localDayKey(ms: number): string { + const d = new Date(ms) + return `${d.getFullYear()}-${d.getMonth()}-${d.getDate()}` +} +function foldCopilotSupplementaryTurns( + sessionId: string, + turns: ClassifiedTurn[], + supplementaryStoreKeys: ReadonlySet | undefined, +): ClassifiedTurn[] { + const shutdownPrefix = `copilot:${sessionId}:shutdown` + let hasSupplementary = false + for (const t of turns) { + for (const c of t.assistantCalls) { + if ( + c.deduplicationKey.startsWith(shutdownPrefix) || + (c.deduplicationKey.startsWith('copilot-store:') && supplementaryStoreKeys?.has(c.deduplicationKey)) + ) { + c.supplementaryAccounting = true + hasSupplementary = true + } + } + } + if (!hasSupplementary) return turns + const anchored: ClassifiedTurn[] = [] + const floating: ClassifiedTurn[] = [] + for (const t of turns) { + ;(t.assistantCalls.some(c => !c.supplementaryAccounting) ? anchored : floating).push(t) + } + if (floating.length === 0) return turns + if (anchored.length === 0) { + // No behavioral turn to fold into (a rollup-only session, or a range + // slice that excluded every behavioral turn). The supplementary turns + // stay SEPARATE: merging them into one container would re-anchor later + // legs' turn-level cost onto the first leg's day. They carry zero + // turn/call weight either way. + return turns + } + // Nearest anchored turn by timestamp via one sorted pass + binary search — + // a long session can hold thousands of turns and this runs on every serve. + const anchorTs = anchored + .map(a => ({ ts: new Date(a.timestamp).getTime(), turn: a })) + .filter(a => !Number.isNaN(a.ts)) + .sort((a, b) => a.ts - b.ts) + const kept: ClassifiedTurn[] = [...anchored] + for (const t of floating) { + const ts = new Date(t.timestamp).getTime() + let best: ClassifiedTurn | null = null + let bestDist = Infinity + let bestAnchorTs = NaN + if (anchorTs.length > 0 && !Number.isNaN(ts)) { + let lo = 0 + let hi = anchorTs.length - 1 + while (lo < hi) { + const mid = (lo + hi) >> 1 + if (anchorTs[mid]!.ts < ts) lo = mid + 1 + else hi = mid + } + for (const idx of [lo - 1, lo]) { + const a = anchorTs[idx] + if (!a) continue + const d = Math.abs(a.ts - ts) + if (d < bestDist) { + bestDist = d + best = a.turn + bestAnchorTs = a.ts + } + } + } + // Never fold across a local-day boundary: turn-level judgments (category + // cost, edit/one-shot counts) are anchored to the TURN's day while + // call-level totals bucket per call, so folding a 00:05 rollup into a + // 23:55 turn would seal its cost under the earlier day's categories while + // the headline counted it on its own — the two would stop reconciling. + const sameDay = best !== null && localDayKey(bestAnchorTs) === localDayKey(ts) + if (best && bestDist <= FOLD_WINDOW_MS && sameDay) { + best.assistantCalls = [...best.assistantCalls, ...t.assistantCalls] + } else { + kept.push(t) + } + } + return kept +} + // ── Cache-Aware Parsing Helpers ──────────────────────────────────────── // Merge the calls of the last cached turn with the calls parsed from the @@ -2891,7 +3016,30 @@ async function parseProviderSources( } const fp = await fingerprintFile(source.path) - if (!fp) continue + if (!fp) { + // A source that was discovered but cannot be fingerprinted is skipped — + // but skipping is only safe when the file is genuinely GONE (discovery + // raced a deletion; nothing to hydrate). An unreadable-but-present file + // (EACCES/EIO) may hold changes no parser ever got to defer on, so the + // pass must not report full hydration (round-6 finding: an unreadable + // store fingerprint silently bypassed the deferral fence). Network + // sources have no file to be unreadable — their synthetic paths always + // stat ENOENT — and never defer here. Virtual-suffix paths (cursor + // `#…`, opencode `:…`) must be classified against the same underlying + // paths the fingerprint read: the compound path itself always ENOENTs. + if (provider.network) continue + for (const candidate of sourcePathStatCandidates(source.path)) { + const code = await stat(candidate).then( + () => null, + (e: unknown) => (e as NodeJS.ErrnoException).code ?? 'UNKNOWN' + ) + if (code !== 'ENOENT' && code !== 'ENOTDIR') { + deferredRetryableSource = true + break + } + } + continue + } const cached = section.files[source.path] const action = reconcileFile(fp, cached) @@ -2968,7 +3116,11 @@ async function parseProviderSources( // Store/merge parsed turns into the cache. // Durable providers use a union-by-deduplicationKey merge: existing turns // are NEVER deleted (preserves data for spans pruned from the DB), and - // only turns whose dedup keys are not already cached are appended. + // only turns whose dedup keys are not already cached are appended. A + // deliberate consequence: capture-only metadata on an already-cached key + // (copilot nanoAiu/requestMultiplier) is not backfilled by a re-parse. + // Safe because the parse version that admits store rows is the same one + // that captures the metadata, and the CLI never rewrites old rows. // Non-durable providers keep the original overwrite-or-append behaviour. if (provider.durableSources) { const existingEntry = section.files[source.path] @@ -3000,6 +3152,12 @@ async function parseProviderSources( ;(diskCache as { _dirty?: boolean })._dirty = true } catch (err) { if (isSqliteBusyError(err)) { + // Deferred, not failed: the cache keeps serving this source's + // previous rows and the next refresh retries. But the data this + // read would have added is MISSING from this parse, so the run is a + // partial hydration — the daily backfill must not finalize history + // built on it (the same fence a stale read-only serve raises). + deferredRetryableSource = true warnProviderReadFailureOnce(providerName, err) continue } @@ -3040,10 +3198,17 @@ async function parseProviderSources( } // 90-day age-out for durable providers: remove entries whose newest call is - // older than 90 days so the cache doesn't grow unboundedly over time. + // older than 90 days so the cache doesn't grow unboundedly. One scoped + // exemption: a source that declared retainWhilePresent and is still + // discovered IS the durable record itself (copilot's session-store.db) — + // pruning it would drop crash-only rows the file still holds and force a + // full re-read on the next refresh. Everything else — orphans, and ordinary + // still-present journal files — ages out on the pre-existing schedule. if (!readOnly && provider.durableSources) { + const retainPaths = new Set(sources.filter(s => s.retainWhilePresent).map(s => s.path)) const cutoffMs = Date.now() - 90 * 24 * 60 * 60 * 1000 for (const [cachedPath, cachedFile] of Object.entries(section.files)) { + if (retainPaths.has(cachedPath)) continue const newestTs = cachedFile.turns .flatMap(t => t.calls) .map(c => new Date(c.timestamp).getTime()) @@ -3056,6 +3221,224 @@ async function parseProviderSources( } } + // Copilot rollup-vs-store reconciliation, enforced at SERVE time — the sole + // precedence mechanism. Parsers cache both representations of a session + // unconditionally (per-request store rows and the shutdown rollup); the + // serve set is the one coherent snapshot, so deciding here cannot be raced + // by writers between a probe and a parse, and heals any path into the cache. + // Per (session, model): when store rows exist, the rollup calls are dropped + // and replaced by the rows PLUS per-leg residual calls for any usage a + // rollup leg carried beyond the rows in ITS OWN interval — rows commit + // strictly before their leg's shutdown line, so a leg at time T covers + // exactly the rows in (previous leg's T, T], and rows outside that interval + // (a crash tail after the last clean shutdown, a later DB reset) can never + // cancel a different leg's missing usage. A store missing requests a leg + // covered therefore still serves that tail exactly once, on the leg's own + // day; a complete store serves pure per-request granularity with every + // residual at zero. The decision reads only cached contents, never + // discovery: an absent or deleted store changes nothing at serve, so a + // finalized daily history can never flip when the store file comes and + // goes. Cached rows of a deleted store stop influencing results only when + // the 90-day age-out removes them. + // + // The same sweep resolves two identity questions from the full cached data + // so that answers are invariant across date ranges and file churn: + // - Row↔per-turn pairing (behavioral weight): a store row and the per-turn + // call of the same request carry no shared id, so rows pair with same- + // model per-turn calls by timestamp adjacency (monotone two-pointer + // matching, 2-minute window — the two are written moments apart). The + // paired rows' dedup keys become supplementary; unpaired rows are + // store-only requests and keep behavioral weight. Computed over the FULL + // serve set, never a range slice, so adjacent day queries agree with the + // lifetime answer. + // - Project identity: every call of a session serves under the session's + // session-state-derived label when the serve set knows it, else the store + // rows' own label — so neither a store row cached before events.jsonl + // existed nor an events.jsonl orphaned after a prune can split the + // session across two grouping keys. + type CopilotStamped = { ts: number; input: number; cacheRead: number; cacheWrite: number; reasoning: number } + let copilotRecon: { + storeKeys: Set + storeCalls: Map + rollupLegs: Map> + supplementaryStoreKeys: Set + sessionProject: Map + storeProject: Map + nanRollupFallbackTs: Map + sessionEarliestValidTs: Map + } | null = null + if (providerName === 'copilot') { + const seenAggKeys = new Set() + const storeKeys = new Set() + const storeCalls = new Map() + const rollupLegs = new Map>() + const storeRowIds = new Map>() + const perTurnTs = new Map() + const sessionProject = new Map() + const storeProject = new Map() + // Stable timestamp fallbacks for rollup calls whose own stamp cannot + // parse. Stability across serves is load-bearing: the daily union seals + // whatever day the call served under, so the fallback must never move as + // the session grows. The preceding valid timestamp in the SAME file is + // immutable (session files are append-only); the session's EARLIEST + // valid timestamp is the stable backstop (appends only add later ones). + // "Latest valid" would move on every resume and double the call across + // sealed days. + const nanRollupFallbackTs = new Map() + const sessionEarliestValidTs = new Map() + // Session-state per-turn calls carry no per-call project (the serve loop + // takes it from source.project), so resolve it the same way here — every + // call of the session must serve under the SAME label, whichever of the + // representations parsed first or survives on disk. + const sourceProjectByPath = new Map(sources.map(s => [s.path, s.project])) + for (const [cachedPath, cachedFile] of Object.entries(section.files)) { + let lastValidTsInFile = '' + for (const turn of cachedFile.turns) { + const shutdownPrefix = `copilot:${turn.sessionId}:shutdown:` + for (const c of turn.calls) { + if (seenAggKeys.has(c.deduplicationKey)) continue + seenAggKeys.add(c.deduplicationKey) + const ts = new Date(c.timestamp).getTime() + const isStore = c.deduplicationKey.startsWith('copilot-store:') + const isRollup = c.deduplicationKey.startsWith(shutdownPrefix) + const aggKey = `${turn.sessionId}\n${c.model}` + if (!Number.isNaN(ts)) { + lastValidTsInFile = c.timestamp + const prev = sessionEarliestValidTs.get(turn.sessionId) + if (!prev || c.timestamp < prev) sessionEarliestValidTs.set(turn.sessionId, c.timestamp) + } else if (isRollup && lastValidTsInFile) { + nanRollupFallbackTs.set(c.deduplicationKey, lastValidTsInFile) + } + if (!isStore && !isRollup) { + const project = c.project ?? sourceProjectByPath.get(cachedPath) + if (project && !sessionProject.has(turn.sessionId)) sessionProject.set(turn.sessionId, project) + if (!Number.isNaN(ts)) { + const list = perTurnTs.get(aggKey) ?? [] + list.push(ts) + perTurnTs.set(aggKey, list) + } + continue + } + if (Number.isNaN(ts)) continue + const stamped: CopilotStamped = { + ts, + input: c.usage.inputTokens, + cacheRead: c.usage.cacheReadInputTokens, + cacheWrite: c.usage.cacheCreationInputTokens, + reasoning: c.usage.reasoningTokens, + } + if (isStore) { + storeKeys.add(aggKey) + const list = storeCalls.get(aggKey) ?? [] + list.push(stamped) + storeCalls.set(aggKey, list) + const ids = storeRowIds.get(aggKey) ?? [] + ids.push({ ts, dedupKey: c.deduplicationKey }) + storeRowIds.set(aggKey, ids) + if (c.project && !storeProject.has(turn.sessionId)) storeProject.set(turn.sessionId, c.project) + } else { + const legs = rollupLegs.get(aggKey) ?? [] + legs.push({ ...stamped, rawTs: c.timestamp }) + rollupLegs.set(aggKey, legs) + } + } + } + } + // Row↔per-turn pairing: monotone two-pointer matching over the sorted + // timestamp lists. Paired rows are the requests whose per-turn call is + // already served; the unpaired excess — the crash tail, or a whole + // store-only history — keeps its weight. The window is TIGHT (2 minutes): + // a request's row and its assistant.message are written at the same + // completion moment, seconds apart, while a crash-only row sits minutes + // to hours from any unrelated call — a wide window would let it pair + // against a neighbor whose own row is missing and hide the crash + // request's call weight. The residual ambiguity (a crash row landing + // within the window of an unrecorded-row request) is a double-failure + // conjunction and affects only call counts, never tokens. + const PAIR_WINDOW_MS = 2 * 60 * 1000 + const supplementaryStoreKeys = new Set() + for (const [aggKey, ids] of storeRowIds) { + const callTs = perTurnTs.get(aggKey) + if (!callTs?.length) continue + ids.sort((a, b) => a.ts - b.ts) + callTs.sort((a, b) => a - b) + let i = 0 + let j = 0 + while (i < ids.length && j < callTs.length) { + const d = ids[i]!.ts - callTs[j]! + if (Math.abs(d) <= PAIR_WINDOW_MS) { + supplementaryStoreKeys.add(ids[i]!.dedupKey) + i++ + j++ + } else if (d < 0) { + i++ + } else { + j++ + } + } + } + copilotRecon = { storeKeys, storeCalls, rollupLegs, supplementaryStoreKeys, sessionProject, storeProject, nanRollupFallbackTs, sessionEarliestValidTs } + } + const copilotServeProject = (sessionId: string): string | undefined => + copilotRecon + ? copilotRecon.sessionProject.get(sessionId) ?? copilotRecon.storeProject.get(sessionId) + : undefined + const reconcileCopilotCalls = (turn: CachedTurn): CachedTurn | null => { + if (!copilotRecon) return turn + const shutdownPrefix = `copilot:${turn.sessionId}:shutdown:` + let changed = false + const kept: CachedCall[] = [] + for (const c of turn.calls) { + if (c.deduplicationKey.startsWith(shutdownPrefix)) { + const tsValid = !Number.isNaN(new Date(c.timestamp).getTime()) + if (tsValid && copilotRecon.storeKeys.has(`${turn.sessionId}\n${c.model}`)) { + // Store rows exist for this (session, model): the rollup is + // replaced by the rows plus the per-leg residuals synthesized at + // session assembly. + changed = true + continue + } + if (!tsValid) { + // A rollup whose timestamp cannot parse never entered the residual + // sweep, so dropping it would silently lose its usage — it serves + // instead (weightless supplementary). But served with the broken + // stamp it is invisible to every date-range filter (and poisons + // day keys), so it adopts a STABLE valid timestamp: the one + // preceding it in its own append-only file, else the session's + // earliest. Stability matters — a moving fallback (e.g. "latest") + // would relocate the call after a resume, doubling it across an + // already-sealed day and its new one. Only a session with no valid + // timestamp anywhere keeps the raw value. + const fallbackTs = + copilotRecon.nanRollupFallbackTs.get(c.deduplicationKey) ?? + copilotRecon.sessionEarliestValidTs.get(turn.sessionId) + if (fallbackTs) { + kept.push({ ...c, timestamp: fallbackTs }) + changed = true + continue + } + } + kept.push(c) + continue + } + if (c.deduplicationKey.startsWith('copilot-store:')) { + const project = copilotRecon.sessionProject.get(turn.sessionId) + if (project && c.project !== project) { + kept.push({ ...c, project }) + changed = true + continue + } + } + kept.push(c) + } + if (!changed) return turn + if (kept.length === 0) return null + // Re-anchor a turn whose own stamp cannot parse to its first surviving + // call, mirroring turnSlicedToRange — day bucketing reads the turn stamp. + const turnTsValid = !Number.isNaN(new Date(turn.timestamp).getTime()) + return { ...turn, calls: kept, ...(turnTsValid ? {} : { timestamp: kept[0]!.timestamp }) } + } + // Query-time: derive SessionSummary from all cached turns. // Uses seenKeys (shared across providers) for cross-provider dedup. const sessionMap = new Map; title?: string }>() @@ -3064,7 +3447,9 @@ async function parseProviderSources( const cachedFile = section.files[source.path] if (!cachedFile) continue - for (const turn of cachedFile.turns) { + for (const rawTurn of cachedFile.turns) { + const turn = reconcileCopilotCalls(rawTurn) + if (!turn) continue const hasDup = turn.calls.some(c => seenKeys.has(c.deduplicationKey)) if (hasDup) continue @@ -3086,7 +3471,7 @@ async function parseProviderSources( const classified = dateRange ? (classifiedTurnSlicedToRange(classifiedFull, dateRange) ?? classifiedFull) : classifiedFull - const project = slicedTurn.calls[0]?.project ?? source.project + const project = copilotServeProject(turn.sessionId) ?? slicedTurn.calls[0]?.project ?? source.project const key = `${providerName}:${turn.sessionId}:${project}` const existing = sessionMap.get(key) @@ -3121,7 +3506,9 @@ async function parseProviderSources( for (const [cachedPath, cachedFile] of Object.entries(section.files)) { if (allDiscoveredFiles.has(cachedPath)) continue // already counted above - for (const turn of cachedFile.turns) { + for (const rawTurn of cachedFile.turns) { + const turn = reconcileCopilotCalls(rawTurn) + if (!turn) continue const hasDup = turn.calls.some(c => seenKeys.has(c.deduplicationKey)) if (hasDup) continue @@ -3141,7 +3528,10 @@ async function parseProviderSources( const classified = dateRange ? (classifiedTurnSlicedToRange(classifiedFull, dateRange) ?? classifiedFull) : classifiedFull - const project = slicedTurn.calls[0]?.project ?? providerName + // Orphaned files lose their source.project; for copilot the recon + // maps restore the session's label so an events.jsonl pruned while + // its store rows live on cannot split the session (round-6 finding). + const project = copilotServeProject(turn.sessionId) ?? slicedTurn.calls[0]?.project ?? providerName const key = `${providerName}:${turn.sessionId}:${project}` const existingEntry = sessionMap.get(key) @@ -3157,11 +3547,121 @@ async function parseProviderSources( } } + // Copilot residuals: for every (session, model) where BOTH representations + // exist, each rollup LEG subtracts only the store rows in its own interval + // (previous leg's timestamp, its own timestamp] — rows commit strictly + // before their leg's shutdown line, so rows outside the interval (a crash + // tail, a later same-path reset) can never cancel a different leg's + // missing usage — and any remainder serves once as a supplementary call + // anchored at that leg's own timestamp, keeping the tail on the day the + // leg actually recorded it. Subject to the same inclusive date-range rule + // as the rollup it stands in for. Derived purely from cached contents, so + // it is identical across refresh and read-only serves and each leg's + // residual shrinks monotonically as later parses append the rows it stood + // in for. + if (copilotRecon) { + const residualsBySession = new Map() + for (const [aggKey, legs] of copilotRecon.rollupLegs) { + const rows = copilotRecon.storeCalls.get(aggKey) + if (!rows?.length) continue + const nl = aggKey.indexOf('\n') + const sessionId = aggKey.slice(0, nl) + const model = aggKey.slice(nl + 1) + legs.sort((a, b) => a.ts - b.ts) + // Legs sharing one timestamp have no interval between them — the + // strict (prev, ts] rule would hand all their rows to the first and + // mint a full-delta residual for the second, double-counting. Coalesce + // them into one leg so the shared instant subtracts its rows once. + const coalesced: Array = [] + for (const leg of legs) { + const last = coalesced[coalesced.length - 1] + if (last && last.ts === leg.ts) { + last.input += leg.input + last.cacheRead += leg.cacheRead + last.cacheWrite += leg.cacheWrite + last.reasoning += leg.reasoning + } else { + coalesced.push({ ...leg }) + } + } + rows.sort((a, b) => a.ts - b.ts) + let rowIdx = 0 + let prevLegTs = -Infinity + for (let legIdx = 0; legIdx < coalesced.length; legIdx++) { + const leg = coalesced[legIdx]! + const covered = { input: 0, cacheRead: 0, cacheWrite: 0, reasoning: 0 } + while (rowIdx < rows.length && rows[rowIdx]!.ts <= leg.ts) { + const row = rows[rowIdx]! + if (row.ts > prevLegTs) { + covered.input += row.input + covered.cacheRead += row.cacheRead + covered.cacheWrite += row.cacheWrite + covered.reasoning += row.reasoning + } + rowIdx++ + } + prevLegTs = leg.ts + const input = Math.max(0, leg.input - covered.input) + const cacheRead = Math.max(0, leg.cacheRead - covered.cacheRead) + const cacheWrite = Math.max(0, leg.cacheWrite - covered.cacheWrite) + const reasoning = Math.max(0, leg.reasoning - covered.reasoning) + if (input === 0 && cacheRead === 0 && cacheWrite === 0 && reasoning === 0) continue + if (dateRange) { + const ts = new Date(leg.rawTs) + if (Number.isNaN(ts.getTime()) || ts < dateRange.start || ts > dateRange.end) continue + } + const calls = residualsBySession.get(sessionId) ?? [] + calls.push({ + provider: 'copilot', + model, + usage: { inputTokens: input, outputTokens: 0, cacheCreationInputTokens: cacheWrite, cacheReadInputTokens: cacheRead, cachedInputTokens: 0, reasoningTokens: reasoning, webSearchRequests: 0 }, + costUSD: calculateCost(model, input, 0, cacheWrite, cacheRead, 0), + tools: [], mcpTools: [], skills: [], subagentTypes: [], + hasAgentSpawn: false, hasPlanMode: false, + speed: 'standard', timestamp: leg.rawTs, bashCommands: [], + deduplicationKey: `copilot:${sessionId}:shutdown-residual:${model}:${legIdx}`, + supplementaryAccounting: true, + }) + residualsBySession.set(sessionId, calls) + } + } + for (const [sessionId, calls] of residualsBySession) { + const project = copilotServeProject(sessionId) ?? providerName + const mapKey = `${providerName}:${sessionId}:${project}` + // One turn PER LEG timestamp: a session's residuals can span days, and + // a single container turn anchored at the first leg would let the fold + // drag a later leg's category cost onto an earlier day's turn. + const byTs = new Map() + for (const call of calls) { + const list = byTs.get(call.timestamp) ?? [] + list.push(call) + byTs.set(call.timestamp, list) + } + const existing = sessionMap.get(mapKey) + const target = existing ?? { project, turns: [] as ClassifiedTurn[] } + for (const [ts, tsCalls] of byTs) { + target.turns.push({ + userMessage: '', + assistantCalls: tsCalls, + timestamp: ts, + sessionId, + category: 'general', + retries: 0, + hasEdits: false, + }) + } + if (!existing) sessionMap.set(mapKey, target) + } + } + const projectMap = new Map() for (const [key, { project, projectPath, workingDirectory, turns, prLinks, title }] of sessionMap) { const sessionId = key.split(':')[1] ?? key - const session = buildSessionSummary(sessionId, project, turns) - const explicitLinks = new Set(turns.flatMap(turn => turn.prRefs ?? [])) + const assembledTurns = providerName === 'copilot' + ? foldCopilotSupplementaryTurns(sessionId, turns, copilotRecon?.supplementaryStoreKeys) + : turns + const session = buildSessionSummary(sessionId, project, assembledTurns) + const explicitLinks = new Set(assembledTurns.flatMap(turn => turn.prRefs ?? [])) for (const link of prLinks ?? []) explicitLinks.add(link) if (explicitLinks.size) { session.prLinks = [...explicitLinks].sort() @@ -3169,7 +3669,9 @@ async function parseProviderSources( } if (workingDirectory) session.workingDirectory = workingDirectory if (title) session.title = title - if (session.apiCalls > 0) { + // Supplementary-only sessions (e.g. a rollup with no per-turn calls) have + // apiCalls 0 by design but their tokens/cost are real and must serve. + if (session.apiCalls > 0 || session.totalCostUSD > 0 || session.totalInputTokens + session.totalOutputTokens + session.totalCacheReadTokens + session.totalCacheWriteTokens + session.totalReasoningTokens > 0) { const existing = projectMap.get(project) if (existing) { existing.sessions.push(session) @@ -3190,7 +3692,7 @@ async function parseProviderSources( const CACHE_TTL_MS = 180_000 const MAX_CACHE_ENTRIES = 10 -const sessionCache = new Map() +const sessionCache = new Map() // Burst reuse for a resident process (codeburn serve). Every payload command // anchors its range end at its own `new Date()`, so two panel fetches issued @@ -3232,6 +3734,7 @@ function burstReuse(dateRange: DateRange, sig: string): ProjectSummary[] | null const validatedClean = parseReuseValidator !== null && age <= VALIDATED_REUSE_CAP_MS && parseReuseValidator(entry.ts) if (!insideBurst && !validatedClean) continue if (endMs < entry.endMs || endMs - entry.endMs > Math.max(windowMs, validatedClean ? VALIDATED_REUSE_CAP_MS : 0)) continue + if (entry.hydrationComplete !== undefined) sessionHydrationComplete = entry.hydrationComplete return filterProjectsByDateRange(entry.data, dateRange) } return null @@ -3264,7 +3767,11 @@ function cachePut(key: string, data: ProjectSummary[]) { const oldest = [...sessionCache.entries()].sort((a, b) => a[1].ts - b[1].ts)[0] if (oldest) sessionCache.delete(oldest[0]) } - sessionCache.set(key, { data, ts: now, ...(putMeta ?? {}) }) + // The hydration verdict is a property OF this result: a memo/burst hit + // must restore the verdict its data was parsed under, or a stale partial + // parse could be served while a later, unrelated parse's `true` lets the + // daily backfill seal history around the gap (round-6 finding). + sessionCache.set(key, { data, ts: now, hydrationComplete: sessionHydrationComplete, ...(putMeta ?? {}) }) putMeta = null } @@ -3707,10 +4214,18 @@ export function isSessionHydrationComplete(): boolean { // chart (gapStart = lastComputedDate + 1 never looks back at them). let readOnlyServedStale = false +// Set when a changed source's read was deferred on a retryable failure (e.g. +// a SQLITE_BUSY store): the parse completed but did not hydrate that source's +// new data, so the run must not report hydration complete even in write mode. +let deferredRetryableSource = false + export async function parseAllSessions(dateRange?: DateRange, providerFilter?: string): Promise { const key = cacheKey(dateRange, providerFilter) const cached = sessionCache.get(key) - if (cached && Date.now() - cached.ts < CACHE_TTL_MS) return cached.data + if (cached && Date.now() - cached.ts < CACHE_TTL_MS) { + if (cached.hydrationComplete !== undefined) sessionHydrationComplete = cached.hydrationComplete + return cached.data + } // The signature is the key minus the range: what must match for a burst // reuse (provider, config env, proxy hash) regardless of the now-anchor. const burstSig = cacheKey(undefined, providerFilter) @@ -3784,6 +4299,7 @@ async function runParse( ): Promise { const { isCold = false, readOnly = false, refreshLock } = options readOnlyServedStale = false + deferredRetryableSource = false const seenMsgIds = new Set() const seenKeys = new Set() const allSources = await discoverAllSessions(providerFilter) @@ -3900,9 +4416,10 @@ async function runParse( } } // Assigned, not forced true: a read-only run that had to skip or stale real - // files reached the end of the scan without hydrating everything, and the - // daily backfill must not finalize history off it. - sessionHydrationComplete = !readOnly || !readOnlyServedStale + // files, or a write run that deferred a changed source on a retryable + // failure, reached the end of the scan without hydrating everything, and + // the daily backfill must not finalize history off it. + sessionHydrationComplete = (!readOnly || !readOnlyServedStale) && !deferredRetryableSource // Merge across providers by normalised project path so the same repository // is not double-counted when it was worked on with more than one tool diff --git a/src/plan-usage.ts b/src/plan-usage.ts index 648d3c91..85444124 100644 --- a/src/plan-usage.ts +++ b/src/plan-usage.ts @@ -1,3 +1,4 @@ +import { behavioralCallCount } from './behavioral-weight.js' import { readPlans, type Plan, type PlanMap } from './config.js' import { parseAllSessions } from './parser.js' import { PLAN_PROVIDERS } from './plans.js' @@ -129,7 +130,7 @@ export function getPlanUsageFromProjects(plan: Plan, projects: ProjectSummary[], } } -function getPlanScopedProjects(plan: Plan, projects: ProjectSummary[], today: Date): ProjectSummary[] { +export function getPlanScopedProjects(plan: Plan, projects: ProjectSummary[], today: Date): ProjectSummary[] { const { periodStart } = computePeriodFromResetDay(plan.resetDay, today) const provider = plan.provider @@ -156,14 +157,17 @@ function getPlanScopedProjects(plan: Plan, projects: ProjectSummary[], today: Da (sum, turn) => sum + turn.assistantCalls.reduce((turnSum, call) => turnSum + call.costUSD, 0), 0, ) - const apiCalls = turns.reduce((sum, turn) => sum + turn.assistantCalls.length, 0) - return apiCalls > 0 ? { ...session, turns, totalCostUSD, apiCalls } : null + const apiCalls = turns.reduce((sum, turn) => sum + behavioralCallCount(turn.assistantCalls), 0) + // Keep on cost as well as calls: a copilot rollup-only session has + // zero behavioral calls but real spend, and dropping it would erase + // that spend from the plan window. + return apiCalls > 0 || totalCostUSD > 0 ? { ...session, turns, totalCostUSD, apiCalls } : null }) .filter((session): session is NonNullable => session !== null) const totalCostUSD = sessions.reduce((sum, session) => sum + session.totalCostUSD, 0) const totalApiCalls = sessions.reduce((sum, session) => sum + session.apiCalls, 0) - return totalApiCalls > 0 ? { ...project, sessions, totalCostUSD, totalApiCalls } : null + return totalApiCalls > 0 || totalCostUSD > 0 ? { ...project, sessions, totalCostUSD, totalApiCalls } : null }) .filter((project): project is NonNullable => project !== null) } diff --git a/src/providers/copilot.ts b/src/providers/copilot.ts index e472fdb7..e48760a0 100644 --- a/src/providers/copilot.ts +++ b/src/providers/copilot.ts @@ -40,6 +40,7 @@ // CODEBURN_COPILOT_WS_STORAGE_DIR — Override VS Code workspaceStorage // CODEBURN_COPILOT_GLOBAL_STORAGE_DIR — Override VS Code globalStorage // CODEBURN_COPILOT_JETBRAINS_DIR — Override the JetBrains github-copilot root +// CODEBURN_COPILOT_SESSION_STORE_DB — Override the ~/.copilot/session-store.db path // // ARCHITECTURE: // discoverSessions() returns OTel sessions and legacy JSONL sessions. When @@ -262,6 +263,10 @@ function getCopilotSessionStateDir(override?: string): string { return override ?? process.env['CODEBURN_COPILOT_SESSION_STATE_DIR'] ?? join(homedir(), '.copilot', 'session-state') } +function getSessionStoreDbPath(override?: string): string { + return override ?? process.env['CODEBURN_COPILOT_SESSION_STORE_DB'] ?? join(homedir(), '.copilot', 'session-store.db') +} + /** * Locate the agent-traces.db file. * @@ -837,12 +842,31 @@ function createJsonlParser( // (and its cost) is not double-counted. Combined with the per-turn // output cost, this yields the full, CLI-measured session cost. if (isTranscript) continue + // When session-store.db holds per-request usage rows for this + // session, those rows are authoritative for input/cache: written + // per request instead of only on clean shutdown, and they describe + // the SAME tokens this rollup lumps together. That precedence is + // enforced at SERVE time, not here: this rollup is always parsed + // and cached, and the reconciliation in parseProviderSources + // decides per (session, model) — store rows replace the rollup, + // with any usage the rollup carried beyond the rows' sum served as + // a residual (see reconcileCopilotCalls there). Read-time + // precedence over one coherent serve set cannot be raced by + // writers between a coverage probe and this parse, and a briefly + // unreadable store never blocks this file. const shutdownData = event.data as SessionShutdownData const modelMetrics = shutdownData.modelMetrics if (!isRecord(modelMetrics)) continue + // Fallback order matters for accounting, not just display: this + // stamp anchors the leg's interval in the serve-time reconciliation, + // which subtracts the store rows written up to it. A shutdown + // happens at the END of a leg, so the last event seen is the + // nearest true anchor; sessionStartTime is BEFORE every row, and + // anchoring there would leave the leg covering nothing and re-mint + // its whole usage as a residual beside the rows it duplicates. const shutdownTimestamp = - (event.timestamp ?? '') || timestampToISO(shutdownData.sessionStartTime) || lastEventTimestamp + (event.timestamp ?? '') || lastEventTimestamp || timestampToISO(shutdownData.sessionStartTime) for (const [model, metrics] of Object.entries(modelMetrics)) { if (!model || !isRecord(metrics)) continue @@ -865,7 +889,14 @@ function createJsonlParser( // its counters (a fresh accounting epoch): delta from zero, else // this leg's post-reset usage would be clamped away entirely. // inputTokens is the monotonic sentinel — it is cache-inclusive, - // so any usage at all grows it. + // so any usage at all grows it. In-session COMPACTION is a + // confirmed reset trigger (CLI 1.0.78: a clean single-process + // 107-request session's sole rollup covered exactly its five + // post-compaction requests), so rollup-only accounting + // undercounts any compacted session — usage between the last + // pre-reset rollup and the reset is simply never written here. + // Only the per-request session-store rows record it; that is why + // they are authoritative for covered sessions. const prev = prevRaw && cumulative.inputTokens < numberOrZero(prevRaw.inputTokens) ? undefined @@ -1866,7 +1897,7 @@ function createOtelParser( outputTokens, cacheCreationTokens, cacheReadTokens, - 0 // reasoningTokens — not exposed in current OTel schema + 0 // webSearchRequests — not applicable to OTel spans ) yield { @@ -1899,6 +1930,273 @@ function createOtelParser( } } +// --------------------------------------------------------------------------- +// Session-store SQLite parser — per-request usage rows from session-store.db +// --------------------------------------------------------------------------- +// +// The Copilot CLI and the GitHub Copilot desktop app both write +// ~/.copilot/session-store.db unconditionally. Its assistant_usage_events +// table records one row per API request AS IT HAPPENS, where the +// session.shutdown rollup in events.jsonl is written only on clean shutdown +// (a crash loses the whole session's input/cache accounting) and lumps a +// session leg into one per-model total. The DB rows are therefore +// authoritative for input/cache tokens; the serve-time reconciliation in +// parseProviderSources replaces the covered (session, model) rollup calls +// with the rows plus a residual for anything the rollup carried beyond them. +// +// The emitted calls mirror the shutdown-call contract exactly: input/cache/ +// reasoning only, output 0 — per-turn output (and its tools/userMessage +// metadata) stays owned by the events.jsonl assistant.message calls, so +// emitting output here would double-count it. The per-request billing +// metadata (total_nano_aiu, request_multiplier) is captured onto the cached +// calls but not priced or displayed — that design is upstream #890; the +// throughput/latency columns are deliberately not read yet. + +// The one REQUIRED usage query, shared verbatim between the discovery probe +// and the parser so the two can never diverge on schema: discovery runs it +// LIMIT 1 (prepare validates every table and column it touches) before +// emitting the source, so a store whose shape the parser cannot read is +// classified absent — its sessions keep their shutdown rollups — instead of +// surfacing a source that could only ever fail. +const SESSION_STORE_USAGE_COLUMNS = `e.id, e.session_id, e.model, + e.input_tokens, e.cache_read_tokens, e.cache_write_tokens, + e.reasoning_tokens, e.created_at, + s.cwd, s.repository, s.created_at AS session_created_at` +const SESSION_STORE_USAGE_FROM = ` + FROM assistant_usage_events e + LEFT JOIN sessions s ON s.id = e.session_id` +const SESSION_STORE_USAGE_SELECT = `SELECT ${SESSION_STORE_USAGE_COLUMNS}${SESSION_STORE_USAGE_FROM}` + +// OPTIONAL enrichment: the billing-metadata columns, tried first at parse +// time and never probed at discovery — older CLI stores predate them, and +// requiring them would classify a perfectly readable store as absent. A +// `no such column` failure falls back to the base select above, so an old +// store parses identically, just without the metadata. +const SESSION_STORE_USAGE_SELECT_BILLING = `SELECT ${SESSION_STORE_USAGE_COLUMNS}, + e.total_nano_aiu, e.request_multiplier${SESSION_STORE_USAGE_FROM}` + +// Type alias, not interface: db.query's Row constraint needs the implicit +// index signature only anonymous object types carry. +type SessionStoreUsageRow = { + id: number + session_id: string + model: string + input_tokens: number | null + cache_read_tokens: number | null + cache_write_tokens: number | null + reasoning_tokens: number | null + created_at: string | null + cwd: string | null + repository: string | null + session_created_at: string | null + // Present only when the billing select succeeded (schema has the columns). + total_nano_aiu?: number | null + request_multiplier?: number | null +} + +// FNV-1a 64-bit over the row's identifying content, base36. Collisions only +// matter between two rows sharing the SAME session_id and row id — i.e. a +// same-path DB reset that happens to reuse an id — where the content strings +// differ; 64 bits keeps even adversarial token tuples from aliasing (32-bit +// FNV collisions between plausible tuples are constructible). +function fnv1a64(s: string): string { + let h = 0xcbf29ce484222325n + const prime = 0x100000001b3n + const mask = 0xffffffffffffffffn + for (let i = 0; i < s.length; i++) { + h ^= BigInt(s.charCodeAt(i)) + h = (h * prime) & mask + } + return h.toString(36) +} + +function createSessionStoreParser( + source: SessionStoreSessionSource, + seenKeys: Set +): SessionParser { + return { + async *parse(): AsyncGenerator { + // Lazy-load the SQLite module (same pattern as the OTel source) + const { openDatabase, isSqliteBusyError } = await import('../sqlite.js') + + // The open sits inside the same classify-and-defer boundary as the + // query: discovery validated this store moments ago, so a failure HERE + // (EACCES/CANTOPEN/EMFILE race) is transient-shaped — letting it + // propagate raw would cache a failed marker at the current fingerprint + // and zero the covered sessions until the file next changes. + let db: ReturnType + try { + db = openDatabase(source.path) + } catch (err) { + if (isSqliteBusyError(err)) throw err + throw Object.assign( + new Error('copilot session-store.db unreadable at open; deferring'), + { code: 'SQLITE_BUSY' } + ) + } + try { + let rows: SessionStoreUsageRow[] + try { + try { + rows = db.query(`${SESSION_STORE_USAGE_SELECT_BILLING} ORDER BY e.id ASC`) + } catch (billingErr) { + // Only the optional billing columns may be missing (older store + // schema): fall back to the discovery-validated base select. + // Anything else re-throws into the defer classification below. + const msg = billingErr instanceof Error ? billingErr.message : String(billingErr) + if (!/no such column/i.test(msg)) throw billingErr + rows = db.query(`${SESSION_STORE_USAGE_SELECT} ORDER BY e.id ASC`) + } + } catch (err) { + // Discovery prepare-validated this exact query moments ago, so any + // failure here means the store became unreadable or changed shape + // mid-run. Yielding nothing would cache an EMPTY success at this + // fingerprint while the covered sessions' rollups stay suppressed + // — a silent under-count that persists until the file changes. + // Every failure defers instead: parseProviderSources + // skips-and-retries on the busy shape without writing the cache. + if (isSqliteBusyError(err)) throw err + throw Object.assign( + new Error('copilot session-store.db unreadable mid-parse; deferring'), + { code: 'SQLITE_BUSY' } + ) + } + + // created_at defaults to SQLite's datetime('now') — UTC but + // timezone-less ('2026-08-07 17:56:38', or with fractional seconds + // under 'subsec'), which Date.parse reads as LOCAL time and would + // shift the request onto the wrong day. The CLI writes explicit + // ISO-Z strings (audited: every observed row), so this normalizes + // only the defensive zoneless shapes to UTC; anything carrying its + // own zone/offset passes through untouched. + const normalizeTimestamp = (raw: string | null): string => + raw + ? timestampToISO( + /^\d{4}-\d{2}-\d{2}[ T]\d{2}:\d{2}:\d{2}(\.\d+)?$/.test(raw) + ? raw.replace(' ', 'T') + 'Z' + : raw + ) + : '' + let prevTimestamp = '' + + for (const row of rows) { + if (!row.session_id) continue + + // A call with an empty timestamp is invisible to every date-range + // filter, so never emit one: fall back from the row's own + // created_at to the previous row's timestamp, then to the + // session's created_at. The previous row deliberately outranks the + // session's own created_at even across sessions — ids are GLOBALLY + // insertion-ordered, so the previous row is the nearest earlier + // clock reading, while a resumed session's created_at can be days + // stale. Both columns carry SQLite defaults, so an empty chain is + // unreachable outside a hand-built store; such a row is skipped, + // and if that desert covers a whole session its rollup simply + // stays unsuppressed at serve time. + const timestamp = + normalizeTimestamp(row.created_at) || + prevTimestamp || + normalizeTimestamp(row.session_created_at) + if (!timestamp) continue + prevTimestamp = timestamp + // TEXT NOT NULL still admits '': a billable row must NEVER be + // dropped for an unnameable model — its session's rollup was + // suppressed on the promise that every billable row is emitted + // (the coverage predicate does not know about models). Price as + // 'unknown' instead; the pricing engine reports unknown models at + // $0 with a fix-it hint rather than silently losing the tokens. + const model = row.model || 'unknown' + + const cacheReadTokens = numberOrZero(row.cache_read_tokens) + const cacheWriteTokens = numberOrZero(row.cache_write_tokens) + const reasoningTokens = numberOrZero(row.reasoning_tokens) + // input_tokens is cache-INCLUSIVE (input + cache_read + cache_write), + // the same convention the shutdown rollup uses — confirmed against + // token_details_json, whose tokenType:"input" entries hold exactly + // this difference. calculateCost expects the uncached remainder with + // cache tokens billed separately, so subtract; clamp guards a future + // schema that reports input non-inclusively. + const inputTokens = Math.max( + 0, + numberOrZero(row.input_tokens) - cacheReadTokens - cacheWriteTokens + ) + + // Nothing this call would add over the per-turn events (output is + // intentionally excluded), so skip it to avoid an empty $0 row. + if (inputTokens === 0 && cacheReadTokens === 0 && cacheWriteTokens === 0 && reasoningTokens === 0) continue + + // `id` is AUTOINCREMENT: stable across re-parses and never reused + // WITHIN one database lifetime — but recreating the DB at the same + // path restarts the sequence, and a bare `:` key would then + // make the durable union swallow the new row as "already cached" + // while its usage differs. A content discriminator (raw created_at + // + token counts + model) makes a genuinely different request under + // a reused id a NEW key; a byte-identical re-insert (backup restore, + // VACUUM INTO) still collapses to the same key. + const dedupKey = `copilot-store:${row.session_id}:${row.id}:${fnv1a64( + `${row.created_at ?? ''}|${row.input_tokens ?? ''}|${row.cache_read_tokens ?? ''}|${row.cache_write_tokens ?? ''}|${row.reasoning_tokens ?? ''}|${row.model}` + )}` + if (seenKeys.has(dedupKey)) continue + seenKeys.add(dedupKey) + + // One DB spans every project, so the project must ride each call + // (the per-source fallback would lump them all together). Prefer + // the label discovery derived from the session's own session-state + // dir (workspace.yaml cwd — the same one its per-turn output calls + // carry, so the session never splits across two projects); the + // store's sessions.cwd/repository names only sessions with no + // session-state dir on this machine. + const project = + source.projectsBySessionId?.get(row.session_id) ?? + (row.cwd + ? basename(row.cwd) + : row.repository + ? basename(row.repository.replace(/\.git$/, '')) + : row.session_id) + + // Tokens are real per-request counts written by the CLI, so this + // cost is measured, not char-estimated. reasoning_tokens rides as + // metadata only, never as a cost line: it is a SUBSET of the row's + // output_tokens (the row's own token_details_json prices exactly + // input/cache_read/cache_write/output, no reasoning entry), and + // output — reasoning included — is billed by the per-turn + // assistant.message call. Pricing reasoning here would double-count. + const costUSD = calculateCost(model, inputTokens, 0, cacheWriteTokens, cacheReadTokens, 0) + + yield { + provider: 'copilot', + sessionId: row.session_id, + project, + model, + inputTokens, + outputTokens: 0, + cacheCreationInputTokens: cacheWriteTokens, + cacheReadInputTokens: cacheReadTokens, + cachedInputTokens: 0, + reasoningTokens, + webSearchRequests: 0, + costUSD, + costIsEstimated: false, + tools: [], + bashCommands: [], + timestamp, + speed: 'standard' as const, + deduplicationKey: dedupKey, + userMessage: '', + // Billing metadata rides as capture-only fields (deliberately + // OUTSIDE the dedup-key content hash: it identifies a charge, not + // the request). Omitted when the schema predates the columns. + ...(typeof row.total_nano_aiu === 'number' ? { nanoAiu: row.total_nano_aiu } : {}), + ...(typeof row.request_multiplier === 'number' ? { requestMultiplier: row.request_multiplier } : {}), + } + } + } finally { + db.close() + } + }, + } +} + // --------------------------------------------------------------------------- // Extended SessionSource for OTel sessions // --------------------------------------------------------------------------- @@ -1912,6 +2210,19 @@ interface JsonlSessionSource extends SessionSource { sourceType: 'jsonl' } +// The Copilot CLI / GitHub desktop-app session store (~/.copilot/session-store.db). +// One source per DB file; the parser iterates every session's usage rows in a +// single DB open, mirroring the OTel source. +interface SessionStoreSessionSource extends SessionSource { + sourceType: 'session-store' + // sessionId → project label derived from the session-state dirs + // (workspace.yaml cwd), attached at discovery. The store's own + // sessions.cwd can lag or miss what the session actually ran in, and the + // per-turn output calls already carry the jsonl-derived label — using the + // same one keeps a session's store rows and output calls in one session. + projectsBySessionId?: Map +} + // A VS Code workspaceStorage transcript. Distinct from 'jsonl' (CLI // session-state) so classification rides provenance, not file contents (#944). interface TranscriptSessionSource extends SessionSource { @@ -1959,6 +2270,10 @@ function isTranscriptSource(source: SessionSource): source is TranscriptSessionS return (source as TranscriptSessionSource).sourceType === 'transcript' } +function isSessionStoreSource(source: SessionSource): source is SessionStoreSessionSource { + return (source as SessionStoreSessionSource).sourceType === 'session-store' +} + // --------------------------------------------------------------------------- // Session discovery: JSONL (original) // --------------------------------------------------------------------------- @@ -2020,6 +2335,69 @@ async function discoverOtelSessions( return [{ path: dbPath, project: 'copilot-chat', provider: 'copilot', sourceType: 'otel' }] } +// --------------------------------------------------------------------------- +// Session discovery: session-store SQLite +// --------------------------------------------------------------------------- + +/** + * Probe session-store.db. This decides only whether a store SOURCE exists; + * which sessions it covers is decided at serve time from what its parse + * actually cached (see reconcileCopilotCalls in parseProviderSources), so + * nothing a writer does between this probe and the parse can change + * accounting. + * + * Permanent absence — no file, no sqlite driver, or a schema the parser's + * own query cannot prepare against ("no such table": CLI builds before the + * store existed; "no such column": a future migration) — returns null: no + * source, no suppression, and the shutdown-rollup path carries the sessions + * exactly as before. + * + * EVERY other failure still emits the source. Measured against the real + * driver (node:sqlite, WAL store): a write lock never blocks readers and a + * hot -wal without its -shm reads fine, so the reachable failures here are + * corruption-class — SQLITE_CORRUPT (11), SQLITE_NOTADB (26, e.g. mid + * atomic-replace), SQLITE_CANTOPEN (14, deleted after the stat) — plus the + * classic busy/locked pair and stat-level EACCES/EIO. None of those prove + * the store is gone, so the path must stay discovered: the parse raises the + * busy shape parseProviderSources skips-and-retries, previously cached rows + * keep serving, and serve-time suppression keeps holding from the cache + * instead of flapping the covered sessions' rollups back in. + */ +async function discoverSessionStoreSource( + dbPath: string +): Promise { + const source: SessionStoreSessionSource = { + path: dbPath, + project: 'copilot', + provider: 'copilot', + sourceType: 'session-store', + // The store IS the durable record: while it sits on disk, its cached rows + // must never age out (crash-only rows have no rollup to fall back to). + // Journal-style sources keep the ordinary durable age-out. + retainWhilePresent: true, + } + try { + await stat(dbPath) + } catch (err) { + const code = (err as NodeJS.ErrnoException).code + return code === 'ENOENT' || code === 'ENOTDIR' ? null : source + } + const { openDatabase, isSqliteAvailable } = await import('../sqlite.js') + if (!isSqliteAvailable()) return null + try { + const db = openDatabase(dbPath) + try { + db.query(`${SESSION_STORE_USAGE_SELECT} LIMIT 1`) + } finally { + db.close() + } + return source + } catch (err) { + const message = err instanceof Error ? err.message : String(err) + return /no such (table|column)/i.test(message) ? null : source + } +} + // --------------------------------------------------------------------------- // Session discovery: JetBrains (IntelliJ IDEA, PyCharm, …) // --------------------------------------------------------------------------- @@ -2373,7 +2751,8 @@ export function createCopilotProvider( sessionStateDir?: string, workspaceStorageDir?: string, globalStorageDir?: string, - jetbrainsDir?: string + jetbrainsDir?: string, + sessionStoreDb?: string ): Provider { // jsonlDir is resolved lazily inside discoverSessions so that env-var // overrides set after module load (e.g. in tests) are respected. @@ -2434,10 +2813,41 @@ export function createCopilotProvider( } } + // 1b. Discover the CLI / GitHub desktop-app session store. Written per + // API request (crash-proof) where the events.jsonl shutdown rollup + // exists only after a clean exit, so its rows are authoritative for + // input/cache tokens; the serve-time reconciliation + // (reconcileCopilotCalls in parseProviderSources) replaces the covered + // (session, model) rollups with the served rows plus a residual. True + // absence (older CLI schema, no sqlite driver) leaves the rollup path + // untouched; an unreadable store still surfaces the source so its + // parse defers and cached rows keep serving (see + // discoverSessionStoreSource). + let storeSource: SessionStoreSessionSource | null = null + try { + storeSource = await discoverSessionStoreSource(getSessionStoreDbPath(sessionStoreDb)) + } catch { + // Unreachable in practice (the probe catches its own errors): a + // throw here means the sqlite module itself is unusable, and + // rollup-only accounting is then the correct mode — the same + // fallback as a missing driver. + storeSource = null + } + if (storeSource) sources.push(storeSource) + // 2. Discover JSONL sessions (fallback — output tokens only) try { const jsonlDir = getCopilotSessionStateDir(sessionStateDir) const jsonlSources = await discoverJsonlSessions(jsonlDir) + if (storeSource) { + // Same sessionId derivation as createJsonlParser: the CLI keys + // session-state dirs and session-store rows by the same id, so the + // store parser can attribute each session's rows to the same + // project its per-turn output calls carry. + storeSource.projectsBySessionId = new Map( + jsonlSources.map(src => [basename(dirname(src.path)), src.project]) + ) + } sources.push(...jsonlSources) } catch { // JSONL discovery failed @@ -2497,6 +2907,9 @@ export function createCopilotProvider( if (isOtelSource(source)) { return createOtelParser(source, seenKeys) } + if (isSessionStoreSource(source)) { + return createSessionStoreParser(source, seenKeys) + } if (isChatSessionSource(source)) { return createChatSessionParser(source, seenKeys) } diff --git a/src/providers/types.ts b/src/providers/types.ts index 8f9e902c..cd5b640a 100644 --- a/src/providers/types.ts +++ b/src/providers/types.ts @@ -8,6 +8,14 @@ export type SessionSource = { sourceLabel?: string sourcePath?: string sourceKind?: 'claude-config' | 'claude-desktop' + // Durable providers only: while this exact path is still discovered, its + // cached entry is exempt from the 90-day age-out — the file IS the durable + // record (copilot's session-store.db), so pruning it would drop crash-only + // rows the source still holds and force a full re-read on the next refresh. + // Orphaned paths (no longer discovered) age out normally, and unflagged + // durable sources age out even while their file remains on disk (the cap + // bounds cache growth for provider-pruned journals). + retainWhilePresent?: boolean } export type SessionParser = { @@ -44,6 +52,13 @@ export type ParsedProviderCall = { locAdded?: number locRemoved?: number editFailed?: number + // Copilot session-store billing metadata, captured only — no report consumes + // these yet (pricing/display design is upstream #890). total_nano_aiu is the + // request's charged AI-credit amount in nano-AIU (1e9 nano-AIU = 1 credit = + // $0.01); request_multiplier is the model's plan multiplier. Captured now so + // a future consumer needs no re-parse of rows the CLI may prune meanwhile. + nanoAiu?: number + requestMultiplier?: number turnId?: string toolSequence?: ToolCall[][] userMessage: string @@ -74,9 +89,11 @@ export type Provider = { // fingerprinted or incrementally cached, so the parser re-fetches every run. network?: boolean // Source data is managed by an external process that may prune old records - // (e.g. VS Code's OTel agent-traces.db). Cached entries for discovered paths - // are never evicted, and orphaned entries (paths no longer discovered) are - // kept and included in query-time aggregation so the monthly total never drops. + // (e.g. VS Code's OTel agent-traces.db). Cached entries are never evicted on + // ordinary refreshes, and orphaned entries (paths no longer discovered) are + // kept and included in query-time aggregation so the monthly total never + // drops. All entries are subject to the 90-day age-out unless their source + // declares retainWhilePresent (see SessionSource). durableSources?: boolean modelDisplayName(model: string): string toolDisplayName(rawTool: string): string diff --git a/src/session-cache.ts b/src/session-cache.ts index 2759520f..5be1c620 100644 --- a/src/session-cache.ts +++ b/src/session-cache.ts @@ -54,6 +54,11 @@ export type CachedCall = { activeDurationMs?: number activeGeneratedTokens?: number toolWaitMs?: number + // Copilot session-store billing metadata (capture-only; no report consumes + // these yet — see ParsedProviderCall). Omitted when the store's schema + // predates the columns. + nanoAiu?: number + requestMultiplier?: number } export type CachedTurn = { @@ -189,6 +194,15 @@ const TEMP_FILE_MAX_AGE_MS = 5 * 60 * 1000 // that only the cache still holds (see DURABLE_PROVIDER_NAMES below). Do not // "complete" the map for copilot until the durable carry-forward learns to // merge instead of drop. +// +// CODEBURN_COPILOT_SESSION_STORE_DB is covered by that ruling too, and needs +// no exception: repointing it cannot serve stale data. Copilot's +// rollup-vs-store reconciliation runs at SERVE time over the cached serve set +// (parseProviderSources), never against a discovery-time snapshot, so a +// repointed path is simply a new source parsed on sight while the old path's +// cached rows persist as durable orphans contributing exactly what they +// always did. There is no cross-file dependency for the fingerprint to catch, +// so declaring it would buy nothing and cost the durable-history loss above. export const PROVIDER_ENV_VARS: Record = { claude: ['CLAUDE_CONFIG_DIRS', 'CLAUDE_CONFIG_DIR', 'CODEBURN_DESKTOP_SESSIONS_DIR', 'APPDATA', 'LOCALAPPDATA'], 'cline-cli': ['CLINE_SESSION_DATA_DIR', 'CLINE_DATA_DIR', 'CLINE_DIR'], @@ -264,7 +278,14 @@ export const PROVIDER_PARSE_VERSIONS: Record = { // source-provenance-v1 (#944): CLI sessions were misread as VS Code // transcripts (both carry producer 'copilot-agent'), skipping the shutdown // input/cache rollup; this bump re-parses them so the missing tokens land. - copilot: 'cli-shutdown-cost-v1-skills-source-provenance-v1', + // session-store-v2: input/cache for sessions covered by session-store.db + // moved from shutdown-rollup calls to per-request DB rows. This bump + // re-parses pre-store caches so the DB rows land; the rollup calls stay + // cached (the durable union merge never deletes) and the serve-time + // reconciliation in parseProviderSources decides per (session, model) what + // they still contribute. v2 (over the never-released v1): store dedup keys + // grew a content discriminator so a same-path DB reset cannot alias rows. + copilot: 'cli-shutdown-cost-v1-skills-source-provenance-v1-session-store-v2', grok: 'estimated-cost-v1', hermes: 'reasoning-output-accounting-v1-est-cost', 'lingtai-tui': 'token-ledger-registry-activity-v3', @@ -739,6 +760,28 @@ export async function fingerprintFile(filePath: string): Promise 0) candidates.push(filePath.slice(0, hashIdx)) + const colonIdx = filePath.lastIndexOf(':') + if (colonIdx > 0) { + // Only a prefix that still looks like a path is a candidate: a plain + // Windows path (`C:\...`) would otherwise yield the bare drive letter, + // and a stat error on that cwd-relative name must never hold hydration. + const prefix = filePath.slice(0, colonIdx) + if (prefix.includes('/') || prefix.includes('\\')) candidates.push(prefix) + } + return candidates +} + // ── Reconciliation ───────────────────────────────────────────────────── export type ReconcileAction = diff --git a/src/sessions-report.ts b/src/sessions-report.ts index 5ce716bb..84fb5a98 100644 --- a/src/sessions-report.ts +++ b/src/sessions-report.ts @@ -1,3 +1,4 @@ +import { behavioralCallCount, behavioralTurnCount } from './behavioral-weight.js' import { getShortModelName } from './models.js' import { CATEGORY_LABELS } from './types.js' import type { ProjectSummary, SessionSummary, TaskCategory } from './types.js' @@ -52,7 +53,7 @@ export function aggregateSessions(projects: ProjectSummary[]): SessionRow[] { cost: session.totalCostUSD, savingsUSD: session.totalSavingsUSD, calls: session.apiCalls, - turns: session.turns.length, + turns: behavioralTurnCount(session.turns), inputTokens: session.totalInputTokens, outputTokens: session.totalOutputTokens, cacheReadTokens: session.totalCacheReadTokens, @@ -286,7 +287,7 @@ export type SessionPrAttribution = { // Minimal structural shape a SessionSummary satisfies, so the state machine is // unit-testable without constructing a full session fixture. type AttributableSession = { - turns: Array<{ prRefs?: string[]; category?: string; assistantCalls: Array<{ costUSD: number; savingsUSD?: number; model?: string }> }> + turns: Array<{ prRefs?: string[]; category?: string; assistantCalls: Array<{ costUSD: number; savingsUSD?: number; model?: string; supplementaryAccounting?: boolean }> }> prLinks?: string[] totalCostUSD: number apiCalls: number @@ -442,7 +443,7 @@ function sessionFingerprint(s: SessionSummary): string { ts: t.assistantCalls[0]?.timestamp ?? t.timestamp ?? '', prRefs: sortedCopy(t.prRefs), cost: t.assistantCalls.reduce((n, c) => n + c.costUSD, 0), - calls: t.assistantCalls.length, + calls: behavioralCallCount(t.assistantCalls), savings: t.assistantCalls.reduce((n, c) => n + (c.savingsUSD ?? 0), 0), models: modelCost, } @@ -673,8 +674,10 @@ export function attributeSessionPrSpend(session: AttributableSession): SessionPr for (const turn of session.turns) { if (turn.prRefs?.length) current = turn.prRefs const cost = turn.assistantCalls.reduce((s, c) => s + c.costUSD, 0) - const calls = turn.assistantCalls.length + const calls = behavioralCallCount(turn.assistantCalls) const savings = turn.assistantCalls.reduce((s, c) => s + (c.savingsUSD ?? 0), 0) + // Cost/savings keep every call, so a supplementary-only turn (calls === 0) + // still attributes its spend here — only a genuinely empty turn is skipped. if (cost === 0 && calls === 0 && savings === 0) continue if (current === null) { unattributed.cost += cost @@ -895,11 +898,13 @@ export function aggregateByBranch(projects: ProjectSummary[]): BranchRow[] { let current: string | null = null for (const turn of session.turns) { if (turn.gitBranch) current = turn.gitBranch + // Raw gate: a supplementary-only turn still carries cost to attribute, + // even though it adds no behavioral calls. if (turn.assistantCalls.length === 0) continue const turnCost = turn.assistantCalls.reduce((sum, call) => sum + call.costUSD, 0) const row = byBranch.get(current) ?? { cost: 0, calls: 0, sessions: new Set() } row.cost += turnCost - row.calls += turn.assistantCalls.length + row.calls += behavioralCallCount(turn.assistantCalls) row.sessions.add(session.sessionId) byBranch.set(current, row) } diff --git a/src/types.ts b/src/types.ts index a51ae672..a4fcb883 100644 --- a/src/types.ts +++ b/src/types.ts @@ -156,6 +156,12 @@ export type ParsedApiCall = { activeDurationMs?: number activeGeneratedTokens?: number toolWaitMs?: number + /// Supplementary accounting: this call's tokens/cost are real but the call is + /// not a distinct behavioral request (copilot shutdown rollups and store rows + /// that pair with an already-counted per-turn call). Aggregates keep its + /// tokens/cost but skip it in apiCalls / model-call / turn counts. Transient: + /// assigned at serve time, never cached. + supplementaryAccounting?: boolean } export type ToolCall = { diff --git a/src/usage-aggregator.ts b/src/usage-aggregator.ts index 82aeb732..53cbdd2c 100644 --- a/src/usage-aggregator.ts +++ b/src/usage-aggregator.ts @@ -1,5 +1,6 @@ import { homedir } from 'node:os' import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory, type DateRange } from './types.js' +import { isBehavioralCall } from './behavioral-weight.js' import { type PeriodData, type ProviderCost, type BreakdownArrays, type MenubarPayload, type ClaudeConfigSelector, buildMenubarPayload } from './menubar-json.js' import { parseAllSessions, filterProjectsByName, filterProjectsByDays, filterProjectsByClaudeConfigSource, isSessionHydrationComplete } from './parser.js' import { findUnpricedModels, getLocalModelSavingsConfigHash, getPriceOverridesConfigHash, getShortModelName, isExpectedFreeModel } from './models.js' @@ -903,11 +904,16 @@ export async function buildMenubarPayloadForRange(periodInfo: PeriodInfo, opts: for (const [m, d] of Object.entries(s.mcpBreakdown)) { mcpMap[m] = (mcpMap[m] ?? 0) + d.calls } for (const turn of s.turns) for (const call of turn.assistantCalls) { if (!call.savingsUSD || call.savingsUSD <= 0) continue + // Saved DOLLARS/tokens keep every call, but the `calls` figures are + // request counts: a supplementary accounting call (copilot rollup / + // paired store row) can carry configured model-savings too and must + // not count as a request. + const callWeight = isBehavioralCall(call) ? 1 : 0 totalSavings += call.savingsUSD - totalSavingsCalls += 1 + totalSavingsCalls += callWeight const modelKey = getShortModelName(call.model) const acc = savingsByModel.get(modelKey) ?? { calls: 0, actualUSD: 0, savingsUSD: 0, baselineModel: call.savingsBaselineModel ?? '', inputTokens: 0, outputTokens: 0 } - acc.calls += 1 + acc.calls += callWeight acc.actualUSD += call.costUSD acc.savingsUSD += call.savingsUSD acc.baselineModel = acc.baselineModel || (call.savingsBaselineModel ?? '') @@ -915,7 +921,7 @@ export async function buildMenubarPayloadForRange(periodInfo: PeriodInfo, opts: acc.outputTokens += call.usage.outputTokens savingsByModel.set(modelKey, acc) const provAcc = savingsByProvider.get(call.provider) ?? { calls: 0, savingsUSD: 0 } - provAcc.calls += 1 + provAcc.calls += callWeight provAcc.savingsUSD += call.savingsUSD savingsByProvider.set(call.provider, provAcc) } diff --git a/tests/audit-report.test.ts b/tests/audit-report.test.ts index 1723712c..2c7a1280 100644 --- a/tests/audit-report.test.ts +++ b/tests/audit-report.test.ts @@ -101,6 +101,27 @@ describe('aggregateAudit', () => { expect(rows[0]!.cost.recomputedTotalUSD).toBe(0) }) + it('gives copilot supplementary accounting no call weight and no phantom reasoning output', async () => { + // One served request recorded twice: the per-turn call carries the full output, the + // paired store row carries that output's reasoning subset plus real input/cache tokens. + const perTurn = makeCall({ inputTokens: 300, outputTokens: 500, cacheReadInputTokens: 1000 }, 1.0, 'claude-sonnet-4-5', 'copilot') + const supplementary: ParsedApiCall = { + ...makeCall({ inputTokens: 40, outputTokens: 0, reasoningTokens: 800, cacheReadInputTokens: 900 }, 0.5, 'claude-sonnet-4-5', 'copilot'), + supplementaryAccounting: true, + } + const rows = await aggregateAudit([makeProject([perTurn, supplementary])]) + + expect(rows).toHaveLength(1) + const r = rows[0]! + expect(r.calls).toBe(1) + expect(r.displayed.outputTokens).toBe(500) + // Raw stays untouched, and tokens/cost keep every call, supplementary included. + expect(r.raw.reasoningTokens).toBe(800) + expect(r.raw.inputTokens).toBe(340) + expect(r.displayed.cacheReadTokens).toBe(1900) + expect(r.attributedCostUSD).toBeCloseTo(1.5) + }) + it('splits buckets by (provider, model)', async () => { const rows = await aggregateAudit([makeProject([ makeCall({ inputTokens: 10 }, 0.1, 'model-a', 'claude'), diff --git a/tests/compare-stats.test.ts b/tests/compare-stats.test.ts index f110264a..17df188a 100644 --- a/tests/compare-stats.test.ts +++ b/tests/compare-stats.test.ts @@ -623,3 +623,61 @@ describe('findModelStat', () => { expect(findModelStat(stats, 'claude-opus-9-9')).toBeUndefined() }) }) + +// Copilot serve sets carry supplementary accounting calls (shutdown rollups, +// residuals, store rows paired with an already-counted per-turn call). They hold +// real tokens/cost but no behavioral evidence, so the compare report — which is +// entirely per-call/per-turn ratios — must not weigh them. +function supplement(turn: ClassifiedTurn, model: string, cost: number): ClassifiedTurn { + turn.assistantCalls.unshift({ + ...turn.assistantCalls[0]!, + model, + costUSD: cost, + supplementaryAccounting: true, + deduplicationKey: `supp-${Math.random()}`, + }) + return turn +} + +describe('supplementary accounting weight', () => { + it('takes the primary model from the first behavioral call, not a leading supplementary one', () => { + const project = makeProject([ + supplement(makeTurn('opus-4-6', 0.10, { hasEdits: true }), 'rollup-model', 0.01), + ]) + const stats = aggregateModelStats([project]) + + expect(stats.find(s => s.model === 'opus-4-6')!.totalTurns).toBe(1) + expect(stats.find(s => s.model === 'rollup-model')?.totalTurns ?? 0).toBe(0) + }) + + it('keeps supplementary cost and tokens but does not count them as calls', () => { + const project = makeProject([ + supplement(makeTurn('opus-4-6', 0.10), 'opus-4-6', 0.04), + ]) + const m = aggregateModelStats([project]).find(s => s.model === 'opus-4-6')! + + expect(m.calls).toBe(1) + expect(m.cost).toBeCloseTo(0.14) + expect(m.outputTokens).toBe(400) + }) + + it('excludes an accounting-only turn from every efficiency surface', () => { + const accountingOnly = makeTurn('opus-4-6', 0.09, { hasEdits: true, category: 'debugging', speed: 'fast', hasAgentSpawn: true }) + accountingOnly.assistantCalls[0]!.supplementaryAccounting = true + const project = makeProject([ + makeTurn('opus-4-6', 0.10, { hasEdits: true, category: 'coding' }), + accountingOnly, + ]) + + const m = aggregateModelStats([project]).find(s => s.model === 'opus-4-6')! + expect(m.totalTurns).toBe(1) + expect(m.editTurns).toBe(1) + + const categories = computeCategoryComparison([project], 'opus-4-6', 'other-model') + expect(categories.map(c => c.category)).toEqual(['coding']) + + const style = computeWorkingStyle([project], 'opus-4-6', 'other-model') + expect(style.find(r => r.label === 'Delegation rate')!.valueA).toBeCloseTo(0) + expect(style.find(r => r.label === 'Fast mode usage')!.valueA).toBeCloseTo(0) + }) +}) diff --git a/tests/dashboard.test.ts b/tests/dashboard.test.ts index 9879c8a1..c49e7efc 100644 --- a/tests/dashboard.test.ts +++ b/tests/dashboard.test.ts @@ -89,6 +89,18 @@ function makeTurn(timestamp: string, costs: number[]): SessionSummary['turns'][n } } +// Copilot serve sets pair a per-turn call with supplementary accounting rows +// (shutdown rollups, residuals, store rows): real cost and tokens, zero +// behavioral weight. +function markSupplementary(turn: SessionSummary['turns'][number], indexes: number[]): SessionSummary['turns'][number] { + for (const index of indexes) { + const call = turn.assistantCalls[index]! + call.supplementaryAccounting = true + call.usage = { ...call.usage, inputTokens: 40 } + } + return turn +} + // Logic replicated from TopSessions component function getTopSessions(projects: ProjectSummary[], n = 5) { const all = projects.flatMap(p => p.sessions.map(s => ({ ...s, projectPath: p.projectPath }))) @@ -231,6 +243,20 @@ describe('Daily Activity history', () => { ]) }) + it('spends supplementary accounting cost without counting it as a call', () => { + const session = makeSession('s1', 0) + session.turns = [ + // One real request plus its paired store row. + markSupplementary(makeTurn('2026-08-05T12:00:00Z', [1.0, 0.5]), [1]), + // A rollup-only turn: cost with no behavioral request at all. + markSupplementary(makeTurn('2026-08-05T13:00:00Z', [0.25]), [0]), + ] + + expect(getDailyActivityRows([makeProject('proj', [session])])).toEqual([ + { day: '2026-08-05', cost: 1.75, calls: 1 }, + ]) + }) + it('pages one viewport and keeps the final page full', () => { expect(pageHistoryCursor(0, 1, 35, 69)).toBe(34) expect(pageHistoryCursor(34, -1, 35, 69)).toBe(0) diff --git a/tests/day-aggregator.test.ts b/tests/day-aggregator.test.ts index 7d70d14b..6cb44ae8 100644 --- a/tests/day-aggregator.test.ts +++ b/tests/day-aggregator.test.ts @@ -559,3 +559,34 @@ describe('daily-cache ↔ report daily-bucket parity', () => { expect(historyByDate[dayB]).toBe(10) }) }) + +describe('supplementary accounting weight (copilot store/rollup calls)', () => { + it('adds cost and tokens but no call or turn weight, matching buildSessionSummary', () => { + // One real request served both ways: the per-turn call is behavioral, the + // paired store row is supplementary. Sealed daily history must agree with + // the live session summary (apiCalls 1), not double the call. + const behavioral = makeCall('2026-08-05T10:00:00Z', 1, 'claude-sonnet-4-5', 'copilot') + const supplementary = { ...makeCall('2026-08-05T10:00:05Z', 2, 'claude-sonnet-4-5', 'copilot'), supplementaryAccounting: true } + const day = aggregateProjectsIntoDays([makeSingleTurnProject([behavioral, supplementary])])[0]! + expect(day.calls).toBe(1) + expect(day.cost).toBeCloseTo(3, 12) + expect(day.inputTokens).toBe(200) + expect(day.models['claude-sonnet-4-5']!.calls).toBe(1) + expect(day.models['claude-sonnet-4-5']!.cost).toBeCloseTo(3, 12) + expect(day.providers['copilot']!.calls).toBe(1) + expect(day.categories['coding']!.turns).toBe(1) + + // A turn made only of supplementary calls (a rollup-only session's + // accounting container): cost and tokens land, weight does not. + const aggOnly = aggregateProjectsIntoDays([makeSingleTurnProject([ + { ...makeCall('2026-08-05T11:00:00Z', 2, 'claude-sonnet-4-5', 'copilot'), supplementaryAccounting: true }, + ])])[0]! + expect(aggOnly.calls).toBe(0) + expect(aggOnly.cost).toBeCloseTo(2, 12) + expect(aggOnly.inputTokens).toBe(100) + expect(aggOnly.categories['coding']!.turns).toBe(0) + expect(aggOnly.editTurns).toBe(0) + expect(aggOnly.models['claude-sonnet-4-5']!.calls).toBe(0) + expect(aggOnly.providers['copilot']!.calls).toBe(0) + }) +}) diff --git a/tests/export.test.ts b/tests/export.test.ts index 81bc97e9..b55f7271 100644 --- a/tests/export.test.ts +++ b/tests/export.test.ts @@ -220,6 +220,49 @@ describe('exportCsv', () => { expect(lines[1]).toContain("'+danger-model") }) + it('keeps supplementary accounting rows in records.csv and marks them', async () => { + const project = makeProject('app') + const turn = project.sessions[0]!.turns[0]! + turn.assistantCalls.push({ + ...turn.assistantCalls[0]!, + supplementaryAccounting: true, + costUSD: 0.5, + deduplicationKey: 'dedup-supp', + }) + const periods: PeriodExport[] = [{ label: '30 Days', projects: [project] }] + + const folder = await exportCsv(periods, join(tmpDir, 'records.csv')) + const lines = (await readFile(join(folder, 'records.csv'), 'utf-8')).trimEnd().split('\n') + + // The column exists on every row (undefined on normal ones), so rowsToCsv — + // which reads headers off the first row — always emits it. + expect(lines[0]!.endsWith(',supplementary')).toBe(true) + expect(lines[1]!.endsWith(',1.23,0,')).toBe(true) + expect(lines[2]!.endsWith(',0.5,0,true')).toBe(true) + expect(lines).toHaveLength(3) + }) + + it('counts only behavioral turns in the sessions.csv Turns column', async () => { + const project = makeProject('app') + const session = project.sessions[0]! + session.turns.push({ + ...session.turns[0]!, + assistantCalls: [{ + ...session.turns[0]!.assistantCalls[0]!, + supplementaryAccounting: true, + deduplicationKey: 'dedup-supp', + }], + }) + const periods: PeriodExport[] = [{ label: '30 Days', projects: [project] }] + + const folder = await exportCsv(periods, join(tmpDir, 'sessions.csv')) + const [header, row] = (await readFile(join(folder, 'sessions.csv'), 'utf-8')).split('\n') + const turns = row!.split(',')[header!.split(',').indexOf('Turns')] + + // Two raw turns, one of them accounting-only. + expect(turns).toBe('1') + }) + it('adds optional subagentType and unambiguous model fields to sessions.csv', async () => { const periods: PeriodExport[] = [{ label: '30 Days', projects: [makeProject('app', 'planner')] }] @@ -258,6 +301,49 @@ describe('exportJson', () => { expect(data.sessions[1]).not.toHaveProperty('subagentType') }) + it('keeps supplementary-accounting tokens/cost in daily rows without counting them as calls', async () => { + const project = makeProject('app') + const turn = project.sessions[0]!.turns[0]! + turn.assistantCalls.push({ + ...turn.assistantCalls[0]!, + supplementaryAccounting: true, + usage: { ...turn.assistantCalls[0]!.usage, inputTokens: 40, outputTokens: 0, cacheReadInputTokens: 900 }, + costUSD: 0.5, + deduplicationKey: 'dedup-supp', + }) + const periods: PeriodExport[] = [{ label: '30 Days', projects: [project] }] + + const saved = await exportJson(periods, join(tmpDir, 'supp.json')) + const data = JSON.parse(await readFile(saved, 'utf-8')) + + expect(data.periods[0].daily).toHaveLength(1) + expect(data.periods[0].daily[0]).toMatchObject({ + 'API Calls': 1, + 'Input Tokens': 140, + 'Cache Read Tokens': 900, + 'Cost (USD)': 1.73, + }) + }) + + it('marks supplementary records and omits the key on normal ones', async () => { + const project = makeProject('app') + const turn = project.sessions[0]!.turns[0]! + turn.assistantCalls.push({ + ...turn.assistantCalls[0]!, + supplementaryAccounting: true, + costUSD: 0.5, + deduplicationKey: 'dedup-supp', + }) + const periods: PeriodExport[] = [{ label: '30 Days', projects: [project] }] + + const saved = await exportJson(periods, join(tmpDir, 'supp-records.json')) + const data = JSON.parse(await readFile(saved, 'utf-8')) + + expect(data.records).toHaveLength(2) + expect(data.records[0]).not.toHaveProperty('supplementary') + expect(data.records[1]).toMatchObject({ supplementary: true, cost: 0.5 }) + }) + it('includes an mcp section with per-server usage', async () => { const project = makeProject('app') project.sessions[0]!.mcpBreakdown = { node_repl: { calls: 3 }, github: { calls: 1 } } diff --git a/tests/format-status-bar.test.ts b/tests/format-status-bar.test.ts new file mode 100644 index 00000000..0a34f924 --- /dev/null +++ b/tests/format-status-bar.test.ts @@ -0,0 +1,66 @@ +import { describe, it, expect } from 'vitest' +import stripAnsi from 'strip-ansi' + +import { formatCost, renderStatusBar } from '../src/format.js' +import type { ProjectSummary } from '../src/types.js' + +// Copilot supplementary accounting calls (shutdown rollups, residuals, store +// rows paired with an already-counted per-turn call) carry real cost but are +// not distinct requests. The status bar must spend their cost and count zero +// calls for them, matching the session summaries and sealed daily history. +describe('renderStatusBar supplementary accounting', () => { + function call(costUSD: number, timestamp: string, supplementaryAccounting: boolean) { + return { + provider: 'copilot', + model: 'claude-sonnet-4-5', + usage: { + inputTokens: supplementaryAccounting ? 40 : 0, + outputTokens: 0, + cacheCreationInputTokens: 0, + cacheReadInputTokens: 0, + cachedInputTokens: 0, + reasoningTokens: 0, + webSearchRequests: 0, + }, + costUSD, + tools: [], + mcpTools: [], + skills: [], + hasAgentSpawn: false, + hasPlanMode: false, + speed: 'standard', + timestamp, + bashCommands: [], + deduplicationKey: `copilot-${timestamp}`, + supplementaryAccounting, + } + } + + it('counts only behavioral calls while keeping every call\'s cost', () => { + // Local noon today, so both turns bucket into today/month on any machine TZ. + const now = new Date() + const at = (minute: number) => new Date(now.getFullYear(), now.getMonth(), now.getDate(), 12, minute).toISOString() + + const projects = [{ + sessions: [{ + turns: [ + // One real request plus its paired store row. + { + timestamp: at(0), + assistantCalls: [call(1.0, at(0), false), call(0.5, at(1), true)], + }, + // Rollup-only turn: cost lands, no request is counted. The raw + // zero-call gate in renderStatusBar keeps this turn in the totals. + { + timestamp: at(2), + assistantCalls: [call(0.25, at(2), true)], + }, + ], + }], + }] as ProjectSummary[] + + const out = stripAnsi(renderStatusBar(projects)) + expect(out).toContain(`Today ${formatCost(1.75)} 1 calls`) + expect(out).toContain(`Month ${formatCost(1.75)} 1 calls`) + }) +}) diff --git a/tests/models-report.test.ts b/tests/models-report.test.ts index 33317fd8..697a6f01 100644 --- a/tests/models-report.test.ts +++ b/tests/models-report.test.ts @@ -32,6 +32,7 @@ function makeCall(opts: { costUSD: number input?: number output?: number + reasoning?: number cacheWrite?: number cacheRead?: number }): ParsedApiCall { @@ -42,6 +43,7 @@ function makeCall(opts: { ...emptyTokens(), inputTokens: opts.input ?? 0, outputTokens: opts.output ?? 0, + reasoningTokens: opts.reasoning ?? 0, cacheCreationInputTokens: opts.cacheWrite ?? 0, cacheReadInputTokens: opts.cacheRead ?? 0, }, @@ -257,6 +259,25 @@ describe('aggregateModels', () => { const rows = await aggregateModels([project]) expect(rows[0]!.outputTokens).toBe(250) }) + + it('gives copilot supplementary accounting no call weight and no phantom reasoning output', async () => { + // One served request recorded twice: the per-turn call carries the full output, the + // paired store row carries that output's reasoning subset plus real input/cache tokens. + const perTurn = makeCall({ provider: 'copilot', model: 'claude-sonnet-4-5', input: 300, output: 500, cacheRead: 1000, costUSD: 1.0 }) + const supplementary: ParsedApiCall = { + ...makeCall({ provider: 'copilot', model: 'claude-sonnet-4-5', input: 40, output: 0, reasoning: 800, cacheRead: 900, costUSD: 0.5 }), + supplementaryAccounting: true, + } + const rows = await aggregateModels([makeProject([makeTurn('feature', [perTurn, supplementary])])]) + expect(rows).toHaveLength(1) + const row = rows[0]! + expect(row.calls).toBe(1) + expect(row.outputTokens).toBe(500) + // Tokens and cost keep every call, supplementary included. + expect(row.inputTokens).toBe(340) + expect(row.cacheReadTokens).toBe(1900) + expect(row.costUSD).toBeCloseTo(1.5, 6) + }) }) describe('aggregateModels byAgent', () => { diff --git a/tests/parser.test.ts b/tests/parser.test.ts index 4bd0c5c2..38ebd95d 100644 --- a/tests/parser.test.ts +++ b/tests/parser.test.ts @@ -4,7 +4,8 @@ // (b) OTel-prune monotonic — OTel DB rows pruned → total unchanged // (c) no double-count — same source parsed twice → counted once // (d) non-durable evicts — deleted source for non-durable provider IS removed -// (e) 90-day age-out — orphan ≥ 91d old is pruned; ≤ 89d is retained +// (e) 90-day age-out — ≥ 91d pruned (orphans AND unflagged discovered +// sources); retainWhilePresent + discovered kept import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest' import { mkdtemp, mkdir, writeFile, rm, unlink } from 'fs/promises' @@ -13,7 +14,10 @@ import { join } from 'path' import { createRequire } from 'node:module' import { isSqliteAvailable } from '../src/sqlite.js' -import { clearSessionCache, parseAllSessions, setParseReuseValidator } from '../src/parser.js' +import { calculateCost } from '../src/models.js' +import { aggregateProjectsIntoDays } from '../src/day-aggregator.js' +import { DAILY_CACHE_VERSION, currentTzKey, ensureCacheHydrated, saveDailyCache } from '../src/daily-cache.js' +import { clearSessionCache, isSessionHydrationComplete, parseAllSessions, setParseReuseValidator } from '../src/parser.js' import { loadCache, saveCache, sessionCachePath } from '../src/session-cache.js' import type { SessionSource, SessionParser, ParsedProviderCall } from '../src/providers/types.js' @@ -354,7 +358,7 @@ describe('(d) non-durable provider evicts deleted sources', () => { // (e) 90-day age-out: orphan ≥ 91d old is pruned; ≤ 89d is retained // ═══════════════════════════════════════════════════════════════════════════ describe('(e) 90-day age-out for durable providers', () => { - it('prunes an orphaned cache entry whose newest call is 91 days old', async () => { + it('prunes a 91-day-old entry even while its unflagged source is still discovered', async () => { const synthFile = join(tmpHome, 'synth-age.txt') await writeFile(synthFile, 'placeholder') @@ -374,17 +378,54 @@ describe('(e) 90-day age-out for durable providers', () => { userMessage: 'old', sessionId: 'synth-old', }] - // First parse: cached with 91d-old timestamp → immediately pruned by 90-day check + // Unflagged durable sources age out on the ordinary schedule even while + // the file remains on disk (the pre-existing cap on cache growth for + // provider-pruned journals). Only a retainWhilePresent source — the + // copilot session-store, which IS the durable record — is exempt. const proj1 = await parseAllSessions(undefined, 'test-synthetic') - expect(totalOutput(proj1)).toBe(0) // pruned right away + expect(totalOutput(proj1)).toBe(0) - // Confirm: entry is not in the persistent cache after first parse + // Stable on the next pass too (re-parsed, then aged out again). clearSessionCache() - _synthSources = [] // no longer discovered const proj2 = await parseAllSessions(undefined, 'test-synthetic') expect(totalOutput(proj2)).toBe(0) }) + it('retains a 91-day-old entry whose still-discovered source declares retainWhilePresent', async () => { + const synthFile = join(tmpHome, 'synth-retain-flag.txt') + await writeFile(synthFile, 'placeholder') + + const ts91dAgo = new Date(Date.now() - 91 * 24 * 60 * 60 * 1000).toISOString() + + _synthDurable = true + _synthSources = [{ path: synthFile, project: 'test', provider: 'test-synthetic', retainWhilePresent: true }] + _synthYields = [{ + provider: 'test-synthetic', model: 'gpt-4o', + inputTokens: 10, outputTokens: 8, + cacheCreationInputTokens: 0, cacheReadInputTokens: 0, + cachedInputTokens: 0, reasoningTokens: 0, webSearchRequests: 0, + costUSD: 0.002, tools: [], bashCommands: [], + timestamp: ts91dAgo, + speed: 'standard', + deduplicationKey: 'synth-retain-flag-91d', + userMessage: 'old', sessionId: 'synth-flag', + }] + + // Flagged + discovered: the file is the durable record; pruning it would + // drop data only it holds. Served and retained across passes. + const proj1 = await parseAllSessions(undefined, 'test-synthetic') + expect(totalOutput(proj1)).toBe(8) + clearSessionCache() + const proj2 = await parseAllSessions(undefined, 'test-synthetic') + expect(totalOutput(proj2)).toBe(8) + + // Once ORPHANED the flag no longer applies — orphan age-out prunes. + clearSessionCache() + _synthSources = [] // no longer discovered + const proj3 = await parseAllSessions(undefined, 'test-synthetic') + expect(totalOutput(proj3)).toBe(0) + }) + it('retains an orphaned cache entry whose newest call is 89 days old', async () => { const synthFile = join(tmpHome, 'synth-retain.txt') await writeFile(synthFile, 'placeholder') @@ -419,6 +460,54 @@ describe('(e) 90-day age-out for durable providers', () => { }) }) +// ═══════════════════════════════════════════════════════════════════════════ +// Emission gate: a reasoning-only copilot session still emits +// ═══════════════════════════════════════════════════════════════════════════ +// A rollup can carry ONLY reasoning tokens (zero input/cache/output, zero +// cost — copilot reasoning is never priced). The gate admits any +// usage-bearing session; dropping this one would silently lose the only +// record of its usage. +describe('reasoning-only copilot session emission', () => { + it('emits a session whose sole rollup carries only reasoning tokens', async () => { + const sessionStateDir = join(tmpHome, 'session-state-reason-only') + await mkdir(sessionStateDir, { recursive: true }) + vi.stubEnv('CODEBURN_COPILOT_SESSION_STATE_DIR', sessionStateDir) + vi.stubEnv('CODEBURN_COPILOT_SESSION_STORE_DB', join(tmpHome, 'no-store.db')) + vi.stubEnv('CODEBURN_COPILOT_DISABLE_OTEL', '1') + vi.stubEnv('CODEBURN_COPILOT_WS_STORAGE_DIR', join(tmpHome, 'no-ws')) + vi.stubEnv('CODEBURN_COPILOT_GLOBAL_STORAGE_DIR', join(tmpHome, 'no-global')) + vi.stubEnv('CODEBURN_COPILOT_JETBRAINS_DIR', join(tmpHome, 'no-jb')) + + const ts = new Date(Date.now() - 2 * 24 * 60 * 60 * 1000).toISOString() + const dir = join(sessionStateDir, 'sess-reason') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-reason\ncwd: /home/user/testproj\n') + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: ts, data: { newModel: 'gpt-5' } }), + JSON.stringify({ + type: 'session.shutdown', + timestamp: ts, + data: { + shutdownType: 'routine', + modelMetrics: { + 'gpt-5': { + requests: { count: 1, cost: 0 }, + usage: { inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0, reasoningTokens: 800 }, + }, + }, + }, + }), + ].join('\n') + '\n') + + const projects = await parseAllSessions(undefined, 'copilot') + const sessions = projects.flatMap(p => p.sessions).filter(s => s.sessionId === 'sess-reason') + expect(sessions).toHaveLength(1) + expect(sessions[0]!.totalReasoningTokens).toBe(800) + // The rollup is supplementary accounting: usage served, zero call weight. + expect(sessions[0]!.apiCalls).toBe(0) + }) +}) + // ═══════════════════════════════════════════════════════════════════════════ // (f) Version-bump survival: a PROVIDER_PARSE_VERSIONS bump (or any env // fingerprint change) must NOT erase durable orphans. The cache is the @@ -767,3 +856,1328 @@ describe('(r) validated parse reuse (setParseReuseValidator)', () => { _synthYields = [] }) }) + +// ═══════════════════════════════════════════════════════════════════════════ +// (i) Growing session-store DB: durable merge appends only the new rows +// ═══════════════════════════════════════════════════════════════════════════ +// session-store.db records one usage row per API request; rows only ever +// append (AUTOINCREMENT ids). This exercises the PRODUCTION path end to end: +// both representations parse and cache, serve-time precedence +// (parseProviderSources) drops the covered session's shutdown rollup, and a +// re-parse after INSERTs appends exactly the new rows under the durable +// union-by-dedup-key merge — totals must equal the DB, not the rollup, and +// never double-count. +function createStoreDb(dbPath: string): void { + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const db = new DatabaseSync(dbPath) + db.exec(` + CREATE TABLE sessions (id TEXT PRIMARY KEY, cwd TEXT, repository TEXT, created_at TEXT); + CREATE TABLE assistant_usage_events ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + session_id TEXT NOT NULL, + model TEXT NOT NULL, + input_tokens INTEGER, + output_tokens INTEGER, + cache_read_tokens INTEGER, + cache_write_tokens INTEGER, + reasoning_tokens INTEGER, + created_at TEXT + ); + `) + db.close() +} + +function insertStoreRow( + dbPath: string, + sessionId: string, + inputTokens: number, // cache-inclusive, as the CLI writes it + cacheRead: number, + cacheWrite: number, + createdAt: string, + reasoning = 0, + cwd = '/home/user/testproj', +): void { + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const db = new DatabaseSync(dbPath) + db.prepare(`INSERT OR IGNORE INTO sessions (id, cwd) VALUES (?, ?)`).run(sessionId, cwd) + db.prepare( + `INSERT INTO assistant_usage_events + (session_id, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, reasoning_tokens, created_at) + VALUES (?, 'claude-sonnet-4-5', ?, 0, ?, ?, ?, ?)` + ).run(sessionId, inputTokens, cacheRead, cacheWrite, reasoning, createdAt) + db.close() +} + +describe.skipIf(!isSqliteAvailable())('(i) growing session-store DB durable merge', () => { + it('totals track the store exactly as rows append, with the rollup reconciled away', async () => { + const sessionStateDir = join(tmpHome, 'session-state') + await mkdir(sessionStateDir, { recursive: true }) + const dbPath = join(tmpHome, 'session-store.db') + vi.stubEnv('CODEBURN_COPILOT_SESSION_STATE_DIR', sessionStateDir) + vi.stubEnv('CODEBURN_COPILOT_SESSION_STORE_DB', dbPath) + vi.stubEnv('CODEBURN_COPILOT_DISABLE_OTEL', '1') + vi.stubEnv('CODEBURN_COPILOT_WS_STORAGE_DIR', join(tmpHome, 'no-ws')) + vi.stubEnv('CODEBURN_COPILOT_GLOBAL_STORAGE_DIR', join(tmpHome, 'no-global')) + vi.stubEnv('CODEBURN_COPILOT_JETBRAINS_DIR', join(tmpHome, 'no-jb')) + + const base = Date.now() - 5 * 24 * 60 * 60 * 1000 + const at = (offsetSec: number): string => new Date(base + offsetSec * 1000).toISOString() + + // The session's events.jsonl carries per-turn output AND a shutdown + // rollup summing exactly the two covered requests (the production shape: + // rows commit before the rollup is written, cache-inclusive on both + // sides). If reconciliation failed to replace the rollup, totals would + // double; if it dropped usage, they would fall short of the rows. + const dir = join(sessionStateDir, 'sess-store') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-store\ncwd: /home/user/testproj\n') + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ type: 'assistant.message', timestamp: at(10), data: { messageId: 'msg-1', outputTokens: 17, toolRequests: [] } }), + JSON.stringify({ + type: 'session.shutdown', + timestamp: at(20), + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 2, cost: 1 }, + usage: { inputTokens: 20000, outputTokens: 17, cacheReadTokens: 17000, cacheWriteTokens: 2400, reasoningTokens: 40 }, + }, + }, + }, + }), + ].join('\n') + '\n') + + createStoreDb(dbPath) + // Row 1 carries reasoning tokens: they are a subset of the session's + // per-turn output and must ride as metadata WITHOUT entering the + // query-path cost recompute (cachedCallToApiCall discards the parser's + // costUSD for copilot and re-derives from tokens — the assertion below + // is the only guard that exercises that production path). + insertStoreRow(dbPath, 'sess-store', 12000, 10000, 1500, at(12), 40) // input 500 + insertStoreRow(dbPath, 'sess-store', 8000, 7000, 900, at(15)) // input 100 + + const sumUsage = (projects: Awaited>) => { + const calls = projects.flatMap(p => p.sessions).flatMap(s => s.turns).flatMap(t => t.assistantCalls) + return { + input: calls.reduce((s, c) => s + c.usage.inputTokens, 0), + cacheRead: calls.reduce((s, c) => s + c.usage.cacheReadInputTokens, 0), + cacheWrite: calls.reduce((s, c) => s + c.usage.cacheCreationInputTokens, 0), + output: calls.reduce((s, c) => s + c.usage.outputTokens, 0), + cost: calls.reduce((s, c) => s + c.costUSD, 0), + } + } + + // The reasoning-free cost of everything above: per-turn output plus the + // two store rows priced on input/cache alone. A higher observed cost + // means the 40 reasoning tokens were billed at the output rate on a call + // that owns no output — double-billing them against the per-turn call. + const expectedCost = + calculateCost('claude-sonnet-4-5', 0, 17, 0, 0, 0) + + calculateCost('claude-sonnet-4-5', 500, 0, 1500, 10000, 0) + + calculateCost('claude-sonnet-4-5', 100, 0, 900, 7000, 0) + + // First parse: input/cache equal the DB rows exactly (the rollup is + // reconciled away, residual zero); output stays with the per-turn event. + const first = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(first.cost).toBeCloseTo(expectedCost, 12) + expect(first).toEqual({ input: 600, cacheRead: 17000, cacheWrite: 2400, output: 17, cost: first.cost }) + + // The session continues: one more API request lands as one more row. + // Re-parse against the warm disk cache — the durable merge must append + // only the new row's key, keeping totals equal to the DB. + clearSessionCache() + insertStoreRow(dbPath, 'sess-store', 5000, 4600, 300, at(30)) // input 100 + + const second = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(second.cost).toBeCloseTo(expectedCost + calculateCost('claude-sonnet-4-5', 100, 0, 300, 4600, 0), 12) + expect(second).toEqual({ input: 700, cacheRead: 21600, cacheWrite: 2700, output: 17, cost: second.cost }) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (k) Serve-time precedence heals a durably double-cached session +// ═══════════════════════════════════════════════════════════════════════════ +// The union merge never deletes, so a rollup cached while the store was +// unreadable (an unsupported-schema epoch, a runtime without node:sqlite, +// restored files) survives the store later becoming readable — and its +// session's rows would then be cached beside it. Serve-time precedence must +// drop the rollup calls whenever store calls exist for the session, healing +// the state instead of double-counting it forever. +describe.skipIf(!isSqliteAvailable())('(k) serve-time precedence over stale cached rollups', () => { + it('stops counting a cached rollup once the store covers its session', async () => { + const sessionStateDir = join(tmpHome, 'session-state') + await mkdir(sessionStateDir, { recursive: true }) + const dbPath = join(tmpHome, 'session-store.db') + vi.stubEnv('CODEBURN_COPILOT_SESSION_STATE_DIR', sessionStateDir) + vi.stubEnv('CODEBURN_COPILOT_SESSION_STORE_DB', dbPath) + vi.stubEnv('CODEBURN_COPILOT_DISABLE_OTEL', '1') + vi.stubEnv('CODEBURN_COPILOT_WS_STORAGE_DIR', join(tmpHome, 'no-ws')) + vi.stubEnv('CODEBURN_COPILOT_GLOBAL_STORAGE_DIR', join(tmpHome, 'no-global')) + vi.stubEnv('CODEBURN_COPILOT_JETBRAINS_DIR', join(tmpHome, 'no-jb')) + + const base = Date.now() - 5 * 24 * 60 * 60 * 1000 + const at = (offsetSec: number): string => new Date(base + offsetSec * 1000).toISOString() + const dir = join(sessionStateDir, 'sess-stale') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-stale\ncwd: /home/user/testproj\n') + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ type: 'assistant.message', timestamp: at(10), data: { messageId: 'msg-1', outputTokens: 25, toolRequests: [] } }), + JSON.stringify({ + type: 'session.shutdown', + timestamp: at(20), + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 1, cost: 1 }, + usage: { inputTokens: 20000, outputTokens: 25, cacheReadTokens: 17000, cacheWriteTokens: 2400, reasoningTokens: 0 }, + }, + }, + }, + }), + ].join('\n') + '\n') + + const sumUsage = (projects: Awaited>) => { + const calls = projects.flatMap(p => p.sessions).flatMap(s => s.turns).flatMap(t => t.assistantCalls) + return { + input: calls.reduce((s, c) => s + c.usage.inputTokens, 0), + cacheRead: calls.reduce((s, c) => s + c.usage.cacheReadInputTokens, 0), + output: calls.reduce((s, c) => s + c.usage.outputTokens, 0), + } + } + + // Run 1: no store exists — the rollup is legitimately the only record + // and gets durably cached. + const first = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(first).toEqual({ input: 600, cacheRead: 17000, output: 25 }) + + // The store now becomes readable WITH rows for the same session — the + // uncovered→covered transition no writer ordering protects (schema + // epoch ending, node:sqlite appearing, restored files). events.jsonl is + // unchanged, so its cached rollup calls survive the merge untouched. + clearSessionCache() + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-stale', 12000, 10000, 1500, at(12)) // input 500 + insertStoreRow(dbPath, 'sess-stale', 8000, 7000, 900, at(15)) // input 100 + + // Run 2: totals must equal per-turn output + store rows — the cached + // rollup (input 600 / cacheRead 17,000) must not ALSO count. + const second = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(second).toEqual({ input: 600, cacheRead: 17000, output: 25 }) + + // Run 3 (warm disk cache, nothing changed): still healed, still once. + clearSessionCache() + const third = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(third).toEqual({ input: 600, cacheRead: 17000, output: 25 }) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (l) Age-out never expires the still-discovered session store +// ═══════════════════════════════════════════════════════════════════════════ +// An idle machine whose session-store rows are all >90 days old: the store is +// still on disk and IS the durable record (crash-only rows have no rollup to +// fall back to), so its discovery declares retainWhilePresent and the age-out +// leaves it alone. Ordinary journal-style sources keep the pre-existing +// schedule — pruned at 90 days whether or not the file remains — and orphaned +// store entries age out normally once the DB itself is gone. +describe.skipIf(!isSqliteAvailable())('(l) age-out exempts still-discovered store data', () => { + it('serves >90d-old store rows while the store is still on disk', async () => { + const sessionStateDir = join(tmpHome, 'session-state') + await mkdir(sessionStateDir, { recursive: true }) + const dbPath = join(tmpHome, 'session-store.db') + vi.stubEnv('CODEBURN_COPILOT_SESSION_STATE_DIR', sessionStateDir) + vi.stubEnv('CODEBURN_COPILOT_SESSION_STORE_DB', dbPath) + vi.stubEnv('CODEBURN_COPILOT_DISABLE_OTEL', '1') + vi.stubEnv('CODEBURN_COPILOT_WS_STORAGE_DIR', join(tmpHome, 'no-ws')) + vi.stubEnv('CODEBURN_COPILOT_GLOBAL_STORAGE_DIR', join(tmpHome, 'no-global')) + vi.stubEnv('CODEBURN_COPILOT_JETBRAINS_DIR', join(tmpHome, 'no-jb')) + + const base = Date.now() - 91 * 24 * 60 * 60 * 1000 + const at = (offsetSec: number): string => new Date(base + offsetSec * 1000).toISOString() + const dir = join(sessionStateDir, 'sess-idle') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-idle\ncwd: /home/user/testproj\n') + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ type: 'assistant.message', timestamp: at(10), data: { messageId: 'msg-1', outputTokens: 25, toolRequests: [] } }), + JSON.stringify({ + type: 'session.shutdown', + timestamp: at(20), + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 1, cost: 1 }, + usage: { inputTokens: 20000, outputTokens: 25, cacheReadTokens: 17000, cacheWriteTokens: 2400, reasoningTokens: 0 }, + }, + }, + }, + }), + ].join('\n') + '\n') + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-idle', 12000, 10000, 1500, at(12)) // input 500 + insertStoreRow(dbPath, 'sess-idle', 8000, 7000, 900, at(15)) // input 100 + + const sumUsage = (projects: Awaited>) => { + const calls = projects.flatMap(p => p.sessions).flatMap(s => s.turns).flatMap(t => t.assistantCalls) + return { + input: calls.reduce((s, c) => s + c.usage.inputTokens, 0), + cacheRead: calls.reduce((s, c) => s + c.usage.cacheReadInputTokens, 0), + output: calls.reduce((s, c) => s + c.usage.outputTokens, 0), + } + } + + // Both runs: the store rows must serve — never zero. The session's + // events.jsonl is itself >90d old and follows the ordinary durable + // schedule (pruned even while on disk), so its per-turn output and + // rollup drop out; only the retainWhilePresent store keeps this + // session's record, exactly the crash-only-rows guarantee the flag + // exists for. + const first = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(first).toEqual({ input: 600, cacheRead: 17000, output: 0 }) + clearSessionCache() + const second = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(second).toEqual({ input: 600, cacheRead: 17000, output: 0 }) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// Shared setup for the serve-time precedence scenarios (m)/(n)/(o): a CLI +// session-state dir + a session-store.db, all env-pinned into tmpHome. +// ═══════════════════════════════════════════════════════════════════════════ +async function setupCopilotStoreEnv(): Promise<{ + dbPath: string + at: (offsetSec: number) => string + writeSession: (sessionId: string, opts: { output: number; rollup?: boolean }) => Promise + sumUsage: (projects: Awaited>) => { input: number; cacheRead: number; cacheWrite: number; output: number } +}> { + const sessionStateDir = join(tmpHome, 'session-state') + await mkdir(sessionStateDir, { recursive: true }) + const dbPath = join(tmpHome, 'session-store.db') + vi.stubEnv('CODEBURN_COPILOT_SESSION_STATE_DIR', sessionStateDir) + vi.stubEnv('CODEBURN_COPILOT_SESSION_STORE_DB', dbPath) + vi.stubEnv('CODEBURN_COPILOT_DISABLE_OTEL', '1') + vi.stubEnv('CODEBURN_COPILOT_WS_STORAGE_DIR', join(tmpHome, 'no-ws')) + vi.stubEnv('CODEBURN_COPILOT_GLOBAL_STORAGE_DIR', join(tmpHome, 'no-global')) + vi.stubEnv('CODEBURN_COPILOT_JETBRAINS_DIR', join(tmpHome, 'no-jb')) + + const base = Date.now() - 5 * 24 * 60 * 60 * 1000 + const at = (offsetSec: number): string => new Date(base + offsetSec * 1000).toISOString() + + // The rollup always uses the maintainer's repro numbers: cache-inclusive + // input 20,000 → uncached 600, cacheRead 17,000, cacheWrite 2,400. + const writeSession = async (sessionId: string, opts: { output: number; rollup?: boolean }): Promise => { + const dir = join(sessionStateDir, sessionId) + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), `id: ${sessionId}\ncwd: /home/user/testproj\n`) + const lines = [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ type: 'assistant.message', timestamp: at(10), data: { messageId: 'msg-1', outputTokens: opts.output, toolRequests: [] } }), + ] + if (opts.rollup) { + lines.push(JSON.stringify({ + type: 'session.shutdown', + timestamp: at(20), + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 1, cost: 1 }, + usage: { inputTokens: 20000, outputTokens: opts.output, cacheReadTokens: 17000, cacheWriteTokens: 2400, reasoningTokens: 0 }, + }, + }, + }, + })) + } + const eventsPath = join(dir, 'events.jsonl') + await writeFile(eventsPath, lines.join('\n') + '\n') + return eventsPath + } + + const sumUsage = (projects: Awaited>) => { + const calls = projects.flatMap(p => p.sessions).flatMap(s => s.turns).flatMap(t => t.assistantCalls) + return { + input: calls.reduce((s, c) => s + c.usage.inputTokens, 0), + cacheRead: calls.reduce((s, c) => s + c.usage.cacheReadInputTokens, 0), + cacheWrite: calls.reduce((s, c) => s + c.usage.cacheCreationInputTokens, 0), + output: calls.reduce((s, c) => s + c.usage.outputTokens, 0), + } + } + + return { dbPath, at, writeSession, sumUsage } +} + +// ═══════════════════════════════════════════════════════════════════════════ +// (m) The probe-to-parse race, at serve level: rows commit, THEN the shutdown +// line lands — counted once, store side wins +// ═══════════════════════════════════════════════════════════════════════════ +// The #946 round-2 repro. Under parse-time suppression, a coverage snapshot +// taken at discovery went stale the moment a session ended between probe and +// parse (rows commit BEFORE the shutdown line is appended), and the rollup +// was emitted beside the rows — doubling input 100 / cacheRead 8,000 / +// cacheWrite 2,000 durably. Serve-time precedence has no snapshot to go +// stale: whatever store rows made it into the serve set drop the session's +// rollups, no matter when either side was parsed or cached. +describe.skipIf(!isSqliteAvailable())('(m) rows-then-shutdown race counted once at serve time', () => { + it('drops the rollup cached after its session ended mid-run', async () => { + const { dbPath, at, writeSession, sumUsage } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + const eventsPath = await writeSession('sess-race', { output: 345 }) + + // Run 1 parses the live session mid-flight: no rows, no rollup yet. + const first = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(first).toEqual({ input: 0, cacheRead: 0, cacheWrite: 0, output: 345 }) + + // The session ends: rows committed FIRST (the CLI's write order), the + // shutdown rollup appended second, both between two codeburn runs. + clearSessionCache() + insertStoreRow(dbPath, 'sess-race', 10100, 8000, 2000, at(15)) // input 100 + await writeFile(eventsPath, JSON.stringify({ + type: 'session.shutdown', + timestamp: at(20), + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 1, cost: 1 }, + usage: { inputTokens: 10100, outputTokens: 345, cacheReadTokens: 8000, cacheWriteTokens: 2000, reasoningTokens: 0 }, + }, + }, + }, + }) + '\n', { flag: 'a' }) + + // Both representations parse and cache; the serve set holds the row, so + // the rollup is dropped: 100/8,000/2,000 once, not twice. + const second = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(second).toEqual({ input: 100, cacheRead: 8000, cacheWrite: 2000, output: 345 }) + + // Warm re-run: still once. + clearSessionCache() + const third = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(third).toEqual({ input: 100, cacheRead: 8000, cacheWrite: 2000, output: 345 }) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (n) A store holding no billable rows for a session never suppresses it +// ═══════════════════════════════════════════════════════════════════════════ +// Two scenarios collapse into this serve-set shape. (1) Atomic replacement: +// discovery probes store A, a writer renames store B over it, the parse +// reads B — under a parse-time coverage snapshot, sessions covered by A but +// absent from B would lose their rollups AND their rows; at serve time only +// what the parse actually produced suppresses. (2) The billable predicate: +// all-zero rows emit no calls, so a session with only those must keep its +// rollup — suppression on mere row-existence would zero its input/cache. +describe.skipIf(!isSqliteAvailable())('(n) no billable store rows → the rollup still counts', () => { + it('keeps the rollup when the served store holds only zero-usage rows for its session', async () => { + const { dbPath, at, writeSession, sumUsage } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + // sess-r: only an all-zero row (emits nothing). sess-other: billable. + insertStoreRow(dbPath, 'sess-r', 0, 0, 0, at(5)) + insertStoreRow(dbPath, 'sess-other', 5050, 5000, 0, at(6)) // input 50 + await writeSession('sess-r', { output: 25, rollup: true }) + + const totals = sumUsage(await parseAllSessions(undefined, 'copilot')) + // sess-r's rollup (600/17,000/2,400) + sess-other's row (50/5,000/0) + // + sess-r's per-turn output. + expect(totals).toEqual({ input: 650, cacheRead: 22000, cacheWrite: 2400, output: 25 }) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (o) Store-absence epoch: deleting the store changes nothing served +// ═══════════════════════════════════════════════════════════════════════════ +// The round-6 finding on the previous design: gating reconciliation on the +// store being DISCOVERED made an absence epoch flip served totals — the +// orphaned rows and the returning rollup both counted, and a daily history +// finalized during the epoch kept the doubled day forever even after the +// store came back. Reconciliation therefore reads only the cached serve set: +// the rows (durable orphans included) keep replacing the rollup, totals are +// identical before and after the deletion, and sealed history can never +// flip. The orphaned rows remain the session's record until the 90-day +// age-out prunes them — at which point the rollup calls, still cached in +// events.jsonl's entry, become the record again with the same exactly-once +// guarantee. +describe.skipIf(!isSqliteAvailable())('(o) absence epoch: served totals are independent of store presence', () => { + it('serves identical totals before and after the store file is deleted', async () => { + const { dbPath, at, writeSession, sumUsage } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-e', 12000, 10000, 1500, at(12)) // input 500 + insertStoreRow(dbPath, 'sess-e', 8000, 7000, 900, at(15)) // input 100 + await writeSession('sess-e', { output: 25, rollup: true }) + + // Store present: rows win, rollup reconciled away. + const first = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(first).toEqual({ input: 600, cacheRead: 17000, cacheWrite: 2400, output: 25 }) + + // The store vanishes; its cached rows become durable orphans. + clearSessionCache() + await rm(dbPath, { force: true }) + + // Nothing changes: the cached rows are still the session's record, the + // rollup stays reconciled away, and no epoch can double-count a day. + const second = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(second).toEqual({ input: 600, cacheRead: 17000, cacheWrite: 2400, output: 25 }) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (p) The reverse race: a row committing after the store read never zeroes +// the refresh +// ═══════════════════════════════════════════════════════════════════════════ +// The #946 round-4 finding 3 shape. The store parses before events.jsonl; a +// row can commit after that read but before the jsonl parse reaches the +// shutdown line. Under parse-time suppression the live coverage re-check +// then saw the row and suppressed the rollup — against a store snapshot +// that had emitted nothing — losing the request's input/cache for the whole +// refresh. Serve-time reconciliation cannot outrun the serve set: the +// rollup stands until rows actually land, partially-landed rows are topped +// up by the residual to the rollup's own totals, and full coverage retires +// the residual — counted once at every step and zero at none. +describe.skipIf(!isSqliteAvailable())('(p) reconciliation never outruns the served store rows', () => { + it('serves the rollup total at every stage while rows progressively land', async () => { + const { dbPath, at, writeSession, sumUsage } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + await writeSession('sess-rev', { output: 25, rollup: true }) + + // Refresh 1: the store was read before the rows committed — it holds + // nothing for this session. The rollup is the only record and must + // count; a zero here is the old design's lost refresh. + const first = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(first).toEqual({ input: 600, cacheRead: 17000, cacheWrite: 2400, output: 25 }) + + // The first row lands: it serves per-request, and the rollup usage it + // does not yet represent serves once as the residual — the total never + // dips below what the rollup proved happened. + clearSessionCache() + insertStoreRow(dbPath, 'sess-rev', 12000, 10000, 1500, at(12)) // input 500 + + const second = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(second).toEqual({ input: 600, cacheRead: 17000, cacheWrite: 2400, output: 25 }) + + // The second row completes coverage: totals unchanged, now served + // entirely by rows — the residual has retired to zero. + clearSessionCache() + insertStoreRow(dbPath, 'sess-rev', 8000, 7000, 900, at(15)) // input 100 + + const third = await parseAllSessions(undefined, 'copilot') + expect(sumUsage(third)).toEqual({ input: 600, cacheRead: 17000, cacheWrite: 2400, output: 25 }) + const keys = third.flatMap(p => p.sessions).flatMap(s => s.turns).flatMap(t => t.assistantCalls).map(c => c.deduplicationKey) + expect(keys.filter(k => k.startsWith('copilot-store:')).length).toBe(2) + expect(keys.some(k => k.includes(':shutdown-residual:'))).toBe(false) + expect(keys.some(k => k.includes(':shutdown:'))).toBe(false) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (s) Behavioral weight: supplementary accounting never fabricates activity +// ═══════════════════════════════════════════════════════════════════════════ +// The round-6 finding 1 + the maintainer's weight contract. A shutdown rollup +// is aggregate accounting, never a request; a store row is a real request but +// pairs 1:1 with its served per-turn call when one exists. Blanket weight 1 +// fabricated apiCalls/turns/modelCalls for every covered request; blanket +// weight 0 would hide crash-recovered requests. The pinned contract: +// JSONL + matching row = 1 call/turn; JSONL + rollup = the JSONL count only; +// store-only request = 1 call; rollup-only = 0 calls with full usage. +describe.skipIf(!isSqliteAvailable())('(s) behavioral weight of store rows and rollups', () => { + it('pins call/turn weight across the four review scenarios', async () => { + const { dbPath, at, writeSession } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + + // A: one real request, both representations served. + await writeSession('sess-paired', { output: 25 }) + insertStoreRow(dbPath, 'sess-paired', 12000, 10000, 1500, at(12)) + // B: pre-store session — per-turn call + rollup, no rows. + await writeSession('sess-rollup', { output: 30, rollup: true }) + // C: store-only request (crash lost the per-turn call; no session-state). + insertStoreRow(dbPath, 'sess-crash', 8000, 7000, 900, at(15)) + + const sessions = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + const byId = new Map(sessions.map(s => [s.sessionId, s])) + const modelCalls = (s: NonNullable>): number => + Object.values(s.modelBreakdown).reduce((sum, m) => sum + m.calls, 0) + + const paired = byId.get('sess-paired')! + expect(paired.apiCalls).toBe(1) + expect(paired.turns.length).toBe(1) + expect(modelCalls(paired)).toBe(1) + expect(paired.totalInputTokens).toBe(500) // the row's tokens count in full + expect(paired.totalCacheReadTokens).toBe(10000) + expect(paired.totalOutputTokens).toBe(25) + + const withRollup = byId.get('sess-rollup')! + expect(withRollup.apiCalls).toBe(1) // the per-turn call only + expect(withRollup.turns.length).toBe(1) + expect(modelCalls(withRollup)).toBe(1) + expect(withRollup.totalInputTokens).toBe(600) // rollup tokens retained + + const crash = byId.get('sess-crash')! + expect(crash.apiCalls).toBe(1) // a real, store-only request + expect(crash.turns.length).toBe(1) + expect(crash.totalInputTokens).toBe(100) + }) + + it('pairs rows per model: a subagent row without its per-turn call still counts', async () => { + const { dbPath, at, writeSession } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + // One sonnet request served both ways, plus a haiku subagent request + // whose per-turn call was lost — the haiku row must not pair against the + // sonnet call. + await writeSession('sess-multi', { output: 25 }) // sonnet per-turn call + insertStoreRow(dbPath, 'sess-multi', 12000, 10000, 1500, at(12)) // sonnet row → paired + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const db = new DatabaseSync(dbPath) + db.prepare( + `INSERT INTO assistant_usage_events + (session_id, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, reasoning_tokens, created_at) + VALUES ('sess-multi', 'claude-haiku-4-5', 5050, 0, 5000, 0, 0, ?)` + ).run(at(14)) + db.close() + + const session = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .find(s => s.sessionId === 'sess-multi')! + expect(session.apiCalls).toBe(2) // the sonnet request + the haiku request + const calls = Object.fromEntries(Object.entries(session.modelBreakdown).map(([m, b]) => [m, b.calls])) + expect(Object.values(calls).reduce((a, b) => a + b, 0)).toBe(2) + expect(session.totalInputTokens).toBe(500 + 50) + }) + + it('serves a rollup-only session with zero call weight but full usage', async () => { + const { dbPath, at } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + // events.jsonl holding ONLY the shutdown rollup — no per-turn events + // survived. Its tokens are real; its "calls" are not. + const dir = join(tmpHome, 'session-state', 'sess-agg') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-agg\ncwd: /home/user/testproj\n') + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ + type: 'session.shutdown', + timestamp: at(20), + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 2, cost: 1 }, + usage: { inputTokens: 20000, outputTokens: 0, cacheReadTokens: 17000, cacheWriteTokens: 2400, reasoningTokens: 0 }, + }, + }, + }, + }), + ].join('\n') + '\n') + + const sessions = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + const agg = sessions.find(s => s.sessionId === 'sess-agg') + expect(agg).toBeDefined() + expect(agg!.apiCalls).toBe(0) + expect(agg!.totalInputTokens).toBe(600) + expect(agg!.totalCacheReadTokens).toBe(17000) + expect(agg!.totalCostUSD).toBeGreaterThan(0) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (t) A deferred store read marks hydration incomplete +// ═══════════════════════════════════════════════════════════════════════════ +// The round-6 finding 2. A changed store whose read defers (busy, EACCES — +// every retryable shape funnels through the same busy-shaped throw) keeps +// serving its cached rows, but the data the read would have added is missing +// from the pass: reporting hydration complete would let the daily backfill +// finalize history without it. The fence must hold until the read lands. +describe.skipIf(!isSqliteAvailable())('(t) deferred store read marks hydration incomplete', () => { + it.skipIf(process.getuid?.() === 0)('holds the fence while the changed store is unreadable, then recovers', async () => { + const { chmod } = await import('fs/promises') + const { dbPath, at, writeSession, sumUsage } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-h', 12000, 10000, 1500, at(12)) // input 500 + await writeSession('sess-h', { output: 25 }) + + const first = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(first).toEqual({ input: 500, cacheRead: 10000, cacheWrite: 1500, output: 25 }) + expect(isSessionHydrationComplete()).toBe(true) + + // The store grows, then becomes unreadable before the refresh reads it. + clearSessionCache() + insertStoreRow(dbPath, 'sess-h', 8000, 7000, 900, at(20)) // input 100 + await chmod(dbPath, 0o000) + try { + const second = sumUsage(await parseAllSessions(undefined, 'copilot')) + // Cached rows keep serving — the deferral is invisible in totals... + expect(second).toEqual({ input: 500, cacheRead: 10000, cacheWrite: 1500, output: 25 }) + // ...but the pass must not claim full hydration: a row is missing. + expect(isSessionHydrationComplete()).toBe(false) + } finally { + await chmod(dbPath, 0o644) + } + + // Readable again: the deferred row lands and the fence lifts. + clearSessionCache() + const third = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(third).toEqual({ input: 600, cacheRead: 17000, cacheWrite: 2400, output: 25 }) + expect(isSessionHydrationComplete()).toBe(true) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (t2) The fence actually HOLDS THE DAILY WATERMARK, end to end +// ═══════════════════════════════════════════════════════════════════════════ +// The maintainer's refinement asked for a production-path regression proving +// "busy store + unchanged cached JSONL => hydration incomplete AND daily +// watermark held" — the flag alone is not the contract. This wires the REAL +// isSessionHydrationComplete into ensureCacheHydrated (no stub) and asserts +// the daily cache refuses to finalize while the store's re-read is deferred. +describe.skipIf(!isSqliteAvailable())('(t2) deferred store holds the daily watermark', () => { + it.skipIf(process.getuid?.() === 0)('leaves the daily cache incomplete until the store is readable', async () => { + const { chmod } = await import('fs/promises') + const { dbPath, at, writeSession } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-wm', 12000, 10000, 1500, at(12)) + await writeSession('sess-wm', { output: 25 }) + + // Seed a daily cache whose watermark is deliberately BEHIND yesterday, so + // every run below faces a real backfill gap to seal while the watermark is + // non-null and its preservation is observable. The session cache stays + // warm and the JSONL unchanged — the reviewer's shape, where no + // session-state parser runs and only the store's own deferral can reach + // the fence. + const dayStr = (n: number): string => { + const d = new Date() + d.setDate(d.getDate() - n) + return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, '0')}-${String(d.getDate()).padStart(2, '0')}` + } + const seededWatermark = dayStr(5) + const seedDailyCache = async () => { + await saveDailyCache({ + version: DAILY_CACHE_VERSION, + savingsConfigHash: '', + tzKey: currentTzKey(), + lastComputedDate: seededWatermark, + complete: true, + watermarkTrusted: true, + days: [], + }) + } + const hydrate = async () => { + clearSessionCache() + return ensureCacheHydrated( + range => parseAllSessions(range, 'copilot'), + aggregateProjectsIntoDays, + '', + // The production wiring: the real flag, not a stub. + isSessionHydrationComplete, + ) + } + + // The store grows and turns unreadable before the refresh reads it. + await seedDailyCache() + insertStoreRow(dbPath, 'sess-wm', 8000, 7000, 900, at(20)) + await chmod(dbPath, 0o000) + let degraded: Awaited> + try { + degraded = await hydrate() + expect(isSessionHydrationComplete()).toBe(false) + // The contract the reviewer asked for: history does not seal, and the + // watermark is HELD where it was — not advanced by a parse that never + // read the new row, and not reset either. + expect(degraded.complete).toBe(false) + expect(degraded.lastComputedDate).toBe(seededWatermark) + } finally { + await chmod(dbPath, 0o644) + } + + // Readable again — and healing must happen IN PLACE, against the same + // persisted incomplete cache the degraded run left behind (no reseed): + // the deferred row lands, the day seals, and the watermark advances. + const healed = await hydrate() + expect(isSessionHydrationComplete()).toBe(true) + expect(healed.complete).toBe(true) + expect(healed.lastComputedDate).not.toBe(seededWatermark) + expect(healed.lastComputedDate! > seededWatermark).toBe(true) + // The row deferred during the outage is in the sealed history. + const sealed = healed.days.reduce((s, d) => s + (d.providers['copilot']?.inputTokens ?? 0), 0) + expect(sealed).toBe(600) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (u) A project learned after rows were cached cannot split the session +// ═══════════════════════════════════════════════════════════════════════════ +// The round-6 companion finding. Store rows cached before their session's +// session-state dir existed carry the store's cwd-derived project; when +// events.jsonl later appears (workspace.yaml cwd), the store file itself may +// be unchanged — its cached calls are reused verbatim. Project is part of the +// session grouping key, so without serve-time unification one real session +// splits in two. The serve pass rewrites cached store calls to the +// session-state project whenever the serve set knows it. +describe.skipIf(!isSqliteAvailable())('(u) late-learned project identity unifies cached store rows', () => { + it('serves one session under the session-state project after it appears', async () => { + const { dbPath, at, writeSession } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-late', 12000, 10000, 1500, at(12), 0, '/home/user/storeproj') + + const s1 = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .find(s => s.sessionId === 'sess-late')! + expect(s1.project).toBe('storeproj') + + // The session-state dir appears; the store file is UNCHANGED, so its + // cached call still carries 'storeproj' until serve-time unification. + clearSessionCache() + await writeSession('sess-late', { output: 25 }) + + const sessions = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .filter(s => s.sessionId === 'sess-late') + expect(sessions).toHaveLength(1) + expect(sessions[0]!.project).toBe('testproj') + expect(sessions[0]!.totalInputTokens).toBe(500) + expect(sessions[0]!.totalOutputTokens).toBe(25) + expect(sessions[0]!.apiCalls).toBe(1) + }) + + it('keeps one session when the session-state dir is pruned and the jsonl orphans', async () => { + // The reverse shape (round-6.5 finding): events.jsonl becomes a durable + // orphan, losing its source.project, while the store rows live on. Both + // sides must adopt the surviving label — the store's own — or one real + // request serves as two sessions with doubled call weight. + const { dbPath, at, writeSession } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-prune', 12000, 10000, 1500, at(12), 0, '/home/user/storeproj') + await writeSession('sess-prune', { output: 25 }) + + const s1 = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .filter(s => s.sessionId === 'sess-prune') + expect(s1).toHaveLength(1) + expect(s1[0]!.apiCalls).toBe(1) + + clearSessionCache() + await rm(join(tmpHome, 'session-state', 'sess-prune'), { recursive: true, force: true }) + + const s2 = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .filter(s => s.sessionId === 'sess-prune') + expect(s2).toHaveLength(1) + // The rows were cached carrying the jsonl-derived label, so the session + // keeps it even after the jsonl orphans — stable, and never split. + expect(s2[0]!.project).toBe('testproj') + expect(s2[0]!.apiCalls).toBe(1) + expect(s2[0]!.totalInputTokens).toBe(500) + expect(s2[0]!.totalOutputTokens).toBe(25) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (v) A same-path store reset cannot swallow new usage +// ═══════════════════════════════════════════════════════════════════════════ +// The round-6 finding 4. Recreating the DB at the same path restarts the +// AUTOINCREMENT sequence; under a bare : key the durable union +// rejected the recreated row as already-cached and its usage vanished. The +// content-discriminated key admits it; the original row remains served as +// real past usage (the durable contract: cached history never shrinks). +describe.skipIf(!isSqliteAvailable())('(v) same-path store reset', () => { + it('serves the recreated row alongside the durable original', async () => { + const { dbPath, at, sumUsage } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-reset', 12000, 10000, 1500, at(12)) // input 500 + const first = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(first).toEqual({ input: 500, cacheRead: 10000, cacheWrite: 1500, output: 0 }) + + clearSessionCache() + await rm(dbPath, { force: true }) + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-reset', 8000, 7000, 900, at(30)) // row id 1 again + const second = sumUsage(await parseAllSessions(undefined, 'copilot')) + expect(second).toEqual({ input: 600, cacheRead: 17000, cacheWrite: 2400, output: 0 }) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (w) Mixed coverage: a crash tail cannot cancel a leg's missing rows +// ═══════════════════════════════════════════════════════════════════════════ +// The round-6.5 converging finding (grok + gpt-5.6-sol independently). A +// lifetime `max(0, rollup − allRows)` residual lets store rows the rollup +// never covered (a crash tail, a later reset) cancel usage the rollup DID +// cover whose rows are missing. Residuals are therefore computed per rollup +// leg over only the rows in that leg's interval — rows commit before their +// leg's shutdown line, so a row after the leg belongs to the tail, never to +// the subtraction. +describe.skipIf(!isSqliteAvailable())('(w) per-leg residual under mixed coverage', () => { + it('serves the covered gap AND the crash tail in full', async () => { + const { dbPath, at } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + + const dir = join(tmpHome, 'session-state', 'sess-mix') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-mix\ncwd: /home/user/testproj\n') + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ type: 'assistant.message', timestamp: at(5), data: { messageId: 'msg-1', outputTokens: 25, toolRequests: [] } }), + // The rollup covers requests R1+R2 (cache-inclusive input 10,200 → + // uncached 200, cacheRead 10,000). + JSON.stringify({ + type: 'session.shutdown', + timestamp: at(20), + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 2, cost: 1 }, + usage: { inputTokens: 10200, outputTokens: 25, cacheReadTokens: 10000, cacheWriteTokens: 0, reasoningTokens: 0 }, + }, + }, + }, + }), + ].join('\n') + '\n') + + // R1's row exists (input 100 / cacheRead 5,000); R2's row is missing + // (pre-store leg). R3 is a crash-tail row AFTER the shutdown (input 300) + // that no rollup ever covered. + insertStoreRow(dbPath, 'sess-mix', 5100, 5000, 0, at(10)) // R1 + insertStoreRow(dbPath, 'sess-mix', 300, 0, 0, at(30)) // R3, after the leg + + const session = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .find(s => s.sessionId === 'sess-mix')! + // Truth: R1 (100/5,000) + R2 (100/5,000, via the leg residual) + R3 (300). + // A lifetime subtraction would have served input 400 — R3's crash input + // cancelling R2's missing row. + expect(session.totalInputTokens).toBe(500) + expect(session.totalCacheReadTokens).toBe(10000) + expect(session.totalOutputTokens).toBe(25) + // Weight: the per-turn call + the unpaired crash row. + expect(session.apiCalls).toBe(2) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (x) Per-leg residuals keep each leg's gap on that leg's own day +// ═══════════════════════════════════════════════════════════════════════════ +// Round-6.5: one residual stamped at the LAST leg moved every earlier leg's +// uncovered usage onto the final day — day 1 sealed 0, day 2 sealed double. +// Each leg's residual is anchored at that leg's own timestamp. +describe.skipIf(!isSqliteAvailable())('(x) multi-leg residual day attribution', () => { + it('serves each leg\'s uncovered usage on its own day', async () => { + const dayN = Date.now() - 5 * 24 * 60 * 60 * 1000 + const at = (offsetHours: number): string => new Date(dayN + offsetHours * 3600 * 1000).toISOString() + const sessionStateDir = join(tmpHome, 'session-state') + await mkdir(sessionStateDir, { recursive: true }) + const dbPath = join(tmpHome, 'session-store.db') + vi.stubEnv('CODEBURN_COPILOT_SESSION_STATE_DIR', sessionStateDir) + vi.stubEnv('CODEBURN_COPILOT_SESSION_STORE_DB', dbPath) + vi.stubEnv('CODEBURN_COPILOT_DISABLE_OTEL', '1') + vi.stubEnv('CODEBURN_COPILOT_WS_STORAGE_DIR', join(tmpHome, 'no-ws')) + vi.stubEnv('CODEBURN_COPILOT_GLOBAL_STORAGE_DIR', join(tmpHome, 'no-global')) + vi.stubEnv('CODEBURN_COPILOT_JETBRAINS_DIR', join(tmpHome, 'no-jb')) + + // A resumed session: leg 1 shuts down on day 1 (cumulative input 500, + // cache-free for arithmetic clarity), leg 2 the next day (cumulative 800 + // → delta 300). The store has one leg-2 row (100); leg 1 predates the + // store entirely and leg 2 is only partially covered, so BOTH legs carry + // a nonzero residual on their own day. + const dir = join(sessionStateDir, 'sess-legs') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-legs\ncwd: /home/user/testproj\n') + const rollup = (ts: string, cumulativeInput: number): string => JSON.stringify({ + type: 'session.shutdown', + timestamp: ts, + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 1, cost: 1 }, + usage: { inputTokens: cumulativeInput, outputTokens: 10, cacheReadTokens: 0, cacheWriteTokens: 0, reasoningTokens: 0 }, + }, + }, + }, + }) + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ type: 'assistant.message', timestamp: at(1), data: { messageId: 'msg-1', outputTokens: 10, toolRequests: [] } }), + rollup(at(2), 500), + rollup(at(26), 800), + ].join('\n') + '\n') + + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-legs', 100, 0, 0, at(25)) + + const inputIn = async (startH: number, endH: number): Promise => { + const projects = await parseAllSessions( + { start: new Date(dayN + startH * 3600 * 1000), end: new Date(dayN + endH * 3600 * 1000) }, 'copilot') + return projects.flatMap(p => p.sessions).flatMap(s => s.turns).flatMap(t => t.assistantCalls) + .reduce((s, c) => s + c.usage.inputTokens, 0) + } + + expect(await inputIn(-1, 12)).toBe(500) // day 1: leg 1's gap, at leg 1's stamp + expect(await inputIn(12, 36)).toBe(300) // day 2: the row (100) + leg 2's residual (200) + expect(await inputIn(-1, 36)).toBe(800) // both: exactly once + + // A range that excludes every behavioral turn (the per-turn call) but + // holds residual turns from different days: they must stay SEPARATE + // turns on their own days — an anchorless merge would re-anchor day-2 + // accounting onto day 1's turn stamp. + const anchorless = (await parseAllSessions( + { start: new Date(dayN + 1.5 * 3600 * 1000), end: new Date(dayN + 36 * 3600 * 1000) }, 'copilot')) + .flatMap(p => p.sessions).filter(s => s.sessionId === 'sess-legs') + const turnDays = anchorless.flatMap(s => s.turns).map(t => new Date(t.timestamp).toISOString().slice(0, 10)) + expect(new Set(turnDays).size).toBeGreaterThanOrEqual(2) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (w2) Residual edge shapes: equal-timestamp legs, unparseable leg stamps, +// and crash rows outside the pairing window +// ═══════════════════════════════════════════════════════════════════════════ +describe.skipIf(!isSqliteAvailable())('(w2) residual and pairing edge shapes', () => { + it('coalesces equal-timestamp legs so their shared rows subtract once', async () => { + const { dbPath, at } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + const dir = join(tmpHome, 'session-state', 'sess-ties') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-ties\ncwd: /home/user/testproj\n') + const rollup = (ts: string, cumulativeInput: number): string => JSON.stringify({ + type: 'session.shutdown', + timestamp: ts, + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 1, cost: 1 }, + usage: { inputTokens: cumulativeInput, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0, reasoningTokens: 0 }, + }, + }, + }, + }) + // Two shutdown legs stamped the SAME second (deltas 100 and 200); the + // store holds one 200-token row at that instant. A strict interval rule + // hands the row to the first leg and mints a full 200 residual for the + // second (serving 400); coalescing subtracts it once (serving 300). + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + rollup(at(20), 100), + rollup(at(20), 300), + ].join('\n') + '\n') + insertStoreRow(dbPath, 'sess-ties', 200, 0, 0, at(20)) + + const input = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .filter(s => s.sessionId === 'sess-ties') + .reduce((s, x) => s + x.totalInputTokens, 0) + expect(input).toBe(300) + }) + + it('serves a rollup with an unparseable timestamp instead of silently dropping it', async () => { + const { dbPath, at } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + const dir = join(tmpHome, 'session-state', 'sess-badts') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-badts\ncwd: /home/user/testproj\n') + // The leg's timestamp cannot parse, so it can never enter the residual + // sweep — dropping it because rows exist would silently lose its usage. + // It serves unchanged (weightless); the bounded overlap with the row is + // the documented corruption-shaped trade (over-serve, never lose). + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ + type: 'session.shutdown', + timestamp: 'not-a-timestamp', + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 1, cost: 1 }, + usage: { inputTokens: 500, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0, reasoningTokens: 0 }, + }, + }, + }, + }), + ].join('\n') + '\n') + insertStoreRow(dbPath, 'sess-badts', 100, 0, 0, at(10)) + + const sessions = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .filter(s => s.sessionId === 'sess-badts') + const input = sessions.reduce((s, x) => s + x.totalInputTokens, 0) + expect(input).toBe(600) // row 100 + un-droppable rollup 500; never 100 + expect(sessions.reduce((s, x) => s + x.apiCalls, 0)).toBe(1) // the row; the rollup stays weightless + + // The rollup adopted a STABLE valid timestamp (its file's preceding + // valid stamp, else the session's earliest), so a RANGED query (the + // shape the daily backfill uses) serves the same 600 — a raw + // unparseable stamp would be invisible to callsInRange and quietly lose + // the 500 from every sealed day. + const base = new Date(at(0)).getTime() + const rangedInput = async (): Promise => + (await parseAllSessions( + { start: new Date(base - 3600 * 1000), end: new Date(base + 3600 * 1000) }, 'copilot')) + .flatMap(p => p.sessions).filter(s => s.sessionId === 'sess-badts') + .reduce((s, x) => s + x.totalInputTokens, 0) + expect(await rangedInput()).toBe(600) + + // The session resumes a day later: the fallback must NOT move with it — + // a day sealed with the rollup would otherwise lose it to the new day + // and the daily union would count it twice. + clearSessionCache() + insertStoreRow(dbPath, 'sess-badts', 50, 0, 0, at(86400)) + expect(await rangedInput()).toBe(600) + }) + + it('anchors a timestamp-less shutdown after its rows, not at session start', async () => { + // Audit finding: the leg's stamp anchors its interval in the residual + // sweep. A shutdown event with no `timestamp` fell back to + // sessionStartTime — BEFORE every row — so the leg covered nothing and + // re-minted its whole usage beside the rows it duplicates (600/17,000 + // served as 1,200/34,000). The fallback now prefers the last event seen, + // which is at the end of the leg where a shutdown actually happens. + const { dbPath, at } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + const dir = join(tmpHome, 'session-state', 'sess-nots') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-nots\ncwd: /home/user/testproj\n') + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ type: 'assistant.message', timestamp: at(18), data: { messageId: 'msg-1', outputTokens: 25, toolRequests: [] } }), + // No `timestamp` on the shutdown; sessionStartTime precedes every row. + JSON.stringify({ + type: 'session.shutdown', + data: { + shutdownType: 'routine', + sessionStartTime: at(0), + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 2, cost: 1 }, + usage: { inputTokens: 20000, outputTokens: 25, cacheReadTokens: 17000, cacheWriteTokens: 2400, reasoningTokens: 0 }, + }, + }, + }, + }), + ].join('\n') + '\n') + insertStoreRow(dbPath, 'sess-nots', 12000, 10000, 1500, at(12)) // input 500 + insertStoreRow(dbPath, 'sess-nots', 8000, 7000, 900, at(15)) // input 100 + + const session = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .find(s => s.sessionId === 'sess-nots')! + expect(session.totalInputTokens).toBe(600) + expect(session.totalCacheReadTokens).toBe(17000) + expect(session.totalCacheWriteTokens).toBe(2400) + }) + + it('keeps a crash row\'s call weight when it sits outside the pairing window', async () => { + const { dbPath, at, writeSession } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + // The per-turn call at at(10) lost its own row; a crash-only row lands 5 + // minutes later. A wide pairing window would pair the two and hide the + // crash request; the tight window leaves the row unpaired and counted. + await writeSession('sess-far', { output: 25 }) + insertStoreRow(dbPath, 'sess-far', 100, 0, 0, at(310)) + + const session = (await parseAllSessions(undefined, 'copilot')).flatMap(p => p.sessions) + .find(s => s.sessionId === 'sess-far')! + expect(session.apiCalls).toBe(2) + expect(session.totalInputTokens).toBe(100) + expect(session.totalOutputTokens).toBe(25) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (y) Pairing is range-invariant: adjacent day queries stay additive +// ═══════════════════════════════════════════════════════════════════════════ +// Round-6.5 (gpt-5.6-sol): weights recomputed from a range slice let a store +// row and its per-turn call each count as a call on opposite sides of a day +// boundary — two calls for one request across adjacent queries. The pairing +// is computed once over the full serve set and arrives as a key set, so a +// slice cannot change a row's verdict. +describe.skipIf(!isSqliteAvailable())('(y) range-invariant behavioral pairing', () => { + it('counts one call across two adjacent ranges that split row from per-turn call', async () => { + const { dbPath, at, writeSession } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + // writeSession puts the per-turn call at at(10); the row lands at at(60), + // both within the pairing window. + await writeSession('sess-split', { output: 25 }) + insertStoreRow(dbPath, 'sess-split', 12000, 10000, 1500, at(60)) + + const base = new Date(at(0)).getTime() + const range = async (fromSec: number, toSec: number) => { + const projects = await parseAllSessions( + { start: new Date(base + fromSec * 1000), end: new Date(base + toSec * 1000) }, 'copilot') + const sessions = projects.flatMap(p => p.sessions).filter(s => s.sessionId === 'sess-split') + return { + apiCalls: sessions.reduce((s, x) => s + x.apiCalls, 0), + input: sessions.reduce((s, x) => s + x.totalInputTokens, 0), + output: sessions.reduce((s, x) => s + x.totalOutputTokens, 0), + } + } + + // Range A holds only the per-turn call; range B only the row. + const a = await range(0, 30) + expect(a).toEqual({ apiCalls: 1, input: 0, output: 25 }) + const b = await range(31, 120) + expect(b).toEqual({ apiCalls: 0, input: 500, output: 0 }) + + // The full range pairs them the same way: still exactly one call. + const full = await range(0, 120) + expect(full).toEqual({ apiCalls: 1, input: 500, output: 25 }) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (z) The hydration verdict travels with its result +// ═══════════════════════════════════════════════════════════════════════════ +describe.skipIf(!isSqliteAvailable())('(z) hydration verdict integrity', () => { + // Round-6.5 (gpt-5.6-sol): a memoized partial parse served after a later, + // complete parse inherited the later parse's `true`, letting the daily + // backfill seal history around the deferred data. + it.skipIf(process.getuid?.() === 0)('a memo hit restores the verdict its data was parsed under', async () => { + const { chmod } = await import('fs/promises') + const { dbPath, at, writeSession } = await setupCopilotStoreEnv() + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-memo', 12000, 10000, 1500, at(12)) + await writeSession('sess-memo', { output: 25 }) + await parseAllSessions(undefined, 'copilot') + expect(isSessionHydrationComplete()).toBe(true) + + // The store grows and becomes unreadable: the re-parse defers (false) + // and that verdict is memoized with the result. + clearSessionCache() + insertStoreRow(dbPath, 'sess-memo', 8000, 7000, 900, at(20)) + await chmod(dbPath, 0o000) + try { + await parseAllSessions(undefined, 'copilot') + expect(isSessionHydrationComplete()).toBe(false) + + // An unrelated provider parses completely and flips the global to true… + await parseAllSessions(undefined, 'claude') + expect(isSessionHydrationComplete()).toBe(true) + + // …but re-serving the deferred copilot result from the memo must + // restore ITS verdict, not inherit the later parse's. + await parseAllSessions(undefined, 'copilot') + expect(isSessionHydrationComplete()).toBe(false) + } finally { + await chmod(dbPath, 0o644) + } + }) + + // Round-6.5 (gpt-5.6-sol): a source whose FINGERPRINT cannot be read was + // skipped before any parser could raise the deferral shape, leaving the + // fence open while the cached (stale) rows served. + it.skipIf(process.getuid?.() === 0)('an unreadable fingerprint defers instead of silently skipping', async () => { + const { chmod } = await import('fs/promises') + const sessionStateDir = join(tmpHome, 'session-state') + await mkdir(sessionStateDir, { recursive: true }) + const storeDir = join(tmpHome, 'store-dir') + await mkdir(storeDir, { recursive: true }) + const dbPath = join(storeDir, 'session-store.db') + vi.stubEnv('CODEBURN_COPILOT_SESSION_STATE_DIR', sessionStateDir) + vi.stubEnv('CODEBURN_COPILOT_SESSION_STORE_DB', dbPath) + vi.stubEnv('CODEBURN_COPILOT_DISABLE_OTEL', '1') + vi.stubEnv('CODEBURN_COPILOT_WS_STORAGE_DIR', join(tmpHome, 'no-ws')) + vi.stubEnv('CODEBURN_COPILOT_GLOBAL_STORAGE_DIR', join(tmpHome, 'no-global')) + vi.stubEnv('CODEBURN_COPILOT_JETBRAINS_DIR', join(tmpHome, 'no-jb')) + const base = Date.now() - 5 * 24 * 60 * 60 * 1000 + const at = (offsetSec: number): string => new Date(base + offsetSec * 1000).toISOString() + + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-fp', 12000, 10000, 1500, at(12)) + const first = await parseAllSessions(undefined, 'copilot') + expect(first.flatMap(p => p.sessions).find(s => s.sessionId === 'sess-fp')!.totalInputTokens).toBe(500) + expect(isSessionHydrationComplete()).toBe(true) + + // The store grows, then its parent dir loses traversal: discovery still + // emits the source (EACCES is not absence), the fingerprint read fails, + // and the pass must report incomplete hydration — the new row is missing. + clearSessionCache() + insertStoreRow(dbPath, 'sess-fp', 8000, 7000, 900, at(20)) + await chmod(storeDir, 0o000) + try { + const second = await parseAllSessions(undefined, 'copilot') + expect(second.flatMap(p => p.sessions).find(s => s.sessionId === 'sess-fp')!.totalInputTokens).toBe(500) + expect(isSessionHydrationComplete()).toBe(false) + } finally { + await chmod(storeDir, 0o755) + } + + clearSessionCache() + const third = await parseAllSessions(undefined, 'copilot') + expect(third.flatMap(p => p.sessions).find(s => s.sessionId === 'sess-fp')!.totalInputTokens).toBe(600) + expect(isSessionHydrationComplete()).toBe(true) + }) +}) + +// ═══════════════════════════════════════════════════════════════════════════ +// (j) Rollup-day reattribution: usage lands on the request days, not the day +// the CLI finally shut down +// ═══════════════════════════════════════════════════════════════════════════ +// Observed in the wild: a session ran entirely on day N (per-request DB rows) +// but its session.shutdown rollup was stamped the NEXT morning when the CLI +// was closed. The rollup path put the whole session's input/cache on day N+1; +// with the store covering the session, the tokens must land on day N and the +// session must contribute NOTHING to day N+1 — while still counting exactly +// once in an unfiltered (lifetime) parse. This is the per-day attribution +// change the daily-cache v18 bump re-derives for. +describe.skipIf(!isSqliteAvailable())('(j) rollup-day reattribution to request days', () => { + it('counts a next-morning-shutdown session on its request day only', async () => { + const sessionStateDir = join(tmpHome, 'session-state') + await mkdir(sessionStateDir, { recursive: true }) + const dbPath = join(tmpHome, 'session-store.db') + vi.stubEnv('CODEBURN_COPILOT_SESSION_STATE_DIR', sessionStateDir) + vi.stubEnv('CODEBURN_COPILOT_SESSION_STORE_DB', dbPath) + vi.stubEnv('CODEBURN_COPILOT_DISABLE_OTEL', '1') + vi.stubEnv('CODEBURN_COPILOT_WS_STORAGE_DIR', join(tmpHome, 'no-ws')) + vi.stubEnv('CODEBURN_COPILOT_GLOBAL_STORAGE_DIR', join(tmpHome, 'no-global')) + vi.stubEnv('CODEBURN_COPILOT_JETBRAINS_DIR', join(tmpHome, 'no-jb')) + + // "Day N" = 5 days ago; the shutdown lands ~19h later ("next morning"). + const dayN = Date.now() - 5 * 24 * 60 * 60 * 1000 + const at = (offsetHours: number): string => new Date(dayN + offsetHours * 3600 * 1000).toISOString() + + const dir = join(sessionStateDir, 'sess-overnight') + await mkdir(dir, { recursive: true }) + await writeFile(join(dir, 'workspace.yaml'), 'id: sess-overnight\ncwd: /home/user/testproj\n') + await writeFile(join(dir, 'events.jsonl'), [ + JSON.stringify({ type: 'session.model_change', timestamp: at(0), data: { newModel: 'claude-sonnet-4-5' } }), + JSON.stringify({ type: 'assistant.message', timestamp: at(1), data: { messageId: 'msg-1', outputTokens: 25, toolRequests: [] } }), + JSON.stringify({ + type: 'session.shutdown', + timestamp: at(19), + data: { + shutdownType: 'routine', + modelMetrics: { + 'claude-sonnet-4-5': { + requests: { count: 2, cost: 1 }, + usage: { inputTokens: 20000, outputTokens: 25, cacheReadTokens: 17000, cacheWriteTokens: 2400, reasoningTokens: 0 }, + }, + }, + }, + }), + ].join('\n') + '\n') + + createStoreDb(dbPath) + insertStoreRow(dbPath, 'sess-overnight', 12000, 10000, 1500, at(1)) // input 500 + insertStoreRow(dbPath, 'sess-overnight', 8000, 7000, 900, at(2)) // input 100 + + const inventory = (projects: Awaited>) => { + const sessions = projects.flatMap(p => p.sessions).filter(s => s.turns.some(t => t.assistantCalls.length > 0)) + const calls = sessions.flatMap(s => s.turns).flatMap(t => t.assistantCalls) + return { + sessions: sessions.length, + input: calls.reduce((s, c) => s + c.usage.inputTokens, 0), + cacheRead: calls.reduce((s, c) => s + c.usage.cacheReadInputTokens, 0), + output: calls.reduce((s, c) => s + c.usage.outputTokens, 0), + } + } + + // Lifetime: exactly one session, tokens counted once, from the store. + const lifetime = inventory(await parseAllSessions(undefined, 'copilot')) + expect(lifetime).toEqual({ sessions: 1, input: 600, cacheRead: 17000, output: 25 }) + + // A range covering only the shutdown stamp (rollup path would have put + // 600/17000 here): the session must contribute nothing at all. + const shutdownDay = inventory(await parseAllSessions( + { start: new Date(dayN + 12 * 3600 * 1000), end: new Date(dayN + 36 * 3600 * 1000) }, 'copilot')) + expect(shutdownDay).toEqual({ sessions: 0, input: 0, cacheRead: 0, output: 0 }) + + // The request day carries everything. + const requestDay = inventory(await parseAllSessions( + { start: new Date(dayN - 1 * 3600 * 1000), end: new Date(dayN + 12 * 3600 * 1000) }, 'copilot')) + expect(requestDay).toEqual({ sessions: 1, input: 600, cacheRead: 17000, output: 25 }) + }) +}) diff --git a/tests/plan-usage.test.ts b/tests/plan-usage.test.ts index eabfe5a5..94cc7521 100644 --- a/tests/plan-usage.test.ts +++ b/tests/plan-usage.test.ts @@ -4,8 +4,8 @@ import { join } from 'node:path' import { describe, it, expect, vi, beforeEach } from 'vitest' -import { savePlan } from '../src/config.js' -import { activePlansFromMap, computePeriodFromResetDay, getPlanUsage, getPlanUsageFromProjects, getPlanUsages } from '../src/plan-usage.js' +import { savePlan, type Plan } from '../src/config.js' +import { activePlansFromMap, computePeriodFromResetDay, getPlanScopedProjects, getPlanUsage, getPlanUsageFromProjects, getPlanUsages } from '../src/plan-usage.js' import type { ProjectSummary } from '../src/types.js' const { parseAllSessionsMock } = vi.hoisted(() => ({ @@ -40,6 +40,75 @@ describe('computePeriodFromResetDay', () => { }) }) +describe('getPlanScopedProjects supplementary accounting', () => { + const plan: Plan = { id: 'custom', monthlyUsd: 100, provider: 'all', resetDay: 1, setAt: '2026-08-01T00:00:00.000Z' } + const today = new Date('2026-08-10T12:00:00.000Z') + + function copilotCall(costUSD: number, timestamp: string, supplementaryAccounting: boolean) { + return { + provider: 'copilot', + model: 'claude-sonnet-4-5', + usage: { + inputTokens: supplementaryAccounting ? 40 : 0, + outputTokens: 0, + cacheCreationInputTokens: 0, + cacheReadInputTokens: 0, + cachedInputTokens: 0, + reasoningTokens: 0, + webSearchRequests: 0, + }, + costUSD, + tools: [], + mcpTools: [], + skills: [], + hasAgentSpawn: false, + hasPlanMode: false, + speed: 'standard', + timestamp, + bashCommands: [], + deduplicationKey: `copilot-${timestamp}`, + supplementaryAccounting, + } + } + + it('weighs calls behaviorally and keeps a cost-bearing zero-call session', () => { + const scoped = getPlanScopedProjects(plan, [ + { + project: 'codeburn', + projectPath: '/tmp/codeburn', + totalCostUSD: 1.75, + totalApiCalls: 2, + sessions: [ + { + // One real request served alongside its paired store row. + turns: [{ + timestamp: '2026-08-05T12:00:00.000Z', + assistantCalls: [ + copilotCall(1.0, '2026-08-05T12:00:00.000Z', false), + copilotCall(0.5, '2026-08-05T12:00:05.000Z', true), + ], + }], + }, + { + // Rollup-only session: real spend, zero behavioral requests. + turns: [{ + timestamp: '2026-08-06T12:00:00.000Z', + assistantCalls: [copilotCall(0.25, '2026-08-06T12:00:00.000Z', true)], + }], + }, + ], + }, + ] as ProjectSummary[], today) + + expect(scoped).toHaveLength(1) + expect(scoped[0]!.sessions.map(session => session.apiCalls)).toEqual([1, 0]) + expect(scoped[0]!.sessions.map(session => session.totalCostUSD)).toEqual([1.5, 0.25]) + expect(scoped[0]!.totalApiCalls).toBe(1) + expect(scoped[0]!.totalCostUSD).toBeCloseTo(1.75, 10) + expect(getPlanUsageFromProjects(plan, scoped, today).spentApiEquivalentUsd).toBeCloseTo(1.75, 10) + }) +}) + describe('getPlanUsage', () => { beforeEach(() => { parseAllSessionsMock.mockReset() diff --git a/tests/provider-env-declarations.test.ts b/tests/provider-env-declarations.test.ts index 2044571f..0ab0aaf6 100644 --- a/tests/provider-env-declarations.test.ts +++ b/tests/provider-env-declarations.test.ts @@ -85,6 +85,15 @@ const FILE_PROVIDERS: Record = { // Copilot has since pruned from the DB that only the cache still holds. // Deferred until the durable carry-forward learns to merge instead of drop. const COPILOT_DEFERRED = 'deferred (Ruling 1): declaring it would force the durable re-parse that loses pruned OTel history' +// The session-store override is allowlisted on its own reasoning, not merely +// by inheriting the copilot deferral: repointing it cannot serve stale data, +// because copilot's rollup-vs-store reconciliation reads only the cached +// serve set (parseProviderSources), never a discovery-time snapshot. A new +// path is a new source parsed on sight, and the old path's cached entries +// persist as durable orphans that keep contributing what they always did — +// there is no cross-file dependency for a fingerprint to catch, so declaring +// it would buy nothing and cost the #927 durable-history loss. +const COPILOT_STORE_DEFERRED = 'deferred (#927): repointing serves no stale data — serve-time reconciliation reads the cached serve set, so a new store path parses on sight and the old path stays a durable orphan' const ALLOWLIST: Record = { 'sqlite-session-parser.ts:CODEBURN_VERBOSE': 'sqlite-session-parser.ts:276 — logging verbosity only; changes no discovered path and no parsed value', 'copilot.ts:CODEBURN_COPILOT_SESSION_STATE_DIR': COPILOT_DEFERRED, @@ -93,6 +102,7 @@ const ALLOWLIST: Record = { 'copilot.ts:CODEBURN_COPILOT_WS_STORAGE_DIR': COPILOT_DEFERRED, 'copilot.ts:CODEBURN_COPILOT_GLOBAL_STORAGE_DIR': COPILOT_DEFERRED, 'copilot.ts:CODEBURN_COPILOT_DISABLE_OTEL': COPILOT_DEFERRED, + 'copilot.ts:CODEBURN_COPILOT_SESSION_STORE_DB': COPILOT_STORE_DEFERRED, 'copilot.ts:APPDATA': COPILOT_DEFERRED, 'copilot.ts:XDG_CONFIG_HOME': COPILOT_DEFERRED, 'copilot.ts:LOCALAPPDATA': COPILOT_DEFERRED, diff --git a/tests/providers/copilot.test.ts b/tests/providers/copilot.test.ts index ae128be9..03c53b0c 100644 --- a/tests/providers/copilot.test.ts +++ b/tests/providers/copilot.test.ts @@ -5,12 +5,25 @@ import { tmpdir } from 'os' import { createRequire } from 'node:module' import { copilot, createCopilotProvider, getVSCodeGlobalStorageDirs, getVSCodeWorkspaceStorageDirs } from '../../src/providers/copilot.js' -import { isSqliteAvailable } from '../../src/sqlite.js' +import { isSqliteAvailable, isSqliteBusyError } from '../../src/sqlite.js' import { calculateCost } from '../../src/models.js' import type { ParsedProviderCall } from '../../src/providers/types.js' let tmpDir: string +// The machine running this suite may itself have a real +// ~/.copilot/session-store.db, which discoverSessions would pick up by +// default and leak into every discovery test's source list. Pin the path to +// a nonexistent file globally; tests that need a store pass an explicit +// fixture path to createCopilotProvider (or re-stub the env themselves). +beforeEach(() => { + vi.stubEnv('CODEBURN_COPILOT_SESSION_STORE_DB', '/nonexistent/session-store.db') +}) + +afterEach(() => { + vi.unstubAllEnvs() +}) + async function createSessionDir(sessionId: string, lines: string[], cwd = '/home/user/myproject') { const sessionDir = join(tmpDir, sessionId) await mkdir(sessionDir, { recursive: true }) @@ -1784,6 +1797,709 @@ describe('copilot provider - OTel cache token parsing', () => { // separate serialized fields. These helpers reproduce that on-disk shape so // tests exercise the real regex/scan extraction path. +// --------------------------------------------------------------------------- +// Session-store tests (~/.copilot/session-store.db) +// +// The Copilot CLI and the GitHub Copilot desktop app write per-request usage +// rows into assistant_usage_events. These tests verify the row → call +// contract (cache-inclusive input decomposed, output excluded), the +// discovery-time shutdown-rollup suppression for covered sessions, and the +// graceful-absence path for stores predating the table. Fixture DBs are +// built programmatically — never committed binaries. +// --------------------------------------------------------------------------- + +/** Creates a minimal session-store.db schema matching the Copilot CLI store. */ +function createSessionStoreDb(dbPath: string): void { + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const db = new DatabaseSync(dbPath) + db.exec(` + CREATE TABLE sessions ( + id TEXT PRIMARY KEY, + cwd TEXT, + repository TEXT, + branch TEXT, + created_at TEXT DEFAULT (datetime('now')) + ); + CREATE TABLE assistant_usage_events ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + session_id TEXT NOT NULL REFERENCES sessions(id), + model TEXT NOT NULL, + input_tokens INTEGER, + output_tokens INTEGER, + cache_read_tokens INTEGER, + cache_write_tokens INTEGER, + reasoning_tokens INTEGER, + created_at TEXT DEFAULT (datetime('now')) + ); + `) + db.close() +} + +interface UsageRowDef { + sessionId: string + model: string + // Cache-INCLUSIVE, as the CLI writes it (input + cache_read + cache_write). + inputTokens: number + outputTokens?: number + cacheReadTokens?: number + cacheWriteTokens?: number + reasoningTokens?: number + // Explicit null writes SQL NULL (exercises the timestamp fallback chain); + // undefined gets a fixed default so unrelated tests stay deterministic. + createdAt?: string | null + cwd?: string + repository?: string + sessionCreatedAt?: string | null +} + +function insertUsageRow(dbPath: string, row: UsageRowDef): void { + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const db = new DatabaseSync(dbPath) + db.prepare(`INSERT OR IGNORE INTO sessions (id, cwd, repository, created_at) VALUES (?, ?, ?, ?)`) + .run(row.sessionId, row.cwd ?? null, row.repository ?? null, row.sessionCreatedAt ?? null) + db.prepare( + `INSERT INTO assistant_usage_events + (session_id, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, reasoning_tokens, created_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?)` + ).run( + row.sessionId, + row.model, + row.inputTokens, + row.outputTokens ?? 0, + row.cacheReadTokens ?? 0, + row.cacheWriteTokens ?? 0, + row.reasoningTokens ?? 0, + row.createdAt === undefined ? '2026-08-01T12:00:00.000Z' : row.createdAt, + ) + db.close() +} + +const storeSource = (path: string) => + ({ path, project: 'copilot', provider: 'copilot', sourceType: 'session-store' }) + +describe.skipIf(!isSqliteAvailable())('copilot provider - session-store parsing', () => { + let dbPath: string + + beforeEach(async () => { + tmpDir = await mkdtemp(join(tmpdir(), 'copilot-store-test-')) + dbPath = join(tmpDir, 'session-store.db') + }) + + afterEach(async () => { + await rm(tmpDir, { recursive: true, force: true }) + vi.unstubAllEnvs() + }) + + it('decomposes cache-inclusive input_tokens per request row, output excluded', async () => { + createSessionStoreDb(dbPath) + // First two requests of a real CLI session: input_tokens is + // cache-INCLUSIVE (24680 = 2 + 0 + 24678), confirmed by the rows' own + // token_details_json split (tokenType:"input" holds the uncached + // remainder). The rows carry output tokens which must NOT be emitted — + // per-turn output is owned by the events.jsonl assistant.message calls. + insertUsageRow(dbPath, { + sessionId: 'sess-a', model: 'claude-sonnet-4-5', + inputTokens: 24680, outputTokens: 81, cacheReadTokens: 0, cacheWriteTokens: 24678, + createdAt: '2026-08-07T17:56:38.756Z', cwd: '/home/user/myproject', + }) + insertUsageRow(dbPath, { + sessionId: 'sess-a', model: 'claude-sonnet-4-5', + inputTokens: 24793, outputTokens: 19, cacheReadTokens: 24678, cacheWriteTokens: 113, + createdAt: '2026-08-07T17:56:40.414Z', + }) + + const calls = await collectCalls(storeSource(dbPath)) + expect(calls).toHaveLength(2) + + const first = calls[0]! + // : plus a content discriminator (created_at + token counts + + // model, hashed) so a same-path DB reset reusing AUTOINCREMENT ids can + // never alias a different request onto a cached key. + expect(first.deduplicationKey).toMatch(/^copilot-store:sess-a:1:[0-9a-z]+$/) + expect(first.model).toBe('claude-sonnet-4-5') + expect(first.inputTokens).toBe(2) // 24680 - 0 - 24678 + expect(first.cacheReadInputTokens).toBe(0) + expect(first.cacheCreationInputTokens).toBe(24678) + expect(first.outputTokens).toBe(0) + expect(first.costIsEstimated).toBe(false) // measured, not estimated + expect(first.costUSD).toBeCloseTo(calculateCost('claude-sonnet-4-5', 2, 0, 24678, 0, 0), 12) + expect(first.costUSD).toBeGreaterThan(0) + expect(first.project).toBe('myproject') // sessions.cwd basename + expect(first.sessionId).toBe('sess-a') + expect(first.timestamp).toBe('2026-08-07T17:56:38.756Z') + + const second = calls[1]! + expect(second.deduplicationKey).toMatch(/^copilot-store:sess-a:2:[0-9a-z]+$/) + expect(second.inputTokens).toBe(2) // 24793 - 24678 - 113 + expect(second.cacheReadInputTokens).toBe(24678) + expect(second.cacheCreationInputTokens).toBe(113) + }) + + it('captures total_nano_aiu and request_multiplier when the schema has them', async () => { + // Billing-schema store (newer CLI): the optional columns ride the calls + // as capture-only metadata — no pricing or display consumes them (#890) + // — and stay OUT of the dedup-key content hash. + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const db = new DatabaseSync(dbPath) + db.exec(` + CREATE TABLE sessions (id TEXT PRIMARY KEY, cwd TEXT, repository TEXT, branch TEXT, created_at TEXT DEFAULT (datetime('now'))); + CREATE TABLE assistant_usage_events ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + session_id TEXT NOT NULL REFERENCES sessions(id), + model TEXT NOT NULL, + input_tokens INTEGER, output_tokens INTEGER, + cache_read_tokens INTEGER, cache_write_tokens INTEGER, reasoning_tokens INTEGER, + total_nano_aiu INTEGER, request_multiplier REAL, + created_at TEXT DEFAULT (datetime('now')) + ); + `) + db.prepare(`INSERT INTO sessions (id, cwd) VALUES ('sess-aiu', '/home/user/proj')`).run() + db.prepare( + `INSERT INTO assistant_usage_events + (session_id, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, reasoning_tokens, total_nano_aiu, request_multiplier, created_at) + VALUES ('sess-aiu', 'claude-sonnet-4-5', 1000, 20, 600, 300, 0, 24594000000, 15.0, '2026-08-07T18:00:00.000Z')` + ).run() + db.close() + + const calls = await collectCalls(storeSource(dbPath)) + expect(calls).toHaveLength(1) + expect(calls[0]!.nanoAiu).toBe(24594000000) + expect(calls[0]!.requestMultiplier).toBe(15) + expect(calls[0]!.inputTokens).toBe(100) // 1000 - 600 - 300 + }) + + it('parses identically when the billing columns are absent (older store schema)', async () => { + createSessionStoreDb(dbPath) // schema predates total_nano_aiu / request_multiplier + insertUsageRow(dbPath, { + sessionId: 'sess-old', model: 'claude-sonnet-4-5', + inputTokens: 500, cacheReadTokens: 0, cacheWriteTokens: 0, + createdAt: '2026-08-07T18:00:00.000Z', + }) + const calls = await collectCalls(storeSource(dbPath)) + expect(calls).toHaveLength(1) + expect(calls[0]!.inputTokens).toBe(500) + expect(calls[0]!.nanoAiu).toBeUndefined() + expect(calls[0]!.requestMultiplier).toBeUndefined() + }) + + it('bills a multi-model (delegating) session per row model', async () => { + createSessionStoreDb(dbPath) + // A delegating CLI session: subagent requests land as their own rows with + // a distinct model, exactly as observed for haiku-backed subagents. + insertUsageRow(dbPath, { + sessionId: 'sess-multi', model: 'claude-sonnet-4-5', + inputTokens: 10100, cacheReadTokens: 8000, cacheWriteTokens: 2000, reasoningTokens: 94, + }) + insertUsageRow(dbPath, { + sessionId: 'sess-multi', model: 'claude-haiku-4.5', + inputTokens: 5050, cacheReadTokens: 5000, cacheWriteTokens: 0, + }) + + const calls = await collectCalls(storeSource(dbPath)) + expect(calls).toHaveLength(2) + + const sonnet = calls.find(c => c.model === 'claude-sonnet-4-5')! + expect(sonnet.inputTokens).toBe(100) + expect(sonnet.reasoningTokens).toBe(94) + // Reasoning is metadata, never a cost line: the CLI's own + // token_details_json prices only input/cache/output, and reasoning + // tokens are a subset of output_tokens — billed by the per-turn + // assistant.message call. A cost above input+cache pricing here means + // reasoning got billed twice. + expect(sonnet.costUSD).toBeCloseTo(calculateCost('claude-sonnet-4-5', 100, 0, 2000, 8000, 0), 12) + const haiku = calls.find(c => c.model === 'claude-haiku-4.5')! + expect(haiku.inputTokens).toBe(50) + expect(haiku.cacheReadInputTokens).toBe(5000) + }) + + it('skips all-zero rows and reads SQL-default timestamps as UTC', async () => { + createSessionStoreDb(dbPath) + // A row with no input/cache/reasoning adds nothing over the per-turn + // events (output is excluded by design) — no empty $0 call. + insertUsageRow(dbPath, { sessionId: 'sess-z', model: 'gpt-5', inputTokens: 0, outputTokens: 42 }) + // created_at written by SQLite's datetime('now') default: UTC but + // timezone-less, with and without subseconds. Neither may be read as + // local time — that would land the request on the wrong day. + insertUsageRow(dbPath, { + sessionId: 'sess-z', model: 'gpt-5', + inputTokens: 500, cacheReadTokens: 200, createdAt: '2026-08-07 17:56:38', + }) + insertUsageRow(dbPath, { + sessionId: 'sess-z', model: 'gpt-5', + inputTokens: 600, cacheReadTokens: 300, createdAt: '2026-08-07 23:59:59.756', + }) + + const calls = await collectCalls(storeSource(dbPath)) + expect(calls).toHaveLength(2) + expect(calls[0]!.timestamp).toBe('2026-08-07T17:56:38.000Z') + expect(calls[1]!.timestamp).toBe('2026-08-07T23:59:59.756Z') + }) + + it('keeps dedup keys stable as the store grows', async () => { + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { sessionId: 'sess-grow', model: 'gpt-5', inputTokens: 1000, cacheReadTokens: 400 }) + + const seen = new Set() + const first = await collectCalls(storeSource(dbPath), seen) + expect(first.map(c => c.deduplicationKey)).toEqual([ + expect.stringMatching(/^copilot-store:sess-grow:1:[0-9a-z]+$/), + ]) + + // Unchanged store re-parsed with the shared dedup set: nothing re-emits — + // the key (including its content discriminator) is stable across parses. + expect(await collectCalls(storeSource(dbPath), seen)).toHaveLength(0) + + // New request row: only it is emitted, under the next AUTOINCREMENT id — + // the append-only shape the durable union-by-key cache merge requires. + insertUsageRow(dbPath, { sessionId: 'sess-grow', model: 'gpt-5', inputTokens: 2000, cacheReadTokens: 900 }) + const grown = await collectCalls(storeSource(dbPath), seen) + expect(grown.map(c => c.deduplicationKey)).toEqual([ + expect.stringMatching(/^copilot-store:sess-grow:2:[0-9a-z]+$/), + ]) + }) + + it('gives a reused row id a NEW key when the DB was recreated with different content', async () => { + // Same path, same session, same AUTOINCREMENT id — but the store was + // deleted and recreated, so row id 1 now describes a DIFFERENT request. + // A bare : key would make the durable union swallow the new + // row as already-cached, losing its usage; the content discriminator + // must split the two. A byte-identical re-insert (backup restore) must + // still collapse to the SAME key. + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { + sessionId: 'sess-reset', model: 'gpt-5', + inputTokens: 100, cacheReadTokens: 0, createdAt: '2026-08-07T10:00:00.000Z', + }) + const before = await collectCalls(storeSource(dbPath)) + expect(before).toHaveLength(1) + + await rm(dbPath, { force: true }) + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { + sessionId: 'sess-reset', model: 'gpt-5', + inputTokens: 200, cacheReadTokens: 0, createdAt: '2026-08-08T10:00:00.000Z', + }) + const after = await collectCalls(storeSource(dbPath)) + expect(after).toHaveLength(1) + expect(after[0]!.deduplicationKey).not.toBe(before[0]!.deduplicationKey) + expect(after[0]!.inputTokens).toBe(200) + + // Identical content re-inserted under the same id: the key must NOT move. + await rm(dbPath, { force: true }) + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { + sessionId: 'sess-reset', model: 'gpt-5', + inputTokens: 200, cacheReadTokens: 0, createdAt: '2026-08-08T10:00:00.000Z', + }) + const restored = await collectCalls(storeSource(dbPath)) + expect(restored[0]!.deduplicationKey).toBe(after[0]!.deduplicationKey) + }) + + it('parses BOTH the store rows and the shutdown rollup for a covered session', async () => { + // Precedence is serve-time only: the parsers cache both representations + // unconditionally, and parseProviderSources drops the rollup calls of + // sessions whose store rows are being served (tests/parser.test.ts (i), + // (k), (m)). Suppressing here would re-open the probe-to-parse races the + // serve-time design closes, so this pins the parse-level contract: no + // parser-side suppression, ever. + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { + sessionId: 'sess-covered', model: 'claude-sonnet-4-5', + inputTokens: 10100, cacheReadTokens: 8000, cacheWriteTokens: 2000, + cwd: '/home/user/myproject', + }) + const eventsPath = await createSessionDir('sess-covered', [ + modelChange('claude-sonnet-4-5'), + userMessage('do the thing'), + assistantMessage({ messageId: 'msg-1', outputTokens: 345 }), + shutdownEvent({ + modelMetrics: { + 'claude-sonnet-4-5': { inputTokens: 71282, outputTokens: 345, cacheReadTokens: 35495, cacheWriteTokens: 35783, reasoningTokens: 31 }, + }, + }), + ]) + + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', dbPath) + const sources = await provider.discoverSessions() + const store = sources.find(s => (s as { sourceType?: string }).sourceType === 'session-store') + expect(store).toBeDefined() + const jsonl = sources.find(s => s.path === eventsPath) + expect(jsonl).toBeDefined() + + const seen = new Set() + const collect = async (src: typeof sources[number]) => { + const out: ParsedProviderCall[] = [] + for await (const call of provider.createSessionParser(src, seen).parse()) out.push(call) + return out + } + const storeCalls = await collect(store!) + const jsonlCalls = await collect(jsonl!) + + expect(storeCalls.map(c => c.deduplicationKey)).toEqual([ + expect.stringMatching(/^copilot-store:sess-covered:1:[0-9a-z]+$/), + ]) + const rollup = jsonlCalls.find(c => c.deduplicationKey === 'copilot:sess-covered:shutdown:claude-sonnet-4-5:1') + expect(rollup).toBeDefined() + expect(rollup!.cacheReadInputTokens).toBe(35495) + expect(storeCalls[0]!.inputTokens).toBe(100) + expect(storeCalls[0]!.cacheReadInputTokens).toBe(8000) + }) + + it('keeps the shutdown rollup for sessions the store does not cover', async () => { + createSessionStoreDb(dbPath) + // The store knows about a DIFFERENT session (e.g. one run under a newer + // CLI); sess-uncovered predates the table's rows and must keep its + // rollup-derived input/cache. + insertUsageRow(dbPath, { sessionId: 'sess-other', model: 'gpt-5', inputTokens: 700, cacheReadTokens: 300 }) + const eventsPath = await createSessionDir('sess-uncovered', [ + modelChange('claude-sonnet-4-5'), + assistantMessage({ messageId: 'msg-1', outputTokens: 100 }), + shutdownEvent({ + modelMetrics: { + 'claude-sonnet-4-5': { inputTokens: 10100, outputTokens: 100, cacheReadTokens: 8000, cacheWriteTokens: 2000 }, + }, + }), + ]) + + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', dbPath) + const sources = await provider.discoverSessions() + const jsonl = sources.find(s => s.path === eventsPath)! + + const calls: ParsedProviderCall[] = [] + for await (const call of provider.createSessionParser(jsonl, new Set()).parse()) calls.push(call) + + const rollup = calls.find(c => c.deduplicationKey === 'copilot:sess-uncovered:shutdown:claude-sonnet-4-5:1') + expect(rollup).toBeDefined() + expect(rollup!.inputTokens).toBe(100) + expect(rollup!.cacheReadInputTokens).toBe(8000) + }) + + it('a locked store still surfaces its source and never blocks session-state parsing', async () => { + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { sessionId: 'sess-locked', model: 'gpt-5', inputTokens: 1000, cacheReadTokens: 400 }) + const eventsPath = await createSessionDir('sess-locked', [ + modelChange('claude-sonnet-4-5'), + assistantMessage({ messageId: 'msg-1', outputTokens: 100 }), + shutdownEvent({ + modelMetrics: { + 'claude-sonnet-4-5': { inputTokens: 10100, outputTokens: 100, cacheReadTokens: 8000, cacheWriteTokens: 2000 }, + }, + }), + ]) + + // Hold an exclusive write transaction across discovery AND both parses, + // the shape of a CLI mid-checkpoint. A lock proves nothing about + // absence, so the source must still surface — its path stays discovered + // and previously cached rows keep serving (and keep suppressing at + // serve time) — while its parse raises the busy shape + // parseProviderSources skips-and-retries. The session-state file no + // longer waits on the store for anything: its parse (rollup included) + // must succeed with the store locked the whole time. + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const locker = new DatabaseSync(dbPath) + locker.exec('BEGIN EXCLUSIVE') + try { + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', dbPath) + const sources = await provider.discoverSessions() + const store = sources.find(s => (s as { sourceType?: string }).sourceType === 'session-store') + expect(store).toBeDefined() + + const consumeStore = async () => { + for await (const _ of provider.createSessionParser(store!, new Set()).parse()) void _ + } + await expect(consumeStore()).rejects.toSatisfy((err: unknown) => isSqliteBusyError(err)) + + const jsonl = sources.find(s => s.path === eventsPath)! + const calls: ParsedProviderCall[] = [] + for await (const call of provider.createSessionParser(jsonl, new Set()).parse()) calls.push(call) + expect(calls.some(c => c.deduplicationKey === 'copilot:sess-locked:shutdown:claude-sonnet-4-5:1')).toBe(true) + } finally { + locker.exec('ROLLBACK') + locker.close() + } + }) + + it('surfaces the source when the store path cannot be stat-ed, and defers its parse', async () => { + // EACCES/EIO on stat must NOT read as absence: a store may exist that + // this run cannot see. The source stays discovered — so serve-time + // suppression keeps holding from previously cached rows — and its parse + // raises the busy shape parseProviderSources skips-and-retries. The + // session-state file parses normally either way. + if (typeof process.getuid === 'function' && process.getuid() === 0) return // root ignores modes + const deniedDir = join(tmpDir, 'denied') + await mkdir(deniedDir, { recursive: true }) + const deniedDb = join(deniedDir, 'session-store.db') + createSessionStoreDb(deniedDb) + const eventsPath = await createSessionDir('sess-denied', [ + modelChange('claude-sonnet-4-5'), + assistantMessage({ messageId: 'msg-1', outputTokens: 10 }), + shutdownEvent({ + modelMetrics: { + 'claude-sonnet-4-5': { inputTokens: 5100, outputTokens: 10, cacheReadTokens: 4000, cacheWriteTokens: 1000 }, + }, + }), + ]) + + const { chmod } = await import('fs/promises') + await chmod(deniedDir, 0o000) + try { + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', deniedDb) + const sources = await provider.discoverSessions() + const store = sources.find(s => (s as { sourceType?: string }).sourceType === 'session-store') + expect(store).toBeDefined() + const consumeStore = async () => { + for await (const _ of provider.createSessionParser(store!, new Set()).parse()) void _ + } + await expect(consumeStore()).rejects.toSatisfy((err: unknown) => isSqliteBusyError(err)) + + const jsonl = sources.find(s => s.path === eventsPath)! + const calls: ParsedProviderCall[] = [] + for await (const call of provider.createSessionParser(jsonl, new Set()).parse()) calls.push(call) + expect(calls.some(c => c.deduplicationKey === 'copilot:sess-denied:shutdown:claude-sonnet-4-5:1')).toBe(true) + } finally { + await chmod(deniedDir, 0o755) + } + }) + + it('defers the store source when the DB becomes unopenable after discovery', async () => { + // An EACCES/CANTOPEN race between discovery and parse must defer, not + // fall through to the generic parse-failure path — that would cache a + // failed marker at the current fingerprint and zero the covered + // sessions until the file next changes. + if (typeof process.getuid === 'function' && process.getuid() === 0) return // root ignores modes + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { sessionId: 'sess-open', model: 'gpt-5', inputTokens: 1000, cacheReadTokens: 400 }) + + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', dbPath) + const sources = await provider.discoverSessions() + const store = sources.find(s => (s as { sourceType?: string }).sourceType === 'session-store')! + + const { chmod } = await import('fs/promises') + await chmod(dbPath, 0o000) + try { + const consume = async () => { + for await (const _ of provider.createSessionParser(store, new Set()).parse()) void _ + } + await expect(consume()).rejects.toSatisfy((err: unknown) => isSqliteBusyError(err)) + } finally { + await chmod(dbPath, 0o644) + } + }) + + it('defers the store parse when the schema changes mid-run', async () => { + // Discovery prepare-validated the schema this run, so a query failure at + // parse time proves a mid-run migration. Falling through to the generic + // parse-failure path would cache an EMPTY success at the current + // fingerprint while cached rows keep suppressing rollups at serve time — + // a silent under-count until the file next changes. Defer instead. + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { sessionId: 'sess-migrate', model: 'gpt-5', inputTokens: 1000, cacheReadTokens: 400 }) + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', dbPath) + const sources = await provider.discoverSessions() + const store = sources.find(s => (s as { sourceType?: string }).sourceType === 'session-store')! + + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const migrator = new DatabaseSync(dbPath) + migrator.exec('ALTER TABLE assistant_usage_events DROP COLUMN reasoning_tokens') + migrator.close() + + const consume = async () => { + for await (const _ of provider.createSessionParser(store, new Set()).parse()) void _ + } + await expect(consume()).rejects.toMatchObject({ code: 'SQLITE_BUSY' }) + }) + + it('emits billable rows with an empty model as unknown instead of dropping them', async () => { + // TEXT NOT NULL admits '': a billable row must never be dropped for an + // unnameable model — serve-time precedence suppresses the session's + // rollup whenever its store rows serve, so a skipped row's tokens would + // simply vanish. Price as 'unknown' instead. + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { + sessionId: 'sess-nomodel', model: '', + inputTokens: 10100, cacheReadTokens: 8000, cacheWriteTokens: 2000, + }) + + const calls = await collectCalls(storeSource(dbPath)) + expect(calls).toHaveLength(1) + expect(calls[0]!.model).toBe('unknown') + expect(calls[0]!.inputTokens).toBe(100) + expect(calls[0]!.cacheReadInputTokens).toBe(8000) + }) + + it('surfaces the source when the store is corrupt, and defers its parse', async () => { + // Corruption-class failures (SQLITE_CORRUPT/NOTADB/CANTOPEN — measured as + // the store's realistic failure modes; WAL write locks don't even block + // readers) must NOT read as absence: the file may be mid atomic-replace + // and readable next run. The source stays discovered — cached rows keep + // serving and keep suppressing at serve time — while its parse defers + // with the busy shape. Session-state files parse normally throughout. + await writeFile(dbPath, 'not a sqlite database at all') + const eventsPath = await createSessionDir('sess-corrupt', [ + modelChange('claude-sonnet-4-5'), + assistantMessage({ messageId: 'msg-1', outputTokens: 100 }), + shutdownEvent({ + modelMetrics: { + 'claude-sonnet-4-5': { inputTokens: 10100, outputTokens: 100, cacheReadTokens: 8000, cacheWriteTokens: 2000 }, + }, + }), + ]) + + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', dbPath) + const sources = await provider.discoverSessions() + const store = sources.find(s => (s as { sourceType?: string }).sourceType === 'session-store') + expect(store).toBeDefined() + const consumeStore = async () => { + for await (const _ of provider.createSessionParser(store!, new Set()).parse()) void _ + } + await expect(consumeStore()).rejects.toMatchObject({ code: 'SQLITE_BUSY' }) + + const jsonl = sources.find(s => s.path === eventsPath)! + const calls: ParsedProviderCall[] = [] + for await (const call of provider.createSessionParser(jsonl, new Set()).parse()) calls.push(call) + expect(calls.some(c => c.deduplicationKey === 'copilot:sess-corrupt:shutdown:claude-sonnet-4-5:1')).toBe(true) + }) + + it('treats a store whose schema the parser cannot read as absent', async () => { + // A schema mismatch ("no such column") is a permanent shape, not a + // transient failure: deferring would stall CLI parsing forever, and the + // rollups ARE the right source for a store the parser can't read. The + // probe runs the parser's exact query, so the mismatch is caught before + // any rollup is suppressed. + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const db = new DatabaseSync(dbPath) + db.exec(` + CREATE TABLE sessions (id TEXT PRIMARY KEY, cwd TEXT, repository TEXT); + CREATE TABLE assistant_usage_events ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + session_id TEXT NOT NULL, + model TEXT NOT NULL, + input_tokens INTEGER + ); + INSERT INTO assistant_usage_events (session_id, model, input_tokens) VALUES ('sess-newschema', 'gpt-5', 900); + `) + db.close() + + const eventsPath = await createSessionDir('sess-newschema', [ + modelChange('claude-sonnet-4-5'), + assistantMessage({ messageId: 'msg-1', outputTokens: 50 }), + shutdownEvent({ + modelMetrics: { + 'claude-sonnet-4-5': { inputTokens: 5100, outputTokens: 50, cacheReadTokens: 4000, cacheWriteTokens: 1000 }, + }, + }), + ]) + + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', dbPath) + const sources = await provider.discoverSessions() + expect(sources.some(s => (s as { sourceType?: string }).sourceType === 'session-store')).toBe(false) + + const jsonl = sources.find(s => s.path === eventsPath)! + const calls: ParsedProviderCall[] = [] + for await (const call of provider.createSessionParser(jsonl, new Set()).parse()) calls.push(call) + expect(calls.some(c => c.deduplicationKey.includes(':shutdown:'))).toBe(true) + }) + + it('treats a store without assistant_usage_events as absent', async () => { + // Older CLI builds create session-store.db without the usage table. The + // source must not surface (and must not throw), and no session gets its + // shutdown rollup suppressed. + const { DatabaseSync } = requireForTest('node:sqlite') as { DatabaseSync: new (path: string) => TestDb } + const db = new DatabaseSync(dbPath) + db.exec('CREATE TABLE sessions (id TEXT PRIMARY KEY, cwd TEXT)') + db.close() + + const eventsPath = await createSessionDir('sess-old-cli', [ + modelChange('claude-sonnet-4-5'), + assistantMessage({ messageId: 'msg-1', outputTokens: 50 }), + shutdownEvent({ + modelMetrics: { + 'claude-sonnet-4-5': { inputTokens: 5100, outputTokens: 50, cacheReadTokens: 4000, cacheWriteTokens: 1000 }, + }, + }), + ]) + + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', dbPath) + const sources = await provider.discoverSessions() + expect(sources.some(s => (s as { sourceType?: string }).sourceType === 'session-store')).toBe(false) + + const jsonl = sources.find(s => s.path === eventsPath)! + const calls: ParsedProviderCall[] = [] + for await (const call of provider.createSessionParser(jsonl, new Set()).parse()) calls.push(call) + expect(calls.some(c => c.deduplicationKey.includes(':shutdown:'))).toBe(true) + }) + + it('never emits an empty timestamp: falls back to the previous row, then sessions.created_at', async () => { + // A call with an empty timestamp is invisible to every date-range filter + // — the tokens would silently vanish from daily/monthly views while the + // session's rollup stays suppressed. Rows are id-ordered, so the nearest + // earlier row is the closest clock reading; a NULL on the very first row + // falls back to the session's own created_at. + createSessionStoreDb(dbPath) + insertUsageRow(dbPath, { + sessionId: 'sess-nots', model: 'gpt-5', + inputTokens: 1000, cacheReadTokens: 400, createdAt: null, + sessionCreatedAt: '2026-08-05T09:00:00.000Z', + }) + insertUsageRow(dbPath, { + sessionId: 'sess-nots', model: 'gpt-5', + inputTokens: 2000, cacheReadTokens: 900, createdAt: '2026-08-05T09:05:00.000Z', + }) + insertUsageRow(dbPath, { + sessionId: 'sess-nots', model: 'gpt-5', + inputTokens: 3000, cacheReadTokens: 1400, createdAt: null, + }) + + const calls = await collectCalls(storeSource(dbPath)) + expect(calls).toHaveLength(3) + expect(calls[0]!.timestamp).toBe('2026-08-05T09:00:00.000Z') // sessions.created_at + expect(calls[1]!.timestamp).toBe('2026-08-05T09:05:00.000Z') // its own created_at + expect(calls[2]!.timestamp).toBe('2026-08-05T09:05:00.000Z') // previous row's + }) + + it('attributes store rows to the jsonl-derived project, over sessions.cwd', async () => { + // The per-turn output calls carry the workspace.yaml-derived project, + // and the session grouping key includes project — so a store row landing + // under any OTHER label (the sessionId fallback for a NULL cwd, or a + // stale/differing sessions.cwd) splits one real session into two. + // Sessions with no session-state dir keep the cwd → repository → + // sessionId fallback chain. + createSessionStoreDb(dbPath) + // The review's verbatim shape: NULL cwd AND repository, jsonl present. + insertUsageRow(dbPath, { + sessionId: 'sess-attr-null', model: 'gpt-5', + inputTokens: 1000, cacheReadTokens: 400, + }) + // A present-but-differing sessions.cwd must also lose to the jsonl label. + insertUsageRow(dbPath, { + sessionId: 'sess-attr-stale', model: 'gpt-5', + inputTokens: 1500, cacheReadTokens: 600, cwd: '/home/user/stale-db-cwd', + }) + insertUsageRow(dbPath, { + sessionId: 'sess-nojsonl', model: 'gpt-5', + inputTokens: 2000, cacheReadTokens: 900, cwd: '/home/user/db-only-proj', + }) + await createSessionDir('sess-attr-null', [ + modelChange('gpt-5'), + assistantMessage({ messageId: 'msg-1', outputTokens: 10 }), + ], '/home/user/jsonl-proj') + await createSessionDir('sess-attr-stale', [ + modelChange('gpt-5'), + assistantMessage({ messageId: 'msg-2', outputTokens: 10 }), + ], '/home/user/jsonl-proj') + + const provider = createCopilotProvider(tmpDir, '/nonexistent/ws', '/nonexistent/global', '/nonexistent/jb', dbPath) + const sources = await provider.discoverSessions() + const store = sources.find(s => (s as { sourceType?: string }).sourceType === 'session-store')! + + const calls: ParsedProviderCall[] = [] + for await (const call of provider.createSessionParser(store, new Set()).parse()) calls.push(call) + expect(calls.find(c => c.sessionId === 'sess-attr-null')!.project).toBe('jsonl-proj') + expect(calls.find(c => c.sessionId === 'sess-attr-stale')!.project).toBe('jsonl-proj') + expect(calls.find(c => c.sessionId === 'sess-nojsonl')!.project).toBe('db-only-proj') + }) +}) + describe('copilot provider - JetBrains parsing', () => { beforeEach(async () => { tmpDir = await mkdtemp(join(tmpdir(), 'copilot-jetbrains-test-')) diff --git a/tests/session-cache.test.ts b/tests/session-cache.test.ts index de8df9c0..0c0afaa8 100644 --- a/tests/session-cache.test.ts +++ b/tests/session-cache.test.ts @@ -22,6 +22,7 @@ import { reconcileFile, saveCache, sessionCachePath, + sourcePathStatCandidates, } from '../src/session-cache.js' // Version-suffixed filename (e.g. session-cache.v5.json) the cache now writes to. @@ -909,3 +910,16 @@ describe('loadCache memo', () => { clearLoadCacheMemo() }) }) + +describe('sourcePathStatCandidates', () => { + it('mirrors the fingerprint fallbacks: plain, #-suffixed, and :-suffixed paths', () => { + expect(sourcePathStatCandidates('/a/b/state.vscdb')).toEqual(['/a/b/state.vscdb']) + expect(sourcePathStatCandidates('/a/b/state.vscdb#cursor-ws=ws1')) + .toEqual(['/a/b/state.vscdb#cursor-ws=ws1', '/a/b/state.vscdb']) + expect(sourcePathStatCandidates('/a/b/db.sqlite:sess-1')) + .toEqual(['/a/b/db.sqlite:sess-1', '/a/b/db.sqlite']) + // A plain Windows path must NOT yield the bare drive letter — a stat + // error on a cwd-relative 'C' must never hold hydration. + expect(sourcePathStatCandidates('C:\\data\\gone.jsonl')).toEqual(['C:\\data\\gone.jsonl']) + }) +}) diff --git a/tests/sessions-report.test.ts b/tests/sessions-report.test.ts index 158daa9b..da930ea9 100644 --- a/tests/sessions-report.test.ts +++ b/tests/sessions-report.test.ts @@ -1,7 +1,7 @@ import { describe, expect, it } from 'vitest' -import { aggregateSessions, renderJson, renderTable } from '../src/sessions-report.js' -import type { ClassifiedTurn, ProjectSummary, SessionSummary } from '../src/types.js' +import { aggregateByBranch, aggregateSessions, attributeSessionPrSpend, renderJson, renderTable } from '../src/sessions-report.js' +import type { ClassifiedTurn, ParsedApiCall, ProjectSummary, SessionSummary } from '../src/types.js' function makeProject(): ProjectSummary { const turn: ClassifiedTurn = { @@ -143,3 +143,66 @@ describe('sessions JSON emitter', () => { expect(Math.max(...lines.slice(0, -1).map(line => line.length))).toBeLessThanOrEqual(80) }) }) + +// A copilot serve set pairs some calls with an already-counted per-turn call +// (shutdown rollups, residuals, store rows). Their tokens and cost are real and +// must survive into every spend surface, but they carry no behavioral weight, so +// no user-visible calls/turns counter may count them. +function copilotCall(overrides: Partial & { deduplicationKey: string }): ParsedApiCall { + return { ...makeProject().sessions[0]!.turns[0]!.assistantCalls[0]!, provider: 'copilot', ...overrides } +} + +function makeSupplementaryProject(): ProjectSummary { + const project = makeProject() + const session = project.sessions[0]! + const mixed = session.turns[0]! + mixed.gitBranch = 'feat/copilot' + mixed.assistantCalls = [ + copilotCall({ deduplicationKey: 'behavioral-1', costUSD: 0.10 }), + copilotCall({ deduplicationKey: 'supp-1', costUSD: 0.02, supplementaryAccounting: true }), + ] + session.turns.push({ + ...mixed, + gitBranch: undefined, + timestamp: '2026-07-10T10:02:00.000Z', + assistantCalls: [copilotCall({ deduplicationKey: 'supp-2', costUSD: 0.05, supplementaryAccounting: true })], + }) + session.apiCalls = 1 + session.totalCostUSD = 0.17 + return project +} + +describe('supplementary accounting weight', () => { + it('counts only turns with a behavioral call in the session rows', () => { + const rows = aggregateSessions([makeSupplementaryProject()]) + expect(rows[0]!.turns).toBe(1) + expect(rows[0]!.calls).toBe(1) + expect(rows[0]!.cost).toBeCloseTo(0.17) + }) + + it('attributes supplementary spend to a PR while counting only behavioral calls', () => { + const url = 'https://github.com/acme/app/pull/7' + const { perUrl } = attributeSessionPrSpend({ + turns: [ + { prRefs: [url], assistantCalls: [{ costUSD: 0.10 }, { costUSD: 0.02, supplementaryAccounting: true }] }, + // Supplementary-only turn: no calls, but its cost still belongs to the PR. + { assistantCalls: [{ costUSD: 0.05, supplementaryAccounting: true }] }, + ], + totalCostUSD: 0.17, + apiCalls: 1, + totalSavingsUSD: 0, + }) + + const pr = perUrl.get(url)! + expect(pr.calls).toBe(1) + expect(pr.cost).toBeCloseTo(0.17) + }) + + it('attributes supplementary spend to a branch while counting only behavioral calls', () => { + const rows = aggregateByBranch([makeSupplementaryProject()]) + expect(rows).toHaveLength(1) + expect(rows[0]!.branch).toBe('feat/copilot') + expect(rows[0]!.calls).toBe(1) + expect(rows[0]!.cost).toBeCloseTo(0.17) + }) +}) diff --git a/tests/usage-aggregator-savings.test.ts b/tests/usage-aggregator-savings.test.ts new file mode 100644 index 00000000..b6d8b39c --- /dev/null +++ b/tests/usage-aggregator-savings.test.ts @@ -0,0 +1,107 @@ +import { describe, expect, it, beforeAll, vi } from 'vitest' + +import { buildMenubarPayloadForRange } from '../src/usage-aggregator.js' +import { getDateRange } from '../src/cli-date.js' +import { loadPricing } from '../src/models.js' +import type { ProjectSummary } from '../src/types.js' + +// The savings block counts REQUESTS: a copilot supplementary accounting call +// (rollup / paired store row) can carry configured model-savings too — its +// saved dollars must be kept while its call weight stays zero. +const ts = new Date().toISOString() + +function makeCall(savingsUSD: number, supplementary: boolean) { + return { + provider: 'copilot', + model: 'llama3.1:8b', + usage: { + inputTokens: 10, + outputTokens: supplementary ? 0 : 20, + cacheCreationInputTokens: 0, + cacheReadInputTokens: 0, + cachedInputTokens: 0, + reasoningTokens: 0, + webSearchRequests: 0, + }, + costUSD: 0.01, + savingsUSD, + savingsBaselineModel: 'gpt-4o', + tools: [], + mcpTools: [], + skills: [], + subagentTypes: [], + hasAgentSpawn: false, + hasPlanMode: false, + speed: 'standard' as const, + timestamp: ts, + bashCommands: [], + deduplicationKey: supplementary ? 'sav-supp' : 'sav-real', + ...(supplementary ? { supplementaryAccounting: true } : {}), + } +} + +const emptyCat = { turns: 0, costUSD: 0, savingsUSD: 0, retries: 0, editTurns: 0, oneShotTurns: 0 } + +function fixtureProjects(): ProjectSummary[] { + return [{ + project: 'proj', + projectPath: 'proj', + sessions: [{ + sessionId: 'sess-sav', + project: 'proj', + firstTimestamp: ts, + lastTimestamp: ts, + totalCostUSD: 0.02, + totalSavingsUSD: 7, + totalInputTokens: 20, + totalOutputTokens: 20, + totalCacheReadTokens: 0, + totalCacheWriteTokens: 0, + apiCalls: 1, + turns: [{ + userMessage: 'hi', + timestamp: ts, + sessionId: 'sess-sav', + category: 'coding', + retries: 0, + hasEdits: false, + assistantCalls: [makeCall(5, false), makeCall(2, true)], + }], + modelBreakdown: {}, + toolBreakdown: {}, + mcpBreakdown: {}, + bashBreakdown: {}, + subagentBreakdown: {}, + categoryBreakdown: { coding: { ...emptyCat, turns: 1, costUSD: 0.02, savingsUSD: 7 } }, + skillBreakdown: {}, + }], + totalCostUSD: 0.02, + totalSavingsUSD: 7, + totalApiCalls: 1, + }] as unknown as ProjectSummary[] +} + +vi.mock('../src/parser.js', async (importOriginal) => { + const mod = await importOriginal() + return { ...mod, parseAllSessions: vi.fn(async () => fixtureProjects()) } +}) + +describe('buildMenubarPayloadForRange: supplementary savings weight', () => { + beforeAll(async () => { + await loadPricing() + }) + + it('keeps supplementary saved dollars but counts only behavioral calls', async () => { + const payload = await buildMenubarPayloadForRange(getDateRange('today'), { provider: 'all', optimize: false }) + const savings = payload.current.localModelSavings! + + expect(savings.totalUSD).toBeCloseTo(7, 10) + expect(savings.calls).toBe(1) + const byModel = savings.byModel.find(m => m.name.includes('llama'))! + expect(byModel.savingsUSD).toBeCloseTo(7, 10) + expect(byModel.calls).toBe(1) + const byProvider = savings.byProvider.find(p => p.name === 'copilot')! + expect(byProvider.savingsUSD).toBeCloseTo(7, 10) + expect(byProvider.calls).toBe(1) + }) +})