Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ NEXT_PUBLIC_APP_URL=http://localhost:3000
# OpenAI API - Get from: https://platform.openai.com/api-keys
OPENAI_API_KEY=
OPENAI_ORG_ID=
# Optional: OpenAI model for the web action agent (Responses API); supports
# Codex/ChatGPT models such as codex-mini-latest. Defaults to gpt-4o-mini.
OPENAI_ACTION_MODEL=

# Anthropic Claude API - Get from: https://console.anthropic.com/settings/keys
ANTHROPIC_API_KEY=
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/AUDIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,4 +66,4 @@ valid. Referenced paths were checked against the working tree:

| `agent-completion-enforcement.yml` | **ADD** | Protected-default-branch verifier that creates the independent **Agent completion enforcement** Check directly against the PR head SHA. It accepts only an exact-head machine-readable report from the configured dedicated GitHub App; missing/stale/mutable evidence, untrusted label provenance, and custom roles all fail closed. The existing `agent-completion/truth-gate` status stays advisory and must not be made required. |

The protected policy at `.github/agent-lock/trusted-publishers.json` starts with empty allowlists and therefore blocks until a repository administrator provisions the dedicated App and trusted actor identities through protected review. The repository ruleset must then require **Agent completion enforcement**, one independent approval, and resolved conversations.
The protected policy at `.github/agent-lock/trusted-publishers.json` starts with empty allowlists. Until a repository administrator provisions the dedicated App and trusted actor identities through protected review, a head with no trusted publication receives a `neutral` advisory Check (`trust_policy_unprovisioned_no_publication`) instead of failing every pull request; a published report against empty allowlists, or a missing publication once any allowlist is populated, still fails closed. The repository ruleset must then require **Agent completion enforcement**, one independent approval, and resolved conversations.
2 changes: 1 addition & 1 deletion .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,4 +129,4 @@ A full audit of this directory was performed (see

`pr-checks.yml` retains the advisory `agent-completion/truth-gate/pr-<number>` status; it is never required. `agent-completion-enforcement.yml` runs protected default-branch code, does not execute PR code, and creates the separate **Agent completion enforcement** Check directly on the PR head SHA. It accepts only an exact-head, machine-readable report published by the configured dedicated GitHub App. Missing, stale, edited/deleted, ambiguous, or untrusted evidence fails closed.

Before enabling the rule, provision `.github/agent-lock/trusted-publishers.json` through protected review with the trusted App and actor allowlists. Empty lists intentionally block. Configure the repository ruleset to require **Agent completion enforcement**, one independent approval, and resolved conversations. Do not require `agent-completion/truth-gate`.
Before enabling the rule, provision `.github/agent-lock/trusted-publishers.json` through protected review with the trusted App and actor allowlists. While a publication exists, empty lists intentionally block; until the App is provisioned (all three allowlists empty) a missing publication publishes a `neutral` advisory Check rather than failing every pull request. Configure the repository ruleset to require **Agent completion enforcement**, one independent approval, and resolved conversations only after provisioning. Do not require `agent-completion/truth-gate`.
19 changes: 11 additions & 8 deletions .github/workflows/agent-completion-enforcement.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,10 @@ jobs:
# output/artifact. This workflow never executes PR-controlled code or
# accepts an artifact produced by a pull_request workflow.
gh api "repos/$REPO/commits/$head/check-runs" --jq '.check_runs[] | select(.name == "Agent Lock trusted publication") | .output.text' > trusted-report.json
if test -s trusted-report.json; then
python3 scripts/ci/agent_completion_enforcement.py trusted-report.json .github/agent-lock/trusted-publishers.json "$head" "$PR" > enforcement-verdict.json
else
printf '%s\n' '{"conclusion":"failure","reason":"missing_trusted_publication","details":{}}' > enforcement-verdict.json
fi
# The verifier fails closed on any provisioned policy; it publishes a
# neutral advisory verdict only when no publication exists and every
# trusted allowlist is verifiably empty (App not yet provisioned).
python3 scripts/ci/agent_completion_enforcement.py trusted-report.json .github/agent-lock/trusted-publishers.json "$head" "$PR" > enforcement-verdict.json
- name: Publish the required head-bound Check run
if: always()
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
Expand Down Expand Up @@ -73,7 +72,9 @@ jobs:
}
const conclusion = verdict.conclusion === 'success'
? 'success'
: 'failure';
: verdict.conclusion === 'neutral'
? 'neutral'
: 'failure';
const summary = JSON.stringify(verdict);
await github.rest.checks.create({
owner: context.repo.owner,
Expand All @@ -85,10 +86,12 @@ jobs:
output: {
title: conclusion === 'success'
? 'Trusted evidence verified'
: 'Trusted evidence blocked',
: conclusion === 'neutral'
? 'Trusted publisher not yet provisioned'
: 'Trusted evidence blocked',
summary: summary.slice(0, 60000)
}
});
if (conclusion !== 'success') {
if (conclusion === 'failure') {
core.setFailed(verdict.reason || 'trusted evidence blocked');
}
3 changes: 3 additions & 0 deletions apps/web/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ UVAI_AI_RATE_LIMIT_PER_MINUTE=12

# AI Gateway (Vercel AI SDK)
OPENAI_API_KEY=sk-...
# Optional: OpenAI model for the action agent (Responses API); supports
# Codex/ChatGPT models such as codex-mini-latest. Defaults to gpt-4o-mini.
# OPENAI_ACTION_MODEL=codex-mini-latest
ANTHROPIC_API_KEY=sk-ant-...

# Supabase
Expand Down
43 changes: 43 additions & 0 deletions apps/web/src/lib/__tests__/action-agent-model.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import { describe, it, expect, afterEach } from 'vitest';
import { resolveOpenAIActionModel, AVAILABLE_TOOL_NAMES } from '@/lib/action-agent';

const ORIGINAL = process.env.OPENAI_ACTION_MODEL;

afterEach(() => {
if (ORIGINAL === undefined) delete process.env.OPENAI_ACTION_MODEL;
else process.env.OPENAI_ACTION_MODEL = ORIGINAL;
});

describe('resolveOpenAIActionModel', () => {
it('defaults to gpt-4o-mini when OPENAI_ACTION_MODEL is unset', () => {
delete process.env.OPENAI_ACTION_MODEL;
expect(resolveOpenAIActionModel()).toBe('gpt-4o-mini');
});

it('uses a configured Codex model on the Responses API', () => {
process.env.OPENAI_ACTION_MODEL = 'codex-mini-latest';
expect(resolveOpenAIActionModel()).toBe('codex-mini-latest');
});

it('trims whitespace and falls back to the default for blank values', () => {
process.env.OPENAI_ACTION_MODEL = ' gpt-5-codex ';
expect(resolveOpenAIActionModel()).toBe('gpt-5-codex');

process.env.OPENAI_ACTION_MODEL = ' ';
expect(resolveOpenAIActionModel()).toBe('gpt-4o-mini');
});

it('resolves per call so runtime env changes take effect', () => {
process.env.OPENAI_ACTION_MODEL = 'codex-mini-latest';
expect(resolveOpenAIActionModel()).toBe('codex-mini-latest');
process.env.OPENAI_ACTION_MODEL = 'gpt-4o-mini';
expect(resolveOpenAIActionModel()).toBe('gpt-4o-mini');
});
});

describe('action agent surface', () => {
it('still exposes the executable tool registry', () => {
expect(AVAILABLE_TOOL_NAMES).toContain('dispatch_agent');
expect(AVAILABLE_TOOL_NAMES.length).toBeGreaterThan(0);
});
});
22 changes: 18 additions & 4 deletions apps/web/src/lib/action-agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ import 'server-only';
*
* Provider routing mirrors the rest of the app: OpenAI (Responses API,
* multi-round function calling) is primary; Gemini is a single-round fallback
* used only when OpenAI is unavailable or quota-limited.
* used only when OpenAI is unavailable or quota-limited. The OpenAI model is
* configurable via `OPENAI_ACTION_MODEL`, so Codex/ChatGPT models served by
* the same Responses API (e.g. `codex-mini-latest`) can drive the agent.
*/

import OpenAI from 'openai';
Expand All @@ -33,11 +35,22 @@ function getOpenAI(): OpenAI {
return _openai;
}

const MODEL_OPENAI = 'gpt-4o-mini';
const DEFAULT_MODEL_OPENAI = 'gpt-4o-mini';
const MODEL_GEMINI = GEMINI_FAST_MODEL;
const MAX_TOOL_ROUNDS = 4;
const MAX_TRANSCRIPT_CHARS = 8000;

/**
* Resolve the OpenAI model used for action extraction. Set `OPENAI_ACTION_MODEL`
* to route the agent to a Codex/ChatGPT model on the same Responses API
* (e.g. `codex-mini-latest` or `gpt-5-codex`); defaults to `gpt-4o-mini`.
* Resolved per call so Next.js runtime env changes take effect without a rebuild.
*/
export function resolveOpenAIActionModel(): string {
const model = process.env.OPENAI_ACTION_MODEL?.trim();
return model || DEFAULT_MODEL_OPENAI;
Comment on lines +49 to +51
}

const SYSTEM_PROMPT = `You are an action agent for a video-to-workflow platform.
Read the transcript and decide which executable tools to call so the viewer can
act on what was said. Call a tool for every concrete, practical item — do not
Expand Down Expand Up @@ -85,10 +98,11 @@ function isPlainObject(v: unknown): v is Record<string, unknown> {
/** Run the OpenAI Responses-API tool-calling loop until the model stops calling tools. */
async function runWithOpenAI(opts: RunActionAgentOptions, ctx: ToolContext): Promise<AgentAction[]> {
const openai = getOpenAI();
const model = resolveOpenAIActionModel();
const actions: AgentAction[] = [];

let response = await openai.responses.create({
model: MODEL_OPENAI,
model,
instructions: SYSTEM_PROMPT,
input: buildUserPrompt(opts.transcript, opts.videoTitle),
tools: toOpenAITools(),
Expand Down Expand Up @@ -150,7 +164,7 @@ async function runWithOpenAI(opts: RunActionAgentOptions, ctx: ToolContext): Pro
}

response = await openai.responses.create({
model: MODEL_OPENAI,
model,
previous_response_id: response.id,
input: toolOutputs,
tools: toOpenAITools(),
Expand Down
2 changes: 1 addition & 1 deletion docs/agent-completion-truth-gate.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The truth gate converts repository evidence into one deterministic verdict. It n

## Required enforcement rollout

`agent-completion/truth-gate` remains advisory and must not be added as a required status. The separate **Agent completion enforcement** workflow is the required, head-bound Check run. It verifies an exact-head machine-readable report published through a dedicated GitHub App and rejects missing, stale, edited, deleted, ambiguous, or self-published agent evidence. The protected policy is `.github/agent-lock/trusted-publishers.json`; empty trusted-publisher or trusted-actor allowlists are intentionally blocking. Custom roles are fail-closed.
`agent-completion/truth-gate` remains advisory and must not be added as a required status. The separate **Agent completion enforcement** workflow is the required, head-bound Check run. It verifies an exact-head machine-readable report published through a dedicated GitHub App and rejects missing, stale, edited, deleted, ambiguous, or self-published agent evidence. The protected policy is `.github/agent-lock/trusted-publishers.json`; while a published report exists, empty trusted-publisher or trusted-actor allowlists are intentionally blocking. When no trusted publication exists and every allowlist in a well-formed fail-closed policy is empty (the dedicated App is not yet provisioned), the workflow publishes a `neutral` advisory Check instead of failing every pull request; any provisioned or malformed policy with a missing publication still fails closed. Custom roles are fail-closed.

The trusted publisher must bind report data to PR number, full head SHA, delivery/run identity, trusted label authorization, trusted human exemption (when applicable), append-only agent events, and per-path passed/failed/error counts. The required verifier never executes PR code. Repository rules must require **Agent completion enforcement**, one independent approval, and resolved conversations. They must not require the advisory custom status.

Expand Down
Loading
Loading