-
Notifications
You must be signed in to change notification settings - Fork 205
feat: scaffold Debate Arena agent kit #197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
HEMANTH-A-7
wants to merge
22
commits into
Lamatic:main
Choose a base branch
from
HEMANTH-A-7:feat/debate-arena-kit
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
5b9e092
Add files via upload
HEMANTH-A-7 62887cf
Add files via upload
HEMANTH-A-7 3be04b2
Add files via upload
HEMANTH-A-7 9b82331
Add files via upload
HEMANTH-A-7 6ad57ae
Add files via upload
HEMANTH-A-7 828f5ec
Add files via upload
HEMANTH-A-7 56e8fbf
Add files via upload
HEMANTH-A-7 d872caa
Add files via upload
HEMANTH-A-7 23d0b60
Add files via upload
HEMANTH-A-7 853b2d9
Add files via upload
HEMANTH-A-7 e2b2075
Add files via upload
HEMANTH-A-7 543f1a2
Add files via upload
HEMANTH-A-7 6eadcc9
Add files via upload
HEMANTH-A-7 1dbff6b
Add files via upload
HEMANTH-A-7 a085e3b
Add files via upload
HEMANTH-A-7 f6e9f7a
Add files via upload
HEMANTH-A-7 01e96c2
Add files via upload
HEMANTH-A-7 aba1c53
Add files via upload
HEMANTH-A-7 1a75b26
Add files via upload
HEMANTH-A-7 831dc52
Use CSS variables for error color in globals.css
HEMANTH-A-7 97b986b
Re-run judge on regenerate to keep verdict/history in sync
HEMANTH-A-7 1d46215
Disable history buttons while a debate is running or regenerating
HEMANTH-A-7 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Lamatic API credentials (Settings -> API Keys / Project / API Docs in Studio) | ||
| LAMATIC_API_URL=YOUR_API_ENDPOINT | ||
| LAMATIC_PROJECT_ID=YOUR_PROJECT_ID | ||
| LAMATIC_API_KEY=YOUR_API_KEY | ||
|
|
||
| # Deployed flow IDs (Studio -> flow -> three-dot menu -> Copy Flow Id) | ||
| DEBATE_SETUP_FLOW_ID=YOUR_DEBATE_SETUP_FLOW_ID | ||
| DEBATE_ROUND_FLOW_ID=YOUR_DEBATE_ROUND_FLOW_ID | ||
| DEBATE_JUDGE_FLOW_ID=YOUR_DEBATE_JUDGE_FLOW_ID |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| .lamatic/ | ||
| node_modules/ | ||
| .next/ | ||
| .env | ||
| .env.local | ||
| *.tsbuildinfo |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| # Debate Arena | ||
|
|
||
| Pose any tradeoff or decision and watch two AI agents argue opposing sides across multiple rounds, then get an impartial judge's synthesis — a pros/cons matrix and a final recommendation. | ||
|
|
||
| ## Why | ||
|
|
||
| Asking a single AI for advice on a hard tradeoff gets you one perspective, delivered with unearned confidence. Real decisions are clearer when you see the strongest case for each side argued out, rebutted, and then weighed by someone neutral. Debate Arena turns that into a repeatable workflow: two persona agents debate, then a judge agent synthesizes. | ||
|
|
||
| ## How it works | ||
|
|
||
| Three flows: | ||
|
|
||
| ``` | ||
| "microservices vs monolith" ──▶ debate-setup ──▶ two clear positions | ||
| │ | ||
| ┌────────────────────┴────────────────────┐ | ||
| ▼ ▼ | ||
| debate-round (Position A) debate-round (Position B) | ||
| opening argument, then rebuttals each round, alternating | ||
| └────────────────────┬────────────────────┘ | ||
| ▼ | ||
| debate-judge ──▶ pros/cons + verdict | ||
| ``` | ||
|
|
||
| ### 1. `debate-setup` | ||
|
|
||
| **Input:** `topic` — a raw decision or question, in plain English (already framed as "X vs Y" or open-ended, e.g. "should I take this job"). | ||
|
|
||
| **What it does:** Neutrally reframes the topic and produces exactly two clearly opposed positions a reasonable person could argue, without picking a side. States any assumptions it made explicitly. | ||
|
|
||
| **Output:** `cleanTopic`, `positionA` (`label`, `stance`), `positionB` (`label`, `stance`), `context`. | ||
|
|
||
| ### 2. `debate-round` | ||
|
|
||
| **Input:** `topic`, `position` (the side this call argues for), `opponentPosition`, `transcript` (all prior statements from both sides), `round`, `isRebuttal`. | ||
|
|
||
| **What it does:** Generates that position's next statement — an opening argument on round 1, or a rebuttal of the opponent's most recent point plus a reinforced case on later rounds. This flow is called multiple times (once per side per round) by the app layer, not chained automatically, since each call needs a different persona/context. | ||
|
|
||
| **Output:** `statement`, `keyPoint`. | ||
|
|
||
| ### 3. `debate-judge` | ||
|
|
||
| **Input:** `topic`, `positionA`, `positionB`, `transcript` (the full debate). | ||
|
|
||
| **What it does:** Extracts the pros and cons actually raised for each side (no inventing new ones), identifies the single strongest argument per side, and gives a final recommendation with an honest confidence level and caveats — including saying "it depends" when that's the truthful answer. | ||
|
|
||
| **Output:** `prosA`, `consA`, `prosB`, `consB`, `strongestArgA`, `strongestArgB`, `recommendation`, `confidence`, `caveats`. | ||
|
|
||
| ## Uniqueness | ||
|
|
||
| Checked against all 79 existing AgentKit registry entries (30 kits, 6 bundles, 43 templates) — nothing else runs a structured multi-agent debate-then-judge pattern. The closest neighbors (`founder-lens`, `system-design-analyzer`) analyze a single input from one perspective; none stage opposing agents against each other and synthesize a verdict. | ||
|
|
||
| ## Running the app locally | ||
|
|
||
| ```bash | ||
| cd apps | ||
| cp .env.example .env.local # fill in your Lamatic credentials + deployed flow IDs | ||
| npm install | ||
| npm run dev | ||
| ``` | ||
|
|
||
| Or deploy directly with the button below once the three flows are built and deployed in Lamatic Studio. | ||
|
|
||
| [](https://vercel.com/new/clone?repository-url=https://github.com/Lamatic/AgentKit&root-directory=kits%2Fdebate-arena%2Fapps&env=DEBATE_SETUP_FLOW_ID,DEBATE_ROUND_FLOW_ID,DEBATE_JUDGE_FLOW_ID,LAMATIC_API_URL,LAMATIC_PROJECT_ID,LAMATIC_API_KEY&envDescription=Your%20Lamatic%20API%20credentials%20and%20deployed%20flow%20IDs%20are%20required.&envLink=https://github.com/Lamatic/AgentKit/tree/main/kits/debate-arena%23readme) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| # Debate Arena | ||
|
|
||
| ## Overview | ||
|
|
||
| Debate Arena turns any tradeoff or decision into a structured, multi-agent debate: two persona agents argue opposing positions across several rounds, rebutting each other, and a third impartial judge agent synthesizes a pros/cons matrix and a final recommendation. It is a three-flow kit (with a small Next.js app in `apps/`). | ||
|
|
||
| ## Flows | ||
|
|
||
| ### 1. `debate-setup` | ||
|
|
||
| **Input:** `topic` (a raw decision or question in plain English). | ||
|
|
||
| **What it does:** Uses an LLM (`groq/llama-3.3-70b-versatile`) to neutrally reframe the topic and produce exactly two clearly opposed positions, without taking a side itself. If the user's input is already framed as a binary ("X vs Y"), it keeps that framing; if open-ended, it frames the affirmative and the alternative case. States assumptions explicitly rather than burying them. | ||
|
|
||
| **Output:** Structured JSON — `cleanTopic`, `positionA` (`label`, `stance`), `positionB` (`label`, `stance`), `context` — meant to seed every subsequent `debate-round` call. | ||
|
|
||
| ### 2. `debate-round` | ||
|
|
||
| **Input:** `topic`, `position` (which side to argue), `opponentPosition`, `transcript` (all statements so far), `round`, `isRebuttal`. | ||
|
|
||
| **What it does:** Uses an LLM (`groq/llama-3.3-70b-versatile`) to generate one persona's next statement. On the opening round it produces a fresh argument for that position; on later rounds, with `isRebuttal: true`, it must directly address the opponent's most recent point before reinforcing its own case. The prompt explicitly forbids fabricated statistics — arguments should reason from principles and well-known tradeoffs, not invented facts. Statements are capped around 120 words to keep the debate readable. This flow is invoked repeatedly (once per side per round) by the calling app, alternating `position`/`opponentPosition`; it is not a chained pipeline step by itself. | ||
|
|
||
| **Output:** `statement`, `keyPoint` (a one-line summary of the point made, useful for the judge and for future rebuttals to target precisely). | ||
|
|
||
| ### 3. `debate-judge` | ||
|
|
||
| **Input:** `topic`, `positionA`, `positionB`, `transcript` (every turn from both `debate-round` calls). | ||
|
|
||
| **What it does:** Uses an LLM (`groq/llama-3.3-70b-versatile`) to review the full transcript and extract only the pros/cons that were actually argued (never inventing new ones), identify the single strongest argument per side, and produce a final recommendation with an honestly-stated confidence level (`low`/`medium`/`high`) and explicit caveats. If the honest answer is "it depends", the flow says so and states exactly what it depends on rather than forcing a false-confidence pick. | ||
|
|
||
| **Output:** `prosA`, `consA`, `prosB`, `consB`, `strongestArgA`, `strongestArgB`, `recommendation`, `confidence`, `caveats`. | ||
|
|
||
| ## Guardrails | ||
|
|
||
| All three flows follow `constitutions/default.md`: no fabricated facts or statistics (arguments must reason from stated principles/tradeoffs), the judge only reports pros/cons that were actually present in the transcript, confidence is stated honestly rather than defaulting to "high", and none of the flows log or retain the debate topic or transcript beyond the request/response cycle. | ||
|
|
||
| ## Integration | ||
|
|
||
| Call `debate-setup` once with the raw topic, then call `debate-round` in a loop — once per side, per round, alternating `position`/`opponentPosition` and passing the growing `transcript` array each time — for as many rounds as desired (1-3 recommended). Finally call `debate-judge` once with the complete transcript. See `README.md` for the flow diagram, or run `apps/` locally for a ready-made UI that drives this exact sequence and renders it as a live back-and-forth debate. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Lamatic API credentials (Settings -> API Keys / Project / API Docs in Studio) | ||
| LAMATIC_API_URL=YOUR_API_ENDPOINT | ||
| LAMATIC_PROJECT_ID=YOUR_PROJECT_ID | ||
| LAMATIC_API_KEY=YOUR_API_KEY | ||
|
|
||
| # Deployed flow IDs (Studio -> flow -> three-dot menu -> Copy Flow Id) | ||
| DEBATE_SETUP_FLOW_ID=YOUR_DEBATE_SETUP_FLOW_ID | ||
| DEBATE_ROUND_FLOW_ID=YOUR_DEBATE_ROUND_FLOW_ID | ||
| DEBATE_JUDGE_FLOW_ID=YOUR_DEBATE_JUDGE_FLOW_ID |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| { | ||
| "extends": ["next/core-web-vitals"] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,195 @@ | ||
| "use server"; | ||
|
|
||
| import { z } from "zod"; | ||
| import { getLamaticClient } from "@/lib/lamatic-client"; | ||
| import kitConfig from "../../lamatic.config"; | ||
|
|
||
| export type ActionResult<T> = | ||
| | { success: true; data: T } | ||
| | { success: false; error: string }; | ||
|
|
||
| const PositionSchema = z.object({ | ||
| label: z.string(), | ||
| stance: z.string(), | ||
| }); | ||
|
|
||
| export type Position = z.infer<typeof PositionSchema>; | ||
|
|
||
| const DebateSetupSchema = z.object({ | ||
| cleanTopic: z.string(), | ||
| positionA: PositionSchema, | ||
| positionB: PositionSchema, | ||
| context: z.string(), | ||
| }); | ||
|
|
||
| export type DebateSetup = z.infer<typeof DebateSetupSchema>; | ||
|
|
||
| const DebateTurnSchema = z.object({ | ||
| round: z.number(), | ||
| side: z.enum(["A", "B"]), | ||
| label: z.string(), | ||
| statement: z.string(), | ||
| keyPoint: z.string(), | ||
| }); | ||
|
|
||
| export type DebateTurn = z.infer<typeof DebateTurnSchema>; | ||
|
|
||
| const DebateRoundResultSchema = z.object({ | ||
| statement: z.string(), | ||
| keyPoint: z.string(), | ||
| }); | ||
|
|
||
| const DebateVerdictSchema = z.object({ | ||
| prosA: z.array(z.string()), | ||
| consA: z.array(z.string()), | ||
| prosB: z.array(z.string()), | ||
| consB: z.array(z.string()), | ||
| strongestArgA: z.string(), | ||
| strongestArgB: z.string(), | ||
| recommendation: z.string(), | ||
| confidence: z.enum(["low", "medium", "high"]), | ||
| caveats: z.array(z.string()), | ||
| }); | ||
|
|
||
| export type DebateVerdict = z.infer<typeof DebateVerdictSchema>; | ||
|
|
||
| /** | ||
| * lamatic.config.ts is the source of truth for which env var holds each | ||
| * step's deployed flow ID. Resolving through the config (instead of | ||
| * hardcoding the env var name at each call site) keeps this file in sync | ||
| * if the config ever changes a step's envKey. | ||
| */ | ||
| type StepId = (typeof kitConfig.steps)[number]["id"]; | ||
|
|
||
| function requireFlowId(stepId: StepId): string { | ||
| const step = kitConfig.steps.find((s) => s.id === stepId); | ||
| if (!step) { | ||
| throw new Error(`Unknown step "${stepId}" -- check lamatic.config.ts`); | ||
| } | ||
|
|
||
| const flowId = process.env[step.envKey]; | ||
| if (!flowId) { | ||
| throw new Error( | ||
| `Missing ${step.envKey}. Set it in .env.local (see .env.example) after deploying the "${stepId}" flow in Lamatic Studio.` | ||
| ); | ||
| } | ||
| return flowId; | ||
| } | ||
|
|
||
| /** | ||
| * Step 1: turn a raw decision/question into a neutral topic framing plus | ||
| * two clearly opposed positions. | ||
| */ | ||
| export async function runDebateSetup( | ||
| topic: string | ||
| ): Promise<ActionResult<DebateSetup>> { | ||
| try { | ||
| if (!topic.trim()) { | ||
| return { success: false, error: "Please describe the decision or question you want debated." }; | ||
| } | ||
|
|
||
| const client = getLamaticClient(); | ||
| const flowId = requireFlowId("debate-setup"); | ||
|
|
||
| const res = await client.executeFlow(flowId, { topic: topic.trim() }); | ||
|
|
||
| if (res.status !== "success" || !res.result) { | ||
| return { success: false, error: res.message || "Could not frame the debate topic. Please try rephrasing it." }; | ||
| } | ||
|
|
||
| const parsed = DebateSetupSchema.safeParse(res.result); | ||
| if (!parsed.success) { | ||
| return { success: false, error: "The debate-setup flow returned an unexpected response shape. Please try again." }; | ||
| } | ||
|
|
||
| return { success: true, data: parsed.data }; | ||
| } catch (err) { | ||
| return { success: false, error: err instanceof Error ? err.message : "Unexpected error framing the debate." }; | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Step 2: generate one agent's next statement (opening argument or | ||
| * rebuttal) for a given side. Called repeatedly by the client -- once per | ||
| * side per round -- with the accumulated transcript so far. | ||
| */ | ||
| export async function runDebateRound(params: { | ||
| topic: string; | ||
| position: Position; | ||
| opponentPosition: Position; | ||
| transcript: DebateTurn[]; | ||
| round: number; | ||
| side: "A" | "B"; | ||
| isRebuttal: boolean; | ||
| }): Promise<ActionResult<DebateTurn>> { | ||
| try { | ||
| const client = getLamaticClient(); | ||
| const flowId = requireFlowId("debate-round"); | ||
|
|
||
| const res = await client.executeFlow(flowId, { | ||
| topic: params.topic, | ||
| position: params.position, | ||
| opponentPosition: params.opponentPosition, | ||
| transcript: params.transcript, | ||
| round: params.round, | ||
| isRebuttal: params.isRebuttal, | ||
| }); | ||
|
|
||
| if (res.status !== "success" || !res.result) { | ||
| return { success: false, error: res.message || "Could not generate the next argument." }; | ||
| } | ||
|
|
||
| const parsed = DebateRoundResultSchema.safeParse(res.result); | ||
| if (!parsed.success) { | ||
| return { success: false, error: "The debate-round flow returned an unexpected response shape. Please try again." }; | ||
| } | ||
|
|
||
| const turn: DebateTurn = { | ||
| round: params.round, | ||
| side: params.side, | ||
| label: params.position.label, | ||
| statement: parsed.data.statement, | ||
| keyPoint: parsed.data.keyPoint, | ||
| }; | ||
|
|
||
| return { success: true, data: turn }; | ||
| } catch (err) { | ||
| return { success: false, error: err instanceof Error ? err.message : "Unexpected error generating this round." }; | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Step 3: given the full transcript, synthesize a pros/cons matrix and a | ||
| * final recommendation. | ||
| */ | ||
| export async function runDebateJudge(params: { | ||
| topic: string; | ||
| positionA: Position; | ||
| positionB: Position; | ||
| transcript: DebateTurn[]; | ||
| }): Promise<ActionResult<DebateVerdict>> { | ||
| try { | ||
| const client = getLamaticClient(); | ||
| const flowId = requireFlowId("debate-judge"); | ||
|
|
||
| const res = await client.executeFlow(flowId, { | ||
| topic: params.topic, | ||
| positionA: params.positionA, | ||
| positionB: params.positionB, | ||
| transcript: params.transcript, | ||
| }); | ||
|
|
||
| if (res.status !== "success" || !res.result) { | ||
| return { success: false, error: res.message || "Could not reach a verdict." }; | ||
| } | ||
|
|
||
| const parsed = DebateVerdictSchema.safeParse(res.result); | ||
| if (!parsed.success) { | ||
| return { success: false, error: "The debate-judge flow returned an unexpected response shape. Please try again." }; | ||
| } | ||
|
|
||
| return { success: true, data: parsed.data }; | ||
| } catch (err) { | ||
| return { success: false, error: err instanceof Error ? err.message : "Unexpected error judging the debate." }; | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.