feat: Implement A2A Protocol and MCP Support for AgentBeats Compatibility

[emooreatx]

Summary

Implement Agent2Agent (A2A) Protocol and Model Context Protocol (MCP) support to enable CIRISNode to function as a compliant "green agent" (evaluator) for the AgentBeats competition and broader agent interoperability.

Background

CIRISNode serves as the evaluation interface where AI agents ("purple agents") connect to take the HE-300 ethical benchmark. To participate in AgentBeats and enable universal agent interoperability, we need to implement two complementary protocols:

"All agents participating in AAA must comply with two open standards: Google's A2A protocol for task management, and MCP for tool and resource access. Any agent conforming to these standards can seamlessly participate in any AAA evaluation."
— AgentBeats Documentation

AgentBeats Requirements

Per the Phase 1 requirements:

Green Agent (Evaluator) Requirements

• Public GitHub repository with complete source code
• Baseline purple agent(s) demonstrating A2A compatibility
• Docker image enabling end-to-end execution without manual intervention
• AgentBeats platform registration

Evaluation Criteria We Must Meet

• Reproducibility: "Consistent results across runs with the same agents"
• Benchmark Design: Tasks that "test genuine agentic abilities—reasoning, planning, multi-step execution"
• Evaluation Methodology: "Automated evaluation where feasible"

Deadline: Phase 1 - January 31, 2026

---

Part 1: A2A Protocol Implementation

Overview

A2A is an open standard (originally Google, now Linux Foundation) enabling communication between AI agents using JSON-RPC 2.0 over HTTP(S).

"A2A tries to complement MCP where A2A is focused on a different problem, while MCP focuses on lowering complexity to connect agents with tools and data, A2A focuses on how to enable agents to collaborate."
— Google Developers Blog

Required Endpoints

1. Agent Card Discovery

GET /.well-known/agent.json

Returns JSON describing CIRISNode's capabilities:

{
  "name": "CIRISNode HE-300 Evaluator",
  "description": "Green agent for HE-300 ethical reasoning benchmark",
  "version": "1.0.0",
  "endpoint": "https://cirisnode.example.com/a2a",
  "skills": [
    {
      "name": "he300_evaluation",
      "description": "Evaluate agent responses against 300 ethical scenarios",
      "input_schema": { ... },
      "output_schema": { ... }
    }
  ],
  "auth": {
    "type": "bearer",
    "scheme": "jwt"
  }
}

2. Task Management (JSON-RPC 2.0)

POST /a2a/rpc
Content-Type: application/json

{
  "jsonrpc": "2.0",
  "method": "tasks/create",
  "params": {
    "skill": "he300_evaluation",
    "input": { "scenario_ids": [...], "model": "..." }
  },
  "id": "task-123"
}

Required methods:

• tasks/create - Start new evaluation task
• tasks/get - Get task status/results
• tasks/cancel - Cancel running task
• skills/list - List available evaluation capabilities

3. Streaming Support (SSE)

For long-running HE-300 evaluations:

GET /a2a/tasks/{task_id}/stream
Accept: text/event-stream

A2A SDK

Use the official Python SDK:

pip install a2a-sdk

Reference: a2aproject/A2A

---

Part 2: MCP Server Implementation

Overview

MCP enables purple agents to access CIRISNode's evaluation tools and benchmark data.

"The November 2025 specification represents a significant shift, expanding MCP beyond synchronous tool calling into an architecture capable of supporting secure, long-running, governed workflows."
— MCP Spec Updates

Required Primitives

1. Tools (Actions)

{
  "tools": [
    {
      "name": "run_he300_scenario",
      "description": "Evaluate agent response against a specific HE-300 scenario",
      "inputSchema": {
        "type": "object",
        "properties": {
          "scenario_id": { "type": "string" },
          "agent_response": { "type": "string" }
        },
        "required": ["scenario_id", "agent_response"]
      }
    },
    {
      "name": "run_he300_batch",
      "description": "Run batch evaluation across multiple scenarios",
      "inputSchema": { ... }
    },
    {
      "name": "get_evaluation_report",
      "description": "Generate signed evaluation report",
      "inputSchema": { ... }
    }
  ]
}

2. Resources (Data)

{
  "resources": [
    {
      "uri": "he300://scenarios",
      "name": "HE-300 Scenario List",
      "description": "Available ethical scenarios for evaluation",
      "mimeType": "application/json"
    },
    {
      "uri": "he300://categories",
      "name": "Scenario Categories",
      "description": "commonsense, deontology, justice, virtue ethics"
    }
  ]
}

3. Tasks (Async Operations - Nov 2025 Spec)

For long-running benchmark evaluations:

{
  "tasks": [
    {
      "name": "full_he300_benchmark",
      "description": "Complete 300-scenario evaluation with report generation"
    }
  ]
}

Transport Options

• STDIO - For local development/testing
• HTTP/SSE - For remote/cloud deployment (required for AgentBeats)

Authentication

Per MCP Nov 2025 spec, implement OAuth 2.1 with Resource Indicators (RFC 8707).

Reference: modelcontextprotocol/servers

---

Implementation Plan

Phase 1: Core A2A (Priority: High)

1. Add /.well-known/agent.json endpoint
2. Implement JSON-RPC 2.0 router at /a2a/rpc
3. Create task lifecycle management (create/get/cancel)
4. Add SSE streaming for task progress

Phase 2: MCP Server (Priority: High)

1. Implement MCP server with tools primitive
2. Expose HE-300 scenarios as resources
3. Add async tasks for full benchmark runs
4. Implement HTTP/SSE transport

Phase 3: AgentBeats Integration (Priority: Medium)

1. Create Docker image with AgentBeats controller
2. Build baseline purple agent for testing
3. Register on AgentBeats platform
4. Submit for Phase 1 evaluation

Phase 4: Security & Production (Priority: Medium)

1. OAuth 2.1 authorization flow
2. Rate limiting and access controls
3. Audit logging for all evaluations
4. Signed evaluation reports (Ed25519)

---

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Purple Agent (Under Test)                 │
└─────────────────────────┬───────────────────────────────────┘
                          │ A2A Protocol (JSON-RPC 2.0)
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                     CIRISNode (Green Agent)                  │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────┐  │
│  │ A2A Endpoints   │  │ MCP Server      │  │ REST API    │  │
│  │ /.well-known/   │  │ Tools/Resources │  │ /api/v1/    │  │
│  │ /a2a/rpc        │  │ /mcp/           │  │ benchmarks  │  │
│  └────────┬────────┘  └────────┬────────┘  └──────┬──────┘  │
│           └────────────────────┴──────────────────┘          │
│                              │                               │
└──────────────────────────────┼───────────────────────────────┘
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                    CIRISBench (Backend)                      │
│            EthicsEngine Enterprise + Infrastructure          │
└─────────────────────────────────────────────────────────────┘

---

Acceptance Criteria

• Agent Card served at /.well-known/agent.json
• A2A JSON-RPC endpoint functional at /a2a/rpc
• MCP server exposes HE-300 tools and resources
• Docker image builds and runs without manual intervention
• Baseline purple agent demonstrates successful evaluation
• All evaluations produce signed, reproducible reports
• Documentation for purple agent developers

---

References

A2A Protocol

• A2A Protocol Specification
• A2A GitHub Repository
• Google A2A Announcement
• Linux Foundation A2A Project

MCP Protocol

• MCP Specification (Nov 2025)
• MCP GitHub
• MCP Server Examples

AgentBeats

• AgentBeats Competition
• AgentBeats Documentation
• A2A Integration Guide
• Green vs Purple Agents

---

Related

• PR Feature/eee integration #26: EEE Integration (merged) - Backend evaluation engine
• PR feat: implement H3ERE-compliant MCP Bridge & Audit Anchoring #27: MCP Bridge attempt - See discussion for audit logging ideas
• CIRISBench - Evaluation infrastructure