Skip to content

feat(attachments): image and PDF attachments for widget and worker #150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
PAMulligan merged 3 commits into from
Sep 1, 2026
Merged

feat(attachments): image and PDF attachments for widget and worker #150

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2c5854a
feat(attachments): image and PDF attachments for widget and worker
Sep 1, 2026
e4fb367
Merge branch 'main' into 53-file-and-image-attachments
Sep 1, 2026
fa6abc4
chore(widget): raise bundle budgets for attachments
Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 26 additions & 4 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ claudius/
│ ├── src/
│ │ ├── index.ts # Hono API routes
│ │ ├── chat.ts # Claude API integration
│ │ ├── attachments.ts # Attachment parsing/validation/content blocks
│ │ ├── attachment-storage.ts # Passthrough vs R2 storage, signed URLs
│ │ ├── attachment-quota.ts # Per-IP / per-tenant daily upload quotas (KV)
│ │ ├── system-prompt.ts # Bot personality/knowledge
│ │ └── __tests__/ # Worker tests
│ ├── wrangler.toml # Cloudflare config
Expand Down Expand Up @@ -102,6 +105,7 @@ pnpm test # Run tests
| `ChatMessage` | Renders individual messages with URL linking |
| `ChatSources` | Slide-out sidebar displaying grouped source links |
| `SourceIcon` | Icon button with badge count to trigger source sidebar |
| `AttachmentPreview` | Image thumbnail / file chip for pending and sent attachments |

### useChat Hook

Expand Down Expand Up @@ -135,8 +139,9 @@ Worker can't stream.

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/chat` | POST | Send message, get AI response |
| `/api/chat/stream` | POST | Same request; streams the reply as SSE (`chunk`/`done`/`error` events) |
| `/api/chat` | POST | Send message, get AI response (JSON, or multipart when uploading files) |
| `/api/chat/stream` | POST | Same request; streams the reply as SSE (`chunk`/`tool`/`done`/`error` events) |
| `/api/attachments/*` | GET | Serve a stored attachment via signed URL (R2 mode only) |
| `/api/health` | GET | Health check |

`/api/chat/stream` shares the rate limiter, validation, and error shapes with
Expand Down Expand Up @@ -175,7 +180,14 @@ renders. Retrieval failures degrade to ungrounded replies. Ingestion:
{
messages: [
{ role: "user", content: "Hello" },
{ role: "assistant", content: "Hi there!" }
{ role: "assistant", content: "Hi there!" },
{
role: "user",
content: "What's on this receipt?",
attachments?: [
{ id: "att-1", name: "receipt.png", mediaType: "image/png", size: 1234, data?: "<base64>", key?: "att/<tenant>/<uuid>" }
]
}
]
}

Expand All @@ -184,10 +196,17 @@ renders. Retrieval failures degrade to ungrounded replies. Ingestion:
reply: "How can I help you today?",
sources?: [
{ url: "https://...", title: "...", type: "blog" | "page" | "external" }
]
],
attachments?: [ { id: "att-1", key: "att/...", url: "https://.../api/attachments/...?exp=&sig=", expiresAt: "..." } ]
}
```

Attachments (images + PDFs) are opt-in on the widget (`attachments` prop) and on by
default in the worker in passthrough mode (forward to Anthropic, store nothing).
Set `ATTACHMENT_STORAGE=r2` plus an `ATTACHMENTS` R2 binding and
`ATTACHMENT_SIGNING_SECRET` to keep uploads for `ATTACHMENT_RETENTION_HOURS`.
See `docs/src/content/docs/configuration/attachments.md`.

## Customization

### System Prompt
Expand Down Expand Up @@ -324,3 +343,6 @@ When finishing a development branch (via the `finishing-a-development-branch` sk
|----------|-------------|
| `ANTHROPIC_API_KEY` | Anthropic API key for Claude |
| `ALLOWED_ORIGIN` | CORS allowed origin (set in wrangler.toml for local dev) |
| `ATTACHMENTS_ENABLED`, `ATTACHMENT_TYPES`, `ATTACHMENT_MAX_BYTES`, `ATTACHMENT_MAX_COUNT`, `ATTACHMENT_MAX_REQUEST_BYTES` | Attachment acceptance limits (all optional) |
| `ATTACHMENT_QUOTA_IP_BYTES`, `ATTACHMENT_QUOTA_TENANT_BYTES`, `TENANT_ID` | Daily upload quotas enforced via the `RATE_LIMIT` KV |
| `ATTACHMENT_STORAGE`, `ATTACHMENT_RETENTION_HOURS`, `ATTACHMENT_SIGNING_SECRET` | `passthrough` (default) or `r2` storage; R2 needs the `ATTACHMENTS` bucket binding and the secret |
41 changes: 40 additions & 1 deletion clients/_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,26 @@
"type": "string",
"enum": ["bottom-right", "bottom-left", "top-right", "top-left"]
},
"accentColor": { "type": "string", "pattern": "^#[0-9a-fA-F]{6}$" }
"accentColor": { "type": "string", "pattern": "^#[0-9a-fA-F]{6}$" },
"attachments": {
"description": "Let visitors attach images and PDFs. true enables the defaults (5 MB, 5 files per message).",
"oneOf": [
{ "type": "boolean" },
{
"type": "object",
"additionalProperties": false,
"properties": {
"maxSizeBytes": { "type": "integer", "minimum": 1 },
"maxCount": { "type": "integer", "minimum": 1 },
"allowedTypes": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
}
}
}
]
}
}
},
"worker": {
Expand All @@ -57,6 +76,26 @@
"systemPrompt": {
"type": "string",
"description": "Relative path to a markdown file containing the system prompt"
},
"attachments": {
"type": "object",
"description": "Worker-side attachment settings; map to the ATTACHMENT_* variables",
"additionalProperties": false,
"properties": {
"enabled": { "type": "boolean" },
"maxBytes": { "type": "integer", "minimum": 1 },
"maxCount": { "type": "integer", "minimum": 1 },
"maxRequestBytes": { "type": "integer", "minimum": 1 },
"allowedTypes": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"storage": { "type": "string", "enum": ["passthrough", "r2"] },
"retentionHours": { "type": "integer", "minimum": 1 },
"quotaIpBytesPerDay": { "type": "integer", "minimum": 0 },
"quotaTenantBytesPerDay": { "type": "integer", "minimum": 0 }
}
}
}
}
Expand Down
88 changes: 85 additions & 3 deletions docs/src/content/docs/api/rest.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ sidebar:
Base URL: your deployed worker, e.g.
`https://claudius-chat-worker.<you>.workers.dev`. CORS restricts callers to
the configured [`ALLOWED_ORIGIN`](/configuration/worker/) list (plus
`http://localhost:*`). Allowed methods: `POST`, `OPTIONS`;
`http://localhost:*`). Allowed methods: `GET`, `POST`, `OPTIONS`;
allowed header: `Content-Type`.

## POST /api/chat
Expand All @@ -32,22 +32,88 @@ Send the conversation so far; receive the assistant's reply.
|-------|------|-------|
| `messages` | array, required | Full conversation history, oldest first. Max **100** messages; each `content` is truncated to **2,000** characters |
| `messages[].role` | `"user" \| "assistant"` | Other roles are rejected |
| `messages[].content` | string | May be empty only when the message has attachments |
| `messages[].attachments` | array, optional | Files on a **user** message; see below |
| `conversationId` | string, optional | Opaque id used only for [analytics](/deployment/worker/#analytics-with-d1-optional) correlation |

#### Attachments

Each entry in `messages[].attachments` is:

```json
{
"id": "att-1",
"name": "receipt.png",
"mediaType": "image/png",
"size": 48213,
"data": "<base64, optional>",
"key": "att/<tenant>/<uuid> (optional)"
}
```

| Field | Notes |
|-------|-------|
| `id` | Client-generated, `[A-Za-z0-9_-]{1,64}`, unique per request. Names the multipart file part |
| `name` | Filename; shown to the model as the document title |
| `mediaType` | Must be on the worker's allowlist **and** match the file's leading bytes |
| `size` | Bytes. Recomputed server-side whenever bytes are present |
| `data` | Inline base64 (no `data:` prefix). Omit when sending multipart or referencing a stored file |
| `key` | Storage key returned by a previous response (R2 mode). The worker loads the file itself |

An attachment with neither `data` nor `key` (or whose stored copy expired) is
described to the model as "no longer available" rather than failing the
request. Attachments are forwarded to Claude as `image` blocks (JPEG, PNG,
GIF, WebP) or `document` blocks (PDF), placed before the message text.

#### Multipart requests

To avoid base64 overhead, send `multipart/form-data` instead of JSON:

- a `payload` text field containing the JSON body above, with attachment
entries **without** `data`;
- one file part per new upload, whose field name is the attachment `id`.

```bash
curl https://<worker>/api/chat \
-F 'payload={"messages":[{"role":"user","content":"What is the total?","attachments":[{"id":"f1","name":"receipt.png","mediaType":"image/png","size":48213}]}]}' \
-F 'f1=@receipt.png;type=image/png'
```

Stray file parts that match no attachment are rejected. The widget's client
switches to multipart automatically whenever a message carries inline bytes.

`POST /api/chat/stream` accepts the same JSON or multipart body. Attachment
errors are returned as plain JSON before the stream opens, and when the R2
backend stored uploads the `done` event carries the same `attachments` array
as the non-streaming response below.

### Response `200`

```json
{
"reply": "We're available Monday through Friday, 9am to 5pm.",
"sources": [
{ "url": "https://example.com/contact", "title": "Contact", "type": "page" }
],
"attachments": [
{
"id": "att-1",
"key": "att/example.com/6f1c…",
"url": "https://<worker>/api/attachments/att/example.com/6f1c…?exp=1750000000&sig=…",
"expiresAt": "2026-06-16T12:00:00.000Z"
}
]
}
```

`sources` is optional and reserved for retrieval-backed backends — the
bundled worker returns only `reply` today (see [RAG](/rag/)).

`attachments` is present only when the worker's
[R2 storage backend](/configuration/attachments/#r2) stored new uploads
during this request. Reference `key` on later turns instead of re-sending the
bytes; `url` is an HMAC-signed download link valid until `expiresAt`.

### Errors

All errors share one envelope:
Expand All @@ -58,12 +124,28 @@ All errors share one envelope:

| Status | `code` | When | Extra |
|--------|--------|------|-------|
| `400` | `VALIDATION_ERROR` | Empty/missing `messages`, more than 100 messages, invalid role | |
| `400` | `VALIDATION_ERROR` | Empty/missing `messages`, more than 100 messages, invalid role, message with neither text nor attachments | |
| `400` | `ATTACHMENTS_DISABLED` | Request carried attachments but `ATTACHMENTS_ENABLED=false` | |
| `400` | `ATTACHMENT_INVALID` | Disallowed or mismatched type, too many files, malformed id/key, or Claude could not process the file | |
| `413` | `ATTACHMENT_TOO_LARGE` | File over `ATTACHMENT_MAX_BYTES`, or this message's uploads over `ATTACHMENT_MAX_REQUEST_BYTES` | |
| `413` | `ATTACHMENT_QUOTA_EXCEEDED` | Daily per-IP or per-tenant upload quota reached | `Retry-After` header (seconds to next UTC midnight) |
| `429` | `RATE_LIMITED` | Per-IP limit exceeded (default 10/min, 50/hr) | `Retry-After` header (seconds); `limitType`: `"minute"` or `"hour"` |
| `500` | `CONFIG_ERROR` | Worker misconfiguration (e.g. bad API key) | |
| `500` | `CONFIG_ERROR` | Worker misconfiguration (e.g. bad API key, R2 mode without bucket/secret) | |
| `503` | `SERVICE_ERROR` | Claude temporarily unavailable/overloaded | |
| `500` | `UNKNOWN_ERROR` | Anything else | |

## GET /api/attachments/{key}

Serves a stored attachment (R2 mode only). The full URL, including the `exp`
and `sig` query parameters, comes from a chat response's `attachments[].url`;
it cannot be constructed by hand.

| Status | When |
|--------|------|
| `200` | Bytes with the original `Content-Type`, `Content-Disposition: inline`, and `Cache-Control: private` |
| `403` | Signature invalid or link expired |
| `404` | Unknown key, object expired/deleted, or storage is passthrough |

## GET /api/health

```json
Expand Down
Loading
Loading