From 0340f6b65171d8bc1c44dfc35df7a559b7c78455 Mon Sep 17 00:00:00 2001 From: Ayman Hamed Date: Mon, 27 Jul 2026 22:20:48 +0300 Subject: [PATCH] feat(agent): runStreaming(_:images:) for vision-capable models Adds a streaming overload that accepts [LLMImage] and attaches them to the user turn; the no-image runStreaming(_:) delegates to it. Enables streamed vision replies (image attachments) without dropping to non-streaming. Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 1 + Sources/SwiftAgentKit/Core/Agent.swift | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index dcdbbdb..5eda496 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ All notable changes to SwiftAgentKit will be documented in this file. - **ContextSift: stop the `artifact_read` retrieval loop.** A large *active* tool result was bounded to `maxActiveResultChars` and spilled to an artifact with a "use artifact_read" hint; when the model then called `artifact_read`, that retrieval result was *itself* re-truncated and re-spilled — so the model kept calling `artifact_read` to "get the full output" that never fully surfaced (observed as ~6 repeated `artifact_read` calls until `maxTurns` ran out). Fix: retrieval tools (`artifact_read`, `artifact_search`) are now exempt from active-display truncation and receipt spilling — their output is shown in full (the tools already page via `offset`/`limit`). Also raised the default `maxActiveResultChars` 2000 → 8000 so ordinary tool outputs fit inline without any artifact round-trip. Regression coverage added (an active `artifact_read` result larger than the bound is shown in full, never re-truncated). ### Added +- **`runStreaming(_:images:)`** — a streaming overload that accepts `[LLMImage]` for vision-capable models (attaches the images to the user turn; the ReAct loop is otherwise identical). The no-image `runStreaming(_:)` now delegates to it. - **Self-improving skills — persist skills the agent authors at runtime.** New `AgentSkillStore` protocol + `FileAgentSkillStore` (one human-editable markdown file per skill: `# name` / `Triggers:` / instructions) and a built-in `LearnSkillTool` (`learn_skill`). Set `agent.skillStore = …` and the agent (a) loads previously authored skills into its `SkillRegistry` and (b) auto-registers `learn_skill` — so it can turn a recurring task, or a corrected mistake, into a reusable keyword-triggered skill that fires immediately (added to the live registry) and persists across sessions. Mirrors the `memoryStore` → `RememberTool` pattern. Regression coverage added (store round-trip; `learn_skill` persists + activates on a matching query). - **`SwiftAgentKitTools` — a new opt-in product of ready-made native tools.** Import it and register the ones you want; the app supplies the confirmation UI/policy via `AgentCallbacks.onToolConfirmation`. - **Filesystem** (Foundation-only, cross-platform): `read_file` (paged), `write_file` (append/overwrite, `requiresConfirmation`), `list_dir`, `search_files` (name and/or content substring, bounded traversal). diff --git a/Sources/SwiftAgentKit/Core/Agent.swift b/Sources/SwiftAgentKit/Core/Agent.swift index 7173824..4b226d2 100644 --- a/Sources/SwiftAgentKit/Core/Agent.swift +++ b/Sources/SwiftAgentKit/Core/Agent.swift @@ -1079,6 +1079,12 @@ public final class Agent: @unchecked Sendable { /// text, but on the final turn the original deltas have already been /// streamed — so a rewrite won't retroactively change what the caller saw. public func runStreaming(_ query: String) -> AsyncThrowingStream { + runStreaming(query, images: []) + } + + /// Streaming variant that accepts images for vision-capable models. The images + /// are attached to the user turn; the rest of the ReAct loop is identical. + public func runStreaming(_ query: String, images: [LLMImage]) -> AsyncThrowingStream { return AsyncThrowingStream { [weak self] continuation in let task = Task { [weak self] in guard let self else { @@ -1088,7 +1094,7 @@ public final class Agent: @unchecked Sendable { do { _ = try await self.runLoop( query: query, - images: [], + images: images, onText: { continuation.yield($0) } ) continuation.finish()