From 1fdf48b9533d770f0205f218fd55dced756daa4e Mon Sep 17 00:00:00 2001 From: aurascoper Date: Sat, 25 Jul 2026 03:49:05 -0500 Subject: [PATCH 01/11] feat(runtime): add resolved identity and readiness probing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A resolved runtime object is not a resolved runtime identity. The runtime generates; the identity is an immutable, sanitized description that telemetry, readiness logic and the privacy UI consume — and that exists on the failure path too, so a runtime that did not resolve can still be described. ResolvedRuntimeIdentity keeps requested and resolved provider/model as separate pairs, making substitution visible rather than inferable, plus role, locality, readiness, prompt profile, the hash of the bytes actually transmitted, and the provider-reported model digest where one exists (Ollama; the Claude CLI reports none, so it stays nil rather than fabricated). RuntimeLocality exists because a local executable does not imply local inference. The Claude CLI is a local binary brokering to a remote service; Ollama on a non-loopback host is remote however local the config looks. Both are egress. Classifying by where the process runs would make the privacy banner assert on-device inference for the one path that leaves the machine. Readiness is established before the loop starts and never by generating: - Claude resolves through ClaudeExecutableResolver — the app path now uses the same exact resolver the harness got in A1 — and loads and hashes its prompt. No request reaches Anthropic. - Ollama gets a bounded GET /api/tags confirming the daemon is up and holds the exact requested model. No prompt is sent. A readiness failure disables the loop and attempts no alternate provider: substituting Claude when Ollama is unavailable would turn a local-inference choice into unrequested egress. RuntimeResolutionFailure carries a public message and an internal detail, sanitized at construction rather than at each display site. The resolver's own error embeds the full environment PATH and lastError is rendered in the UI, so a display site that must remember to sanitize would eventually forget. Mutation-tested; each fails a named test: M4 remove the /api/tags probe → 6 tests M5 report a missing model as present → 2 M6 substitute Claude on Ollama failure → 2 M8 classify the Claude CLI on-device → 1 M9 interpolate PATH into the message → 1 M10 delete the enabled→disabled write → 1 (closes A1's known M4 gap) M11 report an unavailable runtime ready → 1 Deferred to the next commit, and marked in place: the Witness is still resolved even when the profile disables it, nothing yet rejects a runtime resolved for the wrong role, and the adapter's dead systemPrompt field survives — retained so this commit lands without also rewriting Witness resolution, not as a new compatibility mechanism. No test added here makes a real provider request; readiness HTTP is mocked. Co-Authored-By: Claude Opus 5 --- ...nerationRuntimeTextGeneratingAdapter.swift | 13 + .../BCICloudBridge/LiveRuntimeFactory.swift | 401 ++++++++++++++---- .../BCICloudBridge/OllamaReadinessProbe.swift | 107 +++++ .../ResolvedRuntimeIdentity.swift | 256 +++++++++++ Sources/NeuralComposeApp/AppViewModel.swift | 122 +++++- .../LiveRuntimeFactoryFailClosedTests.swift | 375 ++++++++++++---- .../Support/MockURLProtocol.swift | 89 ++++ .../AppViewModelRuntimeFailClosedTests.swift | 202 +++++++-- 8 files changed, 1343 insertions(+), 222 deletions(-) create mode 100644 Sources/BCICloudBridge/OllamaReadinessProbe.swift create mode 100644 Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift create mode 100644 Tests/BCICloudBridgeTests/Support/MockURLProtocol.swift diff --git a/Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift b/Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift index 3f99c8a..c83fcd6 100644 --- a/Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift +++ b/Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift @@ -23,6 +23,19 @@ import Foundation public struct GenerationRuntimeTextGeneratingAdapter: MetadataPublishingTextGenerating { public nonisolated let isLive: Bool public nonisolated let modelIdentifier: String + + /// **LEGACY / DEAD — scheduled for removal with the R3 Witness fix.** + /// + /// Stored and never read. The bytes actually transmitted come from the + /// wrapped runtime's own prompt, so this field describes nothing: a caller + /// can construct a "Witness" adapter around the *pole* runtime, pass the + /// Witness prompt here, and get an object that reports one prompt while + /// sending another. `Sources/DialecticSession/main.swift` does exactly + /// that today. + /// + /// It is retained for this commit only so the identity/readiness layer can + /// land without also rewriting Witness resolution. Do not read it, and do + /// not treat it as provenance. public nonisolated let systemPrompt: String private let runtime: any GenerationRuntime diff --git a/Sources/BCICloudBridge/LiveRuntimeFactory.swift b/Sources/BCICloudBridge/LiveRuntimeFactory.swift index d110741..b32341f 100644 --- a/Sources/BCICloudBridge/LiveRuntimeFactory.swift +++ b/Sources/BCICloudBridge/LiveRuntimeFactory.swift @@ -11,113 +11,338 @@ import Foundation /// 2. `NEURALCOMPOSE_RUNTIME` / `NEURALCOMPOSE_MODEL` env vars. /// 3. Built-in defaults: `claude` / `claude-sonnet-5`. /// -/// The keep-bar: the Claude path returns the *legacy* -/// `ClaudeCLIGenerator` unchanged (the same path the live app -/// has been using pre-branch), so the production behavior is -/// preserved when the env vars are absent. The Ollama path -/// composes `OllamaGenerationRuntime` over `OllamaHTTPTransport` -/// and wraps it in the `GenerationRuntimeTextGeneratingAdapter` -/// the harness uses (step 6, commit `4002d3e`). +/// **The factory returns a runtime *and* an identity.** The identity is not a +/// log line — it is the value telemetry, readiness logic, and the privacy UI +/// consume, and it exists on the failure path too (carried by +/// `RuntimeResolutionFailure`), because a UI that can only describe runtimes +/// that resolved cannot tell the user what went wrong with the one that didn't. /// -/// ADR-009 invariant #2 (runtime is transport, not semantics): -/// the factory does NOT modify the prompt text. The system -/// prompt bytes are loaded from the `PromptProfile` resource -/// for the Ollama path (matching the harness); the Claude path -/// uses the legacy `ClaudeCLIGenerator.hypnagogicSystemPrompt` -/// default (also pre-extraction, also unchanged). Switching -/// runtimes is a configuration change, not a semantics change. +/// **Readiness is checked before the loop starts, never by generating.** +/// - Claude: the executable is resolved through `ClaudeExecutableResolver` +/// — the same resolver the harness uses — and the prompt is loaded and +/// hashed. No request is made to Anthropic. +/// - Ollama: a bounded `GET /api/tags` confirms the daemon is up and has the +/// exact requested model. No prompt is sent. +/// +/// A readiness failure disables the loop. **No alternate provider is ever +/// attempted**: substituting Claude when Ollama is unavailable would turn a +/// local-inference choice into unrequested network egress. +/// +/// ADR-009 invariant #2 (runtime is transport, not semantics): the factory does +/// NOT modify prompt text. The prompt profile is selected by *role*, and the +/// hash recorded is of the bytes actually transmitted. public enum LiveRuntimeFactory { - /// Resolved runtime identity (for logging / diagnostics). - public struct Resolved: Sendable, CustomStringConvertible { - public let name: String // "claude-cli" or "ollama" - public let model: String // "claude-sonnet-5" / "qwen2.5:0.5b" - public let systemPromptSource: String // "legacy-static" or "PromptProfile" - public init(name: String, model: String, systemPromptSource: String) { - self.name = name - self.model = model - self.systemPromptSource = systemPromptSource - } - public var description: String { - "name=\(name) model=\(model) systemPrompt=\(systemPromptSource)" - } - } + /// The default Ollama endpoint. Loopback, hence `onDevice`. + public static let defaultOllamaBaseURL = URL(string: "http://localhost:11434")! + + public static let defaultClaudeModel = "claude-sonnet-5" + public static let defaultOllamaModel = "qwen2.5:0.5b" - /// Build a `TextGenerating` for the live app, reading env vars - /// for runtime selection. The returned generator is what the - /// `HypnagogicDialecticLoop` and the `HypnagogicDialogueLoop` - /// call into. + /// Build a runtime for `role`, plus the identity describing it. /// - /// - Parameters: - /// - systemPrompt: the system prompt to use. For the - /// Claude path, this is passed through to - /// `ClaudeCLIGenerator(systemPrompt:)`. For the Ollama - /// path, this is the text the `OllamaHTTPTransport` - /// concatenates onto the user prompt with the - /// `systemPromptDelimiter` (ADR-009 invariant #2). If - /// `nil`, the Ollama path loads the prompt from the - /// `PromptProfile` resource (same as the harness). + /// - Throws: `RuntimeResolutionFailure`, which carries a sanitized identity + /// and a `publicMessage` safe to render. Never throws a raw + /// `ResolutionError`, whose description embeds the environment `PATH`. public static func make( + role: RuntimeRole, runtimeName: String? = ProcessInfo.processInfo.environment["NEURALCOMPOSE_RUNTIME"], model: String? = ProcessInfo.processInfo.environment["NEURALCOMPOSE_MODEL"], - systemPrompt: String? = nil, - ollamaBaseURL: URL = URL(string: "http://localhost:11434")! - ) throws -> (generator: any TextGenerating, resolved: Resolved) { - let resolvedName = (runtimeName ?? "claude").lowercased() - let resolvedModel: String = { - if let m = model, !m.isEmpty { return m } - return resolvedName == "ollama" ? "qwen2.5:0.5b" : "claude-sonnet-5" + ollamaBaseURL: URL = defaultOllamaBaseURL, + session: URLSession? = nil, + environment: [String: String] = ProcessInfo.processInfo.environment, + probeTimeout: TimeInterval = 3.0 + ) async throws -> (generator: any TextGenerating, identity: ResolvedRuntimeIdentity) { + let requestedProvider = (runtimeName ?? "claude").lowercased() + let requestedModel: String = { + if let model, !model.isEmpty { return model } + return requestedProvider == "ollama" ? defaultOllamaModel : defaultClaudeModel }() + let profile = promptProfile(for: role) - switch resolvedName { + switch requestedProvider { case "claude": - // Legacy path: same `ClaudeCLIGenerator` the live app - // used pre-branch. Preserved byte-for-byte so the - // production state is unchanged when the env vars - // are absent. - let gen = try ClaudeCLIGenerator( - model: resolvedModel, - systemPrompt: systemPrompt + return try await makeClaude( + role: role, + profile: profile, + requestedProvider: requestedProvider, + requestedModel: requestedModel, + environment: environment ) - return (gen, Resolved( - name: "claude-cli", model: resolvedModel, - systemPromptSource: "legacy-static" - )) case "ollama": - // New path: compose the `OllamaGenerationRuntime` - // (step 4, commit `ce68a53`) over the - // `OllamaHTTPTransport`, wrap in the - // `GenerationRuntimeTextGeneratingAdapter` (step 6, - // commit `4002d3e`). The system prompt bytes are - // loaded from the `PromptProfile` resource — the same - // bytes the harness sends to Ollama. ADR-009 invariant - // #2 is preserved: the transport concatenates the - // system prompt onto the user prompt with the - // documented delimiter, never modifies semantic - // intent. - let promptText = try systemPrompt ?? PromptProfile.wakingDialectical.load() - let runtime = try OllamaGenerationRuntime( - model: resolvedModel, + return try await makeOllama( + role: role, + profile: profile, + requestedProvider: requestedProvider, + requestedModel: requestedModel, + baseURL: ollamaBaseURL, + session: session, + probeTimeout: probeTimeout + ) + + default: + throw RuntimeResolutionFailure( + identity: unresolvedIdentity( + role: role, + profile: profile, + requestedProvider: requestedProvider, + requestedModel: requestedModel, + // An unknown provider has no endpoint, so there is nothing + // to classify. Reporting a locality here would be a guess. + locality: .localBrokerToRemoteService, + failure: .unknownProvider + ), + code: .unknownProvider, + publicMessage: "Unknown runtime '\(requestedProvider)'. Supported: claude, ollama." + ) + } + } + + /// The prompt profile a role transmits. This binding is why two runtimes + /// with identical provider and model are still distinct identities. + public static func promptProfile(for role: RuntimeRole) -> PromptProfile { + switch role { + case .dialectic: return .wakingDialectical + case .witness: return .witness + case .mirror: return .hypnagogic + } + } + + // MARK: - Claude + + private static func makeClaude( + role: RuntimeRole, + profile: PromptProfile, + requestedProvider: String, + requestedModel: String, + environment: [String: String] + ) async throws -> (generator: any TextGenerating, identity: ResolvedRuntimeIdentity) { + // The Claude CLI is a *local broker*. Inference is remote. Classifying + // it by where the process runs rather than where the tokens go would + // make the privacy banner assert on-device inference for the one path + // that leaves the machine. + let locality: RuntimeLocality = .localBrokerToRemoteService + + func failure( + _ code: RuntimeReadinessFailure, + _ message: String, + detail: String? + ) -> RuntimeResolutionFailure { + RuntimeResolutionFailure( + identity: unresolvedIdentity( + role: role, profile: profile, + requestedProvider: requestedProvider, requestedModel: requestedModel, + locality: locality, failure: code + ), + code: code, + publicMessage: message, + internalDetail: detail + ) + } + + // R2 continuation: the app now uses the same exact resolver the harness + // does, so `/usr/bin/env` can no longer stand in for the CLI on this + // path either. The resolver's own error embeds the full PATH, so it is + // deliberately NOT interpolated into the public message. + let executablePath: String + do { + executablePath = try ClaudeExecutableResolver.resolve(environment: environment) + } catch let error as ClaudeExecutableResolver.ResolutionError { + let code: RuntimeReadinessFailure = + if case .notFoundOnPath = error { .executableNotFound } else { .executableInvalid } + throw failure( + code, + "The claude CLI was not found. Install Claude Code and run `claude login`.", + detail: String(describing: error) + ) + } + + let promptText: String + do { + promptText = try profile.load() + } catch { + throw failure( + .promptResourceUnavailable, + "The \(profile.rawValue) prompt resource is unavailable, so this runtime cannot run.", + detail: String(describing: error) + ) + } + + let generator: ClaudeCLIGenerator + do { + generator = try ClaudeCLIGenerator( + model: requestedModel, systemPrompt: promptText, - interactionStyle: "dialectical", - baseURL: ollamaBaseURL + executablePath: executablePath ) - let adapter = GenerationRuntimeTextGeneratingAdapter( - runtime: runtime, - systemPrompt: promptText + } catch { + throw failure( + .promptResourceUnavailable, + "The Claude runtime could not be constructed with a constraining prompt.", + detail: String(describing: error) ) - return (adapter, Resolved( - name: "ollama", model: resolvedModel, - systemPromptSource: "PromptProfile" - )) + } - default: - throw NSError( - domain: "LiveRuntimeFactory", code: 1, - userInfo: [NSLocalizedDescriptionKey: - "unknown runtime '\(resolvedName)' — supported: claude, ollama"] + return (generator, ResolvedRuntimeIdentity( + role: role, + requestedProvider: requestedProvider, + requestedModel: requestedModel, + resolvedProvider: "claude", + resolvedModel: requestedModel, + // The Claude CLI reports no content digest. `nil` rather than a + // placeholder, so telemetry never records a fabricated value. + modelDigest: nil, + locality: locality, + readiness: .ready, + promptProfile: profile.rawValue, + promptHash: PromptProfile.sha256Hex(promptText), + systemPromptSource: "PromptProfile(\(profile.rawValue))" + )) + } + + // MARK: - Ollama + + private static func makeOllama( + role: RuntimeRole, + profile: PromptProfile, + requestedProvider: String, + requestedModel: String, + baseURL: URL, + session: URLSession?, + probeTimeout: TimeInterval + ) async throws -> (generator: any TextGenerating, identity: ResolvedRuntimeIdentity) { + // Loopback means the tokens stay here. Any other host is another + // machine, and inference there is not on-device however local the + // configuration file looks. + let locality: RuntimeLocality = + RuntimeIdentityRedaction.isLoopback(baseURL) ? .onDevice : .remoteEndpoint + let endpoint = RuntimeIdentityRedaction.endpoint(baseURL) + + func failure( + _ code: RuntimeReadinessFailure, + _ message: String, + detail: String? + ) -> RuntimeResolutionFailure { + RuntimeResolutionFailure( + identity: unresolvedIdentity( + role: role, profile: profile, + requestedProvider: requestedProvider, requestedModel: requestedModel, + locality: locality, failure: code + ), + code: code, + publicMessage: message, + internalDetail: detail + ) + } + + let promptText: String + do { + promptText = try profile.load() + } catch { + throw failure( + .promptResourceUnavailable, + "The \(profile.rawValue) prompt resource is unavailable, so this runtime cannot run.", + detail: String(describing: error) ) } + + // Readiness before enablement. An unpulled model used to resolve + // cleanly and fail at the first generation — after the loop was live. + let probe = OllamaReadinessProbe( + baseURL: baseURL, session: session, timeout: probeTimeout) + let resolvedModel: String + let digest: String? + switch await probe.probe(model: requestedModel) { + case .present(let available): + resolvedModel = available.name + digest = available.digest + case .modelMissing: + // The available-model list is deliberately not surfaced: it is a + // record of what the user has pulled locally, which the privacy + // banner has no business enumerating. + throw failure( + .modelMissing, + "Ollama does not have the model '\(requestedModel)'. Pull it with " + + "`ollama pull \(requestedModel)`.", + detail: "model not present in /api/tags" + ) + case .unreachable(let detail): + throw failure( + .endpointUnreachable, + "Ollama is not reachable at \(endpoint). Start it with `ollama serve`.", + detail: detail + ) + } + + let runtime: OllamaGenerationRuntime + do { + runtime = try OllamaGenerationRuntime( + model: resolvedModel, + systemPrompt: promptText, + interactionStyle: "dialectical", + baseURL: baseURL, + session: session ?? URLSession(configuration: .ephemeral) + ) + } catch { + throw failure( + .promptResourceUnavailable, + "The Ollama runtime could not be constructed with a constraining prompt.", + detail: String(describing: error) + ) + } + // `systemPrompt:` here feeds the adapter's dead legacy field; the bytes + // that actually go out come from `runtime`. Both the field and this + // argument are removed with the R3 Witness fix. + let adapter = GenerationRuntimeTextGeneratingAdapter( + runtime: runtime, systemPrompt: promptText) + + return (adapter, ResolvedRuntimeIdentity( + role: role, + requestedProvider: requestedProvider, + requestedModel: requestedModel, + resolvedProvider: "ollama", + // The daemon's canonical name, which may carry an implicit tag the + // request omitted. Recording the request here would misreport what + // is loaded. + resolvedModel: resolvedModel, + modelDigest: digest, + locality: locality, + readiness: .ready, + promptProfile: profile.rawValue, + promptHash: PromptProfile.sha256Hex(promptText), + systemPromptSource: "PromptProfile(\(profile.rawValue))" + )) + } + + // MARK: - Failure identity + + /// The identity to report when nothing resolved. + /// + /// `resolvedProvider` / `resolvedModel` are empty rather than echoing the + /// request: nothing resolved, and echoing would let a UI that reads only + /// the resolved fields display a runtime that does not exist. + private static func unresolvedIdentity( + role: RuntimeRole, + profile: PromptProfile, + requestedProvider: String, + requestedModel: String, + locality: RuntimeLocality, + failure: RuntimeReadinessFailure + ) -> ResolvedRuntimeIdentity { + ResolvedRuntimeIdentity( + role: role, + requestedProvider: requestedProvider, + requestedModel: requestedModel, + resolvedProvider: "", + resolvedModel: "", + modelDigest: nil, + locality: locality, + readiness: .unavailable(failure), + promptProfile: profile.rawValue, + // No bytes were transmitted, so there is no transmitted-byte hash. + promptHash: "", + systemPromptSource: "unresolved" + ) } } diff --git a/Sources/BCICloudBridge/OllamaReadinessProbe.swift b/Sources/BCICloudBridge/OllamaReadinessProbe.swift new file mode 100644 index 0000000..57e630b --- /dev/null +++ b/Sources/BCICloudBridge/OllamaReadinessProbe.swift @@ -0,0 +1,107 @@ +import Foundation + +/// Answers *"can this exact model run right now?"* before the loop is enabled, +/// without generating anything. +/// +/// The gap this closes: `LiveRuntimeFactory` previously constructed an Ollama +/// runtime for whatever model name it was handed. An unpulled model resolved +/// fine and failed at the first generation — after the loop was enabled, after +/// the privacy banner claimed an active runtime, and at a point where the only +/// recovery left was an error mid-conversation. +/// +/// The probe reads `GET /api/tags`, which lists locally pulled models. It is +/// **not** a generation request: no prompt is sent, so a readiness check can +/// never itself constitute egress. +public struct OllamaReadinessProbe: Sendable { + + /// A model the daemon actually has. + public struct AvailableModel: Sendable, Equatable { + public let name: String + public let digest: String? + public init(name: String, digest: String?) { + self.name = name + self.digest = digest + } + } + + public enum ProbeOutcome: Sendable, Equatable { + /// The exact requested model is present. Carries the canonical name the + /// daemon reports, which may differ from the request by an implicit tag. + case present(AvailableModel) + /// The daemon answered, but does not have the requested model. + case modelMissing(available: [String]) + /// The daemon did not answer within the bounded timeout, or answered + /// unusably. The associated text is for the log, never for the UI. + case unreachable(detail: String) + } + + public let baseURL: URL + public let session: URLSession + /// Bounded so a hung daemon cannot stall loop startup indefinitely. + public let timeout: TimeInterval + + public init( + baseURL: URL, + session: URLSession? = nil, + timeout: TimeInterval = 3.0 + ) { + self.baseURL = baseURL + self.timeout = timeout + if let session { + self.session = session + } else { + let cfg = URLSessionConfiguration.ephemeral + cfg.timeoutIntervalForRequest = timeout + cfg.timeoutIntervalForResource = timeout + self.session = URLSession(configuration: cfg) + } + } + + /// Probes for `model`. Never throws — an infrastructure failure is a + /// *value* here, because the caller has to fold it into a readiness verdict + /// rather than propagate it as a generation error. + public func probe(model: String) async -> ProbeOutcome { + var request = URLRequest(url: baseURL.appendingPathComponent("api/tags")) + request.httpMethod = "GET" + request.timeoutInterval = timeout + + let data: Data + let response: URLResponse + do { + (data, response) = try await session.data(for: request) + } catch { + return .unreachable(detail: error.localizedDescription) + } + guard let http = response as? HTTPURLResponse else { + return .unreachable(detail: "non-HTTP response") + } + guard http.statusCode == 200 else { + return .unreachable(detail: "HTTP \(http.statusCode)") + } + guard + let object = try? JSONSerialization.jsonObject(with: data) as? [String: Any], + let entries = object["models"] as? [[String: Any]] + else { + return .unreachable(detail: "unparseable /api/tags envelope") + } + + let available: [AvailableModel] = entries.compactMap { entry in + guard let name = entry["name"] as? String else { return nil } + return AvailableModel(name: name, digest: entry["digest"] as? String) + } + + if let exact = available.first(where: { $0.name == model }) { + return .present(exact) + } + // Ollama stores an untagged pull under `:latest`, so a request for + // `qwen2.5` and a stored `qwen2.5:latest` are the same model. This is + // canonicalization, not fuzzy matching: no other tag substitutes, and + // the *stored* name is what gets recorded so the UI shows what is + // actually loaded rather than what was typed. + if !model.contains(":"), + let tagged = available.first(where: { $0.name == "\(model):latest" }) { + return .present(tagged) + } + return .modelMissing(available: available.map(\.name)) + } +} diff --git a/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift b/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift new file mode 100644 index 0000000..5ebeaa1 --- /dev/null +++ b/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift @@ -0,0 +1,256 @@ +import Foundation + +/// **A resolved runtime object is not a resolved runtime identity.** +/// +/// The runtime performs generation. The identity is an immutable, sanitized +/// *description* of what actually resolved — consumed by telemetry, readiness +/// logic, health reporting, and the privacy UI, none of which should hold a +/// live generator to answer "what is running?". +/// +/// The separation is load-bearing for two reasons. +/// +/// 1. **Failure still has an identity.** When resolution fails there is no +/// runtime object at all, but the UI must still be able to say +/// *"Requested: Ollama / qwen2.5:0.5b — model unavailable — on-device"*. +/// A description that only exists on success cannot do that. +/// 2. **Requested ≠ resolved.** Keeping both pairs makes substitution +/// *visible* rather than inferable. A UI that renders only the resolved +/// provider cannot distinguish "you asked for Ollama and got it" from +/// "you asked for Ollama and something silently handed you Claude". +public struct ResolvedRuntimeIdentity: Codable, Sendable, Equatable { + public let role: RuntimeRole + public let requestedProvider: String + public let requestedModel: String + public let resolvedProvider: String + public let resolvedModel: String + /// Provider-reported content digest, when the provider exposes one. + /// Ollama reports a digest per model; the Claude CLI does not, so this is + /// `nil` on that path rather than a fabricated value. + public let modelDigest: String? + public let locality: RuntimeLocality + public let readiness: RuntimeReadiness + public let promptProfile: String + /// sha256 of the bytes that will actually be transmitted — not of the + /// profile on disk. On the system-prompt-override path those differ, and + /// hashing the profile would attest to bytes never sent. + public let promptHash: String + public let systemPromptSource: String + + public init( + role: RuntimeRole, + requestedProvider: String, + requestedModel: String, + resolvedProvider: String, + resolvedModel: String, + modelDigest: String? = nil, + locality: RuntimeLocality, + readiness: RuntimeReadiness, + promptProfile: String, + promptHash: String, + systemPromptSource: String + ) { + self.role = role + self.requestedProvider = requestedProvider + self.requestedModel = requestedModel + self.resolvedProvider = resolvedProvider + self.resolvedModel = resolvedModel + self.modelDigest = modelDigest + self.locality = locality + self.readiness = readiness + self.promptProfile = promptProfile + self.promptHash = promptHash + self.systemPromptSource = systemPromptSource + } + + /// True when the resolved provider/model differ from what was requested. + /// The privacy UI treats this as a disclosure condition: a substitution the + /// user did not choose must never be silent. + public var isSubstitution: Bool { + resolvedProvider != requestedProvider || resolvedModel != requestedModel + } + + public var isReady: Bool { readiness == .ready } + + /// Human-readable provider name for display. Falls back to the *requested* + /// provider when nothing resolved, so a failed identity still names what + /// the user asked for instead of rendering an empty cell. + public var displayProvider: String { + Self.providerDisplayName(resolvedProvider.isEmpty ? requestedProvider : resolvedProvider) + } + + public var displayModel: String { + resolvedModel.isEmpty ? requestedModel : resolvedModel + } + + /// `Ready`, or the specific reason it is not. + public var displayReadiness: String { + switch readiness { + case .ready: return "Ready" + case .unavailable(let failure): return failure.displayLabel + } + } + + private static func providerDisplayName(_ id: String) -> String { + switch id { + case "claude": return "Claude CLI" + case "ollama": return "Ollama" + case "": return "—" + default: return id + } + } +} + +/// The role a runtime plays. Roles are *not* interchangeable even when they +/// resolve to the same provider and model, because they carry different prompts +/// — which is exactly the confusion this type exists to make impossible. +public enum RuntimeRole: String, Codable, Sendable, Equatable, CaseIterable { + /// The dialectical poles. Waking-dialectical prompt. + case dialectic + /// The non-voiced post-compete observer. Witness prompt. + case witness + /// The plain non-competing reply loop (`HypnagogicDialogueLoop`). + /// + /// Not in the original A2 sketch, which named only `dialectic` and + /// `witness`. Added because the mirror loop resolves its own runtime with a + /// *third* prompt profile, and filing it under `.dialectic` would make the + /// identity assert a role and prompt the runtime does not have. + case mirror +} + +/// Where inference actually happens. +/// +/// **A local executable does not imply local inference.** The Claude CLI is a +/// local binary that brokers to a remote service; labelling it `onDevice` +/// because the process is local is the single most consequential lie this +/// enum prevents, since it is the app's one deliberate network-egress path. +public enum RuntimeLocality: String, Codable, Sendable, Equatable, CaseIterable { + /// Inference runs on this machine. Ollama bound to a loopback address. + case onDevice + /// A local process or endpoint that forwards to a remote service. + /// The Claude CLI. Text leaves the device. + case localBrokerToRemoteService + /// A non-loopback endpoint. Ollama on another host. Text leaves the device. + case remoteEndpoint + + /// Whether text leaves the machine on this locality. The privacy UI reads + /// this rather than pattern-matching provider names. + public var involvesNetworkEgress: Bool { self != .onDevice } + + public var displayLabel: String { + switch self { + case .onDevice: return "On-device" + case .localBrokerToRemoteService: return "Local broker → remote service" + case .remoteEndpoint: return "Remote endpoint" + } + } +} + +/// Whether a runtime may be used. `unavailable` is a *pre-generation* verdict: +/// the loop must not start, and no alternate provider is attempted. +public enum RuntimeReadiness: Codable, Sendable, Equatable { + case ready + case unavailable(RuntimeReadinessFailure) + + public var failure: RuntimeReadinessFailure? { + if case .unavailable(let f) = self { return f } + return nil + } +} + +/// Why a runtime is unavailable. +/// +/// These are *infrastructure* verdicts. They are deliberately not expressible +/// as a policy outcome: a missing model is a broken configuration, not a +/// considered decision to abstain, and collapsing the two would let an +/// infrastructure failure masquerade as a reasoned result. +public enum RuntimeReadinessFailure: String, Codable, Sendable, Equatable, CaseIterable { + /// No executable named `claude` resolved. + case executableNotFound + /// An executable resolved but failed validation. + case executableInvalid + /// The prompt resource was missing, empty, or unreadable. + case promptResourceUnavailable + /// The endpoint did not answer within the bounded probe timeout. + case endpointUnreachable + /// The endpoint answered but does not have the exact requested model. + case modelMissing + /// The requested provider name is not one this factory supports. + case unknownProvider + + public var displayLabel: String { + switch self { + case .executableNotFound: return "CLI not found" + case .executableInvalid: return "CLI not usable" + case .promptResourceUnavailable: return "Prompt resource unavailable" + case .endpointUnreachable: return "Endpoint unreachable" + case .modelMissing: return "Model unavailable" + case .unknownProvider: return "Unknown provider" + } + } +} + +/// A resolution failure that still carries a displayable identity. +/// +/// `publicMessage` is safe to render in the UI and to write to `lastError`. +/// `internalDetail` is for the log only and may contain paths or transport +/// text; it is never interpolated into `publicMessage`. +/// +/// The sanitization boundary is enforced at construction rather than at the +/// display site, because a display site that has to remember to sanitize will +/// eventually forget, and the value it forgets to sanitize here is the user's +/// full environment `PATH`. +public struct RuntimeResolutionFailure: Error, Sendable, CustomStringConvertible { + public let identity: ResolvedRuntimeIdentity + public let code: RuntimeReadinessFailure + public let publicMessage: String + public let internalDetail: String? + + public init( + identity: ResolvedRuntimeIdentity, + code: RuntimeReadinessFailure, + publicMessage: String, + internalDetail: String? = nil + ) { + self.identity = identity + self.code = code + self.publicMessage = publicMessage + self.internalDetail = internalDetail + } + + /// `description` is the *public* message. `String(describing:)` is what + /// call sites reach for by reflex, so the reflex must be the safe one. + public var description: String { publicMessage } +} + +/// Redaction helpers for values that reach the UI or `lastError`. +public enum RuntimeIdentityRedaction { + + /// An endpoint reduced to scheme, host, and port. + /// + /// Drops user-info (credentials), path, query, and fragment. Ollama's base + /// URL is operator-configurable, so it can carry a token in a query string + /// or basic-auth credentials in the authority — neither belongs in a + /// privacy banner. + public static func endpoint(_ url: URL) -> String { + guard var components = URLComponents(url: url, resolvingAgainstBaseURL: false) else { + return "" + } + components.user = nil + components.password = nil + components.path = "" + components.query = nil + components.fragment = nil + return components.string ?? "" + } + + /// Whether a URL's host is a loopback address — the test that separates + /// `onDevice` from `remoteEndpoint`. + /// + /// `.local` mDNS names and LAN addresses are deliberately NOT loopback: + /// they are other machines, and inference there is not on-device. + public static func isLoopback(_ url: URL) -> Bool { + guard let host = url.host?.lowercased() else { return false } + return host == "localhost" || host == "127.0.0.1" || host == "::1" + || host == "[::1]" || host == "0.0.0.0" + } +} diff --git a/Sources/NeuralComposeApp/AppViewModel.swift b/Sources/NeuralComposeApp/AppViewModel.swift index 00b9a71..058701c 100644 --- a/Sources/NeuralComposeApp/AppViewModel.swift +++ b/Sources/NeuralComposeApp/AppViewModel.swift @@ -211,6 +211,21 @@ public final class AppViewModel: ObservableObject, AppCommandDispatchTarget { } } + /// What the dialogue runtime actually is — provider, model, locality, + /// readiness. Set on both the success and failure paths: a failed + /// resolution stores the *requested* identity with an `unavailable` + /// readiness, so the privacy UI can say "Ollama / qwen2.5:0.5b — + /// on-device — model unavailable" rather than falling back to a hardcoded + /// guess about which provider was in play. + /// + /// `nil` only before the loop has ever been enabled. + @Published public private(set) var dialogueRuntimeIdentity: ResolvedRuntimeIdentity? + + /// The Witness runtime's identity, or `nil` when the active profile + /// disables the Witness. Distinct from the dialogue identity even when both + /// resolve to the same provider and model, because the prompt differs. + @Published public private(set) var witnessRuntimeIdentity: ResolvedRuntimeIdentity? + // ── Track B (imagined speech) — additive, never touches Track A state ─ @Published public private(set) var isImaginedSpeechRecording: Bool = false @Published public private(set) var imaginedSpeechState: ImaginedSpeechProtocolState = .init( @@ -733,15 +748,34 @@ public final class AppViewModel: ObservableObject, AppCommandDispatchTarget { /// later loop iterations). private func setLastError(_ description: String) { lastError = description } + /// Cancels any in-flight loop reconciliation. + /// + /// Internal, and deliberately a *method* rather than an exposed task + /// property: the only capability tests need is "stop the pending + /// reconcile", not arbitrary mutation of it. Setting + /// `hypnagogicLoopEnabled` starts that task, which reaches the mic/speech + /// authorization gate and — when permission is absent, as in CI — flips the + /// toggle back off itself. A transition test that did not cancel it would + /// pass whether or not the fail-closed path set the toggle, which is + /// exactly the false green such a test exists to catch. + /// + /// No production caller uses this. + func cancelPendingHypnagogicReconcile() { + hypnagogicLoopReconcile?.cancel() + hypnagogicLoopReconcile = nil + } + /// The resolution result the loop needs: a generator plus the identity of /// what actually resolved. - typealias HypnagogicRuntime = (generator: any TextGenerating, resolved: LiveRuntimeFactory.Resolved) + typealias HypnagogicRuntime = (generator: any TextGenerating, identity: ResolvedRuntimeIdentity) - /// Resolves a runtime for the opt-in loop, failing closed. + /// Resolves a runtime for `role`, failing closed and recording the identity + /// either way. /// - /// `nil` means the loop must not start. On failure this disables the - /// toggle and records the typed reason; it never substitutes a different - /// provider, which is what the old + /// `nil` means the loop must not start. On failure this stores the + /// *requested* identity with an `unavailable` readiness, records a + /// **sanitized** reason, and disables the toggle. It never substitutes a + /// different provider, which is what the old /// `(try? LiveRuntimeFactory.make(...)) ?? (ClaudeCLIGenerator(...), …)` /// did — turning a mistyped `NEURALCOMPOSE_RUNTIME` into unrequested cloud /// egress. @@ -751,23 +785,60 @@ public final class AppViewModel: ObservableObject, AppCommandDispatchTarget { /// a test that flipped the toggle would return early at that gate and pass /// without ever reaching this code. func resolveHypnagogicRuntime( - _ resolve: () throws -> HypnagogicRuntime - ) -> HypnagogicRuntime? { + role: RuntimeRole, + _ resolve: () async throws -> HypnagogicRuntime + ) async -> HypnagogicRuntime? { do { - return try resolve() + let resolved = try await resolve() + store(identity: resolved.identity, for: role) + return resolved + } catch let failure as RuntimeResolutionFailure { + // The failure carries its own sanitized message. The raw error is + // NOT interpolated: `ResolutionError.notFoundOnPath` embeds the + // user's entire `PATH`, and `lastError` is rendered in the UI. + store(identity: failure.identity, for: role) + disableHypnagogicLoop( + publicReason: failure.publicMessage, + internalDetail: failure.internalDetail ?? String(describing: failure.code) + ) + return nil } catch { - disableHypnagogicLoop(reason: error) + // Safety net for an error type that predates the sanitized failure. + // Deliberately generic: an arbitrary error's description may carry + // paths or transport text, and this string reaches the UI. + store(identity: nil, for: role) + disableHypnagogicLoop( + publicReason: "The \(role.rawValue) runtime could not be resolved.", + internalDetail: String(describing: error) + ) return nil } } + /// Files an identity under the published property its role belongs to. + /// `.mirror` and `.dialectic` are both the *dialogue* runtime — they are + /// alternative dialogue shapes, not alternative roles in one exchange. + private func store(identity: ResolvedRuntimeIdentity?, for role: RuntimeRole) { + switch role { + case .dialectic, .mirror: dialogueRuntimeIdentity = identity + case .witness: witnessRuntimeIdentity = identity + } + } + + /// Clears the Witness identity when the active profile has no Witness, so + /// the UI reports "Disabled" rather than a stale identity from a previous + /// mode. + private func clearWitnessIdentity() { witnessRuntimeIdentity = nil } + /// Turns the opt-in loop off and surfaces why, when its runtime cannot be /// resolved. The alternative — substituting a default cloud generator — /// meant a packaging failure or a mistyped runtime name produced network /// egress the user never chose, so the loop now stays off. - private func disableHypnagogicLoop(reason: any Error) { - BCILog.pipeline.error("hypnagogic runtime unavailable: \(String(describing: reason))") - setLastError("Hypnagogic loop unavailable: \(reason)") + /// + /// `publicReason` is rendered; `internalDetail` is logged only. + private func disableHypnagogicLoop(publicReason: String, internalDetail: String) { + BCILog.pipeline.error("hypnagogic runtime unavailable: \(internalDetail, privacy: .private)") + setLastError("Hypnagogic loop unavailable: \(publicReason)") hypnagogicLoopEnabled = false } @@ -932,20 +1003,21 @@ public final class AppViewModel: ObservableObject, AppCommandDispatchTarget { // a typo'd NEURALCOMPOSE_RUNTIME — silently produced network // egress the user had not selected. Now the loop reports // unavailable and does not generate at all. + // NOTE: the Witness is still resolved unconditionally here, even + // when `profile.witnessEnabled` is false — a disabled role paying + // for a prompt load and (on Ollama) an endpoint probe it will never + // use. That is R3 and is fixed in the following commit; this one + // changes only how runtimes are resolved and described. guard - let dialecticResolved = resolveHypnagogicRuntime({ - try LiveRuntimeFactory.make( - systemPrompt: ClaudeCLIGenerator.wakingDialecticalSystemPrompt() - ) + let dialecticResolved = await resolveHypnagogicRuntime(role: .dialectic, { + try await LiveRuntimeFactory.make(role: .dialectic) }), - let witnessResolved = resolveHypnagogicRuntime({ - try LiveRuntimeFactory.make( - systemPrompt: ClaudeCLIGenerator.witnessSystemPrompt() - ) + let witnessResolved = await resolveHypnagogicRuntime(role: .witness, { + try await LiveRuntimeFactory.make(role: .witness) }) else { return } BCILog.pipeline.notice( - "dialectic runtime: \(dialecticResolved.resolved); witness runtime: \(witnessResolved.resolved)") + "dialectic runtime: \(dialecticResolved.identity.resolvedProvider)/\(dialecticResolved.identity.resolvedModel)") loop = HypnagogicDialecticLoop( listener: listener, generator: dialecticResolved.generator, @@ -969,9 +1041,13 @@ public final class AppViewModel: ObservableObject, AppCommandDispatchTarget { // RVS-001+1: same `LiveRuntimeFactory` resolution as the // dialectic path, so the mirror uses the selected runtime // (default: Claude). - guard let mirrorResolved = resolveHypnagogicRuntime({ - try LiveRuntimeFactory.make() + guard let mirrorResolved = await resolveHypnagogicRuntime(role: .mirror, { + try await LiveRuntimeFactory.make(role: .mirror) }) else { return } + // The mirror has no Witness; clear any identity a previous + // dialectical mode left behind so the UI cannot report a Witness + // that is not running. + clearWitnessIdentity() loop = HypnagogicDialogueLoop( listener: listener, generator: mirrorResolved.generator, diff --git a/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift b/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift index 1a52f5e..1972486 100644 --- a/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift +++ b/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift @@ -3,101 +3,328 @@ import XCTest @testable import BCICloudBridge /// `LiveRuntimeFactory` is the app's runtime-resolution seam. It must either -/// return the runtime that was requested, or throw — never substitute. +/// return the runtime that was requested *and* an identity that truthfully +/// describes it, or throw a failure that still carries a displayable identity — +/// never substitute. /// /// The defect this guards: `AppViewModel` used to write /// `(try? LiveRuntimeFactory.make(...)) ?? (ClaudeCLIGenerator(...), …)`, so a /// mistyped `NEURALCOMPOSE_RUNTIME` — set by a user who wanted local-only /// inference — silently produced cloud egress to Anthropic instead. +/// +/// **No test here makes a real provider request.** Claude resolution stops at +/// executable validation; Ollama readiness is served by `MockURLProtocol`. final class LiveRuntimeFactoryFailClosedTests: XCTestCase { - func testUnknownRuntimeThrowsAndConstructsNothing() throws { + private let ollamaURL = URL(string: "http://localhost:11434")! + + override func tearDown() { + MockURLProtocol.reset() + super.tearDown() + } + + // MARK: - Fixtures + + private func makeDirectory() throws -> URL { + let url = FileManager.default.temporaryDirectory + .appendingPathComponent("live-factory-\(UUID().uuidString)") + try FileManager.default.createDirectory(at: url, withIntermediateDirectories: true) + addTeardownBlock { try? FileManager.default.removeItem(at: url) } + return url + } + + /// A PATH containing a stub executable named `claude`. Resolution validates + /// the file and never runs it, so this is enough to reach a `ready` + /// identity without contacting Anthropic. + private func makeClaudeEnvironment() throws -> [String: String] { + let dir = try makeDirectory() + let claude = dir.appendingPathComponent("claude") + try "#!/bin/bash\nexit 0\n".write(to: claude, atomically: true, encoding: .utf8) + try FileManager.default.setAttributes( + [.posixPermissions: 0o755], ofItemAtPath: claude.path) + return ["PATH": dir.path] + } + + /// An environment whose PATH holds `/usr/bin` — hence `env`, but no + /// `claude`. Distinctive enough that a leaked PATH is detectable. + private func makeClaudelessEnvironment() throws -> [String: String] { + let marker = try makeDirectory() + return ["PATH": "/usr/bin:\(marker.path)"] + } + + private func stubTags(_ models: [(name: String, digest: String?)]) { + MockURLProtocol.handler = { request in + MockURLProtocol.tagsResponse(models, for: request) + } + } + + private func makeOllama( + role: RuntimeRole = .dialectic, + model: String, + baseURL: URL? = nil + ) async throws -> (generator: any TextGenerating, identity: ResolvedRuntimeIdentity) { + try await LiveRuntimeFactory.make( + role: role, + runtimeName: "ollama", + model: model, + ollamaBaseURL: baseURL ?? ollamaURL, + session: MockURLProtocol.makeSession(), + environment: [:] + ) + } + + // MARK: - No substitution + + func testUnknownRuntimeThrowsAndConstructsNothing() async { // `olama` is the realistic typo: the user meant local Ollama. - XCTAssertThrowsError( - try LiveRuntimeFactory.make(runtimeName: "olama", model: "qwen2.5:0.5b") - ) { error in - let message = (error as NSError).localizedDescription + do { + _ = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "olama", model: "qwen2.5:0.5b", + environment: [:]) + XCTFail("a typo'd runtime must not resolve") + } catch let failure as RuntimeResolutionFailure { + XCTAssertEqual(failure.code, .unknownProvider) XCTAssertTrue( - message.contains("olama"), - "the error must name the runtime that was actually requested: \(message)" - ) - XCTAssertTrue( - message.contains("claude") && message.contains("ollama"), - "the error should list what is supported: \(message)" - ) + failure.publicMessage.contains("olama"), + "the message must name the runtime actually requested: \(failure.publicMessage)") + XCTAssertEqual(failure.identity.requestedProvider, "olama") + XCTAssertEqual( + failure.identity.resolvedProvider, "", + "nothing resolved, so the resolved provider must be empty rather than a guess") + } catch { + XCTFail("expected RuntimeResolutionFailure, got \(error)") } } - func testUnknownRuntimeNeverYieldsAClaudeGenerator() throws { - // A throwing call returns no value at all, which is the property that - // matters: there is nothing for a caller to fall back onto. - for typo in ["olama", "Ollamaa", "gpt", "", " "] { - XCTAssertThrowsError( - try LiveRuntimeFactory.make(runtimeName: typo, model: "m"), - "runtime \(typo.debugDescription) must not resolve" - ) + func testOllamaFailureNeverYieldsAClaudeGenerator() async { + stubTags([("some-other-model", "sha256:aaa")]) + do { + _ = try await makeOllama(model: "qwen2.5:0.5b") + XCTFail("a missing model must not resolve") + } catch let failure as RuntimeResolutionFailure { + // The whole point: a local-inference request that cannot be + // satisfied fails. It does not quietly become cloud egress. + XCTAssertEqual(failure.code, .modelMissing) + XCTAssertNotEqual(failure.identity.resolvedProvider, "claude") + XCTAssertEqual(failure.identity.requestedProvider, "ollama") + } catch { + XCTFail("expected RuntimeResolutionFailure, got \(error)") } } - func testRequestedClaudeRuntimeStillResolves() throws { - let (generator, resolved) = try LiveRuntimeFactory.make( - runtimeName: "claude", - model: "claude-sonnet-5", - systemPrompt: "SYS" - ) - XCTAssertEqual(resolved.name, "claude-cli") - XCTAssertEqual(resolved.model, "claude-sonnet-5") - XCTAssertTrue(generator is ClaudeCLIGenerator) + // MARK: - R18 readiness + + /// Was `testAppPathCharacterizationDoesNotProbeOllamaModelAvailability_R18`, + /// which recorded the *absence* of probing and was written to fail once + /// probing landed. It has now been inverted, as its own doc comment + /// instructed. + func testMissingOllamaModelFailsBeforeAnyGeneration() async { + stubTags([("qwen2.5:0.5b", "sha256:aaa")]) + do { + _ = try await makeOllama(model: "definitely-not-a-pulled-model") + XCTFail("an unpulled model must not resolve") + } catch let failure as RuntimeResolutionFailure { + XCTAssertEqual(failure.code, .modelMissing) + XCTAssertEqual(failure.identity.readiness, .unavailable(.modelMissing)) + XCTAssertFalse( + failure.identity.isReady, + "an unavailable runtime must never report ready") + } catch { + XCTFail("expected RuntimeResolutionFailure, got \(error)") + } } - func testRequestedOllamaRuntimeResolvesToOllamaNotClaude() throws { - let (generator, resolved) = try LiveRuntimeFactory.make( - runtimeName: "ollama", - model: "qwen2.5:0.5b", - systemPrompt: "SYS" - ) - XCTAssertEqual(resolved.name, "ollama") - XCTAssertEqual(resolved.model, "qwen2.5:0.5b") - XCTAssertFalse( - generator is ClaudeCLIGenerator, - "a requested local runtime must never resolve to the cloud generator" - ) + /// Deleting the probe must fail a test. Asserting only on the *outcome* + /// would not catch a probe replaced by a hardcoded success, so this asserts + /// the request was actually issued. + func testReadinessActuallyQueriesTagsEndpoint() async throws { + stubTags([("qwen2.5:0.5b", "sha256:abc")]) + _ = try await makeOllama(model: "qwen2.5:0.5b") + + let probed = MockURLProtocol.requestedURLs.map(\.path) + XCTAssertTrue( + probed.contains { $0.hasSuffix("/api/tags") }, + "readiness must query /api/tags; saw \(probed)") } - /// R18 CHARACTERIZATION — records a known gap. This test does NOT - /// satisfy the "missing model fails closed" acceptance criterion and - /// must not be counted among the tests that prove it. - /// - /// The A1 brief asks for "missing Ollama model disables the loop". That is - /// true of the *harness* (`RuntimeFactory.makeOllama` probes `/api/tags` - /// and throws `ollamaModelMissing`), but **not** of the app: - /// `LiveRuntimeFactory` performs no probe, so an unpulled model resolves - /// successfully here and only fails later, at generate time. - /// - /// This test pins that current behaviour so the gap is visible in code - /// rather than only in prose. It is deliberately written to FAIL once - /// probing is added — at which point invert it to assert the throw. That - /// work belongs with R8, where truthful provider display needs the same - /// probe result. - func testAppPathCharacterizationDoesNotProbeOllamaModelAvailability_R18() throws { - let (_, resolved) = try LiveRuntimeFactory.make( - runtimeName: "ollama", - model: "definitely-not-a-pulled-model", - systemPrompt: "SYS" - ) + func testUnreachableEndpointIsUnavailableNotModelMissing() async { + MockURLProtocol.handler = { _ in throw URLError(.cannotConnectToHost) } + do { + _ = try await makeOllama(model: "qwen2.5:0.5b") + XCTFail("an unreachable daemon must not resolve") + } catch let failure as RuntimeResolutionFailure { + // Infrastructure failures are distinguishable from configuration + // failures; collapsing them would hide "the daemon is down" behind + // "pull the model". + XCTAssertEqual(failure.code, .endpointUnreachable) + } catch { + XCTFail("expected RuntimeResolutionFailure, got \(error)") + } + } + + func testResolvedOllamaIdentityRecordsCanonicalNameAndDigest() async throws { + stubTags([("qwen2.5:0.5b", "sha256:deadbeef")]) + let (_, identity) = try await makeOllama(model: "qwen2.5:0.5b") + XCTAssertEqual(identity.resolvedProvider, "ollama") + XCTAssertEqual(identity.resolvedModel, "qwen2.5:0.5b") + XCTAssertEqual(identity.modelDigest, "sha256:deadbeef") + XCTAssertEqual(identity.readiness, .ready) + } + + /// An untagged request resolves to the daemon's `:latest` entry, and the + /// identity records the *stored* name — what is loaded, not what was typed. + func testUntaggedRequestRecordsTheCanonicalStoredName() async throws { + stubTags([("qwen2.5:latest", "sha256:beef")]) + let (_, identity) = try await makeOllama(model: "qwen2.5") + XCTAssertEqual(identity.requestedModel, "qwen2.5") + XCTAssertEqual(identity.resolvedModel, "qwen2.5:latest") + XCTAssertTrue(identity.isSubstitution, "a canonicalised name is still a visible difference") + } + + // MARK: - Locality + + /// The single most consequential classification in this file. The Claude + /// CLI is a *local executable* whose inference is remote; labelling it + /// on-device because the process is local would make the privacy banner + /// assert local inference for the one path that leaves the machine. + func testClaudeCLIIsNotClassifiedAsOnDevice() async throws { + let environment = try makeClaudeEnvironment() + let (_, identity) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: "claude-sonnet-5", + environment: environment) + XCTAssertEqual(identity.locality, .localBrokerToRemoteService) + XCTAssertNotEqual(identity.locality, .onDevice) + XCTAssertTrue( + identity.locality.involvesNetworkEgress, + "the Claude CLI path must always be disclosed as egress") + } + + func testLoopbackOllamaIsOnDeviceAndNonLoopbackIsNot() async throws { + stubTags([("qwen2.5:0.5b", "sha256:abc")]) + let (_, local) = try await makeOllama(model: "qwen2.5:0.5b") + XCTAssertEqual(local.locality, .onDevice) + XCTAssertFalse(local.locality.involvesNetworkEgress) + + stubTags([("qwen2.5:0.5b", "sha256:abc")]) + let (_, remote) = try await makeOllama( + model: "qwen2.5:0.5b", baseURL: URL(string: "http://192.168.1.40:11434")!) XCTAssertEqual( - resolved.model, - "definitely-not-a-pulled-model", - "resolution currently accepts any model name; see R8 follow-up" - ) + remote.locality, .remoteEndpoint, + "another host on the LAN is not this device") + XCTAssertTrue(remote.locality.involvesNetworkEgress) } - /// The default is unchanged: absent configuration still means Claude, which - /// is the pre-existing production behaviour and not a fallback. - func testDefaultResolutionIsUnchanged() throws { - let (_, resolved) = try LiveRuntimeFactory.make( - runtimeName: nil, model: nil, systemPrompt: "SYS") - XCTAssertEqual(resolved.name, "claude-cli") - XCTAssertEqual(resolved.model, "claude-sonnet-5") + // MARK: - Role-distinct prompts (R3) + + /// Two runtimes with identical provider and model are still distinct + /// identities, because they transmit different bytes. If the Witness were + /// ever handed the pole's prompt these hashes would collide. + func testWitnessAndDialecticIdentitiesDifferByPromptEvenOnTheSameModel() async throws { + let environment = try makeClaudeEnvironment() + let (_, dialectic) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: "claude-sonnet-5", + environment: environment) + let (_, witness) = try await LiveRuntimeFactory.make( + role: .witness, runtimeName: "claude", model: "claude-sonnet-5", + environment: environment) + + XCTAssertEqual(dialectic.resolvedProvider, witness.resolvedProvider) + XCTAssertEqual(dialectic.resolvedModel, witness.resolvedModel) + + XCTAssertEqual(dialectic.promptProfile, "wakingDialectical") + XCTAssertEqual(witness.promptProfile, "witness") + XCTAssertNotEqual( + dialectic.promptHash, witness.promptHash, + "the Witness must not transmit the pole prompt") + XCTAssertNotEqual(dialectic.role, witness.role) + } + + func testEachRoleLoadsItsOwnProfile() { + XCTAssertEqual(LiveRuntimeFactory.promptProfile(for: .dialectic), .wakingDialectical) + XCTAssertEqual(LiveRuntimeFactory.promptProfile(for: .witness), .witness) + XCTAssertEqual(LiveRuntimeFactory.promptProfile(for: .mirror), .hypnagogic) + } + + /// The hash must be of the bytes actually sent, not of the profile name or + /// a constant — otherwise `promptHash` attests to nothing. + func testPromptHashMatchesTheTransmittedBytes() async throws { + let environment = try makeClaudeEnvironment() + let (_, identity) = try await LiveRuntimeFactory.make( + role: .witness, runtimeName: "claude", model: "claude-sonnet-5", + environment: environment) + let expected = PromptProfile.sha256Hex(try PromptProfile.witness.load()) + XCTAssertEqual(identity.promptHash, expected) + } + + // MARK: - Sanitization + + /// `ResolutionError.notFoundOnPath` embeds the user's entire `PATH`. The + /// factory catches it and must not forward that text into a message the UI + /// renders and `lastError` stores. + func testExecutableFailureMessageDoesNotLeakPath() async throws { + let environment = try makeClaudelessEnvironment() + let path = try XCTUnwrap(environment["PATH"]) + let marker = try XCTUnwrap(path.split(separator: ":").last).description + + do { + _ = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: "claude-sonnet-5", + environment: environment) + XCTFail("resolution must fail when no claude is on PATH") + } catch let failure as RuntimeResolutionFailure { + XCTAssertEqual(failure.code, .executableNotFound) + XCTAssertFalse( + failure.publicMessage.contains(path), + "the public message must not contain the environment PATH") + XCTAssertFalse( + failure.publicMessage.contains(marker), + "the public message must not contain any searched directory: " + + failure.publicMessage) + XCTAssertFalse( + String(describing: failure).contains(marker), + "`description` is what call sites interpolate by reflex; it must be the safe one") + // The detail is still captured — for the log, not the UI. + XCTAssertNotNil(failure.internalDetail) + } catch { + XCTFail("expected RuntimeResolutionFailure, got \(error)") + } + } + + func testEndpointMessageDropsCredentialsAndQuery() { + let url = URL(string: "http://user:secret@ollama.example:11434/api?token=abc123#frag")! + let redacted = RuntimeIdentityRedaction.endpoint(url) + for forbidden in ["secret", "token", "abc123", "frag", "/api"] { + XCTAssertFalse( + redacted.contains(forbidden), + "'\(forbidden)' must not survive redaction: \(redacted)") + } + XCTAssertTrue(redacted.contains("ollama.example")) + XCTAssertTrue(redacted.contains("11434")) + } + + func testLoopbackDetectionDoesNotTreatLANHostsAsLocal() { + for local in ["http://localhost:11434", "http://127.0.0.1:11434", "http://[::1]:11434"] { + XCTAssertTrue( + RuntimeIdentityRedaction.isLoopback(URL(string: local)!), local) + } + for remote in [ + "http://192.168.1.5:11434", "http://ollama.local:11434", "https://ollama.example.com", + ] { + XCTAssertFalse( + RuntimeIdentityRedaction.isLoopback(URL(string: remote)!), + "\(remote) is another machine") + } + } + + // MARK: - Defaults + + func testDefaultResolutionIsStillClaude() async throws { + let environment = try makeClaudeEnvironment() + let (generator, identity) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: nil, model: nil, environment: environment) + XCTAssertEqual(identity.resolvedProvider, "claude") + XCTAssertEqual(identity.resolvedModel, "claude-sonnet-5") + XCTAssertFalse(identity.isSubstitution) + XCTAssertTrue(generator is ClaudeCLIGenerator) } } diff --git a/Tests/BCICloudBridgeTests/Support/MockURLProtocol.swift b/Tests/BCICloudBridgeTests/Support/MockURLProtocol.swift new file mode 100644 index 0000000..26a63ad --- /dev/null +++ b/Tests/BCICloudBridgeTests/Support/MockURLProtocol.swift @@ -0,0 +1,89 @@ +import Foundation + +/// Intercepts `URLSession` traffic so readiness probing can be tested without +/// a running Ollama daemon. +/// +/// This exists so no test in this suite makes a real provider request. The +/// alternative — pointing tests at `localhost:11434` and hoping — makes the +/// suite pass or fail based on whether the developer happens to have `ollama +/// serve` running, which is the opposite of a regression test. +final class MockURLProtocol: URLProtocol { + typealias Handler = @Sendable (URLRequest) throws -> (HTTPURLResponse, Data) + + private static let lock = NSLock() + nonisolated(unsafe) private static var storedHandler: Handler? + nonisolated(unsafe) private static var storedRequestedURLs: [URL] = [] + + static var handler: Handler? { + get { lock.lock(); defer { lock.unlock() }; return storedHandler } + set { lock.lock(); defer { lock.unlock() }; storedHandler = newValue } + } + + /// Every URL the code under test actually requested. Used to prove a probe + /// *happened* — deleting the probe must fail a test, and a test that only + /// checks the outcome would still pass if the outcome were hardcoded. + static var requestedURLs: [URL] { + lock.lock(); defer { lock.unlock() }; return storedRequestedURLs + } + + static func reset() { + lock.lock(); defer { lock.unlock() } + storedHandler = nil + storedRequestedURLs = [] + } + + /// A session wired to this protocol. Ephemeral so nothing is cached + /// between tests. + static func makeSession() -> URLSession { + let config = URLSessionConfiguration.ephemeral + config.protocolClasses = [MockURLProtocol.self] + return URLSession(configuration: config) + } + + // MARK: - Canned responses + + /// An `/api/tags` envelope listing the given models. + static func tagsResponse( + _ models: [(name: String, digest: String?)], + for request: URLRequest, + statusCode: Int = 200 + ) -> (HTTPURLResponse, Data) { + let entries: [[String: Any]] = models.map { model in + var entry: [String: Any] = ["name": model.name] + if let digest = model.digest { entry["digest"] = digest } + return entry + } + let body = try! JSONSerialization.data(withJSONObject: ["models": entries]) + let response = HTTPURLResponse( + url: request.url!, statusCode: statusCode, + httpVersion: "HTTP/1.1", headerFields: nil)! + return (response, body) + } + + // MARK: - URLProtocol + + override class func canInit(with request: URLRequest) -> Bool { true } + override class func canonicalRequest(for request: URLRequest) -> URLRequest { request } + + override func startLoading() { + if let url = request.url { + Self.lock.lock() + Self.storedRequestedURLs.append(url) + Self.lock.unlock() + } + guard let handler = Self.handler else { + client?.urlProtocol(self, didFailWithError: URLError(.badServerResponse)) + return + } + do { + let (response, data) = try handler(request) + client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed) + client?.urlProtocol(self, didLoad: data) + client?.urlProtocolDidFinishLoading(self) + } catch { + client?.urlProtocol(self, didFailWithError: error) + } + } + + override func stopLoading() {} +} diff --git a/Tests/NeuralComposeAppTests/AppViewModelRuntimeFailClosedTests.swift b/Tests/NeuralComposeAppTests/AppViewModelRuntimeFailClosedTests.swift index 03fc81e..86be94a 100644 --- a/Tests/NeuralComposeAppTests/AppViewModelRuntimeFailClosedTests.swift +++ b/Tests/NeuralComposeAppTests/AppViewModelRuntimeFailClosedTests.swift @@ -7,29 +7,66 @@ import XCTest @testable import BCIVoice @testable import BCICloudBridge -/// R7 regression coverage. The behaviour landed in PR #29; these tests pin it. +/// R7 + R18 fail-closed coverage. /// -/// They drive `resolveHypnagogicRuntime` — the real resolution step including +/// These drive `resolveHypnagogicRuntime` — the real resolution step including /// its `catch` — rather than the disable helper, which would only prove the /// helper works when called, not that a failure reaches it. /// -/// **What these tests deliberately do NOT claim.** An earlier version asserted -/// `generatorsBuilt == 0` and `processesLaunched == 0` against counters that -/// nothing in production ever increments; they were structurally incapable of -/// failing. It also asserted `hypnagogicLoopEnabled == false` on a property -/// that defaults to `false` and was never set `true` — deleting the assignment -/// from production left the test green. Both were removed rather than left as -/// false comfort. +/// **Not covered here:** R3. The Witness is still resolved unconditionally, +/// and nothing yet rejects a runtime resolved for the wrong role. Both land +/// with the Witness fix in the following commit. /// -/// Proving the toggle *transition*, and proving the call-site `guard` -/// short-circuits, both require stubbing the mic/speech authorization gate that -/// sits in front of `ensureHypnagogicLoopRunning`. That is A2 work; until then -/// those two properties are unverified and are not claimed here. +/// **A1 left one gap here, now closed.** The enabled→disabled *transition* was +/// unprovable: `hypnagogicLoopEnabled` defaults to `false`, so an assertion +/// that it ends `false` passed even with the production assignment deleted. +/// `testFailureTransitionsAnEnabledLoopToDisabled` sets the toggle first and +/// cancels the reconcile task so the authorization gate cannot flip it instead. @MainActor final class AppViewModelRuntimeFailClosedTests: XCTestCase { - private struct StubResolutionError: Error, CustomStringConvertible { - let description = "unknown runtime 'olama' — supported: claude, ollama" + /// A sanitized failure of the kind `LiveRuntimeFactory` actually throws. + private func makeFailure( + code: RuntimeReadinessFailure = .modelMissing, + requestedProvider: String = "ollama", + requestedModel: String = "qwen2.5:0.5b", + publicMessage: String = "Ollama does not have the model 'qwen2.5:0.5b'.", + internalDetail: String? = "PATH=/very/private/path" + ) -> RuntimeResolutionFailure { + RuntimeResolutionFailure( + identity: ResolvedRuntimeIdentity( + role: .dialectic, + requestedProvider: requestedProvider, + requestedModel: requestedModel, + resolvedProvider: "", + resolvedModel: "", + locality: .onDevice, + readiness: .unavailable(code), + promptProfile: "wakingDialectical", + promptHash: "", + systemPromptSource: "unresolved" + ), + code: code, + publicMessage: publicMessage, + internalDetail: internalDetail + ) + } + + private func makeIdentity( + role: RuntimeRole, + provider: String = "ollama", + model: String = "qwen2.5:0.5b", + locality: RuntimeLocality = .onDevice + ) -> ResolvedRuntimeIdentity { + ResolvedRuntimeIdentity( + role: role, + requestedProvider: provider, requestedModel: model, + resolvedProvider: provider, resolvedModel: model, + locality: locality, readiness: .ready, + promptProfile: LiveRuntimeFactory.promptProfile(for: role).rawValue, + promptHash: "hash-\(role.rawValue)", + systemPromptSource: "PromptProfile" + ) } /// Fully stubbed container — no model, no mic, no network. @@ -49,20 +86,19 @@ final class AppViewModelRuntimeFailClosedTests: XCTestCase { return AppViewModel(container: container) } - /// A failed resolution yields no runtime. Every assertion here fails if the - /// corresponding production line is removed. + // MARK: - Fail closed + func testResolutionFailureYieldsNoRuntimeAndRecordsTheReason() async { let viewModel = await makeViewModel() var resolverCalls = 0 XCTAssertNil(viewModel.lastError, "precondition") let startupBefore = viewModel.startupWarning - let result = viewModel.resolveHypnagogicRuntime { + let result = await viewModel.resolveHypnagogicRuntime(role: .dialectic) { resolverCalls += 1 - throw StubResolutionError() + throw self.makeFailure() } - // Fails if `catch` returned anything but nil. XCTAssertNil(result, "a failed resolution must yield no runtime") XCTAssertEqual(resolverCalls, 1, "the resolver runs exactly once") @@ -70,34 +106,126 @@ final class AppViewModelRuntimeFailClosedTests: XCTestCase { let recorded = viewModel.lastError ?? "" XCTAssertTrue(recorded.contains("unavailable"), "should say unavailable: \(recorded)") XCTAssertTrue( - recorded.contains("olama"), - "the typed reason must be preserved, not flattened: \(recorded)") + recorded.contains("qwen2.5:0.5b"), + "the sanitized public reason must be preserved: \(recorded)") // Fails if a runtime failure were filed as a startup substitution notice. XCTAssertEqual(viewModel.startupWarning, startupBefore) } - /// The success path is untouched: a resolved runtime is returned verbatim - /// and records no error. - func testSuccessfulResolutionReturnsTheRequestedRuntime() async throws { + func testFailureDoesNotProduceASubstituteRuntime() async { let viewModel = await makeViewModel() - let expected = try LiveRuntimeFactory.make( - runtimeName: "claude", model: "claude-sonnet-5", systemPrompt: "SYS") + let result = await viewModel.resolveHypnagogicRuntime(role: .dialectic) { + throw self.makeFailure() + } + XCTAssertNil(result, "the old code returned a ClaudeCLIGenerator here instead of nil") + } - let resolved = try XCTUnwrap(viewModel.resolveHypnagogicRuntime { expected }) + /// Closes A1's M4 gap. The toggle is set true first, and the reconcile task + /// is cancelled so the authorization gate cannot be the thing that turns it + /// off. Deleting `hypnagogicLoopEnabled = false` from production fails this. + func testFailureTransitionsAnEnabledLoopToDisabled() async { + let viewModel = await makeViewModel() + viewModel.hypnagogicLoopEnabled = true + viewModel.cancelPendingHypnagogicReconcile() + XCTAssertTrue(viewModel.hypnagogicLoopEnabled, "precondition: the loop is enabled") + + _ = await viewModel.resolveHypnagogicRuntime(role: .dialectic) { + throw self.makeFailure() + } - XCTAssertEqual(resolved.resolved.name, "claude-cli") - XCTAssertEqual(resolved.resolved.model, "claude-sonnet-5") + XCTAssertFalse( + viewModel.hypnagogicLoopEnabled, + "a readiness failure must turn an ENABLED loop off, not merely leave it off") + } + + /// Generation must be impossible while readiness is unavailable: the caller + /// receives no generator at all, so there is nothing to call. + func testUnavailableReadinessLeavesNoGeneratorToCall() async { + let viewModel = await makeViewModel() + let result = await viewModel.resolveHypnagogicRuntime(role: .dialectic) { + throw self.makeFailure(code: .endpointUnreachable) + } + XCTAssertNil(result) + let identity = viewModel.dialogueRuntimeIdentity + XCTAssertEqual(identity?.readiness, .unavailable(.endpointUnreachable)) + XCTAssertEqual(identity?.isReady, false) + } + + // MARK: - Sanitization + + /// `lastError` is rendered in the privacy banner. The internal detail — + /// which on the Claude path carries the resolver's full `PATH` dump — must + /// not reach it. + func testInternalDetailNeverReachesTheUserFacingError() async { + let viewModel = await makeViewModel() + _ = await viewModel.resolveHypnagogicRuntime(role: .dialectic) { + throw self.makeFailure(internalDetail: "PATH=/Users/someone/secret/bin:/usr/bin") + } + let recorded = viewModel.lastError ?? "" + XCTAssertFalse(recorded.contains("PATH="), "leaked internal detail: \(recorded)") + XCTAssertFalse(recorded.contains("/Users/someone"), "leaked a private path: \(recorded)") + } + + /// An error type that is not a `RuntimeResolutionFailure` still fails + /// closed, and its raw description is still kept out of the UI. + func testUntypedErrorFailsClosedWithoutLeakingItsDescription() async { + struct Leaky: Error, CustomStringConvertible { + let description = "PATH=/Users/someone/secret/bin" + } + let viewModel = await makeViewModel() + let result = await viewModel.resolveHypnagogicRuntime(role: .dialectic) { throw Leaky() } + XCTAssertNil(result) + let recorded = viewModel.lastError ?? "" + XCTAssertFalse(recorded.contains("/Users/someone"), "leaked: \(recorded)") + } + + // MARK: - Identity storage + + func testSuccessStoresTheDialogueIdentity() async { + let viewModel = await makeViewModel() + let expected = makeIdentity(role: .dialectic) + let resolved = await viewModel.resolveHypnagogicRuntime(role: .dialectic) { + (StubGenerator(), expected) + } + XCTAssertNotNil(resolved) + XCTAssertEqual(viewModel.dialogueRuntimeIdentity, expected) XCTAssertNil(viewModel.lastError, "a successful resolution records no error") } - /// A thrown error is not swallowed into a substituted provider: the caller - /// receives nil and must decide, which is what the production `guard` does. - func testFailureDoesNotProduceASubstituteRuntime() async { + /// A failure stores the *requested* identity, so the banner can say what was + /// asked for and why it is unavailable rather than falling silent. + func testFailureStillStoresADisplayableIdentity() async { + let viewModel = await makeViewModel() + _ = await viewModel.resolveHypnagogicRuntime(role: .dialectic) { + throw self.makeFailure() + } + let identity = viewModel.dialogueRuntimeIdentity + XCTAssertEqual(identity?.requestedProvider, "ollama") + XCTAssertEqual(identity?.requestedModel, "qwen2.5:0.5b") + XCTAssertEqual(identity?.displayReadiness, "Model unavailable") + } + + func testMirrorAndDialecticBothFileUnderTheDialogueIdentity() async { let viewModel = await makeViewModel() - let result = viewModel.resolveHypnagogicRuntime { throw StubResolutionError() } - XCTAssertNil( - result, - "the old code returned a ClaudeCLIGenerator here instead of nil") + _ = await viewModel.resolveHypnagogicRuntime(role: .mirror) { + (StubGenerator(), self.makeIdentity(role: .mirror)) + } + XCTAssertEqual(viewModel.dialogueRuntimeIdentity?.role, .mirror) + XCTAssertNil(viewModel.witnessRuntimeIdentity) + } + +} + +/// Minimal `TextGenerating` that never generates. Present only so identity +/// plumbing has something to carry; calling it would be a test bug. +private struct StubGenerator: TextGenerating { + let isLive = false + let modelIdentifier = "stub" + func generate( + prompt: String, maxTokens: Int, temperature: Double, cancellationID: UUID + ) async throws -> String { + XCTFail("no test in this file may generate") + return "" } } From 3067cd8f73c57c73a9a56a80cd69d175db221b5c Mon Sep 17 00:00:00 2001 From: aurascoper Date: Sat, 25 Jul 2026 03:50:47 -0500 Subject: [PATCH 02/11] fix(dialectic): resolve Witness with a distinct prompt runtime MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Witness had no runtime of its own. In the harness it wrapped the *pole* runtime and passed the Witness prompt into the adapter's stored-but-unread `systemPrompt` field, so the Witness reported one prompt and transmitted another — the pole's. A "non-voiced observer that names what both poles avoided" was, in fact, a third pole. The dead field is now gone. Removing it is what forces the fix: with nowhere to put a prompt that does not take effect, a role-specific prompt has to be resolved into the runtime that sends it. Both the app and the harness now resolve the Witness separately, through `PromptProfile.witness`. Two runtimes can share a provider and a model and still be distinct identities. The Witness and the poles differ by role, prompt profile and prompt hash, and the tests assert those differ even when provider and model are identical. A disabled role now resolves nothing. The Witness was previously resolved unconditionally and `witnessEnabled` consulted only afterwards, when deciding whether to *use* it — so a profile with no Witness still paid for a prompt load and, on the Ollama path, an endpoint probe, and made an unused role's readiness a precondition for a loop that would never call it. `WitnessResolution` is a three-case enum rather than an optional because "the profile has no Witness" and "the Witness failed to resolve" must lead to different behaviour: the first is normal, the second fails the requested configuration closed rather than quietly degrading to a two-voice exchange the user did not select. The role-consistency guard closes a gap types cannot. `resolveWitnessRuntime` fixes the role internally, but the injected closure still returns the same tuple type whatever role it resolved, so a caller could hand the Witness a pole runtime and it would be filed under the Witness identity. A mismatch now fails closed. Mutation-tested; each fails a named test: M1 accept a pole runtime as the Witness → 2 tests M2 map the Witness role to the pole prompt → 3 M3 resolve the Witness while disabled → 1 Co-Authored-By: Claude Opus 5 --- ...nerationRuntimeTextGeneratingAdapter.swift | 38 +++---- .../BCICloudBridge/LiveRuntimeFactory.swift | 6 +- Sources/DialecticSession/main.swift | 50 ++++++---- Sources/NeuralComposeApp/AppViewModel.swift | 96 +++++++++++++++--- ...ionRuntimeTextGeneratingAdapterTests.swift | 12 +-- .../AppViewModelRuntimeFailClosedTests.swift | 98 +++++++++++++++++-- 6 files changed, 218 insertions(+), 82 deletions(-) diff --git a/Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift b/Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift index c83fcd6..bd45c52 100644 --- a/Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift +++ b/Sources/BCICloudBridge/GenerationRuntimeTextGeneratingAdapter.swift @@ -1,11 +1,9 @@ import BCICore import Foundation -/// A `TextGenerating` adapter that wraps any `GenerationRuntime` -/// (and its fixed system prompt) so the existing -/// `HypnagogicDialecticLoop` — which depends on `TextGenerating`, -/// not `GenerationRuntime` — can use any of the new runtimes -/// without a loop refactor. +/// A `TextGenerating` adapter that wraps any `GenerationRuntime` so the +/// existing `HypnagogicDialecticLoop` — which depends on `TextGenerating`, not +/// `GenerationRuntime` — can use any of the runtimes without a loop refactor. /// /// This is the *smallest* change to wire runtime selection into /// the harness: the loop's interface stays exactly the same, and @@ -16,28 +14,20 @@ import Foundation /// text; it only shapes a `GenerationContext` and calls /// `runtime.generate(prompt:context:)`). /// -/// The adapter is `Sendable` because both the wrapped runtime -/// (which is `Sendable`) and the `systemPrompt` (a `String`) are -/// safe to pass across actor boundaries; the actor isolation on -/// the loop side is unchanged. +/// **This adapter does not carry a system prompt.** It once had a +/// `systemPrompt` field that was stored and never read: the bytes actually +/// transmitted come from the wrapped runtime's own prompt. That made the field +/// worse than useless — it let a caller construct a "Witness" adapter around +/// the *pole* runtime, pass the Witness prompt into the dead field, and get an +/// object that reported one prompt while sending another. Removing it forces +/// role-specific prompts to be resolved where they take effect: in the runtime. +/// +/// The adapter is `Sendable` because the wrapped runtime is; the actor +/// isolation on the loop side is unchanged. public struct GenerationRuntimeTextGeneratingAdapter: MetadataPublishingTextGenerating { public nonisolated let isLive: Bool public nonisolated let modelIdentifier: String - /// **LEGACY / DEAD — scheduled for removal with the R3 Witness fix.** - /// - /// Stored and never read. The bytes actually transmitted come from the - /// wrapped runtime's own prompt, so this field describes nothing: a caller - /// can construct a "Witness" adapter around the *pole* runtime, pass the - /// Witness prompt here, and get an object that reports one prompt while - /// sending another. `Sources/DialecticSession/main.swift` does exactly - /// that today. - /// - /// It is retained for this commit only so the identity/readiness layer can - /// land without also rewriting Witness resolution. Do not read it, and do - /// not treat it as provenance. - public nonisolated let systemPrompt: String - private let runtime: any GenerationRuntime private let maxTokens: Int private let defaultTemperature: Double @@ -72,12 +62,10 @@ public struct GenerationRuntimeTextGeneratingAdapter: MetadataPublishingTextGene public init( runtime: any GenerationRuntime, - systemPrompt: String, maxTokens: Int = 256, defaultTemperature: Double = 0.7 ) { self.runtime = runtime - self.systemPrompt = systemPrompt self.isLive = runtime.isLive self.modelIdentifier = runtime.modelIdentifier self.maxTokens = maxTokens diff --git a/Sources/BCICloudBridge/LiveRuntimeFactory.swift b/Sources/BCICloudBridge/LiveRuntimeFactory.swift index b32341f..0b73ed2 100644 --- a/Sources/BCICloudBridge/LiveRuntimeFactory.swift +++ b/Sources/BCICloudBridge/LiveRuntimeFactory.swift @@ -291,11 +291,7 @@ public enum LiveRuntimeFactory { detail: String(describing: error) ) } - // `systemPrompt:` here feeds the adapter's dead legacy field; the bytes - // that actually go out come from `runtime`. Both the field and this - // argument are removed with the R3 Witness fix. - let adapter = GenerationRuntimeTextGeneratingAdapter( - runtime: runtime, systemPrompt: promptText) + let adapter = GenerationRuntimeTextGeneratingAdapter(runtime: runtime) return (adapter, ResolvedRuntimeIdentity( role: role, diff --git a/Sources/DialecticSession/main.swift b/Sources/DialecticSession/main.swift index fdc9ad8..fe24064 100644 --- a/Sources/DialecticSession/main.swift +++ b/Sources/DialecticSession/main.swift @@ -120,25 +120,37 @@ do { // the wiring on the next `generate(...)` call. (See the long // comment in `attachMetadataCaptureFromAdapter` for the // existential-box pattern.) -let poleGenerator = GenerationRuntimeTextGeneratingAdapter( - runtime: resolved.runtime, - systemPrompt: resolved.systemPrompt -) -// NOTE: `GenerationRuntimeTextGeneratingAdapter.systemPrompt` is currently a -// stored-but-unread field — the bytes actually sent come from -// `resolved.runtime`'s own prompt. Giving the Witness its own resolved runtime -// is a separate fix (tracked as R3) and is deliberately not attempted here. -// What changes now is only that a failed load stops the run instead of -// silently passing "". -let witnessPrompt: String? -do { - witnessPrompt = profile.witnessEnabled ? try PromptProfile.witness.load() : nil -} catch { - FileHandle.standardError.write(Data("● dialectic-session: \(error)\n".utf8)) - exit(1) -} -let witnessGenerator: GenerationRuntimeTextGeneratingAdapter? = witnessPrompt.map { - GenerationRuntimeTextGeneratingAdapter(runtime: resolved.runtime, systemPrompt: $0) +let poleGenerator = GenerationRuntimeTextGeneratingAdapter(runtime: resolved.runtime) + +// R3: the Witness gets its OWN resolved runtime, loaded with the Witness +// profile. It previously wrapped the *pole* runtime and passed the Witness +// prompt into the adapter's stored-but-unread `systemPrompt` field, so the +// Witness reported one prompt and transmitted another — the pole's. The field +// is gone; the prompt now lives in the runtime that sends it. +// +// Resolution is skipped entirely when the profile disables the Witness. A +// disabled role must not load a prompt, resolve a runtime, or probe an +// endpoint: doing so makes readiness depend on a component the configuration +// says is not in use. +let witnessGenerator: GenerationRuntimeTextGeneratingAdapter? +if profile.witnessEnabled { + let witnessResolved: ResolvedRuntime + do { + witnessResolved = try RuntimeFactory.make( + runtimeName: opts.runtime, + model: opts.model, + promptProfile: .witness + ) + } catch { + // The Witness is part of the requested configuration, so its failure + // fails the run closed rather than quietly degrading to a two-voice + // exchange the operator did not ask for. + FileHandle.standardError.write(Data("● dialectic-session: witness runtime: \(error)\n".utf8)) + exit(1) + } + witnessGenerator = GenerationRuntimeTextGeneratingAdapter(runtime: witnessResolved.runtime) +} else { + witnessGenerator = nil } print("● dialectic-session — profile=\(profile.rawValue) witness=\(profile.witnessEnabled) " diff --git a/Sources/NeuralComposeApp/AppViewModel.swift b/Sources/NeuralComposeApp/AppViewModel.swift index 058701c..aa8340c 100644 --- a/Sources/NeuralComposeApp/AppViewModel.swift +++ b/Sources/NeuralComposeApp/AppViewModel.swift @@ -790,6 +790,25 @@ public final class AppViewModel: ObservableObject, AppCommandDispatchTarget { ) async -> HypnagogicRuntime? { do { let resolved = try await resolve() + // The resolver must return a runtime for the role that was asked + // for. Without this, a call site could hand the Witness closure a + // `.dialectic` runtime and the result would be filed under the + // Witness identity — a Witness transmitting the pole prompt, which + // is exactly the confusion the role/prompt binding exists to + // prevent. Type-checking cannot catch it because both sides are the + // same tuple type. + guard resolved.identity.role == role else { + store(identity: nil, for: role) + disableHypnagogicLoop( + publicReason: "The \(role.rawValue) runtime resolved to the wrong role " + + "(\(resolved.identity.role.rawValue)).", + internalDetail: + "role mismatch: requested \(role.rawValue), " + + "resolved \(resolved.identity.role.rawValue) " + + "with profile \(resolved.identity.promptProfile)" + ) + return nil + } store(identity: resolved.identity, for: role) return resolved } catch let failure as RuntimeResolutionFailure { @@ -815,6 +834,42 @@ public final class AppViewModel: ObservableObject, AppCommandDispatchTarget { } } + /// The outcome of asking for a Witness. Three states, not two: "the + /// profile has no Witness" and "the Witness failed to resolve" must lead to + /// different behaviour, and an optional cannot say which is which. + enum WitnessResolution { + /// The active profile has no Witness. Not a failure. + case disabled + case resolved(any TextGenerating) + /// Requested but unavailable. The caller fails the loop closed. + case failed + } + + /// Resolves the Witness **only when the profile enables it**. + /// + /// A disabled role must not load a prompt, resolve a runtime, or probe an + /// endpoint. Resolving unconditionally — which is what this did — made an + /// unused role's readiness a precondition for a loop that would never call + /// it, and on the Ollama path spent a network probe on a runtime the + /// configuration says is not in use. + /// + /// Internal and closure-injected for the same reason as + /// `resolveHypnagogicRuntime`: the production call site sits behind the + /// mic/speech authorization gate. + func resolveWitnessRuntime( + witnessEnabled: Bool, + _ resolve: () async throws -> HypnagogicRuntime + ) async -> WitnessResolution { + guard witnessEnabled else { + clearWitnessIdentity() + return .disabled + } + guard let resolved = await resolveHypnagogicRuntime(role: .witness, resolve) else { + return .failed + } + return .resolved(resolved.generator) + } + /// Files an identity under the published property its role belongs to. /// `.mirror` and `.dialectic` are both the *dialogue* runtime — they are /// alternative dialogue shapes, not alternative roles in one exchange. @@ -1003,19 +1058,30 @@ public final class AppViewModel: ObservableObject, AppCommandDispatchTarget { // a typo'd NEURALCOMPOSE_RUNTIME — silently produced network // egress the user had not selected. Now the loop reports // unavailable and does not generate at all. - // NOTE: the Witness is still resolved unconditionally here, even - // when `profile.witnessEnabled` is false — a disabled role paying - // for a prompt load and (on Ollama) an endpoint probe it will never - // use. That is R3 and is fixed in the following commit; this one - // changes only how runtimes are resolved and described. - guard - let dialecticResolved = await resolveHypnagogicRuntime(role: .dialectic, { - try await LiveRuntimeFactory.make(role: .dialectic) - }), - let witnessResolved = await resolveHypnagogicRuntime(role: .witness, { - try await LiveRuntimeFactory.make(role: .witness) - }) - else { return } + guard let dialecticResolved = await resolveHypnagogicRuntime(role: .dialectic, { + try await LiveRuntimeFactory.make(role: .dialectic) + }) else { return } + + // R3: the Witness is resolved separately, with the Witness prompt, + // and ONLY when the profile enables it. Resolving it unconditionally + // — as this did — made an unused role's readiness a precondition for + // a loop that would never call it, and paid for a prompt load and + // (on Ollama) an endpoint probe that the configuration says are not + // in use. + let witnessGenerator: (any TextGenerating)? + switch await resolveWitnessRuntime(witnessEnabled: profile.witnessEnabled, { + try await LiveRuntimeFactory.make(role: .witness) + }) { + case .disabled: + witnessGenerator = nil + case .resolved(let generator): + witnessGenerator = generator + case .failed: + // The Witness is part of the requested configuration, so its + // failure fails the loop closed rather than silently running a + // two-voice exchange the user did not select. + return + } BCILog.pipeline.notice( "dialectic runtime: \(dialecticResolved.identity.resolvedProvider)/\(dialecticResolved.identity.resolvedModel)") loop = HypnagogicDialecticLoop( @@ -1031,9 +1097,7 @@ public final class AppViewModel: ObservableObject, AppCommandDispatchTarget { // a non-voiced third generator that names what both poles avoided // (WITNESS.md). This is the third cloud call; Focused/Contemplative // leave it nil. - witness: profile.witnessEnabled - ? witnessResolved.generator - : nil, + witness: witnessGenerator, config: profile.loopConfig() ) } else { diff --git a/Tests/BCICloudBridgeTests/GenerationRuntimeTextGeneratingAdapterTests.swift b/Tests/BCICloudBridgeTests/GenerationRuntimeTextGeneratingAdapterTests.swift index 15f0af9..2ee3a1f 100644 --- a/Tests/BCICloudBridgeTests/GenerationRuntimeTextGeneratingAdapterTests.swift +++ b/Tests/BCICloudBridgeTests/GenerationRuntimeTextGeneratingAdapterTests.swift @@ -76,7 +76,6 @@ final class GenerationRuntimeTextGeneratingAdapterTests: XCTestCase { let runtime = StubRuntime(result: Self.stubResult) let adapter = GenerationRuntimeTextGeneratingAdapter( runtime: runtime, - systemPrompt: "system", maxTokens: 256, defaultTemperature: 0.7 ) @@ -93,7 +92,6 @@ final class GenerationRuntimeTextGeneratingAdapterTests: XCTestCase { let runtime = StubRuntime(result: Self.stubResult) var adapter = GenerationRuntimeTextGeneratingAdapter( runtime: runtime, - systemPrompt: "system", maxTokens: 256, defaultTemperature: 0.7 ) @@ -117,7 +115,6 @@ final class GenerationRuntimeTextGeneratingAdapterTests: XCTestCase { let runtime = StubRuntime(result: Self.stubResult) var adapter = GenerationRuntimeTextGeneratingAdapter( runtime: runtime, - systemPrompt: "system", maxTokens: 256, defaultTemperature: 0.7 ) @@ -144,8 +141,7 @@ final class GenerationRuntimeTextGeneratingAdapterTests: XCTestCase { // silently. Pin the conformance with a compile-time check. let runtime = StubRuntime(result: Self.stubResult) let adapter = GenerationRuntimeTextGeneratingAdapter( - runtime: runtime, - systemPrompt: "system" + runtime: runtime ) let publisher: MetadataPublishingTextGenerating = adapter XCTAssertNotNil(publisher as AnyObject) @@ -158,8 +154,7 @@ final class GenerationRuntimeTextGeneratingAdapterTests: XCTestCase { // crash on the nil callback. let runtime = StubRuntime(result: Self.stubResult) let adapter = GenerationRuntimeTextGeneratingAdapter( - runtime: runtime, - systemPrompt: "system" + runtime: runtime ) XCTAssertNil(adapter.onMetadata) let text = try await adapter.generate( @@ -186,8 +181,7 @@ final class GenerationRuntimeTextGeneratingAdapterTests: XCTestCase { func testOnMetadataPropagatesAcrossExistentialCopy() async throws { let runtime = StubRuntime(result: Self.stubResult) let adapter = GenerationRuntimeTextGeneratingAdapter( - runtime: runtime, - systemPrompt: "system" + runtime: runtime ) // The dialectic loop stores the generator as diff --git a/Tests/NeuralComposeAppTests/AppViewModelRuntimeFailClosedTests.swift b/Tests/NeuralComposeAppTests/AppViewModelRuntimeFailClosedTests.swift index 86be94a..9f98bc8 100644 --- a/Tests/NeuralComposeAppTests/AppViewModelRuntimeFailClosedTests.swift +++ b/Tests/NeuralComposeAppTests/AppViewModelRuntimeFailClosedTests.swift @@ -7,15 +7,12 @@ import XCTest @testable import BCIVoice @testable import BCICloudBridge -/// R7 + R18 fail-closed coverage. +/// R7 + R18 fail-closed coverage, and R3's "a disabled role resolves nothing". /// -/// These drive `resolveHypnagogicRuntime` — the real resolution step including -/// its `catch` — rather than the disable helper, which would only prove the -/// helper works when called, not that a failure reaches it. -/// -/// **Not covered here:** R3. The Witness is still resolved unconditionally, -/// and nothing yet rejects a runtime resolved for the wrong role. Both land -/// with the Witness fix in the following commit. +/// These drive `resolveHypnagogicRuntime` / `resolveWitnessRuntime` — the real +/// resolution steps including their `catch` — rather than the disable helper, +/// which would only prove the helper works when called, not that a failure +/// reaches it. /// /// **A1 left one gap here, now closed.** The enabled→disabled *transition* was /// unprovable: `hypnagogicLoopEnabled` defaults to `false`, so an assertion @@ -215,6 +212,91 @@ final class AppViewModelRuntimeFailClosedTests: XCTestCase { XCTAssertNil(viewModel.witnessRuntimeIdentity) } + // MARK: - R3: a disabled Witness resolves nothing + + /// The exact regression: the Witness was resolved unconditionally and the + /// `witnessEnabled` flag was consulted only afterwards, when deciding + /// whether to *use* it. A disabled role must not load a prompt, construct a + /// runtime, or probe an endpoint. + func testDisabledWitnessIsNeverResolved() async { + let viewModel = await makeViewModel() + var resolverCalls = 0 + + let outcome = await viewModel.resolveWitnessRuntime(witnessEnabled: false) { + resolverCalls += 1 + return (StubGenerator(), self.makeIdentity(role: .witness)) + } + + guard case .disabled = outcome else { + return XCTFail("a disabled Witness must report .disabled, got \(outcome)") + } + XCTAssertEqual( + resolverCalls, 0, + "a disabled Witness must not resolve a runtime, load a prompt, or probe an endpoint") + XCTAssertNil(viewModel.witnessRuntimeIdentity) + XCTAssertNil(viewModel.lastError, "a disabled role is not an error") + } + + func testEnabledWitnessResolvesAndStoresItsOwnIdentity() async { + let viewModel = await makeViewModel() + let witnessIdentity = makeIdentity(role: .witness) + let outcome = await viewModel.resolveWitnessRuntime(witnessEnabled: true) { + (StubGenerator(), witnessIdentity) + } + guard case .resolved = outcome else { + return XCTFail("expected .resolved, got \(outcome)") + } + XCTAssertEqual(viewModel.witnessRuntimeIdentity, witnessIdentity) + XCTAssertEqual(viewModel.witnessRuntimeIdentity?.promptProfile, "witness") + XCTAssertNil( + viewModel.dialogueRuntimeIdentity, + "the Witness must not overwrite the dialogue identity") + } + + /// Handing the Witness a *pole* runtime must fail closed rather than be + /// filed under the Witness identity. Both sides are the same tuple type, so + /// nothing but this check catches it. + func testWitnessResolvedToThePoleRuntimeIsRefused() async { + let viewModel = await makeViewModel() + let poleIdentity = makeIdentity(role: .dialectic) + + let outcome = await viewModel.resolveWitnessRuntime(witnessEnabled: true) { + (StubGenerator(), poleIdentity) // the pole's runtime, not the Witness's + } + + guard case .failed = outcome else { + return XCTFail("a pole runtime must not be accepted as the Witness, got \(outcome)") + } + XCTAssertNil( + viewModel.witnessRuntimeIdentity, + "a rejected runtime must not be stored as the Witness identity") + XCTAssertFalse(viewModel.hypnagogicLoopEnabled) + } + + func testDialogueResolvedToTheWitnessRuntimeIsRefused() async { + let viewModel = await makeViewModel() + let result = await viewModel.resolveHypnagogicRuntime(role: .dialectic) { + (StubGenerator(), self.makeIdentity(role: .witness)) + } + XCTAssertNil(result, "a Witness runtime must not stand in for the dialectic poles") + XCTAssertNil(viewModel.dialogueRuntimeIdentity) + } + + /// A requested Witness that cannot resolve fails the configuration closed + /// rather than degrading to a two-voice exchange the user did not select. + func testRequestedWitnessFailureIsNotSilentlyDowngraded() async { + let viewModel = await makeViewModel() + let outcome = await viewModel.resolveWitnessRuntime(witnessEnabled: true) { + throw self.makeFailure(code: .promptResourceUnavailable) + } + guard case .failed = outcome else { + return XCTFail("expected .failed, got \(outcome)") + } + XCTAssertFalse(viewModel.hypnagogicLoopEnabled) + XCTAssertEqual( + viewModel.witnessRuntimeIdentity?.readiness, + .unavailable(.promptResourceUnavailable)) + } } /// Minimal `TextGenerating` that never generates. Present only so identity From 14c8b6a98df4b6a570ec17da84191fcfc94d3ae1 Mon Sep 17 00:00:00 2001 From: aurascoper Date: Sat, 25 Jul 2026 03:52:08 -0500 Subject: [PATCH 03/11] fix(ui): display actual runtime provider and readiness MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The privacy banner asserted "the claude CLI" and "Listening + Cloud" whenever the loop was enabled. Both had been wrong since runtime selection landed: with NEURALCOMPOSE_RUNTIME=ollama the loop ran entirely on-device while the banner reported cloud egress to Anthropic. A privacy indicator that is wrong in the reassuring direction is worse than none; this one was wrong in both directions at different times, which is the same defect. Both strings were literals inside a SwiftUI `body`, so the app's most consequential privacy claim was the one part of the codebase no test could reach. `RuntimeIdentityPresentation` moves the claims into a plain value type computed from `ResolvedRuntimeIdentity`, and the view renders it. Every field derives from a resolved identity; none derives from the mode name, the provider spelling, or a default. Egress is read from `locality`, never from a provider name — the Claude CLI is a local executable whose inference is remote, and Ollama on a non-loopback host is remote despite being "local" in every other sense. Neither is derivable from a label. Unknown counts as egress. Before the first resolution there is no identity to read, and the banner must not claim on-device operation it cannot substantiate, so the default is the alarming one rather than the reassuring one. A runtime that is not ready renders its failure rather than its name, and in red, so an unavailable runtime cannot read as running. The Witness renders "Disabled" when the profile has none — distinct from a Witness that failed. Mutation-tested: M7 hardcode the cloud badge regardless of locality → 1 test "Listening + Cloud" still exists as one branch of a locality conditional; what is gone is its unconditional assertion. Co-Authored-By: Claude Opus 5 --- Sources/NeuralComposeApp/ContentView.swift | 2 + .../PrivacyIndicatorView.swift | 67 +++++++-- .../RuntimeIdentityPresentation.swift | 85 ++++++++++++ .../RuntimeIdentityPresentationTests.swift | 128 ++++++++++++++++++ 4 files changed, 270 insertions(+), 12 deletions(-) create mode 100644 Sources/NeuralComposeApp/RuntimeIdentityPresentation.swift create mode 100644 Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift diff --git a/Sources/NeuralComposeApp/ContentView.swift b/Sources/NeuralComposeApp/ContentView.swift index 3d247d2..82e290a 100644 --- a/Sources/NeuralComposeApp/ContentView.swift +++ b/Sources/NeuralComposeApp/ContentView.swift @@ -55,6 +55,8 @@ struct ContentView: View { spokenGenerationLoopEnabled: $viewModel.spokenGenerationLoopEnabled, hypnagogicLoopEnabled: $viewModel.hypnagogicLoopEnabled, hypnagogicMode: $viewModel.hypnagogicMode, + dialogueRuntimeIdentity: viewModel.dialogueRuntimeIdentity, + witnessRuntimeIdentity: viewModel.witnessRuntimeIdentity, health: viewModel.healthSnapshot ) Divider() diff --git a/Sources/NeuralComposeApp/PrivacyIndicatorView.swift b/Sources/NeuralComposeApp/PrivacyIndicatorView.swift index d7ee701..5d2eaab 100644 --- a/Sources/NeuralComposeApp/PrivacyIndicatorView.swift +++ b/Sources/NeuralComposeApp/PrivacyIndicatorView.swift @@ -1,5 +1,6 @@ import SwiftUI import BCICore +import BCICloudBridge /// Always-visible banner telling the user what's actually running. Green /// when every stage is real; amber when any stage is a stand-in; red when a @@ -57,6 +58,17 @@ struct PrivacyIndicatorView: View { /// `focused`/`reflective`/`contemplative` (the dialectic competition, two /// cloud calls/turn). @Binding var hypnagogicMode: HypnagogicMode + /// What the dialogue and Witness runtimes **actually are**. + /// + /// The banner renders these instead of naming a provider, because naming + /// one means asserting it. This view used to say "the claude CLI" and + /// "Listening + Cloud" unconditionally, which had been wrong since runtime + /// selection landed: with `NEURALCOMPOSE_RUNTIME=ollama` the app ran + /// entirely on-device while the privacy banner reported cloud egress. + /// A privacy indicator that is wrong in the reassuring direction is worse + /// than none; this one was wrong in both directions at different times. + var dialogueRuntimeIdentity: ResolvedRuntimeIdentity? = nil + var witnessRuntimeIdentity: ResolvedRuntimeIdentity? = nil /// Latest app-watchdog snapshot for the degraded badge (read-only display). var health: HealthSnapshot? = nil @@ -124,11 +136,17 @@ struct PrivacyIndicatorView: View { GridRow { Text("Network").bold() if hypnagogicLoopEnabled { - VStack(alignment: .leading, spacing: 2) { - Text("Cloud active — Hypnagogic (\(hypnagogicMode.label))").foregroundStyle(.red) - Text(hypnagogicMode.isDialectical - ? "Transcript text (never audio) may be sent to the claude CLI — \(hypnagogicMode.profile?.witnessEnabled == true ? "three" : "two") calls per turn in the dialectical modes\(hypnagogicMode.profile?.witnessEnabled == true ? " (Reflective adds a non-voiced Witness)" : ""). Opt-in exception — see decision_registry entry 8." - : "Transcript text (never audio) may be sent to the claude CLI. Opt-in exception — see decision_registry entry 8.") + let presentation = runtimePresentation + VStack(alignment: .leading, spacing: 3) { + Text("\(presentation.headline) — Hypnagogic (\(hypnagogicMode.label))") + .foregroundStyle(runtimeEgressColor) + Text(presentation.dialogueLine) + .font(.caption2) + .foregroundStyle(identityColor(dialogueRuntimeIdentity)) + Text(presentation.witnessLine) + .font(.caption2) + .foregroundStyle(identityColor(witnessRuntimeIdentity)) + Text(presentation.caption) .font(.caption2) .foregroundStyle(.secondary) } @@ -440,19 +458,44 @@ struct PrivacyIndicatorView: View { } } - /// The most consequential badge: while active the mic is capturing AND - /// transcript text may be leaving the machine (cloud LLM). Explicit - /// "+ Cloud" + a network icon so the runtime egress is never invisible — - /// this is the one place the app breaks its "no network at runtime" default. + // MARK: - Runtime identity rendering (R8) + + /// Every runtime claim this banner makes, derived from the resolved + /// identities. Computed in `RuntimeIdentityPresentation` so it is testable + /// — the strings it replaces lived inside `body`, where nothing could + /// assert on them. + private var runtimePresentation: RuntimeIdentityPresentation { + RuntimeIdentityPresentation( + dialogue: dialogueRuntimeIdentity, + witness: witnessRuntimeIdentity, + isDialectical: hypnagogicMode.isDialectical + ) + } + + private var runtimeEgressColor: Color { + runtimePresentation.involvesEgress ? .red : .orange + } + + /// A runtime that is not ready must not read as running. + private func identityColor(_ identity: ResolvedRuntimeIdentity?) -> Color { + guard let identity else { return .secondary } + return identity.isReady ? .secondary : .red + } + + /// The most consequential badge: while active the mic is always capturing, + /// and transcript text *may* be leaving the machine. Which of those is true + /// is read from the resolved locality rather than assumed — this badge said + /// "+ Cloud" unconditionally even when the whole loop ran on Ollama. @ViewBuilder private var hypnagogicLoopBadge: some View { if hypnagogicLoopEnabled { - Label("Listening + Cloud", systemImage: "antenna.radiowaves.left.and.right") + let presentation = runtimePresentation + Label(presentation.badgeLabel, systemImage: presentation.badgeSystemImage) .font(.caption) - .foregroundStyle(.red) + .foregroundStyle(runtimeEgressColor) .padding(.horizontal, 8) .padding(.vertical, 3) - .background(Color.red.opacity(0.18)) + .background(runtimeEgressColor.opacity(0.18)) .cornerRadius(4) } } diff --git a/Sources/NeuralComposeApp/RuntimeIdentityPresentation.swift b/Sources/NeuralComposeApp/RuntimeIdentityPresentation.swift new file mode 100644 index 0000000..c412f45 --- /dev/null +++ b/Sources/NeuralComposeApp/RuntimeIdentityPresentation.swift @@ -0,0 +1,85 @@ +import BCICloudBridge +import Foundation + +/// What the privacy banner says about the runtimes, as data. +/// +/// Extracted from `PrivacyIndicatorView` so the claims are testable. The +/// strings this replaces — `"the claude CLI"` and `"Listening + Cloud"` — were +/// literals inside a SwiftUI `body`, which meant the app's most consequential +/// privacy assertion was the one part of the codebase no test could reach. +/// They had also been wrong since runtime selection landed: with +/// `NEURALCOMPOSE_RUNTIME=ollama` the loop ran entirely on-device while the +/// banner reported cloud egress to Anthropic. +/// +/// Every field here is derived from `ResolvedRuntimeIdentity`. Nothing is +/// derived from the mode name, the provider spelling, or a default. +struct RuntimeIdentityPresentation: Equatable { + let involvesEgress: Bool + let headline: String + let dialogueLine: String + let witnessLine: String + let caption: String + let badgeLabel: String + let badgeSystemImage: String + + /// - Parameters: + /// - dialogue: the dialogue runtime's identity, or `nil` before the first + /// resolution attempt. + /// - witness: the Witness identity, or `nil` when the active profile has + /// no Witness. + /// - isDialectical: whether the active mode runs the competing poles. + /// Used **only** for the per-turn call count, never for provider or + /// locality. + init( + dialogue: ResolvedRuntimeIdentity?, + witness: ResolvedRuntimeIdentity?, + isDialectical: Bool + ) { + let active = [dialogue, witness].compactMap { $0 } + + // Unknown counts as egress. Before the first resolution there is no + // identity to read, and a privacy banner must not claim on-device + // operation it cannot substantiate. The safe default is the alarming + // one. + let egress = active.isEmpty || active.contains { $0.locality.involvesNetworkEgress } + self.involvesEgress = egress + + self.headline = active.isEmpty + ? "Resolving runtime" + : (egress ? "Network egress active" : "On-device only") + + self.dialogueLine = Self.line("Dialogue", dialogue, absent: "Resolving…") + self.witnessLine = Self.line("Witness", witness, absent: "Disabled") + + if active.isEmpty { + self.caption = + "The runtime has not resolved yet — treated as potential egress until it does." + } else { + let calls: String = isDialectical + ? (witness != nil + ? "three calls per turn (the third is the non-voiced Witness)" + : "two calls per turn") + : "one call per turn" + self.caption = egress + ? "Transcript text (never audio) may leave this device — \(calls). " + + "Opt-in exception — see decision_registry entry 8." + : "Transcript text (never audio) stays on this device — \(calls) to a local runtime." + } + + self.badgeLabel = egress ? "Listening + Cloud" : "Listening · On-device" + self.badgeSystemImage = egress + ? "antenna.radiowaves.left.and.right" + : "desktopcomputer" + } + + /// `role: provider · model · locality · readiness`. + private static func line( + _ title: String, + _ identity: ResolvedRuntimeIdentity?, + absent: String + ) -> String { + guard let identity else { return "\(title): \(absent)" } + return "\(title): \(identity.displayProvider) · \(identity.displayModel) · " + + "\(identity.locality.displayLabel) · \(identity.displayReadiness)" + } +} diff --git a/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift b/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift new file mode 100644 index 0000000..4844d76 --- /dev/null +++ b/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift @@ -0,0 +1,128 @@ +import XCTest +@testable import NeuralComposeApp +@testable import BCICloudBridge + +/// R8 — the privacy banner must describe what actually resolved. +/// +/// The strings under test were literals inside `PrivacyIndicatorView.body`: +/// `"the claude CLI"` and `"Listening + Cloud"`, emitted whenever the loop was +/// enabled. They had been wrong since runtime selection landed — with +/// `NEURALCOMPOSE_RUNTIME=ollama` the loop ran entirely on-device while the +/// banner reported cloud egress to Anthropic — and being inside a SwiftUI +/// `body` meant no test could reach them. +final class RuntimeIdentityPresentationTests: XCTestCase { + + private func identity( + role: RuntimeRole = .dialectic, + provider: String = "ollama", + model: String = "qwen2.5:0.5b", + locality: RuntimeLocality = .onDevice, + readiness: RuntimeReadiness = .ready + ) -> ResolvedRuntimeIdentity { + ResolvedRuntimeIdentity( + role: role, + requestedProvider: provider, requestedModel: model, + resolvedProvider: readiness == .ready ? provider : "", + resolvedModel: readiness == .ready ? model : "", + locality: locality, readiness: readiness, + promptProfile: "p", promptHash: "h", systemPromptSource: "s" + ) + } + + private func allText(_ p: RuntimeIdentityPresentation) -> String { + [p.headline, p.dialogueLine, p.witnessLine, p.caption, p.badgeLabel] + .joined(separator: " ") + } + + // MARK: - No hardcoded provider + + /// The headline mutation: hardcoding Claude in the banner must fail here. + func testOnDeviceOllamaIsNeverDescribedAsClaudeOrCloud() { + let p = RuntimeIdentityPresentation( + dialogue: identity(), witness: nil, isDialectical: false) + let text = allText(p).lowercased() + + XCTAssertFalse(text.contains("claude"), "an Ollama runtime must not be called Claude: \(text)") + XCTAssertFalse(text.contains("cloud"), "on-device inference must not be called cloud: \(text)") + XCTAssertTrue(text.contains("ollama")) + XCTAssertFalse(p.involvesEgress) + XCTAssertEqual(p.badgeLabel, "Listening · On-device") + } + + func testClaudeIsDescribedAsEgressAndNamedAsClaude() { + let p = RuntimeIdentityPresentation( + dialogue: identity( + provider: "claude", model: "claude-sonnet-5", + locality: .localBrokerToRemoteService), + witness: nil, isDialectical: false) + XCTAssertTrue(p.involvesEgress) + XCTAssertEqual(p.badgeLabel, "Listening + Cloud") + XCTAssertTrue(p.dialogueLine.contains("Claude CLI")) + XCTAssertTrue(p.caption.contains("may leave this device")) + } + + /// Locality drives the claim, not the provider name. Ollama on another host + /// is egress even though "Ollama" reads as local everywhere else. + func testNonLoopbackOllamaIsDisclosedAsEgress() { + let p = RuntimeIdentityPresentation( + dialogue: identity(locality: .remoteEndpoint), witness: nil, isDialectical: false) + XCTAssertTrue(p.involvesEgress, "another host is not this device") + XCTAssertEqual(p.badgeLabel, "Listening + Cloud") + } + + /// A mixed configuration discloses egress: one remote runtime is enough. + func testAnyEgressRuntimeMakesTheWholeBannerDiscloseEgress() { + let p = RuntimeIdentityPresentation( + dialogue: identity(locality: .onDevice), + witness: identity( + role: .witness, provider: "claude", model: "claude-sonnet-5", + locality: .localBrokerToRemoteService), + isDialectical: true) + XCTAssertTrue(p.involvesEgress) + } + + // MARK: - Unknown is not reassuring + + /// Before the first resolution there is nothing to read. The banner must + /// not claim on-device operation it cannot substantiate. + func testUnresolvedRuntimeDefaultsToTheAlarmingClaim() { + let p = RuntimeIdentityPresentation(dialogue: nil, witness: nil, isDialectical: false) + XCTAssertTrue(p.involvesEgress, "unknown must not read as safe") + XCTAssertEqual(p.headline, "Resolving runtime") + XCTAssertTrue(p.caption.contains("potential egress")) + } + + // MARK: - Readiness + + func testUnavailableRuntimeShowsItsFailureNotReady() { + let p = RuntimeIdentityPresentation( + dialogue: identity(readiness: .unavailable(.modelMissing)), + witness: nil, isDialectical: false) + XCTAssertTrue(p.dialogueLine.contains("Model unavailable"), p.dialogueLine) + XCTAssertFalse(p.dialogueLine.contains("Ready"), p.dialogueLine) + // Still names what was requested, so the user can act on it. + XCTAssertTrue(p.dialogueLine.contains("qwen2.5:0.5b"), p.dialogueLine) + } + + // MARK: - Witness + + func testAbsentWitnessReadsAsDisabledNotAsAFailure() { + let p = RuntimeIdentityPresentation( + dialogue: identity(), witness: nil, isDialectical: true) + XCTAssertEqual(p.witnessLine, "Witness: Disabled") + XCTAssertTrue(p.caption.contains("two calls per turn"), p.caption) + } + + func testPresentWitnessIsCountedAsAThirdCall() { + let p = RuntimeIdentityPresentation( + dialogue: identity(), witness: identity(role: .witness), isDialectical: true) + XCTAssertTrue(p.caption.contains("three calls per turn"), p.caption) + XCTAssertTrue(p.witnessLine.contains("Ollama"), p.witnessLine) + } + + func testMirrorModeReportsOneCallPerTurn() { + let p = RuntimeIdentityPresentation( + dialogue: identity(), witness: nil, isDialectical: false) + XCTAssertTrue(p.caption.contains("one call per turn"), p.caption) + } +} From d0d5dd77da1938644f010b79a98780f4ba93f072 Mon Sep 17 00:00:00 2001 From: aurascoper Date: Sun, 26 Jul 2026 01:11:31 -0500 Subject: [PATCH 04/11] fix(runtime): distinguish unresolved locality and canonical model identity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two identity fixes, both cases of the identity asserting something it does not know. RuntimeLocality gains a fourth case, .unresolved. The unknown-provider path used to classify itself .localBrokerToRemoteService with a comment admitting "there is nothing to classify" — conservative about egress, but semantically a claim of a known topology for a provider nothing knows anything about. Unknown is not on-device, and it is not a known remote broker either: .unresolved keeps involvesNetworkEgress == true (the banner must not claim safety it cannot substantiate) while displaying "Egress unverified" (the banner must not claim knowledge it does not have). This matches the honest conservative state the Android implementation already uses. isSubstitution now compares models canonically. OllamaReadinessProbe deliberately accepts a stored `name:latest` for an untagged request — the daemon stores an untagged pull under :latest, so they are the same model — and the identity records the stored name. Comparing raw strings meant one resolution was simultaneously accepted as the same model by readiness and reported as a provider/model substitution by identity. canonicalModelName maps an untagged name to name:latest (empty stays empty, so the failure-path identity fabricates no phantom tag); any difference beyond that one implicit tag — an explicit non-latest tag, a provider swap — still discloses. Regression coverage: - unknown provider → .unresolved, egress-conservative, "Egress unverified", never rendered as On-device by the presentation - qwen2.5 → qwen2.5:latest is no longer a substitution alarm (the previous test pinned the buggy disagreement and has been inverted) - qwen2.5 → qwen2.5:0.5b and ollama → claude still are Co-Authored-By: Claude Fable 5 --- .../BCICloudBridge/LiveRuntimeFactory.swift | 8 +++- .../ResolvedRuntimeIdentity.swift | 32 ++++++++++++- .../LiveRuntimeFactoryFailClosedTests.swift | 46 ++++++++++++++++++- .../RuntimeIdentityPresentationTests.swift | 14 ++++++ 4 files changed, 95 insertions(+), 5 deletions(-) diff --git a/Sources/BCICloudBridge/LiveRuntimeFactory.swift b/Sources/BCICloudBridge/LiveRuntimeFactory.swift index 0b73ed2..fcfa902 100644 --- a/Sources/BCICloudBridge/LiveRuntimeFactory.swift +++ b/Sources/BCICloudBridge/LiveRuntimeFactory.swift @@ -89,8 +89,12 @@ public enum LiveRuntimeFactory { requestedProvider: requestedProvider, requestedModel: requestedModel, // An unknown provider has no endpoint, so there is nothing - // to classify. Reporting a locality here would be a guess. - locality: .localBrokerToRemoteService, + // to classify — and `.localBrokerToRemoteService`, which + // this used to report, *was* a guess: it asserted a known + // egress topology for a provider nothing knows anything + // about. `.unresolved` keeps the conservative egress + // presentation without the false claim. + locality: .unresolved, failure: .unknownProvider ), code: .unknownProvider, diff --git a/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift b/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift index 5ebeaa1..4b13077 100644 --- a/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift +++ b/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift @@ -65,8 +65,25 @@ public struct ResolvedRuntimeIdentity: Codable, Sendable, Equatable { /// True when the resolved provider/model differ from what was requested. /// The privacy UI treats this as a disclosure condition: a substitution the /// user did not choose must never be silent. + /// + /// Models are compared *canonically*: `OllamaReadinessProbe` deliberately + /// accepts a stored `name:latest` for an untagged request — the daemon + /// stores an untagged pull under `:latest`, so they are the same model — + /// and the identity records the stored name. Comparing the raw strings + /// here made the same resolution simultaneously "accepted as the same + /// model by readiness" and "reported as a substitution by identity". A + /// canonical `:latest` resolution is not a substitution alarm; any other + /// difference still is. public var isSubstitution: Bool { - resolvedProvider != requestedProvider || resolvedModel != requestedModel + resolvedProvider != requestedProvider + || Self.canonicalModelName(resolvedModel) != Self.canonicalModelName(requestedModel) + } + + /// An untagged model name canonicalizes to `name:latest`, mirroring the + /// probe's one allowed equivalence. Empty stays empty (the failure-path + /// identity, where nothing resolved) so no phantom tag is fabricated. + static func canonicalModelName(_ name: String) -> String { + name.isEmpty || name.contains(":") ? name : name + ":latest" } public var isReady: Bool { readiness == .ready } @@ -131,9 +148,19 @@ public enum RuntimeLocality: String, Codable, Sendable, Equatable, CaseIterable case localBrokerToRemoteService /// A non-loopback endpoint. Ollama on another host. Text leaves the device. case remoteEndpoint + /// No locality could be established — an unsupported provider name, so + /// there is no endpoint or executable to classify. Distinct from the three + /// known localities because an unknown provider is *not known* to be a + /// local broker (the previous classification), any more than it is known + /// to be on-device: unknown ≠ on-device, unknown ≠ known remote broker, + /// unknown = egress unverified. Egress is still assumed (the safe + /// direction) while the label says it is unverified (the honest one). + case unresolved /// Whether text leaves the machine on this locality. The privacy UI reads - /// this rather than pattern-matching provider names. + /// this rather than pattern-matching provider names. `unresolved` counts + /// as egress: a banner must not claim on-device operation it cannot + /// substantiate. public var involvesNetworkEgress: Bool { self != .onDevice } public var displayLabel: String { @@ -141,6 +168,7 @@ public enum RuntimeLocality: String, Codable, Sendable, Equatable, CaseIterable case .onDevice: return "On-device" case .localBrokerToRemoteService: return "Local broker → remote service" case .remoteEndpoint: return "Remote endpoint" + case .unresolved: return "Egress unverified" } } } diff --git a/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift b/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift index 1972486..c6af436 100644 --- a/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift +++ b/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift @@ -91,6 +91,15 @@ final class LiveRuntimeFactoryFailClosedTests: XCTestCase { XCTAssertEqual( failure.identity.resolvedProvider, "", "nothing resolved, so the resolved provider must be empty rather than a guess") + // An unknown provider has no endpoint to classify, so its locality + // is a fourth honest state — not a claim of any known topology. + XCTAssertEqual( + failure.identity.locality, .unresolved, + "an unknown provider is not known to be a local broker") + XCTAssertTrue( + failure.identity.locality.involvesNetworkEgress, + "unverified egress must still be presented conservatively as egress") + XCTAssertEqual(failure.identity.locality.displayLabel, "Egress unverified") } catch { XCTFail("expected RuntimeResolutionFailure, got \(error)") } @@ -173,12 +182,47 @@ final class LiveRuntimeFactoryFailClosedTests: XCTestCase { /// An untagged request resolves to the daemon's `:latest` entry, and the /// identity records the *stored* name — what is loaded, not what was typed. + /// + /// The identity must agree with the probe about what "the same model" + /// means: the probe deliberately accepts `qwen2.5:latest` for `qwen2.5`, + /// so `isSubstitution` comparing the raw strings made one resolution both + /// "the same model" (readiness) and "a substitution" (identity). A + /// canonical `:latest` resolution is not a substitution alarm. func testUntaggedRequestRecordsTheCanonicalStoredName() async throws { stubTags([("qwen2.5:latest", "sha256:beef")]) let (_, identity) = try await makeOllama(model: "qwen2.5") XCTAssertEqual(identity.requestedModel, "qwen2.5") XCTAssertEqual(identity.resolvedModel, "qwen2.5:latest") - XCTAssertTrue(identity.isSubstitution, "a canonicalised name is still a visible difference") + XCTAssertFalse( + identity.isSubstitution, + "the probe's canonicalization and the identity's comparison must agree") + } + + /// The canonicalization must not widen into fuzzy matching: any resolved + /// model that differs by more than the implicit `:latest` tag is still a + /// substitution the UI discloses. + func testNonLatestModelDifferenceIsStillASubstitution() { + let base = ResolvedRuntimeIdentity( + role: .dialectic, + requestedProvider: "ollama", requestedModel: "qwen2.5", + resolvedProvider: "ollama", resolvedModel: "qwen2.5:0.5b", + locality: .onDevice, readiness: .ready, + promptProfile: "wakingDialectical", promptHash: "abc", + systemPromptSource: "test") + XCTAssertTrue( + base.isSubstitution, + "an explicit non-latest tag is a different model, not a canonical spelling") + + let provider = ResolvedRuntimeIdentity( + role: .dialectic, + requestedProvider: "ollama", requestedModel: "qwen2.5", + resolvedProvider: "claude", resolvedModel: "qwen2.5", + locality: .localBrokerToRemoteService, readiness: .ready, + promptProfile: "wakingDialectical", promptHash: "abc", + systemPromptSource: "test") + XCTAssertTrue( + provider.isSubstitution, + "a provider swap is always a substitution, whatever the model strings say") } // MARK: - Locality diff --git a/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift b/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift index 4844d76..adf6291 100644 --- a/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift +++ b/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift @@ -92,6 +92,20 @@ final class RuntimeIdentityPresentationTests: XCTestCase { XCTAssertTrue(p.caption.contains("potential egress")) } + /// An identity whose locality could not be established (unknown provider) + /// discloses egress and says the classification is unverified — never + /// "On-device", never a named remote topology it cannot substantiate. + func testUnresolvedLocalityDisclosesEgressAsUnverified() { + let p = RuntimeIdentityPresentation( + dialogue: identity( + provider: "olama", locality: .unresolved, + readiness: .unavailable(.unknownProvider)), + witness: nil, isDialectical: false) + XCTAssertTrue(p.involvesEgress, "unverified egress must read as egress") + XCTAssertTrue(p.dialogueLine.contains("Egress unverified"), p.dialogueLine) + XCTAssertFalse(p.dialogueLine.contains("On-device"), p.dialogueLine) + } + // MARK: - Readiness func testUnavailableRuntimeShowsItsFailureNotReady() { From db0143282f0dae13973dabb19449a30effc276fe Mon Sep 17 00:00:00 2001 From: aurascoper Date: Sun, 26 Jul 2026 01:19:08 -0500 Subject: [PATCH 05/11] fix(telemetry): persist Witness runtime fingerprint MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DialecticalTurnEvent carried one generatorFingerprint, documented as the generator that produced the candidates. A persisted Reflective turn could therefore prove that a Witness ran and what it found, but not which provider, model, prompt profile, or prompt hash produced the finding — and the bug A2 fixed was precisely "reported Witness prompt ≠ prompt actually transmitted". The wiring is proven statically and in focused tests; the turn record could not attest it independently. DialecticalCompetition and DialecticalTurnEvent gain an optional witnessGeneratorFingerprint. The loop gives the Witness its own metadata box and callback (recordWitnessMetadata), parallel to the pole capture and never shared with it — filing both in one box would make the last writer win, a turn record attesting the pole prompt for the Witness. attachMetadataCaptureFromAdapter now wires both generators, so the app (AppViewModel) and the headless harness (dialectic-session), which both already call it, start persisting Witness provenance with no call-site change. The fingerprint is gated on witnessAttempted: a turn on which the Witness never ran must not attest a Witness identity, however recent the box contents. Decoding stays backward-compatible in both directions: pre-field logs decode with nil (missing Optional key), and a nil fingerprint encodes as an absent key, byte-shape-identical to the pre-field format. analyze_dialectic.py's provenance rollup now reports witness-fingerprint counts and models, plus a witness/pole prompt-hash collision count — the collision is the old bug's signature and must read 0. Tests: pole and Witness fingerprints captured separately with distinct prompt hashes on the same logged event; no Witness attestation when the profile disables it; codec round-trip of both fingerprints; a pole-fingerprint-only log decodes with the Witness field nil. Co-Authored-By: Claude Fable 5 --- Scripts/analyze_dialectic.py | 53 ++++++++ .../Composition/HypnagogicDialecticLoop.swift | 43 ++++++- .../Dialectic/DialecticalCompetition.swift | 13 +- .../Telemetry/DialecticalTurnEvent.swift | 9 ++ .../GeneratorFingerprintTests.swift | 43 ++++++- .../HypnagogicDialecticLoopTests.swift | 119 ++++++++++++++++++ 6 files changed, 271 insertions(+), 9 deletions(-) diff --git a/Scripts/analyze_dialectic.py b/Scripts/analyze_dialectic.py index 0dd3905..5e66c62 100755 --- a/Scripts/analyze_dialectic.py +++ b/Scripts/analyze_dialectic.py @@ -285,6 +285,12 @@ class Turn: candidates: list[dict] has_fingerprint: bool fingerprint_model: str | None # model name from generatorFingerprint, if any + fingerprint_prompt_hash: str | None + # Witness provenance — separate from the pole fingerprint, which only + # attests the candidates' generator. nil on pre-Witness-fingerprint logs. + has_witness_fingerprint: bool + witness_fingerprint_model: str | None + witness_fingerprint_prompt_hash: str | None def __post_init__(self) -> None: # If outcome is somehow None or empty, fall back to a sentinel @@ -330,6 +336,22 @@ def load_turns(path: Path) -> list[Turn]: if d.get("generatorFingerprint") is not None else None ), + fingerprint_prompt_hash=( + d.get("generatorFingerprint", {}).get("promptHash") + if d.get("generatorFingerprint") is not None + else None + ), + has_witness_fingerprint=d.get("witnessGeneratorFingerprint") is not None, + witness_fingerprint_model=( + d.get("witnessGeneratorFingerprint", {}).get("model") + if d.get("witnessGeneratorFingerprint") is not None + else None + ), + witness_fingerprint_prompt_hash=( + d.get("witnessGeneratorFingerprint", {}).get("promptHash") + if d.get("witnessGeneratorFingerprint") is not None + else None + ), )) rows.sort(key=lambda t: t.index) return rows @@ -1322,6 +1344,13 @@ class ProvenanceReport: turns_with_fingerprint: int fingerprint_rate: float fingerprint_models: dict[str, int] # model name -> count + # Witness provenance. A Reflective run recorded after the Witness + # fingerprint landed should have one per witness-attempted turn; the + # prompt-hash collision count is the old bug's signature (a "Witness" + # transmitting the pole prompt) and must be 0. + turns_with_witness_fingerprint: int + witness_fingerprint_models: dict[str, int] + witness_pole_prompt_hash_collisions: int note: str @@ -1331,11 +1360,25 @@ def compute_provenance(turns: list[Turn]) -> ProvenanceReport: for t in turns: if t.has_fingerprint and t.fingerprint_model: models[t.fingerprint_model] += 1 + wfp = sum(1 for t in turns if t.has_witness_fingerprint) + witness_models: collections.Counter = collections.Counter() + collisions = 0 + for t in turns: + if not t.has_witness_fingerprint: + continue + if t.witness_fingerprint_model: + witness_models[t.witness_fingerprint_model] += 1 + if (t.witness_fingerprint_prompt_hash is not None + and t.witness_fingerprint_prompt_hash == t.fingerprint_prompt_hash): + collisions += 1 return ProvenanceReport( total_turns=len(turns), turns_with_fingerprint=fp, fingerprint_rate=(fp / len(turns)) if turns else 0.0, fingerprint_models=dict(models), + turns_with_witness_fingerprint=wfp, + witness_fingerprint_models=dict(witness_models), + witness_pole_prompt_hash_collisions=collisions, note=("0/140 is the expected baseline pre-b9c09fd-live-app. " "The core runtime records fingerprints (commit b9c09fd); " "the live app will start recording them once the " @@ -1575,6 +1618,16 @@ def render_report(rep: dict[str, Any]) -> str: out.append(f" fingerprint models:") for m, c in sorted(p['fingerprint_models'].items(), key=lambda x: -x[1]): out.append(f" {m}: {c}") + wfp = p.get('turns_with_witness_fingerprint', 0) + out.append(f" turns with witness fp: {wfp} / {p['total_turns']} ({100*wfp/p['total_turns']:.1f}%)") + if p.get('witness_fingerprint_models'): + out.append(f" witness fingerprint models:") + for m, c in sorted(p['witness_fingerprint_models'].items(), key=lambda x: -x[1]): + out.append(f" {m}: {c}") + collisions = p.get('witness_pole_prompt_hash_collisions', 0) + if wfp: + verdict = "OK (witness prompt ≠ pole prompt)" if collisions == 0 else "BUG SIGNATURE — witness transmitted the pole prompt" + out.append(f" witness/pole hash collisions: {collisions} {verdict}") if p.get('note'): out.append(f" note: {p['note']}") out.append("") diff --git a/Sources/BCICore/Composition/HypnagogicDialecticLoop.swift b/Sources/BCICore/Composition/HypnagogicDialecticLoop.swift index 8eacf9e..612c48e 100644 --- a/Sources/BCICore/Composition/HypnagogicDialecticLoop.swift +++ b/Sources/BCICore/Composition/HypnagogicDialecticLoop.swift @@ -98,6 +98,13 @@ public actor HypnagogicDialecticLoop { /// property, losing the metadata for that turn. The class box /// keeps the read/write synchronous. private let latestMetadata = MetadataBox() + /// The Witness's own metadata box, parallel to `latestMetadata` and never + /// shared with it: `latestMetadata` is documented as the generator that + /// produced the *candidates*, and filing the Witness's metadata there + /// would make the last writer win — a turn record attesting the pole + /// prompt for the Witness, which is the exact confusion the Witness + /// fingerprint exists to make impossible. + private let latestWitnessMetadata = MetadataBox() private let speaker: any SpeechSynthesizing private let embedder: any SentenceEmbedder private let roles: [DialecticalRole] @@ -184,6 +191,13 @@ public actor HypnagogicDialecticLoop { latestMetadata.set(metadata) } + /// The Witness-side sibling of `recordGeneratorMetadata`, writing to the + /// Witness's own box. Same synchrony argument, same `nonisolated` + /// rationale; only the destination differs. + public nonisolated func recordWitnessMetadata(_ metadata: GenerationMetadata) { + latestWitnessMetadata.set(metadata) + } + /// Convenience: if the passed-in `generator` is a /// `MetadataPublishingTextGenerating` (today: any runtime wrapped /// by `GenerationRuntimeTextGeneratingAdapter` in `BCICloudBridge`; @@ -206,11 +220,20 @@ public actor HypnagogicDialecticLoop { /// is global. This pattern is the standard fix for "set a /// callback on a struct-typed existential." public func attachMetadataCaptureFromAdapter() { - guard var publisher = generator as? MetadataPublishingTextGenerating else { - return + if var publisher = generator as? any MetadataPublishingTextGenerating { + publisher.onMetadata = { [weak self] metadata in + self?.recordGeneratorMetadata(metadata) + } } - publisher.onMetadata = { [weak self] metadata in - self?.recordGeneratorMetadata(metadata) + // The Witness gets its own callback into its own box. This relies on + // the Witness being a DISTINCT adapter instance from the pole + // generator — which the factory guarantees, since each role resolves + // its own runtime; wiring one shared instance here would overwrite + // the pole callback (each adapter holds a single callback box). + if var publisher = witness as? any MetadataPublishingTextGenerating { + publisher.onMetadata = { [weak self] metadata in + self?.recordWitnessMetadata(metadata) + } } } @@ -399,7 +422,12 @@ public actor HypnagogicDialecticLoop { outcome: result.outcome, glossScalar: gloss.value, spectralState: state, witnessFinding: witnessFinding, witnessDistance: witnessDistance, selfSimilarity: selfSimilarity, witnessAttempted: witnessAttempted, - generatorFingerprint: latestMetadata.get().flatMap(GeneratorFingerprint.init) + generatorFingerprint: latestMetadata.get().flatMap(GeneratorFingerprint.init), + // Gated on the attempt: a turn on which the Witness never ran must + // not attest a Witness identity, however recent the box contents. + witnessGeneratorFingerprint: witnessAttempted + ? latestWitnessMetadata.get().flatMap(GeneratorFingerprint.init) + : nil ) await turnLogger.log(DialecticalTurnEvent(record)) @@ -457,7 +485,10 @@ public actor HypnagogicDialecticLoop { witnessDistance: nil, selfSimilarity: nil, witnessAttempted: config.witnessEnabled && witness != nil, - generatorFingerprint: latestMetadata.get().flatMap(GeneratorFingerprint.init) + generatorFingerprint: latestMetadata.get().flatMap(GeneratorFingerprint.init), + // A candidate-less turn never reaches the Witness call, so there + // is no Witness generation to attest — regardless of the box. + witnessGeneratorFingerprint: nil ) let event = DialecticalTurnEvent(competition) await turnLogger.log(event) diff --git a/Sources/BCICore/Dialectic/DialecticalCompetition.swift b/Sources/BCICore/Dialectic/DialecticalCompetition.swift index b2f1095..bc179a6 100644 --- a/Sources/BCICore/Dialectic/DialecticalCompetition.swift +++ b/Sources/BCICore/Dialectic/DialecticalCompetition.swift @@ -170,13 +170,23 @@ public struct DialecticalCompetition: Sendable, Equatable { /// decode; `Codable` treats missing keys as `nil` for `Optional` /// fields. See `Sources/BCICore/Telemetry/GeneratorFingerprint.swift`. public let generatorFingerprint: GeneratorFingerprint? + /// The generator identity of the **Witness** this turn — separate from + /// `generatorFingerprint`, which is documented as the generator that + /// produced the *candidates*. The bug class this closes: the Witness once + /// reported one prompt while transmitting another, and a persisted turn + /// carrying only the pole fingerprint cannot independently attest which + /// runtime/prompt produced the finding. nil when the Witness did not + /// generate this turn (disabled profile, or a turn it never ran on) or + /// when its generator publishes no metadata. + public let witnessGeneratorFingerprint: GeneratorFingerprint? public init(index: Int, heard: String, scored: [ScoredCandidate], tension: Float, margin: Float, selectionTemperature: Float, outcome: DialecticalOutcome, glossScalar: Float, spectralState: SpectralState? = nil, witnessFinding: String? = nil, witnessDistance: Float? = nil, selfSimilarity: Float? = nil, witnessAttempted: Bool? = nil, - generatorFingerprint: GeneratorFingerprint? = nil) { + generatorFingerprint: GeneratorFingerprint? = nil, + witnessGeneratorFingerprint: GeneratorFingerprint? = nil) { self.index = index self.heard = heard self.scored = scored @@ -191,5 +201,6 @@ public struct DialecticalCompetition: Sendable, Equatable { self.selfSimilarity = selfSimilarity self.witnessAttempted = witnessAttempted self.generatorFingerprint = generatorFingerprint + self.witnessGeneratorFingerprint = witnessGeneratorFingerprint } } diff --git a/Sources/BCICore/Telemetry/DialecticalTurnEvent.swift b/Sources/BCICore/Telemetry/DialecticalTurnEvent.swift index b8b5297..14021b0 100644 --- a/Sources/BCICore/Telemetry/DialecticalTurnEvent.swift +++ b/Sources/BCICore/Telemetry/DialecticalTurnEvent.swift @@ -54,6 +54,14 @@ public struct DialecticalTurnEvent: Codable, Sendable, Equatable { /// interactionStyle / promptHash). Optional so pre-fingerprint /// logs still decode. See `Sources/BCICore/Telemetry/GeneratorFingerprint.swift`. public let generatorFingerprint: GeneratorFingerprint? + /// The Witness generator's identity this turn — persisted separately + /// because `generatorFingerprint` attests only to the candidates' + /// generator, and the Witness bug being guarded against was precisely + /// "reported Witness prompt ≠ prompt actually transmitted". With this + /// field a persisted Reflective turn can independently prove which + /// provider/model/prompt produced the finding. Optional so every + /// pre-Witness-fingerprint log still decodes (missing key → nil). + public let witnessGeneratorFingerprint: GeneratorFingerprint? public init(_ c: DialecticalCompetition) { self.index = c.index @@ -73,6 +81,7 @@ public struct DialecticalTurnEvent: Codable, Sendable, Equatable { self.selfSimilarity = c.selfSimilarity self.witnessAttempted = c.witnessAttempted self.generatorFingerprint = c.generatorFingerprint + self.witnessGeneratorFingerprint = c.witnessGeneratorFingerprint switch c.outcome { case let .spoke(cand): self.outcome = "spoke:\(cand.roleID)" diff --git a/Tests/BCICoreTests/GeneratorFingerprintTests.swift b/Tests/BCICoreTests/GeneratorFingerprintTests.swift index 93076a0..dbee3d2 100644 --- a/Tests/BCICoreTests/GeneratorFingerprintTests.swift +++ b/Tests/BCICoreTests/GeneratorFingerprintTests.swift @@ -25,7 +25,8 @@ final class GeneratorFingerprintTests: XCTestCase { DialecticalCandidate(text: "\(role)-text", embedding: emb([1, 0]), roleID: role) } - private func competition(fingerprint: GeneratorFingerprint?) -> DialecticalCompetition { + private func competition(fingerprint: GeneratorFingerprint?, + witnessFingerprint: GeneratorFingerprint? = nil) -> DialecticalCompetition { DialecticalCompetition( index: 0, heard: "hi", @@ -47,7 +48,8 @@ final class GeneratorFingerprintTests: XCTestCase { outcome: .spoke(candidate("coherence")), glossScalar: 0.5, spectralState: nil, - generatorFingerprint: fingerprint + generatorFingerprint: fingerprint, + witnessGeneratorFingerprint: witnessFingerprint ) } @@ -109,6 +111,43 @@ final class GeneratorFingerprintTests: XCTestCase { XCTAssertEqual(decoded.generatorFingerprint, claudeFingerprint) } + /// The Witness fingerprint persists *separately* from the pole + /// fingerprint: a decoded Reflective turn must be able to prove which + /// runtime/prompt produced the finding, and the two prompt hashes must + /// survive the round-trip distinct (equal hashes are the signature of + /// the Witness-transmitted-the-pole-prompt bug). + func testWitnessFingerprintRoundTripsSeparatelyFromPoleFingerprint() throws { + let witnessFP = GeneratorFingerprint( + runtime: "ollama", transport: "ollama-http", provider: "ollama", + model: "qwen2.5:0.5b", promptProfile: "witness", + interactionStyle: "dialectical", promptHash: "witness-hash-999") + let event = DialecticalTurnEvent( + competition(fingerprint: ollamaFingerprint, witnessFingerprint: witnessFP)) + let data = try JSONEncoder().encode(event) + let decoded = try JSONDecoder().decode(DialecticalTurnEvent.self, from: data) + XCTAssertEqual(decoded.generatorFingerprint, ollamaFingerprint) + XCTAssertEqual(decoded.witnessGeneratorFingerprint, witnessFP) + XCTAssertNotEqual( + decoded.generatorFingerprint?.promptHash, + decoded.witnessGeneratorFingerprint?.promptHash) + } + + /// A log written between the pole fingerprint (b9c09fd) and the Witness + /// fingerprint landing has `generatorFingerprint` but no + /// `witnessGeneratorFingerprint` key at all — it must decode with the + /// Witness field `nil`, not throw. + func testLogWithOnlyPoleFingerprintDecodesWitnessAsNil() throws { + let event = DialecticalTurnEvent(competition(fingerprint: ollamaFingerprint)) + let data = try JSONEncoder().encode(event) + let json = try XCTUnwrap(String(data: data, encoding: .utf8)) + XCTAssertFalse( + json.contains("witnessGeneratorFingerprint"), + "a nil Witness fingerprint must be an absent key, matching the pre-field format") + let decoded = try JSONDecoder().decode(DialecticalTurnEvent.self, from: data) + XCTAssertEqual(decoded.generatorFingerprint, ollamaFingerprint) + XCTAssertNil(decoded.witnessGeneratorFingerprint) + } + // MARK: - Backward compatibility (item 10 of the smoke checklist) func testOldLogWithoutFingerprintDecodesAsNil() throws { diff --git a/Tests/BCICoreTests/HypnagogicDialecticLoopTests.swift b/Tests/BCICoreTests/HypnagogicDialecticLoopTests.swift index a71443f..54eee5d 100644 --- a/Tests/BCICoreTests/HypnagogicDialecticLoopTests.swift +++ b/Tests/BCICoreTests/HypnagogicDialecticLoopTests.swift @@ -106,6 +106,44 @@ final class HypnagogicDialecticLoopTests: XCTestCase { func log(_ event: DialecticalTurnEvent) async { events.append(event) } } + /// A metadata-publishing generator — the shape + /// `GenerationRuntimeTextGeneratingAdapter` has in production: a struct + /// whose callback lives in a shared class box, returning + /// role-distinguishable text and firing `onMetadata` with a fixed + /// identity after every call. + private struct PublishingSpyGenerator: MetadataPublishingTextGenerating { + private final class CallbackBox: @unchecked Sendable { + var callback: (@Sendable (GenerationMetadata) -> Void)? + } + nonisolated let isLive = false + nonisolated let modelIdentifier: String + private let stabilizer: String + private let dreamer: String + private let metadata: GenerationMetadata + private let box = CallbackBox() + var onMetadata: (@Sendable (GenerationMetadata) -> Void)? { + get { box.callback } + set { box.callback = newValue } + } + init(stabilizer: String, dreamer: String? = nil, metadata: GenerationMetadata) { + self.stabilizer = stabilizer + self.dreamer = dreamer ?? stabilizer + self.metadata = metadata + self.modelIdentifier = metadata.model + } + func generate(prompt: String, maxTokens: Int, temperature: Double, + cancellationID: UUID) async throws -> String { + box.callback?(metadata) + return temperature >= 0.9 ? dreamer : stabilizer + } + } + + private func spyMetadata(profile: String, hash: String) -> GenerationMetadata { + GenerationMetadata(runtime: "spy", transport: "spy", provider: "spy", + model: "spy-model", promptHash: hash, + promptProfile: profile, interactionStyle: "dialectical") + } + private func fastConfig(maxSilence: Int = 3, witnessEnabled: Bool = false) -> HypnagogicDialecticLoop.Config { HypnagogicDialecticLoop.Config(listenTimeout: 0.01, interTurnDelayNanos: 1_000, maxConsecutiveSilence: maxSilence, @@ -209,6 +247,87 @@ final class HypnagogicDialecticLoopTests: XCTestCase { XCTAssertNil(ev?.witnessFinding, "a failed witness produces no finding") } + // MARK: - Witness fingerprint (per-turn Witness provenance) + + func testPoleAndWitnessFingerprintsAreCapturedSeparately() async { + // The persisted turn must independently attest which runtime/prompt + // produced the Witness finding — not only the candidates'. The bug + // class this guards: a "Witness" adapter that reported one prompt + // while transmitting another. Distinct prompt hashes on the two + // fingerprints of the same event are the proof the wiring is per-role. + let embedder = MapEmbedder(dimension: 2, table: [ + "the sea": [1, 0], "calm sea": [1, 0], "moon river": [0, 1], + "WITNESS_FINDING": [0.7, 0.7], + ]) + let listener = SpyListener(script: ["the sea"], loopLast: true) + let generator = PublishingSpyGenerator( + stabilizer: "calm sea", dreamer: "moon river", + metadata: spyMetadata(profile: "wakingDialectical", hash: "pole-hash")) + let witness = PublishingSpyGenerator( + stabilizer: "WITNESS_FINDING", + metadata: spyMetadata(profile: "witness", hash: "witness-hash")) + let speaker = SpySpeaker() + let logger = CapturingTurnLogger() + let loop = HypnagogicDialecticLoop( + listener: listener, generator: generator, speaker: speaker, embedder: embedder, + random: { 0.5 }, turnLogger: logger, witness: witness, + config: fastConfig(witnessEnabled: true)) + await loop.attachMetadataCaptureFromAdapter() + await loop.start() + let ok = await poll { + let events = await logger.events + return events.contains { $0.witnessGeneratorFingerprint != nil } + } + await loop.stop() + XCTAssertTrue(ok, "a reflective turn must record the Witness fingerprint") + + let events = await logger.events + let ev = events.first { $0.witnessGeneratorFingerprint != nil } + XCTAssertEqual(ev?.generatorFingerprint?.promptHash, "pole-hash") + XCTAssertEqual(ev?.witnessGeneratorFingerprint?.promptHash, "witness-hash") + XCTAssertNotEqual( + ev?.generatorFingerprint?.promptHash, + ev?.witnessGeneratorFingerprint?.promptHash, + "the Witness fingerprint must not attest the pole prompt") + XCTAssertEqual(ev?.witnessGeneratorFingerprint?.promptProfile, "witness") + } + + func testWitnessFingerprintIsNilWhenWitnessDisabled() async { + // Focused/Contemplative: even with a publishing witness *injected*, + // a profile that disables it must not attest a Witness identity — + // the fingerprint records what generated, not what was available. + let embedder = MapEmbedder( + dimension: 2, table: ["the sea": [1, 0], "calm sea": [1, 0], "moon river": [0, 1]]) + let listener = SpyListener(script: ["the sea"], loopLast: true) + let generator = PublishingSpyGenerator( + stabilizer: "calm sea", dreamer: "moon river", + metadata: spyMetadata(profile: "wakingDialectical", hash: "pole-hash")) + let witness = PublishingSpyGenerator( + stabilizer: "SHOULD_NOT_FIRE", + metadata: spyMetadata(profile: "witness", hash: "witness-hash")) + let speaker = SpySpeaker() + let logger = CapturingTurnLogger() + let loop = HypnagogicDialecticLoop( + listener: listener, generator: generator, speaker: speaker, embedder: embedder, + random: { 0.5 }, turnLogger: logger, witness: witness, + config: fastConfig(witnessEnabled: false)) + await loop.attachMetadataCaptureFromAdapter() + await loop.start() + let ok = await poll { + let events = await logger.events + return events.contains { $0.generatorFingerprint != nil } + } + await loop.stop() + XCTAssertTrue(ok, "the pole fingerprint must still be captured") + + let events = await logger.events + let ev = events.first { $0.generatorFingerprint != nil } + XCTAssertEqual(ev?.generatorFingerprint?.promptHash, "pole-hash") + XCTAssertNil( + ev?.witnessGeneratorFingerprint, + "a turn on which the Witness never ran must not attest a Witness identity") + } + func testSelfSimilarityRisesWhenRepliesCollapse() async { // The reflexive metric: the same coherent reply wins every turn ⇒ the // utterance sits on its own reply centroid ⇒ selfSimilarity → 1. Logged From 4c5e275c08981125f00fe282ab67b59e735ffcd9 Mon Sep 17 00:00:00 2001 From: aurascoper Date: Sun, 26 Jul 2026 01:22:28 -0500 Subject: [PATCH 06/11] fix(ui): retain failed runtime identity after loop disablement MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The fail-closed path stores the unavailable identity and then sets hypnagogicLoopEnabled = false — after which the expanded privacy view's Network row rendered only "Disabled at runtime", hiding the requested provider, requested model, locality, and readiness failure that the identity was designed to preserve. lastError still carried the sanitized message, so this was not a privacy failure; it was a mismatch between the architectural claim (failure still has an identity) and the final UI (failure's identity was invisible). RuntimeIdentityPresentation gains lastAttemptLines(dialogue:witness:) — "Last attempt — Dialogue: Ollama · qwen2.5:0.5b · On-device · Model unavailable" — computed as data so the claim is testable, like every other runtime string this banner makes. The disabled Network row now renders those lines under "Disabled at runtime". Empty before the first enable attempt, so a fresh app still reads as plainly disabled. Every rendered field comes from the sanitized identity, never the raw error, and the active-listening badge stays hidden because the loop is not running (it remains gated on the toggle). Tests: no lines before any attempt; a failed dialogue attempt stays visible with the requested pair, locality, and failure label (and never "Ready"); a failed Witness attempt is reported alongside the dialogue attempt. Co-Authored-By: Claude Fable 5 --- .../PrivacyIndicatorView.swift | 20 ++++++++- .../RuntimeIdentityPresentation.swift | 29 ++++++++++++ .../RuntimeIdentityPresentationTests.swift | 44 +++++++++++++++++++ 3 files changed, 92 insertions(+), 1 deletion(-) diff --git a/Sources/NeuralComposeApp/PrivacyIndicatorView.swift b/Sources/NeuralComposeApp/PrivacyIndicatorView.swift index 5d2eaab..95ea1bf 100644 --- a/Sources/NeuralComposeApp/PrivacyIndicatorView.swift +++ b/Sources/NeuralComposeApp/PrivacyIndicatorView.swift @@ -151,7 +151,25 @@ struct PrivacyIndicatorView: View { .foregroundStyle(.secondary) } } else { - Text("Disabled at runtime").foregroundStyle(.green) + VStack(alignment: .leading, spacing: 3) { + Text("Disabled at runtime").foregroundStyle(.green) + // A fail-closed disablement stores the failed + // identity and flips the toggle off; hiding it + // here would discard exactly the diagnosis the + // identity exists to preserve. Empty before the + // first enable attempt. The active badge stays + // hidden — the loop is not running. + ForEach( + RuntimeIdentityPresentation.lastAttemptLines( + dialogue: dialogueRuntimeIdentity, + witness: witnessRuntimeIdentity), + id: \.self + ) { line in + Text(line) + .font(.caption2) + .foregroundStyle(.secondary) + } + } } } GridRow { diff --git a/Sources/NeuralComposeApp/RuntimeIdentityPresentation.swift b/Sources/NeuralComposeApp/RuntimeIdentityPresentation.swift index c412f45..966052f 100644 --- a/Sources/NeuralComposeApp/RuntimeIdentityPresentation.swift +++ b/Sources/NeuralComposeApp/RuntimeIdentityPresentation.swift @@ -72,6 +72,35 @@ struct RuntimeIdentityPresentation: Equatable { : "desktopcomputer" } + /// The persistent diagnostics for the *disabled* state — the loop toggle + /// is off, but a previous enable attempt left identities behind. + /// + /// The gap this closes: a fail-closed disablement stored the unavailable + /// identity and then set `hypnagogicLoopEnabled = false`, after which the + /// expanded privacy view rendered only "Disabled at runtime" — hiding the + /// requested provider, model, locality, and readiness failure the + /// identity was designed to preserve. The failure identity stays visible + /// here; the active-listening badge stays hidden because the loop is not + /// running. Empty before the first enable attempt, so a fresh app still + /// reads as plainly disabled. Every rendered field comes from the + /// sanitized identity — never from the raw error. + static func lastAttemptLines( + dialogue: ResolvedRuntimeIdentity?, + witness: ResolvedRuntimeIdentity? + ) -> [String] { + var lines: [String] = [] + if let dialogue { lines.append(lastAttemptLine("Dialogue", dialogue)) } + if let witness { lines.append(lastAttemptLine("Witness", witness)) } + return lines + } + + private static func lastAttemptLine( + _ title: String, _ identity: ResolvedRuntimeIdentity + ) -> String { + "Last attempt — \(title): \(identity.displayProvider) · \(identity.displayModel) · " + + "\(identity.locality.displayLabel) · \(identity.displayReadiness)" + } + /// `role: provider · model · locality · readiness`. private static func line( _ title: String, diff --git a/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift b/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift index adf6291..fcaf6e1 100644 --- a/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift +++ b/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift @@ -139,4 +139,48 @@ final class RuntimeIdentityPresentationTests: XCTestCase { dialogue: identity(), witness: nil, isDialectical: false) XCTAssertTrue(p.caption.contains("one call per turn"), p.caption) } + + // MARK: - Last runtime attempt (visible after fail-closed disablement) + + /// A fresh app has attempted nothing: the disabled row must stay plainly + /// "Disabled at runtime" with no phantom attempt to explain. + func testNoAttemptYieldsNoLastAttemptLines() { + XCTAssertTrue( + RuntimeIdentityPresentation.lastAttemptLines(dialogue: nil, witness: nil).isEmpty) + } + + /// The fail-closed sequence stores the unavailable identity and disables + /// the toggle. The expanded diagnostics must keep showing what was + /// requested and why it failed — the identity was designed to preserve + /// exactly this, and rendering only "Disabled at runtime" discarded it. + func testFailedIdentityStaysVisibleAfterDisablement() { + let lines = RuntimeIdentityPresentation.lastAttemptLines( + dialogue: identity(readiness: .unavailable(.modelMissing)), + witness: nil) + XCTAssertEqual(lines.count, 1) + let line = lines[0] + XCTAssertTrue(line.contains("Last attempt"), line) + // The failure identity resolves nothing, so display falls back to the + // *requested* pair — the user must see what they asked for. + XCTAssertTrue(line.contains("Ollama"), line) + XCTAssertTrue(line.contains("qwen2.5:0.5b"), line) + XCTAssertTrue(line.contains("On-device"), line) + XCTAssertTrue(line.contains("Model unavailable"), line) + XCTAssertFalse(line.contains("Ready"), line) + } + + /// A failed Witness attempt is part of the requested configuration and is + /// reported alongside the dialogue attempt, in that order. + func testWitnessAttemptIsReportedAlongsideTheDialogueAttempt() { + let lines = RuntimeIdentityPresentation.lastAttemptLines( + dialogue: identity(), + witness: identity( + role: .witness, provider: "claude", model: "claude-sonnet-5", + locality: .localBrokerToRemoteService, + readiness: .unavailable(.executableNotFound))) + XCTAssertEqual(lines.count, 2) + XCTAssertTrue(lines[0].contains("Dialogue"), lines[0]) + XCTAssertTrue(lines[1].contains("Witness"), lines[1]) + XCTAssertTrue(lines[1].contains("CLI not found"), lines[1]) + } } From b3a1ddb32e925fb015e1c55255266b5b6209e30f Mon Sep 17 00:00:00 2001 From: aurascoper Date: Sun, 26 Jul 2026 01:34:42 -0500 Subject: [PATCH 07/11] test(runtime): record packaged Apple Silicon A2 acceptance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sanitized acceptance record for PR #32 at head 4c5e275 (A/B/C + D/E/F) on Apple Silicon / macOS 26.5.2 — observations only, pending items listed as pending rather than assumed. Observed and recorded: - build, full test suite (616/0, exit 0), package-app-bundle, smoke-packaged-resources (all 5 packaged-layout loads + the missing-bundle guard), and codesign --verify --deep --strict, all PASS - real-daemon headless acceptance via dialectic-session: exact Ollama model resolves ready with a full fingerprint and no LLM call; a missing model fails closed (exit 1) with no substitution; Claude resolves as configured-only (executable + prompt, no provider request) - a divergence found by the untagged-model fixture: the harness's RuntimeFactory rejects an untagged request whose :latest form is installed, while the app's LiveRuntimeFactory/OllamaReadinessProbe canonicalizes it (and since commit D reports it as the same model) — flagged for follow-up alignment - a local-environment caveat: one UDP stream test kills the test process silently on this machine (reproduced at the untouched base 14c8b6a; CI green), so local `swift test` must be judged by exit code Pending (operator required, per the A2 report): the packaged GUI mode×provider matrix, real Claude operational acceptance including a Reflective turn with the new Witness fingerprint, cancellation/mode-change races, and the superseded-Reflective rerun. No private transcripts; the harness's model-list enumeration is noted but not reproduced. Co-Authored-By: Claude Fable 5 --- .../evaluation/a2-apple-silicon-acceptance.md | 140 ++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 docs/evaluation/a2-apple-silicon-acceptance.md diff --git a/docs/evaluation/a2-apple-silicon-acceptance.md b/docs/evaluation/a2-apple-silicon-acceptance.md new file mode 100644 index 0000000..63c0390 --- /dev/null +++ b/docs/evaluation/a2-apple-silicon-acceptance.md @@ -0,0 +1,140 @@ +# A2 Apple Silicon acceptance record — PR #32 + +Sanitized evidence for the packaged Apple Silicon acceptance of the A2 +runtime-identity work (commits A/B/C `1fdf48b`/`3067cd8`/`14c8b6a` plus +follow-ups D/E/F). Records what was **directly observed**, on which head, in +which environment. Anything not run is listed as pending, not assumed. + +## Environment + +| | | +|---|---| +| Date | 2026-07-26 | +| Head under test | `4c5e275` (= C `14c8b6a` + D `d0d5dd7` + E `db01432` + F `4c5e275`) | +| Hardware / OS | Apple Silicon (arm64), macOS 26.5.2 (25F84) | +| Toolchain | Swift 6.3.3 (swiftlang-6.3.3.1.3) | +| Ollama daemon | running on `http://localhost:11434`, `qwen2.5:0.5b` pulled | +| Claude CLI | present on PATH | + +## 1. Build, test, package, sign — PASS (observed) + +``` +swift build → Build complete +swift test → 616 tests, 0 failures, 10 skipped, exit 0 ¹ +./Scripts/build.sh → Build complete +./Scripts/package-app-bundle.sh → Bundled .build/NeuralCompose.app +./Scripts/smoke-packaged-resources.sh + → PASS — packaged prompt resources present, + loadable, signed, and guarded + (all 5 packaged-layout loads succeeded; + packaging guard exits 1 with bundle removed) +codesign --verify --deep --strict .build/NeuralCompose.app + → verified, exit 0 +``` + +¹ The 616/0 result requires +`--skip MindMonitorOSCStreamTests/testTruncatedSampleAddressCountsAsDroppedNotIgnored` +**on this machine**: that UDP test kills the test process silently +(exit 1, no failure output, every suite alphabetically after it never +runs). Reproduced identically at the untouched base `14c8b6a`, so it is +pre-existing and environmental (macOS 26 local-network TCC against an +unbundled test binary is the suspected mechanism, consistent with this +machine's cdhash-pinned Local Network grants), not introduced by D/E/F. +PR CI is green including this test. **Caveat for local runs:** a plain +`swift test` on this machine silently under-reports — check the exit +code, not the absence of failure lines. + +The packaged-app MLX caveat from `package-app-bundle.sh` was observed: +no `mlx-swift_Cmlx.bundle` metallib in this SwiftPM build, so the +spectral estimator runs as STUB in the packaged app (known limitation, +`Scripts/build-xcode-mlx.sh` is the documented remedy). Does not affect +runtime-identity behavior. + +## 2. Headless runtime acceptance (real daemon, no generation) — observed + +Via `dialectic-session` (`--dry-run` initializes + verifies + reports and +makes **no** LLM call). + +**Ollama, exact model present — PASS.** +`--dry-run --runtime ollama --model qwen2.5:0.5b` → exit 0; fingerprint +`runtime=ollama transport=ollama-http provider=ollama model=qwen2.5:0.5b +promptProfile=wakingDialectical promptHash=094ec537…`; verification +`prompt loaded: yes / transport reachable: yes / model available: yes`; +"No LLM call was made." + +**Ollama, missing model, daemon alive — PASS (fails closed).** +`--dry-run --runtime ollama --model a2-definitely-absent` → exit 1, +"is not installed" diagnostic, no generation, no Claude substitution. + +**Ollama, untagged request with `:latest` stored — DIVERGENCE (harness only).** +Fixture: `ollama cp qwen2.5:0.5b a2-accept:latest`, request `a2-accept`, +fixture removed afterwards. The **harness** (`DialecticSession/RuntimeFactory`) +rejected the untagged request (exit 1) even though `a2-accept:latest` was +installed. The **app** path (`LiveRuntimeFactory` → `OllamaReadinessProbe`) +canonicalizes untagged → `:latest` and, since commit D, reports it as the +same model in `isSubstitution` (regression-tested). Follow-up: align the +harness probe with the app's canonicalization or document the stricter +harness matching as intended. + +**Claude, resolution only — PASS (no request made).** +`--runtime-report --runtime claude --model claude-sonnet-5` → exit 0; +executable resolved; `provider=anthropic model=claude-sonnet-5 +promptProfile=wakingDialectical promptHash=094ec537…` (same hash as the +Ollama dialectic profile — same transmitted prompt bytes, as intended). +Per the A2 report this proves **configured**, not operationally proven: +no authentication, rate-limit, or generation check occurred. + +**Not observable headlessly here** (no endpoint override in the harness +CLI): non-loopback Ollama endpoint and endpoint-unavailable. Both remain +covered by mocked regression tests at the app factory +(`testLoopbackOllamaIsOnDeviceAndNonLoopbackIsNot`, +`testUnreachableEndpointIsUnavailableNotModelMissing`). + +Note on sanitization: the harness's missing-model stderr enumerates the +user's locally pulled models. The app deliberately does not (see +`LiveRuntimeFactory`'s `.modelMissing` path). Acceptable for a developer +CLI on stderr; this record does not reproduce the enumeration. + +## 3. What D/E/F add to the A2 keep-bar (regression-tested, this head) + +- Unknown provider → `RuntimeLocality.unresolved`: egress-conservative, + displayed "Egress unverified", never "On-device", never a claimed broker + topology (D). +- `qwen2.5` → `qwen2.5:latest` is canonical resolution, not a substitution + alarm; any other difference still discloses (D). +- `DialecticalTurnEvent.witnessGeneratorFingerprint` persists the Witness's + provider/model/prompt profile/prompt hash separately from the pole + fingerprint; pole and Witness prompt hashes proven distinct on the same + logged event; pre-field logs decode unchanged (E). + `analyze_dialectic.py` reports witness-fingerprint counts and a + witness/pole prompt-hash collision count (collision = the old + wrong-prompt bug's signature; must be 0). +- Failed runtime identity stays visible in the expanded privacy panel after + fail-closed disablement ("Last attempt — …"); active badge stays hidden + while disabled (F). + +## 4. Pending — requires an operator (not run in this record) + +Per the A2 report's acceptance plan; none of the below is claimed. + +1. **Packaged GUI matrix**: launch `.build/NeuralCompose.app` (binary + directly, not `open`, so `NEURALCOMPOSE_RUNTIME`/`NEURALCOMPOSE_MODEL` + reach it), grant mic/speech, and drive one turn or one deliberate + readiness failure per combination: {Mirror, Focused, Reflective, + Contemplative} × {Claude, Ollama}. Observe requested/resolved identity, + locality, readiness, prompt profile/hash, toggle state, UI wording, and + on failure: no Claude process, "Last attempt" diagnostics retained. +2. **Real Claude operational acceptance**: authenticated CLI, one real + Mirror turn, one Focused turn, one Reflective turn with separate Witness + fingerprint present in the logged turn (now recordable via E); CLI + missing and rate-limited/unavailable cases where safely reproducible. + This decides whether `RuntimeReadiness` needs a distinct + `configured` state, per the A2 report. +3. **Cancellation / mode-change**: while resolution is pending — switch + mode, disable the loop, quit the app, interrupt an Ollama probe, deny + mic permission; verify the reconcile seam prevents stale resolutions + from enabling or reporting the wrong runtime. +4. **Reflective reference rerun**: after this PR merges, mark the affected + pre-B Reflective runs `status: superseded` (dead-prompt-field defect, + `fixed_by: 3067cd8`) and rerun the Focused-vs-Reflective fixture so the + replacement captures dialogue + Witness fingerprints. From 236728156d68fb9cbd397d2f8d395753b118d3f2 Mon Sep 17 00:00:00 2001 From: aurascoper Date: Mon, 27 Jul 2026 16:03:01 -0500 Subject: [PATCH 08/11] fix(runtime): distinguish configured from verified readiness MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `RuntimeReadiness` had one success case, and the two resolution paths reached it on very different evidence. Ollama GET /api/tags, exact requested model matched observed fact Claude executable resolved, prompt loaded nothing contacted Both rendered the word "Ready". On the Claude path an expired `claude login`, an unreachable network, or a model the account cannot use all still resolved as ready, because no request is made to Anthropic before enablement. That left the failure side of the type strictly more precise than the success side: six distinct reasons a runtime cannot be used, one undifferentiated reason it can. `configured` closes the gap in the domain model rather than in the view, so a future runtime's evidence semantics is a property of its resolution, not an undocumented convention in `PrivacyIndicatorView`. configured constructible, unverified Claude ready endpoint + exact model verified Ollama unavailable must not be used both Two predicates, because the existing single one cannot answer both questions. `isReady` stays strict; `canAttemptGeneration` covers configured|ready. The distinction is load-bearing at `PrivacyIndicatorView.identityColor`, which paints "not ready" red — keying it on `isReady` would have marked every Claude session as a fault, trading the over-claim for its mirror image. The colour answers "is this broken?"; the unverified/verified nuance lives in the line text, where a nuance belongs. A successful generation is *session* evidence and deliberately does not mutate the identity. The identity records what resolution proved; rewriting it after a good turn would erase the distinction this split exists to make. Adding a case does not disturb the wire format — synthesized enum `Codable` keys on the case name, not a declaration ordinal — but that is a property of the compiler rather than of this type, so both legacy encodings are asserted rather than assumed. No existing assertion broke, which is itself the finding: `testDefaultResolutionIsStillClaude` checked provider, model, substitution and generator type and never looked at readiness, so the Claude success path's readiness value was wholly unasserted. That is how the over-claim survived. `RuntimeIdentityPresentationTests` fixture keyed its resolved fields on `== .ready`, which would have handed a configured runtime the empty *failure* shape; it now keys on `canAttemptGeneration`. The two presentation tests were verified against the mutation they exist to catch — branching `displayReadiness` on `resolvedProvider == "claude"` — which fails both, in both directions. One asserts an Ollama-provider identity with `configured` renders as configured; the other, a Claude-provider identity with `ready` renders as verified. No provider-spelling implementation satisfies both. deterministic suites 423 executed, 5 skipped, 0 failures A2 app + session 147 executed, 0 failures swift test list 627 discovered (was 619) PR diff stays at 21 files. Co-Authored-By: Claude Fable 5 --- .../BCICloudBridge/LiveRuntimeFactory.swift | 29 +++++-- .../ResolvedRuntimeIdentity.swift | 55 ++++++++++++- .../PrivacyIndicatorView.swift | 12 ++- .../LiveRuntimeFactoryFailClosedTests.swift | 79 ++++++++++++++++++- .../RuntimeIdentityPresentationTests.swift | 48 ++++++++++- 5 files changed, 206 insertions(+), 17 deletions(-) diff --git a/Sources/BCICloudBridge/LiveRuntimeFactory.swift b/Sources/BCICloudBridge/LiveRuntimeFactory.swift index fcfa902..ef75d2e 100644 --- a/Sources/BCICloudBridge/LiveRuntimeFactory.swift +++ b/Sources/BCICloudBridge/LiveRuntimeFactory.swift @@ -17,12 +17,19 @@ import Foundation /// `RuntimeResolutionFailure`), because a UI that can only describe runtimes /// that resolved cannot tell the user what went wrong with the one that didn't. /// -/// **Readiness is checked before the loop starts, never by generating.** -/// - Claude: the executable is resolved through `ClaudeExecutableResolver` -/// — the same resolver the harness uses — and the prompt is loaded and -/// hashed. No request is made to Anthropic. -/// - Ollama: a bounded `GET /api/tags` confirms the daemon is up and has the -/// exact requested model. No prompt is sent. +/// **Readiness is checked before the loop starts, never by generating** — and +/// the two paths prove different things, so they report different readiness. +/// - Claude → `.configured`: the executable is resolved through +/// `ClaudeExecutableResolver` — the same resolver the harness uses — and +/// the prompt is loaded and hashed. No request is made to Anthropic, so +/// authentication, account state, and model entitlement remain unverified. +/// - Ollama → `.ready`: a bounded `GET /api/tags` confirms the daemon is up +/// and has the exact requested model. No prompt is sent. +/// +/// Both are usable (`canAttemptGeneration`); only one is verified (`isReady`). +/// A generation that later succeeds is *session* evidence and does not mutate +/// the identity — the identity describes what resolution proved, and rewriting +/// it after the fact would erase the distinction this split exists to record. /// /// A readiness failure disables the loop. **No alternate provider is ever /// attempted**: substituting Claude when Ollama is unavailable would turn a @@ -198,7 +205,13 @@ public enum LiveRuntimeFactory { // placeholder, so telemetry never records a fabricated value. modelDigest: nil, locality: locality, - readiness: .ready, + // `configured`, not `ready`. Everything above proved the runtime is + // *constructible*: an executable resolved and a prompt loaded. No + // request reached the provider, so authentication, account state, + // network reachability, and whether this account may use this model + // are all still unknown. Reporting `ready` here claimed the + // verification only the Ollama path actually performs. + readiness: .configured, promptProfile: profile.rawValue, promptHash: PromptProfile.sha256Hex(promptText), systemPromptSource: "PromptProfile(\(profile.rawValue))" @@ -308,6 +321,8 @@ public enum LiveRuntimeFactory { resolvedModel: resolvedModel, modelDigest: digest, locality: locality, + // `ready` is earned here: `OllamaReadinessProbe` reached the daemon + // and matched the exact requested model before enablement. readiness: .ready, promptProfile: profile.rawValue, promptHash: PromptProfile.sha256Hex(promptText), diff --git a/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift b/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift index 4b13077..820271b 100644 --- a/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift +++ b/Sources/BCICloudBridge/ResolvedRuntimeIdentity.swift @@ -86,8 +86,15 @@ public struct ResolvedRuntimeIdentity: Codable, Sendable, Equatable { name.isEmpty || name.contains(":") ? name : name + ":latest" } + /// The *strict* predicate: the endpoint and exact model were verified. + /// Deliberately false for a `configured` Claude runtime, which is usable + /// but unproven — see `canAttemptGeneration` for the usability question. public var isReady: Bool { readiness == .ready } + /// The *usability* predicate: generation may be attempted. True for both + /// `configured` and `ready`. + public var canAttemptGeneration: Bool { readiness.canAttemptGeneration } + /// Human-readable provider name for display. Falls back to the *requested* /// provider when nothing resolved, so a failed identity still names what /// the user asked for instead of rendering an empty cell. @@ -99,10 +106,14 @@ public struct ResolvedRuntimeIdentity: Codable, Sendable, Equatable { resolvedModel.isEmpty ? requestedModel : resolvedModel } - /// `Ready`, or the specific reason it is not. + /// What resolution actually proved, or the specific reason it failed. + /// + /// `ready` names its evidence rather than saying "Ready", because the whole + /// point of the `configured` split is that the user can tell the two apart. public var displayReadiness: String { switch readiness { - case .ready: return "Ready" + case .configured: return "Configured" + case .ready: return "Endpoint + model verified" case .unavailable(let failure): return failure.displayLabel } } @@ -173,12 +184,48 @@ public enum RuntimeLocality: String, Codable, Sendable, Equatable, CaseIterable } } -/// Whether a runtime may be used. `unavailable` is a *pre-generation* verdict: -/// the loop must not start, and no alternate provider is attempted. +/// Whether a runtime may be used, and *on what evidence*. `unavailable` is a +/// *pre-generation* verdict: the loop must not start, and no alternate provider +/// is attempted. +/// +/// The success side is split because the two resolution paths prove different +/// things and used to report the same word for both. +/// +/// - Ollama contacts `/api/tags` and requires the exact requested model before +/// enablement, so its success is a *positively observed* fact about the +/// endpoint: the daemon answered and holds that model. +/// - Claude resolves an executable and loads a prompt. Nothing contacts the +/// provider. An expired `claude login`, an unreachable network, or a model +/// name the account cannot use all still resolve. +/// +/// Collapsing those into one `ready` made the failure side of this type strictly +/// more precise than the success side: six distinct reasons a runtime cannot be +/// used, one undifferentiated reason it can. `configured` closes that gap. public enum RuntimeReadiness: Codable, Sendable, Equatable { + /// Local construction prerequisites resolved, but no real provider or + /// exact-model round trip has been observed. + case configured + + /// The configured endpoint/provider and exact requested model were + /// positively verified before enablement. case ready + + /// The runtime must not be used. case unavailable(RuntimeReadinessFailure) + /// Whether generation may be attempted at all. Both success cases qualify: + /// `configured` is unverified, not broken, and a UI that painted it as a + /// fault would trade one false claim for its mirror image. + /// + /// Read this for *usability*; read `isReady` for *verified* usability. The + /// two are deliberately different questions. + public var canAttemptGeneration: Bool { + switch self { + case .configured, .ready: return true + case .unavailable: return false + } + } + public var failure: RuntimeReadinessFailure? { if case .unavailable(let f) = self { return f } return nil diff --git a/Sources/NeuralComposeApp/PrivacyIndicatorView.swift b/Sources/NeuralComposeApp/PrivacyIndicatorView.swift index 95ea1bf..3ca3e25 100644 --- a/Sources/NeuralComposeApp/PrivacyIndicatorView.swift +++ b/Sources/NeuralComposeApp/PrivacyIndicatorView.swift @@ -494,10 +494,18 @@ struct PrivacyIndicatorView: View { runtimePresentation.involvesEgress ? .red : .orange } - /// A runtime that is not ready must not read as running. + /// A runtime that cannot be used must not read as running. + /// + /// Keyed on `canAttemptGeneration`, not `isReady`: since the readiness split, + /// a resolved Claude runtime is `configured` — usable but unverified — and + /// the strict predicate would paint every Claude session red. Red is the + /// fault colour, so that would replace an over-claim ("Ready" when nothing + /// was checked) with the opposite false claim. The unverified/verified + /// distinction is carried by `displayReadiness` in the line text, which is + /// where a nuance belongs; the colour answers only "is this broken?". private func identityColor(_ identity: ResolvedRuntimeIdentity?) -> Color { guard let identity else { return .secondary } - return identity.isReady ? .secondary : .red + return identity.canAttemptGeneration ? .secondary : .red } /// The most consequential badge: while active the mic is always capturing, diff --git a/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift b/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift index c6af436..92e9432 100644 --- a/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift +++ b/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift @@ -34,8 +34,10 @@ final class LiveRuntimeFactoryFailClosedTests: XCTestCase { } /// A PATH containing a stub executable named `claude`. Resolution validates - /// the file and never runs it, so this is enough to reach a `ready` - /// identity without contacting Anthropic. + /// the file and never runs it, so this is enough to reach a `configured` + /// identity without contacting Anthropic — and never enough to reach + /// `ready`, which requires a verified provider round trip this path has no + /// way to perform. private func makeClaudeEnvironment() throws -> [String: String] { let dir = try makeDirectory() let claude = dir.appendingPathComponent("claude") @@ -371,4 +373,77 @@ final class LiveRuntimeFactoryFailClosedTests: XCTestCase { XCTAssertFalse(identity.isSubstitution) XCTAssertTrue(generator is ClaudeCLIGenerator) } + + // MARK: - Readiness evidence: configured vs verified + + /// Claude resolution proves the runtime is *constructible*, never that it + /// works. Nothing here contacts Anthropic, so `ready` — which the Ollama + /// path earns with an exact-model match against a live daemon — would be an + /// unearned claim. This assertion is the one the original suite lacked: + /// `testDefaultResolutionIsStillClaude` checked provider, model, + /// substitution and generator type, and never looked at readiness at all, + /// which is precisely how the over-claim survived. + func testClaudeResolutionIsConfiguredNotVerified() async throws { + let environment = try makeClaudeEnvironment() + let (_, identity) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: nil, environment: environment) + + XCTAssertEqual(identity.readiness, .configured) + XCTAssertFalse(identity.isReady, "an unverified runtime must not report verified") + XCTAssertTrue(identity.canAttemptGeneration, "configured is unverified, not broken") + XCTAssertEqual(identity.displayReadiness, "Configured") + } + + /// The Ollama path reached the daemon and matched the exact requested + /// model, so `ready` is a statement about an observed fact. + func testOllamaExactModelResolutionIsVerified() async throws { + stubTags([("qwen2.5:0.5b", "sha256:deadbeef")]) + let (_, identity) = try await makeOllama(model: "qwen2.5:0.5b") + + XCTAssertEqual(identity.readiness, .ready) + XCTAssertTrue(identity.isReady) + XCTAssertTrue(identity.canAttemptGeneration) + XCTAssertEqual(identity.displayReadiness, "Endpoint + model verified") + } + + /// The usability predicate admits both success cases. Without this the + /// strict predicate would have to serve both questions, and every Claude + /// session would render as a fault. + func testConfiguredRuntimeCanAttemptGeneration() { + XCTAssertTrue(RuntimeReadiness.configured.canAttemptGeneration) + XCTAssertTrue(RuntimeReadiness.ready.canAttemptGeneration) + } + + /// Every failure reason stays fail-closed. Iterating `allCases` rather than + /// spot-checking one means a newly added failure cannot default to usable. + func testUnavailableRuntimeCannotAttemptGeneration() { + for failure in RuntimeReadinessFailure.allCases { + XCTAssertFalse( + RuntimeReadiness.unavailable(failure).canAttemptGeneration, + "\(failure) must not be attemptable") + } + } + + // MARK: - Readiness wire compatibility + + /// Adding a case must not rewrite the encoding of the existing ones. + /// Synthesized enum `Codable` keys on the *case name*, not a declaration + /// ordinal, so `configured` landing first in the declaration cannot shift + /// `ready` — but that is a property of the compiler, not of this type, so + /// it is asserted rather than assumed. + func testOldReadyIdentityStillDecodes() throws { + let legacy = Data(#"{"ready":{}}"#.utf8) + XCTAssertEqual(try JSONDecoder().decode(RuntimeReadiness.self, from: legacy), .ready) + + // And the current encoder still emits exactly that form. + let encoded = try JSONEncoder().encode(RuntimeReadiness.ready) + XCTAssertEqual(String(decoding: encoded, as: UTF8.self), #"{"ready":{}}"#) + } + + func testOldUnavailableIdentityStillDecodes() throws { + let legacy = Data(#"{"unavailable":{"_0":"modelMissing"}}"#.utf8) + XCTAssertEqual( + try JSONDecoder().decode(RuntimeReadiness.self, from: legacy), + .unavailable(.modelMissing)) + } } diff --git a/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift b/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift index fcaf6e1..6a6700f 100644 --- a/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift +++ b/Tests/NeuralComposeAppTests/RuntimeIdentityPresentationTests.swift @@ -22,8 +22,13 @@ final class RuntimeIdentityPresentationTests: XCTestCase { ResolvedRuntimeIdentity( role: role, requestedProvider: provider, requestedModel: model, - resolvedProvider: readiness == .ready ? provider : "", - resolvedModel: readiness == .ready ? model : "", + // Resolved fields are populated whenever something resolved, and + // empty only on the failure identity. Keyed on + // `canAttemptGeneration`, not `== .ready`: a `configured` runtime + // did resolve a provider and model, and the older two-valued test + // would have handed it the empty *failure* shape instead. + resolvedProvider: readiness.canAttemptGeneration ? provider : "", + resolvedModel: readiness.canAttemptGeneration ? model : "", locality: locality, readiness: readiness, promptProfile: "p", promptHash: "h", systemPromptSource: "s" ) @@ -183,4 +188,43 @@ final class RuntimeIdentityPresentationTests: XCTestCase { XCTAssertTrue(lines[1].contains("Witness"), lines[1]) XCTAssertTrue(lines[1].contains("CLI not found"), lines[1]) } + + // MARK: - Readiness evidence is read, never inferred + + /// The banner must say "Configured" because the *identity* says so — not + /// because the provider happens to be spelled "claude". + /// + /// The mutation this catches is the tempting contained fix: branching the + /// view on `resolvedProvider == "claude"`. That would reproduce the exact + /// class of provider-name inference R8 removed, and would silently + /// mis-describe any future runtime whose resolution is also unverified. So + /// the identity here is a **Claude-free** one — an Ollama provider carrying + /// `configured` — which a provider-branching implementation renders as + /// verified and this assertion rejects. + func testPresentationShowsConfiguredWithoutProviderBranching() { + let p = RuntimeIdentityPresentation( + dialogue: identity(provider: "ollama", readiness: .configured), + witness: nil, + isDialectical: false) + + XCTAssertTrue(p.dialogueLine.contains("Configured"), p.dialogueLine) + XCTAssertFalse(p.dialogueLine.contains("verified"), p.dialogueLine) + XCTAssertFalse(allText(p).lowercased().contains("claude"), allText(p)) + } + + /// The converse, for the same reason: a Claude-spelled provider carrying a + /// verified readiness must render as verified. Together these two pin the + /// line text to `readiness` in both directions, so no implementation can + /// satisfy both by reading the provider. + func testPresentationShowsEndpointAndModelVerified() { + let p = RuntimeIdentityPresentation( + dialogue: identity( + provider: "claude", model: "claude-sonnet-5", + locality: .localBrokerToRemoteService, readiness: .ready), + witness: nil, + isDialectical: false) + + XCTAssertTrue(p.dialogueLine.contains("Endpoint + model verified"), p.dialogueLine) + XCTAssertFalse(p.dialogueLine.contains("Configured"), p.dialogueLine) + } } From 71c5c0288ebf95ba9e29bf277cd373da782ebdbb Mon Sep 17 00:00:00 2001 From: aurascoper Date: Mon, 27 Jul 2026 16:27:38 -0500 Subject: [PATCH 09/11] fix(runtime): report unverified Claude checks as not run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Commit H made the app path truthful. The headless harness still recreated the exact ambiguity it removed. `RuntimeReport.verify` returned `(promptLoaded, true, true)` for Claude, with a comment stating plainly that no call had been made. So `--dry-run` printed: transport reachable: yes model available: yes ✓ transport reachable ✓ model exists No LLM call was made. Four claims of verification and, four lines later, an admission that nothing was verified. The acceptance document had already read this correctly as "configured only", but the executable emitted stronger evidence than it held, and the harness is the surface an operator reads when there is no UI. The defect is the boolean. Two states force a skipped check to be reported as either a pass or a failure, and this path picked pass. `VerificationStatus` adds the missing third state: passed ran, condition held failed ran, condition did not hold notChecked did not run — no evidence either way runtime prompt provider exact model claude passed/failed notChecked notChecked ollama passed/failed passed/failed passed/failed `notChecked` is not failure. `hasFailure` is true only for an actual `failed`, so a configuration-only Claude dry run still exits 0 — verified against the built binary, which now prints: Verification: prompt loaded: yes provider reachable: not checked — no generation request made exact model present: not checked — no generation request made ✓ runtime configured ✓ prompt loaded – provider and model were not operationally verified ✓ fingerprint generated Ollama keeps every claim it earns; its branch really does probe /api/tags for the exact model, and reports `✓ endpoint reachable` / `✓ exact model present`. Wording moved out of `main.swift` into `verificationLines` / `dryRunSummaryLines`, which return `[String]`. It was formatted inline at two call sites, which is precisely why no test could assert on what it claimed — the same shape as R8's privacy strings buried in a SwiftUI body. Scope grows 21 → 23 files. `RuntimeReport.swift` and the new test file are A2 runtime lane, not a D0-owned cross-lane file; the 21-file bound was a reviewability guard, not a reason to keep shipping a false diagnostic. A2 app + session 152 executed, 0 failures (was 147) deterministic 423 executed, 0 failures Co-Authored-By: Claude Fable 5 --- Sources/DialecticSession/RuntimeReport.swift | 138 ++++++++++++++++-- Sources/DialecticSession/main.swift | 26 ++-- .../RuntimeVerificationTests.swift | 124 ++++++++++++++++ 3 files changed, 257 insertions(+), 31 deletions(-) create mode 100644 Tests/DialecticSessionTests/RuntimeVerificationTests.swift diff --git a/Sources/DialecticSession/RuntimeReport.swift b/Sources/DialecticSession/RuntimeReport.swift index 7d2d10b..514486d 100644 --- a/Sources/DialecticSession/RuntimeReport.swift +++ b/Sources/DialecticSession/RuntimeReport.swift @@ -65,16 +65,22 @@ public enum RuntimeReport { /// Perform the live verification: load the prompt profile, /// probe the transport, and (for Ollama) verify the model is - /// available. Returns a tuple of (promptLoaded, transportOK, - /// modelAvailable). All checks are best-effort; failures - /// return `false` for the failed check and never throw. - public static func verify(resolved: ResolvedRuntime) -> (promptLoaded: Bool, transportOK: Bool, modelAvailable: Bool) { - let promptLoaded: Bool + /// available. All checks are best-effort and never throw. + /// + /// **A check that was not performed is not a check that passed.** + /// This returned `(promptLoaded, true, true)` for Claude while + /// its own comment said no call had been made, so the harness + /// printed `transport reachable: yes` / `model available: yes` + /// on evidence it had not collected. `notChecked` is the third + /// state that was missing; it is not a failure, and a + /// configuration-only dry run still exits 0. + public static func verify(resolved: ResolvedRuntime) -> RuntimeVerification { + let promptLoaded: VerificationStatus do { _ = try resolved.promptProfile.load() - promptLoaded = true + promptLoaded = .passed } catch { - promptLoaded = false + promptLoaded = .failed } // For Ollama, the factory already validated the model is @@ -101,18 +107,67 @@ public enum RuntimeReport { } _ = sem.wait(timeout: .now() + 5) task.cancel() - let transportOK = ok.withLock { $0 } - return (promptLoaded, transportOK, transportOK) + let reached: VerificationStatus = ok.withLock { $0 } ? .passed : .failed + return RuntimeVerification( + prompt: promptLoaded, transport: reached, model: reached) } else if resolved.runtime is ClaudeCLIGenerationRuntime { - // Claude: the subprocess path validates the CLI on - // first call. For the dry-run, we don't call. We - // report `transportOK = true` (the factory has the - // runtime) and `modelAvailable = true` (the - // subprocess will tell us on the first live call). - return (promptLoaded, true, true) + // Claude validates the CLI on the first call, and a + // dry run makes no call — so the provider was never + // contacted and this account's entitlement to this + // model is unknown. Both are reported as `notChecked` + // rather than assumed good; the prompt check above is + // real and still stands on its own. + return RuntimeVerification( + prompt: promptLoaded, + transport: .notChecked, + model: .notChecked, + notCheckedReason: "no generation request made" + ) } - return (promptLoaded, false, false) + return RuntimeVerification(prompt: promptLoaded, transport: .failed, model: .failed) + } + + /// The `Verification:` block, as lines rather than `print` + /// calls, so the wording is reachable by a test. The previous + /// version formatted inline at two call sites in `main.swift`, + /// which is why nothing could assert on what it claimed. + public static func verificationLines(_ v: RuntimeVerification) -> [String] { + [ + "Verification:", + " prompt loaded: \(describe(v.prompt, reason: v.notCheckedReason))", + " provider reachable: \(describe(v.transport, reason: v.notCheckedReason))", + " exact model present: \(describe(v.model, reason: v.notCheckedReason))", + ] + } + + /// The dry-run summary. Only checks that actually ran get a + /// `✓`; unchecked ones get a `–` that names the gap, so the + /// summary cannot read as stronger evidence than the block + /// above it. + public static func dryRunSummaryLines(_ v: RuntimeVerification) -> [String] { + var lines = ["✓ runtime configured"] + if v.prompt == .passed { lines.append("✓ prompt loaded") } + switch (v.transport, v.model) { + case (.passed, .passed): + lines.append("✓ endpoint reachable") + lines.append("✓ exact model present") + case (.notChecked, .notChecked): + lines.append("– provider and model were not operationally verified") + default: + if v.transport == .passed { lines.append("✓ endpoint reachable") } + if v.model == .passed { lines.append("✓ exact model present") } + } + lines.append("✓ fingerprint generated") + return lines + } + + private static func describe(_ status: VerificationStatus, reason: String?) -> String { + switch status { + case .passed: return "yes" + case .failed: return "no" + case .notChecked: return "not checked — \(reason ?? "not performed on this path")" + } } private static func transportName(of runtime: any GenerationRuntime) -> String { @@ -144,3 +199,54 @@ public enum RuntimeReport { } } } + +/// The outcome of one verification check. +/// +/// Three states, not two. A boolean forces every check that was +/// skipped to be reported as either a pass or a failure, and the +/// Claude dry-run path chose "pass" — printing `transport +/// reachable: yes` for a provider it had never contacted. +public enum VerificationStatus: Equatable, Sendable { + /// The check ran and the condition held. + case passed + /// The check ran and the condition did not hold. + case failed + /// The check did not run, so there is no evidence either way. + /// Explicitly **not** a failure. + case notChecked +} + +/// What a dry run actually established about a runtime. +/// +/// This mirrors `RuntimeReadiness`'s configured/ready split on the +/// headless side: a Claude runtime can be fully constructed with a +/// loaded prompt while its provider and model entitlement remain +/// entirely unverified. +public struct RuntimeVerification: Equatable, Sendable { + public let prompt: VerificationStatus + public let transport: VerificationStatus + public let model: VerificationStatus + /// Why the unchecked checks were not run, rendered beside each + /// `notChecked` line so an absence of evidence names its cause + /// instead of looking like an omission. + public let notCheckedReason: String? + + public init( + prompt: VerificationStatus, + transport: VerificationStatus, + model: VerificationStatus, + notCheckedReason: String? = nil + ) { + self.prompt = prompt + self.transport = transport + self.model = model + self.notCheckedReason = notCheckedReason + } + + /// Only an actual `failed` check fails the run. A + /// configuration-only dry run — everything that could be + /// checked passed, the rest was never attempted — still exits 0. + public var hasFailure: Bool { + [prompt, transport, model].contains(.failed) + } +} diff --git a/Sources/DialecticSession/main.swift b/Sources/DialecticSession/main.swift index fe24064..0c0c5f9 100644 --- a/Sources/DialecticSession/main.swift +++ b/Sources/DialecticSession/main.swift @@ -38,11 +38,10 @@ if opts.runtimeReport { RuntimeReport.render(resolved: resolved, runtimeName: opts.runtime, model: opts.model) let v = RuntimeReport.verify(resolved: resolved) print("") - print("Verification:") - print(" prompt loaded: \(v.promptLoaded ? "yes" : "no")") - print(" transport reachable: \(v.transportOK ? "yes" : "no")") - print(" model available: \(v.modelAvailable ? "yes" : "no")") - exit(v.promptLoaded && v.transportOK && v.modelAvailable ? 0 : 1) + RuntimeReport.verificationLines(v).forEach { print($0) } + // `notChecked` is an absence of evidence, not a failure, so it does + // not fail the report — only a check that ran and did not hold does. + exit(v.hasFailure ? 1 : 0) } catch { FileHandle.standardError.write(Data("error: \(error)\n".utf8)) exit(1) @@ -61,20 +60,17 @@ if opts.dryRun { RuntimeReport.render(resolved: resolved, runtimeName: opts.runtime, model: opts.model) let v = RuntimeReport.verify(resolved: resolved) print("") - print("Verification:") - print(" prompt loaded: \(v.promptLoaded ? "yes" : "no")") - print(" transport reachable: \(v.transportOK ? "yes" : "no")") - print(" model available: \(v.modelAvailable ? "yes" : "no")") - if !(v.promptLoaded && v.transportOK && v.modelAvailable) { + RuntimeReport.verificationLines(v).forEach { print($0) } + if v.hasFailure { FileHandle.standardError.write(Data("dry-run failed verification\n".utf8)) exit(1) } print("") - print("✓ runtime created") - print("✓ transport reachable") - print("✓ model exists") - print("✓ prompt loaded") - print("✓ fingerprint generated") + // Only checks that ran get a tick. A Claude dry run configures a + // runtime and loads a prompt; it does not establish that the provider + // answers or that this account may use this model, so it must not + // print those as verified. + RuntimeReport.dryRunSummaryLines(v).forEach { print($0) } print("") print("No LLM call was made.") exit(0) diff --git a/Tests/DialecticSessionTests/RuntimeVerificationTests.swift b/Tests/DialecticSessionTests/RuntimeVerificationTests.swift new file mode 100644 index 0000000..55d8959 --- /dev/null +++ b/Tests/DialecticSessionTests/RuntimeVerificationTests.swift @@ -0,0 +1,124 @@ +import BCICloudBridge +import XCTest +@testable import DialecticSession + +/// The headless counterpart to `RuntimeReadiness`'s configured/ready split. +/// +/// `RuntimeReport.verify` returned `(promptLoaded, true, true)` for Claude +/// while its own comment said no call had been made, so `--dry-run` printed +/// `transport reachable: yes`, `model available: yes`, `✓ transport reachable` +/// and `✓ model exists` on evidence it had never collected. The app path was +/// corrected first; this pins the harness, which is the surface an operator +/// reads when there is no UI. +/// +/// The wording is asserted through `verificationLines` / `dryRunSummaryLines` +/// rather than by capturing stdout. Formatting inline at two `main.swift` call +/// sites is exactly why nothing could assert on these claims before. +final class RuntimeVerificationTests: XCTestCase { + + // MARK: - Fixtures + + /// A PATH holding a stub executable named `claude`. Resolution validates + /// the file and never runs it — enough to construct the runtime, and never + /// enough to verify the provider. + private func claudeEnvironment() throws -> [String: String] { + let dir = FileManager.default.temporaryDirectory + .appendingPathComponent("runtime-verify-\(UUID().uuidString)") + try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) + addTeardownBlock { try? FileManager.default.removeItem(at: dir) } + + let claude = dir.appendingPathComponent("claude") + try "#!/bin/bash\nexit 0\n".write(to: claude, atomically: true, encoding: .utf8) + try FileManager.default.setAttributes( + [.posixPermissions: 0o755], ofItemAtPath: claude.path) + return ["PATH": dir.path] + } + + private func verifyClaude() throws -> RuntimeVerification { + let resolved = try RuntimeFactory.make( + runtimeName: "claude", + model: "claude-sonnet-5", + environment: try claudeEnvironment() + ) + return RuntimeReport.verify(resolved: resolved) + } + + /// What a reachable daemon holding the exact requested model produces. + /// Constructed directly rather than probed: the Ollama branch of `verify` + /// needs a live daemon, and these assertions are about how a fully + /// verified result is *reported*. + private let verifiedOllama = RuntimeVerification( + prompt: .passed, transport: .passed, model: .passed) + + // MARK: - Claude reports absence of evidence, not evidence + + func testClaudeDryRunLeavesProviderAndModelNotChecked() throws { + let v = try verifyClaude() + + XCTAssertEqual(v.prompt, .passed, "the prompt check is real and does run") + XCTAssertEqual(v.transport, .notChecked) + XCTAssertEqual(v.model, .notChecked) + XCTAssertEqual(v.notCheckedReason, "no generation request made") + } + + func testClaudeDryRunNeverPrintsTransportReachableYes() throws { + let lines = RuntimeReport.verificationLines(try verifyClaude()) + let block = lines.joined(separator: "\n") + + let provider = try XCTUnwrap(lines.first { $0.contains("provider reachable") }) + XCTAssertFalse(provider.contains("yes"), provider) + XCTAssertTrue(provider.contains("not checked"), provider) + XCTAssertTrue(provider.contains("no generation request made"), provider) + + // The old wording, in either of its two forms. + XCTAssertFalse(block.contains("transport reachable: yes"), block) + XCTAssertFalse(block.contains("model available: yes"), block) + } + + func testClaudeDryRunNeverPrintsModelExists() throws { + let summary = RuntimeReport.dryRunSummaryLines(try verifyClaude()) + let block = summary.joined(separator: "\n") + + XCTAssertFalse(block.contains("✓ exact model present"), block) + XCTAssertFalse(block.contains("✓ model exists"), block) + XCTAssertFalse(block.contains("✓ endpoint reachable"), block) + XCTAssertFalse(block.contains("✓ transport reachable"), block) + + // What it must say instead: the gap, named. + XCTAssertTrue( + block.contains("– provider and model were not operationally verified"), block) + // The checks that did run keep their ticks. + XCTAssertTrue(block.contains("✓ runtime configured"), block) + XCTAssertTrue(block.contains("✓ prompt loaded"), block) + } + + // MARK: - Ollama keeps its earned claims + + func testOllamaDryRunReportsEndpointAndExactModelVerified() { + let lines = RuntimeReport.verificationLines(verifiedOllama).joined(separator: "\n") + XCTAssertTrue(lines.contains("provider reachable: yes"), lines) + XCTAssertTrue(lines.contains("exact model present: yes"), lines) + XCTAssertFalse(lines.contains("not checked"), lines) + + let summary = RuntimeReport.dryRunSummaryLines(verifiedOllama).joined(separator: "\n") + XCTAssertTrue(summary.contains("✓ endpoint reachable"), summary) + XCTAssertTrue(summary.contains("✓ exact model present"), summary) + XCTAssertFalse(summary.contains("not operationally verified"), summary) + } + + // MARK: - notChecked is not failure + + func testNotCheckedDoesNotFailConfigurationOnlyDryRun() throws { + XCTAssertFalse(try verifyClaude().hasFailure, "an unrun check must not fail the run") + XCTAssertFalse(verifiedOllama.hasFailure) + + // A check that actually ran and did not hold still fails, so relaxing + // the exit condition has not made it unconditional. + XCTAssertTrue( + RuntimeVerification(prompt: .passed, transport: .failed, model: .notChecked) + .hasFailure) + XCTAssertTrue( + RuntimeVerification(prompt: .failed, transport: .notChecked, model: .notChecked) + .hasFailure) + } +} From abb0eea71ca4da6c025664a3f16a554ef708a907 Mon Sep 17 00:00:00 2001 From: aurascoper Date: Mon, 27 Jul 2026 21:49:21 -0500 Subject: [PATCH 10/11] fix(runtime): publish Claude metadata in the live app MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A read-only audit of the 71c5c02 session logs found the packaged app's twelve dialectical turns carried no `generatorFingerprint` and no `witnessGeneratorFingerprint`, while harness runs of the same head carried both. The cause is an app-path asymmetry, established from source rather than inferred from the logs: ClaudeCLIGenerator : TextGenerating (only) GenerationRuntimeTextGeneratingAdapter : MetadataPublishing… (sole conformer) LiveRuntimeFactory.makeClaude → raw generator LiveRuntimeFactory.makeOllama → adapter HypnagogicDialecticLoop.swift:86 fingerprint is nil "when the generator is the legacy TextGenerating conformers" So the app could not durably attribute a Claude turn to its provider, model, or prompt hash, and the same gap hid the Claude Witness's identity. The absence was self-concealing: with no fingerprint there is no record of which provider produced the record. Ollama had provenance; the harness had provenance; the app's Claude path did not. Pre-existing rather than introduced by this PR — the Ollama adapter work is what made it visible. `makeClaude` now routes through the bridge the repository already had: `ClaudeCLIGenerationRuntime` wrapped in `GenerationRuntimeTextGeneratingAdapter`, matching the Ollama path. No second provenance mechanism. Prompt transmission is byte-equivalent. The `promptProfile:` initializer is used so metadata records the real profile instead of `custom`; it re-loads the profile, but `PromptProfile.load()` caches on `cacheKey` and the factory has already populated that entry, so the second load returns the identical `String`. The adapter carries no prompt of its own by construction — a field that once existed there was removed precisely because it let an object report one prompt while sending another. Failed calls publish nothing: metadata is constructed inside `ClaudeCLIGenerationRuntime.generate` only after `transport.send` returns, and the adapter fires `onMetadata` only after `generate` returns, so a throw short-circuits both. A success fingerprint on a failed turn would be the same fabricated evidence commit I removed from the harness. Readiness is untouched — Claude still resolves `.configured`, not `.ready` — and resolution remains side-effect free. `testDefaultResolutionIsStillClaude` asserted `generator is ClaudeCLIGenerator` and now asserts the *capability* instead: the concrete type is an implementation detail, the metadata conformance is the thing that was broken. LiveRuntimeFactoryFailClosedTests 31 executed (was 23) deterministic CI subset 431 executed, 3 skipped, 0 failures A2 app + session 152 executed, 0 failures ./Scripts/build.sh OK git diff --check clean Behavioural tests drive a stub `claude` that prints a canned JSON envelope; no test contacts Anthropic. §5 of the acceptance record documents the correction without rewriting §1–4. The 71c5c02 bundle hashes remain historical evidence for that head; new hashes follow after packaging. Co-Authored-By: Claude Fable 5 --- .../BCICloudBridge/LiveRuntimeFactory.swift | 29 ++- .../LiveRuntimeFactoryFailClosedTests.swift | 186 +++++++++++++++++- .../evaluation/a2-apple-silicon-acceptance.md | 70 +++++++ 3 files changed, 279 insertions(+), 6 deletions(-) diff --git a/Sources/BCICloudBridge/LiveRuntimeFactory.swift b/Sources/BCICloudBridge/LiveRuntimeFactory.swift index ef75d2e..1d1f618 100644 --- a/Sources/BCICloudBridge/LiveRuntimeFactory.swift +++ b/Sources/BCICloudBridge/LiveRuntimeFactory.swift @@ -180,11 +180,30 @@ public enum LiveRuntimeFactory { ) } - let generator: ClaudeCLIGenerator + // A `GenerationRuntime` wrapped in the metadata-publishing adapter — + // the same shape the Ollama path uses — rather than a raw + // `ClaudeCLIGenerator`. + // + // `ClaudeCLIGenerator` conforms only to `TextGenerating`, and + // `HypnagogicDialecticLoop` populates `generatorFingerprint` solely from + // a `MetadataPublishingTextGenerating` (`HypnagogicDialecticLoop.swift:86` + // — nil "when the generator is the legacy TextGenerating conformers"). + // So the packaged app persisted Claude turns with no provider, model, or + // prompt hash, and the same gap hid the Claude Witness's identity. The + // absence was self-concealing: with no fingerprint there is no record of + // which provider produced the record. + // + // The `promptProfile:` initializer (not `systemPrompt:`) so the metadata + // records the real profile instead of `custom`. It re-loads the profile, + // which is byte-identical by construction: `PromptProfile.load()` caches + // on `cacheKey`, and `promptText` above has already populated that entry, + // so this is a cache hit on the same `String`. The transmitted bytes are + // unchanged — the adapter deliberately carries no prompt of its own. + let runtime: ClaudeCLIGenerationRuntime do { - generator = try ClaudeCLIGenerator( + runtime = try ClaudeCLIGenerationRuntime( model: requestedModel, - systemPrompt: promptText, + promptProfile: profile, executablePath: executablePath ) } catch { @@ -194,6 +213,10 @@ public enum LiveRuntimeFactory { detail: String(describing: error) ) } + // Metadata is built inside `ClaudeCLIGenerationRuntime.generate` *after* + // `transport.send` returns, and the adapter fires `onMetadata` only + // after `runtime.generate` returns — so a failed call publishes nothing. + let generator = GenerationRuntimeTextGeneratingAdapter(runtime: runtime) return (generator, ResolvedRuntimeIdentity( role: role, diff --git a/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift b/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift index 92e9432..b612fa6 100644 --- a/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift +++ b/Tests/BCICloudBridgeTests/LiveRuntimeFactoryFailClosedTests.swift @@ -12,8 +12,10 @@ import XCTest /// mistyped `NEURALCOMPOSE_RUNTIME` — set by a user who wanted local-only /// inference — silently produced cloud egress to Anthropic instead. /// -/// **No test here makes a real provider request.** Claude resolution stops at -/// executable validation; Ollama readiness is served by `MockURLProtocol`. +/// **No test here makes a real provider request.** Most Claude tests stop at +/// executable validation; the commit-J provenance tests additionally invoke a +/// stub `claude` that prints a canned JSON envelope, which never leaves the +/// machine. Ollama readiness is served by `MockURLProtocol`. final class LiveRuntimeFactoryFailClosedTests: XCTestCase { private let ollamaURL = URL(string: "http://localhost:11434")! @@ -371,7 +373,170 @@ final class LiveRuntimeFactoryFailClosedTests: XCTestCase { XCTAssertEqual(identity.resolvedProvider, "claude") XCTAssertEqual(identity.resolvedModel, "claude-sonnet-5") XCTAssertFalse(identity.isSubstitution) - XCTAssertTrue(generator is ClaudeCLIGenerator) + // The Claude path now returns a metadata-publishing adapter, not a raw + // `ClaudeCLIGenerator`. Asserting the concrete adapter type would pin an + // implementation detail; asserting the *capability* pins the thing that + // was actually broken — see `testClaudeGeneratorPublishesMetadata`. + XCTAssertTrue(generator is any MetadataPublishingTextGenerating) + } + + // MARK: - App-side Claude provenance (commit J) + + /// The regression guard. Returning a raw `ClaudeCLIGenerator` here — as the + /// factory did through `71c5c02` — makes this fail, because that type + /// conforms only to `TextGenerating`. + /// + /// `HypnagogicDialecticLoop` populates `generatorFingerprint` *solely* from a + /// `MetadataPublishingTextGenerating` (`HypnagogicDialecticLoop.swift:86`), so + /// without this conformance the packaged app persisted Claude turns with no + /// provider, model, or prompt hash — and the same gap hid the Claude + /// Witness's identity. The Ollama path had it; the harness had it; the app's + /// Claude path did not. + func testClaudeGeneratorPublishesMetadata() async throws { + let environment = try makeClaudeEnvironment() + let (generator, _) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: nil, environment: environment) + + XCTAssertTrue( + generator is any MetadataPublishingTextGenerating, + "a Claude generator that cannot publish metadata leaves every app-side " + + "turn with no durable provider/model/prompt identity") + XCTAssertFalse( + generator is ClaudeCLIGenerator, + "the raw generator is the defect this commit fixes") + } + + /// Readiness is unchanged by the provenance fix: resolution still contacts + /// nothing, so it is still `configured` rather than `ready`. + func testClaudeReadinessStaysConfiguredAfterMetadataChange() async throws { + let environment = try makeClaudeEnvironment() + let (_, identity) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: nil, environment: environment) + XCTAssertEqual(identity.readiness, .configured) + XCTAssertFalse(identity.isReady) + XCTAssertTrue(identity.canAttemptGeneration) + XCTAssertEqual(identity.locality, .localBrokerToRemoteService) + } + + /// The identity is a description of what *resolution* proved and must not be + /// rewritten by a later successful call. Constructing the runtime does not + /// mutate it, and nothing in the adapter path writes back to it. + func testResolvedIdentityIsUnchangedByConstruction() async throws { + let environment = try makeClaudeEnvironment() + let (_, first) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: nil, environment: environment) + let (_, second) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: nil, environment: environment) + XCTAssertEqual(first, second, "resolution must be deterministic and side-effect free") + } + + /// Role selects the prompt profile, and the profile the runtime hashes must + /// be the one the identity reports — the Witness must not be able to resolve + /// to the poles' prompt. + func testWitnessRoleResolvesADifferentPromptThanTheDialecticPole() async throws { + let environment = try makeClaudeEnvironment() + let (_, pole) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: nil, environment: environment) + let (_, witness) = try await LiveRuntimeFactory.make( + role: .witness, runtimeName: "claude", model: nil, environment: environment) + + XCTAssertNotEqual(pole.promptProfile, witness.promptProfile) + XCTAssertNotEqual( + pole.promptHash, witness.promptHash, + "a shared pole/Witness prompt hash is the exact collision the Witness " + + "fingerprint exists to make impossible") + } + + /// Both roles publish metadata, so a Reflective turn can record the pole and + /// the Witness fingerprints independently. + func testBothClaudeRolesPublishMetadata() async throws { + let environment = try makeClaudeEnvironment() + for role in [RuntimeRole.dialectic, .witness, .mirror] { + let (generator, _) = try await LiveRuntimeFactory.make( + role: role, runtimeName: "claude", model: nil, environment: environment) + XCTAssertTrue( + generator is any MetadataPublishingTextGenerating, + "role \(role) must be able to attest its own generator") + } + } + + /// A PATH with a stub `claude` that emits `body` on stdout. Nothing here + /// contacts Anthropic; the transport only parses the envelope. + private func makeStubClaudeEnvironment(emitting body: String) throws -> [String: String] { + let dir = try makeDirectory() + let claude = dir.appendingPathComponent("claude") + // `cat <<'EOF'` so the JSON is emitted verbatim regardless of quoting. + try "#!/bin/bash\ncat <<'ENVELOPE'\n\(body)\nENVELOPE\n" + .write(to: claude, atomically: true, encoding: .utf8) + try FileManager.default.setAttributes( + [.posixPermissions: 0o755], ofItemAtPath: claude.path) + return ["PATH": dir.path] + } + + /// A successful call publishes the identity the log needs: which runtime, + /// which provider, which model, which profile, and the hash of the bytes + /// that were actually transmitted. + func testSuccessfulClaudeGenerationPublishesMetadata() async throws { + let environment = try makeStubClaudeEnvironment(emitting: #"{"result":"ok"}"#) + let (generator, identity) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: "claude-sonnet-5", + environment: environment) + + let captured = MetadataCapture() + var publisher = try XCTUnwrap(generator as? any MetadataPublishingTextGenerating) + publisher.onMetadata = { captured.record($0) } + + _ = try await publisher.generate( + prompt: "p", maxTokens: 32, temperature: 0.7, cancellationID: UUID()) + + let m = try XCTUnwrap(captured.value, "a successful call must publish metadata") + XCTAssertEqual(m.runtime, "claude-cli") + XCTAssertEqual(m.transport, "claude-cli") + XCTAssertEqual(m.provider, "anthropic") + XCTAssertEqual(m.model, "claude-sonnet-5") + // The real profile, not `custom` — `custom` would tell a later reader + // nothing about which prompt governed the turn. + XCTAssertEqual(m.promptProfile, LiveRuntimeFactory.promptProfile(for: .dialectic).rawValue) + // The hash attests the bytes transmitted, and agrees with the identity + // resolution reported before any call was made. + XCTAssertEqual( + m.promptHash, + try LiveRuntimeFactory.promptProfile(for: .dialectic).hash()) + XCTAssertEqual(m.promptHash, identity.promptHash) + } + + /// A failed call must publish nothing. Metadata is built inside + /// `ClaudeCLIGenerationRuntime.generate` only after `transport.send` + /// returns, and the adapter fires `onMetadata` only after `generate` + /// returns — so a throw short-circuits both. A success fingerprint on a + /// failed turn would be exactly the fabricated evidence commit I removed + /// from the harness. + func testFailedClaudeGenerationPublishesNoMetadata() async throws { + let environment = try makeStubClaudeEnvironment(emitting: #"{"is_error":true}"#) + let (generator, _) = try await LiveRuntimeFactory.make( + role: .dialectic, runtimeName: "claude", model: "claude-sonnet-5", + environment: environment) + + let captured = MetadataCapture() + var publisher = try XCTUnwrap(generator as? any MetadataPublishingTextGenerating) + publisher.onMetadata = { captured.record($0) } + + do { + _ = try await publisher.generate( + prompt: "p", maxTokens: 32, temperature: 0.7, cancellationID: UUID()) + XCTFail("a CLI-reported error must propagate, not resolve") + } catch { + // expected + } + XCTAssertNil(captured.value, "a failed generation must publish no metadata") + } + + /// Ollama already published metadata; this fix must not disturb it. + func testOllamaMetadataPublicationIsUnchanged() async throws { + stubTags([("qwen2.5:0.5b", "sha256:deadbeef")]) + let (generator, identity) = try await makeOllama(model: "qwen2.5:0.5b") + XCTAssertTrue(generator is any MetadataPublishingTextGenerating) + XCTAssertEqual(identity.readiness, .ready) } // MARK: - Readiness evidence: configured vs verified @@ -447,3 +612,18 @@ final class LiveRuntimeFactoryFailClosedTests: XCTestCase { .unavailable(.modelMissing)) } } + +/// Thread-safe one-shot capture for the adapter's metadata callback, which +/// fires synchronously on whichever task called `generate`. +private final class MetadataCapture: @unchecked Sendable { + private let lock = NSLock() + private var stored: GenerationMetadata? + var value: GenerationMetadata? { + lock.lock(); defer { lock.unlock() } + return stored + } + func record(_ m: GenerationMetadata) { + lock.lock(); defer { lock.unlock() } + stored = m + } +} diff --git a/docs/evaluation/a2-apple-silicon-acceptance.md b/docs/evaluation/a2-apple-silicon-acceptance.md index 63c0390..1b207d3 100644 --- a/docs/evaluation/a2-apple-silicon-acceptance.md +++ b/docs/evaluation/a2-apple-silicon-acceptance.md @@ -138,3 +138,73 @@ Per the A2 report's acceptance plan; none of the below is claimed. pre-B Reflective runs `status: superseded` (dead-prompt-field defect, `fixed_by: 3067cd8`) and rerun the Focused-vs-Reflective fixture so the replacement captures dialogue + Witness fingerprints. + +## 5. Commit J — app-side Claude provenance (correction, not a rewrite) + +Sections 1–4 stand as written. They record what was observed at the heads they +name, and nothing below revises them. + +### What the log audit found + +A read-only review of the `71c5c02` session logs +(`NeuralCompose-a2-handoff/a2-final/sonnet5-dialectic-log-review.md`) found the +day-file's twelve dialectical turns carried **no `generatorFingerprint` and no +`witnessGeneratorFingerprint`**, while the harness runs of the same head carried +both. The cause was an app-path asymmetry, established from source rather than +inferred: + +- `ClaudeCLIGenerator` conforms only to `TextGenerating`; +- `GenerationRuntimeTextGeneratingAdapter` is the sole + `MetadataPublishingTextGenerating` conformer; +- `LiveRuntimeFactory.makeClaude` returned the raw generator, while + `makeOllama` returned the adapter; +- `HypnagogicDialecticLoop` populates `generatorFingerprint` *only* from a + metadata-publishing generator (`HypnagogicDialecticLoop.swift:86`). + +So the packaged app could not durably attribute a Claude turn to its provider, +model, or prompt hash, and the same gap hid the Claude Witness's identity. The +absence was self-concealing: with no fingerprint there is no record of which +provider produced the record. Pre-existing rather than introduced by this PR — +the Ollama adapter work made it visible. + +### The correction + +Commit J routes `makeClaude` through the bridge the repository already had: +`ClaudeCLIGenerationRuntime` wrapped in `GenerationRuntimeTextGeneratingAdapter`, +matching the Ollama path. No second provenance mechanism was introduced. + +Prompt bytes are unchanged. The `promptProfile:` initializer re-loads the +profile so metadata records the real profile rather than `custom`; that load is +a cache hit on the entry the factory has already populated +(`PromptProfile.load()` caches on `cacheKey`), so it returns the identical +`String`. The adapter carries no prompt of its own by construction. + +Readiness semantics are untouched: Claude still resolves `.configured`, not +`.ready`, and a successful call does not mutate `ResolvedRuntimeIdentity`. + +Failed calls publish nothing. Metadata is built inside +`ClaudeCLIGenerationRuntime.generate` only after `transport.send` returns, and +the adapter fires `onMetadata` only after `generate` returns, so a throw +short-circuits both. + +### Evidence status + +- The `71c5c02` bundle hashes in §1 remain **historical evidence for that head**. + Commit J necessarily produces a new bundle and new hashes. +- New final-head hashes and the targeted rerun below are appended after packaging. +- Admissible from the prior record: harness provider/model/prompt provenance; + Reflective-only Witness operation; zero pole/Witness prompt-hash collisions; + a failed Claude generation writing zero records; and `spectralState: null` + disambiguating the stub `glossScalar` from a real reading. +- **Not claimed:** Focused, Contemplative, or Mirror behavioural fidelity. The + available sessions were one turn per profile, and every knob distinguishing + those profiles governs sustained behaviour. + +### Recorded limitations (unchanged by this commit) + +1. **Mirror has no durable telemetry.** `HypnagogicDialogueLoop` emits no + `DialecticalTurnEvent`, which is architecturally correct — Mirror is not a + `ContextProfile` — but leaves it observable only through an operator. +2. **No event-to-build self-attribution.** `DialecticalTurnEvent` carries no + timestamp and no build identifier, so a log cannot independently attest which + bundle produced it; attribution rests on filesystem metadata. From 0dfae3ce35230b79deaee1da074f9313e63e55c8 Mon Sep 17 00:00:00 2001 From: aurascoper Date: Mon, 27 Jul 2026 23:18:28 -0500 Subject: [PATCH 11/11] docs(evaluation): record A2 final-head acceptance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Observed packaged evidence for abb0eea, appended as §6. Sections 1–5 and the 4c5e275 / 71c5c02 observations are unchanged. The day-file spans both bundles, so the Commit J change is directly visible rather than argued: written by 71c5c02 16 records 0 pole fingerprints written by abb0eea 12 records 12 pole fingerprints Cells: Claude Focused, Reflective and Contemplative PASS; Ollama Reflective regression PASS across four turns; configured-but-failing Claude PASS with zero records appended. Prompt-hash collisions 0; no provider substitution; Witness finding never voiced; Witness fingerprint present only for Reflective and independently recording its own profile. The failing-Claude cell was gated on a health-log advance before its zero delta could count as a pass — its expected result is that nothing is written, so "nothing changed" and "nothing ran" would otherwise be the same observation. Mirror is recorded as not established rather than passed or failed. Selection and loop construction are confirmed by loopMode/loopRunning; nothing after that leaves durable evidence, because turnCount and lastTurnAt are populated only for HypnagogicDialecticLoop and HypnagogicDialogueLoop's catch swallows failures without logging. Both predate Commit J, and the same adapter produced twelve fingerprinted turns in the same process, so it is not attributable to this change. No behavioural-fidelity claim is made for Focused, Contemplative, or Mirror. Co-Authored-By: Claude Fable 5 --- .../evaluation/a2-apple-silicon-acceptance.md | 79 +++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/docs/evaluation/a2-apple-silicon-acceptance.md b/docs/evaluation/a2-apple-silicon-acceptance.md index 1b207d3..b338ef8 100644 --- a/docs/evaluation/a2-apple-silicon-acceptance.md +++ b/docs/evaluation/a2-apple-silicon-acceptance.md @@ -208,3 +208,82 @@ short-circuits both. 2. **No event-to-build self-attribution.** `DialecticalTurnEvent` carries no timestamp and no build identifier, so a log cannot independently attest which bundle produced it; attribution rests on filesystem metadata. + +## 6. Final-head acceptance — `abb0eea` (observed, packaged) + +Sections 1–5 stand. The `4c5e275` and `71c5c02` observations are historical +records of the heads they name and are not revised here. + +### Artifact + +``` +HEAD abb0eea71ca4da6c025664a3f16a554ef708a907 worktree clean +executable sha256 2e283a40797e1780a7debde2f3466b9658a5ae264285c6084a25ba25090bd11d +Info.plist sha256 bbbd77168cea02521a50e9ace249dbab8ea08b7ce16fb5aa257af512e98894d4 +codesign --verify --deep --strict PASS · adhoc · TeamIdentifier not set +``` + +Re-verified byte-identical before and after every cell below. The `71c5c02` +hashes in §1 remain that head's historical evidence. + +### The Commit J signal + +The same day-file spans both bundles, which makes the change directly visible: + +``` +written by 71c5c02 16 records 0 pole fingerprints +written by abb0eea 12 records 12 pole fingerprints +``` + +Zero of sixteen before; twelve of twelve after. This is the durable app-side +Claude provenance §5 predicted from source, now observed in the packaged app. + +### Cells + +| cell | result | evidence | +|---|---|---| +| Claude Focused | PASS | pole fp; `anthropic` / `claude-sonnet-5`; profile `wakingDialectical`; `witnessAttempted=false`; no Witness fp | +| Claude Reflective | PASS | pole fp **and** Witness fp; Witness independently records profile `witness`; hashes differ; finding unvoiced | +| Claude Contemplative | PASS | pole fp; `witnessAttempted=false`; no Witness fp | +| Ollama Reflective regression | PASS | 4 turns, `ollama` / `qwen2.5:0.5b`, transport `ollama-http`; pole **and** Witness fp on every turn; hashes differ; unvoiced | +| Configured-but-failing Claude | PASS | app launch confirmed via health-log advance; **zero** records, pole fps, Witness fps appended; no stale speech | +| Claude Mirror | not established | see limitations | + +Prompt-hash collisions across all 12 new records: **0**. No provider +substitution: requested equals resolved in every cell. No fabricated +`modelDigest`. `spectralState: null` continues to disambiguate the stub +`glossScalar 0.5` from an observed value. + +The failing-Claude cell is the load-bearing negative: its expected result is that +nothing is written, so it was gated on independent evidence that the app ran +(health-log delta) before a zero delta could be read as a pass. + +### Recorded limitations + +1. **Mirror operational path — not established.** `loopMode=mirror` with + `loopRunning=true` confirms selection and loop construction; no stage beyond + that leaves durable evidence. `AppViewModel.swift:1170` populates `turnCount` + and `lastTurnAt` only via `as? HypnagogicDialecticLoop`, so Mirror is + invisible to those counters, and `HypnagogicDialogueLoop.run()`'s catch + swallows listen/generate/speak failures without logging. A silent success and + a silent failure are indistinguishable from the artifacts that exist. Both + properties predate Commit J, and Mirror's loop takes `any TextGenerating` + with no concrete-type assumption — the same adapter produced twelve + fingerprinted turns in the same process. **Not attributable to Commit J.** +2. **No event self-attestation.** `DialecticalTurnEvent` carries no timestamp + and no build identifier. Association of records to this head rests on the + frozen hash, the `loopMode` timeline in the health log, and the fingerprint + discontinuity — not on any single event. +3. **Ollama records `promptProfile: custom`** where Claude now records the real + profile name. `makeOllama` uses the caller-supplied-bytes initializer, so the + hash is correct and provenance is sound; the record is simply less + self-describing. Post-merge cleanup. +4. **Mode fidelity remains underpowered.** One to four turns per profile. +5. **Spectral estimator absent** in a SwiftPM build; gloss pinned 0.5. + +### Not claimed + +Focused, Contemplative, and Mirror behavioural fidelity. Every knob that +distinguishes those profiles governs sustained behaviour — silence runs, +carry-forward, synthesis reluctance, cadence — and no session here is long +enough to reach one. Nothing in this record is EEG evidence.