From 3f6e9699ab5a589f1dd5da4e3e3ac8ccdd5a76c2 Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Tue, 25 Aug 2026 16:31:23 +0900 Subject: [PATCH 01/14] chore: track issue 81 remediation --- Docs/issue-remediation-progress.md | 47 ++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 Docs/issue-remediation-progress.md diff --git a/Docs/issue-remediation-progress.md b/Docs/issue-remediation-progress.md new file mode 100644 index 0000000..450ea92 --- /dev/null +++ b/Docs/issue-remediation-progress.md @@ -0,0 +1,47 @@ +# Issue remediation progress + +Base: `main` at `88f36e65223f874e8ce13fa4846ef517f1203146` + +## Delivery order + +1. #81 actionable bounded raw-helper crash diagnostics +2. #83 bounded Objective-C table and loaded-image reads + +Each issue is delivered as an independent Ready PR targeting `main`. The next +issue starts only after the current PR is review-clean and merged. + +## Current issue: #81 + +Branch: `codex/issue-81-actionable-crash-diagnostics` + +Verified evidence: + +- Signal-only helper failures currently persist only a generic termination + sentence, without the child process identity needed to correlate an OS + incident report. +- Long uncaught-exception output retains only the final eight nonempty lines, + which discards the exception name/reason and first relevant frames. +- Successful helper diagnostics already use a typed report and must remain + separate from arbitrary process output. + +Design gate pending: + +- Identify the single owner that observes ordered helper output, process + identity, termination reason, and terminal time. +- Define a strict byte/line bound that preserves an actionable prefix and the + termination tail without retaining unbounded output. +- Carry one failure capsule through raw dumping, persistence, and rendering + without introducing mirror state or a second source of truth. +- Prove correlation fields against real Crash Reporter metadata without + persisting user-private paths. + +Required validation: + +- Long exception fixture retains the exception name/reason and first relevant + frame plus the terminal tail. +- Signal-only fixture explicitly states that no process diagnostic was emitted + and records the exact child identity/timing needed for correlation. +- Bounds hold for long lines, invalid UTF-8, interleaved streams, and high + output volume. +- The capsule survives through `runTargets.failureSummary` and both terminal + and nonterminal failed-target rendering. From 46cb518261143f266d522947a353b43d99dc9f34 Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Tue, 25 Aug 2026 16:42:31 +0900 Subject: [PATCH 02/14] docs: approve issue 81 design gate --- Docs/issue-remediation-progress.md | 40 ++++++++++++++++++++++-------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/Docs/issue-remediation-progress.md b/Docs/issue-remediation-progress.md index 450ea92..e989346 100644 --- a/Docs/issue-remediation-progress.md +++ b/Docs/issue-remediation-progress.md @@ -24,16 +24,36 @@ Verified evidence: - Successful helper diagnostics already use a typed report and must remain separate from arbitrary process output. -Design gate pending: - -- Identify the single owner that observes ordered helper output, process - identity, termination reason, and terminal time. -- Define a strict byte/line bound that preserves an actionable prefix and the - termination tail without retaining unbounded output. -- Carry one failure capsule through raw dumping, persistence, and rendering - without introducing mirror state or a second source of truth. -- Prove correlation fields against real Crash Reporter metadata without - persisting user-private paths. +Design gate approved: + +- No one process can observe every correlation fact for Simulator execution: + `ProcessRunner` owns the `xcrun simctl spawn` wrapper transcript and terminal + observation, while the raw helper owns its actual PID and loaded image. +- The helper writes a separate, invocation-authenticated startup handshake + before loading target metadata. It contains only schema/invocation identity, + actual PID, executable name and LC_UUID, producer version, and Unix epoch + start microseconds. It is atomic, at most 2 KiB, and contains no path, device + UDID, command, environment, or runtime root. +- The diagnostics report remains a completed typed-diagnostics contract. It is + not converted into a two-phase process-state file. +- One bounded process-output value owns combined-stream ordering, head/tail + retention, line and byte omission counts, terminal-safe rendering, and the + inclusive output ceiling. Synthetic termination text is not classified as + process-emitted output. +- `runPrivateHeaderKitRawDump` is the only failure-capsule builder because it + knows execution mode and receives the helper handshake, bounded transcript, + and wrapper termination. The capsule has at most 18 lines and 24 KiB, keeps + the first and last eight diagnostic lines, and ends with one canonical + concise headline. +- The capsule is persisted unchanged in the existing + `runTargets.failureSummary`; no DB column or migration is added. Existing + executor, resume, store, and final-summary paths remain the single transport. +- The current-process LC_UUID primitive moves to + `PrivateHeaderKitExecutableResolution`, which is already shared by Tooling + and RawDumpCore; the Mach-O walk is not duplicated. +- Crash Reporter correlation uses PID, executable UUID/name, helper start, + capture time, termination observation, and signal when available. Incident + ID is assigned after a crash and is therefore not guessed at run time. Required validation: From c73d83f58f52a4634b040c941b72087183def13e Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Tue, 25 Aug 2026 16:59:56 +0900 Subject: [PATCH 03/14] fix(tooling): retain bounded process output --- .../ProcessRunner.swift | 455 +++++++++++++++--- .../main.swift | 24 + .../StreamingProcessRunnerTests.swift | 185 +++++-- 3 files changed, 547 insertions(+), 117 deletions(-) diff --git a/Sources/PrivateHeaderKitTooling/ProcessRunner.swift b/Sources/PrivateHeaderKitTooling/ProcessRunner.swift index 7099a5d..29e7394 100644 --- a/Sources/PrivateHeaderKitTooling/ProcessRunner.swift +++ b/Sources/PrivateHeaderKitTooling/ProcessRunner.swift @@ -19,7 +19,7 @@ struct ProcessGroupTeardownError: Error, Sendable, CustomStringConvertible { private enum CapturedCommandStreamResult: Sendable { case standardOutputComplete - case standardErrorComplete([String]) + case standardErrorComplete(BoundedProcessOutput) } struct WaitableProcessGroupLeader: Sendable { @@ -293,15 +293,125 @@ private func withOwnedProcessGroup< } #endif +/// A terminal-safe, strictly bounded rendering of bytes emitted by a subprocess. +/// +/// Termination and empty-output descriptions belong to ``StreamingCommandResult`` and are not +/// represented as subprocess output here. +public struct BoundedProcessOutput: Equatable, Sendable { + public static let maximumHeadLineCount = 8 + public static let maximumTailLineCount = 8 + public static let maximumRenderedLineByteCount = 1_024 + public static let maximumRenderedLineCount = 17 + public static let maximumRenderedByteCount = 18 * 1_024 + + public let lines: [String] + public let omittedLineCount: UInt + public let omittedByteCount: UInt + + public init(lines: [String]) { + var collector = BoundedProcessOutputCollector() + for line in lines { + collector.consume(logicalLine: line) + } + self = collector.finish() + } + + public var isEmpty: Bool { lines.isEmpty } + + public var text: String { + lines.joined(separator: "\n") + } + + fileprivate init( + lines: [String], + omittedLineCount: UInt, + omittedByteCount: UInt + ) { + precondition( + lines.count <= Self.maximumRenderedLineCount, + "bounded process output exceeded its line count" + ) + precondition( + lines.allSatisfy { $0.utf8.count <= Self.maximumRenderedLineByteCount }, + "bounded process output exceeded its per-line byte count" + ) + precondition( + lines.joined(separator: "\n").utf8.count <= Self.maximumRenderedByteCount, + "bounded process output exceeded its rendered byte count" + ) + precondition( + lines.allSatisfy { terminalSafeProcessOutput($0) == $0 }, + "bounded process output contains terminal-unsafe text" + ) + self.lines = lines + self.omittedLineCount = omittedLineCount + self.omittedByteCount = omittedByteCount + } +} + public struct StreamingCommandResult: Equatable, Sendable { public let status: Int32 public let wasKilled: Bool - public let lastLines: [String] + /// Output emitted by the process. Synthetic termination text is available through + /// ``diagnosticLines`` and ``diagnosticText`` instead. + public let emittedOutput: BoundedProcessOutput + public let terminationObservedAtUnixEpochMicroseconds: UInt64? + + public init( + status: Int32, + wasKilled: Bool, + lastLines: [String], + terminationObservedAtUnixEpochMicroseconds: UInt64? = nil + ) { + self.init( + status: status, + wasKilled: wasKilled, + emittedOutput: BoundedProcessOutput(lines: lastLines), + terminationObservedAtUnixEpochMicroseconds: + terminationObservedAtUnixEpochMicroseconds + ) + } - public init(status: Int32, wasKilled: Bool, lastLines: [String]) { + public init( + status: Int32, + wasKilled: Bool, + emittedOutput: BoundedProcessOutput, + terminationObservedAtUnixEpochMicroseconds: UInt64? = nil + ) { self.status = status self.wasKilled = wasKilled - self.lastLines = lastLines + self.emittedOutput = emittedOutput + self.terminationObservedAtUnixEpochMicroseconds = + terminationObservedAtUnixEpochMicroseconds + } + + public var diagnosticLines: [String] { + guard status != 0 || wasKilled else { return emittedOutput.lines } + var lines = emittedOutput.lines + if emittedOutput.isEmpty { + lines.append("No process diagnostic was emitted.") + } + if wasKilled { + lines.append("Terminated by signal \(status)") + } else { + lines.append("Exited with status \(status)") + } + return lines + } + + public var diagnosticText: String { + diagnosticLines.joined(separator: "\n") + } + + /// Compatibility view preserving the previous synthetic termination rules. + public var lastLines: [String] { + var lines = emittedOutput.lines + if wasKilled { + lines.append("Terminated by signal \(status)") + } else if status != 0, emittedOutput.isEmpty { + lines.append("Exited with status \(status)") + } + return lines } } @@ -339,7 +449,23 @@ public extension CommandRunning { } public struct ProcessRunner: CommandRunning, Sendable { - public init() {} + private let terminationObservationClock: @Sendable () throws -> UInt64 + + public init() { +#if os(macOS) + self.terminationObservationClock = unixEpochMicrosecondsFromRealtimeClock +#else + self.terminationObservationClock = { + throw ToolingError.unsupported("process execution is not available on this platform") + } +#endif + } + + init( + terminationObservationClock: @escaping @Sendable () throws -> UInt64 + ) { + self.terminationObservationClock = terminationObservationClock + } #if os(macOS) public func runCapture( @@ -435,39 +561,36 @@ public struct ProcessRunner: CommandRunning, Sendable { return .standardOutputComplete } group.addTask { - var collector = StreamingOutputCollector() + var collector = BoundedProcessOutputCollector() for try await buffer in execution.standardError { try Task.checkCancellation() collector.consume(buffer.withUnsafeBytes { Array($0) }) } - collector.finish() - return .standardErrorComplete(collector.lastLines) + return .standardErrorComplete(collector.finish()) } - var standardErrorLines: [String] = [] + var standardErrorOutput = BoundedProcessOutput(lines: []) while let streamResult = try await group.next() { - if case .standardErrorComplete(let lines) = streamResult { - standardErrorLines = lines + if case .standardErrorComplete(let output) = streamResult { + standardErrorOutput = output } } - return standardErrorLines + return standardErrorOutput } } } try Task.checkCancellation() let termination = commandTermination(result.terminationStatus) - var standardErrorLines = result.closureResult - if termination.wasKilled { - appendLastLine( - "Terminated by signal \(termination.status)", - to: &standardErrorLines - ) - } + let commandResult = StreamingCommandResult( + status: termination.status, + wasKilled: termination.wasKilled, + emittedOutput: result.closureResult + ) guard termination.status == 0, !termination.wasKilled else { throw ToolingError.commandFailed( command: command, status: termination.status, - stderr: standardErrorLines.joined(separator: "\n") + stderr: commandResult.lastLines.joined(separator: "\n") ) } return @@ -599,7 +722,7 @@ public struct ProcessRunner: CommandRunning, Sendable { error: .combinedWithOutput ) { execution in try await withOwnedProcessGroup(execution: execution) { - var collector = StreamingOutputCollector() + var collector = BoundedProcessOutputCollector() for try await buffer in execution.standardOutput { let bytes = buffer.withUnsafeBytes { Array($0) } if streamOutput { @@ -607,26 +730,18 @@ public struct ProcessRunner: CommandRunning, Sendable { } collector.consume(bytes) } - collector.finish() - return collector.lastLines + return collector.finish() } } // See runCapture: cancellation is observed after group completion. try Task.checkCancellation() let termination = commandTermination(result.terminationStatus) - var lastLines = result.closureResult - if termination.wasKilled { - appendLastLine( - "Terminated by signal \(termination.status)", - to: &lastLines - ) - } else if termination.status != 0, lastLines.isEmpty { - appendLastLine("Exited with status \(termination.status)", to: &lastLines) - } return StreamingCommandResult( status: termination.status, wasKilled: termination.wasKilled, - lastLines: lastLines + emittedOutput: result.closureResult, + terminationObservedAtUnixEpochMicroseconds: + try terminationObservationClock() ) } catch is CancellationError { throw CancellationError() @@ -979,78 +1094,270 @@ final class CancellableStandardOutputWriter: @unchecked Sendable { } } -struct StreamingOutputCollector { - static let maximumLineByteCount = 128 * 1024 +#endif - private var pendingBytes: [UInt8] = [] - private var discardedPendingByteCount = 0 - private(set) var lastLines: [String] = [] +struct BoundedProcessOutputCollector { + private var pendingLine = BoundedProcessLineBytes() + private var headLines: [BoundedProcessLine] = [] + private var tailLines: [BoundedProcessLine] = [] + private var omittedLineCount: UInt = 0 + private var omittedByteCount: UInt = 0 + private var isFinished = false mutating func consume(_ bytes: [UInt8]) { + precondition(!isFinished, "cannot consume process output after finishing") var segmentStart = bytes.startIndex for index in bytes.indices where bytes[index] == UInt8(ascii: "\n") { - appendPending(bytes[segmentStart.. 0 { + mutating func consume(logicalLine: String) { + consume(Array(logicalLine.utf8)) + consume([UInt8(ascii: "\n")]) + } + + mutating func finish() -> BoundedProcessOutput { + precondition(!isFinished, "process output collector finished more than once") + if !pendingLine.isEmpty { consumePendingLine() } - pendingBytes.removeAll(keepingCapacity: false) + isFinished = true + + var lines = headLines.map(\.text) + if omittedLineCount > 0 || omittedByteCount > 0 { + lines.append("[omitted \(omittedLineCount) lines and \(omittedByteCount) bytes]") + } + lines += tailLines.map(\.text) + return BoundedProcessOutput( + lines: lines, + omittedLineCount: omittedLineCount, + omittedByteCount: omittedByteCount + ) + } + + private mutating func consumePendingLine() { + let line = pendingLine.finish() + pendingLine = BoundedProcessLineBytes() + guard let line else { return } + append(line) } - private mutating func appendPending(_ bytes: ArraySlice) { + private mutating func append(_ line: BoundedProcessLine) { + omittedByteCount = saturatingSum(omittedByteCount, line.omittedByteCount) + if headLines.count < BoundedProcessOutput.maximumHeadLineCount { + headLines.append(line) + return + } + + tailLines.append(line) + guard tailLines.count > BoundedProcessOutput.maximumTailLineCount else { return } + let omittedLine = tailLines.removeFirst() + omittedLineCount = saturatingSum(omittedLineCount, 1) + omittedByteCount = saturatingSum( + omittedByteCount, + omittedLine.retainedSourceByteCount + ) + } +} + +private struct BoundedProcessLineBytes { + // Terminal escaping can expand one source scalar substantially. Retaining larger raw edges + // keeps enough source material to fill both rendered halves without retaining the whole line. + private static let edgeByteCount = + BoundedProcessOutput.maximumRenderedLineByteCount * 8 + + private var head: [UInt8] = [] + private var tail: [UInt8] = [] + private var totalByteCount: UInt = 0 + + var isEmpty: Bool { totalByteCount == 0 } + + mutating func append(_ bytes: ArraySlice) { guard !bytes.isEmpty else { return } + totalByteCount = saturatingSum(totalByteCount, UInt(bytes.count)) + + let headCapacity = max(0, Self.edgeByteCount - head.count) + let headBytes = bytes.prefix(headCapacity) + head.append(contentsOf: headBytes) + let remaining = bytes.dropFirst(headBytes.count) + guard !remaining.isEmpty else { return } - let maximumCount = Self.maximumLineByteCount - if bytes.count >= maximumCount { - discardedPendingByteCount += pendingBytes.count + bytes.count - maximumCount - pendingBytes.removeAll(keepingCapacity: true) - pendingBytes.append(contentsOf: bytes.suffix(maximumCount)) + if remaining.count >= Self.edgeByteCount { + tail = Array(remaining.suffix(Self.edgeByteCount)) } else { - let overflow = max(0, pendingBytes.count + bytes.count - maximumCount) + let overflow = max(0, tail.count + remaining.count - Self.edgeByteCount) if overflow > 0 { - pendingBytes.removeFirst(overflow) - discardedPendingByteCount += overflow + tail.removeFirst(overflow) } - pendingBytes.append(contentsOf: bytes) + tail.append(contentsOf: remaining) } + } - // When the retained suffix begins in the middle of a valid scalar, discard the - // continuation bytes too. Invalid bytes elsewhere still follow String(decoding:)'s - // replacement-character contract. - while discardedPendingByteCount > 0, - let first = pendingBytes.first, - first & 0b1100_0000 == 0b1000_0000 - { - pendingBytes.removeFirst() - discardedPendingByteCount += 1 + mutating func finish() -> BoundedProcessLine? { + let retainedByteCount = UInt(head.count + tail.count) + let discardedRawByteCount = saturatingSubtract(totalByteCount, retainedByteCount) + let headText: String + let tailText: String + if discardedRawByteCount == 0 { + headText = normalizedProcessOutputLine(head + tail) + tailText = "" + } else { + headText = normalizedProcessOutputLine(head) + tailText = normalizedProcessOutputLine(tail) + } + guard !headText.isEmpty || !tailText.isEmpty || discardedRawByteCount > 0 else { + return nil } + return boundedProcessLine( + head: headText, + tail: tailText, + discardedRawByteCount: discardedRawByteCount + ) } +} - private mutating func consumePendingLine() { - var line = String(decoding: pendingBytes, as: UTF8.self) +private struct BoundedProcessLine { + let text: String + let sourceByteCount: UInt + let omittedByteCount: UInt + + var retainedSourceByteCount: UInt { + saturatingSubtract(sourceByteCount, omittedByteCount) + } +} + +private func boundedProcessLine( + head: String, + tail: String, + discardedRawByteCount: UInt +) -> BoundedProcessLine { + let maximumByteCount = BoundedProcessOutput.maximumRenderedLineByteCount + let separator = " … " + let headByteCount = UInt(head.utf8.count) + let tailByteCount = UInt(tail.utf8.count) + let sourceByteCount = saturatingSum( + saturatingSum(headByteCount, tailByteCount), + discardedRawByteCount + ) + + if discardedRawByteCount == 0, tail.isEmpty, head.utf8.count <= maximumByteCount { + return BoundedProcessLine( + text: head, + sourceByteCount: sourceByteCount, + omittedByteCount: 0 + ) + } + + let sourceTail = tail.isEmpty ? head : tail + let contentBudget = maximumByteCount - separator.utf8.count + let headBudget = contentBudget / 2 + let tailBudget = contentBudget - headBudget + let retainedHead = prefixFittingUTF8(head, maximumByteCount: headBudget) + let retainedTail = suffixFittingUTF8(sourceTail, maximumByteCount: tailBudget) + let retainedSourceByteCount = UInt(retainedHead.utf8.count + retainedTail.utf8.count) + let omittedByteCount = saturatingSubtract(sourceByteCount, retainedSourceByteCount) + return BoundedProcessLine( + text: retainedHead + separator + retainedTail, + sourceByteCount: sourceByteCount, + omittedByteCount: omittedByteCount + ) +} + +private func normalizedProcessOutputLine(_ bytes: [UInt8]) -> String { + terminalSafeProcessOutput( + String(decoding: bytes, as: UTF8.self) .trimmingCharacters(in: .whitespacesAndNewlines) - if discardedPendingByteCount > 0 { - line = "[truncated \(discardedPendingByteCount) bytes] \(line)" + ) +} + +private func terminalSafeProcessOutput(_ value: String) -> String { + value.unicodeScalars.reduce(into: "") { result, scalar in + switch scalar.value { + case 0x09: + result += #"\t"# + case 0x0a: + result += #"\n"# + case 0x0d: + result += #"\r"# + default: + switch scalar.properties.generalCategory { + case .control, .format, .lineSeparator, .paragraphSeparator: + result += String(format: #"\u{%04x}"#, scalar.value) + default: + result.unicodeScalars.append(scalar) + } } - pendingBytes.removeAll(keepingCapacity: true) - discardedPendingByteCount = 0 - guard !line.isEmpty else { return } - appendLastLine(line, to: &lastLines) } } -private func appendLastLine(_ line: String, to lines: inout [String]) { - lines.append(line) - if lines.count > 8 { - lines.removeFirst(lines.count - 8) +private func prefixFittingUTF8(_ value: String, maximumByteCount: Int) -> String { + var result = "" + var byteCount = 0 + for scalar in value.unicodeScalars { + let scalarByteCount = scalar.utf8.count + guard byteCount <= maximumByteCount - scalarByteCount else { break } + result.unicodeScalars.append(scalar) + byteCount += scalarByteCount } + return result +} + +private func suffixFittingUTF8(_ value: String, maximumByteCount: Int) -> String { + var scalars: [Unicode.Scalar] = [] + var byteCount = 0 + for scalar in value.unicodeScalars.reversed() { + let scalarByteCount = scalar.utf8.count + guard byteCount <= maximumByteCount - scalarByteCount else { break } + scalars.append(scalar) + byteCount += scalarByteCount + } + var result = "" + for scalar in scalars.reversed() { + result.unicodeScalars.append(scalar) + } + return result +} + +private func saturatingSum(_ lhs: UInt, _ rhs: UInt) -> UInt { + let (sum, overflow) = lhs.addingReportingOverflow(rhs) + return overflow ? UInt.max : sum } + +private func saturatingSubtract(_ lhs: UInt, _ rhs: UInt) -> UInt { + lhs >= rhs ? lhs - rhs : 0 +} + +private func unixEpochMicrosecondsFromRealtimeClock() throws -> UInt64 { +#if os(macOS) + var time = timespec() + guard clock_gettime(CLOCK_REALTIME, &time) == 0 else { + throw ToolingError.message( + "failed to observe process termination time (errno \(errno))" + ) + } + guard time.tv_sec >= 0, time.tv_nsec >= 0 else { + throw ToolingError.message("process termination clock returned a negative value") + } + let seconds = UInt64(time.tv_sec) + let (secondMicroseconds, overflow) = seconds.multipliedReportingOverflow(by: 1_000_000) + guard !overflow else { + throw ToolingError.message("process termination clock exceeded UInt64") + } + let nanosecondMicroseconds = UInt64(time.tv_nsec) / 1_000 + let (result, additionOverflow) = secondMicroseconds.addingReportingOverflow( + nanosecondMicroseconds + ) + guard !additionOverflow else { + throw ToolingError.message("process termination clock exceeded UInt64") + } + return result +#else + throw ToolingError.unsupported("process execution is not available on this platform") #endif +} diff --git a/Tests/PrivateHeaderKitToolingTestHelper/main.swift b/Tests/PrivateHeaderKitToolingTestHelper/main.swift index 3ce4798..3e322d2 100644 --- a/Tests/PrivateHeaderKitToolingTestHelper/main.swift +++ b/Tests/PrivateHeaderKitToolingTestHelper/main.swift @@ -39,6 +39,8 @@ private struct PrivateHeaderKitToolingTestHelper { throw HelperError.invalidCommand(command) } try writeLargeStandardErrorFailure(byteCount: byteCount) + case "long-exception-failure": + try writeLongExceptionFailure() case "chunked-output": try writeChunkedOutput() case "buffered-output": @@ -121,6 +123,27 @@ private struct PrivateHeaderKitToolingTestHelper { exit(19) } + private static func writeLongExceptionFailure() throws -> Never { + var lines = [ + "*** Terminating app due to uncaught exception 'FixtureException', reason: 'fixture reason'", + "*** First throw call stack:", + "(", + "0 CoreFoundation fixture", + "1 libobjc fixture", + "2 PrivateHeaderKitToolingTestHelper frame-zero", + "3 PrivateHeaderKitToolingTestHelper frame-one", + "4 PrivateHeaderKitToolingTestHelper frame-two", + ] + lines += (5...24).map { "\($0) filler frame \($0)" } + lines += [ + ")", + "libc++abi: terminating due to uncaught exception of type NSException", + "final-diagnostic-tail", + ] + try writeAll(Array((lines.joined(separator: "\n") + "\n").utf8), to: STDERR_FILENO) + exit(19) + } + private static func writeAll(_ bytes: [UInt8], to descriptor: Int32) throws { var writtenCount = 0 while writtenCount < bytes.count { @@ -153,6 +176,7 @@ private struct PrivateHeaderKitToolingTestHelper { ) guard result.status == 19, !result.wasKilled, + result.emittedOutput.lines == ["buffered-stdout", "buffered-stderr"], result.lastLines == ["buffered-stdout", "buffered-stderr"] else { throw ToolingError.message( diff --git a/Tests/PrivateHeaderKitToolingTests/StreamingProcessRunnerTests.swift b/Tests/PrivateHeaderKitToolingTests/StreamingProcessRunnerTests.swift index d409cc8..92e579a 100644 --- a/Tests/PrivateHeaderKitToolingTests/StreamingProcessRunnerTests.swift +++ b/Tests/PrivateHeaderKitToolingTests/StreamingProcessRunnerTests.swift @@ -522,6 +522,7 @@ struct StreamingProcessRunnerTests { #expect(result.lastLines.contains("stdout-line")) #expect(result.lastLines.contains("stderr-line")) #expect(result.lastLines.contains("tail-line")) + #expect(result.terminationObservedAtUnixEpochMicroseconds != nil) #expect(forwarded.contains("stdout-line")) #expect(forwarded.contains("stderr-line")) #expect(forwarded.contains("tail-line")) @@ -794,7 +795,7 @@ struct StreamingProcessRunnerTests { } } - @Test func nonzeroStreamingExitKeepsCombinedTailAndSimpleMapsItToFailure() async throws { + @Test func nonzeroStreamingExitKeepsBoundedOutputAndSimpleRendersTermination() async throws { let command = [ "/bin/sh", "-c", "printf 'stream-output\\n'; printf 'stream-error\\n' >&2; exit 24", @@ -809,7 +810,13 @@ struct StreamingProcessRunnerTests { #expect(result.status == 24) #expect(!result.wasKilled) + #expect(result.emittedOutput.lines == ["stream-output", "stream-error"]) #expect(result.lastLines == ["stream-output", "stream-error"]) + #expect(result.diagnosticLines == [ + "stream-output", + "stream-error", + "Exited with status 24", + ]) do { try await ProcessRunner().runSimple(command, env: nil, cwd: nil) @@ -854,12 +861,12 @@ struct StreamingProcessRunnerTests { #expect(output.allSatisfy { $0 == UInt8(ascii: "x") }) } - @Test func chunkedCaptureFailureKeepsStatusAndBoundedStandardErrorTail() async throws { + @Test func chunkedCaptureFailureKeepsStatusAndStrictlyBoundedStandardError() async throws { let helper = try testHelperExecutableURL() let command = [ helper.path, "large-stderr-failure", - String(StreamingOutputCollector.maximumLineByteCount + 4_096), + String(BoundedProcessOutput.maximumRenderedLineByteCount + 4_096), ] do { @@ -877,12 +884,16 @@ struct StreamingProcessRunnerTests { } #expect(failedCommand == command) #expect(status == 19) - #expect(standardError.hasPrefix("[truncated 4096 bytes] ")) - #expect(standardError.hasSuffix("\nfinal-diagnostic")) + let lines = standardError.split(separator: "\n").map(String.init) + #expect(lines.first?.hasPrefix("x") == true) + #expect(lines.first?.contains(" … ") == true) + #expect(lines.first?.hasSuffix("x") == true) + #expect(lines.first?.utf8.count == BoundedProcessOutput.maximumRenderedLineByteCount) + #expect(lines.contains("final-diagnostic")) + #expect(lines.contains { $0.hasPrefix("[omitted 0 lines and ") }) #expect( standardError.utf8.count - == StreamingOutputCollector.maximumLineByteCount - + "[truncated 4096 bytes] \nfinal-diagnostic".utf8.count + <= BoundedProcessOutput.maximumRenderedByteCount ) } catch { Issue.record("unexpected error: \(error)") @@ -951,53 +962,90 @@ struct StreamingProcessRunnerTests { #expect(!FileManager.default.fileExists(atPath: streamingMarker)) } - @Test func collectorPreservesUTF8AcrossInjectedChunkBoundariesAndKeepsEightLines() throws { - let complete = Array( - ("ignored-1\nignored-2\n" + (1...7).map { "line-\($0)\n" }.joined() - + "emoji:😀\n").utf8 - ) + @Test func collectorPreservesUTF8AcrossChunksAndRetainsHeadAndTail() throws { + let complete = Array(((1...19).map { "line-\($0)\n" }.joined() + "emoji:😀\n").utf8) let emojiStart = try #require(complete.firstIndex(of: 0xF0)) - var collector = StreamingOutputCollector() + var collector = BoundedProcessOutputCollector() collector.consume(Array(complete[..<(emojiStart + 2)])) collector.consume(Array(complete[(emojiStart + 2)...])) - collector.finish() + let output = collector.finish() - #expect(collector.lastLines == [ - "line-1", "line-2", "line-3", "line-4", - "line-5", "line-6", "line-7", "emoji:😀", + #expect(output.lines == [ + "line-1", "line-2", "line-3", "line-4", "line-5", "line-6", "line-7", + "line-8", "[omitted 4 lines and 27 bytes]", "line-13", "line-14", "line-15", + "line-16", "line-17", "line-18", "line-19", "emoji:😀", ]) + #expect(output.omittedLineCount == 4) + #expect(output.omittedByteCount == 27) } - @Test func collectorBoundsUnterminatedLineAndMarksDiscardedPrefix() throws { - let discardedByteCount = 37 - let bytes = Array( - repeating: UInt8(ascii: "x"), - count: StreamingOutputCollector.maximumLineByteCount + discardedByteCount - ) - var collector = StreamingOutputCollector() + @Test func collectorRetainsBothEndsOfLongLineWithinOneKiB() throws { + let line = "BEGIN-" + String(repeating: "x", count: 5_000) + "-END" + let bytes = Array(line.utf8) + var collector = BoundedProcessOutputCollector() for chunkStart in stride(from: 0, to: bytes.count, by: 997) { collector.consume(Array(bytes[chunkStart.. 0) + #expect(output.text.utf8.count <= BoundedProcessOutput.maximumRenderedByteCount) } - @Test func collectorReplacesInvalidAndIncompleteUTF8AtEOF() { - var collector = StreamingOutputCollector() - collector.consume([UInt8(ascii: "a"), 0x80, 0xF0, 0x9F]) - collector.finish() + @Test func collectorTerminalSafesControlsAndInvalidIncompleteUTF8() throws { + var collector = BoundedProcessOutputCollector() + collector.consume([ + UInt8(ascii: "a"), 0x1B, UInt8(ascii: "\t"), 0x80, 0xF0, 0x9F, + ]) + let output = collector.finish() + + #expect(output.lines == [#"a\u{001b}\t��"#]) + let line = try #require(output.lines.first) + #expect( + line.unicodeScalars.allSatisfy { + switch $0.properties.generalCategory { + case .control, .format, .lineSeparator, .paragraphSeparator: + false + default: + true + } + } + ) + } - #expect(collector.lastLines == ["a��"]) + @Test func resultInitializerCannotBypassOutputBounds() { + let lines = (1...10_000).map { index in + index == 1 + ? "head\u{001B}" + String(repeating: "x", count: 5_000) + : "line-\(index)" + } + let result = StreamingCommandResult(status: 19, wasKilled: false, lastLines: lines) + let output = result.emittedOutput + + #expect(output.lines.count == BoundedProcessOutput.maximumRenderedLineCount) + #expect(output.lines.first?.hasPrefix(#"head\u{001b}"#) == true) + #expect(output.lines[8].hasPrefix("[omitted 9984 lines and ")) + #expect(output.lines.last == "line-10000") + #expect(output.omittedLineCount == 9_984) + #expect(output.omittedByteCount > 0) + #expect( + output.lines.allSatisfy { + $0.utf8.count <= BoundedProcessOutput.maximumRenderedLineByteCount + } + ) + #expect(output.text.utf8.count <= BoundedProcessOutput.maximumRenderedByteCount) + #expect(result.lastLines.last == "line-10000") + #expect(result.diagnosticLines.last == "Exited with status 19") } - @Test func helperOutputPreservesExactBytesAndTailLines() async throws { + @Test func helperOutputPreservesExactBytesAndBoundedLines() async throws { let helper = try testHelperExecutableURL() let passthrough = LockedDataBox() let expected = (1...9).map { "line-\($0)\n" }.joined() + "emoji:😀\nline-10\n" @@ -1009,14 +1057,19 @@ struct StreamingProcessRunnerTests { ) #expect(passthrough.snapshot() == Data(expected.utf8)) - #expect(result.lastLines == [ - "line-4", "line-5", "line-6", "line-7", + #expect(result.emittedOutput.lines == [ + "line-1", "line-2", "line-3", "line-4", "line-5", "line-6", "line-7", "line-8", "line-9", "emoji:😀", "line-10", ]) + #expect(result.lastLines == result.emittedOutput.lines) } - @Test func signalTerminationUsesSignalStatusAndKilledFlag() async throws { - let result = try await ProcessRunner().runStreaming( + @Test func signalOnlyTerminationKeepsEmittedOutputEmptyAndRecordsObservationTime() + async throws + { + let observedAt: UInt64 = 1_777_000_123_456_789 + let runner = ProcessRunner(terminationObservationClock: { observedAt }) + let result = try await runner.runStreaming( ["/bin/sh", "-c", "kill -TERM $$"], streamOutput: false, passthrough: { _ in } @@ -1024,7 +1077,53 @@ struct StreamingProcessRunnerTests { #expect(result.status == SIGTERM) #expect(result.wasKilled) + #expect(result.emittedOutput.isEmpty) + #expect(result.emittedOutput.lines.isEmpty) #expect(result.lastLines == ["Terminated by signal \(SIGTERM)"]) + #expect(result.diagnosticLines == [ + "No process diagnostic was emitted.", + "Terminated by signal \(SIGTERM)", + ]) + #expect(result.terminationObservedAtUnixEpochMicroseconds == observedAt) + } + + @Test func longExceptionRetainsNameReasonFirstApplicationFrameAndTail() async throws { + let helper = try testHelperExecutableURL() + let observedAt: UInt64 = 1_777_000_987_654_321 + let result = try await ProcessRunner( + terminationObservationClock: { observedAt } + ).runBuffered( + [helper.path, "long-exception-failure"], + env: nil, + cwd: nil + ) + + #expect(result.status == 19) + #expect(!result.wasKilled) + #expect( + result.emittedOutput.lines.first + == "*** Terminating app due to uncaught exception 'FixtureException', reason: 'fixture reason'" + ) + #expect( + result.emittedOutput.lines.contains( + "2 PrivateHeaderKitToolingTestHelper frame-zero" + ) + ) + #expect(result.emittedOutput.lines[8].hasPrefix("[omitted ")) + #expect( + result.emittedOutput.lines.suffix(2) == [ + "libc++abi: terminating due to uncaught exception of type NSException", + "final-diagnostic-tail", + ] + ) + #expect(result.emittedOutput.lines.count == 17) + #expect( + result.emittedOutput.text.utf8.count + <= BoundedProcessOutput.maximumRenderedByteCount + ) + #expect(result.lastLines.last == "final-diagnostic-tail") + #expect(result.diagnosticLines.last == "Exited with status 19") + #expect(result.terminationObservedAtUnixEpochMicroseconds == observedAt) } @Test func waitableLeaderAnchorsIdentityThroughProcessGroupCompletion() async throws { From 95ff3888f5b61c6acce3b2c46491def8ce34b348 Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Tue, 25 Aug 2026 17:00:35 +0900 Subject: [PATCH 04/14] Add raw helper startup handshake --- Package.swift | 1 + .../PrivateHeaderGenerationRawDumping.swift | 23 ++ .../CurrentProcessExecutableIdentity.swift | 59 +++++ .../PrivateHeaderKitHelperProtocol.swift | 201 ++++++++++++++++++ .../PrivateHeaderKitRawDumpMain.swift | 80 ++++++- .../ToolCompatibilityIdentity.swift | 54 ++--- ...ivateHeaderGenerationRawDumpingTests.swift | 17 +- .../PrivateHeaderKitHelperProtocolTests.swift | 156 ++++++++++++++ .../PrivateHeaderKitRawDumpTests.swift | 119 +++++++++++ .../ToolCompatibilityIdentityTests.swift | 13 ++ 10 files changed, 680 insertions(+), 43 deletions(-) create mode 100644 Sources/PrivateHeaderKitExecutableResolution/CurrentProcessExecutableIdentity.swift diff --git a/Package.swift b/Package.swift index ffcd26d..d70dfc7 100644 --- a/Package.swift +++ b/Package.swift @@ -93,6 +93,7 @@ let package = Package( .target( name: "PrivateHeaderKitTooling", dependencies: [ + "PrivateHeaderKitExecutableResolution", .product( name: "Subprocess", package: "swift-subprocess", diff --git a/Sources/PrivateHeaderKitCore/PrivateHeaderGenerationRawDumping.swift b/Sources/PrivateHeaderKitCore/PrivateHeaderGenerationRawDumping.swift index 4826033..c079bc1 100644 --- a/Sources/PrivateHeaderKitCore/PrivateHeaderGenerationRawDumping.swift +++ b/Sources/PrivateHeaderKitCore/PrivateHeaderGenerationRawDumping.swift @@ -5,6 +5,15 @@ extension PrivateHeaderGeneration { package enum RawDumping { package static func makeInvocation(_ request: Request) -> Invocation { let helperURL = request.executionMode.helperURL(from: request.helperURLs) + let processHandshakeID = UUID() + let processHandshakeReportURL = request.stagingOutputDirectory + .deletingLastPathComponent() + .appendingPathComponent( + ".privateheaderkit-raw-process-handshake-" + + processHandshakeID.uuidString.lowercased() + + ".json", + isDirectory: false + ) let diagnosticsReportURL = request.stagingOutputDirectory .deletingLastPathComponent() .appendingPathComponent( @@ -17,10 +26,14 @@ extension PrivateHeaderGeneration { helperURL: helperURL, inputPath: request.inputPath, stagingOutputDirectory: request.stagingOutputDirectory, + processHandshakeID: processHandshakeID, + processHandshakeReportURL: processHandshakeReportURL, diagnosticsReportURL: diagnosticsReportURL, command: makeCommand( helperURL: helperURL, request: request, + processHandshakeID: processHandshakeID, + processHandshakeReportURL: processHandshakeReportURL, diagnosticsReportURL: diagnosticsReportURL ), environment: makeEnvironment(for: request) @@ -61,6 +74,8 @@ extension PrivateHeaderGeneration { private static func makeCommand( helperURL: URL, request: Request, + processHandshakeID: UUID, + processHandshakeReportURL: URL, diagnosticsReportURL: URL ) -> [String] { var command: [String] @@ -98,6 +113,12 @@ extension PrivateHeaderGeneration { if request.executionMode.isHost, request.options.preferRuntimeMetadata { command.append("-R") } + command += [ + "--process-handshake-id", + processHandshakeID.uuidString.lowercased(), + "--process-handshake-report", + processHandshakeReportURL.path, + ] command += ["--diagnostics-report", diagnosticsReportURL.path] command.append(request.inputPath) return command @@ -278,6 +299,8 @@ extension PrivateHeaderGeneration.RawDumping { package let helperURL: URL package let inputPath: String package let stagingOutputDirectory: URL + package let processHandshakeID: UUID + package let processHandshakeReportURL: URL package let diagnosticsReportURL: URL package let command: [String] package let environment: [String: String] diff --git a/Sources/PrivateHeaderKitExecutableResolution/CurrentProcessExecutableIdentity.swift b/Sources/PrivateHeaderKitExecutableResolution/CurrentProcessExecutableIdentity.swift new file mode 100644 index 0000000..3f11e12 --- /dev/null +++ b/Sources/PrivateHeaderKitExecutableResolution/CurrentProcessExecutableIdentity.swift @@ -0,0 +1,59 @@ +import Foundation + +#if canImport(Darwin) +import Darwin +import MachO +#endif + +package enum CurrentProcessExecutableIdentityError: Error, Equatable, Sendable { + case imageInspectionFailed + case missingMachOUUID +} + +extension CurrentProcessExecutableIdentityError: CustomStringConvertible, LocalizedError { + package var description: String { + switch self { + case .imageInspectionFailed: + "failed to inspect the running executable image" + case .missingMachOUUID: + "the running executable has no Mach-O UUID" + } + } + + package var errorDescription: String? { description } +} + +package func currentProcessMachOUUID() throws -> UUID { +#if canImport(Darwin) + guard let header = _dyld_get_image_header(0), + header.pointee.magic == MH_MAGIC_64 + else { + throw CurrentProcessExecutableIdentityError.imageInspectionFailed + } + + var cursor = UnsafeRawPointer(header).advanced( + by: MemoryLayout.size + ) + var remainingBytes = Int(header.pointee.sizeofcmds) + for _ in 0..= MemoryLayout.size else { break } + let command = cursor.loadUnaligned(as: load_command.self) + let commandSize = Int(command.cmdsize) + guard commandSize >= MemoryLayout.size, + commandSize <= remainingBytes + else { + break + } + if command.cmd == LC_UUID { + guard commandSize >= MemoryLayout.size else { break } + let uuid = cursor.loadUnaligned(as: uuid_command.self).uuid + return UUID(uuid: uuid) + } + cursor = cursor.advanced(by: commandSize) + remainingBytes -= commandSize + } + throw CurrentProcessExecutableIdentityError.missingMachOUUID +#else + throw CurrentProcessExecutableIdentityError.imageInspectionFailed +#endif +} diff --git a/Sources/PrivateHeaderKitHelperProtocol/PrivateHeaderKitHelperProtocol.swift b/Sources/PrivateHeaderKitHelperProtocol/PrivateHeaderKitHelperProtocol.swift index 9a44b14..4ade95e 100644 --- a/Sources/PrivateHeaderKitHelperProtocol/PrivateHeaderKitHelperProtocol.swift +++ b/Sources/PrivateHeaderKitHelperProtocol/PrivateHeaderKitHelperProtocol.swift @@ -30,6 +30,207 @@ package enum PrivateHeaderKitProducerVersion { } } +package struct PrivateHeaderKitRawDumpProcessHandshake: Codable, Hashable, Sendable { + package static let currentSchemaVersion = 1 + package static let maximumEncodedByteCount = 2 * 1_024 + package static let maximumExecutableNameUTF8Count = 128 + + package let schemaVersion: Int + package let invocationID: UUID + package let processIdentifier: Int32 + package let helperStartedAtUnixMicroseconds: Int64 + package let executableName: String + package let executableMachOUUID: UUID + package let producerVersion: String + + package init( + invocationID: UUID, + processIdentifier: Int32, + helperStartedAtUnixMicroseconds: Int64, + executableName: String, + executableMachOUUID: UUID, + producerVersion: String = PrivateHeaderKitBuildInfo.version + ) throws { + try Self.validateInvocationID(invocationID) + try Self.validateProcessIdentifier(processIdentifier) + try Self.validateStartTime(helperStartedAtUnixMicroseconds) + try Self.validateExecutableName(executableName) + try Self.validateExecutableMachOUUID(executableMachOUUID) + let producerVersion = try Self.validateProducerVersion(producerVersion) + + self.schemaVersion = Self.currentSchemaVersion + self.invocationID = invocationID + self.processIdentifier = processIdentifier + self.helperStartedAtUnixMicroseconds = helperStartedAtUnixMicroseconds + self.executableName = executableName + self.executableMachOUUID = executableMachOUUID + self.producerVersion = producerVersion + } + + package init(from decoder: any Decoder) throws { + let fields = try decoder.container(keyedBy: FieldKey.self) + guard Set(fields.allKeys.map(\.stringValue)) + == Set(CodingKeys.allCases.map(\.rawValue)) + else { + throw ValidationError.invalidFieldSet + } + let container = try decoder.container(keyedBy: CodingKeys.self) + let schemaVersion = try container.decode(Int.self, forKey: .schemaVersion) + guard schemaVersion == Self.currentSchemaVersion else { + throw ValidationError.unsupportedSchemaVersion( + expected: Self.currentSchemaVersion, + actual: schemaVersion + ) + } + + let invocationID = try container.decode(UUID.self, forKey: .invocationID) + let processIdentifier = try container.decode(Int32.self, forKey: .processIdentifier) + let helperStartedAtUnixMicroseconds = try container.decode( + Int64.self, + forKey: .helperStartedAtUnixMicroseconds + ) + let executableName = try container.decode(String.self, forKey: .executableName) + let executableMachOUUID = try container.decode(UUID.self, forKey: .executableMachOUUID) + let producerVersion = try container.decode(String.self, forKey: .producerVersion) + + try Self.validateInvocationID(invocationID) + try Self.validateProcessIdentifier(processIdentifier) + try Self.validateStartTime(helperStartedAtUnixMicroseconds) + try Self.validateExecutableName(executableName) + try Self.validateExecutableMachOUUID(executableMachOUUID) + + self.schemaVersion = schemaVersion + self.invocationID = invocationID + self.processIdentifier = processIdentifier + self.helperStartedAtUnixMicroseconds = helperStartedAtUnixMicroseconds + self.executableName = executableName + self.executableMachOUUID = executableMachOUUID + self.producerVersion = try Self.validateProducerVersion(producerVersion) + } + + package func encoded() throws -> Data { + let encoder = JSONEncoder() + encoder.outputFormatting = [.sortedKeys] + let data = try encoder.encode(self) + guard data.count <= Self.maximumEncodedByteCount else { + throw ValidationError.encodedPayloadTooLarge( + actual: data.count, + maximum: Self.maximumEncodedByteCount + ) + } + return data + } + + package static func decode( + _ data: Data, + expectedInvocationID: UUID + ) throws -> Self { + guard data.count <= maximumEncodedByteCount else { + throw ValidationError.encodedPayloadTooLarge( + actual: data.count, + maximum: maximumEncodedByteCount + ) + } + let handshake = try JSONDecoder().decode(Self.self, from: data) + guard handshake.invocationID == expectedInvocationID else { + throw ValidationError.invocationIDMismatch( + expected: expectedInvocationID, + actual: handshake.invocationID + ) + } + return handshake + } + + private static let zeroUUID = UUID(uuidString: "00000000-0000-0000-0000-000000000000")! + + private static func validateInvocationID(_ value: UUID) throws { + guard value != zeroUUID else { + throw ValidationError.invalidInvocationID + } + } + + private static func validateProcessIdentifier(_ value: Int32) throws { + guard value > 0 else { + throw ValidationError.invalidProcessIdentifier(value) + } + } + + private static func validateStartTime(_ value: Int64) throws { + guard value > 0 else { + throw ValidationError.invalidStartTime(value) + } + } + + private static func validateExecutableName(_ value: String) throws { + guard !value.isEmpty, + value != ".", + value != "..", + value.utf8.count <= maximumExecutableNameUTF8Count, + !value.contains("/"), + value.unicodeScalars.allSatisfy({ scalar in + switch scalar.properties.generalCategory { + case .control, .format, .lineSeparator, .paragraphSeparator: + false + default: + true + } + }) + else { + throw ValidationError.invalidExecutableName + } + } + + private static func validateExecutableMachOUUID(_ value: UUID) throws { + guard value != zeroUUID else { + throw ValidationError.invalidExecutableMachOUUID + } + } + + private static func validateProducerVersion(_ value: String) throws -> String { + do { + return try PrivateHeaderKitProducerVersion.validated(value) + } catch { + throw ValidationError.invalidProducerVersion + } + } + + private enum CodingKeys: String, CodingKey, CaseIterable { + case schemaVersion + case invocationID + case processIdentifier + case helperStartedAtUnixMicroseconds + case executableName + case executableMachOUUID + case producerVersion + } + + private struct FieldKey: CodingKey { + let stringValue: String + let intValue: Int? = nil + + init?(stringValue: String) { + self.stringValue = stringValue + } + + init?(intValue: Int) { + return nil + } + } + + package enum ValidationError: Error, Equatable, Sendable { + case unsupportedSchemaVersion(expected: Int, actual: Int) + case encodedPayloadTooLarge(actual: Int, maximum: Int) + case invocationIDMismatch(expected: UUID, actual: UUID) + case invalidInvocationID + case invalidProcessIdentifier(Int32) + case invalidStartTime(Int64) + case invalidExecutableName + case invalidExecutableMachOUUID + case invalidProducerVersion + case invalidFieldSet + } +} + package struct PrivateHeaderKitRawDumpDiagnostic: Codable, Hashable, Sendable { package static let maximumStringUTF8Count = 2_048 diff --git a/Sources/PrivateHeaderKitRawDumpCore/PrivateHeaderKitRawDumpMain.swift b/Sources/PrivateHeaderKitRawDumpCore/PrivateHeaderKitRawDumpMain.swift index 62bc999..4fb6caf 100644 --- a/Sources/PrivateHeaderKitRawDumpCore/PrivateHeaderKitRawDumpMain.swift +++ b/Sources/PrivateHeaderKitRawDumpCore/PrivateHeaderKitRawDumpMain.swift @@ -98,6 +98,8 @@ struct DumpOptions { var logSkippedClasses: Bool = false var profile: Bool = false var logSwiftEvents: Bool = false + var processHandshakeID: UUID? + var processHandshakeReportURL: URL? var diagnosticsReportURL: URL? let objcDiagnostics = RawDumpObjCDiagnosticsAccumulator() } @@ -128,7 +130,7 @@ public struct PrivateHeaderKitRawDumpCLI { } do { - try await run(parsed: parsed) + try await runRawDumpAfterWritingProcessHandshake(parsed) try writeDiagnosticsReportIfRequested(parsed.options) } catch { do { @@ -199,6 +201,18 @@ func parseArguments( guard nextIndex < args.count else { return nil } options.diagnosticsReportURL = URL(fileURLWithPath: args[nextIndex]) index += 1 + case "--process-handshake-id": + let nextIndex = index + 1 + guard nextIndex < args.count, + let invocationID = UUID(uuidString: args[nextIndex]) + else { return nil } + options.processHandshakeID = invocationID + index += 1 + case "--process-handshake-report": + let nextIndex = index + 1 + guard nextIndex < args.count else { return nil } + options.processHandshakeReportURL = URL(fileURLWithPath: args[nextIndex]) + index += 1 default: if arg.hasPrefix("-") { // ignore unknown flags for compatibility @@ -213,6 +227,11 @@ func parseArguments( guard options.useSharedCache == (options.expectedCacheUUID != nil) else { return nil } + guard (options.processHandshakeID == nil) + == (options.processHandshakeReportURL == nil) + else { + return nil + } if !options.useRuntimeFallback { options.useRuntimeFallback = shouldUseRuntimeFallback(environment: environment) } @@ -241,10 +260,69 @@ private func printUsage() { -R Prefer Objective-C runtime metadata (auto-enabled in simulator) --diagnostics-report Write the versioned Objective-C metadata diagnostics report + --process-handshake-id + Bind the raw-dump invocation to its startup process handshake + --process-handshake-report + Write the bounded startup process handshake before target work """ print(text) } +func runRawDumpAfterWritingProcessHandshake( + _ parsed: ParsedArguments, + writeProcessHandshake: (DumpOptions) throws -> Void = { + try writeProcessHandshakeIfRequested($0) + }, + runOperation: (ParsedArguments) async throws -> Void = run +) async throws { + try writeProcessHandshake(parsed.options) + try await runOperation(parsed) +} + +func writeProcessHandshakeIfRequested( + _ options: DumpOptions, + processIdentifier: () -> Int32 = { getpid() }, + nowUnixMicroseconds: () throws -> Int64 = currentRealtimeUnixMicroseconds, + executableName: () -> String = { Bundle.main.executableURL?.lastPathComponent ?? "" }, + executableMachOUUID: () throws -> UUID = currentProcessMachOUUID, + producerVersion: String = PrivateHeaderKitBuildInfo.version +) throws { + guard let invocationID = options.processHandshakeID, + let reportURL = options.processHandshakeReportURL + else { + return + } + let handshake = try PrivateHeaderKitRawDumpProcessHandshake( + invocationID: invocationID, + processIdentifier: processIdentifier(), + helperStartedAtUnixMicroseconds: nowUnixMicroseconds(), + executableName: executableName(), + executableMachOUUID: executableMachOUUID(), + producerVersion: producerVersion + ) + try handshake.encoded().write(to: reportURL, options: .atomic) +} + +func currentRealtimeUnixMicroseconds() throws -> Int64 { + var time = timespec() + guard clock_gettime(CLOCK_REALTIME, &time) == 0, + let seconds = Int64(exactly: time.tv_sec), + let nanoseconds = Int64(exactly: time.tv_nsec) + else { + throw POSIXError(.init(rawValue: errno) ?? .EIO) + } + let (wholeMicroseconds, multiplyOverflow) = seconds.multipliedReportingOverflow( + by: 1_000_000 + ) + let (result, addOverflow) = wholeMicroseconds.addingReportingOverflow( + nanoseconds / 1_000 + ) + guard !multiplyOverflow, !addOverflow, result > 0 else { + throw POSIXError(.EOVERFLOW) + } + return result +} + private func writeDiagnosticsReportIfRequested(_ options: DumpOptions) throws { guard let reportURL = options.diagnosticsReportURL else { return } try writeRawDumpDiagnosticsReport(options.objcDiagnostics.report, to: reportURL) diff --git a/Sources/PrivateHeaderKitTooling/ToolCompatibilityIdentity.swift b/Sources/PrivateHeaderKitTooling/ToolCompatibilityIdentity.swift index f6fed78..bd40586 100644 --- a/Sources/PrivateHeaderKitTooling/ToolCompatibilityIdentity.swift +++ b/Sources/PrivateHeaderKitTooling/ToolCompatibilityIdentity.swift @@ -1,14 +1,10 @@ import Foundation +import PrivateHeaderKitExecutableResolution #if canImport(CryptoKit) import CryptoKit #endif -#if canImport(Darwin) -import Darwin -import MachO -#endif - package struct ToolArtifactInput: Equatable, Sendable { package let role: String package let url: URL @@ -84,45 +80,21 @@ package struct SwiftPMToolSnapshot: Equatable, Sendable { } } -package func currentProcessExecutableBuildIdentity() throws -> String { +package func currentProcessExecutableBuildIdentity( + resolveMachOUUID: () throws -> UUID = currentProcessMachOUUID +) throws -> String { + do { + let uuid = try resolveMachOUUID() + return "macho-uuid:\(uuid.uuidString.lowercased())" + } catch let error as CurrentProcessExecutableIdentityError { #if canImport(Darwin) - guard let header = _dyld_get_image_header(0), - header.pointee.magic == MH_MAGIC_64 - else { - throw ToolingError.message("failed to inspect the running executable image") - } - - var cursor = UnsafeRawPointer(header).advanced( - by: MemoryLayout.size - ) - var remainingBytes = Int(header.pointee.sizeofcmds) - for _ in 0..= MemoryLayout.size else { - break - } - let command = cursor.load(as: load_command.self) - let commandSize = Int(command.cmdsize) - guard commandSize >= MemoryLayout.size, - commandSize <= remainingBytes - else { - break - } - if command.cmd == LC_UUID { - guard commandSize >= MemoryLayout.size else { - break - } - let uuid = cursor.load(as: uuid_command.self).uuid - return "macho-uuid:\(UUID(uuid: uuid).uuidString.lowercased())" - } - cursor = cursor.advanced(by: commandSize) - remainingBytes -= commandSize - } - throw ToolingError.message("the running executable has no Mach-O UUID") + throw ToolingError.message(error.description) #else - throw ToolingError.message( - "running executable identity is unavailable on this platform" - ) + throw ToolingError.message( + "running executable identity is unavailable on this platform" + ) #endif + } } package func captureToolArtifactSnapshot( diff --git a/Tests/PrivateHeaderKitCoreTests/PrivateHeaderGenerationRawDumpingTests.swift b/Tests/PrivateHeaderKitCoreTests/PrivateHeaderGenerationRawDumpingTests.swift index a1ef021..620abb7 100644 --- a/Tests/PrivateHeaderKitCoreTests/PrivateHeaderGenerationRawDumpingTests.swift +++ b/Tests/PrivateHeaderKitCoreTests/PrivateHeaderGenerationRawDumpingTests.swift @@ -29,11 +29,24 @@ struct PrivateHeaderGenerationRawDumpingTests { invocation.diagnosticsReportURL.deletingLastPathComponent() == stageDirectory.deletingLastPathComponent() ) + #expect( + invocation.processHandshakeReportURL.deletingLastPathComponent() + == stageDirectory.deletingLastPathComponent() + ) + #expect( + invocation.processHandshakeReportURL.lastPathComponent + == ".privateheaderkit-raw-process-handshake-" + + invocation.processHandshakeID.uuidString.lowercased() + + ".json" + ) + #expect(invocation.processHandshakeReportURL != invocation.diagnosticsReportURL) #expect( invocation.command == [ "/opt/privateheaderkit/bin/privateheaderkit", "__raw-dump", "-o", stageDirectory.path, "-b", "-h", "-s", "-c", "--expected-cache-uuid", - cacheUUID.uuidString.lowercased(), "-D", "-R", "--diagnostics-report", + cacheUUID.uuidString.lowercased(), "-D", "-R", "--process-handshake-id", + invocation.processHandshakeID.uuidString.lowercased(), "--process-handshake-report", + invocation.processHandshakeReportURL.path, "--diagnostics-report", invocation.diagnosticsReportURL.path, "/System/Library/PrivateFrameworks/Foo.framework", ]) @@ -63,6 +76,8 @@ struct PrivateHeaderGenerationRawDumpingTests { invocation.command == [ "xcrun", "simctl", "spawn", "SIM-001", "/opt/privateheaderkit/bin/privateheaderkit-sim", "__raw-dump", "-o", stageDirectory.path, "-b", "-h", + "--process-handshake-id", invocation.processHandshakeID.uuidString.lowercased(), + "--process-handshake-report", invocation.processHandshakeReportURL.path, "--diagnostics-report", invocation.diagnosticsReportURL.path, "/System/Library/Frameworks/UIKit.framework", ]) diff --git a/Tests/PrivateHeaderKitHelperProtocolTests/PrivateHeaderKitHelperProtocolTests.swift b/Tests/PrivateHeaderKitHelperProtocolTests/PrivateHeaderKitHelperProtocolTests.swift index 46fb84c..dac3282 100644 --- a/Tests/PrivateHeaderKitHelperProtocolTests/PrivateHeaderKitHelperProtocolTests.swift +++ b/Tests/PrivateHeaderKitHelperProtocolTests/PrivateHeaderKitHelperProtocolTests.swift @@ -5,6 +5,162 @@ import Testing @Suite struct PrivateHeaderKitHelperProtocolTests { + @Test func rawDumpProcessHandshakeRoundTripsItsExactBoundedSchema() throws { + let invocationID = UUID(uuidString: "11111111-2222-3333-4444-555555555555")! + let executableUUID = UUID(uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee")! + let handshake = try PrivateHeaderKitRawDumpProcessHandshake( + invocationID: invocationID, + processIdentifier: 4_242, + helperStartedAtUnixMicroseconds: 1_700_000_000_123_456, + executableName: "privateheaderkit-sim-helper", + executableMachOUUID: executableUUID, + producerVersion: "v1.2.3" + ) + + let data = try handshake.encoded() + let decoded = try PrivateHeaderKitRawDumpProcessHandshake.decode( + data, + expectedInvocationID: invocationID + ) + let object = try #require( + JSONSerialization.jsonObject(with: data) as? [String: Any] + ) + + #expect(decoded == handshake) + #expect(data.count <= PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount) + #expect( + Set(object.keys) == [ + "schemaVersion", + "invocationID", + "processIdentifier", + "helperStartedAtUnixMicroseconds", + "executableName", + "executableMachOUUID", + "producerVersion", + ] + ) + #expect(object["schemaVersion"] as? Int == 1) + #expect(object["processIdentifier"] as? Int == 4_242) + #expect(object["executableName"] as? String == "privateheaderkit-sim-helper") + let wire = String(decoding: data, as: UTF8.self) + #expect(!wire.contains("/Users/")) + #expect(!wire.contains("SIMCTL_CHILD")) + #expect(!wire.contains("RuntimeRoot")) + } + + @Test func rawDumpProcessHandshakeRejectsMismatchedInvocationAndOversizedPayload() throws { + let invocationID = UUID(uuidString: "11111111-2222-3333-4444-555555555555")! + let unexpectedID = UUID(uuidString: "99999999-8888-7777-6666-555555555555")! + let handshake = try PrivateHeaderKitRawDumpProcessHandshake( + invocationID: invocationID, + processIdentifier: 4_242, + helperStartedAtUnixMicroseconds: 1_700_000_000_123_456, + executableName: "privateheaderkit-sim-helper", + executableMachOUUID: UUID(uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee")!, + producerVersion: "v1.2.3" + ) + + #expect( + throws: PrivateHeaderKitRawDumpProcessHandshake.ValidationError.invocationIDMismatch( + expected: unexpectedID, + actual: invocationID + ) + ) { + _ = try PrivateHeaderKitRawDumpProcessHandshake.decode( + handshake.encoded(), + expectedInvocationID: unexpectedID + ) + } + #expect( + throws: PrivateHeaderKitRawDumpProcessHandshake.ValidationError.encodedPayloadTooLarge( + actual: PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount + 1, + maximum: PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount + ) + ) { + _ = try PrivateHeaderKitRawDumpProcessHandshake.decode( + Data( + repeating: 0, + count: PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount + 1 + ), + expectedInvocationID: invocationID + ) + } + } + + @Test func rawDumpProcessHandshakeEnforcesStringBounds() throws { + let invocationID = UUID(uuidString: "11111111-2222-3333-4444-555555555555")! + let executableUUID = UUID(uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee")! + let maximum = try PrivateHeaderKitRawDumpProcessHandshake( + invocationID: invocationID, + processIdentifier: 4_242, + helperStartedAtUnixMicroseconds: 1_700_000_000_123_456, + executableName: String( + repeating: "\\", + count: PrivateHeaderKitRawDumpProcessHandshake.maximumExecutableNameUTF8Count + ), + executableMachOUUID: executableUUID, + producerVersion: String( + repeating: "\\", + count: PrivateHeaderKitProducerVersion.maximumUTF8Count + ) + ) + + #expect( + try maximum.encoded().count + <= PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount + ) + #expect(throws: PrivateHeaderKitRawDumpProcessHandshake.ValidationError.self) { + _ = try PrivateHeaderKitRawDumpProcessHandshake( + invocationID: invocationID, + processIdentifier: 4_242, + helperStartedAtUnixMicroseconds: 1_700_000_000_123_456, + executableName: String( + repeating: "e", + count: PrivateHeaderKitRawDumpProcessHandshake.maximumExecutableNameUTF8Count + 1 + ), + executableMachOUUID: executableUUID, + producerVersion: "v1.2.3" + ) + } + #expect(throws: PrivateHeaderKitRawDumpProcessHandshake.ValidationError.self) { + _ = try PrivateHeaderKitRawDumpProcessHandshake( + invocationID: invocationID, + processIdentifier: 4_242, + helperStartedAtUnixMicroseconds: 1_700_000_000_123_456, + executableName: "privateheaderkit-sim-helper", + executableMachOUUID: executableUUID, + producerVersion: String( + repeating: "v", + count: PrivateHeaderKitProducerVersion.maximumUTF8Count + 1 + ) + ) + } + } + + @Test func rawDumpProcessHandshakeStrictlyRejectsInvalidFields() { + let expectedID = UUID(uuidString: "11111111-2222-3333-4444-555555555555")! + let validFields = #""schemaVersion":1,"invocationID":"11111111-2222-3333-4444-555555555555","processIdentifier":4242,"helperStartedAtUnixMicroseconds":1700000000123456,"executableName":"privateheaderkit-sim-helper","executableMachOUUID":"aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee","producerVersion":"v1.2.3""# + let payloads = [ + "{\(validFields),\"path\":\"/private/var/tmp/helper\"}", + "{\(validFields.replacingOccurrences(of: "\"schemaVersion\":1", with: "\"schemaVersion\":2"))}", + "{\(validFields.replacingOccurrences(of: "11111111-2222-3333-4444-555555555555", with: "00000000-0000-0000-0000-000000000000"))}", + "{\(validFields.replacingOccurrences(of: "\"processIdentifier\":4242", with: "\"processIdentifier\":0"))}", + "{\(validFields.replacingOccurrences(of: "\"helperStartedAtUnixMicroseconds\":1700000000123456", with: "\"helperStartedAtUnixMicroseconds\":0"))}", + "{\(validFields.replacingOccurrences(of: "privateheaderkit-sim-helper", with: "/private/helper"))}", + "{\(validFields.replacingOccurrences(of: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", with: "00000000-0000-0000-0000-000000000000"))}", + "{\(validFields.replacingOccurrences(of: "\"producerVersion\":\"v1.2.3\"", with: "\"producerVersion\":\"\""))}", + ] + + for payload in payloads { + #expect(throws: (any Error).self) { + _ = try PrivateHeaderKitRawDumpProcessHandshake.decode( + Data(payload.utf8), + expectedInvocationID: expectedID + ) + } + } + } + @Test func rawDumpDiagnosticsZeroReportRoundTrips() throws { let report = PrivateHeaderKitRawDumpDiagnosticsReport( producerVersion: "v1.2.3", diff --git a/Tests/PrivateHeaderKitRawDumpTests/PrivateHeaderKitRawDumpTests.swift b/Tests/PrivateHeaderKitRawDumpTests/PrivateHeaderKitRawDumpTests.swift index aed5b65..27fa391 100644 --- a/Tests/PrivateHeaderKitRawDumpTests/PrivateHeaderKitRawDumpTests.swift +++ b/Tests/PrivateHeaderKitRawDumpTests/PrivateHeaderKitRawDumpTests.swift @@ -44,6 +44,8 @@ struct PrivateHeaderKitRawDumpArgumentTests { "--expected-cache-uuid", "11111111-2222-3333-4444-555555555555", "-D", "-R", + "--process-handshake-id", "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", + "--process-handshake-report", "/tmp/handshake.json", "--diagnostics-report", "/tmp/report.json", "/tmp/input" ] @@ -62,9 +64,126 @@ struct PrivateHeaderKitRawDumpArgumentTests { #expect(parsed?.options.expectedCacheUUID == UUID(uuidString: "11111111-2222-3333-4444-555555555555")) #expect(parsed?.options.verbose == true) #expect(parsed?.options.useRuntimeFallback == true) + #expect( + parsed?.options.processHandshakeID + == UUID(uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee") + ) + #expect(parsed?.options.processHandshakeReportURL?.path == "/tmp/handshake.json") #expect(parsed?.options.diagnosticsReportURL?.path == "/tmp/report.json") } + @Test func processHandshakeArgumentsMustBePairedAndValid() { + #expect( + parseArguments( + ["--process-handshake-id", UUID().uuidString, "/tmp/input"], + environment: [:] + ) == nil + ) + #expect( + parseArguments( + ["--process-handshake-report", "/tmp/handshake.json", "/tmp/input"], + environment: [:] + ) == nil + ) + #expect( + parseArguments( + [ + "--process-handshake-id", "not-a-uuid", + "--process-handshake-report", "/tmp/handshake.json", + "/tmp/input", + ], + environment: [:] + ) == nil + ) + } + + @Test func processHandshakeIsWrittenBeforeRawDumpWorkBegins() async throws { + let parsed = try #require( + parseArguments( + [ + "--process-handshake-id", "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", + "--process-handshake-report", "/tmp/handshake.json", + "/tmp/input", + ], + environment: [:] + ) + ) + var events: [String] = [] + + try await runRawDumpAfterWritingProcessHandshake( + parsed, + writeProcessHandshake: { _ in events.append("handshake") }, + runOperation: { _ in events.append("work") } + ) + + #expect(events == ["handshake", "work"]) + } + + @Test func processHandshakeWriteFailurePreventsRawDumpWork() async throws { + let parsed = try #require( + parseArguments( + [ + "--process-handshake-id", "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", + "--process-handshake-report", "/tmp/handshake.json", + "/tmp/input", + ], + environment: [:] + ) + ) + var didBeginWork = false + + await #expect(throws: FixtureProcessHandshakeError.self) { + try await runRawDumpAfterWritingProcessHandshake( + parsed, + writeProcessHandshake: { _ in throw FixtureProcessHandshakeError.writeFailed }, + runOperation: { _ in didBeginWork = true } + ) + } + + #expect(didBeginWork == false) + } + + @Test func processHandshakeWriterUsesOnlyBoundedSelfReportedIdentity() throws { + let root = FileManager.default.temporaryDirectory.appendingPathComponent( + "PrivateHeaderKitRawDumpProcessHandshakeTests-\(UUID().uuidString)", + isDirectory: true + ) + try FileManager.default.createDirectory(at: root, withIntermediateDirectories: true) + defer { try? FileManager.default.removeItem(at: root) } + let reportURL = root.appendingPathComponent("handshake.json") + let invocationID = UUID(uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee")! + let executableUUID = UUID(uuidString: "11111111-2222-3333-4444-555555555555")! + var options = DumpOptions(outputDir: root) + options.processHandshakeID = invocationID + options.processHandshakeReportURL = reportURL + + try writeProcessHandshakeIfRequested( + options, + processIdentifier: { 4_242 }, + nowUnixMicroseconds: { 1_700_000_000_123_456 }, + executableName: { "privateheaderkit-sim-helper" }, + executableMachOUUID: { executableUUID }, + producerVersion: "v1.2.3" + ) + + let data = try Data(contentsOf: reportURL) + let handshake = try PrivateHeaderKitRawDumpProcessHandshake.decode( + data, + expectedInvocationID: invocationID + ) + #expect(data.count <= PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount) + #expect(handshake.processIdentifier == 4_242) + #expect(handshake.helperStartedAtUnixMicroseconds == 1_700_000_000_123_456) + #expect(handshake.executableName == "privateheaderkit-sim-helper") + #expect(handshake.executableMachOUUID == executableUUID) + #expect(handshake.producerVersion == "v1.2.3") + #expect(!String(decoding: data, as: UTF8.self).contains("/private/var/tmp")) + } + + private enum FixtureProcessHandshakeError: Error { + case writeFailed + } + @Test func parseArgumentsIgnoresUnknownFlags() { let parsed = parseArguments(["-Z", "/tmp/input"], environment: [:]) #expect(parsed?.inputPath == "/tmp/input") diff --git a/Tests/PrivateHeaderKitToolingTests/ToolCompatibilityIdentityTests.swift b/Tests/PrivateHeaderKitToolingTests/ToolCompatibilityIdentityTests.swift index da6fd05..1e23c90 100644 --- a/Tests/PrivateHeaderKitToolingTests/ToolCompatibilityIdentityTests.swift +++ b/Tests/PrivateHeaderKitToolingTests/ToolCompatibilityIdentityTests.swift @@ -7,6 +7,19 @@ import Testing @Suite struct ToolCompatibilityIdentityTests { + @Test func runningExecutableIdentityDelegatesToSharedMachOUUIDResolver() throws { + let executableUUID = UUID(uuidString: "11111111-2222-3333-4444-555555555555")! + var resolutionCount = 0 + + let identity = try currentProcessExecutableBuildIdentity { + resolutionCount += 1 + return executableUUID + } + + #expect(resolutionCount == 1) + #expect(identity == "macho-uuid:11111111-2222-3333-4444-555555555555") + } + @Test func toolInputHashingChecksCancellationBetweenBoundedReads() throws { let root = try makeIdentityFixtureRoot() defer { try? FileManager.default.removeItem(at: root) } From 8e69ef9e59fca2829d41dc04d1f8ecbdcade5fdb Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Tue, 25 Aug 2026 17:09:34 +0900 Subject: [PATCH 05/14] docs: tighten issue 81 privacy contract --- Docs/issue-remediation-progress.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Docs/issue-remediation-progress.md b/Docs/issue-remediation-progress.md index e989346..6c88736 100644 --- a/Docs/issue-remediation-progress.md +++ b/Docs/issue-remediation-progress.md @@ -31,8 +31,8 @@ Design gate approved: observation, while the raw helper owns its actual PID and loaded image. - The helper writes a separate, invocation-authenticated startup handshake before loading target metadata. It contains only schema/invocation identity, - actual PID, executable name and LC_UUID, producer version, and Unix epoch - start microseconds. It is atomic, at most 2 KiB, and contains no path, device + actual PID, executable name and LC_UUID, and Unix epoch start microseconds. + It is atomic, at most 2 KiB, and contains no path, producer text, device UDID, command, environment, or runtime root. - The diagnostics report remains a completed typed-diagnostics contract. It is not converted into a two-phase process-state file. From b0edb84c98d43902b177d6603417bdd088dbc345 Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Tue, 25 Aug 2026 17:12:25 +0900 Subject: [PATCH 06/14] fix(tooling): clarify omitted process bytes --- .../ProcessRunner.swift | 108 ++++++++++++------ .../StreamingProcessRunnerTests.swift | 47 ++++++-- 2 files changed, 111 insertions(+), 44 deletions(-) diff --git a/Sources/PrivateHeaderKitTooling/ProcessRunner.swift b/Sources/PrivateHeaderKitTooling/ProcessRunner.swift index 29e7394..c958311 100644 --- a/Sources/PrivateHeaderKitTooling/ProcessRunner.swift +++ b/Sources/PrivateHeaderKitTooling/ProcessRunner.swift @@ -306,7 +306,9 @@ public struct BoundedProcessOutput: Equatable, Sendable { public let lines: [String] public let omittedLineCount: UInt - public let omittedByteCount: UInt + /// A lower bound in raw source bytes. Terminal-safe rendering can expand one source byte, + /// so the exact omitted source count is not always recoverable from a bounded rendering. + public let omittedSourceByteCountLowerBound: UInt public init(lines: [String]) { var collector = BoundedProcessOutputCollector() @@ -325,27 +327,27 @@ public struct BoundedProcessOutput: Equatable, Sendable { fileprivate init( lines: [String], omittedLineCount: UInt, - omittedByteCount: UInt + omittedSourceByteCountLowerBound: UInt ) { precondition( lines.count <= Self.maximumRenderedLineCount, - "bounded process output exceeded its line count" + "BoundedProcessOutputCollector must enforce the rendered line-count bound" ) precondition( lines.allSatisfy { $0.utf8.count <= Self.maximumRenderedLineByteCount }, - "bounded process output exceeded its per-line byte count" + "BoundedProcessOutputCollector must enforce the per-line rendered byte bound" ) precondition( lines.joined(separator: "\n").utf8.count <= Self.maximumRenderedByteCount, - "bounded process output exceeded its rendered byte count" + "BoundedProcessOutputCollector must enforce the total rendered byte bound" ) precondition( lines.allSatisfy { terminalSafeProcessOutput($0) == $0 }, - "bounded process output contains terminal-unsafe text" + "BoundedProcessOutputCollector must emit terminal-safe text" ) self.lines = lines self.omittedLineCount = omittedLineCount - self.omittedByteCount = omittedByteCount + self.omittedSourceByteCountLowerBound = omittedSourceByteCountLowerBound } } @@ -1101,11 +1103,15 @@ struct BoundedProcessOutputCollector { private var headLines: [BoundedProcessLine] = [] private var tailLines: [BoundedProcessLine] = [] private var omittedLineCount: UInt = 0 - private var omittedByteCount: UInt = 0 + private var omittedSourceByteCountLowerBound: UInt = 0 + private var didOmitContent = false private var isFinished = false mutating func consume(_ bytes: [UInt8]) { - precondition(!isFinished, "cannot consume process output after finishing") + precondition( + !isFinished, + "BoundedProcessOutputCollector owns a single consume-before-finish lifecycle" + ) var segmentStart = bytes.startIndex for index in bytes.indices where bytes[index] == UInt8(ascii: "\n") { pendingLine.append(bytes[segmentStart.. BoundedProcessOutput { - precondition(!isFinished, "process output collector finished more than once") + precondition( + !isFinished, + "BoundedProcessOutputCollector owns a single finish transition" + ) if !pendingLine.isEmpty { consumePendingLine() } isFinished = true var lines = headLines.map(\.text) - if omittedLineCount > 0 || omittedByteCount > 0 { - lines.append("[omitted \(omittedLineCount) lines and \(omittedByteCount) bytes]") + if omittedLineCount > 0 || didOmitContent { + lines.append( + "[omitted \(omittedLineCount) lines and at least " + + "\(omittedSourceByteCountLowerBound) bytes]" + ) } lines += tailLines.map(\.text) return BoundedProcessOutput( lines: lines, omittedLineCount: omittedLineCount, - omittedByteCount: omittedByteCount + omittedSourceByteCountLowerBound: omittedSourceByteCountLowerBound ) } @@ -1149,7 +1161,11 @@ struct BoundedProcessOutputCollector { } private mutating func append(_ line: BoundedProcessLine) { - omittedByteCount = saturatingSum(omittedByteCount, line.omittedByteCount) + didOmitContent = didOmitContent || line.didOmitContent + omittedSourceByteCountLowerBound = saturatingSum( + omittedSourceByteCountLowerBound, + line.omittedSourceByteCountLowerBound + ) if headLines.count < BoundedProcessOutput.maximumHeadLineCount { headLines.append(line) return @@ -1158,10 +1174,11 @@ struct BoundedProcessOutputCollector { tailLines.append(line) guard tailLines.count > BoundedProcessOutput.maximumTailLineCount else { return } let omittedLine = tailLines.removeFirst() + didOmitContent = true omittedLineCount = saturatingSum(omittedLineCount, 1) - omittedByteCount = saturatingSum( - omittedByteCount, - omittedLine.retainedSourceByteCount + omittedSourceByteCountLowerBound = saturatingSum( + omittedSourceByteCountLowerBound, + omittedLine.rawSourceByteCountRemainingAfterLowerBound ) } } @@ -1175,12 +1192,18 @@ private struct BoundedProcessLineBytes { private var head: [UInt8] = [] private var tail: [UInt8] = [] private var totalByteCount: UInt = 0 + private var hasPotentialNonWhitespaceContent = false var isEmpty: Bool { totalByteCount == 0 } mutating func append(_ bytes: ArraySlice) { guard !bytes.isEmpty else { return } totalByteCount = saturatingSum(totalByteCount, UInt(bytes.count)) + if !hasPotentialNonWhitespaceContent, + bytes.contains(where: { !Self.isASCIIWhitespace($0) }) + { + hasPotentialNonWhitespaceContent = true + } let headCapacity = max(0, Self.edgeByteCount - head.count) let headBytes = bytes.prefix(headCapacity) @@ -1211,46 +1234,57 @@ private struct BoundedProcessLineBytes { headText = normalizedProcessOutputLine(head) tailText = normalizedProcessOutputLine(tail) } + guard hasPotentialNonWhitespaceContent else { return nil } guard !headText.isEmpty || !tailText.isEmpty || discardedRawByteCount > 0 else { return nil } return boundedProcessLine( head: headText, tail: tailText, - discardedRawByteCount: discardedRawByteCount + rawSourceByteCount: totalByteCount, + omittedSourceByteCountLowerBound: discardedRawByteCount ) } + + private static func isASCIIWhitespace(_ byte: UInt8) -> Bool { + switch byte { + case 0x09...0x0d, 0x20: + true + default: + false + } + } } private struct BoundedProcessLine { let text: String - let sourceByteCount: UInt - let omittedByteCount: UInt + let rawSourceByteCount: UInt + let omittedSourceByteCountLowerBound: UInt + let didOmitContent: Bool - var retainedSourceByteCount: UInt { - saturatingSubtract(sourceByteCount, omittedByteCount) + var rawSourceByteCountRemainingAfterLowerBound: UInt { + saturatingSubtract(rawSourceByteCount, omittedSourceByteCountLowerBound) } } private func boundedProcessLine( head: String, tail: String, - discardedRawByteCount: UInt + rawSourceByteCount: UInt, + omittedSourceByteCountLowerBound: UInt ) -> BoundedProcessLine { let maximumByteCount = BoundedProcessOutput.maximumRenderedLineByteCount let separator = " … " - let headByteCount = UInt(head.utf8.count) - let tailByteCount = UInt(tail.utf8.count) - let sourceByteCount = saturatingSum( - saturatingSum(headByteCount, tailByteCount), - discardedRawByteCount - ) - if discardedRawByteCount == 0, tail.isEmpty, head.utf8.count <= maximumByteCount { + if omittedSourceByteCountLowerBound == 0, + tail.isEmpty, + head.utf8.count <= maximumByteCount + { return BoundedProcessLine( text: head, - sourceByteCount: sourceByteCount, - omittedByteCount: 0 + rawSourceByteCount: rawSourceByteCount, + omittedSourceByteCountLowerBound: 0, + didOmitContent: false ) } @@ -1260,12 +1294,14 @@ private func boundedProcessLine( let tailBudget = contentBudget - headBudget let retainedHead = prefixFittingUTF8(head, maximumByteCount: headBudget) let retainedTail = suffixFittingUTF8(sourceTail, maximumByteCount: tailBudget) - let retainedSourceByteCount = UInt(retainedHead.utf8.count + retainedTail.utf8.count) - let omittedByteCount = saturatingSubtract(sourceByteCount, retainedSourceByteCount) + // Do not infer raw byte counts from terminal-safe UTF-8. Escaping and replacement decoding + // can expand source bytes, so only raw bytes discarded before normalization contribute to + // this lower bound. return BoundedProcessLine( text: retainedHead + separator + retainedTail, - sourceByteCount: sourceByteCount, - omittedByteCount: omittedByteCount + rawSourceByteCount: rawSourceByteCount, + omittedSourceByteCountLowerBound: omittedSourceByteCountLowerBound, + didOmitContent: true ) } diff --git a/Tests/PrivateHeaderKitToolingTests/StreamingProcessRunnerTests.swift b/Tests/PrivateHeaderKitToolingTests/StreamingProcessRunnerTests.swift index 92e579a..b306ff1 100644 --- a/Tests/PrivateHeaderKitToolingTests/StreamingProcessRunnerTests.swift +++ b/Tests/PrivateHeaderKitToolingTests/StreamingProcessRunnerTests.swift @@ -890,7 +890,7 @@ struct StreamingProcessRunnerTests { #expect(lines.first?.hasSuffix("x") == true) #expect(lines.first?.utf8.count == BoundedProcessOutput.maximumRenderedLineByteCount) #expect(lines.contains("final-diagnostic")) - #expect(lines.contains { $0.hasPrefix("[omitted 0 lines and ") }) + #expect(lines.contains { $0.hasPrefix("[omitted 0 lines and at least ") }) #expect( standardError.utf8.count <= BoundedProcessOutput.maximumRenderedByteCount @@ -972,15 +972,16 @@ struct StreamingProcessRunnerTests { #expect(output.lines == [ "line-1", "line-2", "line-3", "line-4", "line-5", "line-6", "line-7", - "line-8", "[omitted 4 lines and 27 bytes]", "line-13", "line-14", "line-15", + "line-8", "[omitted 4 lines and at least 27 bytes]", "line-13", "line-14", + "line-15", "line-16", "line-17", "line-18", "line-19", "emoji:😀", ]) #expect(output.omittedLineCount == 4) - #expect(output.omittedByteCount == 27) + #expect(output.omittedSourceByteCountLowerBound == 27) } @Test func collectorRetainsBothEndsOfLongLineWithinOneKiB() throws { - let line = "BEGIN-" + String(repeating: "x", count: 5_000) + "-END" + let line = "BEGIN-" + String(repeating: "x", count: 20_000) + "-END" let bytes = Array(line.utf8) var collector = BoundedProcessOutputCollector() for chunkStart in stride(from: 0, to: bytes.count, by: 997) { @@ -993,12 +994,42 @@ struct StreamingProcessRunnerTests { #expect(retained.contains(" … ")) #expect(retained.hasSuffix("-END")) #expect(retained.utf8.count == BoundedProcessOutput.maximumRenderedLineByteCount) - #expect(output.lines.last?.hasPrefix("[omitted 0 lines and ") == true) + #expect(output.lines.last?.hasPrefix("[omitted 0 lines and at least ") == true) #expect(output.omittedLineCount == 0) - #expect(output.omittedByteCount > 0) + #expect( + output.omittedSourceByteCountLowerBound + == UInt( + line.utf8.count + - (BoundedProcessOutput.maximumRenderedLineByteCount * 8 * 2) + ) + ) #expect(output.text.utf8.count <= BoundedProcessOutput.maximumRenderedByteCount) } + @Test func omittedWholeLineCountsRawSourceBytesBeforeTerminalEscaping() { + var lines = (1...8).map { "head-\($0)" } + lines.append("\u{001B}") + lines += (1...8).map { "tail-\($0)" } + + let output = BoundedProcessOutput(lines: lines) + + #expect(output.omittedLineCount == 1) + #expect(output.omittedSourceByteCountLowerBound == 1) + #expect(output.lines[8] == "[omitted 1 lines and at least 1 bytes]") + } + + @Test func hugeASCIIWhitespaceLineIsNotClassifiedAsEmittedContent() { + var collector = BoundedProcessOutputCollector() + collector.consume(Array(repeating: UInt8(ascii: " "), count: 100_000)) + + let output = collector.finish() + + #expect(output.isEmpty) + #expect(output.lines.isEmpty) + #expect(output.omittedLineCount == 0) + #expect(output.omittedSourceByteCountLowerBound == 0) + } + @Test func collectorTerminalSafesControlsAndInvalidIncompleteUTF8() throws { var collector = BoundedProcessOutputCollector() collector.consume([ @@ -1031,10 +1062,10 @@ struct StreamingProcessRunnerTests { #expect(output.lines.count == BoundedProcessOutput.maximumRenderedLineCount) #expect(output.lines.first?.hasPrefix(#"head\u{001b}"#) == true) - #expect(output.lines[8].hasPrefix("[omitted 9984 lines and ")) + #expect(output.lines[8].hasPrefix("[omitted 9984 lines and at least ")) #expect(output.lines.last == "line-10000") #expect(output.omittedLineCount == 9_984) - #expect(output.omittedByteCount > 0) + #expect(output.omittedSourceByteCountLowerBound > 0) #expect( output.lines.allSatisfy { $0.utf8.count <= BoundedProcessOutput.maximumRenderedLineByteCount From 9e3a5ddeab9ca3b2f2033369ff4864221c855a75 Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Tue, 25 Aug 2026 17:14:52 +0900 Subject: [PATCH 07/14] Tighten raw helper handshake identity --- Package.swift | 1 + .../CurrentProcessExecutableIdentity.swift | 41 +++++++++++++++++++ .../PrivateHeaderKitHelperProtocol.swift | 18 +------- .../PrivateHeaderKitRawDumpMain.swift | 8 ++-- .../PrivateHeaderKitHelperProtocolTests.swift | 34 +++------------ .../PrivateHeaderKitRawDumpTests.swift | 12 ++++-- 6 files changed, 61 insertions(+), 53 deletions(-) diff --git a/Package.swift b/Package.swift index d70dfc7..4785758 100644 --- a/Package.swift +++ b/Package.swift @@ -192,6 +192,7 @@ let package = Package( .testTarget( name: "PrivateHeaderKitRawDumpTests", dependencies: [ + "PrivateHeaderKitExecutableResolution", "PrivateHeaderKitHelperProtocol", "PrivateHeaderKitRawDumpCore", "PrivateHeaderKitTestSupport", diff --git a/Sources/PrivateHeaderKitExecutableResolution/CurrentProcessExecutableIdentity.swift b/Sources/PrivateHeaderKitExecutableResolution/CurrentProcessExecutableIdentity.swift index 3f11e12..049499d 100644 --- a/Sources/PrivateHeaderKitExecutableResolution/CurrentProcessExecutableIdentity.swift +++ b/Sources/PrivateHeaderKitExecutableResolution/CurrentProcessExecutableIdentity.swift @@ -6,15 +6,21 @@ import MachO #endif package enum CurrentProcessExecutableIdentityError: Error, Equatable, Sendable { + case executablePathInspectionFailed case imageInspectionFailed + case missingExecutableName case missingMachOUUID } extension CurrentProcessExecutableIdentityError: CustomStringConvertible, LocalizedError { package var description: String { switch self { + case .executablePathInspectionFailed: + "failed to inspect the running executable path" case .imageInspectionFailed: "failed to inspect the running executable image" + case .missingExecutableName: + "the running executable path has no file name" case .missingMachOUUID: "the running executable has no Mach-O UUID" } @@ -23,6 +29,41 @@ extension CurrentProcessExecutableIdentityError: CustomStringConvertible, Locali package var errorDescription: String? { description } } +package func currentProcessExecutableName() throws -> String { +#if canImport(Darwin) + var requiredByteCount: UInt32 = 1 + var buffer = [CChar](repeating: 0, count: Int(requiredByteCount)) + var result = buffer.withUnsafeMutableBufferPointer { + _NSGetExecutablePath($0.baseAddress, &requiredByteCount) + } + if result != 0 { + guard requiredByteCount > 1 else { + throw CurrentProcessExecutableIdentityError.executablePathInspectionFailed + } + buffer = [CChar](repeating: 0, count: Int(requiredByteCount)) + result = buffer.withUnsafeMutableBufferPointer { + _NSGetExecutablePath($0.baseAddress, &requiredByteCount) + } + } + guard result == 0, + let nullIndex = buffer.firstIndex(of: 0) + else { + throw CurrentProcessExecutableIdentityError.executablePathInspectionFailed + } + let pathBytes = buffer[.. UUID { #if canImport(Darwin) guard let header = _dyld_get_image_header(0), diff --git a/Sources/PrivateHeaderKitHelperProtocol/PrivateHeaderKitHelperProtocol.swift b/Sources/PrivateHeaderKitHelperProtocol/PrivateHeaderKitHelperProtocol.swift index 4ade95e..7d59904 100644 --- a/Sources/PrivateHeaderKitHelperProtocol/PrivateHeaderKitHelperProtocol.swift +++ b/Sources/PrivateHeaderKitHelperProtocol/PrivateHeaderKitHelperProtocol.swift @@ -41,22 +41,19 @@ package struct PrivateHeaderKitRawDumpProcessHandshake: Codable, Hashable, Senda package let helperStartedAtUnixMicroseconds: Int64 package let executableName: String package let executableMachOUUID: UUID - package let producerVersion: String package init( invocationID: UUID, processIdentifier: Int32, helperStartedAtUnixMicroseconds: Int64, executableName: String, - executableMachOUUID: UUID, - producerVersion: String = PrivateHeaderKitBuildInfo.version + executableMachOUUID: UUID ) throws { try Self.validateInvocationID(invocationID) try Self.validateProcessIdentifier(processIdentifier) try Self.validateStartTime(helperStartedAtUnixMicroseconds) try Self.validateExecutableName(executableName) try Self.validateExecutableMachOUUID(executableMachOUUID) - let producerVersion = try Self.validateProducerVersion(producerVersion) self.schemaVersion = Self.currentSchemaVersion self.invocationID = invocationID @@ -64,7 +61,6 @@ package struct PrivateHeaderKitRawDumpProcessHandshake: Codable, Hashable, Senda self.helperStartedAtUnixMicroseconds = helperStartedAtUnixMicroseconds self.executableName = executableName self.executableMachOUUID = executableMachOUUID - self.producerVersion = producerVersion } package init(from decoder: any Decoder) throws { @@ -91,7 +87,6 @@ package struct PrivateHeaderKitRawDumpProcessHandshake: Codable, Hashable, Senda ) let executableName = try container.decode(String.self, forKey: .executableName) let executableMachOUUID = try container.decode(UUID.self, forKey: .executableMachOUUID) - let producerVersion = try container.decode(String.self, forKey: .producerVersion) try Self.validateInvocationID(invocationID) try Self.validateProcessIdentifier(processIdentifier) @@ -105,7 +100,6 @@ package struct PrivateHeaderKitRawDumpProcessHandshake: Codable, Hashable, Senda self.helperStartedAtUnixMicroseconds = helperStartedAtUnixMicroseconds self.executableName = executableName self.executableMachOUUID = executableMachOUUID - self.producerVersion = try Self.validateProducerVersion(producerVersion) } package func encoded() throws -> Data { @@ -186,14 +180,6 @@ package struct PrivateHeaderKitRawDumpProcessHandshake: Codable, Hashable, Senda } } - private static func validateProducerVersion(_ value: String) throws -> String { - do { - return try PrivateHeaderKitProducerVersion.validated(value) - } catch { - throw ValidationError.invalidProducerVersion - } - } - private enum CodingKeys: String, CodingKey, CaseIterable { case schemaVersion case invocationID @@ -201,7 +187,6 @@ package struct PrivateHeaderKitRawDumpProcessHandshake: Codable, Hashable, Senda case helperStartedAtUnixMicroseconds case executableName case executableMachOUUID - case producerVersion } private struct FieldKey: CodingKey { @@ -226,7 +211,6 @@ package struct PrivateHeaderKitRawDumpProcessHandshake: Codable, Hashable, Senda case invalidStartTime(Int64) case invalidExecutableName case invalidExecutableMachOUUID - case invalidProducerVersion case invalidFieldSet } } diff --git a/Sources/PrivateHeaderKitRawDumpCore/PrivateHeaderKitRawDumpMain.swift b/Sources/PrivateHeaderKitRawDumpCore/PrivateHeaderKitRawDumpMain.swift index 4fb6caf..cc6b80d 100644 --- a/Sources/PrivateHeaderKitRawDumpCore/PrivateHeaderKitRawDumpMain.swift +++ b/Sources/PrivateHeaderKitRawDumpCore/PrivateHeaderKitRawDumpMain.swift @@ -283,9 +283,8 @@ func writeProcessHandshakeIfRequested( _ options: DumpOptions, processIdentifier: () -> Int32 = { getpid() }, nowUnixMicroseconds: () throws -> Int64 = currentRealtimeUnixMicroseconds, - executableName: () -> String = { Bundle.main.executableURL?.lastPathComponent ?? "" }, - executableMachOUUID: () throws -> UUID = currentProcessMachOUUID, - producerVersion: String = PrivateHeaderKitBuildInfo.version + executableName: () throws -> String = currentProcessExecutableName, + executableMachOUUID: () throws -> UUID = currentProcessMachOUUID ) throws { guard let invocationID = options.processHandshakeID, let reportURL = options.processHandshakeReportURL @@ -297,8 +296,7 @@ func writeProcessHandshakeIfRequested( processIdentifier: processIdentifier(), helperStartedAtUnixMicroseconds: nowUnixMicroseconds(), executableName: executableName(), - executableMachOUUID: executableMachOUUID(), - producerVersion: producerVersion + executableMachOUUID: executableMachOUUID() ) try handshake.encoded().write(to: reportURL, options: .atomic) } diff --git a/Tests/PrivateHeaderKitHelperProtocolTests/PrivateHeaderKitHelperProtocolTests.swift b/Tests/PrivateHeaderKitHelperProtocolTests/PrivateHeaderKitHelperProtocolTests.swift index dac3282..b22c9de 100644 --- a/Tests/PrivateHeaderKitHelperProtocolTests/PrivateHeaderKitHelperProtocolTests.swift +++ b/Tests/PrivateHeaderKitHelperProtocolTests/PrivateHeaderKitHelperProtocolTests.swift @@ -13,8 +13,7 @@ struct PrivateHeaderKitHelperProtocolTests { processIdentifier: 4_242, helperStartedAtUnixMicroseconds: 1_700_000_000_123_456, executableName: "privateheaderkit-sim-helper", - executableMachOUUID: executableUUID, - producerVersion: "v1.2.3" + executableMachOUUID: executableUUID ) let data = try handshake.encoded() @@ -36,7 +35,6 @@ struct PrivateHeaderKitHelperProtocolTests { "helperStartedAtUnixMicroseconds", "executableName", "executableMachOUUID", - "producerVersion", ] ) #expect(object["schemaVersion"] as? Int == 1) @@ -56,8 +54,7 @@ struct PrivateHeaderKitHelperProtocolTests { processIdentifier: 4_242, helperStartedAtUnixMicroseconds: 1_700_000_000_123_456, executableName: "privateheaderkit-sim-helper", - executableMachOUUID: UUID(uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee")!, - producerVersion: "v1.2.3" + executableMachOUUID: UUID(uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee")! ) #expect( @@ -87,7 +84,7 @@ struct PrivateHeaderKitHelperProtocolTests { } } - @Test func rawDumpProcessHandshakeEnforcesStringBounds() throws { + @Test func rawDumpProcessHandshakeEnforcesExecutableNameBounds() throws { let invocationID = UUID(uuidString: "11111111-2222-3333-4444-555555555555")! let executableUUID = UUID(uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee")! let maximum = try PrivateHeaderKitRawDumpProcessHandshake( @@ -98,11 +95,7 @@ struct PrivateHeaderKitHelperProtocolTests { repeating: "\\", count: PrivateHeaderKitRawDumpProcessHandshake.maximumExecutableNameUTF8Count ), - executableMachOUUID: executableUUID, - producerVersion: String( - repeating: "\\", - count: PrivateHeaderKitProducerVersion.maximumUTF8Count - ) + executableMachOUUID: executableUUID ) #expect( @@ -118,28 +111,14 @@ struct PrivateHeaderKitHelperProtocolTests { repeating: "e", count: PrivateHeaderKitRawDumpProcessHandshake.maximumExecutableNameUTF8Count + 1 ), - executableMachOUUID: executableUUID, - producerVersion: "v1.2.3" - ) - } - #expect(throws: PrivateHeaderKitRawDumpProcessHandshake.ValidationError.self) { - _ = try PrivateHeaderKitRawDumpProcessHandshake( - invocationID: invocationID, - processIdentifier: 4_242, - helperStartedAtUnixMicroseconds: 1_700_000_000_123_456, - executableName: "privateheaderkit-sim-helper", - executableMachOUUID: executableUUID, - producerVersion: String( - repeating: "v", - count: PrivateHeaderKitProducerVersion.maximumUTF8Count + 1 - ) + executableMachOUUID: executableUUID ) } } @Test func rawDumpProcessHandshakeStrictlyRejectsInvalidFields() { let expectedID = UUID(uuidString: "11111111-2222-3333-4444-555555555555")! - let validFields = #""schemaVersion":1,"invocationID":"11111111-2222-3333-4444-555555555555","processIdentifier":4242,"helperStartedAtUnixMicroseconds":1700000000123456,"executableName":"privateheaderkit-sim-helper","executableMachOUUID":"aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee","producerVersion":"v1.2.3""# + let validFields = #""schemaVersion":1,"invocationID":"11111111-2222-3333-4444-555555555555","processIdentifier":4242,"helperStartedAtUnixMicroseconds":1700000000123456,"executableName":"privateheaderkit-sim-helper","executableMachOUUID":"aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee""# let payloads = [ "{\(validFields),\"path\":\"/private/var/tmp/helper\"}", "{\(validFields.replacingOccurrences(of: "\"schemaVersion\":1", with: "\"schemaVersion\":2"))}", @@ -148,7 +127,6 @@ struct PrivateHeaderKitHelperProtocolTests { "{\(validFields.replacingOccurrences(of: "\"helperStartedAtUnixMicroseconds\":1700000000123456", with: "\"helperStartedAtUnixMicroseconds\":0"))}", "{\(validFields.replacingOccurrences(of: "privateheaderkit-sim-helper", with: "/private/helper"))}", "{\(validFields.replacingOccurrences(of: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", with: "00000000-0000-0000-0000-000000000000"))}", - "{\(validFields.replacingOccurrences(of: "\"producerVersion\":\"v1.2.3\"", with: "\"producerVersion\":\"\""))}", ] for payload in payloads { diff --git a/Tests/PrivateHeaderKitRawDumpTests/PrivateHeaderKitRawDumpTests.swift b/Tests/PrivateHeaderKitRawDumpTests/PrivateHeaderKitRawDumpTests.swift index 27fa391..52ae83c 100644 --- a/Tests/PrivateHeaderKitRawDumpTests/PrivateHeaderKitRawDumpTests.swift +++ b/Tests/PrivateHeaderKitRawDumpTests/PrivateHeaderKitRawDumpTests.swift @@ -2,6 +2,7 @@ import Foundation import Dispatch import MachOKit @_spi(Core) @_spi(Diagnostics) @testable import MachOObjCSection +import PrivateHeaderKitExecutableResolution import PrivateHeaderKitHelperProtocol import Testing #if canImport(PrivateHeaderKitRawDumpRuntimeObjC) @@ -32,6 +33,13 @@ private enum FakeRawMachOLoadError: Error, CustomStringConvertible { @Suite struct PrivateHeaderKitRawDumpArgumentTests { + @Test func currentProcessExecutableNameContainsNoPathComponents() throws { + let executableName = try currentProcessExecutableName() + + #expect(!executableName.isEmpty) + #expect(!executableName.contains("/")) + } + @Test func parseArgumentsPopulatesOptions() { let args = [ "-o", "/tmp/out", @@ -162,8 +170,7 @@ struct PrivateHeaderKitRawDumpArgumentTests { processIdentifier: { 4_242 }, nowUnixMicroseconds: { 1_700_000_000_123_456 }, executableName: { "privateheaderkit-sim-helper" }, - executableMachOUUID: { executableUUID }, - producerVersion: "v1.2.3" + executableMachOUUID: { executableUUID } ) let data = try Data(contentsOf: reportURL) @@ -176,7 +183,6 @@ struct PrivateHeaderKitRawDumpArgumentTests { #expect(handshake.helperStartedAtUnixMicroseconds == 1_700_000_000_123_456) #expect(handshake.executableName == "privateheaderkit-sim-helper") #expect(handshake.executableMachOUUID == executableUUID) - #expect(handshake.producerVersion == "v1.2.3") #expect(!String(decoding: data, as: UTF8.self).contains("/private/var/tmp")) } From 9331c5c1f94fae12cf07d5c13315584e4e915404 Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Tue, 25 Aug 2026 17:20:34 +0900 Subject: [PATCH 08/14] fix: persist actionable raw helper failure capsules --- .../PrivateHeaderKitGenerationClient.swift | 160 +++++++++- .../RawHelperFailureCapsule.swift | 116 +++++++ .../PrivateHeaderGenerationRawDumping.swift | 12 + .../PrivateHeaderKitCLITests.swift | 80 ++++- ...ivateHeaderKitProgressRenderingTests.swift | 57 ++++ .../RawHelperFailureCapsuleTests.swift | 288 ++++++++++++++++++ ...PrivateHeaderGenerationExecutorTests.swift | 57 ++++ 7 files changed, 750 insertions(+), 20 deletions(-) create mode 100644 Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift create mode 100644 Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift diff --git a/Sources/PrivateHeaderKitCLI/PrivateHeaderKitGenerationClient.swift b/Sources/PrivateHeaderKitCLI/PrivateHeaderKitGenerationClient.swift index 5eab085..fe8a80b 100644 --- a/Sources/PrivateHeaderKitCLI/PrivateHeaderKitGenerationClient.swift +++ b/Sources/PrivateHeaderKitCLI/PrivateHeaderKitGenerationClient.swift @@ -105,41 +105,173 @@ func runPrivateHeaderKitRawDump( _ invocation: PrivateHeaderGeneration.RawDumping.Invocation, processRunner: any CommandRunning ) async throws -> PrivateHeaderGeneration.RawDumping.Result { - let processResult = try await processRunner.runBuffered( - invocation.command, - env: invocation.environment, - cwd: nil - ) + let processResult: StreamingCommandResult + do { + processResult = try await processRunner.runBuffered( + invocation.command, + env: invocation.environment, + cwd: nil + ) + } catch { + try? FileManager.default.removeItem(at: invocation.processHandshakeReportURL) + try? FileManager.default.removeItem(at: invocation.diagnosticsReportURL) + throw error + } guard processResult.status == 0, !processResult.wasKilled else { + let handshake = consumeFailedRawDumpProcessHandshake( + at: invocation.processHandshakeReportURL, + expectedInvocationID: invocation.processHandshakeID + ) try? FileManager.default.removeItem(at: invocation.diagnosticsReportURL) + let recognizesSimulatorChildTermination: Bool + if case .simulator = invocation.executionMode { + recognizesSimulatorChildTermination = true + } else { + recognizesSimulatorChildTermination = false + } return PrivateHeaderGeneration.RawDumping.Result( terminationStatus: processResult.status, wasKilled: processResult.wasKilled, - failureSummary: processResult.lastLines.isEmpty - ? nil - : processResult.lastLines.joined(separator: "\n") + failureSummary: RawHelperFailureCapsule( + processResult: processResult, + handshake: handshake, + recognizesSimulatorChildTermination: recognizesSimulatorChildTermination + ).text ) } - let report = try consumeRawDumpDiagnosticsReport( - at: invocation.diagnosticsReportURL - ) + do { + _ = try consumeSuccessfulRawDumpProcessHandshake( + at: invocation.processHandshakeReportURL, + expectedInvocationID: invocation.processHandshakeID + ) + } catch { + try? FileManager.default.removeItem(at: invocation.diagnosticsReportURL) + throw error + } + let report = try consumeRawDumpDiagnosticsReport(at: invocation.diagnosticsReportURL) return PrivateHeaderGeneration.RawDumping.Result( terminationStatus: processResult.status, wasKilled: processResult.wasKilled, - failureSummary: processResult.lastLines.isEmpty - ? nil - : processResult.lastLines.joined(separator: "\n"), + failureSummary: nil, diagnosticsReport: report ) } +private func consumeSuccessfulRawDumpProcessHandshake( + at reportURL: URL, + expectedInvocationID: UUID, + fileManager: FileManager = .default +) throws -> PrivateHeaderKitRawDumpProcessHandshake { + let handshake: PrivateHeaderKitRawDumpProcessHandshake + do { + handshake = try readRawDumpProcessHandshake( + at: reportURL, + expectedInvocationID: expectedInvocationID, + fileManager: fileManager + ) + } catch { + try? fileManager.removeItem(at: reportURL) + throw error + } + + do { + try fileManager.removeItem(at: reportURL) + } catch { + throw PrivateHeaderGeneration.RawDumping.ContractError + .processHandshakeCleanupFailed( + path: reportURL.path, + reason: String(describing: error) + ) + } + return handshake +} + +private func consumeFailedRawDumpProcessHandshake( + at reportURL: URL, + expectedInvocationID: UUID, + fileManager: FileManager = .default +) -> RawHelperFailureCapsule.HandshakeObservation { + defer { try? fileManager.removeItem(at: reportURL) } + do { + return .available( + try readRawDumpProcessHandshake( + at: reportURL, + expectedInvocationID: expectedInvocationID, + fileManager: fileManager + ) + ) + } catch PrivateHeaderGeneration.RawDumping.ContractError.missingProcessHandshake { + return .missing + } catch { + return .invalid + } +} + +private func readRawDumpProcessHandshake( + at reportURL: URL, + expectedInvocationID: UUID, + fileManager: FileManager +) throws -> PrivateHeaderKitRawDumpProcessHandshake { + let path = reportURL.path + guard fileManager.fileExists(atPath: path) else { + throw PrivateHeaderGeneration.RawDumping.ContractError.missingProcessHandshake(path) + } + + let data: Data + do { + let values = try reportURL.resourceValues(forKeys: [ + .isRegularFileKey, + .fileSizeKey, + ]) + guard values.isRegularFile == true else { + throw PrivateHeaderGeneration.RawDumping.ContractError.invalidProcessHandshake( + path: path, + reason: "report is not a regular file" + ) + } + guard let fileSize = values.fileSize else { + throw PrivateHeaderGeneration.RawDumping.ContractError.invalidProcessHandshake( + path: path, + reason: "report size is unavailable" + ) + } + guard fileSize <= PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount else { + throw PrivateHeaderGeneration.RawDumping.ContractError.processHandshakeTooLarge( + path: path, + actual: fileSize, + maximum: PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount + ) + } + data = try Data(contentsOf: reportURL) + guard data.count <= PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount else { + throw PrivateHeaderGeneration.RawDumping.ContractError.processHandshakeTooLarge( + path: path, + actual: data.count, + maximum: PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount + ) + } + return try PrivateHeaderKitRawDumpProcessHandshake.decode( + data, + expectedInvocationID: expectedInvocationID + ) + } catch let error as PrivateHeaderGeneration.RawDumping.ContractError { + throw error + } catch { + throw PrivateHeaderGeneration.RawDumping.ContractError.invalidProcessHandshake( + path: path, + reason: String(describing: error) + ) + } +} + private func consumeRawDumpDiagnosticsReport( at reportURL: URL, fileManager: FileManager = .default ) throws -> PrivateHeaderKitRawDumpDiagnosticsReport { let path = reportURL.path guard fileManager.fileExists(atPath: path) else { + try? fileManager.removeItem(at: reportURL) throw PrivateHeaderGeneration.RawDumping.ContractError.missingDiagnosticsReport(path) } diff --git a/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift b/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift new file mode 100644 index 0000000..5e391af --- /dev/null +++ b/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift @@ -0,0 +1,116 @@ +import Foundation +import PrivateHeaderKitHelperProtocol +import PrivateHeaderKitTooling + +struct RawHelperFailureCapsule: Equatable, Sendable { + static let maximumLineCount = BoundedProcessOutput.maximumRenderedLineCount + 1 + static let maximumRenderedByteCount = 24 * 1_024 + + enum HandshakeObservation: Equatable, Sendable { + case available(PrivateHeaderKitRawDumpProcessHandshake) + case missing + case invalid + } + + let text: String + + init( + processResult: StreamingCommandResult, + handshake: HandshakeObservation, + recognizesSimulatorChildTermination: Bool + ) { + var reportedChildSignal: Int32? + var diagnosticLines = processResult.emittedOutput.lines.filter { line in + guard recognizesSimulatorChildTermination, + let signal = Self.simulatorChildTerminationSignal(in: line) + else { + return true + } + reportedChildSignal = signal + return false + } + if diagnosticLines.isEmpty { + diagnosticLines = ["helper diagnostic output: none emitted"] + } + + let termination: String + if let reportedChildSignal { + termination = "child_signal(\(reportedChildSignal))" + } else if processResult.wasKilled { + termination = "wrapper_signal(\(processResult.status))" + } else { + termination = "exit(\(processResult.status))" + } + + let identity: String + switch handshake { + case .available(let value): + identity = [ + "handshake=available", + "helper=\(value.executableName)", + "lc_uuid=\(value.executableMachOUUID.uuidString.lowercased())", + "pid=\(value.processIdentifier)", + "start_us=\(value.helperStartedAtUnixMicroseconds)", + ].joined(separator: " ") + case .missing: + identity = "identity=unavailable handshake=missing" + case .invalid: + identity = "identity=unavailable handshake=invalid" + } + + let terminationObservedAt = processResult + .terminationObservedAtUnixEpochMicroseconds + .map(String.init) + ?? "unavailable" + let headline = [ + "privateheaderkit raw helper error:", + "capsule=v1", + "termination=\(termination)", + "wrapper_status=\(processResult.status)", + "wrapper_killed=\(processResult.wasKilled)", + identity, + "termination_observed_us=\(terminationObservedAt)", + ].joined(separator: " ") + let text = (diagnosticLines + [headline]).joined(separator: "\n") + precondition( + diagnosticLines.count + 1 <= Self.maximumLineCount, + "raw helper failure capsule exceeded its line count" + ) + precondition( + text.utf8.count <= Self.maximumRenderedByteCount, + "raw helper failure capsule exceeded its rendered byte count" + ) + self.text = text + } + + private static func simulatorChildTerminationSignal(in line: String) -> Int32? { + let prefix = "Child process terminated with signal " + guard line.hasPrefix(prefix) else { return nil } + let suffix = line.dropFirst(prefix.count) + guard let separator = suffix.firstIndex(of: ":") else { return nil } + let signalText = suffix[..= 0x30 && $0 <= 0x39 }), + let signal = Int32(String(signalText)), + signal > 0 + else { + return nil + } + let space = suffix.index(after: separator) + guard space < suffix.endIndex, + suffix[space] == " " + else { + return nil + } + let reasonStart = suffix.index(after: space) + guard reasonStart < suffix.endIndex else { return nil } + let reason = suffix[reasonStart...] + guard !reason.hasPrefix(" "), + !reason.hasSuffix(" "), + line == prefix + String(signal) + ": " + String(reason) + else { + return nil + } + return signal + } +} diff --git a/Sources/PrivateHeaderKitCore/PrivateHeaderGenerationRawDumping.swift b/Sources/PrivateHeaderKitCore/PrivateHeaderGenerationRawDumping.swift index c079bc1..8c844be 100644 --- a/Sources/PrivateHeaderKitCore/PrivateHeaderGenerationRawDumping.swift +++ b/Sources/PrivateHeaderKitCore/PrivateHeaderGenerationRawDumping.swift @@ -147,6 +147,10 @@ extension PrivateHeaderGeneration { extension PrivateHeaderGeneration.RawDumping { package enum ContractError: Error, Equatable, CustomStringConvertible, Sendable { + case missingProcessHandshake(String) + case invalidProcessHandshake(path: String, reason: String) + case processHandshakeTooLarge(path: String, actual: Int, maximum: Int) + case processHandshakeCleanupFailed(path: String, reason: String) case missingDiagnosticsReport(String) case invalidDiagnosticsReport(path: String, reason: String) case diagnosticsReportTooLarge(path: String, actual: Int, maximum: Int) @@ -154,6 +158,14 @@ extension PrivateHeaderGeneration.RawDumping { package var description: String { switch self { + case .missingProcessHandshake(let path): + "raw helper contract failure: successful helper did not write process handshake at \(path)" + case .invalidProcessHandshake(let path, let reason): + "raw helper contract failure: invalid process handshake at \(path): \(reason)" + case .processHandshakeTooLarge(let path, let actual, let maximum): + "raw helper contract failure: process handshake at \(path) is \(actual) bytes; maximum is \(maximum)" + case .processHandshakeCleanupFailed(let path, let reason): + "raw helper contract failure: could not remove process handshake at \(path): \(reason)" case .missingDiagnosticsReport(let path): "raw helper contract failure: successful helper did not write diagnostics report at \(path)" case .invalidDiagnosticsReport(let path, let reason): diff --git a/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitCLITests.swift b/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitCLITests.swift index af58024..2ca835d 100644 --- a/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitCLITests.swift +++ b/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitCLITests.swift @@ -1118,6 +1118,7 @@ struct PrivateHeaderKitCLIExecutionTests { for: inventoryCommand ) await runner.setStreamingHandler { command, _, _ in + try writeRawDumpProcessHandshake(for: command) guard let outputIndex = command.firstIndex(of: "-o"), outputIndex + 1 < command.count else { throw ToolingError.message("raw dump command is missing its output directory") } @@ -1203,7 +1204,20 @@ struct PrivateHeaderKitCLIExecutionTests { #expect(result.terminationStatus == 19) #expect(!result.wasKilled) - #expect(result.failureSummary == "helper-warning\nfatal-tail") + #expect( + result.failureSummary + == "helper-warning\nfatal-tail\n" + + "privateheaderkit raw helper error: capsule=v1 termination=exit(19) " + + "wrapper_status=19 wrapper_killed=false " + + "handshake=available helper=privateheaderkit-raw-helper " + + "lc_uuid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee pid=4242 " + + "start_us=1700000000123456 termination_observed_us=unavailable" + ) + #expect( + !FileManager.default.fileExists( + atPath: invocation.processHandshakeReportURL.path + ) + ) #expect(!FileManager.default.fileExists(atPath: invocation.diagnosticsReportURL.path)) } @@ -1222,6 +1236,7 @@ struct PrivateHeaderKitCLIExecutionTests { ) let runner = RecordingCommandRunner() await runner.setStreamingHandler { command, _, _ in + try writeRawDumpProcessHandshake(for: command) guard let reportIndex = command.firstIndex(of: "--diagnostics-report") else { throw ToolingError.message("missing diagnostics report argument") } @@ -1236,7 +1251,11 @@ struct PrivateHeaderKitCLIExecutionTests { omittedDiagnosticCount: 2 ) ).write(to: URL(fileURLWithPath: command[reportIndex + 1]), options: .atomic) - return StreamingCommandResult(status: 0, wasKilled: false, lastLines: []) + return StreamingCommandResult( + status: 0, + wasKilled: false, + lastLines: ["successful helper stderr is not a persisted failure"] + ) } let result = try await runPrivateHeaderKitRawDump(invocation, processRunner: runner) @@ -1244,6 +1263,12 @@ struct PrivateHeaderKitCLIExecutionTests { #expect(result.diagnostics.count == 1) #expect(result.diagnostics.first?.owner == "Objective-C protocol P") #expect(result.omittedDiagnosticCount == 2) + #expect(result.failureSummary == nil) + #expect( + !FileManager.default.fileExists( + atPath: invocation.processHandshakeReportURL.path + ) + ) #expect(!FileManager.default.fileExists(atPath: invocation.diagnosticsReportURL.path)) } @@ -1266,16 +1291,22 @@ struct PrivateHeaderKitCLIExecutionTests { ) ) let runner = RecordingCommandRunner() - if malformed { - await runner.setStreamingHandler { _, _, _ in + await runner.setStreamingHandler { command, _, _ in + try writeRawDumpProcessHandshake(for: command) + if malformed { try Data("not-json".utf8).write(to: invocation.diagnosticsReportURL) - return StreamingCommandResult(status: 0, wasKilled: false, lastLines: []) } + return StreamingCommandResult(status: 0, wasKilled: false, lastLines: []) } await #expect(throws: PrivateHeaderGeneration.RawDumping.ContractError.self) { _ = try await runPrivateHeaderKitRawDump(invocation, processRunner: runner) } + #expect( + !FileManager.default.fileExists( + atPath: invocation.processHandshakeReportURL.path + ) + ) #expect(!FileManager.default.fileExists(atPath: invocation.diagnosticsReportURL.path)) } } @@ -1294,7 +1325,8 @@ struct PrivateHeaderKitCLIExecutionTests { ) ) let runner = RecordingCommandRunner() - await runner.setStreamingHandler { _, _, _ in + await runner.setStreamingHandler { command, _, _ in + try writeRawDumpProcessHandshake(for: command) try Data( count: PrivateHeaderKitRawDumpDiagnosticsReport.maximumEncodedByteCount + 1 ).write(to: invocation.diagnosticsReportURL) @@ -1304,6 +1336,11 @@ struct PrivateHeaderKitCLIExecutionTests { await #expect(throws: PrivateHeaderGeneration.RawDumping.ContractError.self) { _ = try await runPrivateHeaderKitRawDump(invocation, processRunner: runner) } + #expect( + !FileManager.default.fileExists( + atPath: invocation.processHandshakeReportURL.path + ) + ) #expect(!FileManager.default.fileExists(atPath: invocation.diagnosticsReportURL.path)) } @@ -2635,6 +2672,36 @@ private func testPrivateHeaderKitHelperResolver( ) } +private func writeRawDumpProcessHandshake( + for command: [String], + processIdentifier: Int32 = 4_242, + helperStartedAtUnixMicroseconds: Int64 = 1_700_000_000_123_456, + executableName: String = "privateheaderkit-raw-helper", + executableMachOUUID: UUID = UUID( + uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" + )! +) throws { + guard let identifierIndex = command.firstIndex(of: "--process-handshake-id"), + identifierIndex + 1 < command.count, + let invocationID = UUID(uuidString: command[identifierIndex + 1]), + let reportIndex = command.firstIndex(of: "--process-handshake-report"), + reportIndex + 1 < command.count + else { + throw ToolingError.message("raw dump command is missing its process handshake") + } + let handshake = try PrivateHeaderKitRawDumpProcessHandshake( + invocationID: invocationID, + processIdentifier: processIdentifier, + helperStartedAtUnixMicroseconds: helperStartedAtUnixMicroseconds, + executableName: executableName, + executableMachOUUID: executableMachOUUID + ) + try handshake.encoded().write( + to: URL(fileURLWithPath: command[reportIndex + 1]), + options: .atomic + ) +} + private struct BufferedRawDumpProbeRunner: CommandRunning { func runCapture( _ command: [String], @@ -2674,6 +2741,7 @@ private struct BufferedRawDumpProbeRunner: CommandRunning { env: [String: String]?, cwd: URL? ) async throws -> StreamingCommandResult { + try writeRawDumpProcessHandshake(for: command) if let reportIndex = command.firstIndex(of: "--diagnostics-report") { try Data("not-json".utf8).write( to: URL(fileURLWithPath: command[reportIndex + 1]), diff --git a/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitProgressRenderingTests.swift b/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitProgressRenderingTests.swift index 8d36fa9..a6c43da 100644 --- a/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitProgressRenderingTests.swift +++ b/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitProgressRenderingTests.swift @@ -50,6 +50,63 @@ struct PrivateHeaderKitProgressRenderingTests { ]) } + @Test func rawHelperCapsuleUsesTheSameCanonicalHeadlineLiveAndFinal() { + let headline = "privateheaderkit raw helper error: capsule=v1 " + + "termination=child_signal(11) wrapper_status=11 wrapper_killed=false " + + "handshake=available " + + "helper=privateheaderkit-sim-helper " + + "lc_uuid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee pid=4242 " + + "start_us=1700000000123456 " + + "termination_observed_us=1777000111222333" + let capsule = "exception prefix\nfinal diagnostic tail\n" + headline + let liveOutput = ProgressTextRecorder() + let liveFailures = ProgressTextRecorder() + let liveRenderer = PrivateHeaderKitProgressOutputLogger( + outputLogger: { liveOutput.append($0) }, + failureLogger: { liveFailures.append($0) }, + artifactDirectory: URL(fileURLWithPath: "/tmp/generated-headers/source"), + inlineProgressEnabled: false, + startsTimer: false + ) + liveRenderer.report( + .targetFinished( + index: 1, + total: 1, + displayName: "Foo", + status: .partial, + failureSummary: capsule + ) + ) + + let finalOutput = ProgressTextRecorder() + renderPrivateHeaderKitRunSummary( + .init( + runID: .init(rawValue: "run-capsule"), + status: .partial, + targetCounts: .init(total: 1, partial: 1), + artifactDirectory: URL(fileURLWithPath: "/tmp/generated-headers/source"), + stateDatabaseURL: URL(fileURLWithPath: "/tmp/generation.sqlite"), + targetFailures: [ + .init( + targetID: "framework:Foo.framework", + displayName: "Foo", + status: .partial, + message: capsule + ) + ] + ), + sourceDisplayName: "iOS 27.0 beta (24A5390f)", + targetQuery: "Foo", + title: "Generation completed with failures", + outputLogger: { finalOutput.append($0) } + ) + + #expect(liveOutput.values.isEmpty) + #expect(liveFailures.values == ["[1/1] Foo partial", " " + headline]) + #expect(finalOutput.values.contains(" " + headline)) + #expect(concisePrivateHeaderKitDiagnostic(capsule) == headline) + } + @Test func terminalOutputCyclesDotsAndReusesTheSuccessfulTargetLine() { let output = ProgressTextRecorder() let failures = ProgressTextRecorder() diff --git a/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift b/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift new file mode 100644 index 0000000..00db68c --- /dev/null +++ b/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift @@ -0,0 +1,288 @@ +import Foundation +import PrivateHeaderKitCore +import PrivateHeaderKitHelperProtocol +import PrivateHeaderKitTestSupport +import PrivateHeaderKitTooling +import Testing + +@testable import PrivateHeaderKitCLI + +@Suite +struct RawHelperFailureCapsuleTests { + @Test func longExceptionRetainsCauseFirstApplicationFrameTailAndHeadline() throws { + var lines = [ + "*** Terminating app due to uncaught exception 'FixtureException', reason: 'fixture reason'", + "*** First throw call stack:", + "(", + "0 CoreFoundation fixture", + "1 libobjc fixture", + "2 privateheaderkit-sim-helper frame-zero", + "3 privateheaderkit-sim-helper frame-one", + "4 privateheaderkit-sim-helper frame-two", + ] + lines += (5...24).map { "\($0) filler frame \($0)" } + lines += [ + ")", + "libc++abi: terminating due to uncaught exception of type NSException", + "final-diagnostic-tail", + ] + let capsule = RawHelperFailureCapsule( + processResult: .init( + status: 19, + wasKilled: false, + emittedOutput: BoundedProcessOutput(lines: lines), + terminationObservedAtUnixEpochMicroseconds: 1_777_000_987_654_321 + ), + handshake: .available(try Self.helperHandshake(executableName: "privateheaderkit-sim-helper")), + recognizesSimulatorChildTermination: true + ) + let renderedLines = capsule.text.split(separator: "\n").map(String.init) + + #expect(renderedLines.first == lines.first) + #expect(renderedLines.contains("2 privateheaderkit-sim-helper frame-zero")) + #expect(renderedLines.contains(where: { $0.hasPrefix("[omitted ") })) + #expect(renderedLines[renderedLines.count - 3] == lines[lines.count - 2]) + #expect(renderedLines[renderedLines.count - 2] == "final-diagnostic-tail") + let expectedHeadline = "privateheaderkit raw helper error: capsule=v1 " + + "termination=exit(19) wrapper_status=19 wrapper_killed=false " + + "handshake=available " + + "helper=privateheaderkit-sim-helper " + + "lc_uuid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee pid=4242 " + + "start_us=1700000000123456 " + + "termination_observed_us=1777000987654321" + #expect(renderedLines.last == expectedHeadline) + #expect(renderedLines.count == RawHelperFailureCapsule.maximumLineCount) + } + + @Test func exactSimulatorSignalLineBecomesTerminationWithoutDiagnosticOutput() throws { + let capsule = RawHelperFailureCapsule( + processResult: .init( + status: 11, + wasKilled: false, + lastLines: ["Child process terminated with signal 11: Segmentation fault"], + terminationObservedAtUnixEpochMicroseconds: 1_777_000_111_222_333 + ), + handshake: .available(try Self.helperHandshake(executableName: "privateheaderkit-sim-helper")), + recognizesSimulatorChildTermination: true + ) + + let expected = "helper diagnostic output: none emitted\n" + + "privateheaderkit raw helper error: capsule=v1 " + + "termination=child_signal(11) wrapper_status=11 wrapper_killed=false " + + "handshake=available " + + "helper=privateheaderkit-sim-helper " + + "lc_uuid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee pid=4242 " + + "start_us=1700000000123456 " + + "termination_observed_us=1777000111222333" + #expect(capsule.text == expected) + } + + @Test func similarOrHostSignalTextIsNotAuthoritative() throws { + let similar = "Child process terminated with signal 11 - Segmentation fault" + let simulatorCapsule = RawHelperFailureCapsule( + processResult: .init(status: 19, wasKilled: false, lastLines: [similar]), + handshake: .missing, + recognizesSimulatorChildTermination: true + ) + let hostCapsule = RawHelperFailureCapsule( + processResult: .init( + status: 19, + wasKilled: false, + lastLines: ["Child process terminated with signal 11: Segmentation fault"] + ), + handshake: .missing, + recognizesSimulatorChildTermination: false + ) + + #expect(simulatorCapsule.text.hasPrefix(similar + "\n")) + #expect(simulatorCapsule.text.contains("termination=exit(19)")) + #expect(hostCapsule.text.contains("Child process terminated with signal 11")) + #expect(hostCapsule.text.contains("termination=exit(19)")) + } + + @Test func normalExitWithMissingIdentityIsExplicit() { + let capsule = RawHelperFailureCapsule( + processResult: .init(status: 19, wasKilled: false, lastLines: []), + handshake: .missing, + recognizesSimulatorChildTermination: false + ) + + #expect(capsule.text.hasPrefix("helper diagnostic output: none emitted\n")) + #expect(capsule.text.contains("termination=exit(19)")) + #expect(capsule.text.contains("identity=unavailable handshake=missing")) + #expect(capsule.text.contains("termination_observed_us=unavailable")) + } + + @Test func capsuleBoundsAndEnvelopePrivacyAreInvariant() throws { + let output = BoundedProcessOutput( + lines: (0..<100).map { index in + "line-\(index)-" + String(repeating: "x", count: 4_000) + } + ) + let capsule = RawHelperFailureCapsule( + processResult: .init(status: 19, wasKilled: false, emittedOutput: output), + handshake: .available(try Self.helperHandshake()), + recognizesSimulatorChildTermination: false + ) + let lines = capsule.text.split(separator: "\n", omittingEmptySubsequences: false) + + #expect(lines.count <= RawHelperFailureCapsule.maximumLineCount) + #expect(capsule.text.utf8.count <= RawHelperFailureCapsule.maximumRenderedByteCount) + #expect(!capsule.text.contains("/Users/private/RuntimeRoot")) + #expect(!capsule.text.contains("00000000-1111-2222-3333-444444444444")) + #expect(!capsule.text.contains("SIMCTL_CHILD_DYLD_ROOT_PATH")) + #expect(!capsule.text.contains("xcrun simctl spawn")) + #expect(!capsule.text.contains("producer=")) + } + + @Test func failedRunClassifiesMissingAndInvalidHandshakeAndCleansReports() async throws { + for invalid in [false, true] { + let fixture = try Self.rawDumpInvocationFixture() + defer { try? FileManager.default.removeItem(at: fixture.root) } + let runner = RecordingCommandRunner() + await runner.setStreamingHandler { _, _, _ in + if invalid { + try Data("not-json".utf8).write( + to: fixture.invocation.processHandshakeReportURL, + options: .atomic + ) + } + try JSONEncoder().encode( + PrivateHeaderKitRawDumpDiagnosticsReport(diagnostics: []) + ).write(to: fixture.invocation.diagnosticsReportURL, options: .atomic) + return StreamingCommandResult( + status: 19, + wasKilled: false, + lastLines: [], + terminationObservedAtUnixEpochMicroseconds: 1_777_000_222_333_444 + ) + } + + let result = try await runPrivateHeaderKitRawDump( + fixture.invocation, + processRunner: runner + ) + + #expect(result.failureSummary?.contains(invalid ? "handshake=invalid" : "handshake=missing") == true) + #expect(result.failureSummary?.contains("identity=unavailable") == true) + #expect( + (result.failureSummary?.split(separator: "\n").count ?? 0) + <= RawHelperFailureCapsule.maximumLineCount + ) + #expect( + !FileManager.default.fileExists( + atPath: fixture.invocation.processHandshakeReportURL.path + ) + ) + #expect( + !FileManager.default.fileExists( + atPath: fixture.invocation.diagnosticsReportURL.path + ) + ) + } + } + + @Test func successfulRunRequiresBoundedRegularInvocationBoundHandshake() async throws { + enum FixtureKind: CaseIterable, Sendable { + case missing + case malformed + case oversized + case directory + case wrongInvocation + } + + for kind in FixtureKind.allCases { + let fixture = try Self.rawDumpInvocationFixture() + defer { try? FileManager.default.removeItem(at: fixture.root) } + let runner = RecordingCommandRunner() + await runner.setStreamingHandler { _, _, _ in + switch kind { + case .missing: + break + case .malformed: + try Data("not-json".utf8).write( + to: fixture.invocation.processHandshakeReportURL, + options: .atomic + ) + case .oversized: + try Data( + count: PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount + 1 + ).write( + to: fixture.invocation.processHandshakeReportURL, + options: .atomic + ) + case .directory: + try FileManager.default.createDirectory( + at: fixture.invocation.processHandshakeReportURL, + withIntermediateDirectories: false + ) + case .wrongInvocation: + try Self.helperHandshake(invocationID: UUID()).encoded().write( + to: fixture.invocation.processHandshakeReportURL, + options: .atomic + ) + } + try JSONEncoder().encode( + PrivateHeaderKitRawDumpDiagnosticsReport(diagnostics: []) + ).write(to: fixture.invocation.diagnosticsReportURL, options: .atomic) + return StreamingCommandResult(status: 0, wasKilled: false, lastLines: []) + } + + await #expect(throws: PrivateHeaderGeneration.RawDumping.ContractError.self) { + _ = try await runPrivateHeaderKitRawDump( + fixture.invocation, + processRunner: runner + ) + } + #expect( + !FileManager.default.fileExists( + atPath: fixture.invocation.processHandshakeReportURL.path + ) + ) + #expect( + !FileManager.default.fileExists( + atPath: fixture.invocation.diagnosticsReportURL.path + ) + ) + } + } + + private static func helperHandshake( + invocationID: UUID = UUID(uuidString: "11111111-2222-3333-4444-555555555555")!, + executableName: String = "privateheaderkit-raw-helper" + ) throws -> PrivateHeaderKitRawDumpProcessHandshake { + try PrivateHeaderKitRawDumpProcessHandshake( + invocationID: invocationID, + processIdentifier: 4_242, + helperStartedAtUnixMicroseconds: 1_700_000_000_123_456, + executableName: executableName, + executableMachOUUID: UUID( + uuidString: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" + )! + ) + } + + private static func rawDumpInvocationFixture() throws -> ( + root: URL, + invocation: PrivateHeaderGeneration.RawDumping.Invocation + ) { + let root = FileManager.default.temporaryDirectory.appendingPathComponent( + "RawHelperFailureCapsuleTests-\(UUID().uuidString)", + isDirectory: true + ) + let stage = root.appendingPathComponent("stage", isDirectory: true) + try FileManager.default.createDirectory(at: stage, withIntermediateDirectories: true) + let invocation = PrivateHeaderGeneration.RawDumping.makeInvocation( + try .init( + helperURLs: .init( + host: root.appendingPathComponent("privateheaderkit-raw-helper"), + simulator: root.appendingPathComponent("privateheaderkit-sim-helper") + ), + executionMode: .host, + inputPath: "/System/Library/Frameworks/AppKit.framework", + stagingOutputDirectory: stage + ) + ) + return (root, invocation) + } +} diff --git a/Tests/PrivateHeaderKitCoreTests/PrivateHeaderGenerationExecutorTests.swift b/Tests/PrivateHeaderKitCoreTests/PrivateHeaderGenerationExecutorTests.swift index 80444cb..b99358c 100644 --- a/Tests/PrivateHeaderKitCoreTests/PrivateHeaderGenerationExecutorTests.swift +++ b/Tests/PrivateHeaderKitCoreTests/PrivateHeaderGenerationExecutorTests.swift @@ -1397,6 +1397,63 @@ struct PrivateHeaderGenerationExecutorTests { == .init(rawValue: "run-001")) } + @Test func rawHelperFailureCapsuleSurvivesPartialAndFailedPersistence() async throws { + let capsule = [ + "exception prefix", + "final diagnostic tail", + "privateheaderkit raw helper error: capsule=v1 termination=child_signal(11) " + + "wrapper_status=11 wrapper_killed=false " + + "handshake=available helper=privateheaderkit-sim-helper " + + "lc_uuid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee pid=4242 " + + "start_us=1700000000123456 termination_observed_us=1777000111222333", + ].joined(separator: "\n") + let cases: [( + name: String, + contents: String?, + targetStatus: PrivateHeaderGeneration.RunTargetStatus, + runStatus: PrivateHeaderGeneration.RunStatus + )] = [ + ("partial", "partial header", .partial, .partial), + ("failed", nil, .failed, .failed), + ] + + for testCase in cases { + let fixture = try ExecutorFixture() + defer { fixture.cleanup() } + try fixture.createFramework("Foo.framework") + let runID = PrivateHeaderGeneration.RunID( + rawValue: "run-capsule-\(testCase.name)" + ) + let executor = fixture.executor( + runner: RecordingRunner( + contents: testCase.contents, + result: .init(terminationStatus: 11, failureSummary: capsule) + ), + runID: runID.rawValue, + generationID: "generation-capsule-\(testCase.name)" + ) + + do { + _ = try await executor.run( + plan: try fixture.plan(.query("Foo"), resumeBehavior: .fresh) + ) + Issue.record("raw helper failure unexpectedly returned success") + } catch let PrivateHeaderGeneration.GenerationError.runFailed(failure) { + #expect(failure.summary.status == testCase.runStatus) + #expect(failure.summary.targetFailures.count == 1) + #expect(failure.summary.targetFailures.first?.status == testCase.targetStatus) + #expect(failure.summary.targetFailures.first?.message == capsule) + } + + let store = try GenerationStore(databaseURL: fixture.databaseURL) + let snapshot = try await store.runSnapshot(runID) + #expect(snapshot.status == testCase.runStatus) + #expect(snapshot.targets.count == 1) + #expect(snapshot.targets.first?.status == testCase.targetStatus) + #expect(snapshot.targets.first?.failureSummary == capsule) + } + } + @Test func zeroSuccessfulTargetsNeverCreateOrSwitchPointer() async throws { let fixture = try ExecutorFixture() defer { fixture.cleanup() } From 57e55f5be6fb34f266c02f6ba50f5a4ebb240570 Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Tue, 25 Aug 2026 17:24:18 +0900 Subject: [PATCH 09/14] fix: corroborate simulator child termination --- .../RawHelperFailureCapsule.swift | 23 +++++++------- .../RawHelperFailureCapsuleTests.swift | 30 +++++++++++++++++++ 2 files changed, 43 insertions(+), 10 deletions(-) diff --git a/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift b/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift index 5e391af..d3afb78 100644 --- a/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift +++ b/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift @@ -19,15 +19,18 @@ struct RawHelperFailureCapsule: Equatable, Sendable { handshake: HandshakeObservation, recognizesSimulatorChildTermination: Bool ) { - var reportedChildSignal: Int32? - var diagnosticLines = processResult.emittedOutput.lines.filter { line in - guard recognizesSimulatorChildTermination, - let signal = Self.simulatorChildTerminationSignal(in: line) - else { - return true - } + var diagnosticLines = processResult.emittedOutput.lines + let reportedChildSignal: Int32? + if recognizesSimulatorChildTermination, + !processResult.wasKilled, + let finalLine = diagnosticLines.last, + let signal = Self.simulatorChildTerminationSignal(in: finalLine), + signal == processResult.status + { reportedChildSignal = signal - return false + diagnosticLines.removeLast() + } else { + reportedChildSignal = nil } if diagnosticLines.isEmpty { diagnosticLines = ["helper diagnostic output: none emitted"] @@ -74,11 +77,11 @@ struct RawHelperFailureCapsule: Equatable, Sendable { let text = (diagnosticLines + [headline]).joined(separator: "\n") precondition( diagnosticLines.count + 1 <= Self.maximumLineCount, - "raw helper failure capsule exceeded its line count" + "RawHelperFailureCapsule must enforce its rendered line-count bound" ) precondition( text.utf8.count <= Self.maximumRenderedByteCount, - "raw helper failure capsule exceeded its rendered byte count" + "RawHelperFailureCapsule must enforce its rendered byte-count bound" ) self.text = text } diff --git a/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift b/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift index 00db68c..830902f 100644 --- a/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift +++ b/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift @@ -100,6 +100,36 @@ struct RawHelperFailureCapsuleTests { #expect(hostCapsule.text.contains("termination=exit(19)")) } + @Test func onlyCorroboratedFinalSimulatorSignalLineIsAuthoritative() { + let exactSignalLine = "Child process terminated with signal 11: Segmentation fault" + let nonfinalCapsule = RawHelperFailureCapsule( + processResult: .init( + status: 11, + wasKilled: false, + lastLines: [exactSignalLine, "later helper diagnostic"] + ), + handshake: .missing, + recognizesSimulatorChildTermination: true + ) + let mismatchedStatusCapsule = RawHelperFailureCapsule( + processResult: .init(status: 19, wasKilled: false, lastLines: [exactSignalLine]), + handshake: .missing, + recognizesSimulatorChildTermination: true + ) + let killedWrapperCapsule = RawHelperFailureCapsule( + processResult: .init(status: 11, wasKilled: true, lastLines: [exactSignalLine]), + handshake: .missing, + recognizesSimulatorChildTermination: true + ) + + #expect(nonfinalCapsule.text.hasPrefix(exactSignalLine + "\n")) + #expect(nonfinalCapsule.text.contains("termination=exit(11)")) + #expect(mismatchedStatusCapsule.text.hasPrefix(exactSignalLine + "\n")) + #expect(mismatchedStatusCapsule.text.contains("termination=exit(19)")) + #expect(killedWrapperCapsule.text.hasPrefix(exactSignalLine + "\n")) + #expect(killedWrapperCapsule.text.contains("termination=wrapper_signal(11)")) + } + @Test func normalExitWithMissingIdentityIsExplicit() { let capsule = RawHelperFailureCapsule( processResult: .init(status: 19, wasKilled: false, lastLines: []), From ee1f124ce6bce3219eabeb1a95d8e4fba268e8b2 Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Wed, 26 Aug 2026 06:48:38 +0900 Subject: [PATCH 10/14] fix: decode simulator child signal exit status --- .../RawHelperFailureCapsule.swift | 9 +++++++-- .../RawHelperFailureCapsuleTests.swift | 12 ++++++------ 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift b/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift index d3afb78..bdd5b02 100644 --- a/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift +++ b/Sources/PrivateHeaderKitCLI/RawHelperFailureCapsule.swift @@ -25,7 +25,7 @@ struct RawHelperFailureCapsule: Equatable, Sendable { !processResult.wasKilled, let finalLine = diagnosticLines.last, let signal = Self.simulatorChildTerminationSignal(in: finalLine), - signal == processResult.status + Self.simulatorWrapperStatus(for: signal) == processResult.status { reportedChildSignal = signal diagnosticLines.removeLast() @@ -95,7 +95,8 @@ struct RawHelperFailureCapsule: Equatable, Sendable { guard !signalText.isEmpty, signalText.utf8.allSatisfy({ $0 >= 0x30 && $0 <= 0x39 }), let signal = Int32(String(signalText)), - signal > 0 + signal > 0, + signal < 128 else { return nil } @@ -116,4 +117,8 @@ struct RawHelperFailureCapsule: Equatable, Sendable { } return signal } + + private static func simulatorWrapperStatus(for childSignal: Int32) -> Int32 { + 128 + childSignal + } } diff --git a/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift b/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift index 830902f..153f8af 100644 --- a/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift +++ b/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift @@ -57,7 +57,7 @@ struct RawHelperFailureCapsuleTests { @Test func exactSimulatorSignalLineBecomesTerminationWithoutDiagnosticOutput() throws { let capsule = RawHelperFailureCapsule( processResult: .init( - status: 11, + status: 139, wasKilled: false, lastLines: ["Child process terminated with signal 11: Segmentation fault"], terminationObservedAtUnixEpochMicroseconds: 1_777_000_111_222_333 @@ -68,7 +68,7 @@ struct RawHelperFailureCapsuleTests { let expected = "helper diagnostic output: none emitted\n" + "privateheaderkit raw helper error: capsule=v1 " - + "termination=child_signal(11) wrapper_status=11 wrapper_killed=false " + + "termination=child_signal(11) wrapper_status=139 wrapper_killed=false " + "handshake=available " + "helper=privateheaderkit-sim-helper " + "lc_uuid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee pid=4242 " @@ -104,7 +104,7 @@ struct RawHelperFailureCapsuleTests { let exactSignalLine = "Child process terminated with signal 11: Segmentation fault" let nonfinalCapsule = RawHelperFailureCapsule( processResult: .init( - status: 11, + status: 139, wasKilled: false, lastLines: [exactSignalLine, "later helper diagnostic"] ), @@ -112,7 +112,7 @@ struct RawHelperFailureCapsuleTests { recognizesSimulatorChildTermination: true ) let mismatchedStatusCapsule = RawHelperFailureCapsule( - processResult: .init(status: 19, wasKilled: false, lastLines: [exactSignalLine]), + processResult: .init(status: 11, wasKilled: false, lastLines: [exactSignalLine]), handshake: .missing, recognizesSimulatorChildTermination: true ) @@ -123,9 +123,9 @@ struct RawHelperFailureCapsuleTests { ) #expect(nonfinalCapsule.text.hasPrefix(exactSignalLine + "\n")) - #expect(nonfinalCapsule.text.contains("termination=exit(11)")) + #expect(nonfinalCapsule.text.contains("termination=exit(139)")) #expect(mismatchedStatusCapsule.text.hasPrefix(exactSignalLine + "\n")) - #expect(mismatchedStatusCapsule.text.contains("termination=exit(19)")) + #expect(mismatchedStatusCapsule.text.contains("termination=exit(11)")) #expect(killedWrapperCapsule.text.hasPrefix(exactSignalLine + "\n")) #expect(killedWrapperCapsule.text.contains("termination=wrapper_signal(11)")) } From 1c60f34b94b9f56f4132245c9e7f30baa8585148 Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Wed, 26 Aug 2026 06:59:03 +0900 Subject: [PATCH 11/14] docs: record issue 81 runtime validation --- Docs/issue-remediation-progress.md | 39 ++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/Docs/issue-remediation-progress.md b/Docs/issue-remediation-progress.md index 6c88736..160c630 100644 --- a/Docs/issue-remediation-progress.md +++ b/Docs/issue-remediation-progress.md @@ -65,3 +65,42 @@ Required validation: output volume. - The capsule survives through `runTargets.failureSummary` and both terminal and nonterminal failed-target rendering. + +Implementation completed: + +- `BoundedProcessOutput` now owns terminal-safe combined-stream head/tail + retention and raw-source omission lower bounds. `StreamingCommandResult` + carries that value plus the wrapper termination-observation timestamp. +- Every raw helper invocation has a distinct process-handshake report. The + helper writes its validated PID, executable name, LC_UUID, invocation ID, + and start timestamp before loading the requested target. +- `runPrivateHeaderKitRawDump` consumes and removes both reports on every + success/failure/throw path and builds one bounded failure capsule on a + nonzero helper result. +- Simulator child termination is recognized only from the exact final + `simctl` line when the wrapper's normal exit status corroborates the POSIX + `128 + signal` convention. The wrapper line is then replaced by the typed + child-signal field instead of being duplicated as arbitrary output. +- Executor/store/rendering tests confirm that the exact capsule is the existing + `runTargets.failureSummary`; no persistence schema changed. + +Validation completed: + +- `swift test --force-resolved-versions` passed after integration. +- The focused capsule suite passed with 8 tests after the measured `simctl` + exit-status correction. +- A release-mode run against the exact iOS 27.0 beta `24A5390f` runtime and + `AXSpringBoardServerInstance` reproduced its expected uncaught exception as + run `run-d455b470-6ec7-4955-9157-7bc90c082a47`. +- SQLite retained the exception name/reason, first frames, omission marker, + terminal frames, and canonical headline in 17 lines / 1,725 bytes. Database + integrity was `ok` with no foreign-key violations. +- The headline reported `child_signal(6)`, wrapper status `134`, helper PID + `28709`, LC_UUID `31c43965-06ab-3d01-b413-8db66023c8d9`, start microseconds, + and termination-observation microseconds. +- Crash Reporter independently recorded the same PID, LC_UUID, helper name, + and `SIGABRT`/code 6, with capture time between helper start and observed + termination. +- The run-owned Simulator was deleted and the SDK-runtime override was restored + to its default. The isolated output was moved recoverably to + `/Users/kn/.Trash/privateheaderkit-issue81-runtime-MTbctA`. From a47b0f253f589fdcefd710d045531b0be3d4d7da Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Wed, 26 Aug 2026 07:15:20 +0900 Subject: [PATCH 12/14] fix: read raw helper reports from bounded descriptors --- .../PrivateHeaderKitGenerationClient.swift | 101 ++++------- .../PrivateHeaderKitCLI/RawDumpReportIO.swift | 170 ++++++++++++++++++ .../PrivateHeaderKitCLITests.swift | 62 ++++++- .../RawDumpReportIOTests.swift | 128 +++++++++++++ .../RawHelperFailureCapsuleTests.swift | 54 +++++- 5 files changed, 438 insertions(+), 77 deletions(-) create mode 100644 Sources/PrivateHeaderKitCLI/RawDumpReportIO.swift create mode 100644 Tests/PrivateHeaderKitCLITests/RawDumpReportIOTests.swift diff --git a/Sources/PrivateHeaderKitCLI/PrivateHeaderKitGenerationClient.swift b/Sources/PrivateHeaderKitCLI/PrivateHeaderKitGenerationClient.swift index fe8a80b..ac3db51 100644 --- a/Sources/PrivateHeaderKitCLI/PrivateHeaderKitGenerationClient.swift +++ b/Sources/PrivateHeaderKitCLI/PrivateHeaderKitGenerationClient.swift @@ -167,8 +167,7 @@ private func consumeSuccessfulRawDumpProcessHandshake( do { handshake = try readRawDumpProcessHandshake( at: reportURL, - expectedInvocationID: expectedInvocationID, - fileManager: fileManager + expectedInvocationID: expectedInvocationID ) } catch { try? fileManager.removeItem(at: reportURL) @@ -197,8 +196,7 @@ private func consumeFailedRawDumpProcessHandshake( return .available( try readRawDumpProcessHandshake( at: reportURL, - expectedInvocationID: expectedInvocationID, - fileManager: fileManager + expectedInvocationID: expectedInvocationID ) ) } catch PrivateHeaderGeneration.RawDumping.ContractError.missingProcessHandshake { @@ -210,47 +208,35 @@ private func consumeFailedRawDumpProcessHandshake( private func readRawDumpProcessHandshake( at reportURL: URL, - expectedInvocationID: UUID, - fileManager: FileManager + expectedInvocationID: UUID ) throws -> PrivateHeaderKitRawDumpProcessHandshake { let path = reportURL.path - guard fileManager.fileExists(atPath: path) else { - throw PrivateHeaderGeneration.RawDumping.ContractError.missingProcessHandshake(path) - } - let data: Data do { - let values = try reportURL.resourceValues(forKeys: [ - .isRegularFileKey, - .fileSizeKey, - ]) - guard values.isRegularFile == true else { - throw PrivateHeaderGeneration.RawDumping.ContractError.invalidProcessHandshake( - path: path, - reason: "report is not a regular file" - ) - } - guard let fileSize = values.fileSize else { - throw PrivateHeaderGeneration.RawDumping.ContractError.invalidProcessHandshake( - path: path, - reason: "report size is unavailable" - ) - } - guard fileSize <= PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount else { + data = try RawDumpReportIO.read( + at: reportURL, + maximumByteCount: PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount + ) + } catch let error as RawDumpReportIO.Failure { + switch error { + case .missing: + throw PrivateHeaderGeneration.RawDumping.ContractError + .missingProcessHandshake(path) + case .tooLarge(let actual): throw PrivateHeaderGeneration.RawDumping.ContractError.processHandshakeTooLarge( path: path, - actual: fileSize, + actual: actual, maximum: PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount ) - } - data = try Data(contentsOf: reportURL) - guard data.count <= PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount else { - throw PrivateHeaderGeneration.RawDumping.ContractError.processHandshakeTooLarge( + default: + throw PrivateHeaderGeneration.RawDumping.ContractError.invalidProcessHandshake( path: path, - actual: data.count, - maximum: PrivateHeaderKitRawDumpProcessHandshake.maximumEncodedByteCount + reason: error.description ) } + } + + do { return try PrivateHeaderKitRawDumpProcessHandshake.decode( data, expectedInvocationID: expectedInvocationID @@ -270,47 +256,30 @@ private func consumeRawDumpDiagnosticsReport( fileManager: FileManager = .default ) throws -> PrivateHeaderKitRawDumpDiagnosticsReport { let path = reportURL.path - guard fileManager.fileExists(atPath: path) else { - try? fileManager.removeItem(at: reportURL) - throw PrivateHeaderGeneration.RawDumping.ContractError.missingDiagnosticsReport(path) - } - let data: Data do { - let values = try reportURL.resourceValues(forKeys: [ - .isRegularFileKey, - .fileSizeKey, - ]) - guard values.isRegularFile == true else { - throw PrivateHeaderGeneration.RawDumping.ContractError.invalidDiagnosticsReport( - path: path, - reason: "report is not a regular file" - ) - } - guard let fileSize = values.fileSize else { - throw PrivateHeaderGeneration.RawDumping.ContractError.invalidDiagnosticsReport( - path: path, - reason: "report size is unavailable" - ) - } - guard fileSize <= PrivateHeaderKitRawDumpDiagnosticsReport.maximumEncodedByteCount else { + data = try RawDumpReportIO.read( + at: reportURL, + maximumByteCount: PrivateHeaderKitRawDumpDiagnosticsReport.maximumEncodedByteCount + ) + } catch let error as RawDumpReportIO.Failure { + try? fileManager.removeItem(at: reportURL) + switch error { + case .missing: + throw PrivateHeaderGeneration.RawDumping.ContractError + .missingDiagnosticsReport(path) + case .tooLarge(let actual): throw PrivateHeaderGeneration.RawDumping.ContractError.diagnosticsReportTooLarge( path: path, - actual: fileSize, + actual: actual, maximum: PrivateHeaderKitRawDumpDiagnosticsReport.maximumEncodedByteCount ) - } - data = try Data(contentsOf: reportURL) - guard data.count <= PrivateHeaderKitRawDumpDiagnosticsReport.maximumEncodedByteCount else { - throw PrivateHeaderGeneration.RawDumping.ContractError.diagnosticsReportTooLarge( + default: + throw PrivateHeaderGeneration.RawDumping.ContractError.invalidDiagnosticsReport( path: path, - actual: data.count, - maximum: PrivateHeaderKitRawDumpDiagnosticsReport.maximumEncodedByteCount + reason: error.description ) } - } catch let error as PrivateHeaderGeneration.RawDumping.ContractError { - try? fileManager.removeItem(at: reportURL) - throw error } catch { try? fileManager.removeItem(at: reportURL) throw PrivateHeaderGeneration.RawDumping.ContractError.invalidDiagnosticsReport( diff --git a/Sources/PrivateHeaderKitCLI/RawDumpReportIO.swift b/Sources/PrivateHeaderKitCLI/RawDumpReportIO.swift new file mode 100644 index 0000000..d78acb4 --- /dev/null +++ b/Sources/PrivateHeaderKitCLI/RawDumpReportIO.swift @@ -0,0 +1,170 @@ +import Foundation + +#if canImport(Darwin) +import Darwin +#elseif canImport(Glibc) +import Glibc +#endif + +enum RawDumpReportIO { + enum Failure: Error, Equatable, CustomStringConvertible, Sendable { + case missing + case openFailed(errno: Int32) + case inspectionFailed(errno: Int32) + case notRegularFile + case invalidFileSize + case tooLarge(actual: Int) + case readFailed(errno: Int32) + + var description: String { + switch self { + case .missing: + "report is missing" + case .openFailed(let errorCode): + "could not open report: errno \(errorCode)" + case .inspectionFailed(let errorCode): + "could not inspect report: errno \(errorCode)" + case .notRegularFile: + "report is not a regular file" + case .invalidFileSize: + "report size is invalid" + case .tooLarge(let actual): + "report exceeds its maximum size after \(actual) bytes" + case .readFailed(let errorCode): + "could not read report: errno \(errorCode)" + } + } + } + + final class OpenedFile { + private let descriptor: Int32 + + init(at url: URL) throws { + let openedDescriptor = url.path.withCString { path in + systemOpen( + path, + O_RDONLY | O_CLOEXEC | O_NOFOLLOW | O_NONBLOCK | O_NOCTTY + ) + } + guard openedDescriptor >= 0 else { + let errorCode = errno + if errorCode == ENOENT { + throw Failure.missing + } + throw Failure.openFailed(errno: errorCode) + } + descriptor = openedDescriptor + } + + deinit { + _ = systemClose(descriptor) + } + + func read(maximumByteCount: Int) throws -> Data { + precondition( + maximumByteCount >= 0 && maximumByteCount < Int.max, + "RawDumpReportIO owns a finite nonnegative maximum byte count" + ) + + var metadata = stat() + guard systemFstat(descriptor, &metadata) == 0 else { + throw Failure.inspectionFailed(errno: errno) + } + guard metadata.st_mode & mode_t(S_IFMT) == mode_t(S_IFREG) else { + throw Failure.notRegularFile + } + guard metadata.st_size >= 0, + let declaredByteCount = Int(exactly: metadata.st_size) + else { + throw Failure.invalidFileSize + } + guard declaredByteCount <= maximumByteCount else { + throw Failure.tooLarge(actual: declaredByteCount) + } + + let overflowByteCount = maximumByteCount + 1 + var contents = Data() + contents.reserveCapacity(declaredByteCount) + var buffer = [UInt8]( + repeating: 0, + count: min(64 * 1_024, overflowByteCount) + ) + + while contents.count < overflowByteCount { + let requestedByteCount = min( + buffer.count, + overflowByteCount - contents.count + ) + let readByteCount = buffer.withUnsafeMutableBytes { bytes in + systemRead(descriptor, bytes.baseAddress, requestedByteCount) + } + if readByteCount > 0 { + contents.append(contentsOf: buffer.prefix(readByteCount)) + continue + } + if readByteCount == 0 { + break + } + let errorCode = errno + if errorCode == EINTR { + continue + } + throw Failure.readFailed(errno: errorCode) + } + + guard contents.count <= maximumByteCount else { + throw Failure.tooLarge(actual: contents.count) + } + return contents + } + } + + static func read( + at url: URL, + maximumByteCount: Int + ) throws -> Data { + try OpenedFile(at: url).read(maximumByteCount: maximumByteCount) + } +} + +private func systemOpen( + _ path: UnsafePointer, + _ flags: Int32 +) -> Int32 { +#if canImport(Darwin) + Darwin.open(path, flags) +#elseif canImport(Glibc) + Glibc.open(path, flags) +#endif +} + +private func systemFstat( + _ descriptor: Int32, + _ metadata: UnsafeMutablePointer +) -> Int32 { +#if canImport(Darwin) + Darwin.fstat(descriptor, metadata) +#elseif canImport(Glibc) + Glibc.fstat(descriptor, metadata) +#endif +} + +private func systemRead( + _ descriptor: Int32, + _ buffer: UnsafeMutableRawPointer?, + _ byteCount: Int +) -> Int { +#if canImport(Darwin) + Darwin.read(descriptor, buffer, byteCount) +#elseif canImport(Glibc) + Glibc.read(descriptor, buffer, byteCount) +#endif +} + +private func systemClose(_ descriptor: Int32) -> Int32 { +#if canImport(Darwin) + Darwin.close(descriptor) +#elseif canImport(Glibc) + Glibc.close(descriptor) +#endif +} diff --git a/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitCLITests.swift b/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitCLITests.swift index 2ca835d..56e06e5 100644 --- a/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitCLITests.swift +++ b/Tests/PrivateHeaderKitCLITests/PrivateHeaderKitCLITests.swift @@ -1272,13 +1272,19 @@ struct PrivateHeaderKitCLIExecutionTests { #expect(!FileManager.default.fileExists(atPath: invocation.diagnosticsReportURL.path)) } - @Test func successfulRawDumpRejectsMissingOrMalformedDiagnosticsReport() async throws { + @Test func successfulRawDumpRejectsMissingMalformedOrNonRegularDiagnosticsReport() async throws { + enum FixtureKind: CaseIterable, Sendable { + case missing + case malformed + case directory + } + let root = try temporaryDirectory() defer { try? FileManager.default.removeItem(at: root) } let stage = root.appendingPathComponent("stage", isDirectory: true) try FileManager.default.createDirectory(at: stage, withIntermediateDirectories: true) - for malformed in [false, true] { + for kind in FixtureKind.allCases { let invocation = PrivateHeaderGeneration.RawDumping.makeInvocation( try .init( helperURLs: .init( @@ -1293,14 +1299,51 @@ struct PrivateHeaderKitCLIExecutionTests { let runner = RecordingCommandRunner() await runner.setStreamingHandler { command, _, _ in try writeRawDumpProcessHandshake(for: command) - if malformed { + switch kind { + case .missing: + break + case .malformed: try Data("not-json".utf8).write(to: invocation.diagnosticsReportURL) + case .directory: + try FileManager.default.createDirectory( + at: invocation.diagnosticsReportURL, + withIntermediateDirectories: false + ) } return StreamingCommandResult(status: 0, wasKilled: false, lastLines: []) } - await #expect(throws: PrivateHeaderGeneration.RawDumping.ContractError.self) { - _ = try await runPrivateHeaderKitRawDump(invocation, processRunner: runner) + switch kind { + case .missing: + await #expect( + throws: PrivateHeaderGeneration.RawDumping.ContractError + .missingDiagnosticsReport(invocation.diagnosticsReportURL.path) + ) { + _ = try await runPrivateHeaderKitRawDump( + invocation, + processRunner: runner + ) + } + case .directory: + await #expect( + throws: PrivateHeaderGeneration.RawDumping.ContractError + .invalidDiagnosticsReport( + path: invocation.diagnosticsReportURL.path, + reason: "report is not a regular file" + ) + ) { + _ = try await runPrivateHeaderKitRawDump( + invocation, + processRunner: runner + ) + } + case .malformed: + await #expect(throws: PrivateHeaderGeneration.RawDumping.ContractError.self) { + _ = try await runPrivateHeaderKitRawDump( + invocation, + processRunner: runner + ) + } } #expect( !FileManager.default.fileExists( @@ -1333,7 +1376,14 @@ struct PrivateHeaderKitCLIExecutionTests { return StreamingCommandResult(status: 0, wasKilled: false, lastLines: []) } - await #expect(throws: PrivateHeaderGeneration.RawDumping.ContractError.self) { + await #expect( + throws: PrivateHeaderGeneration.RawDumping.ContractError + .diagnosticsReportTooLarge( + path: invocation.diagnosticsReportURL.path, + actual: PrivateHeaderKitRawDumpDiagnosticsReport.maximumEncodedByteCount + 1, + maximum: PrivateHeaderKitRawDumpDiagnosticsReport.maximumEncodedByteCount + ) + ) { _ = try await runPrivateHeaderKitRawDump(invocation, processRunner: runner) } #expect( diff --git a/Tests/PrivateHeaderKitCLITests/RawDumpReportIOTests.swift b/Tests/PrivateHeaderKitCLITests/RawDumpReportIOTests.swift new file mode 100644 index 0000000..a2c8a9c --- /dev/null +++ b/Tests/PrivateHeaderKitCLITests/RawDumpReportIOTests.swift @@ -0,0 +1,128 @@ +import Foundation +import Testing + +#if canImport(Darwin) +import Darwin +#elseif canImport(Glibc) +import Glibc +#endif + +@testable import PrivateHeaderKitCLI + +@Suite +struct RawDumpReportIOTests { + @Test func openedReportKeepsItsInodeWhenPathIsReplaced() throws { + let root = try Self.temporaryDirectory() + defer { try? FileManager.default.removeItem(at: root) } + let reportURL = root.appendingPathComponent("report.json") + let replacementURL = root.appendingPathComponent("replacement.json") + let openedContents = Data("opened-inode".utf8) + let replacementContents = Data("replacement-inode".utf8) + try openedContents.write(to: reportURL) + + let openedReport = try RawDumpReportIO.OpenedFile(at: reportURL) + try replacementContents.write(to: replacementURL) + let renameResult = replacementURL.path.withCString { replacementPath in + reportURL.path.withCString { reportPath in + systemRename(replacementPath, reportPath) + } + } + #expect(renameResult == 0) + + #expect( + try openedReport.read(maximumByteCount: 1_024) == openedContents + ) + #expect( + try RawDumpReportIO.read(at: reportURL, maximumByteCount: 1_024) + == replacementContents + ) + } + + @Test func oversizedRegularReportIsRejectedWithItsObservedSize() throws { + let root = try Self.temporaryDirectory() + defer { try? FileManager.default.removeItem(at: root) } + let reportURL = root.appendingPathComponent("oversized.json") + try Data(count: 65).write(to: reportURL) + + #expect(throws: RawDumpReportIO.Failure.tooLarge(actual: 65)) { + _ = try RawDumpReportIO.read(at: reportURL, maximumByteCount: 64) + } + } + + @Test func nonRegularAndSymbolicLinkReportsAreRejected() throws { + let root = try Self.temporaryDirectory() + defer { try? FileManager.default.removeItem(at: root) } + + let directoryURL = root.appendingPathComponent("directory", isDirectory: true) + try FileManager.default.createDirectory( + at: directoryURL, + withIntermediateDirectories: false + ) + #expect(throws: RawDumpReportIO.Failure.notRegularFile) { + _ = try RawDumpReportIO.read(at: directoryURL, maximumByteCount: 64) + } + + let fifoURL = root.appendingPathComponent("fifo") + let fifoResult = fifoURL.path.withCString { path in + systemMakeFIFO(path, mode_t(0o600)) + } + #expect(fifoResult == 0) + #expect(throws: RawDumpReportIO.Failure.notRegularFile) { + _ = try RawDumpReportIO.read(at: fifoURL, maximumByteCount: 64) + } + + #expect(throws: RawDumpReportIO.Failure.notRegularFile) { + _ = try RawDumpReportIO.read( + at: URL(fileURLWithPath: "/dev/null"), + maximumByteCount: 64 + ) + } + + let targetURL = root.appendingPathComponent("target.json") + let symbolicLinkURL = root.appendingPathComponent("link.json") + try Data("target".utf8).write(to: targetURL) + try FileManager.default.createSymbolicLink( + at: symbolicLinkURL, + withDestinationURL: targetURL + ) + do { + _ = try RawDumpReportIO.read(at: symbolicLinkURL, maximumByteCount: 64) + Issue.record("expected symbolic-link report rejection") + } catch RawDumpReportIO.Failure.openFailed { + // O_NOFOLLOW rejects the final symbolic-link component before any bytes are read. + } catch { + Issue.record("unexpected symbolic-link report error: \(error)") + } + } + + private static func temporaryDirectory() throws -> URL { + let url = FileManager.default.temporaryDirectory.appendingPathComponent( + "RawDumpReportIOTests-\(UUID().uuidString)", + isDirectory: true + ) + try FileManager.default.createDirectory(at: url, withIntermediateDirectories: true) + return url + } +} + +private func systemRename( + _ source: UnsafePointer, + _ destination: UnsafePointer +) -> Int32 { +#if canImport(Darwin) + Darwin.rename(source, destination) +#elseif canImport(Glibc) + Glibc.rename(source, destination) +#endif +} + +private func systemMakeFIFO( + _ path: UnsafePointer, + _ permissions: mode_t +) -> Int32 { +#if canImport(Darwin) + Darwin.mkfifo(path, permissions) +#elseif canImport(Glibc) + Glibc.mkfifo(path, permissions) +#endif +} diff --git a/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift b/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift index 153f8af..0c336d1 100644 --- a/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift +++ b/Tests/PrivateHeaderKitCLITests/RawHelperFailureCapsuleTests.swift @@ -258,11 +258,55 @@ struct RawHelperFailureCapsuleTests { return StreamingCommandResult(status: 0, wasKilled: false, lastLines: []) } - await #expect(throws: PrivateHeaderGeneration.RawDumping.ContractError.self) { - _ = try await runPrivateHeaderKitRawDump( - fixture.invocation, - processRunner: runner - ) + switch kind { + case .missing: + await #expect( + throws: PrivateHeaderGeneration.RawDumping.ContractError + .missingProcessHandshake( + fixture.invocation.processHandshakeReportURL.path + ) + ) { + _ = try await runPrivateHeaderKitRawDump( + fixture.invocation, + processRunner: runner + ) + } + case .oversized: + await #expect( + throws: PrivateHeaderGeneration.RawDumping.ContractError + .processHandshakeTooLarge( + path: fixture.invocation.processHandshakeReportURL.path, + actual: PrivateHeaderKitRawDumpProcessHandshake + .maximumEncodedByteCount + 1, + maximum: PrivateHeaderKitRawDumpProcessHandshake + .maximumEncodedByteCount + ) + ) { + _ = try await runPrivateHeaderKitRawDump( + fixture.invocation, + processRunner: runner + ) + } + case .directory: + await #expect( + throws: PrivateHeaderGeneration.RawDumping.ContractError + .invalidProcessHandshake( + path: fixture.invocation.processHandshakeReportURL.path, + reason: "report is not a regular file" + ) + ) { + _ = try await runPrivateHeaderKitRawDump( + fixture.invocation, + processRunner: runner + ) + } + case .malformed, .wrongInvocation: + await #expect(throws: PrivateHeaderGeneration.RawDumping.ContractError.self) { + _ = try await runPrivateHeaderKitRawDump( + fixture.invocation, + processRunner: runner + ) + } } #expect( !FileManager.default.fileExists( From 0b80c1d79359dd34311108e0621b4885843b06ea Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Wed, 26 Aug 2026 07:16:09 +0900 Subject: [PATCH 13/14] docs: record bounded report I/O validation --- Docs/issue-remediation-progress.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/Docs/issue-remediation-progress.md b/Docs/issue-remediation-progress.md index 160c630..0860fd1 100644 --- a/Docs/issue-remediation-progress.md +++ b/Docs/issue-remediation-progress.md @@ -77,6 +77,10 @@ Implementation completed: - `runPrivateHeaderKitRawDump` consumes and removes both reports on every success/failure/throw path and builds one bounded failure capsule on a nonzero helper result. +- `RawDumpReportIO` opens each report with no-follow, nonblocking, no-controlling- + terminal, and close-on-exec flags, then performs regular-file inspection and + a `maximum + 1` bounded read on that same descriptor. This removes the + check/reopen race for both handshake and diagnostics reports. - Simulator child termination is recognized only from the exact final `simctl` line when the wrapper's normal exit status corroborates the POSIX `128 + signal` convention. The wrapper line is then replaced by the typed @@ -89,6 +93,10 @@ Validation completed: - `swift test --force-resolved-versions` passed after integration. - The focused capsule suite passed with 8 tests after the measured `simctl` exit-status correction. +- The CLI suite passed 90 tests under both Swift 6.4 / Xcode 27 and Swift 6.3.2 / + Xcode 26.5 after the descriptor-bound report reader was added. Deterministic + tests cover path replacement after open, oversized files, symlinks, + directories, FIFOs, devices, and typed contract-error mapping. - A release-mode run against the exact iOS 27.0 beta `24A5390f` runtime and `AXSpringBoardServerInstance` reproduced its expected uncaught exception as run `run-d455b470-6ec7-4955-9157-7bc90c082a47`. From 39f8140983ee77f4a76c95b5fa48fd2a7ffef8ef Mon Sep 17 00:00:00 2001 From: Kazuki Nakashima <65545348+lynnswap@users.noreply.github.com> Date: Wed, 26 Aug 2026 07:22:57 +0900 Subject: [PATCH 14/14] chore: remove issue 81 progress ledger --- Docs/issue-remediation-progress.md | 114 ----------------------------- 1 file changed, 114 deletions(-) delete mode 100644 Docs/issue-remediation-progress.md diff --git a/Docs/issue-remediation-progress.md b/Docs/issue-remediation-progress.md deleted file mode 100644 index 0860fd1..0000000 --- a/Docs/issue-remediation-progress.md +++ /dev/null @@ -1,114 +0,0 @@ -# Issue remediation progress - -Base: `main` at `88f36e65223f874e8ce13fa4846ef517f1203146` - -## Delivery order - -1. #81 actionable bounded raw-helper crash diagnostics -2. #83 bounded Objective-C table and loaded-image reads - -Each issue is delivered as an independent Ready PR targeting `main`. The next -issue starts only after the current PR is review-clean and merged. - -## Current issue: #81 - -Branch: `codex/issue-81-actionable-crash-diagnostics` - -Verified evidence: - -- Signal-only helper failures currently persist only a generic termination - sentence, without the child process identity needed to correlate an OS - incident report. -- Long uncaught-exception output retains only the final eight nonempty lines, - which discards the exception name/reason and first relevant frames. -- Successful helper diagnostics already use a typed report and must remain - separate from arbitrary process output. - -Design gate approved: - -- No one process can observe every correlation fact for Simulator execution: - `ProcessRunner` owns the `xcrun simctl spawn` wrapper transcript and terminal - observation, while the raw helper owns its actual PID and loaded image. -- The helper writes a separate, invocation-authenticated startup handshake - before loading target metadata. It contains only schema/invocation identity, - actual PID, executable name and LC_UUID, and Unix epoch start microseconds. - It is atomic, at most 2 KiB, and contains no path, producer text, device - UDID, command, environment, or runtime root. -- The diagnostics report remains a completed typed-diagnostics contract. It is - not converted into a two-phase process-state file. -- One bounded process-output value owns combined-stream ordering, head/tail - retention, line and byte omission counts, terminal-safe rendering, and the - inclusive output ceiling. Synthetic termination text is not classified as - process-emitted output. -- `runPrivateHeaderKitRawDump` is the only failure-capsule builder because it - knows execution mode and receives the helper handshake, bounded transcript, - and wrapper termination. The capsule has at most 18 lines and 24 KiB, keeps - the first and last eight diagnostic lines, and ends with one canonical - concise headline. -- The capsule is persisted unchanged in the existing - `runTargets.failureSummary`; no DB column or migration is added. Existing - executor, resume, store, and final-summary paths remain the single transport. -- The current-process LC_UUID primitive moves to - `PrivateHeaderKitExecutableResolution`, which is already shared by Tooling - and RawDumpCore; the Mach-O walk is not duplicated. -- Crash Reporter correlation uses PID, executable UUID/name, helper start, - capture time, termination observation, and signal when available. Incident - ID is assigned after a crash and is therefore not guessed at run time. - -Required validation: - -- Long exception fixture retains the exception name/reason and first relevant - frame plus the terminal tail. -- Signal-only fixture explicitly states that no process diagnostic was emitted - and records the exact child identity/timing needed for correlation. -- Bounds hold for long lines, invalid UTF-8, interleaved streams, and high - output volume. -- The capsule survives through `runTargets.failureSummary` and both terminal - and nonterminal failed-target rendering. - -Implementation completed: - -- `BoundedProcessOutput` now owns terminal-safe combined-stream head/tail - retention and raw-source omission lower bounds. `StreamingCommandResult` - carries that value plus the wrapper termination-observation timestamp. -- Every raw helper invocation has a distinct process-handshake report. The - helper writes its validated PID, executable name, LC_UUID, invocation ID, - and start timestamp before loading the requested target. -- `runPrivateHeaderKitRawDump` consumes and removes both reports on every - success/failure/throw path and builds one bounded failure capsule on a - nonzero helper result. -- `RawDumpReportIO` opens each report with no-follow, nonblocking, no-controlling- - terminal, and close-on-exec flags, then performs regular-file inspection and - a `maximum + 1` bounded read on that same descriptor. This removes the - check/reopen race for both handshake and diagnostics reports. -- Simulator child termination is recognized only from the exact final - `simctl` line when the wrapper's normal exit status corroborates the POSIX - `128 + signal` convention. The wrapper line is then replaced by the typed - child-signal field instead of being duplicated as arbitrary output. -- Executor/store/rendering tests confirm that the exact capsule is the existing - `runTargets.failureSummary`; no persistence schema changed. - -Validation completed: - -- `swift test --force-resolved-versions` passed after integration. -- The focused capsule suite passed with 8 tests after the measured `simctl` - exit-status correction. -- The CLI suite passed 90 tests under both Swift 6.4 / Xcode 27 and Swift 6.3.2 / - Xcode 26.5 after the descriptor-bound report reader was added. Deterministic - tests cover path replacement after open, oversized files, symlinks, - directories, FIFOs, devices, and typed contract-error mapping. -- A release-mode run against the exact iOS 27.0 beta `24A5390f` runtime and - `AXSpringBoardServerInstance` reproduced its expected uncaught exception as - run `run-d455b470-6ec7-4955-9157-7bc90c082a47`. -- SQLite retained the exception name/reason, first frames, omission marker, - terminal frames, and canonical headline in 17 lines / 1,725 bytes. Database - integrity was `ok` with no foreign-key violations. -- The headline reported `child_signal(6)`, wrapper status `134`, helper PID - `28709`, LC_UUID `31c43965-06ab-3d01-b413-8db66023c8d9`, start microseconds, - and termination-observation microseconds. -- Crash Reporter independently recorded the same PID, LC_UUID, helper name, - and `SIGABRT`/code 6, with capture time between helper start and observed - termination. -- The run-owned Simulator was deleted and the SDK-runtime override was restored - to its default. The isolated output was moved recoverably to - `/Users/kn/.Trash/privateheaderkit-issue81-runtime-MTbctA`.