Feature/node store migration - #103
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request advances the NodeStore migration by switching many model surfaces from Node to store-backed NodeReference, adding structural-key utilities for cross-store equality, and tightening memory/performance behavior across indexing/printing/diffing/specialization. It also improves robustness for older Mach-O binaries by adding LC_DYLD_INFO(_ONLY) bind-opcode fallback and ensuring interface printing degrades per-definition rather than blanking whole blocks.
Changes:
- Migrate declaration names/members and many printers/differs to
NodeReference+ explicitmaterialize()at the rendering boundary, including structural hashing for name types. - Add NodeStore-oriented infrastructure (
InternedNodeReferenceCache,StructuralNodeReferenceKey,SymbolRowBucket) plus new/updated regression tests around retention, equivalence, and cross-store dictionary keying. - Improve legacy-binary support and operational stability (dyld bind opcode fallback; per-definition printing catch; more deterministic concurrency tests).
Reviewed changes
Copilot reviewed 103 out of 103 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| Tests/SwiftSpecializationTests/GenericSpecializationTests.swift | Update specialization test to use typeContextDescriptorWrapper directly after model slimming. |
| Tests/SwiftPrintingTests/NodePrinterTests.swift | Materialize NodeReference before using printers that expect Node. |
| Tests/SwiftInterfaceTests/SymbolTableRetentionTests.swift | New regression test ensuring stored symbols don’t retain the shared per-image symbol table. |
| Tests/SwiftInterfaceTests/LegacyDyldInfoBindTests.swift | New regression tests for legacy LC_DYLD_INFO(_ONLY) binaries + per-item print degradation. |
| Tests/SwiftIndexingTests/SymbolTestsCoreIntegrationTests.swift | Update to new conformance name maps (conformingProtocolNamesByTypeName). |
| Tests/SwiftIndexingTests/DeclarationModelInstanceSizeTests.swift | New regression tests for declaration model instance-size ceilings and compact retained references. |
| Tests/SwiftDiffingTests/ABIExtensionAttributionTests.swift | Adapt diff attribution tests to NodeReference + helper interning. |
| Tests/SwiftDiffingTests/ABIDifferTests.swift | Adapt differ tests to NodeReference + helper interning/materialization. |
| Tests/SwiftAttributeInferenceTests/TypeAttributeInferrerTests.swift | Adapt inference tests to NodeReference in fields/variables/subscripts and demangled symbol nodes. |
| Tests/MachOSymbolsTests/SymbolTableEquivalenceTests.swift | New in-process image equivalence tests for the offset-ized symbol table collection/search behavior. |
| Tests/MachOSymbolsTests/StructuralNodeReferenceKeyTests.swift | New tests pinning cross-store structural dictionary/set behavior. |
| Tests/MachOSymbolsTests/InternedNodeReferenceCacheTests.swift | New tests for per-scope interning behavior, concurrency, and eviction semantics. |
| Tests/MachOCachesTests/SharedCacheTests.swift | Replace wall-clock parallelism heuristic with deterministic semaphore-based concurrency proof. |
| Tests/IntegrationTests/MachOSymbols/SymbolIndexStoreTests.swift | Split integration suites per image and add build-time measurement printing. |
| Tests/IntegrationTests/MachOSymbols/SymbolIndexStoreBaselineTests.swift | New baseline metrics suite for NodeStore migration measurement and reclaim verification. |
| Sources/SwiftSpecialization/TypeDefinition+Specialization.swift | Enforce single-wrapper materialization discipline and switch bound-name generation to interned NodeReference. |
| Sources/SwiftSpecialization/GenericSpecializer.swift | Migrate type/protocol descriptor access and name creation to interned NodeReference; adjust generic checks. |
| Sources/SwiftSpecialization/ConformanceProvider.swift | Update conformance lookups and superclass map building with descriptor materialization + interned nodes. |
| Sources/SwiftPrinting/SwiftDeclarationPrinter+Members.swift | Materialize NodeReference field/case payload type nodes at print-time. |
| Sources/SwiftPrinting/SwiftDeclarationPrinter+Headers.swift | Thread materialized wrappers into field layout rendering and materialize nodes where Node is required. |
| Sources/SwiftPrinting/SwiftDeclarationPrinter.swift | Enforce per-operation wrapper materialization; per-extension catch for default-implementation extensions; materialize member nodes for printers. |
| Sources/SwiftLayout/ObjCClassIndex.swift | Switch runtime-name demangle to transient demangle to avoid global NodeCache retention. |
| Sources/SwiftInterface/SwiftInterfaceBuilder.swift | Change printRoot to catch per definition within each block (types/specializations/protocols/extensions). |
| Sources/SwiftInterface/SwiftDiffableInterfaceRenderer.swift | Update enum detection to use typeContextDescriptorWrapper. |
| Sources/SwiftInterface/SwiftDeclarationPrinter+DiffRendering.swift | Materialize wrappers/nodes for diff-header printing while keeping single-materialization discipline. |
| Sources/SwiftInspection/ClassHierarchyDumper.swift | Use transient demangle to avoid global caching; align demangling import SPI. |
| Sources/SwiftDump/Protocols/TypedDumper.swift | Use transient demangle for metatype mangled-name demangling. |
| Sources/SwiftDump/Dumper/ProtocolDumper.swift | Switch default-implementation visited tracking to StructuralNodeReferenceKey and resolve via NodeReference. |
| Sources/SwiftDump/Dumper/ProtocolConformanceDumper.swift | Switch visited tracking + demangle paths to NodeReference and structural keys. |
| Sources/SwiftDiffing/ABIKey.swift | Make key construction generic over DemanglingNode, documenting materialization tradeoffs. |
| Sources/SwiftDiffing/ABIDiffer.swift | Update enum detection to use typeContextDescriptorWrapper. |
| Sources/SwiftDeclarationRendering/SpecializedMetadataNodeSubstitution.swift | Use transient demangle for metatype demangling and document rationale. |
| Sources/SwiftDeclarationRendering/RuntimeFieldLayoutBackend.swift | Use transient demangle and transient node construction during substitution to avoid global caching. |
| Sources/SwiftDeclarationRendering/Extensions/ResolvedTypeReference+.swift | Use Node.createTransient for constructed ObjC reference trees. |
| Sources/SwiftDeclarationRendering/Extensions/Node+.swift | Update NodePrinterTarget conformance to autoclosure-based API and provide a representation-generic semantic print. |
| Sources/SwiftDeclarationRendering/Extensions/GenericContext+Dump.swift | Use transient nodes for constructed ObjC protocol requirement dumps. |
| Sources/SwiftDeclarationRendering/DemangleResolver.swift | Add a representation-generic resolve(for: some DemanglingNode) overload. |
| Sources/SwiftDeclaration/Extensions.swift | Update various name-building utilities to intern into InternedNodeReferenceCache; add structural-key usage in indexing. |
| Sources/SwiftDeclaration/Components/Names/TypeName.swift | Switch node storage to NodeReference and implement structural Hashable/Equatable. |
| Sources/SwiftDeclaration/Components/Names/ProtocolName.swift | Switch node storage to NodeReference and implement structural Hashable/Equatable. |
| Sources/SwiftDeclaration/Components/Names/ExtensionName.swift | Switch node storage to NodeReference and implement structural Hashable/Equatable. |
| Sources/SwiftDeclaration/Components/Names/DefinitionName.swift | Update protocol surface to NodeReference. |
| Sources/SwiftDeclaration/Components/Definitions/VariableDefinition.swift | Store member node as NodeReference. |
| Sources/SwiftDeclaration/Components/Definitions/SubscriptDefinition.swift | Store member node as NodeReference. |
| Sources/SwiftDeclaration/Components/Definitions/ProtocolDefinition.swift | Retain only ProtocolDescriptor and materialize wrapper on demand; use structural visited keys and NodeReference demangle path. |
| Sources/SwiftDeclaration/Components/Definitions/OverrideSymbolMatcher.swift | Switch to structural visited keys and SymbolIndexStore.demangledNodeReference for cross-store matching. |
| Sources/SwiftDeclaration/Components/Definitions/FunctionDefinition.swift | Store member node as NodeReference. |
| Sources/SwiftDeclaration/Components/Definitions/FieldDefinition.swift | Store field type node as NodeReference. |
| Sources/SwiftAttributeInference/TypeAttributeInferrer.swift | Update class descriptor access to use typeContextDescriptorWrapper. |
| Sources/MachOSymbols/SymbolRowBucket.swift | New bucket type optimizing common single-row cases for symbol-row indexes. |
| Sources/MachOSymbols/Symbol.swift | Make Symbol Sendable, drop retained nlist existential, and use transient demangle for demangledNode. |
| Sources/MachOSymbols/StructuralNodeReferenceKey.swift | New structural dictionary/set key for cross-store NodeReference equality/hashing. |
| Sources/MachOSymbols/InternedNodeReferenceCache.swift | New shared per-scope interning cache built on SharedNodeStore, with Mach-O scoping and eviction. |
| Sources/MachOSymbols/DemangledSymbol.swift | Redesign DemangledSymbol to reference symbol-table rows + detachable standalone form; expose retention metric for tests. |
| Sources/MachOExtensions/MachOFile+.swift | Add legacy LC_DYLD_INFO(_ONLY) bind-opcode fallback with lazy per-image indexing and caching. |
| Package.swift | Adopt swift-demangling 0.5.1..<0.6.0, wire MachOSymbols into relevant targets/tests, and enable MachOSymbolsTests. |
| Documentations/Internal/TaskReports/2026-08-08-metadata-reader-cache-retirement.md | Task report documenting MetadataReader cache retirement and migration outcomes. |
| Documentations/Internal/TaskReports/2026-08-06-main-rewind-onto-0.14.1.md | Update task report with post-rebase reconciliation details. |
| Documentations/Internal/TaskReports/2026-08-03-system-framework-rendering-ab.md | Task report documenting system-framework A/B rendering verification process. |
| Documentations/Internal/TaskReports/2026-08-03-legacy-dyld-info-bind-support.md | Task report documenting legacy dyld bind support and per-item print degradation fix. |
| Documentations/Internal/TaskReports/2026-07-29-catalyst-rank-and-row-dedup-followup.md | Task report documenting follow-up fixes around dyld cache ranking and row de-dup. |
| Documentations/Internal/TaskReports/2026-07-26-reference-storage-existential-width.md | Update task report with baseline re-recording notes and verification details. |
| Documentations/Internal/SystemFrameworkRenderingVerification.md | New maintainer doc for mandatory A/B verification workflow after large refactors. |
| Documentations/Evolutions/README.md | Restore evolution links for 0001–0003 now that they’re merged. |
| Documentations/Evolutions/0004-arm64e-signed-vwt-pointer-hardening.md | Update cross-proposal links and post-merge verification notes. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| @@ -108,20 +108,98 @@ extension MachOFile { | |||
|
|
|||
ExtensionDefinition.index(in:)'s two early returns (no conformance
descriptor / conformance without resilient witnesses) returned before
setting isIndexed, so every consumer that gates on the flag re-entered
the whole indexing pass: one interface run touches the same extension
at four points (printExtensionDefinition, printExtensionHeader's
materialization, printDefinition, and the diffable builder), and with
evolution 0002 each re-entry re-parses the conformance and its trailing
objects instead of reading a stored property. The unset-flag gap
pre-exists on main but was free there; the materialization cost is new.
Both early returns are completed indexings ('nothing to index'), so
they now mark the definition indexed; a thrown materialization still
leaves the flag unset so a failed read can be retried.
Found by the PR #103 review (finding H2); regression tests cover the
descriptor-less unit shape and the diffable builder's full bucket sweep.
…r populations Evolution 0002's storage cleanup releases the section-wrapper arrays at the end of prepare(), but the six public statistics accessors (numberOfTypes/Enums/Structs/Classes/Protocols/ProtocolConformances) still computed over them — every post-preparation read silently returned 0, indistinguishable from an empty binary, with zero in-repo consumers to notice (the whole blast radius is downstream panels). prepare() now freezes the six counts into Storage.PreparationStatistics immediately before the release and the accessors read that snapshot; signatures and @inlinable stay unchanged. Evolution 0002's source-compatibility section and decision log record the behavioral impact its first landing missed. Found by the PR #103 review (finding H1); pinned by statisticsRemainAvailableAfterPreparation.
…ind gained resolveBind(fileOffset:) learned to answer from the legacy LC_DYLD_INFO(_ONLY) bind opcode-stream index when chained fixups are absent, but isBind(_:) still routed through the chained-fixups-only resolveBind(at:) — on a pre-chained-fixups binary (every iOS 15.5 simulator framework) the two public APIs contradicted each other for the same slot, so a consumer gating a bind read on isBind got nothing on exactly the binaries the fallback targets. isBind(_:) now splits on the same discriminator as resolveBind(fileOffset:); both stale doc comments (which still asserted the file 'must contain dyldChainedFixups data') updated to describe the fallback. Found by the PR #103 review (finding M4); pinned by isBindAgreesWithResolveBindOnLegacyBinaries on the legacy fixture.
…stile opcode values The bind opcode stream is binary-supplied input, but the file-offset index walked it trusting two raw ulebs: a repeat count of 2^40 spun the do_bind_uleb_times loop to OOM/hang, and a wrapped or oversized segment offset attributed a symbol name to an unrelated file offset (the &+ arithmetic never range-checked the slot). Every slot is now checked against its segment's file size before recording — matching how dyld bounds slots — and a repeat run terminates when the slot walks past the segment end, which bounds the loop regardless of the count value. The review's third claim (Int(segment) trapping on a uleb above Int.max) is refuted and recorded in the findings doc: MachOKit decodes the segment index from the opcode's 4-bit immediate, so it can never exceed 15. The malformed-stream regression test was skipped on the user's instruction; the legitimate path stays pinned by the four LegacyDyldInfoBindTests. Found by the PR #103 review (finding H3, revised).
… PackedNameReference The packed name reference enforced its 22-bit length / 40-bit offset budgets with precondition, but both components are binary-supplied: the length is strlen over the image's string table (a hostile or truncated table with no NUL inside 4,194,303 bytes trapped the process in release), the offset a pointer difference into it — and the public DemangledSymbol(symbol:demangledNode:) packed an unclamped caller name through the same trap. The initializer is now failable: the build sweep refuses (skips the row of) a name whose geometry cannot pack — validated before appending so a refused name leaves no orphan bytes — while the standalone one-row table backing the public initializer clamps to the representable prefix (unreachable for any legitimate mangled name). updateRowInPlace does raw bit surgery on the already-validated reference instead of round-tripping through re-validation. Found by the PR #103 review (finding M3); pinned by PackedNameReferenceBudgetTests (pre-fix: 'Precondition failed: symbol name byte length exceeds the 22-bit budget' killed the test runner). Sweep note (same batch, finding H3): the LC_DYLD_INFO decoder was the other trusting-binary-values site; both are now bounded.
…d failed test runs compare_all_pairs derived its verdict solely from the difference count, which only increments while iterating discovered baseline/*.txt files: with no cache archive, no installed simulator runtime, or a mistyped --frameworks, every scenario degrades to paired .skip markers, the glob yields nothing, and the script printed 'RESULT: all pairs byte-identical.' and exited 0 over an empty comparison. run_macho_image_part compounded it by printing swift test's exit code without propagating it, so a failed MachOImage leg could not fail the run either. A harness that reads failure as success retroactively weakens every conclusion that cited its green light. The verdict now requires a non-empty examined-pair count, reports how many pairs backed it, and any non-zero test invocation is a run-level hard failure regardless of the diff outcome. Found by the PR #103 review (finding H4). Demonstrated pre-fix: an empty output root returned difference_count 0 (success path); post-fix the same input takes the zero-pair FAILED branch.
… up its directory LegacyDyldInfoBindTests compiled its fixture with standardError piped but read the pipe only AFTER waitUntilExit: diagnostics beyond the ~64 KB pipe buffer would deadlock compiler and parent — and because the compilation result is a static let under a serialized suite, a toolchain/SDK mismatch would hang the whole swift test run instead of reporting a failure. The pipe is now drained to EOF before reaping. The per-run LegacyDyldInfoBindFixture-<UUID> directory (source + dylib) also leaked on every run; it is now removed at process exit (the fixture must outlive every test in the suite, so per-test cleanup is not an option; a crashed run leaks at most one uniquely-named directory). Found by the PR #103 review (finding L2; deadlock shape mechanism-verified, not reproduced — forcing 64 KB of swiftc diagnostics is not practical in a fixture).
…stically offsetQueriesRebuildSymbolsWithQueriedOffset capped its check at 500 entries while iterating a deliberately-unordered Dictionary whose iteration order is seeded per process — every run sampled a DIFFERENT 500 offsets, so a regression touching only cache-adjusted keys could pass one run and fail the next in the very test written to pin that rebuild. It now checks every offset in sorted order (0.065 s on the fixture — the cap bought nothing). Found by the PR #103 review (finding L4).
…e indexer deinit evicted three caches (symbol store, interned-name store, demangle memo) under one flag that only proves this indexer populated the symbol-store entry first — with indexer A owning image X and a later indexer B live on the same image, A's deinit wiped the caches out from under B: B's already-built names keep an orphaned store alive while later names land in a fresh one, so structurallyEquals' store === fast path stops firing for the pre-eviction population and every context demangle is re-paid. Per-cache ownership flags would not help (whoever populates first owns all three and the wipe-under-B is identical), so ownership is now claimed per IMAGE in a process-wide registry and the eviction runs in the deinit of the image's LAST live indexer. Entries built by non-indexer callers are still never claimed and never evicted — the pre-existing contract, enforced per image. Found by the PR #103 review (finding M6); pinned by PerImageCacheEvictionTests on the SymbolTestsHelper fixture (new MachOFileName case, kept out of shared suites so the membership assertions cannot race a parallel suite's indexer lifecycle; pre-fix the survivor test failed on all three caches). MetadataReader gains a package-visibility non-creating cacheExists(for:) probe for the test.
…nload hazard Two PR #103 review findings resolve as adjudications rather than the suggested code changes, with the reasoning and re-adjudication conditions recorded in ReviewAdjudications.md (A4/A5) and revision notes on the findings document: - M5 (detachedFromSharedTable does not detach the node store): the suggested node-copy is refuted — the definition storing the detached symbol keeps the SAME NodeReference in its own node field (the intended per-image recycling model), so a copy reclaims nothing while the model lives. Landed as doc-comment precision (the detach is symbol-table-layer only; node-store sharing is deliberate) plus storedDeclarationSymbolsShareTheDefinitionsNodeStore pinning the sharing so a future copy must be a deliberate, measured decision. - M2 (raw mapped-string-table pointers outliving dlclose): WON'T FIX — the open half of its Q1 resolved NO by experiment (macOS 26): dyld pins every Swift/ObjC-content image as never-unload (dlclose leaves even a class-less Swift dylib mapped), and the only images that do unmap (pure C) contain no Swift-prefixed names so they never mint mapped rows. The images that could dangle have no rows; the images that have rows cannot dangle. Both suggested mitigations refused (remove-image hook = permanent dead code for pinned images; copy-on-vend breaks the 32-byte shared-table design for an unreachable scenario). SymbolTable's lifetime-constraint comment now records the empirical guarantee.
…intExtensionHeader try? collapsed a thrown materializedProtocolConformance(in:) into the same nil as 'this extension has no conformance', so the public printExtensionHeader emitted a confidently wrong extension header with the conformance clause, @retroactive, and global-actor markers silently missing. In-repo the swallow is unreachable — the only caller indexes first, and index(in:) runs the same materialization with a bare try — which is exactly the defect: a public entry holding a weaker error contract than the index(in:) it follows. The materialization now propagates; external callers hitting a genuinely unreadable conformance get the error instead of a silently degraded header. Found by the PR #103 review (finding M1, revised during implementation: the originally-claimed rendering-path failure is unreachable in-repo and the finding was downgraded to Low with the public-contract framing; see the findings doc's revision note). Pinned by printExtensionHeaderPropagatesMaterializationFailure via a real descriptor layout re-wrapped at an out-of-bounds offset (new package raw-descriptor initializer on ExtensionDefinition as the test surface).
… loops printTypeDefinition and printExtensionDefinition iterated their nested type/protocol children with a bare try await: one nested child whose descriptor could not be read escaped the enclosing definition and the top-level per-definition catch discarded the WHOLE outer type or extension. The nested loops now catch per child (printCatchedThrowing, the same helper printRoot's contract uses), so a corrupt child drops only itself — one more step in the direction the per-block -> per- definition change already moved. Healthy-path output is byte-identical (snapshot suites: 175 tests in 21 suites green). Found by the PR #103 review (finding L1); pinned by corruptNestedChildDropsOnlyItself via a real struct-descriptor layout re-wrapped at an out-of-bounds offset (new package raw-descriptor initializer on TypeDefinition as the test surface; pre-fix the child's offsetOutOfBounds failed the whole parent).
…dicate the early exit The shape all three ranking rounds missed — a plain .dylib name, which can never reach bestMatchRank and always pays the full multi-cache scan — now has end-to-end coverage against the current system's dyld shared cache: libswiftCore resolves to /usr/lib/swift/libswiftCore.dylib and SwiftUI to the native canonical framework binary (never the iOSSupport Catalyst build). The review's suggested achievable-rank early exit is adjudicated as unsound and not landed: holding a rank-2 dylib hit, an unscanned subcache can still contain the rank-0 framework binary, so stopping early reintroduces exactly the cross-subcache order-dependence the ranking was built to remove; and the full scan measures at 43 ms on the current cache (at most once per CLI invocation), so the only sound alternative (replicating MachOKit's image enumeration to rank paths before constructing files) is not worth its drift risk. Found by the PR #103 review (finding L3, revised); reasoning and re-adjudication conditions in ReviewAdjudications.md (A6).
- ProjectEvolutionLog gains its missing sections for evolutions 0002 (descriptor slimming) and 0003 (row-bucket flattening) plus a section for the review-fix batch itself, and the TaskReports link that named a nonexistent file (2026-07-25-dyld-cache-... vs the actual 2026-07-25-cache-...) now resolves. - Task report for the whole implementation arc (Chinese, per convention): research, refuted premises, per-batch execution with pre-fix failure evidence, verification, and divergences. - The findings document's status line records the outcome (fixed or adjudicated per finding, B1 deliberately left to the user). - AGENTS.md synced with the changed facts: isBind/resolveBind parity + hostile-input bounds in MachOExtensions, the nested per-child print catch, and the per-image cache-eviction registry.
…ename Upstream swift-demangling renamed SymbolStore/SymbolStoreBuilder to NodeStore/NodeStoreBuilder (the library's domain concept is the demangled Node tree; there is no Symbol abstraction in its API) and the branch to feature/node-store. Update the migration plan accordingly. The local SymbolIndexStore type keeps its name — it is the migration target, not part of the upstream rename.
…a storage (Stages 1-2) Storage now owns a frozen per-image NodeStore arena instead of leaking every demangled tree into the process-global NodeCache: - Build sweep demangles cache-free onto a transient tree, classifies on it, and interns into a NodeStoreBuilder; indexes are collected NodeIndex-keyed (PendingStorage) and converted to NodeReference form after freeze(). - All indexes key/carry NodeReference (16-byte handle) instead of Node. - Node-taking query APIs (memberSymbols(of:for:node:), opaqueTypeDescriptorSymbol(for:)) bridge via structurallyEquals bucket matching — the frozen store drops its intern tables. - Late symbols (outside the build sweep) demangle into per-symbol mini stores; demangledNodeReference(for:in:) is the new zero-materialization lookup, demangledNode(for:in:) keeps the Node contract via materialize(). - Consumers: matchers use demangleSymbolReference + OrderedSet<NodeReference> visited sets; DefinitionBuilder/TypeDefinition lookups key by NodeReference (O(1) hash after hash-consing); renderer boundaries materialize. isGlobal/isAccessor/hasAccessor/accessorKind/isStoredVariable generalized over DemanglingNode. Measured on the SwiftUI image (debug, like-for-like vs main): NodeCache growth 19,345 leaves / 559,976 subtrees -> 0 / 0; release reclaims 236 MB of 302 MB (legacy 180 MB of 272 MB); NodeStore body 7 MB for 579,291 unique nodes; build 30.7s vs 36.2s under equal parallel load. Tests: enable MachOSymbolsTests target with SymbolIndexStoreFixtureTests (8 cases: cache-free invariant via leaf identity, byte-identical printing vs the Node pipeline, structurallyEquals bucket hits for every key, demangledNode/reference agreement, late-symbol mini-store fallback); add manually-run SymbolIndexStoreBaselineTests (Stage 0 metrics); make SharedCacheTests concurrency proof deterministic via semaphore barrier instead of wall-clock thresholds. Snapshot acceptance: 60/60 byte-identical against the main-verified baseline.
Stage 3 of the NodeStore migration — the Symbol-side residency item the arena does not cover: - Symbol drops the (any NlistProtocol)? stored existential (40 B per copy; its only consumer was the collection-time external-undefined filter, which runs on the MachOKit symbol before Symbol is even constructed). The bit is captured as a stored isExternal Bool; Symbol is explicitly Sendable and shrinks 64 B -> 32 B. - Storage keeps one flat symbolTable: [Symbol] row per unique symbol name (canonical cache-adjusted offset), a parallel rootNodeIndexByTableRow array, and tableRowByName whose keys share row string storage. Every classification index (symbolsByKind, member x3, global, opaque, symbolsByOffset) stores 4-byte UInt32 rows; member/opaque keys shrink from 16-byte NodeReference to 4-byte NodeStore.NodeIndex. Raw and cache-adjusted offset keys share one row; symbols(for:in:) rebuilds each Symbol with the queried offset, byte-identical to the former per-key copies. - DemangledSymbol becomes a 32-byte value: shared-table reference + row + NodeReference, with symbol computed. The public init(symbol:demangledNode:) stays as a single-row-table compat path. - Indexes accumulate in final row form during the sweep (RowIndexes), so freeze() is followed by a plain move into Storage — the pending->populate double-index conversion pass (+30 MB transient peak) is gone and all Storage index fields are now lets. - demangledNodeReference(for:) resolves name -> row + canonical-offset check, strictly equivalent to the former [Symbol: NodeReference] keying; misses still fall through to the late mini-store path. Stage 4 re-measurement (SwiftUI image, debug, same methodology): build 24.5s (legacy 28.6s, Stage 1+2 31.0s); build-phase phys_footprint delta 68 MB (was 302 MB); post-release residual 49 MB; NodeCache growth stays 0/0; NodeStore identical at 7 MB / 579,291 nodes; index entry counts match the baseline item for item. Tests: fixture suite adapted to the row layout plus two new cases — compactValueLayouts (stride <= 32 B invariants) and offsetQueriesRebuildSymbolsWithQueriedOffset (shared-row exit semantics); baseline test reports symbolTable rows/stride. Acceptance: 79 tests in 5 suites passed (60 snapshots byte-identical + 10 fixture cases + SharedCacheTests). Note: struct-layout changes in MachOSymbols require a clean rebuild — SwiftPM incremental builds linked stale downstream objects (runtime SIGSEGV in outlined destroy of the old Symbol layout).
…o NodeStore migration plan
…ansient demangling de-pins NodeCache (Stages 5a/5c) Stage 5c — every unbounded demangle/construction site off the global cache: MetadataReader (28 Node.create sites -> createTransient, 5 demangleAsNode -> demangleAsNodeTransient incl. the symbolic-reference resolver path), RuntimeFieldLayoutBackend, TypedDumper, ClassHierarchyDumper, Symbol.demangledNode, ResolvedTypeReference+, GenericContext+Dump. NodeCache.shared no longer grows while browsing; bounded singletons keep interning. SemanticString adopts the lazy @autoclosure scope hook. Stage 5a — declaration layer holds NodeReference instead of Node: DefinitionName/TypeName/ProtocolName/ExtensionName, Variable/Function/ SubscriptDefinition.node, FieldDefinition.typeNode and ExtensionDefinition.genericSignature. Member definitions reference the per-image symbol store directly (build-time materialize() dropped); the extension-member path passes main-store keys straight through; metadata-derived trees are wrapped via NodeReference(interning:) mini stores, with TypeDefinition.index(in:) batching all field type trees of one type into a single shared store. Name types customize Hashable to structural semantics and keep Codable wire-compatible. ABIKey and the synthesized-member dedup helpers are generic over DemanglingNode; memberSymbols(of:for:node:) gains a NodeReference overload; NodeReference.printSemantic provides zero-materialization rich printing. Five explicit materialize() bridges remain at printer entry points for Stage 5b to remove. Acceptance: 98 tests / 15 suites green, interface snapshot byte-identical.
…ting and resolver (Stage 5b-lite) - DemanglingNode.printSemantic protocol extension: zero-materialization rich printing for any representation through the shared generic engine (store-backed nominal scopes materialize lazily via the autoclosure scope hook). - DemangleResolver.resolve(for: some DemanglingNode): .options prints straight from the node's representation; .builder keeps its public Node closure signature and materializes only on that path. - SwiftDump dumpers pass DemangledSymbol.demangledNode references straight to the resolver — the six highest-frequency transient materialization sites are gone. Full genericization of the async NodePrintable stack is deliberately deferred: after Stage 5a the printer-entry materializations are transient-only (no resident cost), and the stack synthesizes helper Nodes in five places that a generic SomeNode cannot express. Decision and re-entry criteria recorded in NodeStoreMigrationPlan.md. Acceptance: 98 tests / 15 suites green, interface snapshot byte-identical; baseline re-measurement clean (NodeCache delta 0/0, NodeStore 7 MB / 579,291 nodes unchanged, symbolTable 202,603 rows).
… not store identity Stage 5a replaced the `[Node: …]` method-descriptor and vtable-offset lookups in `TypeDefinition.index(in:)` with bare `[NodeReference: …]` dictionaries. `NodeReference`'s intrinsic `Hashable`/`==` is store-identity based (`(store, index)`), not structural, and the two sides of these lookups do not share a store: the keys come from override descriptors' implementation symbols, which `SymbolIndexStore.demangledNodeReference(for:)` hands back from a per-symbol mini store whenever the symbol falls outside the build sweep, while the member side queries them with references from the shared image store. Structurally equal keys from different stores missed, silently dropping both the `override` keyword and the `// VTable offset:` comment. The offset-keyed fallback tables are built only from `methodDescriptors`, so override methods had no second path and lost the annotations entirely. Add `StructuralNodeReferenceKey` — the same `structurallyEquals` + `structuralHash` treatment the `Name` types already carry — and key both lookups on it, wrapping at the five insertion sites and the four query sites. Containers that group or dedup within a single `memberSymbols` batch keep bare `NodeReference` keys: they live in one hash-consed store, where structural equality coincides with index equality. Caught by a whole-file dump/interface A/B diff against main: iOS 18.5 SwiftData's `Schema.Attribute` rendered 5 overrides on main and 0 on the branch, while iOS 26.5 SwiftData and SwiftUI/SwiftUICore were byte-identical. After the fix all three reader sources match main byte for byte — MachOFile 38/38 (iOS 15.5–27.0b2), DyldCache 18/18 (host + iOS 27.0b3/b4), MachOImage 6/6.
…work binary wins
Leaf names are not unique inside a shared cache, but `machOFile(by: .name(_:))`
compared `imagePath.lastPathComponent.deletingPathExtension` and took the first
hit. iOS 27 ships both
/System/Library/Frameworks/SwiftUI.framework/SwiftUI
/System/Library/AccessibilityBundles/SwiftUI.axbundle/SwiftUI
so on the simulator caches, which enumerate the accessibility bundle first,
`swift-section --dyld-shared-cache -n SwiftUI` silently resolved to a payload
carrying no Swift metadata: an empty dump, a four-line interface, and exit code
zero. Nothing distinguished it from a framework that genuinely has no types.
Replace the boolean match with a rank (`matchRank(forImagePath:)`): the binary
inside a `<name>.framework` directory — including the macOS `Versions/A/` shape —
scores best, a `.dylib` next, any other same-leaf payload last. `bestMatch(in:)`
returns the best-ranked image and short-circuits on the first best-rank hit, so
an exact path or a present framework binary costs no more than the previous
first-match scan; `.path` lookups always score best rank and are unchanged. Ties
keep the earliest image, so a given cache resolves deterministically.
`DyldCacheImageSearchTests` pins the ranking on synthetic paths (no cache on
disk needed), which is why MachOCachesTests now depends on MachOExtensions.
End to end, `-n SwiftUI` on the iOS 27 simulator cache goes from 0 to 9,131,212
bytes and matches the `-p`-generated baseline byte for byte, with no change to
`-n SwiftUICore`, `-n SwiftData`, or the macOS host cache.
… registration lacked `addSubIndexer(_:)` had only an index-based inverse (`removeSubIndexer(at:)`), which callers holding the indexer itself cannot use without tracking positions. Add the identity overload. This is what lets a consumer actually release per-image state: registration is what keeps a sub-indexer — and therefore its whole declaration graph, including the `NodeStore` its definitions reference — alive for the aggregate's lifetime. Dropping the last reference lets the sub-indexer deinit, which evicts its `SymbolIndexStore` entry (see this type's `deinit`), so the memory is genuinely reclaimed rather than merely unreferenced from one dictionary. Also documents the dyld cache name-ranking fix in the migration plan and adds the task report covering both that fix and the RuntimeViewer index-lifecycle review (including the one subagent finding that did not survive verification).
The rebase brought proposals 0001-0003 and main's 0004 into one history: restore 0004's row in the status table, relink its related-proposal references, and record that the merge precondition for the RV real-device verification step is now satisfied.
… adjudications
ExtensionDefinition.index(in:)'s two early returns (no conformance
descriptor / conformance without resilient witnesses) returned before
setting isIndexed, so every consumer that gates on the flag re-entered
the whole indexing pass: one interface run touches the same extension
at four points (printExtensionDefinition, printExtensionHeader's
materialization, printDefinition, and the diffable builder), and with
evolution 0002 each re-entry re-parses the conformance and its trailing
objects instead of reading a stored property. The unset-flag gap
pre-exists on main but was free there; the materialization cost is new.
Both early returns are completed indexings ('nothing to index'), so
they now mark the definition indexed; a thrown materialization still
leaves the flag unset so a failed read can be retried.
Found by the PR #103 review (finding H2); regression tests cover the
descriptor-less unit shape and the diffable builder's full bucket sweep.
…r populations Evolution 0002's storage cleanup releases the section-wrapper arrays at the end of prepare(), but the six public statistics accessors (numberOfTypes/Enums/Structs/Classes/Protocols/ProtocolConformances) still computed over them — every post-preparation read silently returned 0, indistinguishable from an empty binary, with zero in-repo consumers to notice (the whole blast radius is downstream panels). prepare() now freezes the six counts into Storage.PreparationStatistics immediately before the release and the accessors read that snapshot; signatures and @inlinable stay unchanged. Evolution 0002's source-compatibility section and decision log record the behavioral impact its first landing missed. Found by the PR #103 review (finding H1); pinned by statisticsRemainAvailableAfterPreparation.
…ind gained resolveBind(fileOffset:) learned to answer from the legacy LC_DYLD_INFO(_ONLY) bind opcode-stream index when chained fixups are absent, but isBind(_:) still routed through the chained-fixups-only resolveBind(at:) — on a pre-chained-fixups binary (every iOS 15.5 simulator framework) the two public APIs contradicted each other for the same slot, so a consumer gating a bind read on isBind got nothing on exactly the binaries the fallback targets. isBind(_:) now splits on the same discriminator as resolveBind(fileOffset:); both stale doc comments (which still asserted the file 'must contain dyldChainedFixups data') updated to describe the fallback. Found by the PR #103 review (finding M4); pinned by isBindAgreesWithResolveBindOnLegacyBinaries on the legacy fixture.
…stile opcode values The bind opcode stream is binary-supplied input, but the file-offset index walked it trusting two raw ulebs: a repeat count of 2^40 spun the do_bind_uleb_times loop to OOM/hang, and a wrapped or oversized segment offset attributed a symbol name to an unrelated file offset (the &+ arithmetic never range-checked the slot). Every slot is now checked against its segment's file size before recording — matching how dyld bounds slots — and a repeat run terminates when the slot walks past the segment end, which bounds the loop regardless of the count value. The review's third claim (Int(segment) trapping on a uleb above Int.max) is refuted and recorded in the findings doc: MachOKit decodes the segment index from the opcode's 4-bit immediate, so it can never exceed 15. The malformed-stream regression test was skipped on the user's instruction; the legitimate path stays pinned by the four LegacyDyldInfoBindTests. Found by the PR #103 review (finding H3, revised).
… PackedNameReference The packed name reference enforced its 22-bit length / 40-bit offset budgets with precondition, but both components are binary-supplied: the length is strlen over the image's string table (a hostile or truncated table with no NUL inside 4,194,303 bytes trapped the process in release), the offset a pointer difference into it — and the public DemangledSymbol(symbol:demangledNode:) packed an unclamped caller name through the same trap. The initializer is now failable: the build sweep refuses (skips the row of) a name whose geometry cannot pack — validated before appending so a refused name leaves no orphan bytes — while the standalone one-row table backing the public initializer clamps to the representable prefix (unreachable for any legitimate mangled name). updateRowInPlace does raw bit surgery on the already-validated reference instead of round-tripping through re-validation. Found by the PR #103 review (finding M3); pinned by PackedNameReferenceBudgetTests (pre-fix: 'Precondition failed: symbol name byte length exceeds the 22-bit budget' killed the test runner). Sweep note (same batch, finding H3): the LC_DYLD_INFO decoder was the other trusting-binary-values site; both are now bounded.
…d failed test runs compare_all_pairs derived its verdict solely from the difference count, which only increments while iterating discovered baseline/*.txt files: with no cache archive, no installed simulator runtime, or a mistyped --frameworks, every scenario degrades to paired .skip markers, the glob yields nothing, and the script printed 'RESULT: all pairs byte-identical.' and exited 0 over an empty comparison. run_macho_image_part compounded it by printing swift test's exit code without propagating it, so a failed MachOImage leg could not fail the run either. A harness that reads failure as success retroactively weakens every conclusion that cited its green light. The verdict now requires a non-empty examined-pair count, reports how many pairs backed it, and any non-zero test invocation is a run-level hard failure regardless of the diff outcome. Found by the PR #103 review (finding H4). Demonstrated pre-fix: an empty output root returned difference_count 0 (success path); post-fix the same input takes the zero-pair FAILED branch.
… up its directory LegacyDyldInfoBindTests compiled its fixture with standardError piped but read the pipe only AFTER waitUntilExit: diagnostics beyond the ~64 KB pipe buffer would deadlock compiler and parent — and because the compilation result is a static let under a serialized suite, a toolchain/SDK mismatch would hang the whole swift test run instead of reporting a failure. The pipe is now drained to EOF before reaping. The per-run LegacyDyldInfoBindFixture-<UUID> directory (source + dylib) also leaked on every run; it is now removed at process exit (the fixture must outlive every test in the suite, so per-test cleanup is not an option; a crashed run leaks at most one uniquely-named directory). Found by the PR #103 review (finding L2; deadlock shape mechanism-verified, not reproduced — forcing 64 KB of swiftc diagnostics is not practical in a fixture).
…stically offsetQueriesRebuildSymbolsWithQueriedOffset capped its check at 500 entries while iterating a deliberately-unordered Dictionary whose iteration order is seeded per process — every run sampled a DIFFERENT 500 offsets, so a regression touching only cache-adjusted keys could pass one run and fail the next in the very test written to pin that rebuild. It now checks every offset in sorted order (0.065 s on the fixture — the cap bought nothing). Found by the PR #103 review (finding L4).
…e indexer deinit evicted three caches (symbol store, interned-name store, demangle memo) under one flag that only proves this indexer populated the symbol-store entry first — with indexer A owning image X and a later indexer B live on the same image, A's deinit wiped the caches out from under B: B's already-built names keep an orphaned store alive while later names land in a fresh one, so structurallyEquals' store === fast path stops firing for the pre-eviction population and every context demangle is re-paid. Per-cache ownership flags would not help (whoever populates first owns all three and the wipe-under-B is identical), so ownership is now claimed per IMAGE in a process-wide registry and the eviction runs in the deinit of the image's LAST live indexer. Entries built by non-indexer callers are still never claimed and never evicted — the pre-existing contract, enforced per image. Found by the PR #103 review (finding M6); pinned by PerImageCacheEvictionTests on the SymbolTestsHelper fixture (new MachOFileName case, kept out of shared suites so the membership assertions cannot race a parallel suite's indexer lifecycle; pre-fix the survivor test failed on all three caches). MetadataReader gains a package-visibility non-creating cacheExists(for:) probe for the test.
…nload hazard Two PR #103 review findings resolve as adjudications rather than the suggested code changes, with the reasoning and re-adjudication conditions recorded in ReviewAdjudications.md (A4/A5) and revision notes on the findings document: - M5 (detachedFromSharedTable does not detach the node store): the suggested node-copy is refuted — the definition storing the detached symbol keeps the SAME NodeReference in its own node field (the intended per-image recycling model), so a copy reclaims nothing while the model lives. Landed as doc-comment precision (the detach is symbol-table-layer only; node-store sharing is deliberate) plus storedDeclarationSymbolsShareTheDefinitionsNodeStore pinning the sharing so a future copy must be a deliberate, measured decision. - M2 (raw mapped-string-table pointers outliving dlclose): WON'T FIX — the open half of its Q1 resolved NO by experiment (macOS 26): dyld pins every Swift/ObjC-content image as never-unload (dlclose leaves even a class-less Swift dylib mapped), and the only images that do unmap (pure C) contain no Swift-prefixed names so they never mint mapped rows. The images that could dangle have no rows; the images that have rows cannot dangle. Both suggested mitigations refused (remove-image hook = permanent dead code for pinned images; copy-on-vend breaks the 32-byte shared-table design for an unreachable scenario). SymbolTable's lifetime-constraint comment now records the empirical guarantee.
…intExtensionHeader try? collapsed a thrown materializedProtocolConformance(in:) into the same nil as 'this extension has no conformance', so the public printExtensionHeader emitted a confidently wrong extension header with the conformance clause, @retroactive, and global-actor markers silently missing. In-repo the swallow is unreachable — the only caller indexes first, and index(in:) runs the same materialization with a bare try — which is exactly the defect: a public entry holding a weaker error contract than the index(in:) it follows. The materialization now propagates; external callers hitting a genuinely unreadable conformance get the error instead of a silently degraded header. Found by the PR #103 review (finding M1, revised during implementation: the originally-claimed rendering-path failure is unreachable in-repo and the finding was downgraded to Low with the public-contract framing; see the findings doc's revision note). Pinned by printExtensionHeaderPropagatesMaterializationFailure via a real descriptor layout re-wrapped at an out-of-bounds offset (new package raw-descriptor initializer on ExtensionDefinition as the test surface).
… loops printTypeDefinition and printExtensionDefinition iterated their nested type/protocol children with a bare try await: one nested child whose descriptor could not be read escaped the enclosing definition and the top-level per-definition catch discarded the WHOLE outer type or extension. The nested loops now catch per child (printCatchedThrowing, the same helper printRoot's contract uses), so a corrupt child drops only itself — one more step in the direction the per-block -> per- definition change already moved. Healthy-path output is byte-identical (snapshot suites: 175 tests in 21 suites green). Found by the PR #103 review (finding L1); pinned by corruptNestedChildDropsOnlyItself via a real struct-descriptor layout re-wrapped at an out-of-bounds offset (new package raw-descriptor initializer on TypeDefinition as the test surface; pre-fix the child's offsetOutOfBounds failed the whole parent).
…dicate the early exit The shape all three ranking rounds missed — a plain .dylib name, which can never reach bestMatchRank and always pays the full multi-cache scan — now has end-to-end coverage against the current system's dyld shared cache: libswiftCore resolves to /usr/lib/swift/libswiftCore.dylib and SwiftUI to the native canonical framework binary (never the iOSSupport Catalyst build). The review's suggested achievable-rank early exit is adjudicated as unsound and not landed: holding a rank-2 dylib hit, an unscanned subcache can still contain the rank-0 framework binary, so stopping early reintroduces exactly the cross-subcache order-dependence the ranking was built to remove; and the full scan measures at 43 ms on the current cache (at most once per CLI invocation), so the only sound alternative (replicating MachOKit's image enumeration to rank paths before constructing files) is not worth its drift risk. Found by the PR #103 review (finding L3, revised); reasoning and re-adjudication conditions in ReviewAdjudications.md (A6).
- ProjectEvolutionLog gains its missing sections for evolutions 0002 (descriptor slimming) and 0003 (row-bucket flattening) plus a section for the review-fix batch itself, and the TaskReports link that named a nonexistent file (2026-07-25-dyld-cache-... vs the actual 2026-07-25-cache-...) now resolves. - Task report for the whole implementation arc (Chinese, per convention): research, refuted premises, per-batch execution with pre-fix failure evidence, verification, and divergences. - The findings document's status line records the outcome (fixed or adjudicated per finding, B1 deliberately left to the user). - AGENTS.md synced with the changed facts: isBind/resolveBind parity + hostile-input bounds in MachOExtensions, the nested per-child print catch, and the per-image cache-eviction registry.
main extracted the in-repo `MachOExtensions` target into the upstream `MachOKitExtensions` package (so `MachOObjCSection` can depend on it too without a package-level cycle). Rebasing this branch onto that resolved the two modify/delete conflicts by taking main's deletion, which left three `import MachOExtensions` sites and two test-target dependencies pointing at a module this repo no longer has. The four behaviors this branch had landed in those two deleted files — cross-subcache dyld image ranking, the legacy LC_DYLD_INFO(_ONLY) bind index, its hostile-input bounds check (PR #103 finding H3), and isBind's matching fallback (M4) — were ported into MachOKitExtensions first, from this branch's tip state rather than commit by commit, so dropping the intermediate hunks during the rebase loses nothing. `DyldCacheImageSearchTests` drops `@testable`: `matchRank` / `bestMatchRank` are public upstream, and an external package dependency is not built with testability anyway. Docs updated in the same batch. AGENTS.md was already stale on main (the extraction commit did not touch it) and this branch had added the LC_DYLD_INFO paragraph to the same bullet, so it now described a module this repo does not contain; it is rewritten as the upstream package plus the two behaviors this repo's tests still pin. Two implementation notes had dead `Sources/MachOExtensions/` paths. Historical records (task reports, review findings, changelogs, proposals) are left as written. Verified: MachOKitExtensions builds standalone; this repo's suite passes with exit code 0 (1408 tests, 264 suites, IntegrationTests skipped), built against the local siblings with --manifest-cache none — SwiftPM's cached manifest evaluation otherwise silently keeps resolving the upstream package remotely.
The dyld-cache image ranking and the legacy LC_DYLD_INFO(_ONLY) bind index were developed on this branch against the in-repo `MachOExtensions` target and ported upstream when that target was extracted; 0.1.1 is the release carrying them. This is the real floor, not a preference — `DyldCacheImageSearchTests` and `LegacyDyldInfoBindTests` do not compile against 0.1.0, which is what the inherited open `from: "0.1.0"` bound would have allowed a resolver to pick. Verified against the published tag rather than the local checkout: the two ported files at 0.1.1 are byte-identical to the copies the suite passed with, and with the local sibling symlink removed so the package resolves remotely, the suite passes at exit code 0 (1408 tests, 264 suites, IntegrationTests skipped). Local resolution and CI now agree.
aa0a412 to
02a935f
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 109 out of 109 changed files in this pull request and generated no new comments.
Suppressed comments (1)
Sources/SwiftSpecialization/TypeDefinition+Specialization.swift:356
boundGenericTypeNameinternswrappedNodeintoInternedNodeReferenceCache’s process-scoped store. Since the resultingTypeNameis stored on a specializedTypeDefinition, this can pin the entire process-scoped store (which is intended for transient, no-MachO-handle call sites) and reduce the effectiveness of per-image cache eviction.
If you can’t thread a machO handle in here to use the image-scoped reference(interning:in:), it’s safer to avoid the shared process store for a long-lived value and instead mint a standalone NodeReference for the specialized name.
When both sides failed but with DIFFERENT exit codes, each leg unlinked its .txt and wrote its own .skip, so the pair was invisible to both .txt globs: neither counter moved and the skip loop printed nothing. With any other framework identical the run reported "all N pairs byte-identical" and exited 0 — over a candidate that had, for instance, started trapping where the baseline merely errored. This is the same shape as the zero-pairs hole fixed earlier in this branch (a harness that cannot fail is worse than no harness), so the verdict logic now has unit tests of its own. They are standard-library only and run in seconds.
Issue #102 asked for three things when a definition cannot be printed: keep the partial result, dispatch a failure event, and stop writing to stdout. Only the first landed. A run that lost 8,375 definitions therefore dispatched zero definitionPrintFailed events, and its only signal was a bare `unexpected(at: 8)` on stdout — the very stream the CLI writes the generated Swift to, so the diagnostic corrupted the interface it was reporting on and, being buffered in a pipe, surfaced far from its cause. printCatchedThrowing now takes the dispatcher and a printing context, and every call site that has a definition identity supplies one. The two globals blocks deliberately do not: printVariable / printFunction are non-throwing and already catch per member, so there is no identity to attribute a block-level failure to. Sweeping for the same defect found four more library-side print(error) sites, all of them on the interface-generating path and all now on stderr. Sources/ no longer contains a single one.
header(_:_:) swallowed a throw into an empty SemanticString, but renderType / renderProtocol compute their body units unconditionally and hand both to DiffContainerAssembler. A type whose header failed was therefore emitted as members and braces with no `struct Foo` line above them — structurally invalid Swift, produced silently, with no event and no error. Header rendering genuinely throws: it reads the declaration's name, generic signature and superclass and demangles each (issue #102 is the field evidence that print-time DemanglingErrors happen on real binaries), and since evolution 0002 it also re-materializes the wrapper from its descriptor. The nil now means "this declaration exists but could not be rendered", which drops it whole. An absent side stays an empty string, since that is the case the added/removed markers are built on.
Two fixes on the per-image index, both from the same review round. opaqueTypeDescriptorSymbol bucketed its entries by member identifier and scanned the bucket structurally. The assumption was that an identifier picks out "normally exactly one" descriptor; it does not. The identifier is the member name, so in SwiftUI the `body` bucket alone holds hundreds of entries, and the caller queries once per printed `some`-returning declaration with no memoization — quadratic in a count that runs into the thousands, with a fresh visited-pair set allocated per comparison. StructuralNodeReferenceKey now also accepts a bare query Node, hashing it through the structural hash upstream keeps in step with the reference's, so the lookup is one probe. Cache eviction claimed all three per-image caches from one sample of the symbol store. Only that one is necessarily an indexer's: the interned-name store and the demangle memo are also populated by SwiftLayout, the renderers and SwiftSpecialization, so a "dump, then build the interface" sequence fills both with no symbol store at all — and the combined claim read that state backwards, evicting caches live non-indexer work was still using. Each is now claimed separately. Registration is keyed on indexer identity rather than counted, so a concurrent second prepare() cannot strand the population above zero and leak all three for the process lifetime. Also renames a `cls` binding this branch carried over, per the no- abbreviations rule.
Evolution 0001's compatibility section claimed "purely additive / no breakage" and "no superseded public API" while the landed code had replaced Symbol.nlist with isExternal. An accepted proposal stating the opposite of the shipped code is worse than the API change itself, so it now records the break honestly — including the mitigating fact that the default argument keeps Symbol(offset:name:) compiling. AGENTS.md follows the two code changes in this batch (structural opaque index, per-cache eviction claims) and gains the A/B harness self-test. Adjudications A7/A8 record the two findings deliberately not fixed: the print-options divergence between the indexer and dump witness matchers (no reproduction, identical on main, no fixture) and updateConfiguration's re-prepare being a no-op (unreachable for every known consumer). Node+.swift's printSemantic gains a warning not to "modernize" onto the new runPrintWalk requirement: that hook returns String because it dispatches print(using:), so a custom target is outside its remit by design, and the engine's static entry point — including its stack guard — is unchanged.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 116 out of 117 changed files in this pull request and generated no new comments.
Suppressed comments (3)
Tests/IntegrationTests/MachOSymbols/SymbolIndexStoreBaselineTests.swift:76
Tests/IntegrationTests/is intended for manual inspection without assertions.#expect(demangledSymbolCount > 0)makes this behave like a normal unit test and can fail the suite for environmental reasons; prefer logging the metric (or move this intoMachOSymbolsTestsif it should be enforced).
Tests/IntegrationTests/MachOSymbols/SymbolIndexStoreBaselineTests.swift:36Tests/IntegrationTests/is documented as a manual-inspection-only target (no assertions/preconditions). Using#requirehere turns the run into a hard failure and can break unrelated CI invocations; consider either moving this suite into a non-Integration test target or degrading to a soft failure (e.g.guard let … else { print(...); return }).
This issue also appears on line 76 of the same file.
Sources/SwiftSpecialization/TypeDefinition+Specialization.swift:355
boundGenericTypeNamestores the resultingTypeNameon specialized definitions, but it internswrappedNodeinto the process-scopedInternedNodeReferenceCache(reference(interning:)). That bypasses the per-image scope/eviction model used for other names (e.g.TypeContextDescriptorWrapper.typeName(in:)usesreference(interning:…, in: machO)), and can keep interned nodes alive beyond the image/indexer lifecycle. Consider threadingmachOinto this helper so the bound name is interned into the same image scope as the unbound name.
No description provided.