Skip to content

Feature/node store migration - #103

Open
Mx-Iris wants to merge 77 commits into
mainfrom
feature/node-store-migration
Open

Feature/node store migration#103
Mx-Iris wants to merge 77 commits into
mainfrom
feature/node-store-migration

Conversation

@Mx-Iris

@Mx-Iris Mx-Iris commented Aug 9, 2026

Copy link
Copy Markdown
Member

No description provided.

Copilot AI lite review requested due to automatic review settings August 9, 2026 11:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request advances the NodeStore migration by switching many model surfaces from Node to store-backed NodeReference, adding structural-key utilities for cross-store equality, and tightening memory/performance behavior across indexing/printing/diffing/specialization. It also improves robustness for older Mach-O binaries by adding LC_DYLD_INFO(_ONLY) bind-opcode fallback and ensuring interface printing degrades per-definition rather than blanking whole blocks.

Changes:

  • Migrate declaration names/members and many printers/differs to NodeReference + explicit materialize() at the rendering boundary, including structural hashing for name types.
  • Add NodeStore-oriented infrastructure (InternedNodeReferenceCache, StructuralNodeReferenceKey, SymbolRowBucket) plus new/updated regression tests around retention, equivalence, and cross-store dictionary keying.
  • Improve legacy-binary support and operational stability (dyld bind opcode fallback; per-definition printing catch; more deterministic concurrency tests).

Reviewed changes

Copilot reviewed 103 out of 103 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Tests/SwiftSpecializationTests/GenericSpecializationTests.swift Update specialization test to use typeContextDescriptorWrapper directly after model slimming.
Tests/SwiftPrintingTests/NodePrinterTests.swift Materialize NodeReference before using printers that expect Node.
Tests/SwiftInterfaceTests/SymbolTableRetentionTests.swift New regression test ensuring stored symbols don’t retain the shared per-image symbol table.
Tests/SwiftInterfaceTests/LegacyDyldInfoBindTests.swift New regression tests for legacy LC_DYLD_INFO(_ONLY) binaries + per-item print degradation.
Tests/SwiftIndexingTests/SymbolTestsCoreIntegrationTests.swift Update to new conformance name maps (conformingProtocolNamesByTypeName).
Tests/SwiftIndexingTests/DeclarationModelInstanceSizeTests.swift New regression tests for declaration model instance-size ceilings and compact retained references.
Tests/SwiftDiffingTests/ABIExtensionAttributionTests.swift Adapt diff attribution tests to NodeReference + helper interning.
Tests/SwiftDiffingTests/ABIDifferTests.swift Adapt differ tests to NodeReference + helper interning/materialization.
Tests/SwiftAttributeInferenceTests/TypeAttributeInferrerTests.swift Adapt inference tests to NodeReference in fields/variables/subscripts and demangled symbol nodes.
Tests/MachOSymbolsTests/SymbolTableEquivalenceTests.swift New in-process image equivalence tests for the offset-ized symbol table collection/search behavior.
Tests/MachOSymbolsTests/StructuralNodeReferenceKeyTests.swift New tests pinning cross-store structural dictionary/set behavior.
Tests/MachOSymbolsTests/InternedNodeReferenceCacheTests.swift New tests for per-scope interning behavior, concurrency, and eviction semantics.
Tests/MachOCachesTests/SharedCacheTests.swift Replace wall-clock parallelism heuristic with deterministic semaphore-based concurrency proof.
Tests/IntegrationTests/MachOSymbols/SymbolIndexStoreTests.swift Split integration suites per image and add build-time measurement printing.
Tests/IntegrationTests/MachOSymbols/SymbolIndexStoreBaselineTests.swift New baseline metrics suite for NodeStore migration measurement and reclaim verification.
Sources/SwiftSpecialization/TypeDefinition+Specialization.swift Enforce single-wrapper materialization discipline and switch bound-name generation to interned NodeReference.
Sources/SwiftSpecialization/GenericSpecializer.swift Migrate type/protocol descriptor access and name creation to interned NodeReference; adjust generic checks.
Sources/SwiftSpecialization/ConformanceProvider.swift Update conformance lookups and superclass map building with descriptor materialization + interned nodes.
Sources/SwiftPrinting/SwiftDeclarationPrinter+Members.swift Materialize NodeReference field/case payload type nodes at print-time.
Sources/SwiftPrinting/SwiftDeclarationPrinter+Headers.swift Thread materialized wrappers into field layout rendering and materialize nodes where Node is required.
Sources/SwiftPrinting/SwiftDeclarationPrinter.swift Enforce per-operation wrapper materialization; per-extension catch for default-implementation extensions; materialize member nodes for printers.
Sources/SwiftLayout/ObjCClassIndex.swift Switch runtime-name demangle to transient demangle to avoid global NodeCache retention.
Sources/SwiftInterface/SwiftInterfaceBuilder.swift Change printRoot to catch per definition within each block (types/specializations/protocols/extensions).
Sources/SwiftInterface/SwiftDiffableInterfaceRenderer.swift Update enum detection to use typeContextDescriptorWrapper.
Sources/SwiftInterface/SwiftDeclarationPrinter+DiffRendering.swift Materialize wrappers/nodes for diff-header printing while keeping single-materialization discipline.
Sources/SwiftInspection/ClassHierarchyDumper.swift Use transient demangle to avoid global caching; align demangling import SPI.
Sources/SwiftDump/Protocols/TypedDumper.swift Use transient demangle for metatype mangled-name demangling.
Sources/SwiftDump/Dumper/ProtocolDumper.swift Switch default-implementation visited tracking to StructuralNodeReferenceKey and resolve via NodeReference.
Sources/SwiftDump/Dumper/ProtocolConformanceDumper.swift Switch visited tracking + demangle paths to NodeReference and structural keys.
Sources/SwiftDiffing/ABIKey.swift Make key construction generic over DemanglingNode, documenting materialization tradeoffs.
Sources/SwiftDiffing/ABIDiffer.swift Update enum detection to use typeContextDescriptorWrapper.
Sources/SwiftDeclarationRendering/SpecializedMetadataNodeSubstitution.swift Use transient demangle for metatype demangling and document rationale.
Sources/SwiftDeclarationRendering/RuntimeFieldLayoutBackend.swift Use transient demangle and transient node construction during substitution to avoid global caching.
Sources/SwiftDeclarationRendering/Extensions/ResolvedTypeReference+.swift Use Node.createTransient for constructed ObjC reference trees.
Sources/SwiftDeclarationRendering/Extensions/Node+.swift Update NodePrinterTarget conformance to autoclosure-based API and provide a representation-generic semantic print.
Sources/SwiftDeclarationRendering/Extensions/GenericContext+Dump.swift Use transient nodes for constructed ObjC protocol requirement dumps.
Sources/SwiftDeclarationRendering/DemangleResolver.swift Add a representation-generic resolve(for: some DemanglingNode) overload.
Sources/SwiftDeclaration/Extensions.swift Update various name-building utilities to intern into InternedNodeReferenceCache; add structural-key usage in indexing.
Sources/SwiftDeclaration/Components/Names/TypeName.swift Switch node storage to NodeReference and implement structural Hashable/Equatable.
Sources/SwiftDeclaration/Components/Names/ProtocolName.swift Switch node storage to NodeReference and implement structural Hashable/Equatable.
Sources/SwiftDeclaration/Components/Names/ExtensionName.swift Switch node storage to NodeReference and implement structural Hashable/Equatable.
Sources/SwiftDeclaration/Components/Names/DefinitionName.swift Update protocol surface to NodeReference.
Sources/SwiftDeclaration/Components/Definitions/VariableDefinition.swift Store member node as NodeReference.
Sources/SwiftDeclaration/Components/Definitions/SubscriptDefinition.swift Store member node as NodeReference.
Sources/SwiftDeclaration/Components/Definitions/ProtocolDefinition.swift Retain only ProtocolDescriptor and materialize wrapper on demand; use structural visited keys and NodeReference demangle path.
Sources/SwiftDeclaration/Components/Definitions/OverrideSymbolMatcher.swift Switch to structural visited keys and SymbolIndexStore.demangledNodeReference for cross-store matching.
Sources/SwiftDeclaration/Components/Definitions/FunctionDefinition.swift Store member node as NodeReference.
Sources/SwiftDeclaration/Components/Definitions/FieldDefinition.swift Store field type node as NodeReference.
Sources/SwiftAttributeInference/TypeAttributeInferrer.swift Update class descriptor access to use typeContextDescriptorWrapper.
Sources/MachOSymbols/SymbolRowBucket.swift New bucket type optimizing common single-row cases for symbol-row indexes.
Sources/MachOSymbols/Symbol.swift Make Symbol Sendable, drop retained nlist existential, and use transient demangle for demangledNode.
Sources/MachOSymbols/StructuralNodeReferenceKey.swift New structural dictionary/set key for cross-store NodeReference equality/hashing.
Sources/MachOSymbols/InternedNodeReferenceCache.swift New shared per-scope interning cache built on SharedNodeStore, with Mach-O scoping and eviction.
Sources/MachOSymbols/DemangledSymbol.swift Redesign DemangledSymbol to reference symbol-table rows + detachable standalone form; expose retention metric for tests.
Sources/MachOExtensions/MachOFile+.swift Add legacy LC_DYLD_INFO(_ONLY) bind-opcode fallback with lazy per-image indexing and caching.
Package.swift Adopt swift-demangling 0.5.1..<0.6.0, wire MachOSymbols into relevant targets/tests, and enable MachOSymbolsTests.
Documentations/Internal/TaskReports/2026-08-08-metadata-reader-cache-retirement.md Task report documenting MetadataReader cache retirement and migration outcomes.
Documentations/Internal/TaskReports/2026-08-06-main-rewind-onto-0.14.1.md Update task report with post-rebase reconciliation details.
Documentations/Internal/TaskReports/2026-08-03-system-framework-rendering-ab.md Task report documenting system-framework A/B rendering verification process.
Documentations/Internal/TaskReports/2026-08-03-legacy-dyld-info-bind-support.md Task report documenting legacy dyld bind support and per-item print degradation fix.
Documentations/Internal/TaskReports/2026-07-29-catalyst-rank-and-row-dedup-followup.md Task report documenting follow-up fixes around dyld cache ranking and row de-dup.
Documentations/Internal/TaskReports/2026-07-26-reference-storage-existential-width.md Update task report with baseline re-recording notes and verification details.
Documentations/Internal/SystemFrameworkRenderingVerification.md New maintainer doc for mandatory A/B verification workflow after large refactors.
Documentations/Evolutions/README.md Restore evolution links for 0001–0003 now that they’re merged.
Documentations/Evolutions/0004-arm64e-signed-vwt-pointer-hardening.md Update cross-proposal links and post-merge verification notes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@@ -108,20 +108,98 @@ extension MachOFile {

Mx-Iris added a commit that referenced this pull request Aug 9, 2026
Copilot AI review requested due to automatic review settings August 9, 2026 15:38
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
ExtensionDefinition.index(in:)'s two early returns (no conformance
descriptor / conformance without resilient witnesses) returned before
setting isIndexed, so every consumer that gates on the flag re-entered
the whole indexing pass: one interface run touches the same extension
at four points (printExtensionDefinition, printExtensionHeader's
materialization, printDefinition, and the diffable builder), and with
evolution 0002 each re-entry re-parses the conformance and its trailing
objects instead of reading a stored property. The unset-flag gap
pre-exists on main but was free there; the materialization cost is new.

Both early returns are completed indexings ('nothing to index'), so
they now mark the definition indexed; a thrown materialization still
leaves the flag unset so a failed read can be retried.

Found by the PR #103 review (finding H2); regression tests cover the
descriptor-less unit shape and the diffable builder's full bucket sweep.
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
…r populations

Evolution 0002's storage cleanup releases the section-wrapper arrays at
the end of prepare(), but the six public statistics accessors
(numberOfTypes/Enums/Structs/Classes/Protocols/ProtocolConformances)
still computed over them — every post-preparation read silently
returned 0, indistinguishable from an empty binary, with zero in-repo
consumers to notice (the whole blast radius is downstream panels).

prepare() now freezes the six counts into Storage.PreparationStatistics
immediately before the release and the accessors read that snapshot;
signatures and @inlinable stay unchanged. Evolution 0002's
source-compatibility section and decision log record the behavioral
impact its first landing missed.

Found by the PR #103 review (finding H1); pinned by
statisticsRemainAvailableAfterPreparation.
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
…ind gained

resolveBind(fileOffset:) learned to answer from the legacy
LC_DYLD_INFO(_ONLY) bind opcode-stream index when chained fixups are
absent, but isBind(_:) still routed through the chained-fixups-only
resolveBind(at:) — on a pre-chained-fixups binary (every iOS 15.5
simulator framework) the two public APIs contradicted each other for
the same slot, so a consumer gating a bind read on isBind got nothing
on exactly the binaries the fallback targets.

isBind(_:) now splits on the same discriminator as
resolveBind(fileOffset:); both stale doc comments (which still asserted
the file 'must contain dyldChainedFixups data') updated to describe the
fallback.

Found by the PR #103 review (finding M4); pinned by
isBindAgreesWithResolveBindOnLegacyBinaries on the legacy fixture.
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
…stile opcode values

The bind opcode stream is binary-supplied input, but the file-offset
index walked it trusting two raw ulebs: a repeat count of 2^40 spun the
do_bind_uleb_times loop to OOM/hang, and a wrapped or oversized segment
offset attributed a symbol name to an unrelated file offset (the &+
arithmetic never range-checked the slot). Every slot is now checked
against its segment's file size before recording — matching how dyld
bounds slots — and a repeat run terminates when the slot walks past the
segment end, which bounds the loop regardless of the count value.

The review's third claim (Int(segment) trapping on a uleb above
Int.max) is refuted and recorded in the findings doc: MachOKit decodes
the segment index from the opcode's 4-bit immediate, so it can never
exceed 15. The malformed-stream regression test was skipped on the
user's instruction; the legitimate path stays pinned by the four
LegacyDyldInfoBindTests.

Found by the PR #103 review (finding H3, revised).
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
… PackedNameReference

The packed name reference enforced its 22-bit length / 40-bit offset
budgets with precondition, but both components are binary-supplied: the
length is strlen over the image's string table (a hostile or truncated
table with no NUL inside 4,194,303 bytes trapped the process in
release), the offset a pointer difference into it — and the public
DemangledSymbol(symbol:demangledNode:) packed an unclamped caller name
through the same trap.

The initializer is now failable: the build sweep refuses (skips the row
of) a name whose geometry cannot pack — validated before appending so a
refused name leaves no orphan bytes — while the standalone one-row
table backing the public initializer clamps to the representable prefix
(unreachable for any legitimate mangled name). updateRowInPlace does
raw bit surgery on the already-validated reference instead of
round-tripping through re-validation.

Found by the PR #103 review (finding M3); pinned by
PackedNameReferenceBudgetTests (pre-fix: 'Precondition failed: symbol
name byte length exceeds the 22-bit budget' killed the test runner).

Sweep note (same batch, finding H3): the LC_DYLD_INFO decoder was the
other trusting-binary-values site; both are now bounded.
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
…d failed test runs

compare_all_pairs derived its verdict solely from the difference count,
which only increments while iterating discovered baseline/*.txt files:
with no cache archive, no installed simulator runtime, or a mistyped
--frameworks, every scenario degrades to paired .skip markers, the glob
yields nothing, and the script printed 'RESULT: all pairs
byte-identical.' and exited 0 over an empty comparison.
run_macho_image_part compounded it by printing swift test's exit code
without propagating it, so a failed MachOImage leg could not fail the
run either. A harness that reads failure as success retroactively
weakens every conclusion that cited its green light.

The verdict now requires a non-empty examined-pair count, reports how
many pairs backed it, and any non-zero test invocation is a run-level
hard failure regardless of the diff outcome.

Found by the PR #103 review (finding H4). Demonstrated pre-fix: an
empty output root returned difference_count 0 (success path); post-fix
the same input takes the zero-pair FAILED branch.
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
… up its directory

LegacyDyldInfoBindTests compiled its fixture with standardError piped
but read the pipe only AFTER waitUntilExit: diagnostics beyond the
~64 KB pipe buffer would deadlock compiler and parent — and because the
compilation result is a static let under a serialized suite, a
toolchain/SDK mismatch would hang the whole swift test run instead of
reporting a failure. The pipe is now drained to EOF before reaping.

The per-run LegacyDyldInfoBindFixture-<UUID> directory (source + dylib)
also leaked on every run; it is now removed at process exit (the
fixture must outlive every test in the suite, so per-test cleanup is
not an option; a crashed run leaks at most one uniquely-named
directory).

Found by the PR #103 review (finding L2; deadlock shape
mechanism-verified, not reproduced — forcing 64 KB of swiftc
diagnostics is not practical in a fixture).
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
…stically

offsetQueriesRebuildSymbolsWithQueriedOffset capped its check at 500
entries while iterating a deliberately-unordered Dictionary whose
iteration order is seeded per process — every run sampled a DIFFERENT
500 offsets, so a regression touching only cache-adjusted keys could
pass one run and fail the next in the very test written to pin that
rebuild. It now checks every offset in sorted order (0.065 s on the
fixture — the cap bought nothing).

Found by the PR #103 review (finding L4).
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
…e indexer

deinit evicted three caches (symbol store, interned-name store,
demangle memo) under one flag that only proves this indexer populated
the symbol-store entry first — with indexer A owning image X and a
later indexer B live on the same image, A's deinit wiped the caches out
from under B: B's already-built names keep an orphaned store alive
while later names land in a fresh one, so structurallyEquals' store ===
fast path stops firing for the pre-eviction population and every
context demangle is re-paid. Per-cache ownership flags would not help
(whoever populates first owns all three and the wipe-under-B is
identical), so ownership is now claimed per IMAGE in a process-wide
registry and the eviction runs in the deinit of the image's LAST live
indexer. Entries built by non-indexer callers are still never claimed
and never evicted — the pre-existing contract, enforced per image.

Found by the PR #103 review (finding M6); pinned by
PerImageCacheEvictionTests on the SymbolTestsHelper fixture (new
MachOFileName case, kept out of shared suites so the membership
assertions cannot race a parallel suite's indexer lifecycle; pre-fix
the survivor test failed on all three caches). MetadataReader gains a
package-visibility non-creating cacheExists(for:) probe for the test.
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
…nload hazard

Two PR #103 review findings resolve as adjudications rather than the
suggested code changes, with the reasoning and re-adjudication
conditions recorded in ReviewAdjudications.md (A4/A5) and revision
notes on the findings document:

- M5 (detachedFromSharedTable does not detach the node store): the
  suggested node-copy is refuted — the definition storing the detached
  symbol keeps the SAME NodeReference in its own node field (the
  intended per-image recycling model), so a copy reclaims nothing while
  the model lives. Landed as doc-comment precision (the detach is
  symbol-table-layer only; node-store sharing is deliberate) plus
  storedDeclarationSymbolsShareTheDefinitionsNodeStore pinning the
  sharing so a future copy must be a deliberate, measured decision.

- M2 (raw mapped-string-table pointers outliving dlclose): WON'T FIX —
  the open half of its Q1 resolved NO by experiment (macOS 26): dyld
  pins every Swift/ObjC-content image as never-unload (dlclose leaves
  even a class-less Swift dylib mapped), and the only images that do
  unmap (pure C) contain no Swift-prefixed names so they never mint
  mapped rows. The images that could dangle have no rows; the images
  that have rows cannot dangle. Both suggested mitigations refused
  (remove-image hook = permanent dead code for pinned images;
  copy-on-vend breaks the 32-byte shared-table design for an
  unreachable scenario). SymbolTable's lifetime-constraint comment now
  records the empirical guarantee.
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
…intExtensionHeader

try? collapsed a thrown materializedProtocolConformance(in:) into the
same nil as 'this extension has no conformance', so the public
printExtensionHeader emitted a confidently wrong extension header with
the conformance clause, @retroactive, and global-actor markers silently
missing. In-repo the swallow is unreachable — the only caller indexes
first, and index(in:) runs the same materialization with a bare try —
which is exactly the defect: a public entry holding a weaker error
contract than the index(in:) it follows. The materialization now
propagates; external callers hitting a genuinely unreadable conformance
get the error instead of a silently degraded header.

Found by the PR #103 review (finding M1, revised during implementation:
the originally-claimed rendering-path failure is unreachable in-repo
and the finding was downgraded to Low with the public-contract framing;
see the findings doc's revision note). Pinned by
printExtensionHeaderPropagatesMaterializationFailure via a real
descriptor layout re-wrapped at an out-of-bounds offset (new package
raw-descriptor initializer on ExtensionDefinition as the test surface).
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
… loops

printTypeDefinition and printExtensionDefinition iterated their nested
type/protocol children with a bare try await: one nested child whose
descriptor could not be read escaped the enclosing definition and the
top-level per-definition catch discarded the WHOLE outer type or
extension. The nested loops now catch per child (printCatchedThrowing,
the same helper printRoot's contract uses), so a corrupt child drops
only itself — one more step in the direction the per-block -> per-
definition change already moved. Healthy-path output is byte-identical
(snapshot suites: 175 tests in 21 suites green).

Found by the PR #103 review (finding L1); pinned by
corruptNestedChildDropsOnlyItself via a real struct-descriptor layout
re-wrapped at an out-of-bounds offset (new package raw-descriptor
initializer on TypeDefinition as the test surface; pre-fix the child's
offsetOutOfBounds failed the whole parent).
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
…dicate the early exit

The shape all three ranking rounds missed — a plain .dylib name, which
can never reach bestMatchRank and always pays the full multi-cache
scan — now has end-to-end coverage against the current system's dyld
shared cache: libswiftCore resolves to /usr/lib/swift/libswiftCore.dylib
and SwiftUI to the native canonical framework binary (never the
iOSSupport Catalyst build).

The review's suggested achievable-rank early exit is adjudicated as
unsound and not landed: holding a rank-2 dylib hit, an unscanned
subcache can still contain the rank-0 framework binary, so stopping
early reintroduces exactly the cross-subcache order-dependence the
ranking was built to remove; and the full scan measures at 43 ms on the
current cache (at most once per CLI invocation), so the only sound
alternative (replicating MachOKit's image enumeration to rank paths
before constructing files) is not worth its drift risk.

Found by the PR #103 review (finding L3, revised); reasoning and
re-adjudication conditions in ReviewAdjudications.md (A6).
Mx-Iris added a commit that referenced this pull request Aug 9, 2026
- ProjectEvolutionLog gains its missing sections for evolutions 0002
  (descriptor slimming) and 0003 (row-bucket flattening) plus a section
  for the review-fix batch itself, and the TaskReports link that named
  a nonexistent file (2026-07-25-dyld-cache-... vs the actual
  2026-07-25-cache-...) now resolves.
- Task report for the whole implementation arc (Chinese, per
  convention): research, refuted premises, per-batch execution with
  pre-fix failure evidence, verification, and divergences.
- The findings document's status line records the outcome (fixed or
  adjudicated per finding, B1 deliberately left to the user).
- AGENTS.md synced with the changed facts: isBind/resolveBind parity +
  hostile-input bounds in MachOExtensions, the nested per-child print
  catch, and the per-image cache-eviction registry.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 108 out of 108 changed files in this pull request and generated no new comments.

…ename

Upstream swift-demangling renamed SymbolStore/SymbolStoreBuilder to
NodeStore/NodeStoreBuilder (the library's domain concept is the demangled
Node tree; there is no Symbol abstraction in its API) and the branch to
feature/node-store. Update the migration plan accordingly. The local
SymbolIndexStore type keeps its name — it is the migration target, not
part of the upstream rename.
…a storage (Stages 1-2)

Storage now owns a frozen per-image NodeStore arena instead of leaking every
demangled tree into the process-global NodeCache:

- Build sweep demangles cache-free onto a transient tree, classifies on it,
  and interns into a NodeStoreBuilder; indexes are collected NodeIndex-keyed
  (PendingStorage) and converted to NodeReference form after freeze().
- All indexes key/carry NodeReference (16-byte handle) instead of Node.
- Node-taking query APIs (memberSymbols(of:for:node:),
  opaqueTypeDescriptorSymbol(for:)) bridge via structurallyEquals bucket
  matching — the frozen store drops its intern tables.
- Late symbols (outside the build sweep) demangle into per-symbol mini
  stores; demangledNodeReference(for:in:) is the new zero-materialization
  lookup, demangledNode(for:in:) keeps the Node contract via materialize().
- Consumers: matchers use demangleSymbolReference + OrderedSet<NodeReference>
  visited sets; DefinitionBuilder/TypeDefinition lookups key by
  NodeReference (O(1) hash after hash-consing); renderer boundaries
  materialize. isGlobal/isAccessor/hasAccessor/accessorKind/isStoredVariable
  generalized over DemanglingNode.

Measured on the SwiftUI image (debug, like-for-like vs main): NodeCache
growth 19,345 leaves / 559,976 subtrees -> 0 / 0; release reclaims
236 MB of 302 MB (legacy 180 MB of 272 MB); NodeStore body 7 MB for
579,291 unique nodes; build 30.7s vs 36.2s under equal parallel load.

Tests: enable MachOSymbolsTests target with SymbolIndexStoreFixtureTests
(8 cases: cache-free invariant via leaf identity, byte-identical printing
vs the Node pipeline, structurallyEquals bucket hits for every key,
demangledNode/reference agreement, late-symbol mini-store fallback);
add manually-run SymbolIndexStoreBaselineTests (Stage 0 metrics);
make SharedCacheTests concurrency proof deterministic via semaphore
barrier instead of wall-clock thresholds. Snapshot acceptance: 60/60
byte-identical against the main-verified baseline.
Stage 3 of the NodeStore migration — the Symbol-side residency item the
arena does not cover:

- Symbol drops the (any NlistProtocol)? stored existential (40 B per copy;
  its only consumer was the collection-time external-undefined filter,
  which runs on the MachOKit symbol before Symbol is even constructed).
  The bit is captured as a stored isExternal Bool; Symbol is explicitly
  Sendable and shrinks 64 B -> 32 B.
- Storage keeps one flat symbolTable: [Symbol] row per unique symbol name
  (canonical cache-adjusted offset), a parallel rootNodeIndexByTableRow
  array, and tableRowByName whose keys share row string storage. Every
  classification index (symbolsByKind, member x3, global, opaque,
  symbolsByOffset) stores 4-byte UInt32 rows; member/opaque keys shrink
  from 16-byte NodeReference to 4-byte NodeStore.NodeIndex. Raw and
  cache-adjusted offset keys share one row; symbols(for:in:) rebuilds
  each Symbol with the queried offset, byte-identical to the former
  per-key copies.
- DemangledSymbol becomes a 32-byte value: shared-table reference + row +
  NodeReference, with symbol computed. The public
  init(symbol:demangledNode:) stays as a single-row-table compat path.
- Indexes accumulate in final row form during the sweep (RowIndexes), so
  freeze() is followed by a plain move into Storage — the pending->populate
  double-index conversion pass (+30 MB transient peak) is gone and all
  Storage index fields are now lets.
- demangledNodeReference(for:) resolves name -> row + canonical-offset
  check, strictly equivalent to the former [Symbol: NodeReference] keying;
  misses still fall through to the late mini-store path.

Stage 4 re-measurement (SwiftUI image, debug, same methodology): build
24.5s (legacy 28.6s, Stage 1+2 31.0s); build-phase phys_footprint delta
68 MB (was 302 MB); post-release residual 49 MB; NodeCache growth stays
0/0; NodeStore identical at 7 MB / 579,291 nodes; index entry counts
match the baseline item for item.

Tests: fixture suite adapted to the row layout plus two new cases —
compactValueLayouts (stride <= 32 B invariants) and
offsetQueriesRebuildSymbolsWithQueriedOffset (shared-row exit semantics);
baseline test reports symbolTable rows/stride. Acceptance: 79 tests in
5 suites passed (60 snapshots byte-identical + 10 fixture cases +
SharedCacheTests).

Note: struct-layout changes in MachOSymbols require a clean rebuild —
SwiftPM incremental builds linked stale downstream objects (runtime
SIGSEGV in outlined destroy of the old Symbol layout).
…ansient demangling de-pins NodeCache (Stages 5a/5c)

Stage 5c — every unbounded demangle/construction site off the global cache:
MetadataReader (28 Node.create sites -> createTransient, 5 demangleAsNode
-> demangleAsNodeTransient incl. the symbolic-reference resolver path),
RuntimeFieldLayoutBackend, TypedDumper, ClassHierarchyDumper,
Symbol.demangledNode, ResolvedTypeReference+, GenericContext+Dump.
NodeCache.shared no longer grows while browsing; bounded singletons keep
interning. SemanticString adopts the lazy @autoclosure scope hook.

Stage 5a — declaration layer holds NodeReference instead of Node:
DefinitionName/TypeName/ProtocolName/ExtensionName, Variable/Function/
SubscriptDefinition.node, FieldDefinition.typeNode and
ExtensionDefinition.genericSignature. Member definitions reference the
per-image symbol store directly (build-time materialize() dropped); the
extension-member path passes main-store keys straight through;
metadata-derived trees are wrapped via NodeReference(interning:) mini
stores, with TypeDefinition.index(in:) batching all field type trees of
one type into a single shared store. Name types customize Hashable to
structural semantics and keep Codable wire-compatible. ABIKey and the
synthesized-member dedup helpers are generic over DemanglingNode;
memberSymbols(of:for:node:) gains a NodeReference overload;
NodeReference.printSemantic provides zero-materialization rich printing.
Five explicit materialize() bridges remain at printer entry points for
Stage 5b to remove.

Acceptance: 98 tests / 15 suites green, interface snapshot byte-identical.
…ting and resolver (Stage 5b-lite)

- DemanglingNode.printSemantic protocol extension: zero-materialization
  rich printing for any representation through the shared generic engine
  (store-backed nominal scopes materialize lazily via the autoclosure
  scope hook).
- DemangleResolver.resolve(for: some DemanglingNode): .options prints
  straight from the node's representation; .builder keeps its public
  Node closure signature and materializes only on that path.
- SwiftDump dumpers pass DemangledSymbol.demangledNode references
  straight to the resolver — the six highest-frequency transient
  materialization sites are gone.

Full genericization of the async NodePrintable stack is deliberately
deferred: after Stage 5a the printer-entry materializations are
transient-only (no resident cost), and the stack synthesizes helper
Nodes in five places that a generic SomeNode cannot express. Decision
and re-entry criteria recorded in NodeStoreMigrationPlan.md.

Acceptance: 98 tests / 15 suites green, interface snapshot
byte-identical; baseline re-measurement clean (NodeCache delta 0/0,
NodeStore 7 MB / 579,291 nodes unchanged, symbolTable 202,603 rows).
… not store identity

Stage 5a replaced the `[Node: …]` method-descriptor and vtable-offset lookups
in `TypeDefinition.index(in:)` with bare `[NodeReference: …]` dictionaries.
`NodeReference`'s intrinsic `Hashable`/`==` is store-identity based
(`(store, index)`), not structural, and the two sides of these lookups do not
share a store: the keys come from override descriptors' implementation symbols,
which `SymbolIndexStore.demangledNodeReference(for:)` hands back from a
per-symbol mini store whenever the symbol falls outside the build sweep, while
the member side queries them with references from the shared image store.
Structurally equal keys from different stores missed, silently dropping both
the `override` keyword and the `// VTable offset:` comment. The offset-keyed
fallback tables are built only from `methodDescriptors`, so override methods had
no second path and lost the annotations entirely.

Add `StructuralNodeReferenceKey` — the same `structurallyEquals` +
`structuralHash` treatment the `Name` types already carry — and key both
lookups on it, wrapping at the five insertion sites and the four query sites.
Containers that group or dedup within a single `memberSymbols` batch keep bare
`NodeReference` keys: they live in one hash-consed store, where structural
equality coincides with index equality.

Caught by a whole-file dump/interface A/B diff against main: iOS 18.5
SwiftData's `Schema.Attribute` rendered 5 overrides on main and 0 on the branch,
while iOS 26.5 SwiftData and SwiftUI/SwiftUICore were byte-identical. After the
fix all three reader sources match main byte for byte — MachOFile 38/38
(iOS 15.5–27.0b2), DyldCache 18/18 (host + iOS 27.0b3/b4), MachOImage 6/6.
…work binary wins

Leaf names are not unique inside a shared cache, but `machOFile(by: .name(_:))`
compared `imagePath.lastPathComponent.deletingPathExtension` and took the first
hit. iOS 27 ships both

    /System/Library/Frameworks/SwiftUI.framework/SwiftUI
    /System/Library/AccessibilityBundles/SwiftUI.axbundle/SwiftUI

so on the simulator caches, which enumerate the accessibility bundle first,
`swift-section --dyld-shared-cache -n SwiftUI` silently resolved to a payload
carrying no Swift metadata: an empty dump, a four-line interface, and exit code
zero. Nothing distinguished it from a framework that genuinely has no types.

Replace the boolean match with a rank (`matchRank(forImagePath:)`): the binary
inside a `<name>.framework` directory — including the macOS `Versions/A/` shape —
scores best, a `.dylib` next, any other same-leaf payload last. `bestMatch(in:)`
returns the best-ranked image and short-circuits on the first best-rank hit, so
an exact path or a present framework binary costs no more than the previous
first-match scan; `.path` lookups always score best rank and are unchanged. Ties
keep the earliest image, so a given cache resolves deterministically.

`DyldCacheImageSearchTests` pins the ranking on synthetic paths (no cache on
disk needed), which is why MachOCachesTests now depends on MachOExtensions.
End to end, `-n SwiftUI` on the iOS 27 simulator cache goes from 0 to 9,131,212
bytes and matches the `-p`-generated baseline byte for byte, with no change to
`-n SwiftUICore`, `-n SwiftData`, or the macOS host cache.
… registration lacked

`addSubIndexer(_:)` had only an index-based inverse (`removeSubIndexer(at:)`),
which callers holding the indexer itself cannot use without tracking positions.
Add the identity overload.

This is what lets a consumer actually release per-image state: registration is
what keeps a sub-indexer — and therefore its whole declaration graph, including
the `NodeStore` its definitions reference — alive for the aggregate's lifetime.
Dropping the last reference lets the sub-indexer deinit, which evicts its
`SymbolIndexStore` entry (see this type's `deinit`), so the memory is genuinely
reclaimed rather than merely unreferenced from one dictionary.

Also documents the dyld cache name-ranking fix in the migration plan and adds
the task report covering both that fix and the RuntimeViewer index-lifecycle
review (including the one subagent finding that did not survive verification).
Mx-Iris added 18 commits August 10, 2026 20:25
The rebase brought proposals 0001-0003 and main's 0004 into one history:
restore 0004's row in the status table, relink its related-proposal
references, and record that the merge precondition for the RV real-device
verification step is now satisfied.
ExtensionDefinition.index(in:)'s two early returns (no conformance
descriptor / conformance without resilient witnesses) returned before
setting isIndexed, so every consumer that gates on the flag re-entered
the whole indexing pass: one interface run touches the same extension
at four points (printExtensionDefinition, printExtensionHeader's
materialization, printDefinition, and the diffable builder), and with
evolution 0002 each re-entry re-parses the conformance and its trailing
objects instead of reading a stored property. The unset-flag gap
pre-exists on main but was free there; the materialization cost is new.

Both early returns are completed indexings ('nothing to index'), so
they now mark the definition indexed; a thrown materialization still
leaves the flag unset so a failed read can be retried.

Found by the PR #103 review (finding H2); regression tests cover the
descriptor-less unit shape and the diffable builder's full bucket sweep.
…r populations

Evolution 0002's storage cleanup releases the section-wrapper arrays at
the end of prepare(), but the six public statistics accessors
(numberOfTypes/Enums/Structs/Classes/Protocols/ProtocolConformances)
still computed over them — every post-preparation read silently
returned 0, indistinguishable from an empty binary, with zero in-repo
consumers to notice (the whole blast radius is downstream panels).

prepare() now freezes the six counts into Storage.PreparationStatistics
immediately before the release and the accessors read that snapshot;
signatures and @inlinable stay unchanged. Evolution 0002's
source-compatibility section and decision log record the behavioral
impact its first landing missed.

Found by the PR #103 review (finding H1); pinned by
statisticsRemainAvailableAfterPreparation.
…ind gained

resolveBind(fileOffset:) learned to answer from the legacy
LC_DYLD_INFO(_ONLY) bind opcode-stream index when chained fixups are
absent, but isBind(_:) still routed through the chained-fixups-only
resolveBind(at:) — on a pre-chained-fixups binary (every iOS 15.5
simulator framework) the two public APIs contradicted each other for
the same slot, so a consumer gating a bind read on isBind got nothing
on exactly the binaries the fallback targets.

isBind(_:) now splits on the same discriminator as
resolveBind(fileOffset:); both stale doc comments (which still asserted
the file 'must contain dyldChainedFixups data') updated to describe the
fallback.

Found by the PR #103 review (finding M4); pinned by
isBindAgreesWithResolveBindOnLegacyBinaries on the legacy fixture.
…stile opcode values

The bind opcode stream is binary-supplied input, but the file-offset
index walked it trusting two raw ulebs: a repeat count of 2^40 spun the
do_bind_uleb_times loop to OOM/hang, and a wrapped or oversized segment
offset attributed a symbol name to an unrelated file offset (the &+
arithmetic never range-checked the slot). Every slot is now checked
against its segment's file size before recording — matching how dyld
bounds slots — and a repeat run terminates when the slot walks past the
segment end, which bounds the loop regardless of the count value.

The review's third claim (Int(segment) trapping on a uleb above
Int.max) is refuted and recorded in the findings doc: MachOKit decodes
the segment index from the opcode's 4-bit immediate, so it can never
exceed 15. The malformed-stream regression test was skipped on the
user's instruction; the legitimate path stays pinned by the four
LegacyDyldInfoBindTests.

Found by the PR #103 review (finding H3, revised).
… PackedNameReference

The packed name reference enforced its 22-bit length / 40-bit offset
budgets with precondition, but both components are binary-supplied: the
length is strlen over the image's string table (a hostile or truncated
table with no NUL inside 4,194,303 bytes trapped the process in
release), the offset a pointer difference into it — and the public
DemangledSymbol(symbol:demangledNode:) packed an unclamped caller name
through the same trap.

The initializer is now failable: the build sweep refuses (skips the row
of) a name whose geometry cannot pack — validated before appending so a
refused name leaves no orphan bytes — while the standalone one-row
table backing the public initializer clamps to the representable prefix
(unreachable for any legitimate mangled name). updateRowInPlace does
raw bit surgery on the already-validated reference instead of
round-tripping through re-validation.

Found by the PR #103 review (finding M3); pinned by
PackedNameReferenceBudgetTests (pre-fix: 'Precondition failed: symbol
name byte length exceeds the 22-bit budget' killed the test runner).

Sweep note (same batch, finding H3): the LC_DYLD_INFO decoder was the
other trusting-binary-values site; both are now bounded.
…d failed test runs

compare_all_pairs derived its verdict solely from the difference count,
which only increments while iterating discovered baseline/*.txt files:
with no cache archive, no installed simulator runtime, or a mistyped
--frameworks, every scenario degrades to paired .skip markers, the glob
yields nothing, and the script printed 'RESULT: all pairs
byte-identical.' and exited 0 over an empty comparison.
run_macho_image_part compounded it by printing swift test's exit code
without propagating it, so a failed MachOImage leg could not fail the
run either. A harness that reads failure as success retroactively
weakens every conclusion that cited its green light.

The verdict now requires a non-empty examined-pair count, reports how
many pairs backed it, and any non-zero test invocation is a run-level
hard failure regardless of the diff outcome.

Found by the PR #103 review (finding H4). Demonstrated pre-fix: an
empty output root returned difference_count 0 (success path); post-fix
the same input takes the zero-pair FAILED branch.
… up its directory

LegacyDyldInfoBindTests compiled its fixture with standardError piped
but read the pipe only AFTER waitUntilExit: diagnostics beyond the
~64 KB pipe buffer would deadlock compiler and parent — and because the
compilation result is a static let under a serialized suite, a
toolchain/SDK mismatch would hang the whole swift test run instead of
reporting a failure. The pipe is now drained to EOF before reaping.

The per-run LegacyDyldInfoBindFixture-<UUID> directory (source + dylib)
also leaked on every run; it is now removed at process exit (the
fixture must outlive every test in the suite, so per-test cleanup is
not an option; a crashed run leaks at most one uniquely-named
directory).

Found by the PR #103 review (finding L2; deadlock shape
mechanism-verified, not reproduced — forcing 64 KB of swiftc
diagnostics is not practical in a fixture).
…stically

offsetQueriesRebuildSymbolsWithQueriedOffset capped its check at 500
entries while iterating a deliberately-unordered Dictionary whose
iteration order is seeded per process — every run sampled a DIFFERENT
500 offsets, so a regression touching only cache-adjusted keys could
pass one run and fail the next in the very test written to pin that
rebuild. It now checks every offset in sorted order (0.065 s on the
fixture — the cap bought nothing).

Found by the PR #103 review (finding L4).
…e indexer

deinit evicted three caches (symbol store, interned-name store,
demangle memo) under one flag that only proves this indexer populated
the symbol-store entry first — with indexer A owning image X and a
later indexer B live on the same image, A's deinit wiped the caches out
from under B: B's already-built names keep an orphaned store alive
while later names land in a fresh one, so structurallyEquals' store ===
fast path stops firing for the pre-eviction population and every
context demangle is re-paid. Per-cache ownership flags would not help
(whoever populates first owns all three and the wipe-under-B is
identical), so ownership is now claimed per IMAGE in a process-wide
registry and the eviction runs in the deinit of the image's LAST live
indexer. Entries built by non-indexer callers are still never claimed
and never evicted — the pre-existing contract, enforced per image.

Found by the PR #103 review (finding M6); pinned by
PerImageCacheEvictionTests on the SymbolTestsHelper fixture (new
MachOFileName case, kept out of shared suites so the membership
assertions cannot race a parallel suite's indexer lifecycle; pre-fix
the survivor test failed on all three caches). MetadataReader gains a
package-visibility non-creating cacheExists(for:) probe for the test.
…nload hazard

Two PR #103 review findings resolve as adjudications rather than the
suggested code changes, with the reasoning and re-adjudication
conditions recorded in ReviewAdjudications.md (A4/A5) and revision
notes on the findings document:

- M5 (detachedFromSharedTable does not detach the node store): the
  suggested node-copy is refuted — the definition storing the detached
  symbol keeps the SAME NodeReference in its own node field (the
  intended per-image recycling model), so a copy reclaims nothing while
  the model lives. Landed as doc-comment precision (the detach is
  symbol-table-layer only; node-store sharing is deliberate) plus
  storedDeclarationSymbolsShareTheDefinitionsNodeStore pinning the
  sharing so a future copy must be a deliberate, measured decision.

- M2 (raw mapped-string-table pointers outliving dlclose): WON'T FIX —
  the open half of its Q1 resolved NO by experiment (macOS 26): dyld
  pins every Swift/ObjC-content image as never-unload (dlclose leaves
  even a class-less Swift dylib mapped), and the only images that do
  unmap (pure C) contain no Swift-prefixed names so they never mint
  mapped rows. The images that could dangle have no rows; the images
  that have rows cannot dangle. Both suggested mitigations refused
  (remove-image hook = permanent dead code for pinned images;
  copy-on-vend breaks the 32-byte shared-table design for an
  unreachable scenario). SymbolTable's lifetime-constraint comment now
  records the empirical guarantee.
…intExtensionHeader

try? collapsed a thrown materializedProtocolConformance(in:) into the
same nil as 'this extension has no conformance', so the public
printExtensionHeader emitted a confidently wrong extension header with
the conformance clause, @retroactive, and global-actor markers silently
missing. In-repo the swallow is unreachable — the only caller indexes
first, and index(in:) runs the same materialization with a bare try —
which is exactly the defect: a public entry holding a weaker error
contract than the index(in:) it follows. The materialization now
propagates; external callers hitting a genuinely unreadable conformance
get the error instead of a silently degraded header.

Found by the PR #103 review (finding M1, revised during implementation:
the originally-claimed rendering-path failure is unreachable in-repo
and the finding was downgraded to Low with the public-contract framing;
see the findings doc's revision note). Pinned by
printExtensionHeaderPropagatesMaterializationFailure via a real
descriptor layout re-wrapped at an out-of-bounds offset (new package
raw-descriptor initializer on ExtensionDefinition as the test surface).
… loops

printTypeDefinition and printExtensionDefinition iterated their nested
type/protocol children with a bare try await: one nested child whose
descriptor could not be read escaped the enclosing definition and the
top-level per-definition catch discarded the WHOLE outer type or
extension. The nested loops now catch per child (printCatchedThrowing,
the same helper printRoot's contract uses), so a corrupt child drops
only itself — one more step in the direction the per-block -> per-
definition change already moved. Healthy-path output is byte-identical
(snapshot suites: 175 tests in 21 suites green).

Found by the PR #103 review (finding L1); pinned by
corruptNestedChildDropsOnlyItself via a real struct-descriptor layout
re-wrapped at an out-of-bounds offset (new package raw-descriptor
initializer on TypeDefinition as the test surface; pre-fix the child's
offsetOutOfBounds failed the whole parent).
…dicate the early exit

The shape all three ranking rounds missed — a plain .dylib name, which
can never reach bestMatchRank and always pays the full multi-cache
scan — now has end-to-end coverage against the current system's dyld
shared cache: libswiftCore resolves to /usr/lib/swift/libswiftCore.dylib
and SwiftUI to the native canonical framework binary (never the
iOSSupport Catalyst build).

The review's suggested achievable-rank early exit is adjudicated as
unsound and not landed: holding a rank-2 dylib hit, an unscanned
subcache can still contain the rank-0 framework binary, so stopping
early reintroduces exactly the cross-subcache order-dependence the
ranking was built to remove; and the full scan measures at 43 ms on the
current cache (at most once per CLI invocation), so the only sound
alternative (replicating MachOKit's image enumeration to rank paths
before constructing files) is not worth its drift risk.

Found by the PR #103 review (finding L3, revised); reasoning and
re-adjudication conditions in ReviewAdjudications.md (A6).
- ProjectEvolutionLog gains its missing sections for evolutions 0002
  (descriptor slimming) and 0003 (row-bucket flattening) plus a section
  for the review-fix batch itself, and the TaskReports link that named
  a nonexistent file (2026-07-25-dyld-cache-... vs the actual
  2026-07-25-cache-...) now resolves.
- Task report for the whole implementation arc (Chinese, per
  convention): research, refuted premises, per-batch execution with
  pre-fix failure evidence, verification, and divergences.
- The findings document's status line records the outcome (fixed or
  adjudicated per finding, B1 deliberately left to the user).
- AGENTS.md synced with the changed facts: isBind/resolveBind parity +
  hostile-input bounds in MachOExtensions, the nested per-child print
  catch, and the per-image cache-eviction registry.
main extracted the in-repo `MachOExtensions` target into the upstream
`MachOKitExtensions` package (so `MachOObjCSection` can depend on it too
without a package-level cycle). Rebasing this branch onto that resolved
the two modify/delete conflicts by taking main's deletion, which left
three `import MachOExtensions` sites and two test-target dependencies
pointing at a module this repo no longer has.

The four behaviors this branch had landed in those two deleted files —
cross-subcache dyld image ranking, the legacy LC_DYLD_INFO(_ONLY) bind
index, its hostile-input bounds check (PR #103 finding H3), and isBind's
matching fallback (M4) — were ported into MachOKitExtensions first, from
this branch's tip state rather than commit by commit, so dropping the
intermediate hunks during the rebase loses nothing. `DyldCacheImageSearchTests`
drops `@testable`: `matchRank` / `bestMatchRank` are public upstream, and
an external package dependency is not built with testability anyway.

Docs updated in the same batch. AGENTS.md was already stale on main (the
extraction commit did not touch it) and this branch had added the
LC_DYLD_INFO paragraph to the same bullet, so it now described a module
this repo does not contain; it is rewritten as the upstream package plus
the two behaviors this repo's tests still pin. Two implementation notes
had dead `Sources/MachOExtensions/` paths. Historical records (task
reports, review findings, changelogs, proposals) are left as written.

Verified: MachOKitExtensions builds standalone; this repo's suite passes
with exit code 0 (1408 tests, 264 suites, IntegrationTests skipped),
built against the local siblings with --manifest-cache none — SwiftPM's
cached manifest evaluation otherwise silently keeps resolving the
upstream package remotely.
The dyld-cache image ranking and the legacy LC_DYLD_INFO(_ONLY) bind index
were developed on this branch against the in-repo `MachOExtensions` target
and ported upstream when that target was extracted; 0.1.1 is the release
carrying them. This is the real floor, not a preference —
`DyldCacheImageSearchTests` and `LegacyDyldInfoBindTests` do not compile
against 0.1.0, which is what the inherited open `from: "0.1.0"` bound would
have allowed a resolver to pick.

Verified against the published tag rather than the local checkout: the two
ported files at 0.1.1 are byte-identical to the copies the suite passed
with, and with the local sibling symlink removed so the package resolves
remotely, the suite passes at exit code 0 (1408 tests, 264 suites,
IntegrationTests skipped). Local resolution and CI now agree.
@Mx-Iris
Mx-Iris force-pushed the feature/node-store-migration branch from aa0a412 to 02a935f Compare August 11, 2026 00:28
Copilot AI review requested due to automatic review settings August 11, 2026 00:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 109 out of 109 changed files in this pull request and generated no new comments.

Suppressed comments (1)

Sources/SwiftSpecialization/TypeDefinition+Specialization.swift:356

  • boundGenericTypeName interns wrappedNode into InternedNodeReferenceCache’s process-scoped store. Since the resulting TypeName is stored on a specialized TypeDefinition, this can pin the entire process-scoped store (which is intended for transient, no-MachO-handle call sites) and reduce the effectiveness of per-image cache eviction.

If you can’t thread a machO handle in here to use the image-scoped reference(interning:in:), it’s safer to avoid the shared process store for a long-lived value and instead mint a standalone NodeReference for the specialized name.

When both sides failed but with DIFFERENT exit codes, each leg unlinked
its .txt and wrote its own .skip, so the pair was invisible to both .txt
globs: neither counter moved and the skip loop printed nothing. With any
other framework identical the run reported "all N pairs byte-identical"
and exited 0 — over a candidate that had, for instance, started trapping
where the baseline merely errored.

This is the same shape as the zero-pairs hole fixed earlier in this
branch (a harness that cannot fail is worse than no harness), so the
verdict logic now has unit tests of its own. They are standard-library
only and run in seconds.
Issue #102 asked for three things when a definition cannot be printed:
keep the partial result, dispatch a failure event, and stop writing to
stdout. Only the first landed. A run that lost 8,375 definitions
therefore dispatched zero definitionPrintFailed events, and its only
signal was a bare `unexpected(at: 8)` on stdout — the very stream the
CLI writes the generated Swift to, so the diagnostic corrupted the
interface it was reporting on and, being buffered in a pipe, surfaced
far from its cause.

printCatchedThrowing now takes the dispatcher and a printing context,
and every call site that has a definition identity supplies one. The two
globals blocks deliberately do not: printVariable / printFunction are
non-throwing and already catch per member, so there is no identity to
attribute a block-level failure to.

Sweeping for the same defect found four more library-side print(error)
sites, all of them on the interface-generating path and all now on
stderr. Sources/ no longer contains a single one.
header(_:_:) swallowed a throw into an empty SemanticString, but
renderType / renderProtocol compute their body units unconditionally and
hand both to DiffContainerAssembler. A type whose header failed was
therefore emitted as members and braces with no `struct Foo` line above
them — structurally invalid Swift, produced silently, with no event and
no error.

Header rendering genuinely throws: it reads the declaration's name,
generic signature and superclass and demangles each (issue #102 is the
field evidence that print-time DemanglingErrors happen on real
binaries), and since evolution 0002 it also re-materializes the wrapper
from its descriptor.

The nil now means "this declaration exists but could not be rendered",
which drops it whole. An absent side stays an empty string, since that
is the case the added/removed markers are built on.
Two fixes on the per-image index, both from the same review round.

opaqueTypeDescriptorSymbol bucketed its entries by member identifier and
scanned the bucket structurally. The assumption was that an identifier
picks out "normally exactly one" descriptor; it does not. The identifier
is the member name, so in SwiftUI the `body` bucket alone holds hundreds
of entries, and the caller queries once per printed `some`-returning
declaration with no memoization — quadratic in a count that runs into
the thousands, with a fresh visited-pair set allocated per comparison.
StructuralNodeReferenceKey now also accepts a bare query Node, hashing
it through the structural hash upstream keeps in step with the
reference's, so the lookup is one probe.

Cache eviction claimed all three per-image caches from one sample of the
symbol store. Only that one is necessarily an indexer's: the
interned-name store and the demangle memo are also populated by
SwiftLayout, the renderers and SwiftSpecialization, so a "dump, then
build the interface" sequence fills both with no symbol store at all —
and the combined claim read that state backwards, evicting caches live
non-indexer work was still using. Each is now claimed separately.
Registration is keyed on indexer identity rather than counted, so a
concurrent second prepare() cannot strand the population above zero and
leak all three for the process lifetime.

Also renames a `cls` binding this branch carried over, per the no-
abbreviations rule.
Evolution 0001's compatibility section claimed "purely additive / no
breakage" and "no superseded public API" while the landed code had
replaced Symbol.nlist with isExternal. An accepted proposal stating the
opposite of the shipped code is worse than the API change itself, so it
now records the break honestly — including the mitigating fact that the
default argument keeps Symbol(offset:name:) compiling.

AGENTS.md follows the two code changes in this batch (structural opaque
index, per-cache eviction claims) and gains the A/B harness self-test.
Adjudications A7/A8 record the two findings deliberately not fixed: the
print-options divergence between the indexer and dump witness matchers
(no reproduction, identical on main, no fixture) and updateConfiguration's
re-prepare being a no-op (unreachable for every known consumer).

Node+.swift's printSemantic gains a warning not to "modernize" onto the
new runPrintWalk requirement: that hook returns String because it
dispatches print(using:), so a custom target is outside its remit by
design, and the engine's static entry point — including its stack guard
— is unchanged.
Copilot AI review requested due to automatic review settings August 13, 2026 03:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 116 out of 117 changed files in this pull request and generated no new comments.

Suppressed comments (3)

Tests/IntegrationTests/MachOSymbols/SymbolIndexStoreBaselineTests.swift:76

  • Tests/IntegrationTests/ is intended for manual inspection without assertions. #expect(demangledSymbolCount > 0) makes this behave like a normal unit test and can fail the suite for environmental reasons; prefer logging the metric (or move this into MachOSymbolsTests if it should be enforced).
    Tests/IntegrationTests/MachOSymbols/SymbolIndexStoreBaselineTests.swift:36
  • Tests/IntegrationTests/ is documented as a manual-inspection-only target (no assertions/preconditions). Using #require here turns the run into a hard failure and can break unrelated CI invocations; consider either moving this suite into a non-Integration test target or degrading to a soft failure (e.g. guard let … else { print(...); return }).

This issue also appears on line 76 of the same file.
Sources/SwiftSpecialization/TypeDefinition+Specialization.swift:355

  • boundGenericTypeName stores the resulting TypeName on specialized definitions, but it interns wrappedNode into the process-scoped InternedNodeReferenceCache (reference(interning:)). That bypasses the per-image scope/eviction model used for other names (e.g. TypeContextDescriptorWrapper.typeName(in:) uses reference(interning:…, in: machO)), and can keep interned nodes alive beyond the image/indexer lifecycle. Consider threading machO into this helper so the bound name is interned into the same image scope as the unbound name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants