Skip to content

printRoot() discards an entire definition block when a single definition throws #102

Description

@vkharion

Summary

Two issues in one causal chain. The second is the trigger; the first is what makes
it catastrophic, and I think it is the more important of the two.

  1. printRoot() wraps each of its six definition blocks in
    printCatchedThrowing, which print(error)s and returns nil. Because
    BlockList keeps no partial result, a throw on element k discards
    0…k-1 as well — the entire block is lost.
  2. Two StructuredQueriesCore extension definitions throw
    DemanglingError.unexpected(at:) while rendering their associated-type
    records.

Net effect on a real binary: 2 unprintable definitions out of 8,375 cost all
8,375
, and the resulting output looks entirely healthy.

Environment

  • MachOSwiftSection 0.11.0
  • swift-demangling 0.4.0
  • Swift 6.3.3 (swiftlang-6.3.3.1.1), macOS 27.0
  • Subject: an iOS Simulator Debug build, arm64 slice, ~333 MB, statically linking
    pointfreeco/swift-structured-queries

1. One throwing definition costs the whole block

Measured on the same binary, before and after switching from printRoot() to
driving the six blocks per definition (both slices, arm64 + x86_64):

printRoot() per definition
Swift interface 5,512,116 chars 10,231,900 chars
extension declarations 0 16,752
definitions rendered 23,820 / 23,824

So 46.1% of the interface was missing, and the part that was missing was
every extension in the product.

Nothing about the output suggests truncation — it is multi-megabyte, contains
classes, structs, enums and protocols, and simply ends after the last protocol.
The only signal is a bare unexpected(at: 8) on stdout, with no context, which
in a piped/CI log is fully buffered and therefore surfaces far from its cause.

Three suggestions, in rough order of value:

  • Keep partial results. Rendering per definition means one bad definition
    costs one definition. That is what I ended up doing downstream, and it also
    makes the loss countable — I now compare the indexer's per-block definition
    count against the printed count and hard-fail if a non-empty block renders
    zero.
  • Dispatch an event on failure. dispatchingCatchedThrowing does dispatch,
    but the block-level printCatchedThrowing does not — so passing
    eventHandlers does not surface this. I measured 0
    definitionPrintFailed events for a run that lost 8,375 definitions.
  • Don't print(error) to stdout from a library.

Iterating block 6 (typeExtensionDefinitions + protocolExtensionDefinitions +
typeAliasExtensionDefinitions + conformanceExtensionDefinitions) and calling
printer.printExtensionDefinition per definition, on the arm64 slice:

extensions = 8375
failures   = 2

Blocks 1–2 cannot contribute: printVariable and printFunction are
non-throwing. So block 6 is where this originates.

2. The trigger: associated-type records carrying symbolic references

Throw path:

printExtensionDefinition
  └─ AssociatedTypeDumper.mergedRecords(of:using:in:)   // SwiftInterfacePrinter.swift:230
       └─ DemanglingError.unexpected(at:)

The two failing definitions and their associated-type records. Control bytes —
MangledName.Element.lookup, i.e. symbolic references, which rawString
collapses to a single UnicodeScalar(reference.kind) — are shown as <n>:

FAIL 1 — DemanglingError.unexpected(at: 8)
  protocolTypeName:     $s21StructuredQueriesCore27PrimaryKeyedTableDefinitionP
  substitutedTypeName:  10PrimaryKey<2>Qz
  substitutedTypeName:  <1>yxq___G

FAIL 2 — DemanglingError.unexpected(at: 6)
  protocolTypeName:     $s21StructuredQueriesCore15QueryExpressionP
  substitutedTypeName:  xSg
  protocolTypeName:     $s21StructuredQueriesCore27PrimaryKeyedTableDefinitionP
  substitutedTypeName:  10PrimaryKey<2>QzSg
  substitutedTypeName:  <1>yx__G

Both involve a symbolic reference in a position the parse does not appear to
expect: <1> leads a bound-generic mangling (<1>yxq___G), and <2> sits
between a length-prefixed identifier and an associated-type operator
(10PrimaryKey<2>Qz).

Worth noting: the offsets differ (8 vs 6) for near-identical inputs
(10PrimaryKey<2>Qz vs 10PrimaryKey<2>QzSg), so at: tracks parse progress
rather than a fixed index into the input — it is not directly usable to locate
the offending character.

I did not determine whether the defect is a grammar gap in swift-demangling or
a resolver/handling issue in AssociatedTypeDumper, which is why this is an
issue rather than a pull request.

Reproduction

Any Mach-O statically linking swift-structured-queries should reproduce it:

let builder = try SwiftInterfaceBuilder(configuration: .init(), eventHandlers: [], in: machO)
try await builder.prepare()
let text = try await builder.printRoot().string
// text contains no `extension` declarations at all;
// `unexpected(at: 8)` is printed to stdout once per call

To confirm the amplification, iterate indexer.typeExtensionDefinitions /
protocolExtensionDefinitions / typeAliasExtensionDefinitions /
conformanceExtensionDefinitions and call printer.printExtensionDefinition
per definition: 8,373 succeed.

Happy to run further diagnostics against this binary if it would help — I can
dump any additional state from the two failing definitions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions