Summary
Two issues in one causal chain. The second is the trigger; the first is what makes
it catastrophic, and I think it is the more important of the two.
printRoot() wraps each of its six definition blocks in
printCatchedThrowing, which print(error)s and returns nil. Because
BlockList keeps no partial result, a throw on element k discards
0…k-1 as well — the entire block is lost.
- Two
StructuredQueriesCore extension definitions throw
DemanglingError.unexpected(at:) while rendering their associated-type
records.
Net effect on a real binary: 2 unprintable definitions out of 8,375 cost all
8,375, and the resulting output looks entirely healthy.
Environment
- MachOSwiftSection 0.11.0
- swift-demangling 0.4.0
- Swift 6.3.3 (
swiftlang-6.3.3.1.1), macOS 27.0
- Subject: an iOS Simulator Debug build, arm64 slice, ~333 MB, statically linking
pointfreeco/swift-structured-queries
1. One throwing definition costs the whole block
Measured on the same binary, before and after switching from printRoot() to
driving the six blocks per definition (both slices, arm64 + x86_64):
|
printRoot() |
per definition |
| Swift interface |
5,512,116 chars |
10,231,900 chars |
extension declarations |
0 |
16,752 |
| definitions rendered |
— |
23,820 / 23,824 |
So 46.1% of the interface was missing, and the part that was missing was
every extension in the product.
Nothing about the output suggests truncation — it is multi-megabyte, contains
classes, structs, enums and protocols, and simply ends after the last protocol.
The only signal is a bare unexpected(at: 8) on stdout, with no context, which
in a piped/CI log is fully buffered and therefore surfaces far from its cause.
Three suggestions, in rough order of value:
- Keep partial results. Rendering per definition means one bad definition
costs one definition. That is what I ended up doing downstream, and it also
makes the loss countable — I now compare the indexer's per-block definition
count against the printed count and hard-fail if a non-empty block renders
zero.
- Dispatch an event on failure.
dispatchingCatchedThrowing does dispatch,
but the block-level printCatchedThrowing does not — so passing
eventHandlers does not surface this. I measured 0
definitionPrintFailed events for a run that lost 8,375 definitions.
- Don't
print(error) to stdout from a library.
Iterating block 6 (typeExtensionDefinitions + protocolExtensionDefinitions +
typeAliasExtensionDefinitions + conformanceExtensionDefinitions) and calling
printer.printExtensionDefinition per definition, on the arm64 slice:
extensions = 8375
failures = 2
Blocks 1–2 cannot contribute: printVariable and printFunction are
non-throwing. So block 6 is where this originates.
2. The trigger: associated-type records carrying symbolic references
Throw path:
printExtensionDefinition
└─ AssociatedTypeDumper.mergedRecords(of:using:in:) // SwiftInterfacePrinter.swift:230
└─ DemanglingError.unexpected(at:)
The two failing definitions and their associated-type records. Control bytes —
MangledName.Element.lookup, i.e. symbolic references, which rawString
collapses to a single UnicodeScalar(reference.kind) — are shown as <n>:
FAIL 1 — DemanglingError.unexpected(at: 8)
protocolTypeName: $s21StructuredQueriesCore27PrimaryKeyedTableDefinitionP
substitutedTypeName: 10PrimaryKey<2>Qz
substitutedTypeName: <1>yxq___G
FAIL 2 — DemanglingError.unexpected(at: 6)
protocolTypeName: $s21StructuredQueriesCore15QueryExpressionP
substitutedTypeName: xSg
protocolTypeName: $s21StructuredQueriesCore27PrimaryKeyedTableDefinitionP
substitutedTypeName: 10PrimaryKey<2>QzSg
substitutedTypeName: <1>yx__G
Both involve a symbolic reference in a position the parse does not appear to
expect: <1> leads a bound-generic mangling (<1>yxq___G), and <2> sits
between a length-prefixed identifier and an associated-type operator
(10PrimaryKey<2>Qz).
Worth noting: the offsets differ (8 vs 6) for near-identical inputs
(10PrimaryKey<2>Qz vs 10PrimaryKey<2>QzSg), so at: tracks parse progress
rather than a fixed index into the input — it is not directly usable to locate
the offending character.
I did not determine whether the defect is a grammar gap in swift-demangling or
a resolver/handling issue in AssociatedTypeDumper, which is why this is an
issue rather than a pull request.
Reproduction
Any Mach-O statically linking swift-structured-queries should reproduce it:
let builder = try SwiftInterfaceBuilder(configuration: .init(), eventHandlers: [], in: machO)
try await builder.prepare()
let text = try await builder.printRoot().string
// text contains no `extension` declarations at all;
// `unexpected(at: 8)` is printed to stdout once per call
To confirm the amplification, iterate indexer.typeExtensionDefinitions /
protocolExtensionDefinitions / typeAliasExtensionDefinitions /
conformanceExtensionDefinitions and call printer.printExtensionDefinition
per definition: 8,373 succeed.
Happy to run further diagnostics against this binary if it would help — I can
dump any additional state from the two failing definitions.
Summary
Two issues in one causal chain. The second is the trigger; the first is what makes
it catastrophic, and I think it is the more important of the two.
printRoot()wraps each of its six definition blocks inprintCatchedThrowing, whichprint(error)s and returnsnil. BecauseBlockListkeeps no partial result, a throw on element k discards0…k-1as well — the entire block is lost.StructuredQueriesCoreextension definitions throwDemanglingError.unexpected(at:)while rendering their associated-typerecords.
Net effect on a real binary: 2 unprintable definitions out of 8,375 cost all
8,375, and the resulting output looks entirely healthy.
Environment
swiftlang-6.3.3.1.1), macOS 27.0pointfreeco/swift-structured-queries1. One throwing definition costs the whole block
Measured on the same binary, before and after switching from
printRoot()todriving the six blocks per definition (both slices, arm64 + x86_64):
printRoot()extensiondeclarationsSo 46.1% of the interface was missing, and the part that was missing was
every extension in the product.
Nothing about the output suggests truncation — it is multi-megabyte, contains
classes, structs, enums and protocols, and simply ends after the last
protocol.The only signal is a bare
unexpected(at: 8)on stdout, with no context, whichin a piped/CI log is fully buffered and therefore surfaces far from its cause.
Three suggestions, in rough order of value:
costs one definition. That is what I ended up doing downstream, and it also
makes the loss countable — I now compare the indexer's per-block definition
count against the printed count and hard-fail if a non-empty block renders
zero.
dispatchingCatchedThrowingdoes dispatch,but the block-level
printCatchedThrowingdoes not — so passingeventHandlersdoes not surface this. I measured 0definitionPrintFailedevents for a run that lost 8,375 definitions.print(error)to stdout from a library.Iterating block 6 (
typeExtensionDefinitions+protocolExtensionDefinitions+typeAliasExtensionDefinitions+conformanceExtensionDefinitions) and callingprinter.printExtensionDefinitionper definition, on the arm64 slice:Blocks 1–2 cannot contribute:
printVariableandprintFunctionarenon-throwing. So block 6 is where this originates.
2. The trigger: associated-type records carrying symbolic references
Throw path:
The two failing definitions and their associated-type records. Control bytes —
MangledName.Element.lookup, i.e. symbolic references, whichrawStringcollapses to a single
UnicodeScalar(reference.kind)— are shown as<n>:Both involve a symbolic reference in a position the parse does not appear to
expect:
<1>leads a bound-generic mangling (<1>yxq___G), and<2>sitsbetween a length-prefixed identifier and an associated-type operator
(
10PrimaryKey<2>Qz).Worth noting: the offsets differ (8 vs 6) for near-identical inputs
(
10PrimaryKey<2>Qzvs10PrimaryKey<2>QzSg), soat:tracks parse progressrather than a fixed index into the input — it is not directly usable to locate
the offending character.
I did not determine whether the defect is a grammar gap in
swift-demanglingora resolver/handling issue in
AssociatedTypeDumper, which is why this is anissue rather than a pull request.
Reproduction
Any Mach-O statically linking
swift-structured-queriesshould reproduce it:To confirm the amplification, iterate
indexer.typeExtensionDefinitions/protocolExtensionDefinitions/typeAliasExtensionDefinitions/conformanceExtensionDefinitionsand callprinter.printExtensionDefinitionper definition: 8,373 succeed.
Happy to run further diagnostics against this binary if it would help — I can
dump any additional state from the two failing definitions.