fix: a fold target must not itself be a source (#723)

[raeq]

Summary

044B ы mapped to ƅi, and ƅ (U+0185) is a source folding to b:

	ы
canonicalize, normalize_confusables — iterate	bi
strip_obfuscation, get_pipeline's confusables step — one pass	ƅi

Why it survived

The issue's title names it: the exhaustive idempotence gate tests the one function that iterates. A gate aimed at the forgiving caller cannot see a defect that only the strict one meets.

Fixed in the data, not in a consumer

That is the part that keeps it closed.

• scripts/gen_confusables.py resolves every target through the map until it is a fixed point.
• build.rs's "a target must not itself be a source" assert now checks every character of every target. It previously checked only if target.len() == 1 — exactly the hole ƅi fell through.

The assert can hold only because the data satisfies it.

Two rows resolved: 044B → bi, and 1D14 → eo (was ǝo) — a second chain the issue does not name, which the widened assert found. strip_obfuscation is now a fixed point over the whole BMP, where it had two exceptions.

Keys move

90 rows, all strip_obfuscation, all toward the answer the iterating callers already gave — Apy6ƅi becomes Apy6bi. canonicalize, normalize_confusables, search_key, catalog_key and sort_key are byte-identical.

Carries an Upgrade note. KEY_SCHEMA_VERSION stays at 2, already bumped for this cycle.

Tests

tests/test_target_chains.py asserts the property from three sides: no target contains a source (naming the row — a build-script panic names only the file), the single-pass and iterating callers agree, and strip_obfuscation is a fixed point over the BMP.

U+04F9 is deliberately excluded from the agreement test. It carries a diaeresis, so strip_obfuscation and canonicalize differ there for a correct and unrelated reason, and comparing them would assert that accent preservation is a bug. Its own claim — that the single-pass answer is now stable — is tested separately.

Closes #723

🤖 Generated with Claude Code

[Copilot]

🟢 Approval recommended

The behavioral change is applied at the data source, enforced by a stronger build-time invariant, and backed by targeted regression tests, with only minor test/message nits noted.

Pull request overview

This PR closes a confusables-table chaining hole where a fold target could itself be a fold source, causing single-pass callers (notably strip_obfuscation) to stop short of the fixed point that iterating callers already reached. The fix is applied at data-generation time, enforced at build time, and covered by a new regression test.

Changes:

• Resolve transitive target chains during scripts/gen_confusables.py generation so fold targets are fixed points.
• Strengthen build.rs’s “target must not itself be a source” assertion to check every character of multi-character targets.
• Update affected TSV rows, add a regression test suite, and document the upgrade impact in CHANGELOG.md.

File summaries

File	Description
tests/test_target_chains.py	Adds regression coverage for target-chaining, single-pass vs fixed-point agreement, and strip_obfuscation BMP fixed-point behavior.
src/tables/data/confusables_to_latin.tsv	Updates the two resolved rows (044B -> bi, 1D14 -> eo).
scripts/gen_confusables.py	Introduces _resolve_target_chains and applies it to generated mappings before writing TSVs.
CHANGELOG.md	Adds upgrade note + fixed entry describing key movement for stored strip_obfuscation outputs and the root cause/fix.
build.rs	Expands the build-time assertion to detect sources embedded anywhere in multi-character targets.

Review details

• Files reviewed: 5/6 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.