perf: eliminated strict lookup allocations and sorting

[alyldas]

Summary

• Replace per-token candidate cloning, lookup and de-duplication sets, and sorting with per-invocation scratch state and an ordered bucket merge.
• Preserve numeric precedence across fallback, case-variant, and split-token buckets while rebuilding indexes after runtime dictionary mutation.
• Keep scratch state local to each strict scan so reentrant iterators and multiple filter instances remain independent.
• Add targeted compatibility coverage for metadata precedence, duplicate buckets, UTF-16 ranges, mutation, and scanner early stop.
• Add a focused benchmark that reports strict-token timing plus per-call Set construction and Array sorting counts.

Validation

• npm ci
• npm test -- tests/strict-token-index.spec.ts tests/api.spec.ts tests/scanner.spec.ts tests/runtime-literals.spec.ts tests/normalization.spec.ts tests/internal-rules.spec.ts
• npm run check
• npm run benchmark:strict-lookup with the same harness on origin/main and this branch
• npm run benchmark:profanity on origin/main and this branch with the same machine and command shape

Benchmark Evidence

Baseline origin/main:

• createProfanityFilter(): 7.0241 avg ms
• unique clean tokens: 1.1341 avg ms; 600 Set constructions and 300 Array sorts per call
• repeated clean tokens: 0.8620 avg ms; 600 Set constructions and 300 Array sorts per call
• strict early match: 0.1452 avg ms; 2 Set constructions and 1 Array sort per call
• strict late match: 0.4411 avg ms; 602 Set constructions and 301 Array sorts per call
• general long clean: 0.9487 avg ms
• general long late match: 0.2607 avg ms

This branch:

• createProfanityFilter(): 7.1659 avg ms
• unique clean tokens: 1.0218 avg ms; 0 Set constructions and 0 Array sorts per call
• repeated clean tokens: 0.7701 avg ms; 0 Set constructions and 0 Array sorts per call
• strict early match: 0.1472 avg ms; 0 Set constructions and 0 Array sorts per call
• strict late match: 0.3542 avg ms; 0 Set constructions and 0 Array sorts per call
• general long clean: 0.9382 avg ms
• general long late match: 0.2336 avg ms

The unique-token negative control improved by about 9.9%, repeated clean text by about 10.7%, and the strict late match by about 19.7%. Construction and the strict early hit remained within normal same-machine run variance.

Compatibility Notes

• Public APIs, exports, scanner inputs, and dictionary formats are unchanged.
• Strict rule order, fallback precedence, taxonomy metadata, UTF-16 analyzer ranges, scanner code-point ranges, and early-stop behavior remain compatible.
• Runtime mutation rebuilds the ordered token index, while per-invocation scratch state prevents stale state from crossing filters or reentrant iterators.
• Symbol-token, phrase, loose-index, normalization, and boundary behavior are unchanged.
• Release Please PR chore(main): release 0.16.2 #175 remains open and is reserved for the single milestone release.

Closes #171

No publish, no merge, no tag/release.