Skip to content

perf: reused prepared input across scanners and profiles - #179

Merged
alyldas merged 1 commit into
mainfrom
perf/reuse-prepared-input
Jul 23, 2026
Merged

perf: reused prepared input across scanners and profiles#179
alyldas merged 1 commit into
mainfrom
perf/reuse-prepared-input

Conversation

@alyldas

@alyldas alyldas commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Reuse invocation-local same-length normalized views across package-owned scanners and composed profiles that share a normalization strategy.
  • Cache loose candidate facts by matcher index so runtime dictionary mutation keeps isolation and cannot reuse stale evidence.
  • Route scanner check, legacy analysis, and sink streaming through the prepared capability while preserving custom-filter fallbacks, early stop, source ordering, taxonomy metadata, provenance, and UTF-16 ranges.
  • Corroborate shared length hints against the actual input and document filter-instance reuse outside the per-message hot path.
  • Extend regression coverage and the benchmark harness for same-strategy reuse, mixed strategies, scanner paths, fallback behavior, and construction cost.

Validation

  • npm ci
  • npm test -- tests/prepared-input.spec.ts tests/scanner.spec.ts tests/multilanguage-filter.spec.ts tests/normalization.spec.ts tests/english-language-pack.spec.ts tests/api.spec.ts
  • npm run check
  • npm run benchmark:profanity

Benchmark Evidence

Baseline origin/main:

  • createProfanityFilter(): 7.3146 avg ms
  • single-profile scanner check, long clean: 0.9102 avg ms
  • single-profile scanner scan, long late match: 1.2322 avg ms
  • same-strategy composed check, long clean: 0.6574 avg ms
  • same-strategy composed analyze, match: 0.4884 avg ms
  • same-strategy composed censor, match: 0.5374 avg ms
  • same-strategy composed scanner scan, match: 0.6448 avg ms

This branch:

  • createProfanityFilter(): 7.5875 avg ms
  • single-profile scanner check, long clean: 0.8920 avg ms
  • single-profile scanner scan, long late match: 1.0560 avg ms
  • same-strategy composed check, long clean: 0.5373 avg ms
  • same-strategy composed analyze, match: 0.4670 avg ms
  • same-strategy composed censor, match: 0.5304 avg ms
  • same-strategy composed scanner scan, match: 0.4599 avg ms

Within the final branch run, reused versus public-fallback preparation measured 0.5373 vs 0.6136 avg ms for long-clean checks, 0.3735 vs 0.5746 for matching checks, 0.4670 vs 0.5055 for analysis, 0.5304 vs 0.6666 for censoring, 0.4522 vs 0.4715 for scanner checks, and 0.4599 vs 0.5113 for scanner scans. Filter-construction cost remains visible, and the documentation directs consumers to reuse constructed filters.

Compatibility Notes

  • Public filter, profile, scanner, dictionary, and @textfilters/core contracts are unchanged; the prepared capability remains package-internal.
  • Custom filters without the capability retain the public check() and analyze() fallback, and missing, partial, or conflicting hints do not change matching behavior.
  • check(), analyze(), censor(), scanner ranges, streaming order, legacy metadata ordering, profile provenance, taxonomy metadata, runtime mutation isolation, and UTF-16 offsets remain compatible.

Closes #173

No publish, no merge, no tag/release.

@alyldas
alyldas merged commit ff288c2 into main Jul 23, 2026
4 checks passed
@alyldas
alyldas deleted the perf/reuse-prepared-input branch July 23, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Reuse prepared input across scanners and composed profiles

1 participant