IntlCollator: honor locale-tailored caseFirst default (da, mt) - #365
IntlCollator: honor locale-tailored caseFirst default (da, mt)#365robobun wants to merge 1 commit into
Conversation
CLDR's collation data for da (Danish) and mt (Maltese) specifies [caseFirst upper], so a default collator for those locales should sort uppercase before lowercase and report resolvedOptions().caseFirst === "upper". initializeCollator previously always called ucol_setAttribute( UCOL_CASE_FIRST, UCOL_OFF) when no caseFirst option or -u-kf- extension was supplied, overwriting the locale tailoring, and derived m_caseFirst only from the option/extension value. resolveLocale already leaves the Kf extension as a null String in that case (see the early-continue in IntlObject.cpp), so skip the setAttribute when it is null and read the actual UCOL_CASE_FIRST back from the collator to populate m_caseFirst. This mirrors the existing handling of UCOL_ALTERNATE_HANDLING / m_ignorePunctuation directly above it.
|
Warning Review limit reached
Next review available in: 4 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
Comment |
There was a problem hiding this comment.
Beyond the inline note, I also verified: (1) resolveLocale in IntlObject.cpp:1048 does leave extensions[Kf] as a null String when neither an option nor a -u-kf- subtag is present, so the isNull() gate is sound; (2) the ASCII fast path stays safe — canDoASCIIUCADUCETComparisonWithUCollator independently checks UCOL_CASE_FIRST == UCOL_OFF on the collator, and m_caseFirst now agrees with it for da/mt.
Extended reasoning...
Confirmed the two load-bearing assumptions the fix rests on. First, the early continue at IntlObject.cpp:1048 (keyPos == notFound && !optionsValue) skips writing resolved.extensions[Kf], so caseFirstString.isNull() reliably distinguishes the no-option/no-extension path from an explicit "false". Second, the ASCII UCADUCET fast path was already double-gated: even before this change, canDoASCIIUCADUCETComparisonWithUCollator read UCOL_CASE_FIRST directly from the collator and would have rejected da/mt once the setAttribute overwrite is removed — the read-back into m_caseFirst just makes the earlier m_caseFirst != CaseFirst::False short-circuit consistent with it. No new fast-path risk introduced.
| if (!caseFirstString.isNull()) | ||
| ucol_setAttribute(m_collator.get(), UCOL_CASE_FIRST, caseFirst, &status); |
There was a problem hiding this comment.
🟣 Pre-existing edge case not covered by this fix: an unrecognized -u-kf- value (e.g. new Intl.Collator('da-u-kf-xyz') or bare 'da-u-kf') still forces UCOL_OFF and reports caseFirst: "false", because resolveLocale falls back to keyLocaleData[0] — which sortLocaleData hardcodes as "false" regardless of locale — so caseFirstString is non-null and the read-back is skipped. The PR doesn't regress this (it was already "false" before) and correctly fixes the far-more-common no-extension path; a possible follow-up is to make sortLocaleData's Kf list locale-aware, or to also read back when the resolved value is "false" but no explicit option/extension requested it.
Extended reasoning...
What the bug is
The PR's read-back of UCOL_CASE_FIRST fires only when caseFirstString.isNull(). That correctly catches the no-extension / no-option case, but there is one more path where the caller did not actually specify a caseFirst yet caseFirstString comes back non-null: a -u-kf- Unicode extension whose value is not one of upper / lower / false (e.g. 'da-u-kf-xyz', or a bare 'da-u-kf'). On that path the locale's tailored [caseFirst upper] is still overwritten with UCOL_OFF.
The specific code path
In resolveLocale (IntlObject.cpp:1041–1069):
- For
'da-u-kf-xyz',extensionSubtagscontainskf, sokeyPos != notFoundand the earlycontinueat line 1048 is not taken. keyLocaleData = sortLocaleData("da", Kf)returns["false", "lower", "upper"]— hardcoded independent of locale (IntlCollator.cpp:104).value = keyLocaleData[0]→"false"(line 1054).- The requested value
"xyz"is not found inkeyLocaleData(line 1060–1061), sovaluestays"false". For a bare'da-u-kf',keyLocaleDatadoesn't contain"true"(line 1065), sovaluealso stays"false". resolved.extensions[Kf]is the non-null string"false".
Back in initializeCollator:
caseFirstString == "false"→m_caseFirst = CaseFirst::False→ localcaseFirst = UCOL_OFF.!caseFirstString.isNull()is true, soucol_setAttribute(UCOL_CASE_FIRST, UCOL_OFF)still runs, clobberingda's CLDR[caseFirst upper]tailoring.- The new read-back block is skipped because
caseFirstString.isNull()is false.
Why existing code doesn't prevent it
Per ECMA-402 §10.2.3, [[LocaleData]].[[<locale>]].kf's first element should be the locale's default ("upper" for da/mt), so an unrecognized extension value should fall back to that default. sortLocaleData/searchLocaleData instead hardcode ["false", "lower", "upper"] for every locale, so keyLocaleData[0] is always "false". This is a pre-existing spec-compliance gap; the PR's isNull() gate can't distinguish “user explicitly asked for false” from “resolveLocale fell back to element 0.”
Impact
After this PR:
new Intl.Collator('da').resolvedOptions().caseFirst === 'upper'✅ (fixed)new Intl.Collator('da-u-kf-xyz').resolvedOptions().caseFirst === 'false'and sorts lowercase-first, whereas V8/ICU return'upper'.
This is unchanged from before the PR — 'da-u-kf-xyz' already returned "false" and forced UCOL_OFF. The PR is a strict improvement and does not regress this edge case. The trigger (users passing an invalid -u-kf- value to a da/mt collator) is quite obscure.
Step-by-step proof
Input: new Intl.Collator('da-u-kf-xyz')
requestedLocales = ['da-u-kf-xyz']
caseFirstOption = null (no options object)
→ resolveLocale:
extensionSubtags = ['kf', 'xyz']
keyPos('kf') = 0 ≠ notFound → skip line-1048 continue
keyLocaleData = sortLocaleData('da', Kf) = ['false','lower','upper']
value = keyLocaleData[0] = 'false'
'xyz' ∉ keyLocaleData → value stays 'false'
resolved.extensions[Kf] = 'false' (non-null)
→ initializeCollator:
caseFirstString = 'false' → m_caseFirst = False → caseFirst = UCOL_OFF
!caseFirstString.isNull() → ucol_setAttribute(UCOL_CASE_FIRST, UCOL_OFF) ← overwrites da tailoring
caseFirstString.isNull() == false → read-back block skipped
Result: resolvedOptions().caseFirst == 'false', compare('a','A') == -1
Possible fixes (follow-up)
Either would close the gap:
- Make
sortLocaleData/searchLocaleDatalocale-aware forKf(return["upper", ...]forda/mt) sokeyLocaleData[0]is the actual locale default per spec; or - Also take the read-back path when
caseFirstString == "false"_s && caseFirstOption.isNull()(i.e.,"false"came from fallback rather than an explicit{caseFirst: 'false'}/-u-kf-false). Explicit-u-kf-falsewould still round-trip correctly because it is inkeyLocaleDataand produces the same non-null"false", so an extra flag distinguishing “explicit vs fallback” may be needed for full precision.
Preview Builds
|
CLDR's collation data for
da(Danish) andmt(Maltese) specifies[caseFirst upper]: a default collator for those locales should sort uppercase before lowercase and reportresolvedOptions().caseFirst === "upper"(V8/ICU honor this). JSC instead reported"false"and sorted lowercase first.Cause
initializeCollatoralways calleducol_setAttribute(UCOL_CASE_FIRST, UCOL_OFF)when nocaseFirstoption or-u-kf-extension was supplied, overwriting the ICU locale tailoring, and derivedm_caseFirstonly from the option/extension value (never from the opened collator).Fix
resolveLocalealready leaves theKfextension as a nullStringwhen neither an option nor a Unicode extension requests a value (see the earlycontinueinIntlObject.cpp). When it is null:ucol_setAttribute(UCOL_CASE_FIRST, ...)call so the locale's tailored default stays in effect, anducol_getAttribute(UCOL_CASE_FIRST)back intom_caseFirstsoresolvedOptions()reports the actual value andupdateCanDoASCIIUCADUCETComparison()gates the ASCII fast path correctly.This mirrors the existing handling of
UCOL_ALTERNATE_HANDLING/m_ignorePunctuationdirectly above it.Explicit
{ caseFirst: "false" }or-u-kf-falsestill produce a non-nullKfextension and continue to forceUCOL_OFFas before.Repro
Branched from
549170099226(Bun's currentWEBKIT_VERSION) so the paired Bun PR can pin the preview build without pulling in unrelated WebKit changes.