You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Unspaced CJK names ("毛泽东", "김민준") can't be parsed today because there's nothing to split into pieces. For Chinese and Korean, segmentation is cheap and fits the library's constants-driven style, because surnames come from small closed sets while given names are open-ended — only a surname dictionary is needed:
Korean: 286 surnames total (2015/2019 census); Kim/Lee/Park alone cover ~45% of the population; about a dozen two-syllable surnames. Complete coverage in ~300 entries.
Chinese: top 100 surnames cover ~86% of the population, ~6,000 in current use; surnames are 1 character, occasionally 2 (compound: 欧阳, 司马, …). Excellent coverage in ~500 entries.
Proposal:
A new surnames constants set (same SetManager machinery as titles/prefixes), shipping curated Chinese and Korean surname lists.
A pre-processing step (opt-in, like all localization) that longest-matches a surname off the front of an unspaced string and inserts the space — compound/two-syllable surnames matched before single ones (欧阳明 must split 欧阳+明, not 欧+阳明, even though 欧 is itself a surname).
Opt-in is required for a second reason here: a pure-Han string cannot indicate whether it's Chinese or Japanese (林 is Lin or Hayashi), so the caller must select the language. Japanese itself is explicitly out of scope — ~138,500 surnames with genuinely ambiguous split boundaries — and is covered by #272.
Depends on the name_order order-spec (#270). Would resolve the unspaced cases behind #83 and (with ('last','middle','first')) #146-style Vietnamese input; supersedes the analysis in #34.
Merged in #294, shipping in 2.1.0: native-script name order (Han and Hangul, family-first) and unspaced-hangul segmentation are on by default; unspaced Chinese segmentation is opt-in via the zh locale pack. Romanized input is unchanged by design — family-first romanized data remains explicit config (name_order=FAMILY_FIRST); see the migration notes in docs/migrate.rst for the v1-facade behavior changes. Japanese segmentation continues in #272.
Activity
Simplify and speed up the #271 hot paths
Sort differential rules most-specific-first, dropping the ordering rule
Merge pull request #294 from derek73/v2.1/cjk-defaults
derek73 commentedon Jul 29, 2026
Merged in #294, shipping in 2.1.0: native-script name order (Han and Hangul, family-first) and unspaced-hangul segmentation are on by default; unspaced Chinese segmentation is opt-in via the zh locale pack. Romanized input is unchanged by design — family-first romanized data remains explicit config (name_order=FAMILY_FIRST); see the migration notes in docs/migrate.rst for the v1-facade behavior changes. Japanese segmentation continues in #272.
Add the kana Script members and the effective_script kana license (#272)
Classify the ideographic iteration mark as Han (#272)
Tighten the compound rule to names the order flip can reach
Reclassify the FAMILY-comma honorific row to comma-family (#308)
Pin the 殿 exclusion, the one nothing held (#308)
Put a non-ASCII shape in the scaling benchmark (#308)
14 remaining items
Document script-scoped CJK defaults and the ZH pack (derek73#271)
Classify the derek73#271 CJK fixes in the differential harness
Correct the migration doc's CJK baseline claims (derek73#271)
Fix the fork example and document the hangul render change (derek73#271)
Move _SCRIPT_RANGES to _policy, beside Script
Give the differential harness a CJK corpus generated from the case table
Polish: the CJK rule slug owns derek73#298; generator docstring tense
Retire the canonical-CJK-rule selector, keep its table guards
Label the CJK honorific rows for the issues they depend on (derek73#324)
fix(derek73#322): the lookup fold reads every full stop and composes NFC
tooling(differential): re-measure four exemption arguments that were …
docs(design): the contest check covers nesting, not every overlap
docs(design)+tests: W2 keeps its normative half; the period is not ig…
tooling(differential): the six 2.x contests get their winners
docs(design): the derek73#501 adjudication