diff --git a/.claude/agents/design-docs-reviewer.md b/.claude/agents/design-docs-reviewer.md index 2f270f26..ce3c4a13 100644 --- a/.claude/agents/design-docs-reviewer.md +++ b/.claude/agents/design-docs-reviewer.md @@ -1,6 +1,6 @@ --- name: design-docs-reviewer -description: Reviews changes to docs/design/ (rules.md, decisions.md, mechanisms.md, and AGENTS.md itself, which carries the conventions the other three rest on) for the failure modes that this doc system's own tests cannot catch — unreproducible measurements, rules that contradict the decision entries they rest on, statements that contradict their own examples, and prose edits that change what the doc parsers see. Use before merging any PR that touches docs/design/, and after distilling a design session into those files. +description: Reviews changes to docs/design/ (rules/, decisions/, mechanisms.md, and AGENTS.md itself, which carries the conventions the other three rest on) for the failure modes that this doc system's own tests cannot catch — unreproducible measurements, rules that contradict the decision entries they rest on, statements that contradict their own examples, and prose edits that change what the doc parsers see. Use before merging any PR that touches docs/design/, and after distilling a design session into those files. tools: Bash, Read, Grep, Glob --- @@ -26,7 +26,7 @@ It is the only copy on purpose. This agent used to restate the axes and drifted three ways against it — a missing axis, a narrowed one, and a corpus count whose correction was recorded there and never reached the copy here (#473, -decisions.md#review-agent-single-source). Do not reintroduce a +decisions/review-agent-single-source). Do not reintroduce a summary. ## How to run diff --git a/AGENTS.md b/AGENTS.md index 44d301ea..5992b8ff 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,17 +18,19 @@ After a review round, **review the fix commit too** — second-round passes on # ## Rules documentation (docs/design/) -Three committed contributor docs carry the parser's normative rules and their reasons, enforced by the doc tests in tests/v2/ (rules_doc.py and the test_doc_*/test_rules_doc* modules). Before proposing a design or a fix that touches parser behavior, read docs/design/mechanisms.md and the relevant docs/design/rules.md sections — the catalog is keyed by problem shape, and the pattern you are about to invent is often already there. +Three committed contributor docs carry the parser's normative rules and their reasons, enforced by the doc tests in tests/v2/ (design_docs.py, rules_doc.py and the test_doc_*/test_rules_doc*/test_design_docs modules). Before proposing a design or a fix that touches parser behavior, read docs/design/mechanisms.md and the relevant docs/design/rules/ sections — the catalog is keyed by problem shape, and the pattern you are about to invent is often already there. -- **rules.md** — NORMATIVE rules (intended behavior, domain-topic sections, executable examples; `deviates:` markers track known parser gaps). Cite as `rules.md#P1`; code comments quote a verbatim excerpt in double quotes, checked by tests/v2/test_doc_citations.py. -- **decisions.md** — the ADR-style record: dated entries, Declined: (rejected WITH evidence), Excluded: (what must stay out of a wordlist and why), Open: (issue links, never restated; also keyed to a vocabulary set for contested memberships — the right-rule-wrong-set class), 3-0-reevaluations (append whenever a design cites 1.4 parity as load-bearing). +- **rules/** — NORMATIVE rules (intended behavior, domain-topic sections, executable examples; `deviates:` markers track known parser gaps), one file per rule and per section Background. Cite as `rules/P1`; code comments quote a verbatim excerpt in double quotes, checked by tests/v2/test_doc_citations.py. +- **decisions/** — the ADR-style record, one file per key: dated entries, Declined: (rejected WITH evidence), Excluded: (what must stay out of a wordlist and why), Open: (issue links, never restated; also keyed to a vocabulary set for contested memberships — the right-rule-wrong-set class), 3-0-reevaluations (append whenever a design cites 1.4 parity as load-bearing). - **mechanisms.md** — problem-shape catalog with citable contract statements; stage-attribution claims in comments must cite an entry verbatim, never restate it in fresh words. -**Same-PR amendment rule.** Any PR that changes or clarifies parser behavior — or the boundaries around a documented rule, since a neighboring change can invalidate a rule's stated limits without touching its code — amends rules.md in the same diff (the doc diff is part of the reviewable change, like tests). A change that resolves or reverses a design question adds a decisions.md entry; a new reusable pattern adds a mechanisms.md entry; a fixed deviation removes its `deviates:` marker in the same PR (the examples test forces this). Issues proposing behavior changes should be drafted in rule shape — rationale, statement, examples with boundaries, accepted consequences, open questions, exclusions — so landing an accepted proposal is a copy, not a rewrite. +**Each rule and decisions entry is its own named file** (`rules/S2-case-signal-at-the-suffix-slot.md`, `decisions/vocabulary-collisions.md`), and each folder's README.md indexes them; a new rule or a new decisions key is a new file plus its index line, which `tests/v2/test_design_docs.py` holds to the file's heading and name, and tests/v2/design_docs.py is the one module that knows the layout (#713). Citations carry the ID only, never the name, so a file can be renamed freely. **When you cite a rule or decision in chat, a commit or a PR, give its name with its ID** — "S2 (the case signal at the suffix slot)" — since the ID alone tells the reader nothing about which rule it is. + +**Same-PR amendment rule.** Any PR that changes or clarifies parser behavior — or the boundaries around a documented rule, since a neighboring change can invalidate a rule's stated limits without touching its code — amends the rules in the same diff (the doc diff is part of the reviewable change, like tests). A change that resolves or reverses a design question adds a decisions entry; a new reusable pattern adds a mechanisms.md entry; a fixed deviation removes its `deviates:` marker in the same PR (the examples test forces this). Issues proposing behavior changes should be drafted in rule shape — rationale, statement, examples with boundaries, accepted consequences, open questions, exclusions — so landing an accepted proposal is a copy, not a rewrite. **Counting claims.** A bare count in prose is either an assertion or a liability, keyed by who observes its staleness: asserted counts (a test holds the number) fail CI at change time — the useful kind; dated snapshots ("51 sites at spec time") cannot go stale; standing present-tense prose counts are the forbidden class — promote to an assertion, add a date, or state the invariant and let a test count. After changing how many times something runs, sweep for counts, not for the thing's name. -**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the snapshot's diff since the last release, be verified against rules.md examples, or -- for a view the snapshot cannot see -- carry a recompute recipe stored with the design entry the bullet cites; never write one from memory. `uv run python tools/differential/snapshot.py --since v` prints every corpus name the release moved, grouped by changed field; v2.4.1 is the first tag carrying a snapshot, and for an older release `compare.py --baseline ` prints the same report against the PyPI wheel, closing with the entries compared and how many moved names are Latin-only. Both say how many moved names only `corpus_cldr.jsonl` holds. The snapshot records the seven role fields, ambiguity kinds and `initials()` for every name, so an initials count is read off it directly. `corpus_cldr.jsonl`'s fictional renderings are in it: count them out of a release figure (both tools print how many moved names only that file holds), or say the figure includes them. `capitalized()` and any other render view stay invisible to the snapshot (decisions.md#R4, #R3), and for those the first two sources cannot reach a claim: the snapshot is byte-identical across the change, and an example line witnesses an output without counting anything. A recipe names the corpus files, the policy sweep, and -- the part that is easy to omit and fatal -- THE COMPARATOR, which must be something the shipped tree is not: #408's first recipe said to compare `initials()` against a folded-first partition, which is what `initials()` now IS, so it reproduced 0 where the bullet claimed 660 and was the only stated provenance for the number. Run the recipe as written before shipping the bullet. Cross-version numbers (a released wheel, the pre-change tree) are dated snapshots under Counting claims, since nothing in the repository re-runs them. The same standard binds decisions entries and PR bodies: a claim that a change restores the released reading is measured over the whole moved population against the wheel, never spot-checked (#625 claimed its 774 moves landed on the 1.4.0 and 2.3.0 readings from three names; measured, the tree matched 2.3.0 on 0 of 258 moved texts, master on 152), and a population size is re-counted from its recipe, never copied from another entry (the same entry carried #624's 137,810 for a grid of 395,148). Code comments citing rule IDs follow the excerpt discipline, checked by tests/v2/test_doc_citations.py: a rule's statement and its `Accepted:` clauses are quotable, its example lines are not; `rules.md#H` names the H section, whose Background is quotable; a decisions.md citation may paraphrase, but what it quotes must be verbatim; and a reference with no quote must still name something that exists. The check's reach is a quote within four words of the ID or chained to one that is: a quote further off reads exactly like a name written in double quotes, so it is NOT checked, and the second review of #632 found 14 such quotes stale by hand. Keep a quote beside its ID. Quotes of AGENTS.md itself are not checked at all. +**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the snapshot's diff since the last release, be verified against rules examples, or -- for a view the snapshot cannot see -- carry a recompute recipe stored with the design entry the bullet cites; never write one from memory. `uv run python tools/differential/snapshot.py --since v` prints every corpus name the release moved, grouped by changed field; v2.4.1 is the first tag carrying a snapshot, and for an older release `compare.py --baseline ` prints the same report against the PyPI wheel, closing with the entries compared and how many moved names are Latin-only. Both say how many moved names only `corpus_cldr.jsonl` holds. The snapshot records the seven role fields, ambiguity kinds and `initials()` for every name, so an initials count is read off it directly. `corpus_cldr.jsonl`'s fictional renderings are in it: count them out of a release figure (both tools print how many moved names only that file holds), or say the figure includes them. `capitalized()` and any other render view stay invisible to the snapshot (decisions/R4, #R3), and for those the first two sources cannot reach a claim: the snapshot is byte-identical across the change, and an example line witnesses an output without counting anything. A recipe names the corpus files, the policy sweep, and -- the part that is easy to omit and fatal -- THE COMPARATOR, which must be something the shipped tree is not: #408's first recipe said to compare `initials()` against a folded-first partition, which is what `initials()` now IS, so it reproduced 0 where the bullet claimed 660 and was the only stated provenance for the number. Run the recipe as written before shipping the bullet. Cross-version numbers (a released wheel, the pre-change tree) are dated snapshots under Counting claims, since nothing in the repository re-runs them. The same standard binds decisions entries and PR bodies: a claim that a change restores the released reading is measured over the whole moved population against the wheel, never spot-checked (#625 claimed its 774 moves landed on the 1.4.0 and 2.3.0 readings from three names; measured, the tree matched 2.3.0 on 0 of 258 moved texts, master on 152), and a population size is re-counted from its recipe, never copied from another entry (the same entry carried #624's 137,810 for a grid of 395,148). Code comments citing rule IDs follow the excerpt discipline, checked by tests/v2/test_doc_citations.py: a rule's statement and its `Accepted:` clauses are quotable, its example lines are not; `rules/H` names the H section, whose Background is quotable; a decisions citation may paraphrase, but what it quotes must be verbatim; and a reference with no quote must still name something that exists. The check's reach is a quote within four words of the ID or chained to one that is: a quote further off reads exactly like a name written in double quotes, so it is NOT checked, and the second review of #632 found 14 such quotes stale by hand. Keep a quote beside its ID. Quotes of AGENTS.md itself are not checked at all. **Writing the user docs (docs/*.rst) has its own AGENTS.md too.** `docs/AGENTS.md` carries the style the docs are written in — tables as indexes, subheadings per task, measured claims, recipe doctests — distilled from PR #588. It loads the same way as the docs/design/ one below; if your tool does not do nested discovery, read it before editing a `.rst` file under docs/. @@ -36,11 +38,11 @@ Three committed contributor docs carry the parser's normative rules and their re **Triaging a design backlog: shape before value.** When a pile of open questions has to be ordered, relatedness will not partition it — in a parser nearly everything touches particles or suffixes. The line that does is whether a decision changes WHAT THE MODEL CAN EXPRESS or fills in a value in a shape already fixed. Settle the shape questions first, as a batch: they are few, they are usually independent of each other (so their own ordering does not matter), and each one collapses or re-frames a run of the value questions below it. The rest are leaves needing one measurement and one answer apiece, in any order. Worked example, 2026-08-16 (#386): 30 open issues and ~13 open design questions reduced to three shape decisions, which decided or re-framed fourteen of them — and two of the three turned out smaller than their issues claimed, because the issues had gone stale. Check the age of an issue's premises before believing its cost estimate. -**Markdown line width: don't hard-wrap prose.** Measured 2026-08-16 — wrapped and unwrapped render identically, docs/design/ is not in the Sphinx build, and unwrapping is byte-identical (newline→space is 1:1), so the width buys nothing and costs diff churn (74% of #386's decisions.md deletions were reflow, not change). AGENTS.md and docs/design/ were unwrapped in one whitespace-only commit on 2026-08-16; keep new prose unwrapped rather than reformatting again. The blame cost was 43 lines of 1717, measured, not the wholesale reset predicted before doing it — git's diff maps most joined lines back to whichever commit wrote their opening text. `.git-blame-ignore-revs` lists the commit and is worth enabling (`git config blame.ignoreRevsFile .git-blame-ignore-revs`) for future format-only commits, but it recovered none of those 43: `--ignore-revs-file` cannot pass blame through a commit that MERGES lines, because the joined line has no counterpart in the parent to re-attribute to. Exceptions, both enforced: rules.md's example and pointer lines are parsed per line, so a wrapped example is a hard error naming its rule; and rules.md's rule BODIES stay wrapped because a rule block is a record format, aligning 4-space statement prose above 6-space examples, not a paragraph. +**Markdown line width: don't hard-wrap prose.** Measured 2026-08-16 — wrapped and unwrapped render identically, docs/design/ is not in the Sphinx build, and unwrapping is byte-identical (newline→space is 1:1), so the width buys nothing and costs diff churn (74% of #386's decisions.md deletions were reflow, not change). AGENTS.md and docs/design/ were unwrapped in one whitespace-only commit on 2026-08-16; keep new prose unwrapped rather than reformatting again. The blame cost was 43 lines of 1717, measured, not the wholesale reset predicted before doing it — git's diff maps most joined lines back to whichever commit wrote their opening text. `.git-blame-ignore-revs` lists the commit and is worth enabling (`git config blame.ignoreRevsFile .git-blame-ignore-revs`) for future format-only commits, but it recovered none of those 43: `--ignore-revs-file` cannot pass blame through a commit that MERGES lines, because the joined line has no counterpart in the parent to re-attribute to. Exceptions, both enforced: a rule's example and pointer lines are parsed per line, so a wrapped example is a hard error naming its rule; and rule BODIES stay wrapped because a rule block is a record format, aligning 4-space statement prose above 6-space examples, not a paragraph. **Guard tests** SHOULD carry a recorded negative control — the answer with the guard off, stored as data (the `_UNGUARDED_EFFECT` shape in tests/v2/pipeline/test_state.py; see mechanisms.md's Verification shapes). Re-take it after the last commit that adds tests, before merge: a review round that adds rows moves the count, and three recorded controls in a row went stale that way (#625 27 → 30, #636 4 → 9, #637 5 → 7 → 11), each caught only by the next review. -**Invariant tests over a generated grid share ONE parsed grid, and a grid pairs a text only with configurations that can read it differently.** Both rules are in `tests/v2/test_properties.py` (`_connective_findings`, `_off_switch_findings`, `_rows`) and both were bought at a price: four tests each walking the same 170,100-row grid, and a grid that was a full cross product of texts × lexicons × policies, together took that module from 11s to 115s and CI's build jobs from ~5 minutes to 17–25 (under coverage every parse costs several times more, and every later PR pays it). A new invariant over an existing grid joins that grid's walk instead of opening its own; a new grid states its measured runtime where it is built. Trim a grid by SHAPE — a variant lexicon reaches only the texts holding the word it adds — never by sampling, and prove the dropped rows were duplicates by signature before dropping them. Mutation counts in the tests' docstrings are counts over the grid, so re-measure them whenever the grid moves -- and so do the DESIGN DOCS that quote them. A recorded control reaches `docs/design/decisions.md` and `docs/design/mechanisms.md` as well as the docstring, nothing checks those copies, and a grid change silently makes them wrong about a number they present as measured: the #397 second review moved four such figures at once (2026-09-20). Move them in the same commit, and where the older number is worth keeping, label it with the population it was measured over rather than deleting it. +**Invariant tests over a generated grid share ONE parsed grid, and a grid pairs a text only with configurations that can read it differently.** Both rules are in `tests/v2/test_properties.py` (`_connective_findings`, `_off_switch_findings`, `_rows`) and both were bought at a price: four tests each walking the same 170,100-row grid, and a grid that was a full cross product of texts × lexicons × policies, together took that module from 11s to 115s and CI's build jobs from ~5 minutes to 17–25 (under coverage every parse costs several times more, and every later PR pays it). A new invariant over an existing grid joins that grid's walk instead of opening its own; a new grid states its measured runtime where it is built. Trim a grid by SHAPE — a variant lexicon reaches only the texts holding the word it adds — never by sampling, and prove the dropped rows were duplicates by signature before dropping them. Mutation counts in the tests' docstrings are counts over the grid, so re-measure them whenever the grid moves -- and so do the DESIGN DOCS that quote them. A recorded control reaches `docs/design/decisions/` and `docs/design/mechanisms.md` as well as the docstring, nothing checks those copies, and a grid change silently makes them wrong about a number they present as measured: the #397 second review moved four such figures at once (2026-09-20). Move them in the same commit, and where the older number is worth keeping, label it with the population it was measured over rather than deleting it. ## Commands @@ -79,7 +81,7 @@ uv run sphinx-build -b html docs dist/docs # sentence only where a caller would trip on it, then the issue link(s): # :issue:`123` or :issue:`123`, :issue:`124` (sphinx.ext.extlinks; a PR # number works too). The mechanism, boundary cases and measurements live -# in the issue and docs/design/decisions.md, never in the bullet -- +# in the issue and docs/design/decisions/, never in the bullet -- # 2.4.0's entry was cut from 10,765 words to 1,491 on 2026-10-11 # - Merge bullets a user would see as one change, with several links # - Version is decided at release time (patch/minor/major per semver) @@ -107,7 +109,7 @@ uv run sphinx-build -b html docs dist/docs # "Reads:". That is checkable, so check it rather than reading it: # compare it against grep -oE '\b(policy|lexicon)\.[a-z_]+' on the module # - tests/v2/cases.py notes, which explain why a row lands where it does -# - docs/design/rules.md, decisions.md and mechanisms.md -- READ, don't +# - docs/design/rules/, decisions/ and mechanisms.md -- READ, don't # grep: the excerpt and example tests catch citation and example drift, # but statement and Background prose can still be wrong about behavior # that changed @@ -165,15 +167,15 @@ logging.getLogger('HumanName').setLevel(logging.DEBUG) The library has two layers: `nameparser/config/` (data) and `nameparser/parser.py` (logic). -**Design philosophy — positional and language-agnostic.** The parser assigns parts by *position* plus small sets of words that join to neighbors; it never detects language. A name's language can't be reliably inferred from Latin-script transliteration ("Ali" is Arabic or Italian; "Van"/"Della"/"Bin" are first names in some cultures, particles in others), so language-specific rules belong in opt-in `Constants` config, never global defaults. Many "wrong for language X" reports (#133, #150, #130, #85, #103, #146, #83) are irreducible ambiguities — e.g. `de Mesnil` (want last name) vs `Van Johnson` (want first name) are the same `[prefix][word]` shape. Before adding a rule, confirm it doesn't break the opposite case (run the full suite — Portuguese and "Van Johnson" tests are the usual canaries). **The one scoped exception (2.1, #271/#272): script-conditional behavior is permitted exactly where the SCRIPT ITSELF — not statistics about it — determines the convention.** The never-detect-language rule above is about Latin *transliteration*, where the signal genuinely is destroyed; native script is a different question, and it is answered per behavior rather than per script. Five defaults fall out of it, plus a sixth that applies the same not-a-guess standard to specific WORDS rather than to a script (#308's honorific peel, below). Wholly-Han, wholly-Hangul and kana-licensed names read family-first (`Policy.script_orders`) — no language detection needed, because zh and ja both write family-first in native script, so order cannot be misread even though the language is unknowable. Unspaced hangul splits into surname + given name (`config/surnames.py` ships the Korean census list as DEFAULT vocabulary) — nothing but Korean is written in hangul and the surnames are a closed census set, and the vocabulary is self-selecting besides: a hangul entry can only ever match hangul text. Hiragana licenses Japanese (#272) — a name whose characters stay inside Han∪kana while carrying at least one kana cannot be Chinese (the kana rules it out) and is not a transcription (foreign names are transcribed in katakana ALONE, マイケル has no kanji), so 高橋みなみ and 山田 エミ read family-first too; mechanically they resolve to the HIRAGANA entry, the license's carrier key. PURE katakana — full-width or halfwidth, the halfwidth block U+FF65–FF9F being katakana in the script table since #594 — is excluded and keeps the positional default: マイケル・ジャクソン is a transcribed foreign name in its source order, and a Japanese reading written in katakana (legacy halfwidth data spells Japanese names that way too) cannot be told from one, so the script settles nothing and a caller who knows opts in through `script_orders`. And the 间隔号 U+00B7 (#298) is the transcription marker for scripts that HAVE no transcription script: a name it divides (威廉·莎士比亚 — flanked by classified characters on both sides, so Catalan's Gal·la is untouched) keeps the declared order and never segments — the orthography names the convention, with the divider carrying the signal that stands the script's family-first reading down, so the default reads it in its written order and a declared `FAMILY_FIRST` reverses it like any other name; pure katakana reaches the same reading by the opposite route, its script settling nothing so the declared order stands. And a listed CJK honorific glued to the END of a name token is split off it (#308) — 田中さん is 田中 plus さん — on the same orthography-settles-it test, narrowed for the glued position: an entry peels only where it can never end a name, so 씨/님/さん/様/先生 peel while 양/군/氏/博士/殿 stay spaced-only (김지양 and 田中博士 are names, and ~90 Japanese surnames end in 殿) and 君 is in NEITHER set (王君 is a complete Chinese name), though its kana spelling くん peels. Like the nakaguro's tokenize-level separation described next, it is reached by neither policy opt-out — but for its own reason: the vocabulary carries the license itself rather than borrowing the script's, so `segment_scripts` has nothing to say about it. Since #312 it also crosses the 间隔号, which still stops the surname split standing right beside it: it answers where a name DIVIDES into surname and given, and the peel never asks that question. Whether it also crosses the FAMILY comma is tolerated rather than settled: the 2026-09-01 demotion (rules.md#W3) narrowed that half from contract to best-effort, since no CJK writing system's own convention puts a comma between family and given at all — so `김, 민준씨` reads today exactly as the spaced `김 민준씨` does (family 김, given 민준, suffix 씨) while the split stands down as before, but that reading is watched in the snapshot rather than pinned as contract. Its site is accordingly the name-bearing segment runs — `segments[:2]` under a family comma, and `segments[0]` as before otherwise, the family comma being the one structure that splits the name itself across two runs, with the honorific as often glued to the given side as to the family. That is the whole reach and nothing past it (`김, 민준 지훈씨` peels; `김, 민준, 지훈씨` and `김,, 민준씨` do not, both landing in a third run), and whether `segments[1]` is name text at all is now ASKED rather than inferred from the structure — `segment` does not guarantee it, since a one-word part before the comma reads as FAMILY_COMMA even when the part after it is entirely suffix-shaped, and the peel walking into such a run took `V.` for its site, found no listed tail and abandoned (#319). The question is asked in two parts. First the comma's own decision: `_comma.decide` on a classified copy of the name as written, since #613's /simplify -- a hand copy of it here drifted from decide four times in that PR's review (the ambiguous class's count, the numeral refusal, a suffix before the comma, the split `Ph. D.`), each drift dividing a name decide read as a family comma or leaving whole one it read as a suffix comma, and asking the real decision is what ended it. Where that leaves the family comma, C1's suffix-word predicate, lifted into `_vocab.is_wholly_suffix` so the stages cannot drift: a wholly suffix-shaped second run is declined and the scan stays in `segments[0]`, so `田中さん, V.` and `田中さん, Ph. D.` give さん up as `田中さん, PhD` always did. The test is necessary but NOT sufficient, and the second condition is not decoration: every honorific tail is also a suffix word, so a glued honorific is itself part of what makes its run read as suffix-shaped, and declining a run that holds the ONLY site loses the peel outright. `segments[0]` must therefore offer a peel site of its own before the second run is declined — `이, J.씨` and `선생님, J.씨` pass the suffix test and are scanned anyway, keeping the pre-#319 reading, while `김민준씨, J.씨` has a site on both sides and peels the person's own 씨 rather than the junk one behind the comma. Uniform in the PEEL, that is — where the credential itself lands is `assign`'s question and still differs by spelling (`V.` → `given`, `PhD` and `Ph. D.` → `suffix`). Not `_is_post_nominal` pluralized: the run predicate says yes both to what the token predicate vetoes (`V.`, `V`, `I` — the class the defect was reported as) and to what the token predicate never sees at all, since `period_joined_vocab` and the delimiter routes are the run predicate's alone (`Msc.Ed.` and `J.씨` reach it that way, and `田中さん, Msc.Ed.` moves with the rest). `Policy(lenient_comma_suffixes=False)` drops this call to the strict token test too — so those three read as name text again and keep the pre-#319 answer, while `Ph. D.` peels under the knob regardless, its merged `phd` passing the strict test. `田中さん, 太郎` is unchanged, and not because of its comma — the honorific there is not at the end of the name, 太郎 is. The nakaguro belongs to the same doctrine but is decided a layer down: U+30FB and its halfwidth twin U+FF65 separate tokens like whitespace, unconditionally and in tokenize, so neither policy opt-out (`script_orders={}`, `segment_scripts=()`) reaches it — the codepoints are CJK-only and appear in no other script's names, which is what licenses a tokenize-level rule where U+00B7 (also the Catalan punt volat, interior to Gal·la) needs the flanked-by-classified-script guard `_tokenize_region` gives it (#298). Han segmentation stays OPT-IN (`locales.ZH` for Chinese, `locales.JA` for Japanese) — a zh surname list corrupts Japanese kanji names, since 高 is a common Chinese surname and 高橋一郎 would split 高+橋一郎 where the correct reading is 高橋+一郎; no surname list divides a kanji name at all, so `locales.JA` activates the stage and a pluggable `Parser(segmenter=...)` does the dividing. Latin-script input is never touched by any of this: "Kim Min-jun" is genuinely order-ambiguous and stays governed by `name_order` and opt-in packs. Before adding a script-conditional rule, work out which of the three it is — certain, certain for this one behavior only, or a statistical guess wearing a script's clothes. +**Design philosophy — positional and language-agnostic.** The parser assigns parts by *position* plus small sets of words that join to neighbors; it never detects language. A name's language can't be reliably inferred from Latin-script transliteration ("Ali" is Arabic or Italian; "Van"/"Della"/"Bin" are first names in some cultures, particles in others), so language-specific rules belong in opt-in `Constants` config, never global defaults. Many "wrong for language X" reports (#133, #150, #130, #85, #103, #146, #83) are irreducible ambiguities — e.g. `de Mesnil` (want last name) vs `Van Johnson` (want first name) are the same `[prefix][word]` shape. Before adding a rule, confirm it doesn't break the opposite case (run the full suite — Portuguese and "Van Johnson" tests are the usual canaries). **The one scoped exception (2.1, #271/#272): script-conditional behavior is permitted exactly where the SCRIPT ITSELF — not statistics about it — determines the convention.** The never-detect-language rule above is about Latin *transliteration*, where the signal genuinely is destroyed; native script is a different question, and it is answered per behavior rather than per script. Five defaults fall out of it, plus a sixth that applies the same not-a-guess standard to specific WORDS rather than to a script (#308's honorific peel, below). Wholly-Han, wholly-Hangul and kana-licensed names read family-first (`Policy.script_orders`) — no language detection needed, because zh and ja both write family-first in native script, so order cannot be misread even though the language is unknowable. Unspaced hangul splits into surname + given name (`config/surnames.py` ships the Korean census list as DEFAULT vocabulary) — nothing but Korean is written in hangul and the surnames are a closed census set, and the vocabulary is self-selecting besides: a hangul entry can only ever match hangul text. Hiragana licenses Japanese (#272) — a name whose characters stay inside Han∪kana while carrying at least one kana cannot be Chinese (the kana rules it out) and is not a transcription (foreign names are transcribed in katakana ALONE, マイケル has no kanji), so 高橋みなみ and 山田 エミ read family-first too; mechanically they resolve to the HIRAGANA entry, the license's carrier key. PURE katakana — full-width or halfwidth, the halfwidth block U+FF65–FF9F being katakana in the script table since #594 — is excluded and keeps the positional default: マイケル・ジャクソン is a transcribed foreign name in its source order, and a Japanese reading written in katakana (legacy halfwidth data spells Japanese names that way too) cannot be told from one, so the script settles nothing and a caller who knows opts in through `script_orders`. And the 间隔号 U+00B7 (#298) is the transcription marker for scripts that HAVE no transcription script: a name it divides (威廉·莎士比亚 — flanked by classified characters on both sides, so Catalan's Gal·la is untouched) keeps the declared order and never segments — the orthography names the convention, with the divider carrying the signal that stands the script's family-first reading down, so the default reads it in its written order and a declared `FAMILY_FIRST` reverses it like any other name; pure katakana reaches the same reading by the opposite route, its script settling nothing so the declared order stands. And a listed CJK honorific glued to the END of a name token is split off it (#308) — 田中さん is 田中 plus さん — on the same orthography-settles-it test, narrowed for the glued position: an entry peels only where it can never end a name, so 씨/님/さん/様/先生 peel while 양/군/氏/博士/殿 stay spaced-only (김지양 and 田中博士 are names, and ~90 Japanese surnames end in 殿) and 君 is in NEITHER set (王君 is a complete Chinese name), though its kana spelling くん peels. Like the nakaguro's tokenize-level separation described next, it is reached by neither policy opt-out — but for its own reason: the vocabulary carries the license itself rather than borrowing the script's, so `segment_scripts` has nothing to say about it. Since #312 it also crosses the 间隔号, which still stops the surname split standing right beside it: it answers where a name DIVIDES into surname and given, and the peel never asks that question. Whether it also crosses the FAMILY comma is tolerated rather than settled: the 2026-09-01 demotion (rules/W3) narrowed that half from contract to best-effort, since no CJK writing system's own convention puts a comma between family and given at all — so `김, 민준씨` reads today exactly as the spaced `김 민준씨` does (family 김, given 민준, suffix 씨) while the split stands down as before, but that reading is watched in the snapshot rather than pinned as contract. Its site is accordingly the name-bearing segment runs — `segments[:2]` under a family comma, and `segments[0]` as before otherwise, the family comma being the one structure that splits the name itself across two runs, with the honorific as often glued to the given side as to the family. That is the whole reach and nothing past it (`김, 민준 지훈씨` peels; `김, 민준, 지훈씨` and `김,, 민준씨` do not, both landing in a third run), and whether `segments[1]` is name text at all is now ASKED rather than inferred from the structure — `segment` does not guarantee it, since a one-word part before the comma reads as FAMILY_COMMA even when the part after it is entirely suffix-shaped, and the peel walking into such a run took `V.` for its site, found no listed tail and abandoned (#319). The question is asked in two parts. First the comma's own decision: `_comma.decide` on a classified copy of the name as written, since #613's /simplify -- a hand copy of it here drifted from decide four times in that PR's review (the ambiguous class's count, the numeral refusal, a suffix before the comma, the split `Ph. D.`), each drift dividing a name decide read as a family comma or leaving whole one it read as a suffix comma, and asking the real decision is what ended it. Where that leaves the family comma, C1's suffix-word predicate, lifted into `_vocab.is_wholly_suffix` so the stages cannot drift: a wholly suffix-shaped second run is declined and the scan stays in `segments[0]`, so `田中さん, V.` and `田中さん, Ph. D.` give さん up as `田中さん, PhD` always did. The test is necessary but NOT sufficient, and the second condition is not decoration: every honorific tail is also a suffix word, so a glued honorific is itself part of what makes its run read as suffix-shaped, and declining a run that holds the ONLY site loses the peel outright. `segments[0]` must therefore offer a peel site of its own before the second run is declined — `이, J.씨` and `선생님, J.씨` pass the suffix test and are scanned anyway, keeping the pre-#319 reading, while `김민준씨, J.씨` has a site on both sides and peels the person's own 씨 rather than the junk one behind the comma. Uniform in the PEEL, that is — where the credential itself lands is `assign`'s question and still differs by spelling (`V.` → `given`, `PhD` and `Ph. D.` → `suffix`). Not `_is_post_nominal` pluralized: the run predicate says yes both to what the token predicate vetoes (`V.`, `V`, `I` — the class the defect was reported as) and to what the token predicate never sees at all, since `period_joined_vocab` and the delimiter routes are the run predicate's alone (`Msc.Ed.` and `J.씨` reach it that way, and `田中さん, Msc.Ed.` moves with the rest). `Policy(lenient_comma_suffixes=False)` drops this call to the strict token test too — so those three read as name text again and keep the pre-#319 answer, while `Ph. D.` peels under the knob regardless, its merged `phd` passing the strict test. `田中さん, 太郎` is unchanged, and not because of its comma — the honorific there is not at the end of the name, 太郎 is. The nakaguro belongs to the same doctrine but is decided a layer down: U+30FB and its halfwidth twin U+FF65 separate tokens like whitespace, unconditionally and in tokenize, so neither policy opt-out (`script_orders={}`, `segment_scripts=()`) reaches it — the codepoints are CJK-only and appear in no other script's names, which is what licenses a tokenize-level rule where U+00B7 (also the Catalan punt volat, interior to Gal·la) needs the flanked-by-classified-script guard `_tokenize_region` gives it (#298). Han segmentation stays OPT-IN (`locales.ZH` for Chinese, `locales.JA` for Japanese) — a zh surname list corrupts Japanese kanji names, since 高 is a common Chinese surname and 高橋一郎 would split 高+橋一郎 where the correct reading is 高橋+一郎; no surname list divides a kanji name at all, so `locales.JA` activates the stage and a pluggable `Parser(segmenter=...)` does the dividing. Latin-script input is never touched by any of this: "Kim Min-jun" is genuinely order-ambiguous and stays governed by `name_order` and opt-in packs. Before adding a script-conditional rule, work out which of the three it is — certain, certain for this one behavior only, or a statistical guess wearing a script's clothes. -**A constant's membership is a question you may reopen.** Proposing that a word be ADDED, REMOVED or MOVED between vocabulary sets is ordinary design work — a shipped entry is not evidence that anyone judged it. `SUFFIX_ACRONYMS` arrived in `af5bdab` as a bulk Wikipedia import never reviewed against surname collisions: 572 of its 577 alphabetic entries leave `family` empty in `"John "` against five ambiguous-gated exceptions (recomputed 2026-09-07; the fifth is `ba`; 568 of 575 against seven, measured 2026-09-25 after #540), and `sa`, `se` and `om` are borne as surnames (measured 2026-08-23), as were `rai`, `cha`, `ba` and `mc` before `decisions.md#suffix-acronym-collisions` decided all four — `rai` and `cha` removed, `ba` marked ambiguous, `mc` left alone as no borne name at all. The same entry marked `meng` and `lac` ambiguous on 2026-09-25 (#540). When a fix starts to look like new machinery, check the vocabulary first. Criterion: `decisions.md#vocabulary-collisions`, with #360's positional qualifier. +**A constant's membership is a question you may reopen.** Proposing that a word be ADDED, REMOVED or MOVED between vocabulary sets is ordinary design work — a shipped entry is not evidence that anyone judged it. `SUFFIX_ACRONYMS` arrived in `af5bdab` as a bulk Wikipedia import never reviewed against surname collisions: 572 of its 577 alphabetic entries leave `family` empty in `"John "` against five ambiguous-gated exceptions (recomputed 2026-09-07; the fifth is `ba`; 568 of 575 against seven, measured 2026-09-25 after #540), and `sa`, `se` and `om` are borne as surnames (measured 2026-08-23), as were `rai`, `cha`, `ba` and `mc` before `decisions/suffix-acronym-collisions` decided all four — `rai` and `cha` removed, `ba` marked ambiguous, `mc` left alone as no borne name at all. The same entry marked `meng` and `lac` ambiguous on 2026-09-25 (#540). When a fix starts to look like new machinery, check the vocabulary first. Criterion: `decisions/vocabulary-collisions`, with #360's positional qualifier. **Sweep the forms a change can reach: comma shapes, then `name_order`.** No comma, a FULL name before the comma, a ONE-WORD name before the comma and NOTHING before the comma are four paths, and the last two are the misses — #429, #430 and #432 are all the one-word path disagreeing with the full-name path on inputs the full-name path reads correctly, and #613's PR review found `, PhD` and `, Jr.` (an empty surname field, as CSV data writes it) read as given and title by a decision that returned early on an empty head, after two review rounds whose every grid had a name before the comma. Put an empty head and a nickname-only head (`(Bob), PhD`) in every comma grid. Count the one-word path the way C1 counts, which is neither tokens nor pieces: a particle run and the one word it attaches to are one word (#575), a connective join and a bound given-name pair count as their words (#613, so `Ortega y Gasset, Dr.` is a whole name), and a suffix beside the surname counts as no name word at all. #603's review found the suffix half (`Vega y Lopez Jr., PhD Jones` lost its family where `Vega y Lopez, PhD Jones` kept it); #613 then reversed the connective half, and both now read given 'Vega', middle 'y', family 'Lopez' — so test a particle surname, a connective surname and a surname with a suffix, each before a comma. For orders the sweep already exists (`tests/v2/test_cases.py` runs every row under all three) but carries one assertion, R2's family partition, so it checks nothing a new change moves; coverage there has been vacuous before (PR #394's review found the suite passed with `name_order` discarded from grouping). Parse your change's names in each comma shape and each order, and read the ones you did not predict. And read a generated grid's moves by SHAPE, not by corpus membership: #614's first design passed the suite, the five gates and a fingerprint of every corpus name, because its grid moves were sorted into 'corpus name moved' and 'garbage', and the review found the moves that mattered among the 'garbage' -- a bare salutation (`Mr. and Mrs.` lost its connective to the family name) and a name with one trailing word (`Freiherr von Berg X.Y.Z.`). Pull out the head-plus-one-word slice of the grid and a salutation with nothing after it, and read every move there. Two more were the second review's (#614): no grid put TWO words that are both a title and a particle at the head (`Freiherr St van Berg MA`, where an index into the pieces was applied to a merged view of them -- `TITLES ∩ particles_ambiguous` is `{freiherr, st}`, so put both in every head list, each in its bare and its period-marked spelling -- #637's grid had `St` and `Freiherr` alone and missed that `St.` and `Freiherr.` are declined by the bound join and then attached by P6), and the grids compared ambiguity KINDS, so a report telling the caller the wrong field (`Freiherr von Berg Ma`, 'read as a given name' of a word in the family) passed every pin. #701's review added one: put a title/suffix dual (`Ms`, `Sr.`, `Cpl`) SECOND in a title run, behind an ordinary title. A stop that asks only "is this word suffix vocabulary" passes every row that opens with the dual and fails `Rev. Sr. Smith, Mary`; it was found by comparing `T Smith, John` against `T John Smith` over every title word, and a comma-versus-comma-less comparison is the cheap check for any rule that moves a word between the two slots. Compare each report's `detail` against the field its tokens land in -- since #626 `test_a_report_names_the_field_its_word_lands_in` (tests/v2/test_cases.py) does it over every non-locale case row under all three orders (a row carrying its own policy only as declared), for the phrasings its `_FIELD_CLAIMS` lists -- so a row with the shape is the test only where the report's wording is listed: a new emitter's wording goes in that list, and segment's report on a part past the second comma names no field (#629). #626's own first grid then missed an emitter for want of one comma shape (`, Ma Dr.`: an empty head, an ambiguous word, a trailing title, which H1 moves to the family): a trailing title belongs in every empty-head comma grid too. And #620's: an exclusion list in code being replaced is a set of rules, not noise -- #614's merged copy excluded every word the trailing read weighs from its units, the first #620 prototype dropped the list as a symptom of the copy, and its review found the list had been keeping positional counting's rule that a member in front counts as a word (`Freiherr von Berg MA X.Y.Z.` lost its suffix). Before deleting one, mutate each entry out and read what moves. **Then sweep the spellings: a class reached by vocabulary is also reached by SHAPE.** A test that uses only the listed spelling of a word class walks only the vocabulary path, and the shape path is a separate branch that can be missing while every test passes. Roman numerals are the sharpest case: `ii`, `iii` and `iv` are suffix vocabulary; `i` and `v` are vocabulary too but initial-shaped, so `is_suffix_piece` vetoes them and only the numeral fork reads them; and `vi` through `x` are in no list at all, read by the fork's shape test alone (checked 2026-10-04 against `Lexicon.default()`) -- which stops at `x`: `_vocab._ROMAN` matches I to X, so `John Smith XI` reads `XI` as a name word (re-checked 2026-10-07, #614's /simplify, which found a docstring claiming "any length"). So a change that handles `V` and `III` can still miss `VI` — #610 found exactly that, the maiden take's candidate loop admitting a shape-only numeral in one position where the peel reads it in two (`Jane Doe nee Smith VI Prof.` kept maiden `Smith VI` while `V` and `III` worked). The same split runs through the credentials (`MA` listed, `X.Y.Z.` by the dotted shape, `XYZ` by the caps shape, and `Ph. D.` merged by group into one flagged piece the peel's walk never holds -- #603 found #602's run-start predicate refusing it too, so `Smith, John Ph. D. Jones` kept middle 'Jones' where `Smith, John PhD Jones` read suffix 'PhD Jones') and the titles (`Prof.` period-marked and read at the end of a name, `Prof` bare and a name word there). When a change touches a word class, test one spelling from each path. The same holds for a separator's SPACING: a comma (or any punctuation) can stand alone, glue to the word before it or glue to the word after, and each is a different token stream. #653's first fix swept `(née, Jones)` and missed `(née , Jones)`, where the kept comma became a word after the marker and M3 read the clause as maiden ', Jones'; its review found it. Put all three spacings in any grid around a separator. -**And the encoding: a letter class written `[^\W\d_]` or tested with `isalpha()` is blind to a combining mark.** Python's `\w` and `isalpha()` do not match Mn, so a letter written as base plus accent (NFD) ends a word early; the vocabulary path is immune because `_normalize` NFC-folds, the shape path is not. #652: `Dr. inž. Janez Novak` read `inž.` as the given name when the ž was decomposed, and `Janez Novak, ŽŠČ` reversed, in two families that share the blind spot — the letter-shape regexes (`period_abbreviation`, `_DOTTED`, `_CHUNKED`, `_PAIRED_INITIALS`), now spelled with `_vocab._LETTER` (a letter plus the general combining blocks; `test_regex_sync` compares the two `_MARKS` strings and pins `period_abbreviation` against config's copy), and the caps predicates, which NFC-compose a non-ASCII word before `isalpha()` (`is_caps_word`, behind C-level `isalpha()`/`is_normalized` tests, so already-composed text pays no frame — measured 2026-10-12, `Smith, José` was 193 frames before and after; the first version called it for every non-ASCII word and cost one frame each, and no guard saw it because the reference name is ASCII). A width test must count letters, not code points (`composed_len`): admitting a decomposed word to the caps set put `John Smith, MÁ` on the pair exception's wrong side, and the review's grid found it where the rows (all three letters long) did not — put a two-letter word in every caps grid. The initial shape (`_INITIAL`, `\w\.`, v1-verbatim and public) is not widened but asked of the COMPOSED word, in `is_initial_shaped` and in `_render`'s two copies; and the single-letter connective tests count composed letters, since a decomposed `й` is two code points (#665, where `É. y É. Smith` and `John й Smith` read differently decomposed). Compose the WHOLE predicate, not one half: `is_initial` is a shape ANDed with the #320 script veto, and composing the shape alone made a decomposed `씨.` an initial, since decomposed hangul is conjoining jamo outside every script range (#665's first version; `in_initialless_script` composes too now). Not covered, deliberately: the marks only some scripts write (Indic vowel signs, Arabic and Hebrew points; rules.md#H2's Accepted clause). The guard is `test_a_decomposed_name_renders_as_its_composed_twin` over the CLDR corpus plus `test_a_decomposed_abbreviation_reads_as_its_composed_twin` (rows × three orders × EVERYWHERE); the corpus guard reaches a shape only if some corpus name does, which is how the caps shape was missed until review, so when adding a shape rule add a row there. Four of #652's rows and two of #665's pass with every fix reverted (measured) and are parity pins, and the test compares ambiguity kinds and `initials()` too, since #665's `John й Smith` differed in its report; `_DOTTED` has no observable input and is pinned by a direct call. +**And the encoding: a letter class written `[^\W\d_]` or tested with `isalpha()` is blind to a combining mark.** Python's `\w` and `isalpha()` do not match Mn, so a letter written as base plus accent (NFD) ends a word early; the vocabulary path is immune because `_normalize` NFC-folds, the shape path is not. #652: `Dr. inž. Janez Novak` read `inž.` as the given name when the ž was decomposed, and `Janez Novak, ŽŠČ` reversed, in two families that share the blind spot — the letter-shape regexes (`period_abbreviation`, `_DOTTED`, `_CHUNKED`, `_PAIRED_INITIALS`), now spelled with `_vocab._LETTER` (a letter plus the general combining blocks; `test_regex_sync` compares the two `_MARKS` strings and pins `period_abbreviation` against config's copy), and the caps predicates, which NFC-compose a non-ASCII word before `isalpha()` (`is_caps_word`, behind C-level `isalpha()`/`is_normalized` tests, so already-composed text pays no frame — measured 2026-10-12, `Smith, José` was 193 frames before and after; the first version called it for every non-ASCII word and cost one frame each, and no guard saw it because the reference name is ASCII). A width test must count letters, not code points (`composed_len`): admitting a decomposed word to the caps set put `John Smith, MÁ` on the pair exception's wrong side, and the review's grid found it where the rows (all three letters long) did not — put a two-letter word in every caps grid. The initial shape (`_INITIAL`, `\w\.`, v1-verbatim and public) is not widened but asked of the COMPOSED word, in `is_initial_shaped` and in `_render`'s two copies; and the single-letter connective tests count composed letters, since a decomposed `й` is two code points (#665, where `É. y É. Smith` and `John й Smith` read differently decomposed). Compose the WHOLE predicate, not one half: `is_initial` is a shape ANDed with the #320 script veto, and composing the shape alone made a decomposed `씨.` an initial, since decomposed hangul is conjoining jamo outside every script range (#665's first version; `in_initialless_script` composes too now). Not covered, deliberately: the marks only some scripts write (Indic vowel signs, Arabic and Hebrew points; rules/H2's Accepted clause). The guard is `test_a_decomposed_name_renders_as_its_composed_twin` over the CLDR corpus plus `test_a_decomposed_abbreviation_reads_as_its_composed_twin` (rows × three orders × EVERYWHERE); the corpus guard reaches a shape only if some corpus name does, which is how the caps shape was missed until review, so when adding a shape rule add a row there. Four of #652's rows and two of #665's pass with every fix reverted (measured) and are parity pins, and the test compares ambiguity kinds and `initials()` too, since #665's `John й Smith` differed in its report; `_DOTTED` has no observable input and is pinned by a direct call. ### Configuration layer (`nameparser/config/`) @@ -181,9 +183,9 @@ Most modules define a `frozenset` of known name pieces; `capitalization.py` and - `titles.py` — `TITLES` (prenominals) and `GIVEN_NAME_TITLES` (e.g. "Sir", which treat the following name as given, not family) - `suffixes.py` — `SUFFIX_ACRONYMS` (with periods, e.g. "M.D.") and `SUFFIX_WORDS` (e.g. "Jr."), plus `GLUED_HONORIFICS` (#308), the subset of `SUFFIX_WORDS` the peel may split off the END of a name token — a separate, harsher set, since the glued position has no writer-drawn boundary to lean on -- `particles.py` — `PARTICLES` (family-name particles, e.g. "de", "van") and `NON_GIVEN_NAME_PARTICLES`, the curated subset that is *never* a standalone given name. What the subset licenses is rules.md#P1's fold in `post_rules` (the "rule 1b" of older issue comments — decisions.md maps the old numbers), and the fold has ONE site: the piece that OPENS the name. A lone never-given particle there, with at least one more name word behind it, starts the family. Titles do not move the opening position (rules.md#P4), so `Sir de Mesnil` reads family `de Mesnil` under every `name_order`, as `de Mesnil` does. How far the fold reaches depends on the order the name was read under: by default it takes the rest of the name (`de Mesnil Jean` → family `de Mesnil Jean`); under either family-first order it takes the particle run plus ONE name unit and lays the rest out by the order (`de Mesnil Jean` → family `de Mesnil`, given `Jean`); after a family comma it takes the rest of that part whatever the order (`Smith, de Mesnil Jean` → family `Smith de Mesnil Jean`). The leading particle stays a lone piece for the fold to find — `group` does not chain it forward, and the fold walks the particles after it itself — so `de la Vega` reads family `de la Vega` under all three orders. The GIVEN position is not a site. It was a second one until #467, which removed it because under a family-first order that slot holds what the caller DECLARED to be the given name: `Mesnil de` under `FAMILY_FIRST` or `FAMILY_FIRST_GIVEN_LAST` reads given `de`, family `Mesnil`. Nor is a particle inside the name: `Juan de la Vega` under a family-first order reads family `Juan`, given `de la Vega`, with no ambiguity recorded — what the vocabulary forbids is the bare particle as a given name, not a name part that begins with one. A bare `de` has nothing to fold into and is placed by position like any one-word name: given `de` by default, family `de` under both family-first orders. A leading particle OUTSIDE the set is left to position — `van Gogh` is given `van`, family `Gogh` by default and family `van`, given `Gogh` under both family-first orders — and records a `PARTICLE_OR_GIVEN` ambiguity under all three; a never-given particle opening the name records none (`de Mesnil`), and neither does an ambiguous one inside it (`Vincent van Gogh`). `Lexicon.particles_ambiguous` is its complement within `PARTICLES`, so the two mark OPPOSITE sets — see the flip warning in `docs/migrate.rst` before translating either -- `bound_given_names.py` — `BOUND_GIVEN_NAMES` (bound given-name prefixes, e.g. "abdul", "abu"); a group-stage rule joins the first non-title piece to its following piece before roles are assigned, reserving a family word unless a family comma has already fixed it or a given-name title stands ahead (#369, rules.md#P5) (v1's `_join_bound_first_name`, ported into `_pipeline/_group.py` and gone from the tree — the v1 descriptions further down are history, not current code) -- `conjunctions.py` — `CONJUNCTIONS` (e.g. "and", "of") used to chain multi-word titles and to join name parts (rules.md#P3), plus `CONJUNCTIONS_AMBIGUOUS` (#383/#479), the subset of single cased letters that read as an INITIAL in a name written wholly in one case — a marker set with no v1 `Constants` attribute of its own, so the only v1 knob that reaches it is deleting the conjunction +- `particles.py` — `PARTICLES` (family-name particles, e.g. "de", "van") and `NON_GIVEN_NAME_PARTICLES`, the curated subset that is *never* a standalone given name. What the subset licenses is rules/P1's fold in `post_rules` (the "rule 1b" of older issue comments — decisions/legacy-rule-numbers maps the old numbers), and the fold has ONE site: the piece that OPENS the name. A lone never-given particle there, with at least one more name word behind it, starts the family. Titles do not move the opening position (rules/P4), so `Sir de Mesnil` reads family `de Mesnil` under every `name_order`, as `de Mesnil` does. How far the fold reaches depends on the order the name was read under: by default it takes the rest of the name (`de Mesnil Jean` → family `de Mesnil Jean`); under either family-first order it takes the particle run plus ONE name unit and lays the rest out by the order (`de Mesnil Jean` → family `de Mesnil`, given `Jean`); after a family comma it takes the rest of that part whatever the order (`Smith, de Mesnil Jean` → family `Smith de Mesnil Jean`). The leading particle stays a lone piece for the fold to find — `group` does not chain it forward, and the fold walks the particles after it itself — so `de la Vega` reads family `de la Vega` under all three orders. The GIVEN position is not a site. It was a second one until #467, which removed it because under a family-first order that slot holds what the caller DECLARED to be the given name: `Mesnil de` under `FAMILY_FIRST` or `FAMILY_FIRST_GIVEN_LAST` reads given `de`, family `Mesnil`. Nor is a particle inside the name: `Juan de la Vega` under a family-first order reads family `Juan`, given `de la Vega`, with no ambiguity recorded — what the vocabulary forbids is the bare particle as a given name, not a name part that begins with one. A bare `de` has nothing to fold into and is placed by position like any one-word name: given `de` by default, family `de` under both family-first orders. A leading particle OUTSIDE the set is left to position — `van Gogh` is given `van`, family `Gogh` by default and family `van`, given `Gogh` under both family-first orders — and records a `PARTICLE_OR_GIVEN` ambiguity under all three; a never-given particle opening the name records none (`de Mesnil`), and neither does an ambiguous one inside it (`Vincent van Gogh`). `Lexicon.particles_ambiguous` is its complement within `PARTICLES`, so the two mark OPPOSITE sets — see the flip warning in `docs/migrate.rst` before translating either +- `bound_given_names.py` — `BOUND_GIVEN_NAMES` (bound given-name prefixes, e.g. "abdul", "abu"); a group-stage rule joins the first non-title piece to its following piece before roles are assigned, reserving a family word unless a family comma has already fixed it or a given-name title stands ahead (#369, rules/P5) (v1's `_join_bound_first_name`, ported into `_pipeline/_group.py` and gone from the tree — the v1 descriptions further down are history, not current code) +- `conjunctions.py` — `CONJUNCTIONS` (e.g. "and", "of") used to chain multi-word titles and to join name parts (rules/P3), plus `CONJUNCTIONS_AMBIGUOUS` (#383/#479), the subset of single cased letters that read as an INITIAL in a name written wholly in one case — a marker set with no v1 `Constants` attribute of its own, so the only v1 knob that reaches it is deleting the conjunction - `maiden_markers.py` — `MAIDEN_MARKERS` (e.g. "née", "geb.") routing the following name to `maiden` - `surnames.py` — `KOREAN_SURNAMES`, the census list the 2.0 API splits unspaced hangul on (#271). With `maiden_markers.py` it is one of the two data modules `Constants` has **no** attribute for: both reach the parse only through `Constants._snapshot()` → `Lexicon`, so the v1 surface stays frozen and there is no v1 knob to turn either off (the opt-out is the 2.0 `Policy`) - `capitalization.py` — `CAPITALIZATION_EXCEPTIONS` mapping of case MASKS (e.g. `{'phd': 'PhD'}`): each value is its key's own letters and digits recased, laid over the word as written, so `ph.d.` repairs to `Ph.D.` and repair never writes the value's own punctuation into the word (#459); `Lexicon` raises `ValueError` on a value that spells anything else @@ -222,15 +224,15 @@ The 2.0 rewrite lands as underscore-private modules alongside the v1 code. These - **Method organization**, fixed section order in every class: fields + `__post_init__` validation → alternative constructors → dunders (construction/equality → protocol → operators) → properties → public methods by concern (access → editing → comparison → rendering delegates) → private helpers last, except a helper serving exactly one section may sit at that section's head. Sanctioned deviation, facade layer only: `HumanName` and the shim `Constants` organize by v1 concern groups (`# -- render defaults --`, `# -- config / parsing --`, `# -- fields --`, ..., dunders and pickle last) — the classes mirror v1's own surface and die in 3.0; the canonical order still binds every core type. - **Validation is eager and fail-loud**: every `raise` states the offending value, the expected form, and the fix. Exception taxonomy: wrong type — including wrong element type inside a collection, bare `str` where an iterable of strings is expected, or a `Mapping` where a plain iterable is expected — raises `TypeError`; well-typed but unacceptable values raise `ValueError`; failed enum lookups stay `ValueError` for any input (stdlib `EnumType` precedent). **When the message hands the reader code to paste, that code has to survive a type checker, and then RUN** — nameparser ships `py.typed`, and a placeholder can type-check and still raise: the 2.4 release sweep's first fix to the `capitalization_exceptions` hint offered `('key', 'Value')`, which passes mypy and fails #459's mask check, so `test_add_capitalization_exceptions_raises_pointing_at_replace` now executes the offered code. #337's segmenterless warning offered `Policy(segment_scripts=())`, an `arg-type` error, because these fields are annotated with what they STORE rather than everything the constructor accepts. Prefer the `frozenset()` / `()` spellings in messages and docstrings, and pin the offered spelling in a test — the warning tests matched on `ja_segmenter` and never checked the actionable half of the message. **A warning emitted in `Parser.__post_init__` needs `parser_for` to re-emit it from its own frame** (the `catch_warnings(record=True)` block at its return): `__post_init__`'s `stacklevel` is sized for direct `Parser(...)` construction, and through `parser_for`'s extra frame the default one-line rendering attributes the warning to the library's own `return Parser(...)` — the exact call the message tells the user to change becomes invisible. No single stacklevel serves both entry points; a new construction warning gets the re-emission for free, but a new CONSTRUCTION SITE for `Parser` inside this package needs its own re-emission or its callers get library-attributed warnings (#337 review). - **Guard, hint, and emit for the WHOLE family, and parametrize the test over it**: a check added to one member of a set belongs on all of it, and the test must sweep the family, not one example. This session shipped `_reject_str_and_mapping` on `Policy` but not `PolicyPatch`, the bytes decode hint on three of five config entry points, and a regex-sync roster missing four of its copies — each a separate follow-up bug that a `{class} × {field} × {bad-value}` parametrization would have caught and a per-example test hid. When you find you're guarding member N, grep for the other members first. -- **Ambiguities are emitted at the DECISION site**: an `Ambiguity` records a fork the parse had to call, not a token that sits in an ambiguous vocabulary. Emit where the branch is taken — the trailing-suffix peel (read once in `_group` for a main segment since #614, its picks reported by `_assign` once the roles they took are known, except a pick the particle chain takes into the name, which the chain reports and the read then gives up), the delimiter escape's follow-up in `classify` — never by scanning for a `vocab:*-ambiguous` tag. The same tagged token is a genuine fork in one position and unremarkable in another (`do` mid-name in "Joao da Silva do Amaral de Souza" chooses nothing). **A branch that runs but changes nothing is not a decision either** -- the prefix chain's `merge_pieces(pieces, ptags, k, j)` executes even when `j == k + 1`, folding a piece into itself, and keying on "the code got here" reported a fork for all ambiguous particles on "Do Van Jr." (`Dr.` when that was written, before #367 made a plain title transparent and put the shape out of the loop's reach entirely), where the particle stayed a lone leading name piece — the GIVEN name under the default order, the family name under `FAMILY_FIRST` — and `_assign` reported the same token again. Check that the branch actually claimed something (`j > k + 1`, or a run of particles group merged into the piece ahead of S2's read, which the chain then claims nothing past, #625) before recording. Structure often settles the question before it arises, which is why `PARTICLE_OR_GIVEN` is not emitted on the `FAMILY_COMMA` path's WHOLLY-FAMILY read -- the comma fixed which piece is the family -- and `SUFFIX_OR_NAME` is not emitted for "Ma, Jack". A tail segment is the same case from the other side: assign reads it as suffixes, a maiden marker there included since #601, and since #603 a title word past the second comma as a title (rules.md#C2, so `Freiherr` below reads title) (`Jane Doe, PhD, Jr nee van Ma` reads suffix `PhD, Jr nee van Ma`, the marker an ordinary word, rules.md#M2), so group makes no particle chain there at all (#636; its two emitters had been handed no report list there), after either comma — `John Smith, Jr., Freiherr von Richthofen` leaves `von` unchained and reports only `comma-structure` (rules.md#C2, 2026-09-28). Read that scope narrowly: the comma settles nothing about a particle trailing the given name, so P6's attachment decides that fork on the same path and reports it (#405; in `post_rules` until #613, in `_assign` since, right after the walk that reads the given part), in the kind naming the reading it OVERRODE, which is the reading assign made and not the word's vocabulary: `SUFFIX_OR_NAME` where assign had read the run as a post-nominal (`vd`, `mc`), else `PARTICLE_OR_GIVEN` where the run holds an ambiguous particle (`van`, and `do`, which is in the suffix vocabulary too but in its AMBIGUOUS half, so no credential reading was overridden), else silence. The decision site also has the token index and the detail text in hand, which the tag scan would have to reconstruct -- all but the FIELD a report names, which a rule in post_rules can still change (H1's move behind a title, P1's family-first fold, P6's attachment, and under opt-in policies the patronymic rotations and `middle_as_family`'s fold): an emitter naming one passes the rest of its sentence as `field_tail` and assemble words the field from the final role (#626, where `Kim Min Do` under `FAMILY_FIRST` was told 'Do' was a middle name with 'Do' in the family). **If a fork's two branches are taken in DIFFERENT stages, every one of them needs the emitter** -- `PARTICLE_OR_GIVEN` is decided in `_assign` when the ambiguous particle stays a lone leading piece, in `_group` when something shifts it off the name's leading piece and the prefix chain claims it, and wherever P6's attachment takes a trailing particle into the family -- in `_assign` after a comma that names a family (since #613), in `post_rules` at the end of a name read family-first (#467) and after a comma with nothing before it, where H1 and M4 must read the name first -- so every one of them reports; for two years only the first did. What can still do the shifting is narrow, and #367 is why: a plain title no longer can (`Dr. Van Johnson` reads as `Van Johnson` does and reports from `_assign`), so the `_group` emitter needs a word that is BOTH a title and a particle — measured, `TITLES ∩ particles_ambiguous` is `{freiherr, st}` in the default vocabulary (`do` left TITLES in #296's audit; decisions.md's Excluded block records the before and after), plus any overlap a caller's config creates — standing ahead of the chained particle as the LEADING NAME word. Titles may precede it, so `Dr. St van Johnson` reaches the emitter and `St van Johnson` does too; a given name may not, so `Jan Freiherr von Richthofen` does not reach it while `Freiherr von Richthofen` and `Dr. Freiherr von Richthofen` do. Two shapes that look like they should reach it and do NOT, both measured by stepping `STAGES` and watching where `ambiguities` grows: `Dr. Do van Johnson` and `Do St Johnson` report from `assign`, not `group`, because `do` is no longer a title and so stays the leading name piece assign reports on — a both-vocabulary word CHAINED (`Jan St Johnson`) reports nothing at all. When checking whether that emitter is dead, a both-vocabulary word in the leading name position is the thing to look for, and the answer is that it is not dead. The stage-ownership map in `tests/v2/pipeline/test_state.py` must list `ambiguities` for each such stage, and it passes vacuously until a case row exercises the path, so add the row too. Report BOTH directions of a two-way fork — "John Smith MA" (read as a suffix) and "Jack MA" (read as the family name) are equally guesses. Every kind needs a trigger in `tests/v2/test_contracts.py::_AMBIGUITY_TRIGGERS` (an explicit `None`, strict-xfail, while reserved), and case-table rows pin expected kinds exactly, so a new emitter shows up in both immediately. **Pin the decision, not the vocabulary**: the only titled-particle test used an UNAMBIGUOUS particle, so it walked the right code path and proved nothing about the branch under test -- two criticals passed 1539 tests. A row contrasting the two readings ("John Smith V" against "John Smith B") is what makes an emitter's absence meaningful. +- **Ambiguities are emitted at the DECISION site**: an `Ambiguity` records a fork the parse had to call, not a token that sits in an ambiguous vocabulary. Emit where the branch is taken — the trailing-suffix peel (read once in `_group` for a main segment since #614, its picks reported by `_assign` once the roles they took are known, except a pick the particle chain takes into the name, which the chain reports and the read then gives up), the delimiter escape's follow-up in `classify` — never by scanning for a `vocab:*-ambiguous` tag. The same tagged token is a genuine fork in one position and unremarkable in another (`do` mid-name in "Joao da Silva do Amaral de Souza" chooses nothing). **A branch that runs but changes nothing is not a decision either** -- the prefix chain's `merge_pieces(pieces, ptags, k, j)` executes even when `j == k + 1`, folding a piece into itself, and keying on "the code got here" reported a fork for all ambiguous particles on "Do Van Jr." (`Dr.` when that was written, before #367 made a plain title transparent and put the shape out of the loop's reach entirely), where the particle stayed a lone leading name piece — the GIVEN name under the default order, the family name under `FAMILY_FIRST` — and `_assign` reported the same token again. Check that the branch actually claimed something (`j > k + 1`, or a run of particles group merged into the piece ahead of S2's read, which the chain then claims nothing past, #625) before recording. Structure often settles the question before it arises, which is why `PARTICLE_OR_GIVEN` is not emitted on the `FAMILY_COMMA` path's WHOLLY-FAMILY read -- the comma fixed which piece is the family -- and `SUFFIX_OR_NAME` is not emitted for "Ma, Jack". A tail segment is the same case from the other side: assign reads it as suffixes, a maiden marker there included since #601, and since #603 a title word past the second comma as a title (rules/C2, so `Freiherr` below reads title) (`Jane Doe, PhD, Jr nee van Ma` reads suffix `PhD, Jr nee van Ma`, the marker an ordinary word, rules/M2), so group makes no particle chain there at all (#636; its two emitters had been handed no report list there), after either comma — `John Smith, Jr., Freiherr von Richthofen` leaves `von` unchained and reports only `comma-structure` (rules/C2, 2026-09-28). Read that scope narrowly: the comma settles nothing about a particle trailing the given name, so P6's attachment decides that fork on the same path and reports it (#405; in `post_rules` until #613, in `_assign` since, right after the walk that reads the given part), in the kind naming the reading it OVERRODE, which is the reading assign made and not the word's vocabulary: `SUFFIX_OR_NAME` where assign had read the run as a post-nominal (`vd`, `mc`), else `PARTICLE_OR_GIVEN` where the run holds an ambiguous particle (`van`, and `do`, which is in the suffix vocabulary too but in its AMBIGUOUS half, so no credential reading was overridden), else silence. The decision site also has the token index and the detail text in hand, which the tag scan would have to reconstruct -- all but the FIELD a report names, which a rule in post_rules can still change (H1's move behind a title, P1's family-first fold, P6's attachment, and under opt-in policies the patronymic rotations and `middle_as_family`'s fold): an emitter naming one passes the rest of its sentence as `field_tail` and assemble words the field from the final role (#626, where `Kim Min Do` under `FAMILY_FIRST` was told 'Do' was a middle name with 'Do' in the family). **If a fork's two branches are taken in DIFFERENT stages, every one of them needs the emitter** -- `PARTICLE_OR_GIVEN` is decided in `_assign` when the ambiguous particle stays a lone leading piece, in `_group` when something shifts it off the name's leading piece and the prefix chain claims it, and wherever P6's attachment takes a trailing particle into the family -- in `_assign` after a comma that names a family (since #613), in `post_rules` at the end of a name read family-first (#467) and after a comma with nothing before it, where H1 and M4 must read the name first -- so every one of them reports; for two years only the first did. What can still do the shifting is narrow, and #367 is why: a plain title no longer can (`Dr. Van Johnson` reads as `Van Johnson` does and reports from `_assign`), so the `_group` emitter needs a word that is BOTH a title and a particle — measured, `TITLES ∩ particles_ambiguous` is `{freiherr, st}` in the default vocabulary (`do` left TITLES in #296's audit; the particle-line block in decisions/vocabulary-exclusions records the before and after), plus any overlap a caller's config creates — standing ahead of the chained particle as the LEADING NAME word. Titles may precede it, so `Dr. St van Johnson` reaches the emitter and `St van Johnson` does too; a given name may not, so `Jan Freiherr von Richthofen` does not reach it while `Freiherr von Richthofen` and `Dr. Freiherr von Richthofen` do. Two shapes that look like they should reach it and do NOT, both measured by stepping `STAGES` and watching where `ambiguities` grows: `Dr. Do van Johnson` and `Do St Johnson` report from `assign`, not `group`, because `do` is no longer a title and so stays the leading name piece assign reports on — a both-vocabulary word CHAINED (`Jan St Johnson`) reports nothing at all. When checking whether that emitter is dead, a both-vocabulary word in the leading name position is the thing to look for, and the answer is that it is not dead. The stage-ownership map in `tests/v2/pipeline/test_state.py` must list `ambiguities` for each such stage, and it passes vacuously until a case row exercises the path, so add the row too. Report BOTH directions of a two-way fork — "John Smith MA" (read as a suffix) and "Jack MA" (read as the family name) are equally guesses. Every kind needs a trigger in `tests/v2/test_contracts.py::_AMBIGUITY_TRIGGERS` (an explicit `None`, strict-xfail, while reserved), and case-table rows pin expected kinds exactly, so a new emitter shows up in both immediately. **Pin the decision, not the vocabulary**: the only titled-particle test used an UNAMBIGUOUS particle, so it walked the right code path and proved nothing about the branch under test -- two criticals passed 1539 tests. A row contrasting the two readings ("John Smith V" against "John Smith B") is what makes an emitter's absence meaningful. - **A kind is worth adding only if a reader would hesitate too**: the test is not "does the code take a branch" but whether a person reading that input would genuinely be unsure. "Smith, John V" reads as a middle initial to anyone -- the comma settles it -- so reporting it would be noise that teaches callers to ignore the field, which costs more than the missing report. Reachability of the second branch is necessary, not sufficient. Prefer leaving a fork silent and documenting the omission over emitting on input nobody finds ambiguous. - **Parser owns config-dependent conveniences**: `Parser.matches`/`Parser.capitalized`/`Parser.revise` exist because the `ParsedName` equivalents fall back to DEFAULT config for str/omitted arguments (documented loudly in both docstrings). `revise` harvests tokens from a full sub-parse of each replacement value (tags kept minus `FOLDED_TAG`, roles forced, the R1 entry pass `suffix_entries` re-run over the forced state so a suffix value's entries follow its own commas, ambiguities discarded); the merge tail is shared with `replace()` via `ParsedName._with_field_tokens`. `Parser.capitalized` delegates through `name.capitalized(self.lexicon)` specifically so `_parser` never imports `_render` — keep it that way. - **Per-word vocabulary fields warn on multi-word entries** (`_normset`/`_normpairs` via `_warn_dead_entry`, UserWarning, never a raise — see the given_name_titles Gotcha for why raising is wrong). `given_name_titles` is the one multi-word-matched field and is exempt; `_edit` passes `warn=False` (add() warns once via the new instance's `__post_init__`; remove() stores nothing). The default vocabulary and every locale pack must stay warning-free (`test_default_lexicon_builds_warning_free`, `test_pack_vocabulary_entries_are_single_words`). - **Invariants guard harm, not no-ops**: add a constructor check when violating it produces a *wrong parse*, not when it produces *nothing*. A false positive costs a working configuration; a true positive on an inert condition costs the user nothing, so that trade is never worth taking. `suffix_acronyms_ambiguous ∩ suffix_words` is guarded because the overlap loses a family name; `given_name_titles` is not, because an unreachable entry is simply never consulted (see Gotchas). Before adding one, construct the config it forbids and check what actually breaks. **Between raise and silence sits the construction-time `UserWarning`**, for a gap that is statically decidable, harmless to SOME deliberate caller, and indistinguishable-from-working for everyone else: the segmenterless activation (#337 — `parser_for(locales.JA)` without a segmenter behaved exactly like a working parser minus the feature) warns rather than raises because the inert JA registration is itself a pinned property, and a warning is filterable by the caller who wants exactly that. The message must carry every applicable remedy and no inapplicable one (the `ja_segmenter` hint fires only when a Japanese script is among the dead ones). Test fuzzers that legitimately construct such configs suppress the warning by MESSAGE, never by category — a blanket `UserWarning` ignore would mask the next construction diagnostic (`_quiet_parser` in `tests/v2/test_properties.py` is the pattern). - **The shim TRANSLATES; it never raises on a config v1 accepted (one decided exception, #459, at the end of this bullet), and never silently changes the parse**: `Constants._snapshot()` is a translation boundary between v1's model and v2's invariants, and every transformation there carries its v1-reachability argument in a comment. These exist today — `first_name_titles` re-folded per word (v1 joins-then-`lc`, v2 normalizes-then-joins), `suffix_acronyms_ambiguous ∩ acronyms` (a provable no-op), `suffix_words − ambiguous` (v1 already accepts the word via the acronym branch, so the addition is inert there), `particles_ambiguous ∪ (bound ∩ particles)` (a pinned deviation, `test_bound_never_given_prefix_deviates_on_two_pieces`), `honorific_tails = GLUED_HONORIFICS ∩ suffix_words` (#308 behavior with no v1 manager of its own, so the one v1 knob that reaches it is deleting the suffix word — which turns the peel off, `test_snapshot_removing_a_honorific_word_turns_the_peel_off`), `conjunctions_ambiguous = CONJUNCTIONS_AMBIGUOUS ∩ conjunctions` (#383/#479 behavior with no v1 manager of its own either, so the one v1 knob that reaches it is deleting the conjunction — which turns the marking off, `test_snapshot_removing_a_conjunction_turns_the_marker_off`), and `maiden_delimiters − nickname_delimiters` on the POLICY half of the same method (v1 precedence: a pair in both v1 buckets parses as a nickname, while `Policy` resolves the overlap the other way, so the subtraction is what keeps the facade at v1 behavior, `test_snapshot_overlap_keeps_v1_nickname_precedence`), and `_v1_matchable`, which runs ahead of every other one: an entry v1 could never match — empty after `lc()`, or holding edge whitespace, a whitespace run or any whitespace character other than a single space, since no v1 piece does — is dropped from all nine set fields and from the `capitalization_exceptions` keys and named in one `UserWarning` whose remedy runs (#541; translated, it started matching once `Lexicon` stripped it or tripped a `Lexicon` invariant the set algebra could not see, `test_an_entry_v1_could_not_match_is_dropped_with_one_warning`), except the two whitespace typos in 1.4.0's own shipped `TITLES`, which a restored 1.4 pickle carries and are dropped without the warning. The same filter drops an entry `Lexicon` folds to empty — since 2.3.0's `FULL_STOPS` fold, a lone CJK full stop, which 1.4.0 through 2.2.0 could match on a token made of that stop but which since 2.3.0 can be neither held nor matched, so translating it raised wherever `Lexicon` received it (#582, the one departure from 1.4.0 this filter makes) — and the two cross-set computations `Lexicon` re-checks after its own fold (`suffix_words − ambiguous` and `bound ∩ particles`) compare `_normalize`d spellings, while every other field passes raw as before (an edge full stop, `'ma。'` beside the ambiguous `ma`, had raised `Lexicon`'s gate-bypass check since 2.3.0; folding EVERY field instead was measured to move readings on configs that never raised, `test_an_edge_full_stop_entry_outside_the_two_checks_reads_as_before`). Note that the `maiden_delimiters − nickname_delimiters` one is on the `Policy`, not the `Lexicon` — the roster is per-`_snapshot()`, not per-vocabulary-field, so a sweep that only reads the `Lexicon(...)` call misses it. When a v1 config cannot satisfy a v2 invariant, work out what v1 actually *does* with it — usually nothing — and reproduce that; weakening the invariant or letting the raise through are both wrong. **Test the case the translation decides**, not one where both branches agree: a test using an input v1 parses identically with and without the config pins nothing. A `capitalization_exceptions` value that does not spell its key's letters and digits raises `ValueError` at snapshot rather than being translated (#459), because v1 substituted such a value for the word and 2.4 repair only recases, so there is no v1 behavior to reproduce; pinned by `test_a_mismatched_exception_value_raises_at_the_first_parse`. - **Reprs are bounded**: render which fields deviate from a named baseline and by how much, never contents (`Lexicon(default + titles: +2)`). `PolicyPatch`'s repr shows only set (non-UNSET) fields; `_order_repr` must never raise even on an unvalidated patch's garbage `name_order` (PolicyPatch defers validation to apply time); the sweep test in `tests/v2/test_reprs.py` pins that no config repr leaks the UNSET sentinel. -- **Every pipeline stage is one module holding one public function of the same name** (`_tokenize.py`/`tokenize`), with the exceptions named where they stand — `_group.py`'s `marker_run_length`, a shared predicate (mechanisms.md#ONE-PREDICATE-PER-QUESTION), and since #511 `_post_rules.py`'s `suffix_entries`, the R1 entry pass as a function, because `Parser.revise` runs that one pass over a forced-role sub-parse and a stage's tail cannot be called on its own — **and its module docstring declares the contract in three labelled lines**: `Consumes:` what it takes from `ParseState`, `Produces:` what it hands back, `Reads:` which `Policy`/`Lexicon` fields it consults. `Reads:` is the load-bearing one — it makes "which stage do I touch for this feature?" a grep rather than a read-through. The authority for the stage set is `_pipeline/__init__.py`'s `STAGES` (eight, and not public API) with the field-ownership map in `ParseState`'s docstring, pinned by `tests/v2/pipeline/test_state.py`; NOT rules.md, which is implementation-free by its own preamble and whose `implemented:` names modules honoring a rule rather than stages. A `_pipeline/` module that is not a stage says so in its first line instead (`_assemble.py`: "Not a stage: …", omitting `Reads:` because it consults neither), so absence of the three lines is a claim about the module rather than an oversight. Provenance: §5 of the 2026-07-11 conventions spec, recorded here 2026-08-16. -- **Frames are a budget for hot and scaling paths; everywhere else, prefer the shared predicate.** The parser's cost is counted in Python frames (`sys.setprofile` "call" events — deterministic and load-immune, which is why #475 chose them over wall-clock timing), but one frame is small: measured 2026-09-28 on py3.11, a trivial call costs about 13 ns against 40–80 µs for a whole parse, so one frame per parse is roughly 0.03% of it. What the budget exists to catch is how a cost SCALES and how it ACCUMULATES, so weigh an extra frame by four questions, in order. (1) Does it multiply with the input — per token, per member, per pair? Then it matters, and a hand-inlined fast path can earn its place (the inline tag tests ahead of `_pieces.starts_a_credential_run` in its loops, each a necessary half of the predicate so nothing can drift; `_vocab.run_word_fold` was the measured example until #613 deleted the run test it served). Folding a copy into a shared helper is where such a pre-check gets lost: #617 replaced `_comma._title_chains`, whose loop tested the conjunction tag inline, with a call into group's loop, which asks `is_conj_piece` per piece, and every family-comma parse paid two frames per word of the part (`Smith, John` +2) while the reference row fell -- review caught it; the fix gates the call on the tag walk the caller already makes, and `test_a_comma_part_with_no_free_connective_skips_the_join` holds it. Count a comma name, not only the reference row, when a fold touches the comma path. (2) How many names pay it — every parse, where plain names like `Smith, John` must keep their cost, or only a rare shape? (3) Does it move an enforced guard — the ±2% reference band in `test_benchmark.py` (`_CALL_BASELINE`, decisions.md#parse-cost) or a scaling ratio? (4) What does the inline alternative cost to maintain? A hand copy of a predicate that can drift is the defect class mechanisms.md#ONE-PREDICATE-PER-QUESTION names, and it compounds; a NARROWER question stated exactly is not a copy and may stay inline on that ground alone. A constant frame on a rare shape that no guard sees loses to readability: #544's cleanup declined `is_title_piece` for one frame on comma credential runs (24 ns a parse, no guard moved), a stricter call than this rule asks for; by this rule the inline test is still right, but because it asks the narrower question (a lone word in both vocabularies), not because of the frame. The converse holds too: the frame counter cannot see C-level work (`x in some_list`, slices, string concatenation), so "no new frames" is not "no new cost" — #553's quadratic was a list membership test inside a loop, invisible to every frame guard. +- **Every pipeline stage is one module holding one public function of the same name** (`_tokenize.py`/`tokenize`), with the exceptions named where they stand — `_group.py`'s `marker_run_length`, a shared predicate (mechanisms.md#ONE-PREDICATE-PER-QUESTION), and since #511 `_post_rules.py`'s `suffix_entries`, the R1 entry pass as a function, because `Parser.revise` runs that one pass over a forced-role sub-parse and a stage's tail cannot be called on its own — **and its module docstring declares the contract in three labelled lines**: `Consumes:` what it takes from `ParseState`, `Produces:` what it hands back, `Reads:` which `Policy`/`Lexicon` fields it consults. `Reads:` is the load-bearing one — it makes "which stage do I touch for this feature?" a grep rather than a read-through. The authority for the stage set is `_pipeline/__init__.py`'s `STAGES` (eight, and not public API) with the field-ownership map in `ParseState`'s docstring, pinned by `tests/v2/pipeline/test_state.py`; NOT the rules, which are implementation-free by their own preamble and whose `implemented:` names modules honoring a rule rather than stages. A `_pipeline/` module that is not a stage says so in its first line instead (`_assemble.py`: "Not a stage: …", omitting `Reads:` because it consults neither), so absence of the three lines is a claim about the module rather than an oversight. Provenance: §5 of the 2026-07-11 conventions spec, recorded here 2026-08-16. +- **Frames are a budget for hot and scaling paths; everywhere else, prefer the shared predicate.** The parser's cost is counted in Python frames (`sys.setprofile` "call" events — deterministic and load-immune, which is why #475 chose them over wall-clock timing), but one frame is small: measured 2026-09-28 on py3.11, a trivial call costs about 13 ns against 40–80 µs for a whole parse, so one frame per parse is roughly 0.03% of it. What the budget exists to catch is how a cost SCALES and how it ACCUMULATES, so weigh an extra frame by four questions, in order. (1) Does it multiply with the input — per token, per member, per pair? Then it matters, and a hand-inlined fast path can earn its place (the inline tag tests ahead of `_pieces.starts_a_credential_run` in its loops, each a necessary half of the predicate so nothing can drift; `_vocab.run_word_fold` was the measured example until #613 deleted the run test it served). Folding a copy into a shared helper is where such a pre-check gets lost: #617 replaced `_comma._title_chains`, whose loop tested the conjunction tag inline, with a call into group's loop, which asks `is_conj_piece` per piece, and every family-comma parse paid two frames per word of the part (`Smith, John` +2) while the reference row fell -- review caught it; the fix gates the call on the tag walk the caller already makes, and `test_a_comma_part_with_no_free_connective_skips_the_join` holds it. Count a comma name, not only the reference row, when a fold touches the comma path. (2) How many names pay it — every parse, where plain names like `Smith, John` must keep their cost, or only a rare shape? (3) Does it move an enforced guard — the ±2% reference band in `test_benchmark.py` (`_CALL_BASELINE`, decisions/parse-cost) or a scaling ratio? (4) What does the inline alternative cost to maintain? A hand copy of a predicate that can drift is the defect class mechanisms.md#ONE-PREDICATE-PER-QUESTION names, and it compounds; a NARROWER question stated exactly is not a copy and may stay inline on that ground alone. A constant frame on a rare shape that no guard sees loses to readability: #544's cleanup declined `is_title_piece` for one frame on comma credential runs (24 ns a parse, no guard moved), a stricter call than this rule asks for; by this rule the inline test is still right, but because it asks the narrower question (a lone word in both vocabularies), not because of the frame. The converse holds too: the frame counter cannot see C-level work (`x in some_list`, slices, string concatenation), so "no new frames" is not "no new cost" — #553's quadratic was a list membership test inside a loop, invisible to every frame guard. - **A claim about WHICH STAGE or WHICH LAYER does something is checkable — check it before writing it.** The pipeline is eight stages with a written ownership map (`ParseState`'s docstring, pinned by `tests/v2/pipeline/test_state.py`), and `parse(s).tokens` prints every token's role and tags, so "extract assigns this", "classify never sees that", "group consumes it" each have a one-command answer. #329's prose claimed delimited maiden content is *"claimed whole before classify has tagged anything inside it"*; measured, `classify` tags the marker fine and only the CONSUMING is missing, because `_group`'s rule walks `pieces` and a token that already carries a role is not in `pieces`. Two different mechanisms, one plausible sentence covering both. That single claim then shipped SIX times across three correction rounds, which is the part worth internalizing: **when a mechanism claim turns out wrong, sweep for where else you wrote it, and sweep again at the END of the change over the words the change itself just added.** Prose density here means one idea lives in a docstring, a case note, a release-log entry and this file at once; the implementer working against a wrong mechanism is the person most likely to restate it; and rewriting a mechanism claim is writing one, so the correction earns the same one-command check as the original — two of the six instances were fresh errors introduced by the sentence fixing the previous one. **Adjacency is the trap.** The claim that feels already-known is the one about the neighbouring stage or the neighbouring layer: one comment block in `tests/v2/test_facade_cases.py` got the exception type, the raising layer, the skip mechanism, the count of skipped rows, and which row was blocked all wrong at once — every one a claim about `_config_shim` versus real 1.4.0, written from reasoning, in the file whose whole job is translating between them. What finally held was not better prose but moving the claim into a test (`_CORE_ONLY_IDS`), which cannot be wrong the way a sentence can. - **Typing/docs**: `from __future__ import annotations`; `frozen=True, slots=True` on every public dataclass; strict-profile mypy flags via per-module overrides in pyproject (`strict = true` itself is not valid per-module). Docstrings state contracts in prose with **no doctest blocks** — `--doctest-modules` makes every example a test; behavior examples go to unit tests per the lean-docs rule. **Document the positive direction of a partial property**: "a non-empty `ambiguities` is a signal to act on" is checkable, while "an empty one means no fork occurred" is a universal negative needing exhaustive verification -- that claim was written twice and falsified twice, at sites the author had not audited. - **The segmenter contract**: the optional `Parser(segmenter=...)` hook is parse-totality's ONE exception (locales spec section 4). Everything inside that exception is a bug in USER CODE, never a fact about the name, so it is surfaced rather than absorbed: the segmenter's own exceptions propagate, and the two protocol violations the stage can detect for itself — an answer of the wrong type, and one cutting at or past the end of the token it was handed — raise `TypeError`/`ValueError` from `_script_segment` for the same reason. The line to hold when adding a check there: a protocol violation by the segmenter's AUTHOR raises, while an adapter's defense against its own third-party library (`locales/ja.py`'s repertoire, length, reconstruction and score guards) declines with `None`, because what those catch is a fact about the content. @@ -253,7 +255,7 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_ **Unknown-key attribute access on `TupleManager`/`RegexTupleManager` warns (1.4, #256) — 2.0 note: the warning became a hard `AttributeError` naming the known keys (shim `TupleManager`); the paragraph below describes the deleted v1 machinery and applies only when reading v1 history.** — a key not currently in the dict emits `DeprecationWarning` naming the miss and the known keys (`_warn_unknown_key`), before falling back to the same `None`/`EMPTY_REGEX` default as before; `.get()` stays silent. Dunder probes (`__deepcopy__`) still raise `AttributeError` outright, and single-underscore probes (`_repr_html_`, IPython's `_ipython_canary_method_should_not_exist_`, etc.) are excluded from the warning too — no real config key starts with `_`, so both guards just keep protocol/introspection probes from misfiring as "typo" warnings. This means internal parser code that reads `self.C.regexes.` unconditionally (e.g. `squash_bidi`'s `bidi`) now warns if a caller's custom `regexes` dict omits that key — a previously-silent partial-override pattern is on the same deprecation path as an actual typo. -**Adding a word to a config set** — first check the *other* sets for the same word (grep `nameparser/config/` or intersect the sets in a `python3 -c`). Real overlaps exist: `do`/`st`/`mc` ∈ `PARTICLES` ∩ `TITLES`/`SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PARTICLES` ∩ `BOUND_GIVEN_NAMES` (position-dependent: leading token → given-name join, mid-name → family-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; dropping `abd` from `BOUND_GIVEN_NAMES`). Then check the word against real names, not memory: `uv run python tools/name_usage.py ` counts its bearers as a given name and as a family name (US Census surnames, SSA given names, Wikidata with countries of citizenship). Pass a name as it is written (`McDonald`): Wikidata labels match exactly, and no rule rebuilds an internal capital from `mcdonald`. It reports and does not decide — read the counts against the position the set's claim acts on. A word that would peel GLUED (`GLUED_HONORIFICS`) asks a different question, whether any name ENDS in it, and for a CJK word the tool answers that too: an ends-with row from a cached index of Wikidata's CJK name labels (built on first use; 33 minutes on 2026-10-17), plus the names the word ends across the family/given join (阿部 + 長 for 部長). Read the tool's hits by item before counting them, because labels include transcriptions and mislabels: #707's one hit for 二世 was 厄德二世, the given name Odo with a regnal numeral in its Chinese label, borne by 109 Western people named Odo. Then ask a third question the tool cannot: does a common longer word end in it? The peel matches the end of a token, spaced or glued, so a listed tail splits every unlisted compound that ends in it — #707 vetted 議員, 社長 and 医学博士 against names and its review found 衆議院議員, 副社長 and 獣医学博士 cut at the tail (`田中 副社長` → given 副); no corpus held a compound office word until #707's rules.md examples added two, so the snapshot was silent about the class. List the common compounds whole or record the split as a cost. And check the predicates that test a member's SPELLING rather than its membership: they encode what the set's members happened to look like when they were written. `_pieces.starts_a_credential_run` implemented rules.md#S2's "Latin" as `isascii()`, exact until #661 added `mlađi`, which would have read `mlađi Jones` differently from `stariji Jones`; the design-docs review caught it, after a first draft had credited the test to the wrong clause of S2 and proposed deleting it on a mutation that moved nothing because nothing pinned it. (#679 then dropped the script test on purpose, after measuring what it protected.) Before adding the first member of a new script, or the first accented one, grep the set's consumers for `isascii`, `isalpha` and script tests, and read the rule each one implements. The same goes for the first member holding PUNCTUATION: #700 added `d'` and `az-`, the parse read them right and every snapshot move was an intended one, and `capitalized()` still wrote `D' Ivoi`, because case repair splits a token at its apostrophe or hyphen and asks the vocabulary of the stem. `capitalized()` is invisible to the snapshot (`initials()` is recorded there), so parse such a member's names through `capitalized()` and `capitalized(force=True)` in each case before calling it done. +**Adding a word to a config set** — first check the *other* sets for the same word (grep `nameparser/config/` or intersect the sets in a `python3 -c`). Real overlaps exist: `do`/`st`/`mc` ∈ `PARTICLES` ∩ `TITLES`/`SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PARTICLES` ∩ `BOUND_GIVEN_NAMES` (position-dependent: leading token → given-name join, mid-name → family-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; dropping `abd` from `BOUND_GIVEN_NAMES`). Then check the word against real names, not memory: `uv run python tools/name_usage.py ` counts its bearers as a given name and as a family name (US Census surnames, SSA given names, Wikidata with countries of citizenship). Pass a name as it is written (`McDonald`): Wikidata labels match exactly, and no rule rebuilds an internal capital from `mcdonald`. It reports and does not decide — read the counts against the position the set's claim acts on. A word that would peel GLUED (`GLUED_HONORIFICS`) asks a different question, whether any name ENDS in it, and for a CJK word the tool answers that too: an ends-with row from a cached index of Wikidata's CJK name labels (built on first use; 33 minutes on 2026-10-17), plus the names the word ends across the family/given join (阿部 + 長 for 部長). Read the tool's hits by item before counting them, because labels include transcriptions and mislabels: #707's one hit for 二世 was 厄德二世, the given name Odo with a regnal numeral in its Chinese label, borne by 109 Western people named Odo. Then ask a third question the tool cannot: does a common longer word end in it? The peel matches the end of a token, spaced or glued, so a listed tail splits every unlisted compound that ends in it — #707 vetted 議員, 社長 and 医学博士 against names and its review found 衆議院議員, 副社長 and 獣医学博士 cut at the tail (`田中 副社長` → given 副); no corpus held a compound office word until #707's rules examples added two, so the snapshot was silent about the class. List the common compounds whole or record the split as a cost. And check the predicates that test a member's SPELLING rather than its membership: they encode what the set's members happened to look like when they were written. `_pieces.starts_a_credential_run` implemented rules/S2's "Latin" as `isascii()`, exact until #661 added `mlađi`, which would have read `mlađi Jones` differently from `stariji Jones`; the design-docs review caught it, after a first draft had credited the test to the wrong clause of S2 and proposed deleting it on a mutation that moved nothing because nothing pinned it. (#679 then dropped the script test on purpose, after measuring what it protected.) Before adding the first member of a new script, or the first accented one, grep the set's consumers for `isascii`, `isalpha` and script tests, and read the rule each one implements. The same goes for the first member holding PUNCTUATION: #700 added `d'` and `az-`, the parse read them right and every snapshot move was an intended one, and `capitalized()` still wrote `D' Ivoi`, because case repair splits a token at its apostrophe or hyphen and asks the vocabulary of the stem. `capitalized()` is invisible to the snapshot (`initials()` is recorded there), so parse such a member's names through `capitalized()` and `capitalized(force=True)` in each case before calling it done. **Before adding a short/common word to `PARTICLES` globally**, test it mid-string against realistic 3-token names, not just check for English-word collisions: Korean/Vietnamese given names put a short syllable in the middle slot (`Park In Hwan`, `Nguyen To Nga`), and Western names put a bare initial there (`John V. Smith`). A word that looks safe ("nobody is named 'to'") can still swallow a real middle name/initial into the last name once it's a global prefix — confirmed regressions for `to`/`in`/`an`/`ten`/`then` and bare `v` this way (PR #191). @@ -273,9 +275,9 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_ **`HumanName.C` is a property backed by `_C`, but pickles under the public key `'C'`** — `__init__`/direct assignment route through the `C` setter, which calls the shared `_validate_constants` staticmethod (also used by `__init__`) so an invalid value raises `TypeError` immediately instead of surfacing later as an unrelated `AttributeError` deep in parsing (#239). `__getstate__`/`__setstate__` deliberately translate `self._C` ↔ a `'C'` key in the pickled dict (with the usual `CONSTANTS`-singleton-becomes-`None` sentinel) rather than pickling `_C` directly, so the on-disk pickle format hasn't changed across this fix — don't "simplify" that translation away or old pickles/tests that hand-build a state dict with a `'C'` key will break. -**Titles permanently shadow first names — be conservative** — any word in `TITLES` is always consumed as a title and can never be parsed as a first name. `"Dean"` is the canonical example: it's a common academic title *and* a common given name, so it is intentionally absent from the default titles (see `docs/customize.rst` — users who need it add it via opt-in `Constants`). Before adding a word to `TITLES`, ask: "Could this plausibly be someone's given name in any culture?" If yes, don't add it globally; it belongs in caller-supplied `Constants` instead. This same caution applies to international honorifics — `Prince`, `Sheikh`, `Frau` are all first names in some contexts. It also applies to any prefix sub-set gated on "never a first name": obscure-looking foreign particles are surprisingly often real given names — `Von` (Von Miller), `Vander` (Brazilian, also the Arcane character). When unsure, exclude — a missing member just means that name isn't auto-handled, whereas a wrong member misparses a real person. The decided exception is a word borne as a name only RARELY in the leading position — mostly a surname written last (`graf`, `greve`, `knight`), a few with small given-name or surname-first counts (`herra`, `hrabia`): the title claim acts only in front, so it ships, and a name that begins with it (`Greve Anna`, `Herra Wijaya` → title) is the accepted cost. That is a judgment by incidence, made per word with the counts recorded in `decisions.md#salutation-titles` (rules.md#H1's Accepted clause states the cost), not a rule a sweep can apply: a word that commonly leads a real name (Vietnamese `Ông`, the given names `thiru`, `puan`, `marshal`) is still out. `tools/name_usage.py` produces the counts this judgment starts from (role counts, plus Wikidata citizenship to tell where a family name is written first); its Wikidata column is a lower bound (Shehu Shagari has no given-name statement), so a small count there proves little and a zero proves nothing. +**Titles permanently shadow first names — be conservative** — any word in `TITLES` is always consumed as a title and can never be parsed as a first name. `"Dean"` is the canonical example: it's a common academic title *and* a common given name, so it is intentionally absent from the default titles (see `docs/customize.rst` — users who need it add it via opt-in `Constants`). Before adding a word to `TITLES`, ask: "Could this plausibly be someone's given name in any culture?" If yes, don't add it globally; it belongs in caller-supplied `Constants` instead. This same caution applies to international honorifics — `Prince`, `Sheikh`, `Frau` are all first names in some contexts. It also applies to any prefix sub-set gated on "never a first name": obscure-looking foreign particles are surprisingly often real given names — `Von` (Von Miller), `Vander` (Brazilian, also the Arcane character). When unsure, exclude — a missing member just means that name isn't auto-handled, whereas a wrong member misparses a real person. The decided exception is a word borne as a name only RARELY in the leading position — mostly a surname written last (`graf`, `greve`, `knight`), a few with small given-name or surname-first counts (`herra`, `hrabia`): the title claim acts only in front, so it ships, and a name that begins with it (`Greve Anna`, `Herra Wijaya` → title) is the accepted cost. That is a judgment by incidence, made per word with the counts recorded in `decisions/salutation-titles` (rules/H1's Accepted clause states the cost), not a rule a sweep can apply: a word that commonly leads a real name (Vietnamese `Ông`, the given names `thiru`, `puan`, `marshal`) is still out. `tools/name_usage.py` produces the counts this judgment starts from (role counts, plus Wikidata citizenship to tell where a family name is written first); its Wikidata column is a lower bound (Shehu Shagari has no given-name statement), so a small count there proves little and a zero proves nothing. -**The period-abbreviation title inference runs at the head of the GIVEN-NAME part, not the head of the name** — an unrecognized multi-letter word ending in a single trailing period (`_pieces._PERIOD_ABBREV`, a hand copy of the `period_abbreviation` regex, `{2,}` letters — it was assign's until #424 and group's until #439) is treated as a title in the leading title run, e.g. `"Insp. Jane Morse"` → `title='Insp.'`. "Leading" is per SEGMENT: `_peel_leading_titles` is called for NO_COMMA segment 0, SUFFIX_COMMA segment 0, and FAMILY_COMMA **segment 1**, so `"Morse, Det. Insp. Jane"` → `title='Det. Insp.'` and a lone `"Smith, Xyz."` → `title='Xyz.'` — long-standing, verified against 1.4.0, and the mechanism behind #296 (`"Smith, Jr."` read title in 1.4.0; the comma's credential read takes it as suffix `Jr.` now). The docs said "leading word" until 2026-08-01 and were wrong for every comma path. FAMILY_COMMA **segment 0** peels too since #701, but by VOCABULARY only (rules.md#H6, in assign's family-comma branch rather than `_peel_leading_titles`): `Dr. Smith, John` → `title='Dr.'`, while `Xyz. Smith, John` keeps family `Xyz. Smith`, and a title-particle (`St`, `Freiherr`) stops the run, as does a title with nothing but post-nominals behind it (`Dr. Jr., John` keeps family `Dr.`); a title/suffix dual with a name word behind it still peels (`Rev. Sr. Smith, Mary`). It does not mutate `C.titles`, so the periodless form (`"Insp"`) is unaffected elsewhere. The `{2,}` length requirement — not a separate initials check — is what excludes single-letter initials like `"J."`; the same word after the given name is left as a middle name. **The inference OUTRANKS vocabulary where it runs**: `"Esq. Smith"` → `title='Esq.'` even though `esq` is suffix-only vocabulary, because the shape rule fires before anything consults the suffix sets. **The INFERENCE still runs in one direction only, and that is what the two slots share and where they part** (rewritten 2026-09-08, #316): a period-marked word is claimed by SHAPE at the front and by VOCABULARY at the back, so an unlisted abbreviation opening a name is a title while an unlisted abbreviation ending one is a NAME word (`"John Smith Xyz."` → `family='Xyz.'`). A trailing period-marked word the vocabulary knows as a title now IS one (`"John Smith Prof."` → `title='Prof.'`, `rules.md#H5`) — the comma path always read it that way and the two agree now — and a trailing period-marked word the SUFFIX vocabulary knows is still a suffix, the suffix run being peeled first (`"John Smith Esq."` → `suffix='Esq.'`). A BARE trailing title word stays a name word (`"John Smith Sir"`, `"Mary Jane King"`), which is the doctrine line below. Meanwhile `period_joined_vocab` resolves INTERIOR-period tokens (`Lt.Gov.`, `Msc.Ed.`) to title-or-suffix by vocabulary, and `_extract._suffix_shaped` treats any period-final delimited content as not-a-nickname. Four sites, and the trailing one now resolves by two vocabularies in order rather than one; unifying the rest is open design work, not settled. (#109; see `docs/usage.rst` "Titles you didn't configure") +**The period-abbreviation title inference runs at the head of the GIVEN-NAME part, not the head of the name** — an unrecognized multi-letter word ending in a single trailing period (`_pieces._PERIOD_ABBREV`, a hand copy of the `period_abbreviation` regex, `{2,}` letters — it was assign's until #424 and group's until #439) is treated as a title in the leading title run, e.g. `"Insp. Jane Morse"` → `title='Insp.'`. "Leading" is per SEGMENT: `_peel_leading_titles` is called for NO_COMMA segment 0, SUFFIX_COMMA segment 0, and FAMILY_COMMA **segment 1**, so `"Morse, Det. Insp. Jane"` → `title='Det. Insp.'` and a lone `"Smith, Xyz."` → `title='Xyz.'` — long-standing, verified against 1.4.0, and the mechanism behind #296 (`"Smith, Jr."` read title in 1.4.0; the comma's credential read takes it as suffix `Jr.` now). The docs said "leading word" until 2026-08-01 and were wrong for every comma path. FAMILY_COMMA **segment 0** peels too since #701, but by VOCABULARY only (rules/H6, in assign's family-comma branch rather than `_peel_leading_titles`): `Dr. Smith, John` → `title='Dr.'`, while `Xyz. Smith, John` keeps family `Xyz. Smith`, and a title-particle (`St`, `Freiherr`) stops the run, as does a title with nothing but post-nominals behind it (`Dr. Jr., John` keeps family `Dr.`); a title/suffix dual with a name word behind it still peels (`Rev. Sr. Smith, Mary`). It does not mutate `C.titles`, so the periodless form (`"Insp"`) is unaffected elsewhere. The `{2,}` length requirement — not a separate initials check — is what excludes single-letter initials like `"J."`; the same word after the given name is left as a middle name. **The inference OUTRANKS vocabulary where it runs**: `"Esq. Smith"` → `title='Esq.'` even though `esq` is suffix-only vocabulary, because the shape rule fires before anything consults the suffix sets. **The INFERENCE still runs in one direction only, and that is what the two slots share and where they part** (rewritten 2026-09-08, #316): a period-marked word is claimed by SHAPE at the front and by VOCABULARY at the back, so an unlisted abbreviation opening a name is a title while an unlisted abbreviation ending one is a NAME word (`"John Smith Xyz."` → `family='Xyz.'`). A trailing period-marked word the vocabulary knows as a title now IS one (`"John Smith Prof."` → `title='Prof.'`, `rules/H5`) — the comma path always read it that way and the two agree now — and a trailing period-marked word the SUFFIX vocabulary knows is still a suffix, the suffix run being peeled first (`"John Smith Esq."` → `suffix='Esq.'`). A BARE trailing title word stays a name word (`"John Smith Sir"`, `"Mary Jane King"`), which is the doctrine line below. Meanwhile `period_joined_vocab` resolves INTERIOR-period tokens (`Lt.Gov.`, `Msc.Ed.`) to title-or-suffix by vocabulary, and `_extract._suffix_shaped` treats any period-final delimited content as not-a-nickname. Four sites, and the trailing one now resolves by two vocabularies in order rather than one; unifying the rest is open design work, not settled. (#109; see `docs/usage.rst` "Titles you didn't configure") **Cyrillic suffix regexes need `re.I` even when the pattern is suffix-only** — a Latin title-cased word (`Ivanovich`) keeps its suffix lowercase, so `re.I` seemed skippable; but an irregular Cyrillic suffix can be nearly the whole word (`ильич`), so title-casing capitalizes into the suffix itself (`Ильич`). `east_slavic_patronymic_cyrillic` shipped without `re.I` on the Latin reasoning and silently failed on capitalized irregular forms — don't assume Latin's title-case safety transfers to Cyrillic. (#185) @@ -283,11 +285,11 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_ **Comparing against v1 needs `PYTHONSAFEPATH=1` AND a directory outside the worktree** — `uv run --isolated --no-project --with 'nameparser==1.4.0'` still puts the checkout's `nameparser/` ahead of the pinned wheel on `sys.path`, so the "v1" side silently imports the branch and every comparison reports parity. Run it as `cd && PYTHONSAFEPATH=1 uv run --isolated --no-project --with 'nameparser==1.4.0' python -c "..."`. This produces false confidence rather than an error, so it invalidates results without ever looking wrong. **A `tests/v2/cases.py` row's `parity` means 1.4.0, and nothing checks it** -- the table runs every row against the tree, so a row labelled `parity` that only matches master passes for good. #614 added eight such rows, compared against master; #620 found them by running the wheel this way and relabelled each with the change its reading came from, found by `git bisect run` in a scratch worktree (`git worktree add --detach`, never this one). Run the wheel before writing `parity`. -**A group-side reading of a family comma's given part that asks `tail_reading` disagrees with the given part's own walk, and the disagreement is accepted.** `tail_reading` reads the part as the comma-less name reads it; assign's walk over the given part after a family comma reads by its own predicates -- a shape-only numeral VI to X stays a name word there (`Smith, John van der Berg VI` reads middle 'van der Berg VI'), V before a third comma part is read strictly (#144), and P6 attaches a period-marked title-particle (`St.`) the comma-less reading takes as a title. So a bound or a decline group computes with `tail_reading` acts where the walk then reads no title: #623's chain bound (`Smith, John Mc Prof. VI`) and #637's join decline (`Smith, abdul Prof. VI` → middle 'Prof. VI', `Smith, abdul St.` → family 'St. Smith'). Both were recorded rather than narrowed (Derek: garbage input; decisions.md#H5 and #P5, rules.md#P5's Accepted clause). Before writing another group-side predictor of the given part, sweep those three shapes and decide, rather than rediscovering the gap in review. +**A group-side reading of a family comma's given part that asks `tail_reading` disagrees with the given part's own walk, and the disagreement is accepted.** `tail_reading` reads the part as the comma-less name reads it; assign's walk over the given part after a family comma reads by its own predicates -- a shape-only numeral VI to X stays a name word there (`Smith, John van der Berg VI` reads middle 'van der Berg VI'), V before a third comma part is read strictly (#144), and P6 attaches a period-marked title-particle (`St.`) the comma-less reading takes as a title. So a bound or a decline group computes with `tail_reading` acts where the walk then reads no title: #623's chain bound (`Smith, John Mc Prof. VI`) and #637's join decline (`Smith, abdul Prof. VI` → middle 'Prof. VI', `Smith, abdul St.` → family 'St. Smith'). Both were recorded rather than narrowed (Derek: garbage input; decisions/H5 and #P5, rules/P5's Accepted clause). Before writing another group-side predictor of the given part, sweep those three shapes and decide, rather than rediscovering the gap in review. **A comparison that runs both sides in one tree reports 0 differences, which is what success looks like.** `tools/differential/compare.py` has been hardened against this — it generates its baseline worker into a temp dir, strips `PYTHONPATH` from the child, and aborts unless both the version AND the resolved path check out on each side; its README's three invocation traps are the analysis behind that design, worth reading before changing the harness. **The exposure is ad-hoc comparisons you write yourself**, where the same collapse has a cause the harness cannot disarm for you: **the shell's working directory persists between tool calls**, so a two-tree comparison written as two `cd`s silently runs both halves in whichever tree it landed in. Pin absolute paths, and assert `nameparser.__file__` on both sides the way `compare.py` does. The general rule outlives any particular trap: before believing a null result, prove the harness can report a difference — a clean run and a broken harness are the same output. The proof has to come from the input class the null result is about: #683 moved `tools/name_usage.py` to a POST and checked it with `shehu`, which is ASCII and so cannot show that the server decodes a Cyrillic body. A misdecoded query still runs and matches nothing, so the Cyrillic zeros it was meant to vouch for proved nothing until `Иван` (17,119 given) went through the same path. The converse trap is a mutation that moves nothing: it shows the code is unexercised by your grids, not that it is dead. #638 was filed calling group's chain re-ask dead on a grid with no title-particle head before a particle run and a dotted acronym; the next review found 400 moves (`Smith, Freiherr van Berg X.Y.Z.`) and the issue closed the same day. Before calling code dead, look for the input that reaches it. -**`SUFFIX_ACRONYMS` and `SUFFIX_WORDS` are asserted DISJOINT at import (since 2026-09-08) — a bulk post-nominal import that re-creates an overlap fails there, and the fix is to choose a set, not to relax the assert** — the two branches normalize differently (the word test strips only edge periods, the acronym test strips all of them), so a word in both is matched by two rules and which one fired is unreadable from outside. `esq` was the single overlap and left `SUFFIX_ACRONYMS`, which is what made the assert possible; the whole algebra that had made the overlap safe, and the criterion that decided which set a word belongs to, are in the `suffix-acronym-collisions` entry of `docs/design/decisions.md` if anyone wants them. +**`SUFFIX_ACRONYMS` and `SUFFIX_WORDS` are asserted DISJOINT at import (since 2026-09-08) — a bulk post-nominal import that re-creates an overlap fails there, and the fix is to choose a set, not to relax the assert** — the two branches normalize differently (the word test strips only edge periods, the acronym test strips all of them), so a word in both is matched by two rules and which one fired is unreadable from outside. `esq` was the single overlap and left `SUFFIX_ACRONYMS`, which is what made the assert possible; the whole algebra that had made the overlap safe, and the criterion that decided which set a word belongs to, are in the `suffix-acronym-collisions` entry of `docs/design/decisions/` if anyone wants them. **`Lexicon.given_name_titles` is deliberately unvalidated against `titles` — do not add a check** — the lookup key is the space-joined run of `Role.TITLE` tokens, built by the parse — matched whole OR by that key's LAST word since #489, a run addressing the way its final title does — and a conjunction inside a run is itself tagged `Role.TITLE`, so `"sir and dame"` is a matchable key whose middle word lives in `conjunctions`. A whole-entry check rejected multi-word entries; a per-word check rejected that one. No static relation over the vocabulary sets decides reachability. An unreachable entry is inert — nothing consults it and nothing misparses — so the condition being "guarded" costs the user nothing while each guard cost a working configuration. If a diagnostic is wanted, it must be non-blocking — the multi-word UserWarning in `_normset` is the shipped example; a raise remains wrong. Note the SHIPPED data cannot carry a multi-word given-name title without a spurious warning: `GIVEN_NAME_TITLES ⊆ TITLES` puts the entry in `titles` too, which is per-word warned; user-supplied v2 Lexicons are unaffected (`add(given_name_titles=...)` alone is silent). @@ -323,7 +325,7 @@ Don't use the bare `python3 -m doctest .rst` CLI (no `optionflags`) to che **Prefix-join uses value-based `list.index()`** in `join_on_conjunctions` — fragile when a token value repeats (e.g. a trailing title that's also a suffix acronym, or two `van`s); constrain such lookups to start at `i + 1`. See #100. -**Title vs suffix is positional for BARE words, and the leading period-abbreviation rule overrides even that** — a word matching `TITLES` at the front of a name becomes `title`; the same word matching `SUFFIX_ACRONYMS`/`SUFFIX_WORDS` at the end becomes `suffix` (never both, regardless of the word's real-world meaning). The `TITLES`/suffix overlap was audited in #296 (2026-08-23): the pure postnominals (`jr`, `junior`, `phd`, `do`, `se`) left `TITLES`, the v1-residue `dr`/`sra` left the suffix sets, and the twelve words still in both (`md`, `ms`, `sa`, `sr`, `lt`, `ra`, `vc`, and the ranks `cpl`, `cpo`, `cpt`, `csm`, `sgm`) are deliberate duals that position decides. External test sources (old issue gists, etc.) sometimes assert `suffix` for a leading professional abbreviation like `RA`/`PD`/`Dipl.-Ing.` — that's the source data being wrong, not a parser bug. Verify position before "fixing" it. Two qualifications the older "purely positional" wording papered over, the first measured 2026-08-01 and the second answered 2026-09-08 (#316): a PERIOD-marked leading word is claimed by the shape rule before any vocabulary is read (`"Esq. Smith"` → `title`, though `esq` is suffix-only), and a PERIOD-marked trailing word is claimed by VOCABULARY — first the suffix sets, which the peel reads before anything else (`"John Smith Esq."` → `suffix`), then the titles (`"John Smith Prof."` → `title='Prof.'`, `rules.md#H5`) — while an unlisted abbreviation there stays a name part (`"John Smith Xyz."` → `family='Xyz.'`) unless it is written as two or more period-separated chunks, which since 2.4 IS a trailing shape rule (`Policy.unlisted_dotted_suffixes`, default on: `"John Smith X.Y.Z."` → `suffix='X.Y.Z.'`, read by the same words-to-spare count a bare ambiguous acronym takes, `rules.md#S3`). The SINGLE trailing period is still not one, deliberately — it is the abbreviation shape any word can wear. The bare word is where "positional" still holds whole -- outside a part past the second comma, which holds no name word, so C2 reads a bare title word there as a title since #603 (`Eric H. Holder, Jr., Attorney General`) -- and the reason it must: `TITLES` holds words that are in no suffix set — 787 of them on 2026-10-11 (746 before #606's salutations), measured with `L = Parser().lexicon; len(L.titles - L.suffix_acronyms - L.suffix_words)`, a figure that grows with the vocabulary and never shrinks the argument — and many are ordinary surnames (`king`, `bishop`, `prince`, `pope`, `judge`, `sheriff`, `baron`, `master`, ...), so a vocabulary-first rule over BARE trailing words would read `"Mary Jane King"` as `title='King'`, `family='Jane'`. The period is what separates the safe case from that one, and it separates it by being a WRITING convention rather than by making the collision go away: `"Mary Jane King"` keeps `family='King'` while `"Mary Jane King."` reads `title='King.'`, `family='Jane'`, a cost accepted under the input-is-a-name premise rather than one the rule prevents (decisions.md#P5's trailing-position bullet, measured 2026-09-09). Since #316 that sentence describes the shipped rule rather than an aspiration. +**Title vs suffix is positional for BARE words, and the leading period-abbreviation rule overrides even that** — a word matching `TITLES` at the front of a name becomes `title`; the same word matching `SUFFIX_ACRONYMS`/`SUFFIX_WORDS` at the end becomes `suffix` (never both, regardless of the word's real-world meaning). The `TITLES`/suffix overlap was audited in #296 (2026-08-23): the pure postnominals (`jr`, `junior`, `phd`, `do`, `se`) left `TITLES`, the v1-residue `dr`/`sra` left the suffix sets, and the twelve words still in both (`md`, `ms`, `sa`, `sr`, `lt`, `ra`, `vc`, and the ranks `cpl`, `cpo`, `cpt`, `csm`, `sgm`) are deliberate duals that position decides. External test sources (old issue gists, etc.) sometimes assert `suffix` for a leading professional abbreviation like `RA`/`PD`/`Dipl.-Ing.` — that's the source data being wrong, not a parser bug. Verify position before "fixing" it. Two qualifications the older "purely positional" wording papered over, the first measured 2026-08-01 and the second answered 2026-09-08 (#316): a PERIOD-marked leading word is claimed by the shape rule before any vocabulary is read (`"Esq. Smith"` → `title`, though `esq` is suffix-only), and a PERIOD-marked trailing word is claimed by VOCABULARY — first the suffix sets, which the peel reads before anything else (`"John Smith Esq."` → `suffix`), then the titles (`"John Smith Prof."` → `title='Prof.'`, `rules/H5`) — while an unlisted abbreviation there stays a name part (`"John Smith Xyz."` → `family='Xyz.'`) unless it is written as two or more period-separated chunks, which since 2.4 IS a trailing shape rule (`Policy.unlisted_dotted_suffixes`, default on: `"John Smith X.Y.Z."` → `suffix='X.Y.Z.'`, read by the same words-to-spare count a bare ambiguous acronym takes, `rules/S3`). The SINGLE trailing period is still not one, deliberately — it is the abbreviation shape any word can wear. The bare word is where "positional" still holds whole -- outside a part past the second comma, which holds no name word, so C2 reads a bare title word there as a title since #603 (`Eric H. Holder, Jr., Attorney General`) -- and the reason it must: `TITLES` holds words that are in no suffix set — 787 of them on 2026-10-11 (746 before #606's salutations), measured with `L = Parser().lexicon; len(L.titles - L.suffix_acronyms - L.suffix_words)`, a figure that grows with the vocabulary and never shrinks the argument — and many are ordinary surnames (`king`, `bishop`, `prince`, `pope`, `judge`, `sheriff`, `baron`, `master`, ...), so a vocabulary-first rule over BARE trailing words would read `"Mary Jane King"` as `title='King'`, `family='Jane'`. The period is what separates the safe case from that one, and it separates it by being a WRITING convention rather than by making the collision go away: `"Mary Jane King"` keeps `family='King'` while `"Mary Jane King."` reads `title='King.'`, `family='Jane'`, a cost accepted under the input-is-a-name premise rather than one the rule prevents (the trailing-position bullet in decisions/vocabulary-exclusions, measured 2026-09-09). Since #316 that sentence describes the shipped rule rather than an aspiration. ### Tests (`tests/`) diff --git a/docs/customize.rst b/docs/customize.rst index 6a330fd1..3b0e345e 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -273,8 +273,8 @@ The exception is an acronym that is also a name word, such as ``meng`` or ``edd``. A mask applies wherever its word stands, so it would re-spell a person called Meng or Edd, and these get none: they repair in plain capitals. The reasoning is the Excluded block for -``CAPITALIZATION_EXCEPTIONS`` under ``R4`` in -``docs/design/decisions.md``. +``CAPITALIZATION_EXCEPTIONS`` (meng, edd, lac, ded) in +``docs/design/decisions/vocabulary-exclusions.md``. .. doctest:: diff --git a/docs/design/AGENTS.md b/docs/design/AGENTS.md index fb039a15..f8837f2f 100644 --- a/docs/design/AGENTS.md +++ b/docs/design/AGENTS.md @@ -2,34 +2,36 @@ Loaded when a session touches these documents; the root AGENTS.md carries what fires from elsewhere (the same-PR amendment rule, prose counting claims, release-log claims, backlog triage, markdown line width, guard-test negative controls). -rules.md states intended parsing behavior and is NORMATIVE; decisions.md is the dated ADR log behind it; mechanisms.md catalogs reusable patterns and verification shapes. Each declares its own conventions at the top — read those before describing what one of them contains (axis 4). +rules/ states intended parsing behavior and is NORMATIVE; decisions/ is the dated ADR log behind it; mechanisms.md catalogs reusable patterns and verification shapes. rules/ and decisions/ hold one named file per entry, indexed by their README.md. Each declares its own conventions at the top (for the two folders, in the README) — read those before describing what one of them contains (axis 4). **Starting a design: pin the negative space first.** A spec's "X is not supported" is a claim about today's parser; hold it with a case row for X written BEFORE the implementation, so a row that passes unexpectedly kills the premise before any code exists. The 2026-07-30 comma-suffix spec did this for one item (`suffix_phrase_is_comma_scoped`) and not for its central premise, which is the half that cost a full plan (#291). -**Rethinking a rule that grew by patching.** Three signs together mean the next fix should be a rethink, not another patch: the rule works by scanning and stopping; each later fix added a stop, an exception or a check on one; and a guard exists whose job is to model what another stage would do. The third sign is the decisive one — a rule that must predict other rules' behavior to stay correct is defending an invariant it should be built from. #601 is the worked case. Five issues (#424, #533, #535, #538, #549) had each added a stop or a release check to M2, and its decisions.md section reached about 13,000 words. The rethink (mechanisms.md#READ-WITHOUT-THEN-BIND, #LICENSED-BY-NEIGHBOR) deleted the release machinery, cut the rule's statement to 883 words (counted 2026-10-04), and turned the invariant from a checked property into the construction. The rethink was drafted in rule shape and prototyped in seven variants behind one switch before anything landed (#601's comments); the variants that failed did so on measured counts, not on review opinion. Rule length alone is not the signal: S2 and R4 have each gathered 35 dated decisions.md entries, and R4's are mostly vocabulary and rendering cases rather than added stops (counted 2026-10-04). +**Rethinking a rule that grew by patching.** Three signs together mean the next fix should be a rethink, not another patch: the rule works by scanning and stopping; each later fix added a stop, an exception or a check on one; and a guard exists whose job is to model what another stage would do. The third sign is the decisive one — a rule that must predict other rules' behavior to stay correct is defending an invariant it should be built from. #601 is the worked case. Five issues (#424, #533, #535, #538, #549) had each added a stop or a release check to M2, and its decisions.md section reached about 13,000 words. The rethink (mechanisms.md#READ-WITHOUT-THEN-BIND, #LICENSED-BY-NEIGHBOR) deleted the release machinery, cut the rule's statement to 883 words (counted 2026-10-04), and turned the invariant from a checked property into the construction. The rethink was drafted in rule shape and prototyped in seven variants behind one switch before anything landed (#601's comments); the variants that failed did so on measured counts, not on review opinion. Rule length alone is not the signal: S2 and R4 have each gathered 35 dated decisions entries, and R4's are mostly vocabulary and rendering cases rather than added stops (counted 2026-10-04). -**Landing a design.** The gitignored spec (docs/superpowers/specs/) is the working medium; it dies with the branch, and the docs are the record. Before a design PR merges, walk its spec (including amendments) and distill the durable residue: decisions made or reversed → decisions.md entries; proposals rejected with evidence → Declined:; vocabulary that must stay out → Excluded:; behavior the design settled → rules.md (with a deviates: marker if unshipped); reusable patterns → mechanisms.md; options weighed → a weighing entry. Then check the spec cites nothing the docs don't now carry — a spec section with no committed home when the PR merges is lost, not deferred (a 2026-08-16 sweep of eight weeks of specs recovered nine such items). The same-PR amendment rule above covers code-driven changes; this covers the design-driven ones. +**Landing a design.** The gitignored spec (docs/superpowers/specs/) is the working medium; it dies with the branch, and the docs are the record. Before a design PR merges, walk its spec (including amendments) and distill the durable residue: decisions made or reversed → decisions entries; proposals rejected with evidence → Declined:; vocabulary that must stay out → Excluded:; behavior the design settled → rules/ (with a deviates: marker if unshipped); reusable patterns → mechanisms.md; options weighed → a weighing entry. Then check the spec cites nothing the docs don't now carry — a spec section with no committed home when the PR merges is lost, not deferred (a 2026-08-16 sweep of eight weeks of specs recovered nine such items). The same-PR amendment rule above covers code-driven changes; this covers the design-driven ones. -**A count in a dated entry is evidence, not a live fact.** decisions.md entries are snapshots by convention, so measurements belong in them — but a reader wanting TODAY's number must not have to trust the snapshot's date. Where an entry quotes something that drifts (vocabulary sizes, corpus counts, set compositions), give the one-liner that recomputes it, and phrase the argument so it survives the digits moving — "the two shares differ by orders of magnitude" outlives "58% vs 0.65%". A count that carries no argument is better deleted than dated: "over every name in the corpus, no prefilter" says what "over all 782 names" says, and cannot go stale. Do NOT reach for a test asserting the count — that is the constant-content pattern, and it fails on every legitimate vocabulary addition. #326 is the cautionary case: it quoted a vocabulary composition, carried a date, and was stale in five days. +**Moving text between files breaks every positional reference in it, and in what pointed at it.** "The bullet above", "this file's Excluded block", "the W2 section" are true only where the text stood. #713 split decisions.md by heading and then moved the vocabulary-keyed blocks into decisions/vocabulary-exclusions, and its last review found thirteen such references left pointing at nothing: three inside the moved blocks, ten in entries elsewhere that located a block by position. The citation check cannot see them, because none is a citation. After a move, grep the moved text and the whole folder for `above`, `below`, `this file`, `this section` and the moved block's name, and turn each hit into a key-form pointer (`decisions/P3's bullet of that date`). + +**A count in a dated entry is evidence, not a live fact.** decisions entries are snapshots by convention, so measurements belong in them — but a reader wanting TODAY's number must not have to trust the snapshot's date. Where an entry quotes something that drifts (vocabulary sizes, corpus counts, set compositions), give the one-liner that recomputes it, and phrase the argument so it survives the digits moving — "the two shares differ by orders of magnitude" outlives "58% vs 0.65%". A count that carries no argument is better deleted than dated: "over every name in the corpus, no prefilter" says what "over all 782 names" says, and cannot go stale. Do NOT reach for a test asserting the count — that is the constant-content pattern, and it fails on every legitimate vocabulary addition. #326 is the cautionary case: it quoted a vocabulary composition, carried a date, and was stale in five days. **Primary-source review.** When doc content is distilled from a session's work, have that session (or its transcript) review its own sections before or soon after landing — attribution flattening and inverted arguments are visible only to the source. Reviewers state which tree each measurement ran on (stale fetches produced three rounds of already-fixed findings), and landed corrections are re-verified here before committing. Two shapes, needing different instruments. CROSS-SESSION distillation is what the rule above is written for — the #381 arc harvested nine named sessions, and for each the source transcript is an artifact independent of the reviewing session, so "review your own sections" is a two-artifact comparison. SAME-SESSION design work (#386) has no such artifact: spec and docs were written in one context, so the same instruction degenerates into re-reading the working memory that produced any error, and the rationale inverts — when the source is NOW, the source is what got it wrong. There, compare against artifacts instead, in this measured order of yield: re-derive every number with a freshly written script (2 wrong claims on #386); re-read the written spec against a checklist for FIDELITY, not just presence (1); interrogate NAMED contested rule pairs one at a time (2). Unaided prose reading found none — so a general "coherence read" by the author is the one form to distrust, because a green report from the weakest instrument manufactures confidence rather than supplying it. -**Reviewing a docs/design change: the eleven axes.** Run these before merging anything that touches rules.md, decisions.md, mechanisms.md — or THIS FILE, which carries the conventions those three rest on and has the same failure modes. Axes 3, 5 and 6 are rules.md-specific (they need rule statements, examples and `interacts:`); the other eight apply to any of the four — axis 11 included, since an invariant over two inputs is prompted by what a rule implies but is RUN without reading one, which is the whole of its value. Say which axes you ran and which you skipped as inapplicable — a checklist that asks for something meaningless teaches the reader to skim it. Axes 1–9 were each earned by a defect that survived every other check on #386, where the review passes surfaced ten issues in total — nine of them by these axes, one by the spec self-review above — and unaided prose reading surfaced none. Axes 10 and 11 were earned the same way on other arcs — #291's, which PREDATES this list, and #461/#466's, which follows it. The parenthetical is the case that earned the axis. +**Reviewing a docs/design change: the eleven axes.** Run these before merging anything that touches rules/, decisions/, mechanisms.md — or THIS FILE, which carries the conventions those three rest on and has the same failure modes. Axes 3, 5 and 6 are rules-specific (they need rule statements, examples and `interacts:`); the other eight apply to any of the four — axis 11 included, since an invariant over two inputs is prompted by what a rule implies but is RUN without reading one, which is the whole of its value. Say which axes you ran and which you skipped as inapplicable — a checklist that asks for something meaningless teaches the reader to skim it. Axes 1–9 were each earned by a defect that survived every other check on #386, where the review passes surfaced ten issues in total — nine of them by these axes, one by the spec self-review above — and unaided prose reading surfaced none. Axes 10 and 11 were earned the same way on other arcs — #291's, which PREDATES this list, and #461/#466's, which follows it. The parenthetical is the case that earned the axis. -Skip what the suite already enforces. `tests/v2/test_rules_doc.py` and `tests/v2/test_doc_citations.py` machine-check example lines and their `deviates:` today-values, boundaries, the one-of `implemented:`/`tracked:` rule, `implemented:` against the modules that actually cite the rule, `interacts:` and cross-doc anchor IDs, citation excerpts quoted verbatim, and legacy citation forms. Assume they pass; a finding one of them would catch is a false positive. Read those two files rather than trusting this sentence — it named five of the nine guards when written, and goes stale as guards are added. +Skip what the suite already enforces. `tests/v2/test_rules_doc.py`, `tests/v2/test_doc_citations.py` and `tests/v2/test_design_docs.py` machine-check example lines and their `deviates:` today-values, boundaries, the one-of `implemented:`/`tracked:` rule, `implemented:` against the modules that actually cite the rule, `interacts:` and cross-doc anchor IDs, citation excerpts quoted verbatim, legacy citation forms (the pre-#713 file-anchor form among them), and that every rule and decisions file is indexed once, under a title matching its heading and file name. Assume they pass; a finding one of them would catch is a false positive. Read those three files rather than trusting this sentence — it named five of the nine guards when written, and goes stale as guards are added. 1. **Recompute every number.** Any count, ratio or "N names" in the diff gets recomputed by a script written now, never read. (A filter described as returning one name returned three; a rule's stated blast radius listed two names its own scope excluded.) 2. **Your detector is a second, unreviewed implementation.** It must read the same vocabulary and boundaries the rule reads. (A particle-run detector that walked only the never-given half — the rule chains through ANY particle — reported 50 false movers where the true count was 1, printing cleanly both times.) 3. **Rule vs. its own examples.** (P1 said the fold takes "the particle and the one name word"; its example `de la Vega` is two particles onto one word.) -4. **Any claim about a document vs. what that document says about itself.** Two directions, both bitten. Inward: follow every `history:` and read the entry (P6's rationale asserted what decisions.md#vocabulary-collisions denies, and the rule's own guard exists because the assertion is false). Outward: when prose anywhere — this file, a code comment, another design doc — describes what one of these files contains or does, read that file's preamble or entry conventions first (an AGENTS.md bullet claimed rules.md names the pipeline stages; its preamble forbids exactly that — "Statements are implementation-free: no stage names, no function names, no regexes"). Those declarations sit at the top of files people cite without reading. Mind their scope before calling a violation: rules.md's ban binds STATEMENTS, which `_statements()` in test_doc_citations.py defines as rule-block text up to the first example, so Backgrounds and Not-in-scope may name functions and do. +4. **Any claim about a document vs. what that document says about itself.** Two directions, both bitten. Inward: follow every `history:` and read the entry (P6's rationale asserted what decisions/vocabulary-collisions denies, and the rule's own guard exists because the assertion is false). Outward: when prose anywhere — this file, a code comment, another design doc — describes what one of these files contains or does, read that file's preamble or entry conventions first (an AGENTS.md bullet claimed the rules name the pipeline stages; their preamble forbids exactly that — "Statements are implementation-free: no stage names, no function names, no regexes"). Those declarations sit at the top of files people cite without reading. Mind their scope before calling a violation: the rules' ban binds STATEMENTS, which `_statements()` in test_doc_citations.py defines as rule-block text up to the first example, so Backgrounds and Not-in-scope may name functions and do. 5. **Does the change move another rule's OUTPUT?** A marker lands on the rule whose STATEMENT changed; a rule can move another's output without touching its statement, and the runner cannot see it. Walk `interacts:`. (family_base moved initials(); R3 had no marker.) 6. **Contested inputs: is precedence stated?** `interacts:` is advisory and pins nothing. This document states precedence in the rule's own statement (H2, M1, W3). (P6 and S2 both claimed `Berg, Jan vd`.) 7. **General clause vs. adjudicated scope.** Enumerate the vocabulary a shape-clause reaches. (P6's precedence was argued for `vd` and swept in `do` and `mc`.) 8. **Guard docstring vs. what the guard enforces**, especially when the promise spans two test modules. 9. **Prose is input to the doc parsers.** A line starting with `"` inside a rule block is an example; a comment's quoted values join the citation block above them. 10. **Re-run every capability claim.** Axis 1 recomputes numbers; this one recomputes CAN and CANNOT. A statement that the parser never or cannot produce some reading is measured by a parse run written now, never derived from a mechanism — most of all when it justifies building something. (#291 derived "a multi-word credential is unparseable" from "vocabulary is matched one token at a time" — which `given_name_titles` had been an exception to since 2026-07-19, a month before that arc, and `maiden_markers` is a second since #434 — and specified a vocabulary set, a matching unit, a predicate and a `Lexicon` field on it; `parse("John Smith, MD PhD").suffix` had been `'MD PhD'` since 1.4.0.) -11. **State the invariant that relates two DIFFERENT inputs.** Axes 1–10 compare a document to another document, to its own examples, or to the implementation, so every one of them can only find code that DISAGREES with a rule — none of them can find a rule that is simply wrong. When a rule implies that two spellings of the same name must parse alike, write that as an invariant over both inputs and run it: it consults no rule statement, so it is not blind that way. Distinct from an invariant over two VIEWS of one parse, like `test_the_family_partitions_into_particles_and_base` in tests/v2/test_cases.py — that one holds however wrong the rule is. (Two review rounds a month apart found real defects and missed the premise under them: #461 was backed out after the rule turned out wrong, #466 closed unmerged for the same reason, and in both, every reviewer checked the code against the rule and THE RULE was at fault. The worked invariant: the family-comma writing under the default order and the comma-less writing under FAMILY_FIRST must parse alike. Measured, it holds for the great majority of the particle vocabulary and fails exactly on the words that are also suffix vocabulary — recovering P6's contested-membership set from behavior, having read no rule. Composition, measurement and recompute recipe: decisions.md#two-input-invariants.) +11. **State the invariant that relates two DIFFERENT inputs.** Axes 1–10 compare a document to another document, to its own examples, or to the implementation, so every one of them can only find code that DISAGREES with a rule — none of them can find a rule that is simply wrong. When a rule implies that two spellings of the same name must parse alike, write that as an invariant over both inputs and run it: it consults no rule statement, so it is not blind that way. Distinct from an invariant over two VIEWS of one parse, like `test_the_family_partitions_into_particles_and_base` in tests/v2/test_cases.py — that one holds however wrong the rule is. (Two review rounds a month apart found real defects and missed the premise under them: #461 was backed out after the rule turned out wrong, #466 closed unmerged for the same reason, and in both, every reviewer checked the code against the rule and THE RULE was at fault. The worked invariant: the family-comma writing under the default order and the comma-less writing under FAMILY_FIRST must parse alike. Measured, it holds for the great majority of the particle vocabulary and fails exactly on the words that are also suffix vocabulary — recovering P6's contested-membership set from behavior, having read no rule. Composition, measurement and recompute recipe: decisions/two-input-invariants.) -Also: before reporting "N names move", report the size of the population that COULD move — a small count over a corpus blind to the shape is evidence about the corpus. For rules.md#P6 the population is the comma-bearing names that END in a particle — a small fraction of the comma-bearing names, which are themselves under a third of the corpus; decisions.md#P6 carries the dated measurement and the recompute. Do not restate a corpus count here. Axis 1 over this very sentence once caught it conflating the population with the movers, and the corrected figures that replaced it were superseded within the month as the corpus grew — a standing count in this file is wrong twice over, once for the class and once for the digits. +Also: before reporting "N names move", report the size of the population that COULD move — a small count over a corpus blind to the shape is evidence about the corpus. For rules/P6 the population is the comma-bearing names that END in a particle — a small fraction of the comma-bearing names, which are themselves under a third of the corpus; decisions/P6 carries the dated measurement and the recompute. Do not restate a corpus count here. Axis 1 over this very sentence once caught it conflating the population with the movers, and the corrected figures that replaced it were superseded within the month as the corpus grew — a standing count in this file is wrong twice over, once for the class and once for the digits. diff --git a/docs/design/decisions.md b/docs/design/decisions.md deleted file mode 100644 index 2ce2ef6c..00000000 --- a/docs/design/decisions.md +++ /dev/null @@ -1,1778 +0,0 @@ -# Decisions - -The why behind the rules: a lightweight ADR log, keyed by rule ID (`### P2 — `), by mechanism slug, or by a short slug for cross-cutting and tooling decisions. Section headings carry the key, so every `decisions.md#P2` reference in code or rules.md is a live anchor. - -The gitignored planning specs are the working medium and die with their branches; this file is the record. A landing design PR distills its spec's durable residue here (the checklist is AGENTS.md's "Landing a design"). - -Entry conventions: - -- **Dated decision entries**, append-only in spirit: a reversed decision is not edited, a later entry supersedes it. Each entry cites its issue or PR. The date is the DECISION's, never the release's (add "shipped in X" alongside if useful), and git author dates outrank remembered ones — two review rounds corrected exactly these two errors. -- **Harvested entries** (content landed from a session's report) keep their provenance: a "measurements are that session's, spot-checked at landing" framing line, the contributor's measured-vs-remembered markings where they matter, and — the hard-won one — a flagged uncertainty is resolved by reading the source artifact, never by inference from the neighboring argument. -- **`Declined:`** — proposals rejected WITH the evidence that killed them. Resolved-as-no is a decision; without a home for it, the next person re-derives the rejected proposal and its measurement. -- **`Excluded:`** — standing prohibitions with indefinite lifetime, keyed by vocabulary set: entries that must stay OUT of a wordlist, each with its reason. Distinct from Declined because the failure mode differs — nobody re-derives a rejected proposal, but someone sweeping a wordlist ships the excluded entry as a bug. -- **`Open:`** — unresolved questions as issue links with one-line handles. The ISSUE is canonical; this block never restates it. Two keyings: under a rule ID for questions about the rule, and — like `Excluded:` — keyed to a VOCABULARY SET for contested memberships, the category neither `deviates:` nor `Excluded:` covers: the rule is right and a word's set membership is the question (the shape: `rai` in the suffix acronyms and `swami` absent from the given-name titles, both since answered — see #suffix-acronym-collisions and #indic-honorifics). Place the block beside the set's `Excluded:` entries so a wordlist editor meets both. -- **Weighing entries** for contested questions: the options considered, each option's intended effect, and the accepted costs of the option chosen. The costs accepted here are the artifacts rules.md lists under the rule's `Accepted:` consequences; the two link by rule ID. - -### A1 — the ambiguity report - -- 2026-09-20 #397 (second review) — A REPORT MUST NOT CONTRADICT THE READING BESIDE IT, and the shape could not arise before this cycle. `classify`'s `conjunction-or-initial` fork offers exactly two readings and its detail says which it took ("it is read as an initial"). Where the rest of the parse then roles that letter SUFFIX, the fork resolved to NEITHER branch: `JOHN QUINCY SMITH I` carried the connective-or-initial beside a `suffix-or-name` saying the same token reads as a generational suffix, and `john smith i`, `HENRY I`, `SMITH, JOHN I`, `JOSEP CAROD I ROVIRA III` and `abdul e i` were the same. `i` is the first word that is both a MARKED connective and suffix vocabulary, so no parse before #397 could reach it — which is why the rule is written down now rather than having been. - DECIDED: the report is WITHDRAWN where the role is SUFFIX or TITLE, in `_pipeline/_assemble.py`, the one place that sees final roles and the ambiguity list together — and beside a withdrawal that was already there, for a report whose referents did not survive assembly. That is the narrow reading of mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE rather than an exception to it: classify still owns the fork and still takes it where it is taken; what goes is a report whose subject the parse went on to read as something else. Emitting later was the alternative and was rejected as more invasive for the same answer — the fork's inputs (the case class, the own-words span) live in classify and moving the emitter moves them. - MEASURED: over the review's 43,925-name grid under eight configurations, 9,806 rows on 1,637 distinct names lose exactly one `conjunction-or-initial` and nothing else — no report of any other kind moves and none is gained. On the deduped corpus-union-cases population the only name affected is `JOHN QUINCY SMITH I`, a row this round added, so the ledger's three-name `fix(#397) the Catalan link reports a connective-or-initial in a one-case name` rule is untouched: all three of its names role the letter MIDDLE and all three still report. The invariant is `tests/v2/test_properties.py::test_no_report_contradicts_the_reading_beside_it`, which states it twice — no connective-or-initial on a token roled SUFFIX or TITLE, and no token named by two reports whose details assert different readings — and fails on 8,204 rows at `dc3bdf9c`, 0 here. -- 2026-10-08 #626 — A REPORT NAMES THE FIELD ITS WORD LANDS IN. Six of assign's reports worded the field from the role the word held at assign (`read as a given name`), and rules in post_rules still move a word after that: H1 moves the name behind a title to the family, P1's family-first fold takes the particle run and one unit and lays the rest out by the order, P6's attachment takes a trailing name word into the family at the end of a name read family-first, and under opt-in policies the patronymic rotations (O1, O2) and `middle_as_family`'s fold (O3) move one too. So `Attorney General of Minnesota` reported 'General of Minnesota' "read as a given name by convention" with the unit in the family (since 2.3.0, #491's join clause); `Kim Min Do` under FAMILY_FIRST told the caller 'Do' was "read as a middle name" with 'Do' in the family, and `de Kim Ma` under FAMILY_FIRST said 'Ma' was a middle name with 'Ma' the given name (both this cycle, since #289 made the Title-case word a name word); `, Ma Dr.` said 'Ma' after the comma was "read as the given name" with 'Ma' in the family (this cycle, since #289 added that comma report; 2.3.0 reads the same fields and reports nothing); and `Van Ivan Petrovich` under the East Slavic patronymic rule said the leading 'Van' was "read as a given name" after O1 rotated it into the family (at 2.3.0 too). #614's second review had fixed one instance of the class (`Freiherr von Berg Ma`) by moving its report to the stage that knows the outcome. - DECIDED: the DECISION stays at its site and the WORDING of the field moves to assemble, the one place that knows final roles (the same reasoning as the 2026-09-20 withdrawal above). An emitter whose detail names a field a later rule could change writes the sentence up to the field and hands the rest as `PendingAmbiguity.field_tail` (`""` where the field ends the sentence); assemble joins them around `FIELD_READING` of the first referent's final role. All six such emitters in assign use it -- H4's all-suffix name, title-or-name's join clause, O5's lone word, the trailing peel's picks, the leading ambiguous particle and the comma report on the word after a family comma -- including those right on every measured input, so no later rule can make them wrong. A report in a name emptied for want of content is withdrawn rather than shipped half-worded (no emitter reaches that today). A placeholder in the detail was declined: token text can carry any characters, and a name containing the placeholder would corrupt its own report. Reports worded where the field is settled keep their literal wording: a comma's credential reading, the suffix run's and the numeral fork's, H4's "the name" and P6's own "joins the family". The comma report's given-side wording moves from "the given name" to "a given name" with the rest. - MISSED, then caught in review: the first draft converted five emitters and its grid had no empty-head comma with a trailing title, the shape AGENTS.md's comma sweep already asks for; four of the review's five readers found the sixth emitter on `, Ma Dr.`. The draft also named only H1 and P6 as movers; P1's fold was found by stepping the stages, and the review of the fix found the opt-in movers O1 to O3, which no grid over the three orders reaches -- reverting the leading-particle emitter passed the whole suite until rows under those policies pinned it. - MEASURED 2026-10-08, each report's field phrase compared against its tokens' final roles. RECIPE: texts are the 1,536 names of `tools/differential/corpus*.jsonl` and 347 `tests/v2/cases.py` texts (the table at 0c54662b, the parent of #614's first commit, less the corpus names; today's table adds rows this entry's review wrote for the shapes the grid missed, so it reads more wrong claims on master, 6 more as measured on the review), plus every head in {'John', 'John Smith', 'John Q. Smith', 'Mary Ann Smith', 'John van Smith', 'Juan de la Vega', 'John van', 'anh van', 'Jan Freiherr von Berg', 'Freiherr von Berg', 'Freiherr St', 'St', 'Dr. St', 'Freiherr', 'Dr.', 'Mr. and Mrs.', 'Attorney General of', 'abdul salam', 'Juan y Garcia', 'Jane Doe nee van der Berg', 'Dr. John Smith', 'JOHN SMITH', 'john smith', 'Smith', 'de Mesnil', 'Dr. de Mesnil', 'Kim Min', 'Kim Min Jun', 'Ortega y Gasset', 'van', 'Do', ''} followed by one or two words (repetition allowed) from the list [PhD, MA, Ma, DO, Do, do, vd, mc, Jr, III, V, V., VI, X.Y.Z., Ph. D., Esq., Prof., Sir, Ed, van, and, Jones, ba, John, Smith, Minnesota, de, St, Freiherr], space-joined and stripped; 20,000 draws after Python's `random.seed(626)`, each `random.choice(heads)`, then `random.randint(3, 4)` words each `random.choice(words)`, space-joined and stripped, both lists in the order written here; then, unstripped, `{head}, {word}`, `Smith, {head} {word}` with any double space replaced by one, and `{word}, {head}` for every head and word, and `, {w1} {w2}` and `(Bob), {w1} {w2}` for every ordered pair of words; the empty text dropped and duplicates removed -- 53,517 texts; each parsed under the three orders, so the opt-in movers are outside these figures; a claim is a match of `(reads? as|joins) ()` for the phrases `_FIELD_CLAIMS` in tests/v2/test_cases.py lists plus "the given name", and it is wrong when its tokens' roles are not among the fields the phrase names. On master (26cdb891) 2,725 of 89,092 claims were wrong, in three emitters: title-or-name's join clause (1,575), the trailing peel's picks (1,114) and the comma report (36); on the tree, 0. The comparator is master, which words the field at assign; the tree words it at assemble. Fingerprint against master over the differential corpora, master's case-table names and a seeded comma grid (13,090 names) under six policies (the three orders, `lenient_comma_suffixes=False`, a `/` suffix delimiter and `CapsSuffixes.OFF`): no role, kind, initials or rendering moves; the details that change are 6,395 comma reports on 1,086 names reworded from "the given name" to "a given name", 85 on 15 names now naming the family they land in, and 40 title-or-name join reports on 10 names now naming the family. Pinned by `test_a_report_names_the_field_its_word_lands_in` in tests/v2/test_cases.py, over every case row under all three orders except the locale rows (a row with its own policy only as declared), whose recorded control is 14 failures on master, and by `test_the_field_sweep_sees_the_claims_it_checks`, which records how many claims that sweep checks; the opt-in movers by the rows `Van Ivan Petrovich` (East Slavic), `Van Ali Veli oglu` (Turkic) and `Do Bishop Do` (FAMILY_FIRST with `middle_as_family`), which the sweep reads as declared. - -- 2026-10-10 #629 — A REPORT ON WORDS IN TWO FIELDS NAMES NEITHER. Segment's `comma-structure` report on a part past the second comma said the part was "consumed as suffix best-effort", true through 2.3.0, when the whole part read as suffix. Since #603 (rules.md#C2) a title word there reads as a title, so `John Smith, Jr., Dr. Bart` was told 'Dr.' was a suffix with 'Dr.' in the title (this cycle, never shipped). - DECIDED: the report names no field: "segment 'Dr. Bart' beyond the recognized comma structures; read best-effort". It says what is certain, that the part is outside the recognized comma structures, and the fields say where each word went. The input is malformed, so the report is not worth machinery: the fix is one string, and a part that reads wholly as suffix loses the word "suffix" from its detail, which is not contract (kinds are). - Declined: naming every field the part's words land in, worded at assemble from the final roles like #626's `field_tail` ("consumed as title and suffix best-effort"). It was built (0d5fea40, PR #630) and reviewed: it needed a second tail on `PendingAmbiguity`, an assemble branch and an equality check in the field guard, and the branch withdrew the report from a name emptied of content (`", , -"`), against rules.md#A2's "ambiguities born from its punctuation survive on the empty result", which no grid caught because every grid word had letters. More code to say more about garbage input; the user-facing gain did not pay for it. - MEASURED 2026-10-10, tree against master bc243869 over the names of `tools/differential/corpus*.jsonl` plus every `tests/v2/cases.py` text, each under the three orders, comparing every report's kind, detail and referents and the seven fields: only the detail of `comma-structure` reports moves, and no report appears or goes. - -### A2 — the empty name - -- 2026-07 (v2 core, PR #288) — v1 kept parse(".") as first="."; 2.0 empties content-free input instead. The born-empty-ambiguity survival was a review fix in the rc1 arc: an unbalanced-delimiter report must outlive the emptying, or malformed input becomes indistinguishable from blank input. - -### legacy-rule-numbers — the old docstring numbering - -Before rules.md, the post_rules stage docstring numbered its rules locally, and historical issue comments (#359, #364, #365, #367) use those numbers. The mapping: rule 1 → H1, rule 1b → P1, rule 2 → O1, rule 3 → O2, rule 4 → O3. Older still: bare "#11"-style citations from v1 comments live in the GOOGLE CODE namespace, not GitHub — GC issue 11 is decisions.md#P3's provenance — so a v1-era number that doesn't fit its GitHub issue is probably a GC number. - -### P1 — lone particle fold - -A lone PIECE is the whole test — deliberately narrower than "a never-given particle is never reported as the given name," which would be false (the over-broad invariant shipped to five sites before #361's review falsified it with three counter-examples). The counter-example set has since shrunk, and its shrinkage is the section's history: "Sir de Mesnil" fell to #367 (titles became transparent); "Juan de la Vega" under family-first — the whole chain in the given position — was called working-as-intended by -#359, contested by #368, and is working-as-intended again as of -the 2026-08-16 entries below. The survivor is the degenerate bare -"de". - -- 2026-08 #359 — the opening site is read from joining structure (pieces), not from assigned roles, so the fold holds under every name_order. Before this, the role-only read let "de Mesnil" split under a family-first order. -- 2026-08 #367 — titles are transparent to the fold: "Sir de Mesnil" now reads like "de Mesnil". Fixed by removing the title→particle chain in grouping, not by touching this rule. -- 2026-08-16 (order-precedence keystone; #364, #365, #368) — the stage split is the decision, and the three issues are one question seen from three angles. GROUPING is vocabulary's job and is order-independent: a particle joins forward through consecutive particles and stops at the first non-particle, and no name_order moves that stopping point. ASSIGNMENT is name_order's job: groups take roles by the declared order. The bugs existed because the implementation runs the two in the opposite dependency — `assign` hands out positions from `_effective_order` and `post_rules` then inspects a fixed list of ROLES, so P1's fold sites and P2's chain had coverage that varied with the declared order by accident. Consequences, each recorded in its own right below: the fold takes only the particle's own group (#364); the middle position needs no third site once grouping is order-independent (#365); and #368 reverses. -- 2026-08-16 #364 — the fold takes the particle RUN and the ONE name word it attaches to, not every remaining word. - "de Mesnil Jean" is family="de Mesnil" plus given="Jean". Run, - not particle: "de la Vega" is two particles onto one word, and an earlier wording here said "the particle", which its own example contradicted (rule-vs-decision-record review). Nothing ever argued for - "takes everything"; it was the shape of v1's - handle_non_first_name_prefix, not a decision. Measured before deciding, over every name in the three differential corpora with NO string prefilter: exactly ONE family holds words beyond its particle's group — "de Mesnil Garcia". - #364's own body warns the change "breaks the v1 parity - tools/differential protects" and that "each ledger would need re-examining"; measured, it is one name and one ledger entry. A shape filter (no comma, leading never-given particle, three or more words) returns THREE candidates, of which two do not move: - "de la Vega" is one group start to finish, and "de Mesnil Jr." - has only two name words because Jr. is a suffix. The pre-merge fact-check caught this stated as "the filter gives one name", which it does not. Measurement trap, recorded because the fact-check fell into it twice: the particle group runs through ANY particle, not only the never-given ones — "de la Vega" chains never-given "de" through AMBIGUOUS "la" onto "Vega". A detector that walks only the never-given run splits the group after "de la" and reports 50 false movers. -- 2026-08-16 #368 REVERSED — shipped behavior is correct. - "Juan de la Vega" under FAMILY_FIRST groups [Juan][de la Vega] - and assigns family="Juan", given="de la Vega". The earlier decision ("the particle wins ... whatever order was declared") was made before the grouping/assignment split was stated and cannot survive it: a mid-name chain HAS a head word and is positioned like any other group. What NON_GIVEN_NAME_PARTICLES guarantees is that the bare word never reads as a given name, not that no name part may begin with one. The asymmetry with the leading case is P4's, not an exception invented here: a leading particle chains nothing, so without the fold pure position makes the BARE particle the given name. That is measurable today on the ambiguous half, where no fold fires — - "van Mesnil Juan" gives given="van", middle="Mesnil", - family="Juan". given="de" is the reading the vocabulary exists to forbid, and the fold is what prevents it. The all-orders agreement in P1 is deliberate and is W4's shape: a wholly-hangul name reads family="김" under every declared order because the script carries a signal the order does not override, and a leading never-given particle is the Latin-script analogue. decisions.md#O4 already draws the line — "Words no vocabulary has claimed read by position" — so name_order governs the unclaimed remainder, which is most inputs. - -- 2026-08-17 — SUPERSEDES the order-independence half of the 2026-08-16 order-precedence keystone entry above (#364, #365, #368). That entry says "no name_order moves that stopping point"; it may. Declaring FAMILY_FIRST or FAMILY_FIRST_GIVEN_LAST is precisely an assertion that what follows the family is NOT more surname, and a particle run's stopping point is exactly the question of where surname material ends — so the declaration is evidence about it, not merely about which slot the result lands in. The grouping/assignment split survives everywhere else; what is narrowed is the claim that the stopping point is one of the things order cannot touch. - Concretely: under GIVEN_FIRST a leading particle run keeps joining to the end of the name. (This entry originally said that makes "de Mesnil Jean" read "exactly like pennie von bergen wessels — the same shape". Measured while implementing #395, that is false and is corrected here rather than left standing: pennie is the GIVEN name there, von is ambiguous so this fold can never fire on it, and the surname reading comes from P2's mid-name chain. What the real name does support is the weaker claim the argument needs — that a particle followed by several words can be all surname, "von Bergen Wessels" being one.) Under a family-first order the run stops after its first non-particle name word, because the caller has said the rest is given or middle. - Accepted, and it is the cost that decided #364 three separate ways before this: "de la Vega Juan" under the DEFAULT order stays wholly a surname. A caller who means family "de la Vega" plus given "Juan" writes the comma. The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language. - Why the question kept thrashing, worth recording so a fourth attempt does not start from scratch: "how much does a leading particle run take" is a LANGUAGE judgement being forced through a POSITION heuristic, in a parser that has correctly refused to detect language (decisions.md#O4, rules.md's Not-in-scope). Every argument in the thread — `de`/`do` as Vietnamese surnames, `von` as German, `dos` as Portuguese — is really about which tradition the name belongs to. The order declaration is the one place a CALLER supplies that information instead of the parser guessing, which is why keying on it is better than any position rule we tried. - Implementation consequence: the stopping point may now read name_order, which the superseded sentence forbade. Whether that lives in grouping (order-aware) or as a split in assignment is open — PR #391 showed assignment cannot split an existing piece today, so choosing assignment means giving it token-level slicing. - Supersedes nothing else: #368's reversal, the trailing-orphan rule P6, and the claim that a leading never-given particle takes the FAMILY rather than a given name all stand. -- 2026-08-18 #395 — WHERE THE STOP LIVES, settling what the entry above left open. Not grouping and not assignment: the fold in post_rules, which already retags tokens and is the only site that can express the stop without moving a piece boundary. Grouping was PR #394 (four regressions) and assignment was PR #391, where the finding was that assign cannot split an existing piece — and the piece is the problem: "de la Cruz Juan Carlos" groups as [de][la Cruz Juan Carlos], the ambiguous particle having chained forward, so any rule that stops "after the first name word" must cut INSIDE a piece. post_rules can, because roles are per token and nothing downstream reads pieces (measured: only _assign, which runs before it, and post_rules itself). Grouping therefore stays order-independent, which keeps the 2026-08-16 keystone intact everywhere except the reach of this one fold. - The order is READ, not re-derived: assign now records the order it actually used on ParseState.order, and the fold keys on that. policy.name_order would have been wrong — a script_orders entry can put the family first under a given-first policy, and the roles assign already wrote would then disagree with the roles the fold computes. The same reasoning already appears one function away, where the PARTICLE_OR_GIVEN emitter reads the role off the token rather than assuming given. - A unit, not a token: a particle chain (P2), a conjunction join (P3) and a bound given-name pair (P5) each count once. All three are read off the TAGS rather than the pieces, but for two different reasons, and the difference matters to anyone who tries to simplify this. The conjunction join grouping DID build and the prefix chain then swallowed: the name reaches the fold as [de][la Vega y Santos Juan], the ambiguous particle having chained forward over the join, so the JOIN's boundary is gone. The leading particle keeps its own piece, and must — the fold's site test wants a lone piece, so a one-piece name would not fold at all. The bound-given join grouping never built at all: P5 joins only where the bound word is the first non-title piece, and at a fold site the first piece is the particle, so "ibn Awf abdul Rahman" arrives as four separate pieces. Restoring piece boundaries in group would fix the first and silently split the second. It also makes P5's new implemented: pointer stronger than a citation — this is the only place that join happens in this shape. This is what keeps "de la Vega y Santos Juan" from stopping between Vega and Santos — the case rules.md#P3 was amended for on 2026-08-17, now executable. - Measured, with the denominator this log requires: of 751 corpus names, SEVEN reach P1's leading fold (DE MESNIL, De Groot, de Mesnil, de Mesnil Garcia, de Mesnil Jr., de la Vega, dos Santos) and exactly ONE of those has anything past the run to lose, so one moves (de Mesnil Garcia) under each family-first order and none under the default. The single mover is a fact about the corpus holding no family-first listing with leftovers, not about the blast radius. The differential harness cannot see any of it — it runs the corpus under the default policy against 1.4.0, which has no name_order — so its exit 0 is evidence for the accepted cost and for nothing else. The verification that counts is the two-leftover shape: mutation-checked, discarding name_order from the leftover placement fails three tests across three layers — the FAMILY_FIRST_GIVEN_LAST case row (one of the two, the FAMILY_FIRST row being blind to it by construction), test_post_rules.py's test_the_two_family_first_orders_differ_at_two_leftovers, and rules.md's P1 example for that order. An earlier draft of this sentence said "exactly one, and no other test in the suite": true when measured, false one commit later, because two of the three guards were added after the measurement and it was not re-run. Recording the failure mode as well as the number, since it is this log's recurring one. -- 2026-09-07 #471 DECLINED — the greedy reach under the default order is the READING, not a gap in one, and what decides it is a principle rather than a limit: a name order is a property of the DATA SOURCE, not of a string. The caller declares it to match how their records are written: the declaration governs what no vocabulary has claimed and a declared family-first order outranks the shape; rules.md's Name order Background states it. decisions.md#O1's 2026-09-07 entry applies the same principle to the patronymic rotations. Applied here: a never-given particle marks where a surname BEGINS, multi-word surnames are ordinary — Spanish double surnames above all — and under the default order nothing has said this surname ends before the string does. So `de la Torre Vega` is a surname-only string read right, and `de la Cruz Juan Carlos` is a family-first listing read under the wrong order. The remedy is the declaration or the comma: FAMILY_FIRST already gives family `de la Cruz`, given `Juan`, middle `Carlos` (#395, the 2026-08-18 entry above), and the family comma fixes the surname whatever order is declared. `de la Family Family Given` and `de la Family Given Given` are undecidable under EITHER order, which is what makes the comma the answer rather than a better heuristic. - This SUPERSEDES the REASON the 2026-08-17 entry above gives for the same outcome — "The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language." That was right and weaker. The order is not a language judgement at all: it is a declaration about the data, and a caller who has one supplies it whether or not a parser could ever have guessed it. The outcome that entry reached, and everything else in it, stands — and so does its "Why the question kept thrashing" paragraph: the REACH question is a language judgement, which is why the order declaration is where the caller supplies it. What is superseded is only the claim that the parser's blindness to language is the reason. - It also answers the 2026-08-16 #364 entry's "Nothing ever argued for 'takes everything'; it was the shape of v1's handle_non_first_name_prefix, not a decision." This entry argues for it. The v1 provenance is unchanged and the argument is new, which is the only thing that had been missing. - Measured 2026-09-07 on the tree, and on the 1.4.0, 2.0.0, 2.1.0 and 2.2.0 wheels, which agree on every default-order row: `de la Torre Vega` gives family `de la Torre Vega`; `de la Cruz Juan Carlos` gives family `de la Cruz Juan Carlos`; `de Mesnil Jean` gives family `de Mesnil Jean`; and the comma form `de la Torre Vega, Juan` gives family `de la Torre Vega`, given `Juan` under both orders. Under FAMILY_FIRST `de la Torre Vega` reads family `de la Torre`, given `Vega` — the cost the declaration accepts, and the reason a surname-only record in a family-first source still wants its comma. Recompute with `Parser().parse(name)` and `Parser(policy=Policy(name_order=FAMILY_FIRST)).parse(name)`; for the wheels, `HumanName(name).last` from a directory outside the checkout so the tree cannot shadow them. - Nothing moves. No parse changes; the `parity` row `leading_never_given_particle_two_leftovers` in tests/v2/cases.py and the guard `test_the_family_first_fold_is_not_explained_under_the_default_order` in tests/v2/test_ledger_guards.py stay as the pins of the decided behavior; no release-note bullet is written, there being no behavior to note and no by-design precedent in docs/release_log.rst. rules.md#P1 gains `de la Torre Vega` as an example, which puts the name in the contract corpus corpus_rules.jsonl. Its parity is over the SEVEN roles, not the `initials` view: at the 1.4.0 baseline only, the name falls into the already-recorded class `fix(initials-per-word) a particle chain inside a name part` beside its sibling `de la Cruz Juan Carlos` — the v1→v2 per-word initialling shipped since 2.0.0, not a regression — so the 1.4.0 intentional count goes 367 → 368 and that rule's recorded claim goes 107 → 108 names with a new digest. The three 2.x baselines are unmoved at 263 / 175 / 37 (296 / 210 / 72 after #449 and #491 landed the same day; #471 itself moved none), and `unexplained: 0` at all four. - The `Open:` pointer to #471 that stood at the end of decisions.md#P6's 2026-08-30 #467 entry is removed with this entry, which is where the question is now answered. P1's own `Open:` block below is #360's and is untouched. - Deliberately NOT decided here: reporting an ambiguity on the shapes neither order settles — the honest-output flag — would be an AmbiguityKind and belongs with the ambiguity bundle (#449/#491/#348) if anywhere. Declining the reach change does not decline that. Two thirds of that bundle shipped 2026-09-07 as decisions.md#O5 and decisions.md#H4; #348 was left OUT by choice, being a question about the TITLES vocabulary rather than about a decision site, and the shapes this sentence is about are neither one — no rule fixes them by convention, so there is no branch to report at. - -Declined: - -- A strict xfail asserting "de Mesnil" → family under FAMILY_FIRST (#359 review) — #359 deliberately left those semantics open, and a strict xfail decides the question by the back door. -- Keying the leading-particle exception on "the first piece that is not a title" — the obvious implementation, and wrong: st, do and freiherr are titles AND ambiguous particles, so it collapsed - "St John Smith" into one given name and broke test_add_title - (which adds "te", also a particle). The shipped predicate is - "not a title or a prefix". -- 2026-08-16 — deleting P4 so a leading particle chains and is then positioned, which is the only way to make "de Mesnil Jean" vary by declared order. It avoids given="de" (the group would be - [de Mesnil]) but breaks "de la Vega": measured, a single group is - positioned by the declared order — "Cher" reads given under GIVEN_FIRST — so "de la Vega" would read given="de la Vega" unless a further rule forced a particle-headed group into the family. Add that rule and [de Mesnil][Jean] yields the #364 reading anyway, so the deletion buys nothing and costs P4. - -Decided 2026-08-18 (was Open: [#360](https://github.com/derek73/python-nameparser/issues/360), which particles count as never-given): #360 applied the criterion at decisions.md#vocabulary-collisions and moved `mc` and `ste` into the set (from the may-be-given half), adding the Spanish/Portuguese articles; later vocabulary work extends the set under the same criterion (#604's Irish particles). - -### P2 — particles join forward - -- 2026-08 #367 — REMOVED a chain, it did not create one: before, the title displaced the particle out of the leading position, so - "Sir de Mesnil" grouped [Sir][de Mesnil] — a chain — and - reported given="de Mesnil" with no family at all. After, "de" is the leading name piece, and a leading particle chains nothing (rule P4): the family reading comes from P1's fold, which is why P1's fold needed no change (its interacts: points here). Grouped today: [Sir] [de] [Mesnil]. - -- Group distribution is load-bearing, not incidental: the v1 Portuguese tests (test_portuguese_dos, test_portuguese_prefixes) require multiple particle groups to land across middle and family, while #132 wanted the combined double-surname reading — the same shape with opposing wants, which is why the combined reading lives in the surnames VIEW and the split in the fields. - -- 2026-08-22 #424 — the chain stops where assign's trailing peel begins. The chain ran "until a trailing suffix begins" and asked with the suffix-piece test, which vetoes a bare `V` as an initial — the #401 question at a third site — so `John van der Berg V` read family 'van der Berg V' (1.4.0 read it so too: shipped since 1.x), and `John van der Berg Ma` read family 'van der Berg Ma' where 1.4.0 read suffix 'Ma' — a 2.0 regression on S2's other fork. Both now stop where `_trailing_start` says the run begins: the S2 peel (`_peel_trailing` over `_peel_walk`, shared with assign and P5's reserve since #425) read over the pieces as they stand from the first name piece, once per segment, kept as a length from the end that the chain's merges ahead of it do not move (the benchmark's `particles` shape is the guard). The chain takes both forks — its merges leave the acronym at least the three pieces the fork counted, so assign peels it after as the walk did before — where the maiden walk takes only the numeral (the M2 entry says why). The fork reads the piece before the numeral as it stands, so `John van der J. V` keeps family 'van der J. V', as `John J. V` reads the V as a name; and a numeral with a suffix behind it is not last in the walk, so `John van der Berg V Jr` keeps family 'van der Berg V'. No differential corpus name had either shape; the rules examples carry them — the numeral against every baseline, the acronym against 2.0.0 and 2.1.0. Two more sites were asking assign's question with a test of their own, both found by the reviews of this change. The walk's starting point and P4's leading-particle scan asked group's `title()`, which does not see H2's unlisted abbreviations — assign peels those as titles all the same — so `Xyz. van Johnson` chained where `Dr. van Johnson` did not (#367 had keyed the exception on the first piece of the name, by group's test), and `Xyz. van Berg MA` read family 'van Berg MA' on every 2.x tree — the chain had taken the acronym, as it took every acronym; the draft that stopped before it left assign two pieces where the fork had counted three, and MA became the family. Both ask assign's own test now — `_is_leading_title` and the period-abbreviation pattern moved down into group, with `_leading_titles` as the one definition of where the name begins: the chain's walk, the bound join's first name piece (the second docs review found that one still on group's test — `Xyz. abdul John Smith` joined nothing where `Dr. abdul John Smith` read given 'abdul John') and, through `_is_leading_title`, the scan and the PARTICLE_OR_GIVEN report — so `Xyz. van …` reads as `Dr. van …` does. `Esq. van Gogh`, a corpus name, moves with it (1.4.0 read family 'van Gogh' for both, as #367's rule records; its own rule at every baseline, and one for the P4 example). A particle of the unambiguous suffix vocabulary too (vd, mc) is a suffix piece to the peel: where it opens the trailing run the chain stops before it as before any suffix word and the peel takes it — `John Smith Mc V` reads suffix 'Mc, V' where master and 1.4.0 read family 'Mc V', a class the second docs review found unrecorded (2,554 of its 117,306 constructed names, an eighteen-word pool that puts the words in every position) — and where it continues a prefix run, the run takes it as a particle, so `John van Mc` keeps family 'van Mc', every baseline's reading and the one P6 chose for the shape after a comma. Both are Accepted under P2. The fork's count was the last thing the chain asked on its own authority, and the reviews found it wrong behind a title-and-particle word: #367 stops the leading-particle scan at `Freiherr`, `St`, `Do` — the name's own first piece — where assign's title peel steps over them, so the chain takes the name's first word, and the acronym the fork counted with three pieces met assign with two: `Freiherr von Berg MA` read given 'von Berg', family 'MA' (1.4.0's reading, as it happens; master read family 'von Berg MA'). So the chain asks the peel again over the pieces it leaves — `chain(tail)`, then `_trailing_start` over what it built, and a second run without the stop where the verdict changed, the snapshot being one copy per segment with a trailing run — and takes what assign will not peel: family 'von Berg MA'. The numeral cannot flip, a chain group never being initial-shaped, so `Freiherr von Richthofen V` keeps suffix 'V', and its chain, the one name piece left, reads as `Dr. Smith V` reads — where 1.4.0 and master read family 'von Richthofen V'; the code review asked for that boundary to be on record, and P2 carries it. #410 (2026-08-25) then moved the name half of both: a title with one name word behind it names the family whatever stands beside it, so the pair reads family 'von Richthofen' and family 'Smith' today, where each read `given` with no family when this entry was written. The boundary is unchanged — the chain still stops before the numeral, and the two names still read alike; only the field the one name piece lands in moved. This name is the fifth corpus name #410 moves, and the only one already classified, its ledger rule's `fields` spanning the narrower diff. -- 2026-10-02 (Derek), #573 — the part before a family comma: a particle that is suffix vocabulary too heads the name word behind it in any case. Decided with P6's third exception and S2's never-given lead; the record is the P6 section's entry of the same date. -- 2026-10-07 #614 — CROSS-REFERENCE: the chain's stop is the trailing run S2 reads once, after P3's joins and before the chain is made, counting the chain's own run -- a particle run, the words both particles and suffix vocabulary behind it included, and the plain words after it -- as the one name word it will make; the re-ask after the chain's own merges (the 'Freiherr von Berg Ma' rollback above) runs only where the run was not read there. decisions.md#S2, this date; rules.md#P2's statement amended. -- 2026-10-10 (Derek), #624 — WHERE THE NAME STARTS IS DECIDED ONCE, AND OF TWO TITLE-PARTICLES THE SECOND LEADS. `TITLES ∩ particles_ambiguous` is `{freiherr, st}`. Such a word in the leading titles is the chain's leading position (P4), so the particle behind it is inside a name (`Freiherr von Berg` → family 'von Berg'). Group's chain found that position with a scan of its own that stopped at the FIRST such word, and chained the second as a surname particle, while assign read both as titles and the trailing read's unit count (`_chain_units`, #620) opened a unit inside the titles to follow the chain: `Freiherr St John Smith MA` read title 'Freiherr', family 'St John Smith', with a title-or-name report saying it was "read as a family name by convention", where `Dr. St John Smith` and `Sir St John Smith` read title 'Dr. St' / 'Sir St', given 'John', family 'Smith'. - DECIDED (Derek: consistency, one shared answer, no special case for garbage): `_pieces.chain_lead` gives the leading position once -- the LAST title in assign's own title run (`leading_titles`) that is also a particle, else the first piece past the run -- and group's chain and the read's count both take it. The run is read as WRITTEN, before H3's give-back (Derek, the review's option (a)): a title the run hands back to the name because only suffix words follow it is still a title to the chain, so the particle behind it is the name's leading piece and chains nothing (P4). `leading_titles` stops at a piece that is itself a leading title only where it gave that piece back or at the segment's last piece (a title needing a following piece), so `chain_lead` recognises the give-back from `n` alone, the last piece ruled out by its position, behind the same inline tag test, and no second title walk is needed. No particle stands between the lead and the run's end, so the chain opens no unit inside the titles: the read's `at` and its `q > k + 1` gate are gone, and group's chain skips every piece up to the lead (`k <= leading`, equivalent to the old `k == leading` wherever the lead was the first title-particle, since no particle stood before it). `Freiherr St John Smith MA` → title 'Freiherr St', given 'John', family 'Smith', suffix 'MA'; `Freiherr St van Berg MA` → title 'Freiherr St', family 'van Berg'. - NOT DONE: the issue also proposed retiring assign's re-read fallback (`read = None` where group's joins left no name word past the titles). Measured, it still does work this change does not touch: disabled, sixteen or seventeen more tests fail depending on how it is disabled, most of them as an IndexError, so it guards totality and not only a reading -- among them the case row `a_title_read_before_the_chain_can_strand_the_name` (`Freiherr von Berg Dr. and Ed. Prof.`, where a CONNECTIVE join makes the name a title), the locale non-interference tests and two case-repair property tests. It stays. - MEASURED 2026-10-10 against 8e6e524a (#631's merge), py3.11, `nameparser.__file__` asserted on each side. A title-particle head grid -- every sequence of one to four words over `Freiherr St Dr. Sir Xyz. von van de do Mc John Smith MA PhD VI Prof. née Jr. and`, plus `Smith, ` and `, Jr.` over `Freiherr St Dr. von John`, 137,810 names under the three orders, comparing the seven fields and every report: 5,964 role moves, no report-only move. 5,868 have two title-particles in the leading title run (a period-shaped word such as `Jr.` and a connective title such as `and Freiherr` counting as titles there), all now titles -- where H3 then gives the last back, the word behind it is the post-nominal H3 saw rather than a word that title chained (`Dr. Freiherr St Mc` → title 'Dr. Freiherr', family 'St', suffix 'Mc'; master family 'St Mc'). 96 are H3's give-back before a run of dual particles, read as H3 states (`Dr. Mc Mc` → given 'Dr.', suffix 'Mc Mc'; master given 'Dr.', middle 'Mc', family 'Mc', the read having counted the second 'Mc' as bound into a run the chain never built). The first draft read the give-back as the chain's lead and gave `Dr. Mc Mc` title 'Dr.', family 'Mc Mc', chaining the name's leading particle behind a plain title against P4, and with a caller lexicon (`esq` added as an ambiguous particle) `Dr. esq vd` chained 'esq' and reported it; the review caught both, and option (a) is what this entry records. A comma-tail grid, `{head} {w1 … wk}` for k from 1 to 3 over the heads `John Smith, Jr.,`, `Smith, John,`, `Smith, Jr.,`, `,,`, `Smith,,`, `John Smith, PhD, Jr.,` and the words `Dr. Dr Attorney`, `Secretary of State` (one word of the list), `Lt.Gov. Freiherr St von`, `van der` (one word), `Bart Jones Jr. PhD MA Ma X.Y.Z. nee VI V 'Bob' (Bob) Prof.`, `Mr. and Mrs.` (one word), `I y` -- 97,650 texts, three orders: 1,368 parses on 456 texts move, every one a part past the second comma led by two title-particles, now all titles as C2 reads a title word there (`,, Freiherr Freiherr Bart` → title 'Freiherr Freiherr', suffix 'Bart'; master title 'Freiherr', suffix 'Freiherr Bart'). The S2 stress grid (#614's recipe): 0 moves. Every corpus name and case text at 8e6e524a under the three orders: only the case rows #624 rewrites move. The five differential gates exit 0. Cost: every frame baseline moves DOWN by 4 (decisions.md#parse-cost, this date). - ROWS: five rows pinned the old decision and now read the new one (`fix(#624)`, each note giving the earlier reading): `Freiherr St van Berg MA`, `Freiherr Freiherr Prof do`, `St St VI`, `Freiherr St MA`, `Freiherr St Prof.`; `Freiherr St John Smith MA`, the issue's example, is added, and `Dr. Mc Mc` pins the give-back half (the review found that half unpinned, a revert passing the suite). The marker row that held group's particle-or-given emitter on `St St née` moves to `St van née`, the same structure with a chained particle that is not a title (parity), since `St St née` no longer reaches that emitter. - -### P6 — the trailing orphan particle - -- 2026-08-23 — `Jong, vd` is C1's now, not P6's: C1's listing-form clause (#296's bundle) reads a post-comma part that is nothing but suffix words as the credential run, and `vd` is suffix vocabulary, so the row reads family 'Jong', suffix 'vd' by statement (it read so before, by the lone-piece deviation). The #380 scope note below calling it deliberately unresolved is superseded for that one row; the P6 question — a tussenvoegsel with no given word left — is unchanged. -- 2026-08-18 #379/#380 IMPLEMENTED — the rule shipped as stated, both `deviates:` markers out. Keyed on the token's VOCABULARY rather than its assigned role, which is what gives the S2 precedence its effect: assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma. The words-to-spare guard is a PIECE test, not a count — every trailing piece that is wholly particles attaches, and the run must leave a GIVEN word ahead of it, so "Nguyen, Van" keeps its only given word. Not "a name word": see the defect below, where that wider reading passed on family text P1 had just produced. MIDDLE was in the test briefly and came out — 0 hits over 740,552 instrumented guard sites, and structurally unreachable, since the only rule that leaves a MIDDLE with no GIVEN ahead of it is gated on state.order, which assign never records on the family-comma path P6 runs on. - Measured: ONE of 751 corpus names moves ("Vega, Juan de la"), at all three baselines, and it needed a new ledger rule in each. The LEDGER rule's regex — not the parse rule, which has no regex and fires on the whole particle vocabulary through the tag — is a PARTIAL copy of PARTICLES, deliberately: the words that actually end a Dutch, Iberian or German listing, not all 70. The ledger guards forced that narrowing: the first draft used the neighbours' bare comma regex, which claimed 215 names and reached two protected exclusions. A member `des` had to come out as well, and for a reason worth keeping straight: it is not in PARTICLES at all, so it could copy nothing, and the text it reached was the opening of "Designated Judge David A. Ezra" rather than a person's name. `_reaches_non_vocabulary("des", PARTICLES)` prints the FRAGMENT it matched, `['Des']`, not the name containing it — which is how the retracted wording arose in the first place. - Two defects found in review, both from the rule's neighbours rather than from its own logic. P1's fold runs EARLIER in the same function and retags all of segment 1 to FAMILY, so a words-to-spare test written as "some name role remains" passed on family text P1 had just produced — and the attachment then hoisted the particle in front of a base it never preceded ("Smith, de Mesnil van" → 'van Smith de Mesnil'). 1.4 gives 'de Mesnil van Smith' for that input and 2.1 gives 'Smith de Mesnil van', so neither the defect nor the fix restores 1.4 — what the fix restores is 2.1, the pre-comma-family reading having moved back in 2.0. The test has to name GIVEN, which is what the rule says. And a post-nominal sits BEHIND the tussenvoegsel in this listing, so the backward walk stopped on it and the attachment never fired: "Berg, Jan van Jr." and "Berg, Jan van, Jr." parsed two ways on nothing but a comma. The walk now steps over a trailing piece holding no name, unless it is itself particle vocabulary — `vd` arrives suffix-roled and IS the run. - Fallout worth its own record: "Vega, Juan de la" was the input tests/test_initials.py used to build an ALL-PARTICLE middle name, and this rule takes that shape away from THAT input. It does not remove the shape — "Vega, Santa de la", "Berg, Van der" and "Nguyen, Van Le" all still produce one in the default order, because the trailing run swallows the given word too and the attachment then declines. The test is repointed at the last of those rather than deleted. Whether such a middle should exist at all is #402 — for a never-given particle it is #365's defect, and for an ambiguous one the parser reports it as a middle name and then declines to initial it, which is a miscount either way. - -- 2026-08-16 (order-precedence keystone; #379, #380, #365) — a particle ending the name has nothing to link forward to, and no particle is a name by itself, so it attaches to the family name standing beside it and renders BEFORE it. The distinction from a chain is what makes this a rule rather than an exception: a chained group has a head word and can be positioned; an orphan has no head, so position has nothing to work with. -- Scope: the COMMA form only, deliberately. "Jong, Anke de" is unambiguous — the comma has already named the family. Without the comma the written shape is not settled: "Jong Anke de" may be a misformatted listing (arguably a missing comma under a declared family-first order) and "Jong de" may be a given name beside a particle. Those keep their positional reading and are not tracked as deviations. -- The words-to-spare guard is load-bearing, not incidental. #379's own subject is "van", which is in the AMBIGUOUS half — so a rule keyed to never-given particles alone would not fix the issue it was filed for, while a rule with no guard breaks Vietnamese - "Nguyen, Van" (given="Van") by eating the only given word. The - guard is S2's shape, reused: consume only when the name has words to spare. -- Rendering before the family has precedent in rules.md#R1 — folded family words under O3 already "render before the rest of the family wherever they stood in the string". The surnames view renders backwards today ("Vega de la", "Jong de") and flips with this rule. -- Measured under this rule's OWN comma scope, over 782 corpus names (245 of them comma-bearing): exactly one moves, - "Vega, Juan de la" → family="de la Vega". A pre-merge fact-check - corrected an earlier count of three here — "Smith van der" and - "Sander van" have no comma, so the rule as scoped does not reach - them; they were measured before the comma scoping was chosen and carried forward unfiltered. -- What that number means is the opposite of reassuring. The corpus holds 245 comma names, TWO of which end in a particle and one of which clears the words-to-spare guard ("Nguyen, Van" is the other, and the guard is what keeps it a given name), so it is very nearly blind to the shape this rule governs — Dutch and Flemish listings are barely represented. Treat the one as evidence about the corpus, not about the blast radius, and run tools/differential at all three baselines in the implementing PR regardless of how small this looks. -- 2026-08-16 (pre-merge coherence pass) — P6 and S2 CONTEST - "Berg, Jan vd": today S2 wins and reports suffix="vd", while - P6's marker asserts family="vd Berg". P6 wins, and the rule says so in its statement rather than leaving the pair to file order. The reason is C-ii's, not a new judgement: vd as the British Volunteer Decoration is rarer than vd as van der, and a trailing abbreviation AFTER A FAMILY COMMA is the tussenvoegsel position specifically. Scope check on the precedence, so it cannot creep: it reaches only words that are both particle and suffix vocabulary, in the trailing-orphan position, under a family comma, with a given word to spare. "John Smith, PhD" and "Smith, Jr." are untouched (not particles), and "Jong, vd" is untouched (no given word remains), which is why that row stays out of scope rather than becoming a counter-example. Recorded because the pair was declared in `interacts:` and left unresolved — `interacts:` is advisory by design and pins nothing, so a declared interaction is a prompt to state the outcome, not a substitute for stating it. -- 2026-08-16 — P6 is the first rule in rules.md that nothing implements. Legitimate per the preamble (the document is normative, and a gap is a tracked deviation), but it left the rule pointing at nothing, so the shape got a pointer rather than an exception: `tracked: #379, #380` in place of `implemented:`, with exactly one of the two required of every rule. An unimplemented rule can no longer sit here untracked, and a shipped rule cannot keep a stale tracking pointer after its issues close. Mutation-tested three ways before being believed (drop the pointer, carry both, malformed refs); each fails. -- The RATIONALE first shipped here was wrong and is corrected: it read "no particle is a name by itself", which is true only of the never-given half — decisions.md#vocabulary-collisions says the opposite in as many words ("most particles are short words that double as names"). The error mattered rather than merely reading badly: the words-to-spare guard exists BECAUSE the rule reaches ambiguous particles (#379's own subject is "van"), so the rationale undercut its own guard. Caught in review, after a coherence pass that interrogated rule-vs-rule pairs and never checked rule-vs-decision-record — which is the gap to close next time, not a one-off. - -Decided 2026-08-19 (was Open: [#380](https://github.com/derek73/python-nameparser/issues/380), "Berg, Jan vd" under this rule): the attachment reads `vd` as *van der* and joins it to the family the comma named (family "vd Berg"); the vd reading itself is decisions.md#vocabulary-collisions (C-ii). The no-given-word case "Jong, vd" is still deliberately unresolved — see the scope note. - -- 2026-08-22 (late review of #425's docs, PR #426) — the bound join runs before P6 sees the name, and P6 now says so. P5's new title-word clause took `do`, `freiherr` and `st` — title AND particle vocabulary, until #296's audit took `do` out of TITLES, since when the join takes `do` as the particle it is — away from the attachment: `Berg, abdul do` had read family 'do Berg' since #379 and reads given 'abdul do' now. That is 1.4.0's reading, and it is what every other particle already did: `Berg, abdul van` read given 'abdul van' at 1.4.0, on 2.0 and 2.1, and on master before #426 — the old reserve's title exclusion had held only the title-word trio back. The precedence is the stage order (P5 joins in group; P6 attaches in post_rules, over what is left -- in assign since #613, after the given part is read, still over what P5 left) rather than a judgement about tussenvoegsels behind bound words, and both rules carry it as Accepted with `Berg, abdul van` as the example, parity at every baseline. The precedence has one exception, found by the review of #424's docs when the Accepted clause claimed the attachment "never sees" the particle: a trailing particle of the unambiguous suffix vocabulary too — `vd`, `mc`; `do` is ambiguous, and a name word to the join — is a suffix piece to the join, which never takes one (#421), so the attachment does see it, and `Berg, abdul vd` reads family 'vd Berg' where 1.4.0 read given 'abdul vd'. Both rules say so, with the example; at 1.4.0 the fix(comma-precomma-family) rule already reaches it (a rule written for it was shadowed, and dropped). - -- 2026-08-29 #405 (the A1 deviation, reversed) — the attachment reports the fork it decides, and the Accepted clause saying it does not is deleted rather than re-tracked. What that clause rested on was assign's own scoping comment: after a comma that fixed the family, a leading given-position particle is "not meaningfully ambiguous". True of the FAMILY — the comma names which piece that is — and not of the particle standing behind the given name, which P6 decides on its own a stage later, out of the same vocabulary that makes `Van Johnson` report `particle-or-given` with no comma in sight. The old state was the shape A1 forbids: `Van Johnson` reported the fork, `Nguyen, Thi Van` decided the identical fork silently and lost a given name doing it. - TWO kinds, keyed on WHAT THE ATTACHMENT OVERRODE and not on what the words are, because `mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE` asks the emitted fork to be the one taken. Where assign had read the run as a post-nominal, the overridden reading is the post-nominal — `suffix-or-name`, which is exactly the S2 precedence this rule already claims in writing. Otherwise, where the run holds an ambiguous particle (`van`, `Van`, `la`), the overridden reading is that word as a name word — `particle-or-given`. One kind for both would assert "particle or given" about a word the parse had read as a credential and never weighed as a name, naming a branch nobody considered. - The key is the READING and not the vocabulary, and the docs review caught a first draft that said otherwise ("a word that is also suffix vocabulary declines a post-nominal"). That wording is wrong for both of the words it most obviously describes. `do` is the third member of the particle ∩ suffix-acronym overlap and reports `particle-or-given`: it sits in the AMBIGUOUS acronym half, which had already left it a name word, so no credential reading was ever taken for the attachment to override. And a run of several words is read whole — `Berg, Jan de vd` reports NOTHING though `vd` is suffix vocabulary, because behind `de` the run was read as name words, where the lone `Berg, Jan vd` reports `suffix-or-name`. Both are pinned in tests/v2/cases.py; neither is in any corpus, so nothing else would have caught the wrong statement. - The two arms are ORDERED rather than asserted disjoint, and that is a correction to the plan, which recorded them as disjoint by construction ("a suffix-vocabulary particle arrives suffix-roled and IS the whole run"). That premise is a property of the SHIPPED vocabulary, not of the rule: `vd` and `mc` are the only words in both the particle and the unambiguous suffix sets and neither is ambiguous particle vocabulary, so no shipped input reaches both arms — measured, 0 both-arm runs over 29,400 generated names (every ordered pair of the 70-word particle vocabulary in six family-comma shapes: bare, behind a given word, behind a title, before `Jr.`, before `MD`, and with the pair split across the comma). But a caller's `Lexicon` may put one word in both, and the `assert` the plan asked for DID fire on `Berg, Jan zz` under a Lexicon adding `zz` to `particles_ambiguous` and `suffix_acronyms`. `rules.md#A1` says parsing never fails on any input and names exactly one exception, a caller's segmenter; an `AssertionError` out of a stage would be a second. So the code decides instead: the suffix arm wins, because assign had read the word as a post-nominal and the name-word reading was never on the table for the attachment to decline. Pinned in tests/v2/pipeline/test_post_rules.py. - The never-given half stays SILENT, and that is the substantive half of the decision. `Jong, Piet de` has no live alternative: `de` is not a name in any reading and is not suffix vocabulary either, so the attachment chose between one reading and nothing, which is not a fork. Reporting it would be "the code got here" keying, the failure this project already has on record (a fork reported for every ambiguous particle at once on `Dr. Van Jr.` — the count is left out on purpose, since decisions.md#vocabulary-collisions says to recompute before quoting it). - Measured over the 1094-name differential corpus: P6's attachment fires on 9 names, of which 5 take the ambiguous arm (`Beethoven, Ludwig van`, `Berg, Jan van der`, `Nguyen, Thi Van`, `Vega, Juan de la`, `der, y van`), 2 the suffix arm (`Berg, Jan vd`, `Berg, abdul vd`) and 2 stay silent (`Jong, Anke de`, `Jong, Piet de`). Seven names gain an ambiguity kind and no role field moves that was not already moving. Read that seven as evidence about the corpus rather than about the blast radius, the caution the 2026-08-16 entry above already gives for this rule: 304 of the 1094 names carry a comma and only 13 end in a particle, 11 of those being rows the #379/#380 arc added itself (this change added four corpus names and not one of them carries a comma, so the 11 is unchanged by it). The reach is a SHAPE — every family-comma listing ending in an ambiguous particle, which is to say ordinary Dutch, Flemish and unaccented Vietnamese listings — and a caller holding such records sees a report on all of them, not on seven. The LEDGER shape that follows from that is not the one the plan predicted, and the difference is worth recording because it will recur for any report added to a rule whose fields already move. A new `fields = ["_ambiguities"]` rule explains none of the seven: `classify` matches on a rule's `fields` being a SUPERSET of the whole diff, and against 2.0.0 and 2.1.0 these names diff in `middle`/`family` (or `family`/`suffix`) from #379 and #380 as well. So the repair is to WIDEN those two existing rules to `{middle, family, _ambiguities}` and `{family, suffix, _ambiguities}` in both ledgers, not to add a rule. Each widened rule still passes the #452 equality check, since the union over the names it explains is exactly what it declares — the two silent `Jong` names contribute `{middle, family}` and the five ambiguous ones contribute `_ambiguities` on top. 1.4.0 needs nothing at all, having no ambiguity surface to diff. Gate before and after: byte-identical output at all three baselines apart from the provenance header — 1094 names, 229/194/102 intentional, 0 unexplained, and no per-heading count moved. The seven names were already in the diff; only their field sets grew. - The most visible consequence, and it is not the Vietnamese one: `Beethoven, Ludwig van` — the textbook-correct Dutch listing, read exactly right — now carries a report. That is honest rather than regrettable. The parser cannot separate it from `Nguyen, Thi Van`; the two are the same string shape over the same vocabulary, and a report on one is a report on the other. A caller who wants only the doubtful cases has nothing here to filter on, because there is nothing to filter on in the input either. - -- 2026-08-30 #467 (the no-comma site, and the given position given back) — a declared family-first order names the family as a comma does, so P6's attachment fires without a comma too. It is keyed on TWO things, and review showed the second is not implied by the first: the run must stand in a MIDDLE — the one position that means nothing for a particle, middles being further given names — and it must END the name. Together they name the order without asking it: the default order ends with the FAMILY and FAMILY_FIRST_GIVEN_LAST with the GIVEN name, so only FAMILY_FIRST can put a name's last word in a middle. This DECIDES the parenthesis the 2026-08-16 Scope bullet left open ("arguably a missing comma under a declared family-first order") and narrows that bullet to the default order. - P1 LOSES ITS GIVEN-POSITION SITE in the same change, and that half is a correction rather than a narrowing. That slot holds what the caller DECLARED to be the given name; the never-given vocabulary supplies a reading where position leaves the question open and vetoes none that position has already given. `Ménil de` under either family-first order now reports given `de`. - THE PREMISE THIS REPLACES WAS FALSE, and it had been stated in three places (rules.md#P6's rationale, `_lexicon.py`, `config/particles.py`): that a never-given particle "cannot be a name at all". Measured, all 33 members report given == themselves when parsed bare. This is the SECOND correction of the same over-claim in this entry — the 2026-08-16 note above records dropping "no particle is a name by itself" for the identical reason. The conclusions drawn from it were mostly right; the reason was not, and it is load-bearing, because stating it as a property of the VOCABULARY invites the same inference for a particle ending the name, where there is nothing to join forward to and the writing is all there is. - NO VOCABULARY TEST AT THE NEW SITE, and that is the substance of the design. The comma path needs one because a comma cannot separate the Dutch reading from the Vietnamese; the declared order can, and does both at once — `Beethoven Ludwig van` under FAMILY_FIRST gives family 'van Beethoven' though `van` is ambiguous vocabulary, and `Nguyen Thi Van` under FAMILY_FIRST_GIVEN_LAST keeps given 'Van' though it is the same word in the same set. A never-given test here would have excluded 37 ambiguous particles — von, di, da, del, le, and `van` itself, the flagship word of the listing the rule is named for. An earlier attempt (#466) carried exactly that test and failed on the Dutch case because of it. - NO RE-LAYOUT EITHER, and this is a property of ONE order rather than a simplification: under FAMILY_FIRST the roles run family, given, middle, middle…, so dropping a trailing MIDDLE leaves every other piece's role untouched. It is false of FAMILY_FIRST_GIVEN_LAST (family, middle…, given) and of the default order, which is the second thing the ends-the-name test secures. #466 removed a piece and re-laid the leftover out, which lost a given name outright on `van Berg Jan de` and promoted a post-nominal into the given slot on `Berg Jan Jr. de`. Both defects are structurally unreachable here. The one guard that remains is the base: attaching to an all-particle family renders one particle in front of another, and R2 reads those words as ordinary name words, which no rule reorders. - THE INVARIANT THIS RESTS ON, and the reason to trust it over the argument it replaced: for a name of the form `Family Given [Particle]`, the family-comma writing under the default order and the comma-less writing under FAMILY_FIRST parse identically. Measured over ALL 70 particles × 3 families × 3 givens = 630 names: 0 of 630 agreed before this change, **603 of 630** after. The invariant draws the never-given/ambiguous line by itself instead of having it asserted, which is why no vocabulary test appears at the site. - The 27 failures are the three words that are BOTH particle and suffix vocabulary — `vd`, `mc`, `do` — and they are a real limit, not noise: P6's precedence over S2 is stated inside the comma paragraph and justified by the comma ("a trailing abbreviation after a family comma is the tussenvoegsel far more often"), so the comma form gives family 'vd Berg' where the comma-less family-first form gives suffix 'vd'. The rule now says so where it states the correspondence. - A FIRST DRAFT OF THIS ENTRY CLAIMED 216 of 216, and the number came from a script that sliced the particle vocabulary — `sorted(particles)[:14]` — which excluded exactly those three words. That is AGENTS.md's axis 2 ("your detector is a second, unreviewed implementation") in its purest form: the detector agreed with the design because it inherited the design's blind spot. Recompute over the WHOLE vocabulary, never a sample. - Scope the invariant to that SHAPE as well: 219 of the 304 comma-bearing corpus names parse differently with the comma simply removed, because a comma also delimits titles and suffixes. (An earlier draft said "181 of 247", which reproduces under no definition of "comma name" — the 304 matches the count the 2026-08-29 entry above already records.) - An asymmetry falls out of it rather than being carved: shape 2 places the given name BEFORE the middles, so it corresponds to FAMILY_FIRST only. There is no comma format that puts the given name last, so FAMILY_FIRST_GIVEN_LAST has no twin to inherit an equivalence from. That is why the trailing particle is one order's business (#469). - ONE EXAMPLE LINE WAS DELETED RATHER THAN UPDATED: `"Nguyen Thi Van" family-first → middle="Van"`. The name is written in FAMILY_FIRST_GIVEN_LAST; its FAMILY_FIRST reading is wrong by construction, so the line pinned a value nobody wants and nothing should depend on. Recorded here because the general question — whether a rules.md example should declare its format, and whether the doc test should refuse to run one outside it — is #470 and is NOT decided by this change. - REACH: 30 of 6606 parses move — this tree's 1101-name corpus × three `name_order` values × `middle_as_family` off and on — over twelve names, NONE under the default order. Recompute by parsing the `tools/differential/corpus*.jsonl` files of this tree, deduped, under each of the six policies, and comparing the seven role fields against a checkout of the parent commit reading the SAME corpus files; the comparator is the pre-change code, never a construction of this rule. Dated snapshot, measured 2026-08-30. Note the corpus grew from master's 1094 by this change's own example lines, which is why the count must name the tree it was taken on. - The differential gate cannot see any of it: `compare.py` parses every name as `HumanName(name)` and sweeps no policy, so no non-default `name_order` behavior has ever been compared across versions. Output is byte-identical at all three baselines apart from the added corpus names, none of which diffs. - WHAT REVIEW FOUND, because the corpus could not. Four agents on PR #472 found the site firing outside the order it was reasoned from, in two directions, and the differential corpus reported neither: 52 default-order names (a conjunction stops a particle's forward chain and leaves it standing in a middle — `Maria Luisa y de la Cruz` gave family 'de la Cruz' where master gives 'la Cruz') and 366 FAMILY_FIRST_GIVEN_LAST folds of a particle that does not end the name (`de Anke van y` gave family 'van de Anke' with the given name still behind it). Of 1099 corpus names only 6 hold a conjunction-then-particle adjacency and none in this shape, so the REACH recipe honestly reported 0 default-order movers throughout. The count was right and the population was blind; "none under the default order" was a property of the corpus, not of the code. Both are now `· boundary` examples in rules.md so the doc runner holds them. - The repair is one clause — the run must END the name — and it subsumes the order test a first repair added: measured over 542,592 generated parses, an explicit `state.order[0] is Role.FAMILY` never decides anything the ends-the-name test has not already decided, so it is not in the code. What IS kept is `state.order is not None`, which is what keeps this site off the family-comma path (0 co-firings with P6's comma site over ~892,000 parses). - A stale index list was repaired with it: O1 and O2 retag between roles without recomputing, and this site was the first consumer of `families` after them (33,840 stale observations instrumented). No shipped input reaches a wrong answer through it — no `_TURKIC`/`_EAST_SLAVIC`-matching word is particle vocabulary — so the fix is unwitnessed by any test and is recorded here instead. -- 2026-09-18 #531 (cross-amendment) — THE PRECEDENCE OVER S2 GAINS ONE EXCEPTION, AND THE RUN WALK GAINS ONE STAND-DOWN CONDITION. An all-caps member of the AMBIGUOUS credential class, in a name written in more than one case, reads as the credential and this attachment stands down: `Doe, John DO` is suffix `DO` where `Doe, John Do` and `SMITH, JOHN DO` still attach. The argument, the accepted cost (`SMITH, JOHN DO` paired with `NASCIMENTO, EDSON ARANTES DO`) and the two carve-outs are in decisions.md#S2's 2026-09-18 `do` bullet; what belongs here is the CONDITION, which is narrowed to the ambiguous tag rather than to the suffix role: a suffix-roled single-token piece carrying that tag is not part of the run, while `vd` and `mc` — unambiguous suffix vocabulary, also particles, also suffix-roled — stay inside it and are unaffected. Keying on the role instead would have stood the attachment down for them too, and keying on nothing would have let it silently override the slot's reading (`Doe, John DO` read family `DO Doe` with the condition absent, verified). The 2026-08-18 entry's "assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma" is unchanged for `vd` and `mc` and now has this one exception for `do`. - -- 2026-10-02 (Derek), #573 — A WORD OF BOTH THE PARTICLE AND THE UNAMBIGUOUS SUFFIX VOCABULARY READS ALIKE IN EVERY CASE AT THE THREE SITES WHERE THE UNIFORM CASE SPLIT IT. Resolves the shapes the C1 section's 2026-10-01 #554 entry left open. `vd` and `mc` split in names written wholly in one case, in three shapes, each split one stage acting on a reading the next never took. The mixed-case spellings were not all right either: `Smith vd Ma, John` read family 'Smith Ma', suffix 'vd' in 2.2.0 and 2.3.0, #530 having fixed it only in this cycle, and the fixes below move mixed-case names as well as uniform ones (blast radius, below). - BEFORE A FAMILY COMMA (P2). `SMITH VD MA, JOHN` read family 'SMITH MA', suffix 'VD' (2.2.0, 2.3.0 and this cycle's tree; 2.0.0 and 2.1.0 read family 'SMITH VD MA'). In one case the S2 trailing reading takes `VD MA`, so the particle chain stops before `VD`; the part before a family comma then takes strict suffix words one piece at a time, and the lone `VD` was peeled from between two family words. DECIDED in that per-piece read: a particle that is suffix vocabulary too, with a name piece behind it in the part, heads that name and stays family (P2's join), so `VD MA` reads family whatever the case. Not decided in the chain: a comma followed by no name word reads that same part positionally, trailing credential included (`Berg de MA, Prof.` keeps suffix 'MA' and its report), and the chain runs before that choice is made. A first draft skipped the chain's stop for the whole family part and moved exactly that shape; a widened grid caught it. Declined: reading `VD MA` as credentials, family 'SMITH', which the comma's naming of the family argues against. `SMITH VD, JOHN` and `SMITH VD JR, JOHN` keep the suffix: nothing, or a suffix word, stands behind the particle. - BETWEEN CREDENTIALS AFTER A FAMILY COMMA (P6). `DOE, JANE PHD VD MA` read family 'VD DOE', suffix 'PHD, MA': assign had read `PHD VD MA` whole, and P6's attachment, which outranks the suffix reading for such a word by design, reached into the run and pulled `VD` out. DECIDED: P6 gains a third exception — a lone such word with a post-nominal read in front of it and another behind, a title between it and the one in front transparent (H5), keeps the post-nominal reading in any case, reported as S2's credential fork. The test is the roles assign gave, asking whether it read the run whole, so a member read as the credential in front speaks as a degree does (`DOE, JANE MA VD PHD` stays whole); a first draft asked S2's company query instead, which counts only an unambiguous credential in front, and missed that shape and the titled one (`DOE, JANE PHD PROF. VD MA`). A generation counts on either side, and that is a decision rather than a side effect: P6's tussenvoegsel stands right behind the given name, so any post-nominal between them says the word is not one, and `Berg, Jan PhD vd Jr.` reads suffix 'PhD vd Jr.' where 2.2.0 and 2.3.0 read family 'vd Berg', suffix 'PhD, Jr.' — the shape `Berg, Jan van Jr.` P6's rationale names has no post-nominal in front and still attaches. No title is skipped BEHIND the word: a title there leaves it a name in assign (`DOE, JANE PHD VD PROF. MA` reads middle 'VD PROF.', as before), so a skip there was dead code, measured never to fire over the grid below, and was removed. Boundaries: with nothing behind it the word ends the name and attaches (`Doe, Jane PhD vd` → family 'vd Doe', 2.2.0's and 2.3.0's reading); two such words side by side are one particle run, which P6 takes whole as its first exception says (`DOE, JANE PHD VD MC MA` → family 'VD MC DOE', unchanged); a plain particle between credentials was never read as a post-nominal and attaches as before (`Doe, Jane PhD de PhD` → family 'de Doe'). Declined: middle 'VD MA', 2.0.0's and 2.1.0's reading — under #554 run membership asks what the word IS. - ONE NEVER-GIVEN PARTICLE OPENING THE GIVEN PART (S2, P1). `SMITH, VD MA` read family 'SMITH VD', suffix 'MA'. S2's given-part slot reads a member ending the given part as the credential because "the words to spare are there by construction", the first word after the comma being the given name; a never-given particle is not one — P1's fold takes it forward into the family — so `MA` was read as a credential with no given name left, and the fold took `VD` alone. DECIDED: where the given part opens with a never-given particle and nothing but particles stands before the member, chained (`VD DE LA`) or not, the member is the word the particle attaches to and reads as a name: family 'SMITH VD MA', the reading of 2.0.0 through 2.3.0 and of `Smith, vd Ma`. The particle test is P1's own predicate (`_pieces.is_lone_never_given_particle`), so the slot cannot drift from the fold it predicts. Plain never-given particles share the shape (`smith, de ma`). Boundaries: capitals in a mixed-case name still make the member the credential, as they do with no words to spare anywhere (`Smith, de MA` keeps suffix 'MA'); an ambiguous particle can be a given name (`SMITH, VAN MA` keeps given 'VAN', suffix 'MA'); and a name word between the particle and the member is the word it attaches to, leaving the member a word to spare (`SMITH, VD BERG MA` keeps suffix 'MA'). Declined: family 'SMITH', suffix 'VD MA' (as `SMITH, PHD MA` reads) — under #554 `VD` speaks for nothing, so only the missing given name could have made `MA` a credential, and it is the particle's surname. - NOT DECIDED HERE: the mixed-case `Doe, Jane PhD vd Ma` (middle 'vd Ma', silent), the issue's fourth bullet. Its reading is unchanged; whether the family-comma path should report the chain's fork there is a separate question. Open: #578 (decided the same day, the S2 section's #578 entry: silent, as it reads). Nor the positional read of a part before a comma followed by no name word, where the case still splits the word: `Smith vd Ma, Prof.` reads given 'Smith', family 'vd Ma', and `SMITH VD MA, PROF.` family 'SMITH', suffix 'VD MA', on this tree and on `e56dd296` alike — that part reads as the comma-less name does, and the comma-less name splits the same way. - BLAST RADIUS, measured 2026-10-02 with `e56dd296` (the pre-change master) as the comparator. The gate's output is byte-identical at all five baselines apart from the tree path: no corpus name holds these shapes, the rules.md examples aside. Over a generated grid, 5,676 of 409,498 parses move, every one in one of two classes — 3,534 where a family gains the words behind its particle, 2,142 where `vd` or `mc` leaves the family for the credential run — and no list of ambiguity kinds changes; 1,310 of the movers are written in mixed case. Recompute: from the pool `vd mc de van do Ma MA PhD Jr John Berg V la Prof.`, build every `<1 or 2 words>, `, `<3 words>, <1 word>`, `Smith , John`, `Doe, Jane <2-4 words>` and `Smith, <2-4 words>`; deduplicate, then parse each as written, `.upper()` and `.lower()`, keyed by the resulting text; keep the entries whose `as_dict()` or sorted ambiguity kinds differ between the trees, and class each by whether the new family holds every old family word and more, or the old family lost only `vd`/`mc` while the suffix kept every word. Cost: the given-slot test was QUADRATIC as first written, rescanning the particle run once per member — the second review measured 29,670 → 165,591 frames on `SMITH, VD ` + `DE `×k + `MA `×k at k = 256 against `e56dd296`, where an earlier wall-clock check at k = 100 and 400 had read 3.18× and passed for linear. The run's end is now kept once per title set: 1.90–1.99× per doubling of k from 16 to 256, and `_REREAD_SHAPES["lead"]` in tests/v2/test_benchmark.py guards it (generator frames at 8 units against 32: 3.38× kept, 7.53× rescanning, bound 6.0). The frame-count reference band did not move. -- 2026-10-06 #613 — THE COMMA SITE IS DECIDED IN THE PASS THAT READS THE GIVEN PART. #611 found P6 showing the decisive sign smaller: assign held back from #602's run exactly the particle tail post_rules would attach a stage later, by asking `particle_tail` over roles that meant "not yet read" there and "suffix" in post_rules. The attachment and its third exception (`_inside_a_credential_run`) moved into assign, right after the given-part walk, and `particle_tail` is asked ONCE, at the moment the walk reaches #602's run: the pieces in front of the run hold the walk's roles and the run's words none, the state in which its answer equals the one over the finished roles (argued piece by piece in its docstring). post_rules ran P1's leading fold before P6, so assign asks P1's condition itself (`is_lone_never_given_particle` on the given piece), which is what keeps `Smith, de Mesnil van` the fold's. H1 and M4, the other rules ahead of the old site, are gated on there being no family, which a family comma always has. The no-comma site (#467) stays in post_rules, since it runs over the order P1's family-first redistribution leaves. NOTHING MOVES, measured 2026-10-06 against the commit before it: the P6 grid (103,786 distinct texts: each of the heads `Berg, Jan`, `Berg, Jan Piet`, `BERG, JAN`, `Berg, Dr. Jan`, `Berg, de`, `Berg, van`, `Berg, abdul`, `Smith, de Mesnil`, `de la, Jan` and `Nguyen, Thi`, followed by every sequence of zero to three of `van de der vd VD mc do DO Do PhD Jr. MA Ma Prof. V la Piet`, with and without a trailing `, Jr.` -- under the default policy, `lenient_comma_suffixes=False`, a family-first order and `middle_as_family`; comparator every token's role and folded or unjoined tag, every ambiguity's kind, detail and token IN ORDER, and `initials()`) and the #613 harness's 80,423 texts are identical, the gate's output byte-identical at all five baselines but for the baseline path, and the suite unchanged. ONE SHAPE DID MOVE, and the PR review found it on a grid with an EMPTY part before the comma, a head no measurement above held: there no family is named, H1 and M4 are not gated off, and they had made the given word the family before the old site attached -- `, Mr. Jones vd` read title 'Mr.', family 'Jones', suffix 'vd', and assign attaching first read given 'Jones', family 'vd' (177 texts on that grid). Reading P6 strictly instead (no family named, no attachment) moved 1,302 texts against master, `, Jan van` among them, so the attachment behind an empty family part stays in post_rules, after H1 and M4, through the same helper; that grid now differs from b547c247 only in C1's decided classes. Frames drop on every family-comma name, post_rules' block having paid a second `particle_tail` and a generator-driven search for the given word on every one, tail or not, where assign now attaches only behind a non-empty tail: `Smith, John` 169 → 160, `Smith, John, Jr.` 213 → 204, `Doe, Jane nee Smith` 302 → 293, a no-comma name unchanged. -- 2026-10-17 #700 — THE ELIDED PARTICLE AND THE HYPHEN-ATTACHED ARABIC ARTICLE ARE PARTICLE VOCABULARY, NEVER-GIVEN, AND THE RULE IS UNCHANGED. Library catalogs file `Paul d'Ivoi` as `Ivoi, Paul d'` and `Ḥusnī az-Zaʿīm` as `Zaʿīm, Ḥusnī az-`, the particle split off with its mark still on it; both read as a middle name (1.4.0 and 2.4 alike) because neither written form was in `PARTICLES`. P6 asks only the particle tag, so the fix is the vocabulary alone: no code moved. Three decisions, each taken by Derek on the #700 session's options. ELIDED SET: `d'`, `l'`, `dell'`, each also in its U+2019 spelling, and `de’` and `’t` beside the existing `de'` and `'t` (`’t` added in the PR review, a gap the test review found), since nothing folds the typographic apostrophe to the ASCII one; particles.py asserts at import that every elided entry has both spellings. ARTICLES: a CLOSED list — `al-`, `ul-`, the article assimilated to each of the fourteen sun letters in ASCII and DMG spellings (`ar-`, `az-`, `aš-`, `ash-`, `aṭ-`, …; lām's being `al-` itself), and the Egyptian `el-` with its assimilations in ASCII spellings only (`es-`, `esh-`, `ed-`, …; no `eš-` or `eṣ-`) — all never-given, where the bare `al` stays ambiguous (Al Gore): written apart with its hyphen, the word is the article and nothing else. RENDERING: the attached family joins with a space like every particle, `d' Ivoi` and `al- Rashid`. - Declined: a shape rule (a vowel, consonants, a hyphen) in place of the list. It would need a new predicate and the bare-word collision check the issue named, for a population the list already covers; the corpora attest seven article spellings (`al- az- aš- ar- es- as- aṭ-`), all in it — recompute: every whitespace- or comma-bounded token of `tools/differential/corpus*.jsonl` matching `\w{1,4}-` at a name's end. - Not decided here: the GLUED forms (`Paul d'Ivoi`, `Harun al-Rashid`) stay one word with no particle split, out of #700's scope by its own statement. Hyphen-minus only: U+2010 and the non-breaking hyphen are not listed. The Scottish Gaelic `a’` (`Mac a’ Ghobhainn`, in corpus_cldr.jsonl) is mid-name and not an index-form particle, and was not added. Nor are the elisions no corpus holds (`dall'`, `nell'`, `degl'`): the list is closed, so such a spelling reads as a middle name until it is listed. - RENDERING FOLLOWED THE VOCABULARY, found in review: R4's `capitalized()` split a token at its mark and asked the particle test of the stem, so `IVOI, PAUL D'` repaired to `Paul D' Ivoi` — 30 of the 32 first-listed spellings, all but `al-` and `de’`, whose stems are particles themselves. The whole token is now asked the particle arm's question first, and lowercased only where that arm would lowercase it; wherever the arm declines (an all-particle part, the given or suffix role, a recorded casing) the token goes to the per-word walk as before — a first version sent every whole-token particle down the arm's other clauses instead, which moved `smith, john, ed-` from 'ED-' to 'Ed-' and a configured `van-der` from 'Van-Der' to 'Van-der' (the fix commit's own review). The one shipped word outside #700 it reaches is `'t`, whose stem is `t`: `van 't Hoff` repairs to 'van 't Hoff', where 1.4.0 through 2.4.2 wrote 'van 'T Hoff' — R4's reading, and the right Dutch one. A casing configured for a stem (`capitalization_exceptions` `al` → `AL`) no longer reaches the marked spelling, which is now a particle in its own right: `al-` is lowercased. The snapshot cannot see a render view (decisions.md#R4); measured instead by `capitalized()` and `capitalized(force=True)` against `00dd2759`, over every name in `de6c045d`'s `tools/differential/corpus*.jsonl` as written, lowercased and uppercased (28 names move, every move lowercasing one token ending in `'` or `-`), and over every particle in eight positions under all three orders (adding `'T` → `'t`, which no corpus name held until R4's `van 't hoff` example joined corpus_rules.jsonl). The boundary `Ivoi, d'` was taken out of rules.md#P6: P6 does not attach there, and the reading it gets instead (family `Ivoi d'`, as `Jong, de` has read since 2.0) is the never-given vocabulary's family-comma reading, which P6's statement does not describe — a P6 example would have pinned another rule's output. It moves from given `d'` (1.4.0 through 2.4.2, the word being no particle there), and so does every new entry standing alone after a comma (`Smith, al-`); it stays pinned as a case row. - Measured 2026-10-17 against the parent commit `b4a004c3`: 39 corpus names move, every one the same shape — a trailing `d'` or article leaves the middle name and joins the family (38 in corpus_gnd.jsonl, `Orval, Pierre d'` in corpus_conventions.jsonl), and no ambiguity kind changes. Recompute: `uv run python tools/differential/snapshot.py --since b4a004c3`. That is evidence about the corpus as much as the change: these are the comma-bearing names ending in one of the new forms, and GND's index forms are where they occur. - -### P7 — a one-letter particle against the initial it spells - -- 2026-10-04 #604 — the PERIOD decides, stated as a shape rather than as a property of one word. `ó` is the first one-letter particle the vocabulary has held, and the particle match folds edge periods away, so `ó` in PARTICLES alone also claimed `Ó.` — the initial of Óscar, Ólafur, Ólöf. Measured on the branch before the veto: `Juan Ó. Pérez` and `J. Ó. Pérez` read family `Ó. Pérez`, `Pérez, Juan Ó.` read family `Ó. Pérez` too (P6 attaching the trailing "particle" behind the comma), and both the core's and the facade's `initials()` dropped the Ó. With the veto every one reads middle `Ó.` (given `Ó.` for `Ó. Pérez`) and initials keep it. A one-letter word written with its period is an initial, whatever particle the letter spells; The same line already stands for a single-letter connective, pinned in the code: 'e.' in 'john e. smith' is tagged an initial, not the Spanish conjunction (classify's initial veto on the conjunction tag). rules.md#P3 does not state it — P3 decides that fork by case, not by the period — and the only rules.md statement of the period line is R4's hyphen clause, scoped to a hyphenated word in case repair. -- WHERE the veto lives, and why it is one helper asked at four sites: `_lexicon._spells_an_initial`. Besides classify's tags, the particle vocabulary is read directly by `_vocab.surname_unit_tags` (the comma unit count, #575), `_render._cap_word`'s case repair, and the facade's `_is_particle` (the v1 last-name split and initials), and a site that kept bare membership read `Ó.` as a particle where classify does not: `Ó. Pérez, Ed` became family `Ó. Pérez`, given `Ed`, where `Q. Pérez, Ed` reads given `Q.`, family `Pérez`, suffix `Ed`. _render and the facade may not import the pipeline (tests/v2/test_layering.py), so the helper sits in the one module all of them already import. Every site asks it AFTER its membership hit, so a word no particle set holds pays no frame for it (AGENTS.md's frame budget: the classify call is per token). A fifth reader, segment's settled-run test in the comma credential walk (#562, deleted with segment's decision in #613), was NOT a site: an initial ends that walk before the particle question is asked, so a veto there changed nothing and was left out. Measured 2026-10-04 while the draft still asked the helper there (the call has since been removed, so that instrument no longer exists): with the helper answering False at that site alone, over every two-to-four-word arrangement of `Ó. ó. de vd Pérez Juan PhD MD VD DO do Do MA Ma Jr.` holding `Ó.` or `ó.`, in every comma position, under both `lenient_comma_suffixes` settings, no reading moved. To re-check it on this tree, wrap `_segment._normalize` so a one-letter period-written word never matches the particle set — the second review did, over 49,999 texts from `Ó. de vd DO do Pérez Juan MA PhD Ed Jr. MD VD van Q.` written as-is, upper and lower, under both settings: 0 moved, against 48 for a positive control that vetoes every particle at that site. The guard is the two-input invariant in tests/v2/test_properties.py — a name holding `Ó.` reads as the same name holding `Q.` in every view — with `_P7_SITE_EFFECT` recording, per site, how many grid texts break with that site's veto removed. -- 2026-10-04 (design review; Derek) — THE PERIOD IS THE ASCII ONE, matching `_vocab.is_initial`'s shape. The first draft took any of `FULL_STOPS`, so `Ó。` was vetoed although is_initial calls it no initial; now it is the particle, as `Q。` is no initial. The second half of is_initial — the scripts that write no initials (#320, `_policy._NO_INITIALS`) — is NOT reproduced: `_lexicon` may not import `_policy` (tests/v2/test_layering.py), and `_render` and the facade, which must ask the same question, can reach neither. Accepted as the residue: a CALLER's one-character particle in Han, Hangul or kana, written with a period, is vetoed where is_initial would call it no initial (`Lexicon.default().add(particles={'之'})` reads `王 之. 明` with `之.` outside the particle). No shipped particle is such a character. A hand copy of the script check, pinned by a sync test, was the alternative and was declined for a case only a caller can construct. -- Accepted: a bare `Juan Ó Pérez`, an Óscar initial written without its period, reads family `Ó Pérez`. Accented initials normally carry the period, and the bare letter is exactly how the particle is written. - -Declined: - -- A per-entry "matches only when written bare" flag (the issue's first framing, a new Lexicon field or set) — the period convention is a fact about INITIALS, not about `ó`, so it is stated once for every one-letter particle, including any a caller configures. A field would be one more vocabulary axis to translate through the v1 shim and to sweep in every guard roster, for a question no entry answers differently. - -### M2 — the maiden-marker rule - -- 2026-07-03 (maiden-bucket design; #274 filed 2026-07-07, landed in the v2 core, PR #288) — the marker takes everything after it up to a trailing suffix, greedily: "née Jones Smith" is a two-word maiden name, matching how the marker is actually used in running text. The marker itself is dropped as structural, like a delimiter character. - -- 2026-08-19 #399 (PR #409) — a maiden marker bounds a particle join reaching it from the left, where the marker is actually taken as a marker. rules.md carried the opposite as an Accepted limitation ("a particle chain swallows a marker in its path, the join (P2) running first"), but that was a consequence of stage order rather than a judgment: the chain merges pieces before the marker is consumed, and the consumer looks for a lone marker piece which by then no longer exists. So `Ursula von der Leyen geb. Albrecht` lost the maiden name that `Ursula Leyen geb. Albrecht` kept. - - The gate on the stop is the whole substance of the rule and was added in review, not in the first cut. Sharing one marker-piece predicate between the stop and the consumer settles what a marker piece IS; it does NOT settle whether the consumer takes, because the consumer also needs a non-suffix piece after the marker. Stopping without that left the marker as a piece of its own, and a lone trailing piece takes a role field: `Jane van der Berg née` read middle 'van der Berg', family 'née' — the marker became the surname and the real surname was demoted, which is the stranding the stop exists to prevent, one field over. Three consequences of the ungated form, each measured and each fixed by the gate: `Jane van der Nee` moved a Dutch bearer of the attested surname Nee out of the family (M1 already names Nee as a surname); `Ursula von der Leyen geb.`, an ordinary truncated record, reported the marker as the surname; and `St St née` stopped reporting its particle-or-given fork, because group's emitter is guarded on the chain having merged something and the stop made it merge nothing for a different reason than the guard assumes — an A1 violation of the shape #405 tracks, and worse than #405's, since it REMOVES a report callers already see. The gate tests the consumer's own condition rather than restating it, which is what makes the two halves agree. M2 states the gate ("a marker left as a word bounds nothing"); the trailing-marker rows pin it. - - Weighed against moving the maiden handler ahead of the chain, and the recorded reason for preferring the stop has been corrected twice. It is NOT that the reorder disturbs names whose maiden side carries its own particle chain: `Anna Müller geb. von der Berg` is identical before #399, after it, and under a reconstructed reorder, and mechanically cannot differ — no suffix and no conjunction on the maiden side means merged and unmerged pieces hand the consumer the same words. The reason is scope: the reorder changes what the consumer receives for every name rather than adding one condition, and two review sweeps found no test and no corpus name that separates the two options, so it would ship unmeasurable. A review sweep also found the reorder would reach maiden names the stop does not (the P3/P5 join-swallow below), and would drop P3's rootname count when the maiden words leave early. That second effect was recorded here as a COST and #411 has since shown it is the opposite: P3's count has the same defect P5's reserve had, counting words a later pass removes, so `juan y garcia nee jones` loses its family name entirely. Correcting the ledger rather than leaving it: the reorder's real price is scope — it changes what the consumer receives for every name — not that consequence. It stays available, and #418 tracks the P3 count on its own. - - Two things fell out rather than being decided separately: the same issue's open question about the leftover given slot (the marker used to compete for it, giving `de la Cruz née Vega` given "née" under a family-first order; consumed and dropped, it never reaches the placement — though a marker the consumer declines still does, which is M2's "just a word" reading and not a leak), and the wholly-particle family a stop can leave, which #404's R2 reading already covers — "Jane de la née Jones" reports family "de la" with a non-empty base. A 305,364-parse sweep at review found no empty-base or partition violation either side of the change. - - Two limits stay, now recorded in M2 rather than left implicit: the bound reaches only a marker standing as a word of its own, so P3's connective join and P5's bound-given join each still absorb a marker before the bound can see it — the deleted Accepted clause covered a class of which this fixed one instance and left two. (P5's was closed by #411 on 2026-08-21, and P3's the same day by the #412 / #417 / #418 entry below, which takes the reorder this entry set aside.) Tests carry the whole verification: of the marker-bearing differential corpus names none has a particle ahead of the marker, so the gate is blind to this class and its clean exit says nothing about the change. A synthetic cross-script row was written to reach the CJK corpus builder — the only route a case row has into any corpus — and then removed: it used that corpus's provenance as a back door, and it pinned nothing the Latin rows did not. The honest route is [#413](https://github.com/derek73/python-nameparser/issues/413), where the harvester's pattern turns out to miss every name written in backticks, which is how this tracker writes them. `Ursula von der Leyen geb. Albrecht` sat in #399's own title the entire time. - -- 2026-08-21 #412 / #417 / #418 (PR #420) — the marker pass runs before every join in the grouping stage. The three issues were one shape: a join rule computed something — P3's word count (#418), the chain's stop condition (#417), the connective join's range (#412) — against a piece list the marker pass would rewrite a few lines later. #411 had fixed the same shape in P5's reserve by hand, excluding the departing words and declining a join onto the marker, and #418 argued the general fix was a guard at the merge primitive: no join may span a marker piece. Stated unconditionally that guard is the ungated stop the 2026-08-19 entry above measured and rejected (it strands `Jane van der Berg née`), so it would have to be gated on "a marker the consumer will take", evaluated live at every merge — #417's shared predicate, plus a per-merge scan the `"and " * n` benchmark shape would have to tolerate, plus the count exclusions kept in step. The reorder set aside above gets the property by construction: the consumer runs first, what it takes is gone before any count or join looks, and the stop, the exclusion and the shared-predicate question all disappear, a net deletion. P5's decline-onto-a-marker test stays, because it is about what a name word is, not about counting — measured: removing it re-read `Berg, abdul née PhD` as given 'abdul née'. It tests the LONE marker piece, which is the only marker the consumer can have declined on its own; a declined marker the particle chain has already taken travels with the chain, and P5 joins the chain — `Abd van der Berg née Jr Jones` reads given 'Abd van der Berg née' where the old stop had kept the marker out of it. Review found this; rules.md#P5 now says "standing as a word of its own" for that reason. - - The recorded ground for setting the reorder aside was that nothing separated it from the stop. That stopped being true when #413's harvest landed, two hours after #418 was filed: eleven corpus names flip P3's carve-out under an appended maiden clause (eight losing the family — #418 counted seven and six, before the harvest), and seven show the connective join swallowing the marker — `johnny y`, `Juan de y`, `e and e` with the marker behind the connective, and `Claus SchroederVP of`, `Jon Dough and`, `Jon Dough and of`, `jones, john e` with the marker ahead of it — where #412 recorded no corpus name behind it at all (review counted the seven; the first draft of this entry had three). Measured over every corpus name, every corpus name with ` nee jones` / ` née Jones` / ` née Jr Jones` appended (comma names included) and every case row, 4,180 inputs: 55 differ, three of them inputs this change itself adds (the P3 example, its appended forms, the new case row), and each of the rest either a shape of the three issues or the clause ceasing to change how the rest of the name reads — `The Hon nee jones` reads given 'The', family 'Hon' because `The Hon` does; `der, y van nee jones` reads family 'van der' because `der, y van` does. No ambiguity report moves. The suffix-comma tails, the ordering cost the #411 entry left unmeasured, were byte-identical in every shape tried — under the default policy, which review showed was the wrong population. With `extra_suffix_delimiters` configured, a delimiter core after a marker in a tail became a word the pass could take (`Smith, John, PhD née - Jones` read maiden '- Jones'), and a core the pass left standing alone stopped being dropped (`Smith, John, - née Jones` read suffix '-', maiden 'Jones'): the tail's core drop used to run before the pass and now ran after it. Closed before merge by having the pass read a tail segment with its cores screened out and leave them for the drop — the old order restored for cores — with both shapes pinned at the piece level. Two corpus names move at the 2.0.0 and 2.1.0 baselines, the #412 pair; at 1.4.0 fix(#274) already classifies them. A third, `Juan y Garcia née Jones`, is the example this change adds to P3 and so to the rules corpus, and moves at all three — the one #418 shape the gate can see, the other eight showing it only under an appended clause. - - The one reading the order costs: the consumer sees unjoined pieces, so "up to any trailing suffix" now means the first suffix WORD after the marker, and a connective beside it cannot un-suffix it first. `Jane née Jr y Jones` read maiden 'Jr y Jones' and now declines (family 'Jr y Jones'). The shape is a suffix homograph ANYWHERE in the maiden name with a connective beside it, not only in first position — review measured `Jane Smith née Jones Jr y Smith` going from maiden 'Jones Jr y Smith' to maiden 'Jones', family 'Jr y Smith', middle 'Smith': the maiden walk stops at the homograph, and the connective then builds a family name out of what is left, demoting the real one. The first draft of this entry priced it as the first-word shape, from a sweep that appended only ` née Jr Jones`. Accepted rather than built around, and the one place a merge-time guard would have read differently. - - #417's question — whether `van der Berg née` is a family name — is answered by consistency rather than decided: a marker the consumer declines is M2's "just a word" and rides inside whatever join reaches it, which is the reading `Jane van der Berg née` already had. The fields of `Jane van der Berg née Jr Jones` do not change at all (middle 'van der Berg née Jr', family 'Jones'); only the pieces do, which is why its pin is at the piece level. The invariant test that hid the shape now lists it. - - Review added two pins the first cut had left to prose. A corpus-wide assertion that appending a maiden clause moves no other field (tests/v2/test_parser.py), with #410's lone-residual shape as its one boundary until #410 closed it on 2026-08-25 — `Dr. Jane née Smith` reads family 'Jane' now, as `Dr. Jane` does, so the assertion covers every corpus name that parses to anything, and the only skip left is M2's own (nothing at all before the marker). And a field-level row for the chain-carried marker. One guard is deliberately left without a pin: `_is_maiden_marker_piece`'s lone-piece half has no default-vocabulary input that reaches it now that the pass runs first (dropping it leaves the suite and a 337k-name review sweep identical), and it stays as M2's own definition — "standing as a word of its own" — rather than as a guard a test holds. - -Open (M2): -[#317](https://github.com/derek73/python-nameparser/issues/317) -the fullwidth-colon marker (旧姓：佐藤 arrives as one word; the head-peel question). - -- 2026-08-22 #424 — the walk stops before the trailing numeral assign reads as the suffix. "Up to any trailing suffix" was asked with the suffix-piece test, whose initial veto does not see a bare `V`: `John née Jones Smith V` read maiden 'Jones Smith V' where `John Smith V` reads suffix 'V'. The walk now also stops at `_trailing_start` — the S2 peel read from the marker on, over the pieces as they stand, stepping over a tail segment's delimiter cores as the walk does — but at the NUMERAL only. The bare-acronym fork counts pieces, and the walk removes the very pieces it counted: `John née Jones Smith Ma` peeled over the pieces as they stand reads the acronym as a credential with words to spare, and once 'Jones Smith' has left the name it is the family of a two-piece name — maiden 'Jones Smith', family 'Ma', which is worse than the maiden 'Jones Smith Ma' it had always read. The numeral fork reads the piece before the numeral, and the take removes that very piece: afterwards assign sees the piece before the MARKER there, and where that is initial-shaped the fork does not fire — the first draft stopped the walk all the same, and `J. née Jones Smith V` read maiden 'Jones Smith', family 'V' (both reviews found it: 27 of the design-docs review's 144 marker shapes, every one a head ending in an initial; the pool is unrecorded). The walk runs the peel again over the view the take would leave — the pieces before the marker and the numeral on, built as P5's reserve builds its view — so it stops only where the numeral reads as the suffix both as written and as left: `J. née Jones Smith V` keeps maiden 'Jones Smith V', as `J. V` reads the V as a name, and `Dr. née Jones Smith V` keeps maiden 'Jones Smith V', since assign peels the title first and a numeral alone is no fork (the first re-ask checked the preceding piece only and handed that V to the given name — the code review). As written too: `Jane née Jones J. V` keeps maiden 'Jones J. V' though the take would leave `Jane V`, the words as written being what the marker takes; the second docs review found the statement saying only the second half. The acronym is left to assign, recorded as Accepted. The peel is read from the marker rather than after it, so `Jane Smith née V` declines as `Jane Smith née PhD` does — nothing after the marker but a suffix, the marker stays a word, suffix 'V' — which is 1.4.0's reading (the first draft read maiden 'V', and the 1.4.0 gate is what showed it); and `Jane née Jones J. V` keeps maiden 'Jones J. V' (the piece before is initial-shaped). 1.4.0 had no maiden support, so the examples fall under the #274 rule at that baseline where its fields reach, and `J. née Jones Smith V` (v1 read suffix 'V') under an accepted rule of its own; against 2.0.0 and 2.1.0 the examples that move are classified — `John née Jones Smith V` in maiden and suffix, `Jane Smith née V` in the five fields the marker's survival touches — and `J. née Jones Smith V` is 2.x parity. - -- 2026-08-25 #410 — the maiden name no longer makes H1 decline; the rule counts name words and a marker-announced name is not one. The decision and its width are recorded at decisions.md#H1, since the term it removed was never maiden-specific. - -- 2026-08-26 #434 — a maiden marker may be a PHRASE, and the Polish "z domu" ships as the first one. The rule had assumed a marker is one word everywhere it was written down, and the library's own advice made that assumption load-bearing: the dead-entry warning told a caller with a multi-word entry to "split it into separate entries", which for this field produces two wrong readings rather than one inert entry. With z and domu stored separately, "Maria Kowalska z domu Nowak" reads maiden "domu Nowak" — the marker's own second word inside the value #434 reports — and "Anna z Nowak" loses its family name outright, because a bare preposition is then a marker and M2 hands a marker every word after it. So the warning was not merely unhelpful for this field; following it was the defect. A phrase entry matched whole is the only shape that fixes both: z is an ordinary Polish preposition and domu an ordinary noun, and neither ever claims anything alone. - - A run predicate — identify a run of marker words, then look the joined run up — was rejected, and the reason is the difference between the two exceptions this makes. given_name_titles is the STORAGE precedent and NOT the matching one. Its entries are stored space-joined and folded per word (_lexicon._title_key, since whole-phrase normalization would leave 'lt. col' with its interior period), and markers copy exactly that. But its MATCHING identifies the run per word first — 'lt' and 'col' are each title vocabulary, so the parse has already read both as titles before anything is joined — and z and domu are markers in no vocabulary at all, so no run can be identified before the lookup. Markers therefore need genuine multi-token lookahead, longest first, which is a different mechanism wearing the same storage rule. Worth knowing that until this change NO shipped vocabulary set held a single multi-word entry, given_name_titles included: the exception existed for caller-configured vocabulary only, and "z domu" is the first phrase the library itself ships. - - That the phrase is safe where its words are not is the C-i question answered one level up, and the contrast worth drawing is with the neighbour in this section's Excluded block. #vocabulary-collisions C-i asks whether a word is borne as an ordinary name in the position the vocabulary claim acts on; roz fails it — Roz is a diminutive of Rosalind and stands exactly where M2's claim acts, the word after a given name — and MAIDEN_MARKERS has no ambiguous subset to express a partial answer in, so the only remedies available there were ship and do not ship, and it was removed. z fails C-i at least as badly. What changes is that the claim need not be made about z at all: the phrase is the unit that claims, and "z domu" is borne as no name in any position. C-i is a test on the claiming unit, and a phrase entry is a way of choosing a different unit rather than a way of passing the test with the old one. This is not a general escape — a phrase whose words are common and whose pairing is also common would fail C-i as a phrase — but it is the answer whenever the collision is with a word that only a longer phrase makes unambiguous. - - Markers are consulted at FOUR sites, which is one more than the title-run key's three, and _title_key's own docstring names the failure mode ("a divergence between them fails silently: the entry simply stops matching"). So one predicate answers "does a marker start here, and where does it end" — _pipeline._vocab.maiden_marker_run, in the vocabulary layer because it is a question about token TEXTS, per mechanisms.md#ONE-PREDICATE-PER-QUESTION's rule that the destination follows the layer. The four are classify's tagging, extract's clause test, and group's TWO — the piece test M2's take is built on and the drop inside an extracted clause (#329), which is the pair that makes four where titles have three. classify calls the predicate in a sequence pass, tagging a run's head "vocab:maiden-marker" as before and its continuations "vocab:maiden-marker-cont"; extract calls it over a clause's whitespace words; both of group's sites read those tags rather than re-deriving the run, which is the same mechanism's recorded-answer half, available because group runs after classify. tests/v2/pipeline/test_group.py::test_every_marker_site_ends_the_run_in_the_same_place is the contract, modelled on the P5/H1 title-run test: it collects each site's answer to "how many words is the marker" over nine spellings and asserts they cannot disagree, and each of the four sites has been mutated to confirm it fails. - - One divergence between those sites is deliberately PRESERVED and must survive any future unification. extract splits a clause on WHITESPACE while the tokenizer splits punctuation off, so "née," is marker-led to classify and not to M3. That is what keeps group's role filter reachable, and cases.py::marker_glued_to_punctuation_keeps_the_clause_a_nickname is the row that pins it — 'Jane "née, Janie" Smith (née Jones)' reads nickname 'née Janie', maiden 'Jones'. Sharing the predicate did not merge the two questions, because each site hands it a different sequence of words; confirmed by mutation, where making _maiden_marked strip commas leaves that row (and its facade twin) the only failures. SUPERSEDED IN PART 2026-10-12 by #653: inside a clause the tokenizer now keeps a comma on its word, so 'née,' is no marker to either site and that row reads nickname 'née, Janie'. The divergence survives through rules.md#T2's nakaguro, which tokenize divides at and the whitespace split does not, and cases.py::marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname is the row that pins it now ('佐藤 花子 "旧姓・田中" (旧姓 鈴木)' reads nickname '旧姓 田中', maiden '鈴木'). - - The commit order carries a finding about this repo's tests worth stating on its own. Taken in the plan's order — predicate, then storage plus the warning carve-out, then the sites — the middle commit would have exempted maiden_markers from the dead-entry warning before anything could match a phrase. That is not merely untidy: pyproject sets filterwarnings = ["error"], so a test that configures a phrase entry FAILS until the exemption lands, which means the carve-out cannot be deferred past the sites either. The two constraints leave one arrangement — the predicate alone, then storage and the four sites together, then the shipped entry — and the middle commit is fused for that reason rather than by preference. Shipping "z domu" last is what keeps any bisected revision from holding a phrase entry nothing can match. - - Review found one real defect, and it is the one this whole design exists to prevent. A marker run is TAGGED over the whole span-sorted token stream and CONSUMED over one segment, and the two populations are not the same: segment's role-None filter keeps only tokens extract has not already given a role, then buckets what is left by the commas before them. So a phrase written across a bracketed clause — "Anna z (domu) Nowak" — had both its words tagged while the piece walk could see only the first, and M2 took a proper PREFIX of the phrase for the whole marker: family 'Anna', maiden 'Nowak', a bare Polish preposition eating the name, and "Maria z (domu) Kowalska Nowak" losing its family name outright. Reachable on shipped vocabulary under the default policy, and it falsified three of this branch's own claims at once, in three files. The fix is a rule about the TAG rather than a guard on the consumer: classify refuses to tag a run whose tokens are not structurally contiguous — same role, so a clause edge ends a run, and the same comma bucket, computed the way segment computes it. Refusing rather than truncating, because truncating hands M2 the same wrong prefix one word shorter. The lookahead is bounded at the boundary before the predicate is asked, so a two-word entry refused at a clause edge still leaves a one-word entry starting there free to match, and an all-single-word vocabulary skips the walk entirely. rules.md#M2 now states the qualifier — a phrase is recognized only whole AND only where its words stand together — which it did not, and that gap is part of why the defect was invisible. - - The defensive complement was declined and the reason is worth recording: having _marker_run_pieces re-derive the run from token indices and refuse a mismatch would be a guard that, with the tagging rule in place, can never fire — untestable code standing in for an invariant one stage already guarantees. What it would have bought is written into that function's docstring instead, which now names segment's role-None filter as the second reason `seen` can skip a token; the sentence that said the continuation "is always in `seen`" without naming that reason is what made the gap invisible to review. - - The contract test should have caught this and did not, and the hole is worth naming because it is a property of how the contract was FORMULATED. It varied the marker's SPELLING across nine forms and held its PLACEMENT fixed, so every run it built was structurally contiguous by construction and no probe ever asked what the sites do when a run straddles a boundary — which is precisely where two sites walking different token populations can disagree. The placement axis is now its own test over eight placements, asserting two things no spelling can reach: that every tagged run has one role and one comma bucket, and that group drops a run whole or declines it. Mutation-checked in three pieces, which is what shows the two halves of the rule are both load-bearing: removing the contiguity bound outright fails both new case rows and all three straddling placements; removing only the ROLE half fails the two clause placements and both rows; removing only the COMMA half fails exactly one placement and NO case row, because a comma-split run happens to be declined downstream for an unrelated reason today. That last one is the argument for a tag-level contract test in its own right — the only witness to half the rule is an assertion about tags, not about fields. - - Perf, and how it was measured, because the first attempt at it was wrong in a way worth recording. Timing four source trees in separate processes and comparing them across sessions gave figures whose spread was the size of the effect: merged master alone measured 0.579s, 0.593s and 0.607s on one machine inside an hour. Every number below is a median of five INTERLEAVED rounds — every variant timed once per round, in order, so drift lands on all of them alike — over 8000 parses of a four-name mix (plain, titled-with-initials, phrase-marked, comma-and-particle). Against master: the run pass alone +6.8%, a head-word fast path inside the predicate +1.4%, the contiguity rule on top of that +1.6%, and after the simplify round -0.3% and -1.3% on two separate five-round runs. So the shipped mechanism has no cost this benchmark can resolve. That is not a free lunch and should not be read as one: classify no longer does the per-token `n in lexicon.maiden_markers` membership test master did — the marker decision moved into one pass that answers for every token at once — so what the marker pass adds and what it removed are within noise of each other. The reviewer reproduced the +7.2% and +1.6% stages independently on other hardware, which is the corroboration a percentage of this size needs. - - Four structural choices got it there, and each is a smaller version of the same move: ask the question once and pass the answer along. The fast path lives INSIDE the predicate rather than at its callers, so it stays an implementation detail and not a second answer. maiden_marker_head takes the FOLD rather than the text (suffix_as_written's shape, "n is _normalize(text), passed in so callers normalize once"), and classify folds once per token for the marker pass and the vocabulary tags together, which took the common token from two folds to one — fewer than before this work. maiden_marker_run folds each word once and keys from a prefix, where _title_key(words[:n]) per candidate length re-folds the whole prefix and made a ONE-word hit cost more than a two-word one; sixteen of the seventeen shipped entries are one word. And the marker pass returns its decision as index -> tag for classify's single token-building loop to write, rather than rebuilding every marker token a second time with dataclasses.replace, which is the most expensive call in a parse. The comma-bucket sweep the contiguity rule needs is deferred until a marker head is actually found, so a single-word vocabulary and a phrase vocabulary over a name holding no marker both skip it entirely. - - One limit is recorded rather than closed. The contiguity test is role plus comma bucket, and role is one-directional: a role change is always a clause edge, but two ADJACENT clauses of the same role are indistinguishable, so 'Jane (z) (domu) Jones' does tag a run across two separate nickname clauses. Nothing reads it — the piece walk never sees role-bearing tokens and the clause drop is scoped to one span — and the parse is identical with and without the phrase mechanism. The clean fix is a per-token structural UNIT id derived from state.segments and state.extracted, which would replace role-plus-bucket with one comparison and close the adjacent-clause case as a side effect. Deliberately not taken here: it changes a mechanism four correctness reviewers had just cleared, inside a pass whose stated business was not to change behavior. It is the shape to reach for if this test ever needs to grow a third term. - -- 2026-09-19 (Derek), #533 — THE CLAUSE ENDS AT A TRAILING CREDENTIAL, AND THE DOUBLE CHECK THE 2026-08-22 BULLET DEFERRED NOW EXISTS FOR THE ACRONYM TOO. That bullet stopped the walk before the trailing numeral and left the bare-acronym fork alone, with its reason stated: the acronym fork COUNTS pieces and the walk removes the very pieces it counted, so a peel read over the pieces as they stand said "credential with words to spare" about `John née Jones Smith Ma` and would have left maiden 'Jones Smith', family 'Ma'. The reason was right and is unchanged; what it argued for was a deferral, and the answer is the one the numeral got — ask the peel TWICE, once over the pieces as written and once over the name the take would leave, and stop only where both read the word as the credential. "Left to assign" meant "read in silence", assign never getting the question at all, the clause's tokens carrying the maiden role before pieces exist for it: the end of a maiden clause was the last trailing position in the library where a member of the ambiguous credential class was read without a report. WHICH RULE reads the name left standing is the part the numeral's version did not have to answer, because a numeral reads the same way everywhere and a credential does not. With no comma it is the trailing peel; after a family comma it is #531's slot, where the comma has already settled the count and the writing decides alone; before that comma the words are the family the comma named and a stop would hand one of them to `family` rather than to `suffix`; past a second comma the segment is read as credentials whole and no trailing rule is consulted. The last two READ AS NONE, which is a statement and not a default — the first prototype reported on `Smith, John, Jr nee Jones MA`, where the walk had asked nothing. A FIFTH CASE, found by the design-docs review of this commit and not by the plan, and THE PR REVIEW THEN SETTLED IT THE OTHER WAY — see the review bullet below, which is where the reasoning now lives. The shape: after a family comma the reader may take the word and nothing may then report it. `Jane Doe, Jr nee Smith MA` is a family comma whose segment 1 is `Jr nee Smith MA`, so the given-slot reader is asked and would take the member, but the take leaves that segment as `Jr MA`, post-nominals only, which the credential-run gate reads whole without ever reaching the emitter — and the released word therefore lands in `given`, not in `suffix`. This commit read it as maiden 'Smith', suffix 'Jr MA', silently; the review round declines the stop instead, so the clause keeps 'Smith MA' and reports, which is 2f57ff21's reading. The KEPT direction always reported here, the clause's own emitter being what raises it: `Jane Doe, Jr nee Smith Ma` and `Jane Doe, Jr nee MA` both say so, and now so does `Jane Doe, Jr nee Smith MA`. Pinned by a unit test rather than by a rules.md example line: the name differs from the tree at all five baselines and most of the diff is not this change's (1.4.0 reads title 'Jr', given 'nee', family 'Jane Doe'; 2.0.0 and 2.1.0 title 'Jr', family 'Jane Doe'; 2.2.0 and 2.3.0 already read given 'Jane', family 'Doe', suffix 'Jr'), so an example line would have put five ledger rules into this docs commit to attribute four other changes' readings. THE SECOND CHECK ASKS WHETHER THE READER TAKES THAT WORD, not whether it takes something, and the distinction is not academic: the numeral's existing re-ask tests whether the view's trailing run starts at the view's end, exactly right for a fork that reads one piece, and under it `JOHN NEE JONES SMITH MA PHD` leaves `JOHN MA PHD`, whose peel takes 'PHD' and then DECLINES 'MA' for want of words to spare — so the weaker question answers yes while the member becomes the FAMILY name, which is the 2026-08-22 disaster one word further on. Measured both ways on the prototype; the shipped check compares the view's run start against the member's own index in that view, and `JOHN NEE JONES SMITH MA PHD` keeps maiden 'JONES SMITH MA' with suffix 'PHD' and reports. AFTER A FAMILY COMMA THE CHECK NEEDS #531'S FLOOR, NOT ONLY ITS LEAN: `Doe, Jane nee Smith MA do` leaves `Jane MA do`, where 'MA' leans credential but the 'do' behind it does not read as a suffix (P6 keeps it, and `Doe, Jane MA do` reads middle 'MA', family 'do Doe'), so #531's slot would read 'MA' as a MIDDLE name — and a check that asked only about 'MA' released it from the clause into the current name's middle, a word crossing from one person's name to another's. With the floor the clause keeps 'MA do' whole and reports the 'do'. THE FLOOR ON THE FIRST WORD IS A CLAMP, NOT A VETO, and the first draft got two names wrong by making it one: stated as "a member that is the ONLY word after the marker stays the maiden name" and implemented as "cancel the stop when nothing would be left", it cancels too much wherever the peel consumed that first word AND words behind it — `Doe, J. nee MA ba` peels 'ba' and then 'MA', so the first piece the peel took IS the only maiden word, and the veto handed 'ba' back to the clause, giving maiden 'MA ba' where `Doe, J. ba` reads suffix 'ba'. Clamped to the piece after the marker instead, that name keeps maiden 'MA' and reads suffix 'ba', reporting both. The floor itself is deliberate and diverges from the `née V` / `née PhD` precedent, where the marker declines and stays an ordinary word: certain suffix vocabulary declines the marker, while an ambiguous word is kept by the clause it ends, because that class is the one carrying no evidence of which it is — its members are borne surnames as well as credentials, and nobody writes a credential straight after the marker. 1.4.0 read `Jane Doe nee MA` as last 'nee', suffix 'MA' (measured on the wheel), so this half restores nothing and does not try to. THE `do` PAIR is #531's reading, unchanged and now SHARED as a predicate rather than spelled twice (`credential_at_the_given_slot` in `_pipeline/_pieces.py`; the drift would have been silent, each site's own tests going on passing): `Doe, Jane nee Smith do` keeps maiden 'Smith do', `Doe, Jane nee Smith Do` keeps 'Smith Do', `Doe, Jane nee Smith DO` gives maiden 'Smith' with suffix 'DO', and the comma-less `Jane Doe nee Smith do` reads suffix 'do' — right for the reason the whole change is, since `John Doe do` reads suffix 'do' today, P6 does not run without a comma, and a clause must not change how a word outside it reads. ACCEPTED COSTS, each measured on this tree: `JANE DOE NEE YO-YO MA` reads maiden 'YO-YO', suffix 'MA', a two-word birth surname losing its last word in a one-case record — and `JANE YO-YO MA` reads suffix 'MA' too, so the cost is the one-case reading's rather than this rule's, and in mixed case the writing saves it (`Jane Doe nee Yo-Yo Ma` keeps maiden 'Yo-Yo Ma' and reports); `Jane Doe nee Smith X.Y.Z.` reads suffix where 1.4.0 read last 'X.Y.Z.', the same trade #531 took at its own slot and for agreement with `John Doe X.Y.Z.` rather than for parity; (two costs listed here were WITHDRAWN by the PR review and are no longer costs at all — `Doe, Dr. nee Smith MA` and `Berg, abdul nee Jones MA` both keep their maiden words and report; the bullet below carries the reason); `Jane Smith nee Jones and MA` reads maiden 'Jones and' with suffix 'MA', the take running before the joins (#420) so the connective is a piece of its own when the walk stops; and a report on a declined word adds noise without moving a field, which is #530's stated rule and the trade #531 took at its slot. NOT TRANSPARENT HERE: a trailing title. `Jane Doe nee Smith MA Prof.` is unchanged and silent while `Jane Doe nee Smith Prof. MA` gives maiden 'Smith Prof.' with suffix 'MA' and reports, because the walk reads the trailing credential run and the H5 title chain is read by the rule the walk does not run. Accepted for #533, pinned as a pair of boundary example lines under M2; moving the walk onto the title-aware reading would change the numeral half too, which the 2026-08-22 bullet settled with its own measurements, and the pair belongs to one decision about what "trailing" means inside a clause — a follow-up is drafted for Derek rather than filed here. THE `one_case` PLUMBING AT THESE SITES IS NOW LIVE, and the 2026-09-18 bullet under `### S2` headed THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT stands as what was true then: `numeral_only` answered off the peeled numeral and the numeral fork is decided before the peel reads a lean, so the fact reached only the bare-acronym fork, which that reading discarded. The acronym fork is asked now. Re-measured 2026-09-19 with the same wrapper shape over the same population that entry's recipe names — AND THE PAIR IS THE FINDING: over the corpus as it stood the day before this change, dropping the argument inside the walk moved 0 parses; over the corpus WITH this change's own rows it moves 36 of 10,752 (1,792 names), on six distinct names (`Doe, Jane nee Smith DO`, `Doe, Jane nee Smith Ma`, `Jane Doe nee Smith Ma`, `Jane Doe nee Smith Ma JD`, `Jane Doe nee Yo-Yo Ma`, `John née Jones Smith MA`). The population, the six policies and the denominator are recorded at the call site in `_pipeline/_group.py`, as the 2026-09-18 entry records its own; both were re-run on 2026-09-19 after the review found the figures first written there (1,790 names, 10,740 and 21,480 parses) did not reproduce from the recipe as stated — they had been taken two corpus names earlier in the same day's work. The FINDING was unharmed, 36 parses on the same six names. The plumbing was live either way and the corpus could not show it, which is the blindness mechanisms.md's corpus field note asks to be measured before any "N names move" is written down. - -- 2026-09-19 (Derek), #533 review — A WORD THE CLAUSE GIVES UP READS AS A POST-NOMINAL OR THE CLAUSE KEEPS IT, AND THAT INVARIANT REPLACES THE ACCEPTED SILENT MOVERS ABOVE. The PR review measured what the released words actually do, and three of them do not land in `suffix`: `Doe, Prof. nee Smith A.B.` read given 'A.B.' (66 distinct names in the review's own sweep, the same shape reaching `ba`, `X.Y.Z.`, caps-on `XYZ` and the lower-case spellings); `Berg, Jane van der nee Smith DO` read family 'van der DO Berg' and `Jane Doe nee Smith DO DO` read family 'DO DO', a word crossing from the BIRTH name into the current one, which is #424's failure from the other side; and `Berg, abdul nee Jones MA` read given 'abdul MA'. Two of those had been written up as ACCEPTED COSTS in the bullet above, on the argument that the clause form AGREES with the clause-less one. The agreement is real and it is not the question: the clause-less `Berg, abdul MA` puts the word in the given name because nobody promised otherwise, while a CLAUSE promised that the word was either the birth name or a credential, and a third answer is the promise broken in silence. So the direction is the parent's, conservative: THE STOP IS RIGHT ONLY WHERE THE RELEASED WORD ENDS THE PARSE SUFFIX-ROLED, and where the take cannot promise that, the clause keeps the word (and reports it, the emitter asking whether a trailing rule reads these words at all rather than whether the view check then passed). Two view checks carry it, each modelling a LATER decision rather than predicting one: the given-slot reader requires the view to still hold a name word AHEAD of the member, since a trailing slot needs a part to be the end of; and a release is withdrawn where a join reached below the marker pass would absorb the word — P2's chain, for a particle-tagged member behind a particle piece or beside another released one, and P5's bound-given join, for the word after the bound one. Both over-decline rather than predict, which is the direction the invariant asks for. MEASURED, and the pair of numbers is the argument: over the review's 142,518-name corpus under six policies, 855,108 parses, the invariant holds 0 violations at 2f57ff21 and 0 here, against 1,446 in 40 shapes at the commit this round started from; and a token-level diff against 2f57ff21 finds 128,424 moved parses on 24,958 names with ZERO of the moved tokens unnamed by a `suffix-or-name` report, against 4,100 silent moved tokens in 59 shapes before. `tests/v2/test_properties.py` carries the invariant as a property over a generated grid rich enough to hold every shape above; the grid fails on 790 of its 8,466 parses at d97d3eb7 and passes at 2f57ff21, which is what makes it a test rather than a restatement. TWO ACCEPTED SILENCES REMAIN AND ARE NOT SILENCES OF THIS KIND: a member with a name word BEHIND it was never asked about, and a delimited clause is not a fork at all (below). ONE ROUGH EDGE IS RECORDED RATHER THAN FIXED, and it PREDATES this change: with `extra_suffix_delimiters=('-',)`, `Jane Doe nee Smith - MA` gives maiden 'Smith -' — a clause ending on a bare delimiter token. 2f57ff21 does the same thing one word later (`Jane Doe nee Smith - PhD` gives maiden 'Smith -' there too), so what #533 changes is only which words reach the edge. Walking the stop back over it is not the two-line change it looks like: the delimiter-core set the walk already steps over is EMPTY off a tail segment, which is exactly where this name's `-` sits, so the fix would have to widen what counts as a core on every path and would move the parent's reading as well. Left for its own issue. - -- 2026-09-19 (Derek), #533 review — THE DELIMITERS TRUMP ANY OTHER READING, and that is a settled position rather than an un-asked fork. `Jane Doe (nee Smith Ma)` and `Jane Doe (nee Smith MA)` both keep the WHOLE span as the maiden name and report nothing, in every release including this one; the marker INSIDE the delimiters is what tells the parser the span is a maiden clause, even where the pair is not configured in `maiden_delimiters` (M3). The writer drew the boundary, so no fork is called and there is nothing to report — which is a different kind of quiet from the silences M2 and `AmbiguityKind.SUFFIX_OR_NAME` document, where a fork WAS available and the parser declined to ask it. The boundary cuts both ways: `Jane Doe (nee Smith) MA` gives suffix 'MA' and reports, the member standing outside the span being an ordinary trailing credential. Behavior is unchanged from 2f57ff21 and was not touched; what the review added is the three rules.md example lines, the case rows, and the statement in M2 and in the `SUFFIX_OR_NAME` docstring, because an undocumented settled position reads as an oversight to the next person who measures it. - -- 2026-09-20 (Derek), #397 review — A LINK INSIDE A MAIDEN CLAUSE STAYS IN THE BIRTH NAME, AND THE SPEC'S Q3 WAS WRONG ABOUT THIS BY MEASUREMENT. The walk has ended the birth name at the first suffix WORD after the marker since the #420 reorder, and the Catalan link is also the roman numeral, so `Jane Doe nee Puig i Soler` read maiden 'Puig' and left 'i Soler' outside the clause — middle 'Doe i', family 'Soler' at master 46651750, and the same truncation at every 2.x release. THE TRUNCATION PREDATES #397. What #397 added is the harm: with the link joining, those leaked words merged into the CURRENT surname, and 0fbcaa0b read family 'Doe i Soler' — a word of one person's birth name inside the surname they carry now, which is the class of failure #424 and #533 exist to prevent, arrived at from a third direction. The comma form leaks into the given part instead (`Doe, Jane nee Puig i Soler` read given 'Jane i Soler'), and a Catalan head loses the distinction between its own link and the clause's (`Maria Carod i Rovira nee Puig i Soler` read family 'Carod i Rovira i Soler'). THE SPEC'S Q3 said this interaction was unchanged; it was written from the rule and not from a parse, and the four names above are what a measurement said instead. Derek decided on 2026-09-20 to fix it in #397's own PR rather than file it, on the ground that the PR is what turned a truncation into a leak. - THE FIX IS P3'S OWN CLASS TEST, ASKED WITH THE CLAUSE'S BOUNDS. The walk does not stop at a suffix piece that is a connective PLACED TO JOIN inside the clause — a name word of the clause on its left and one on its right, which is the condition the 41de3df6 join already asks through `_name_word_beside`, called here rather than restated (mechanisms.md#ONE-PREDICATE-PER-QUESTION). Two bounds make it the CLAUSE's question and not the segment's, and each is a control: the left bound is the first piece after the whole marker run, so the marker is no name word (`Jane Doe nee i Soler` declines as before, family 'nee i Soler'); the right bound is where assign's trailing run begins over the pieces as written, so a generation or credential ending the clause is outside it (`Jane Doe nee Puig i III` keeps suffix 'i III', `Jane Doe nee Puig i MA` suffix 'i MA' — and the MA one needs the BOUND, carrying no `vocab:suffix` tag for the piece test to refuse it by). The right bound is `trailing_start`'s own answer read off the peel pair already in hand, not a second call. #533's trailing-credential stop and #424's numeral stop are byte-identical across the change, measured. - MEASURED, and the pair of numbers is the argument. A new property grid — 4,590 clause texts (five heads including a Catalan one with its own link, all seventeen shipped markers, six clause bodies, five trailing runs, comma and no-comma) against the default policy, both family-first orders and strict comma, 15,810 rows, of which 10,540 hold a link the `y` spelling puts between two birth-name words — carries two invariants in `tests/v2/test_properties.py`. THE y TWIN (`y` is the same connective with the ambiguity removed, so it is an oracle and not a second subject): the `i` parse's seven fields equal the `y` parse's, letter for letter apart from the link. It fails on ALL 10,540 guarded rows at 0fbcaa0b AND at the parent 46651750 — no clause shape holding a joining link read as its twin did — and on 0 here. NO BIRTH-NAME WORD IN THE CURRENT NAME, delimited by the twin's own maiden tokens: 28,985 tokens at 0fbcaa0b against 22,185 at 46651750, and 0 here. The GAP between those two is exactly what this branch added, and it is why the y twin alone would not have told the story: the twin fails identically at both commits while the harm doubled between them. - THE GUARD IS NARROWER THAN "the twin keeps the link", by measurement rather than by preference. `Jane Doe nee Puig y` keeps its `y` in maiden 'Puig y' while `Jane Doe nee Puig i` reads maiden 'Puig' with suffix 'i' — deliberately, the link joining nothing there — so a guard asking only whether the twin kept the letter would demand the two agree where the rules say they must not. `Puig y III` and `y Soler` are the same shape from the other two sides, and the grid keeps all three as the bodies the guard has to refuse. - INV6's off-switch exemption is NOT widened, and the reason is the one AGENTS.md gives for not widening an exemption a grid cannot reach: measured 2026-09-20, 1,680 rows of that grid carry both a class letter and a marker and 0 of them put the letter after the marker, its generator placing the connective among the name's own words. A clause link has its own grid instead. - TWO SECOND-ORDER MOVEMENTS, both recorded rather than repaired. The two one-case spellings part company, which is P3's own Accepted clause reaching a taken marker rather than a declined one: classify leaves a letter after a marker to the mixed-case rule, so `JANE DOE NEE PUIG I SOLER` never moved (the capital is an initial, and the initial veto kept the walk going all along) while `jane doe nee puig i soler` goes from family 'doe i soler' to maiden 'puig i soler'. And a report is GAINED where a longer clause ends on an ambiguous credential: `Jane Doe nee Puig i Ma` keeps maiden 'Puig i Ma' and says `suffix-or-name` where 0fbcaa0b read family 'Doe i Ma' in silence — the clause's own emitter, reaching a word the truncation had put out of its reach. A THIRD, found by the second review and the mirror of that one: a report is LOST where the clause takes a class member it used to end at. `Jane Doe nee MA i Soler` reads maiden 'MA i Soler' in silence where the parent 46651750 read maiden 'MA' with a `suffix-or-name` on it. That FOLLOWS from the link no longer ending the clause and is not a second decision: the report is the walk's own, raised on the LAST word the clause kept where a trailing rule was asked about it, and with the link joining, 'MA' has a name word behind it and is no longer that word — which is exactly what `Jane Doe nee MA Smith` has always done with the same acronym. The `y` twin is the oracle and it AGREES, in fields and in reports: `Jane Doe nee MA y Soler` reads maiden 'MA y Soler' in silence at the parent and here. Measured 2026-09-20 over the review grid, 24 names lose the report this way and all 24 agree with their `y` twin on every field and every report. - -- 2026-09-22 #397 follow-up — A DELIMITER CORE PAST THE CLAUSE'S FIRST WORD PASSES FOR THE NAME WORD BESIDE A LINK, RECORDED AS A DEVIATION RATHER THAN REPAIRED. The link exception above wants "a connective standing between two name words of the clause", and a separator the caller declared through `Policy.extra_suffix_delimiters` is structure, not a name word — so a link with one beside it joins nothing and should end the clause like any other suffix word. Between the marker and the clause's first word that already holds, the bound refusing the core before either piece test is asked. PAST that first word it does not: the core is an ordinary index to the run walk, which steps over connectives and nothing else, so it stands in for the name word on the link's left and the clause runs on past a title it would otherwise stop at. MEASURED 2026-09-22 under `extra_suffix_delimiters=(" - ",)`: `Smith, John, PhD née Puig Mr. - i Soler` reads maiden 'Puig Mr. i Soler' where its separator-less twin `Smith, John, PhD née Puig Mr. i Soler` stops at 'Puig Mr.'. THE POPULATION is the branch's own sweep, recorded with the code it describes (2026-09-21, corpus ∪ cases.py ∪ the property grids ∪ a 50,925-name generated set with cores, under thirteen core-bearing policies): the predicate is asked about a core in 51,072 of 900,023 calls, the answer differs from a core-skipping reading in 8,094 parses over 1,278 texts, and 1,824 of those move `maiden` on 288 texts — none of the 288 reachable at the default policy, `extra_suffix_delimiters` being empty there. NOT REPAIRED HERE: the fix threads the core set through three call sites into the run walk and moves the parent's reading as well, which makes it its own change rather than a rider on a review round. Open: #538. PINNED TWICE MEANWHILE. rules.md#M2 carries the shape as a `deviates: #538` example under an `extra_suffix_delimiters-dash` annotation — the first entry `tests/v2/rules_doc.py`'s registry has had for that field, named after the Policy field and carrying the delimiter in the suffix because the field's value is a set rather than a flag. And `tests/v2/pipeline/test_group.py::test_a_core_beside_a_link_wrongly_passes_for_a_word_until_538` holds the pair at the piece level, named so nobody reads it as the contract. ONE COST OF THE DOC EXAMPLE, worth knowing before the repair lands: its string enters `corpus_rules.jsonl`, where the differential gate parses it with the DEFAULT facade — no delimiter declared, so the dash is an ordinary name word and #538's reading is off the path entirely. It moves there for the 2026-09-20 link fix instead, and is classified as that at all five baselines: added to the `fix(#397) a link inside a maiden clause stays in the birth name` alternation at the four 2.x ledgers (suffix 'PhD i Soler' → 'PhD', maiden 'Puig Mr. -' → 'Puig Mr. - i Soler', identical at each), and given its own rule at 1.4.0, where v1 had no maiden markers and read the whole suffix-comma tail as one suffix. -- 2026-09-26 #538 — A DELIMITER CORE IS STEPPED OVER BY THE LINK'S NEIGHBOUR SEARCH, AS A CONNECTIVE IS (Derek, 2026-09-26). The 2026-09-22 follow-up above recorded the deviation; this resolves it. `_run_neighbours` treats a lone core like a connective, so the word on a link's side is the one past the core and a maiden clause reads exactly as the same text written without the core: `Smith, John, PhD née Puig Mr. - i Soler` under `extra_suffix_delimiters=(" - ",)` now stops at maiden 'Puig Mr.' as its separator-less twin does. INVARIANT, pinned by `tests/v2/test_properties.py::test_a_delimiter_core_reads_as_if_it_were_not_written` over 81 generated texts, maiden field: 6 disagreed at e0f1a2fa, 0 after. Declined: THE BOUNDARY READING, which the 2026-09-22 entry's wording implied ("a link with one beside it joins nothing") — a core ending the neighbour search on its side. It fixes the reported shape equally, and it moves `Smith, John, PhD née Puig - i Soler` and `… Puig i - Soler` from maiden 'Puig i Soler' to maiden 'Puig', suffix 'PhD i Soler' — a birth-name link pushed into the credentials, where the skip reading leaves both as they read. Default-unreachable either way. THE SAME STEPPING APPLIES OUTSIDE A CLAUSE: the `frozen` loop's own `_run_neighbours` call also takes `cores`, so a connective beside a core looks past it; where a credential or nothing stands beyond, it stays a lone suffix word rather than joining, and the core it stood beside -- now a lone piece with nothing joined to it -- is dropped as #206 drops any lone core, the same way it already drops one between two ordinary post-nominals: `Smith, John, PhD - i Soler` 'PhD - i Soler' -> 'PhD, i Soler', `Smith, John, PhD - i - MD` 'PhD - i - MD' -> 'PhD, i, MD', `Smith, John, - i Puig` '- i Puig' -> 'i Puig', `Smith, John, Puig i -` 'Puig i -' -> 'Puig i'. MEASURED 2026-09-26: 191 of 8,097 generated core-bearing tail texts (the three heads `Smith, John,`, `Smith, John, Jr.,` and `John Smith,`, each followed by every 2-4-word product of {PhD, MD, Puig, i, y, -, Jr., Soler, Mr.} containing a core) move `suffix` and no other field, none reachable at the default policy; the comparison is each text parsed under `extra_suffix_delimiters=(" - ",)` with `_run_neighbours` handed its cores and with it handed none. Not repaired here: a core the join MERGES into a joined piece -- interior (`… Puig Dr. i - y Soler`, still 'PhD i - y Soler') or at the edge a link joins across, a name word standing beyond it rather than a credential or nothing (`Smith, John, Puig - i Soler` 'Puig - i Soler', its separator-less twin 'Puig i Soler'; `Smith, John, Puig i - Soler` 'Puig i - Soler', measured 2026-09-26) -- is not dropped from the suffix text; the join is right, the surviving separator is the gap. Also in that family: an ORDINARY connective's join still takes a core as its neighbour, the stepping above being for generational vocabulary alone (rules.md#P3) -- `Smith, John, PhD - and MD` keeps suffix 'PhD - and MD' (measured 2026-09-26), pre-existing and unchanged here. Open: #549, which carries both the surviving core and this one. -- 2026-10-03 #549 — SUPERSEDED: the skip reading the #538 entry above settles is reversed, and a declared delimiter in a trailing part now parts it as a comma would, so `Smith, John, PhD née Puig - i Soler` reads maiden 'Puig' with suffix 'PhD, i Soler', the reading #538 declined and the one 2.3.0 gave, and a marker with a core straight before or after it takes no clause (`Smith, John, PhD née - Jones` reads suffix 'PhD née, Jones', reversing the #418 entry's repair). The decision, its measurement and the invariant that replaces #538's are under C1's 2026-10-03 entry. -- 2026-09-26 #535 — THE MAIDEN WALK READS THE TRAILING TITLE CHAIN, AND EVERY STOP ASKS ONE RELEASE QUESTION OF THE SPAN IT GIVES UP (Derek, 2026-09-26). This resolves the NOT TRANSPARENT HERE clause of the 2026-09-19 #533 entry above, which recorded `Jane Doe nee Smith MA Prof.` and `Jane Doe nee Smith Prof. MA` disagreeing and deferred the pair as one decision about what "trailing" means inside a clause. BOTH HALVES WERE TAKEN TOGETHER: a trailing title ends the clause (`Jane Doe nee Smith Prof.` reads title 'Prof.', maiden 'Smith', where 2.3.0 read maiden 'Smith Prof.'), and a credential or numeral standing in front of that title gets the stop it gets with the title absent (`… Smith MA Prof.` gives suffix 'MA', `… Smith V Prof.` suffix 'V'). Only the first half would have left the pair disagreeing in the other direction; H5's transparency is a statement about the peel and the chain read together to their fixed point, so the walk now reads the end of the clause the way assign reads the end of the name — `tail_reading`, the shared predicate mechanisms.md#ONE-PREDICATE-PER-QUESTION names — and over the forms the transparency property test below exercises, the two spellings give one answer. They split where something in or ahead of the clause stands to take the title — for example a particle chain (ahead of the clause or inside it), a bound-given join, a name left with no name word, or, after a family comma, a title word in front of a class member with a title behind it, which the given part's chain stops short of exactly as it does with no marker (`Doe, Jane nee Smith Rev. MA Prof.` reads maiden 'Smith Rev.', title 'Prof.', suffix 'MA', where `… Rev. Prof. MA` reads title 'Rev. Prof.', maiden 'Smith', suffix 'MA'; bare `Doe, Jane Rev. MA Prof.` reads middle 'Rev.') — and the Accepted pairs recorded here and in rules.md#M2 are examples of that, not a complete list. `tests/v2/test_properties.py::test_a_trailing_title_is_transparent_to_the_maiden_clause` holds that as an invariant over the two inputs, over heads and runs where the credential is given up and nothing ahead would take the title, with its negative control at e0f1a2fa in its docstring. ACCEPTED, THE TRANSPARENCY BOUNDARY: where the clause KEEPS the credential the spellings differ, because a clause is one contiguous run and a title inside the kept text cannot leave without the words behind it — `Doe nee Smith ba Prof.` reads title 'Prof.', maiden 'Smith ba', and `Doe nee Smith Prof. ba` maiden 'Smith Prof. ba'; `abdul nee Smith MA Prof.` reads title 'Prof.', family 'abdul', maiden 'Smith MA', and `abdul nee Smith Prof. MA` given 'abdul', maiden 'Smith Prof. MA' (2.3.0 kept every word in all four). The report differs with them: `Doe, Dr. nee Smith MA` and `… Prof. MA` report `suffix-or-name` on the kept MA, `… MA Prof.` reports nothing, the kept member no longer ENDING the clause, which is what M2's report is asked of. rules.md#M2 carries the first pair as an Accepted example. ACCEPTED, FURTHER SPLITS, measured 2026-09-26. Where something ahead of the clause would take the title, the spellings differ even with the credential given up in both, because the first-suffix-word stop ends the clause before the title check runs and the title check alone declines: `Jane van der Berg nee Smith PhD Prof.` reads title 'Prof.', maiden 'Smith', suffix 'PhD', and `… Prof. PhD` maiden 'Smith Prof.', suffix 'PhD' — H5's accepted particle-chain boundary inherited, the bare `Jane van der Berg Smith PhD Prof.` / `… Prof. PhD` splitting the same way — and `Berg, abdul nee Smith PhD Prof.` (bound-given join) and `Doe, Dr. nee Smith PhD Prof.` (no name word left) split likewise; the parent d9d80492 and 2.3.0 read all of these as the tree does, so only the claim is new. And a released particle with a title behind it is withdrawn: `Jane Doe nee Smith DO Prof.` reads title 'Prof.', maiden 'Smith DO', where `… Prof. DO` reads title 'Prof.', maiden 'Smith', suffix 'DO' (2.3.0 maiden 'Smith DO Prof.' and 'Smith Prof. DO'); `do` likewise. A particle INSIDE the clause does it from the other side, its chain able to take the credential behind it: `Jane Doe nee Smith do MA Prof.` reads title 'Prof.', maiden 'Smith do MA', and `… do Prof. MA` title 'Prof.', maiden 'Smith do', suffix 'MA' (2.3.0 kept every word in both). rules.md#M2 carries a pair of each shape as Accepted examples. ACCEPTED, H5'S REACH: `Jane Doe nee Smith King.` reads title 'King.', maiden 'Smith' (2.3.0 maiden 'Smith King.'), H5's accepted `Mary Jane King.` cost now reaching the last word of a birth name as it reaches the last word of a current one; rules.md#M2 carries it as an Accepted example. READER SCOPE IS UNCHANGED: the chain is consulted only where a trailing rule reads the clause's words (no comma, the part before a suffix comma, the given part after a family comma); before a family comma and in a tail segment the walk reads the peel alone and the clause keeps the title (`Doe nee Smith Prof., Jane` keeps maiden 'Smith Prof.'). THE FIRST-WORD FLOOR MOVED INTO THE CHAIN. `trailing_titles` and `tail_reading` take a `floor` (1 everywhere else), and the walk passes the position just past the marker's first word, so the chain never takes that word: `Jane Doe nee King.` keeps its maiden name (TITLES holds borne surnames and the marker announced one), and `Jane Doe nee Prof. Dr.` keeps 'Prof.' and gives up 'Dr.'. The first draft applied the floor as a clamp on the title stop after the chain had run, and that is measurably wrong rather than merely inelegant: a chain allowed to take the first word has already spliced it out of the count the re-peel reads, so `Jane Doe nee King. ba` kept 'ba' in the clause where `Jane Doe nee Smith ba` gives it up. `test_a_title_first_word_counts_as_a_word` is the invariant (a title-vocabulary first word against an ordinary one, the credential behind it read alike) and its docstring records the clamp's failure count. ONE RELEASE CHECK FOR THREE STOPS. The numeral, the credential and the title stop each ask `_release_reads_off` whether the name the take would leave reads what they give up as titles or suffixes with no join below the take absorbing it — rules.md#M2's invariant, "a word the clause gives up reads as a post-nominal or the clause keeps it", which the title stop now answers to as well. It is asked of the whole SPAN a stop gives up, not of the stop's own word: a first draft that checked the word alone broke the invariant on 513 of a 5,198-parse sweep (measured 2026-09-26 on that draft, which is not in the tree to recompute), because a stop gives up every word behind it — `DOE NEE SMITH PROF. MA` stopped at the title and handed the MA to the family, where the name left standing reads MA as a name; it keeps maiden 'SMITH PROF. MA' now. The question is asked per reader, the way that reader reads: the TRAILING reader is assign's own `tail_reading` over the view; the GIVEN_SLOT reader has its count settled by the comma, so it is the writing alone with a name word ahead. The name-word-ahead half is what keeps `Dr. nee Jones Smith Prof.` whole — the take would leave `Dr. Prof.`, whose Prof. would be the family name. THE JOINS. A released span holding a title with a particle ahead of it declines, because P2's chain runs on over a trailing title (rules.md#H5's Accepted `John van der Berg Prof.`): `Jane van der Berg nee Smith Prof.` keeps maiden 'Smith Prof.', and a released particle with a title behind it is withdrawn for the same reason (`Jane Doe nee Smith MA do Prof.` keeps maiden 'Smith MA do'). The bound-given (P5) half of `_join_takes_the_member` is asked for the GIVEN_SLOT reader alone: P5 is `BoundJoin.LENIENT` only after a family comma, and before one its STRICT reserve already refuses a join that would change a suffix reading, so asking there over-declined — `abdul nee Smith V` kept 'V' in the clause, where the scoped check lets it go to suffix as 2.3.0 did. The consequence is recorded by a case row rather than prevented: `abdul nee Smith Dr.` reads title 'Dr.', family 'abdul', maiden 'Smith', which is how `abdul Dr.` reads bare (2.3.0 read given 'abdul', maiden 'Smith Dr.'). THREE NUMERAL-STOP READINGS MOVE WITH IT, decided in rather than deferred, since each is the same invariant broken at the stop this change was already rewriting. The numeral stop never asked the join question: `Berg, abdul nee Smith V` read given 'abdul V' at 2.2.0 and 2.3.0 (given 'abdul nee', suffix 'V' at 2.0.0 and 2.1.0, #411's reserve differing between those pairs before the walk runs) and reads maiden 'Smith V' now. After a family comma the given slot reads a lone numeral as a suffix only where the given part is the LAST comma part — assign's own two-segment condition from #144 — and the walk now asks it too (`tail_follows`): `Doe, Jane nee Smith V, PhD` read middle 'V' at 2.2.0, 2.3.0 and e0f1a2fa, an M2 violation predating this change that its first draft had extended to `… V Prof., PhD`, and it reads maiden 'Smith V' now, as 2.0.0 read it. The withdrawal reaches through a title behind the numeral, so `Doe, Jane nee Smith Prof. V, PhD` keeps maiden 'Smith Prof. V' where `Doe, Jane nee Smith V Prof., PhD` gives up the title — the same asymmetry the bare given slot has with no marker in it (`Doe, Jane Prof. V, PhD` against `Doe, Jane V Prof., PhD`). And the numeral stop reads FROM the marker, unlike the other two, so a numeral straight after the marker is not held to the first-word floor, and it may decline the clause; examples, not a rule over every shape: `Dr. nee V` and `Doe, J. nee V` keep maiden 'V' as 2.3.0 did, though `Doe, J. V` reads suffix 'V', and `Doe nee V, Jane` and `Smith, John, PhD nee V` read no clause at all (family 'Doe nee V', suffix 'PhD nee V', as at 2.3.0) — `Jane Doe nee V Prof.` read maiden 'V Prof.' at 2.3.0 and reads family 'nee', suffix 'V', title 'Prof.' now, as `Jane Doe nee V` reads plus the title. After a family comma with another comma part behind the given one, the same #144 condition that keeps `Doe, Jane nee Smith V, PhD` whole now keeps a lone numeral in the clause as well: `Doe, Jane nee V, PhD` read given 'Jane', middle 'nee V', suffix 'PhD' at 2.3.0 and at the parent d9d80492 and reads maiden 'V', suffix 'PhD' now, as 2.0.0 and 2.1.0 read it, `Doe, Jane nee V, Jr.` likewise — the marker had been read as a name word there at 2.2.0 and 2.3.0, and a case row records it. A LINK THE WALK STOPS AT ASKS THE RELEASE QUESTION TOO. Reading the end of the clause through the title chain makes the link exception refuse a link it used to join — in `Jane Doe nee Smith i DO Prof.` the DO is the peel's once the title is chained — and a stop at a link gives up the words behind it. Unchecked, that put 'Doe i' in the middle name and 'DO Prof.' in the family (e0f1a2fa read maiden 'Smith i DO Prof.'), and `Berg, abdul nee Smith i V Prof., MD` read middle 'V'. So a link the exception refuses asks `_release_reads_off` of the run it would give up — only where the exception, bounded by the peel over the words as WRITTEN, would have joined it (the refusal is the title chain's), where a trailing rule reads the clause, and where the link is not the first word after the marker (a first-word stop declines the clause and gives nothing up); where that fails the clause keeps the link and walks on. Scoped that narrowly because a first version asked it of every refused link and kept links the name left standing reads off: `Doe, Jane nee Smith i V` kept maiden 'Smith i' where every release gives up suffix 'i V' (23,472 of a 411,936-parse link grid moved clean readings, measured 2026-09-26). With it, the given-part model reads the lenient numeral in both of assign's passes — the literal last piece first, then the last one standing once the chain has taken the titles behind it — and only where no comma part follows (`Doe, Jane nee Smith i V Prof.` gives up 'i V' and the title, as bare `Doe, Jane i V Prof.` reads them; `… i V Prof., PhD` keeps maiden 'Smith i V'). The same model moves the credential stop where a lenient numeral follows the credential: `Doe, Jane nee Smith MA V` now gives up 'MA V' (suffix 'MA V', maiden 'Smith') as bare `Doe, Jane MA V` reads it, where d9d80492 and 2.3.0 read maiden 'Smith MA', suffix 'V'; `… MA V, PhD` keeps maiden 'Smith MA V'. Over that link grid (links i, y, e, and, i y, y i, and i × 8 heads × bodies × `, MD` tail × 3 cases × 3 name orders), against the tree before the link check: 0 new violations, 6,195 fixed, and the clean readings that move now match the bare given part's (`DOE, JANE NEE SMITH MA I` gives up 'MA I', as `DOE, JANE MA I` reads suffix 'MA I'). `Jane Doe nee Smith i DO Prof.` now reads title 'Prof.', maiden 'Smith i DO' and reports the kept DO, and `Berg, abdul nee Smith i V Prof., MD` maiden 'Smith i V Prof.', suffix 'MD'. A RELEASED TITLE THAT IS ALSO A PARTICLE IS KEPT AFTER A FAMILY COMMA, because P6 attaches a particle trailing the given part to the family: released by the title stop, 'St.' in `Doe, Jane nee Smith St.` reads family 'St. Doe', so `_join_takes_the_member` declines it and the name reads maiden 'Smith St.', as e0f1a2fa and 2.3.0 read it, and so does `Doe, Jane nee Smith MA St.`. Titles only — a credential that is also a particle ('DO') is the given slot's own lean, which the credential stop has already asked (#533). A FOURTH MEASUREMENT, against e0f1a2fa rather than d9d80492: the M2 invariant over a fuzz of twelve heads ({`Jane Doe`, `Doe, Jane`, `Doe, Prof.`, `Jane Doe, PhD`, `Doe, J.`, `Berg, Jane van der`, `Jane van der Berg`, `Berg, abdul`, `abdul Berg`, `J. Doe`, `Prof. Jane Doe`, `Dr.`}), ` nee Smith ` and every one-to-three-word sequence over {MA, Ma, V, Prof., St., King., PhD, ba, do, DO, Jr., i, van, y} holding at least one of Prof., St., King., then nothing or `, MD`, as written, upper- and lower-cased (107,352 parses, measured 2026-09-26 on the narrowed tree; the intermediate tree above read 2,228): 588 texts that violated it at e0f1a2fa no longer do, and 12 newly do, all `Dr. nee Smith i MA|V` followed by `Prof.`, `King.` or `St.`, with or without `, MD`. Each is the title-carrying twin of `Dr. nee Smith i MA` / `Dr. nee Smith i V`, which read family 'i' at e0f1a2fa already: the bare-title head leaves no name word, the unguarded stop's class (#548), which the title now reaches because it leaves the clause. ACCEPTED: `Jane Doe (nee Smith Prof.)` reads title 'Prof.', maiden 'Smith' — bracket content ending in a period is suffix-shaped (S1), so the brackets are dropped and the clause is read bare, outside the reach of the delimiter precedence; rules.md#M2 carries it as an Accepted example. MEASURED 2026-09-26, the tree against its parent d9d80492, over three generated grids, and these are dated snapshots. Recipe: grid A is each head in {`Doe, Jane`, `Doe, J.`, `Doe, Prof.`, `Berg, abdul`, `Doe, Jane van`, `Jane Doe`, `J. Doe`, `Dr.`, `abdul`, `Jane van der Berg`, `John`}, then ` nee `, then every sequence of one to three words (repetition allowed) from {Smith, Jones, Prof., Dr., Sir, MA, DO, PhD, V, III, i, do, van, Ma, M.A., King., Rev., ba}, then either nothing or `, PhD`, each text as written, upper-cased and lower-cased, duplicates removed — 327,936 texts; grid B, aimed at the bound-given join, is the same construction over heads {`Dr. abdul`, `abdul rahman`, `Dr. abdul rahman`, `Berg, Dr. abdul`, `Berg, abdul rahman`, `abu`, `Berg, abu`, `Mr. abu`, `Dr.`, `Berg, Dr.`, `abdul`, `Berg, abdul`}, words {Smith, Prof., MA, V, do, van, PhD, Ma, ba, III, Jr., M.A.} and tails {nothing, `, PhD`, `, Jr., MD`}, plus these fourteen texts, as written only: `Jane Doe nee Ph. D. Prof.`, `Jane Doe nee Ph. D. Smith Prof.`, `Jane Doe z domu King. ba`, `Jane Doe z domu Smith Prof.`, `Jane Doe nee King. Prof. ba`, `Jane Doe nee King. Prof.`, `Doe, Jane nee Smith V,`, `Doe, Jane nee Smith V, ` (with the trailing space), `Doe, Jane, nee Smith V`, `Doe, Jane nee Smith V Prof., PhD`, `Doe, Jane nee Smith Prof. V, PhD`, `Doe, Jane nee Smith MA, PhD`, `Doe, Jane nee Smith V (Jr.)`, `Doe, Jane nee Smith V "Bo"` — 173,057 texts. The invariant tested is M2's: in a parse with a non-empty maiden field, every token written after the ` nee ` marker is roled maiden, title or suffix (so the two `z domu` texts are parsed and not checked). Grid C, a four-word probe of the title chain after a family comma, is each head in {`Doe, Jane`, `Jane Doe`, `Doe, J.`, `John Smith`, `Doe, Jane Mary`}, then ` nee `, then every sequence of four words (repetition allowed) from {Smith, Jones, Prof., Dr., King., MA, PhD, V, Jr., ba, do}, then either nothing or `, PhD`, as written only — 146,410 texts. Per grid, the counts are: texts that newly violate the invariant at the tree, texts that violated it at the parent and no longer do, texts that violate it at both. Grid A: 0, 2,012 and 17,854; grid C: 0, 1,368 and 15,534; grid B: 0, 1,222 and 11,149 (all re-measured 2026-09-26 on the tree with the link and particle-title checks below, as narrowed; an intermediate tree whose link check fired on every refused link read grid A 0, 3,772 and 16,094, its extra 1,760 fixes being links the as-written reading refuses too, which are #548's class), one of those last being the check itself counting the quoted nickname in `Doe, Jane nee Smith V "Bo"`. In grids A and B, every other remaining violation has the clause ending immediately before a word of the unambiguous suffix vocabulary written the way the plain suffix-word stop takes it (PhD, III, M.A., Jr., and a lower-case i or v; never a bare capital, which reads as an initial), which is where that stop ends a clause without asking a release question. DEFERRED: the walk's plain suffix-word stop — the one ending the clause at the first suffix word, as distinct from the trailing numeral, credential and title stops — asks no release question at all, and where the words behind it cannot read as post-nominals they land in a name part: on degenerate heads the stop word itself does (`Dr. nee Smith PhD Prof.` reads family 'PhD', `Doe nee Smith Jr. Prof., Jane` family 'Doe Prof.'), and with an ordinary head the words behind it do (`Doe, Jane nee Smith PhD Smith` reads middle 'Smith'). Unchanged here, and what "the clause keeps it" should mean where the kept words would follow a credential the clause itself ended at is its own question; rules.md#M2 carries `Doe nee Smith Jr. Prof., Jane` as an Accepted example meanwhile. The trailing numeral's stop has the same gap before a family comma, where it is made over the peel alone with no release question: `Doe nee Smith V, Jane` reads family 'Doe V', maiden 'Smith' (so did 2.2.0, 2.3.0 and the parent d9d80492; 2.0.0 and 2.1.0 read maiden 'Smith V'), and rules.md#M2 carries it beside the other. Open: #548. COST, measured 2026-09-26 on CPython 3.11.16 as profiler call events in one `Parser.parse` after a warm-up parse, parent → tree: `John Smith` 171 unchanged, as are `Jane Doe Prof.` and `John Smith MA`; `Jane Doe nee Smith` 244 → 246; `Jane Doe nee Smith MA` 377 → 388; `Jane Doe nee Smith Prof.` 276 → 372, the one shape that now runs the chain and a release check it never ran. Recompute: count `sys.setprofile` call events around the second of two `Parser().parse` calls, with the tree and d9d80492 each first on `sys.path`. -- 2026-09-27 (Derek), #544 — S2'S COMPANY CLAUSE DOES NOT REACH ACROSS A CLAUSE, recorded as an Accepted boundary rather than repaired. `Jane Doe Jr. nee Smith Ma` keeps maiden 'Smith Ma' and reports it, the clause's own words standing between 'Jr.' and 'Ma', where the clause-less `Jane Doe Jr. Ma` reads suffix 'Jr. Ma'. tests/v2/test_properties.py's clause-agreement walk pins the pairs that differ for exactly this reason as its `anchored_head` class — 810 of its 20,412 pairs, recorded 2026-09-27, 0 with the anchor off. Inside the clause the company reads as it does anywhere: `Jane Doe nee Smith PhD MEng` ends the clause at 'PhD' and reads suffix 'PhD MEng', maiden 'Smith'. - -- 2026-10-04 (Derek), #601 — A MARKER COUNTS ONLY BEHIND A SURNAME, AND IT TAKES THE WORDS AFTER IT UP TO THE TRAILING RUN THE NAME WOULD END WITH IF THE CLAUSE WERE NOT WRITTEN. This SUPERSEDES the release machinery of the #533 and #535 entries above (the release question, the per-stop checks, the anchored-head boundary of the #544 entry) and the given-part and suffix-comma reach the 2026-07-03 rule had; the #399, #420, #434 and delimiter entries stand. The trigger was #548, `Dr. nee Smith PhD Prof.`, where 2.3.0 read family 'PhD', maiden 'Smith': a clause whose head was nothing but a title, and a release check asking whether each word it gave up could still read as a post-nominal. #548 was closed into this issue and #602 (S2's run) because both halves of its answer were rules rather than repairs. THE HEAD RULE (Derek): a marker is one only in the name before any comma or the surname part before a family comma, behind at least one name word past the leading title run, and not straight behind an unambiguous suffix word or a connective. Anywhere else it is an ordinary word — in the given part after a family comma (`Doe, Jane nee Smith` reads middle 'nee Smith', 1.4.0's reading) and in a part after a suffix comma (`Smith, John, PhD née Jones` reads suffix 'PhD née Jones', also 1.4.0's). Derek's framing was that a maiden marker always follows a family name, and that a marker following something the parser has decided is a title, suffix, particle or connective is not a marker. THE PARTICLE HALF WAS DROPPED BY MEASUREMENT: prototype variant 1 refused a marker behind particle vocabulary and so declined `Anh Do née Tran` and `Mai Le née Nguyen`, Do, Le and Van being particles that are also the commonest Vietnamese surnames; a particle counts as the surname (`Jane van nee Smith` reads family 'van', maiden 'Smith'). THE BOUND (Derek): the clause takes its words up to the run of post-nominals and titles that the trailing rules read at the end of the name with the clause removed, and the words it gives up take the roles that reading gives them — the TRAILING rule's reading of the clause-free name, not the full parse of it, so a head the full parse would read differently (H1's lone title, a P5 join) cannot change where the clause ends. The first word after the marker is always taken, the marker having announced a name — a lone roman numeral included, so `Jane Smith née V` reads maiden 'V' where the clause-free `Jane Smith V` reads suffix 'V', accepted rather than given a numeral exception (rules.md#M2's Accepted block) — unless it is an unambiguous suffix word, where the marker declines (`Jane Smith nee PhD` reads family 'nee', suffix 'PhD', as before). Before a family comma the clause keeps the 2026-07-03 stop at the first suffix word, since no trailing rule reads that part's end. VARIANTS, prototyped 2026-10-03 behind an environment switch on one tree (the numbers are in #601's comments): v2/v3 grew the clause until the existing release check accepted it and reached 0 invariant violations but inherited the check's caution about particles (`Anh Do geb. de la Vega PhD Prof.` kept 'PhD Prof.' in the maiden name); v4 read the run once over the name as written without binding it, and is the recorded negative control in tests/v2/test_properties.py (350 of the clause grid's 4,482 parses fail under it, measured 2026-10-04); v5/v6 bound that reading as roles but still counted the clause's words among the words to spare, so `John nee Smith ba` read 'ba' as a credential where `John ba` reads a name; v7, the clause-free reading bound as roles, was adopted. Against #535's grids v7 left 0 violations of the M2 invariant where the tree then had 15,586 (grid A, 327,936 texts) and 8,713 (grid B, 173,057), measured 2026-10-03 with an invariant check written to #535's recipe. WHAT THE PROTOTYPE MISSED, found implementing it: the numeral fork. A trailing roman numeral is a suffix only by its shape against the word before it, and v7 read the clause's last word without that test, so `John née Jones Smith VI` kept 'VI' in the maiden name; the take now asks `is_trailing_numeral_suffix` of the last piece, and cases.py's `a_numeral_the_clause_free_name_reads_ends_the_clause` pins it. And the head's title run is measured over the words before the marker, a lone title excepted: measured over the whole segment, `Lord Chancellor née Jones` counted 'Chancellor' as a title and refused the marker. TRIAGE, 2026-10-04: the prototype's 68 failing case rows (40 facade failures among them) were re-pinned as `fix(#601)` with their group named in the note, bar the head-rule bug above; tests/v2/test_parser.py's corpus-append invariant became two-sided (appending ` née X` moves no other field, OR the take records why it declined), with a named set for the suffixes that themselves hold a marker. DEGENERATE INPUT, NO READING DESIGNED FOR (Derek, 2026-10-04: garbage in, garbage out). Two names that read worse by eye moved, and they are recorded here so the move is not mistaken for a regression, not because either reading is wanted: `Berg, Jane van der nee Smith DO` reads middle 'van der nee Smith', family 'Berg' where 2.3.0 read family 'van der Berg', maiden 'Smith DO' — the marker in a given part being a word, the particles no longer trail the given name (P6); and `Jane Doe, PhD née Smith` reads given 'PhD', middle 'née Smith', family 'Jane Doe' where 2.3.0 read given 'Jane', family 'Doe', suffix 'PhD', maiden 'Smith', which 2.3.0 reached only through the clause taking the marker (the comma part opening with a credential is #603's shape). Measured 2026-10-04 against 2.3.0. COST: the take is one function and no release helper survives; the #533 review's comma-head grids could no longer reach a take at all and were replaced by counting heads (tests/v2/test_properties.py's `_maiden_clause_grid`, 4,482 parses). -- 2026-10-04 #610 — THE MAIDEN TAKE'S CANDIDATE WORDS ARE ONE FUNCTION BESIDE THE PEEL, AND SHARING IT FIXED A GAP THE HAND COPY HAD. `_maiden_take` chose the clause words to read its clause-free view over with a filter of its own (`_clause_tail_word` plus a numeral clause), a hand copy of the peel's and the chain's admissions whose contract — a superset of what `tail_reading` takes — nothing stated or tested. It is now `_pieces.trailing_candidates`, written beside `peel_trailing`, and tests/v2/pipeline/test_pieces.py pins the superset at the take itself, on the unjoined pieces the take is handed, over a generated grid (37,044 clauses through `group`). The grid found the copy wrong twice. The numeral fork reads a shape-only roman numeral (`VI`, in no wordlist) as the WALK's last piece, and two things leave it last without standing at the end of the name: a period-marked title behind it, which the chain takes first, and the merged `Ph. D.`, which group flags and the walk never holds. The copy admitted the numeral only as the clause's last word, so `Jane Doe nee Smith VI Prof.` kept maiden 'Smith VI' where `John Smith VI Prof.` reads suffix 'VI', and `Jane Doe nee Smith VI Ph. D.` the same way. Both now end the clause at the numeral, as rules.md#M2's statement already said they should. Recorded negative control: the loop as #601 shipped it misses 432 of that grid's clauses, every one this shape. The test's first version read plain names through all of `group`, on pieces the joins had already merged, which review caught: there the old loop missed 288 names, and a first draft of the fix, covering the title half only, still missed 111 — the `Ph. D.` half, which that grid is what found. Against master 8a8459a8 over the 95,813-text equivalence grid the change moved 10 readings, all of this class. The same issue lifted P6's particle-tail walk into `_pieces.particle_tail`, read off roles, so post_rules' attachment and assign's given-part credential run (#602) ask one walk; that half moved none of the 95,813 readings. - -### N1 — the default delimiter pairs - -- 2026-10-03 #597 — THE HALFWIDTH CORNER BRACKETS ARE A DEFAULT NICKNAME PAIR. `｢｣` (U+FF62/U+FF63) join `DEFAULT_NICKNAME_DELIMITERS` beside `「」`, and the v1 facade gains the matching `halfwidth_corner_brackets` key, so `HumanName` reads them as `parse()` does. They are the same punctuation as `「」` in JIS X 0201's narrower encoding, the legacy data halfwidth katakana comes from (#594), where they were the only corner brackets available, and they appear in no non-CJK text. Before this every release kept the clause in the name, brackets and all — a middle where it stood spaced (`山田 ｢ﾀﾛｰ｣ ﾀﾛｳ`), part of one given word where it was glued (`山田｢ﾀﾛｰ｣太郎`) — and since #594 the bracket also cost the spaced name its order: the halfwidth kana now classify, but the unclassified bracket kept `山田 ｢ﾀﾛｰ｣ ﾀﾛｳ` from being wholly East Asian script, so it read given 山田 (in released versions the unclassified kana did that on their own). A v1 `Constants` restored from a pickle keeps the keys it was saved with and so does not gain the pair, the same policy #273's pairs followed: restored configuration is never widened behind the caller's back. A pair is not script-gated, so `John ｢Jack｣ Smith` reads nickname Jack as `John 「Jack」 Smith` does. Nothing else needs a halfwidth twin: Unicode has no halfwidth form of `『』` or of the fullwidth parentheses. MEASURED 2026-10-03: no corpus line held a halfwidth bracket before this change, so the three `fix(#597)` case rows are the whole population, and they move at every baseline. RECOMPUTE: parse `山田 ｢ﾀﾛｰ｣ ﾀﾛｳ`, `山田｢ﾀﾛｰ｣太郎` and `John ｢Jack｣ Smith` here and on each baseline from PyPI and read `nickname` beside the name fields. - -- 2026-10-12 #653 — A DELIMITED CLAUSE KEEPS ITS COMMA. `Jane (Janey, Jo) Smith` reads nickname 'Janey, Jo' again, as 1.4.0 did, where 2.0.0 through 2.4.0 gave 'Janey Jo'. The comma was lost in tokenize, not in the view: `_tokenize_region` serves the main stream and the extracted regions alike, and treated `COMMA_CHARS` as a separator in both, although only the main stream records a segmentation offset for it -- inside a region it divided the clause into words and recorded nothing, so the view's space join had no comma left to keep. It shipped that way with the first tokenizer (7e341a95, 2026-07-12) and no entry chose it; `;`, `/` and `&` were always kept. Now a comma separates only where it records an offset, so inside a clause it is word text on the word it touches (`Janey,`), and the view's space join restores the writer's spacing (`Janey,Jo`, `Janey , Jo` and `,Jo` all read as 1.4.0 read them). WHAT ELSE MOVED, all by the same token change: a marker glued to a comma is no marker inside a clause, as it already was in the main stream (`Jane Smith née, Jones` reads no maiden), so under `maiden_delimiters` `Jane (née, Jones) Smith` keeps maiden 'née, Jones' where the dropped comma had let the drop pass take 'née'; and a phrase marker divided by a comma inside a clause no longer matches, which the M Background already stated (`Maria Kowalska (z, domu Nowak)` keeps maiden 'z, domu Nowak' under `maiden_delimiters`; rules.md#M1 now pins it). A punctuation-only clause reads as its punctuation (`Jane (,) Smith` → nickname ','), as `(;)` and `(-)` already did and as 1.4.0 did — and a name holding one now has a nickname for rules.md#N3 to count, so `Jones (,)` reads family 'Jones' with no ambiguity report where 2.x read given 'Jones', reporting GIVEN_OR_FAMILY since #449 in 2.3 (1.4.0: last 'Jones', nickname ','; the same for `(,) Jones` and `Jones ","`). A COMMA AFTER A MARKER UNMARKS IT (rules.md#M3, decided in review 2026-10-12): once the clause kept its comma, a spaced one after the marker (`Jane (née , Jones) Smith`) passed M3's whitespace test, since ',' was a word after the marker, and the drop pass then took the marker and left maiden ', Jones'; `(née , ) Smith` gave maiden ','. Rather than read the spaced comma as glued `née,` is not, M3 and M1's drop now ask the one predicate `_vocab.comma_follows_marker` of the word after the marker run, so a comma standing alone or glued to either word unmarks it: `Jane (née , Jones) Smith` reads nickname 'née , Jones', which is 1.4.0's reading, where 2.2 through 2.4 read maiden 'Jones'; under `maiden_delimiters` the clause keeps its marker, `(née ,Jones)` reading maiden 'née ,Jones'. Declined: dropping the comma with the marker, which restores 2.4's maiden 'Jones' for the spaced comma while the glued `(née, Jones)` stays a nickname, so the two spellings would disagree. A trailing comma rides as written (`(née Jones,)` → maiden 'Jones,'). The two sites hand the predicate different strings — extract a whitespace word, group a token — so it reads past any leading character that is neither a comma nor a letter or digit: a second review found `Jane (née \u200f, Jones) Smith`, visually the spaced-comma name, read maiden 'née , Jones' while it tested only the first character, M3 seeing the right-to-left mark tokenize drops and group the bare comma (the same for an emoji or a nakaguro before the comma). cases.py's marker_followed_by_a_comma_behind_a_bidi_mark_is_no_marker pins it. ACCEPTED: a whitespace run inside a clause still collapses to one space (1.4.0 kept 'Janey,Jo Bo' verbatim), and the v1 `nickname_list` stays one element per word as it has been since 2.0 (1.4.0 gave one per clause), the comma now riding on its word (`['Janey,', 'Jo']`) as `;` always did. THE DROP-PASS WITNESS MOVED: `marker_glued_to_punctuation_keeps_the_clause_a_nickname` had been the only case row reaching the `role is not Role.MAIDEN` branch of `_group.group`'s drop pass, and it reached it through the dropped comma; `marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname` (`佐藤 花子 "旧姓・田中" (旧姓 鈴木)`) reaches it through rules.md#T2's nakaguro instead, and is now the one row that fails with the branch removed (measured 2026-10-12). MEASURED 2026-10-12: before the rules.md examples landed no corpus name held a comma inside a clause, and the gate at 2.4.0 read 0 diffs with the fix applied; with them, the two examples are the 2 intentional diffs at each of 2.0.0 through 2.4.0 under the `fix(#653)` rule, and parity at 1.4.0. The witness row joins `corpus_cjk.jsonl` and diffs at 1.4.0, 2.0.0 and 2.1.0 only, classified under the #335 rules there (a new literal-keyed `fix(#335/#272)` rule at 1.4.0, one more literal in the existing #335 rules at 2.0.0 and 2.1.0); every baseline's gate exits 0. - -### N3 — the lone-word nickname rule - -- 2026-07 (v2 core, PR #288; recorded plan deviation #2 of the core plan) — v1's rule counted pieces before grouping; the v2 port fires only when the nickname accompanies exactly ONE piece in total — a title counts against it, so "'Smitty' Dr. Jones" reaches H1 with a title and one name word left standing — through 2.1 that meant given="Jones" with the family empty, and since #410 (2026-08-25) H1 names the family, so it reads family="Jones". The count is unchanged; what moved is what happens after it declines. The rule lives in assignment because that is where the piece count is settled. (An earlier wording here said "one non-title piece", predicting the opposite output; the coherence review measured the truth.) - -### N2 — same-character quotes and shared-character conventions - -- 2026-07-18 #273 (7ee6e3a) — the typographic delimiter pairs shipped with a per-pair full-text scan; three independent review agents broke it the same day: with two shared-character conventions genuinely present, the earlier-sorted pair stole the close and the legitimate match dropped silently, zero ambiguity. -- 2026-07-18 (05fe693, the same day) — the interleaved leftmost-match restructure replaced it (an altitude review called it the right depth); the author's own first fix — post-hoc offset filtering of unbalanced candidates alone — was superseded within the day and survives only as the bulk-recorded dangler filter rules.md#N2 now names. The word-internal carve-out covers ONLY the straight apostrophe because it is the one delimiter character that occurs mid- and end-of-word in real names; Derek's distinguishing test: with double quotes the same shape ("Mari\" Aube\"") is genuinely ambiguous, while Mari' Aube' is not. A cached delimiter-charset prescreen landed in the same commit, measured: no-delimiter names parse faster than the pre-#273 baseline. - -Declined: - -- The offset-filter as the whole fix — it repaired the reported symptom (spurious unbalanced flags) while leaving the stolen- close mis-extraction in place. - -### ma-do — ambiguous acronyms by decision - -- 2026-07-17 (M12, Derek-approved) — ma and do joined the ambiguous acronym set because both are common surnames; the two-word "Jack Ma" is kept intact by S2's words-to-spare guard, while the periods gate governs the dotted spellings ("M.A." counts unambiguously). Documented side effect: parenthesized bare - "(MA)"/"(DO)" no longer escape to suffix as in 1.x. -- 2026-09-15 #289 — AMENDS the sentence above for one spelling and leaves it standing for the other. "The two-word 'Jack Ma' is kept intact by S2's words-to-spare guard" is still true of `Jack Ma`, and false of `Jack MA`: an all-caps member of the ambiguous set inside a mixed-case name is now read as a credential with nothing to spare, so `Jack MA` gives given 'Jack', suffix 'MA' — v1's reading was last 'MA', so this is a deliberate parity break and not a restoration. The periods gate is untouched and still governs the dotted spellings. The two ledger comments that quote this sentence carry the same amendment. See #S2. -- 2026-09-18 #531 — AND `do` IS NOW SPELLING-DEPENDENT AFTER A FAMILY COMMA, which neither sentence above anticipated: it is the one member of this set that is also particle vocabulary, so `Doe, John DO` reads suffix `DO` (capitals, mixed-case name) while every other spelling attaches to the family under rules.md#P6. The pairing and its accepted cost are in #S2's 2026-09-18 `do` bullet. - -### vocabulary-collisions — when a word earns the ambiguous marking - -The mechanism shipped twice before anyone wrote down its criterion. Sizes as of 2.2.0dev: particles 67 with 39 ambiguous (58%, PARTICLE_OR_GIVEN); suffix_acronyms 613 with 4 ambiguous (0.65%, SUFFIX_OR_NAME); titles 711 with no ambiguous subset and no AmbiguityKind at all. Those counts are evidence from the decision date, not live facts — they drift with every vocabulary addition. The argument does not depend on the digits (it depends on the two shares differing by orders of magnitude), but recompute before quoting them: - - uv run python -c "from nameparser import Parser; L=Parser().lexicon; print({s: len(getattr(L,s)) for s in ('particles','particles_ambiguous','suffix_words','suffix_acronyms','suffix_acronyms_ambiguous','titles')})" - -Recomputed 2026-09-07 with that recipe, after this section's own #342 decision landed: particles 70 with 37 ambiguous, suffix_acronyms 610 with 5 ambiguous, titles 758. All THREE of the snapshot's figures were audited against the released 2.2.0 wheel, not the acronym one alone — run the recipe under a PEP 723 script pinning nameparser==2.2.0, from a directory outside the checkout so the tree cannot shadow the wheel, and it reports particles 70 with 37 ambiguous, suffix_acronyms 612 and titles 706. So the snapshot's particles and its titles were off at their own date exactly as its 613 was, and the one that matters to a reader subtracting the two rows is the particles: they have NOT moved since 2.2.0, where "67 with 39" against today's "70 with 37" reads as three additions and two demotions that never happened. What did move: rai and cha left the acronyms and ba joined the ambiguous subset (#suffix-acronym-collisions), and the titles gained 52 over the wheel. The snapshot above stays as written, being evidence from its own date; the argument survives either set of digits, since the two shares still differ by orders of magnitude. - -- 2026-08-16 (collision keystone; #348, #360, #342, #385) — the 58%-vs-0.65% gap is BASE RATE, not disagreement. Both sets apply the same test; most particles are short words that double as names (van, bin, le, do, bar, mac) while most credential acronyms are not (abpp, acp). Recorded because the gap reads as an inconsistency and is not one — a reviewer who "harmonizes" the two shares will break one of them. -- **C-i, vocabulary vs. name.** A word belongs in its set's ambiguous subset iff it is borne as an ordinary name IN THE POSITION THE VOCABULARY CLAIM ACTS ON. Existence of a bearer anywhere is not the test, and the first draft of this criterion (recorded 2026-08-16, corrected 2026-08-17) got that wrong. Under uncertainty, default to AMBIGUOUS. This generalizes the evidence standard already stated in NON_GIVEN_NAME_PARTICLES' docstring — a wrong unambiguous claim misparses a real person, a wrong ambiguous marking only adds a flag — from "which set" to "which subset". Applies uniformly to particles, suffix_acronyms and titles. -- 2026-10-04 #606 — C-i has ONE recorded exception, for TITLES only: #salutation-titles ships words measured as borne in the leading position at small incidence (herra, batoni, hrabia, kreivi), a weighing C-i itself does not make. It extends to no other set and no other word without its own counts. -- **C-ii, vocabulary vs. vocabulary.** Where two sets claim a word and NEITHER reading is a name, precedence is a frequency judgment recorded per word. vd is the live case: never-given particle AND credential acronym (the British Volunteer Decoration), neither of them a name. Decision: the Dutch van der reading, as the more common. That is what unblocks #380, whose trailing-orphan half is a separate decision recorded under its own rule. -- 2026-08-17 — C-i CORRECTED: the position qualifier. Writing the - per-word records for the never-given particles falsified the first - draft on the set's most load-bearing member. "De" is a borne Bengali - and Odia surname (parse("Bimal De").family == "De"), so C-i as first - written said mark `de` AMBIGUOUS -- which would break "de la Vega" - and every leading-particle reading. Nothing had tested the criterion - against `de`, and the error pointed at the safe side, so it would - never have announced itself as a misparse. - The qualifier the existing set was already obeying: P1 acts on the - LEADING position or a lone particle in the GIVEN role, so what - matters is whether the naming use occupies THAT position. - van Vietnamese Văn in given position collides ambiguous - bar Bar Refaeli, given position collides ambiguous - do Đỗ leads a surname collides ambiguous - de "De" is a TRAILING surname no clash never-given - Consequence, and the reason this is not merely tidier: `das` was - called ambiguous on #360 under the old reading because Das is a - borne Bengali surname. Measured, "Anjali Das" and "Bimal Das" are - unchanged by never-given `das` (the surname is trailing, the rule - acts leading) while "Maria das Neves" GAINS its particle -- - family="Neves" today, family="das Neves" with it. The old reading - would have declined a fix. `lo` and `el` need re-judging on the same - axis; `Lo` does lead in romanized Chinese ("Lo Wei"), so it may - genuinely collide where `das` does not. - Corrected on #360, which carried the wrong table publicly. -- The concrete demonstration is "do", which three vocabularies claimed — titles, particles/ambiguous, suffix_acronyms/ambiguous. Two mark it ambiguous; the third, TITLES, was the one that actually decided "Do Quang Minh" (title="Do", given="Quang") and reported nothing — until #296's audit (2026-08-23) took it out of TITLES, since when the name reads given "Do" and reports the fork. Same word behind #385's "Anh Do". -- Applications, each still its own work: #360 (mc, ste — neither is a borne given name, so both leave the ambiguous half); #342 (rai — Rai IS a borne surname, so it earns the marking rather than moving); #385 (do — resolved at decisions.md#R2). (#342's clause — "rai — Rai IS a borne surname, so it earns the marking rather than moving" — is SUPERSEDED 2026-09-07: Rai is borne, and what that earned was removal from the vocabulary rather than the marking. See #suffix-acronym-collisions for the relative-frequency criterion that decides between the two.) -- C-ii's per-word framing versus a rule stated for a SHAPE: measured 2026-08-16, the words that are both particle and suffix vocabulary are vd, do and mc — three, not the one this criterion adjudicated. rules.md#P6 states its precedence for the shape, so do and mc inherit vd's answer without being weighed. Recorded rather than papered over: stating a per-word judgement as a general clause is how an unexamined word acquires a decision, and the two are named here so the next reader knows which one was actually argued. -- 2026-10-04 #604 — five particles, judged by C-i one at a time. NEVER-GIVEN: `ó` (Irish "descendant of") and `ní` ("daughter of") — grammatical patronymic particles, no given-name use in the position P1 acts on, and the measured gain is the leading and mid-name surname both (`Liam Ó Murchú` and `Ó Murchú` read family `Ó Murchú`, where today Ó was a middle initial or the given name). AMBIGUOUS: `ua`, the older spelling of `ó` — no given-name use was found, but the search was thinner than for `ó`/`ní`, and C-i says default to ambiguous under uncertainty. Measured 2026-10-04, the two markings agree mid-name (`Seán Ua Buachalla` reads family `Ua Buachalla` either way) and differ in three places. Where `Ua` opens the name: the leading `Ua Buachalla` keeps given `Ua` and reports particle-or-given rather than becoming surname-only, and `Ua Buachalla Seán` under FAMILY_FIRST keeps family `Ua`, given `Buachalla` and reports. Where it opens the part after a family comma (P1's after-comma clause): `Seán, Ua Buachalla` reads given `Ua`, middle `Buachalla`, family `Seán`, where never-given would give family `Seán Ua Buachalla`. And where it ends the name (P6): `Seán, Seán Ua`, and `Seán Seán Ua` under FAMILY_FIRST, report particle-or-given only while `ua` is ambiguous, the fields being the same. Recompute: parse each with `Lexicon.default()` against `Lexicon.default().remove(particles_ambiguous={'ua'})`. The cost is small because `Ua` is the rarer spelling, but it is not leading-only, which matters to anyone re-pricing the marking under #360. `binti` and its Singapore spelling `binte` (Malay "daughter of") — ambiguous like `bin`, Binti being a Swahili given name ("daughter"); `Ina binti Navalamar` reads family `binti Navalamar`, `Binti Jua` keeps given `Binti` and reports. The one-letter `ó` needed a rule of its own against the initial it spells: rules.md#P7, decisions.md#P7. -- Caution when applying C-i to the particle set: TITLES ∩ ambiguous — {do, freiherr, st} until #296's audit took 'do' out of TITLES (2026-08-23; `Do Quang Minh` reads given 'Do' with the fork now), {freiherr, st} since — is load-bearing, per the Excluded note in the W2 section. Emptying it makes the particle-or-given emitter dead code. - -Open: [#348](https://github.com/derek73/python-nameparser/issues/348) applying C-i to the 711 title entries, then titles_ambiguous plus a TITLE_OR_GIVEN kind. Blocked on data, not on judgement — the census needs a given-name frequency corpus this repo does not have, which is why the criterion is recorded here and the census is not attempted. - -### v1-xfail-triage — the eight inherited aspirations, four retired and four kept (three of those eight since fixed, 2026-09-08 and 2026-09-23) - -The reconciled v1-style banks (`tests/test_*.py`) carried eight `@pytest.mark.xfail` tests inherited from v1, each an aspiration nobody had judged since. `xfail_strict = true`, so all eight were live claims that 2.0 still does not do the thing — but a bare marker says only that, never whether the thing is WANTED, and that is the gap this triage closes. After it, every surviving marker cites an issue and every retired one is a deliberate pin of current behavior. The parses quoted below are this entry's evidence and its own recompute: - - uv run python -c "from nameparser import HumanName; [print(repr(s), repr(HumanName(s))) for s in ('Maier, Amy I, Jr.', 'Dr King Jr', 'King, Dr Jr', 'Ahmad ben Husain', \"The Right Hon. the President of the Queen's Bench Division\")]" - -- 2026-09-01 (Derek's triage) — NOT FIXED, four. The aspiration is rejected, the marker is gone, and the test now pins what the parser does with the grounds recorded beside it so nobody re-derives the rejected proposal: - - `Maier, Amy I, Jr.` — v1 wanted suffix "I, Jr."; it reads given Amy, middle I, family Maier, suffix "Jr.". A middle initial `I` is far more common than an ordinal I borne without a Sr./Jr.-style companion, so where an explicit suffix comma has already named the suffix, the trailing I stays a middle initial. That is #vocabulary-collisions' commonality reasoning applied to a SHAPE rather than to a word — the criterion is written per-word, and the extension is named here rather than smuggled in. **This amends rules.md#C1 in the same diff.** C1's one-character clause said a bare single letter behind a name word is the generation, full stop, which predicts exactly the suffix "I, Jr." being rejected here. The discriminator the parser has always applied and C1 omitted is the FURTHER COMMA: `Maier, Amy I` reads suffix 'I' while `Maier, Amy I, Jr.` reads middle 'I', and `Smith, John V` against `Smith, John V, Jr.` is the same pair on a name C1 already exemplifies — all four measured. So the rule was incomplete, not the behavior: a rules.md defect rather than a parser one, which is the call this file already made for `'Donald mc'` in the fields-only arc entry (2026-08-27, #451). The qualifier is normative prose carrying no example line of its own; C1's Accepted block records why and names the bank pair as its executable witness. - - `Dr King Jr` — **FIXED 2026-09-08 (the #316/#489 bundle), and this is the one bullet of the four that moved out of NOT FIXED.** v1 wanted title 'Dr', family 'King', suffix 'Jr', and that is what the parse gives; the leading title peel now leaves a name word a suffix cannot be (#H3). The bullet is amended in place rather than moved, because the half it DECIDED is untouched and is still the reason the shape exists at all. What follows is the 2026-09-01 record, with the reading it describes corrected where the fix falsified it. It read title 'Dr King', family 'Jr' then. `king` stays in the titles vocabulary: it is there for the addressing forms ("King Charles"), and taking it out to serve the surname reading trades a common use for a rarer one, which is the direction #vocabulary-collisions cuts. TITLES has no ambiguous subset and no AmbiguityKind, so — as with MAIDEN_MARKERS and `roz` — the only two expressions of the criterion available here are ship and do not ship; #348 is the open work that would give this set a third answer. The comma format is the road to the surname reading, and is now pinned alongside rather than left as prose — `King, Dr Jr` reads title 'Dr', family 'King', suffix 'Jr'. The test cites [#27](https://github.com/derek73/python-nameparser/issues/27), which is closed; this is the decision it never got. - Two halves, and only one was decided. DECIDED: `king` stays in TITLES, which is unchanged and is what still puts `King` inside the title run in the first place. RECORDED, not endorsed: what becomes of the leftover `Jr`. rules.md#S2 predicts suffix 'Jr' with an empty family — its Accepted clause consumes an unambiguous suffix even when nothing is left to be the family (`Smith Jr.` → family "") — but once the title chain had taken `Dr King`, H1 claimed the one remaining word and it read family 'Jr', suffix ''. `Dr Smith Jr` isolated the cause: family 'Smith', suffix 'Jr', exactly as S2 states. S2 carried a descriptive note saying so. This entry then said that a future change moving `Dr King Jr` toward S2's prediction is an IMPROVEMENT and updates the pin — that change is #H3's peel floor and it landed 2026-09-08. The reading is now title 'Dr', family 'King', suffix 'Jr': the run gives back its last word where everything behind it is post-nominal and that word is not itself suffix vocabulary, so `Dr King Jr` reads like `Dr Smith Jr` rather than against it, and S2's note is replaced by the argument it asked for. The pin in `tests/test_suffixes.py::test_king` moved with it, by decision rather than by drift, and `king` being title vocabulary now buys the row a `title-or-name` report (#H4) rather than a different reading. - - `Ahmad ben Husain` — v1 wanted family "ben Husain"; it reads given Ahmad, middle ben, family Husain. Already decided in v0.2.5, when `ben` came out of the prefixes, and for the reason that still holds: `ben` collides with the given name Ben, in the position the particle claim would act on — `Ahmad Ben Husain` reads middle 'Ben' today, which is exactly the token a forward-joining particle claim would take. That is C-i's position test, and it keeps `ben` out. Recorded a second time as a standing keep-out in this file's Excluded block for the particle set, because that is where a wordlist sweep meets it: a keep-out that lives only in a triage entry is one the next Arabic/Hebrew patronymic-particle sweep never reads. Worth naming as a failure mode of its own — the marker was an aspiration that outlived its own resolution, and nothing in a bare xfail says which of the eight were like that. - - `The Right Hon. the President of the Queen's Bench Division` — v1 wanted the whole string as one title; it reads title "The Right Hon. the President of the Queen's Bench", family 'Division'. This is a name parser, not a title parser: handed an input that is all titles it assumes the last title-word is the name. Accepted as convention rather than defended as correct — and since 2026-09-07 the guess is no longer silent: rules.md#H4 states the convention and the parse reports `title-or-name`, which is what [#491](https://github.com/derek73/python-nameparser/issues/491) asked for and is a report rather than a change of reading. See decisions.md#H4. -- 2026-09-01 — FIX CANDIDATES, four. Each carries its issue, so `pytest -rx` names the work instead of listing anonymous aspirations. Two markers remain, both #490's (re-counted 2026-09-23, #492); #489 was fixed 2026-09-08 and #492 on 2026-09-23 and their markers are gone, which is why this line no longer says the marker stays: - - [#489](https://github.com/derek73/python-nameparser/issues/489) — **FIXED 2026-09-08 (the #316/#489 bundle).** `Her Majesty Queen Elizabeth` should address by given name (`tests/test_conjunctions.py::test_conjunction_in_an_address_with_a_first_name_title`), and it does: a title RUN addresses as its LAST title does, `queen` being a given-name title (#H1). The xfail marker is removed and the test is an ordinary passing pin. - - [#490](https://github.com/derek73/python-nameparser/issues/490) — `E.T. Smith` (`tests/test_conjunctions.py::test_two_initials_conflict_with_conjunction`) and `U.S. District Judge Marc Thomas Treadwell` (`tests/test_titles.py::test_chained_title_first_name_title_is_initials`). One issue for two tests deliberately: each test's own comment names the other's shape as what blocks a fix — dotted initials against dotted title and credential vocabulary — so they are one question, and fixing either alone is what has failed before. - - [#492](https://github.com/derek73/python-nameparser/issues/492) — `capitalize()` leaves `juan garcia III` lowercase (`tests/test_capitalization.py::test_capitalization_exception_for_already_capitalized_III_KNOWN_FAILURE`). The `_KNOWN_FAILURE` suffix is kept: it is still true, and beside the annotated reason it reads as redundant rather than misleading. -- 2026-09-23 [#492](https://github.com/derek73/python-nameparser/issues/492) — FIXED (the #459/#492/#478 bundle): `juan garcia III` repairs to `Juan Garcia III`, R5's gate leaving the suffixes out (decisions.md#R5, 2026-09-23). The marker is gone and the test is renamed `tests/test_capitalization.py::test_capitalization_exception_for_already_capitalized_III`, the `_KNOWN_FAILURE` suffix the bullet above kept being no longer true. Two markers remain, both #490's, and the marker counts above — this entry's heading, the FIX CANDIDATES line and the #485 bullet — were amended in place on this date, following the #489 fix's precedent. `tools/differential/corpus.jsonl` still carries the old test name as a label: that file is v1's own suite harvested at a pinned ref (tools/differential/README.md), so the label names where the row came from and is left as it is. -- [#485](https://github.com/derek73/python-nameparser/issues/485) is superseded. It proposed prefixing corpus labels with `xfail:` so a radar diff on a name like `Dr King Jr` reads as a known-bad parse improving rather than a regression. The premise was that a v1 xfail marker is a usable triage signal; after this triage it is not, because the marker no longer tracks the disposition — `Dr King Jr` is now a PIN, so a radar diff on it means a decided reading moved, which is precisely the signal the prefix would have suppressed. The four that remain carry issue numbers, which is the same information in a place that cannot go stale against the pinned historical ref. The 2026-09-08 fix STRENGTHENS this rather than unsettling it, and the argument is worth stating because the fix looks at first like the counterexample: `Dr King Jr` moved, so a diff on it is exactly what #485's prefix predicted — but it moved by DECISION, with a ledger rule and a rewritten pin, which is a classified change and not a known-bad parse quietly improving. Had the prefix shipped, that diff would have read as the aspiration finally coming true and nobody would have been asked to classify it. Two markers remain, both #490's (re-counted 2026-09-23, #492), and they still carry issue numbers. - -### suffix-field-composition — three kinds of thing in one field - -- 2026-08-16 (suffix keystone; #326) — measured composition of suffix_words (40 entries): 11 generational (i, ii, iii, iv, v, jr, jnr, sr, snr, junior, 2), 5 neither (dr, esq, esquire, ret, vet), and 24 POSTNOMINAL HONORIFICS — 20 CJK (さん, さま, くん, ちゃん, 様, 殿, 氏, 先生, 博士, 教授, 女士, 小姐, 씨, 양, 군, 님, 박사, 박사님, 교수님, 선생님) and 4 Hebrew (ז"ל, ז״ל, שליט"א, שליט״א). The honorifics are the LARGEST group. -- Decision: do NOT split the field. Record the composition; #296, - #291, #325 and #289 proceed on their own terms rather than - waiting on it. -- Why #326 cannot be taken at face value: it argues the split is tractable because "the vocabulary is already split — suffix_acronyms is credentials, entirely; suffix_words is generational plus a handful". That was true when written (2026-08-02); the 2.1.0 East Asian work landed 2026-08-07 and put the 24 honorifics in the same set. CLDR's two buckets do not cover what the set now holds, so "adopt CLDR's model" is not available as the cheap answer. The issue's table is corrected on the issue. -- The unexamined question this leaves: whether the honorific bucket belongs in `suffix` at all. rules.md#W3 already argues an honorific is "no part of the name on either side" — the same language the H section uses for prenominal titles — so the conflation with PhD may be worse than the generation/credentials one #326 was filed about. Deliberately not decided here. -- Recorded as a documentation failure mode, not just a fact: `suffix` started generational, absorbed credentials, then absorbed postnominal honorifics, each step locally reasonable and none recorded as a widening of the field's MEANING. rules.md#S2's Background still calls it "two different things", which was accurate when written. The rules doc pinned the behavior faithfully; what slipped is the field's definition, which no rule states because no rule owns it. Field definitions need the same discipline rule statements get. -- The 11/5/24 split above is this entry's own version of the hazard it describes: a quoted vocabulary composition, dated, exactly as - #326 quoted one. A date does not stop rot — #326 carried one too - — so before relying on the split, check the set still looks like it: - - uv run python -c "from nameparser import Parser; print(sorted(Parser().lexicon.suffix_words))" - - What is durable here is the SHAPE of the finding — three kinds of thing, the honorifics the largest — not the three integers. A test asserting the counts is deliberately not the answer: that is the constant-content pattern, and it would fail on every legitimate vocabulary addition. - -### deviates-registry — packs stay pure data (option C) - -- 2026-07-18 (d4aaafa; the DEVIATES design note) — a `deviates` field ON Locale was rejected: callables break value equality and pickle-by-reference, and DEVIATES is acceptance metadata, not runtime behavior (revisit only if a runtime consumer like explain() appears). The REGISTRY became the contract instead: test_registry_is_the_pack_contract fails structurally unless every registered pack ships DEVIATES and its rotator list. This is why LOCALE-PACKS-PURE-DATA can promise "no code paths of their own" and mean it. - -### normalization-fold — lower(), not casefold(); comparison folds harder - -- 2026-07-17/19 — vocabulary storage normalizes with lower(), NOT casefold(): casefold mutated stored spellings (κος→κοσ, großfürst→grossfürst) while lower() keeps authored forms; the accepted cost, pinned deliberately, is that ASCII-SS GROSSFÜRST no longer matches. The paired half: comparison_key()/matches() use casefold() ON PURPOSE — comparison is the one surface that wants aggressive folding; this is a release-log-classified 2.0 CHANGE from 1.4's lower() (ß and final-sigma forms compare equal only since the v2 core; v1.4 has no casefold anywhere). The 1.x line was symmetric BY DESIGN — lower() everywhere, chosen for symmetry with the parser's lc() convention — so the storage-vs-comparison split is a v2 decision moving away from a real, deliberate alternative, not a fix of an oversight. Do not - "fix" it symmetric: symmetric is what 1.x was. -- 2026-07 (rc1 arc) — the fold must reach a FIXED POINT, and anything built on it must converge too: _title_key joined per-word folds and kept empty slots, so given_name_titles with a foldable word stored a key match-time could never rebuild — silently inert, and pickle round-trips then rejected the state. Shipped in the rc, caught in review, fixed by dropping words that fold away. - -### nfd-shapes — shape tests read letters, and a decomposed letter is one letter - -- 2026-10-12 (#652) — a shape regex spelled a letter `[^\W\d_]`, and `\w` does not match a combining mark, so a name typed in NFD (`inž.` as i, n, z, U+030C) ended its word at the z: `Dr. inž. Janez Novak` made `inž.` the given name, and `Janez Novak, ŽŠČ` reversed because `str.isalpha()` refuses the caps shape. Vocabulary was never affected (`_normalize` NFC-folds); only the shape path was. The bug is as old as 1.4.0, whose `period_abbreviation` had the same class, so this is a fix and not a restoration. DECIDED: a letter in the four letter-shape regexes is `_vocab._LETTER` (letter plus the general combining blocks), `config`'s `period_abbreviation` carries the same class, and the caps predicates NFC-compose a word that is not already NFC before asking `isalpha()`, and the two-letter pair test counts letters rather than code points (`composed_len`; the first version of the fix split `John Smith, MÁ` the other way, found in review). The tests as written (`str.isalpha()`, `unicodedata.is_normalized`) are C-level, so a word already composed pays no frame: `Smith, José` 193 and `Janez Novak, Žiga` 237 frames before and after, default and EVERYWHERE (measured 2026-10-12, py3.11). Output keeps the encoding written; only the reading is made encoding-independent. MEASURED MOVES the release log cannot see, against the released 2.4.0: NFD `Janez Novak, ŽŠČ` went from given `ŽŠČ` (reversed) to suffix `ŽŠČ`, and NFD `Janez Novak, Ž.Š.` went from suffix `Ž.Š.` to given `Ž.Š.`, family `Janez Novak` — the composed twin's reading, which is C1's (paired initials are a name), so the second moves AWAY from the suffix reading and toward parity. Neither is in a corpus. -- DECLINED to widen the class to every script's marks: Indic vowel signs and Arabic and Hebrew points are marks too, but rules.md#H2's Accepted clause already records that an abugida abbreviation never reads as a title by shape, and a general `\p{M}` would change that without a measurement. KNOWN LIMITS, left open (#665): `_INITIAL` / `_render._INITIAL` / `config` `initial` (`\w\.`) are v1-verbatim and public, and an NFD single initial (`É.`) is not initial-shaped there. No role or `initials()` result differs for a lone initial (`Janez Ž. Novak`, 17 shapes × 3 orders), but one does beside a connective: NFC `É. y É. Smith` reads given `É.`, middle `y É.`, NFD reads given `É. y É.`. The same width problem makes the decomposed Cyrillic conjunction `й` (two code points) miss the single-letter tests at `_classify.py` (`len(token.text) == 1`) and `_pieces.py` (`len(text) == 1`): NFD `John й Smith` reads family `John й Smith`. Both predate #652 and read identically on 2.4.0 and master; neither is in a corpus. -- 2026-10-13 #665 — DONE, closing both KNOWN LIMITS above. The initial shape is asked of the composed spelling: `_vocab.is_initial_shaped` NFC-composes a non-ASCII word before `_INITIAL`, as `_render` already did for its two copies (#542). Composing was chosen over widening the class because the pattern is the public v1 `initial` regex, pinned verbatim by `test_regex_sync`, and because it gives parity BY CONSTRUCTION — a decomposed word reads as the word it composes to, and a letter-plus-mark with no precomposed form has no composed twin to disagree with, so it is left as the pattern reads it — and likewise the width tests count composed code points, so a caller's one-letter connective with no precomposed form (`q́`, added to `conjunctions`) is two letters to them and joins as a longer connective does (`John q́ Smith` reads given `John q́ Smith`), as rules.md#P3's "composes to one character" states. The three single-letter width tests (`_classify`'s connective fork and its one-case report, `_pieces`' three-word carve-out) count letters: the classify pair asks `composed_len` only for non-ASCII text that the vocabulary membership has already admitted and the C-level `len(...) == 1` has declined, and `_pieces` composes in C before `isalpha()`. ASCII text pays no frame and the width tests cost composed text nothing (`Dr. Juan Q. Xavier de la Vega III` 366, `Smith, José` 194, `Juan y Eva Garcia` 254, `John й Smith` 255 composed, all equal before and after); the veto below costs one frame per non-ASCII word that reaches it, its matcher being called through a wrapper (`É. y É. Smith` 305 → 307, `Janez Ž. Novak` 237 → 238, `田中. 太郎` 207 → 208; measured 2026-10-13, py3.11, all composed). Kept rather than composing inline at its four call sites: a rare shape no guard counts, against four copies of the one step whose omission this entry is about (AGENTS.md's frame rule, question 4). Decomposed `John й Smith` reads given John, middle й, family Smith, where 2.3.0 through 2.4.1 read the whole name as the given name and reported given-or-family (the KNOWN LIMITS bullet above says family; measured on 2.4.0, 2.4.1 and f9b2b81c it was given, and this bullet corrects it) — a regression of 2.3.0, whose NFC lookup fold (#cjk-full-stops) let the decomposed й match the conjunction vocabulary while the width tests still counted two code points; 1.4.0 through 2.2.0 never matched it and read it as middle й, so this restores their reading. Decomposed `É. y É. Smith` reads given É., middle `y É.`, where every release from 1.4.0 read given `É. y É.`, so that half is a fix. Both checked on the 1.4.0, 2.0.0, 2.2.0, 2.3.0, 2.4.0 and 2.4.1 wheels (2026-10-13). The #320 veto beside the shape is asked of the composed spelling too (`_vocab.in_initialless_script`, ASCII answering before the matcher): decomposed hangul is conjoining jamo, which no `_NO_INITIALS` range holds, so composing the shape alone made a decomposed `씨.` an initial — `김민준, 씨.` read title `씨.` — which the first version of this fix did, caught by `test_suffix_lookup_reads_every_full_stop`. The same veto read on jamo had long let a decomposed hangul word with a period take the leading-abbreviation shape (#H2): `김. 민준` read title `김.` decomposed in every release from 1.4.0 (checked on the 1.4.0, 2.0.0, 2.2.0, 2.3.0 and 2.4.0 wheels), and reads family `김.`, given `민준` as composed now; rules.md#H2's veto clause states it and carries the decomposed example. An UNSPACED decomposed hangul name is still not split (docs/usage.rst's documented exception), so `김민준. 이서연` keeps differing by encoding. The guard is `tests/v2/test_parser.py`'s twin test, which now compares ambiguity kinds and `initials()` as well as roles; its docstring records which rows each of the four sites holds. - -### render-default — the default view's format - -- 2026-07-18 (bf1141c) — the default spec is '{title} {given} "{nickname}" {middle} {family} ({maiden}) {suffix}'. Declined the same day: a née-template maiden ('née {maiden}') that round-tripped LOSSLESSLY (née is a maiden marker), rejected on presentation grounds. Accepted cost of the shipped form: a parenthesized maiden re-parses as a nickname. - -### P3 — connectives - -- 2026-07-30 #267 — the four-word single-letter asymmetry: a bare Latin capital connective is vetoed as an initial while a Cyrillic capital joins. #267's closure ("v2.0 behaves this way by default … verify it was the right call") showed only the Cyrillic half; the Latin-capital veto was never separately adjudicated, which rules.md#P3 records as an Accepted consequence. -- 2026-08-17 — a conjunction-joined run counts as ONE name word in every rule that counts them. Raised while bounding the leading particle run (#395): a run that stops "after the first name word" would otherwise be able to stop INSIDE a part P3 built, splitting "de la Vega y Santos Juan" between "Vega" and "Santos". This is not new behavior — H1 already counts this way, measured: "Mr. Jack and Jill" gives family="Jack and Jill" while "Mr. Jack Jill" gives given="Jack", family="Jill". What is new is that rules.md says so, which is what binds #395's implementation. Recorded because the first draft of the clause asserted that no rule could show it today and shipped an Accepted note to that effect; the shipped H1 pair was simply never run. Detector failure of the usual shape — reasoning about which rules count instead of parsing a name and looking. - -- 2026-08-21 #418 (PR #420) — the three-word carve-out counts the name that remains once the maiden name leaves. The count was taken while the marker and the maiden name were still pieces, so `juan y garcia nee jones` counted five, 'y' joined, and the family was empty once the clause left — given 'juan y garcia', family ''. Nine corpus names show it under an appended clause. Closed by order, not by an exclusion: the marker pass now runs before the count (decisions.md#M2, 2026-08-21), and the three-piece gate ahead of the count, which had the same exposure, is closed the same way. rules.md#P3 now says the count is of the name's own words. The question #397 and #411 left open — whether a word's membership in one vocabulary should suppress it from a count gating a rule keyed on another — is untouched: this was never a membership question, the words are simply not in the name. - -- 2026-09-13 #383 + #479 — BLESSED for the mixed-case half, and DECIDED BY VOCABULARY for the one-case half, which is the half the question never reached. A single-letter connective joins only on positive evidence. Where a name is written in more than one case the writing is that evidence and nothing moves: a bare LATIN capital reads as an initial ("Jose E Maria Santos" → middle "E Maria"), which is #383 answered with "bless". The shape is still LATIN and the phrase "Latin-capital veto" does not retire with this change — `_INITIAL` is ASCII-only, so a Cyrillic capital is never initial-shaped, joins, and #267's reading above is untouched. An earlier draft of this work claimed the phrase retires; it was overstated, and what became evidence-based is the one-case half alone. - WHERE THE WRITING SAYS NOTHING. A name written wholly in ONE case — all upper and all lower alike — carries no case evidence about any letter in it. The reading there comes from a new marker subset `Lexicon.conjunctions_ambiguous` ⊂ `conjunctions`, default `{e}`: a member reads as an INITIAL and reports the new `AmbiguityKind.CONJUNCTION_OR_INITIAL`, a non-member JOINS even written as a bare capital and reports nothing. That is the third instance of the ambiguous-subset pattern, after `particles_ambiguous` and `suffix_acronyms_ambiguous`, so no mechanisms.md entry is owed. - ROW 1 OF #479 IS THE DEFECT, NOT ROW 4 (Derek, 2026-09-12). The one-case lowercase reading is the one that was wrong, not the one-case uppercase reading that looked wrong beside it. - WHY `e` AND NOT `y`. A bare E initial is common (Edward, Elizabeth) and an "e" between two surnames is rare outside couple listings; a bare Y initial is rare and "y" between two surnames is the commonest Hispanic compound and this library's oldest fixture ("Velasquez y Garcia"). The 2026-08-18 LOCALE CONTROL bullet under #3-0-reevaluations already said that which letters a tradition wants joined is per-word POLICY rather than membership; this subset is that policy made configurable, which is the smaller half of what that bullet asked for and ships without locale packs. `i` is not here because it is not conjunction vocabulary at all yet; it ships in the subset when #397 adds it, a bare I initial being as common as a bare E. - MEASURED (2026-09-13, this branch, over the deduped `tools/differential/corpus*.jsonl` glob — 1174 distinct names). SEVENTEEN names are written in one case and carry a cased single-letter connective: that is the WHOLE population this fork can reach, and ten of them move something against 2.3.0 (a role, `initials()`, or the report). Recompute the population by keeping every corpus name whose space-joined text equals its own `upper()` or `lower()` and which holds a one-character token that is in `conjunctions` after `_lexicon._normalize` and whose `upper()` and `lower()` differ; recompute the movers by parsing each of them here and under a released wheel and diffing the seven role fields, `initials()` and the ambiguity kinds. Of the EIGHT such names the corpora already held before this PR added its own examples, not one moves a role — the three-word carve-out or an existing initial reading covers every one of them. The two role moves are both new example names, and they break 1.4.0 parity in OPPOSITE directions: `jose e maria santos` read given "jose e maria" at every release from 1.4.0 through 2.3.0 and now reads given "jose", middle "e maria"; `JUAN GARCIA Y LOPEZ` read middle "GARCIA Y" and now reads family "GARCIA Y LOPEZ" (measured on the released 1.4.0 and 2.3.0 wheels from a throwaway environment, 2026-09-13 — this tree shadows the wheel otherwise). The ledgers at all five baselines classify them. - THE RULE IS PER TOKEN AT CLASSIFY TIME AND REACHES EVERY NAME LENGTH (Derek, 2026-09-13), the three-word shape included: `john e smith` reads middle "e" tagged as an initial, so `parse(...).initials()` gives "j. e. s." where 2.3.0 gave "j. s." and `capitalized()` gives "John E Smith" where 2.3.0 gave "John e Smith". P3's three-word carve-out is untouched — it already says a single letter in a short name is more likely an initial, and this is the same judgement extended to the letter's case — so no ROLE moves at three words and the visible change is in the two derived views. - BOTH QUESTIONS ARE ASKED OF THE NAME'S OWN WORDS — the case class AND the fork, not just the three-word count. Tokens standing before the first maiden-marker run head and not already carrying a role from a delimited clause are the name; the rest are not. Found by measurement rather than by design: appending " née Jones" made an all-caps name mixed-case and flipped the fork, failing the repo's invariant that a maiden clause changes nothing else, and a delimited `"Y"` was being conjunction-tagged (so R4 lowercased it) while a delimited `"E"` reported. ACCEPTED edge, and it is WIDER than an early draft of this entry said (measured 2026-09-13, in review): a maiden marker's run and every word AFTER it leave the own-words span the moment classify tags the marker, and they stay outside whether or not the marker pass later DECLINES the marker and leaves it a word. So a declined marker does two things, not one. It still does not count toward the case class, which is visible only when the declined marker word is the only differently-cased token (`JUAN Y GARCIA née` joins where `JUAN Y GARCIA née Jones` keeps "Y" a name word) — that much the early draft had. But it also leaves any connective standing after it to the MIXED-CASE rule rather than to the one-case fork, so a bare Latin capital there is an initial and its lowercase spelling joins. That is the one shape in which a name's OWN words read differently in its two one-case spellings: `JUAN NÉE JR Y LOPEZ` reads given "JUAN", middle "NÉE JR Y" with its "Y" tagged `initial`, while `juan née jr y lopez` reads given "juan", middle "née", family "jr y lopez" with its "y" tagged `conjunction`. SCOPE THAT PHRASE TO THE OWN WORDS, because a clause's words are not covered by it and a re-review caught the over-reach: a letter inside a delimited or maiden clause is read as it always was, which still means a different TAG in the two spellings, and case repair shows it — measured 2026-09-13, `JANE VAN DER BERG NÉE Y JONES` capitalizes to `Jane van der Berg (Y Jones)` where `jane van der berg née y jones` gives `(y Jones)`, and `JUAN "Y" GARCIA LOPEZ` keeps nickname `"Y"` where the lowercase spelling keeps `"y"`. The FIELDS agree in both pairs; what differs is the repaired case of a clause's word, which is that clause's rules doing their job rather than this one's. ACCEPTED rather than repaired, on two grounds: classify cannot know what group will decline, and a clause's words are read by the clause's own rules — which is the same doctrine the own-words span states, applied to the one shape where it bites. rules.md#P3's Accepted clause carries the same wording and the same example pair. (2026-10-11, release sweep: #602's credential run now reads `JR Y LOPEZ` as suffixes in the capitals spelling, so P3 carries the pair `DR. NÉE JUAN Y LOPEZ` / `dr. née juan y lopez` instead, the marker declined behind a title the same way: middle "JUAN Y" with "Y" an initial, against family "juan y lopez".) - REPORT SCOPE IS SUBSET MEMBERS ONLY. Both readings are live only there; "y" in a one-case name is not in doubt, and reporting it would be noise on every lowercase Hispanic record. Matches how `particles_ambiguous` and `suffix_acronyms_ambiguous` report, and matches AGENTS.md's "a kind is worth adding only if a reader would hesitate too". - CASELESS LETTERS NEVER ENTER THE FORK. Arabic و has no case, so `token.upper() != token.lower()` is false and today's reading stands. A caseless-script input counts as "one case" under the helper, harmlessly, because the fork also requires a cased token. - NO SWITCH. The subset is the knob: remove "e" to restore the joining reading, add "y" for a Dutch-style "every single letter is an initial". That is also this shape's answer to #516's switch question. - ACCEPTED, AND RECORDED RATHER THAN FIXED: a locale pack cannot express "remove e". `Locale.lexicon` is unioned onto the base and never removes — its own field comment says "a pack never removes base vocabulary" — so an `nl` pack CAN add "y" while a `pt` caller has to write `Parser(lexicon=Lexicon.default().remove(conjunctions_ambiguous={"e"}))` by hand. The first (A) bullet added to #3-0-reevaluations on this date asks whether packs should be able to remove vocabulary at all. - ACCEPTED, AND DEFERRED TO A FOLLOW-UP ISSUE (Derek, 2026-09-13): the facade and the core now disagree about the same letter. `HumanName.initials()` does not follow the parse's tags — `_facade._process_initial` re-derives "is this the connective" from vocabulary plus initial shape on the part's raw text — so `HumanName("john e smith").initials()` stays "j. s." while `parse("john e smith").initials()` gives "j. e. s.", and `JUAN Y GARCIA` splits the other way, facade "J. Y. G." against core "J. G.". Both measured here 2026-09-13 and pinned by a contrastive test rather than left to prose. `_facade.py` is untouched in this PR; the fix has R4's shape — consult the PARSED token wherever the part maps to one and fall back to the vocabulary only where it does not — and it is a follow-up issue, not yet filed, so no number is cited here. `capitalize()` is not split: it follows the parse on both surfaces, so `HumanName("john e smith").capitalize()` gives "John E Smith" too. - 2026-09-13 #528 — AMENDS the paragraph above: CLOSED THE SAME DAY. `_facade.py` was touched after all — #528 shipped the fix the paragraph predicted, with the R4 shape it predicted: the facade consults the PARSED token wherever the part maps to one and falls back to the vocabulary only where it does not. Both views give "j. e. s." and "J. G." now (measured 2026-09-13 after the fix); the contrastive test the paragraph above describes became the agreement test `tests/v2/test_render.py::test_facade_initials_follow_the_one_case_fork`. `decisions.md#R3`'s 2026-09-13 entry carries the fix, its accepted costs and the one corpus name where the two views still differ. `capitalize()` was never split: it followed the parse on both surfaces throughout the window the paragraph above describes, giving "John E Smith" for that whole window — not "all along", since 2.3.0, before #383/#479 landed, gave "John e Smith" for the same name. - TWO OTHER CAUSES SHARE THE `_initials` FIELD IN THE LEDGERS AND NEITHER IS THIS PR'S, recorded because a reader meeting them under these names will reach for this entry. The Arabic `محمد و علي` diffs on `_initials` at 1.4.0 only (`م. و. ع.` → `م. ع.`), measured byte-identical either side of the fork: it is a pre-existing #269 consequence — a recognized non-Latin connective contributes no initial — surfaced by the row entering the contract corpus, and `expected_since_1.4.0.toml` ledgers it as `feat(#269)`. `JUAN Y GARCIA` diffs on `_initials` at 2.0.0 through 2.2.0 for a DIFFERENT reason than it does at 2.3.0: at those three baselines `fix(#462)` already admits the name (the facade moved `J. G.` → `J. Y. G.` there), and this PR's core move `J. Y. G.` → `J. G.` gets its own rule at 2.3.0 alone. Two causes, one field, documented in a dated paragraph on `fix(#462)` in those three ledgers rather than as a competing rule. - Out of scope, each its own issue: #492 (whether a cased suffix token counts as case evidence — `is_one_case` is written so R5 and #492 can share it later, but render does not import it here); #478 (hyphenated connective repair — its spaced-form claim now depends on "y" staying OUT of the subset); #461 (R3's clause); #289 and #516 (the same case-class fact read at the suffix and post-comma slots); the render-side conjunction fallback for spliced raw text, which stays vocabulary-keyed as rules.md#R4 states. - - 2026-09-15 #289/#516 — AMENDS the out-of-scope list above: #289 and #516 are no longer out of scope, having shipped in 2.4 on the fact this entry's own fork computes. The fact is now a `ParseState` field rather than a classify local, taken over the SAME own-words span this entry defines, and `_pieces.own_words` is where that span lives for both readers. #492 stays out of scope and stays deferred. See #S2. - - 2026-09-23 #492/#478 — AMENDS the out-of-scope list above again: both have landed, in RENDER only. #492: R5's gate leaves the suffixes out (decisions.md#R5, 2026-09-23); this entry's fork is untouched and still counts a cased suffix, which is the R5/parser split that entry records, and render still does not import `is_one_case`. #478: a connective part inside a hyphenated word keeps its lowercase (decisions.md#R4, 2026-09-23); the spaced-form claim it rested on is unchanged, and still depends on `y` staying out of the subset. - -- 2026-09-13 #383/#479 — `conjunctions_ambiguous` is deliberately NOT registered in `_SUBSET_FIELDS`, so an orphan marker entry raises nothing. An orphan there is INERT rather than harmful: the classify fork tests membership in `conjunctions` before it reads the subset, and the emitter tests it again, so a marker entry whose base word is gone is never consulted by either. AGENTS.md's invariants rule guards harm, not no-ops, and the precedent is this file's #given-name-titles Declined entry, where two attempts at a check each cost a working configuration to forbid a condition that costs nothing. What holds the SHIPPED constant to the subset relation is an import-time assert in `nameparser/config/conjunctions.py` (which also holds it to cased single letters, the only entries the fork can read), and the v1 shim's intersection, which is a provable no-op. So `remove(conjunctions={"e"})` simply works, leaving a stale marker entry behind that does nothing. Pinned by `test_removing_a_conjunction_leaves_its_ambiguous_marker_alone` in tests/v2/test_lexicon.py and by the orphan behavioral pin in tests/v2/pipeline/test_classify.py, both of which cite this entry. - -- 2026-09-13 #383/#479 — AMENDS the 2026-08-27 #445 entry under M4 below, which calls `initial` "the `initial` shape tag": since this change the tag is no longer purely a shape claim, because a marked single-letter connective in a one-case name carries it without being initial-SHAPED. rules.md#M4's statement now says "read as an initial" for that reason, and M4's carve-out itself is unchanged — it still reads the tag classify recorded rather than a predicate of its own. - -- 2026-09-20 #397 — DECIDED AND SHIPPED: `i` joins `conjunctions` and `conjunctions_ambiguous` together, which is what the 2026-09-13 entry above said would happen and which half of it the Open block below called contested. `parse("Josep Carod i Rovira")` gives family "Carod i Rovira" where every release from 1.4.0 through 2.3.0 gave middle "Carod i" with family "Rovira"; `parse("Josep Lluis Carod i Rovira")` gives middle "Lluis" with that same family; and `parse("Carod i Rovira, Josep")` gives family "Carod i Rovira" where they read family "Carod Rovira" and took the link into `suffix` as a generation marker (all three measured 2026-09-20 on the released 1.4.0, 2.0.0, 2.1.0, 2.2.0 and 2.3.0 wheels from a throwaway environment, this tree shadowing a wheel otherwise). - THE OFF SWITCH is `Lexicon.default().remove(conjunctions={"i"})`, and on the v1 surface deleting `i` from `C.conjunctions`; both restore all three readings above byte for byte, measured 2026-09-20. That is a PROPERTY rather than an anecdote, and it is this change's durable statement, and it takes THREE exemptions rather than two once case repair is included — the earlier wording of this sentence named two and the release note named none. A name's FIELDS equal its reading with the letter out of the vocabulary unless (1) a link joined between two name words or (2) the two readings disagree about the letter being an initial; its REPAIR, plain and forced, equals the off switch's unless one of those two holds or (3) the parse placed the letter among the NAME words as a connective, where R4 keeps it lower-case and the off switch — for which it is no connective at all — title-cases it: `parse("Dr. John i Smith").capitalized(force=True)` keeps 'i' against the off switch's 'Dr. John I Smith', and `Carod i Rovira` and `Josep i Rovira` are the same shape. Measured 2026-09-20 over the shipped grid's 28,448 class-letter rows: 1,462 take the join exemption, 9,296 the initial-reading one, and 2,228 more reach the third, which is INV7's alone. `tests/v2/test_properties.py::test_a_link_that_joins_nothing_changes_no_field` runs it over a 16,576-text grid paired with the configurations that can read each text differently — 66,752 rows, of which 28,448 carry a class letter and are parsed twice — and carries the recorded control: 1,480 failing parses at `c8550b64`, the commit the first review was written against, 154 at `dc3bdf9c`, and 0 here. (Quoted over the full cross product of texts × lexicons × policies the earlier numbers were 2,360 and a 12,880-name grid times eight parsers; the shipped grid is trimmed by SHAPE and grew again with the second review, so the test's own docstring is the live figure and this is a dated snapshot.) - THE COUNT REACHES THE CLASS (Derek, 2026-09-20), which ANSWERS the C-ii question the 2026-08-21 #418 bullet above left open — "whether a word's membership in one vocabulary should suppress it from a count gating a rule keyed on another". It does not: a single-letter connective counts as a name word toward P3's three-word carve-out whatever else the vocabulary says it is, and `i` is also the roman numeral. The alternative was rejected on its own arithmetic rather than on taste — counting the link as the generation it also spells lets a connective raise the bar for ITS OWN join, so "Josep Carod i Rovira", the shortest linked name and the commonest one, would be the only shape that failed to link while the four-word spellings joined. The arithmetic runs the other way too, which is why the count is scoped to where the word is PLACED TO JOIN: a connective joining nothing would otherwise LOWER the bar for an unrelated one, and "Carod y Rovira i" is that shape — it counted the trailing generation and let the "y" join. It reads given "Carod", middle "y", family "Rovira", suffix "i" here, the same FIELDS the parent gave — its initials moved, but under R3's rule rather than this one. - ACCEPTED CONSEQUENCE OF REACHING THE CLASS, and the one row the design's reading tables show it touching: under `Lexicon.default().add(particles={"y"})`, `Juan Velasquez y Garcia` goes from given "Juan" / middle "Velasquez" / family "y Garcia" to given "Juan" / family "Velasquez y Garcia" — the custom lexicon now AGREES with the default one, which is the right direction — and its `family_base` becomes "Velasquez Garcia", a pre-existing R2 limit (`family_base` drops a particle wherever it stands, not only leading) surfaced rather than created here. Measured 2026-09-20 on this branch. - WIDER THAN ONE ROW, and the second review measured how much: the design's reading tables show one name because they hold one of this shape, not because one moves. Over the review's 43,925-name generated grid, `add(particles={"y"})` moves a word the parent roled SUFFIX into a name part on 130 names BEYOND what the default lexicon already moves there — "ABDUL Y ROVIRA MA" goes from given "ABDUL" / family "Y ROVIRA" / suffix "MA" to given "ABDUL Y ROVIRA" / family "MA" — and 71 of those 130 still report `suffix-or-name` on the moved word, so the reading is marked rather than silent. Four more lose a title, all four the same name in its four spellings ("Jr. y Soler Puig MA" and its casings). The direction is the one the row above describes: the custom lexicon converges on the default one, the particle joining what the connective already joined. Measured 2026-09-20. - THE BOTH-SIDES SCOPE, AND THE REVIEW THAT FIXED IT. A connective that is also generational vocabulary joins only where a NAME WORD stands on each side of it — a word the rest of the parse reads as a name word rather than as a credential, the trailing numeral or a title, looked for past any run of connectives standing between. The FIRST implementation tested POSITION (an interior piece) rather than the neighbours' CLASS, and swallowed a credential: "Josep Lluis Carod i III" read family "Carod i III" where the "III" is the generation. An independent review found it. It is recorded here because the invariant that ought to have caught it cannot: INV1 inspects the part the join PRODUCED, where the absorbed credential is itself the name word standing on the right, so that name SATISFIES it — the general lesson is a field note in mechanisms.md of this date, and the instrument is `tests/v2/test_properties.py::test_a_trailing_credential_never_joins_into_a_name_part`, which asks the off-switch parse instead and carries its own control (639 failing parses at `c8550b64` over the shipped grid, 105 at `dc3bdf9c`, 0 here; 960 over the full cross product, a dated snapshot). - THE BOUND THE CONDITION READS, AND THE SECOND REVIEW THAT FIXED IT (2026-09-20). "A name word" is looked for between assign's leading-title run and the start of its trailing suffix run, and that second bound was read by `_pieces.trailing_start`, the peel over the pieces as WRITTEN. A trailing TITLE hides the suffix run from that peel — it takes nothing and answers `len(pieces)` — so the condition was told a credential was a name word and the join swallowed it: `John Quincy Adams i MA Prof.` read family 'Adams i MA' with no report and an `initials()` of "J. Q. A. M.", where `John Quincy Adams i MA`, the same name one title shorter, reads family 'Adams' with suffix 'i MA' and reports the acronym. `John Quincy i MA Prof.`, `Josep Lluis i MA Prof.` and their one-case spellings are the rest of it. THE FIX IS H5's OWN READING, `_pieces.trailing_start_past_titles`, which runs the peel and the title chain to the fixed point `tail_reading` already computes for assign — so the join and the placement agree about where the name ends, which is what H5's sentence says they must. Asked once per segment and only where such a connective was found, so no ordinary name pays it; frame-neutral on `tools/perf/call_count.py` (parse=406.00, facade=443.00). Measured over the review's 43,925-name grid under eight configurations: 27 rows move, on 9 distinct names, every one of them onto the reading the off-switch parser gives. - AND THE CONDITION IS ASKED ONCE PER RUN, not once per member, which is a THIRD fix in the same clause and was a defect the second review left standing (found in its own report, fixed 2026-09-20). `_name_word_beside` walked the run of connectives beside a link to find the first name word past it, and both callers ask it per member, so a name holding a run of n links walked it n times and the stage went quadratic in the run's length: `"Josep " + "i " * n + "Rovira"` grew 3.8x per doubling against the 2.0x every other shape holds — 59ms at n=800 and 214ms at 1,600, where the parent 46651750 reads 8.3ms — and `"Jane Doe nee Puig " + "i " * n + "Soler"` the same, the clause's own caller. Every member of one contiguous run has the SAME nearest name word on each side, so `_group._run_neighbours` answers it once for the whole segment in two linear passes, built on the same gate as the two bounds. All four shapes now read 1.97-2.03x per doubling to n=1,600. THE PRICE IS A CONSTANT ON SHORT NAMES and is recorded rather than hidden: one call plus `_is_conj_piece` for the pieces the walk never reached, so `Josep Carod i Rovira` goes 304 -> 307 frames and `Jane Doe nee Puig i Soler` 307 -> 312, against 6,741 -> 2,652 for that second name with a run of 64. A name with no generational connective is untouched (`John Smith` 163, `Smith, John` 185, `Juan Garcia y Lopez` 285, `Jane Doe nee Smith` 233, all unmoved). Byte-identical over 1,748,904 parses — 145,742 names, the corpus, the case table, all three property grids and a run-heavy generated set, under three lexicons and four policies — same digest on both trees. Guarded twice, because one instrument cannot reach both callers: `tests/v2/test_benchmark.py`'s thirteenth `_SHAPES` row (`link_run`, `"i Und "`) times the join's caller, and `test_a_clause_link_run_does_not_cost_quadratically` counts frames for the clause's, which no repeated unit can express. - NOT FIXED AND RECORDED INSTEAD, because it is neither this rule's nor this branch's: the PARTICLE CHAIN reads the same bound the same blind way, so a trailing title lets it run to the end of the segment. `de la ma prof.`, `de la e ma prof.`, `van der berg ma prof.` and `Josep de Carod y Rovira MA Prof.` all put the title inside the family at the parent 46651750 and at 1.4.0, with no letter of this class in them at all. What the branch does is send MORE names down it — once `i` is marked, a one-case `de la i ma` stops ending in a suffix piece and the chain reaches past it — so `de la i ma prof.` and `de la i rovira prof.` now read as their `e` twins always have. 25 names of the review grid are in this family and none of them is a `_name_word_beside` case. It is the #535 boundary and belongs to its own change; `tests/v2/test_properties.py`'s INV1-strengthened carries a LINK-FREE CONTROL so the invariant can tell the two apart, and says so where it is asked. - THE CLASS TEST IS NOT A LENGTH TEST, corrected by the second review (2026-09-20). The stage's own filter carried a `len(tok.text) != 1` beside the two tag tests, narrowing this clause to ONE-LETTER connectives where the rule names a class. Caller-reachable: under `Lexicon.default().add(conjunctions={"og"}, suffix_words={"og"})`, `John Quincy Smith og` read family 'Smith og' with the filter and reads family 'Smith' with suffix 'og' without it — the rule's own answer. DROPPED, measured byte-identical over every oracle in use (fields, reports, `initials()`, `capitalized()` plain and forced, every token's role) across 359,053 parses: the 351,400-parse review grid under eight lexicon/policy/locale configurations, the 2,175-parse sweep of `tests/v2/cases.py` under three orders and the 5,478-parse sweep of the differential corpora under six. It has to be — `i` is the only member of the class in the default vocabulary and in every locale pack, and it is one letter. The THREE-WORD carve-out stays single-letter, because that is what its own sentence says. - THE SCOPE'S ALTERNATIVES, MEASURED 2026-09-20 rather than argued, by building each alternative from this tree and sweeping the 1558 non-empty corpus-union-cases names under eight configurations (default, `lenient_comma_suffixes=False`, both family-first orders, and the four locale packs). ABSENT — no both-sides condition at all — differs on five names, and every one of them is a row this bundle added: "John Quincy Smith i" would read family "Smith i", "Josep Lluis Carod i III" family "Carod i III", "Josep Lluis Carod i V" family "Carod i V", "Carod y de Rovira i" given "Carod y de" with family "Rovira i", and "Rovira, Josep Carod i Jr." middle "Carod i Jr.". Nothing that stood in the corpus at the parent separates absent from narrow, so those rows ARE the evidence and pruning one retires it. WIDE — the same condition with the generational-vocabulary test dropped, so it reaches every single-character connective — differs on five as well, and THREE of those were in the corpus already: "Mr. & Mrs. John Smith" would read title "Mr." with given "&" and middle "Mrs. John", "1 & 2, 3 4 5, Mr." would lose "2" out of the family, and "Jane née Jr y Jones" would read middle "née Jr y" with family "Jones". So the class test is not decoration — widening the rule past the link-and-generation class breaks "Mr. & Mrs." — and narrow is both what the design asked for and what measurement supports. (A "0 differences over 1545 names" figure was measured for this scope before the review supplied the class test and before those rows existed; it is superseded here.) - ACCEPTED, THE TWO-WORD SHAPE: "Carod i" keeps given "Carod" with suffix "i", and "Josep Carod i" keeps family "Carod" with suffix "i", because a connective with nothing to its right is connecting nothing. 2.0.0 through 2.3.0 read BOTH of those the same way; 1.4.0 read the two-word one differently, giving family "i" with given "Carod" (measured on the wheels 2026-09-20, correcting an earlier draft of this bullet that claimed 1.4.0 parity for it). That does not move the accepted cost, because the 1.4.0 difference is not this rule's: the two-word shape's routing left 1.4.0's reading in 2.0, which is why the case row carries `fix(suffix-routing)` rather than a connective classification. What this rule decides is that the link does not join there, and 2.0 through 2.3 agree with it. The ledger rule that called this NOT WANTED is rewritten in place rather than deleted — it still explains the name, and the standing is decided now rather than unwanted. - ACCEPTED, A CONNECTIVE RUN: "Henry i of England" reads given "Henry i of England" with a `given-or-family` report, where the parent read given "Henry" with family "i of England". The "i" and the "of" merge as a run and the run joins the whole name into one piece — the same thing "Duke of Edinburgh" already did. - ACCEPTED, THE LOWER-CASE REGNAL NUMERAL, and it is the shape that makes the join FIRE with no report at all on input nobody would call Catalan: a mixed-case name with a lowercase `i` between two name words joins like any other link, so "John Paul i Smith" reads family "Paul i Smith" where the parent read middle "Paul i", and "john i Smith Jones" reads given "john i Smith" where it read middle "i Smith". Nothing marks the reading, because nothing is ambiguous by this rule's lights — the letter IS connective vocabulary and it IS joining two name words — and a report there would fire on every Catalan name too (rules.md#A1's hesitating-reader test). The frequency is the argument for accepting it: measured 2026-09-20 over the deduped corpus-union-cases population of 1,568 non-empty names under the three orders, the join fires on SIX names and five of them are the Catalan rows this bundle added; the sixth is "Henry i of England", the paragraph above. A regnal numeral written lowercase between two name words is the shape that would pay, and the corpus holds one. - ACCEPTED, THE PARTICLE ON THE RIGHT. A particle is a name word for the both-sides test, so the link joins across one: "Josep de Carod i Rovira" reads family "de Carod i Rovira", which is the Catalan reading, while "Josep Carod i de Rovira" reads middle "Carod i de" with family "Rovira" where the parent read middle "Carod i" with family "de Rovira" — the join takes the particle as its right-hand neighbour and the particle chain then follows the joined piece instead of the tail word. Recorded rather than repaired: which side a particle standing between a link and a surname belongs to is P2's question and not this one. - ACCEPTED, AND THE HONEST STATEMENT OF THE ONE-CASE HALF, written out because this branch's own first commit message overclaimed that one-case names keep their reading. `i` is in the MARKED subset, so in a name written wholly in one case it reads as an INITIAL and reports `conjunction-or-initial`, exactly as `e` has since #383/#479. For "JOSEP CAROD I ROVIRA" and "josep carod i rovira" the fields are indeed unchanged and only the report is new. But a one-case name reports wherever a bare `i`/`I` stands among the NAME'S OWN WORDS, which is a great many more names than the Catalan ones — "JOHN I SMITH" and "john i smith" one report each, "JOHN SMITH I" and "john smith i" two, "HENRY I" and "henry i" three. A letter inside a maiden clause is read by the clause's rules and stays silent, exactly as `e` does ("JANE DOE NEE I JONES" reports nothing), which is the own-words scope the 2026-09-13 #383/#479 entry above defines. And in ALL-LOWER names the initial reading MOVES FIELDS wherever the parent read the lower-case letter as the generation. Each such name now reads as its ALL-CAPS twin already did: "rovira, i" gives given "i" where it gave suffix "i" ("ROVIRA, I" already gave given "I"); "john smith i jr" gives middle "smith" with family "i" where it gave family "smith" with suffix "i jr" ("JOHN SMITH I JR" already did); "maier, amy i, jr." gives middle "i" with suffix "jr." where it gave suffix "i, jr." (the corpus name "Maier, Amy I, Jr." reads middle "I" and does not move); and "josep de carod i rovira" gives family "de carod i rovira" where it gave middle "de carod i" with family "rovira". None of those four is a corpus name; all are measured 2026-09-20 on this tree and against the parent's. - NO CHANGE TO THE EXCLUDED BLOCK, recorded as a decision rather than left implicit (Derek, 2026-09-20). It now lists "y" against TWO marked letters rather than one, and that is still right: the argument that put "y" outside is about "y" — the commonest Hispanic compound and this library's oldest fixture — while "i" matches "e" exactly, a bare I initial being as common as a bare E. - -- 2026-09-21 #397 follow-up — AMENDS the "AND THE CONDITION IS ASKED ONCE PER RUN" paragraph above, whose frame figures stand as measured at `6048eb5d`. BOTH SIDES ARE NOW ONE CALL. A connective is placed to join only where a name word stands on EACH side of it, so neither caller ever wanted one side's answer: `_name_word_beside(k, step, ...)` was asked twice and the two answers ANDed at the call site, in the `frozen` loop and again in `_link_joins_inside_the_clause`. It is `_group._between_name_words(k, lo, hi, ...)` now and answers for both — one Python frame per generational connective instead of two, with `step` and its ternary gone. Nothing else moved: the left arm still short-circuits the right, and the arrays, the bounds and the two piece tests are the same tests in the same order. Byte-identical over 3,931,700 parses — 157,188 names (the corpus, cases.py, the property grids and a run-heavy generated set) under six lexicons and four policies, plus the facade surface and the #528 no-parse paths — same sha256 for all twenty-six digest rows and the dumps compare equal. - THE OLD NAME STILL STANDS WHERE IT WAS WRITTEN, and this sentence is the grep trail: `_name_word_beside` is the spelling in decisions.md#M2's 2026-09-20 #397 bullet and in the "NOT FIXED AND RECORDED INSTEAD" paragraph above, and dated entries are not edited, so neither is corrected — a search for either name reaches the whole arc. One place the old name was NOT a citation and is renamed: `tests/v2/test_properties.py`'s per-side mirror, now `_name_word_on_the_side`. That helper is a deliberately independent second implementation, reading spans and off-switch roles where the parser reads pieces and tags, and carrying the implementation's own name made it read as a call into the thing it checks rather than as a model of it. - THE FIGURES, AND THE INTERPRETER, because the pair above spliced two and this one does not. Re-measured 2026-09-21 on py3.11 through `tests/v2/test_benchmark.py`'s own `_frames_for` shape, `b9ed1429` → `9fd84463` → this tree: `Josep Carod i Rovira` 311 → 314 → 313, `Josep Lluis Carod i III` 377 → 381 → 380, `Jane Doe nee Puig i Soler` 315 → 320 → 319, `John Quincy Adams i MA Prof.` 438 → 443 → 442, and the clause guard's own pair 1,125 → 875 → 859 at sixteen links and 6,741 → 2,651 → 2,587 at sixty-four (3.01x, re-pinned in the test). `tools/perf/call_count.py` is unmoved at parse=406.00 facade=443.00 — its reference name carries no link — and so are `John Smith` 172, `Smith, John` 203, `Juan Garcia y Lopez` 291, `John and Jane Smith` 293, `Jane Doe nee Smith` 245 and `Jane Doe nee Smith PhD` 340, none of which reaches the predicate. WHICH OF THE EARLIER FIGURES REPRODUCE, stated exactly where this sentence first said only that one half "very nearly" did: re-measured 2026-09-22 on py3.11, the short-name pairs (304 → 307, 307 → 312) do not reproduce at all, and of the run-of-64 pair (6,741 → 2,652) the LEFT half does and the right does not — `b9ed1429` reads 6,741 on the nose, while `6048eb5d`, the tree its 2,652 was taken on, reads 2,651 here, the same figure `9fd84463` reads. So the splice runs through a single arrow rather than between two of them, which is exactly the spliced table `tools/perf/call_count.py`'s docstring was written about. Timings are unmoved: `"Josep " + "i " * n + "Rovira"` and `"Jane Doe nee Puig " + "i " * n + "Soler"` both read 1.96-2.02x per doubling from n=200 to n=1,600, 11.36ms and 9.39ms at the top end against 11.58ms and 9.58ms at `9fd84463`. - AND THE FOLD BOUGHT A TEST, which is the part worth keeping: while the predicate was called once per side, a mutation of the suffix or the title test hit BOTH sides at once and the left-hand rows killed it, so the right-hand halves were never separately covered — their own rows (`Josep Lluis Carod i Jr.`, `i Mr.`) stand at the END of the name, where `hi` refuses them before either piece test is asked. Folded, the two sides mutate independently and both right-hand tests survived the whole suite. `test_a_credential_or_honorific_mid_name_on_the_right_too` is the pair that kills them, and every arm of the folded predicate now dies by a named test (recorded in its docstring). - -- 2026-09-24 #478 — CROSS-REFERENCE, no change to this rule: case repair's hyphen clause is the one place a marked letter in a name written wholly in one case is read as the connective rather than an initial — `maria silva-e-sousa` repairs to `Maria Silva-e-Sousa` beside the spaced `Maria Silva E Sousa`, and `conjunctions_ambiguous` does not reach the hyphenated form. Decided there, with its cost (`J-E-P DUPONT` → `J-e-P Dupont`): decisions.md#R4's 2026-09-24 #478 bullet. - -- 2026-09-26 #538 — CROSS-REFERENCE: rules.md#P3's separator sentence (a declared delimiter is read past as a connective is) is decided and measured in the 2026-09-26 #538 entry under `### M2`. -- 2026-10-03 #549 — CROSS-REFERENCE: that separator sentence is reversed; it now says a declared separator ends the search as a comma would, a connective never joining across one, and the decision is C1's 2026-10-03 entry. - -- 2026-10-06 #617 — the comma decision joins title words with GROUP'S connective loop, not a copy of it. `_comma.decide` reads the part after the first comma before group joins anything (#613), and the one join the part can hold — a connective binding title words into one title, `John Smith, Mr. and Mrs.` — was made by `_comma._title_chains`, a second copy of the P3 loop in `_group_segment`. The two had already parted three ways: the copy merged no connective run before joining, skipped a single-letter connective unconditionally where group skips it only under the rootname count's carve-out, and derived the title kind alone. Both now call `_pieces.join_connectives` (mechanisms.md#ONE-PREDICATE-PER-QUESTION), which takes the carve-out's answer from its caller (`letter_stays`: group's `total < 4`, always true for the comma decision's own reading, and reaching a LONE single-letter connective only, one beside another connective merging into their run first), a `titles_only` mode for the comma part (a join is made only into a title and takes no prefix kind, any other connective left a word for the reading to place, which is what the copy did), and the `frozen` token set group already passed itself. The merge and the piece predicates it needs moved with it (`merge_pieces`, `joined_tags`, `is_conj_piece`, `is_prefix_piece`), and `_comma._reading_pieces` builds the reading's input, calling the shared loop only where the part holds a connective left free — nearly no part does, and the first cut, calling it always, cost every family-comma parse two frames per word of the part, one `is_conj_piece` call in each of the loop's two passes (`Smith, John` +2, `Smith, John` with sixteen more given words +34, measured inside decide in #617's review); `test_a_comma_part_with_no_free_connective_skips_the_join` holds the gate, with that control recorded. - DECIDED (Derek, 2026-10-06): (1) A CONNECTIVE RUN merges in the comma part as it does in the name — P3's own "connective runs included" — so a title holding `of the` reads whole there too, and a single letter beside another connective joins as a member of their run. `John Smith, Minister of the Interior and Mrs.` reads given 'John', family 'Smith', title 'Minister of the Interior and Mrs.' (83f4e914: family 'John Smith', the copy having joined 'Minister of the' alone and left 'Interior' a name word); `Smith, Chancellor of the Exchequer MP` reads suffix 'MP' (83f4e914: given 'MP'); `John Smith, Mr. y and Mrs.` reads title 'Mr. y and Mrs.' behind given 'John' (83f4e914: family 'John Smith'). The garbage case the issue named, `Prof. Smith, and and Dr.`, reads title 'Prof. and and Dr.', family 'Smith' (83f4e914: title 'and and Dr.', family 'Prof. Smith'). Every one of these reads as 2.3.0 read it, checked against the released wheel, and so did b547c247: what moves on realistic input was #613's copy regressing names it never meant to touch, between releases, so no release note follows. A LONE single-letter connective is unchanged: the comma decision's reading joins no title across one, so a part of titles it separates is no titles-alone part and the listing form reads it, where P3's own count decides the join — `Juan Garcia Lopez, Sr. y Sra.` keeps given 'y' and title 'Sr. Sra.', while `Smith, Sr. y Sra. Juan Maria` joins title 'Sr. y Sra.' behind given 'Juan', both as 2.3.0 and 83f4e914 read them. A case row pins the first. No rules.md text states it, because it is no rule of the comma's: the first review round added a C1 clause saying a lone letter after a comma always stays a word, and the second struck it with the `Juan Maria` reading, P3 deciding in the listing part as everywhere else. (2) EVERY CONNECTIVE OF THE GENERATIONAL CLASS IS FROZEN in the part the comma decision reads, P3's "joins only where a name word stands on each side of it" applied to a part read for credentials, which holds no name word — group freezes the ones it finds without a name word beside them, the same rule. Frozen means no join or run of its own; a neighbour's join may still take one in, as in group (`Smith, Dr. and i` reads title 'Dr. and i'). The default class is `i`, which as one letter the copy never joined and which reaches the freeze only through a run: without it the shared run merge took `Rovira, Dr. i i` into title 'Dr. i i', which `test_a_link_that_joins_nothing_changes_no_field` caught. A generational word of more letters reaches it in a plain join too, which only caller vocabulary can spell, and there the copy DID join it: under `Lexicon.default().add(conjunctions={"og"}, suffix_words={"og"})`, `Smith, Mr. og Mrs.` reads title 'Mr. Mrs.', suffix 'og' (83f4e914 and 2.3.0: title 'Mr. og Mrs.'; b547c247 as now), and as a plain connective `og` still joins. - MEASURED 2026-10-06 against 83f4e914, py3.11, over that tree's population — each branch-added case row and the new rules.md example are fixes against it by construction and not counted here. Two session harnesses, recorded as recipes: each parses its names on both trees and compares the seven role fields, every ambiguity's kind and detail, and `initials()` (the fingerprint adds `capitalized()` and each ambiguity's token spans). The FINGERPRINT (78,282 parses): every corpus name and case row, plus 6,000 seeded draws (`random.seed(7)`) of a front from `'', Smith, John Smith, John Q. Smith, Dr. Smith, de la Vega, SMITH, LLOYD WEBBER, Smith Jr., Vega y Lopez, 김민준 박, 김민준, Dr 김민준` and one to four words behind its comma from the 40 words `Smith John Jane de van vd VD mc do DO MA Ma PhD Ph. D. MD Jr. Jr V V. VI III G.J. XY XYZ X.Y.Z. Dr. Mr. and Prof. Sir i Und RN/CRNA / Jones 김민준 박 J.씨 씨`, three in ten with a third part (each draw in that order: `choice(fronts)`, `randint(1, 4)`, `choice(words)` per word, `random() < .3`, `choice(words)`), plus every two-word pair from that list behind `John Smith, `, `, ` and `Doe, Jane `, under six policies (default, both family-first orders, strict comma suffixes, `/` as a suffix delimiter, caps suffixes off); 30 parses differ, every one a generated name with two connectives side by side after the comma, and no corpus name and no case row moves. The CONNECTIVE GRID (485,300 parses): every three-word string over the 30 words `and of the y e i Und & Mr. Mrs. Dr. Prof. Secretary State Sir PhD MA MD Jr. III V de van der la John Smith Jones J. Ma`, bare and behind `John Smith, `, plus 40,000 seeded draws (`random.seed(617)`) of a front from `'', Smith, John Smith, Dr. Smith, Prof. Smith, de la Vega, Vega y Lopez, SMITH` and one to five of those words, written both as `front, words` and `words front`, three in ten with a third part (drawn in the fingerprint's order, `randint(1, 5)`), under four policies (default, family-first, strict comma suffixes, `&` as a suffix delimiter), and 2,700 parses under the `og` lexicon, every pair of those words in the three templates `John Quincy Smith og {a} {b}`, `John Smith, og {a} {b}` and `Smith, {a} og {b}`; 5,911 differ, 5,590 with two connectives side by side after the comma counting `og` as one, the other 321 the `og` freeze; 4,131 move a role and 1,780 only reports, and in every one of those 1,780 each report the branch adds names a run of two or more connectives, reported once as one run (`John Smith, III and and PhD` reports `'and and' follows a credential` where it reported each `and`). The five differential gates exit 0 and no diff moves but the new example's; that one new rules.md example reads as every 2.x baseline reads it and moves three 1.4.0 rules' corpus reach by one each. Frame cost falls by 10 a parse on every interpreter's reference row, and by 39 more once the /simplify pass gated group's call as the comma's is gated (decisions.md#parse-cost, this date); per name on 3.11, warm, against 83f4e914, `Smith, John`, `John Smith, PhD` and `John Smith, PhD MD MBA RN` cost what they did (161, 206, 294), a long given part less (`Smith, John Quincy Adams Bob` 332 → 297), and a comma part holding a connective pays the shared loop (`John Smith, Mr. and Mrs.` 262 → 269). - AMENDS decisions.md#C1's 2026-10-06 #613 entry, whose decision (3) names `_comma._title_chains` as where the title join is made and describes the join as P3's connective binding title words: it is `_comma._reading_pieces` calling `_pieces.join_connectives` now, and the join merges a connective run first and freezes the generational class, so (3)'s account holds of the join's PRODUCT, one title, and no longer of its reach — rules.md#C1 is amended to match. The `_PREFIXED_SHAPES` `title_chain` row, recorded there as the clock guard on the copy's slice rebuild, guards the shared loop on the same input (`'Mr. and ' * n` behind `John Smith, `), its control re-taken on the moved code (tests/v2/test_benchmark.py records it); the `_SHAPES` `conjunctions` row guards group's calls as before. -- 2026-10-07 #614 — CROSS-REFERENCE: P3's joins are made before S2's trailing run is read, so the run counts a joined part as one name word, as this rule says every count does (`Juan y Garcia X.Y.Z.` keeps family 'X.Y.Z.'), and a salutation keeps its connective (`Mr. and Mrs.` reads title 'Mr. and Mrs.'). #614's first design read before these joins too and broke both; #614's review reversed it (decisions.md#S2, this date). - -- Provenance: the single-letter-connective guard is v1's fix for Google Code issue 11 ("john e smith", 2013, commit 33676c9) — the "#11" citations that circulated pointed at a GitHub accident, not the real source. Recorded so the archaeology stays done. - -Excluded (Lexicon.conjunctions_ambiguous, the marked half of nameparser/config/conjunctions.py — an entry here reads as an INITIAL in a name written wholly in one case): - -- `y` — the commonest Hispanic double-surname link and this library's oldest fixture ("Velasquez y Garcia"); marking it would read the surname link of every lowercase Hispanic record as a middle initial. A caller who wants the Dutch reading adds it (2026-09-13, #383/#479). -- Cyrillic `и`, `і`, `й` — #267 blessed their joining and this change narrows nothing about it; marking one would break the reading #267 adjudicated, in the one-case spellings where nothing else decides (2026-09-13, #383/#479). -- `و` and any other caseless letter — inert rather than harmful, and excluded so nobody adds one believing it does something: the fork requires a token whose `upper()` and `lower()` differ, so a caseless member is never consulted (2026-09-13). -- Multi-letter entries (`and`, `та`, `και`) and `&` — the fork tests a single CHARACTER, so an entry longer than one letter is silently inert. `nameparser/config/conjunctions.py` asserts against it at import (2026-09-13). - -Excluded (Lexicon.conjunctions — nameparser/config/conjunctions.py): - -- `a` — the Māori link in `Te Awanui-a-Rangi` is the one hyphen-interior connective the #478 rule (rules.md#R4) leaves capitalized under force, `Te Awanui-A-Rangi Black`, and it must not be fixed by adding `a` here: a spaced `a` is an initial (`John A Smith`) or a name word elsewhere, and a connective entry would join or lowercase it in every one of those names. The non-reach is recorded under R4 with that name as its witness (2026-09-23, #478). - -Open (Lexicon.conjunctions — contested membership; the issue is canonical): none. The block held one entry, [#397](https://github.com/derek73/python-nameparser/issues/397) — whether Catalan `i` belongs in the conjunctions at all — answered 2026-09-20 by the bullet of that date above: it does, and it joins `conjunctions_ambiguous` in the same change, which was never the contested half. The heading stays with nothing under it on purpose, the same reason an empty ledger roster section is a statement and a missing one is nobody having looked. - -### given-name-titles — deliberately unvalidated - -Declined (rc1 arc; the full argument is AGENTS.md's gotcha): - -- Validating given_name_titles against titles, twice: the whole-entry check rejected legitimate multi-word entries; the per-word check rejected "sir and dame" (a conjunction inside a title run is itself a TITLE token, so the key is matchable while its middle word lives in conjunctions). No static relation over the vocabulary sets decides reachability, and an unreachable entry is inert — each guard cost a working configuration to forbid a condition that costs nothing. - -### phd-merge — the "Ph. D." split - -- 2026-07 (v2 core, PR #288; recorded plan deviation #1 of the core plan) — "Ph. D." tokenizes as two words and is merged back by vocabulary (v1 fix_phd), so the spaced and unspaced spellings read alike. (Scoped by #371 below: alike everywhere EXCEPT at the head of the string, where the merge declines and `Ph. D. Van Johnson` reads title 'Ph.' where `Ph.D. Van Johnson` reads given 'Ph.D.'.) - -- 2026-08-31 #371 (the merge declines at the head) — a suffix never BEGINS a name: position outranks the vocabulary match, and `rules.md#S2` now states it. The merge is what made a leading credential possible at all — every other suffix-shaped word standing first already falls out as a title (H2's abbreviation clause, or TITLES membership: `Jr.`, `MD`, `Esq.`, `Sr.`) or as a name word (`PhD`, `III`), so the `Ph.`/`D.` pair was the only shape that reached the defect. It reached it by emptying the family: `Ph. D. Van Johnson` read given 'Van Johnson' with no surname at all, which is #371's report. - THIS RESTORES 1.4.0 RATHER THAN DEVIATING FROM IT — on three of the four names, and the fourth is named below. v1's fix_phd healed the pair by regex on the raw string, `\s(ph\.?\s+d\.?)`, whose leading `\s` made a head match impossible while it fired everywhere else, mid-name as readily as trailing. (A first draft of this entry said "only where it TRAILED", which is false in both directions and would license narrowing the merge to the trailing position — `John Ph. D. Smith` reads suffix 'Ph. D.' at 1.4.0 and here.) At the head v1 split it — measured on the released wheel, `Ph. D. Van Johnson` gives title 'Ph.', first 'D.', last 'Van Johnson', which is exactly what this change gives. So `expected_since_1.4.0.toml` LOSES its fix(leading-credential) rule — and READ THE DIAGNOSIS, not just the error: the gate reported it SHADOWED, not reverted. `compare.py` separates the two ("shadowed -- an earlier rule claimed every diff it would have" versus "reverted -- matches no diffing name") because the fixes differ. Three of its four names became parity and the fourth, `Ph. D., Jr.`, still diverges under `fix(comma-precomma-family)`, which sits above it. #426 is the precedent for dropping a shadowed rule rather than keeping it. A first draft of this entry generalized EXPLAINED NOTHING into "how a parity restoration announces itself", which would teach the next maintainer to delete a shadowed rule whose behavior is still diverging and the 2.0.0 and 2.1.0 ledgers GAIN one, claiming the same four names, the same five roles and the same digest. What shipped in 2.0 was a regression from applying the merge unconditionally, not a reading anyone chose. - OPENING THE NAME IS NOT OPENING A PIECE LIST, and the first draft got that wrong: a credential run legitimately opens segment 1 after a family comma (`Smith, Ph. D. Jr.`), which C1 reads as a listing rather than a name beginning. Testing `k == 0` alone broke fifteen tests and took the gate red at all three baselines. The caller passes the distinction in, since `_group_segment` cannot see which segment it holds. - ACCEPTED, and found by review rather than by design: a TITLE before the pair keeps the credential reading, because `k` is a piece index and a title is a piece. `Sir Ph. D. Van Johnson` still reads given 'Van Johnson' with an empty family — #371's own symptom, surviving one word to the left. It is 1.4.0 parity, and the rule states it as a boundary rather than deviating: "the first piece of the name" cannot be computed before this merge, since H2's abbreviation test is true of `Ph.` itself, so a scan that stepped over titles would step over the very piece being judged. rules.md#S2 therefore says OPENS THE STRING, which is what the code holds and what v1 held. - THE FIRST DRAFT GOT "THE HEAD" WRONG IN THE OTHER DIRECTION, and the corpus could not see it. Testing the PIECE index alone declined the merge for `"Bob" Ph. D. John Smith`: `extract_delimited` removes a quoted or bracketed clause before grouping, so the pair reaches the merge at `k == 0` with a word standing before it in the INPUT. v1 merges there (its regex needed only a preceding space), and 112 measured names broke parity — every one opening with a quote or a bracket, and not one of them in any corpus. The test is now `a[0] == 0`, the first TOKEN, which is v1's boundary exactly. - ACCEPTED: `Parser.revise(suffix="Ph. D.")` now renders 'Ph., D.'. revise() runs a full sub-parse of the string it is given, and a field value has no head for a head-position rule to consult; a second draft carved that out by requiring a name to displace, and the carve-out made the head reading depend on what FOLLOWED it — appending a maiden clause changed whether 'Ph.' was a title, which `test_a_maiden_clause_changes_nothing_else` caught. Dropping the carve-out removed the inconsistency with it. The merge exists for a credential someone TYPED after a name; a caller who writes the spaced form into the suffix field is taken at their word (Derek's call, 2026-08-31). SUPERSEDED 2026-09-06 by #511, and half of the reasoning here still holds: the MERGE is a head-position rule and still does not fire in a field value. The other half — a caller who writes the spaced form is taken at their word — is what #511 reverses (Derek's call, 2026-09-06, choosing the entry rule over the accepted split when the round-trip limit was measured). What changed is that rules.md#R1's entry pass now runs inside `revise()` over the forced roles, and `Ph.` and `D.` share a comma bucket with nothing between them, so the ENTRY pass joins the pair for the reason it joins `MD` and `PhD` — a different rule reaching the same two words. `Parser.revise(suffix="Ph. D.")` renders 'Ph. D.' (decisions.md#C1, 2026-09-06 #511). - What the leading `Ph.` becomes is H2's business, not this rule's: an abbreviation before a name is almost always a title, which is the same clause that reads `Esq. van Gogh` as title 'Esq.' — so the pair reads title 'Ph.', given 'D.'. The issue proposed given 'Ph.', middle 'D.'; H2 claims the first word before the positional read sees it, and the two answers differ only in which field holds `Ph.`. - -### O5 — the lone name word's convention, and reporting it - -- 2026-09-07 #449 — the convention is REPORTED, as `given-or-family`, with `detail` naming the field it chose. rules.md#O5 has said since #445 that the one-word reading is a guess fixed in advance rather than a determination; what #449 asked is what the library should SAY about it, and the answer is the one A1 gives everywhere else — a caller can only act on doubt that is reported. `detail` names the field rather than the kind naming it, because the field follows the read order and the kind does not: that is the PARTICLE_OR_GIVEN precedent, recorded at that kind's docstring. - What the measurement settled was the POPULATION, not the principle. The naive condition — one name piece placed, nothing else looked at — fires on 143 of the 1123 distinct corpus names, because a title peel, a nickname, a maiden clause, a trailing credential run or a script order can each leave exactly one piece standing where the convention decided nothing. The 2026-08-27 #445 entry below records "19 of 1,085" for a differently written naive condition on a smaller corpus; both numbers are naive and neither is O5's. Guarded on O5's own carve-out list the population is 27 corpus names, 12 contract and 15 radar, and the other 116 report no `given-or-family`, 8 of them reporting `title-or-name` instead (H4's two branches are ahead of O5's at the same line). A first draft wrote 110, which is the subtraction done wrong. Recompute by wrapping `_name_positions` with a recorder that fires at `count == 1`, parsing the deduped `corpus*.jsonl` glob through it, and comparing that set against the names whose parse reports the kind; measured 2026-09-08 on this branch. The guard itself is the code, which is where it should be read. - Two conditions are in the guard that O5's statement had not named, and both are in it now. A script whose own convention settles the order decided the reading, so W4 outranks this the way H1 does — that is what keeps 毛泽东, 高橋みなみ, 씨 and the rest of the script-ordered names out. Asking that question needed a shape change: `_effective_order` used to return the roles triple alone, and a caller comparing that triple against the declared `name_order` cannot tell a script rule DECIDING from a declaration standing unopposed — under a declared family-first order a Han name's W4 entry and the declaration agree. It now returns an `Order(roles, by_script)` and the emitter reads the flag. The second condition is content: a piece with no letter or digit in it is no name word, A2 empties such an input, and a convention report on an empty name would describe a reading nobody got — so `parse("(")` keeps its unbalanced-delimiter report and gains nothing. - Five CJK-bearing names DO report, and none of them is a script rule deciding. Three are a script rule DECLINING: マイケル (W4 gives katakana no order of its own, a wholly-katakana name being read as a transcription), 王·Smith (T3's interpunct suppresses the script_orders lookup whole) and 田中、太郎 (the comma sits inside the token, so the text resolves to no single script). Two are a glued honorific peeled off a Latin stem — Andersonさん and Anderson선생님 — where what is left is Latin and has no script order at all. Recorded because "no CJK name reports" was the expectation going in, and it is wrong in a way that is right: where the script rules decline, what is left IS the one-word convention. - Four conditions were drafted into the guard and none of them shipped, but they were dropped for TWO different reasons, and a first draft of this entry flattened both into "inert". Each was measured the same way: restore that one clause on the O5 branch of a `git archive` copy of this commit, reparse the deduped `corpus*.jsonl` glob, and count which of the 27 lose the report. Two are genuinely INERT, silencing ZERO names (measured 2026-09-08, and again after the 2026-09-08 review round below). `len(state.segments) == 1` is inert OVER THE CORPUS rather than by construction, and a first draft of this line gave the construction reason, which is false: a family comma that names a family does read segment 0 wholly as the family and never positionally, but a comma followed by NO name word hands segment 0 back to the positional read, and `Smith V, Dr.` reaches this very site that way. What silences it is not the clause but the title in segment 1 — after the review round below, which is what tells the read that a title stands there. `not flagged`: a split credential decides the field no more than a whole one does. The general shape is the one this file keeps recording — a guard written from the cases that prompted it carries terms the code already excludes — and the instrument that finds them is mutation, not reading. - The other two were dropped DELIBERATELY, and the same mutation prices each. `not suffix_pieces` silences NINE of the 27 — `Smith Jr.`, `'Smitty' Jones Jr.`, `Carod i`, `Donald mc`, `Jack M.A.`, `John V`, `Mohamad X`, `Andersonさん`, `Anderson선생님` — and `not has_nickname` silences ONE, `'Smitty' Jones Jr.`. Both were dropped on one principle: a suffix or a nickname standing beside the lone name word decides nothing about that word's FIELD, which is the only question this report is about. `Smith Jr.` and `'Smitty' Jones Jr.` are S2's peel taking the suffix and the convention placing what is left, so they report. A first draft justified the nickname clause with "N3 returns before this site", and that is false as written: `_assign.py`'s N3 branch tests `len(pieces) == 1 and len(rest) == 1 and has_nickname`, so it returns only where the segment holds ONE piece, and `'Smitty' Jones Jr.` holds two and reaches this line. What N3 takes is the nickname-plus-one-word shape, not every name a nickname stands in. - Not reported, deliberately: `abdul`, where bound-given vocabulary claimed the word; `de`, where a lone particle's reading is P4's; and `J.`, where the initial's shape is the claim. Those are M4's `_NEVER_FLIPPED` pair plus the particle, read off the tags classify already recorded rather than off a predicate of this emitter's own — literally so since 2026-09-08: `_NEVER_FLIPPED` moved to `_state` beside `WorkToken.tags` (post_rules imports _assign, so _assign cannot reach the other way) and `_WORD_ALREADY_CLAIMED` is that frozenset unioned with `particle`, so the two sets cannot drift. Also not reported: `Sir John`, `'Smitty' Jones` and `abd née Jones`, which O5's example block lists because the convention is what H1, N3 and M4 each left behind on them. The rule's SCOPE and the report's scope differ there, and the difference is stated in the rule rather than left to be discovered: the report is narrowed to an input nothing else stands in, because a report on a name where a title, a nickname or a maiden name is visible in the output tells the caller less than the output already does. - The report is order-independent, which is not the same as the FIELD being order-independent: `Andrew` reports under the default order with `detail` naming given and under FAMILY_FIRST with `detail` naming family. Measured over the whole corpus under the default order, FAMILY_FIRST and FAMILY_FIRST_GIVEN_LAST, the set of names reporting this kind is the same 27 in all three; the only names whose reported kinds move with the order at all are four pre-existing `particle-or-given` reports. - Nothing moves but the report. Every role is identical before and after — measured by parsing all 1123 corpus names under all three orders against a `git archive` of the parent commit, 3369 parses, zero role differences — and the three 2.x ledgers classify the 27 names on `_ambiguities` alone; `_ambiguities` cannot enter a diff below baseline 2.0, so `expected_since_1.4.0.toml` is untouched and its 368 intentional diffs are unchanged. SIX ledger rules rather than one, because an alternation carrying a script-classified member is claimed by the honorific pin in tests/v2/test_ledger_guards.py, which would demand it be a hand copy of GLUED_HONORIFICS: the twenty-two Latin-and-Arabic names ride one alternation, declared in `_NOT_A_VOCABULARY_COPY`, and the five CJK-bearing names take a literal-anchored rule each, which is also where their arguments differ. The three ledger copies are NOT byte-identical, which the drafting expected them to be: in `expected_since_2.0.0.toml` alone the two glued-honorific rules declare `["_ambiguities", "given", "suffix"]`, because at that baseline #308's peel has not shipped and its diff arrives in the same rule as this report, and the gate's over-declared check refuses a rule declaring a role no diff it explains moves. - -- 2026-09-08 #518 review round (the entry above ran into a second day; its own measurements are dated 2026-09-08 under a 2026-09-07 header) — a title standing after a family comma now silences the report. `John V, Dr.` is the shape: a comma with no name word after it hands segment 0 back to the positional read, the leading-title peel counts only what stands in segment 0, so `n` is 0 and the report named `given` for a word H1 then wrote to `family` — the caller was told the convention chose a field the output does not show. `_assign_main` takes a keyword-only `titled`, which the family-comma call fills from the gate's own reading (a False in it is a title, not a suffix), and the field-deciding emitters require `not titled`. Measured on this branch: `John V, Dr.`, `Smith V, Prince` and `Smith V, Dr.` keep every role and lose the `given-or-family` report, keeping the roman numeral's; `John V, Sir` does too, and it is the one worth naming — a given-name title leaves the word in `given`, which is the field the report would have named, so the report and the reading agreeing is what a silenced clause looks like when the rule that decided it happens to agree. Zero corpus names move: no corpus name reaches the site under a comma at all, which is also why the `len(state.segments) == 1` clause above measures inert. - Consolidated the same day, 2026-09-08, and BEHAVIOR IDENTICAL. Segment 1 is now assigned BEFORE segment 0's positional read, so the title standing after the comma already carries a `Role.TITLE` when that read reaches its emitters; the keyword is gone and `n == 0`, `not titled`, the maiden scan and `not resolved.by_script` are one `field_undecided` predicate that both field-deciding emitters read. The token scan subsumes `n == 0` because nothing before assign sets `Role.TITLE` and assign sets it in exactly two places, the leading-title peel and the post-comma reading. Measured against a `git archive` of the previous commit: every distinct name in the deduped `corpus*.jsonl` glob plus the shapes named in this entry and in the #491 entries — 1136 names, 5680 parses — reads the same seven roles and reports the same kinds, details and token texts under the default order, FAMILY_FIRST and FAMILY_FIRST_GIVEN_LAST. One shape constraint came out of the measurement rather than the reading: the two token scans sit UNDER the peel count rather than beside it, because run on every parse they cost 10 calls/name and put the facade 457 over the 456 ceiling of its band (decisions.md#parse-cost). - -### O4 — positional assignment and declared order - -- 2026-08-07 #83 — romanized Chinese order is answered by DECLARATION, not detection: pinyin carries no signal and diaspora makes locale no guide (the issue's own 2019/2021 conclusions); Policy(name_order=FAMILY_FIRST) is the answer the thread wanted. -- 2026-08-07 #146 — Vietnamese needs the THIRD order constant: measured on "Nguyễn Thị Minh Khai", FAMILY_FIRST strands given="Thị" while FAMILY_FIRST_GIVEN_LAST reads given="Khai", middle="Thị Minh", family="Nguyễn". This is why three order constants exist rather than two. - -- Provenance: the three-constants argument originates in #270's own body (2026-07-07): "A boolean family_name_first flag was considered and rejected: Vietnamese order is - [family][middle][given]." The 2026-08-07 #146 measurement below - is the verification; #270 is the origin. - -Declined: - -- Free-form order tuples (('last','middle','first')-style, #270's original draft shape) — the shipped design is exported order constants with tuple rejection at construction (rule D2's - [bad-name-order] example is the pinned message). -- middle_as_family as the way to suppress the middle slot for Vietnamese (2026-08-07 #146) — measured: it merges the middles into the family, giving family="Thị Minh Nguyễn" for "Nguyễn Thị Minh Khai" under family-first-given-last — a plausible-looking wrong answer. There is no policy field that suppresses middle, and name_order rejects a two-role tuple ("name_order must be one of the exported orders"); given_names (given + middle) is the view that stays correct wherever the internal boundary falls. - -### W4 — script-scoped order - -- 2026-07-27 (script-scoped order amendment) — the family-first override is keyed to the SCRIPT of the written name, never to a guessed language: wholly-Han, wholly-Hangul, and kana-licensed Japanese read family-first because zh, ko and ja all write family-first in native script; wholly-katakana names are predominantly transcriptions and keep the declared order. Latin transliterations are never touched. -- 2026-07-29 #272 — the kana license: Han∪kana with at least one kana cannot be Chinese and is not a transcription, so 高橋みなみ reads family-first though it is written in two scripts. -- 2026-10-03 #594 — HALFWIDTH KATAKANA IS KATAKANA. The script table classifies the halfwidth kana block U+FF65–FF9F as `Script.KATAKANA`, so `山田 ﾀﾛｳ` reads family 山田 as `山田 エミ` does; every release through 2.3.0 left the block unclassified and read it given 山田, family ﾀﾛｳ, the positional reading. Legacy data written for JIS X 0201 systems, which had no kanji, spells names this way — bank, payroll and CSV exports. DECIDED by widening the table rather than by folding: nothing rewrites the text before parsing (rules.md's T Background), so normalization at tokenize was never available, and an NFKC fold at classification — the other place the issue offered — reaches far past the kana (`ＪＯＨＮ` to `JOHN`, `㈱` to `(株)`), and it would move a wholly-katakana name across W4's line: a voicing mark with no precomposed form to join (`ｱﾞ`, `ﾝﾞ`) folds to the COMBINING U+3099, which sits in the HIRAGANA block, so the folded token reads as kana-licensed Japanese and `ｱﾞｲ ﾀﾛｳ` would read family-first where the unfolded one keeps the declared order (measured 2026-10-03 by parsing the NFKC spelling `ア゙イ タロウ`, which reads family `ア゙イ`; a two-input sweep of halfwidth names against their NFC(NFKC) spellings differed only where such a mark stood; RECOMPUTE: parse each halfwidth name and `unicodedata.normalize("NFC", unicodedata.normalize("NFKC", name))` and compare the fields). The interpunct flank guard is NOT the hazard, though an earlier wording of this bullet said it was: it reads raw text and asks only whether a character is classified at all, which hiragana and katakana both are. The same measurement shows a pre-existing gap the fold would have widened rather than created: full-width katakana typed with an uncomposable combining U+3099 already takes the kana license today. A block range is how the table already decides (U+30FB and the voicing marks are in by block), and it takes the voicing marks U+FF9E/U+FF9F in because they are spacing characters following their base. The span starts at U+FF65, the halfwidth nakaguro, which tokenize already treats as a separator (rules.md#T2), and stops short of U+FF61–FF64, whose U+FF61 is a full stop. Everything keyed on the table follows: the kana license, the second-East-Asian-word count that keeps a segmenter from being asked (`高橋一郎 ﾀﾛｳ`), the 间隔号's flank guard (`ﾀﾛｳ·ﾔﾏﾀﾞ` divides), and `_NO_INITIALS`, which closes the limit #cjk-full-stops recorded (`ﾀﾅｶ. John` read title `ﾀﾅｶ.`). THE RATIONALE FOR WHOLLY-KATAKANA NAMES IS AMENDED, NOT THE BEHAVIOR: a wholly-halfwidth name keeps the declared order exactly as a wholly-katakana one does (`ﾔﾏﾀﾞ ﾀﾛｳ` reads given ﾔﾏﾀﾞ), but the 2026-07-27 reason — that such names are predominantly transcriptions — is weaker here, since a system with no kanji wrote Japanese names in katakana too, family-first. What stands is the narrower claim: the script cannot tell a Japanese reading from a transcription, so it settles no order, and a caller who knows their katakana is Japanese adds `(Script.KATAKANA, FAMILY_FIRST)` to `script_orders` (docs/usage.rst's Boundaries section carries the recipe). `name_order=FAMILY_FIRST` works too, at the cost of reversing Latin names as well. MEASURED 2026-10-03: the differential gate saw nothing before this change because no corpus line held halfwidth kana; the eight case rows #594 added are its whole population, five of them contract movers (`山田 ﾀﾛｳ`, `山田 ﾀﾞｲｽｹ`, `山田 ﾍﾟｰﾀｰ`, the unspaced `山田ﾀﾛｳ`, `ﾀﾛｳ·ﾔﾏﾀﾞ`), two tolerated radar movers (`ﾀﾅｶ. John`, `ﾏｲｹﾙ.`) and one parity row (`ﾔﾏﾀﾞ ﾀﾛｳ`). RECOMPUTE: parse the eight on this tree and on 2.3.0 from PyPI and read `title` beside the name fields. -- 2026-10-03 #596 — A VOICING MARK TAKES THE SCRIPT OF THE KANA BEFORE IT. The four kana voicing marks U+3099–309C are in the hiragana BLOCK, which is how the script table decides, but none belongs to either syllabary: a mark voices the kana it follows. NFC composes one only where a precomposed voiced kana exists (カ + U+3099 is ガ), so `ア゙イ`, `ン゙` and every spacing ゛/゜ kept a hiragana-block character inside a katakana word, the word took the kana license, and a wholly-katakana name read family-first — the gap the #594 entry above measured, present since 2.1.0. DECIDED by deleting, from the classification copy only, every mark standing after a KANA, so the word reads as its base's script; token text is untouched. A mark after anything else — hangul, Han, Latin, or nothing — voices no kana and keeps the table's answer, as before. That scope is a review finding, not a nicety: the first cut deleted every mark with ANY character before it, which handed `김゙민준` the hangul script, and the surname division, which cuts the raw text, then split the mark off its base — given 김, family `゙민준`, unreported, where 2.3.0 left the word whole and reported given-or-family; `parser_for(locales.ZH)` cut `王゙小明` the same way. A case row watches `김゙민준` — marked tolerated, the reading of a mark after a non-kana being unpromised (rules.md's W Background) — so the radar tier, not the contract, sees it move. Declined: taking the marks out of the hiragana range instead. Measured 2026-10-03 on patched copies of this tree, it fixes the katakana names as well but leaves a word holding a mark in NO script, so `あ゙い たろう` loses family-first, `山田 ア゙イ` loses the kana license and reads family `ア゙イ`, and the unspaced `山田ア゙イ` reads given; the two parity case rows #596 added pin those readings. Declined too: narrowing to the two combining marks, which leaves `ア゛イ タロウ` family-first. NFD-typed text was the issue's worry and is not affected by either option, since classification composes first: NFD `やまだ がくと`, `ジョン ガイ` and `김 민준` parse identically under all of them. A mark with nothing before it keeps the table's answer, so `゙アイ タロウ` still reads family-first; that is degenerate input, recorded in rules.md's W Background as unpromised rather than given a branch. ONE ARGUMENT IN THE #594 ENTRY ABOVE NO LONGER HOLDS: an NFKC fold would no longer move a wholly-katakana name across W4's line, since the combining mark it produces (`ｱﾞ` to `ア゙`) now reads as the katakana it follows — #594's own RECOMPUTE, run 2026-10-03 on this tree over `ｱﾞｲ ﾀﾛｳ`, `ﾝﾞ ﾀﾛｳ`, `山田 ｱﾞｲ`, `ﾏｲｹﾙ ｱﾞｲ`, `ｱﾟｲ ﾀﾛｳ`, `山田 ﾀﾛｳ` and `ﾔﾏﾀﾞ ﾀﾛｳ` under all three orders, finds no role differing. The range decision stands on its other ground, the fold's reach past the kana (`ＪＯＨＮ`, `㈱`). The interpunct clause of rules.md#W4 was corrected in the same change: a name the 间隔号 divides keeps the DECLARED order, not its source order — under `name_order=FAMILY_FIRST` `威廉·莎士比亚` reads family 威廉 — and an example pins it. MEASURED 2026-10-03: no corpus line held an uncomposable mark before this change, so the case rows #596 added are the whole population. At 2.1.0, 2.2.0 and 2.3.0 the three `fix(#596)` rows move; at 2.0.0, 2.1.0 and 2.2.0 `김゙민준` moves in its report alone, #449's given-or-family being new in 2.3, and is parity at 1.4.0, which has no report. At 1.4.0 and 2.0.0, which had no kana license, the three movers are parity, while `あ゙い たろう` and `山田 ア゙イ` move as every native-script CJK name does there, under the existing `fix(#271/#272/#298)` rule. RECOMPUTE: parse `ア゙イ タロウ`, `ア゛イ タロウ`, `マイケル ア゙イ`, `あ゙い たろう`, `山田 ア゙イ` and `김゙민준` here and on each baseline from PyPI. - -### D1 — the segmenterless-activation warning - -- 2026-08 #337 — WARN rather than raise, deliberately: registering the JA pack without the extra must stay safe (the inert registration is itself a pinned property), and a warning is filterable by the caller who wants exactly that inertness. The Japanese install hint is conditional — it appears only when a Japanese script is among the dead scripts; a hangul-only gap gets the generic remedies. -- 2026-08 #337 — the warning exists because parser_for(locales.JA) without segmenter= used to build a parser that silently behaved like a working one minus the feature; it re-emits from the parser_for frame so the reported location is the caller's own call. -- 2026-08-07 #339 — diagnostics that hand the reader code must hand code that type-checks: the message's offered deactivation used a bare tuple literal for segment_scripts — an arg-type error under mypy in a py.typed package — and became frozenset(), pinned by a test asserting the offered spelling. The known-bad spelling is in the denylist test. The structural fact behind the recurrence: autodoc renders docstrings into the API reference, so docs/*.rst is NOT the boundary of "the docs" — a guide and the reference can teach different spellings on the same rendered page, invisibly to any .rst-only sweep. - -### D2 — construction raises, parse never does - -- 2026-07 (v2 core, PR #288) — every raise in the locale-apply path is a plain TypeError/ValueError so the wrap-with-locale-code rewrap cannot break on exotic exception signatures. - -Declined (ambiguity kinds for script-resolved names, 2026-07-27): - -- A "han-script" zh-vs-ja kind — applying the ZH pack IS the disambiguation; per-name flags after an explicit opt-in are noise. -- ORDER emission for script-resolved names — native-script order is convention, not a guess; the reserved kind stays unemitted. SEGMENTATION fires only on a genuine multi-split vocabulary fork (夏侯惇, 남궁민수). - -### H1 — a title and one name word - -- 2026-08-25 (#410) — H1's "and nothing else" counted a suffix, a nickname and a maiden name as further name words, so a title-plus-surname name reported no family the moment any of them stood beside it: `Dr. Smith` reads family 'Smith' and `Dr. Smith née Jones` read given 'Smith' with the family empty. #410 reports the maiden flavor; measured, all three roles suppress the rule identically and the fix is one term in one guard, so the term went rather than the maiden role being special-cased. What decided the width is H1's own rationale — a title addresses by surname — which says nothing about what stands BESIDE the name. The rationale is stated for H1 rather than as a doc-wide principle about what a suffix is, deliberately: N3 counts a suffix the other way (`'Smitty' Jones Jr.` reads given 'Jones', family ''), and which of the two readings is right for a nickname-led name is not decided here. #399 is what made this urgent rather than latent: stopping the particle chain at the marker routed the canonical title-and-particle shape (`Freiherr von Richthofen geb. Albrecht`) into H1 for the first time. N3 moves with it, without N3 changing: `'Smitty' Dr. Jones` declines N3's one-piece count as it always did, and H1 now names the family behind it, so it reads family 'Jones' where it read given 'Jones' through 2.1. Five corpus names change reading at every baseline, of which four arrive as new diffs and are classified with the fix. The fifth, `Freiherr von Richthofen V`, was already classified under the `fix(#424)` rule: #410 narrows its diff from {given, family, suffix} to {family, suffix}, and `classify()` takes the first rule whose declared `fields` are a SUPERSET of the observed diff, so a shrinking diff kept matching and no run ever named it. A real movement behind a green gate, and the second of that kind here: the fields-only `fix(suffix-routing)` catch-all was absorbing three names at 1.4.0 the same way, its heading growing with nothing to announce it. Both are one shape — a rule broader than the diff it explains — and that is the lesson worth keeping rather than either fix. It is also, being a title, one name word and a suffix, the suffix flavor of this very shape, which two ledger comments had claimed no corpus carries. At 1.4.0 three of the four were being absorbed by the fields-only `fix(suffix-routing)` rule (its heading went 14 -> 17) — invisible to `_CORPUS_CLAIMS`, which records the whole corpus for a rule with no `name_regex`, so the guard could not see them arrive. They have their own rule now, as #372's names got one off the same catch-all. The v1 suite shipped the nickname reading as a strict xfail (`tests/test_nicknames.py::test_nickname_and_last_name_with_title`), which passes now. Known gap between the statement and the guard: H1 says "exactly one name word" and "that word", but the guard never counts units. It tests which ROLES are unoccupied — no middle, no family — and then retags every given token, so a name whose given run is two name WORDS (units, not tokens) reaches it and moves both: `Freiherr de V Jr` reads family 'de V'. The distinction matters for reading the claim that follows: a multi-token given run is ordinary — the rules.md example `Freiherr von Richthofen V` has one, its two tokens being a single particle unit — and it is a run of two UNITS that no real input produces. A 58,338-name sweep found no corpus name and no rules.md example with one — but it is recorded because decisions.md elsewhere leans on H1's "exactly one name word" as a settled scope when narrowing P5's licence, and that reading of H1 is stronger than the code behind it. Not fixed here: the widening this entry records is about which roles disqualify the rule, not about how it counts. - -- 2026-09-08 [#489](https://github.com/derek73/python-nameparser/issues/489) (the #316/#489 bundle) — a title RUN addresses as its LAST title does. `Her Majesty Queen Elizabeth` read family `Elizabeth` because the `given_name_titles` lookup was keyed on the whole run, and `her majesty queen` is no shipped entry. A run is written as several titles and is addressed the way its final one is, so the run's folded key is now matched whole OR by its last word. **This AMENDS the 2026-08-22 #369 entry under #P5**, which keyed both sites on the whole run so they could never disagree; the amendment keys both on the LAST word and the invariant is untouched — H1's retag and P5's licence still cannot read one run two ways, through one predicate beside `_title_key`. What the amendment reopens is that entry's EXAMPLE and not its reasoning: `'mr sir'` was #369's own instance of a run neither site treats as a given-name title, and it now IS one to both, `sir` being the last word. - The whole-run arm is kept for a caller's multi-word phrase entry — `lt col` is stored and matched as one key — and its order carries nothing: the `or` short-circuits but decides nothing, since for a ONE-WORD run the key IS its last word. Every shipped given-name title is a single word, asserted in tests/v2/test_lexicon.py rather than counted here, so under the default vocabulary that arm can only ever match a one-word run, which the last-word arm reads the same way. Recompute the single-word property with `L = Parser().lexicon; sorted(t for t in L.given_name_titles if " " in t)`, which gives `[]` on 2026-09-08; the set holds 47 entries that day, and the argument survives the digit moving. - Measured over the corpus glob as it stood at the fix: ONE name moves, `Her Majesty Queen Elizabeth`. TWO after the docs commit that follows, which puts `Dr. Sir John` in corpus_rules.jsonl as an H1 example — re-measured there with the predicate's last-word arm dropped to a whole-run key, so the diff read forwards is this amendment's. Probes that move with it: `Her Majesty Queen Elizabeth II`, `Reverend Mother Teresa` (`mother` is a given-name title), `Dr. Sir John`, `Mr Sir John`, `Xyz. Sir John` — H2's unlisted word in the run does not change the run's last word — and, through P5's licence, `Sir Sheikh abdul rahman`, which reads given `abdul rahman` with an EMPTY FAMILY, the Accepted outcome H1 already records for `Sir John`. - VOCABULARY SCOPE, recorded so the boundary is not read as a gap this bundle left. `prince`, `princess`, `lord` and `lady` are not given-name titles, so `His Excellency Lord Duncan` still reads family `Duncan` and `Her Royal Highness Princess Anne` family `Anne`; `king`, `queen` and `mother` are. Whether the first four should join is a vocabulary question with its own frequency argument — it would move `Prince Harry` and `Lady Gaga` — and is deliberately out of this bundle, filed as [#519](https://github.com/derek73/python-nameparser/issues/519). Recompute the membership with `sorted(Parser().lexicon.given_name_titles)`. - Frame delta measured at zero on both entry points: each site's read sits inside the branch it serves, and the reference name `Dr. Juan0000 de la Vega III` enters neither — H1's guard wants an unoccupied family, P5's a bound given-name word. - TRAILING REACH, added 2026-09-09 in review of the docs commit. H1's site asks which ROLES are unoccupied and never where the title stands, so a run that #H5 chained in BEHIND the one name word decides that word's field exactly as a run in front of it does. Measured on this tree: `Smith Sir.` reads given `Smith` with an empty family and `Smith Queen.` reads given `Smith`, both runs ending in a given-name title, while `Smith Dr.` reads family `Smith`. No code moved for this — the reading has been the shipped one since the trailing walk landed — but the STATEMENT said "a title followed by exactly one name word" and described only the front slot, so what the review changed is the statement, an example line (`"Smith Sir." → given="Smith"`, which puts the name in corpus_rules.jsonl), and the `interacts:` lines, H1 gaining H3 and H5 and H5 gaining H1. The trailing half is where H1 and H5 meet: H5 decides which words leave the name, H1 decides the field of the one left standing. - -- 2026-09-09 (the #316/#489 bundle, review round 2) — WHICH run addresses, where a name carries a title run at BOTH ends. The TRAILING REACH paragraph above is right about the trailing slot and was incomplete about the front one: H1 collected every `Role.TITLE` token into ONE run, and with #316(a) shipped that run can span the whole name. `Sir John Prof.` keyed `sir prof`, whose last word `prof` is no given-name title, so it read title `Sir Prof.`, family `John` where `Sir John` reads given `John` — the title written behind the name changing what the title in front of it addressed by. `Queen Elizabeth Prof.` lost its given name the same way, and `Dr. Smith Sir.` moved in the other direction, the trailing `sir` making the leading `dr` address by given name and turning family `Smith` into given `Smith`. - THE RULE: the run standing BEFORE the one name word is the run that addresses; a run standing behind it decides only when none stands before. The split is at the NAME WORD, and the first fix split at the first token of ANOTHER ROLE instead, which a nickname written in front of the titles satisfies: `'Smitty' Sir Jones Prof.` then found no leading run and read family `Jones` where `'Smitty' Sir Jones` reads given, and `Dr. 'Smitty' Sir John` lost the run it had — the same defect one nickname away, caught by the design review's two-input invariant. H1's own rationale is what decides it, saying in as many words that what stands beside the name word does not decide this reading; so a run written around a nickname is ONE run and `Dr. 'Smitty' Sir John` reads given `John` as `Dr. Sir John` does. That is #H5's transparency principle applied to the one-word name — `X Prof.` is `X` plus a title, so adding the title cannot change how the words in front of it are read — and it leaves the trailing slot's own reading untouched, `Smith Sir.` still reading given `Smith` because no run stands in front of it. rules.md#H1's statement carries the ordering and gains two examples, `"Sir John Prof." → given="John"` and `"Dr. Smith Sir." → family="Smith"`; #H5's one-name-word clause is conditioned the same way. - FOUND BY THE PR REVIEW'S P5 FINDING, and that is the part worth keeping. The reserve fix under #P5 was correct on its own terms and left `Sir abdul rahman Prof.` reading family `abdul rahman`: the join fired for the right reason and the field was still wrong. What the review then asked was which run each site reads, and the answer was that they read DIFFERENT runs — group's licence has always read only the pieces ahead of the bound word (`range(fk)`), while H1 read every title token — so the 2026-08-22 #369 invariant under #P5, that H1 and P5 cannot read one run two ways, had been broken by the composite keying without any site's own tests noticing. Both now read the leading run, through the same predicate, and `nameparser/_pipeline/_group.py` records that beside the licence. - MEASURED 2026-09-09, over every distinct name in `tools/differential/corpus*.jsonl`, parsed on the fix's parent (`e489dc1`) and on the fix and diffed across the seven fields and the ambiguity kinds: over the corpus AS THE PARENT HAD IT, not one name moves; over the corpus as this commit leaves it, exactly TWO do and they are the two rules.md examples this entry adds (`Sir John Prof.`, `Dr. Smith Sir.`). The composite shape needs a title run at BOTH ends and no corpus name had one before those two, which is why a defect this plain sat behind four green gates. `Dr. King Sir.` is untouched and stays #H4's: its leading run is `dr king`, whose last word `king` IS a given-name title, so the word the peel left standing reads given `Sir.` with `title-or-name` reported. The end-to-end invariant is `tests/v2/test_parser.py::test_the_p5_licence_and_h1_read_a_title_run_the_same_way`, whose trailing-title half now requires `f"{title} abdul rahman Prof."` and `f"{title} John Prof."` to give the same given and family as the spellings without the trailing title; restoring the whole-`titles` key fails six of its eleven rows, which is the mutation that proves it discriminates. It carries the nickname spellings as well, and those are the only rows that separate the two splits above. Frame delta zero on both entry points: the run is built inside H1's guard, after the role counts have short-circuited, and the reference name `Dr. Juan0000 de la Vega III` has a family and never enters it. - -- 2026-09-10 [#519](https://github.com/derek73/python-nameparser/issues/519) — `prince` and `princess` join the given-name titles; `lord` and `lady` do not. This RESOLVES the VOCABULARY SCOPE paragraph of the 2026-09-08 #489 bullet above, which is left as written: it named the four as not given-name titles and filed the question, and two of the four have now moved. The criterion is rules.md H Background's — membership follows how the title addresses — and `prince` and `princess` address by the given name (`Prince Harry`, `Princess Anne`), with no surname reading of the word behind it. The clause is for the two words and not for a royal class: TITLES holds `emperor`, `tsar`, `sultan`, `pharaoh`, `archduke`, `maharani` and their pairs, none of them weighed here, and `Emperor Akihito` still reads family `Akihito`. `lord` and `lady` split the same way as each other, by the bearer's rank, which the text does not carry: the given-name form is a courtesy style for children of the senior ranks alone — younger sons of dukes and marquesses (`Lord Peter`), daughters of dukes, marquesses and earls (`Lady Diana`) — and every peer and every wife takes the title or surname (`Lord Byron`, `Lady Thatcher`). [Debrett's "The trouble with titles"](https://debretts.com/the-trouble-with-titles/) and [Bratman's peerage primer](https://www.dbratman.net/nobility.html), both read 2026-09-10, agree on that boundary, Debrett's naming `Lady Jane Debrett` for a baronet's wife as the typical error. The issue gave `lord` the surname reading outright, hedged with a "mostly"; the hedge is this class. The set has no way to say "sometimes" — the `venerable` reasoning under #indic-honorifics — so Derek's call was to leave both out, decided rather than deferred, and the issue's proposed frequency test on `lady` was not run. Both are pinned as negative controls (`Lord Byron`, `Lady Gaga`) beside `His Excellency Lord Duncan`. - Both read sites move, through the one predicate: H1's fold reads `Prince Harry` given `Harry` and `Her Royal Highness Princess Anne` given `Anne` (the #489 bundle's negative control, flipped), and P5's licence reads `Prince abdul Rahman` given `abdul Rahman` with an empty family, as `Sir abdul Rahman` does. What membership does NOT decide is whether the word is a title at all: `Prince Fielder` reads given `Fielder` now and read family `Fielder` before, wrong both ways, and that is [#348](https://github.com/derek73/python-nameparser/issues/348)'s collision. - Measured 2026-09-10 over every name in `tools/differential/corpus*.jsonl` (1308 rows), parsed on this tree and on the same tree with a Lexicon whose `given_name_titles` lacks the two words, diffed across the seven fields and the ambiguity kinds: exactly TWO names move, `Prince Charles` and `Prince Fielder`, both radar-tier in corpus_issues.jsonl, both `{family, given}`. THREE after the docs commit, which puts `Her Royal Highness Princess Anne` in corpus_rules.jsonl as an H1 example. `Prince of Wales Jr` does not move — the joined run addresses by `wales`. Recompute by parsing the corpus glob twice with the two Lexicons and diffing. TITLES membership is unchanged, TITLES being the union. - -### H2 — the leading-abbreviation title - -- 2026-06-30 (leading-period-title design; v2 core, PR #288) — the shape test is v1 parity (period_abbreviation): two-plus letters then a period, bare initials exempt. Its site is the head of the part CARRYING THE GIVEN NAME — the whole name, or the post-comma part under a family comma — not "the head of the name"; that scope correction is PR #315 (2026-08-01, docs-only), verified against 1.4.0 from PyPI, so the parity claim is real and the narrower description never was. The extraction litmus (2026-08-15): the spec drafted this rule as - "recognized by vocabulary, not by written shape" and the live - parser falsified that framing — the shape heuristic is real, and what the abugida gap (#343/#344) shows is its LIMIT, not its absence. Recorded as the rule's Accepted consequence — and the 2026-08-15 landing initially re-narrowed the scope to "name- opening", correcting one error while preserving another; the eighth review round fixed it. - -Decided 2026-09-08 (was Open: [#316](https://github.com/derek73/python-nameparser/issues/316), what a trailing title-vocabulary word should do while the comma paths disagreed): the trailing slot reads VOCABULARY, not shape (#H5), and the leading inference here stays unconditional — #316's open question 2, a symmetric leading rule that would read "Esq. Smith" as a suffix, is DECLINED. #109 shipped the leading inference on purpose, the family comma already carries the credential case ("Smith, Esq." → suffix, this rule's own Accepted clause), and reversing S2's "a suffix never opens the string" for period-marked words for the sake of one word is the wrong trade. The two comma paths agree now: "John Smith Prof." and "Smith, Prof." both read title Prof. See #H5. - -- 2026-09-10 — the shape declines a word carrying a script with no initials (#323); the argument, the halfwidth-kana limit and the measurements are in #cjk-full-stops. That limit was closed on 2026-10-03 by #594, which classifies halfwidth katakana (decisions.md#W4). - -### H3 — the title run's floor - -- 2026-09-08 (the #316/#489 bundle) — the leading title run leaves a name word standing, and a post-nominal is not one. Derek's question is what opened it, verbatim: "Why does `Dr King Jr` need to parse `king` differently than `Dr King`? Jr is a recognized suffix, so it could not count as a following name." The mechanism behind the old reading is an ORDER: the leading peel runs before the trailing suffix peel and its only floor was "a title needs a following piece", so it took `Dr King` whole and left `Jr` to be the name — title `Dr King`, family `Jr`, no suffix at all. The floor added here is a second one, asked of the run's last word: where everything behind the run is a suffix piece and that word is not itself suffix vocabulary, the run gives it back. `Dr King Jr` now reads title `Dr`, family `King`, suffix `Jr` — what v1 wanted (#v1-xfail-triage), what rules.md#S2's descriptive note predicted, and what the comma spelling `King, Dr Jr` has always given. -- Measured over the differential corpus glob as it stood at the fix — 1123 distinct names, 1263 rows, before this bundle's own rules.md examples entered corpus_rules.jsonl and made it 1136 and 1284 — tree against the same tree with the floor removed: exactly one name changes, `Dr King Jr`. FOUR after the docs commit, which puts `Dr. King MD`, `Dr Jr` and `Sir Jr` in corpus_rules.jsonl as examples of the floor and its edge (THREE from 2026-09-09, the /simplify round having taken `Sir Jr` out of rules.md — see #H5 — where `Dr Jr` pins the same reading) — re-measured there, and `Prince of Wales Jr` enters with them and does NOT move, which is the one-word gate witnessed in the corpus. The case row `Dr. King MD` moves the same way. Recompute by parsing every name in `tools/differential/corpus*.jsonl` twice, once on the tree and once with `_pieces.leading_titles` monkeypatched back to the bare "a title needs a following piece" loop, and diffing the seven name fields plus `ambiguities`; 1.4.0 gives the degenerate reading for both names, so the change diffs at every baseline. -- The word given back must be a NAME CANDIDATE, and that is what leaves the degenerate inputs alone. All-suffix inputs are untouched because the run's last word is itself suffix vocabulary — `MD DDS` keeps title `MD`, family `DDS`, and `Jr. Ph. D.` keeps title `Jr.`, suffix `Ph. D.` — and all-title inputs are untouched because there is no rest for the floor to read: `Dr.`, `Marquess of Bath` and `Coach` are unchanged. -- ONE WORD is the gate, and it is a Task 1 review outcome rather than the drafting's shape. A joined title unit given back would lose the title entirely, which is worse than the reading it replaces: `Prince of Wales Jr` keeps title `Prince of Wales`, family `Jr` rather than becoming given `Prince of Wales` with no title at all. `Lord Chancellor Jr`, whose run is two separate words, does move — title `Lord`, family `Chancellor`, suffix `Jr`. -- The edge accepted: where the run's whole content is the word given back, no title is left to make the reading H1's, so the word stands as the NAME rather than as the family. `Dr Jr` reads given `Dr`, suffix `Jr` and `Sir Jr` given `Sir`, suffix `Jr`, both reporting `title-or-name` by H4's lone-title-word convention (2026-09-09: `Sir Jr` is no longer written down anywhere but this sentence — once the floor empties the run no branch reads `vocab:given-title`, so the two readings are one and `Dr Jr` carries it). That is the residue rules.md#S2's note is now scoped to. -- Two residuals the floor does not reach, recorded because they look like misses and are not. A run whose LAST word is itself suffix vocabulary is not given back, so `Dr King MD PhD` still reads title `Dr King MD`, family `PhD` — the floor asks about one word, not about the run's contents. And the floor asks `is_suffix_piece`, which vetoes a bare initial-shaped numeral, so `Dr King V` keeps the whole run as the title and reads given `V` (`king` being a given-name title and the run's last word, which is #H1's 2026-09-08 amendment), where `Dr Smith V` reads family `Smith`, suffix `V`. That numeral fork is #401/#421's territory and is deliberately outside this floor. -- The floor lands in `leading_titles` rather than in assign because that predicate is the ONE answer to where the leading run ends (mechanisms.md#ONE-PREDICATE-PER-QUESTION): assign sets the roles from it and group's chain guard reads the same count, so a floor in assign would have given the two sites different runs. One cost measured and one shape forced by it: the branch's entry gate is two inline tag reads rather than `is_suffix_piece`, because `leading_titles` runs four times per parse and asking the authoritative predicate first cost 8 calls per parse of the reference name against a band with room for two (#parse-cost). The tag reads are the cheapest NECESSARY condition for the next piece to be a suffix piece at all, so an ordinary titled name leaves the branch without entering a frame; `is_suffix_piece` stays the predicate that ANSWERS. Frame delta measured at zero on both entry points. - -### H4 — an input that is nothing but vocabulary still has to name somebody - -- 2026-09-07 #491 — the reading is unchanged and the silence is what ends. Handed a string the title peel eats down to one last word which is itself title vocabulary, the parser reads that word as the name; decisions.md#v1-xfail-triage recorded it in the fourth of its NOT FIXED entries — "this is a name parser, not a title parser" — and said in the same breath that what is actually wrong is the guess being silent. It now reports `title-or-name`, with `detail` naming the word that was made into the name. - Six corpus names gain it when this is written: the Queen's Bench string, `Lord Chancellor`, `Dr. King`, `The Rt Hon`, `His Holiness` and `His Holiness the Dalai Lama`. Three read contract-tier and three radar, and the tier split is an artifact of the documentation rather than of the shape — the three contract ones are contract because this bundle made them rules.md examples, which puts them in corpus_rules.jsonl. (2026-09-08, the #316/#489 bundle: the peel half's population was TEN, and the count moved for two different reasons. `Dr King Jr` and `Dr. King MD` gained the report because the title run's floor now leaves a name word standing where the run used to swallow it (#H3), and `Dr Jr` gained it as the same floor's residue; those three also ENTERED the corpus in that commit, as rules.md examples of the floor. `Sir Jr` gained the report too and reads exactly as `Dr Jr` does, but it is NO corpus member and is counted nowhere here: it entered corpus_rules.jsonl in the same commit as a rules.md example and left again on 2026-09-09, when the /simplify round dropped the example as a duplicate of `Dr Jr`'s (#H5). So the peel half is NINE from that round, six contract and three radar. Recompute rather than trusting any of these numbers: parse every name in the `tools/differential/corpus*.jsonl` glob and collect the ones whose `ambiguities` carry the kind, keeping the tier the FIRST file holding the name declares with the contract files sorted first, as `main()` loads them — the README's table is where the tiers are, and sorting the glob plainly instead reads `Dr. King`, `Dr King Jr` and the Queen's Bench string as radar and gives three and six.) `Dr. King` is the one worth arguing about and it is deliberate: `king` is in the titles vocabulary for the addressing forms, which the triage entry above decided and did not reopen, so `Dr. King` IS an input whose last standing word is title vocabulary and the rule claims it. Reporting there is honest rather than noisy — the reading came from a convention, not from anything the input says — and a caller who wants only the exotic cases has `detail` to filter on. - ONE emitter, and it is at assign's lone-name-word site rather than at H1's retag, which is where the drafting put it. H1 is not the site: under a declared family-first order the assignment places the word in the family directly and H1 never runs, so an emitter there would report under one order and not the other for a reading that is the same either way. Measured under the default order, FAMILY_FIRST and FAMILY_FIRST_GIVEN_LAST, the six report identically. That placement is also why the `detail` names no field, unlike O5's: under the default order H1 retags the word after assign, so a field named at the emitter would be the one the word was placed in and not the one it ends in — and the fork the kind reports is title-versus-name, which no field answers either way. - What is silent, all measured 2026-09-08. A lone title word: `Dr.`, `Sir`, `King` and the chained `Prince of Wales` are a title run with nothing behind it, the peel takes the whole string, no word is left standing to be read as a name, and nothing was chosen — mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE's "a branch that runs but changes nothing is not a decision". A title with an ordinary word behind it: `Dr. Smith` and `King Charles` leave a word standing, but not a title-vocabulary one, so H1 alone explains them. And a title followed by post-nominal vocabulary: `Dr King Jr` and `Dr. King MD` peel `Dr King` and `Dr. King` WHOLE, leaving a credential that the bare-suffix carve-out makes the name — a different convention, and its report is scoped to inputs no title stands in (spelled `n == 0` when this was written, one clause of the `field_undecided` predicate since the 2026-09-08 consolidation recorded under O5), so a title in front of the run takes the input out of it and leaves the reading H1's. **Corrected 2026-09-08 (the #316/#489 bundle):** the two example names no longer read that way and the sentence is wrong about them, though the SCOPE it describes is unchanged and still true of the suffix half. The title run's floor gives back a one-word last piece where everything behind the run is post-nominal (#H3), so `Dr King Jr` reads title `Dr`, family `King`, suffix `Jr` and `Dr. King MD` title `Dr.`, family `King`, suffix `MD`. Both now leave a NAME word standing, and both report through this rule's TITLE half rather than being silent — `king` being title vocabulary is what makes them the peel shape. What is still true, and is what the sentence was written to say, is that the SUFFIX half is reached only where no title was peeled first: `MD DDS` has a title peeled and reports nothing, `DDS` being no title. rules.md#H4's Accepted clause is rewritten to match, and `Dr. King MD` is an example line there now. The peeled titles are never tested for anything: H2 makes an unlisted abbreviation a title by SHAPE, and `Xyz. Smith` is not this input. What the rule turns on is the word left standing. - A refinement of Derek's, made after the population was measured and widening the rule past the all-titles shape it was drafted for: a lone name word that is a JOIN (P3) carrying title vocabulary reports `title-or-name` too, the fork there being whether the title word inside the unit is a title at all rather than which field the unit takes. `John of Prince` and `Smith and Prince` are the measured inputs; no corpus name reaches that branch, because a join LED by a title word is chained into a title run by H3 (`Prince of Wales`), so the two are pinned as case rows rather than as rules.md examples. It sits on O5's branch and takes precedence there, which is why O5's statement says a title silences THIS kind and not every report at the site. - The suffix half is the same argument on the other vocabulary and needed no new kind. An input whose every word is post-nominal vocabulary reads its first word as a name — assign's "everything suffix-shaped after titles: first one is the name" carve-out — and that is the doubt SUFFIX_OR_NAME already names, so it reports that. `Rinpoche` and `QC MP` are the corpus names; `PhD`, `MBA` and `III` are the same shape. `Jr.` alone is NOT the shape at all: H2's opening-abbreviation rule reads it as a title before the suffix vocabulary is consulted, which is that rule's stated precedence and is recorded here because the expectation going in was that `Jr.` alone read as a name. The guard carries two exclusions of its own. A maiden name beside the credential says the input is not post-nominal vocabulary and nothing else, so `abd née Jones` is out for the reason M4 keeps it out of O5's report. And the report is scoped to names no script order placed, so a lone glued CJK honorific — さん, 씨, 선생님 — reports nothing: that is the same shape read through the glued-honorific rules (W2, #271/#308) and the script's own order, and whether those readings should report is left to the arc that revisits them rather than settled here. - An asymmetry on the boundary between this rule and O5, noted and deliberately not fixed. `MA` and `Ma` alone report `given-or-family`; `PhD` alone reports `suffix-or-name`. Both are a bare credential with nothing beside it, and what separates them is which gate reads them: `ma` is an AMBIGUOUS acronym, so S2's gate declines to peel it and the word stands as the one name word, which is O5's branch; `phd` is unambiguous, so it peels to suffix, leaves no name piece, and reaches the bare-suffix carve-out, which is this rule's. Two conventions, two kinds, and the reading each name gets is the same either way — the caller sees a report in both cases and only the kind differs. Fixing it would mean one of the two gates changing what it reads, which moves fields for a bundle that moves none. - Nothing moves but the report, and the rules go in the three 2.x ledgers only — two of them at first and a third after the 2026-09-08 round below, which is where the ten names and the join clause's own rule come from. None is a copy of a wordlist and all three are declared in `_NOT_A_VOCABULARY_COPY`: a member copying TITLES would reach `Dr. Smith`, `Rabbi Cohen` and every other title-plus-surname name, and a member copying the suffix sets would reach `John Smith MD` and `Smith Jr.`, none of which moves. What selects these names is a SHAPE — the peel leaving one unit, and vocabulary standing in it — which no wordlist expresses. The population is EIGHT distinct names when this is written, TWELVE after the #316/#489 bundle and ELEVEN from 2026-09-09, when the /simplify round on the bundle's PR took `Sir Jr` out of rules.md and so out of corpus_rules.jsonl (#H5); the ten this entry says twice above names two OTHER quantities — the ten corpus ROWS the eight names occupy, `Dr. King` and the Queen's Bench string each appearing in two corpus files, and the peel half's ten after the bundle, itself nine from that same round. Recompute all three by parsing every name in the `tools/differential/corpus*.jsonl` glob, collecting the ones whose `ambiguities` carry the kind, and counting rows and distinct names separately (2026-09-09: eight names in ten rows at a0b93f0, eleven names in fourteen rows on this branch, nine of the eleven the peel half and two the join half). - -- 2026-09-08 #518 review round — the join clause is HOISTED beside the peel clause instead of sitting under O5's. It had been written inside O5's `n == 0` leg, so a leading title, a maiden marker or a vocabulary claim on the word silenced it — and every one of those decides which FIELD the unit takes, which is not what this clause asks. Measured before and after on this branch: `Lord Chancellor née Jones` and `Dr. King née Jones` (maiden), `Dr. Smith and Prince` and `Mr. John and King` (a peeled title), `van and Prince` and `J. and Prince` (a claimed word) were all silent and all now report `title-or-name`, with no role moving on any of them. The maiden pair takes the peel half and the other four the join half. TWO CORPUS NAMES MOVE, which the drafting expected to be zero: `Attorney General of Minnesota` and `Deputy Secretary of State`, both a title peeled in front of a joined unit that stands last — and a title needs a following piece, so H3 cannot chain the join into a title run the way it chains `Prince of Wales`. "No corpus name reaches the branch" was true only of the guarded version, and the claim is corrected in rules.md#H4 as well. They take a third `feat(#491)` ledger rule of their own rather than joining the all-titles alternation, whose issue line describes an all-titles input, which neither of these is. Also this round: the emitter comment's carve-out list no longer names "a group-flagged credential" (no such condition is in the code, and `phd_split` pins the opposite), and the `Order` NamedTuple is `EffectiveOrder` with field `order`, `roles` having been the name of two different things three lines apart. - -### H5 — the trailing title run - -- 2026-09-08 (the #316/#489 bundle; closes #316(a)) — a trailing run of period-marked title words is a title. `John Smith Prof.` read family `Prof.` and lost the surname while `Smith, Prof.` read title `Prof.`; the two comma paths disagreed and #316 is that question. They agree now. -- **The doctrine, stated rather than left to be inferred.** A period-marked word is claimed by SHAPE at the front and by VOCABULARY at the back. At the front an unlisted abbreviation is a title (#H2) and the shape outranks the vocabulary, which is why `Esq. Smith` reads title though `esq` is post-nominal vocabulary. At the back there is no shape rule, so only a listed title word chains into the title and `John Smith Xyz.` keeps family `Xyz.`. A BARE title word at the back is a name word — `John Smith Sir` and `Mary Jane King` are untouched — TITLES holding ordinary surnames being exactly why, which is the Excluded block under #P5 ("the trailing-position rule that must NOT be adopted"). The asymmetry is honest rather than an oversight: #109 shipped the leading inference on purpose, and no blanket trailing vocabulary rule is safe while king, judge and bishop are titles. -- **Q3 declined: H2 stays unconditional.** #316's open question 2 asked for the symmetric leading rule, `Esq. Smith` → suffix. Declined: it reverses rules.md#S2's "a suffix never opens the string" for period-marked words for the sake of one word, and the family comma already carries the credential case (`Smith, Esq.` → suffix). Recorded at #H2, where the `Open: #316` hook was. -- **Q2 decided: a RUN, mirroring #H3.** Successive single words that wear the abbreviation shape AND are title vocabulary chain into the title from the end, floor one name piece, so `John Smith Prof. Dr.` reads title `Prof. Dr.` rather than making `Prof.` a name word. Not chosen: one word only, which would have left `Prof.` standing as a name word in exactly the input the rule is about. The title field joins in INPUT order and the rendering needed no change — the title view joins TITLE tokens in token order already, so `Dr. John Smith Prof.` reads title `Dr. Prof.` for free. -- **The predicate refuses `is_leading_title` and that refusal is the rule.** The walk uses `is_title_piece`, the vocabulary read, shared with the leading run so the two cannot disagree about what a title WORD is while disagreeing, deliberately, about what a title SHAPE is. `is_leading_title` carries H2's unlisted-abbreviation inference; with it, `John Smith Xyz.` would lose its family name to a title. That is the plan's mutation check for this commit. -- **No fork is reported**, which is #316's open question 4. Under the input-is-a-name premise a period-marked title word is not a reading a reader would hesitate over; where the doubt is real it is the word left STANDING that carries it, and #H4 already reports that. No `AmbiguityKind` is added by this bundle and `tests/v2/test_contracts.py::_AMBIGUITY_TRIGGERS` is unchanged. -- **A1, the peel order, decided by Derek: the first peel is PROVISIONAL and a trailing title is TRANSPARENT to the suffix reading.** `X Prof. Y` reads exactly as `X Y` reads, plus the title. SCOPED 2026-09-09, in review of the docs commit: that is a claim about inputs where a name word still stands on both sides of the chain — two or more name words. Where the chain leaves ONE name word there is no second reading for it to be transparent to, and #H1 decides the field instead: `Smith Prof.` reads family `Smith` where `Smith` alone reads given `Smith` and reports `given-or-family`, and `Smith Sir.` reads given `Smith` with an empty family, `sir` being a given-name title. All three measured on this tree; rules.md#H5's statement carries the scope and `_assign.py`'s excerpt of it was updated in the same edit. The problem is order: with ONE peel, `Prof.` standing behind `Jr.` stopped the suffix peel before `Jr.`, and the title walk then removed the very word that had been blocking it — `John Smith Jr. Prof.` read family `Jr.`, a generational suffix promoted to the family name, which is worse output than the reading it replaced. So the pieces the walk takes are spliced out and ONE peel runs over what stands, in original order, and that second answer alone places a piece or reports a fork. Measured: `John Smith Jr. Prof.` reads suffix `Jr.`; `John Prof. MA` reads the family `MA` that `John MA` reads, S2's reserve keeping a bare ambiguous acronym the family of a two-word name where a second peel laid over a first read family `John`, suffix `MA`; and `John Smith V Prof. VI` reads what `John Smith V VI` reads — middle `Smith V`, family `VI`, nothing reported — where two peels each reporting their own last piece reported twice. The choice is CORPUS-NEUTRAL: both variants move the same eight names, no more and no fewer, so it is a question about output quality alone. -- **A2, the family-comma segment-1 path, decided: it gets the walk.** The spec's condition was "if the comma segment gate already routes those, say so and leave it". Measured, it does not: `Smith, John Prof.` read middle `Prof.` at 1.4.0, 2.0.0, 2.1.0, 2.2.0 and at this branch's parent — all five measured — while `Smith, John Prof. Dr.` read middle `Prof.`, suffix `Dr.` through 2.1.0 and middle `Prof. Dr.` from 2.2.0, `dr` having left the suffix vocabulary in #296. The eleven comma rows the spec calls "already routed" are the `Smith, Prof.` shape, where segment 1 holds NO name word and `segment_suffix_reading` reads it piece by piece — a different gate and a different mechanism. So the segment-1 walk was genuinely missing; with it, `Smith, John Prof.` reads title `Prof.`, given `John`, family `Smith`. It moved NO corpus name at the fix; `Smith, John Prof.` is a rules.md#H5 example, so it ENTERS the corpus in the docs commit that follows and is one of the seventeen in the population bullet below. Case rows pin it either way. The segment's walk follows the same transparency principle with its own lenient loop, which is why the candidates are the pieces that walk would not read as a suffix rather than the strict suffix test alone (`Smith, John Prof. Jr.` must reach past the post-nominal, `Smith, John Prof. V` past the numeral the lenient tail test claims, #144). -- **What A2 does NOT reach, and why that is right.** `Smith, John Prof. MA` still reads middle `Prof. MA`: after a family comma a bare ambiguous acronym is a MIDDLE name and has been since 2.0 (the `Smith, Ed` cost S2 already accepted), so `MA` stands at the end of the segment, is not a period-marked title word, and stops the walk before it starts. The walk reads from the end; it does not hunt. -- **Measured population: FIVE corpus names at the fix, SEVENTEEN after the docs commit that follows it and the two review rounds on it.** The five are the four planted no-comma rows in corpus_issues.jsonl (`John Smith Dr.`, `John Smith Mr.`, `John Smith Prof.`, `John Smith Rev.`) plus `Andrew Perkins (Mgr.)`, which the drafting did not expect. Eleven of the other twelve are rules.md examples entering corpus_rules.jsonl, eight of them this rule's own — `Smith Prof.`, `Dr. John Smith Prof.`, `John Smith Prof. Dr.`, `John Smith Prof. Jr.`, `John Smith Jr. Prof.`, `John Prof. MA`, `Smith, John Prof.` and `Mary Jane King.`, the last of those added by the first 2026-09-09 review of that commit — plus three of #H1's, which reach this walk because the run each puts behind the name word is what the walk takes: `Smith Sir.`, and `Sir John Prof.` and `Dr. Smith Sir.` from the second review round the same day. The twelfth is `毛 泽东 Dr.`, which entered corpus_cjk_tolerated.jsonl in the FIRST review round and is the one member of the population in a native script: the walk runs before the positional read, so the pieces the script test sees are all Han and the family-first order stands. It is a corpus name the walk moves like any other, and the reason it took its own ledger rule rather than a seventeenth alternative is a guard about alternations, not a difference in the argument (the ledger comment beside it says which). So the growth is documentation rather than reach, and both numbers come off the same recipe — parse every corpus name twice, once on the tree and once with the walk disabled, and diff — run on the two trees. **The disabling stub is NOT the same on the two trees, corrected 2026-09-09 in review of the /simplify round below.** At the fix (`5568fdc`) `trailing_titles` returned the count the chain TAKES and both call sites were in assign, so a stub returning 0 took nothing and disabled the walk. Since the /simplify round it returns the count it LEAVES STANDING, so 0 now takes EVERYTHING and `John Smith` raises IndexError before any measurement happens; the disabling stub on this tree is `lambda rest, *a: len(rest)`, and it has to be bound over both `_pieces.trailing_titles`, which is what `tail_reading` calls, and the name `_assign` imported for the segment-1 walk, patching either alone leaving the other walk live. Re-run that way the two figures hold: FIVE corpus names at `5568fdc`, SEVENTEEN on this branch (2026-09-09). That fifth is a genuine hit rather than a misfire: rules.md#S1 drops the brackets and reads the content exactly as if written bare, so `(Mgr.)` is a trailing period-marked title word and reads as one; its own test stays green. Recompute by parsing every name in `tools/differential/corpus*.jsonl` on the tree and on the branch's third commit's parent and diffing the seven name fields plus `ambiguities`. Baselines differ per name — `John Smith Dr.` read suffix `Dr.` at 1.4.0, 2.0.0 and 2.1.0 (`dr` left the suffix vocabulary in 2.2, #296) and family `Dr.` at 2.2.0 — so the ledger entries are per-baseline. -- **The inline frame-free gates at the two walk sites were REMOVED in review**, and this is the ONE-PREDICATE-PER-QUESTION half of the entry. Each site had a cheap inline test written to match the walk's own first condition; that is a second implementation of the question, and the measurement that justified it did not survive re-running. The band test runs early in a session where the facade sits at 453, so the "one frame of headroom" claim did not reproduce. With the gates gone the walk costs +1 frame on each entry point, inside the plan's target of two and inside `test_facade_cost_stays_within_its_band`. The walk's own cheapness is where the saving lives instead: the abbreviation-shape test is a compiled regex (a C call, no Python frame) and runs BEFORE the vocabulary call, and almost no name ends in a period-marked word, so the ordinary parse pays one match and stops. -- **A pre-existing detail mismatch, recorded and NOT fixed.** #H4's join shape reports `title-or-name` with a `detail` that says the unit was "read as a given name by convention", while under the default order H1 retags the unit to the family — so `Dr. John of Prince` reports that text with the unit in `family`. This rule adds a second input with the same mismatch, `John of Prince Prof.`, and fixes neither: the wording predates this bundle, the fork the kind reports is title-versus-name which no field answers either way, and #H4 already records why the detail names no field for the peel shape. -- 2026-09-09 (review round 2) — **A1's transparency holds for the ONE-name-word case too, and the scope written a day earlier was the defect rather than the boundary.** A1 was scoped that morning to inputs where a name word still stands on both sides of the chain, handing the one-word case to #H1 unconditionally: "the title behind that word decides its field as a title in front of it would". #H1 was then reading BOTH ends of the name as one title run, so `Sir John Prof.` read family `John` where `Sir John` reads given `John` — the trailing title changing what the leading one addressed by, which is exactly the non-transparency A1 denies. The clause is now conditioned on there being no run in FRONT of the word: where one stands it addresses and the chained trailing title only joins the title field, so `X Prof.` reads as `X` plus the title for one name word as for two. Where no run stands in front the trailing one still decides, unchanged — `Smith Prof.` reads family `Smith`, `Smith Sir.` given `Smith`. Nothing in the walk moved; the fix is entirely in which run H1 keys, and #H1's 2026-09-09 entry carries it, the measurement and the mutation. The #P5 clause below reads the same way afterwards and gains its second half: a bound given-name word behind a trailing title joins as it joins with the title absent AND lands in the same field. Said of the COMMA-LESS writing, and the review that found the H1 defect measured why the qualifier is needed: after a family comma the reserve reads no peel at all, so the join fires over the trailing title word and takes it into the given name — `Berg, abdul Prof.` reads given `abdul Prof.` where `Berg, John Prof.` reads title `Prof.`. That is PARITY (1.4.0 reads it the same, measured) and a gap in the segment path rather than a boundary of the rule, so it is recorded rather than fixed here: the clause carries the scope, and the case row `title_word_trailing_behind_a_bound_pair_after_a_comma` pins it without making it normative. Tracked as part of #316. (FIXED by #637, decisions.md#P5, 2026-10-11: the join declines the trailing title word, and the row pins the title.) -- 2026-09-09 (the /simplify round on the bundle's PR) — **the trailing peel and the title chain are ONE reading run to a FIXED POINT, and A1's "ONE peel runs over what stands" was a truncation of it.** Two contrived readings contradicted clauses this section states, both found by an altitude review reading the code against the rule rather than by a test. (1) THE SUBTRACTION MODEL. #P5's reserve modelled assign's reading as the peel's count minus the chain's take, on each of the two views the join compares. That arithmetic equals assign's answer only while the chain's take does not change what the peel would do, and at #S2's bare-ambiguous reserve it does: `abdul rahman MA` reads given `abdul`, family `rahman`, suffix `MA` — the join declining because peeling the acronym unjoined and not joined is a suffix reading the join would change — while `abdul rahman MA Prof.` read given `abdul rahman`, family `MA`, the subtraction having counted `MA` a name word to spare. The Accepted clause below, "a bound given-name word behind one joins exactly as it joins with the title absent", was false as stated. (2) THE TRUNCATED FIXED POINT. The re-peel ran ONCE, so a second title was read half way: `John Prof. MA` reads title `Prof.`, given `John`, family `MA`, and `John Prof. MA Prof.` read title `Prof.`, given `John`, family `Prof.`, suffix `MA` — the re-peel un-peeled the acronym and re-exposed the FIRST title, which nothing then took. Both measured on the branch tip, 2026-09-09. -- **The fix is one function, `_pieces.tail_reading`:** peel, chain, splice the chained pieces out, peel again over what is left, until the chain takes nothing. Assign calls it in place of its inline sequence and all its reports still come from the final peel; the reserve calls it on both views and compares the suffix lists it returns, and both subtractions are deleted. `abdul rahman MA Prof.` now reads given `abdul`, family `rahman`, suffix `MA` under title `Prof.`, and `John Prof. MA Prof.` reads title `Prof. Prof.`, given `John`, family `MA`. One further clause had to be said in the code rather than left to the arithmetic: the join must not take a piece the chain takes (#P5 — "a trailing title word the run takes is no word to spare"), which the subtraction had enforced by accident, by leaving that piece in the tail it compared. Without it `Sir abdul Prof.` joins the title into the given name; with it the reading is title `Sir Prof.`, given `abdul`, as before. `trailing_titles` returns the count it LEAVES STANDING in the same edit, the way `peel_trailing` counts, so the two compose at the call site without arithmetic at all. -- **Cost, and what paid it.** The shared helper is one frame on the common path, where the inline sequence was none. It is paid by `_effective_order` taking the segment's pieces and the name-piece indices — the shape every piece-layer predicate takes — instead of a list built for it, which on 3.11 is a comprehension frame on every parse; the reference name stays at 416 parse / 453 facade. The helper returns a bare 3-tuple rather than a NamedTuple for the same budget: a NamedTuple's `__new__` is itself a frame, measured at +1 here. -- **How big each defect's class is, measured as the rule states it.** The two readings named above are the shapes that made the defects visible; the class both sit in is the TRANSPARENCY class A1 is stated over, and the two defects reach different parts of it — an earlier wording of this bullet flattened them into one width and named the wrong shape for it. Recipe: take the alphabet `John Smith abdul rahman Prof. Dr. Sir MA Jr. V Xyz. King de née`, form every 3- and 4-word arrangement ending in `Prof.` or `Dr.` whose spelling WITHOUT that last word still fills two of given/middle/family (3,210 of them — the two-or-more-name-words scope the A1 clause is stated for), and compare the seven fields other than `title` against that shorter spelling. rules.md#H5 says they must be equal. On the branch tip 786 differ; on this commit 698 do, none of them new, and all 698 are the P2/M2 residue the rule already accepts as a boundary — a particle chain or a maiden marker took the trailing word before the H5 chain could see it (`John de Prof.`, `Mary née Prof.`). The 88 that close PARTITION by defect, and that partition is what makes them two blast radii rather than one. 56 of them carry a SINGLE period-marked title word, and every one of those carries a bound given-name word as well — the subtraction model's reach, `abdul rahman MA Prof.` itself and `abdul John MA Prof.` beside it, where the arithmetic counted the acronym a name word to spare. The other 32 carry TWO, which is the truncated fixed point's reach: `John Prof. MA Prof.` and `John Dr. MA Dr.`, where the single re-peel re-exposed a title nothing then took. The two-title members of the 3,210 number 784, which is the population that could have witnessed the fixed point at all: 164 of them differed on the branch tip and 132 still do, every one of those 132 the same P2/M2 residue the 698 are, so the defect accounted for exactly the 32 that closed and no more. Over the same sweep's 82,712 inputs (both comma-less and family-comma spellings, 2 to 4 words) exactly 156 parses move, splitting the same way — 96 carry two period-marked title words, 60 carry one, and all 60 of those are bound-given — and not one mover is a comma spelling, the family-comma segment path reaching neither defect. Every figure in this bullet re-measured 2026-09-09 on this tree against `7ac2218`, the round's parent. - -- **Nothing else moved, and it was measured rather than argued.** Snapshot all seven fields, the ambiguity kinds and the recorded `order` for every name in `tools/differential/corpus*.jsonl` plus every string literal in `tests/v2/cases.py` (`ast.walk` over that file, which sweeps up the notes and the case ids too — harmless, they parse like anything else), on the branch tip and on the fix, and diff: 2682 strings before and 2681 after, 2674 of them on both trees, and NONE of those 2674 reads differently. The seven that differ are the case rows this round added and dropped, not parses that moved. The two readings above are inputs no corpus holds — as is every other member of the class the bullet below measures — and each takes a case row (`title_word_trailing_run_is_read_to_a_fixed_point`, `title_word_trailing_behind_a_bound_pair_at_the_peel_reserve`). The same round dropped the `Sir Jr` example from rules.md#S2 and its case row: it reads by the same branches as `Dr Jr`, the same roles and the same reported kind, the run being empty by then and no branch reading `vocab:given-title` — so the pair pinned one reading twice. Its leaving corpus_rules.jsonl moves two ledger rosters: the jr suffix-routing rule 7 → 6 corpus names in the 1.4.0 ledger, and the #489 peel-floor rule 4 → 3 in all four. - -- 2026-09-26 #535 — CROSS-REFERENCE: the maiden half of rules.md#H5's "the chain reads PIECES" Accepted row now reads the title — a maiden clause is no longer a join that puts the trailing word out of the chain's reach, so `Mary Smith née Jones Prof.` reads title 'Prof.' — decided in the 2026-09-26 #535 entry under `### M2`. -- 2026-10-07 #614 — the accepted cost 'the chain reads PIECES, so a join that ran earlier puts the word out of reach' is retired for the particle chain on a main segment: the chain is read with S2's run, once, after P3's connective joins and before the particle chain is made, so a particle chain stops in front of a trailing title -- `John van der Berg Prof.` reads title 'Prof.', family 'van der Berg' (every release from 1.4.0: family 'van der Berg Prof.'), the example rules.md#H5 carried for the cost now pinning the reading. It stands for a connective's join, made before the read, so a title it joined into a longer unit is no single word (`John Smith Prof. and Dr.` keeps family 'Prof. and Dr.', the case row that pins the one-word gate), and after a family comma, whose given part is read once all its joins are made (`Smith, John van der Berg Prof.` keeps middle 'van der Berg Prof.'; SUPERSEDED IN PART by #623, below: there the chain now stops in front of the title). decisions.md#S2, this date. -- 2026-10-10 (Derek), #623 — AFTER A FAMILY COMMA THE PARTICLE CHAIN STOPS IN FRONT OF A TRAILING TITLE TOO. Assign's walk over the given part already reads the trailing title run, with its own suffix reading (#144's lenient tail with two parts, #602's run, P6's attachment), so `Smith, John Prof.` reads title 'Prof.'. What it lacked was a chain that left the title standing: group bounded the chain there by `trailing_start`, the suffix peel alone, so the chain took the title into the particle's name before the walk met it, and `Smith, John van der Berg Prof.` read middle 'van der Berg Prof.' where `John van der Berg Prof.` reads title 'Prof.' since #614 (1.4.0 and 2.3.0 read middle 'van der Berg Prof.' for the comma spelling). At that site the bound is now `_pieces.trailing_run_start`, where the run `tail_reading` takes begins, its titles included, so the chain stops in front of the title and the walk reads it: title 'Prof.', middle 'van der Berg'. Nothing new is counted -- only the chain's stop moves -- so the issue's question of which words-to-spare floor the given part uses does not arise, and the comma's own readings of a numeral or a bare acronym there (`Smith, John van der Berg VI` middle 'van der Berg VI') do not move. P6 (the issue's open question) reads after the title run, as though the title were absent: `Smith, Jan van Prof.` → title 'Prof.', family 'van Smith', as `Smith, Jan van` reads. The bound is asked only where a particle stands past the chain's lead, by an inline tag scan, since the chain otherwise merges nothing -- which also drops the `trailing_start` call master made for every family-comma part. - DECIDED (Derek): the narrower bound. - Declined: the issue's proposal, reading the given part's run once ahead of the chain and splitting it off as the main segment's is (#614). It would have assign place the given part off group's read, a second reader beside the walk's own predicates, for a class the bound alone closes: the transparency measurement below finds no text the narrower fix leaves broken that the read would mend. - MEASURED 2026-10-10 against 4d5631c2 (#634's merge), py3.11, `nameparser.__file__` asserted on each side. A given-part grid -- `{family}, {head} {middle} {tail}`, and the same with `, Jr.` appended, for family in `Smith`, `van der Berg`, `Ortega y Gasset`, (empty), `(Bob)`; head in `John`, `Dr. John`, `Freiherr`, `abdul`, `Freiherr St`, `J.`; middle in (none), `van`, `van der`, `de la`, `van Berg`, `van der Berg`, `Mc Mc`, `von und zu`, `Berg`, `Berg van`, `do`, `St`, `y Lopez`; tail in (none), `Prof.`, `Dr.`, `Prof. Dr.`, `Prof`, `Sir`, `Prof. PhD`, `PhD Prof.`, `Jr. Prof.`, `Prof. Jr.`, `MA`, `Ma`, `VI`, `V`, `X.Y.Z.`, `Ph. D.`, `Esq.`, `née Jones`, `Prof. Jones`, `Xyz.`, `King.`, `Prof. V`, `MA Prof.`, `Lt.Gov.` -- 18,480 texts under the three orders and `lenient_comma_suffixes=False`, comparing the seven fields and every report: 17,340 of the 73,920 parses move a role (4,335 texts under each configuration), and no parse moves a report alone. All but 500 parses (125 texts) put a trailing period-marked title into the title field. H5's transparency over the grid's default-order texts whose given part ends in `Prof.`, `Dr.`, `King.` or `Prof. Dr.` (5,390; the parse with the title against the parse without it, plus the title): broken on 392, against 3,048 on master, and on no text master kept. By cause, 352 of the 392 are H1's move of the name behind a title into an empty family (an empty or nickname-only head), and 40 are P5's lenient post-comma join taking the title into a bound given name -- 24 after a named family (`Smith, abdul Dr.` reads given 'abdul Dr.') and 16 behind an empty head (`, abdul Prof.`), the gap rules.md#H5's Accepted P5 paragraph records (fixed by #637, decisions.md#P5, 2026-10-11). The 500 put a title in front of a numeral with a third comma part (`(Bob), Freiherr Mc Mc Prof. V, Jr.`): the bound reads 'V' as a numeral and stops before 'Prof.', where the walk keeps 'V' a name word with a third part (#144), so 'Prof.' lands in the middle 'Prof. V' rather than the given 'Mc Mc Prof.' -- a name word either way, garbage. The S2 stress grid's comma half (#614's recipe): 56 moves, all a trailing title behind a particle after a family comma (`Smith, Jan Freiherr von Berg Prof.`); #624's title-particle grid: 24, the same class (`Smith, John Freiherr Dr.` → title 'Dr.', family 'Freiherr Smith'). #625's particle grid, every corpus name and case text, and #624's comma-tail grid: 0. The five gates exit 0 with the new `fix(#623)` rule in every ledger. Frames (3.11, warm): the reference rows do not move; a family-comma name with a particle pays 2 with no trailing run (`Smith, John van der Berg` 282 → 284), 4 with a suffix (`Smith, John van der Berg Jr.`), and the moved shapes 38 to 73 in a sample (`Smith, John van der Berg Prof. PhD` +38, `Smith, John van der Berg Prof.` 310 → 360, `Smith, Jan van Prof.` +73), with an unmoved particle name ending in a suffix and a title 20 to 22 (`Smith, John van der Berg PhD Prof.` 382 → 402, `Smith, John van der Berg Jr. Prof.` 382 → 404), rare shapes no guard sees; one without a particle pays less (`Smith, John Quincy` 221 → 213, `Doe, John MA` 254 → 227), which `test_a_given_part_with_no_particle_skips_the_chain_bound` holds (the bound skipped, 'Smith, John Prof.' 250 frames with it asked against 207). Recorded negative control: with the old bound, 8 tests fail -- the two `fix(#623)` rows on both surfaces, the three new rules.md#H5 examples and the field-sweep count. - THE REVIEW (2026-10-10). The bound is `tail_reading`'s run, read as the comma-less name reads it, and the walk reads the given part by its own predicates, so where the two disagree on a word behind a title the chain stops for a title the walk does not take, or cannot cross one to reach a particle P6 takes: a title in front of a shape-only numeral (`Smith, John Mc Prof. VI` → suffix 'Mc', middle 'Prof. VI'; master middle 'Mc Prof. VI'; the walk never reads 'VI' as a suffix after a comma), in front of a numeral with a third comma part (the 500 above), in the middle of the given part in front of a particle (`Smith, John van Prof. Mc` → title 'Prof.', middle 'van', family 'Mc Smith'), and a period-marked title-particle ending it, which P6 attaches (`Smith, John van MA St.` → suffix 'MA', family 'St. Smith'). Every such shape is garbage, most read as master already reads the text without the title (`Smith, John Mc VI` suffix 'Mc'), and none is a reading a well-formed name loses: over about 1.6 million generated rows under every order, the opt-in policies, one-case writing and caller lexicons, and over a 134,720-pair transparency grid (`Smith, ` and `, ` heads; one to three words from `John`, `van`, `der`, `von und zu`, `Berg`, `Ma`, `MA`, `V`, `VI`, `Jr.`, `PhD`, `Ph. D.`, `Mc`, `St.`, `St`, `Freiherr`, `X.Y.Z.`, `Esq.`, `abdul`, `Prof.`; a trailing ` Prof.`, with and without `, Jr.` after it; the three orders and `lenient_comma_suffixes=False`; transparent where given, middle, family, nickname and maiden equal the title-less text's and the title gains exactly ` Prof.`; master broken on 53,384 pairs, the tree on 9,734), no text transparent to its title on master is broken now. (The first review named `Smith, John Ph. D. St. Prof.` as one, its oracle reading the direction backwards: master gives suffix 'Ph. D. St. Prof.' against the title-less 'family St. Smith', and the tree is transparent.) Recorded, not fixed (Derek): garbage, and narrowing the bound would add a condition for it. The review also found the bound's particle gate held by no test, two of its narrowings undoing readings master gives -- a scan from the name's start rather than the chain's lead (`Smith, Freiherr von Berg Prof.`) and a test of single-token particles alone (`Smith, John von und zu Berg Prof.`) -- now rows of their own, with an empty-head row (`, Jan van Prof.`, AGENTS.md's comma sweep), and the frame saving now held as above. Each of the three mutations fails a test. The re-ask after the chain (`left < tail`) still decides readings, though its documented case (`Freiherr von Berg Ma`) takes the read-once path since #614: removed, it moves 400 parses of the given-part grid on master and here alike, a title-particle head before a particle run and a dotted acronym (`Smith, Freiherr van Berg X.Y.Z.` → given 'van Berg X.Y.Z.', as 1.4.0 reads it; without it suffix 'X.Y.Z.'). It decides a main segment's run with a name piece inside it too (`Freiherr van Berg Dr. X.Y.Z.` → family 'van Berg Dr. X.Y.Z.', as master and 2.3.0 read it; without it given 'van Berg Dr.', family 'X.Y.Z.', 1.4.0's reading), a garbage shape no row holds. The first review reported it changing nothing over a grid with no such shape, and #638 was filed on that; the fix review found it live and held by no test, and the row `the_chain_runs_again_where_its_merges_free_the_acronym` (parity) holds it now. Left for follow-ups, two gaps outside this change: a part past the second comma reads a trailing title behind a particle into the suffix (`John Smith, Jr., van der Berg Prof.`, where `John Smith, Jr., Prof.` reads title 'Prof.'; #636), and P5's post-comma join taking a title (the 40 above; #637). - -### H6 — a title before a family comma - -- 2026-10-17 (Derek), #701 — A TITLE OPENING THE PART BEFORE A FAMILY COMMA IS THE TITLE. `Dr. Smith, John` read family 'Dr. Smith', and every release did the same: 1.4.0 from PyPI gives last 'Dr. Smith' and 'Prof. Brühl' (suffix 'Dr.') for `Prof. Dr. Brühl, Ada Cornelia` (measured 2026-10-17, `PYTHONSAFEPATH=1`, wheel path asserted). The reading was written down on purpose as Accepted clauses of rules.md#H2 and #H3 ("before a family comma the pre-comma text is wholly the family name") and assumed by C1's Freiherr clause ('Prof. Cruz, Ed'); all three are rewritten here. The listing form moves the surname to the front and leaves the title where it was written, and CLDR's index renderings write exactly that in several locales (`prof. dr. von Selimović Hodžić, Izeta Amra`, `г-н Вернер, Иван Асен`). - DECIDED (Derek, the five points of the design proposal): (1) VOCABULARY ONLY. H2's shape does not fire before a family comma, so `Xyz. Smith, John` keeps family 'Xyz. Smith' — the slot is a surname's slot too, as the trailing slot is (#H5), and every CLDR case is a listed title. (2) The run leaves the family its last piece (`Dr., John`, `Greve, Anna`). (3) A word that is a particle too stops the run, bare or period-marked, so `St Pierre, Ed` and the 1919 legal-surname reading of `Freiherr von Berg, Ed` (C1's Accepted clause) stand, and a title in front of them still peels. (4) The peeled titles join the title field in input order, and the comma decision (`_comma.decide`) is untouched — the count before the comma already treated a title as no name word. (5) ACCEPTED COST: a bare surname-like title opening a two-word surname is the title, `King Smith, John` → title 'King', as `Greve Anna` already reads (#H1). - ADDED IN THE PROTOTYPE, not in the proposal: #H3's floor. The first version peeled `MD` from `MD PhD Jr., John`, leaving family 'PhD' — which a case row (`family_comma_segment_zero_is_not_the_run`) caught. A title addresses somebody, and a post-nominal is not somebody, so the run stops where nothing but post-nominals stands behind it: `Dr. Jr., John` keeps family 'Dr.'. The second version stopped in front of ANY piece of the suffix vocabulary, and the design-docs review found it too broad: a title/suffix dual standing SECOND in the run blocked it and then read as a suffix inside the family (`Rev. Sr. Smith, Mary` → family 'Rev. Smith', suffix 'Sr.', where the comma-less `Rev. Sr. Mary Smith` reads title 'Rev. Sr.'), and so did `Mc` and `vd`, which the family loop's own P2 clause reads as a surname's head (`Dr. Mc Donald, John`). Its check compared `T Smith, John` with `T John Smith` over every title word, bare and period-marked, alone, behind `Dr.` and ahead of it (4,788 pairs over the 798 title words before #705's merge): 36 disagreed, 12 the decided particle class and 24 the twelve duals in second position. The floor asks the H3 question instead — is everything behind this title post-nominal? — so the duals peel wherever a name word follows them, as they do in front of a name with no comma. - Declined: excluding the title/suffix duals from the run (it would keep `Ms Smith, Jane` whole); peeling by H2's shape too (the trailing doctrine, #H5, says only vocabulary claims a word in a surname's slot). - MEASURED 2026-10-17 against master `eb2c1fa5`, py3.11. Snapshot (`uv run python tools/differential/snapshot.py --since master`, before this change's own rows entered the corpora): 20 entries move, all one shape — a title leaves the family for the title field — and 18 of their names are held only by corpus_cldr.jsonl. Several of those are names C1 reads as a family comma though the part after it is a credential CLDR's vocabulary lacks (`hr. Hans Henrik Sven Åge Møller, MF`); the title still comes out right there, and the family stays as wrong as before. CLDR agreement: 29 fields now agree and 28 more get closer to CLDR's value, with none regressing. Merging master after #705 listed `bapak` adds one more of the same shape, `Bapak Adinata, Dwi Putro bin MP` (title 'Bapak' out of family 'bin Bapak Adinata'), two more CLDR fields agreeing and one closer. Cost (`sys.setprofile` call events, one warm-up): `Dr. Juan Q. Xavier de la Vega III` 366 → 366 (the benchmark's `_REFERENCE` row, 307, is unmoved too), `Smith, John` 161 → 161, `Smith Jones, John` 203 → 203 (an inline tag test ahead of `is_title_piece` keeps a part no title opens free of frames), `Dr. Smith, John` 200 → 201 (one `is_title_piece`; the floor's `is_suffix_piece` on 'Smith' replaces the family loop's, which no longer asks the first family piece), `Prof. Dr. Brühl, Ada` 279 → 280. The family loop does not re-ask a piece at or past the floor, so a long post-nominal run costs one frame more than on master, not one per member: `Dr. Smith Jr. PhD MD, John` 306 → 307 (310 before the /simplify round), `Dr. Smith` plus twenty `PhD`, `, John` 903 → 904 (924 before). Those two changes moved no reading on a grid of 88,740 family-comma heads (one to four words from `Dr.`, `Prof`, `St`, `Freiherr`, `Sr.`, `Ms`, `Smith`, `van`, `Vd`, `PhD`, `Jr.`, `MA`, `V`, `de`, `Rev.`, `Mc`, `née`, before `, John`) under the default and `FAMILY_FIRST` orders, comparing fields, ambiguity kinds and `initials()`. - THE PR REVIEW (2026-10-17), three decisions. (a) `gov` JOINS TITLES (Derek). `Lt. Gov. Smith, John` read title 'Lt.', family 'Gov. Smith' — a compound split across two fields, where master kept it whole in the family and the comma-less `Lt. Gov. John Smith` reads title 'Lt. Gov.' by H2's shape. The run is vocabulary-only, so the fix is the vocabulary: `gov` is the abbreviation of the listed `governor`. Counts (`uv run python tools/name_usage.py Gov`, 2026-10-17): 236 US Census 2010 surname bearers, rank 80,419, 91.53% Asian/Pacific Islander; SSA given names under 5 a year; Wikidata 3 family-name bearers, all Israeli. The claim acts only in front, so `John Gov` keeps family 'Gov', and a surname written first loses it (`Gov Sokha` → title 'Gov'), the graf-class trade (#salutation-titles) at a smaller incidence. No corpus name moved. The other unlisted second words the review found (`insp`, `sen`, `supt`, `cmdr`, `atty`, `msgr`) still split; each is a vocabulary question on its own. (b) `Dr. PhD Smith, John` KEEPS the branch reading, family 'PhD Smith' (Derek): the first piece the run leaves is family whatever it is, as piece 0 is with no title (`PhD Smith, John`, read that way by 1.4.0 too); master and 1.4.0 gave `Dr. PhD Smith, John` family 'Dr. Smith', suffix 'PhD'. Garbage input, so no row. (c) `Graf Lambsdorff, Otto` reads title 'Graf' while `Freiherr von Berg, Ed` keeps the rank, and the clause first written for it gave the 1919 legal-surname argument as the reason — which holds for every German rank alike. The reason is the particle stop: `freiherr` is the one rank in `particles`. rules.md#H6's Accepted clause now says so, and the vocabulary question (should `graf`, `fürst`, `prinz` and the rest read as `freiherr` does, with and without a comma) is [#709](https://github.com/derek73/python-nameparser/issues/709). - -### W1 — unspaced CJK division - -- 2026-07-27 #271 (decision; shipped in 2.1.0 via PR #294) — Korean division ships as a default: the census surname list is closed, hangul is self-selecting (a hangul entry can only match hangul text), and being unsplit is recoverable while a wrong split is not — which is also why an unrecognized name stays whole. The filed proposal (#271, 2026-07-07) asked for OPT-IN segmentation for Korean too, "like all localization"; default-on is the later refinement, and the census/self-selecting argument above is what justified promoting Korean past the blanket opt-in stance. -- 2026-07-29 #272 (the ja amendment; shipped in 2.1.0 via PR #297) — Han division is opt-in per language pack because Han text does not identify its language (高橋一郎 under a Chinese list divides wrongly); a pluggable segmenter takes what the vocabulary declines, so pack + segmenter compose mechanically: a listed surname is a dictionary certainty and wins. - -- 2026-07-29 (ja amendment §1a) — script CLASSIFICATION NFC-normalizes its input (NFD katakana carries combining marks outside the block; NFD hangul decomposes to jamo, which would miss the order rule for macOS-origin names) while SEGMENTATION stays raw — chosen over offset-mapping complexity because NFD then degrades to no-split, never to a wrong split. The read-only folds are the deliberate exceptions to "nothing rewrites the text" (rules.md T Background), and neither ever enters a token. (Corrected 2026-09-10, #cjk-full-stops: this read "segmentation MATCHING stays raw" and "the one deliberate exception". Vocabulary matching composes NFC too since #322 — `_lexicon._normalize` folds every lookup and every stored entry the same way — so the folds are two, and what stays raw is segmentation proper: the surname site's membership test and the peel's tail slice, which index the token's own text. The no-split degradation is what still holds there.) - -Declined: - -- JMnedict as bundled segmentation data (#272's body, 2026-07-07) — the pip packaging (jamdict-data) was last compiled 2021-04; JMnedict carries no frequency data, so it cannot resolve the ambiguous 2+2/3+1 splits that motivate a segmenter at all; and bundling raised licensing questions the core avoids by delegating to namedivider — whose code and GBDT model are MIT, whose surname data carries Myoji-Yurai terms permitting exactly this name-dividing use, and whose CC-BY-SA BERT model is unused (the 2026-07-29 amendment corrected an earlier flat-"MIT" description; so does this entry). The kind of decline someone re-proposes in two years. - -- 2026-08 — zh and ja packs are corpus ALTERNATIVES, one per corpus; stacking them (parser_for(ZH, JA, segmenter=...)) is for genuinely mixed data that accepts the trade: a listed Chinese surname wins before the segmenter is consulted, so 高橋一郎 still divides 高 + 橋一郎 under ZH+JA exactly as under ZH alone. Kana gating resolves through the same script function order uses, so kana-licensed composites (高橋みなみ) gate in under JA while pure-katakana tokens never do (amendment 2026-07-27: activation is per script because the ambiguity is per script). - -### W3 — the writer's divisions are respected - -- 2026-08 (segmenter amendment 2026-07-29) — a segmenter answers where an UNDIVIDED name divides, so it is consulted only when the gated token is the name part's only script-written one: "山田 太郎" was divided by its writer and must not have its family divided again. The peeled tail is exempt (that boundary was manufactured, not written); a SPACED honorific is not — at that position it cannot be told from a spaced name element. The trade was measured before deciding: counting spaced honorifics keeps four real surnames whole (佐藤 氏, 田中 様, 鈴木 先生, 中村 教授) and costs the one division 山田太郎 様. -- 2026-08 #312 — under FAMILY_COMMA the whole stage stood down until the peel moved in front of the gate: the comma doctrine is about the input's structure, not the split's source, so it covers vocabulary and segmenter identically, while an honorific is no part of the name whichever side of the comma it glues to. -- 2026-07 — the stage runs AFTER comma segmentation on purpose: running before would make the comma structure depend on the split ("김민준, Jr." pre-split would read suffix-comma on vocabulary alone; as written it stays the listing form). - -### W2 — the glued-honorific peel - -- 2026-08 #308 — an entry peels only where it can never end a name: 씨/님/さん/様/先生 peel; 양/군/氏/博士/殿 stay spaced-only; 君 is in neither set while its kana spelling くん peels. The vocabulary carries the license, so no structural or per-script gate stands over the peel. -- 2026-08 #312 — the peel crosses the family comma and the 间隔号: both answer where a name DIVIDES into surname and given, a question the peel never asks. Provenance matters here: neither gate was argued for the peel specifically — both came with the placement (#312's own framing) — so the crossing was a repair of inherited gates, not a designed-in property. -- 2026-08 #319 — a wholly suffix-shaped second run is declined as a peel site (the "田中さん, V." shape), but only when the name's own run offers a site, since a glued honorific is itself part of what makes a run read as suffix-shaped. -- 2026-10-17 #671 — SPACED AND AFTER-COMMA ONLY, no glued peel (Derek, 2026-10-17). 議員/议员, 国会議員/國會議員, 太平紳士/太平绅士, 医学博士/醫學博士, 二世 and the job titles 社長/部長/課長 and the simplified 社长/部长/课长 (Derek, 2026-10-17: the characters are shared and the offices are real in both languages; no borne-name data either way, #707) join `SUFFIX_WORDS`; none joins `GLUED_HONORIFICS`. The spaced bar is only that a word standing alone is no name, so they ship; the glued bar ("can it END a name?") needs real-name counts that this change did not get — the Wikidata suffix query timed out and the native-script Wikidata corpus holds none of the words — and 部長 fails it on argument alone (長 is a borne given name, 阿部長). A follow-up can vet 医学博士, 議員 and 太平紳士 one at a time. Consequence measured on `parser_for(locales.ZH)`: after a listed surname each word divides off, `王二世` family 王, suffix 二世 (it read given 二世) and `李部長` family 李, suffix 部長, because the segmenter's division is not a peel and the split-off 二世 is then a whole-token suffix word, the way 王博士 already reads; `陳怡君二世` still reads given 怡君二世 (glued, nothing divides it; superseded for the glued question by the #707 entry below). ジュニア was already shipped by #661, and #679 declined 二世 as CLDR's generation word ("regnal, and glued"); here it ships spaced-only as the Chinese/Japanese numeral, which does not reverse that. Excluded as the issue stated: 小 (a prefix and a common given-name character) and 冯/馮 for "von" (a top-30 surname). -- 2026-10-17 #707 — the #671 words meet the glued bar, all but 部長/部长 (Derek, 2026-10-17). 議員/议员, 国会議員/國會議員, 太平紳士/太平绅士, 医学博士/醫學博士, 二世, 社長/課長 and 社长/课长 join `GLUED_HONORIFICS`; 部長/部长 stay spaced-only on the #671 argument, since 長 is a borne given name (Wikidata holds it as a Chinese given name, Q96240389; the Japanese reading rests on the argument) and 阿部長 can be 阿部 + 長 — the measurement below found no such bearer, and a zero that is a lower bound does not refute the argument. The measurement is `tools/name_usage.py`'s ends-with row, new in #707: a local index of every CJK label on eleven name-item classes (family name, given name and their Japanese, Chinese, Korean, male, female, unisex and undifferentiated-kanji subclasses; 215,798 index entries on 128,695 items, 215,696 distinct item-label pairs, fetched 2026-10-17 by the first index, whose character class stopped at the Basic Multilingual Plane; the rebuilt index that takes supplementary-plane Han too (215,820 entries, same day) gave the same answer for every word, 殿 and the boundary pairs below), matched locally, with the matching items' bearers counted live. Two questions per word: does any name item's label END in it, and does any person bear a family name ending in the word's head and a given name equal to its tail (the 阿部 + 長 shape)? Neither found anything for any spelling, with one exception that is a data error: the only item ending in 二世 is Q18339792, the given name Odo, whose Chinese label carries a regnal numeral (厄德二世), and its 109 bearers are Western people named Odo. Of the boundary shapes, only 二+世, 社+長, 社+长, 部+長 and 部+长 had a family name and a given-name item to pair at all, and each counted 0 people. What makes the zeros evidence: the index agreed exactly with a live `STRENDS` query on all 20 class × word cells it was cross-checked on (five subclasses × 殿, 子, 郎, 田, the largest 403 items); 子 found 439 items and 2,436 given-name bearers, 2,081 of them Japanese; and 田 + 太郎 counted 22 people across the boundary. What limits them: Wikidata's coverage, not the index's — 殿 ends 13 name items where a surname dictionary lists about 90, and Japanese family-name items number 4,919. Two shortcuts were measured and rejected before the index: `STRENDS` over every label of a name class timed out at WDQS's 60s with no person join, and WDQS full-text search found 4 family names ending in 殿, matching only where its tokenizer split the character out. Consequence: the snapshot moved four CLDR renderings, each losing its glued word to `suffix` (`王雅婷婷婷二世議員`, both spellings of `…小惠儀太平紳士`, `馮·陳·怡君·達印·二世醫學博士`), and `陳怡君二世`, which the case table and rules.md W2 pin; cldr_agreement records one suffix field newly agreeing (`二世 醫學博士`, where the peel takes 醫學博士 and the 二世 left behind is a whole-token suffix word) and none regressing. The peel still takes one word: `王雅婷婷婷二世議員` peels 議員 and keeps 二世 in the name. Recompute: `uv run python tools/name_usage.py 議員 议员 国会議員 國會議員 太平紳士 太平绅士 医学博士 醫學博士 二世 社長 部長 課長 社长 部长 课长`; the first run builds the index (33 minutes on 2026-10-17, resumable), and `--refresh-index` rebuilds it. ACCEPTED COST, found by the design-docs review and kept (Derek, 2026-10-17): the vetting asked whether a NAME ends in each word and never whether a longer, unlisted OFFICE word does, and common ones do — 衆議院議員 and 参議院議員, 副社長 and 代表取締役社長, 元議員 and 市议员, 副課長, 獣医学博士. The peel matches at the end of the token, spaced or glued, so each splits at the listed tail: `岸田文雄衆議院議員` reads family 岸田文雄衆議院, suffix 議員, and `田中 副社長` given 副, suffix 社長, where before the word stayed whole in the name. Neither reading is right; the compound's head is what the peel leaves behind. No corpus held a compound office word when the review found this, which is why the snapshot was silent; the two W2 examples now put `岸田文雄衆議院議員` and `田中 副社長` in corpus_rules.jsonl. Older entries carry the same cost unrecorded until now (`김민준의학박사` reads given 민준의학, suffix 박사). Listing a compound whole would peel it whole, longest first. - -Excluded (the never-given / ambiguous particle line, nameparser/config/particles.py — #360 owns the vocabulary question): - -- Only 9 of the ambiguous members were individually justified at #360's audit (2026-08); the rest sit there by the conservative default (ambiguous unless argued never-given). #604 (2026-10-04) argued three more on entry — ua, binti, binte — under vocabulary-collisions. -- mc, ste — measured misparses ("Mc Donald" → given "Mc"), tracked in #360; st is inert at the head because TITLES claims it first; mac must stay ambiguous because Mac is a real given name. -- ben — OUT of the set entirely, removed in v0.2.5 and re-affirmed 2026-09-01 (#v1-xfail-triage). Ben is a common given name standing exactly where a forward-joining particle claim would act: "Ahmad Ben Husain" reads middle 'Ben' today, and admitting `ben` would take that token into the family. This is the C-i position test, and the entry is here because the pressure to re-add is real and recurring — "Ahmad ben Husain" wants family "ben Husain", the v1 suite carried that aspiration as an xfail for years, and a sweep of Arabic and Hebrew patronymic particles is exactly how it comes back. A caller who needs the patronymic reading adds it to their own Lexicon. -- nic, unaccented o, bt, bte — left OUT by #604 (2026-10-04). `nic` is the Irish "daughter of" beside `ní`, but also an abbreviation of Nicolas and Nicole and a given name of its own (Nic Cage): a leading `Nic` would be marked ambiguous at best, and its natural home is an opt-in Irish pack. Bare `O` is a common periodless initial, so `o` as a particle would turn `Liam O Murchú`'s middle initial — and every `John O Smith` — into a surname; the anglicized form is glued (`O'Neil`) and needs no particle. `bt`/`bte`, the abbreviations of binti/binte, were not reviewed, and `bt` is not an unknown: it is already shipped as a suffix acronym (Baronet) with a `Bt` case mask, so admitting it as a particle would open a new particle ∩ suffix overlap, the contested class rules.md#P6 decides for vd, do and mc, and under R4's mask precedence its mask would capitalize the particle too. -- Encoding rationale (#293, predating #360's membership questions): the data layer stores the NEVER-GIVEN set and derives the ambiguous one, because that is safe-by-default for new particles — a one-place addition — and the v1 shim translates by one-directional complement. And the constants are FROZEN specifically to kill the cached-Lexicon.default()-vs-fresh- Constants desync that runtime module-constant mutation caused. -- Load-bearing dependency: TITLES ∩ ambiguous — {do, freiherr, st} until #296's audit, {freiherr, st} since — is what keeps the particle-or-given ambiguity emitter reachable at all; moving them all would leave the shipped vocabulary unable to reach it, which is why test_the_chained_emitter_is_reachable_by_construction (tests/v2/test_parser.py) pins reachability by construction rather than by shipped vocabulary — an empty intersection fails no test and does not mean the emitter is unreachable. - -Open (contested vocabulary memberships — the rule is right, the word's set is questioned; the issue is canonical): none. The block held three entries and all three were answered inside a fortnight — #346, #343 and #344 together on 2026-09-06 (see #indic-honorifics) and #342 on 2026-09-07 (see #suffix-acronym-collisions). The heading stays with nothing under it on purpose, the same reason an empty ledger roster section is a statement and a missing one is nobody having looked. sa, se and om are named as the next candidates by #342's own comment and carry no issue, so they are not entries here. - -Excluded (SUFFIX_ACRONYMS — esq, removed 2026-09-08, #316/#489 bundle): - -- esq is OUT of SUFFIX_ACRONYMS and must not be put back by a sweep that finds "Esq." parsing and assumes the acronym set is what carries it. The SUFFIX_WORDS membership carries every single-token spelling — Esq, Esq., ESQ, esq — and stops being inert with the acronym entry gone. What the acronym entry uniquely covered is the multi-dot spelling, so "John Smith E.S.Q." reads family E.S.Q. where every release since 1.4.0 read suffix; that is the whole of the cost and it is a spelling nobody writes. The criterion is #suffix-acronym-collisions', asked of the WORD rather than of the frequency: does the entry describe the word or the machinery. Esquire is a contraction, not an initialism, so the initialism set was never its home; it arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. This SUPERSEDES the dual-membership entry that stood here from the 2.2 cycle, which called the acronym membership load-bearing and the word membership inert and concluded that the singleton "is why the two sets cannot carry a disjointness assert". The assert now exists — `not (SUFFIX_ACRONYMS & SUFFIX_WORDS)` in suffixes.py's guard block — and it is what the removal buys: the two sets normalize differently (the word test strips edge periods, the acronym test strips all of them), so a word in both is matched by two rules and which one fired is unreadable from outside. AGENTS.md's esq gotcha, the two defending comment blocks in suffixes.py, the `# NOT asserted:` note and the `suffix_acronym_multidot_spelling` case row all retire with it; a row pinning `John Smith E.S.Q.` → family replaces the last of those. Measured over the corpus glob as it stood at the change (1123 distinct names, 1263 rows; the docs commit that follows adds this bundle's rules.md examples and makes it 1136 and 1284): exactly one moves, `John Smith E.S.Q.` itself, and it diffs at every baseline. Classified a behavior change, a 2.x parity break, on all four ledgers. - -Excluded (SUFFIX_ACRONYMS — ph, removed 2026-09-13, #459/#521): - -- ph is OUT of SUFFIX_ACRONYMS and must not be put back by a sweep that finds "Ph. D." parsing and assumes the acronym set is what carries it. The split spelling is merged by group's own rule (the `PH`/`D` regexes of `_pipeline/_vocab.py`, v1's fix_phd; see #phd-merge, whose "merged back by vocabulary" means that regex stage and not a lexicon lookup), which never consults acronym membership, and the single-token spellings `phd`/`Ph.D.` are the `phd` entry's. The fragment arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. It did two things. One was to let the merged `Ph. D.` token pass the acronym test on its first piece, which nothing read until #459's all-caps repair did — with `ph` in the set that repair read `john smith ph. d.` as `John Smith PH. D.` on the default path, not only under force, and that is what forced the removal. The other, found by the docs review of the fix-up (2026-09-22) and not at the removal, was to read a bare trailing `Ph.` — no `D.` behind it — as a credential: `John Smith Ph.` gave family Smith, suffix Ph. on every release from 1.4.0 through 2.3.0 (wheels measured), and out of the set it falls to the positional read, middle Smith, family Ph., suffix empty; `Smith, John Ph.` moves the same way, to middle Ph. The cost is that CLASS and not one spelling, measured 2026-09-22 on HEAD and on the five wheels with `Lexicon.default().add(suffix_acronyms={"ph"})` as the comparator: the acronym test strips periods, so the undotted `John Smith Ph`/`PH`/`ph` move identically; inside a credential run `Ph.` no longer joins it, so `John Smith Ph. MD` reads family Ph., suffix MD, and `John Smith MD Ph.` reads middle `Smith MD`, family Ph., the real credential leaving the suffix field with the fragment; `John Smith Ph. Jr.` keeps only the Jr.; and the comma form `Smith, Ph.` reads title Ph. where 2.2.0 and 2.3.0 read suffix. `John Smith P.H.` does not move, the by-shape class reading it either way. Every shape in the class is a bare `ph` with no `D.` behind it, which nobody writes — the same criterion as esq's, asked of the machinery rather than a surname: the entry described the merge, not a word. Classified a behavior change, a 2.x parity break, on all five ledgers (`change(suffix-acronym-collisions) ph leaves the acronym set`), with the cases.py row `removed_ph_fragment_leaves_a_bare_trailing_ph_a_name_word` written to carry the plain shape; the ledger rule is literal to that one name, so a future row on any other shape of the class arrives unexplained at every baseline and is classified by widening that rule, not filed as a regression. No corpus name had any shape of the class, which is why the removal measured as 0 role movers at review time (2026-09-11, 1143 names), the corpus-blindness AGENTS.md warns of. Under `capitalized()` the 13 `PH. D.` shapes the repair had moved under `force=True` go back to `Ph. D.`; `john smith phd` and `ph.d.` still give `Ph.D.`. - -### suffix-acronym-collisions — the trailing-position collision class, decided (2026-09-07, #342/#454) - -Closes #342 (a wordlist question) and #454 (a rules.md question) together, because they are the same question asked of two different words and answering one without the other would leave the criterion half-stated. No parser code moves in either. Two existing forks carry the whole thing and both were confirmed on the pre-bundle tree with a throwaway override before any wordlist was edited: `Parser(lexicon=Lexicon.default().remove(suffix_acronyms={"rai","cha"}).add(suffix_acronyms_ambiguous={"ba"})).parse(text)`. - -- **The criterion, and it is a comparison of RELATIVE FREQUENCY.** #vocabulary-collisions C-i decides whether a word is borne as an ordinary name in the position the vocabulary claim acts on, and answers ship or mark. This entry adds the third answer and says when to reach for it. Being borne at all is only C-i's entry ticket; what chooses among the three answers is how common the word is as a borne name in the TRAILING position weighed against how common it is as a credential. Three outcomes. The NAME reading dominating removes the entry from the vocabulary altogether — a caller who needs it adds it back, which is the answer this file already gives for every locale-specific vocabulary — and that is rai and cha, far more common as surnames than RAI and CHA are as credentials. Rough balance earns the ambiguous marking, both readings being genuinely live so that a flag is the honest output, and that is ba, about as common a surname as BA is a credential. The CREDENTIAL dominating leaves the entry unambiguous, which is where the great majority of the set sits. LENGTH is a CORRELATE and not the test: a short acronym is more often a common credential and more often a name, which is why the four entries carrying the marking before this one — ma, do, ed, jd — are all two letters, and reading that as the rule is what this bullet exists to stop. -- **rai and cha left SUFFIX_ACRONYMS.** RAI is "RETA Authorized Instructor"; CHA is Certified Hotel Administrator or Certified Healthcare Auditor. Rai is a common surname across Hindi- and Bengali-speaking regions and Cha is the Korean 차. Both entries arrived in the 2019-12-11 bulk wikipedia post-nominal import (af5bdab) and neither was reviewed against a surname. Measured 2026-09-07: "Aishwarya Rai" reads family Rai where 2.0.0 through 2.2.0 read suffix Rai with no family name at all; "Lala Lajpat Rai" reads middle Lajpat, family Rai; "Kim Cha" reads family Cha. This SUPERSEDES the #vocabulary-collisions sentence "rai — Rai IS a borne surname, so it earns the marking rather than moving", which has a dated supersession note there rather than a rewrite; the sentence was right that Rai is borne and wrong about which of the three answers that earns. -- **The accepted cost, stated rather than discovered.** "John Smith RAI" reads family RAI with Smith a middle name, and "Ahmad Jayadi, CHA" reads given CHA, family "Ahmad Jayadi" — with no credential after the comma, C1 reads an ordinary family comma. Both are case rows, so the reversal is one measurement away for anyone who wants it back. The judgment behind accepting it: a wrong credential reading destroys a real person's surname on every record they appear in, and a missed credential reading leaves the letters in a name field where a human can still see them. -- **The roman-numeral accident, recorded because it looks like a survival and is not.** "John Smith R.A.I." still reads suffix R.A.I. after the removal — but by ACCIDENT, not by vocabulary: rules.md#S3 splits a token with interior periods on those periods and reads the chunks (`_pipeline._vocab.period_joined_vocab`), and the last chunk `i` is a Roman numeral in SUFFIX_WORDS, so "John Smith X.Y.I." and "John Smith R.A.V." read suffix while "John Smith R.A.X." does not — measured 2026-09-07, `period_joined_vocab('R.A.I.')` returning "suffix" and the chunks being r, a, i, never `ra`. "John Smith C.H.A." shows the other half: it read suffix BEFORE the removal, the lookup fold stripping the periods to reach the cha entry, and reads family C.H.A. after. Nothing is pinned on the accident; it is evidence for the parking lot below. -- **ba joined SUFFIX_ACRONYMS_AMBIGUOUS.** BA is a common credential and Ba a real surname (Vietnamese; Senegalese Fula) — the ma/Ma shape exactly. Measured: "Anna Ba" reads family Ba with a suffix-or-name flag (1.4.0's reading plus the flag), "John Smith BA" keeps suffix BA and is newly flagged, "John Smith B.A." is an unflagged suffix through the period gate, and "Smith, BA" reads given BA — the "Smith, Ed" cost, already Accepted under S2 and not reopened here. No corpus name carries a trailing ba, so nothing moves in the differential. -- **mai left SUFFIX_ACRONYMS (2026-10-16, #687).** The relative-frequency criterion above, asked of a third word and answered as rai and cha were. MAI is "Member, Appraisal Institute", a specialized US real-estate designation; Mai is a Vietnamese given and family name, a Chinese surname (麥) and a Japanese given name. `tools/name_usage.py Mai`: US Census 2010 17,445 family-name bearers (rank 2,075), SSA 5,697+ given-name bearers, Wikidata 822+ family and 458+ given. The entry arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. Measured: `Tran Thi Mai` under `FAMILY_FIRST_GIVEN_LAST` read suffix `Mai` with no given name (it now reads given `Mai`, middle `Thi`, family `Tran`), and `Nguyen Mai` read suffix `Mai` under every order (family `Mai` by default, 1.4.0's reading). The accepted cost is `John Smith MAI`, which reads middle `Smith`, family `MAI`, as `John Smith RAI` does; `Smith, John, MAI` still reads suffix (reported `comma-structure`) and `John Smith M.A.I.` still reads suffix by the dotted shape (rules.md#S3). A caller who needs it back adds it: `Lexicon.default().add(suffix_acronyms={"mai"})`. Marking it ambiguous, the `ba` outcome, was the alternative and was not taken: a trailing `Mai` is a name far more often than a credential. One corpus name moves, `Tran Thi Mai` under `FAMILY_FIRST` (snapshot, 2026-10-16). -- **#454 is BY DESIGN, and the words are mc and vd.** mc has no vowel and is not a borne name — which reverses #342's comment, where mc is listed among the entries "borne as surnames" and is the one member of that list its model-recall hedge does not cover; #360 measured it: mc and ste are contractions of Mac and Sainte, borne in neither position, which is why both left the ambiguous particle half. The Scottish prefix never detaches from the name it belongs to, so "Donald Mc" is not a name anyone writes and its suffix reading costs nothing real. vd needs a surname after it, so a bare trailing vd is the decoration. Neither joins the ambiguous subset and the `fix(suffix-routing)` ledger rule — a two-token name ending in a credential acronym keeps it in `suffix` — stays. What #454's example actually turns on is a different membership: "Mc Donald" reads family "Mc Donald" and "John van Mc" reads family "van Mc" because mc is a never-given PARTICLE (#360), which is why the trailing bare shape is the only one at issue. -- **What #454 got right, and it is a rules.md defect that is now fixed.** rules.md#P6's Accepted clause said that under the default order a comma-less name keeps its positional reading, with "Jong Anke de" → family "de" as its only example. Measured, that holds for de and for do and fails for exactly the two words that are in BOTH the particle vocabulary and the UNAMBIGUOUS suffix vocabulary: "Donald mc" and "Smith vd" lose the family entirely to `suffix`. The clause is repaired to say so, and the two places that already conceded it in prose — the `'Donald mc'` bullet of #differential-ledger's fields-only arc and the corresponding comment in expected_since_1.4.0.toml — are amended from "recorded here rather than fixed" to point at the repair. No example line moves: the clause's own example is de-shaped and stays true, which is why no test caught the defect and why regenerating corpus_rules.jsonl is a no-op here. Recompute the membership with `L = Parser().lexicon; sorted(L.particles & (L.suffix_acronyms - L.suffix_acronyms_ambiguous))`, which gives ['mc', 'vd'] on 2026-09-07, and `sorted(L.particles & L.suffix_acronyms_ambiguous)`, which gives ['do']. -- **Parking lot: the shape-plus-position credential heuristic.** #490's idea without the "collides with vocabulary" part, and what would recover "John Smith RAI" without any wordlist at all. Two shapes: an ALL-CAPS acronym standing in the suffix position of a MIXED-CASE name ("John Smith XYZ", "John Smith, XYZ"), where the case contrast is the signal; and a DOTTED acronym in that position regardless of case ("John Smith X.Y.Z.", "john smith x.y.z."), where the periods are. Measured 2026-09-07, none of the four is read as a suffix today — the bare form gives family XYZ, the comma form given XYZ with family "John Smith", and the second two family X.Y.Z. — while the roman-numeral accident above reads three dotted forms as suffixes on a fork that is about generations, not credentials. That contrast is the evidence the dotted shape is unhandled: the parser already produces the wanted answer for X.Y.I. and R.A.V. by coincidence and the unwanted one for R.A.X. A follow-up issue carries it; nothing in this entry depends on it. -- **esq left SUFFIX_ACRONYMS (2026-09-08, the #316/#489 bundle).** The same criterion asked of a word the frequency argument cannot reach: does the entry describe the WORD or the machinery. Esquire is a contraction, not an initialism, so the initialism set was the wrong home for it whatever its frequency — this is the criterion's third kind of answer, and it is the one rai and cha did not need. The entry arrived with the same 2019-12-11 bulk import (af5bdab, #93) that brought rai and cha and was never reviewed either; its only unique coverage was the multi-dot spelling `E.S.Q.`, the SUFFIX_WORDS membership carrying every single-token spelling. Measured over the corpus glob: one name moves, `John Smith E.S.Q.`, from suffix to family, at every baseline. What the removal buys is the invariant this entry's neighbours could not have: `SUFFIX_ACRONYMS ∩ SUFFIX_WORDS == ∅` is asserted at import, esq having been the only member of that intersection and the reason the assert could not exist. Recompute the intersection with `L = Parser().lexicon; sorted(L.suffix_acronyms & L.suffix_words)`, which gives `[]` on 2026-09-08. The standing keep-out is in the Excluded block above. -- **Not decided here:** sa stays where #296's audit put it (title and suffix dual, position decides); se and om stay unambiguous, there being no surname evidence worth standing behind and OM being the Order of Merit; the S2 comma-form boundary ("Smith, Ed" → given Ed) is not reopened. #342's own comment lists sa, se and om beside ba and cha as model recall rather than corpus-attested, and per the #360 lesson they need a human before they move. -- **Measurement (2026-09-07).** Five corpus names move, every one from the radar-tier corpus_issues.jsonl, in three diff shapes with one cause: Aishwarya Rai moves {family, suffix}, Lala Lajpat Rai and John Smith RAI move {middle, family, suffix}, and John Smith, RAI and Ahmad Jayadi, CHA move {given, family, suffix}. Recompute by parsing every name in the tools/differential/corpus*.jsonl glob twice — once with the shipped lexicon, once with `Lexicon.default().add(suffix_acronyms={"rai","cha"})`, which RESTORES the two entries this bundle removed so the diff read forwards is the removal's — and diffing the seven name fields plus `ambiguities`; the ba step of the same sweep moves nothing. Aishwarya Rai is the one name whose diff DISAPPEARS at 1.4.0, that release having read family Rai too, which is why the `fix(#342) NOT WANTED` ledger rule was deleted rather than rewritten and why the 1.4.0 gate lists four names under the replacement where the three 2.x ledgers list five. Read today's intentional counts off the `corpus:` line of `uv run python tools/differential/compare.py --baseline X`; they rose by three at 1.4.0 and by five at each 2.x baseline. -- **2026-09-15 (#289/#516) — the parking lot is CLOSED, and this entry's "measured 2026-09-07, none of the four is read as a suffix today" sentence is superseded rather than rewritten.** The shape-plus-position heuristic this entry parked is what shipped: the DOTTED shape as `Policy.unlisted_dotted_suffixes`, default on, and the ALL-CAPS shape as `Policy.unlisted_caps_suffixes`, default off. Of the four names the bullet measured, `John Smith X.Y.Z.` and `john smith x.y.z.` now read suffix at the default, and `John Smith XYZ` / `John Smith, XYZ` read suffix only with the opt-in switch. See #S2 for the whole decision and for why the two halves have different defaults. -- **2026-09-15 (#289/#516) — the roman-numeral accident is RETIRED, and here is what narrowly means.** This entry's bullet recorded the accident as evidence rather than as behavior anyone relied on. It now fires no longer where every chunk the vocabulary matches is a single ASCII character — measured, `{i, v, 2}` — so `John Smith R.A.I.` and `John Smith X.Y.I.` reach the shape class instead and read by position, which gives the same fields for the first and 1.4.0's family reading for `Jack X.Y.I.`. `John Smith C.H.A.`, the bullet's other half, is unaffected: nothing claimed it before or after. The WIDE retirement was measured and rejected; #S2 records the three names that pay for it. -- **2026-09-25 (#540) — meng and lac joined SUFFIX_ACRONYMS_AMBIGUOUS (Derek: the marking over removal, and no masks).** #vocabulary-collisions C-i asked at the position the suffix claim acts on, the last word of a name: Meng (孟) is a common Chinese surname and given name, and Lac (Lạc) is a Vietnamese given name — the trailing word in native order, `Nguyễn Văn Lạc` — and a French surname; both are borne in that slot (`wang meng`, `tran lac`, `nguyen van lac`). MEng (Master of Engineering) and LAc (Licensed Acupuncturist) are live credentials, and neither reading is rare enough beside the other to give it the word — the rough balance this entry's criterion marks (where it is uncertain, C-i's default is the marking too). They are the set's first entries longer than two letters, which is the LENGTH clause of the criterion bullet applied, not an exception to it. Neither was checked against a surname when it arrived: `lac` with the af5bdab import (2019-12-11, #93), `meng` with 3e14ea20 (2026-07-01, the German/Dutch title and degree batch, first shipped in 1.3.0). No rule changes: rules.md#S2 reads the two words exactly as it reads `ma` and `ba`. Measured 2026-09-25 on this tree, the parent 120033b5 and the five released wheels (each wheel run as the Gotchas in AGENTS.md prescribe, from a directory outside the checkout under `PYTHONSAFEPATH=1`, asserting `nameparser.__file__`). THE FIX: `wang meng`, `li meng` and `tran lac` read given + family with a `suffix-or-name` report, where every release from 2.0.0 read suffix and no family name — reporting nothing at 2.0.0 through 2.2.0, and `given-or-family` at 2.3.0 and the parent, the one-word-name fork, which was the wrong fork — and 1.4.0 read the family, unflagged. The family-comma spelling returns to 1.4.0 with them, `Wang, Meng` reading given `Meng`, family `Wang` where 2.0.0 through 2.3.0 read suffix `Meng`; and under `FAMILY_FIRST` the bare `Wang Meng` reads given `Meng` where 2.0.0 through 2.3.0 and the parent read family `Wang`, suffix `Meng`, so the comma spelling under the default order and the bare one under `FAMILY_FIRST` still agree with each other. WITH WORDS TO SPARE the credential stays: `john smith meng`, `JOHN SMITH MENG` and `nguyen van lac` keep their suffix, now reported, and so does `john smith m.eng.`, which S2's period gate does not settle, that gate counting one period after each letter. DECIDED (Derek, 2026-09-25): the bare `Wang M.Eng.` reading — family `M.Eng.`, nothing to spare — is accepted and recorded rather than widening the period gate to cover a chunked dotted spelling; S2's period gate counts a member as unambiguous only when it is written one period per letter (`M.A.`), and MEng and LAc are the first members whose conventional dotted spelling is chunked instead. `John Smith M.Eng.` keeps the suffix by the words-to-spare count, unaffected. THE ACCEPTED COST is the one `ma` and `ba` already carry, and it falls on the conventional spellings: in a name written in more than one case S2 reads a member written neither in capitals nor all lower as the name even with words to spare, and MEng and LAc are written exactly that way. `john smith MEng` reads middle `smith`, family `MEng`, where every release from 1.4.0 read suffix `MEng`; `John Smith Meng` and `John Smith LAc` move the same way; and the Title-case `Nguyen Van Lac` reads family `Van Lac` where every release read family `Van`, suffix `Lac` (for this Vietnamese name the gain, the mechanism being the cost's) — only the all-lower `nguyen van lac` keeps its suffix, one case saying nothing. A CREDENTIAL RUN ending in one of them pays the same walk-stopping cost: `Mary Jones PhD MEng` reads middle `Jones PhD`, family `MEng`, and `Jane Doe MS LAc` reads middle `Doe MS`, family `LAc`, where every release read the whole run as a suffix (`PhD, MEng` and `MS, LAc` through 2.2, `PhD MEng` and `MS LAc` in 2.3); `John Smith MEng PhD` reads middle `Smith`, family `MEng`, suffix `PhD`, the unambiguous credential behind the declined pick still peeled; the mechanism is S2's own — the case signal costs a genuine suffix standing behind a name-leaning acronym, the walk stopping at the declined pick rather than continuing past it — pinned by `tests/v2/cases.py`'s `a_declined_ambiguous_pick_stops_the_walk` and, for the family-comma shape, `a_declined_member_ends_the_trailing_run` (`Doe, John MA Ma`). A QUIETER COST sits behind a suffix comma: once the run's last word is a bare ambiguous member, the run is no longer wholly suffix-shaped, so C1 reads the comma as a family comma instead and the run's own words fall into the usual post-comma name slots — `John Smith, PhD MEng` reads given `PhD`, middle `MEng`, family `John Smith`, and `john smith, phd meng` reads given `phd`, family `john smith`, suffix `meng` (the count then peels `meng` with a word to spare); neither case form saves it, `JOHN SMITH, PHD MENG` reading the same way. A title-listed word standing in front of the run reads as a TITLE instead, with no report: `Jane Doe, MS LAc` reads title `MS`, given `LAc`, family `Jane Doe`, unflagged, where every release read suffix `MS LAc`. The path is not new: `john smith, phd ma` and `Jane Doe, MS Ma` already read this way, `ma` having been ambiguous all along. Pinned by `tests/v2/cases.py`'s `comma_credential_run_ending_in_meng_re_reads_the_comma` and `comma_lower_credential_run_ending_in_meng_re_reads_the_comma`. The comma re-read (with its silent title-dual case) and whether a listed member's chunked dotted spelling should pass the period gate are [#544](https://github.com/derek73/python-nameparser/issues/544)'s questions. After a comma, `Smith, MEng` reads given `MEng` (1.4.0's reading; 2.0.0 through 2.3.0 read suffix) — and so does the all-lower `Smith, meng`, a restored reading rather than a cost — and `Smith, John MEng` middle `MEng`, pinned as case rows (`Smith, meng`, `Smith, MEng`, `Smith, John MEng`) and classified by the `fix(#540)` rules, while `John Smith, MEng` keeps the suffix on C1's name-word count; bracketed, `John Smith (MEng)` reads nickname `MEng` with a `suffix-or-nickname` report, S1's escape declining an ambiguous member, where every release read suffix `MEng`. `John Smith MENG` keeps the credential on the capitals lean. The judgment is the one behind rai and cha: a missed credential leaves the letters in a name field where a human can still see them, and a wrong credential reading destroys a surname on every record it appears in. edd and ded are NOT marked: both are borne as GIVEN names (`Edd Smith`, `Ded Gjo Luli`), which is not the position the suffix claim acts on, and no measurement shows either losing a family name — a trailing bearer turning up reopens them. Recomputed with #vocabulary-collisions' recipe on 2026-09-25: suffix_acronyms 608 (unmoved by this change) with 7 ambiguous, 5 before this change (particles 70 with 37 and titles 758, unmoved). -- **Declined 2026-09-25 (#540), with measurement: removing the two words, and masking them.** REMOVAL, the rai/cha answer, measured with `Parser(lexicon=Lexicon.default().remove(suffix_acronyms={"meng", "lac"}, suffix_acronyms_ambiguous={"meng", "lac"}))`: `john smith meng` reads middle `smith`, family `meng`, and `JOHN SMITH MENG` family `MENG`, while `john smith m.eng.` stays a suffix by its dotted shape (S3), reported. Removal loses the bare credential outright where the marking keeps it with words to spare, and C-i's default under uncertainty is the marking. MASKS `meng → MEng` and `lac → LAc`, measured with both added to a private lexicon on top of the marking: the exceptions map is role-free (#R4, 2026-09-23), so `wang meng` renders `Wang MEng`, `tran lac` `Tran LAc` and `meng li` `MEng Li` — the mask re-spells the very surnames the marking restores. A suffix-gated mask would reverse the 2026-09-23 role-free decision for two words and is not taken. Without one, the default path renders `wang meng` as `Wang Meng` (the parent gave `Wang MENG`, the credential clause writing a suffix in capitals), `tran lac` as `Tran Lac`, `john smith meng` as `John Smith MENG` and `nguyen van lac` as `Nguyen Van LAC` by the acronym clause, and keeps a writer's `MEng` as written (#R5, 2026-09-24) — `john smith MEng` gives `john smith MEng`, and `John Smith Meng` under `force=True`, the word being the family name there. Pinned by `tests/v2/test_render.py::test_a_listed_acronym_that_is_a_name_word_gets_no_mask`, whose recorded negative control carries the two masks. -- **Measurement (2026-09-25, #540).** No corpus name written before this change carries a bare trailing `meng` or `lac` (the `tools/differential/corpus*.jsonl` glob at 120033b5), so the population that could move is this change's own shape-tagged case rows, and every mover is one of them. Classified by `fix(#540)` rules: at each 2.x baseline `wang meng` and `tran lac` move {family, suffix, _ambiguities}, `john smith MEng` and `John Smith MEng PhD` move {middle, family, suffix, _ambiguities}, and `john smith meng` and `nguyen van lac` `_ambiguities` alone; `Smith, John MEng` moves {middle, suffix, _ambiguities} at 2.x and {middle, suffix} at 1.4.0; `Smith, meng` and `Smith, MEng` move {given, suffix, _ambiguities} at 2.x and nothing at 1.4.0 (1.4.0's own reading); `John Smith, PhD MEng` moves {given, middle, family, suffix, _ambiguities} and `john smith, phd meng` {given, family, suffix, _ambiguities} at 2.x (without `_ambiguities` at 1.4.0); `Nguyen Van Lac` moves {family, suffix, _ambiguities} at 2.x and {family, suffix} at 1.4.0; and `Wang M.Eng.` moves {family, suffix, _ambiguities} at 2.x only, 1.4.0 reading the same family for an unrelated reason (its two-piece rule -- a lone word after the given name is the family -- not S2's gate). `nguyen van lac` also diffs at 1.4.0 on `_initials` (`n.` → `n. v.`) for fix(#385/#402)'s reason — its family `van` is a one-particle part — and joins that rule's literal list; the same case-insensitive regex also reaches `Nguyen Van Lac` at every baseline, without explaining anything there (its own family move, at 1.4.0 too, is explained by the Title-case Lac rule instead). Read today's intentional counts off the `corpus:` line of `uv run python tools/differential/compare.py --baseline X`; this change raised them by seven at 1.4.0 and by thirteen at each 2.x baseline. -- **2026-09-27 (#544) — #540's two pending questions are answered, and the #540 bullet stands as it landed.** The chunked dotted spelling passes S2's period gate — `Wang M.Eng.` reads suffix 'M.Eng.' as `Wang M.A.` does, and as 2.0.0 through 2.3.0 read it — and the comma is kept: a credential run after a full name that ends in a member is the credential run by C1's name-word count (`John Smith, PhD MEng` suffix 'PhD MEng'; `Jane Doe, MS LAc` suffix 'MS LAc' where #540 left title 'MS', given 'LAc', at the cost, accepted under S2, of `John Smith, Ms Ma` reading suffix 'Ms Ma'). The walk-stopping cost that bullet lists for a run ENDING in a member goes with them, by S2's company clause: `Mary Jones PhD MEng` and `Jane Doe MS LAc` read the whole run as a suffix again. Where the member LEADS the run nothing speaks for it, so `John Smith MEng PhD` keeps its cost. The `fix(#540)` ledger rules for the comma run and the dotted spelling are retired. `fix(#544)` rules classify the comma-run names; `Wang M.Eng.`, whose roles every release from 2.0.0 read, is classified by `feat(#449)`'s given-or-family report rule at 2.0.0 through 2.2.0 and by the dotted `fix(suffix-routing)` rule beside `Jack M.A.` at 1.4.0, and differs from 2.3.0 not at all. The case rows the #540 bullet cites were renamed with the readings they pin: `a_declined_ambiguous_pick_stops_the_walk` is `a_credential_in_front_anchors_a_declined_pick` (`abdul Smith Jr Ma`, suffix 'Jr Ma'), and `comma_credential_run_ending_in_meng_re_reads_the_comma` and `comma_lower_credential_run_ending_in_meng_re_reads_the_comma` are `comma_credential_run_ending_in_meng_keeps_the_suffix_comma` and `comma_lower_credential_run_ending_in_meng_keeps_the_suffix_comma`. The decision and its measurements are the #544 entry under S2. - -### S2 — the case signal at the suffix slot - -- 2026-09-14 #289 + #516 (this PR) — A BARE AMBIGUOUS ACRONYM IS READ BY THE EVIDENCE THE WRITING CARRIES, and written case is evidence the words-to-spare count does not have. In a name written in more than one case, a member of `suffix_acronyms_ambiguous` written in CAPITALS leans credential and is taken as a suffix with nothing to spare (`Jack MA` → given 'Jack', suffix 'MA', as `Jack MD` has always read); one written in any other cased form that is not wholly lower leans SURNAME and stays a name word with plenty to spare (`John Smith Ma` → middle 'Smith', family 'Ma'). An all-lower member carries no lean, a name written wholly in one case carries none at all, and both take the positional rule unchanged — `JOHN SMITH MA` suffix, `ANH DO` family, `john smith ma` suffix. Switch-free and scoped to the listed set: `ba do ed jd ma`. - THE PREDICATE KEEPS THE JUDGED TOKEN IN THE SPAN, and that is what gives the right answers here. `JOHN SMITH Ma` is MIXED because of `Ma`, and a Title-case word against all-caps neighbours is exactly the surname signal this design wants; excluding the judged token would make the name one-case and hand `Ma` to the positional rule. Same for `Jack MA` read the other way — included, the name is mixed and the capitals lean credential; excluded, `own == ['Jack']` is one case and the peel keeps the family. The general question — whether a cased suffix token counts as case evidence — is #492's and stays deferred; this entry records that the INCLUDING reading is what this design rests on. -- 2026-09-14 (Derek) — THE LEAN REACHES THE POST-COMMA GIVEN SLOT: `Smith, MA` → family 'Smith', suffix 'MA', while `Smith, Ma` and `Smith, ma` stay given. Positive evidence outranks position there, so the credential reading fires with ONE word before the comma where the positional rule would not. - AND THE COUNT AT A COMMA IS OF NAME WORDS. For this class, "words to spare" in `X, Y` form means the part before the comma holds two or more NAME words: `Smith Jr., MA` has two tokens and one name word, and a token count hands its family to `given` (measured on the prototype). rules.md#C1 already said both things — "more than one word precedes the comma" in its opening clause and "a part before the comma with more than one name word" later — so the doc edit is a reconciliation inside C1 rather than a new claim. -- 2026-09-15 (spec review) — THE COUNT REACHES THE LISTED SET TOO — one uniform rule for the whole ambiguous class rather than two. `John Smith, Ed` → given John, family Smith, suffix Ed; `john smith, ma` and `JOHN SMITH, MA` move the same way, each being a one-case name that takes the positional rule, which now reaches the comma structure. All three are 1.4.0 RESTORATIONS — measured on the wheel, v1 reads all three as suffixes — so today's tree is what deviates and this restores parity rather than opening distance from it. Corpus population: ZERO (verified by grepping `tools/differential/corpus*.jsonl` for all three; `Royce, Ed` is the corpus's only member of the sibling one-word shape), so no corpus mover count moves and `Davis Royce, Ed` is the case row that carries the decision. -- 2026-09-17, corrected — A CASELESS SCRIPT IS INERT ONLY AS A LEAN, NOT AS THE NAME-WORD COUNT. `is_one_case` answers True trivially for text with no case to write a contrast in, so `ambiguous_lean` returns `None` for `毛泽东, MA`, `마틴 킹, MA` and `田中 太郎, MA` alike — the LEAN really is inert by construction, as first written. But the comma count above is of NAME words "whatever case the name is written in", which is orthogonal to case and reaches a caseless script the same as any other: `마틴 킹, MA` and `田中 太郎, MA` each hold TWO pre-comma name words, so the structure flips and both read suffix 'MA' — a PARITY RESTORATION, not a fresh deviation (1.4.0 read both as suffix on the wheel), with the Hangul/Han surname split then running positionally over what the comma leaves (`마틴 킹` → given '틴', middle '킹', family '마'; `田中 太郎` → given '太郎', family '田中'), unrelated to this design and #271/#272's pre-existing behavior. `毛泽东, MA` does not move — one pre-comma token, the count never reaches two. Pinned `a_caseless_script_wrote_no_contrast_hangul` and `a_caseless_script_wrote_no_contrast_japanese` (tests/v2/cases.py), both `tolerated=True`. -- 2026-09-17, corrected — THE COMMA-PATH REPORT TRACKS THE FORK CONSULTED, NOT THE LEAN. A first pass gated the family-comma report on whether `ambiguous_lean` fired, which wrongly silenced `Smith, ma` (all-lower, no lean) and `毛泽东, MA` (caseless, no lean) — the TRAILING slot has never gated its own report on the lean either (a bare ambiguous acronym reported before #289 too, pick declined or not), because the report is about a fork the parse CONSULTED, and the fork is consulted whenever a class member sits in the slot, lean or no lean. Fixed: the guard is class MEMBERSHIP on the first post-comma piece (`vocab:suffix-ambiguous` or `shape:acronym`, never the lean) — `Smith, ma` reports `"'ma' after the comma is also an ordinary name word; read as the given name"` (since #626, 2026-10-08: "read as a given name", the field worded from where the word lands), `毛泽东, MA` reports the same wording for `'MA'`, and `Smith, MA PhD` (a two-piece post-comma part) reports through its FIRST piece regardless of what follows it, exactly as a one-piece part would. The structure decision reports in `segment`, where that branch is taken, and the family-comma reading in `assign`, so one DECISION never reports twice — a second ambiguous token elsewhere is a second fork and reports on its own (`Smith MA, Ed` carries two `suffix-or-name` reports, one per fork, plus a `given-or-family` for the lone remaining word). -- 2026-09-14 (Derek) — #516's DOTTED HALF IS A SWITCH, `Policy.unlisted_dotted_suffixes`, DEFAULT ON: an unlisted token of two or more period-separated chunks, any chunk length (`X.Y.Z.`, `B.Tech.`, `Q.W.E.R.T.`), joins the ambiguous class BY SHAPE; whole-token vocabulary still wins (`M.A.`, `Ph.D.`, `A.B.C.` — `abc` IS a suffix acronym). The positional rule then decides and reports either way: `John Smith X.Y.Z.` → suffix, `Jack X.Y.Z.` → family, `Smith, A.B.` → given, `John Smith, A.B.` → suffix. Case is irrelevant — the periods are the evidence — so `john smith x.y.z.` reads as its mixed-case twin. Switch OFF: name material everywhere as 2.3 read it, and the fork is STILL reported, the parser having chosen the name reading over a credential one. -- 2026-09-15 (Derek) — THE ROMAN-CHUNK ACCIDENT RETIRES, NARROWLY. rules.md#S3's chunk rule survives except where every chunk the vocabulary matches is a SINGLE ASCII CHARACTER — the accident exactly. Measured, `{w for w in (L.suffix_acronyms | L.suffix_words) if len(w) == 1}` is `{i, v, 2}` plus the glued CJK honorific tails `{様, 殿, 氏, 군, 님, 씨, 양}`, so "single ASCII character" names the first three and only those — CHARACTER because `2` is a digit and is in the set, ASCII because `씨` is the one that must KEEP its chunk claim (`J.씨`). So `John Smith R.A.I.` and `John Smith J.u.n.i.o.r.` reach the shape class and read suffix by POSITION with the same fields plus the report, `Jack X.Y.I.` reads family where it read suffix (1.4.0's reading), and `Msc.Ed.`, `JD.CPA`, `Lt.Gov.`, `J.씨` keep their chunk-derived readings. The WIDE retirement — any chunk match yielding to the shape — was MEASURED AND REJECTED: it moves three more corpus names, `Doe, John Msc.Ed.` losing a real credential to `middle` and the two `김민준씨, J.씨` rows re-routing their honorific peel, for nothing this design wants. -- 2026-09-17/18, #516 review rounds — THE SHAPE VERDICT NEEDS TWO FURTHER GATES, both found by measuring after the shape verdict shipped. (i) ALPHABETIC: `period_joined_vocab`'s "shape" branch now requires every chunk to be `.isalpha()` — a bare digit chunk is not an acronym letter by any reading, so `Smith, 1.4`, `John Smith 1.4` and `John Smith, 1.4` stay unclaimed (no report, no role move), where an unguarded shape verdict had wrongly admitted them; the delimited `Bridge (1.4)` control only proved digits reach extract's escape, saying nothing about the same digits undelimited, which is the gap this closes. (ii) INITIALLESS-SCRIPT: the same test `is_title_shaped` already makes for H2's own leading-abbreviation inference (#323) — `text.isascii() or not in_initialless_script(text)` — now gates the shape verdict too, so `John Smith 田.中.`, `김 민준 이.박.` and `John Smith たな.か.` stay family: a script with no period abbreviations at all has nothing for interior periods to abbreviate, so a CJK word glued into period-separated characters is not spelling an acronym either. `John Smith 田.中.` is pinned `tolerated=True` rather than `shape=` — a Latin period convention glued onto CJK characters is a composed form no writing system produces, and the case table's own validator refuses a Latin shape tag on CJK text; `이.박.` and `たな.か.` are covered by direct `period_joined_vocab` assertions instead of a second corpus row each. -- 2026-09-17, #516 review round — `SMITH, J.R.` READS BY SHAPE FOR A REASON UNRELATED TO WHY IT DECLINES THE LISTED WORD. `_normalize` keeps interior periods ON PURPOSE (v1 parity, pinned 2026-07-17: `'J.R.'` must not collapse to the listed suffix word `'jr'`), so `ambiguous_class_member` correctly answers False for `Smith, J.R.`, `Smith, E.S.Q.` and `Smith, E.T., Jr.` for a reason that has nothing to do with #516. What #516 adds is a SEPARATE route: each chunk (`'j'`/`'r'`, `'e'`/`'s'`/`'q'`, `'e'`/`'t'`) is an unlisted single-letter chunk nothing in the vocabulary claims, so the token joins the class BY SHAPE instead, exactly as `'K.D.'` does in `J.A. K.D.`. The READING is unchanged in all three — one pre-comma name word is never enough to flip the structure — but each now gains `suffix-or-name`, the class having been considered and declined rather than never reached at all. - `is_wholly_suffix` NEVER READS THE BY-SHAPE CLASS. A first version admitted a by-shape member there unconditionally, reasoning it was needed for `John Smith, A.B.`'s structure flip; proved wrong by mutation — that flip already goes through `_segment.py`'s own `pre_comma_names >= 2` disjunct alone, and nothing else in the suite depended on the admission except this predicate's own two unit tests. Left in, the admission combined with C1's LEGACY token-count disjunct (which counts WORDS, not NAME words, and predates the name-word-count decision above) to flip `Smith Jr., A.B.` to given `'Smith'`, suffix `'Jr., A.B.'` with a self-contradicting "holds 1 name words" report. Fixed by dropping the admission outright: the by-shape class reaches the comma form ONLY through `_vocab.ambiguous_class_candidate`, never through `is_wholly_suffix`. `Smith Jr., A.B.` now reads given `'A.B.'`, family `'Smith'`, suffix `'Jr.'` — the same shape the LISTED class reads for `Smith Jr., Ma` and `Smith Jr., ma` (given `'Ma'`/`'ma'`, family `'Smith'`, suffix `'Jr.'`), not for `Smith Jr., MA` (given `''`, family `'Smith'`, suffix `'Jr., MA'` — the caps lean has already moved that one into the suffix run, measured on this tree; at 2.3.0 it read given `'MA'`, family `'Smith'`, suffix `'Jr.'`, the very shape `Smith Jr., A.B.` reads now) — with the report; classification `parity`, not `fix(#516)`: FULL PARITY on the 1.4.0 wheel (first A.B., last Smith, suffix Jr.), only the report being new. -- 2026-09-14 (Derek), corrected 2026-09-18 — #516's ALL-CAPS HALF IS A SWITCH TOO, `Policy.unlisted_caps_suffixes`, DEFAULT OFF, and the asymmetry is the decision. An unlisted all-caps word of two or more letters, with no period ANYWHERE (`isalpha()` rejects a period anywhere, not merely an interior one, so a single TRAILING period, H2's own shape, is unaffected either way), in a name `one_case` says is written in more than one case, joins the class by shape (`_vocab.caps_shape_candidate`). - UNLISTED MEANS IN NO WORDLIST AT ALL, not merely "no whole-token suffix vocabulary" — found only by measuring after a narrower, tag-based guard shipped and reversed: `titles`, `given_name_titles`, `particles`, `particles_ambiguous`, `conjunctions`, `conjunctions_ambiguous`, `bound_given_names`, `suffix_acronyms`, `suffix_acronyms_ambiguous`, `suffix_words` and `maiden_markers` — ELEVEN lists, checked by direct membership rather than by tag, since `segment` calls the shape test before `classify` has tagged anything. Measured with the narrower guard alone (`John Smith DE`, `John Smith NEE` wrongly moved to suffix), and stated here as a recipe rather than a fixed count, since a raw per-list membership tally and a de-duplicated one (a word appearing in two lists, `abu`/`أبو`/`ابو` across `bound_given_names` and `particles_ambiguous`, is one leak, not two) give different totals: `len([w for w in getattr(lexicon, name) if w.isascii() and len(w) >= 2 and w.isalpha()])` per excluded list, run over the shipped `Lexicon.default()`, gives on the order of dozens for `particles` and `particles_ambiguous` and single digits for `conjunctions` and `bound_given_names` — every one of them a wordlist entry the caps shape test's earlier, narrower guard did not check at all. The one figure that reproduces exactly, because ASCII case is what the test needs and every other maiden marker is a non-Latin or non-ASCII spelling the shape test's own `isupper()`/`isalpha()` gate excludes on its own: FOUR maiden markers are ASCII (`geb`, `geboren`, `geborene`, `nee`), out of `len(lexicon.maiden_markers)` total (17, measured 2026-09-18) — `'NEE'`/`'GEB'` are two of those four, capitalized, which is what the narrower guard let through. ACCEPTED GAP, recorded rather than chased: an all-caps German title spelled with `SS` for `ß` (`GROSSFÜRST`) is unlisted by this same lookup too, for a reason predating this switch entirely — `_normalize` uses `lower()`, never `casefold()`, specifically so it does not corrupt `großfürst` to the misspelling `grossfürst`, and classify's own existing `vocab:title` check has never recognized this spelling either. Left for whoever next generalizes the ß-fold question. - THE RUN IS THE CAPS CLASS'S ALONE. The multi-token shape `'LEED AP'` needs a RUN test (`John Smith, LEED AP` → suffix `'LEED AP'`), but the listed and dotted halves are single-token classes by construction — the listed set is inherently bare words and the dotted half never needed more than one token — so testing the run over either of them asks a question this design never posed. An un-narrowed run test measured three reversals before this was caught: `John Smith, Ed Ma` (both listed, Title-case, which #289's OWN lean reads as NAME), `John Smith, ma do` (both listed, lower) and `John Smith, X.Y.Z. A.B.` (both dotted) all wrongly moved to suffix as a run. Narrowed to the caps branch alone; `Smith, LEED AP` still declines and reports, and all three reversals are now pinned identical OFF/ON. - `JACK VI` AND `JOHN SMITH VI` DO NOT MOVE, AND NOT FOR THE REASON A FIRST DRAFT GAVE. `VI` passes the caps shape test and carries both `SHAPE_ACRONYM_TAG` and `vocab:suffix-ambiguous` with the switch on — the predicate reaches it, contrary to what an early write-up of this entry said. What actually holds it unchanged is rules.md#S2's OWN roman-numeral fork (`_pieces.is_trailing_numeral_suffix`), which claims a bare `'VI'` piece and decrements the peel's walk BEFORE the bare-ambiguous/shape branches beneath it are ever reached — a mechanism this switch does not touch and never held `VI` back through non-membership. - THE COST IS A TWO-WORD CONTRAPOSITIVE, not a flat one. Off by default because French and Korean records write the SURNAME in capitals — `Jean DUPONT`, `Minjun KIM` and `Jean Pierre DUPONT` all read family at 1.4.0, at 2.3 and at this default — and `isupper()` is script-agnostic, so the same decline holds in ANY script with a case contrast (`Иван ИВАНОВ`, `Jean ÉCOLE`, measured). Turning it on does not cost every candidate alike: at TWO words there are no words to spare, so `Jean DUPONT` and `Minjun KIM` report the fork and KEEP their family — the class is considered and declined, the mirror of the listed lean's own words-to-spare rule. At THREE words there ARE, and `Jean Pierre DUPONT` gives family `'Pierre'`, suffix `'DUPONT'` — the one genuine cost, and a swallowed family name is the worse of the two failures #vocabulary-collisions C-i already argues from. Default-off is 1.4.0 parity for the whole class regardless; on, nothing but the report changes at two words, and only the three-word shape actually moves. Off, NOTHING is reported either. -- 2026-09-14 (Derek), scope corrected 2026-09-18 — NO NEW AmbiguityKind. Every decision at these slots emits the existing `AmbiguityKind.SUFFIX_OR_NAME` — no new kind — including the comma path, which is the first report of the comma's OWN decision (listing or credential run) in the library (`COMMA_STRUCTURE` already reports on the comma path, but as a shape the parse could not recognize, not a fork it called); the comma-quiet policy in `_types.py` and rules.md#C1 gain an exception scoped to the ambiguous class. Switch B off emits nothing. This is narrower than an earlier wording of this entry claimed: an attachment decided AFTER a family comma is a separate fork, P6's, and it has reported on its own since 2.3 — measured on this tree and at master `1f78bef` (2.3.0), `parse("Berg, Jan vd")` gives given `'Jan'`, family `'vd Berg'`, `ambiguities=['suffix-or-name']`, emitted from `_pipeline/_post_rules.py`'s attachment fork, and `AmbiguityKind.SUFFIX_OR_NAME`'s own docstring in `_types.py` already lists that family-comma case. - THREE READING SITES, NOT TWO. The trailing suffix slot (`_pieces.peel_trailing`/`listed_lean`) and the post-comma given slot (`_pieces.segment_suffix_reading`) are the two the decision names first; `_vocab.is_wholly_suffix` over a `groups[2:]` TAIL segment is the third, and part of the blast radius rests on it: `Steven Hardman, MD, DO, DDS` loses its `comma-structure` flag only because the lean makes segment 2 (`DO`) wholly suffix. That is the one place this arc QUIETS a report rather than adding one, and it has a case row of its own because a quieted flag is invisible to a reader sweeping for new ones. -- 2026-09-14, corrected 2026-09-17 and 2026-09-18 — WHERE THE FACT LIVES, AND WHY THERE. The comma forms need the class decision at SEGMENT time, two stages before the own-words span is computable from marker TAGS. Four options were measured on a runtime probe (baseline 410.00 frames): segment computing it with a direct marker-head walk, +15; the same re-running the whole marker pass, +16; the fact staying in classify while classify RE-DECIDES `structure`, +0 but refused ON ORDERING — `structure` is read between segment and classify by `_script_segment.py`, which returns early on FAMILY_COMMA, so a flip decided in classify would leave the surname-split gate un-run on a name that turns out to have a suffix comma (measured: `Smith 김민준씨, MA` reads given 'Smith', middle '김', family '민준', suffix '씨, MA' on this tree — the figure first written here, given 'MA' with family 'Smith 김민준씨', was the reading before the flip moved to segment, re-measured 2026-09-18 — and a DIFFERENT wrong split again with the flip decided before script_segment — two stages disagreeing about one parse is what mechanisms.md#PIPELINE-STAGE-CONTRACTS exists to forbid); the marker pass and the fact MOVING to segment, +8, refused because killing the duplicated fold means carrying it on the state through CJK splits, a new bug surface in the one stage that changes token COUNT. SHIPPED: one shared helper, `_pieces.own_words(tokens, comma_offsets, markers, marker_tags=None)`, called by classify always (passing its own tag map) and by segment only when a comma form could turn on the fact. Every figure below is a TOTAL against `1f78befd` (the merge base this whole arc branched from), re-measured 2026-09-18 rather than accumulated per-commit: a `sys.setprofile` harness shaped like `tools/perf/call_count.py`'s `calls_for` (one warm-up parse, mean of 50, module-level `nameparser.parse`, run as a SCRIPT FILE outside either checkout so neither run's `sys.path[0]` masks the other — a `-c` one-liner instead would let the invoking shell's cwd shadow the older snapshot and read the current tree twice), against a `git archive 1f78befd nameparser` extraction on one side and this tree's own editable install on the other, same resolved interpreter both sides. Reference name (`tools/perf/call_count.py`'s own): 411 → 412 (+1). A comma-less, non-trigger control, `John Smith`: 175 → 176 (+1) — segment's lazy `own_words` call never fires without a comma, so this is classify's own always-on share of the cost. Comma names: a non-trigger, `Smith, John`: 203 → 207 (+4); `John Smith, Jr.`: 229 → 233 (+4); `Smith, John, Jr.`: 253 → 257 (+4) — every non-trigger comma name pays the SAME +4 total, dominated by classify's shared span cut plus the cheap case-free candidacy tests the three reading sites each add, not by the lazy fact itself (which these names never force). A genuine trigger, `Smith, MA`: 203 → 225 (+22) — the lazy fact plus its three readers' own work once a real candidate is found. (An earlier draft of this paragraph reported "+1 to +2 on non-trigger comma names" and "+19 to +20" on `Smith, MA`; those were PER-COMMIT deltas from the fix-round commentary above, accumulated inexactly rather than measured as one total, and are superseded by the totals here.) `own_words` shares its exact span-cutting walk with classify's own cut through a second shared function, `_vocab.tag_marker_runs` (moved out of `_classify.py` so `_pieces.py`'s own layering rule — it may import `_state` and `_vocab` only — is not broken): both call sites run the identical walk over the identical inputs, including `comma_offsets`, so a marker phrase like `z domu` is bucketed the same way at a comma either side. A corpus probe over the 1174 corpus entries confirms 0 cut differences and 0 verdict differences between the two call shapes (an earlier, text-only approximation that predated sharing `tag_marker_runs` differed on the CUT for 2 names — `Anna z Nowak`, `Anna z (domu) Nowak`, where `z` opens the phrase entry `z domu` but no run completes — while agreeing on the BOOLEAN for all 1174 even then). WITH SWITCH B ON, a genuine caps candidate pays more, recorded rather than chased to zero since nothing gates it — re-measured 2026-09-18 with `Parser().parse` on both sides of the on/off comparison (the convention `tests/v2/test_policy.py`'s own test uses), against this tree's own default: `Smith, John` 206 → 213 (+7), `Smith, XYZ` 205 → 245 (+40). Recompute (`tools/perf/call_count.py`'s own `calls_for` counts only its fixed reference name, so a chosen comma name needs the same shape spelled inline): `uv run python -c "import sys; from nameparser import Parser, Policy -def calls_for(fn, name, n=50): - fn(name); calls=0 - def counter(frame,event,arg): - nonlocal calls - if event=='call': calls+=1 - sys.setprofile(counter) - try: - for _ in range(n): fn(name) - finally: sys.setprofile(None) - return calls/n -off=Parser(); on=Parser(policy=Policy(unlisted_caps_suffixes=True)) -for n in ('Smith, John','Smith, XYZ'): print(n, calls_for(off.parse, n), calls_for(on.parse, n))"` (both cheaper than an unconsolidated spelling would be, after the caps predicate was consolidated into one shared function, `_vocab.caps_shape_candidate`, called from classify, `ambiguous_class_candidate` and `_segment.py`'s run test alike — the usual "a shared call costs the default a frame" objection does not apply, because every caller's own first conjunct is `policy.unlisted_caps_suffixes` itself, False by default; `tests/v2/test_policy.py`'s own dated comment carries the same two figures and the recipe for `tools/perf/call_count.py` reference readers). The band never moved at either policy — `test_benchmark.py` gates only the comma-less reference name, which no comma-path cost can reach — and `_CALL_BASELINE` is untouched throughout. - NO MECHANISMS ENTRY IS OWED, and here is why: the promoted fact is STATE-OFFSET-CHANNELS' "a pre-token fact is recorded on the ParseState and consulted later" applied to a name-level boolean rather than to offsets, and `ParseState.order` is the precedent that entry and ONE-PREDICATE-PER-QUESTION both already name. That entry's own sentence — "recorded as offsets … or by presence alone where the fact is name-level" — already covers it; the one edit made there is to name `one_case` as the third channel. -- 2026-09-15 (Derek) — ACCEPTED, NOT REPAIRED, TWO SHAPES. (i) `Freiherr von Berg MA` MOVES: it was listed as unreachable by the peel and as staying fix(#424)'s, and measured, the caps lean makes it reachable, because the lean ends P2's chain with no words to spare where the count would not. Today title 'Freiherr', family 'von Berg MA'; designed title 'Freiherr', family 'von Berg', suffix 'MA'; 1.4.0 first 'von Berg', last 'MA'. Three readings exist and the designed one ships; it moves a rules.md#P2 example line and re-scopes fix(#424)'s ledger rule, whose `fields` gains `suffix`. (ii) `abdul Smith Jr Ma` reads `Jr` as a MIDDLE NAME: the surname lean breaks the peel AT `Ma`, so the unambiguous `Jr` in front of it is never reached. Today given 'abdul', family 'Smith', suffix 'Jr Ma'; designed given 'abdul Smith', middle Jr, family 'Ma'. One case row rather than real data, but the shape is real — a name-leaning ambiguous acronym BLOCKS the peel of a genuine suffix behind it — and the walk stopping at the declined pick is the rule, not a bug the peel owes a repair. -- OUT OF SCOPE, each its own issue or entry: no tokenizer split (`J.R.R.`, `A.B.` stay single tokens); the trailing-period-only shape is NOT the signal (`John Smith Xyz.` → family, and AGENTS.md's trailing-abbreviation paragraph stands for bare words and sentence-final stops); no new `AmbiguityKind`; no locale packs; #492 stays deferred. NEITHER SWITCH REACHES THE v1 SHIM, and that is a consequence stated rather than repaired: `lenient_comma_suffixes` has no v1 manager either and `_build_snapshot()` simply does not translate it. Switch A is DEFAULT ON, so `HumanName` changes with it and a v1 caller cannot turn it off — the deliberate reading of the bridge, where the facade tracks the parser's defaults and 2.x behavior changes reach it (the #528 precedent). -- 2026-09-18 — RULES.MD STATEMENTS NEVER NAME A POLICY FIELD, AND THIS ARC MADE THAT LOAD-BEARING. rules.md's own preamble binds statements to be implementation-free ("no stage names, no function names, no regexes"), and no existing statement names a `Policy` field either — C1 says "strict mode" and leaves `strict-comma-suffixes` to the annotation slot. S2's and S3's switch clauses follow that precedent rather than setting a new one: the prose describes `unlisted_dotted_suffixes` and `unlisted_caps_suffixes` as caller configuration and points at the example lines, and the ANNOTATIONS carry the exact field names (`unlisted_dotted_suffixes-off`, `unlisted_caps_suffixes-on`, registered in `tests/v2/rules_doc.py`'s `POLICIES` beside `strict-comma-suffixes`). Recorded here rather than as a silent edit to the preamble or to two rules' prose: if a future design wants a field named directly in a statement, that is a preamble amendment for someone to propose, not a convention this entry changes on its own authority. -- 2026-09-18, corrected — `BUILD_RULES_CORPUS.PY` HARVESTING A POLICY-ANNOTATED-ONLY EXAMPLE IS LONG-STANDING, NOT NEW, AND THIS ARC ADDS TWO MORE INSTANCES OF IT. Its own docstring says "Policy annotations are deliberately ignored... a family-first-scoped example is simply one more name to diff" — an earlier draft of this bullet claimed that sentence was written when every annotated example had a default-policy sibling; measured false, at the commit that wrote it and on this tree alike. Recompute: parse every rule's examples with `tests/v2/rules_doc.py`'s own `parse_rules_doc`, group by `text` across the WHOLE document, and keep every text whose every appearance's annotation resolves (`resolve_annotation`) to kind `"policy"` rather than `"locale"`/`"gated_locale"` and never appears bare. That gives **19** pre-existing examples (`Mesnil de`, `Smith, de Mesnil Jean`, `de la Vega y Santos Juan`, `ibn Awf abdul Rahman`, `Nguyen Thi Van`, `Jane Smith (Nee)`, `Jane Smith (Nee) (Jones)`, `Garcia Juan Carlos`, `Nguyễn Thị Minh Khai`, `Del Toro`, `Anh Van Do`, `Beethoven Ludwig van`, `Berg Jan de Jr.`, `Garcia`, `Ménil Christophe de`, `Ménil de`, `Smith (Jones)`, `de Anke van y`, `van Berg Jan de`) plus this arc's own **2** (`Jean DUPONT`, `Jean Pierre DUPONT`, both `unlisted_caps_suffixes-on`, rules.md#S2) — **21** total on this tree. So the shape was accepted the day the docstring was written, not discovered here; what this arc adds is two more members, checked here for the reason the advice below names: `Jean Pierre DUPONT`'s DEFAULT reading (given Jean, middle Pierre, family DUPONT — measured; the annotated ON reading, given Jean, family Pierre, suffix DUPONT, is the only one rules.md states) never appeared in the doc before, and it produces no diff at any of the five baselines either way. Recorded per the same over-collection principle that keeps `(` and `.,` in the corpus (`tools/differential/README.md`'s provenance table): an annotated example's TEXT is a default-policy corpus name regardless of its annotation, full stop, and every one of the 21 — not only this PR's two — is a standing instance a reader should check for a default-policy diff on before assuming it free. -- 2026-09-18 — THE BLAST RADIUS IS THE GATE'S OWN COUNT, NOT A PREDICTION. Measured at 2.3.0 (`uv run python tools/differential/compare.py`, the default baseline, the previous release), and RE-MEASURED after the review round later the same day and again after the verification round that followed it, which is why several sets of figures carrying one date would otherwise sit here: corpus 1234 names, intentional diffs 51, unexplained 0, radar unclassified 7 — SUFFIX_OR_NAME rows the gate reports but no rule yet claims, named so a release note is never quantified from the intentional count alone. The round added TEN corpus names, not the eight first written here (corrected in the verification round of 2026-09-18): `Doe, John X.Y.Z.`, `Dr. 김민준씨, MA`, `John Smith, MD, Ma`, `John Smith, MD, R.A.I.`, `John de Ma`, `Jo김민준씨, DO`, `Kim김민준씨, MA`, `STEVEN HARDMAN, MD, DO, DDS`, `anh van mc`, `毛泽东 MA` — case rows and rules.md example lines both. The verification round itself then added THREE more (`Anh van Do`, `Doe, John MA`, `John Smith, MD, XYZ`, all rules.md boundary examples), taking the gate's count at this baseline from 1231 to 1234. Recompute the delta rather than trusting either figure: load every `tools/differential/corpus*.jsonl` through compare.py's own `_load_entries`, dedup by name in file order, and diff that list against the same list built from `git show :tools/differential/corpus*.jsonl`. That by-name count runs four below the gate's at the 2.x baselines, the gate deduping by (name, order) and four entries carrying a declared order; the DELTAS agree, which is what a "the round added N" sentence is claiming. The round also added a FOURTH rule, `fix(#289/#516) the glued CJK honorific peel reads the case lean` (3 composed CJK names: `Dr. 김민준씨, MA`, `Jo김민준씨, DO`, `Kim김민준씨, MA` — radar-tier by demotion and classified anyway, since a radar diff this arc intended is one a release note may be written from). The four rules carry 20 + 9 + 9 + 3 = 41 of the 51, `John de Ma` having joined the first. The pre-round reading of this paragraph, kept for the shape of the argument rather than the digits: `fix(#289) a written case contrast decides a bare ambiguous acronym` (19 names, incl. `Davis Royce, Ed`, `Freiherr von Berg MA`, `Jack MA`, `Jack Wei Ma`, `John Prof. MA`, `John Smith Ma`, `John Smith, Ed`, `John Smith, MA`, `John Smith, Ma`, `John van der Berg Ma`); `fix(#516) an unlisted dotted acronym is read by position` (9: `Jack X.Y.I.`, `John Smith Q.W.E.R.T.`, `John Smith X.Y.Z.`, `John Smith, A.B.`, `john smith x.y.z.`, `John Smith B.Tech.`, `John Smith C.H.A.`, `John Smith E.S.Q.`, `Smith, E.S.Q.`); `fix(#289/#516) the ambiguous credential class reports at slots that were silent` (9: `John Smith J.u.n.i.o.r.`, `Royce, Ed`, `Smith, A.B.`, `Smith, Ma`, `J.A. K.D.`, `Jack X.Y.Z.`, `John Smith R.A.I.`, `Smith Jr., A.B.`, `Steven Hardman, MD, DO, DDS`). This is LARGER than the plan's original "11 role + 4 report-only" prediction at the default policy — entirely by corpus growth (this arc's own commits admitted the case-row texts above as contract-tier names) and by two mechanisms the runtime prototype that produced that figure predated (`ambiguous_class_candidate`'s existence and the family-comma report's shape-tag branch), never by a wider rule. Recompute with the command above and read the count the gate returns; the numbers here are a snapshot dated 2026-09-18, not a promise. -- 2026-09-18, WHOLE-PR REVIEW ROUND — WHAT IT MOVED, MEASURED BEFORE AND AFTER. Ten findings; the five that changed behavior are here with the shape each was found by, since the shape is what generalizes. (1) A LOST REPORT. Where the prefix chain absorbed an acronym the S2 peel had DECLINED, nobody emitted the flag S2 promises for either reading: `assign` reports from `peel_trailing`'s picks and a pick reaches it only as a LONE piece, so the token it would have reported on no longer existed. Before: `John van der Berg Ma`, `John de Ma`, `Dr. John van Smith Ma`, `John van Smith Ma Jr.` and `John Smith Mc Ma` gave their family names with `ambiguities` empty; after, each names `suffix-or-name` once and no role moves. The emitter is at the chain's own merge and declines where the PARTICLE run took the word (`anh van do`, `anh van mc` — P6's fork, not S2's, and 1.4.0 read both silently). Recompute: parse the five names and read `ambiguities`. (2) TWO STAGES ASKING ONE QUESTION WITH DIFFERENT INPUTS. `_script_segment` asked `is_wholly_suffix` of the post-comma run WITHOUT the `one_case` fact `segment` had recorded one stage earlier, so `Kim김민준씨, MA` read family `Kim김민준씨`, suffix `MA` (씨 unpeeled) while `Kim김민준씨, PhD` read family `Kim김민준`, suffix `씨, PhD` — one name, two credential spellings, two divisions. After: both peel. `Jo김민준씨, DO` and `Dr. 김민준씨, MA` move with them; `김민준씨, MA` does NOT, being wholly one case (Hangul is caseless and `MA` contrasts with nothing), which is the doctrine holding rather than an exception. rules.md#W3's claim that the two stages differ "in what else they require, not in what they ask of the words" was false until this and is true now. (3) A REPORT ABOUT THE PARSER'S OWN NEW READING. rules.md#S3's narrow retirement moved `R.A.I.`, `X.Y.I.` and `J.u.n.i.o.r.` from the vocabulary verdict into the shape class, and `segment`'s tail-segment test reads `is_wholly_suffix`, which is blind to that class — so `John Smith, MD, R.A.I.`, `John Smith, Ph.D., R.A.I.` and `John Smith, MD, J.u.n.i.o.r.` gained a `comma-structure` flag 2.3 did not raise. After: quiet, at the default. The test is narrowed to the BY-SHAPE half deliberately — a LISTED member reaches that reading through the case lean, and `STEVEN HARDMAN, MD, DO, DDS` (one case, nothing leans) keeps its flag as the lean's own recorded negative control; `John Smith, MD, Ma` keeps it too. With `unlisted_dotted_suffixes=False` the flag stands, the token being name material there. C2 is amended to state all of this. (4) THE WORDLIST ROSTER. `_vocab.caps_shape_candidate`'s exclusion named ELEVEN of `_lexicon._VOCAB_FIELDS`' thirteen: `Lexicon.default().add(surnames={"dupont"})` with the caps switch on still read `Jean Pierre DUPONT` as given `Jean`, family `Pierre`, suffix `DUPONT` — a caller listing a word as a SURNAME and getting a credential, which is this switch's own worst failure arriving through the one wordlist that says "family name". After: family `DUPONT`, no report. The roster is derived from the field tuple, with a per-field test; `honorific_tails`, the other omission, was already excluded transitively (⊆ `suffix_words` by Lexicon invariant) and joins for completeness. (5) A LISTED MEMBER READ BY SHAPE. A caller may list a DOTTED entry in `suffix_acronyms_ambiguous` ('a.b'), and the whole token then matches that set while `suffix_as_written`'s period-free acronym lookup ('ab') misses it — so the chunk view reached classify's shape branch, `SHAPE_ACRONYM_TAG` rode along, and `_pieces.listed_lean` (which declines wherever that tag is) silenced the caller's own case lean: `Jack A.B.` read family where `Jack MA` reads suffix. After: suffix `A.B.`, the lean applying. Nothing default moves, the shipped ambiguous vocabulary carrying no periods. Also in the round and behavior-neutral: `Policy.unlisted_dotted_suffixes` and `unlisted_caps_suffixes` moved to the END of `Policy` and `PolicyPatch` — inserted mid-class, they had re-bound every positional argument from `strip_emoji` on, which is API for a dataclass that is not `kw_only`; the bool-validation roster is derived from `dataclasses.fields`, the hand-written twin in the tests having named neither new flag; `ambiguous_class_candidate` lost an optional `one_case` parameter no production caller ever passed, and with it a caps branch that answered False for every name the library has parsed; and `one_case` is REQUIRED on the four piece predicates that own it. -- 2026-09-18 — DIGIT-DOTTED TOKENS MOVE, SILENTLY, AND IT IS ACCEPTED. The narrow retirement's set of single-ASCII-character suffix entries is `{i, v, 2}` — the roman numerals AND the lone digit — so a dotted token whose only chunk match is that digit loses the chunk claim, and the SHAPE class cannot take it either: `period_joined_vocab`'s shape verdict wants every chunk `isalpha()`, which no digit is. Measured against 1f78bef: `John Smith 1.4.2` suffix `1.4.2` → family `1.4.2`; `Smith, 1.4.2` family `Smith`/suffix `1.4.2` → given `1.4.2`/family `Smith`; `John Smith, 1.4.2` given `John`/family `Smith`/suffix `1.4.2` → given `1.4.2`/family `John Smith`. No report at any policy, and the switch does not reach it either way. ACCEPTED: a version string read as a credential was the same accident the retirement removes, and the three readings are pinned as case rows. `John Smith 1.4` was already family at 1f78bef and is unaffected — two chunks, neither a match. docs/release_log.rst's wording is corrected from "a single-character roman numeral" to the digit-inclusive form, which is what the code tests. -- 2026-09-18 — THE COMMA REPORT'S REACH IS THE FIRST POST-COMMA PIECE, AND THE GIVEN SEGMENT'S TRAILING SLOT IS AN OPEN FOLLOW-UP. `assign`'s family-comma emitter reads `state.pieces[1][0]` — the first piece after the comma — so a class member standing at the END of the given part is never asked about. PRE-EXISTING and unchanged by this arc: `Doe, John MA` reads middle `MA` and `Doe, John X.Y.Z.` middle `X.Y.Z.`, both silent, at 1f78bef and on this tree alike. What this arc adds is names that MOVED into that silence: `Doe, John J.u.n.i.o.r.` and `Doe, John X.Y.I.` went suffix → middle, and `John Smith nee Jones R.A.I.` suffix → maiden (the maiden walk absorbs it). `Freiherr von Berg X.Y.I.` was listed here too and does not belong: corrected in the verification round of 2026-09-18, it moved suffix → family but GAINED a report rather than losing one. At 1f78bef and on the 2.3.0 wheel it read title `Freiherr`, family `von Berg`, suffix `X.Y.I.`, reporting `particle-or-given` alone; on this tree it reads title `Freiherr`, family `von Berg X.Y.I.` and reports `particle-or-given` AND `suffix-or-name`, the chain emitter taking it by its shape tag. The role moved into a slot that does not report and the report came from the chain instead, so the name is evidence for the emitter rather than for the silence. The universal claims are narrowed to what ships rather than the emitters widened: `nameparser/_types.py`'s `SUFFIX_OR_NAME` member now names the slots it covers and the one it does not, and rules.md#S3's "and the same at a comma" now says which parts of a comma form it means. Widening it is a MAINTAINER DECISION and is left open, filed as [#531](https://github.com/derek73/python-nameparser/issues/531): the slot is `_assign`'s given-part walk, the fork is genuine there, and whether a middle initial's neighbourhood should start reporting is a judgement about noise (rules.md#A1's "a kind is worth adding only if a reader would hesitate too" applies to a SLOT the same way). And the open question is a 1.4.0 PARITY question as well as a noise one, which the verification round of 2026-09-18 established and the entry had backwards: `Doe, John MA` reads suffix `MA` on the 1.4.0 wheel and middle `MA` at 2.0.0 through 2.3.0 and here, so the silence at this slot is a 2.0-era reading that has never matched v1 — "every release" was the wrong span and it is now "every 2.x release" wherever this is stated (`nameparser/_types.py`'s `SUFFIX_OR_NAME` member, docs/release_log.rst). The dotted sibling does not divide that way: `Doe, John X.Y.Z.` is middle `X.Y.Z.` at 1.4.0 too. So a maintainer widening the slot is choosing between reporting the fork and restoring v1's ROLE, which are different repairs, and only the bare-acronym half has a parity claim behind it. Recorded under 3-0-reevaluations' standing rule that a design citing 1.4 parity as load-bearing says so where the design lives. Two more report movements recorded here rather than repaired: `van MA` LOSES its `particle-or-given` (1f78bef reported both kinds; the caps lean now takes `MA` to suffix, leaving `van` the only name piece, and assign's leading-particle emitter wants more pieces following), and `John Smith Ma Ma` reports ONCE where 1f78bef reported twice (the first `Ma` is a middle name now, not a second declined pick). And a reach of the default-on dotted switch worth naming: `John Smith www.example.com` reads suffix `www.example.com` with a report, where 1f78bef read family — a URL is three alphabetic period-separated chunks and the shape rule cannot tell it from an acronym. -- 2026-09-18 — A KNOWN ROUGH EDGE UNDER `lenient_comma_suffixes=False`, FOR DEREK'S ATTENTION. Measured: `Smith V., Ma` gives given `Smith`, family `V.`, suffix `Ma`, and `Smith V, Ma` gives given `Smith`, suffix `V, Ma`. The cause is the strict knob rather than the credential class: under strict, the initial-shaped `V.` fails the suffix test, so `name_word_count` sees TWO name words before the comma, flips the structure, and `Smith V.` is then read positionally. The control that shows it is the knob's and not this arc's: `Smith V., PhD` reads given `Smith`, family `V.`, suffix `PhD` under the same knob on master. Consistent, then, and left as it is — recorded with case rows so a future change to either the knob or the count has to notice it. -- 2026-09-18 — THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT, kept because the failure mode is this log's recurring one. A review round called THREE `one_case` plumbing sites in `_group.py` inert and proposed removing them. Measured with a runtime wrapper over 1,642 names under six policies — the default, both family-first orders, strict commas, and each 2.4 switch flipped — 9,852 parses: the maiden walk's PAIR is inert, 0 parses move, and it is inert BY CONSTRUCTION rather than by corpus luck (`numeral_only` answers off `peeled.numeral`, and the numeral fork is decided before the peel reads a lean). The chain-tail measure is not: dropping it there moves 18, on `John van der Berg Ma`, `John de Ma` and `Freiherr von Berg MA` under every one of the six. Both figures are at the call sites. The population is the DISTINCT union, in file order, of every `tools/differential/corpus*.jsonl` entry, every `tests/v2/cases.py` text, and `tests/test_variations.TEST_NAMES` with the three comma permutations `test_variations_of_TEST_NAMES` builds (no-comma, family-comma, and suffix-comma where the name has a suffix), empty strings dropped; the wrapper patches the call sites in a copy of the module source and rebinds `_pipeline.STAGES`, which binds stage functions at import and would otherwise leave the mutation inert. This entry first carried 2,338 names and 14,028 parses, which that recipe does not reproduce by any reading of it — the distinct union is 1,642, summing the three sources' distinct counts gives 2,202, and not deduping at all gives 2,690 — so both figures were re-measured on this tree in the verification round of 2026-09-18 and restated here and at the call sites. The FINDING was unharmed: 0 and 18 reproduce exactly, on the same three names, under all six policies, and the control that drops all four sites at once moves those same 18, which is what shows the wrapper can report a difference at all. The argument was right for two sites out of three and would have taken a third with it. -- 2026-09-18 — THE COMMA HARNESS AND THE FRAME BAND, AFTER THE ROUND. Unchanged at the default: 412/449 on the reference name (`uv run python tools/perf/call_count.py`), and the module-`parse` totals above hold exactly — `John Smith` 176, `Smith, John` 207, `John Smith, Jr.` 233, `Smith, John, Jr.` 257, `Smith, MA` 225. WITH SWITCH B ON the two figures recorded above each drop by one: `Smith, John` 213 → 212 and `Smith, XYZ` 245 → 244, `Parser().parse` on both sides as before. The cause is the dead caps branch leaving `ambiguous_class_candidate`: `segment`'s single-token call used to reach a `caps_shape_candidate` call that could only ever answer False, and now does not. Recompute with the recipe above. -- 2026-09-18, VERIFICATION ROUND — THE REPORT INVENTORY AN UPGRADER SEES, 2.3.0 → TREE, AND ITS RECIPE. docs/release_log.rst's comma-report bullet claimed TWO reports go away and named `Steven Hardman, MD, DO, DDS` and `John Smith, MD, R.A.I.`. The second is false and was never measured against the wheel: `John Smith, MD, R.A.I.` reports NOTHING at 2.3.0, having carried a `comma-structure` flag only inside this arc's own development, where rules.md#S3's narrow retirement raised it and the C2 amendment quieted it again. Over the whole corpus exactly ONE report is lost, and one more is SWAPPED. Measured: LOST — `comma-structure` ×1 (`Steven Hardman, MD, DO, DDS`), `given-or-family` ×2 (`Jack X.Y.I.`, and `e and e`, which is PR #527's conjunction change and not this arc's). GAINED — `suffix-or-name` ×37, `conjunction-or-initial` ×9 (#527's again), `given-or-family` ×2 (`Jack MA`, `Jack MA.`). So `Jack X.Y.I.` is the only name whose report CHANGED KIND on this arc's account, `given-or-family` → `suffix-or-name`, and the bullet now names it as the swap rather than counting it among the losses. Recompute: collect the corpus glob through compare.py's `_load_entries` deduped by name, parse each name on the tree and on the 2.3.0 wheel (a PEP 723 `uv run --no-project` script pinning `nameparser==2.3.0`, per AGENTS.md's warning that a same-tree comparison reports parity), and take the per-name multiset difference of the `ambiguities` kind values in both directions. The kind counts are multisets, not name counts — `e and e` contributes two `conjunction-or-initial` gains on its own — which is why a bullet quantified from the gate's per-rule name lists would not answer this question. -- 2026-09-18 (Derek), #531 — CAPITALS DECIDE FOR `do`, AND THE PARTICLE RULE KEEPS EVERY OTHER SPELLING. `do` is the one member of the ambiguous credential class (`ba`, `do`, `ed`, `jd`, `ma` — recompute with `sorted(Lexicon.default().suffix_acronyms_ambiguous)`) that is also particle vocabulary, so the trailing slot of the given part and P6's attachment want the same word. Where the writing carries a positive credential lean — an all-caps member in a name written in more than one case — the credential reading wins and P6 stands down: `Doe, John DO` reads suffix `DO` and reports `suffix-or-name`. Every other spelling attaches exactly as it did before, with P6's own `particle-or-given` and no second report: `Doe, John do`, `Doe, John Do`, `DOE, JOHN DO`, `doe, john do` all read family `do Doe` in their own case. THE PAIRING IS THE ARGUMENT, and the accepted cost and the reason are the two halves of it. In ONE CASE the rule cannot tell `NASCIMENTO, EDSON ARANTES DO` from `SMITH, JOHN DO` and reads both as the particle — right about the Brazilian record, wrong about the osteopath, whose `DO` 1.4.0 read as a suffix; both report `particle-or-given`, which is how a caller finds the second. In MIXED CASE it can, and does: `Nascimento, Edson Arantes do` keeps family `do Nascimento` while `Doe, John DO` gives suffix `DO`. Accepted rather than repaired: the one-case reading is the commoner of the two collisions, and the Nascimento record is the name a wrong answer would damage. The other four members are not particles and are unaffected. This is the exception rules.md#P6's "this attachment outranks the suffix reading (S2)" clause now names. Implementing it takes TWO carve-outs rather than one, which is worth recording because the second is in the REPORT and not the reading: the reading carve-out takes a particle-tagged member only on a POSITIVE credential lean, since `Doe, John do` leans nothing and the positional reading would otherwise take it — the wrong ANSWER, not merely a stray report; and the emitter beside it skips a particle-tagged member it did not itself take, since an emitter gated on tag membership alone reported both kinds on every `do` row. Where P6 owns the fork, P6 reports it, which is mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE read strictly. ONE SHAPE THE PAIRING DOES NOT REACH — a shape, not a spelling, and the difference is the review finding: ANY particle or particle chain standing in front of the caps `DO` takes it out of this slot, because grouping's prefix chain merges the two into ONE particle piece before the slot exists and there is then no lone member for the capitals to lean on. It is a fact about this WORD being particle vocabulary itself, not about particles generally — the caps `MA` in `Doe, John van MA` is not swallowed, reads as the credential, and leaves `van` to attach behind it. `Doe, John van DO` reads family `van DO Doe`, and so do the `de`, `von`, `del`, `la` and `van der` spellings of it (measured 2026-09-19 — parse them and read `family`). Pre-existing and unmoved by #531; `Doe, John van DO` is the case row and rules.md#P6's example line, and P6's statement carries the carve-out. The mirror shape is silent rather than merely unreached and is recorded on `AmbiguityKind.SUFFIX_OR_NAME`'s fourth boundary: a particle BEHIND the member ends the given part where this walk reads it, so `Doe, John MA do` keeps middle `MA` beside family `do Doe` though the capitals would otherwise have taken it, the attachment that moves `do` running a stage later. -- 2026-09-18 (Derek), #531 — THE COMMA REPORT'S REACH NOW INCLUDES THE GIVEN SEGMENT'S TRAILING SLOT, AND THE OPEN FOLLOW-UP IS CLOSED. The 2026-09-18 bullet above headed THE COMMA REPORT'S REACH IS THE FIRST POST-COMMA PIECE recorded this slot as an open maintainer decision and named the two questions it turned on — whether a middle initial's neighbourhood should start reporting (a noise judgement, rules.md#A1) and whether the silence was a 1.4 parity gap. Both are answered here, and that bullet stands as it landed. The slot now reads and reports: `Doe, John MA` gives suffix `MA` and `Doe, John Ma` keeps middle `Ma`, each saying which way it went. Derek chose to restore the ROLE and report both ways — one rule for both spellings — over a report-only change and over a capitals-only one, because the issue exists in the first place because two spellings of one name disagree. The noise question was settled by MEASURING THE DISAGREEMENT rather than by argument: over a generated sweep of 78 pairs — five listed members and two by-shape tokens in three cased spellings each, plus five controls in one spelling apiece, so 26 words against three name shapes — 48 pairs disagreed about whether the word was a credential or a name, and every one of the 48 disagreed in the same direction, the comma form declining what the comma-less form took. After this change 3 disagree and all three are the lower-case `do` rows P6 owns. The sweep and its allowlist are `tests/v2/test_properties.py`; a slot that answers differently from the same name written without a comma is not a quiet slot, it is an inconsistent one. Recorded under `3-0-reevaluations`' standing rule because v1 parity is LOAD-BEARING for one half of this and explicitly NOT for the other: `Doe, John MA` reads suffix `MA` on the 1.4.0 wheel, so the bare-acronym half RESTORES v1's role and the report is all that is new there — its 1.4.0 ledger rule neither retired nor narrowed to `_ambiguities` (nothing below baseline 2.0 can diff on that pseudo-field) but was RE-POINTED to the four names where the writing declines the credential, which keep the 2.0-era middle name against v1. `Doe, John X.Y.Z.` reads middle `X.Y.Z.` at 1.4.0 too, so the dotted half LEAVES v1 and carries a 1.4.0 ledger rule of its own; it moves to match the comma-less `John Doe X.Y.Z.` and rules.md#S3's shape rule, not to restore anything. BLAST RADIUS, stated the way this log's own rule asks: twenty corpus names move, and only TWO of them were in any corpus before this branch (`Doe, John MA` and `Doe, John X.Y.Z.`, both admitted by #530's own arc) — the other eighteen are this change's own case rows, so what the differential measures on pre-existing data is two names, and the population the rule reaches is a SHAPE (every family-comma listing whose given part ends in a member of this class) that the corpora barely sample. ACCEPTED COSTS, all measured on the differential corpora: `SMITH, JOHN DO` keeps family `DO SMITH` where 1.4.0 read suffix `DO`, paired with the Nascimento record in the bullet above; one-case and caseless names take the credential with no case evidence at all, so `DOE, MARY JO MA`, `doe, john ma`, `田中, 太郎 MA` and `김, 민준 MA` all read a suffix, which is what 1.4.0 read for each of the four — "caseless is inert" holds for the LEAN and not for the outcome, since `is_one_case` answers True for a script with no case and the positional reading then decides; `Doe, John van MA` loses its middle to the family, reading family `van Doe`, suffix `MA` with two reports where it read middle `van MA` in silence, which is `Berg, Jan van Jr.`'s reading arriving through a shape it could not reach before (Derek accepted it 2026-09-18 as P6 working correctly rather than as a cascade to carve out); `Doe, John Prof. MA` gains a TITLE role `Prof.` never had, H5's transparency reaching it once `MA` leaves the walk, landing it on the same answer as `Doe, John MA Prof.`; `Smith, LEED AP` moves under the default-off caps switch alone, to given `LEED`, family `Smith`, suffix `AP` with two reports, so no default reading is at stake; and a genuine middle name that is also a class member is now a credential wherever it ends the given part and is not Title-cased — `DOE, JOHN ED` reads suffix `ED` — which is the same cost the comma-less form has carried since 2.0. NOT REPAIRED HERE and left open: the maiden walk claims `Smith MA` whole in `Doe, Jane nee Smith MA` before this slot exists, so the rule cannot reach it and the name stays silent with maiden `Smith MA` — the same gap #530's close-out saw from the other side with `John Smith nee Jones R.A.I.`, and it is out of scope for #531. -- 2026-09-19 (Derek), #533 — THE CLASS'S LAST SILENT TRAILING POSITION IS CLOSED, AND #531'S READING NOW LIVES IN ONE PLACE. The slot list in rules.md#S2 gains the trailing slot of a maiden marker's clause, and #S3's enumeration gains it with the by-shape spellings; the rule that governs what the clause does with those words is M2's, and the entry under `### M2` above is where its reasoning lives. The gap the bullet above left open as out of scope for #531 is the one this closes, from both sides: `Doe, Jane nee Smith MA` now gives maiden 'Smith' with suffix 'MA' and reports, and `John Smith nee Jones R.A.I.` gives suffix 'R.A.I.' again — a RESTORATION rather than a change, since 2.3.0 read it that way (measured on the wheel) and this unreleased cycle moved it into the maiden name when #516 took dotted tokens out of the certain-suffix class, where no corpus file held the name and no gate could see it. Two things belong here rather than under M2. FIRST, `credential_at_the_given_slot` in `_pipeline/_pieces.py` is now where #531's reading of a class member ending the given part is written, with two callers — assign's walk over that part, and the maiden walk's second check over the name a take would leave. Spelling it twice is precisely the "condition written to match it" mechanisms.md#ONE-PREDICATE-PER-QUESTION names, and the drift would have been silent, since each site's own tests would have gone on passing. It is a text-and-tags question, which is what puts it in `_pieces` rather than beside either caller — the destination follows the LAYER, not the topic. It costs one frame PER MEMBER asked at that slot: measured 2026-09-19 against 2f57ff21 per `Parser.parse`, `Doe, John MA` goes 310 → 311 and `Doe, John MA Ma MA`, which asks four times, 439 → 443, while a name with no member there never reaches it and pays nothing. The reference band does not move (412/449 on `uv run python tools/perf/call_count.py`), no test pins 310, and Derek took the trade rather than keep two conditions in step across two stages with no test that asks them both. The maiden path got one frame CHEAPER in the same pass, the numeral reading calling the peel pair it had wrapped rather than the wrapper: `Jane Doe nee Smith` 249 → 248. SECOND, THE ONE-CASE HEAD IS AN ACCEPTED EXCEPTION AND IT IS THE M2 INSTANCE OF #492'S DEFERRED QUESTION about whether a cased suffix token counts as case evidence. `DOE, JANE nee Smith Ma` reads suffix 'Ma' where the clause-less `DOE, JANE Ma` keeps middle 'Ma', because the own-words span stops at the marker — rules.md#P3 puts "a maiden marker's run and every word after it" outside the name's own words from the moment the marker is tagged — so the member's own Title-casing is not in the span `one_case` is computed over and the clause HIDES the contrast. That is THE PREDICATE KEEPS THE JUDGED TOKEN IN THE SPAN, the 2026-09-14 #289/#516 entry at the head of this section, failing structurally rather than by oversight: at this slot the judged token is never in the span, and that entry's own answer — include it — cannot be had here. Widening the span for this one question would change `one_case` for the whole name, and three sites read it, so the exception is accepted instead. Measured 2026-09-19: 114 of 2016 generated pairs — every listed member and both by-shape spellings, in three cased spellings, against sixteen heads and six clause bodies of NAME WORDS ONLY — against 186 allowlisted and 984 disagreeing outside the class before the change, and 0 outside it after. The PR review widened that grid to three markers (`nee`, `née`, `geb.`) and three policies (the default and each 2.4 switch), and the class is INDIFFERENT to both: 1,026 of 18,144, exactly 9x the original in both columns, and still 0 disagreeing outside it. `tests/v2/test_properties.py` carries the sweep with the class defined STRUCTURALLY (`one_case` true of the clause form and false of the clause-less one) rather than as a name list. The recorded control beside it is now the SET and not only its size, as a digest over the members: a count cannot notice a swap, one pair leaving and another arriving, which is the same blindness the count was added to close one level up. -- 2026-09-23 #492 — THE DEFERRED QUESTION the 2026-09-14 predicate paragraph and the 2026-09-19 #533 bullet above name — whether a cased suffix token counts as case evidence — IS ANSWERED FOR RENDER ONLY. R5's gate now leaves the suffixes out (decisions.md#R5, 2026-09-23); the parse-time fact this section rests on keeps counting the judged token exactly as written above, so `Jack MA` still leans credential and `DOE, JANE nee Smith Ma` still reads suffix `Ma`. The split is decided and recorded under R5, with the shapes it makes visible there (`jack MA` at this slot, `john e jones III` at P3's). -- 2026-09-27 (Derek), #544 — A CREDENTIAL RUN KEEPS ITS AMBIGUOUS MEMBERS: C1'S NAME-WORD COUNT READS A RUN AS IT READS ONE WORD, AN UNAMBIGUOUS CREDENTIAL IN FRONT OF A MEMBER SPEAKS FOR IT AT EVERY TRAILING SLOT, AND A LISTED MEMBER'S CHUNKED DOTTED SPELLING PASSES THE PERIOD GATE. Three readers never looked past the member itself: C1 flipped a part to the suffix comma by the name-word count only when it was ONE token, and the trailing readers — the no-comma peel, the comma part read wholly as credentials, the given part's trailing slot — met the member first, took its name lean and stopped, so the credential in front of it was never reached. #540 marked `meng` and `lac`, whose conventional spellings lean name, which turned the one-case-row cost this section's 2026-09-15 item (ii) accepted into the ordinary spelling of two degrees. DECIDED, ten forks, (9) and (10) on 2026-09-28. (1) THE WHOLE RUN AT C1: two or more words after the comma, every one suffix vocabulary or a class candidate, at least one a candidate, behind two or more name words, is the credential run — all-member runs (`John Smith, Ed Ma`) and a leading member (`John Smith, MEng PhD`) included; offered and declined were an anchored-only rule and the issue's own "last word ambiguous" wording, which contradicted its `Ed Ma` consequence. (2) ALL TRAILING PATHS, ANCHORED IN FRONT: a listed member with an unambiguous credential in front of it, through other members, reads as the credential on the no-comma peel, in the comma part read wholly as credentials and at the given slot; a credential BEHIND it anchors nothing (`Wang Ma PhD` keeps family 'Ma'). (3) THE CHUNKED DOTTED GATE: a listed member written in two or more period-closed letter chunks (`M.Eng.`, `L.Ac.`) passes as `M.A.` does; one trailing period (`Ma.`, `Ed.`) still does not. (4) SINGLE-LETTER ROMAN NUMERALS neither anchor nor count toward a C1 run, S3's single-character retirement being the precedent; they peel as before, and the anchor's half of the exclusion is a matter of their shape, boundary (d) below. (5) NO NEW REPORT WHERE THE WRITING SETTLED IT: a run whose every member is LISTED and leans credential — S2's lean, capitals in a mixed-case name — is not flipped, the family-comma path already reading it as the credential run with the fields and the report it always had (`John Smith, PhD MA`) — narrowed 2026-10-01 by the #562 entry under C1: a part holding two particles side by side is read by the count instead, and reports; the alternative, flipping such a run and suppressing only the flip's report, removes reports the family-comma path makes today, `John Smith, MA Jr` losing the one it makes for 'MA'. Because the lean is the listed set's, a by-shape member in capitals never settles a run: `John Smith, X.Y.Z. MA` flips to suffix 'X.Y.Z. MA' and reports. (6) THE ANCHOR DOES NOT REACH ACROSS A MAIDEN CLAUSE: `Jane Doe Jr. nee Smith Ma` keeps maiden 'Smith Ma' where `Jane Doe Jr. Ma` reads suffix 'Jr. Ma' — rules.md#M2's Accepted boundary, and the M2 bullet of this date. (7) AN ANCHOR OVERRIDES P6 AT THE GIVEN SLOT: `doe, jane v phd do` reads suffix 'v phd do' where it read family 'do doe'; the bullet below amends #531's pairing. (8) THE DUAL EXCLUSION IS SCOPED TO THE GIVEN PART'S LEADING TITLE RUN, NOT THE C1 RUN: after a full name the dual counts as the suffix vocabulary it is, as the legacy disjunct already counted it (`John Smith, MS MA`, `John Smith, MD`), and that is what reads the issue's headline `Jane Doe, MS LAc` as suffix 'MS LAc'. (9) A MEMBER THE COMPANY DECIDES REPORTS WHERE IT WAS DECIDED, in a family comma's part read wholly as credentials as at the peel: each member whose own writing declined and which the credential in front made the credential reports `suffix-or-name` at that reading. `Smith, PhD Ma`, `Smith, PhD MEng` and `Smith, Ph. D. MEng` report the member; `Smith, Dr. PhD LAc` and `John Quincy Smith, Prof. PhD LAc` report 'LAc'; `john smith, v phd ma` reports 'ma'; `John Smith, PhD Ma Prof.`, which C1 does not flip for its trailing title, reports 'Ma' where its twin `John Smith, PhD Ma` reports C1's flip once over the part. A member its own capitals made the credential stays silent (`Smith, PhD MA`), and the part's first piece, which nothing stands in front of, is the first-slot report's word and never this one's. (10) A DUAL IN THE GIVEN PART'S LEADING TITLE RUN SILENCES THE WHOLE PART: refinement (b) below reads such a dual as a title that speaks for nothing, and in a part whose leading title run holds one no credential behind the dual speaks for a member of the ambiguous class either, so the company never reads such a part wholly as credentials: the titles give it its reading, the dual a title and the next word the given name, and behind that given name the given slot reads as it reads any (`Smith, MD PhD Jr Ma` reads suffix 'Jr Ma', as `Smith, MD Jane Jr Ma` does). A part with no member to speak for, or only members whose capitals decide them, reads as it did: `Smith, MD PhD` suffix 'MD PhD', `Smith, Ms MD MA` suffix 'Ms MD MA'. `Smith, Ms MD Ma`, `Smith, MD MS Ma` and `SMITH, MD MS BA` read title 'Ms MD' / 'MD MS', given 'Ma' / 'BA'; `Smith, MD PhD Ma` reads title 'MD', given 'PhD', middle 'Ma', the given slot reporting 'Ma'; e10e83b4 read each of them so. The run is the walk's leading title run, so a dual behind a plain title stands in it too (`Smith, Prof. MD Ma` and `smith, prof. md ma` read title 'Prof. MD', given 'Ma'; `Smith, Dr. MD PhD Ma` keeps given 'PhD'), while a plain title alone silences nothing (`Smith, Dr. PhD LAc` reads title 'Dr.', suffix 'PhD LAc'). Behind a real given name the company reads as ever (`Doe, Jane MD Ma` and `Doe, Jane Ms Ma` read suffix 'MD Ma' and 'Ms Ma'), and past the title run a dual speaks like any suffix word (`Smith, PhD Ms Ma` reads suffix 'PhD Ms Ma'); `Smith, Ms Ma`, `Nguyen, Sr Ba` and `Smith, MD Do` are unchanged. - WHAT MAY ANCHOR, four boundaries of the company clause, each a rule. (a) A CONNECTIVE (P3) anchors nothing and ends the run: the generational `i` is also Catalan's conjunction, and `Jane Doe nee Puig i Ma` keeps its link, maiden 'Puig i Ma'. (b) A TITLE/SUFFIX DUAL IN THE GIVEN PART'S LEADING TITLE RUN after a one-word family comma is a title there and anchors nothing — and, by (10), no credential behind it speaks for a member in that part — so the given slot's pass starts past the leading title run: `Smith, Ms Ma`, `Nguyen, Sr Ba` and `Smith, MD Do` keep their given names, and `Smith, MD MA Ma` keeps middle 'Ma', no unambiguous credential standing in its run once the title is set aside. Everywhere else a dual anchors like any suffix word (`John Smith MD MEng`, and (8)). (c) THE WALK'S OWN LEADING PIECE NEVER ANCHORS. In a name with no comma before the run, the leading piece is the name word H4's carve-out keeps whatever its vocabulary, and letting it anchor spends that name: in `Om Ma` and `PhD Ma` 'Ma' stays a name word under every name order (family 'Ma' in the default order, given 'Ma' under both family-first orders), where an anchoring `PhD` would leave given 'PhD', suffix 'Ma' and no family at all. One name word in front frees the same word — `John Om Ma` reads suffix 'Om Ma'. A family comma's given part has no such position, its first piece being read by the comma part's own credential reading, so `Smith, PhD MEng` reads family 'Smith', suffix 'PhD MEng'. (d) A SINGLE-LETTER ROMAN NUMERAL, IN ANY CASE (`V`, `v`, `I.`), anchors nothing for its SHAPE — one letter is how a middle initial is written, and S3 retired single-character vocabulary matches for the same reason (`John Smith PhD V Ma` keeps family 'Ma') — not because a generation is no credential. The rule is stated by that shape and the code tests exactly it (`_vocab.is_single_letter_numeral`); "initial-shaped" was the wrong word for it, `is_initial_shaped` answering False for a lower-case `v` that the exclusion covers too. And the exclusion is not about generations: a multi-letter numeral or generational word anchors as any suffix word does (`John Smith PhD III Ma` reads suffix 'PhD III Ma', `abdul Smith Jr Ma` suffix 'Jr Ma'). - ACCEPTED COSTS, each a reading this design chooses. `John Smith, Ms Ma` → suffix 'Ms Ma', reported, as `John Smith, Ms` alone already reads suffix (8). `Om Jr Ma` → given 'Om', suffix 'Jr Ma' and no family name: 'Jr' is not the leading piece, so it anchors 'Ma', and this section's standing Accepted clause already consumes an unambiguous suffix that leaves no family (`Om Jr` → given 'Om', suffix 'Jr'); every release from 2.0.0 made that same role assignment. `doe, jane v phd do` reports `suffix-or-name` where it reported `particle-or-given`, the fork reported being the credential pick rather than P6's attachment (7). And a one-word family comma whose given part OPENS with a capitalised `MA` in a mixed-case name, with an unambiguous credential behind it, reads the whole part as a credential run, family 'Smith' and no given name, where the parent kept given 'MA': `Smith, MA PhD Ma` read given 'MA', middle 'Ma', suffix 'PhD' and reads suffix 'MA PhD Ma'. 'MA' leans credential on its capitals, so it is read as the credential it is written as, and what follows it is then the company's (or, for `M.Eng.`, the chunked gate's). Measured 2026-09-28 on the grid below: 138 parses (46 texts, three orders each) go from given 'MA' to no given name, 72 of them with no `M.Eng.` in the run; 12 of the 138 are the parses the attribution below leaves unattributed, the rest falling in the anchor and `M.Eng.` classes. Each reports 'MA' at the first slot, and a member behind the credential reports as the company's pick (9). - ACCEPTED LIMITS, each keeping the member a name word, rules.md#S2's Accepted block and case rows: the merged `Ph. D.` is outside the no-comma walk (`John Smith Ph. D. MEng` family 'MEng'; the comma and given-slot spellings do anchor it, `Smith, Ph. D. MEng` and `Doe, Jane Ph. D. MEng` reading suffix 'Ph. D. MEng'), a title between the credential and the member breaks the run (`John Smith PhD Prof. Ma`), and a particle member P2 has chained before the peel is out of reach (`John Smith PhD Do Do` family 'Do Do'; `Smith, PhD Do Ma` given 'PhD', middle 'Do Ma'). - THE TWO-INPUT CHECK. tests/v2/test_properties.py's M2 clause-agreement walk asks every parse it takes both directions of the company clause: a listed member behind a qualifying credential reads as one, and a member read as a credential because of the word in front has that word in the suffix too. Recorded negative controls, from that test's docstring: 48 failing parses with the anchor off, 30 with the walk's leading piece allowed to anchor, 0 on this tree; re-run 2026-09-28 after (9) and (10), unmoved. The walk holds no comma head that opens its part with a dual, so (10) is pinned by case rows and a unit test instead (`a_dual_opening_the_given_part_turns_the_anchor_off`, `a_dual_opening_the_given_part_silences_a_later_degree`, test_pieces' `test_a_dual_in_the_leading_title_run_turns_the_anchor_off`). - REVERSED BY THIS ENTRY rather than edited: this section's 2026-09-15 ACCEPTED item (ii) (the bullet after this entry says so); the three "wrongly moved" reversals the 2026-09-14 paragraph THE RUN IS THE CAPS CLASS'S ALONE records — `John Smith, Ed Ma`, `John Smith, ma do` and `John Smith, X.Y.Z. A.B.` are credential runs now by C1's own count under every policy that keeps their words in the class (with `unlisted_dotted_suffixes` off, `X.Y.Z.` and `A.B.` are no class members and the third keeps the listing form), while that paragraph's narrowing of the CAPS run test stands; and #540's two pending questions (the `suffix-acronym-collisions` bullet of this date). - MEASURED 2026-09-28, the #544 tree against its parent e10e83b4, py3.11, after (9) and (10); the 2026-09-27 figures this paragraph first carried were taken before them and are superseded. Recompute: every run of one to three words from {PhD, MD, Jr, MA, Ma, MEng, M.Eng., LAc, Ed, Do, ba, MS, V, Prof.} behind each of `Wang {}`, `John Smith {}`, `John Quincy Smith {}`, `Dr. John Smith {}`, `John Smith, {}`, `John Quincy Smith, {}`, `Smith, {}`, `Doe, Jane {}`, `Jane Doe nee Smith {}`, `Doe, Jane nee Smith {}` and `Jan van der Berg {}`, each written as given, lower-cased and upper-cased and deduped, parsed under the three name orders at otherwise default policy, the seven role fields and the ambiguity kinds compared against the same grid parsed by e10e83b4. THE DETECTOR, which both the invariants and the attribution read, looks only at the run's own words, compared case-free: {phd, md, jr, ms, m.eng.} are the unambiguous credentials, {ma, meng, lac, ed, do, ba} the members; a member is ANCHORED where an unambiguous credential stands in front of it in the run with nothing but members between, except that `ms` or `md` opening a comma part anchors nothing; and a run word is IN A NAME FIELD where it appears among the space-split words of given, middle, family or maiden more often than the run's unconstrained occurrences of it account for. The two invariants count parses with an anchored member in a name field, and with an unambiguous credential in one. THE ATTRIBUTION puts each moved parse in ONE class, the first that holds, in this order: THE ANCHOR, where the tree's two violation counts are each no higher than the parent's and one is lower; THE CHUNKED `M.Eng.`, where the run holds `M.Eng.`; A LONE `v`, where re-running the first test with `v` counted as an unambiguous credential makes it hold; THE C1 RUN, a comma shape with two or more name words before the comma whose run is only credentials and members, at least one a member; and anything left unattributed. 254,496 parses over 84,832 texts; 89,825 move (29,943 texts, 19,123 of them with a field move): 39,261 the anchor, 14,676 the C1 run, 35,696 the chunked `M.Eng.` (32,472 of those only losing the report it raised as a pick), 180 a lone `v` (`john smith, v phd ma`, suffix 'v phd ma'), and 12 unattributed, the `Smith, MA PhD ba` parses above. Clean-to-clean — no run word in a name field on either side — 23,454 move, every one `M.Eng.`'s dropped report, and none elsewhere. The two invariants, 0 NEW violations of either: an anchored member in a name field, 29,199 → 1,530 (the residue is a particle member P2 has chained, a dual inside a leading title run, `smith, prof. md ma`, and the member behind a credential in a part a dual opens, `Smith, MD PhD Ma`, which (10) keeps a name); an unambiguous credential in a name field, 36,675 → 4,734 (the residue is an interior single-letter `V`, a non-final `Prof.`, the `PhD` a comma part keeps as its given name when a `V` or a title follows it, and the same `PhD` behind a dual that opens the part, (10) again). Against the tree before (9) and (10), the same grid moves 5,076 parses: 4,536 (1,512 texts, every one a comma shape whose part reads wholly as credentials) gain a `suffix-or-name` report and move nothing else, and 540 (180 texts, every one `Smith, ` then `MD` or `MS` opening the run, in each case form) return to e10e83b4's reading, fields and reports alike, from a silent suffix reading of the whole part. The differential corpora at the parent (1,418 names, three orders) move four names, every one intended: `John Smith, PhD MEng`, `john smith, phd meng`, `Wang M.Eng.`, `abdul Smith Jr Ma`. Frames per parse through `Parser().parse`, py3.11: the reference band is unmoved (370/407, `uv run python tools/perf/call_count.py`), and so is every ordinary name measured (`Smith, John Quincy` 260, `Doe, John MA` 283, `Doe, Jane nee Smith PhD` 338) except a comma name with two or more words on each side of the comma, which enters the C1 run test and pays `run_word_fold`'s one call before its first name word ends it (`Doe Smith, Jane Q.` 317 → 318, `Garcia Lopez, Maria Jose` 330 → 331, measured 2026-09-28); the new questions cost where they are asked — `John Smith Ma` 244 → 250, the anchor pass that finds nothing; `John Smith, MA Jr` 355 → 382, the case fact forced to see the capitals settle the run; `John Smith PhD MEng` 286 → 326; `Smith, MD PhD Ma` 349 → 362, the given slot's pass that (10) leaves with nothing to find — and `John Smith, PhD MEng` gets cheaper, 358 → 314, as does `Smith, PhD Ma`, 286 → 271. - NO MECHANISMS ENTRY IS OWED: the anchor pass is ONE-PREDICATE-PER-QUESTION's `_pieces.credential_anchors`, which `segment_suffix_reading` reads inline for the frame budget under a keep-in-step note, and its linearity is the answer #531's trailing floor and #397's `_run_neighbours` already give — one forward pass per question, never a look-behind per member. tests/v2/test_benchmark.py's `credential_run` shape guards it: the per-member look-behind measured 15.8× for 4× the input at base 800, against 4.13–4.21× on this tree at every base (that module's own record). - ADDENDUM 2026-09-28 (Derek). (e) A PARTICLE ANCHORS NOTHING, a fifth boundary beside the four WHAT MAY ANCHOR lists: a word of both the particle and the unambiguous suffix vocabulary (`vd` and `mc` in the shipped lexicon; `do` sits in the ambiguous half, so it is a member and never an anchor) heads the family name behind it (P2), and anchoring there split the family around a suffix — `Smith vd Ma, John` read family 'Smith Ma', suffix 'vd'; `Jan vd Ma` suffix 'vd Ma' and no family; `Smith Mc Ma, John` and `D. Mc Ba Ed, Smith` the same way. Each reads as e10e83b4 reads it again (family 'Smith vd Ma', 'vd Ma', 'Smith Mc Ma', 'D. Mc Ba Ed'). The exclusion is of the word in front only: a particle MEMBER behind a credential is still spoken for (`doe, jane v phd do` keeps suffix 'v phd do'), and a C1 run holding the particle is C1's reading rather than the anchor's (`Jan Berg, vd Ma` reads suffix 'vd Ma', as `Jan Berg, vd` reads suffix 'vd'). Measured over the grid of MEASURED above with `vd` and `Mc` added to its words (127,578 texts, 382,734 parses), against the tree before this addendum: 9,426 parses (3,142 texts) move, every one holding `vd` or `Mc` directly in front of a member; 8,940 return to e10e83b4's reading exactly, and the other 486 all hold `M.Eng.`, whose own change is (3) — 406 of them take e10e83b4's fields and differ only in the pick `M.Eng.` no longer reports, and 80 take the fields e10e83b4 gives the same text with `PhD` in `M.Eng.`'s place (`Jane Doe nee Smith M.Eng. vd Ma` reads middle 'Doe M.Eng.', family 'vd Ma', as `Jane Doe nee Smith PhD vd Ma` reads middle 'Doe PhD'). The differential corpora move 0 of 1441 names, and the grid of MEASURED without the two words moves 0 parses, so every figure there stands. THE PASS IS ASKED ONLY WHERE A SUFFIX PIECE IS IN REACH: `_pieces.anchor_in_reach` walks back through the lone members in front of a declined member on tags alone and answers False wherever the pass must — the first piece past the members is no suffix piece, or there is none — and it is asked only before the pass exists, one lookup answering after that, so a run of members stays linear. An ordinary name ending in a declined member stops paying for the pass (frames, py3.11, before → after, e10e83b4 in brackets): `John Smith Ma` 250 → 246 (244), `Smith, John Ma` 285 → 279 (275), `Doe, Jane MA do` 374 → 369 (363), `Doe, John Q. Ma` 328 → 320 (316), `Jan vd Ma` 308 → 243 (238); a name the pass does read pays the test on top, `John Smith PhD MEng` 326 → 329 and `Doe, Jane PhD MEng` 359 → 362. On the no-comma peel the reach starts past the walk's leading piece, which never anchors — so a by-shape member, which reaches the pass only at the peel's second position (a lean of None with a word to spare is consumed first), finds nothing in reach, and the by-shape guard that stood there was deleted as unreachable. `segment_suffix_reading` calls `credential_anchors` (`first_kept=False`, a comma part's first piece being a run member like any other) in place of the inline copy NO MECHANISMS describes, and the keep-in-step note goes with it. The pass is built only where the reach test finds a suffix piece in front of a declined member, which no member opening the part has, and a part whose name word stands ahead of any member returns before one is asked: a plain comma name pays nothing (`Smith, John` 182, `Doe, John MA` 283, `Smith, J. Q.` 246, `Smith, Ed` and `Smith, Ma` 213, `Smith, Ed John` 270, all as before the call replaced the copy; built on every declined member instead, the last three paid 215, 215 and 273), a title in front pays the reach test alone (`Smith, Dr. Ma` 253 → 254), and a part the company decides pays three frames over the copy (`Smith, PhD MEng` 273 → 276, `Smith, PhD Ma` 271 → 274; `Smith, PhD Ma John` 330 → 335, its pass built for 'Ma' before 'John' ends the part). THE TWO-INPUT CHECK's second control reads 42, not 30, with no change to the property: the walk's leading piece is now held out of the pass twice on the no-comma peel, by `credential_anchors` and by the reach test, either alone suffices, and with both removed the by-shape heads the deleted guard held back (`PhD X.Y.Z.`) fail beside the listed ones; the first control is unmoved at 48. And the 2026-09-28 C2 bullet's "a TAIL segment, which assign reads wholly as suffixes" holds outside a maiden clause standing in it — `Jane Doe, PhD, Jr nee van Ma` keeps maiden 'van Ma' — which rules.md#C2's statement now says. The reach test's one-way exactness is held by tests/v2/pipeline/test_pieces.py's `test_anchor_in_reach_never_hides_an_anchor`, over the case table's texts and runs of one to three words behind two heads; its recorded negative control, the reach test reading a `vocab:suffix` token as no suffix piece, fails at 878 member positions. -- 2026-09-27 (#544) — THE 2026-09-15 ACCEPTED ITEM (ii) IS REVERSED, and the bullet stands as it landed. `abdul Smith Jr Ma` reads given 'abdul', family 'Smith', suffix 'Jr Ma' — 2.3.0's reading — because the unambiguous 'Jr' in front of the Title-case 'Ma' anchors it (the entry above): the peel takes both, and P5's reserve, now seeing the family the join would take, declines the join. Item (i) stands. The case row is now `a_credential_in_front_anchors_a_declined_pick`, and rules.md#S2's Accepted block names the shapes that still keep the company out of reach. -- 2026-09-27 (Derek), #544 — THE #531 PAIRING GAINS A SECOND EXCEPTION, and CAPITALS DECIDE FOR `do` above stands as it landed. That bullet let only a positive credential lean override P6's attachment at the given slot; an unambiguous credential IN FRONT of the member now overrides it as well, the degree being a second and stronger signal: `doe, jane v phd do` reads suffix 'v phd do' and reports `suffix-or-name` where it read family 'do doe' and reported P6's fork. The one-case record the pairing protects has nothing in front of its particle, so `NASCIMENTO, EDSON ARANTES DO` still reads family 'DO NASCIMENTO' and reports `particle-or-given`. -- 2026-09-30 (Derek), #563 — PAIRED INITIALS ARE THE EXCEPTION TO C1'S NAME-WORD COUNT, AND A FLIP ON THE DOTTED SHAPE ALONE IS SILENT. The 2026-09-14 (Derek) entry above, #516's DOTTED HALF IS A SWITCH, read `John Smith, A.B.` as suffix 'A.B.' on the count of two name words before the comma. The count cannot tell a given name and a family name from ONE surname written in two words, and the dotted shape's own most common member is a person's initials, so the unreleased tree read `García Márquez, G.J.` as given 'García', family 'Márquez', suffix 'G.J.', `De La Cruz, M.J.` with no given name at all, and `van der Berg, A.J.` as given 'van' — where 1.4.0 through 2.3.0 read each as initials. A particle check would have caught only the last two; `García Márquez` and `Lloyd Webber` look exactly like `John Smith`. What the parser CAN see is the token, so the line is drawn there: two single letters joined by a period, with or without one after the second (`_vocab.is_paired_initials`; `García Márquez, G.J` reads as `G.J.` does), read as the given name after the comma and report the fork, and only an UNAMBIGUOUS suffix word IN FRONT of them that is not also title vocabulary, or a second word the class admits by dotted shape in the same part, makes them the credential run (`John Smith, PhD X.Y.`, `John Smith, X.Y. P.Q.`). Three letters or more (`X.Y.Z.`) and any longer chunk (`B.Tech.`) read by the count as before. Three choices made with it (Derek, 2026-09-30): (i) FRONT ONLY — a word behind the pair does not speak, so `De La Cruz, M.J. PhD` keeps given 'M.J.' and `John Smith, A.B. PhD` reads given 'A.B.', the direction rules.md#S2's company clause already takes; a LISTED dotted word behind does not speak either (`John Smith, A.B. Ph.D.` → given 'A.B.'), the second-word exception being for words the class admits by shape; (ii) SILENT FLIP — a flip resting only on words the class admits by dotted shape reports nothing, `B.Tech.` included, because the only such word a reader takes for a name is a pair of initials, and the comma flips a pair only where a word beside it has already said otherwise; a LISTED member in the same part still reports the flip (`John Smith, PhD MEng`, `John Smith, X.Y.Z. MA`); (iii) THE COMMA SLOT ONLY — the trailing slot (`John Smith R.T.`), the given part's trailing slot after a family comma (`Doe, John R.T.`) and the maiden clause keep #516's reading and its report; #563 leaves the no-comma question open. A generational word in front speaks like any unambiguous suffix word (`García Márquez, Jr. G.J.` → suffix 'Jr. G.J.'), as S2's company clause has it — unless it is also a title, so `García Márquez, Sr G.J.` reads title 'Sr', given 'G.J.'. ACCEPTED: `García Márquez, G.J.R.` reads suffix 'G.J.R.' — initials are conventionally written apart, as separate words this rule never reads (`García Márquez, G. J. R.` gives given 'G.', middle 'J. R.'), and three letters run together are far likelier a credential. REVERSED BY THIS ENTRY rather than edited, for the dotted half at the first slot after a comma only: the #516 SWITCH entry's `John Smith, A.B.` → suffix and its "reports either way"; the 2026-09-14 NO NEW AmbiguityKind entry's "Every decision at these slots emits" SUFFIX_OR_NAME; and rules.md#C1's "A decision either way at this comma is reported". ALSO FIXED: the trailing slot's report described a by-shape pick in the listed member's words ("written without periods is both a post-nominal"), false of `X.Y.Z.` on both counts; it now says the word is shaped like a post-nominal but listed in no vocabulary. - REVIEW ROUND, same day. The docs review found the first version let a word of both the title and the suffix vocabulary speak for the pair, since C1 counts such a word opening the part as a suffix word: `García Márquez, Ms G.J.` read given 'García', family 'Márquez', suffix 'Ms G.J.' with NO report — #563's own defect, made silent — where 2.3.0 read title 'Ms', given 'G.J.'. The twelve title/suffix duals (`ms`, `md`, `sr`, `lt`, `sa`, `ra`, `vc`, `cpl`, `cpo`, `cpt`, `csm`, `sgm`) all reached it. Fixed by letting only a suffix word that is not title vocabulary speak: in front of a pair, a dual is the title of the given part the pair opens, as S2 already reads a dual in that part's title run, and `Ms G.J.`, `MD G.J.`, `Lt G.J.` now read title plus given as every release did. The title lookup runs only once a pair is met. The code review's frame count found the single-token path asking `ambiguous_class_member` a second time, +2 frames on every reporting comma name (`John Smith, MA`, 252 → 254, profiler call events); for a word past the candidate test LISTED is exactly "no period" (`ambiguous_class_member` declines any period), which is now asked inline. Measured after both fixes against master, same counter: `Smith, John` 183 and `John Smith, MA` 252 on both trees, `John Smith, X.Y.Z.` 295 → 286 (the report it no longer builds), and `García Márquez, G.J.` 338 → 350, the family-comma reading and its report costing more than the flip did. - SECOND REVIEW ROUND, same day, on the first round's fix commit. (a) TWO PAIRS (Derek): `De La Cruz, M.J. K.L.` read family 'De La Cruz', suffix 'M.J. K.L.' — no given name — in silence, the silent-flip rationale ("only paired initials are taken for a name") failing on exactly the case where each word speaking for a pair is itself a pair. The reading stands, two dotted groups not being how anyone writes a person's initials, but that flip now REPORTS: when the only words speaking for a pair are other pairs, every shape word in the part being one. `John Smith, X.Y. P.Q.` gains the report with it; `John Smith, X.Y.Z. G.J.` and `John Smith, PhD G.J. K.L.`, where something else speaks, stay silent. The same rule reports `García Márquez, Ms G.J. K.L.`, whose honorific the run takes, where the first round's fix took it in silence. (b) A CLASS MEMBER IN FRONT SPEAKS FOR NOTHING (Derek): `García Márquez, Ed G.J.`, `Ma G.J.` and `MA G.J.` flipped on the member in front, where S2's company lets only an unambiguous credential speak for a member. Now the comma keeps the family, given 'Ed' — and 'G.J.' then ends the given part, the slot (iii) left alone, so it reads suffix 'G.J.' as `Doe, John R.T.` does, both forks reported; 1.4.0 and 2.3.0 gave middle 'G.J.'. Resolving that slot is the no-comma question #563 leaves open. (c) WORDING, no behavior: C1's sentence counting a dual opening a part as a suffix word now defers to the pair's titles; the pair's report reaches it only where it opens the part, `Smith, Ms G.J.` having always been silent (the comma report's reach is the first post-comma piece, 2026-09-18); and the silence is stated as "no listed word takes part", which covers `García Márquez, PhD G.J.`, where "dotted-shape words alone" did not. - SIMPLIFY ROUND, same day, behavior-identical (0 diffs over 21,604 parses: every corpus name, every quoted string in tests/v2/cases.py and 115 composed `pre, post` probes, under four policies, comparing fields and ambiguity details against the pre-round commit c125f69b; the same harness finds 252 diffs against master). One finding was a cost, not a style point: the speaker test scanned every word in front of EACH pair for a non-title, so `John Smith, MD MD ... G.J. G.J. ...` cost duals × pairs `_normalize` calls (163 at 8 of each, 1,387 at 32, py3.11). Only the first pair's scan can change the answer, since every later pair has the same words in front and more, so it is asked once: 107 and 395. `tests/v2/test_benchmark.py::test_the_paired_initials_title_scan_does_not_cost_quadratically` guards the ratio and fails at c125f69b. The run loop also asks LISTED as "no period", as the single-token test does, and `flip_reports` is set once after the run decision rather than piecemeal. - MEASURED 2026-09-30 against master b98b26e3, every name in this branch's `tools/differential/corpus*.jsonl` parsed on both trees with `nameparser.__file__` asserted on each side: 11 of 1453 distinct names differ, every one of them a name this change's rules.md examples and case rows put in the corpus (the two-pair names `De La Cruz, M.J. K.L.` and `John Smith, X.Y. P.Q.` are not among them: after the second round they read and report exactly as master does). THE POPULATION THAT COULD MOVE is the shape's, and the corpus barely holds it: over master's 1441 distinct names, 13 have a pair opening the part after the first comma, and `John Smith, A.B.` is the only one behind two or more NAME words with an unlisted, non-CJK pair (`Smith Jr., A.B.` has one name word, `Kenneth Clarke Q.C., M.P.` and `Virginia G. Essandoh, J.D.` hold listed acronyms, the rest one word) — so it is the only mover over that corpus, and the count is evidence about the corpus rather than about the rule's reach. Recompute: check out the parent into a separate worktree, parse every corpus name in each tree under `PYTHONSAFEPATH=1` with the tree's root first on `sys.path`, and diff `as_dict()` plus the sorted ambiguity kinds; for the population, take each name whose text after its first comma opens with a token matching `[^\W\d_]\.[^\W\d_]\.?` whole. -- 2026-10-01 (Derek), #564 — THE ALL-CAPS HALF READS THE COMMA POSITION BY DEFAULT, AND THE SWITCH HAS THREE SETTINGS. Supersedes the default of the 2026-09-14 entry above (its reasoning stands for the positions it was argued over). That entry turned the whole caps half off because French and Korean records write the SURNAME in capitals; but the convention writes them at the end of a name (`Jean DUPONT`) or before a comma (`DUPONT, Jean`), never after a comma behind a full name, so the reason for the off default never reached the comma position and that position was switched off with it. The corpus held three names of exactly that shape — `Ahmad Jayadi, CHA`, `John Smith, RAI`, `The Rt Hon Kenneth Clarke QC MP, HMG` — all credentials, all read as the given name at 2.3.0. - DECIDED, with Derek's calls marked. (1) `Policy.unlisted_caps_suffixes` becomes a `CapsSuffixes` StrEnum, OFF / AFTER_COMMA / EVERYWHERE, default AFTER_COMMA (Derek: an enum over two booleans or a mixed `bool | "comma"`, so every state is named and no meaningless combination exists). The field was new in 2.4 and unreleased, so no deprecation is owed; a bool raises a TypeError naming both replacements. The third value exists because redefining "off" would have left no setting that keeps a given name written in capitals (Derek's question, `García Márquez, GABRIEL`). (2) AFTER_COMMA reads only the part right after the first comma, behind two or more name words — the comma's own structure decision, which segment makes from the text before classify runs. The given part's last word after a family comma (`Doe, John XYZ`) and a maiden clause's last word are trailing slots and stay behind EVERYWHERE, as the comma-less end of a name does. (3) A lone two-letter word is never admitted at the comma, in any setting (Derek): it is the undotted form of #563's paired initials, and two words before the comma may be one surname, so `García Márquez, MJ` keeps given 'MJ'. Inside a longer part a two-letter word is read exactly as #563 reads the dotted pair — measured 2026-10-01 on the shipped tree by swapping `MJ` ↔ `M.J.` in runs of one to three words from {MJ, PhD, XYZ, CPA, Jr., MA, LEED, III} after seven name prefixes whose own words carry the contrast (`García Márquez`, `Smith Jones`, `Mr García Márquez`, `de García Márquez`, `García Márquez née Smith`, `John st Smith`, `García y López`): the comma's structure decision differed on 0 of 1295. After five all-caps prefixes it differed on 325 of 925, by design, `MJ` being no class member without the contrast while the dotted shape is case-free (`GARCÍA MÁRQUEZ, CPA MJ` keeps given 'CPA' where `CPA M.J.` makes the run). Recompute: run segment and read `ParseState.structure` for each pair. The equivalence is the comma's decision only: at the given part's last word after a family comma a dotted `M.J.` is a credential by shape (S3) where `MJ` is one only under EVERYWHERE, so `García Márquez, MA M.J.` reads suffix 'M.J.' and `MA MJ` middle 'MJ' — so `García Márquez, MJ PhD` keeps given 'MJ' as `De La Cruz, M.J. PhD` does, and C1 says so by pointing at #563's own sentences rather than restating them. Two restatements were wrong: the first draft's "three letters or more, or a run holding such a word" (both reviews), and its replacement, "with only a credential behind it ... while a credential in front makes the run", which #563's vocabulary rules contradict (`John Smith, MA MJ` reads given 'MA'; `García Márquez, MJ XYZ` makes the run). (3b) An unlisted all-caps word is a by-shape member of C1's run (#544), beside listed credentials, not only in a run of all-caps words (Derek, after review): before, `John Smith, XYZ` read suffix while `John Smith, PhD XYZ` read given 'PhD' and `John Smith, XYZ Jr.` given 'XYZ' — more evidence for a credential producing a name reading, already true under the old `True` and made a default by this change. The contrast has to be the NAME's, and it is carried only by one of the name's own words before the comma (`own_words`, so no maiden clause or delimited content) holding a capital and whose last LETTER is lowercase (`_vocab.written_as_a_name`), with no period and not claimed by a wordlist as a title, particle, connective, credential, generation, maiden marker or honorific (`_vocab.claimed_as_non_name`) (Derek). Every earlier draft asked for "a lowercase letter somewhere, less some exclusions", and each leaked a class the next review found: a mixed-case credential (`LLOYD WEBBER, ANDREW PhD`), a maiden clause or title (`LLOYD WEBBER née Smith, ANDREW PhD`, `Mr LLOYD WEBBER, ANDREW`), a generation, connective or shape title (`LLOYD WEBBER Jr., ANDREW`, `GARCÍA y LÓPEZ, ANDREW`, `Insp. LLOYD WEBBER, ANDREW`), and glued particles and unlisted lowercase words (`GISCARD d'ESTAING, VALÉRY`, `HAFEZ al-ASSAD, BASHAR`, `LLOYD ap RHYS, DAFYDD`) — no closed list separates an all-caps record from a mixed-case name. A Title-case draft (`str.istitle()`) closed those but read too narrowly, missing interior capitals and elisions (`DiCaprio`, `IJzerman`, `al-Rashid`, `d'Estaing`, `McDonald` — every one a mixed-case name that kept a given 'XYZ') and taking a lone capital as Title case (`de GAULLE C, CHARLES` lost its given name). Derek weighed "contains a capital" (an all-caps record with any lowercase word then always carries the contrast) and "not all capitals but contains a capital" (which takes `d'ESTAING` and `al-ASSAD`, the lowercase standing before the capitals) and first chose a capital directly followed by a lowercase letter, with a leading `Mc`/`Mac` skipped. Its review found the skip was one member of a class: any capitalized prefix glued to a surname written in capitals supplies such a pair (`LLOYD FitzGERALD, RONALD`, `PAOLO DeVITO, MARCO`, `JEAN LaFLEUR, PIERRE`, `DICK VanDYKE, JOHN`, `PAUL DuBOIS, JEAN`, `JEAN St-PIERRE, MARC`, and `LLOYD SMITH-McDONALD, RONALD`, the skip applying only at a word's start), as does `ß`, which has no capital (`MÜLLER WEIß, HANS`) — each lost its given name — while titlecase digraphs failed it (`Džokić Ljubić, XYZ`) and it cost a frame per letter. The shipped test, Derek's: a capital in the word and its last letter lowercase. A surname written in capitals ends in a capital whatever is glued in front of it, so every one of those records keeps its given name with no prefix list, and `McDonald` and `Džokić` pass. "Last letter" is the last letter that carries CASE, not the last character: the word is composed (NFC) first; every non-letter and every caseless letter is passed over (`Jones'`, `Smith2`, `Smith)`, `Jonesʼ` with U+02BC, the transliteration letters in `Wafāʾ`, a trailing `李`); and so is a lowercase letter whose capital is not one character (`ß` → `SS`, `ĸ` with none, the `fi` ligature), which is no evidence of case — `WEIß` is written in capitals, `Weiß` is not. Two cuts got this wrong: the first tested the last character, and its review found every name typed with decomposed accents and ending in an accented letter losing the contrast (`José André, XYZ`, `Lê Thị Hà, XYZ`, `René Noé, XYZ`, `Chloé Zoé, MBA XYZ` read given where their composed spellings read suffix), with `Jones'`, `Smith2` and `Smith)`; the second skipped non-letters but stopped at a caseless letter, so `Asmāʾ Wafāʾ, XYZ` and `JOHN Jonesʼ, XYZ` kept given 'XYZ' (its review). MEASURED 2026-10-02 against `a16927c6`, the capital-then-lowercase commit: the seven capitalized-surname prefixes (`FitzGERALD`, `DeVITO`, `LaFLEUR`, `VanDYKE`, `DuBOIS`, `St-PIERRE`, `SMITH-McDONALD`), `MÜLLER WEIß` and `KUJAĸ de PETERSEN` keep their given names, `Džokić Ljubić` gains the contrast, `JOHN O'NEILL's` and `JEAN-pierre DUPONT` gain one (accepted below), and nothing else moves; the decomposed-accent names, `Asmāʾ Wafāʾ` and `JOHN Jonesʼ` read as `a16927c6` reads them. OFF on this tree equals `unlisted_caps_suffixes=False` on `7394078a` (origin/master at the time) on all 4286 inputs. RECIPE, the comparator being those two commits (`git archive` each): the inputs are every name in `tools/differential/corpus*.jsonl`, every match of the regex `"([^"\n]{2,60})"` in `tests/v2/cases.py` holding a comma (a text match, not a literal parser, so fragments of case notes are included, harmlessly; extracting the literals exactly with `tokenize` gave 4273 inputs and the same conclusions), and each of these prefixes joined by ", " to each of the parts `XYZ`, `ANDREW`, `ANDREW PhD`, `PhD XYZ`, `MJ`, `XYZ Jr.` — `John Smith`, `DiCaprio LaBeouf`, `IJzerman IJsselmeer`, `al-Rashid al-Hassan`, `d'Estaing d'Orléans`, `McDonald MacLeod`, `Džokić Ljubić`, `Asmāʾ Wafāʾ`, `John Smith-Jones`, `john smith`, `García y López`, `LLOYD WEBBER`, `GISCARD d'ESTAING`, `HAFEZ al-ASSAD`, `LLOYD McDONALD`, `LLOYD FitzGERALD`, `PAOLO DeVITO`, `JEAN LaFLEUR`, `DICK VanDYKE`, `PAUL DuBOIS`, `JEAN St-PIERRE`, `LLOYD SMITH-McDONALD`, `MÜLLER WEIß`, `KUJAĸ de PETERSEN`, `LLOYD ap RHYS`, `de GAULLE C`, `Mr LLOYD WEBBER`, `LLOYD WEBBER née Smith`, `JOHN O'NEILL's`, `JOHN SMITHs`, `JEAN-pierre DUPONT`, `JOHN Jones'`, `JOHN Jonesʼ`, `JOHN Smith)`, `JOHN Smith2`, and the NFD forms of `José André`, `Lê Thị Hà`, `René Noé`, `Chloé Zoé`. Parse each under OFF, AFTER_COMMA and EVERYWHERE and compare `as_dict()`, the ambiguity kinds and `capitalized(force=True)`; the population is a fact of the tree it runs on (4286 here), since the corpora and the case table grow. ACCEPTED with it (Derek): a name written wholly in lowercase reads the listing form, `john smith, XYZ` keeping given 'XYZ', an unlisted title written in mixed case (`Doña GARCÍA LÓPEZ, MARÍA`) still carries the contrast, and so do two writings in an otherwise all-caps record that end in a lowercase letter: a glued possessive or plural (`JOHN O'NEILL's, XYZ`, `JOHN SMITHs, XYZ` read suffix `XYZ`) and a hyphenated part written in mixed case or lowercase (`JOHN SMITH-Jones, XYZ`, `JEAN-pierre DUPONT, XYZ`). ACCEPTED with (4): in a mixed-case name a capitalized given name followed by a generation goes the same way as one alone, `García Márquez, JUAN Jr.` reading suffix 'JUAN Jr.', reported; and behind a particle-led surname of two name words the flip leaves no given name at all, `De La Cruz García, MARÍA` reading family 'De La Cruz García', suffix 'MARÍA', because P1 reads a never-given particle's part as all surname. (4) ACCEPTED (Derek): a given name written in capitals behind a two-word surname without particles reads as the credential, `García Márquez, GABRIEL` giving given 'García', family 'Márquez', suffix 'GABRIEL', reported; capitalizing the given and not the family is not a convention, and OFF is the escape hatch. #575 was the prerequisite: before it, the count read `Van Buren, MARTIN` and `De La Cruz, MARIA` as two or three name words and this default would have taken their given names too. - WHAT MOVES, measured 2026-10-01 with the gate at all five baselines. The three corpus names above, plus the C1 examples `John Smith, XYZ` and `John Smith, LEED AP` — the latter closing a `deviates: #291` marker rules.md had carried under a closed issue — classified by `fix(#564)` in every ledger; the radar-unclassified count is what it was before the change at every baseline. `John Smith, RAI` and `Ahmad Jayadi, CHA` read suffix again by their capitals, as they did by vocabulary before #342 removed both words: parity at 1.4.0, only the comma's report at 2.0 through 2.2, so the #342 rule's `fields` lose `given` (the OVER-DECLARED check) and the watched shape for `John Smith, RAI` is re-recorded at those four baselines. `Smith, XYZ` keeps given 'XYZ' and, at the default, reports nothing; EVERYWHERE still reports the declined fork there, as it did. - CASE REPAIR FOLLOWS THE READING. The comma decision is segment's, made from the text, while case repair keeps a word in capitals only where classify wrote the shape tag (rules.md#R4); the first draft tagged only under EVERYWHERE, so `parse("John Smith, XYZ").capitalized(force=True)` rendered 'John Smith Xyz' at the default and 'John Smith XYZ' under EVERYWHERE (found by the docs review, axis 5). Classify now tags a caps-shaped word in the part a suffix comma opened under AFTER_COMMA too, so both settings render that part alike, and rules.md#R4 names the caps shape beside the dotted one; `test_render`'s forced-repair test fails with the tagging removed (an R4 example could not witness it, R5 leaving a mixed-case name unrepaired unless forced). A caps word in a third or later comma part is still tagged only under EVERYWHERE, so `John Smith, MD, XYZ` renders 'Xyz' at the default, as master does. - COST, `sys.setprofile` call counts (mean of 50, after one warm-up) against `git archive origin/master`: `Smith, John` 183 → 183, `Smith, JOHN` 183 → 183, `Smith, XYZ` 182 → 182, `John Smith, PhD` 217 → 217, `John Smith, MA` 252 → 252, `John Smith, Ph. D.` 254 → 254, `John Smith, CPA` 217 → 217, `John Smith, MD PhD` 260 → 260, `John Smith, MBA CPA` 261 → 261, `Dr. Juan de la Vega III` 366 → 366, `John Smith, XYZ` 251 → 266, and an all-caps record with a capitalized given name and a credential, `LLOYD WEBBER, ANDREW PhD`, 338 → 364, a constant +26 that does not grow with the name's length (`'LLOYD'*64 + ' WEBBER, ANDREW PhD'` 653 → 679; the second draft's per-character contrast test had made that +350). Past the all-caps check the contrast is linear in the words before the comma, about four frames a word (the generator, the case test's call, and the own-words walk's fold and marker test; `'GAULLE ' + 'ap '*k + 'GAULLE, CHARLES'` is +7, +26, +38, +54 over master at k = 0, 1, 4, 8), and constant in a word's letters (`'de ' + 'GAULLE'*k + ', CHARLES'` 253 → 275, 343 → 365 and 631 → 653 at k = 1, 16, 64), where the capital-then-lowercase draft grew a frame per letter; all paid only once a caps word is in hand. The comma test is on by default, so three C-level checks go before any call: two or more words before the comma (the first draft lacked it, and `Smith, JOHN`, a common record format, paid +25 for a flip it can never make — the code review), the first word in capitals, and the lone-two-letter length. Every caller of the caps predicate asks first, in C, what it would decline anyway — alphabetic capitals, not a listed suffix word — so a listed credential never pays for the call (the review of the first fix found `John Smith, CPA` +4 and `John Smith, Ph. D.` +5 before these). The 2026-09-14 entry's recompute recipe predates the enum: its `on` is `Policy(unlisted_caps_suffixes=CapsSuffixes.EVERYWHERE)` now, and its `off=Parser()` is `Policy(unlisted_caps_suffixes=CapsSuffixes.OFF)`, `Parser()` being AFTER_COMMA. -- 2026-10-02 (Derek), #573 — a never-given particle opening the given part after a family comma leaves no word to spare; the member behind it is the surname it heads, unless mixed-case capitals say credential. The record is the P6 section's entry of the same date. -- 2026-10-02 (Derek), #578 — A PARTICLE BETWEEN A CREDENTIAL AND A TITLE-CASE WORD STAYS SILENT, AS IT READS. `Doe, Jane PhD vd Ma` reads given 'Jane', middle 'vd Ma', family 'Doe', suffix 'PhD', reporting nothing. No new rule decides that: S2's given-part slot already says a particle in front of a member takes the member out of the slot, and where the word behind reads as a name the two are one name, asked nothing and reporting nothing — `vd` reads as `van` does (`Doe, Jane PhD van Ma`, the same reading), and S2's own boundary example is `Doe, John van Ma`. The writing is malformed whichever reading is meant: a Title-case `Ma` behind a lowercase `vd` behind a `PhD` is neither a Dutch listing nor a credential run as anyone writes one. A reader leaning anywhere leans suffix (Derek: both words follow a known credential, and the writer likely did not know how suffixes are capitalized), but that is a guess about a writer's mistake, not a convention the writing carries. Declined: (a) reading `vd Ma` as suffixes, which needs code to make a credential speak across a particle — against P2 (a particle joins the word behind it) and #554 (`vd` speaks for nothing behind it); (b) reporting the fork, which needs an emitter on the family-comma path for a decision the particle chain makes before assign knows which read the part gets (AGENTS.md: prefer leaving a fork silent and documenting the omission over machinery for input nobody reads with confidence). Where the writing does speak the parser already follows it: `Doe, Jane PhD vd MA` reads suffix 'PhD vd MA' and reports (#573). Pinned by the case row `a_particle_after_a_credential_heads_a_title_case_name`. -- 2026-10-04 (Derek), #602 — A CREDENTIAL AFTER THE NAME CORE STARTS A RUN TO THE END OF ITS PART. An unambiguous Latin suffix word (the Latin restriction retracted by #679, below) of two or more letters standing after two name words with no comma (a lone particle not counting toward the two; qualified by #627, below), or after the given word in the part after a family comma, makes every later word of that part a suffix, except a title word, which reads as a title: `John Smith PhD Jones` → suffix 'PhD Jones', `Eric H. Holder Jr. Attorney General` → suffix 'Jr.', title 'Attorney General'. A name word the run absorbs is reported `suffix-or-name`, a reader hesitating over `Jones` there. Before this the three comma shapes disagreed — middle 'Smith PhD' with no comma, middle 'Jones' and suffix 'PhD' after a family comma, and suffix 'Jr., Jones' after a suffix comma (C2's tail) — and the no-comma reading made a credential a middle name, the one reading nobody intends. The suffix-comma path already read it this way, and the comma spelling of the Holder name always did. - WHAT STARTS THE RUN is one predicate, `_pieces.starts_a_credential_run`, read by the trailing peel (`credential_run`, applied to every finished peel, so assign, P5's reserve, P2's chain stop and the maiden walk see one answer) and by the given-part walk. EXCLUDED, each with its reason: a TITLE WORD — 746 title words are in no suffix set (measured 2026-10-04: `len(L.titles - L.suffix_acronyms - L.suffix_words)` for the default lexicon), many of them surnames (`King`, `Bishop`, `Judge`), so `Mary Jane King Smith` keeps family 'Smith'; a title is read from the end of a name only behind a comma or with a period (H5, Derek). The AMBIGUOUS CLASS, which carries no evidence of which it is. A SINGLE LETTER, in any case: a lowercase `i` or `v` carries no `initial` tag, and found by the v1 off-switch test, `Josep Carod i Rovira` with `i` removed from the conjunctions read suffix 'i Rovira'. A CONNECTIVE. A word that is also PARTICLE or BOUND GIVEN-NAME vocabulary — found by the suite, not foreseen: `abd` is in the credential list (ABD) and heads `Abd Allah`, and `Mohamed Ali Abd Allah` read suffix 'Abd Allah' until it was excluded. The NON-LATIN HONORIFIC words, an honorific followed by a name word being a different question from a credential followed by one (retracted by #679, below). A word that is also a title (`MD`, `Ms`, `Sr.`) does start the run: position decides, and after the name core it is the suffix. - DECLINED: lifting the credential out — `John Smith PhD Jones` as given 'John', middle 'Smith', family 'Jones', suffix 'PhD', the family-comma path's old reading. It keeps every name word in a name field, but it reads a credential as something written in the middle of a name, which is the reading the rule rejects. - REVERSED BY THIS ENTRY rather than edited: the #578 bullet above (`Doe, Jane PhD vd Ma` now reads suffix 'PhD vd Ma', the credential starting the given part's run), and two of the three shapes #544's ACCEPTED LIMITS kept out of the company's reach — a title standing between the credential and the member, and a member a particle chain had taken. The case rows `a_title_between_degree_and_member_keeps_it_a_name` and `a_member_the_particle_chain_took_is_out_of_reach` keep their ids and now pin the run. The third, the merged `Ph. D.`, stays out of reach: that piece is outside the walk the run is read over. - MEASURED 2026-10-04, every name in `tools/differential/corpus*.jsonl` parsed on this tree and on aa401d80 with `nameparser.__file__` asserted on each side: 7 of 1,514 move, four of them this entry's own new rules.md#S2 examples. The three already in the corpus: `Jane van der Berg née Jr Jones` (suffix 'Jr Jones', where the marker's declined clause had left middle 'van der Berg née Jr'), and `John Doe, MD - PhD - FACS` and `Smith, MD - PhD - FACS`, whose dash inside the given part, a word there under every policy (#549), is now in the run: suffix 'PhD - FACS' where they read middle '-', suffix 'PhD, FACS'. The comma-part question — whether a part OPENING with a credential makes the comma a suffix comma (`John Smith, PhD Jones` still reads family 'John Smith') — is #603's. - -- 2026-10-07 (Derek), #614 — S2'S TRAILING RUN IS READ ONCE, BETWEEN P3'S CONNECTIVE JOINS AND THE PARTICLE CHAIN, AND BINDS. The issue's diagnosis held on the day: the words-to-spare count is taken over pieces and group's joins make pieces, so group re-asked the peel before and after its own joins -- the particle chain's stop and its re-ask and rollback, the bound-given reserve's two views (P5) -- and assign then read the run again. Now the run of a main segment (no comma, or the part before a suffix comma) of three or more pieces is read once, after the Ph. D. merge, the maiden take and P3's connective joins, which do not depend on the run and make the units it counts, and before the particle chain and the bound-given join, which do; the run is split off, the chain and the join group the name in front of it, and the run is put back unchanged (`_pieces.read_trailing_run`). The chain's stop is the end of what is left and the reserve counts the name pieces the read left; assign places and reports the run off the read (`ParseState.tail_reads`, `_assign._placed`). The read COUNTS a particle run and the plain name words after it as the one unit the chain will make of them (`_counting_view`, a copy; rules.md#P3's own count, mechanisms.md#UNIT-PARTITION), the words both particles and suffix vocabulary straight behind a particle among them, while anything the read weighs stays a word of its own -- suffix vocabulary, the ambiguous class and its shape-only members, roman numerals by shape, initials, period-marked titles. - STALE IN THE ISSUE, measured before designing: segment no longer decides the comma or emits S2 reports (#613 moved C1 to `_comma.decide`); assign predicts P1's fold but no longer P6, which it decides since #613; the two comma restart inputs (`John Smith, PhD vd DO`, `PhD DO DO`) no longer reach the chain, decide binding that part first; `credential_run` already applied in one place; and #613's `name_word_count` reads particle and suffix facts at P1's reach and ignores connective and bound-given joins, so S2 could not share it as it stood. The issue's 'licensed by neighbour', read literally, moves about 25 inputs (`JOHN SMITH MA`, six S3 example lines) and still five read as scoped to the duals, so it was not taken; the counting view's particle run is the one part of it the design needed. - THE FIRST DESIGN AND THE REVIEW THAT REVERSED IT. Derek decided first that the read WINS OVER EVERY JOIN, connectives included -- prototyped both ways in a scratch copy -- and that implementation passed the suite, the gates and the fingerprint. #614's PR review then found it broke what the corpus never tests: a salutation alone read its connective as the family name (`Mr. and Mrs.` gave title 'Mr. Mrs.', family 'and' -- the H5 chain took 'Mrs.' before P3 could join), a joined leading salutation counted as three words (`Mr. and Mrs. Jr` gave family 'and', `Mr. and Mrs. Do` a spurious report), and the read counted a particle chain's or a connective join's words one by one where rules.md#P3 says a joined part is one name word wherever another rule counts (`Freiherr von Berg X.Y.Z.` and `Juan y Garcia X.Y.Z.` took the acronym as a credential against every release, while `Freiherr Berg X.Y.Z.` kept it the family). DECIDED (Derek, 2026-10-07): connectives join first, and the read counts the chain's units -- which returns every one of those to master's reading, and most of the first design's 'tail beats connectives' class with them. A second-round finding the counting view itself made: folding the plain words after a particle into its unit swallowed a shape-only roman numeral (`Juan de la Vega VI` read family 'de la Vega VI'), the #610 class AGENTS.md's spelling sweep records, so a word the numeral fork reads by shape stays out of the unit. - DECIDED BEFORE THE REVERSAL AND STANDING: (1) the read wins over the particle chain and the bound-given join; (2) the consequences below, accepted as a class. - WHAT MOVES. One realistic reading: a trailing period-marked title behind a particle chain is the title, `John van der Berg Prof.` reading title 'Prof.', family 'van der Berg' (every release 1.4.0-2.3.0 and 0c54662b: family 'van der Berg Prof.'), which retires the cost rules.md#H5 recorded as accepted; the six heads of the stress grid below that end in a particle chain move the same way with `Dr.` or `Prof.` behind them. The garbage `Freiherr von Berg Dr. and Ed. Prof.` reads title 'Freiherr von Berg Dr. and Ed.', family 'Prof.' (0c54662b family 'von Berg Dr. and Ed. Prof.'): the title is read off before the chain, which then joins a piece carrying the joined title and reads as a leading title, so assign reads the name again. P5 no longer re-reads the joined view: `abdul J. V` and `Sir abdul J. V` read given 'abdul J.', family 'V' -- 1.4.0's reading restored; 2.0 and 2.1 read suffix 'V', 2.2 and 2.3 kept 'J.' a middle name -- the initial in front suppressing the numeral fork in the one reading taken, three name words standing and the join leaving two; and `Sheik abdul Ma` reports the pick the read weighed (decisions.md#P5, this date). Reports: a pick the particle chain takes into the name is reported by the chain, in its own words, and group takes it out of the read's picks so assign does not report it again (#614's /simplify: assign first skipped a pick it found already reported, inferring group's decision from group's output), so `Jan vd Ma`, `John de Ma`, `John van der Berg Ma` and `Nguyen Van Lac` report as on 0c54662b. The second review found why it must be the chain: assign words a pick by the role it holds before post_rules, and under a leading title-particle that is the given role the fold then moves, so with assign reporting `Freiherr von Berg Ma` told the caller 'read as a given name' of a word in the family. The chain also reports a member the read never weighed, one it takes short of the run, its scan ending at the next particle (`Freiherr von Berg Ed Do` reports 'Ed'). One report moves for the better: a pick the run keeps is reported as the suffix it reads as, `Juan y Garcia Mc Do MD van Ed` reporting 'Ed' 'read as a suffix' where 0c54662b said the particle chain took it into the name, of a word in the suffix. - THE SECOND REVIEW (bfa798fb, 2026-10-07) found four defects in the counting view and the report hand-off, every one on a shape the corpus, the gates and the first stress grid held no instance of, and each now has a case row that dies without its fix. (1) The read started at the end of the leading titles counted in PIECES, applied to the view: a title that is also a particle opens a unit inside the titles, so the index landed past the name and `Freiherr St van Berg MA` read family 'St van Berg MA' (0c54662b suffix 'MA'); the view now returns where the name starts in its own indices -- the first unit opened at or before the titles' end, since a unit opened inside them makes a name of them (SUPERSEDED IN PART by #624, decisions.md#P2, 2026-10-10: the chain now opens no unit inside the titles) even where it ends short (`Freiherr Freiherr Prof do` lost its report on 'do' until that second half). (2) Assign reported the chain's picks, in the wrong field's words, above. (3) The view made units INSIDE #602's credential run, which hid a title word from the run's title test: `John Smith MD van Secretary Jones` lost title 'Secretary' to the suffix. Past a run starter a unit is now the bare particle run, as the chain's own merge leaves it. (4) A `title` piece-tag exclusion in `_counted_as_one` made garbage differ from 0c54662b (`Freiherr von Mr. and Mrs. X.Y.Z.`) and guarded nothing (3) did not; removed rather than pinned. - NOT THE SAME READ AS M2'S: the maiden take reads its own clause-free view of the words before any join, word by word, as rules.md#M2 states, and its Accepted clause already covers what that does where a particle chain would take the run; S2's read counts the chain's units after P3's joins. Two reads of one name answer different questions -- where the clause ends, where the name's run starts -- and are kept apart rather than made one here. - RESIDUALS, the paths that read as before #614: a segment of fewer than three pieces (no join reaches it); a run with a name piece inside it (an H5 title in front of a name word the peel declined, `John Dr. G.J.`, `John van V Dr. V`), which is not split -- group re-asks and assign reads for itself; and a name group's joins leave without a name word in front of the run, which assign reads again so it keeps one -- the same procedure as before #614, not always the same answer, the Freiherr garbage above being the measured case. - MEASURED 2026-10-07 against 0c54662b (#618's merge), py3.11, `nameparser.__file__` asserted on each side; #619, merged between, moved no reading. The two #617 harnesses as decisions.md#P3's 2026-10-06 entry records them (the fingerprint, 78,324 parses; the connective grid, 485,300), and an S2 stress grid built for this entry: the 22 heads `John`, `John Smith`, `John Q. Smith`, `Mary Ann Smith`, `John van Smith`, `Juan de la Vega`, `John van`, `anh van`, `Jan Freiherr von Berg`, `Freiherr von Berg`, `abdul salam`, `abdul rahman al-said`, `Juan y Garcia`, `Josep Carod i Rovira`, `Jane Doe nee van der Berg`, `Dr. John Smith`, `JOHN SMITH`, `john smith`, `Smith`, `de Mesnil`, `Kim Min Jun`, `Ortega y Gasset`, each followed by every one- and two-word tail over the 35 words `PhD MA Ma DO Do do vd VD mc Mc MD Jr Jr. Sr III V V. VI I X.Y.Z. XYZ Ph. D. Esq. Prof. Dr. Sir MBA RN G.J. Ed van and Jones Ed. ba` (`Ph. D.` one word of the list); then 25,000 draws seeded `random.seed(614)`, each `choice(heads)`, `randint(3, 5)` and that many `choice(words)`, joined by spaces; then every head before a comma with every two-word pair behind it, and `Smith, ` and ` , John` for every word; deduplicated, under four policies (default, both family-first orders, strict comma suffixes) -- 81,138 names, 324,552 parses, comparing the seven roles, every ambiguity's kind and detail, `initials()` and `capitalized()`. Fingerprint: 6 parses move, `John van der Berg Prof.` under six policies, and nothing else. Connective grid: 1,248 move, no corpus or case name among them: 1,212 the title class, and 36 garbage shapes holding a title or a credential run behind a leading particle (`de der Dr. MA PhD Vega y Lopez` reads suffix 'PhD Vega y Lopez', #602's run, where 0c54662b kept every word the family), 8 of them moving a report's grouping alone (below). Stress grid: 8,472, no corpus or case name; 48 move reports alone, 20 the suffix wording above and 28 the grouping of an absorbed word's report where the run the view stops unit-making at is not the one the peel reads (`anh van Do Sr DO Jones MBA` reports 'DO' and 'Jones' where 0c54662b reported 'DO Jones'); restricted to a head and one tail word under the default policy, 12 parses, the six particle-chain heads with `Dr.` and `Prof.`, all the title class. (At bfa798fb, before the second review, these were 30, 2,936 and 20,532, the slice 33.) Whole tail readings per corpus name (`tail_reading` calls, 1,536 names): 1,125 read once, 56 twice, none three times, against 1,095, 55 and 30 on 0c54662b; peel passes, which count H5's fixed point's own re-peels, 1,215 names at one against 583. Five differential gates exit 0 with the one move classified `fix(#614)` at every baseline. - COST, `tools/perf/call_count.py` on each interpreter against adf6da88 (#619's merge): `parse` 317 -> 308 on 3.11 and 296 -> 290 on 3.12-3.15, `facade` 354 -> 345 and 333 -> 327 (decisions.md#parse-cost, this date). Per name on 3.11, warm: the Vega name 376 -> 368, and every segment of fewer than three pieces unchanged, never reaching the read (`John Smith` 152, `Smith, John` 161, `John Smith, PhD` 206). -- 2026-10-08 (Derek), #620 — S2'S READ COUNTS THE CHAIN'S UNITS OVER THE PIECES AS WRITTEN, RATHER THAN MERGING A COPY OF THEM. The #614 read above counted through a merged copy of the pieces, a second statement of P2's chain with its own stop list, and its reviews found three defects that were that copy's: an index applied across the two layouts (`Freiherr St van Berg MA`), a unit formed inside #602's run hiding a title (`John Smith MD van Secretary Jones`), and a shape-only numeral folded into a unit (`Juan de la Vega VI`), each patched in the copy. Now `_pieces._chain_units` marks each piece by the unit the chain will put it in -- OPENS, JOINED (a name word the chain joins to the run in front of it) or BOUND (a particle inside the run, a word the run took; retired by #625, this section, 2026-10-10, group merging the run first) -- asking `chain_run_end`, the chain's run end, which `_group`'s chain now calls too. The peel's words to spare (`second_unit`) and #602's run start count in those marks and the peel stops at any piece the chain joins into a run, which is a name word whichever mark it carries; everything else reads the pieces as written, so nothing is hidden, and the copy, its view index and its credential-run gate are gone. - WHAT THE COUNT KEEPS. A word the reading weighs -- a suffix word of either kind, the ambiguous class, a word in it by shape, an initial, a roman numeral by shape, a period-marked title word (`_weighed`) -- keeps a unit of its own though the chain may join it, as a position always counted it; only the plain words behind a particle run are JOINED, and a unit with nothing past its opener moves no name start into the titles (SUPERSEDED IN PART by #624, decisions.md#P2, 2026-10-10: no unit opens inside the titles, so the gate is gone). The first prototype marked every word the chain reaches JOINED, by the chain's own rule alone, and the review found it wrong in two shapes the earlier count read right: `Freiherr von Berg MA X.Y.Z.` kept 'MA X.Y.Z.' in the family (a member the chain would join counted as no word in front of the next), and `St St VI` and `Freiherr St MA` read title 'St St', 'Freiherr St' with the numeral or acronym the whole family (a unit of words the read then takes, which the chain never builds). Both masters read all three as they read now. (#624 since reads `St St VI` and `Freiherr St MA` as title 'St St' / 'Freiherr St' again, by a different route: the chain and the count now share one leading position, the last title-particle, so neither builds the unit -- decisions.md#P2, 2026-10-10.) The copy's exclusion list had been carrying the count positions always kept -- a member in front counts -- and the marks keep it without the copy. The numeral fork still reads the piece in front of the numeral as written -- reading the unit's first word broke rules.md#P2's `John van der J. V` -- so `John van B and Smith X` reads family 'van B and Smith X' again, the reading #614's copy had moved. - ALSO COUNTED IN UNITS: #602's run start, a particle opening a chain run counting once and its bound particle not at all (`John der la Jr. Prof. Smith` keeps its run, `van la Smith Secretary Jr. Dr. Smith` starts none, as on master). And `credential_run`, on the read's path, gathers adjacent particles into the one absorbed piece the chain makes of them (`Dr. John Smith Esq. RN Mc Mc` reports 'Mc Mc'; SUPERSEDED by #625, this section, 2026-10-10: group merges the run before the read, which absorbs it as one piece); M2's clause-free view reports word by word as it always has. - DECLINED, measured: a units test in `tail_reading`'s resume condition. A splice lowers a count only by taking out titles that open units of their own, and with fewer than two units in front of it every piece there is in the first piece's chain run, which the titles behind it join; fuzzed over 372,330 names, the test changed nothing. - CLASSIFICATION, found writing this entry's rows: eight case rows #614 added as `parity` did not match 1.4.0 -- they had been compared against master -- and are relabelled with the change each reading came from, bisected (#424, #289, #296, #516, #602 twice; R2 and P2 for two that read so since the 2.0 pipeline). - MEASURED 2026-10-08 against 4278693d (#621's merge), py3.11, `nameparser.__file__` asserted, with the #614 harnesses and recipe above: fingerprint 0 moves; connective grid 0; S2 stress grid 12, three garbage names under four policies, none in the head-plus-one-word slice -- `Freiherr von Berg Dr. X.Y.Z. VD` reads title 'Freiherr Dr.', suffix 'X.Y.Z. VD' (H5's trailing title, the class #614 moved; master family 'von Berg Dr. X.Y.Z.'), `Jan Freiherr von Berg VD V and ba I` reads family 'VD V and ba I' as adf6da88 did (master suffix 'I'), and `de Mesnil Ma do mc and Ph. D.` reports the 'do' the peel took beside adf6da88's 'Ma'. Five gates exit 0. Frames unchanged, 308 on 3.11 and 289 on 3.12-3.15 (decisions.md#parse-cost). - /SIMPLIFY (2026-10-08), measured. Each pass of `tail_reading` had rescanned for the second unit from the front, a C-level quadratic no frame guard saw: `Freiherr von` + `Berg `*n + `MA Dr. `*n read 5.77x for 4x the input against master's 4.08x; the lookup is now made once per read and handed to every pass (4.03x), and `test_the_second_unit_is_found_once_per_read` counts it. `_weighed` is pinned to the reading by `test_a_word_the_read_takes_is_never_one_the_count_joined` (no piece marked JOINED is ever in the run the read takes; 320 grid names fail with the test answering False). The gathering of an absorbed particle run reads the BOUND marks, and #602's run start one condition. Grids byte-identical, frames unchanged. Left for a follow-up, each moving readings: merging the tail's particle runs in group before the read, which would retire the BOUND stop and the gathering (80 grid names move, `Dr. mc mc`, and the chain's particle report needs re-keying; done by #625, this section, 2026-10-10, where #624 had already moved those 80); and the name start decided once, the item below. - LEFT OPEN, by Derek's scoping: reading a family comma's given part the same way, and deciding the name's start once for a title-particle head (#620's related paths, which change readings). The second was settled by #624 (decisions.md#P2, 2026-10-10), the first by #623 with a narrower bound rather than a second read (decisions.md#H5, 2026-10-10). -- 2026-10-10 (Derek), #627 — A LONE PARTICLE WITH THE RUN RIGHT BEHIND IT COUNTS AS A NAME WORD. #602's run start needs two name words before the credential, and "a lone particle is not a name word for the count", for `de Mesnil`, where the particle and the word behind it are one surname. Since #620 the count is taken in the chain's units, so a particle that opens a unit counts once (`van der`, `von Berg`) and a P3-joined phrase is one piece (`von und zu`); the lone particle stayed uncounted only where nothing joins it, which is exactly where a credential stands behind it. So `John von PhD Jones` read middle 'von PhD', family 'Jones' -- the chain later taking the credential as the particle's surname -- while `John van der PhD Jones` and `John von und zu PhD Jones` read family 'van der' / 'von und zu', suffix 'PhD Jones', and the comma spelling `Smith, John von PhD Jones` already read suffix 'PhD Jones'. The issue's question was which way to make them agree. - DECIDED (Derek): the particle stops at the credential, so it is a name word of its own and the rest is suffix: `John von PhD Jones` → family 'von', suffix 'PhD Jones'. The input is malformed (a particle is not written before a post-nominal), so the decision is whichever reading the shared count already gives the other forms, not a special case for this one: the existing clause gains its own limit, a lone particle counting when the run starts right behind it, in `run_start`'s one loop. The caller without the chain's units (`trailing_start`) shares the expression so the two cannot disagree, but no reading shows that half: it changes the position it returns for `Smith, John von PhD Jones` (2 where master gave 4), so group's chain takes its stopped path and asks `trailing_start` again, reaching the same reading at 16 more frames (339 → 355 on py3.11; a rare shape no frame guard sees, decided for the shared expression over a per-caller copy); a mutant ignoring the flag there passed the suite with the grid byte-identical (#627's reviews). The 1.4.0 and 2.3.0 wheels read middle 'von PhD', as they read `von und zu PhD`; the joined forms changed this cycle with #602. - Declined: counting every unit, a lone particle included. It moved 2,836 S2-grid parses rather than 2,432, the extra ones all `de Mesnil `, where it counted the surname as two words, and it split the two callers: a leading `van` counted before another particle (`van la Smith Jr. Jones` → suffix 'Jr. Jones') but not before a name word (`van Smith Jr. Jones`), and the case row `van la Smith Secretary Jr. Dr. Smith` caught it. - MEASURED 2026-10-10 against 195c4081 (#630's merge), py3.11, `nameparser.__file__` asserted on each side. S2 stress grid (the #614 entry's recipe above, 324,552 parses): 2,432 role moves, no report-only move; every move is a lone particle -- in that grid `van`, or a dual `do`/`mc`/`vd` in any case -- with an unambiguous credential right behind it becoming the family (under the family-first orders the given name, the family staying first), the run taking the rest. The grid's comma shapes do not move; the review found the same move in shapes the grid lacks, as intended: other particles (`von`, `de`, `du`, `la`, `bin`), and a head before a suffix comma (`John von PhD Jones, Jr.` → family 'von', suffix 'PhD Jones, Jr.'). Fingerprint of every `tools/differential/corpus*.jsonl` name and `tests/v2/cases.py` text at 195c4081 under the three orders (1,925 texts, the seven fields and every report): 0 moves; this change adds one text, its own example, which moves. The five differential gates exit 0. Held by the case row `John von PhD Jones` (`fix(#627)`) against the row `van la Smith Secretary Jr. Dr. Smith`, whose lone particle has a name word behind it and still does not count, and the row `Freiherr von vd PhD Jones`, where the particle standing right before the credential is one the run took and the flag is not asked of it (a mutant asking it of every piece passed the suite until that row). -- 2026-10-10 (Derek), #625 — THE TAIL'S PARTICLE RUNS ARE MERGED IN GROUP BEFORE S2'S READ, SO THE READ NEEDS NO BOUND MARK AND NO GATHERING. Since #614 split the trailing run off ahead of the chain, the read sees particles the chain has not merged yet, and #620 re-created the chain's particle-to-particle merge in the reader: `_chain_units` marked a particle standing straight behind another BOUND, the peel stopped at it, and `credential_run` gathered an absorbed run of them into one report. Now group merges each run of two or more adjacent particles past the chain's leading position (`_pieces.merge_particle_runs`, at the trailing site, after P3's joins and before the read) into the piece the chain would open its run with, so the read counts, stops at and absorbs the run as one piece, and the BOUND mark, the peel's stop for it and the gathering are gone. The merged piece keeps `prefix` and drops `title`: a P3 join carries the title tag in from a word in both vocabularies (`St und Berg`), and on the merged run it read the run as a leading title. The title run and the lead are read once, before the merge, and handed to the read, which had asked both again. The chain's PARTICLE_OR_GIVEN report keyed on its run claiming something past the particle (`j > k + 1`), which a merged run has done already though the chain then claims nothing more (`Freiherr von der`); it now keys on that or on the run being one the merge made (`premerged`) -- not on the piece holding several tokens, which a P3-joined phrase does too, and which moved 11,358 reports on the grid below (`Dr. St Do und`). The rule text is unchanged: rules.md#S2 already counts a particle run, the duals straight behind a particle included, as the one name word the chain makes. What grows is code, not concepts: the read loses a mark and two branches and re-creates nothing of the chain, group gains the merge (about 45 more lines than it removes across `_pieces.py` and `_group.py`, docstrings included). - DECIDED (Derek): retiring the two reader-side mechanisms is worth the moves -- every one is garbage input, now read consistently, and none is a released reading restored. - MEASURED 2026-10-10 against 505d6dec (#632's merge), py3.11, `nameparser.__file__` asserted on each side. A particle-run grid -- `{head} {run} {tail}` for every head in (none), `Dr.`, `Freiherr`, `St`, `John`, `Dr. John`, `Freiherr von`, `John Smith`, `Sir`, `Freiherr St`; every run of one to three words over `van der de la von und mc Mc vd do Do St du`; every tail in (none), `Berg`, `MA`, `PhD`, `Ma`, `Jr.`, `Berg MA`, `Prof.`, `Berg Ph. D.`, `V`, `VI`, `X.Y.Z.`, `née Smith`, `Berg Jr. PhD`, `Berg van`, `, Jr.`, `, John` (a comma tail written against the run) -- 395,148 texts, 1,185,444 parses under the three orders, comparing the seven fields and every report (kind, detail, tokens): 774 role moves (258 texts under each order), no report-only move. Every move holds `St und`, the joined piece carrying the title tag, which master merged into the particle run in front with the tag kept and read as a title (`Freiherr Do St und Berg MA` → title 'Freiherr Do St und Berg', family 'MA'); now title 'Freiherr', family 'Do St und Berg', suffix 'MA', the name part read alike with and without the trailing 'MA'. Not a released reading: 1.4.0 reads first 'Do St und Berg', last 'MA', and 2.3.0 family 'Do St und Berg MA', and over the 258 moved texts under the default order master's seven fields equal 2.3.0's on 152, the tree's on none. The S2 stress grid (#614's recipe, above): no role move, and one text's report under its four policies -- `de Mesnil Ma do mc and Ph. D.` no longer reports 'do': P3 joins `mc and Ph. D.` into one suffix-tagged piece, the merge folds 'do' into it, and the peel takes the piece whole rather than picking 'do', the report adf6da88 did not make either and #620 added (its entry above). The #624 title-particle head grid (decisions.md#P2, 2026-10-10): 0. Every corpus name and case text, and #624's comma-tail grid, under the three orders: 0. The five differential gates exit 0, their reports identical line for line to the tree before. The 2026-10-08 prototype's 80 moves (`Dr. mc mc`) are not in the population: #624 already reads `Dr. Mc Mc` as H3 states. Recorded negative controls (2026-10-10): with the merge off, 30 tests in tests/v2 fail (32 over the whole suite, measured at the review's fix), `test_an_absorbed_particle_run_reports_as_the_one_word_the_chain_makes` among them; with the report keyed on `j > k + 1` alone, the case rows `a_particle_takes_the_dual_behind_it_at_the_head` and `a_particle_run_before_the_run_stays_one_name_word`; with the merge starting at the segment's second piece rather than past the lead, 13; with `title` kept, only `test_a_merged_particle_run_is_a_particle_and_no_title`, the unit test added for it. Cost: +3 frames on the reference row, which holds `de la` (decisions.md#parse-cost, this date). - THE REVIEW (2026-10-10) found the change reaching further, every shape garbage, none given a row (Derek): no move goes the other way, and nothing moved over comma shapes (an empty or nickname-only head, a family comma's given part), connective-separated runs, runs beside a maiden marker or holding a nickname, bound given names, one-case names, or caller lexicons adding title or suffix words as particles. (1) The `St und` class is any P3 join whose left word is in both the title and particle vocabularies, `und`, `and` or `of` (`y` moves nothing: master already reads `Freiherr Do St y Berg MA` as now), a caller's title-particle included (`Freiherr Do Freiherr and Berg MA`: master title 'Freiherr Do Freiherr and Berg', family 'MA'; now title 'Freiherr', family 'Do Freiherr and Berg', suffix 'MA'). (2) A title-particle head, a particle run, then a title and a dotted word (`Freiherr van der Dr. G.J. MA`, 88 texts on its grid under each order): the run now counts as one name word, so 'G.J.' has none to spare, the read takes nothing and group's own chain takes 'Dr. G.J.' -- family 'van der Dr. G.J.', suffix 'MA', as the one-particle `Freiherr van Dr. G.J. MA` reads (master title 'Freiherr Dr.', family 'van der', suffix 'G.J. MA'). (3) H3's give-back behind a leading title where every word is suffix vocabulary, the last two a run of duals (`Dr. vd Jr. mc mc`, about 1,800 parses over two grids): the merged run is a particle piece and no suffix word, so the title stays -- title 'Dr.', family 'vd', suffix 'Jr. mc mc', where master gave 'Dr.' back as the given name with 'vd' still standing; `Dr. vd Jr. mc` still gives it back. The review also found two runs in one name unreached by any test (an off-by-one in the merge's bookkeeping passed the suite), now the rows `two_particle_runs_merge_each` and `a_second_run_of_duals_is_the_family` (parity) and a half of the merge's unit test, which also holds that a lone particle is no merged run; and the read's title run taken before the merge held only by the frame bands, now `test_the_read_takes_the_title_run_as_written_before_the_merge` (`Dr. mc Jr mc mc`, which with the run asked again after the merge, by the read or by group, reads family 'mc Jr mc mc'; the fix's own review found the group-side half unheld by a read-level test, so it pins the parse too). -- 2026-10-14 (Derek), #679 — ANY UNAMBIGUOUS SUFFIX WORD OF TWO OR MORE LETTERS STARTS THE RUN, WHATEVER ITS SCRIPT; THE HONORIFIC EXCEPTION IS RETRACTED. #602 excluded "the NON-LATIN HONORIFIC words, an honorific followed by a name word being a different question from a credential followed by one", and drew that line by script, the vocabulary having no honorific marker. #661 then added Cyrillic and Devanagari GENERATIONAL words the script line refused too, splitting Serbian by the script it is written in (#generational-words). Measured 2026-10-14 over every non-Latin suffix word (51) in six shapes under three orders (918 parses), the exception protected no correct reading: an honorific followed by a name word read as a MIDDLE name, in silence (`राम शर्मा साहब मोहन` gave middle 'शर्मा साहब', family 'मोहन'; `משה כהן ז"ל דוד` middle 'כהן ז"ל'), and dropping it reads the honorific as the suffix it is with the word behind it absorbed and reported, as `John Smith PhD Jones` reads. 360 of the 576 honorific parses move and 285 of the 342 generational ones, the same 120 and 95 under each order; the one-character honorifics (씨 様 氏 殿 군 님 양, and जी, whose vowel sign is no letter) never move, S2's two-letter minimum keeping them out already. No realistic name with a name word after an honorific was found, and no corpus name outside rules.md's own examples moves; of the three S2 examples that had pinned the refusal, two now pin the run, and the mixed-script `Taro Tanaka さん Jones` gives way to `राम शर्मा साहब मोहन`. Recompute: the words are master's `Lexicon.default().suffix_words` (after #681) whose letters are not all LATIN by Unicode name; each goes between same-script name words in the shapes `{a} {b} {w} {c}`, `{b}, {a} {w} {c}`, `{a} {b}, {w} {c}`, `{a} {b} {w} {c} {c}`, `{a} {w} {c}` and `{b} {a} {w} {c}`, under each of the three orders, keyed by the order's NAME (the first count keyed by its first role and lost an order, two orders both beginning with FAMILY); compare `as_dict()` and the ambiguity kinds with `_pieces.starts_a_credential_run`'s script test in and out. Generational words are not a class of their own here: a suffix word is a suffix word, which is what drops `_pieces._all_latin` (#681, the same day) and the need for any honorific or generational marker as #679's roster adds non-Latin words. - -### indic-honorifics — the renunciate class and the Indic honorific vocabulary (2026-09-06, #346/#344/#343) - -Closes #346, #344 and #343 as one bundle: each CLASS is decided once, and each script's membership in it is then argued word by word, so the three lists overlap without matching. No parser code moves in it. Four existing forks carry the whole thing — the renunciate fold (rules.md#H1), the spaced trailing suffix path, title vocabulary beating the one-letter-plus-period initial shape, and the unvalidated given_name_titles — and all four were confirmed on the pre-bundle tree with a throwaway override before any wordlist was edited — the recipe is `Parser(lexicon=Lexicon.default().add(given_name_titles={...}, titles={...}, suffix_words={...})).parse(text)`, and the fourth fork is the one a reader will not expect: `Lexicon` deliberately does not validate `given_name_titles` against `titles` (see #given-name-titles), so an entry in the given-name set alone is never recognized as a title at all. In the shipped data that is free, TITLES being GIVEN_NAME_TITLES | {...}. - -- **The renunciate class (#346).** What is decided once is the CLASS — renunciate given-name titles — against rules.md's H Background, which already carried the two criteria: membership in the given-name-title list follows HOW THE TITLE ADDRESSES, and whether an empty family is correct output follows SURNAME RETENTION. Renunciation answers both at once, so "Swami Vivekananda" reads given Vivekananda, family "". Each script's MEMBERSHIP is its own list, and the three overlap rather than coincide: Latin swami/guru/baba/lama (moved out of the TITLES-only block), Devanagari स्वामी/गुरु/बाबा/संत, Bengali স্বামী/শ্রীল/গুরু/বাবা. lama is a Tibetan title whose native script is Tibetan, and the Hindi and Bengali borrowings लामा/লামা are rare enough that no entry was argued for either — "Lama Zopa" folds, "लामा जोपा" and "লামা জোপা" do not. संत (Sant) and শ্রীল (Srila) run the other way: renunciate titles native to those scripts with NO Latin entry, — Latin sant collides with a real given name (the Excluded block below) and srila has no transliteration in use — "संत कबीर" folds, "Sant Kabir" does not. A sweep "harmonizing" the three lists must therefore argue each word on the two criteria, never on symmetry. rabbi and imam address by TITLE rather than by given name, so they fail the first criterion and stay out; the second agrees, those traditions keeping surnames. "Rabbi Cohen" keeps family Cohen and is pinned as the control beside the Swami row. The four Latin entries were already TITLES vocabulary, so the promotion changes only which field the one name word lands in, and C-i's position test was not re-run for them; the known leading-position collision is Guru Dutt, whose given name is Guru — family went from Dutt to "" — accepted as the pre-existing cost of guru being a title at all, and pinned by the row renunciate_title_swallows_a_real_given_name. -- **venerable is left in TITLES only, and is deliberately NOT an Excluded entry.** The traditions split on surname retention — Buddhist monastics drop the family name, Anglican archdeacons and Catholic candidates keep it — so neither answer is right for the single-name case. TITLES has no ambiguous subset and no AmbiguityKind, so, as for king in #v1-xfail-triage and roz in the maiden markers, the only two expressions available are ship and do not ship; this is the third thing, a membership that is right and an OUTPUT that is undecidable, and it stays where it is. -- **Rinpoche is postpositional** ("Sogyal Rinpoche", "Lama Zopa Rinpoche"), so it joins SUFFIX_WORDS and no title set. The Tibetan -la particle is vocative and glued and needs no entry. The Tibetan PREFIX titles (Tulku, Khenpo, Geshe, Kyabje, Jetsun) are out of this bundle — each re-asks the surname-retention question for a different tradition — and are deferred to a follow-up issue (#514). -- **The trailing set ships spaced only.** rinpoche, जी/साहब/साहिब/साहेब/महाराज and সাহেব/বাবু/মহারাজ are SUFFIX_WORDS entries and none is a honorific_tails entry. The standing prohibition below is what generalizes: it was recorded for जी alone and now covers the set, on the same evidence and with the same accepted cost. -- **Bengali বাবু is trailing where Devanagari बाबू is leading** ("অমল বাবু" against a leading बाबू). Different codepoints, so the two cannot interact; the divergence is stated in both wordlist comments so that a sweep "harmonizing" them has to argue with the data module rather than discover the split afterwards. -- **The Latin md history, corrected.** md entered TITLES in 8cd3123 (2015-09-05, #32) as the medical degree written before the name, alongside do, which #296's audit later took out (#vocabulary-collisions). The Muhammad reading is a 2026 reinterpretation, recorded in the case row audit_md_leading_stays_a_title; both readings agree on the field, which is why the entry survived #296's audit as the one deviation from that audit's table. Recorded here so the record stops implying md was added for Muhammad. -- **মোঃ/মো and মোসাঃ/মোছাঃ go into TITLES**, mirroring Latin md. Semantically they are religious name prefixes rather than titles. TITLES is the FUNCTIONAL home because `given` must stay the name the person is addressed by — মোঃ আবদুল করিম is called আবদুল করিম — and the parser has no name-prefix field. The bound-given-name set is the wrong home for the same kind of reason it is the right one for Arabic abdul: abdul is an inseparable half of one name, মোঃ is detachable. Latin Mst waits for the opt-in pack (#345); the most already in TITLES is the English "Most Rev." word and is unrelated. -- **Abbreviation marks and periods.** An entry carrying an abbreviation mark is spelled with it: the Bengali visarga ঃ (U+0983) is a spacing combining mark (Unicode category Mc, not a letter and not punctuation), and `_lexicon._normalize` lowercases, composes NFC for a non-ASCII word and strips edge whitespace and the four full stops of `FULL_STOPS` — no NFKC, no casefold anywhere in that module — so the mark is simply never touched and মোঃ reaches the lexicon and matches intact (corrected 2026-09-10, #cjk-full-stops: the fold was ASCII periods only and no NFC when this was written; the visarga argument is unchanged, ঃ having no canonical decomposition, so NFC leaves it exactly where it was). Recompute with `Lexicon.default().add(titles={"মোঃ"}).titles & {"মোঃ"}`, which is non-empty iff the mark survived, and `unicodedata.category("ঃ")`, which is `Mc`. Period spellings (মো., মোসা., ডা., प्रो., पं.) match the bare stem through the same edge-period strip, so every BENGALI abbreviation carrying the mark ships as two entries, the visarga form and the bare stem (মোঃ and মো, মোসাঃ and মোসা, মোছাঃ and মোছা, মিঃ and মি), while the Devanagari abbreviations carry no such mark and ship as the bare stem alone (प्रो, पं, प्रा, डा); the dotted form is never listed in either script. -- **Vocabulary beats the initial reading.** A one-letter-plus-period Bengali abbreviation is is_initial-shaped and the title entry wins: "ড. মুহাম্মদ ইউনূস" reads title ড., given মুহাম্মদ, family ইউনূস, while "র. কে. নারায়ণ" — real initials with no entry behind them — is unchanged. #343 verified the same fork at 2.1.0; it is re-verified here on the shipped set. This is MEASURED behavior pinned by rows, not a normative rule — no rule in rules.md states that title vocabulary outranks the one-letter-plus-period initial shape, rules.md#H2 speaking only to the bare-initial and the shape-versus-suffix cases — so a future shape veto must re-check "ড. মুহাম্মদ ইউনূস" before it lands. -- **rules.md#H2's Accepted clause keeps its claim and loses its example.** The clause says an unlisted abugida honorific stays a name word, because the shape rule cannot see an unbroken run of letters through combining vowel signs; that is still true. Its example used প্রফেসর, which this bundle lists, so the example is now "প্রকৌশলী. Sen" → given="প্রকৌশলী." — Engineer, a loanword deliberately not shipped. The example is a contract-tier name in corpus_rules.jsonl, so the swap landed in the Bengali commit rather than in the records commit; leaving it would have exited the gate 1 there. -- **Not recorded as excluded:** the Tier 4 candidates — transliterated military ranks, Advocate, Engineer, Marhum, and the kinship forms — are skipped pending a corpus check, not declined, and मां/माता/মা and the hafiz/syed/shahid forms are left alone the same way. An Excluded entry is a standing prohibition, and none of these has been argued to that standard. One of them carries a duty anyway: প্রকৌশলী (Engineer) is rules.md#H2's executable example of an unlisted abugida honorific, so whoever ships it must move the example to another unlisted word first, or the doc test and the contract-tier gate fail together. -- **Measurement (2026-09-06).** Four corpus names move, all under the renunciate fold, all in one direction (the family empties and the given fills): Swami Vivekananda, Guru Nanak, Baba Ramdev and Lama Zopa, every one from the radar-tier corpus_issues.jsonl. Recompute by parsing every name in the tools/differential/corpus*.jsonl glob twice — once with the shipped lexicon, once with `Lexicon.default().remove(given_name_titles={"swami","guru","baba","lama"})` — and diffing the seven name fields. The multi-name spellings are the boundary and do not move: Swami Vivekananda Saraswati and Guru Gobind Singh keep families Saraswati and Singh, rules.md#H1's fold requiring exactly one name word. The Devanagari and Bengali additions move no corpus name at all, the corpora holding no name written in either script apart from rules.md's own H2 example, which this bundle rewrote. The gate's intentional counts rose by four at every baseline; read today's from the `corpus:` line of `uv run python tools/differential/compare.py --baseline X`. - -### salutation-titles — salutations, offices and ranks for the languages TITLES had none for (2026-10-04, #606) - -Closes #606 (#662 later added owura and bapak, last bullet). No parser code moves: 41 words join TITLES and one, knt (Knight, beside kt), joins SUFFIX_ACRONYMS with a `Knt` case mask. The class is decided once — a word that addresses or ranks a person and is written in FRONT of the name — and each word is then argued on #vocabulary-collisions C-i, the position test, because TITLES has no ambiguous subset and the only two answers available are ship and do not ship. The per-word lists, grouped by language, are the #606 block in nameparser/config/titles.py. - -- **The graf class, decided (Derek, 2026-10-04).** A word borne as a name, but RARELY as the first word of one, ships anyway, on the precedent of graf, which shipped though Steffi Graf exists. This is a judgment on each word's incidence in the leading position — the position C-i asks about — and not a rule, and it is an EXCEPTION to C-i rather than an application of it: C-i is an iff with no threshold, and herra, batoni, hrabia and kreivi were measured as borne in that position, so C-i says do not ship them (TITLES having no ambiguous subset to mark them in). What the exception adds is a weighing C-i does not make, a small measured incidence against the benefit of the title reading, and it is scoped to TITLES and to the words this entry names. It licenses nothing in the particle or suffix sets, and a later TITLES word that wants it is argued on its own counts. The title claim acts only in front, so most bearers are untouched in the forms they write: `Gladys Knight` and `Knight, Gladys` keep family Knight. The cost falls on a name that BEGINS with the word, the cost `Graf Steffi` already paid: `Greve Anna` reads title Greve, family Anna, and a given name is lost the same way, `Herra Wijaya` reading title Herra, family Wijaya. Declaring `FAMILY_FIRST` does not help, since the title claim is read before the order (`Hrabia Anna` reads title Hrabia under it); the family comma does. Two further limits, both pre-existing: a trailing period makes the word the title by rules.md#H5 (`Gladys Knight.` reads title `Knight.`, as `Mary Jane King.` does), and every word this entry adds widens that rule's reach, H5 claiming any listed title behind a period. Accepted rather than prevented, and stated at rules.md#H1's Accepted clause. The issue proposed the class for three words — greve (Scandinavian Count), graaf (Dutch Count), knight. The spot-check the issue asked for (forebears.io, 2026-10-04, worldwide counts rounded) found ten more of its "no known collision" words borne, and Derek extended the decision to all of them rather than excluding them: herra (~3,000 surnames, Costa Rica and the Dominican Republic; ~670 given-name uses, Indonesia and the Philippines), batoni (~1,100, the painter Pompeo Batoni; ~300 given-name uses, Tanzania), ponas (~4,000, Iran), counsel (~730, Australia, the US, England), ponia (~560), the Filipino ginoo, ginang and binibini (~500 each, Philippine lists writing Surname, Given with a comma), and two whose registers often write the surname first, hrabia (~750, Poland) and kreivi (~270, Finland). Those last two, and the given-name uses of herra and batoni, are the leading-position bearers this class accepts, and were decided on with their counts in hand. The line is NOT reproducible from this record, and that is said rather than hidden: the accepted side was counted, while the excluded side — thiru, puan, pan, rodina, ông/bà — was judged from general knowledge of each word, as #606 drafted it, with no count taken (the count source refused further lookups the same day). Each was judged a COMMON leading-position name, against counts in the hundreds on the accepted side; gróf was excluded for want of any evidence at all. Whoever decides κόμης or the next salutation should count the excluded words first, and is free to find the line in a different place. The comment `# borne` in titles.py marks each member, so a sweep meets the decision where it meets the word. -- **Accents are not folded, and two words depend on it.** `_lexicon._normalize` lowercases and composes NFC and does nothing else to letters, so paní and härra cannot match Pani (~76,000 surnames and ~65,000 given names, India and Indonesia) or Harra (~1,100). Accent-folding the lexicon would turn both into collisions; whoever proposes it re-argues this entry. frú depends on nothing: the unaccented fru (Swedish/Danish Mrs) already ships, so the Cameroonian surname Fru (~10,800, where the surname is often written first) already reads as a title in `Fru Anna` and did before this entry; that pre-existing collision is recorded here rather than argued, being no part of #606. -- **gravos was dropped, not excluded.** The issue listed it as the Latin-script Greek for Count; the spot-check could not confirm it is the Greek word at all, which is κόμης (kómis). Nothing was shown, so there is nothing to keep out — it is recorded here so that a later sweep starts from that rather than from the issue's table. κόμης itself is NOT added and NOT excluded: it is a Greek surname (~420 bearers in Greece, official lists writing the surname first), which by its counts sits inside the class above beside hrabia, and it was left out of this change for having no use in current Greek address rather than for its collision. Adding it is a decision still to make. -- **Measurement (2026-10-04).** One corpus name moves, `Greve Anna`, given Greve to title Greve: the rules.md#H1 example this entry added, and the only one of its three examples that moves. Recompute: take the 41 entries of the #606 block in titles.py — the lines that are a quoted string, not the words quoted inside its comments, which name existing entries and excluded ones — parse every distinct name in the tools/differential/corpus*.jsonl glob (1,518 on the day) with the shipped parser and with `Lexicon.default().remove(titles=, suffix_acronyms={"knt"})`, and diff `as_dict()`. Before the rules.md examples were added, the same diff moved nothing over the 1,809 corpus lines of the tree before #604 merged, gravos still in the word list. The old readings the release-log bullet quotes were checked against the 1.4.0 and 2.3.0 wheels. -- **2026-10-17 #662 — owura and bapak join TITLES; malam and bay are excluded.** Akan and Indonesian Mr., found in the CLDR corpus (`Owura Bertram…`, `Bapak Dwi Putro bin Adinata MP` read the salutation as the given name). Counts from `tools/name_usage.py` on 2026-10-17: owura and bapak each under 100 US Census bearers and none found on Wikidata (a zero proves nothing, per the tool). The two exclusions are in the Excluded (TITLES) block below. Measurement: two corpus names move, both in corpus_cldr.jsonl and both toward CLDR's fields (`Owura Bertram Wilberforce Henry Robert Wooster MP` and `Bapak Dwi Putro bin Adinata MP`, salutation from given to title); recompute with `uv run python tools/differential/snapshot.py --since `. The release-log reading was checked against the 1.4.0, 2.0.0 and 2.4.2 wheels. - -### generational-words — junior and senior in other languages (2026-10-14, #661) - -Closes #661. 31 spelled-out generational words join SUFFIX_WORDS, and one predicate moves with them (the run-start bullet below), so `Antonio Pavlović mlađi` reads family Pavlović, suffix mlađi, where 1.4.0 and 2.4.2 read middle Pavlović, family mlađi (checked against both wheels). The class is decided once — a word meaning junior/senior (or the Younger/the Elder) written spaced AFTER the name — and each word was then checked two ways. First, collisions: a suffix claim acts on the trailing word, so the risk is a borne name there, and `tools/name_usage.py` found none (US Census, SSA, Wikidata; nuorempi's one Wikidata bearer of each kind is noise, as the issue said). Second, usage, because the issue's list came partly from ilib's machine-translated test data: each word was found in the trailing spaced position in real titles on its own language's Wikipedia (2026-10-13, e.g. "Hans Holbein mlađi", "Плиний Младший", "Pieter Brueghel vanhempi", "बलबीर सिंह सीनियर"). The per-word list, grouped by language, is the #661 block in nameparser/config/suffixes.py. Feminine forms ship beside the masculine wherever the language has them (mlađa, младшая, молодша, jaunesnioji); lt vyresnioji is the grammatical feminine with no titled bearer found, and ships on grammar. ru/uk and cs/sk were not in the issue and were added on the same evidence (Derek, 2026-10-14). - -- **CLDR is corroboration, not a source.** Its person-name samples write mlađi/млађи (hr, sr — on a woman's name too), младши/младша (bg), ifj. (hu) and जूनियर (hi), but its `generation` field also holds translations of the field LABEL rather than of the word: isizukulwane (zu), जेनेरेसन (ne), نسل (ps) and jẹnereṣọn (pcm) all mean "generation". A word is not added because CLDR has it. The wider roster (fils, yngri, الابن and the rest) is open at [#679](https://github.com/derek73/python-nameparser/issues/679), to be vetted word by word the same way. -- **Excluded, each with its reason.** kıdemli (tr): a rank word (Kıdemli onbaşı); Turkish writes Jr. or a parenthesis, (oğul). sóisearach/sinsearach (ga): rank words too; Irish writes Óg and Mór, which are also inside given names (Seán Óg Ó hAilpín), so they are a separate decision and not taken here. ifj./id. (hu): written BEFORE the name (ifj. Kovács János, which H2's shape rule already reads as a title), and a bare `id` would make any trailing ID a suffix. млада (bg feminine of млади): Mlada is a borne given name, and under a family-first order the given name is the trailing word. ml./мл./st.: the abbreviations stay under the #269 deferral in suffixes.py, st. colliding with Saint besides. -- **Spaced only, and what that leaves.** bg, ru and uk write a modern Jr. hyphen-glued (Теодор Рузвелт-младши, Дональд Трамп-молодший), which no whole-token set reaches; for those three the gain is mainly the historical the Elder/the Younger names, written title case (Младший), which the lowercase fold matches. -- **The #602 run start reads the letters' script, not ASCII (Derek chose the Latin-script test, 2026-10-14). SUPERSEDED the same day by #679 (decisions.md#S2): no script is refused now, so the split this bullet accepts is gone.** rules.md#S2 lets "an unambiguous Latin suffix word of two or more letters" start the run, and `_pieces.starts_a_credential_run` had implemented Latin as `letters.isascii()`, exact while no accented Latin word was suffix vocabulary. This entry adds the first ones (mlađi, mlađa, mladší, starší), so without a change `Pieter Bruegel stariji Jones` would read suffix 'stariji Jones' while `Antonio Pavlović mlađi Jones` kept family Jones -- one language, one construction, split by the `đ`. The test now asks a non-ASCII word whether every letter's Unicode name is a LATIN one; the ASCII test stays first, so an ASCII word pays no frame for it. The same script test is what refuses the non-Latin honorifics (`Taro Tanaka さん Jones` keeps family Jones), and it refuses this entry's Cyrillic and Devanagari generational words with them, though the reason S2 gives for the refusal -- an honorific carries no credential mark -- is not true of them. So Serbian splits by script: `Ana Horvat stariji Jones` reads suffix 'stariji Jones' and `Ана Хорват старији Џонс` family Џонс, and after a comma C1 asks the same predicate, so `Ана Хорват, старији Џонс` reads given 'старији'. ACCEPTED for this entry (Derek, 2026-10-14) and stated at rules.md#S2's Accepted clause: the vocabulary does not mark which suffix words are honorifics, a script test is the only line available, and only a name word written after the generational word reaches it. Settling it is open at [#679](https://github.com/derek73/python-nameparser/issues/679), where the non-Latin generational words multiply. The first draft of this entry credited the ASCII test to S2's honorific exclusion instead and proposed deleting it, on the evidence that deleting it moved no test and no corpus name. The design-docs review found the attribution wrong and the evidence empty: nothing pinned the test, and deleting it reads `Taro Tanaka さん Jones` as suffix 'さん Jones' and `Гай Плиний Старший Иванов` as suffix 'Старший Иванов'. Both are rules.md#S2 boundary examples now. #678 was filed on the first draft and corrected. -- **Measurement (2026-10-14).** Six corpus names move, all in corpus_cldr.jsonl and all the CLDR hr/sr renderings carrying mlađi/млађи, each gaining suffix mlađi (or млађи) from a name field; cldr_agreement records 14 changes, every one toward CLDR's fields. The bg and hi renderings do not move because those locales' formats drop the generation field. The run-start change moves no name the corpus held before this entry; its one mover is S2's own `Antonio Pavlović mlađi Jones`. Of the 161 rows of tools/differential/cldr_fields.jsonl whose `generation` value `cldr_agreement.occurs()` in the text, none writing a non-ASCII generation word has a name word after it in the same part (two write given names after `II` across a comma, which ends the run's part). Recompute: `uv run python tools/differential/snapshot.py --since `. - -- **2026-10-14 #679 — the rest of CLDR's roster, vetted word by word (Derek).** 76 more words join SUFFIX_WORDS (counted against master's set, none removed): the #679 block in suffixes.py, 30-odd languages. Vetting as above, run by three research passes over each language's Wikipedia (langlinks of the Jr./Sr./the Younger/the Elder articles, then the language's own search) and `tools/name_usage.py`. CLDR's `generation` field proved unreliable in about a third of languages, beyond the field-label translations: an adjective written BEFORE the name (ta இளைய, kn ಕಿರಿ, si බාල, az Kiçik, uz Kichik, tk kiçi, ba кесе, ky Кенже, kk кіші), an abbreviation nobody writes after a name (mr ज्यु., pa ਜੂ., kok धा.), a word that means "second" (chr ᏔᎵᏁ, hi द्वितीय, zh 二世 — regnal, and glued), or a word no Wikipedia uses where another is standard. Where the language's Wikipedia writes a different word after names, that word ships instead of CLDR's: lo ຈູເນຍ for ຮຸ່ນລູກ, ti ጁንየር (and am ጁኒየር) for ማንጁስ, or ଜୁନିଅର/ଜୁନିୟର୍ for ଜେଆର୍, ta இளையவர் and ஜூனியர், si ජූනියර්, mk помладиот for јуниор. EXCLUDED, each with its reason: he סניור, which Israelis bear as a surname (אלי סניור and בוריס סניור, found on he Wikipedia), and ja シニア, whose only evidence is weaker: `tools/name_usage.py` gives both spellings the same 96 Wikidata bearers as Latin Senior, with the same citizenships, because Wikidata's Senior surname item carries he and ja labels, so the count measures the English surname and not Japanese writing; left out anyway (Derek, 2026-10-14), beside its Hebrew twin — English `senior` is no suffix either, Senior being a surname (3,864 US Census); hy Ավագ (an Armenian given name, in lists that write the surname first); km ជូនៀ (a given name; ជូនៀរ ships); Ajala (a Yoruba surname); kk Джуниор and mk јуниор (a given name or a club name there); mn Жр (the initials ЖР); be мал. (also "figure"); lt jaun.; the single-letter abbreviations mr क. and as ক., which are the initial shape; ne/kok जेआर (rare; also JR); fa پدر/پسر ("father"/"son" without the article that marks ar الابن and he הבן as never a name); and the CLDR forms that are label translations or unattested (cv, chr, ak, blo, kok-Latin, kk-Arabic). PACK-SHAPED, NOT SHIPPED: fr fils/père and the Latin abbreviations ml./st. (cs, sk, sl, hr) are real and standard, but their collisions are with OTHER languages written in the same script — `Marie Fils` (767 US Census bearers of Fils) and `Jiří Dienstbier ml.` beside ML initials and St. as Saint — which is the case a locale pack exists for (the #604 `nic` precedent). No fr or cs pack exists; proposing them is open at [#684](https://github.com/derek73/python-nameparser/issues/684). The Cyrillic мл. does ship: ru, bg and sr all write it after names and nothing collides with it in Cyrillic, so a pack would protect nobody; ст. is barely attested after a name and stays out. Neither existing pack takes anything: tr and az put the junior word BEFORE the name or in a parenthesis, and Turkish baba collides inside Turkish. Measured 2026-10-14: 16 corpus names move, all CLDR renderings in is, ar, fa, gu, kn, ml, ru and cv (cv's rendering through мл.) gaining their generation word as the suffix out of a name field; cldr_agreement's default view records 4 suffix fields newly agreeing (cv, fa, is, ru) and no regression, and 40 still-wrong fields changing across the 16 renderings: 12 suffixes now the generation word alone where CLDR also holds the comma-led credential (`yngri` for `yngri þingmaður`), and 28 family, family_base and given_names fields that lost the word but still disagree. No rendering newly agrees on every role. The Burmese sample glues the Burmese comma ၊ (U+104A) to ဂျူနီယာ and is not reached, the tokenizer not splitting on it. - -### cjk-comma-demotion — the script shapes are pure, the wrappers are tolerated (2026-09-01, #469) - -Closes #469, and continues the corpus-tier arc below rather than standing apart from it: the tier split gave the differential somewhere to WATCH a name without promising it, and this is the first doctrine narrowing to spend that. No parser behavior moves anywhere in it. Counts are this session's and every one is recomputable from the checked-in tree — `wc -l` over tools/differential/, `uv run python tools/differential/build_shapes_corpus.py --coverage`, and one `uv run python tools/differential/compare.py --baseline X` run per baseline, read off its `corpora:` and `corpus:` lines. - -- 2026-09-01 #469 — THE LINE IS FORMAT PURITY, and not comma class. Shapes 6 ("Family Given [Honorific]") and 7 (the interpunct/katakana transcription in source order) admit wholly classified-script text and refuse a comma and every ASCII letter; each composed form around a CJK name — the family-comma listing W3 reads, the honorific-comma probe zoo, a leading Latin title, a trailing credential comma — is TOLERATED input instead: parsed best-effort, contract-exempt, changeable without notice. What the inventory is FOR is the whole argument. It tabulates written ARRANGEMENTS, and a comma-wrapped or Latin-wrapped CJK name is not an arrangement any of the three writing systems produces — it is a listing convention from elsewhere with a CJK name inside it. So the admission test is a property of the FORMAT, and a rule reading such input describes rather than promises. - Supersedes an in-session fork of this same day's design, recorded because it is a line the next reader would redraw the same wrong way. The first cut kept trailing CREDENTIAL commas contract while demoting the family comma, on the theory that the two comma classes differ in kind — a credential comma being a Latin convention wrapped around anything, where a family comma makes a claim about the name's own division. The shape lens replaced it: both are wrappers, neither is a form the script writes, and a doctrine promising one while tolerating the other has to keep explaining a distinction its own admission test cannot state. Purity states the line in one sentence and needs no comma taxonomy at all. - -- 2026-09-01 #469 — SCOPES the FAMILY_COMMA bullet under 3-0-reevaluations below, which reads "inherited from v1's lastname-comma but correct on its own terms — an explicit comma is stronger evidence than script". It stands unchanged for LATIN script, which is the script it was argued about. For CJK script it is narrowed to a description: an explicit comma still beats script and the parser still reads it that way, but every input carrying one is a form no East Asian writing system produces, so what the parser makes of it is current behavior and not a promise. The bullet is not edited — it was right about what it was reasoning over, and this entry is the scope it never had to state while Latin was the only script in the room. - -- 2026-09-01 #469 — the 2026-08-23 (PR #428) snapshot under decisions.md#C1 below attributes `田中さん, Dr.` keeping its honorific glued to "(C1 Accepted)". That clause now lives in rules.md#W3's `tolerated:` note: the BEHAVIOR is unchanged and only its home moved, W3 being where the CJK comma forms are set down now that they are described rather than promised. The snapshot stays as written — it was true when it was taken, and this is the note that says where to read the clause instead. - -- CLDR, PULLED VERBATIM 2026-09-01 — #469 asked for the citation and asked that it be a pull rather than a recollection, so the failures are recorded with the findings. The paths named in the plan (`https://raw.githubusercontent.com/unicode-org/cldr/main/common/personNames/ko.xml`, `.../zh.xml`, `.../ja.xml`) all 404: there is no `common/personNames/` directory in the repository, and the data is a `` element inside `common/main/.xml`. Read there instead, from unicode-org/cldr at `main`, commit `8e4fb0fa1e287c4612b512cf2ef77e890467da28`; the grammar from https://www.unicode.org/reports/tr35/tr35-personNames.html. The grammar first, verbatim, because it decides whether absence is evidence: "A namePattern is composed of a sequence of field IDs, each enclosed in curly braces, and separated by zero or more literal characters (eg, space or comma + space)." A comma between surname and given is therefore expressible, and its absence below is a choice the locale data makes rather than a limit of the format. The `order="surnameFirst" usage="referring" formality="formal"` patterns, verbatim, long then medium then short: - ko {surname} {given} {given2} {credentials} · {surname} {given} {credentials} · {surname} {given} - zh {generation}{surname} {given} {given2} {credentials}{title} · {generation}{surname} {given} {given2-initial}{credentials} · {surname} {given} {given2} - ja {surname} {given2} {given}{title} · {surname} {given}{title} · {surname} {given}{title} - Counted rather than eyeballed: each of the three files holds 42 `namePattern` elements, and not one of the 126 contains a comma of any width (U+002C, U+FF0C, U+3001) or a semicolon. The delimiter between surname and given is a single space throughout. - THE ONE COMMA IN REACH, recorded because the record must not overclaim. It is in the locale-neutral root, whose sorting pattern reads verbatim `{surname} {surname2}, {title} {given} {given2} {credentials}`. `sorting` is a list-ordering format and not a referring one — UTS #35 verbatim: "Used to format names for a sorted list. example: “Brown, William” [medium, informal]" — and ko and zh override all six of their sorting slots comma-free (`{surname} {given}` and kin), while ja overrides five and leaves `order="sorting" length="medium" usage="referring" formality="formal"` as `↑↑↑`, inheriting root's comma-bearing pattern. So the honest reading, and it is the stronger one for being stated with its exception: no native pattern in any of the three writes a comma between surname and given, and the sole comma any of them can reach is INHERITED from the locale-neutral default, in the format whose entire job is sorted lists. That is the "listing convention carried in from elsewhere" the W Background already named, arriving from CLDR rather than from this project's own reasoning — the pull CONFIRMS the writing-system rationale, and complicates it only by showing the exact seam where a comma does get in. - Two findings worth keeping past the comma question. zh writes `{surname}{title}` with no delimiter at all (and ja `{surname} {given}{title}`), which corroborates rules.md#W2's glued honorific from the formatter's side, independently of the peel's own vocabulary argument. And ja declares `・` — a foreign name's spaces become the nakaguro — which is shape 7's divider reached from the formatting direction rather than the parsing one. - -- 2026-09-01 — the SHIPPED 2.2.0 release notes are left untouched, deliberately, and this bullet is the record of that choice. docs/release_log.rst describes the comma-crossing as a fix in two 2.1.0 entries ("Fix a comma or a 间隔号 stopping the glued-honorific peel", and #319's `田中さん, V.` entry). Both were true statements about what those releases did, made while the crossing was contract and #312 had argued it. Demoting the rule does not make a released note false about its release, and editing shipped notes to agree with a later doctrine would cost the log the one property it exists for. The demotion is 2.3's news and belongs in 2.3's notes; the log is a history, not a mirror of the current contract. - -- COUNTS, 2026-09-01, with the recompute beside each. 29 case rows carry `tolerated=True` over 26 distinct texts — the flag is per ROW and the corpus is per TEXT, which is why the two numbers differ (recompute: read `CASES` and count `c.tolerated`). The 26th text arrived with the W3 demotion itself: the given-side listing `지훈, 남궁민수` was a rules.md example with no case row, and would have left the harness altogether when the rules corpus stopped carrying W3, so it was given a tolerated row rather than dropped. `corpus_cjk.jsonl` went 98 → 73 and the radar-tier `corpus_cjk_tolerated.jsonl` was created at 25, then 26; the identity is the check worth keeping, since 73 + 25 = 98 exactly says the split MOVED names and dropped none, and the 99th is that one rescued row. `corpus_rules.jsonl` went 252 → 248: six comma-bearing example texts left (`田中さん, Dr.`, `田中さん, PhD`, `남궁민수, 지훈`, `지훈, 남궁민수` when the builder skipped W3 whole, plus `김, 민준씨` and `田中さん, V.` from W2's swap) and two pure ones arrived (`김민준씨`, `田中さん 様.`). `corpus_shapes.jsonl` stands at 37, with the new shapes covered 5 names (shape 6) and 2 (shape 7). And the stop condition held: intentional diffs are UNMOVED at 226/205/113/0 across 1.4.0/2.0.0/2.1.0/2.2.0, with unexplained 0, radar unclassified 0 and exit 0 at each — 1113 names compare at 1.4.0 (7 skipped, shapes 4/5) and 1120 above it. That the counts do not move is the substance and not a formality: a classified diff counts identically on either tier, so demoting a name changes which file it loads from and nothing about what the gate makes of it. - -- 2026-09-05 — THE SWEEP'S CRITERION WAS NARROWER THAN THE DOCTRINE IT IMPLEMENTED, and three rows sat in the gap. The 2026-09-01 sweep asked, over the CJK-bearing case rows, for a comma or an ASCII letter. A trailing ASCII period is neither, so `田中さん 様.`, `김민준 씨.` and `김민준 양.` — the three `fix(#320)` probes — stayed contract; and `田中さん 様.` was then chosen as rules.md#W2's replacement example for its "ignores surrounding punctuation" clause on the strength of passing that same test, so the miss propagated into the normative document the same day. A period standing after a CJK honorific is a listing artifact no writing system produces, which is this entry's own class in its own words: the admission test judges the FORMAT, and `様.` is no more an arrangement Japanese writes than `田中さん, PhD` is. The three rows carry `tolerated=True` now, and nothing about the parse moves — the rows still pin #320's mechanism at HEAD, and the texts are still compared and classified at every baseline, from the radar file instead of the contract one. CORRECTED IN REVIEW, 2026-09-05: a first draft of this bullet called all three "the spaced twin of a comma row that WAS demoted", and only two are. `田中さん, 様.` and `김민준, 씨.` are both case rows and both in `corpus_cjk_tolerated.jsonl`; `김민준, 양.` is no row at all — enumerating the CASES texts containing `양` returns `김지양`, `양 미선`, `김민준 양`, `김민준 양.` and `양 지훈`, and nothing with a comma. `김민준 양.` is demoted on the class argument alone, which is what carries the other two as well: an existing demoted twin is corroboration, never the warrant, or the class would end wherever the case table happens to stop. (Recompute: read `CASES` and collect the distinct texts containing the character.) - -- 2026-09-05 — THE CRITERION IS RESTATED AS THE DOCTRINE rather than as a sample of it. A SHAPE-TAGGED row (shapes 6/7, `Case.__post_init__` in tests/v2/cases.py) now admits no ASCII character but the space between two name words, so a period, a digit and a bracket are refused beside the comma and the Latin letter the check already refused, and the next composed form does not need a fresh criterion to catch it. The gate is a property of the shape TAG and not of CJK text, which is the half that keeps it honest: an UNTAGGED row may carry ASCII, and several deliberately do. The four mixed-script rows commit `d562164` reviewed one by one — `John 王`, `王·Smith`, `Andersonさん`, `Anderson선생님` — keep the contract tier on the reasons recorded there, untouched by this. - -- 2026-09-05 — `山田 太郎 (マイケル・ジャクソン)` IS KEPT contract: it is the one text the wider sweep surfaces and this entry declines to demote. Its ASCII parentheses are not a convention carried in from elsewhere — they are one SPELLING of a delimiter Japanese writing does produce, and the parser reads the spellings identically. Measured 2026-09-05 under the default facade: `(マイケル・ジャクソン)`, the fullwidth `（マイケル・ジャクソン）` and the corner-bracketed `「マイケル・ジャクソン」` all give family 山田, given 太郎, nickname `マイケル ジャクソン` (recompute: parse the three; ASCII square brackets are NOT in that set and give family `ジャクソン]`, which is what makes this a delimiter roster rather than a rule about brackets). What the row pins is the nakaguro INSIDE a delimited nickname (`fix(#272)`), and which of the three delimiters spells it is incidental to that. - -- 2026-09-05 — W2's PUNCTUATION HALF MOVED to W3 with its example; ITS SECOND HALF IS NORMATIVE AND STAYED, and this bullet is the within-branch correction of a first draft that took both. The clause read "The split-off ignores surrounding punctuation, but never treats a part that is not name text as the name's end". The FIRST half had no pure witness and could not be given one — native CJK writing puts no punctuation around a name for a split-off to ignore, so every input it describes is a listing form, which is W3's ground — and it moved there as description, rewritten to what is measured rather than to what it asserted (the bullet below). The SECOND half is a different claim and the reason given for cutting it does not reach it: it is what the parser does with NO comma and no period anywhere in the input, and contract-tier pure names depend on it. Measured 2026-09-05 by forcing `_is_post_nominal` to False in memory — the predicate the site scan `_peel_site` reads in nameparser/_pipeline/_script_segment.py — and re-parsing (family, given, suffix, title): `田中さん 様` ships `('田中', '', 'さん, 様', '')` and gives `('田中さん', '', '様', '')` without the step; `김민준 박사님` ships `('김', '민준', '박사님', '')` and gives `('김', '민준', '박사, 님', '')`; `선생님` ships `('선생님', '', '', '')` and gives `('선', '생', '님', '')`. The three break DIFFERENT clauses of the half, and a first draft of this bullet said all three were dissection. The restored sentence makes two claims — the post-nominal word is never the name's END, and it is never DISSECTED — and `田中さん 様` fails only the first: `様` comes out WHOLE as the suffix, undissected, but it is what the split-off takes as its SITE, so the peel lands on the post-nominal word and `田中さん` behind it is left unpeeled, where shipped the scan steps past `様` and peels `さん` off `田中さん`. `김민준 박사님` and `선생님` are the dissection failures, `박사님` coming apart into `박사, 님` and `선생님` into `선, 생, 님`. Between them the three witness both clauses, and neither clause is witnessed twice over. Two of the three, `김민준 박사님` and `선생님`, are `corpus_cjk.jsonl` names on the CONTRACT tier, and `ko_honorific_glued_doctor_spaced` is the #308/#271 regression row for the first; that row pins `suffix="박사님"`, which is what `parse('김민준 박사님').suffix` returns under the default facade, measured 2026-09-05. Under W2's remaining FIRST statement alone — a listed honorific glued to the end of the name's last name word splits off once — `박사님` ends in the listed tail `님` and reads as peelable, giving `박사` + `님`. So a W2 cut back to that one sentence would have BEGUN contradicting the regression row its own example line pins, not stopped: the second half is what keeps rules.md agreeing with the shipped parse, and that is the argument for restoring it. Restored in W2's register as "A part that is not name text — a post-nominal word standing on its own — is never the name's end: the split-off steps past it to the name word behind, and never dissects it." — witnessed by `김민준 박사님` → `suffix="박사님"` and the boundary line `선생님` → `family="선생님"`, both texts `corpus_cjk.jsonl` already held, so the two example lines arrive in `corpus_rules.jsonl` without adding a name to the pool. `田中さん 様` would have been the third witness and is NOT added: it is not a case row, and a new contract name needs a 1.4.0 classification this bundle does not have. `_script_segment.py` carries the restored sentence as a second W2 citation, on `_peel_site` where the scan-back lives. RECOMPUTE: rebind `_is_post_nominal` to `lambda state, i: False` on the module and re-parse the three. - -- 2026-09-05, FOUND while rewriting the clause — A PERIOD ON THE NAME'S OWN LAST WORD IS NOT IGNORED; IT IS DECISIVE, and the moved half asserted the opposite. The half said the split-off "ignores surrounding punctuation", and W3's first descriptive rewrite of it kept that shape, attributing the crossing to the punctuation. Five parses, 2026-09-05 under the default facade, say otherwise — counted by INPUT STRING, the convention the rest of this bullet uses when it says "the five inputs" and "parse the five strings". A first draft counted four by counting the SENTENCES below instead, the last of which carries two inputs (`田中さん.` and `김민준씨.`). Six, if the no-period twin `田中さん 太郎` quoted inside the third sentence as its control counts as a parse of its own rather than as part of that item. `田中さん 様` → family `田中`, suffix `さん, 様`: the crossing happens with no punctuation in the input at all, so the vocabulary licenses it and not the period. `田中さん 様.` → family `田中`, suffix `さん, 様.`: the period rides along and moves nothing, `様.` reading post-nominal-strict exactly as `様` does. `田中さん 太郎.` → family `太郎.`, given `田中さん`: a period on a NAME word, and no crossing — and not ignored either, since the same input without it is family `田中さん`, given `太郎`, the order reading flipped by the period alone. `田中さん.` → title `田中さん.`, and `김민준씨.` → title `김민준씨.`: a period glued to the honorific's own word stands between the honorific and that word's end, no listed tail matches, the peel declines and the whole text reads as a title — the parse a period changes most, not least. Of the five inputs, the ONLY one where a period changes nothing is the one where it sits on a word the split-off was going to step past regardless. So W3 now says the step past a post-nominal word is W2's and needs no punctuation, that a period on a separate post-nominal word rides into the suffix, and that a period glued to the honorific's own word decides against the split-off. AMENDED 2026-09-10 by #cjk-full-stops, and it is the summary that goes rather than the measurements: every parse above keeps its date and is what the parser did on 2026-09-05 (the two suffix values render space-joined since #436/#437, the next day, `さん 様` for `さん, 様`). Of the four period-bearing inputs, ALL FOUR now divide exactly as their no-period twins — `田中さん 太郎.` reads family `田中さん`, given `太郎.`, the classification fold making the period invisible to the order rule, and `田中さん.` and `김민준씨.` peel their honorific with the stop riding on it — so the period changes no division at all here, and the third clause of the sentence before this one is superseded: a period glued to the honorific's own word rides WITH the honorific now. What survives is the first two clauses and the finding this bullet is titled for, inverted in its consequence rather than in its content: the period was decisive because a site was reading it, and the answer was to stop three more sites reading it. RECOMPUTE: parse the five strings, each also without its period, and read `title` alongside the name fields — a rewrite that reads only `family` and `suffix` sees two empty results and can call them a null effect. - -- COUNTS, 2026-09-05, with the recompute beside each. `corpus_cjk.jsonl` went 73 → 70 and `corpus_cjk_tolerated.jsonl` 26 → 29 — the same identity as the first demotion, 99 names either side, so the split MOVED three texts and dropped none. `corpus_rules.jsonl` went 248 → 247 → 249: W2 gave up the `田中さん 様.` line, then took the two the review round restored the second half with (`김민준 박사님`, `선생님`). Both arrivals are texts `corpus_cjk.jsonl` already held, so the file grew and the compared POOL did not — the same shape as the shapes-corpus growth of 2026-09-01, and the reason a witness had to be picked from names already inside. 32 case rows carry `tolerated=True` over 29 distinct texts (recompute: read `CASES` and count `c.tolerated`, per row and per text). `_CORPUS_FLOORS["corpus_cjk.jsonl"]` is lowered 70 → 67 deliberately, and the tolerated file's floor of 22 still stands under 29. The stop condition held again: intentional diffs UNMOVED at 352 / 247 / 155 / 14 across 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, unexplained 0, radar unclassified 0 and exit 0 at each. RECOMPUTE THE SWEEP, which is the part worth having rather than the digits: it is the CJK-bearing (`_has_cjk`) rows that do NOT declare `tolerated`, carrying any ASCII character other than a space, grouped by the kind of character that trips them and joined to the corpus files holding each text. Over the tree before this bundle it returns eight distinct texts — the three period rows, the parenthesized nickname, and the four mixed-script rows — and over this one, five: the period class is gone and the letter class is untouched, which is the shape of the answer to look for rather than the count. - -### P5 — bound given names - -- 2026-06-30 (first-name-prefix-join design; v1-era, carried into the v2 port) — the join is vocabulary-driven and deliberately tiny. - -- 2026-08-18 — `abd` ADDED, and the reserve count corrected to let it work. It is the transliteration for the spellings that write the article as its own word ("Abd Allah", "Abd al-Rahman"), which abdul/abdel/abdal do not match; the Arabic-script عبد has covered the same word since #269, so only the Latin side was short. Not a C-ii collision, though it looks like one: C-ii settles a PRECEDENCE by frequency where neither reading is a name, and here one reading is a name and neither had to be given up. It is C-i's positional qualifier (2026-08-17) applied to a bound-given word — the naming use is LEADING, the credential use trailing, which is the `de` row of that table ('a TRAILING surname — no clash') rather than the `vd` row. `abd` is also the postnominal ABD ("All But Dissertation") in SUFFIX_ACRONYMS, and all three credential spellings still parse ("Jane Smith ABD", "Jane Smith, ABD", "Jane Smith A.B.D."). - -- 2026-08-21 #411 (PR #416) — the reserve counts the name the join will actually leave behind, and the join never absorbs a marker. P5 has always said the join "needs a name word to spare, so two name words alone do not join"; the count was taken while the maiden marker and the maiden name were still pieces, because group's marker pass runs later in the same stage. `abdul Berg née Jones` counted four words, joined, and left no family name at all — given 'abdul Berg', family ''. That spelling is the pre-existing witness: it reads the same way at 2.0.0 and 2.1.0. `abd` does NOT — it reads correctly at 2.1.0 and acquired the defect only when #400 made it a bound given-name word this cycle, so for that spelling this is a regression #400 introduced and this closes. #399's chain stop widened the class again by putting the particle spellings in front of the reserve for the first time. - - Excluding the departing words is not sufficient on its own, and review is what established that. Where the maiden walk stops at an inner suffix the excluded span is short, enough words survive it to clear the threshold, and the join fires and takes the MARKER — after which nothing leaves and the count that authorised the join was reasoning about a name that never came to be. So the join now declines outright when the piece it would absorb is a marker, which is what P5 meant all along: it joins the bound word to a name word, and a marker is not one. Tested directly rather than through the span, because the two differ exactly where it matters — a marker with nothing but a suffix after it has no span at all, yet the join would still have absorbed it (`Berg, abdul née PhD` read given 'abdul née'). - - What sharing `_maiden_span` between the count and the marker pass buys is one definition of what a maiden span IS. It does not make the two agree about what leaves: they run at different times on different piece lists, and the join between them could invalidate the span the count was taken against. Forbidding the join to absorb a marker is what closes that, not the sharing — the sharing only keeps the two from disagreeing about the span itself. - - Two consequences. It closes the P5 half of the join-swallow #412 records, so that issue narrows to P3's connective join (closed the same day by the #412 / #417 / #418 entry under M2, which also retires this entry's by-hand exclusion). And where the bound word is also suffix vocabulary, a declining join after a family comma leaves the S2 suffix reading and the name has no given name at all — `Berg, abd née Jones` reads family 'Berg', suffix 'abd', maiden 'Jones', which is exactly how `Berg, abd` alone has always parsed. `abd` is the only shipped word in both sets. - - Two alternatives to the exclusion, recorded so "reorder" is not read as one option with one price. The rejected one moves the maiden handler ahead of the PARTICLE CHAIN, which changes what the consumer receives for every name — that is the scope objection, and it stands. A narrower one was not weighed at the time: move the marker pass to run after P2/P3's joins but BEFORE P5, inside _group_segment rather than in group(). Under that arrangement the reserve counts the surviving name directly, so the span prediction, the exclusion and the marker guard all disappear, and P2/P3 are untouched. Its real cost is ordering against the tail-segment delimiter-core drop, which today runs between the joins and the marker pass and would be reversed for comma tails — unmeasured, which is why this records the option rather than taking it. - - No corpus name produces a #411-attributable diff — every baseline already reads these names the way the fix does — so the ledger rule records the side effect on `van der Berg, abdul née Jones` rather than the defect. That name became an M2 example in the #399 fix and reached the corpus when #414's rules-doc corpus harvested it the following day. - What made it look impossible: adding the word alone changed nothing on the common three-word shape. The reserve count in the join gate excluded suffix-vocabulary pieces, and `abd` IS one, so its own piece did not count toward the three it needed — the rule declined silently. Counting the claimed piece as a name piece fixes it, and is right on its own terms: the reserve asks whether enough OTHER words are left to spare. Measured, the vocabulary-only alternatives both cost something the count fix does not — moving `abd` into SUFFIX_ACRONYMS_AMBIGUOUS loses "Jane Smith, ABD", and dropping it from SUFFIX_ACRONYMS loses the postnominal entirely. - Two limits, both measured and both accepted. The three-token "abd al rahman" still does not join — `al` is a particle and chains forward, which is the half of the original exclusion that survives. And position decides at the two ENDS only: in the given slot of a family-comma name the credential still wins, so "Smith, Abd" reports suffix 'Abd' with no given name where "Smith, Abdul" reports the given name, and no ambiguity is emitted for it. - Same shape as #397 (Catalan `i`, where dual membership shrinks the rootname count P3's carve-out tests), and the second instance the SAME DAY, #397 having been filed two and a half hours earlier. The general question — whether a word's membership in one set should suppress it from a count gating a rule keyed on another — is open there. - -- 2026-08-21 #418 (PR #420) — #411's by-hand exclusion of the departing words is gone, subsumed by the marker pass running before the reserve is counted (decisions.md#M2, 2026-08-21). Its other half stays: the join still declines when the piece it would absorb is a lone marker, because the only marker left by then is one the consumer declined, and P5 says a marker is not a name word — measured, removing the test re-read `Berg, abdul née PhD` as given 'abdul née'. "Lone" is load-bearing and review is what made it explicit: a declined marker the particle chain has already taken is a word of the chain, and the join takes the chain — `Abd van der Berg née Jr Jones` reads given 'Abd van der Berg née', where the old chain stop had kept the marker outside it. P5's statement now says "standing as a word of its own". The narrower reorder recorded above (marker pass after P2/P3's joins, before P5) was overtaken by the full one ahead of every join, which closes #412 and #418 as well, and the tail-segment ordering it priced as unmeasured was measured: byte-identical under the default policy, a real regression with a suffix delimiter configured, found in review and closed before merge (the M2 entry has the shapes). One row re-pinned: `abd née Jones Jr Smith Berg` reads given 'abd Jr', as `abd Jr Smith Berg` already does — P5 absorbing a suffix piece when the reserve allows is pre-existing and independent of markers, noted rather than fixed here. - -- 2026-08-22 #369 — a given-name title licenses the join with one word to spare. #367 regressed `Sheik Abu Bakar` from given 'Abu Bakar' to given 'Abu', family 'Bakar', and its release note recorded why the old reading had been an accident: `abu` is a particle as well as a bound word, and the title used to displace it out of the leading position so the particle CHAIN took 'Bakar'. `Sheik abdul salam` showed that a bound word alone never joined behind a title — the reserve wanted three name words and a title is not one. The question the issue put was whether `Sheik abdul salam` is a two-word given name with no surname, or given 'abdul' plus family 'salam'. Decided on the signal the title already carries: `sheik` is a GIVEN-NAME title, and H1 reads that membership as the assertion that the one word after it is the given name, family empty. P5 now reads it the same way — behind a given-name title the family comma's LENIENT reserve applies, 'abdul salam' joins, and `family=''` is the Accepted outcome H1 records for "Sir John". Not "after any title": `Dr. abdul salam` keeps given 'abdul', family 'salam', because `dr` addresses by family and the reserve exists for exactly that word. Keyed on the whole title run through `_title_key`, exactly as post_rules keys H1, so the two rules cannot read one run two ways: 'mr sir' is not a given-name title to either. Only STRICT relaxes — the family comma's DISABLED segment has no given name to join, and its post-comma segment is LENIENT already. - - AMENDED 2026-09-08 (#489, the #316/#489 bundle): the key is no longer the whole run. Both sites now match the run's folded key WHOLE or by its LAST WORD, and the invariant this entry rests on — that H1 and P5 cannot read one run two ways — is what the amendment preserves rather than what it costs, both sites still reading through one predicate beside `_title_key`. What moves is this entry's own example: `'mr sir'` IS a given-name-title run to both sites now, `sir` being its last word, so `mr sir abdul rahman` joins where the sentence above says it does not. The leaf test that pinned the old reading, test_group's `test_the_title_run_is_one_key_as_h1_reads_it`, is renamed `test_the_title_run_is_read_as_h1_reads_it`; it runs the GROUP stage alone, so what it pins is P5's side — that `mr sir abdul rahman` joins — and not the agreement between the two sites. The agreement rests on the shared predicate, `_run_addresses_by_given` beside `_title_key`, which is the one place either site asks the question, and it is ASSERTED end to end by `tests/v2/test_parser.py::test_the_p5_licence_and_h1_read_a_title_run_the_same_way`: eleven title runs, each parsed in both spellings, requiring `parse(f'{title} John').family == ''` and `parse(f'{title} abdul rahman').family == ''` to agree run by run. Nothing else in this entry changes: `Dr. abdul salam` still keeps given `abdul`, family `salam`, `dr` addressing by family. See #H1 for the measurement and the vocabulary scope. - - Measured at all three baselines: `Sheik abdul salam` moves (given, family), and with it the `Sheik abdul salam Jr` rules example — the same move with the suffix standing — and no other corpus name's fields do; `Sheik Abu Bakar` returns to the fields it had at 1.4.0, 2.0.0 and 2.1.0 — a restoration, which the ledgers record as a non-event — with one v2-surface change: the PARTICLE_OR_GIVEN report it carried through 2.1 is gone, because the join that takes 'Bakar' is P5's rather than the chain's, and a bound word read as the bound word is not a fork. The Arabic-script spelling comes along: `الشيخ عبد الله` read given 'عبد', family 'الله' and now reads given 'عبد الله', which is the name. The previous #369 ledger rules at all three baselines classified the regression "because the cause is known and recorded, NOT because the reading is wanted"; they are rewritten for the fix, and the 1.4.0 one is what keeps `Sheik abdul salam` explained at all: it landed on the fields-only suffix-routing catch-all before #451 deleted that rule, and with no fields-only rule left in any ledger the name now arrives UNEXPLAINED without it — measured by driving `classify()` over the 1.4.0 ledger minus this rule, which returns None for the name and for the `Sheik abdul salam Jr` example. A stronger reason to keep the rule than the one this entry first gave: the alternative is a red gate, not a mislabel. - - Two things review added before merge. The licence lifts the reserve for two WORDS, so the piece it lets the join take must be a single word: a particle chain is one piece, but it is the family name P2 built, and without that clause `Sir abdul van der Berg` read given 'abdul van der Berg', family '' — where the untitled name keeps family 'van der Berg', and so does `Sir John van der Berg`. H1's own scope is "exactly one name word", and the licence now matches it. `Sheik abdul al Rahman` therefore stays given 'abdul', family 'al Rahman', the same limit the `abd` entry above records for the three-token spelling. `Sir abdul van der Berg` is a rules.md example and so sits in the rules corpus, where it reads byte-identical to every baseline — the gate witnesses the clause; `Sheik abdul al Rahman` is in no corpus, and the group test is its only witness. And P5 now states the precedence `Sheik Abu Bakar` only exercised: a bound word that is also a particle is read as the bound word, the join outranking P4's leading-position reading, with no fork reported. That is the reading `Abu Bakar Salim` has always had — P5's join and P4's "the words stay separate" were never reconciled in writing — and the two rules now point at each other. - - Three more from the PR review. The licence takes a name WORD, not a suffix: lowered to LENIENT, the reserve would let the join absorb a suffix piece — `Sheik abdul Jr Smith` read given 'abdul Jr' — which is the shape #421 records for the post-comma LENIENT path and which the STRICT reserve had kept away from titled names; the licence now declines a suffix piece on its own, and #421 will make that decline general. One shape it cannot decline belongs to #401: `Sir abdul V` reads given 'abdul V', because the reserve's initial veto counts a bare `V` as a name word, and the lowered threshold now lets that count fire where STRICT did not — recorded in #401 rather than fixed here. And the join is order-blind, so under a family-first order the licensed pair lands in the FAMILY with no given name: `Sheik abdul salam` reads family 'abdul salam' under both `FAMILY_FIRST` and `FAMILY_FIRST_GIVEN_LAST` (was family 'abdul', given 'salam'), as `Sir John` reads family 'John' there — H1's exemption never runs under a family-first order, so the two rules still agree. Accepted rather than scoped to the given-first read: the effective order is settled in assign, after group, and the outcome is the one H1 already gives the same shape. Two corpus names move under each family-first order, `Sheik abdul salam` and `Sheik Abu Bakar`, measured by hand — the gate reads the default order only. - -- 2026-08-22 #401 / #421 — the reserve and the join ask the same question, and it is the one assign answers. Two defects, one cause: P5 says the join "needs a name word to spare" and joins "the word after it", and neither site knew what a name word was. The reserve counted the pieces that were not suffix pieces, and `_is_suffix_piece` vetoes on the initial tag — right for assign's middle-initial question, wrong for the reserve's: a bare `V` carries both `vocab:suffix` and `initial`, so `abdul Smith V` counted three name words, joined, and assign then read the `V` as the suffix it always was — given 'abdul Smith', family '' (#401; shipped since 1.x — 1.4.0 read first 'abdul Smith', last 'V' — every bound word, only the single-letter numerals V, I, X). The join never asked at all: with a word to spare it took a suffix piece as "the word after" — `abdul Jr Smith Berg` read given 'abdul Jr' (#421; 1.4.0 parity) — and where the suffix was the split credential, merge() unioned the piece tags, the joined piece became a suffix piece, and assign sent the bound word to the suffix field: `abdul Ph. D. Smith Berg` read suffix 'abdul Ph. D.', a 2.0 regression (v1's fix_phd extracted the credential before parsing, so position never mattered). - - Fixed with one predicate, `reads_as_suffix`: a suffix piece, or the trailing roman numeral assign's fork takes — and that fork is now `_vocab.is_trailing_numeral_suffix`, lifted out of assign so the two stages share its definition instead of the reserve carrying a copy (the `_ROMAN` pattern moved with it; the regex-sync pin follows it). The reserve counts a piece out when it reads as a suffix, and the join declines the piece it would take when it does — the same decline it already made for a marker. The mirror is assign's fork condition for condition: the numeral is the LAST piece and not the first name piece — not "the piece before it is not a title", which the first draft wrote and the design-docs review falsified, `jr` being title vocabulary as well as a suffix, so `abdul Smith Jr V` lost its family exactly as before — and the piece before it is read as the join would leave it, the joined pair's first token being the bound word (`abdul J. V`, the design-docs review's other find, otherwise suppressed the fork for the reserve alone and read given 'abdul J.', family ''; the reserve now declines, and assign, seeing the unjoined pieces, reads given 'abdul', middle 'J.', family 'V', exactly as it reads `John J. V`). Two more the code review found: assign drops the group-flagged credential pieces (the Ph. D. merge) from its walk at any position before the trailing peel, so a numeral can be last in that walk without being the last piece — `abdul Smith V Ph. D.` lost its family until "last" and "the piece before" were read over the pieces assign keeps — pinned at the piece level only, because a trailing split credential is the shape the 1.4.0 ledger's `[[never]]` entry refuses to explain any diff on, and v1's reading of that name carried the #401 family loss; the example stayed out of the rules corpus rather than the never-entry being narrowed for one bound-word spelling; and the post-comma walk has no numeral fork at all (a trailing numeral there is read by a lenient last-of-two rule, or as a middle initial with a third part), so a mirror applied under LENIENT declined `Berg, abdul V` and `Berg, abdul V, jr` against every baseline with nothing classifying it — the numeral branch is scoped to STRICT, and after a family comma the reserve counts suffix pieces alone, as it always did. So `abdul Smith V Jr` still joins and assign reads family 'V', and `abdul V Smith` still joins an initial, which is a name word. What is NOT mirrored: S2's other positional fork, the bare ambiguous acronym. Where the join leaves two pieces no family is lost — `abdul Smith Ma` reads family 'Ma' where `John Smith Ma` reads suffix 'Ma', assign keeping the acronym as the family of a two-piece name — but with a suffix word between, the join leaves three, assign peels the acronym and then the suffix, and the family is gone: `abdul Smith Jr Ma` reads given 'abdul Smith', family '', suffix 'Jr, Ma' where `John Smith Jr Ma` keeps family 'Smith'. Pre-existing and byte-identical before this fix, and the whole of what remains of #401's shape on the main walk: a review sweep of the STRICT path found 132 family-losing shapes, every one ending in a bare ambiguous acronym. Left for #425 rather than widened into here, and framed there as sharing assign's peel over a post-join VIEW rather than mirroring a second fork by hand: with a view the count is the view's length, the remap and the first-piece carve-out fall away, and the other two sites that stop at a suffix with the same initial veto — P2's chain and M2's walk, which take a trailing numeral the same way (#424) — would ask the question once. The walk was also computed once rather than per piece after review measured the per-piece rebuild as quadratic (13x growth for 4x the input against the benchmark's 6x bound, on a shape no benchmark unit led with); `bound_given` is that unit now. Readings: `abdul Smith V` → given 'abdul', family 'Smith', suffix 'V', the SUFFIX_OR_NAME report unchanged; `abdul Jr Smith Berg` → given 'abdul', middle 'Jr Smith', where `John Jr Smith Berg` puts it; `abdul Ph. D. Smith Berg` → given 'abdul', middle 'Smith', suffix 'Ph. D.' — the SUFFIX field is 1.4.0's again, the given is not: v1's fix_phd extracted the credential before parsing and the join then took 'Smith', reading given 'abdul Smith' — which the issue reports correctly, and then mislabels its own proposed reading as "1.4.0's reading restored". The decline is chosen over reaching past the credential for the next name word, because P5 joins "the word after it": a join that skipped a suffix would read a shape no one writes as if the suffix were not there, and the marker decline already sets the precedent that the join takes the word after it or nothing. A judgement, recorded as one; `Berg, abdul Jr Smith` → suffix 'Jr'; and `Sir abdul V`, the shape #369's licence had widened #401 onto, reads suffix 'V'. #369's licence-local suffix clause is retired, subsumed by the general decline. `bound_given_join_sees_only_the_surviving_name` re-pins to given 'abd', middle 'Jr Smith', as the #418 entry anticipated. No differential corpus name has either shape — measured: the gate was byte-identical at all three baselines before the rules.md examples went in — so the examples carry the shapes into the rules corpus, where they move against every baseline and are classified there. The 1.4.0 `fix(#401)` rule lists three fields where the 2.x rules list two, because 1.4.0 did not even read the V as a suffix; a draft with two fields handed the name to the fields-only suffix-routing catch-all, whose capped printout hid it, and this entry briefly called #401 a 2.0 regression on that inert measurement — the design-docs review caught it. EXPLAINED NOTHING is not byte-identical. - -- 2026-08-22 #425 — the reserve runs assign's peel over the view the join would leave. #423 had the reserve re-derive assign's trailing peel by hand — the flagged-credential drop, the walk's start, the numeral fork's conditions, the preceding piece remapped to "as the join would leave it" — with each seam held by a pin, and the bare-ambiguous-acronym fork not mirrored at all: `abdul Smith Jr Ma` read given 'abdul Smith', family '', suffix 'Jr, Ma' where `John Smith Jr Ma` keeps family 'Smith', the whole residue of #401's shape on the main walk (every family-losing bound-word shape in the sweep below contains one). The altitude review's framing is taken: the peel is one function, `_peel_trailing` over the list `_peel_walk` builds (the walk's input defined once too, after the /simplify pass found it spelled at three sites), housed in group beside the piece predicates because assign imports those and group cannot import assign; assign calls it and reports what it returns (the numeral fork's piece, the acronym picks, in the order it always reported them — byte-identical over the suite and the three gates); and the reserve builds the VIEW merge() would leave — the pair one piece, its tags the union — and calls the same function. `BoundJoin`'s "the value IS the threshold" trick survives restated as post-join name pieces: STRICT 2, LENIENT 1. The remap and the `k == first_name_k` carve-out are gone, the joined piece being two tokens and never a suffix piece. -- 2026-10-07 #614 — THE RESERVE READS NO VIEW: S2's trailing run is read once, after P3's joins and before the bound-given join is tried, and split off, so the words to spare are the name pieces it left, and the join can move no suffix reading because nothing in group reads the run again (rules.md#P5 amended; decisions.md#S2, this date). SUPERSEDES the two-view comparison, which survives only where the run is not read there (a run with a name piece inside it), and with it two pinned readings: `abdul J. V` and `Sir abdul J. V` now join, given 'abdul J.' with family 'V' -- unjoined the initial in front suppresses the numeral fork, and that is the one reading taken now, where the joined view had read the V as a suffix and declined; 1.4.0 read the same, 2.0 and 2.1 suffix 'V', 2.2 and 2.3 middle 'J.' -- and `Sheik abdul Ma` reports the pick the read weighed (`suffix-or-name`), superseding the #425 bullet's application of the #369 precedent to the pair's suffix fork: the read decides 'Ma' before the join, so the fork is a fork. #369's own precedent, a bound word that is also a particle read as the bound word with no PARTICLE_OR_GIVEN report, is untouched (`Sheik Abu Bakar` reports nothing). Accepted by Derek as a class with the rest of #614's moves. - - One principle came out of writing the ledger that the view alone did not state. `abdul Smith Ma` had read given 'abdul Smith', family 'Ma' since 2.0, and the #423 entry called that the lossless shape; the 1.4.0 gate's catch-all count said otherwise — 1.4.0 reads first 'abdul', last 'Smith', suffix 'Ma', as `John Smith Ma` reads, and the fields-only catch-all had absorbed the diff since the example went in (the count is the tell, per the #401 entry). Unjoined, the acronym has words to spare and peels as a credential; joined, the view is two pieces and the fork keeps it as the family — the join was CHANGING what assign reads. So the rule is stated as P5 now has it: the join joins two name words into one and changes nothing else, which the code asks directly: the pieces the peel takes are the same either way (a count proxy, "the view leaves exactly one name word fewer", was the first spelling; the altitude review pointed out it held only because the peel is a tail walk, and the comparison states the rule). That comparison subsumes the unjoined-piece check #423 added for `abdul V` (unjoined the peel takes the V, joined nothing) and restores `abdul Smith Ma` to 1.4.0's reading; `abdul Smith Berg Ma` keeps its join, the acronym peeling either way. Every #423 pin holds. Measured: the 1.4.0 gate is byte-identical at the code change, before the examples went in — the three `Ma`-final examples are v1 parity, the catch-all back at 15 — and `abdul Smith Jr Ma` and `abdul Smith Ma` move against 2.0.0 and 2.1.0, classified there; no hand corpus has the shapes. The population the corpus cannot see, from the design-docs review's sweep of 67,860 constructed main-walk shapes (three bound words — abdul, abd, abu — × twelve words — Smith, Berg, Ed, J., Jr, M.A., Ma, PhD, V, X, 'Ph. D.', 'van der', none a title word — × tails of one to four; no comma shapes; a different twelve-word pool gives different counts, so the pool is the claim): before this change 2,160 lost the family and every one contained a bare ambiguous acronym (864 with further suffixes behind it, so "ending in" was an artefact of #423's shorter sweep); after it none does, and all 67,860 match the family and suffix their ordinary-given twin reads. 5,166 move against master in all — 3,006 where the acronym was the family and is now the suffix (`Sir abdul Smith Ma`), 2,160 where the family is restored. The title-word class below was outside that population and moves after a family comma too (`Berg, abdul Sir`); so was the licence class at the end of this entry. - - Three things the principle does not reach, recorded so nobody applies it there. The post-comma LENIENT path is untouched (AMENDED by #637, below: its decline now reads H5's trailing run where a title word follows the bound word), and NOT for the reason first written here ("its walk has no peel" — it reads a trailing numeral by the lenient last-of-two rule, as the #423 entry says): the family is fixed there and the pair is the given whatever follows, so the principle is not applied, and `Berg, abdul V` keeps given 'abdul V' though the V would be the suffix unjoined — 1.4.0 parity, and exactly the regression #423 measured when a mirror was applied there; P5 now carries it as Accepted. The vocabulary decline is the UNAMBIGUOUS vocabulary: a bare ambiguous acronym the peel does not take is a name word, so `abdul Ma Smith` reads given 'abdul Ma' (1.4.0 read middle 'Ma' — a 2.x difference no corpus witnesses, now an Accepted example). And the principle is about the suffix reading, not the report: `Sheik abdul Ma` reports no SUFFIX_OR_NAME where `Sheik John Ma` does, the #369 precedent — a bound word read as the bound word is not a fork. One more class moves, found by the code review and then measured against 1.4.0: the 2.x reserve had excluded TITLE pieces from its count — an unrecorded deviation from v1, whose join counted suffixes out and nothing else — and the shared peel, following assign, reads a mid-name title word as the name word it is. `Berg, abdul Sir` (given 'abdul', middle 'Sir' since 2.0) and `abdul Smith Sir` (declined since 2.0) read given 'abdul Sir' and given 'abdul Smith', family 'Sir' — 1.4.0's readings, both; `abdul Sir Smith Berg` never stopped reading given 'abdul Sir'. No corpus name has the shapes that move, and no rules example carries one — `abdul Sir Smith Berg`, P5's example of the reading, is byte-identical at every baseline, so the class is pinned in tests alone (the release log says so; the late review of this entry found the sentence here implying otherwise). The same review found the clause had taken `do`, `freiherr` and `st` from P6 — recorded under P6. What the review actually caught was the tag artefact that exclusion had been hiding: the conjunction merge derives a `title` piece tag for a chain such as "Sheikh and Ahmad", merge()'s union handed it to the pair, and assign peeled the pair as a leading title — `abdul Sheikh and Ahmad Bakar Smith` read title 'abdul Sheikh and Ahmad' on 2.0 and 2.1 (v1 read it as the given), and `abdul Sheikh and Ahmad Bakar`, which the exclusion had kept unjoined, would have too once it went. The bound join drops the tag, and the view drops it the same way — the pair is a given name whatever tag the word carried. One more class the comparison moves, found by the code review's classification of every residual master-to-branch diff (12 of 17,640 constructed names — a fourteen-word pool, bare and behind `Sir`, tails of two and three, 6 × (14² + 14³); the review did not record the pool — one shape; 360 of 360 over {Sir, Sheikh, Dame, Sheik, King, Father} × {abdul, abd, abu, umm} × {J., V, X, A., I} × {V, X, I}): behind a given-name title, a numeral after an initial-shaped piece. `Sir abdul J. V` read given 'abdul J.', suffix 'V' since #369 — the fork suppressed unjoined, fired joined, and the licence's threshold of one let the join through — and the comparison declines it: given 'abdul', middle 'J.', family 'V', exactly as `Sir John J. V`. Pinned at both levels; it was the only shape a looser comparison let through under mutation at the test review. #424 — P2's chain and M2's walk taking a trailing numeral — is the remaining asker of the same question and now has `_peel_trailing` to ask it of. -- 2026-10-11 (Derek), #637 and #640 — THE BOUND JOIN DECLINES A TITLE WORD A TRAILING RULE READS AS A TITLE. P5 joins a bound given-name word to the word after it, and declined a maiden marker and a suffix piece; a title word it took, so where a trailing rule would have read that word as a title the joined piece was no title word to read. After a family comma the join reads no peel (the LENIENT reserve, #423), so `Berg, abdul Prof.` read given 'abdul Prof.' where `Berg, John Prof.` reads title 'Prof.' (#637, the gap the 2026-09-09 A1 entry recorded); in a part past the second comma after a suffix comma, where the join runs STRICT, `John Smith, Jr., abdul Dr. Smith` read suffix 'Jr., abdul Dr. Smith' where `John Smith, Jr., Berg Dr. Smith` reads title 'Dr.' (#640). Now the join declines a title word a trailing rule reads as a title: after a family comma, one H5's trailing run takes from the given part (`tail_reading`, asked only when a title word follows the bound word), and in a part past the second comma any title word, which C2 reads as a title. `Berg, abdul Prof.` → title 'Prof.', given 'abdul'; `John Smith, Jr., abdul Dr. Smith` → title 'Dr.'. - DECIDED (Derek): this narrower clause (a′) rather than declining every title word (a). P5 states that a title word standing in the name is a name word and joins like one (H3), and (a) undid it: it failed six tests, a P5 rules example and the two tests named for that clause among them, and over the bound-given grid below moved 4,044 parses, 3,000 of them a join split with no title read (`(Bob), Abu Bishop` → given 'Abu', middle 'Bishop') beside the 1,044 that (a′) moves (six tests measured against 4455962f's suite). (a′) leaves every join on a title word inside the name standing -- `abdul Sir Smith Berg`, `Berg, abdul Prof. Ali`, `Smith, abdul Bishop`. - MEASURED 2026-10-11 against 4455962f (#639's merge), py3.11, `nameparser.__file__` asserted on each side. A bound-given grid -- `{head} {bound} {next} {rest}` for head in (none), `Dr.`, `Sir`, `Smith,`, `van der Berg,`, `,`, `(Bob),`, `Smith, John,`, `John Smith, Jr.,`, `John Smith, PhD,`, `John`, `Mr. and Mrs.`; bound in `abdul`, `Abu`, `abd al`; next in `Dr.`, `Prof.`, `Prof`, `Bishop`, `King`, `King.`, `Sir`, `St`, `Freiherr`, `Salam`, `Rahman`, `Lt.Gov.`, `Mr.`, `van`, `MA`, `J.`; rest in (none), `Smith`, `Ali Smith`, `Dr.`, `Smith Dr.`, `Jr.`, `MA`, `Smith, Jr.`, `Prof. Smith` -- 5,184 texts under the three orders, comparing the seven fields and every report: 1,044 parses move, every one a title word into the title field and no report moved alone -- 660 in a part past the second comma (`John Smith, Jr., Abu Bishop Ali Smith` → title 'Bishop', as C2 reads `John Smith, Jr., Bishop`), the rest after a family comma or an empty head (`(Bob), Abu King.` → title 'King.', given 'Abu'). #623's given-part grid (decisions.md#H5): 280 -- 260 the same class (`(Bob), abdul Dr.` → title 'Dr.', family 'abdul', H1's move behind a title), and 20 that split the join with no title read (`Smith, abdul Prof. V, Jr.` → given 'abdul', middle 'Prof. V'), the class below. Every corpus name and case text at 4455962f: only `Berg, abdul Prof.`, the row that had pinned the gap. The S2 stress grid, #624's title-particle grid and #625's particle grid: 0. The five gates exit 0, with `fix(#637)` and `fix(#640)` rules in each 2.x ledger; at 1.4.0 the broad `fix(comma-family) lone post-comma piece routes to suffix/title` rule explains both new examples. Frames: the reference rows do not move; a comma-less name with a bound given word pays nothing, the site asked before the title test (`abdul salam ahmed salem` 241), a family-comma one with no title behind it one for the title test (`Berg, abdul salam` 208 → 209), and one with a title word behind it after a family comma about ten to twenty for the H5 read (`Berg, abdul Bishop` 208 → 218, `Berg, abdul Prof.` 207 → 228), rare shapes no guard sees. Recorded negative controls (with the review's rows): with no title declined, 11 tests fail (the four #637/#640 rows on both surfaces, H5's example and two P5 examples); with the past-the-second-comma half alone removed, 5 (its two rows on both surfaces and one P5 example). - THE REVIEW (2026-10-11). The decline after a family comma asks H5's run as the comma-less name reads it (`tail_reading`), and the given part's own walk reads some words behind a title otherwise, the disagreement #623's entry (decisions.md#H5) records for the chain's bound: there the join is declined and no title is read. A shape-only numeral VI to X, or V before a third comma part, which the walk keeps a name word (`Smith, abdul Prof. VI` → given 'abdul', middle 'Prof. VI', where master and 1.4.0 read given 'abdul Prof.', middle 'VI'); a period-marked title that is also a particle, which P6 then attaches with its report (`Smith, abdul St.` → family 'St. Smith', particle-or-given); and a bound word that is also a particle before one (`Smith, Abu St.` → title 'St.', where `Smith, Zed St.` reads family 'St. Smith'). Every such shape is garbage; the first two read as the same name with an ordinary given word does. Recorded, not narrowed, as #623's disagreement was (Derek, #623), and rules.md#P5 states it as an Accepted clause; the review's sweep found nothing reading worse than master on a real name and no comma-less name moving. It also found two mechanisms no test held, now rows: the bare-title half past the second comma (`John Smith, Jr., abdul Sir Smith`) and the whole trailing run after a family comma rather than the last word (`Berg, abdul Prof. Dr.`). - -Excluded (BOUND_GIVEN_NAMES): - -- mohamad — a standalone given name in its own right; binding it would eat the middle name. -- abd — ADDED 2026-08-18, see the entry above; the exclusion stood on two reasons and only one of them survived. The collision with the post-nominal "ABD" turned out to cost nothing (position decides at both ends). The other reason stands as a known limit: the three-token spelling "abd al rahman" is still not joined, `al` being a particle that chains forward, so only the two-token "abd X" and the hyphenated "abd al-Rahman X" join. - -Excluded (DEFAULT_NICKNAME_DELIMITERS): - -- Curly single quotes ('‘','’') are deliberately absent from the default pairs: U+2019 is the typographic apostrophe (O'Connor in curly type), so shipping the pair would eat real names. #273's own proposal excluded them; pinned by the curly_apostrophe_stays_literal case. A sweeper "completing the typographic set" ships the regression. - -Excluded (given-name titles and post-nominals — the 2026-07-19 transliteration deferrals, each living in a data-module comment until now): - -- Arabic bare سيد/شيخ/أمير/سلطان — given-name collisions (Sayyid, Shaikha, Amir, Sultan); the honorific forms with the article (الدكتور, الشيخ) ship instead. -- The abbreviation د. — edge-period normalization leaves bare د, the single-letter trap; Greek bare κ deferred on the same grounds. -- Ottoman post-nominals باشا/بك/أفندي — surname collisions. -- Hebrew bare רב (an ordinary word, "many") and בר (Bar is a common modern Israeli given name) — deferred, #269's territory. -- Latin sri/shri deliberately absent while Devanagari श्री ships: the transliteration collides (Sri Mulyani), the native script cannot. -- Latin sant and pandit deliberately absent while Devanagari संत and पंडित ship: Sant is a Punjabi given name (Sant Singh) and Pandit a common surname (Vijaya Lakshmi Pandit), so the Latin forms fail #vocabulary-collisions C-i's position test at the position the claim acts on, and TITLES has no ambiguous subset to express a partial answer in. The native spellings carry neither collision. The Latin forms belong, if anywhere, to the opt-in packs of #345 (2026-09-06, #344). - -Excluded (multi-word vocabulary entries — a set outside #M2's two exceptions matches one written word, so a multi-word entry there is silently inert and warns at configuration; given_name_titles and, since #434, maiden_markers are the exceptions, and they match phrases by different mechanisms): - -- Eight entries shipped unmatchable from 2013 to 2.0. chargé d'affaires was SPLIT into the chainable chargé + d'affaires, plus unaccented charge (the attaché/attache precedent, Derek's call); leed ap, nicet i–iv and psm i/ii were REMOVED rather than split, on the name-swallowing measurements recorded in #291 (see the comma-suffix-arc Declined entry). The multi-word UserWarning is this story's enforcement. - -Excluded (Lexicon.honorific_tails — a glued tail peels only if it could never end a name; per-entry reasons live in nameparser/config/suffixes.py's vetting block): - -- 양, 군 — 김지양 and 김지군 are given names ending in these syllables, and 양 is a top-tier surname (Yang) besides. (The surname-LEADS argument is a different job: it is why both are safe in the SPACED set — a leading surname never meets the trailing-only suffix gate, so 양 미선 keeps family 양.) -- 氏 — 王氏 is a historical name form ("the Wang woman"). -- 博士 — 田中博士 is Tanaka Hiroshi as readily as Doctor Tanaka. -- 殿 — Japanese surnames end in it (鵜殿, 真殿, four-figure populations); peeling it would cut a real family name in two. -- 君 — 王君 is a complete Chinese name; its kana spelling くん does peel. -- 部長, 部长 — 長 alone is a borne given name, so 阿部長 can be 阿部 + 長 (#707). The measurement found no such bearer, as it found none for the words that shipped; the exclusion rests on the argument, which a lower-bound zero does not refute, so a re-measure that again comes back zero is no reason to add them. -- जी, and with it the whole spaced trailing set the 2026-09-06 Indic bundle shipped — जी, साहब, साहिब, साहेब, महाराज, সাহেব, বাবু, মহারাজ and Latin rinpoche. None is a honorific_tails entry and none may become one. The evidence is जी's and it carries the set: Banerjee, Mukherjee and Chatterjee end in the substring (बनर्जी/मुखर्जी/चटर्जी), and glued peeling strands a fragment on a bare virama (बनर् + जी). The 殿 criterion, in a non-CJK script — which also shows the criterion is not CJK-specific. The accepted cost is that गांधीजी stays unpeeled, and it is the right trade: an unrecognized honorific costs one field, a strand cuts a real family name in two. Both shapes are pinned in tests/v2/test_locales.py. See #indic-honorifics. - -Excluded (TITLES): - -- ঠাকুর — a genuine Bengali honorific (lord/master) that is also Tagore, the surname (#343 records it so a wordlist sweep does not ship it). -- ठाकुर — the Devanagari spelling of the entry above, excluded on the same ground and recorded with it so a sweep of one script does not ship the other (2026-09-06, #344). -- कुमारी / কুমারী (Kumari) — a given name and a family name in the very script the honorific would be written in, so #vocabulary-collisions C-i's position test fails at the position the claim acts on. TITLES has no ambiguous subset to express a partial answer in, so under C-i's "under uncertainty, default to ambiguous" the only available answer is do not ship (2026-09-06, #344/#343). -- बेगम / বেগম (Begum) — a trailing name element far more often than a leading title, so the leading claim would be wrong more often than right (2026-09-06, #343/#344). -- शेख / শেখ (Sheikh) — a clan name and a family name in both scripts ("শেখ হাসিনা"). The divergence from Latin is deliberate and is the sri/shri case run backwards: the Latin sheikh/sheik/shaykh/shaikh cluster SHIPS as given-name titles for the Arabic addressing form, while the Indic spellings name the family (2026-09-06, #344/#343). -- आचार्य (Acharya) — a Brahmin surname (2026-09-06, #344). -- राजा / रानी (Raja/Rani) — common given names (2026-09-06, #344). -- ông / bà (Vietnamese Mr / Mrs) — Ông is a Vietnamese surname and Vietnamese names lead with the surname, the position the title claim acts on; pinned by the case row salutation_that_leads_a_surname_stays_a_name (2026-10-04, #606, #salutation-titles). -- thiru (Tamil Mr) — a Tamil given name (2026-10-04, #606). -- puan (Malay Mrs) — an Indonesian given name, Puan Maharani (2026-10-04, #606). -- pan (Polish/Czech Mr) — the Chinese surname Pan, which leads (2026-10-04, #606). The feminine paní ships, its accent keeping it off the name Pani; the unaccented pani does not. -- rodina (Czech "family") — a Russian surname, often written surname-first (2026-10-04, #606). familie ships. -- gróf (Hungarian Count) — possibly a surname in a surname-first culture; unverified, and under uncertainty the answer is do not ship (2026-10-04, #606). -- cik, paron, tikin, janab (Latin spellings) — unverified, so not shipped. Latin janab follows the sri/shri pattern: Devanagari जनाब ships, and the Latin spelling needs its own check (2026-10-04, #606). -- marshal — a given name (Marshal Yanda). The accepted cost is that `Field Marshal Bernard Montgomery` keeps reading title Field, given Marshal (2026-10-04, #606). -- justice — a given name (Justice Smith). Likewise `Chief Justice John Roberts` keeps title Chief, given Justice (2026-10-04, #606). -- malam (Hausa teacher/Mr.) — a US Census surname (119), 5+ Wikidata family bearers and 1 Wikidata given-name bearer (Nigeria), a lower bound (2026-10-17, #662, `tools/name_usage.py malam`). CLDR's Hausa data files Malam as the title (`grep '"title": "Malam"' tools/differential/cldr_fields.jsonl`), the opposite of the evidence that decided shehu, so this is a decline on the counts, not a shehu-style removal: Derek dropped it for the lower-bound given-name use, and whoever re-argues it should count Hausa given-name use first. The accepted cost is `Malam Shehu Bello` reading given Malam. -- bay (Turkish Mr.) — a US surname with 6,350 bearers and 120 on Wikidata, 11 of them Hungarian; Hungarian writes the surname first, where the title claim acts, so it fails C-i (2026-10-17, #662). -- shehu (Hausa honorific) — a common Hausa given name and an Albanian and Nigerian surname, so the leading title claim took the given name of `Shehu Shagari`, a former president of Nigeria, who read title Shehu, family Shagari in every release from 1.4.0 through 2.4.1. It arrived with an early bulk import (1a51555f) and was never put to C-i; removed 2026-10-13 (#660). Counts from `tools/name_usage.py shehu` on 2026-10-13: 561 US Census surname bearers; on Wikidata 180 family-name bearers (Nigeria 20, Albania 11) and 1 given-name bearer (Nigeria), a lower bound, since Shagari's own entry has no given-name statement. Not weighed under the graf class (#salutation-titles), and the counts above would not settle that weighing: the one leading-position figure, the single Wikidata given-name bearer, is a lower bound and sits below herra's ~670, while the 561 and 180 are family-name bearers, a position C-i does not ask about. The evidence that Shehu leads a name is not a count. It is Shagari himself, and CLDR's Hausa sample data in this tree, whose six renderings carrying the word all file it as the given name (`grep '"given": "Shehu"' tools/differential/cldr_fields.jsonl`). Whoever re-argues the word should count Hausa given-name use first. The accepted cost is the honorific use, `Shehu Abubakar` reading given Shehu as any untitled name does; it belongs in an opt-in pack if anyone asks for it. Moved three CLDR corpus names (`Shehu Muhammed`, `Muhammed, Shehu`, `Modibbo, Shehu Bello`), all three now agreeing with CLDR on every role, so they left `tools/differential/cldr_agreement.json` (moves: `uv run python tools/differential/snapshot.py --since v2.4.1`). -- The trailing-position rule that must NOT be adopted, and 2026-09-08 says what the line is: TITLES holds hundreds of words in no suffix set, at least nineteen of them ordinary English surnames (king, judge, bishop, baron, sheriff, ...), so a blanket "vocabulary outranks position in the trailing slot" reading would turn "Mary Jane King" into title="King" with the family name gone. The leading half of this argument is AGENTS.md's "Dean is deliberately absent" gotcha; this is the trailing half, and it shadows the family name rather than the given (#316). What #316 settled is that the prohibition is on BARE words: "Mary Jane King" still reads family King, while "John Smith Prof." reads title Prof. **The second half of that argument is RETRACTED 2026-09-09**, in review of the docs commit: "a period-marked trailing word being one nobody writes as a surname" claimed the period keeps the collision out of the trailing slot, and it does not. Measured on the branch tree — "Mary Jane King." reads title "King.", given Mary, family Jane, the family name shadowed exactly as the bare reading would have shadowed it, and "John Smith Judge." reads title "Judge." The reach is the vocabulary rather than a handful: of the 746 titles in no suffix set, 627 read as a trailing title once a period is written behind them, and every one of the 119 that do not is kept out by the abbreviation SHAPE — a digit, a hyphen, an apostrophe, or a script whose letters carry combining marks (rules.md#H2 records that half) — no plain ASCII-letter title missing at all. So king, judge, bishop, baron and sheriff ARE reachable in the trailing slot; what the prohibition above buys is the bare spelling and nothing more. The period is a WRITING convention the rule can read, not evidence about the word, and reading it is an ACCEPTED COST under the premise that the input is a name (rules.md H Background): someone who ends a name with a period-marked word has written an abbreviation. rules.md#H5 carries it as an Accepted clause with "Mary Jane King." as the executable example. Recompute the reach with `L = Parser().lexicon; ws = [w for w in L.titles - L.suffix_acronyms - L.suffix_words if " " not in w]; len([w for w in ws if parse("Mary Jane %s." % w).title])` against `len(ws)` — 627 of 746 on 2026-09-09 — the digits move as the vocabulary grows and neither argument does. See #H5. - -Excluded (Policy.script_orders defaults): Script.KATAKANA is deliberately absent — a pure-katakana token is predominantly a transcribed foreign name kept in its source order, so nothing defaults on it (rule W4's boundary). Noted 2026-08-15: of the three Script-keyed axes, this is the one with no force-a-decision guard (mechanisms.md#FORCE-A-DECISION-TABLE), so a new Script member silently gets no order. - -Excluded (MAIDEN_MARKERS, per nameparser/config/maiden_markers.py): - -- Polish "z domu" — the exclusion is LIFTED and the phrase SHIPS; see the INCLUDED entry below for the argument. It was provisional from the day it was written, 2026-07-27, pending the multi-token matching decision — first tracked in #291, repointed to #434 on 2026-09-07 per Derek's amendment on #342, markers having no run predicate, so the multi-token question does not dissolve there the way the suffix one did — and #434 shipped whole-phrase marker matching on 2026-08-26, which is what lifted it. Kept rather than deleted because a provisional exclusion that was answered reads, once deleted, as an exclusion nobody ever considered. -- Contrast entry — unaccented "nee" SHIPS as a marker despite - #274 flagging "is nee safe as a default (it's also a rare - surname)" as open; the question resolved silently with the shipped set. Recorded here because the included risky member deserves its analysis as much as the excluded ones; M1's (Nee) boundary covers only the enclosure path, not this marker path. -- "born" — never shipped: a release-log drafting invention, caught by the 2.0 milestone audit and corrected (5ccf9f3). Recorded so nobody "restores" it; if ever proposed for real, Max Born is the counterexample to analyze. -- Scandinavian "f." — collides with the initial F.; only the full participles (født/fødd/född) are safe. Czech masculine "rozený" awaits the same vetting, on its own merits rather than following the feminine form in. -- Polish "z domu" — INCLUDED 2026-08-26 as the set's first phrase entry, and the contrast to draw against the roz removal below rather than against any included word. Its words fail C-i separately and it passes as a pair; the argument is in the #434 entry under #M2. The point to carry away is that the entry below could not have been rescued the same way: roz is a whole marker as written, so there is no longer unit to move the claim onto. -- Czech/Slovak "roz." — SHIPPED through 2.1 and removed 2026-08-26 (Derek's call). Roz is an ordinary English diminutive of Rosalind, and matching is whole-token, case-folded and period-insensitive, so "Roz", "roz" and "roz." are one string to this set. The collision is not theoretical and it is not new: on the bare path, which predates the 2.2 work entirely, "Rosalind Roz Smith" read maiden "Smith" and NO family name at all, and "Rosalind Roz Jones Smith" read maiden "Jones Smith", because M2 hands the marker every word after it. Grounded in #vocabulary-collisions C-i — a word belongs in its set's ambiguous subset iff it is borne as an ordinary name IN THE POSITION THE VOCABULARY CLAIM ACTS ON, and under uncertainty default to ambiguous. Roz is borne exactly there: the word after a given name, which is the only position M2's claim acts on. What C-i cannot do for this set is the marking: MAIDEN_MARKERS has no ambiguous subset, and no code path consults one, so the only two expressions of C-i available here are ship and do not ship. That is why the remedy is removal rather than an ambiguous marking, and it is worth knowing before someone proposes marking the next collision instead. The full participle rozená stays and is unambiguous; a Czech caller who needs the abbreviation adds it to their own Lexicon, which is the same answer this section gives for every locale-specific vocabulary. The cost is accepted and stated: "Anna Nováková roz. Svobodová" now reads middle "Nováková roz.", family "Svobodová" — which is how 1.4.0 read it — and "Rosalind Roz Smith" is back to 1.4.0's given/middle/family. Pinned by tests/v2/cases.py::diminutive_that_was_a_marker_keeps_the_family and ::full_participle_marker_still_consumes. - -### C1 — the suffix-comma decision - -- 2026-07 (v2 core, PR #288) — v1 parity: only the second segment decides (v1 parser.py:1318), and ">1 word before the first comma" is v1's guard, which is why "Smith, PhD" keeps the listing form. The strict/lenient knob's blast radius on default vocabulary is exactly the single-letter Roman numerals i and v — the only initial-shaped members of the shipped suffix words — which is what makes rules.md#C1's "V." examples the canonical pair. -- 2026-07 (plan deviation #3, recorded) — the decision is definitionally vocabulary-dependent: there is no way to recognize a credential run without consulting the suffix word lists, so the structural stage reads vocabulary through one predicate. -- 2026-07-12/13 (v2 Policy work, PR #288) — the lenient token test is the default and `lenient_comma_suffixes=False` restores the strict one. (An earlier entry here credited #291/#296 — git author dates place both the field and its wiring in the Policy commits, and the #291/#296 design doc never mentions the knob.) -- 2026-08 #319 — the wholly-suffix predicate was lifted into the vocabulary layer so the comma decision and the honorific peel's segment test cannot drift apart. -- 2026-08-23 #296/#325 — the listing form's post-comma part is read for what it is. Three commits of the approved #291/#296 bundle (spec 2026-07-30, plan 2026-08-01; see the comma-suffix arc below), in the plan's order. (1) A comma followed only by titles fixed no family boundary, so the pre-comma name keeps its positional read: `John Smith, Mr.` reads given 'John', family 'Smith', title 'Mr.' where 2.0 had merged the family ('Bob Jones, author' moves with it at every baseline). "Nothing but titles" is title-shaped AND not suffix vocabulary — the plan's helper said only the first, and would have read `Smith, PhD Jr.` as the title run 'PhD Jr.' and `Smith, Mr. Jr.` as 'Mr. Jr.' (both master's suffix 'Jr.'; the gate and a measurement against master caught them). (2) The TITLES/suffix overlap audit and the ordering, one commit because each alone is wrong (the plan's F1): reading the credential run first would make `Smith, Dr.` a suffix while 'dr' was SUFFIX_WORDS residue, and the audit alone leaves `Smith, Jr.` a title by the period-abbreviation inference. The audit table's dispositions ship as approved — 'do', 'jr', 'junior', 'phd', 'se' out of TITLES; 'dr' and 'sra' out of the suffix sets; 'ms' and 'sa' into the ambiguous set — with ONE deviation on evidence the table did not have: 'md' keeps dual membership, because bare 'Md' before a name is the Bengali and South Asian Muslim abbreviation of Muhammad (#343/#345's corpus rows `Md Abdul Karim`, `Md. Abdul Karim`; the drop read the first as given 'Md', middle 'Abdul') while 'MD' after the name is the degree — position decides, as for 'sr'. The ordering is wider than the plan's lone piece: a post-comma part that is nothing but suffix pieces is the credential run, whole, which is what #325 needed (`Smith, Ph. D. Jr.` had put the split credential in the given name once a second suffix made it no longer lone — a 1.4.0 regression; v1 read suffix 'Ph. D.', title 'Jr.'). Vocabulary decides which words qualify and position breaks the tie for the genuine duals: `Smith, Sr.` is Senior where `Sr. Garcia` is Señor, `Smith, Esq.` is the postnominal where H2 still reads `Esq. Smith` as a title (H2 carries that as Accepted). `Smith, Dr. Jr.` keeps the 2.0 deviation's other case — a title, then the last piece as the suffix (v1 read first 'Jr.'; 2.0 through master suffix 'Jr.') — and `Smith, John Jr.` its given name. Consequences the audit carries, all measured and pinned: a leading 'PhD' or 'Jr' is a name word; `Do Quang Minh` reads given 'Do' with the particle-or-given fork, and `Dr. Do Van Johnson, MD` given 'Do', family 'Van Johnson' (the #367 rule for that name is replaced; TITLES ∩ ambiguous is {freiherr, st} now, which keeps the chained emitter reachable); a trailing bare 'Dr' is a name word, joining every other title-only word there (#316 is the open question); `田中さん, PhD` reads suffix 'さん, PhD'; the #100 prefix-join tests use 'lt', a genuine dual, as their token. #291 (multi-word credentials) is the bundle's fourth commit and its own PR. -- 2026-08-23 (review of PR #428, four reviewers) — the repair's condition is "no name word", not "all titles": `John Smith, Mr. Jr.` (a title and a postnominal) had kept the merge the rationale argues against, and reads given 'John', family 'Smith', suffix 'Jr.' now. Its guard counts NAME pieces: the positional read peels a trailing suffix first, so `Smith Jr., Mr.` — two pieces, one name — read positionally lost its family (the code review); it keeps family 'Smith', suffix 'Jr.' at every baseline. The positional read records its ORDER, and post_rules' leading-piece scan keys on "assign records no order after a family comma" rather than on the structure, so P1's family-first fold reaches the pre-comma name as it does without the comma — `de Mesnil Jean, Dr.` under a family-first order had read family 'de' (the test review). The credential run does NOT step over a delimiter core the policy names (`Smith, RN - CRNA` reads given 'RN' under `extra_suffix_delimiters` as without it): the code review asked for it, a draft did it, and tests/test_suffixes.py's "the delimiter must not affect parsing at all" outside the suffix-comma form is v1 parity pinned on purpose — C1 carries the limit as Accepted. A second deviation from the audit table: `ms` and `sa` do NOT join the ambiguous set — the gate is position-blind and the collision is not; gated, `John Smith, MS` lost its suffix-comma route and read title 'MS', and `Smith, Ms.` passed the gate on its one period and read as a credential anyway. Both are genuine duals, and position decides: `Smith, Ms.` reads suffix 'Ms.' (title at every baseline; C1 carries it as Accepted, with `Smith, Ms. Jane` keeping the title) and `John Smith, MS` keeps suffix 'MS'. And the gate itself now means what S2 says: an ambiguous acronym counts when written with ITS periods, one after each letter (`M.A.`, `J.D.`), not on any period — `Smith, Ed.` had become a credential on a name's trailing period, and `Jack Ma.` had read suffix 'Ma.' since 2.0 (S2 carries the boundary; 1.4.0 read family 'Ma.'). Two limits recorded rather than fixed: `田中さん, Dr.` keeps the honorific glued, joining master's `田中さん, Mr.` — the honorific peel runs in script_segment on the suffix-comma and no-comma structures, before group or assign can say the comma fixed nothing (C1 Accepted); and group's chain emitter still reports no PARTICLE_OR_GIVEN on the family-comma path, so `Freiherr von Berg Ma, Mr.` reads as its no-comma twin but without the report — group runs before assign decides which read applies, and plumbing a deferred report through the state is more than the shape is worth today. -- 2026-08-23 #429 — one comma segment is one suffix entry, and the two stages disagreed about which segments those are. group asked by segment INDEX (`tail_start`: 1 after a suffix comma, 2 after a family comma), which is right wherever assign reads the segment as suffixes for the same structural reason — but not after a family comma, where segment 1 is a name slot assign re-reads by CONTENT with `_segment_holds_no_name` and routes piece by piece (#296/#325). So the roles were right and the render was not: `Smith, MD PhD` gave suffix 'MD, PhD' with a comma the writer never typed, and `Smith, CBE MC`, `Smith, BSc MBA` and `Smith, PhD Jr.` the same, while the full-name `John Smith, MD PhD` has rendered 'MD PhD' since 1.4.0. The predicate moved down into group beside the piece predicates it already reads — the same direction `_leading_titles` moved in #424, and the only direction available, since assign imports group. (#436/#437 took group's call away again on 2026-09-06, leaving assign the one caller; the predicate itself stays in the piece layer.) - SCOPE, stated as the condition actually reads rather than as the first draft described it: any family comma whose following segment holds no name word, NOT only a one-word family. `John Smith, Jr. III` renders suffix 'Jr. III' (1.4.0's reading), and a title-led segment is in too — `Smith, Dr. MD PhD` renders 'MD PhD'. The differential cannot witness the widening: of 1069 corpus names, 303 carry a structure comma, the predicate is true for 36, only 9 of those have two or more pieces in the segment, and none of the 9 combines a multi-word pre-comma name with two suffix pieces. 0 unexplained is therefore evidence about the corpus, not about the class — the case table carries it instead. - THE REGRESSION THE REVIEW FOUND, and the reason this entry is long: the first draft let ANY piece open an entry, as the tail block always had. That is safe on a tail segment, where assign routes every piece to SUFFIX — which is what `tail` means — and wrong off it, where a title piece routes to TITLE. Two failures, one cause. The `joined` tag is role-BLIND and the facade heals it for every role, so a title opening an entry collapsed `title_list` from ['Rev.', 'Dr.'] to ['Rev. Dr.'] on `Smith, Rev. Dr.`; and where the pre-comma name left a suffix, the tag glued the next suffix BACKWARD across the writer's own comma — `Smith Jr., Mr. Jr.` rendered suffix 'Jr. Jr.', the exact inverse of the bug being fixed. Neither is visible to the differential, which compares strings, nor to the case table as it stood, which asserts the title STRING (space-joined either way). The rule that replaces it separates two joins that had been one: WITHIN a piece the tag renders a merged piece as one unit, whatever role it holds — that half still stands; BETWEEN pieces it continued an entry, and only a piece that rendered into the same run could do that. The entry was sticky across a piece that was not in it, so an interleaved title did not split the run it sat in (`Smith, MD Dr. PhD` renders 'MD PhD'), and a delimiter core closed it, which was the one thing that separated two entries. That between-piece half is GONE since 2026-09-06: #436/#437 replaced it with a rule over the commas the writer typed, the bullet below. Both readings it was built for are unchanged — the stickiness holds by construction now rather than by code. - Scoped deliberately, and the scoping is the other decision: `tail` still gates delimiter-core dropping, which is v1 `expand_suffix_delimiter` parity (#206, commit 021823e — three code comments say #191, which is the German/Dutch vocabulary PR, and are corrected here) and a TAIL rule. The core set is threaded to the block as `seg_cores` so the widened condition cannot drop cores where `tail` says not to. Worth knowing before anyone tries to pin that: for every realistic delimiter the two spellings are indistinguishable by construction, because the core must first pass the very predicate that gates the block — the only reachable class is a policy whose delimiter core is itself suffix or title vocabulary (`Policy(extra_suffix_delimiters={" jr "})`, `Smith, MD jr`), which the case table does not enshrine. - Cost, measured rather than optimised away: `_segment_holds_no_name` now runs TWICE per family-comma parse, once here and once in assign, on the same pieces — verified pure recomputation, not two questions (408 of 506 comma-bearing test literals reach both sites, 0 disagreements, and off-tail `seg_cores` is empty so `kept` never filters, which is why assign sees exactly the pieces group tested). Removing the second call is worth 0.6–1.4 µs, 1.2–2.2% of a family-comma parse and 0% of every other parse (NO_COMMA and SUFFIX_COMMA make no call at all). Caching it needs a derived ParseState field the two sites would otherwise not share, so the number is recorded instead. The order is cheapest-first per 7009fcf: two bool loads and an int compare gate the call. - R1 gained the separator sentence with this change. That the invariant it states — one comma segment is one suffix entry — was written down in no rule is a large part of why two stages disagreed about it for years: C1 governs routing, R1 governed word ORDER and never mentioned separation, and the only statement of the rule was a code comment. - Left alone, measured and pre-existing: the NO-COMMA path comma-joins a space-separated run too (`John Smith MD PhD` renders 'MD, PhD' at 1.4.0, 2.0.0, 2.1.0 and master alike). The fix requires FAMILY_COMMA and does not reach it; that path has no writer-drawn boundary and is a separate question, recorded here and NOT yet filed. It is load-bearing for one thing: `str()` of a fixed parse is a no-comma string, so re-parsing it re-inserts the comma — round-tripping is not stable on these shapes, and an earlier draft of the release bullet claimed it was. CLOSED 2026-09-06 by #436/#437, the bullet below — every sentence above stands as what that tree measured, and all three of its open clauses are now spent: the question WAS filed, as #436 and #437; the boundary moved from segment shape to the comma the writer typed, which reaches the no-comma path because it never asks about segments at all; and round-tripping is stable on these shapes, pinned by `test_a_suffix_run_survives_a_render_and_a_reparse` in tests/v2/test_render.py. - The name `Smith, MD PhD` entered the corpora as this fix's C1 example — a FIRING one, carrying no `· boundary` annotation, which marks the non-firing example a rule must have — so its ledger entries arrive with it. A fourth written for the 1.4.0 ledger EXPLAINED NOTHING and was dropped: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name there, its fields covering all three that move. That is #373's dormant-rule diagnosis doing its job, and #426 is the precedent for dropping rather than keeping a shadowed rule. -- 2026-08-25 #430 / #432 — one boundary, fixed as one: inside a comma segment a ONE-CHARACTER suffix word — the only kind a reader could take for an initial — is read by WHAT STANDS BEFORE IT. Behind another suffix it describes that suffix and continues the run, period included — `Smith, PSM I` and `Smith, PSM I.` are suffix 'PSM I'/'PSM I.' where they read given 'PSM' plus suffix 'I', an initial in that slot being no shape anyone writes. Behind a name word the period is decisive, marking an abbreviation and so name material — `Smith, John V.` is middle 'V.' where it read suffix 'V.', which is what `Smith, John B.` always did; the only thing that made V differ is that V is also suffix vocabulary. The bare form is the boundary and does not move: `Smith, John V` stays suffix 'V' (v1 parity, #144), which is why the test is the PERIOD and not the numeral. SCOPE, stated as the code reads rather than as the intent was argued: the class is one-character SUFFIX VOCABULARY, not "a roman numeral". It sweeps in `2` (`Smith, John 2.` is middle '2.'), misses `X` and `IX` (roman, but not vocabulary — `Smith, PSM X` is unchanged), and never touched `II`/`III`/`IV` at all, which are two characters and so were never initial-shaped and never vetoed. Anyone adding a one-character suffix in a script that has initials extends it. And the strict knob is consulted: under `lenient_comma_suffixes=False` the initial veto stands and the run ends at the numeral, which the first draft silently overrode by reading no policy at all. - #430's cause was rules.md#S2's initial veto in `is_suffix_piece`, which keeps a numeral out of a suffix reading — right after a name, wrong inside a credential run — so the segment did not look like a run at all and fell to the given-and-suffix walk. Two wrong answers came out of that one cause, and a fix verified on `Smith, PSM I` alone would have left the other: where the run's first word is TITLES vocabulary too (`Smith, MD I`, and `Smith, Jr. I` through the period-abbreviation inference) the leading-title peel took it instead, giving title 'MD' plus given 'I'. - THREE readers, one reading, which is the part worth copying. The gate that says a segment is a credential run, assign's router that assigns its roles, and group's one-entry join that renders it must agree PIECE FOR PIECE, and #429 shipped the inverse of its own fix by deriving that agreement twice. `segment_suffix_reading` returns the per-piece verdicts and all three read it (mechanisms.md#ONE-PREDICATE-PER-QUESTION); the intermediate state where only the gate and the router had been converted rendered `Smith, PSM I` as 'PSM, I', the render splitting a run whose roles were already right — the same tell as #429, one predicate along. TWO readers since 2026-09-06: #436/#437 took the render join out of group, and `segment_suffix_reading`'s remaining readers are assign's gate and assign's router, one call site between them. The predicate is not moved — one call site is no reason to relocate what two sites will contest again — and what is worth carrying away is unchanged, since the reason the three had to agree is the reason the two still do. - Declined, and the v1 suite caught it: asking `is_trailing_numeral_suffix` for #432's half. It answers the period question — `_ROMAN` is anchored and admits no period — but it also refuses a numeral standing behind an initial-shaped piece, which is a NO-COMMA rule and the opposite of this path's parity: `Chang, Andy C I` is first 'Andy', middle 'C', suffix 'I', and asking that predicate here made the numeral a middle. decisions.md's #401/#421 entry records the same fork declining to transfer to this walk under LENIENT. The half that applies is the period alone. - Blast radius, and the corpus IS blind to the moving classes: of the 1069 names on master, seven carry a one-character numeral behind a word inside a comma segment — `Chang, Andy C I`, `Doe, Rev. John V, Jr.`, `Maier, Amy I, Jr.`, `Maier, Amy Lauren I`, `Smith, John I`, `Smith, John V`, `Berg, abdul V` — every one of them BARE, and NONE moves. Zero pre-existing corpus names move at all; the only movers are the three this change adds as C1 examples. So the gate's 0-unexplained is evidence about the corpus, not about the change, and the case table is the whole coverage. `Berg, abdul V` cannot reach the walk at all: the bound-given join makes `abdul V` one piece, which is why the LENIENT mirror #401/#421 declined does not collide here. The moved names entered the corpora as C1's new examples, and carry ledger entries at all three baselines except #430's at 1.4.0, where `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name and a second rule would be dormant. - Two things the /simplify round settled by measurement rather than argument. The walk asks `is_suffix_piece(pieces[m - 1])` — a VOCABULARY test — where the reading carries the verdict it just recorded, and unifying them looks obviously right until you run it: the walk assigns `PSM` to GIVEN in `Smith, PSM I.`, so a reading-style flag says "behind a name" and sends the numeral to middle, losing the strict parity restored two commits earlier. The two sites ask different questions on purpose, and the vocabulary answer is the load-bearing one. The cost is a reading the walk cannot reach: `Smith, John PSM I I.` moves from middle 'I' + suffix 'PSM, I.' to middle 'I I.' + suffix 'PSM', two consecutive one-character numerals behind a credential — accepted, both readings being defensible for a shape nobody writes, and recorded here rather than chased at the price of two correct pins. - The walk is also strict-blind, pre-existing and unchanged: it calls is_suffix_lenient unconditionally, so `Smith, John V` reads suffix 'V' under `lenient_comma_suffixes=False` as under the default. The reading consults the knob and the walk does not, which is now conspicuous where it was merely unexamined. Not this change's to fix. - Not done, and recorded rather than left implicit: neither half emits an ambiguity, though the no-comma `John Smith V` reports `suffix-or-name` for the same fork. The comma path has never reported it (decisions.md#C1 records PARTICLE_OR_GIVEN suppressed there for a related reason), and adding an emitter needs a trigger, a stage-ownership entry and a case row — more than this fix, and #432 raises it as a secondary observation rather than a requirement. -- 2026-09-06 #436/#437 — the entry boundary is the comma the writer typed, not the shape of the segments. Two consecutive post-nominals are one entry iff they sit in the same comma bucket and nothing between them parts the run; `comma_bucket` is the function `segment` builds segments with, so "same part" here is an identity with segment's answer rather than a resemblance to it (mechanisms.md#ONE-PREDICATE-PER-QUESTION). What PARTS a run is a name word — given, middle or family — or a dropped delimiter core (#206). What does NOT part it is a token whose role renders into some other field altogether: the pass names all three of them beside itself as `_RENDERS_ELSEWHERE` — title, nickname and maiden — and such a token is not standing in the run at all, which is what keeps `Smith, MD Dr. PhD` one entry (#429's stickiness, now by construction) and `Smith, MD "Doc" PhD` and `Smith, MD (nee Jones) PhD` with it. Nor does a dropped token that is not a core part it: that is the bare maiden marker of `Smith, MD nee Jones PhD`, dropped with no role at all, so the role test cannot see it. Which is why the dropped arm reads the core SET rather than treating every dropped index as a boundary — a core is the one dropped token the writer typed AS a separator — and the set is `delimiter_cores` in the vocabulary layer, the same derivation off `Policy.extra_suffix_delimiters` that group's drop site reads, imported rather than repeated so the drop site and this pass cannot disagree about what a core is. That is this file's own ONE-PREDICATE-PER-QUESTION again, one level down from the readers #430 counted. FOUND, and it is the reason this is a fix rather than a change: R1 stated the rule in words on 2026-08-23 and the NO-COMMA path had violated it since the sentence was written — #429 added the sentence and recorded the violation without fixing it — its bullet above opens "Left alone, measured and pre-existing" and says of the no-comma path that it "is a separate question". `John Smith MD PhD` rendered suffix 'MD, PhD' at 1.4.0, 2.0.0, 2.1.0 and 2.2.0 alike, a comma nobody typed, and it is now R1's own example line. This bullet is R1's recorded history as well as C1's — `rules.md#R1`'s `history:` field points at `decisions.md#C1` and at nothing else, so the separator rule's provenance has always lived under the suffix-comma decision, and a reader arriving from R1 arrives here. - WHAT GROUP LOST, and it is the whole between-piece half: `tail` by index, #429's `segment_suffix_reading` verdict ORed into it, the per-piece gate, and the stickiness across an interleaved title. Three shape-derived answers to one question was the defect, and the review of #429 had already found that shape wrong in both directions at once. The stickiness needs no code now: `Smith, MD Dr. PhD` has no comma between MD and PhD and the title between them renders into another field, so they join, while `Smith Jr., Mr. Jr.` has the writer's own comma and they do not — both still pinned. What stays in group is core DROPPING keyed on `tail` through `seg_cores` (v1 `expand_suffix_delimiter` parity, #206, a TAIL rule) and the within-piece tag that heals the `Ph. D.` merge, which is role-blind and right for every role. `segment_suffix_reading` is left where it is with ONE call site, in assign; the #430 entry above records three readers, and the render join was the third. - AMENDED FROM THE SPEC TWICE, both times by measurement, and this is the half worth reading twice. The approved predicate parted an entry only at a DROPPED delimiter core. Measured 2026-09-06, the two corpus names the spec names as that class — `Smith, MD - PhD - FACS` and `John Doe, MD - PhD - FACS` — drop no core at all under the default policy the corpora compare under, the dash being no delimiter there: the dashes survive as ordinary given/middle words between PhD and FACS, and a rule reading `dropped` alone joins them where 'PhD, FACS' is pinned. `cases.py`'s `suffix_delimiter_detection` had the same shape pinned already. So the predicate asks what STANDS between the two rather than only what was dropped, and both halves are pinned, one row each, because a dropped-core test alone passes on the rule that broke the corpus. Reading the dropped arm against the core set also retires the spec's open question of how the pass tells a core from a maiden marker: no `WorkToken` tag and no new `ParseState` field, the set being the one group already derives. - THEN AMENDED AGAIN, in review of the FIRST DRAFT of this bundle's code commit — caught before that commit was rewritten, so the commit as it shipped already names all three transparent roles — and the second find is worth as much as the decision. That first draft's predicate made TITLE the only transparent role and treated every dropped index as a boundary, and it was wrong in both directions at once. Measured 2026-09-06 against the parent tree — parse the strings on a pristine checkout of the parent, then against a copy of the pass with `_RENDERS_ELSEWHERE` cut to the title alone and the dropped arm reading any drop as a boundary — three comma-path shapes that render 'MD PhD' at the parent went to 'MD, PhD': `Smith, MD "Doc" PhD`, `Smith, MD (nee Jones) PhD` and `Smith, MD nee Jones PhD`, a nickname, a parenthesised maiden clause and a bare maiden marker, one regression apiece. And the no-comma twin of the first, `John Smith MD "Doc" PhD`, rendered 'MD, PhD' at the parent and stayed there under that predicate — the very bug being fixed, left unfixed on a nickname-carrying shape. The differential witnessed none of the four, so the gate was green over all of it and the code review is what caught it; the general form is that a transparency written from the roles a corpus happens to hold reaches only the roles that corpus happens to hold. Four stage tests pin the four shapes now, and the pass renders 'MD PhD' for all four. - MEASURED 2026-09-06, before and after in the same worktree, the gate exiting 0 at every baseline with `unexplained: 0` and `radar unclassified: 0` throughout: intentional diffs 352 → 360 at 1.4.0, 247 → 254 at 2.0.0, 155 → 166 at 2.1.0 and 14 → 28 at 2.2.0, Latin-only 247 → 255, 151 → 158, 151 → 158 and 14 → 24. The corpus itself grows by exactly one name, the R1 example `John Smith MD PhD` entering `corpus_rules.jsonl` — 1113 → 1114 at 1.4.0 and 1120 → 1121 at the other three — and that is the +1 above the mover count at every baseline. Recipe: `tools/differential/compare.py --baseline B` at each of the four, whole output kept rather than the summary line; the movers themselves from the spec's Recompute block extended to read `state.dropped`, the token roles and the core set, which the bare comma-bucket version cannot do — it reports 15, the two delimiter-core names being the two it joins wrongly. Exactly ONE name changes RULE, at two baselines and no more: `abdul Smith Jr V` at 2.0.0 and 2.1.0, off `fix(#401) the bound-given reserve counts the trailing numeral assign reads as the suffix` and onto the compound rule, its widened diff no longer being a subset of `fix(#401)`'s `fields`. Checked by spying `compare.classify` and diffing the WHOLE (name → rule) map per baseline, not the gate's printed sections, which list each rule's first ten names and would hide a swap inside a larger one. Six new rules across the four ledgers at fourteen placements: the Latin alternation in all four, the bound-given compound at 2.0.0 and 2.1.0, and four one-name CJK rules at 2.1.0 and 2.2.0. The Latin alternation sits FIRST in each ledger on purpose — moved LAST in a scratch copy of the ledger with `compare._allowlist_for` redirected at the copy, the run reports three undeclared order-decided contests at 1.4.0, three at 2.0.0, three at 2.1.0 and none at 2.2.0. - VIEWS, over the 1117 corpus names — the 1116 that were there plus the R1 example: exactly 13 of the 1116 pre-existing names move, and the new example `John Smith MD PhD` makes 14; each of them moves in `suffix_list` and in the v2 dictionary and token-tag list and nowhere else. No other `*_list` view and no v2 role's token list moves on any name in the corpora. Recipe: for every corpus name dump the facade's eleven `*_list` properties plus `initials_list()` (a method, not a property) and, from `parse(name)`, `as_dict()`, `tokens_for(role)` for each role and every token's tags, before and after, and diff key by key. The tags are dumped beside the views because the change IS a tag change, and a view that renders the same over a token whose tags moved is what a view-only snapshot cannot see. Every mover goes from a comma-joined render to a space-joined one; none goes the other way, and none has a comma in the original between the two words. - RE-PINNED, nine `cases.py` rows and not the two the spec expected: `title_plus_one_word_two_suffixes`, `family_segment_multiple_suffixes`, `suffix_run`, `family_comma_segment_zero_is_not_the_run`, `family_comma_numeral_behind_a_suffix_is_not_an_initial` and the four CJK honorific rows. Eight assertions outside the case table moved with them, four of those in the v1-parity suite (`tests/test_capitalization.py`, `tests/test_suffixes.py`), and they are a deliberate v1 deviation rather than a parity break: v1 inserted a comma into a run the writer had spaced, its own comments calling that "not ideal but at least its in the right bucket", and the release-log bullet names the deviation. Four rules.md example lines outside R1 — P2's `John Smith Mc V`, P5's `abdul Smith Jr Ma`, C1's `Smith, John PhD I.` and W3's `田中さん 様.` — carried the comma too; all four names are differential corpus entries, the last of them in `corpus_cjk_tolerated.jsonl`, so the movement was observed by the gate and not only by the doc test that executes them. `family_comma_segment_zero_is_not_the_run` keeps its NAME and its point — segment 0 is the family segment even when wholly credential-shaped, which is what the mutation matrix found and what the row still guards — and only its separator moves. The comma-joined values the nine carried were the shape machinery's artifact and never a reading anyone chose. - ROUND-TRIPPING is stable on the shapes #429 fixed, which is what that entry recorded as the one load-bearing consequence of leaving this open: `str()` of a fixed parse is a no-comma string, and the no-comma path put the comma back. `test_a_suffix_run_survives_a_render_and_a_reparse` in tests/v2/test_render.py asserts `parse(str(parse(s))).suffix == parse(s).suffix` over the three writings. NOT stable in general, and the general claim is the one to resist — measured 2026-09-06 by evaluating that same equality over every corpus name, at the parent tree and here: 35 of 1116 names failed at the parent and 23 of 1117 fail here, thirteen fixed and ONE newly failing. The one is `Smith, John PhD I.`, and it is this same artifact seen from the other side: it round-tripped only because the render emitted 'John Smith PhD, I.' with a comma the writer never typed, which the no-comma path read back as a run. The render writes 'John Smith PhD I.' now, and that string reads middle 'Smith PhD', family 'I.' at the parent and here alike — a bare trailing initial behind a name is name material, #430's own boundary — so the two spellings genuinely disagree and the comma had been hiding it. The 23 that remain are the standing class: `str()` is a rendering, not a canonical form, and no rule in this file promises it re-parses. - NOT FIXED, and named because this change makes it reachable on more inputs: `revise(n, suffix=n.suffix)` is not the identity on a space-joined run. `Parser.revise` classifies each value by a full sub-parse of that value alone — its own docstring records the limit, the value being "classified ON ITS OWN" — and a bare 'MD PhD' carries no comma for the entry rule to route by, so the two words come back as two entries and the field re-renders 'MD, PhD'. Pre-existing, and the same shape as the `str()` limit above rather than a new one. Measured 2026-09-06 over every corpus name, comparing `revise(parse(n), suffix=parse(n).suffix).suffix` against `parse(n).suffix` at the parent tree and here: 24 of the parent's 1116 names fail and 38 of the 1117 here do, none of the parent's 24 recovering, and the 14 added are exactly the 14 movers — the 13 pre-existing names plus the R1 example. Not this change's to fix: the entry rule reads the commas in a whole name, and what `revise` hands its sub-parse is a field, so closing it means deciding what a field value's separators mean, which is a `revise` question. Answered, and FIXED, by #511 — the next bullet. - DECLINED, all four with the evidence: marking the boundary at the core-drop site (#437's MARK-DONT-STRIP shape) — `dropped` already holds the fact with its span, and a second recording of it is the duplication that mechanism exists to prevent, one level up; making the `"joined"` tag role-aware — within a piece it is role-blind and correct for every role, `Smith, Ph. D. Smith` giving `first_list == ['Ph. D.']`, and only the between-piece half was ever a suffix concept; a render-time span scan instead of the recorded tag — `_facade.__setstate__` and `ParsedName.replace()` synthesise span-less tokens, so an unpickled name has nothing to scan and the tag IS the entry structure a pickle carries; and fixing the no-comma path inside group's block as a third branch beside `tail` and `reading`, which is the shape #429 took. -- 2026-09-06 #511 — a suffix value handed to `revise()` derives its entries from its own commas, by the rule a whole name uses. `Parser.revise` sub-parsed each value and forced every harvested token to the named role AFTER the sub-parse had run R1's entry pass, which keys on Role.SUFFIX; a bare 'MD PhD' reads there as a title and a family name, so the pass joined nothing and the field rendered 'MD, PhD'. The fix runs the same pass again: `revise` now sub-parses to a ParseState, forces the role on every non-dropped token, and calls `suffix_entries` — the pass, lifted unedited out of post_rules' tail into a function the two callers share (mechanisms.md#ONE-PREDICATE-PER-QUESTION) — over the forced state, then assembles and harvests as before. So a comma in the value parts two credentials and a space joins them. TWO SPELLINGS of the lifted pass, and the reason is the call budget: a first draft had post_rules call the state-in/state-out wrapper, and the second ParseState build cost three more calls per parse against the band tests/v2/test_benchmark.py holds (py3.11, 2026-09-06, tools/perf/call_count.py: 450 calls/name before the move, 451 with the in-place worker `_mark_suffix_entries` that post_rules now calls, 454 with the draft; the facade band tops at 455.9), so the worker writes in place as every other post rule does and the wrapper exists for the one caller with no token list of its own. NO `_run` HELPER for the same reason: a draft routed `parse()` through a private state builder shared with `revise`, and that cost one frame per parse on the hot path (py3.11, 2026-09-06: 415 calls/name against 414 without it, in a 402-418 band), so the four-line construction is spelled twice and `parse()` is untouched. MEASURED 2026-09-06 over the 1117 distinct names in `tools/differential/corpus*.jsonl`, comparing `revise(p, suffix=p.suffix).suffix` against `p.suffix` under the default parser for every name with a non-empty suffix (368 of the 1117): 38 differed before and 1 after; the differential gate is byte-identical at all four baselines, `revise` not being on the compare path. RECOMPUTE: parse each corpus name, skip an empty suffix, revise the parse with its own suffix, count the names whose suffix moved (the script is in the #511 issue body). THE ONE LEFT is '김민준씨, J.씨', and it is not entry structure: the whole-name parse keeps 'J.씨' one glued suffix token, suffix '씨, J.씨', while the sub-parse of the bare value '씨, J.씨' peels the honorific off the initial, so the revised field renders '씨, J. 씨' — right entries, the spurious comma of before ('씨, J., 씨') gone, one word read differently by the value's own parse than by the whole name's. That is the "classified ON ITS OWN" limit `revise`'s docstring has always recorded, and CJK honorific peeling is a W-rule question this change does not move; pinned in `test_revise_reads_a_glued_honorific_on_its_own`. SUPERSEDES the phd-merge acceptance of 'Ph., D.' on this path (its bullet says how). STALE TAGS, tried and backed out: a draft cleared the sub-parse's own "joined" with the forcing and re-derived it, because the pass only ADDS the tag and `revise(n, family="Jones MD PhD")` carries the sub-parse's between-piece suffix mark on 'PhD' onto a FAMILY token. Measured 2026-09-06 against `330ee55`, the clear also destroyed every WITHIN-piece mark on a non-suffix value — 'D.' of `revise(n, family="John Ph. D. Smith")` lost the merge mark the #436 bullet's DECLINED list calls role-blind and correct for every role — while on a ParsedName only the suffix string view reads "joined" (`_text_for`'s suffix_join gate) — the facade's `_list_for` heals it for every role, but no path puts a revised name into a HumanName, the v1 setters going through `replace()`, so wiring those setters onto `revise` is the change that would show the stale mark, as `last_list == ['Jones', 'MD PhD']` — `initials()` and every field string being identical at both trees for every shape measured. So the tags are kept minus FOLDED_TAG as before; the between-piece mark on a forced non-suffix role is a tag-only oddity that predates this change and stays, and for a suffix value nothing depends on the sub-parse's marks, every pair it joined sharing a bucket with no parting token and the pass setting it again; pinned in `test_revise_keeps_the_sub_parses_within_piece_mark`. Dropped tokens keep their role and tags through the forcing, because the pass filters them by index and assemble omits them. ONE MORE LIMIT, pinned in `test_revise_leaves_a_policy_delimiter_unparted_without_a_tail_segment`, and it is the "classified ON ITS OWN" limit again rather than a rule of revise's: a delimiter the policy names through `extra_suffix_delimiters` is dropped, and so parts entries, only on a segment after a comma that the reading of the words makes a tail, and a value with no comma of its own has none — under `Policy(extra_suffix_delimiters=frozenset({" - "}))` the whole name 'Doe, John, MD PhD - FACS' renders 'MD PhD, FACS' while `revise(n, suffix="MD PhD - FACS")` renders 'MD PhD - FACS', the dash surviving as a token and the forced role making it a suffix word; measured 2026-09-06, a value whose own words read with a tail segment does part ('John Doe, MD - FACS' revises to 'John Doe, MD, FACS') and one after a suffix comma does not ('MD, PhD - FACS' stays), which is how the value's words read as a name deciding it. The round-trip is unaffected, the whole-name view having already rendered that boundary as a comma. DECLINED: a list-valued `revise(p, suffix=["MD PhD", "FACS"])`, which widens the API and leaves the string path where it was; documenting the limit with pins alone, the docstring already recording it and #511's measurement showing it reachable on every space-joined run #436 produced; and a text-only read of the delimiter cores inside `revise`, a second rule for a case no round-trip reaches. Out of scope and left as is: `ParsedName.replace(suffix="MD, PhD")` renders 'MD,, PhD', `replace` whitespace-splitting by contract and the facade setters riding on it for v1 parity. -- 2026-09-27 (Derek), #544 — THE NAME-WORD COUNT READS A RUN. The single-token rule this section records for the ambiguous class generalizes to a post-comma part of two or more words, every one suffix vocabulary or a class member (listed or by shape), at least one a member and none a single-letter roman numeral: behind two or more name words the part is the credential run, and the flip reports once over the whole part (`John Smith, Ed Ma`, `Jane Doe, MS LAc`). A title/suffix dual opening the part counts as suffix vocabulary there, and a run whose every member is listed and leans credential is left to the family-comma path, which already reads it whole. The forks and their measurements are the #544 entry under S2. -- 2026-09-28 (Derek), #544 — A PART READ WHOLLY AS SUFFIXES REPORTS NO NAME READING OF ITS WORDS. group's particle chain runs over every comma segment, and its two emitters — `particle-or-given` when a particle behind a word of both the title and the particle vocabulary chains (since 2.0.0, de264af1) and `suffix-or-name` when the chain takes an ambiguous acronym into the name (#289/#516, 59d8f38a, in no release) — reported inside a TAIL segment, which assign reads wholly as suffixes. Each such report named a reading the parse never made, against rules.md#A1's "A report names the reading the parse took". A family comma's tail was already silent, since group hands the chain no report list anywhere after a family comma; the suffix comma's tails were not. Measured on the released wheels: `John Smith, Jr., Freiherr von Richthofen` reports `particle-or-given` on 'von', a token in the suffix role, at 2.0.0, 2.1.0, 2.2.0 and 2.3.0; `John Smith, Jr., PhD van Ma` and `John Smith, Jr., PhD Do Ma` report it on 'van' and 'Do' at 2.0.0 and 2.1.0 only. The `suffix-or-name` half reached `John Smith, Jr., PhD Do Ma`, `John Smith, MA, PhD Do Ma` and `John Smith, Jr., PhD van Ma` on master (e10e83b4), and the run rule of the bullet above made it reachable behind ONE comma: `John Smith, PhD Do Ma` carried C1's flip and a second report on 'Ma'. FIXED by scope, not by a new test at the emitter: group passes the chain no report list in a tail segment either, so both emitters go quiet there together, and rules.md#C2 states the boundary for any part consumed wholly as suffixes. (Mechanism SUPERSEDED by #636, decisions.md#C1, 2026-10-11: group makes no chain in a tail segment at all, and the part after a suffix comma had reached none since #613, so the report list is withheld only after a family comma.) The maiden channel is a separate parameter and is untouched: a tail segment's reader is NONE, so the maiden walk reports nothing there to begin with. No mechanisms.md entry: this is AMBIGUITY-AT-THE-DECISION-SITE's own contract (a report fires only where the parse chose between live readings) applied to a stage whose reading a later stage overrides for the whole segment. MEASURED 2026-09-28, the tree against the same tree with `None if family_comma else ambiguities` restored in `group()` (the comparator), each parse recorded as its seven fields plus `(kind, [(token text, token role)])` per report, under all three name orders: 0 of the 1441 differential-corpus names move, so the gate has nothing to classify; over a comma grid — the prefixes `John Smith, `, `John Smith, Jr., ` and `Smith, John, ` times every run of one to three words drawn with repetition from {PhD, MA, Ma, Do, van, de, Jr, MEng, Ed, y, i}, each text as written, lowercased and uppercased, deduplicated to 11,049 texts — 360 parses (120 texts, every one of them mixed case) lose one `suffix-or-name` report apiece, every removed report on a token in the suffix role, 0 reports added, 0 field moves. 21 of the 120 texts carry one comma and are the run rule's reach; the other 99 carry two and moved the same way (297 parses) when the same one-line change was applied to master e10e83b4; the `Smith, John, ` prefix moves nothing, being a family comma. The grid holds no word of both the title and the particle vocabulary, so the `particle-or-given` half is witnessed by the case row alone. Pinned by the case rows `a_credential_run_after_the_comma_reports_no_chain_fork` and `a_part_past_the_second_reports_no_particle_fork`; the rows the chain still reports on outside a tail are `the_chain_reports_the_acronym_it_takes` and `titled_particle_chain_survives_a_title_that_is_also_a_particle`. -- 2026-10-01 (Derek), #562 (mechanism and accepted reports superseded by #613, below: the decision now binds the part before any chain runs, so the pair reports nothing) — A PARTICLE CHAIN UNSETTLES A RUN THE CAPITALS SETTLED, AND THE COUNT READS IT. The 2026-09-27 bullet above left a run whose every member is listed and leans credential to the family-comma path, "which already reads it whole". That promise fails wherever two particles stand side by side in the part: group chains them into one particle run (P2), assign reads the part as name text, and P6 attaches the chain to the family — `John Smith, PhD DO DO` read given 'PhD', family 'DO DO John Smith', and `John Smith, DO DO DO` given 'DO', middle 'DO DO'. rules.md#S2 already said the capitals do not decide a member chained behind another particle ("the run attaches whatever the capitals say (P6)"), so C1's shortcut was resting on a premise S2 denies. Of the two fixes #562 weighed, the one taken narrows the shortcut and leaves S2 as written: a part holding two particles side by side is read by the count, which two name words before the comma flip to the credential run, reported (`suffix-or-name`). The other — letting C1's evidence or S2's credential-in-front company outrank the chain — would have contradicted S2's sentence and P6's `Doe, John van DO` example, so it needed S2 amended rather than a gap filled. The test is ANY two adjacent particles, not a member behind one: `vd` is a particle and an unambiguous suffix word, so `John Smith, PhD vd DO` and `John Smith, MA vd vd` chained and misread the same way, the second with no member behind a particle at all. Segment runs before classify, so it asks classify's own predicate (`_normalize(text) in lexicon.particles`) and only while the run is still settled. The test does not ask whether the family-comma path would actually have misread the part, which it could not without reading ahead to group: where that path did read the part whole — for example a pair opening the part with an unambiguous particle-and-suffix word (`John Smith, vd DO`, `John Smith, VD DO`), a credential that is also a title in front (`John Smith, MD DO DO`), or a credential closed by a period in front (`John Smith, Esq. DO DO`, `John Smith, Jr. DO DO`), the list being by example rather than a census — the count flips the part to the same fields and reports the call, as every flip at this comma does (rules.md#C1's "A decision either way at this comma is reported"). Those reports are ACCEPTED (Derek, 2026-10-01: none of these is a name anyone would write on purpose, so a report is the right signal): with the capitals no longer settling the run, the call is the count's, and the report says so; `tests/v2/cases.py` pins `John Smith, vd DO`. 1.4.0 read every one of these names as the fix does; 2.0.0 and 2.1.0 read `John Smith, PhD DO DO` as title 'PhD', given 'DO DO', and 2.2.0 and 2.3.0 as the issue describes. MEASURED 2026-10-01 against master 0eadedeb, py3.11, `nameparser.__file__` asserted on each side, each parse compared as its seven fields plus its sorted ambiguity kinds: 0 of the 1453 differential-corpus names move (the two names this change adds are the gate's only movers, under a new fix(#562) rule in the four 2.x ledgers); over tests/v2/test_properties.py's settled grid (5,580 texts) 15 move, 10 of them role moves, and the other 5 (`John Smith, MD DO DO`, `MS`, `Esq.`, `Sr`, `Ms` in front) keep their fields and gain the flip's report; over a wider grid — the prefixes `John Smith, `, `Smith, ` and `Doe, John ` times every run of one to three words drawn with repetition from {PhD, MA, Ma, DO, Do, do, vd, van, Jr, MD, Ms}, 4,389 texts — 30 move, 17 of them role moves, every mover a `John Smith, ` text now reading given 'John', family 'Smith' and the whole part as suffix, and every one reporting `suffix-or-name`. Recompute: check out the parent into a separate worktree, parse each grid in both trees under `PYTHONSAFEPATH=1` with the tree's root first on `sys.path`, and diff. The settled grid's own pin moves with it: tests/v2/test_properties.py's `_SETTLED_COUNT` reads 2,994 where it read 3,024, the ten `_SETTLED_EXCEPTIONS` that pinned #562 are gone, and its two recorded negative controls read 1,142 and 215 — the second had already moved from 751 to 215 with #563, before this change. LEFT OPEN, as #562 asked: `Smith, PhD DO DO` (ONE name word before the comma, so the count keeps the listing form, and P6 attaches the chain: given 'PhD', family 'DO DO Smith'), where no rule states S2's credential-in-front company against a chain; and `John Smith, PhD van der`, whose particles are no members and never reach the run test, reading given 'PhD', family 'van der John Smith' as before. -- 2026-10-01 (Derek) — A CAPITALS-SETTLED RUN OPENED BY A WORD OF THE CLASS REPORTS, AND RULES.MD#C1 NOW SAYS SO. C1 said a run whose every class word is written in capitals "reads whole in silence". That held only behind another credential: a class word OPENING the part is the first word after the comma, whose decision C1 reports either way, so `Doe, MA PhD`, `John Smith, MA MA` and `Smith, MA PhD` read wholly as suffixes and report `suffix-or-name` once, on that word, while `John Smith, PhD MA` and `Smith, PhD MA` are silent. The behavior predates #562 and is kept: nobody repeats `MA` at the end of their name on purpose, so the report is the right signal. Statement corrected, no parse moved. MEASURED 2026-10-01 on master b39c370c, the prefixes `John Smith, ` and `Smith, ` times every run of two or three words drawn with repetition from {MA, BA, ED, DO, JD, PhD, MD, Jr, Esq.} holding at least one class word: all 900 runs opening with a class word report, 895 of them read whole (14 of those hold a #562 particle pair, whose count flip reports over the whole part rather than on the first word) and the other five being `Smith, MA DO DO` and its like, the one-name-word chain #562 left open; of the 560 opening with another credential, 554 read whole in silence, and the other six hold a #562 particle pair — four reading whole and reporting (`John Smith, PhD DO DO`), two being `Smith, PhD DO DO` and `Smith, Jr DO DO`. Recompute: parse that grid and bucket by whether the first word is a class word, whether `ambiguities` is empty, and whether the suffix is the whole part. The same day (Derek) rules.md#S2's first-slot precedence sentence gained the pointer to that exception, so the next reader does not rediscover it: S2 says the count decides the first slot after a family comma before case, true for one word (`John Smith, MA` flips by the count) and false for a capitals-settled run, where C1's shortcut reads the capitals first and the comma keeps its family reading. And S2's list of reporting slots lost "the segments beyond it": no `suffix-or-name` report lands past the second comma (`Smith, John, MA` and `John Smith, Jr., MA` are silent). It once did: at #530's merge cc78c960 `John Smith, Jr., PhD Do Ma` reported `suffix-or-name` on 'Ma' beside its `comma-structure` flag, from group's chain emitter, which the 2026-09-28 #544 bullet above silenced in a tail. S2 now points at C2 for what such a part does report, which is not only `comma-structure` — an unclosed delimiter there adds `unbalanced-delimiter` (`John Smith, Jr., (Bob`), and a maiden clause standing in it is read as one (`Smith, John, Jr nee Jones MA`, maiden 'Jones MA'). Derek, 2026-10-01: the silence is right. -- 2026-10-01 (Derek), #554 — A WORD OF BOTH THE PARTICLE AND THE SUFFIX VOCABULARY COUNTS IN THE RUN, AND STILL SPEAKS FOR NOTHING. After #544 the run test (`_segment.py`) counts `vd` and `mc` as suffix words, while S2's company (`_pieces._anchors`) refuses to let them speak for the word behind them, an exclusion added within PR #552 after that PR's own anchor pass had brought back a split of `Smith vd Ma, John` into family 'Smith Ma', suffix 'vd'. That split is not only an intermediate state of #552: 2.2.0 and 2.3.0 shipped it (2.0.0 and 2.1.0 read family 'Smith vd Ma'), #530 removed it in this cycle, and #552 kept it removed. The issue framed that as two answers to one question and offered (b): end the run at such a particle, so that one "has a live name reading" predicate decides both. DECIDED (a), the shipped behavior, and no parse moves. They are two questions. Run membership asks what the word IS, and `vd` is unambiguous suffix vocabulary; company asks whether the word SPEAKS FOR the one behind it, and a particle cannot, P2 joining it forward to exactly that word. Both answers follow from the word's vocabulary, and they differ by design rather than by drift; S2's "end the run" is the company's run, not the part C1 counts. The clause reaches exactly `vd` and `mc`: `do`, also in both vocabularies, is in the ambiguous half and so is already a word of C1's class, reporting where these two do not (`John Smith, Jr do` against `John Smith, Jr vd`). Recompute the set: `L = Parser().lexicon; L.particles & (L.suffix_acronyms | L.suffix_words) - L.suffix_acronyms_ambiguous`, and drop the subtraction to see `do` with them. - The evidence is the writing that carries no other signal. Neither reading of `vd Ma` after a comma is realistic, a particle in front of a Chinese family name being as rare as `vd` or `mc` as a credential, so the mixed-case writing is an arbitrary edge case and decides nothing. Written in one case, where S2 has no capitals to read and the vocabulary is all there is, the part reads as the credential run, and that is the reading a reader would take: measured 2026-10-01 on master (fc682e36), `JOHN SMITH, VD MA` and `john smith, vd ma` read given 'JOHN'/'john', family 'SMITH'/'smith', suffix 'VD MA'/'vd ma' and report `suffix-or-name`, as `John Smith, vd Ma` does, and `John Smith, mc Ma` reads the same in every casing. It is also 1.4.0's reading, verified against the released wheel the same day: 1.4.0 read suffix 'vd Ma' and 'VD MA', and 2.0.0 through 2.3.0 read the whole name as the family. (b) would add a particle exception to C1, and to #563's paired-initials test besides, in order to give `JOHN SMITH, VD MA` family 'JOHN SMITH VD MA' with no given name: the worse reading, on exactly the input where nothing else speaks. - #563's site needs nothing of its own. `García Márquez, vd G.J.` reads as `García Márquez, PhD G.J.` does, in every casing: given 'García', family 'Márquez', suffix 'vd G.J.', and neither reports. If a surname split by a credential run in silence is a defect, it is the run's, not the particle's. Open: #573 — uniform-case `vd`/`mc` before a comma and after a family comma (decided 2026-10-02, the P6 section's entry), and the silent mixed-case `Doe, Jane PhD vd Ma` (now #578). Recompute: `parse(s)` with `as_dict()` and `[a.kind.value for a in parse(s).ambiguities]` over each string above and its `.upper()` and `.lower()`. -- 2026-10-01 (Derek), #575 (where the count runs superseded by #613, below) — THE COUNT BEFORE THE COMMA COUNTS A PARTICLE SURNAME ONCE. Every count rules.md#C1 makes of the part before the comma — v1's "more than one word" for an unambiguous credential, the ambiguous class's name-word count (#289, #544), and assign's two-name-word test for the positional read after a comma followed by no name word (#296/#325) — counted tokens or pieces, so `De La Cruz` was three words and `van der Berg` two (the ambiguous `van` stands as its own piece, P1's fork). This cycle's count had therefore read `De La Cruz, Ed` as a credential comma with no given name and split `van der Berg, MA` into given 'van', family 'der Berg' — an unreleased regression against 2.3.0, which read both as the listing form — and 1.4.0 through 2.3.0 all split `van der Berg, PhD`. DECIDED: a particle run and the one name word it attaches to are one word in all three counts, so a particle surname reads as a one-word surname does. - THE COMMA SETTLES P1's FORK, and that is the decision rather than a side effect. Standing alone, `Van Buren` is given 'Van', family 'Buren' with a `particle-or-given` report, so counting it as one surname makes the comma's reading disagree with the standalone one. Derek's call, after weighing the narrow alternative (count only a part P1 already reads as all surname, i.e. one led by a never-given particle like `de`): the listing form puts a surname before the comma, so the comma is evidence the standalone parse does not have, and the readings it gives are the ones a person takes — `Van Buren, Ed` given 'Ed', family 'Van Buren'; `van der Berg, MA` family 'van der Berg', suffix 'MA' (one name word, so C1 reads the case, and capitals in a mixed-case name make the credential). `Van Johnson, Dr.` moves with it, family 'Van Johnson', title 'Dr.', where 2.2.0 and 2.3.0 read given 'Van' and `tests/v2/pipeline/test_assign.py` had pinned the positional fork; that test now uses `Van Johnson Smith, Dr.`, which still has two name words. - THE PARTICLE REACHES ONE WORD, as P1's fold does, not to the end of the part as P2's chain does: `de Mesnil Jean, Dr.` under a family-first order is family 'de Mesnil', given 'Jean', two name words. The cost, and it is real for an ambiguous particle: a `van`-led part with a word after the surname counts two and keeps the positional read with P1's fork, so `van Buren John, Ed` reads given 'van', middle 'Buren', family 'John' and `van der Berg Smith, PhD` given 'van', family 'der Berg Smith', both unchanged from master. Under the default order a never-given particle loses nothing (P1's fold makes `de Mesnil Jean` all surname anyway). `_vocab.unit_ends` carries both reaches behind a `chain` flag, post_rules' fold reading the full chain as before. - ONLY PARTICLE CHAINS, not mechanisms.md#UNIT-PARTITION's full set — stated there as the Contract's one exception, and in P3's statement, whose "ONE name word wherever another rule counts them" now names C1's count as the exception. A bound given-name pair builds a GIVEN name, and P5 gives up a family word where the name has no other (`abdul Salam` alone is given 'abdul', family 'Salam'), so `abdul Salam, Ed` keeps the credential reading. Whether P3 joins a single-letter connective depends on the words of the whole name — `Carod i Rovira, Josep` joins as four words where `Ortega y Gasset` alone, three, does not — and the count is part of deciding what the whole name is, so segment could reach P3's answer only by copying P3; `Ortega y Gasset, Ed` keeps the credential reading. Approved as "particle chains and connective joins", narrowed to particles when P3's exception surfaced in implementation (the first draft joined `John e Smith, III` into one family name, caught by `tests/test_conjunctions.py`). - A WORD THAT IS ALSO A TITLE IS NO PARTICLE WHERE IT OPENS THE PART. `TITLES ∩ particles` is `{freiherr, st}` and `bound_given_names ∩ particles` is `{abu, أبو, ابو}` (recompute: `L = Parser().lexicon; L.particles & L.titles`, `L.particles & L.bound_given_names`). The first draft counted every particle, and the docs review found it reading `Freiherr von Berg, PhD` as family 'Freiherr von Berg' and `St John, PhD` as family 'St John', where every release reads the title. The first fix then excluded both overlaps in EVERY position, and the review of that fix found two errors in it. Position: inside a surname a title-particle chains as P1 and P2 chain it, and the blanket exclusion brought #575's own defect back for `de St Pierre, Ed`, `De St. Croix, Ed` and `de Abu Bakar, Ed` (family only, no given name, where 2.3.0 read given 'Ed'). And direction: 2.0 through 2.3 read `Abu Bakar, Ed` as given 'Ed', family 'Abu Bakar', so excluding the bound-given particle gave released behavior up rather than keeping it — and Abu Bakar is a common Malay surname. DECIDED: a title-particle is no particle only where it LEADS the part, and a bound given-name particle is a particle. So `Freiherr von Berg, PhD` and `St John, PhD` keep the title, `de St Pierre, Ed` reads given 'Ed', and `Abu Bakar, Ed` reads given 'Ed', family 'Abu Bakar' while `Abu Bakar, PhD` reads family 'Abu Bakar', suffix 'PhD' where 1.4.0 through 2.3.0 read given 'Abu'. The agreement test compares each token in both positions; it cannot see which position a caller passes, so case rows pin each count's flag: `de St Pierre, Ed` segment's, and `St John, Dr.` and `von St Johann, PhD` assign's, one per direction (the review of this round found assign's flag fixed to either constant passing every test, and both mutants now fail). `Freiherr von Berg, Ed` reads family 'Freiherr von Berg', given 'Ed' — 2.0.0's through 2.3.0's reading too, so no change against a release; only this cycle's count had read title 'Freiherr', family 'von Berg', suffix 'Ed'. The surname counts once and the title is no name word, so the count reads the listing form, which keeps a leading title in the family as it already did for `Prof. Cruz, Ed` (superseded for listed titles by #H6, 2026-10-17: the count is unchanged, and the listed title then peels). For `Prof.` that is a cost (rules.md#C1's Accepted line); for a German rank it is the right reading (Derek, 2026-10-01): since 1919 a former noble title is part of the legal surname, written between the given name and the particle (Karl-Theodor Freiherr von und zu Guttenberg), so `Freiherr von Berg` is the family name. That is also why `freiherr` is in PARTICLES at all — added with the German prefixes in 3e14ea20 (#18) without a stated reason, and the reason is this one. `Freiherr von Berg, PhD` keeps title 'Freiherr', the unambiguous credential's count taking the title as a word; both are readings of one name, and neither is changed here. A title in front is otherwise outside the fix: v1's count for an unambiguous credential counts the title as a word, so `Dr. van der Berg, PhD` reads as `Dr. van der Berg` does alone, given 'van'. - `De La Cruz, M.J. K.L.` reads given 'M.J.', suffix 'K.L.', reporting twice, as `Cruz, M.J. K.L.` does (Derek, 2026-10-01): with one name word before the comma the paired-initials count no longer reaches it, and C1's two-dotted-groups sentence now says "behind two or more name words". #563 had read it as a flipped credential run with no given name. - SEGMENT RUNS BEFORE CLASSIFY, so its count builds the two facts it reads (particle, suffix) from the vocabulary (`_vocab.surname_unit_tags`), while assign derives them from classify's tags (`_vocab.surname_unit_facts`) over every token of the part, so a suffix word stops a particle in both: the first draft's assign count dropped suffix pieces before walking and read `van Jr. Berg, Mr.` as family 'van Berg' (the code review). `test_classify.test_surname_unit_tags_agree_with_classify` sweeps every single-word vocabulary entry in three casings; its first run caught `JD.CPA` and `Msc.Ed.`, which classify tags as suffixes through the period-joined derivation, now mirrored, and that pair is the recorded negative control (`_SURNAME_UNIT_CONTROL`, asserted with the mirror patched out). - BLAST RADIUS, measured 2026-10-01: the gate exits 0 at all five baselines, and of the 1459 names in master's corpora exactly one moves, `De La Cruz, M.J. K.L.` (above, decided). Comparator: `parse(n).as_dict()`, the ambiguity kinds and `initials()` for every name in every `tools/differential/corpus*.jsonl` at `git archive origin/master`, under all three name orders, on master's tree against this one. The population that could move is small and the corpus is evidence about itself, not the rule: two corpus names have a part before the comma that is one particle surname of several words, with the part after the comma holding a credential — that one and `De La Cruz, M.J. PhD`, which keeps given 'M.J.'. Recompute: corpus names whose part before the first comma has more than one token and `_vocab.surname_unit_count` 1 (24 at master, nearly all `de la Vega, Juan`-type listings), then keep those whose second segment holds a word `_vocab.ambiguous_class_candidate` admits (a listed or dotted member of the credential class) — 2 at master. A filter on any suffix word keeps 17, the `de la Vega, Juan … III` listings among them. Against 2.3.0, `fix(#575)` classifies one name, `van der Berg, PhD`; the other particle-surname example names diff there only by this cycle's #289 count and report. Outside the corpora the move against 2.3.0 is a class, not a list: a leading ambiguous particle and one word, before a comma followed by an unambiguous credential or a title alone, now reads as one surname (`Abu Bakar, PhD`, `bin Laden, PhD`, `Mac Donald, PhD`, `van Gogh, Jr.`, `Van Johnson, Dr.`), where 2.3.0 read the particle as the given name; `Freiherr von Berg, Ed` and `Abu Bakar, Ed` move only against this cycle's master. The boundary case rows added in review carry no shape tag on purpose: their diffs against the older baselines come from earlier changes (#296's positional read, #289's count), so admitting them to the contract corpus would have stretched unrelated ledger rules over them; the case table asserts them either way. - COST, measured 2026-10-01 by `sys.setprofile` call counts (mean of 50, after one warm-up) against `git archive origin/master`: `Smith, John` 183 → 183, `John Smith, MA` 252 → 252, `John Smith, PhD` 209 → 217, `Dr. Juan de la Vega III` (the benchmark reference) 366 → 366. The +8 is the unambiguous credential path building its token list and asking each token whether it is a particle; a part with no particle never builds the units, and a frame-free approximation of `_normalize` was declined as a second spelling of it. -- 2026-10-01 (Derek), #564 — AN UNLISTED ALL-CAPS WORD IS A CREDENTIAL IN THE PART AFTER A COMMA BEHIND TWO NAME WORDS, BY DEFAULT. The decision, its two-letter rule and the mixed-run membership are recorded under decisions.md#S2 (#564), where the caps shape's switch has always been decided; this bullet is C1's pointer to it. -- 2026-10-03 (Derek), #549 — A DECLARED DELIMITER PARTS A TRAILING SUFFIX PART AS A COMMA WOULD, AND THIS REVERSES #538'S SKIP READING (M2, 2026-09-26). The question #549 asked was whether a core a connective join swallows should be skipped (the text read as if the core were absent) or should end the run it stands in; Derek's ground for the second was the one a caller has when declaring one: a delimiter set to ` - ` is meant to behave like a comma around suffixes, and the simplest parser that honours that is one where it IS a comma there. So `group` cuts a tail segment at its cores before grouping anything, groups each part as the comma twin's own segment would be grouped, and drops the cores, which post_rules' entry pass still reads back as boundaries. No join, maiden walk or link search can reach across a core because none of them ever sees one, and the machinery that let them step over it is gone: the `cores` parameter of `_group_segment`, `_maiden_take`, `_run_neighbours` and `_link_joins_inside_the_clause`, `_maiden_take`'s `seen` index list, the `skip` arguments of `peel_walk` and `trailing_start`, and the #206 drop block that ran after the joins. One call per parse fewer (`tools/perf/call_count.py`, py3.11, 2026-10-03: parse 369 against 370 at 061f02da). - WHAT MOVES, against 1.4.0 and against 2.3.0. A connective beside a core no longer joins across it: `HumanName("Smith, John, PhD - and MD", suffix_delimiter=" - ").suffix` is 'PhD, and MD', and `... Puig - y Soler` 'Puig, y Soler' -- 1.4.0's answers (its `expand_suffix_delimiter` split the part's text before anything joined), where 2.0.0, 2.1.0, 2.2.0 and 2.3.0 each gave 'PhD - and MD' and 'Puig - y Soler' (all measured 2026-10-03 with the released wheels). The generational link moves back too: 2.3.0 read `Smith, John, Puig - i Soler` as 'Puig, i Soler' and `Smith, John, PhD née Puig - i Soler` as maiden 'Puig', suffix 'PhD, i Soler', `i` being no connective there, and #397/#538 had moved them to 'Puig - i Soler' and maiden 'Puig i Soler' before either shipped; both read as 2.3.0 did again. An interior core still parts: `... PhD née Puig Dr. i - y Soler` gives suffix 'PhD i, y Soler'. A MAIDEN MARKER BESIDE A CORE TAKES NO CLAUSE, and this is the larger cost by count: with the core straight before the marker (`Smith, John, MD - née Jones Smith`) or straight after it (`Smith, John, PhD née - Jones`), the comma twin's marker has no name word ahead of it in its part, or nothing behind it, and M2 declines, so the parse reads suffix 'MD, née Jones Smith' and 'PhD née, Jones' with no maiden, where 2.0.0 through 2.3.0 read maiden 'Jones Smith' and 'Jones' (measured 2026-10-03; 1.4.0 had no maiden markers and gave the suffix reading). It is the one rules.md#C1 records as Accepted. At the default policy nothing moves, `extra_suffix_delimiters` being empty: the gate exits 0 at all five baselines. - THE DECLINED READING, TAKEN. #538 declined the boundary reading because it moves `Smith, John, PhD née Puig - i Soler` from maiden 'Puig i Soler' to maiden 'Puig' with suffix 'PhD, i Soler', "a birth-name link pushed into the credentials". That is what the same text with a comma typed in the core's place reads, and it is what 2.3.0 read. The marker beside a core (WHAT MOVES, above) is the same trade at a higher price, a maiden name lost rather than a link moved, and it is taken on the same ground; the review of this change found it, the first draft of this entry having weighed only the link shape. The comma reading is accepted as the cost of one rule for a declared separator, where the skip reading kept two (a lone core dropped like a comma, a core beside a connective stepped over like a word) and #549 was the seam between them. - INVARIANT, pinned by `tests/v2/test_properties.py::test_a_delimiter_core_in_a_tail_reads_as_its_comma_twin`: every ROLE field of a name with a ` - ` core in a part after the suffix comma equals that of the same text with a comma in its place, over 129 generated texts; 102 disagreed at 061f02da, 0 after. It replaces #538's `test_a_delimiter_core_reads_as_if_it_were_not_written`, whose invariant this decision reverses. Scoped to TRAILING parts on purpose: before the first comma, or in the given part of the listing form, the core is a word (C1's Accepted entry, v1 parity), and a comma typed there changes the name's structure rather than parting a suffix run, so the twin is a different parse. The ambiguity reports are not part of it: C2 reports `comma-structure` once per extra comma part, so a typed comma can add a report the core does not (`Smith, John, Puig - Puig` against `Smith, John, Puig, Puig`). - MEASURED 2026-10-03, branch against 061f02da. POPULATION: each head in {`Smith, John,`, `Smith, John, Jr.,`, `John Smith,`, `John Smith`} followed by every 2-, 3- and 4-word product of {PhD, MD, Puig, i, y, -, Jr., Soler, Mr., née, and} that contains `-`, 19,972 texts, plus the 1,505 distinct names of `tools/differential/corpus*.jsonl` at 061f02da. Under `extra_suffix_delimiters=(" - ",)` 2,624 of the generated parses move (suffix alone 2,384, suffix and maiden 240 -- every one of the 240 a maiden name lost to a marker beside a core: 100 with `née -`, 108 with `- née`, 32 with both); under `(" / ",)`, with every `-` word rewritten to `/`, the same 2,624 (2,384 and 240), the move being a property of the core's position rather than its spelling; and under `(" and ",)` over the texts as written, where `and` is now the core and the dash a word, 2,528 (2,468 and 60). No other field and no ambiguity report moves under any of the three. In the corpus, one name moves under each of two policies: `Smith, John, PhD née Puig - i Soler` under ` - ` (above), and `Doe, Jane, and Jr.` under ` and `, suffix 'and Jr.' to 'Jr.', a leading core dropped as any lone core is. COMMA-TWIN AGREEMENT over the generated texts under ` - `, comparing the seven role fields of each against the text with ` - ` replaced by `, ` under the default policy and keeping only texts whose core is neither first nor last after the head nor beside another core: `Smith, John,` 752 of 2,100 disagreed before and 0 after, `Smith, John, Jr.,` 752 of 2,100 and 0; `John Smith,` 1,956 of 2,100 both before and after, the head where the typed comma moves the name's structure. (A first draft counted 973 of 3,310 for the `Jr.,` head, its filter reading `Jr.,` as part of the body; the review of this change caught it.) RECOMPUTE: parse each text under each policy at both trees (pin `nameparser.__file__` on each side, AGENTS.md's two-tree gotcha), compare the seven role fields and the sorted ambiguity kinds, and count the texts whose record differs; for the twin, compare the role fields alone. - SUPERSEDES, under M2: the 2026-09-26 #538 entry (its stepping, its invariant and its Not-repaired list, which this closes); the 2026-09-22 #397 follow-up's bound argument (a core can no longer stand between a marker and the clause's first word, the segment being cut at it first); and the #418 reorder entry's closing repair, which screened a tail's cores out of the marker pass so that `Smith, John, PhD née - Jones` read maiden 'Jones' -- it reads suffix 'PhD née, Jones' now, as above. Entries there that describe the walk stepping over a tail's cores (the #424 and #533 bullets) describe machinery this change deletes; they are dated and left as written. Not changed: a core OUTSIDE a tail segment is still a word (C1's Accepted entry), and a core inside a token (`RN/CRNA` under `/`) still reads by `splits_into_suffixes` and keeps the token whole. - -- 2026-10-04 (Derek), #603 (decided at group's head since #613, below) — A CREDENTIAL OPENING THE PART AFTER THE COMMA MAKES IT THE POSTNOMINAL PART, WHATEVER FOLLOWS; AND A TITLE WORD PAST THE SECOND COMMA IS A TITLE. One word that is neither suffix nor title vocabulary used to flip the whole comma reading: `John Smith, PhD Jones` read family 'John Smith', given 'PhD', where `John Smith, MD FACS` and `Eric H. Holder, Jr. Attorney General` were already read as a name and its postnominals. Now the first suffix word of the part, with only titles in front of it, opens the part when it starts S2's run (`_pieces.starts_a_credential_run`, #602's own predicate, so the two runs cannot disagree about what starts one), and every word after it reads as that run reads: a title word as a title, any other as a suffix. The mechanism is the one #296 built for a part holding no name word, extended rather than added beside: `segment` still decides FAMILY_COMMA, and `segment_suffix_reading` returns a reading instead of None, so assign's no-name path reads the part before the comma positionally where it holds two name words. Every word the opened part takes that would otherwise have made it the given part is reported (`suffix-or-name`), as #602's run reports what it absorbs. - THE ONE-WORD PATH, DECIDED (Derek): the whole part is suffixes. `Doe, PhD Jones` reads family 'Doe', suffix 'PhD Jones', no given name, as `Smith, PhD` already reads. DECLINED: reading the credential as a suffix and the next name word as the given name (`Smith, PhD Jones` → given 'Jones'), proposed by analogy with a title at the head of the given part. Asked whether that is a real format, it is not -- nobody writes `Family, Credential Given` on purpose -- and the one realistic input it served, a misplaced generation (`Smith, Jr. John` for `Smith Jr., John`), is garbage in, garbage out. One reading for both comma paths was the deciding ground: the one-word path disagreeing with the full-name path is the shape #429, #430 and #432 each found. Accepted cost: `Smith, Jr. John` loses 'John' to the suffix, where 2.3 read given 'John' with 'Jr.' misfiled as a title. - A TITLE/SUFFIX DUAL OPENS NOTHING (Derek). A word in both vocabularies (`md`, `ms`, `sr`, the ranks `lt`, `cpl` and the rest) is the listing form's title in front of a given name, `Smith, Ms Jane`, and a credential behind it opens nothing either (`Smith, Ms PhD Jones` keeps given 'PhD', as `Smith, MD PhD Ma` keeps given 'Ma' under #544). DECLINED after building it: a dual opening the part behind two or more words before the comma. Derek first chose it for `John Smith, MD Jones`; the reviews found what the count cannot see -- a surname of two words (`García Márquez, Ms Gabriela`, `Smith Jones, Sr. Maria`), a title in front of a one-word surname (`Dr. Smith, Ms Jane`, the count taking the title as a word), and spaced initials (`García Márquez, Ms G. J.`) each lost the given name -- and with the word explained (a dual is a word in both lists, decided by position), Derek chose that duals never open. So `John Smith, MD Jones` and the `MD - PhD - FACS` delimiter rows keep their 2.3 reading. An ambiguous-class member opens nothing either (`John Smith, Ma Jones` keeps given 'Ma'), nor does a suffix word S2's run start refuses (`John Smith, vd Jones`). - STRICT MODE: the veto refuses an initial-shaped word the suffix READING, not the run. `Smith, PSM I.` under `lenient_comma_suffixes=False` reads suffix 'PSM I.' (2.3: given 'PSM', suffix 'I.'), reporting the absorbed 'I.', which matches #602's no-comma run (`John Smith PhD V.` reports; its given-part run does not, a disagreement #602 left and this does not touch). - THE SPELLING GAP THIS FOUND, FIXED HERE. `starts_a_credential_run` refused any piece longer than one token, so the split credential `Ph. D.` group merges started neither #602's run nor this one while `PhD` started both: `Smith, John Ph. D. Jones` read middle 'Jones' where `Smith, John PhD Jones` read suffix 'PhD Jones'. The merged piece now starts the run, the "suffix" piece tag being the Ph./D. merge's alone; `trailing_candidates` had already admitted it, so its pre-check and the predicate now agree. The no-comma spelling `John Smith Ph. D. Jones` still reads family 'Jones': the trailing peel's walk never holds the merged piece, which is a separate path and is left (AGENTS.md's spelling sweep). - C2, WIDENED (Derek): a word of the title vocabulary in a part past the second comma that is not also suffix vocabulary is a title -- the part stands behind a comma, the line H5 draws for reading a title from the end of a name -- and a piece holding a suffix word stays a suffix, a dual and a connective join that took one (`PhD - and MD`) alike. `Eric H. Holder, Jr., Attorney General` reads title 'Attorney General', and a part of title and suffix words is recognized rather than flagged. The flag's test is asked word by word against the title vocabulary, a period-joined word through `period_joined_vocab` as classify tags it (`John Smith, Jr., Lt.Gov.`, which the code review found still flagged), so `Secretary of State` reads its title but keeps `comma-structure`: knowing that group's title chain takes the connective would be `segment` modelling `group`, the sign #611 is about. Every release through 2.3.0 read these as suffixes, 1.4.0 included (`Eric H. Holder, Jr., Attorney General` suffix 'Jr., Attorney General', `Smith, John, Prof.` suffix 'Prof.', measured with the released wheels 2026-10-04); a first draft of this entry said 1.4.0 read them as titles, inferring it from the 1.4.0 run reporting no unexplained diff, when the reason was the broad `fix(comma-family)` rule there claiming them, which #603's two rules now precede by declaration. Accepted consequence: a noble rank there reads as a title, `John Smith, Jr., Freiherr von Richthofen` → title 'Freiherr', suffix 'Jr., von Richthofen'. - WHAT MOVES, measured 2026-10-04 against master at 97d36e0d, after the dual decision above. The differential corpora at 97d36e0d (1,519 name and order entries over 1,515 distinct names): four names, every one intended -- the dash-delimited `Steven Hardman, RN - CRNA` now reads given 'Steven', family 'Hardman', suffix 'RN - CRNA', and `John, Smith, Dr.`, `Andrew Perkins, Jr., Col. (Ret)` and `1 & 2, 3 4 5, Mr.` read their tail title. A generated grid: every name of `tools/differential/corpus*.jsonl` at 97d36e0d, bare-string and object lines alike, plus 120,000 one-to-seven-word texts drawn with `random.Random(601)` from the 54 words {Jane, Doe, Smith, John, nee, née, geb., z domu, PhD, MA, Ma, MD, Jr., Jr, III, V, VI, i, y, de, de la, van, vd, mc, do, Do, DO, abd, abdul, Dr., Prof., King., Le, Attorney, General, Jones, `,`, ba, R.A.I., (nee, Smith), Ó, binti, Ph., D., Chief, Justice, Ms, Sr., RN, -, G.J., MJ, Col.}, plus each of the heads {Jane Doe, Doe,, Smith, John, John Smith, Dr., Mai Le, Jane van der Berg} followed by every three-word product of {nee Smith, PhD, MA, Prof., de, Jr., Jones, VI, do, i Soler, nee}; 98,630 texts, each parsed under the default policy, `name_order` family-first and a ` - ` delimiter, the seven role fields, the ambiguity kinds and details and `initials()` compared against 97d36e0d. 589 texts move: 581 have a part after a comma whose first non-title word is a suffix word that is not also a title, or a title word past the second comma, or both; one, `Jr. i Ó VI Attorney, Ph.`, is the two-name-piece count reading a connective-joined piece beside a suffix word as the family (title 'Ph.', family 'Jr. Ó VI Attorney', suffix 'i', where master read title 'Jr. Ph.', given 'i'); the other seven were read by hand -- a leading period-shape title (`geb.`) in front of the opening credential, and #602's given-part run started by `Ph. D.` (`, abdul Ph. D. Do binti Smith`). Frames per parse through `Parser().parse`, py3.11: the reference band and plain names unchanged (`Dr. Juan Q. Xavier de la Vega III` 449, `Smith, John` 179, `John Smith, Jr.` 210, `Smith, John, Jr.` 223, `Smith, Ms Jane` 256); the positional count gets cheaper, leaving in C for a single piece and building its name set in a loop rather than a comprehension (`Smith, Jr.` 163 → 156, `Smith, MD PhD` 260 → 253, `John Smith, Dr.` 261 → 260); and the opened part pays for what it reads (`John Smith, PhD Jones` 338 → 361). Measured after the /simplify round, which also made the count ask for two name PIECES as well as two units (`Vega y Lopez Jr., PhD Jones` had lost its family). -- 2026-10-04 #611 — C1, S2 AND P6 GREW BY PATCHING, AND EACH SHOWS THE DECISIVE SIGN; THE RETHINKS ARE FILED, NOT DONE. The question was docs/design/AGENTS.md's, asked of the three longest rules after M2 (#601): does a guard model what another stage would do? C1: `segment` predicts group's particle chain (the #562 pair test) and assign's capitals reading (the skip #562 broke), mirrors classify's tags in its unit counts, and decides the structure in two stages, assign re-deciding #296's positional read with a second count -- the copy #603 extended. S2: the words-to-spare count runs over pieces group merges, so the chain re-asks the peel and rolls back (`John van Mc`), P5 compares two views, group emits S2's report, and the peel runs up to five times a parse. P6: assign holds back exactly the particle tail post_rules will attach (`particle_tail(..., sticky_from)`). Decided: #603 shipped as a targeted change inside today's machinery; the rethinks are #613 (C1, read the comma once after classify and bind, P6 folded in) and #614 (S2, read the trailing run once over units and bind), C1 first because S2's unit count should be shared with it, neither for 2.4. -- 2026-10-06 (Derek), #613 — C1 IS DECIDED ONCE, AFTER CLASSIFY, AND THE PART IT DECIDES IS BOUND BEFORE ANY JOIN. #611 found the decisive sign in C1: segment decided the comma structure before any word was tagged, so it modelled what later stages would do -- group's particle chain unsettling a run the capitals settled (#562's pair), assign's reading of a capitals-settled run -- and assign re-decided #296's positional read with a second name-word count. The decision now runs at the head of group (`_comma.decide`), once classify has tagged every word: the part after the first comma is read ONCE by `segment_suffix_reading`, the reading a part holding no name word gets, with the ambiguous words the count licenses passed in, and a postnominal part's SUFFIX and TITLE roles are bound there, so no later stage can read it differently (mechanisms.md#READ-WITHOUT-THEN-BIND, its second instance after M2's #601). segment hands every comma form over as FAMILY_COMMA and keeps C2's flag; script_segment, which runs between, asks its own narrower question (`_postnominal_behind_a_whole_name`, the vocabulary-only suffix test behind two surname units, its peeled tail excluded). Deleted with it: segment's decision block and #564's contrast (moved into `_comma`), classify's comma-run caps branch, assign's comma reading, positional count and second report, `_vocab.run_word_fold`, and the settled-run mirror test in `test_properties.py`, whose whole job was to hold segment's model of group equal to group -- with one decider there is no model left to hold. - DECIDED (Derek, 2026-10-06), each measured before the code was written: (1) a CONNECTIVE surname counts its words before the comma, as #575 decided for every surname but a particle chain -- `Ortega y Gasset, Dr.` reads title 'Dr.', given 'Ortega', middle 'y', family 'Gasset' (b547c247: family 'Ortega y Gasset'), and PR #615's Vega rows, which had pinned the connective as one unit, reverse; (2) nothing after the decision unsettles what it bound, so #562's pair report goes silent (`John Smith, PhD vd DO` keeps suffix 'PhD vd DO' and loses `suffix-or-name`) and a part of credentials holding particles side by side is the credential run behind any name (`De La Cruz, DO DO DO` reads suffix 'DO DO DO', b547c247 given 'DO', middle 'DO DO'); (3) no particle run, connective join or bound given name reaches into a part read as postnominal -- `John Smith, Ph. D. and Mary Jones` reads suffix 'Ph. D. and Mary Jones' (b547c247: family 'John Smith', given 'Ph. D. and Mary', middle 'Jones'), `De La Cruz, MA DO Mr.` reads suffix 'MA DO', title 'Mr.' (b547c247: given 'MA', middle 'DO Mr.', the chain having taken the title), and the one join kept inside the part is P3's connective binding title words into one title (`John Smith, Mr. and Mrs.` keeps title 'Mr. and Mrs.', `_comma._title_chains`; the first draft lost it, and the corpus now carries the name); (4) the positional read of a name before a suffix comma reports as the comma-less name does (`Freiherr von Berg, DO` gains the chain's `particle-or-given`, as `Freiherr von Berg` reports), and a part before the comma that is one name word with a suffix in it is the listing form's family rather than a positional read: `Smith Jr., Esq.` reads family 'Smith', suffix 'Jr., Esq.', and `Smith Jr., V` given 'V', family 'Smith', suffix 'Jr.', as `Smith Jr., John` reads (b547c247: given 'Smith' with `given-or-family`, and given 'Smith', suffix 'Jr., V'). - ACCEPTED WITH THEM, found in measurement or review rather than decided ahead. A connective between a credential and a title joins nothing, since it takes its kind from the credential in front: `John Smith, PhD and Dr.` reads suffix 'PhD and', title 'Dr.' with a report (b547c247: given 'PhD and Dr.'), garbage either way, pinned as the cost of (3). The split credential written around another word is not merged, merging being for adjacent halves: `John Smith, Ph. Ma D.` reads title 'Ph.', given 'Ma', middle 'D.', family 'John Smith' (b547c247: suffix 'Ph. Ma D.'), garbage too. A LISTED member written with a period reads as it does without one behind a whole name, the count licensing it before the lone-piece route's period inference can (`John Smith, Ma.` suffix 'Ma.' as `John Smith, Ma`; b547c247 title 'Ma.'), and `John Smith, PhD Ma.` reports the count's flip over the part where b547c247 reported the anchored pick, as `John Smith, PhD Ma` always did. The garbage `Ph. D., Jr.` reads family 'Ph. D.' (b547c247: title 'Ph.', family 'D.'), its watched row re-recorded in compare.py. - THE CJK RADAR TIER, settled in review and then by Derek. script_segment runs before group, so it cannot read decide's answer, and it needs one: whether the part after the comma is the postnominal part of a whole name decides whether the name before it divides and where a glued honorific peels from. The first draft asked a hand copy of C1 there, by the vocabulary alone, and the copy drifted from decide in four review rounds -- the ambiguous class's count (`김민준 박, MA` read as a credential run behind a two-word name while the name stayed whole), decide's numeral refusal and policy-selected suffix test (`김민준 박, MA V` divided behind a family comma, family '김 민준 박'), a suffix before the comma counted as a name word (`김민준 Jr., PhD` divided, the division then making decide agree), and the split `Ph. D.` (`김민준 Ph. D., PhD`) -- each drift dividing a name decide read as a family comma, or leaving whole one it read as a suffix comma. DECIDED (Derek, 2026-10-06, the /simplify pass): script_segment asks `_comma.decide` itself, on a classified copy of the name as written, and only where the answer can change something (the peel once its two candidate sites differ, the division once there is a token to divide), so no Latin or accented-Latin comma name pays for it. The division now follows decide in both directions (`test_the_division_never_outruns_the_comma_decision`: on 3320d700, the last tree with the copy, 0 of its 300 parses divided behind a family comma and 44 stayed whole behind a suffix comma; now 0 and 0). Against b547c247 that restores every reading the copy had moved (`Smith 김민준씨, XYZ`, `John 田中さん, XYZ`, `김민준 박, X.Y.Z.` divide and peel as master read them), and moves one class master read otherwise: a title and one CJK name word before a credential the capitals license, which decide reads as a suffix comma -- `Dr 김민준, MA` reads title 'Dr', given '민준', family '김', as `Dr 김민준` does (b547c247: family '김민준', the division declined and H1 making the one word the family), and the pre-existing tolerated row `Dr. 김민준씨, MA` moves with it; so does a glued honorific in a part decide reads as postnominal, `John Smith, Dr. J.씨` reading title 'Dr.', given 'John', family 'Smith', suffix 'J.씨' as `John Smith, J.씨` always did (b547c247 peeled the 씨 from the part and read given 'J.', family 'John Smith'). Measured on the /simplify pass's 78,282-parse fingerprint: 1,106 parses move, every one a CJK name before the comma or a `J.씨` after it. `마틴 킹, MA` reads as master did; its report counts three name words, decide reading the divided name. `_vocab.surname_unit_count`, which only the copy asked, is deleted. - DELIMITER CORES are dropped only behind a whole name, which is C1 Accepted's v1 parity (`Smith, MD - PhD - FACS` keeps its cores): the first draft dropped them behind a one-word family too. - WHAT MOVES, measured 2026-10-06 against b547c247 (PR #615's merge), py3.11, with the review round's fixes in. The harness: every differential corpus name at b547c247 plus a comma grid -- for each head in `Smith`, `John Smith`, `van der Berg`, `García Márquez`, `De La Cruz`, `Dr. Smith`, `John Q. Smith`, `SMITH`, `JOHN SMITH`, `Smith Jr.`, `Vega y Lopez`, `LLOYD WEBBER`, `abdul Salam`, `john smith` and `Freiherr von Berg`, the forms `H, A`, `H, A, Jr.` and `H, John A` for every A in `PhD MD Jr. III V V. I MA Ma ma Ed DO Do vd de X.Y.Z. XYZ LEED AP G.J. MJ Ms Mr. Dr. Prof. Jones John Q. Ph. D. Esq. RN - nee i y Sr. and MEng 씨`, `H, A1 A2` for every ordered pair of those, `H, C1 C2 C3` and `H, C1, C2` over `PhD MA Ma DO vd G.J. MJ Ms Mr. Jones V XYZ Jr. Ph. D.` -- 80,423 distinct texts, each parsed under the default policy, `lenient_comma_suffixes=False` and a family-first order; comparator the role fields, the ambiguity kinds and details, and `initials()`. 3,631 texts move, 1,773 with a role. Under the default policy the 1,739 role movers fall in the classes above and nowhere else, sorted by the first of these tests a text meets, in this order (the classifier is shape-only, so a later class can hold texts an earlier one would have taken): `Ph.` and `D.` in the part with a word between them, 31 (ACCEPTED); a connective `y` or `i` in the part before the comma, 936 (1); a part before the comma ending in a suffix word, 456 (4); a connective `and`, `y` or `i` in the part after it, 65 (3); a particle (`do vd mc de van`, any case) in the part after it, 250 (2, 3); and `Ph. D., Jr.`. The report-only movers, compared IN ORDER: 1,177 gain `particle-or-given`, every one the `Freiherr von Berg` head (4); 166 lose `suffix-or-name` (2's pairs) and 4 lose `given-or-family` (`Smith Jr., Jr., Mr.` and its kin, 4); 182 keep the same reports in a new order, `comma-structure` now written before the comma's `suffix-or-name` because segment emits the former and the decision runs after it (report order is no contract, and nothing pins it); 266 change a kind or a detail, the shapes being the first-word report taking the place of the count's flip report on a part a class word opens (`García Márquez, DO DO`), the absorbed-word report no longer naming a particle the chain had joined to the word (`De La Cruz, PhD DO D.` reports 'D.' where it reported 'DO D.'), and the count's flip taking the place of an anchored pick (`John Smith, PhD Ma.`); 63 move under a non-default policy alone, 34 of them in a role. The corpora at b547c247 (1,526 distinct names): four move -- `John Smith, PhD DO DO` and `John Smith, PhD vd DO` lose (2)'s report, `Ph. D., Jr.` as above, and `마틴 킹, MA` in its report detail alone. The gate exits 0 at all five baselines; intentional diffs 631 → 632 at 1.4.0 and 285 → 286 at 2.3.0, the corpus having gained `Ortega y Gasset, Dr.` and `John Smith, Mr. and Mrs.` as C1 examples, and radar unclassified gains `Ph. D., Jr.` at both. - COST, `sys.setprofile` call counts after one warm-up, b547c247 → the tree with this entry's P6 fold and review fixes in (the P6 bullet under P6 gives the fold's own share): `John Smith` 151 → 151, `Smith, John` 179 → 160, `Smith, John Q.` 246 → 230, `Smith, Jr.` 156 → 156, `Smith, John, Jr.` 223 → 204, `John Smith, MA` 249 → 253, `John Smith, Dr.` 260 → 212, `Smith, MD PhD` 253 → 185, `John Smith, PhD Jones` 361 → 262, `Doe, Jane nee Smith` 309 → 293, the reference parse `Dr. Juan Q. Xavier de la Vega III` 449 → 449. A no-comma name never enters `decide`, decide returns before C1's counts where the part holds a name word and nothing of the ambiguous class (guarded by `test_a_listing_comma_with_nothing_to_decide_skips_the_counts`), and group filters its segments for bound roles only when `decide` bound something. The paired-initials speaker scan came back quadratic in the move (`is_suffix_piece` 120 → 1,620 calls for 4× the input on `'John Smith, ' + 'MD '*k + 'G.J. '*k`) while #563's guard, which counted `_normalize`, passed; the scan now finds the speaker once, and the guard counts what the scan calls. - THE PR REVIEW, after the two rounds above. An EMPTY part before the comma had decide return early, leaving the part to the listing walk: `, PhD` read given 'PhD', `, Jr.` title 'Jr.', `, MD PhD` title 'MD', family 'PhD', where b547c247 and 2.3.0 read suffix -- decide now reads the part behind an empty head, which names no whole name, so the comma stays the family comma with the part bound (the heads of every grid above were non-empty, so none saw it). And three more costs the frame guards could not see, each now a clock row in `_PREFIXED_SHAPES` (base 1600, broken 9.9-11.9 against fixed 4.0-4.3): `_title_chains` rebuilding the growing title from a slice at each join, the paired-initials test copying the shape set per pair and scanning a list of the class members, and the core filter scanning a list of the cores. And one render view: decide cleared its caps-shape marks whenever C1's run test declined, so a part #603's opener bound anyway kept its all-caps word without them, and case repair wrote `John Smith, PhD Jones XYZ` as '... PhD Jones Xyz' (rules.md#R4); the marks now stay on every bound caps word that carries the name's contrast, roles and reports unmoved over the harness (`test_a_bound_caps_word_keeps_its_capitals_in_case_repair`). - SUPERSEDES: the 2026-10-01 #562 bullet's mechanism and its accepted reports (its pair report and the settled-run test); the 2026-10-04 #603 bullet's "segment decides" wording; and the #575 bullet's account of WHY segment builds its unit facts from vocabulary -- `_comma._whole_name` counts over classify's tags, and `name_word_count`, which reads the vocabulary, is asked by decide and by script_segment, the agreement sweep `test_surname_unit_tags_agree_with_classify` keeping the vocabulary view equal to the tags. -- 2026-10-06 #617 — CROSS-REFERENCE: the title join decision (3) above makes inside a part read as postnominal is no longer `_comma._title_chains` but group's own loop, `_pieces.join_connectives`, and it now merges a connective run first (`Smith, Chancellor of the Exchequer MP` reads suffix 'MP') and freezes the generational class; decided and measured in decisions.md#P3's 2026-10-06 #617 entry, which amends (3). -- 2026-10-11 (Derek), #636 — NO PARTICLE CHAIN IS MADE IN A PART THAT HOLDS NO NAME WORD. A part past the second comma, after either comma, is read wholly as suffixes, its title words as titles (C2, #603), but group still ran the particle chain over it, which builds a name unit no reading uses (the part after a suffix comma, which C2 covers too, had reached no chain since #613, `_comma.decide` binding it and emptying its segment before any join; a suffix-comma grid moves nothing here -- `{head} {w1 … wk}` for k from 1 to 3 over the heads `John Smith, Jr.`, `John Smith, PhD`, `, PhD`, `John Smith, MD Ma`, `John Smith, Jr. MA`, `Smith, PhD`, `, Jr.`, `John Smith, X.Y.Z.` and the words `van der de la von und St Freiherr do mc vd du Dr. Dr Prof. Prof Attorney Sir Lt.Gov. Mr. MA Ma PhD X.Y.Z. VI V Jr. Berg Jones`, 606,216 parses under the three orders, a sample reaching both comma structures): a particle past the part's first word took the words behind it, a title word among them, so `John Smith, Jr., van der Berg Prof.` read suffix 'Jr., van der Berg Prof.' where `John Smith, Jr., Prof.` and `John Smith, Jr., van Berg Prof.` (whose leading `van` chains nothing, P4) read title 'Prof.'. Now group makes no chain in such a part, and asks no chain bound there, so C2 reads every title word in it, bare (`Prof`) or mid-part (`van der Prof. Berg`) alike. DECIDED (Derek): no chain there, rather than the issue's proposal, #623's bound (`trailing_run_start`) at this site; that reads trailing titles only (H5), narrower than C2's rule, and adds a call where removing the chain takes one away. The chain's two report emitters had been handed no report list in these parts for the same reason (rules.md#C2's no-name-reading clause, 2026-09-28); with no chain there is nothing to suppress, and the handoff now suppresses reports only after a family comma (removing the tail half moved nothing over the suite, #624's comma-tail grid's 292,950 parses, every corpus name and case text, and the suffix-comma grid above). - MEASURED 2026-10-11 against 3b2b6f0d (#635's merge), py3.11, `nameparser.__file__` asserted on each side. #624's comma-tail grid (its recipe, decisions.md#P2, 2026-10-10; 97,650 texts under the three orders), with every corpus name and case text at 3b2b6f0d: 18,318 parses on 6,106 texts move a role, every one in the grid and none of 3b2b6f0d's corpus names or case texts, and no parse moves a report alone. 15,330 put a title word that stood behind a particle run into the title field (`,, 'Bob' van der Attorney` → title 'Attorney', suffix 'van der'; master suffix 'van der Attorney'), and 2,988 take a particle back out of a title the chain had joined it to, where the chain merged it with a P3-joined title and kept the tag (`,, 'Bob' van der Mr. and Mrs.` → title 'Mr. and Mrs.', suffix 'van der'; master title 'der Mr. and Mrs.', suffix 'van'). The S2 stress grid, #624's title-particle grid, #623's given-part grid and #625's particle grid: 0. The five gates exit 0, a `fix(#636)` rule in each 2.x ledger; at 1.4.0 the broad `fix(comma-family) lone post-comma piece routes to suffix/title` rule explains the new example. Frames: the reference rows and family-comma names do not move; a tail part with a particle costs less (`John Smith, Jr., Freiherr von Richthofen` 440 → 428, `John Smith, Jr., van der Berg Prof.` 462 → 452, py3.11 warm). Recorded negative control: with the chain run in these parts again, 9 tests fail -- the three `fix(#636)` rows on both surfaces, the two new rules.md#C2 examples, and `a_part_past_the_second_reports_no_particle_fork`, whose chain report the handoff no longer suppresses, so that row guards the change too -- and with it run only after a family comma, the review's rows `Smith, John, van der Berg Prof.` and `,, van der Berg Prof.` (both surfaces), which it added after that mutant passed the suite. - THE REVIEW (2026-10-11), measured, all garbage-neutral or better, no reading worse over the review's generated tail-part sweeps under thirteen configurations (fields, reports, `initials()`, `capitalized()` and the facade's views compared; the populations were the reviewer's and are not recorded here, so no count is quoted): a word in both the title and particle vocabularies inside a tail part now reads as a title as a plain title there already did (`John Smith, Jr., Berg St Smith` → title 'St'); a particle between two titles there becomes a suffix entry of its own (`John Smith, Jr., St von Mr. and Mrs.` → title 'St Mr. and Mrs.', suffix 'Jr., von'). Two gaps outside this change, already on master: P5's bound join still runs in a tail part after a suffix comma (`John Smith, Jr., abdul Dr. Smith` keeps 'Dr.' in the suffix where `John Smith, Jr., Berg Dr. Smith` reads title 'Dr.'), and rules.md#C2's no-name-reading clause is contradicted for the part after a suffix comma by #603's per-word report (`John Smith, PhD van der Berg` reports suffix-or-name on each word). - Open: #640, #641. (Both settled 2026-10-11: #640 by #637's entry, decisions.md#P5; #641 below.) -- 2026-10-11 (Derek), #641 — C2'S NO-NAME-READING CLAUSE MEANS A WORD READ AS A NAME, AND C1'S PER-WORD REPORT IS THE PART'S OWN. rules.md#C2 said a part the parse consumes wholly as suffixes "raises no report about reading a word of it as a name", whether it is the part after a suffix comma or a part beyond the second; but the part after a suffix comma reports each word its opening credential takes (`John Smith, PhD van der Berg` → suffix-or-name on 'van', 'der' and 'Berg', "follows a credential, so it reads as part of the suffix run; it may be a name word"), the report rules.md#S2 states ("A name word the run absorbs is reported `suffix-or-name`"), which C1's credential-opening branch reaches by reading the words after it as that run reads them, and which C1's example pins (`John Smith, PhD Jones` → `suffix-or-name`). The rule text and the code had disagreed since #603 added the report (in `assign`; #613 later moved it into `_comma.decide`), and no test held either side's wording. DECIDED (Derek): the report stays -- a word behind a credential in a part the comma made postnominal is the fork it reports in `John Smith PhD Jones` -- and C2's sentence is narrowed to what it meant: no report that a word of such a part was READ AS a name (a particle chained onto a name word, an acronym taken into the name). The part's own reports are listed by their rules: C1's report of the decision at its comma (the flip, or the first word after the comma), S2's report on each word a credential in front of it reads as a suffix although it may be a name word (a name word the run absorbs, a member the company decides), and C2's flag. Measured by the review over about 1.2 million parses of suffix-comma and tail parts: those are the only reports such a part carries. A text change only: no reading or report moves (the suite and the five gates unchanged), and the one code comment quoting the old sentence quotes the new. - -### T1 — separators, not joiners - -- 2026-07 (v2 core, PR #288) — v1's squash_emoji/squash_bidi REMOVED the character and joined its neighbors. (v1.3.0 had no bidi handling at all: squash_bidi entered late v1 via #266, 2026-07-07, on the emoji precedent's shape.) ('A😀B' → 'AB'); v2 makes an ignorable character a separator ('A😀B' → 'A', 'B'). The unavoidable consequence of every part being an exact positioned piece of the input: with no rewriting stage, nothing can splice two half-words together. - -### T3 — the interpunct's flank guard - -- 2026-07-30 #298 — codepoint scope, chosen over a blanket - "dot = transcription" rule that would have flipped 高橋・一郎, a - correct and deliberately pinned #272 reading: U+00B7 records the transcription fact and suppresses division; U+30FB/U+FF65 stay pure separators whose pieces the script license reads (rule W4's Accepted pair demonstrates both). Cross-convention input reads by the codepoint it was actually typed with — a chosen limitation, recorded in #298's comments. - -Declined: - -- Ambiguity emission on a half-flanked interpunct ("王·Smith") — proposed in the 2.1 PR review, declined because the dot decides silently under the no-emission decision; the undivided-dot-stays-in-the-word behavior went to docs instead. - - -- 2026-07-30 #298 — U+00B7 divides only between classified-script characters because it is also the Catalan punt volat, interior to legitimate names (Gal·la). The nakaguro (T2) needs no such guard: its codepoints are CJK-only and appear in no other script's names, which is what licenses an unconditional rule. - -### M1 — delimited maiden names - -- 2026-07-03 (maiden-bucket design, landed via the v2 core, PR - #288) — the maiden reading of a delimiter pair is opt-in because - enclosure conventions genuinely vary; there is no default pair. -- 2026-07-18 (cc7063f; Derek's proposal out of a docs-review pain point — routing parens to maiden used to require editing both buckets in tandem, and the nickname default's contents were not discoverable) — bucket overlap is canonicalized before parsing: a pair listed for maiden is dropped from the effective nickname set (maiden wins), and the public DEFAULT_NICKNAME_DELIMITERS constant landed with it. The v1 facade restores v1's nickname-wins reading by pre-subtracting on its side. Weighed and ruled the same day: maiden-wins applies THROUGH apply_patch too — a pack's maiden pair silently removes a user's unrelated explicit nickname pair; warn-on-removal was considered (a silent-failure review flagged the site) and rejected, the silence ruled intended. -- 2026-08-04 #329 (PR #331) — the marker word inside a delimited clause is dropped from a multi-word clause during grouping; the extraction itself keeps the whole enclosed span, so nothing is lost when there is no marker. Three decisions argued separately: the clause-size guard exists because Nee is an attested surname (Irish Ní/Nee, a Chinese romanization) — load-bearing, not defensive, and mutation-proven; clauses are independent — "(Nee) (Jones)" reads maiden "Nee Jones", each clause dropping or keeping its own marker (the first implementation leaked across clauses, a real defect); and the contentless "(née —)" alone now parses to every field empty and bool() False (an explicit alternative keeping maiden "née —" was rejected; pinned by the maiden_marker_delimited_content_free case). -- M1 and M2 are disjoint by construction, which is the no-conflict guarantee: M2's walk covers joining structure that role-bearing words never enter, while M1's drop reaches only extracted content. Verified independently at review over 7,775 records: 967 diffs, every one in the maiden field. - -Declined: - -- Neighbour-scoping the drop (drop a marker whose next word also reads maiden) — implemented, reviewed and rejected: it also fires on the bare-marker path, eating the surname out of "Jane Smith nee Nee Jones" (maiden "Jones" instead of "Nee Jones"), and it leaks across adjacent clauses. It is the obvious implementation, which is why this entry exists. - -- 2026-08-05 #329/#335 — marker auto-detection inside a nickname-delimited clause was deferred to #335 on a corpus measurement: 山田 花子（旧姓 佐藤） is in the CJK differential corpus so the #329 change was gate-visible, while "Jane Smith (née Jones)" is in no corpus — shipping auto-detection in 2.1 would have let a real Latin-affecting change ride under a "0 Latin-only" gate report. - -- 2026-08-26 #335 — correction to the 2026-08-05 entry above, which reasoned from "the #329 change was gate-visible". It was not. Under the default policy the corpora run, no name reached the drop pass at all, so 山田 花子（旧姓 佐藤） moving between 2.0.0 and 2.1.0 was the East Asian order flip and not #329. Nor is #329 gate-visible now that M3 routes marker-led clauses to maiden without configuration: a ledger rule narrows by which FIELDS move and never by what they hold, so reverting the drop leaves the same six names moving the same fields and all three gates green (measured by mutation 2026-08-26). The {nickname, maiden} pair is the 2.1.0 ledger's; the other two baselines classify the CJK name differently, which is why those ledgers are split and must stay so. The deferral #329 argued for was still the right call — it was about a Latin-affecting change riding under a gate report, and that half held — but the premise is corrected here rather than left to be re-used. tools/differential/README.md carries the same correction beside the table it belongs to. -- 2026-08-26 #335 (M3) — the opt-in this section opens with narrowed, and the boundary is not where the 2026-07-03 entry drew it. A clause whose content announces itself no longer needs its pair declared, so what configuring maiden_delimiters buys is now markerless content and one-word clauses. See #M3 for the whole reasoning; M1 still governs every clause whose pair is configured, and M3 defers to it for the marker drop itself. - -- 2026-08-26 #434 — correction to the 2026-08-04 #329 entry above, on the two statements a phrase entry falsified, and a decision that entry's reasoning does not reach. Its "dropped from a multi-word clause" is no longer the condition: "(z domu)" under a configured pair is a two-word clause whose marker is NOT dropped, because both words ARE the marker. The condition has always been "the clause holds a word past its marker" and reads that way in the code; with one-word markers the two were indistinguishable, which is why the entry could say the wrong one for two years without cost. Nor is there a "clause-size guard" any more, load-bearing or otherwise — the containment test now spans the whole marker run, so its subject is the token AFTER the run rather than the second token of the clause. What survives unchanged is the REASON that guard was written: Nee is an attested surname (Irish Ní/Nee, a Chinese romanization), so a clause of nothing but its marker keeps its words, and that argument is mutation-proven still. Recorded here rather than only under #M2, because a future PR reading M1's own history pointer would otherwise find "the clause-size guard" described as load-bearing and restore something that reintroduces the phrase defect while passing every doc test. - - The decision the rewrite forces, stated so a later PR has something to weigh against. Under a configured maiden pair, "Maria Kowalska (z domu)" emits maiden "z domu" — the marker itself as the whole maiden name. That is not what the phrase argument under #M2 says about those words: "z domu" is borne as no name in any position, which is exactly why shipping it as a phrase is safe. So the value is a name nobody has. It is emitted anyway, and deliberately: M1's carve-out is that a clause holding nothing past its marker keeps its WORDS, and the parser has no way to know a caller's configured pair did not enclose an unfamiliar surname — the same ignorance that makes "(Nee)" keep its word. The alternative readings, an empty maiden or a nickname, both require the parser to judge its own vocabulary's words unnameable, which nothing else in the library does. Cheap to revisit: the shape is one rules.md example and one case row away from a decision either way, and no corpus name has it. - -### M3 — the marker-led clause - -- 2026-08-26 #335 (M3, PR on fix/335-maiden-marked-clause) — a bracketed clause whose content opens with a recognized marker word and carries a word after it reads as the maiden name, whichever bucket the enclosing pair sits in. The parse already had the information: "Jane Smith née Jones" gave maiden "Jones" while "Jane Smith (née Jones)" gave nickname "née Jones", because ( ) is a nickname pair by default and nothing looked inside it. The clause says "maiden" out loud, so the caller should not have to say it in Policy. -- 2026-08-26 #335 — the MECHANISM, and the issue proposed the other one. Masking the delimiters and letting M2's bare-marker rule consume the content would avoid a second implementation, and it was rejected on measurement: the two readings agree on a trailing clause and diverge on an interior one. "Jane (née Jones) Smith" reads given "Jane", family "Smith", maiden "Jones" under extraction; masking turns it into the bare "Jane née Jones Smith", which reads maiden "Jones Smith" and family "" (measured 2026-08-26), because M2's take runs to the end of the name. The closing delimiter is a boundary the writer typed and masking throws it away. Extraction was also the SMALLER change — extract already emits Role.MAIDEN regions and group already drops the marker inside one (#329), so this is a role swap at the existing emit site plus one predicate, in the branch S1 has already declined. That placement is what keeps S1 first, but be exact about the mechanism, because the obvious mutation does not test it: hoisting the swap above the _suffix_shaped call changes nothing at all (measured 2026-08-26), since the S1 branch never reads `role` -- it masks the two delimiter spans and hands the content to the token stream, and the role it would have carried is never consulted. What inverts the precedence is giving M3 the clause first, an early `extracted.append((Role.MAIDEN, inner))` ahead of the S1 test: that reads "Jane Smith (née Jr.)" as family "Smith", maiden "Jr." instead of family "née", suffix "Jr.", and M3's own example line for it fails. So the precedence is pinned, and by that example rather than by the branch shape. -- 2026-08-26 #335 — the condition is a word AFTER the marker, not the marker alone. The reason is M1's: a one-word clause keeps its word because Nee is an attested surname (Irish Ní/Nee, and a Chinese romanization), and a marker with nothing after it marks nothing. Be precise about what pins it, because the obvious answer is wrong: M1's own boundary line carries the maiden-parens annotation, so it asserts the CONFIGURED reading and passes either way — built and measured 2026-08-26, a len(words) > 0 predicate still gives it maiden "Nee". What the loosened predicate actually breaks, measured by mutation 2026-08-26, is M3's own boundary example ("Jane Smith (née)" → nickname "née") and tests/v2/cases.py::maiden_marked_clause_one_word_stays_a_nickname in both runners — three pytest failures — AND the 2.1.0 differential gate, which goes red with three unexplained names: "Jane Smith (Nee)", "Jane Smith (Nee) (Jones)" and "Jane Smith (née)". The gate is the stronger signal and the one to look at first. The middle name is why: loosened, it reads maiden "Nee" and nickname "Jones", splitting one bracket pair into the maiden field and the next into the nickname field, which is the direct contradiction of M1's "two enclosures read as one maiden name". Anyone re-litigating the second-word test should check those, not M1's line. So the real division of labour is NOT markerless versus marker-led, which is how it was first written into rules.md and had to be corrected at review: it is one word versus a marker plus a word. "Jane Smith (Nee)" is marker-led and still reads nickname "Nee" by default, maiden "Nee" only with the pair configured (both measured 2026-08-26). -- 2026-08-26 #335 — accepted consequence: the word taken after the marker is not tested for being a name word, so M3 does not stop at a suffix word where M2's bare take does. "Jane Smith (née V)" reads maiden "V" while the bare "Jane Smith née V" reads suffix "V" and family "née" (both measured). The reason is structural rather than an oversight: M3 hands the WHOLE clause to Role.MAIDEN and the bracket is its right boundary, so the trailing-suffix rule never sees the word as trailing the name at all. Pinned as M3's Accepted line in rules.md, which puts "Jane Smith (née V)" in the differential corpus and classified in all three ledgers. -- 2026-08-26 #335 — rules.md#M3 lists implemented: nameparser/_pipeline/_extract.py alone, and that is deliberate rather than an omission. M3 decides a REGION'S ROLE; the marker drop it describes ("the marker itself dropped, as M1 drops it") is M1's mechanism in _group.py, which cites M1 and is governed by it. Adding _group.py to M3's list would mean inventing an M3 citation there for behavior M1 already owns — and test_doc_citations requires implemented: to equal the set of modules that cite the rule, so the two would have to move together. Do not re-open this. -- 2026-08-26 #335 — the durable finding, and the one worth reading before touching either side: _maiden_marked asks a WHITESPACE-SPLIT, _normalize'd question of the clause's first word, while tokenize and classify ask a token-level one. _normalize strips a trailing period but not a comma, so 'née,' is not a marker to M3 while the tokenizer splits the comma off and still tags the token vocab:maiden-marker. That divergence is the only thing keeping the "role is not Role.MAIDEN" branch of _group.group's clause-scoped drop pass reachable at all: without it, a marker-led NICKNAME clause could no longer exist, since M3 would have converted every one to MAIDEN before group ran. Anyone "unifying" the two tests would orphan that filter silently. The pin is tests/v2/cases.py::marker_glued_to_punctuation_keeps_the_clause_a_nickname, and it needs BOTH of its clauses — the drop pass is gated on the name holding a maiden region at all, so a marker-glued nickname clause standing alone leaves the branch unexercised (measured by mutation, 2026-08-26). The row that used to pin the filter, maiden_marker_delimited_beside_a_nickname_clause, lost the job to M3: both its clauses are marker-led, so both are maiden now and there is no contrast left in it. SUPERSEDED IN PART 2026-10-12 by #653: a clause's comma now stays on its word, so 'née,' is no marker to the tokenizer either and the comma row no longer reaches the branch; the divergence lives on through rules.md#T2's nakaguro, and the pin is cases.py::marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname (see #N1's #653 bullet). -- 2026-08-26 #335 — M3 makes the marker vocabulary act in a THIRD position. M2's claim acts on the word after a name word; M1's on the content of a configured pair; M3's now on the first word of any bracketed clause, under the default policy. #vocabulary-collisions C-i is keyed on the position a claim acts on, so extending where a rule acts re-asks C-i of the whole set it reads, and nothing in the process prompts that — the rule was designed, reviewed and approved without the set being re-examined. Review afterwards found one entry that fails C-i, the Czech abbreviation roz, and its disposition is this file's Excluded (MAIDEN_MARKERS) block, which is keyed to the vocabulary set rather than to any rule: it was removed. Two things worth separating, since the finding surfaced during this work and is easy to file under it — the defect was M2's and predates #335 (bare "Rosalind Roz Smith" lost its family name on master), and M3's own boundary already excluded the common one-word "(Roz)" spelling. What belongs to M3 is only the widening, and the lesson is the general one: re-ask C-i of a vocabulary set whenever a rule extends where its claim acts. -- 2026-08-26 #335 — what configuring maiden_delimiters still buys, since M3 removes the commonest reason to reach for it: markerless clauses, and one-word clauses. "Cherice J. (Johnson) Williams" is a real US convention and a corpus name, and nothing in that clause says maiden, so it stays a nickname by default — only a caller who knows their data can say otherwise. The knob is a declaration about the DATA; M3 is a reading of what a clause declares about itself. - -### M4 — a maiden marker makes the lone name word the family - -- 2026-08-27 #445 (M4, PR on fix/445-maiden-marked-lone-name) — Derek's rationale, in his terms: normally if there is only one name we assume it is a given name because we have to pick one, but maiden markers mark a previous surname before marriage, so it would not make sense to include one if there was no other surname for it to clarify — and given that, when a marker leaves only one name word we switch the assumption and read that word as the family name. `parse("Smith née Jones")` reported given 'Smith' with an EMPTY family through 2.1 and reads family 'Smith', maiden 'Jones' since. The rule is written in post_rules as H1's sibling and placed directly under it, so the interaction is decidable by reading rather than by running: where H1 fired there is no GIVEN left for M4 to move, and where H1 declined because the title addresses by given name ("Sir John née Jones") the `not titles` guard keeps M4 off the same word. A titled name is H1's at both outcomes, which is what keeps H1's given-name-title carve-out intact. -- 2026-08-27 #445 — the third time in the 2.2 bundle that a name carrying a maiden clause reported no family, and the three share a SYMPTOM rather than a cause. State that carefully, because the first draft of this entry claimed one cause for all three and was wrong. #410 (H1): a title plus one name word reported no family the moment a suffix, nickname or maiden name stood beside it. #411 (P5's reserve): the bound given-name join counted the marker and the maiden name as words it could spend, and spent the family name. Those two ARE one cause -- a count that included words which are no part of the name -- and the grep that would find a fourth is a rule counting name words without first asking which of them are name words. #445 is not that: it changes no count anywhere (post_rules gains a new block, assign gains only a comment), and `parse("Smith")` with no clause beside it at all already read given 'Smith' with an empty family, so nothing was being swallowed. Its cause is the one the next entry gives: O4 decides nothing at one name word, and the reading came from a convention nobody had written down. The site that grep would miss is `_name_positions`'s `count == 1` branch, silent for every order and every name -- which is exactly what O5 now records. -- 2026-08-27 #445 — O4 was SILENT at exactly one name word, which is why the shipped reading had no rule to point at and why O5 now exists. O4 reads a name by comparing where its words stand ("the first name word is the given name, the last is the family name"); with one word the first IS the last, so there is nothing to compare and the rule decides nothing. O5 records the reading as the convention it is — a guess fixed in advance so that the same input always reads the same way, not a determination about the word — and names H1, N3 and M4 as the rules that DO decide such a name. Written that way deliberately: a rule asserting "a lone name word is the given name" as a fact would have made #445 a contradiction of the documented behavior instead of an exception to a documented guess, and would make #449 one too. -- 2026-08-27 #445 — the two carve-outs are not inventions, and both rest on mechanisms.md#TWO-LAYER-ASSIGN: the positional layer never overrides a vocabulary claim, and M4 changes only what POSITION decided. A word the vocabulary claims as a given name keeps that reading ('abd née Jones' — `vocab:bound-given`), and so does a word written as an initial ('J. née Jones Smith V' — the `initial` shape tag; amended 2026-09-13 under P3: the tag is no longer purely a shape claim). Both are read off tags classify already recorded rather than off a predicate of M4's own, and both witnesses are real corpus names rather than constructed cases, so each carve-out has something in the differential that would notice its loss. Mutation-checked on a scratch copy: dropping the `initial` carve-out fails that name's case row, its facade twin, M4's own boundary example and the pre-existing assertion in test_the_chain_and_the_walk_stop_where_the_peel_begins; dropping `vocab:bound-given` fails the other name's two runners, M4's boundary, P5's own doc example, O5's own `abd née Jones` line and the corpus-wide maiden-clause property on 'abdul' -- six in all, the O5 line having been added by the review round, and the list is exhaustive as re-run against the final tree. -- 2026-08-27 #445 — the two halves of Derek's answer differ in their relationship to 1.4.0, and a reader should not have to re-derive it. The bracketed spelling RESTORES v1: `Smith (née Jones)` read family 'Smith', nickname 'née Jones' on 1.4.0, 2.0.0 and 2.1.0 alike, and now reads family 'Smith', maiden 'Jones' — the clause changes hands and the family name stays put, so the 1.4.0 gate's diff on that name SHRANK to {nickname, maiden}. The interior spelling is a NEW reading: `Jane née Jones Smith` read first 'Jane', middle 'née Jones', last 'Smith' on 1.4.0 and reads family 'Jane', maiden 'Jones Smith' now, the real surname being inside the maiden value by M2's greedy take. That the marker stands inside the name changes nothing for M4, which counts what the take LEAVES rather than where the marker stood — one name word left that way is one name word, which is the widest half of the decision and the half no earlier version agrees with. -- 2026-08-27 #445 — a precedence claim corrected while drafting, and it is exactly the class this bundle keeps producing. The first draft of M4 said a name carrying a nickname is N3's rather than this rule's. Measured false: `'Smitty' Jones Jr. née Smith` reads family 'Jones', because N3's count does not set a suffix aside, so N3 declines and M4 fires. A nickname holds nothing off — where N3 has already named the family M4 finds nothing left to move, and where N3 declined M4 names it. The general lesson is the one #410 and #411 already taught from the other side: a rule that counts name words and a rule that counts something else will disagree at the edges, and the disagreement is only visible if the interaction is measured rather than asserted. -- 2026-08-27 #445 — what the ledger's `# revisit when #445 lands` markers bought, and what they got wrong. Four blocks carried them, and each predicted that a "keep the family" fix would leave the diff a subset of the fields its rule already declared, so the rule would go on explaining the name and the change would be absorbed with the gate green. That was wrong in a way the rule's own shape settles: M4 MOVES the one name word rather than adding one, so `given` empties as `family` fills, the diff outgrows every declaration, and eight names arrived UNEXPLAINED at all three baselines. Two counts run through this and they are different sets, so name which is which: NINE corpus names change reading, of which eight arrive as new diffs — the ninth, 'Smith (née Jones)', has its 1.4.0 diff SHRINK instead, its family agreeing with v1 again. The four markers between them named NINE names to re-measure, of which seven moved: the two that did not are M4's carve-outs, and the two movers the markers could not name were not corpus names until this change's own examples added them. The markers earned their keep all the same — they named the right rule, the right field and very nearly the right set — so they are corrected in place rather than deleted, and the correction says what actually happened. THREE of the four moved in the opposite direction from their own prediction, and getting that number right took two passes. The fix(#335) rule for 'Smith (née Jones)' appears in all three ledgers, and every copy feared absorbing a WIDENED diff while the diff in fact SHRANK to {nickname, maiden}. The first pass narrowed only the 1.4.0 copy, on a measurement of that wheel, and left the other two at four fields on the reasoning that 2.0.0 and 2.1.0 'read `given`, and the baseline cannot move'. They did not: only the BARE spelling ever read `given`, and this rule holds the bracketed one, which reads family 'Smith', nickname 'née Jones' on every released version. So two rules stood over a two-field diff declaring four, and classify's subset test would have explained a `given`/`family` regression on that name in silence -- the absorption those very notes existed to prevent, surviving the round that answered them. A third reviewer found it by instrumenting the comparator to dump real diff-field sets, and that is the lesson worth keeping: ledger prose cannot be checked by reading it, and the round that corrects a marker is exactly where a wrong premise gets written down with confidence. -- 2026-08-27 #445 — the corpus-wide property test is the repair worth copying. tests/v2/test_parser.py's `test_a_maiden_clause_changes_nothing_else` asserts over the corpus that appending " née Jones" adds a maiden name and moves no other field, and M4 falsifies it for fourteen names. Skipping them would have bought a green suite and lost the check; instead the test computes M4's guard from the base parse's tokens and asserts the flip — family takes what `given` held, every other field standing still — so the class that used to be a hole is now the strongest witness the rule has, fourteen names against the six rows cases.py carries. It is also what executes M4's nickname precedence, `'Smitty' Jones Jr.` being one of the fourteen. -- 2026-08-27 #445 (review round) — M4 is keyed on the maiden NAME, not on a marker, and the statement was corrected to say so after two reviewers measured the same defect independently. The guard tests for a token in the MAIDEN role, and M1's caller-configured pair produces one with no marker anywhere: under `Policy(maiden_delimiters=frozenset({("(", ")")}))`, `Smith (Jones)` reads family 'Smith', maiden 'Jones', and so do M1's own boundary examples `Smith (Nee)` and `Smith (z domu)`. The BEHAVIOUR is right — the rationale transfers to a declared pair without a word of change, since what announces a former surname is the clause, not the vocabulary that marked it — so the fix was to the rule, which had said "a recognized maiden marker that takes its name and leaves exactly one name word" and now says "a maiden name standing beside exactly one name word". That covers M1, M2 and M3 uniformly, M1 is in M4's `interacts:` and M4 in M1's, and the configured path has an example line of its own. The general lesson is the one this bundle keeps teaching from a new angle: a statement written from the case that prompted the change describes that case, not the code, and only measuring the OTHER paths into the same guard finds the difference. -- 2026-08-27 #445 (review round) — a known gap recorded rather than closed, the way H1 carries its own. M4's guard counts GIVEN tokens; H1 counts nothing at all (it tests which roles are unoccupied). So a name word another rule has joined counts as several here and M4 declines where H1 fires: `Dr. Dean of Chemistry` reads family 'Dean of Chemistry' while `Dean of Chemistry née Jones` keeps given 'Dean of Chemistry', the connective join (P3) having left three GIVEN tokens. rules.md#P3 says the joined part is one name word wherever another rule counts them, so the two siblings genuinely disagree and this is a gap rather than a boundary. Not closed here: widening the count to units moves zero corpus names (measured), and a behaviour change nobody has approved does not belong in a branch whose blast radius was settled. The sibling claim in the code comment and in this entry is narrowed to match — sibling except in what it counts. -- 2026-08-27 #445 — the relationship to #449, briefly, because the two look like the same question and are not. #449 asks whether a lone name word with nothing to decide it should REPORT an ambiguity; #445 removes a class of names from that population by giving them something that decides. So #445 reduces #449's reach rather than competing with it, and #449's measurement — a naive "one name word, nothing decided it" condition fires on 19 of 1,085 corpus names, only three or four of them the case it is about — is re-taken after this lands. O5 cites #449 in prose and deliberately carries no `deviates:` marker: that marker asserts an intended output the runner then checks strictly, and #449 had not decided one. It has since — 2026-09-07, decisions.md#O5 — and the answer arrived as ordinary example lines rather than as a marker, the report being shipped behavior and not a deviation. The re-measurement this bullet promises is taken there: the naive condition is wider than either number suggests, firing on 143 of the 1123 corpus names, and O5's own population is 27. - -### O1 — East Slavic rotation - -- (v1 era, PR #154) — why patronymic handling is OPT-IN at all: unconditional detection breaks ordinary Latin names whose endings collide (Martin, Franklin, Benjamin), the finding that forced v1's Russian work behind a flag and set the opt-in shape v2 inherited. -- 2026-07-12 (landed in the v2 core, PR #288) — v1 parity pinned live: the rotation reconstructs token position from assigned roles, which is faithful to v1 only under the default given-first order. - -- 2026-08-15 — the rotation × non-default name_order interaction question rode #270, which closed 2026-07-28 with the order constants and no recorded answer for the rotations. -- 2026-09-07 #384 — OPTION 1: the rotations are scoped to the DEFAULT order, and a gate on the read order is what makes that true. The principle is decisions.md#P1's 2026-09-07 entry and rules.md's Name order Background — a name order is a property of the data source, the caller declares it, the declaration governs what no vocabulary has claimed, and a declared family-first order outranks the shape. This rotation exists to RESTORE the default reading from a family-first listing, which is the whole of its job, so a caller who has DECLARED a family-first order has already supplied what it would have inferred and position decides instead. - The first draft of this entry said the scoping was by CONSTRUCTION and there was nothing to implement, on the reasoning that the rotation asks whether the word standing in the FAMILY position carries a patronymic ending and only the default order puts a name's last word there. That is false, and measuring it is what found it: under a family-first order the family slot holds the FIRST word, and a patronymic-DERIVED surname standing first carries the same ending, so such a name reached the rotation and was rotated out of the reading the caller declared. Measured 2026-09-07 before the gate, `Мицкевич Адам Юзеф` under `parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST)))` read family `Адам`, given `Юзеф`, middle `Мицкевич`, where plain FAMILY_FIRST reads family `Мицкевич`, given `Адам`, middle `Юзеф`; `Рабинович Иван Лев` read family `Иван` the same way. After the gate both read as plain FAMILY_FIRST does. The condition is on `state.order` — the order assign actually READ — rather than `policy.name_order`, for the reason decisions.md#P1's fold gives: a script_orders entry can override the policy, and the roles the rotation reads are the ones assign made. No corpus name moves at any of the four baselines, and sweeping the corpora under each pack with FAMILY_FIRST declared shows no difference either: they hold no three-word name whose FIRST word carries an East Slavic patronymic ending, and no four-word name whose FIRST word is a standalone Turkic marker (both counts 0 over the `tools/differential/corpus*.jsonl` union, measured 2026-09-07 by applying `_post_rules`'s own four patronymic patterns to the first word). The general lesson is docs/design/AGENTS.md's axis 10, re-run every capability claim — an option chosen because it needs no code is exactly where the no-code claim goes unmeasured. Two of the gate's three conditions have no possible witness and are recorded here instead: `read_order is None` cannot co-occur with a rotation shape, since assign leaves the order unrecorded only where it positioned at most one name word, and `state.order` differs from `policy.name_order` only through `script_orders`, whose scripts are East Asian and so cannot match the patronymic regexes. Unwitnessed by any test, on the precedent decisions.md#P6's 2026-08-30 #467 entry sets. - Measured 2026-09-07 with `parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST)))`. The family-first listing reads the same either way: `Сидоров Иван Петрович` gives family `Сидоров`, given `Иван`, middle `Петрович` under the default order by rotation, and the identical parse under FAMILY_FIRST by position. The divergence is on natural-order input — `Иван Петрович Сидоров` gives family `Сидоров` by rotation under the default order, and family `Иван`, given `Петрович`, middle `Сидоров` under FAMILY_FIRST. That row is ACCEPTED, not a defect: the caller said this source is written family-first and is being taken at their word. rules.md#O1 carries it as an Accepted consequence. - O2 is the same decision on the Turkic marker and its 2026-09-07 entry says so. The shapes differ, which is why the two rules state their own consequences rather than sharing one: under FAMILY_FIRST `Ali Ahmad Vali oglu` reads family `Ali` exactly as the rotation would, and it is under FAMILY_FIRST_GIVEN_LAST that it reads given `oglu`, middle `Ahmad Vali` — the marker in a name field, which is O2's existing Accepted consequence reached by a second route. Both readings are pinned in tests/v2/test_locales.py beside O1's row (measured 2026-09-07). - Where the row is pinned, and why not in rules.md. The example grammar admits ONE annotation, and `tests/v2/rules_doc.py`'s `resolve_annotation` resolves it to a policy, a locale or a gated locale — never a pair. Composite keys exist for other pairings (`family-first+middle_as_family`, `[ja+segmenter]`), but no registered annotation combines a pack with an order, so the two cannot be written on the same example line. Registering a combined policy would be a test change made inside a docs decision. #470 asked whether an example line should declare its format and was closed not-planned on 2026-09-02 — "no notation is needed. The Accepted clause is already the right home for this" — so prose in the rule is the decided home here, not a workaround. The parse is a `tests/v2/test_locales.py` row beside the pack's other rows, and rules.md#O1 states it in prose. - -Declined: - -- Option 2, suppressing the rotation and re-deriving the order from the patronymic trace under a declared family-first order — the declaration already carries the information this would recompute, and recomputing it means overriding the caller with a string-level guess, which is the principle above inverted. -- Option 3, reporting an ambiguity wherever a patronymic trace disagrees with the declared order — the report would fire on natural-order input inside a family-first source, and the caller cannot tell that from a mis-declared record, which is the only thing the report would be for. It is NOT true that it would fire on every correct family-first parse: a family-first listing CARRYING THE TRACE reads the same under both orders — the rotation fires only where the last word carries the ending, and it then makes the first word the family exactly as position does — so the trace and the position agree there and nothing is reported. Scoped deliberately, the unscoped claim being false: `Сидоров Иван Лев` is a family-first listing with no trace, and it reads family `Лев` under the ru pack's default order against family `Сидоров` under FAMILY_FIRST (measured 2026-09-07). What is left is a signal on the one shape where the declaration is doing its job, indistinguishable from the shape where the declaration is wrong. A report that cannot be filtered on is the failure decisions.md#P6's 2026-08-29 entry records for a different rule; here it is avoidable by not adding it. - -### O2 — Turkic rotation - -- 2026-07-02 (Turkic design; landed in the v2 core, PR #288) — shape fixed at exactly four name words (1 given + 2 middle + 1 marker), v1 parity; other shapes keep their positional reading even when that leaves the marker in a name field (see the rule's Accepted consequence). - -- 2026-07-02 (design, orthography coverage) — dotless ı does not case-fold to i, so qızı is a literal alternative in the pattern (re.I cannot bridge it; all-caps QIZI falls to the ASCII alternative); the patterns are NFC literals and NFD input does not match. Scoping decline: suffix-attached Kazakh/Uzbek patronymics (Әбішұлы) are bound-suffix morphology, not a standalone marker, and deliberately out. -- 2026-08-15 — same rotation/name_order interaction status as O1. -- 2026-09-07 #384 — OPTION 1, the same answer as decisions.md#O1's entry of the same date and on the same principle: the rotation restores the default reading from a family-first listing, so a declared family-first order supplies that reading instead and the rotation stands down. It stands down by the same gate on the read order, for the reason O1's entry records — the marker test reads the word in the FAMILY position, which under a family-first order is the FIRST word, so a name opening with the marker reached it. Measured 2026-09-07 before the gate, `oglu Ahmad Vali Ali` with Turkic handling and FAMILY_FIRST read family `Ahmad`; after it, family `oglu`, which is what plain FAMILY_FIRST reads and this rule's own 2026-07-02 Accepted consequence — the marker in a name field. The natural-order twin `Ali Ahmad Vali oglu` is decisions.md#O1's entry of the same date, which measures it under both family-first orders and pins it in tests/v2/test_locales.py. Options 2 and 3 are Declined for O1's reasons. - -### differential-ledger — tooling decisions (2.1.0 release arc) - -Harvested 2026-08-15 from the release-arc session; measurements are that session's, spot-checked at landing. - -- 2026-07-24 (rc1 arc, predating this section's window) — the TWO-CORPUS design: corpus_issues.jsonl (198 tracker-harvested names, 166 absent from the v1-test-bank corpus) exists because v1 test banks are structurally blind to anything 2.0 added; compare.py globs corpus*.jsonl and fails loudly on none. Its first catch was the leading-credential case ("Ph. D. John Smith" → suffix). - -- 2026-08-05 #332 — ledger field vocabulary is Role's names, not the facade's: canonicalizing to first/last would have put an eighth place naming roles differently from Role inside the durable record, and the facade vocabulary is removed at 3.0. - -Declined: - -- Policy annotations widened to input unions (2026-08-05 #334) — five documented spellings fail mypy and every one has a type-clean equivalent; widening would make every READER see a union, and reading is the commoner operation. A .pyi stub typing __init__ wide and attributes narrow was deferred as a 3.0 candidate (needs a parallel signature list with its own sync test). -- Closing the differential gate's default-policy ceiling (2026-08-04 #332) — a per-row policy in the corpus needs defined behavior for baselines that cannot construct newer policy fields; tests/v2/cases.py already covers opt-in paths per row, so the ceiling is documented instead. -- `_check_tree` as is_relative_to(REPO_ROOT) (2026-08-05 #332) — accepts .venv/, build/ and dist/ copies inside the repo; the invariant is "is the source package", so the predicate is is_relative_to(REPO_ROOT / "nameparser"). -- Empty-string probe as the ledger over-match guard (2026-08-05 - #332) — `.`, `.+`, `\b`, `[\s\S]` all decline "" and still match - every corpus name; replaced by the sentinel set (mechanisms.md#SENTINEL-SET-OVER-MATCH-CHECK). -- "Lists every role" checked against all eight fields entries (2026-08-05 #332) — a seven-role list passed while omitting _ambiguities, which below baseline 2.0 cannot enter a diff at all; the check is against V2_FIELDS. -- A seed ledger rule with name_regex and no fields (2026-08-05 - #332) — it matched every CJK-bearing name and would have - classified all 89 diffs on the first pass, exiting 0 having distinguished nothing. -- 2026-08-07 #333 — the canonical-rule selector (keyed on literal - #271/#272 slug substrings) was deleted rather than repaired, - option A of three: split "every hand copy equals the table" (the sweep's job) from "the table did not change shape without a decision" (the span-bearing test's job), so a selector break can never take the decision gate out as collateral. This retired the slug taboo an earlier mechanisms.md entry carried. -- The B7 sanctioned-extra span (pre-#332 arc) — added, then removed as a pure loss when review measured that every B7-divided name already matches through its guaranteed classified flanks, so the span's only effect was absorbing punt-volat Latin regressions ("Gal·la Marcet" probes on both sides, recorded in PR #305's history). The cleanest "sanctioned extras must be earned" precedent the ledger has. -- An issue for rotating DEFAULT_BASELINE (2026-08-07) — it recurs every release; it lives in the AGENTS.md release checklist beside the VERSION bump instead (#333 must land first). - -Excluded (ledger name_regex patterns — one over-matching rule shadows the whole ledger, since name_regex rules sort first): "", -"(?:)", ".", ".+", "\b", "[\s\S]". Enforced by the sentinel-set -check. - -### differential-ledger, the dormancy arc (2026-08, #328–#376) - -The second ledger arc; measurements are that session's, the machinery verified present at landing. - -Decisions that landed: - -- 2026-08 #328 — `[[never]]` exclusions: ledger vocabulary for - "this shape must never be explained". classify() consults - exclusions before rules, which makes them MONOTONE — an entry only ever removes a name, never moves one between rules — so an exclusion's blast radius is exactly the names it captures, independent of rule order. -- 2026-08-12 #373 — `dormant = ""`: a rule explaining nothing fails the run in both directions, with three diagnoses (reverted / shadowed by X / refused by a [[never]] entry). Always on, not behind --strict: a check nobody passes a flag to is a check that doesn't exist. -- 2026-08 #373 — `--baseline 2.0.0` joined the release checklist: that ledger's rules had no dynamic coverage at all; measured clean at 90/0, so the gap was closed rather than documented. - -Declined: - -- Ambiguity reporting on multi-matching rules (#372/#373) — measured: 28% of claimed name×role pairs already have ≥2 matching rules (732 of 2619; 432 are one pair of rules alone). A report firing on 28% of what it inspects is wallpaper. The actionable slice shipped as the "shadowed by " diagnosis, which speaks only on FULL shadowing. -- A specificity floor for fields-only rules (#372/#373) — when this was proposed, exactly one fields-only rule existed across the ledgers, naming 3 of 7 roles; a six-of-seven floor matched nothing, and nothing would reveal it vacuous. Still declined: #451 banned the SHAPE instead (the fields-only arc below), which is a different proposal and leaves no ledger with a rule for a floor to measure. -- Specificity reordering of the rule sort (#328) — measured across all 751 names, on the ledger as it then stood: width-then-regex-length moved five rule populations and sent 17 names into the generic fields-only rule, draining the CJK-specific ones. No reading of the sort produces the "exactly one label changes" originally claimed; that figure was corrected on the PR. -- Unanchoring the honorific-suffix rule to reach glued forms (#376) — it would make a future suffix regression on 김지양 classify as a recognized honorific. A confidently wrong label is worse than a catch-all's honest breadth, and the gate reads the same either way. - -### differential-ledger, the fields-only arc (2026-08-27, #451) - -The third ledger arc, and the one that ends a shape rather than adding vocabulary. Measurements are this session's, at the counts each commit records. - -Decisions that landed: - -- 2026-08-27 #451 — `validate_rules` REJECTS a rule carrying `fields` and no `name_regex`. Such a rule claims every name whose diff fits those fields, and `_CORPUS_CLAIMS` records a regexless rule's reach as the WHOLE corpus — already at its maximum, so arrivals never move it and `test_every_rule_claims_the_recorded_share_of_the_corpus` passes whether it explains 4 names or 40. The one rule with the shape drifted from the four names its prose described to 25 before #372 cut it back, and then to SEVENTEEN after — decisions.md#H1 records that 14 -> 17 step and #410 taking three back — standing at fourteen across six unrelated behaviour families when this bundle opened, every guard green throughout, absorbing two open v2.2 bugs (#342, #397) as intentional 2.0 changes while AGENTS.md names the 1.4 summary as the release log's source for Behavior Changes. Free to state only because the fourteen were given rules first; a role-count floor was the earlier proposal and stays declined (#372/#373, vacuous). -- 2026-08-27 #451 — a rule whose declared `fields` EXCEED every diff it explains is a defect in its own right, not untidiness. Measured on the catch-all: it declared `{given, family, suffix}`, and by the time this bundle's first three commits had given its `{given, family}` names rules of their own, nothing it still explained moved `given`. Note WHEN the excess appeared, because it is the whole mechanism — the declaration was earned when the rule was written and became excess as its explained set SHRANK under it, which is exactly `fix(#424)` at decisions.md#H1, where #410 narrowed a diff from three roles to two and the rule kept matching by superset with no run naming it. A rule does not have to widen to become over-declared; the diffs beneath it only have to leave. The two together are why #452 was filed — nothing counts what a rule actually explains, only whether it explains anything. -- 2026-08-27 #451 — a ledger rule may classify a reading NOBODY WANTS, provided its `issue` says so. `fix(#342)` and `fix(#397)` carry NOT WANTED in the issue text, the cause-not-consent sentence decisions.md#P5 established for the #369 regression, and a delete-when-fixed instruction the dormancy check enforces once the fix lands. -- 2026-08-27 #451 — one vocabulary per alternation, FORCED rather than chosen. `_LatinCopy` carries one `vocabulary` and `test_latin_alternations_mean_something_the_vocabulary_ships` asserts exactly one roster key per alternation, so `jr` (SUFFIX_WORDS) and `mc`/`mp` (SUFFIX_ACRONYMS) cannot share one. That constraint forces one of the three splits, and forces it only given the roster as written: `_LatinCopy.vocabulary` is a hand-supplied frozenset, so declaring `SUFFIX_WORDS | SUFFIX_ACRONYMS` would let `(jr\.?|v|mp|mc)` pass the member checks — measured. Declining to write a union vocabulary is a judgement, that a rule's alternation should name one wordlist a reader can go and check, and it is recorded here as one rather than as a constraint. The other two are forced separately and are recorded below under the declined single-rule tail: `m\.?a\.?` cannot be an alternation member at all, and the numeral alternation copies no wordlist. Four rules, three independent reasons, none of them taste. -- 2026-08-27 #451 — a ledger rule's character class must not reach a codepoint the IMPLEMENTATION does not act on, and is pinned to that. A SUBSET, not a copy: `test_the_emoji_boundary_rule_copies_the_dividing_ranges` asserts `claimed - divides == set()` and nothing in the other direction, deliberately — `_EMOJI_RANGES` has FOUR blocks and the rule copies the two astral ones, leaving the BMP pair out because no corpus name reaches them through its token anchor. Reading it as equality would instruct the next author to pre-claim the BMP half, which is the #372 shape this same section leans on. The emoji rule's first draft spanned `U+1F300-U+1FAFF` while the two ASTRAL blocks of `_tokenize.py`'s `_EMOJI_RANGES` are narrower than that span: it claimed 1072 codepoints where the parser leaves the token whole, so its own prose ("an emoji inside a token divides it") was false for every one of them and a `{given, family}` diff with another cause would have classified as intended. `test_the_emoji_boundary_rule_copies_the_dividing_ranges` holds it, and fails on the draft it replaced. -- 2026-08-27 #451 — `_sorted_rules` is KEPT although the ban makes it the identity on every ledger that loads. Verified against all three: the ban removes the only shape that could occupy its second tier, so file order settles every tie there is and the function reorders nothing. Kept because it is what makes the ban safe to state for any reader that does not call `validate_rules` first — a future tool, a REPL, a test fixture. Deleting it would move the guarantee from the code into a convention, which is the trade #451 was filed to undo. Recorded as a decision rather than as a consequence precisely so a later dead-code sweep finds the reasoning before the `return`. -- 2026-08-27 #451 — past-tense counts stay in a ledger comment; live ones do not. "It claimed 25 until #372" is a fixed fact. "Four corpus names, and the prose fits all four" was written when the rule explained four and still read that way at fourteen, because nothing recomputes it. Where a count is owned by another comment, point at the owner rather than restating it — two copies means one gets updated and the other does not. -- 2026-08-28 #452 — a rule's declared `fields` must EQUAL the union of the diffs it explains, checked by `compare.py` at the end of every run and failing it like an unexplained diff. The statement is exact rather than heuristic, which is what makes it cheap: `classify()` already requires `declared >= union` for the rule to match the names it matches, so the only possible error is the other direction, and the union is simultaneously the check and the repair. Narrowing to it cannot orphan a name, since every name a rule explains contributed to it. Measured before landing: 3 of 67 explaining rules over-declared at 1.4.0, 5 of 58 at 2.0.0, 6 of 51 at 2.1.0 — all fourteen narrowed first, so the check was silent the day it arrived. -- 2026-08-28 #452 — over-declaration is BASELINE-RELATIVE, and each ledger is measured on its own run rather than copied. `fix(#296) a lone post-comma credential is a suffix` declared `{family, given, suffix, title}` in all three ledgers: exactly exercised at 1.4.0, where v1 reads the pre-comma word as `first` and all four roles move, and over-declared at both 2.x baselines, where the same behaviour moves only `{suffix, title}` — which is all it declares in those two files now. A reader comparing the three sees one rule with TWO different field lists and should read that as correct rather than as drift. Two and not three, measured: the 2.x pair narrowed to the same set, so the split is 1.4.0 against both 2.x ledgers, not one list per file. -- 2026-08-28 #452 — NO escape hatch, decided rather than deferred. `dormant` already covers the explains-nothing case in both directions, a rule with no `fields` has nothing to over-declare, and the ledger's own doctrine — "a rule that pre-claims shapes it has never seen is the #372 failure mode" — makes strictness the existing principle. Accepted cost, stated so it is not rediscovered as a surprise: the first rule that genuinely needs a wider declaration has to argue for a key the way `dormant` was argued for in #373, rather than reaching for one that already exists. -- 2026-08-29 #456 — a rule carrying `name_regex` and no `fields` is REJECTED, the symmetric twin of #451's ban. It narrows by name and by nothing else, so on any name its regex reaches it claims every diff shape there is — measured, 255 shapes from baseline 2.0 on and 127 below it. #452 is what made it urgent rather than merely untidy: `over_declared_rules` skips a rule with no `fields`, correctly, since one declaring no roles cannot over-declare them — so deleting the line is the cheapest way to silence an OVER-DECLARED failure AND the most permissive thing that can be done to the rule, which is the #372 failure mode reached by following a gate error message. Free to enforce on #451's own terms: measured, 0 of 179 rules across the three ledgers had the shape. After it, every rule carries both keys, and the three rejections in `validate_rules` read as one rule — narrow by name and by role, or it is not a rule. NO escape hatch for a "genuinely unbounded" rule, declined until one appears, same call and same reason as #452's. - -Found rather than decided, and worth as much: - -- **A fields-only rule silently falsifies OTHER comments.** The 2.x ledgers' idiom "such a name should arrive UNEXPLAINED and be read once" holds there because no rule above the one making the promise claims that diff shape. A fields-only rule is the extreme way to break it, not the only way: measured while fixing #452, a `{family, maiden}` diff on `fix(#367)`'s own probe goes to `fix(#274)` in the 1.4.0 ledger, which has no fields-only rule anywhere. Note the shape, not the role: `{given, family, maiden}` on the same probe is claimed by nothing, because `fix(#274)`'s `fields` stop at maiden/middle/family. That is the point — the question is only ever answerable per shape. The reusable check is "does anything above me claim this shape", which is `classify()` driven directly — NOT "is there a catch-all". In the 1.4.0 ledger it was false for anything of the catch-all's shape, and nothing said so. Measured while the rule still lived, by driving `compare.classify()` directly over the seven `_MUST_NOT_MATCH` probes of the two rules #451 ported, at diff shape `{given, family}`: ZERO arrived unexplained. Four went to the catch-all ('Mcintyre Smith Jr.', 'Los Santos', 'John Sir de Mesnil', 'Sir Smith'), two to `fix(comma-precomma-family)` ('McDonald, Ronald', 'Smith, Sir de Vaux'), and one ('Ste Marie') to the neighbouring `fix(#360) ste` rule that exists to claim it. Driven through `classify()` rather than read off a gate run, deliberately: six of the seven are in no corpus, so a run would never reach them — which is also why the promise they carried could go false unnoticed. The blast radius of the shape was never just its own heading. -- **Copy-paste is how a ledger comment goes false without being edited.** All three ledgers carried the same `fix(#379)` comment; four of its sentences were true only in the 1.4.0 file they were written for, including one citing `fix(suffix-routing)`, which exists in no 2.x ledger, and one saying a name "falls to the fields-only catch-all" twenty lines after the same file says no fields-only rule exists there. Corrected in `8237b54`. A cross-ledger claim needs re-deriving per file, not copying. - -Declined: - -- A `[[never]]` for #342 and #397 (2026-08-27) — honest, and it reddens the gate: an excluded name reports UNEXPLAINED and exits non-zero, so every release would block until both bugs are fixed. Both are on v2.2, so the window is real. The cause-not-consent rule says the same thing in the ledger and keeps the gate green. -- One replacement rule with a `(jr|v|mp|x|m\.?a\.?|mc)` tail (2026-08-27) — measured, not argued: `m\.?a\.?` matches the corpus fragment `'M.A.'`, which `_normalize` leaves as `'m.a'` and is not a `SUFFIX_ACRONYMS` entry, so the member guard rejects it — the same finding the `ambiguous-surname-acronym` rule already records. `x` is in no vocabulary set at all, reaching `suffix` through the trailing-roman-numeral fork, so it could not be a member either. -- A literal alternation of all 14 absorbed names (2026-08-27), on #413's literal-anchored precedent — kept as the fallback if a reading turned out contested, and not needed: every family had an existing rule in another ledger, a decision to cite, or an open issue. -- Extending #451 to the trailing-numeral ambiguity (2026-08-27) — `'Mohamad X'` loses its family to `_ROMAN`, and by C-i's "under uncertainty, default to AMBIGUOUS" a trailing `X` IS borne as an ordinary name in that position (Malcolm X). Numerals reach `suffix` through a pattern rather than a vocabulary set, so there is no ambiguous SUBSET for `x` to join — but the fork does not go unreported: measured, `parse('Mohamad X')` and `parse('John V')` both carry `AmbiguityKind.SUFFIX_OR_NAME`, which `_assign` raises from the fork itself because "V/X/I are ordinary middle initials, so taking it as a suffix is a call, not a fact". The contrast worth keeping is with the acronym half: `parse('Aishwarya Rai')` loses its family and reports NOTHING, because `rai` is unambiguous vocabulary. So the numeral fork already flags its own uncertainty and the vocabulary path does not — which is an argument about `SUFFIX_ACRONYMS_AMBIGUOUS`, not about `_ROMAN`, and it belongs to #342 rather than here. `'Malcolm X'` is in no corpus (`'Mohamad X'` is, and is what this bundle classified); no open issue on the numeral half. -- `explains = N` per rule (2026-08-28, #452's own first sketch) — catches a rule explaining fewer NAMES even while its fields stay exercised, which the field check does not. Declined on maintenance cost rather than merit: 179 numbers across the three ledgers, every one of them liable to move whenever a corpus does rather than only when a rule does. Measured on the precedent — #414's harvest moved 17 of the 30 `_CORPUS_CLAIMS` entries that existed then and added 14 more, parsing the roster at `7a10689^` and `7a10689`. (That roster's own comment said "Twenty of these moved" and was wrong; measured three ways — that commit, the whole of PR #415, and the backtick-harvest commit after it — the figure is 17 every time, and the comment now says so with the recipe.) The field check needs no number and cannot go stale. This is the design to revisit if a name-count shrink ever bites. -- Extending it to `'Donald mc'` (2026-08-27) — `mc` is `SUFFIX_ACRONYMS` AND `PARTICLES`, the collision C-ii decided for `vd`, and C-ii reaches this shape only through P6's attachment, which is scoped to the comma form. The authority is rules.md#S2's Accepted — "an unambiguous suffix is consumed even when that leaves no family name at all" — and NOT P6's scope note, which was this entry's first citation and does not support it. P6 promises the comma-less shapes "keep their positional reading", and for the words that are both particle and suffix vocabulary it does not hold: measured, `'Jong de'` and `'Smith do'` keep theirs (family 'de', family 'do') while `'Donald mc'` and `'Smith vd'` lose the family entirely to `suffix`. So `'Donald mc'` is out of #451's scope on S2's statement, and P6's own Accepted clause is falsified by two of the three words it sweeps in — its only example, `"Jong Anke de"`, is `de`-shaped, which is why no test catches it. Recorded here as a rules.md defect rather than a ledger one, and FIXED 2026-09-07 under #454: P6's Accepted clause now states the exception, naming mc and vd as the two words the positional reading does not hold for. The measurement above is unchanged and is what the repaired clause says. See #suffix-acronym-collisions. - -### differential-ledger, the corpus-tier arc (2026-09-01, #468/#469) - -The fourth ledger arc, and the first to change what the gate is FOR rather than what a rule may say. Mechanics — the tier table, the promotion recipe, the shape key, the `corpora:` line — are owned by tools/differential/README.md and not restated here; these are the decisions behind them. Counts are this session's, each recomputable by running `uv run python tools/differential/compare.py` (add `--baseline 1.4.0` for the skip figures) over the checked-in corpora and reading its `corpora:` and `corpus:` lines. - -Decisions that landed: - -- 2026-09-01 #468 — every corpus carries a TIER, and only the CONTRACT tier can fail the run. `corpus.jsonl` (scraped from v1's test banks) and `corpus_issues.jsonl` (harvested from the tracker) became RADAR: their diffs still classify against the ledger, so release notes keep their grouping, but an unmatched one prints under `UNCLASSIFIED (radar)` and the run still exits 0. `corpus_cjk.jsonl`, `corpus_rules.jsonl` and `corpus_shapes.jsonl` keep the full discipline. The reason is a VOCABULARY gap and not a volume one: the gate demanded a rule-precision explanation for names nobody chose, and the honest explanation for many of them was "this string exercises nothing" — which no ledger key can say: a rule means "this diff is intended and here is why", an exclusion means "this shape must never be explained", and neither is the sentence wanted. A rule invented to say it would be the #372 shape by construction, since the only thing it could narrow on is which file the name happens to sit in. `_CORPUS_TIERS` in compare.py is the roster and is fail-closed like `_CORPUS_FLOORS`: a corpus added without a tier is a hard error, so the split forces a decision when a corpus arrives rather than defaulting one. The tier rule is about UNMATCHED diffs, and two recorded things outrank it on a radar name — a `[[never]]` exclusion (two bullets below) and a contest row in `_RECORDED_DIFFS` (the watched-shapes arc, 2026-09-05, which states the rule and its ground); a watched row that fails does so on an entry whose tier reads contract — the default-order entry when the name has one, the first-loaded otherwise — which is this rule applied and not an exception to it. -- 2026-09-01 #468 — nothing is DELETED to quiet the gate; promotion is the answer to "then how does a radar name ever bind again". A name enters the contract by being CHOSEN — a tests/v2/cases.py row plus a shape tag, which puts it in `corpus_shapes.jsonl` at the next regeneration. Deleting the string was the alternative and is worse in both directions: it loses the radar signal, and it makes the gate's own population editable by whoever finds the gate inconvenient, which is the one edit a differential gate must not make cheap. A meaningless radar string costs one parse and one report line against a comparison of 1120 names today (1113 when this bullet was written, hours earlier the same day; #486's bullet below carries the arithmetic). -- 2026-09-01 (decided in review) — a `[[never]]` exclusion OUTRANKS the tier: a diff it refuses reports UNEXPLAINED and exits 1 on either tier. Not a carve-out but the tier's own principle applied — the split governs names nobody looked at, and an exclusion is chosen exactly as a rule is, someone having written its `why` and its `examples`. It is also what the shipped ledgers already promise, and the measurement is what settled it: the 1.4.0 ledger's trailing-`Ph. D.` exclusion, whose `why` says "a diff here is a regression", reaches exactly three corpus names (`John Smith Ph. D.`, `John Smith, Ph. D.`, `John Smith, Ph.D.`) and every one of them lives in `corpus.jsonl` or `corpus_issues.jsonl` alone. Reading the tier as outranking the exclusion would have retired that entry outright, silently, in the commit that split the corpora. Recompute by matching each `[[never]]` entry's `name_regex` over the corpus files and reading which files the matches come from. -- 2026-09-01 (decided in review) — a ledger RULE may narrow by comparison order, through a new `orders` key; it is optional, and its absence is the order-blind behavior every rule written before shape-tagged entries has. Forced by the first order-scoped rule anyone wrote: `feat(#395)`'s fold on `de la Cruz Juan Carlos` moves `{family, given, middle}` under the two family-first orders, and the most plausible regression of that very change — the fold leaking into the DEFAULT order — moves those same three roles on that same string, which `corpus_rules.jsonl` carries as a bare string and therefore compares. Order-blind, the rule describing the feature would have absorbed the regression of the feature and called it intentional: #372's failure mode reached from a new direction, and on the one name best placed to hide it. The legal set is BORROWED from shapes.py rather than hand-copied (`_legal_orders`), the same call build_cjk_corpus.py makes for the script table — an order no shape declares is an order no comparison runs under, so a rule scoped to a typo could only ever be dormant. Amended 2026-09-01 (decided in review): the legal set gains a `DEFAULT` SENTINEL, and the two legacy rules that needed it were scoped. Found when the reviewer traced the absorption running the OTHER way from the one above — the order-blind `fix(#399) a maiden marker bounds the particle chain that swallowed it` rule reaches `de la Cruz née Vega`, which `corpus_shapes.jsonl` compares under both family-first orders, and it sorts AHEAD of the two rules written for that name, so a family-first-only `{family, maiden}` regression there would have classified as intentional. The fix a rule wants is `orders`, and no rule could say "default order only": the members are constant NAMES, the default order is the absence of one, and TOML has no null inside an array. So `"DEFAULT"` names it. `fix(#399) …that swallowed it` took `["DEFAULT"]` in both 2.x ledgers and `fix(#296) … 'John Smith, Dr.' keeps its split and its title` took `["DEFAULT", "FAMILY_FIRST"]`, those being the comparisons each one's diffs actually come from. Order-blind rules stay legal — every ledger written before shape tags is full of them — so the absorption is made VISIBLE rather than illegal: a run prints an `ORDER-BLIND` block for each (issue, name, order) where a rule with no `orders` key explained an order-bearing diff, informational and outside the exit code. Recompute by running the gate at each of the four baselines and reading that block; it is empty at all four today. -- 2026-09-01 (decided in review) — EXCLUSIONS stay order-blind, and `validate_exclusions` rejects an `orders` key as unknown. The dormancy arc's monotone-refusal reasoning is the whole argument: an exclusion only ever removes a name from classification, so the widest thing an over-wide one can do is make a name report UNEXPLAINED and fail the run. There is no silent direction to protect against, and the narrowing a rule needs is the one an exclusion must not have — "never explain this, but only under FAMILY_FIRST" is a promise with a hole in it. -- 2026-09-01 #469 — "some names are only supported under one order" is answered by the SHAPE rather than by a ledger exception. shapes.py records, per input shape, the `name_order` the shape is an input FOR and the oldest baseline whose worker can honor that order; `compare.py` parses a tagged entry under that order on both sides, compares it on the v2 surface alone (the facade is the v1-compat surface, and a family-first name is not a v1 contract), and leaves out an entry whose run predates its `min_baseline` — reported as `skipped N names tagged shape(s) [...]` and counted in the `corpora:` line, so a shrunken comparison is never silent. Structural, and that is the point: nothing has to be written into any ledger to say a family-first name is not a default-order promise. Today's arithmetic, all of it recomputable from the checked-in files and one gate run: 13 shape entries, of which 4 dedupe into names already compared under the default order, for 1113 comparisons; shapes 4 and 5 declare `min_baseline` 2.0.0, so 7 entries are skipped at `--baseline 1.4.0` and 1106 names compare there. -- 2026-09-01 #486 (later the same day, so read this bullet as superseding the arithmetic in the one above rather than the two figures disagreeing) — the shape 1-3 variation matrix filled the given-first half of the inventory, which the feature work that authored shapes 4 and 5 had left at whatever the pipeline PRs happened to tag. 14 existing case rows gained a tag and 3 rows were authored for slots no row instantiated, taking `corpus_shapes.jsonl` from 13 entries to 30 and shapes 1/2/3 from 1/4/1 names to 9/11/3. The arithmetic moves with it and the recipe is unchanged — run `uv run python tools/differential/compare.py` and read its `corpora:` and `corpus:` lines, adding `--baseline 1.4.0` for the skip figures: 30 shape entries, of which 14 dedupe into names already compared under the default order, for 1120 comparisons; shapes 4 and 5 still declare `min_baseline` 2.0.0, so 7 entries are skipped at `--baseline 1.4.0` and 1113 names compare there. The dedupe count is the half worth reading, because it is what the tier promotion looks like from this side: 8 of the newly tagged names were in a RADAR corpus only, so the tag moved them into the contract tier without adding a comparison, and 2 more were already contract through the rules corpus. None of the 17 names #486 tagged or authored needed a ledger rule — the seven of them that were in no corpus at all diff at no baseline, and intentional stayed 226/205/113/0 across the change. Not a claim about all 30 entries: the seven shape 4/5 entries are classified by ledger rules today, and scoping exactly those rules is what the `orders` bullet three above is about. -- 2026-09-01 #469 — the CJK arrangement is deliberately OUTSIDE the shape inventory. Whether an unspaced CJK name is a third family-first shape is the open question on #469, and `corpus_cjk.jsonl` — contract-tier already, generated from the case table already — covers that ground meanwhile. Tagging it in would have to ANSWER the question first, since a shape row cannot be written without a `name_order` and a `min_baseline` for it, and the inventory is a table of arrangements rather than where that argument belongs. - -- 2026-09-01 #469 (later the same day, so read this bullet as SUPERSEDING the one above rather than the two disagreeing) — the CJK arrangement is IN the inventory after all, as shapes 6 and 7, and the bullet above is right about why it could not be until the question was answered: a shape row cannot be written without settling what it admits. What settled it was splitting the question the bullet treats as one. There is no single "unspaced CJK name" shape — there is a pure family-first arrangement (6) and a source-order transcription listing (7), and the composed forms are not arrangements at all. Both take `order=None`, which is the part that had looked impossible: the family-first reading is SCRIPT-carried rather than declared, so a pure shape 6/7 string already parses correctly under the DEFAULT policy and there is nothing for `order` to name. `min_baseline` is then documentary rather than a skip trigger, an asymmetry with shapes 4/5 that tools/differential/shapes.py's docstring states so it reads as designed. The doctrine half is its own entry (cjk-comma-demotion above); what belongs here is that `corpus_cjk.jsonl` no longer "covers that ground meanwhile" — the ground is split between it and a radar file now. -- 2026-09-01 #469 — the `tolerated` flag on a case row is the demotion's VEHICLE, and it is recorded here rather than as a mechanisms.md entry. It is an explicit declaration, reviewed one row at a time exactly as a `shape=` tag is, mutually exclusive with one, and restricted to rows whose text bears a classified codepoint; `build_cjk_corpus.py` partitions on it, and a text marked on one row and not another is a hard error, the flag being per-text in effect. Why not a mechanisms entry: that catalog is keyed by RECURRING problem shape and its contract statements are citable verbatim from code comments, and this is one arc's device with no second instance and no comment needing to cite it — the tier mechanics it rides on are already owned by tools/differential/README.md, so an entry would restate a source rather than say something the source does not (the #473 lesson under decisions.md#review-agent-single-source). If a second arc ever needs "record the behavior, promise nothing", that is the point to promote it and this bullet is the first instance to cite. - -### differential-ledger, the initials view (2026-09-01, #484/#462) - -The fifth ledger arc: the gate gains a column it never had. `initials()` is a derived render view, so a change in it with every role identical was invisible to the seven-field comparison by construction — the `middle_as_family_fold_order` row's `notes` in `tests/v2/cases.py` say as much, and #408 moved initials for a whole minor with the gate green. Mechanics (which rows carry `_initials`, the roles-identical guard, the one-field rule constraint) are owned by tools/differential/README.md; these are the decisions. - -- 2026-09-01 #484 — `_initials` is a pseudo-field on BOTH surfaces, the facade's at every baseline (1.4.0 has `initials()`) and the core's from 2.0 on, and it enters a name's diff ONLY when the seven roles and `_ambiguities` agree on every compared surface. The alternative — strict subset semantics, as `_ambiguities` has — was measured and rejected: names whose roles AND initials both move are claimed by 53 of the 72 explaining rules at 1.4.0, 43 of 66 at 2.0.0 and 36 of 59 at 2.1.0 (recompute below), every one of which would have had to grow `_initials`, and a rule declaring it absorbs any initials change on its names anyway, so the churn bought no discrimination. A role move drags its initials with it; the pseudo-field is a render-layer instrument, not a second copy of the role comparison. Consequences a rule author must know: `fields = ["_initials"]` is a rule of its own kind and lists nothing else — validate_rules refuses the mix as dead; and a count taken from the `_initials` summary is a FLOOR on initials movement, not the population (AGENTS.md carries the 83 / 28 / 55 measurement). `family_base` was considered as a second derived view and NOT added (YAGNI; adding one is the same two rows per surface, recorded here rather than on the issue). -- WHAT IT FOUND, measured 2026-09-01 by patching compare.py's own worker to emit `initials()` and comparing against the tree — and measured on THIS BRANCH'S BASE, before `31623d9` fixed #462, which is the tree the parenthetical describes: there the tree matched 2.2.0 exactly, 0 diffs of any kind. Three clusters of initials-only names, 140 at 1.4.0 and 28 at each of 2.0.0 and 2.1.0. On the SHIPPED tree the same recipe gives 126 at 1.4.0 and 42 / 42 / 14 at 2.0.0 / 2.1.0 / 2.2.0 (re-measured 2026-09-02; the gate itself is the recipe now, see RECOMPUTE). The two readings reconcile through cluster (ii) alone — 126 + 14 = 140, 28 + 14 = 42, 0 + 14 = 14 — the 14 #462 names having moved from the 1.4.0 side to the 2.x side when the fix landed. (i) The R2 readmission, 27 names at 1.4.0, 2.0.0 and 2.1.0 — `Anh Do` → `A. D.`, `Juan van der` → `J. v. d.` — shipped in 2.2.0 with a release note and the R2 entry above but no ledger rule, because none could see it; now `fix(#385/#402)`, a literal list of the 27 (the shape "a part of nothing but particles" is no property of the raw string), plus `de los Santos` under its own `fix(#360)` rule since ITS initials moved when `los` became a particle, not with R2. (ii) The facade dropping a middle `E`/`Y` initial, 14 names at 1.4.0 only, all but one radar tier — the population beside the count, per docs/design/AGENTS.md's population rule, is the names carrying such a letter in the MIDDLE or FAMILY group, which is the only position the bug reached, and there are 14 of them: 14 of 14 in that position moved. The `fix(#462)` rule's regex (`(?:^|[\s,])(?:[EY]|[EeYy]\.)(?=[\s,]|$)`) reaches 18 corpus names, four more than the population, and the other four (`E Anne D`, `E Jones`, `E Maria`, `Y. L.`) carry the letter in the GIVEN group, which has always initialed every word it holds, so nothing moved. The discriminator is the GROUP and not the position: `E Anne D,Leonardo` leads with the E too, but its comma re-roles the run into the family group, and it is one of the 14 that move: #462, a 1.4 parity break shipped in 2.0.0 and recorded nowhere. Fixed in the same PR (R3's bullet below); the 14 then move against the three 2.x baselines instead and carry a `fix(#462)` rule in each. (iii) Per-word grouping, 98 names at 1.4.0 only: 1.4.0 initialed one group per `*_list` element and rendered a multi-word element's letters with the separator and no delimiter — `Juan Velasquez y Garcia` → `J. V G.` at 1.4.0 and `J. V. G.` here, `Abdul Salam Hassan` → `A S. H.` and `A. S. H.` — and the 2.0 facade initials each word. Measure with the FULL name, not with the run alone: `Velasquez y Garcia` and `Abdul Salam` as whole inputs give `V. G.` and `A. S.` at 1.4.0 and here alike, the run being the family and initialed per word either way, so the bare forms this bullet illustrated with until 2026-09-02 reproduced nothing. Strip periods and spaces and all 98 agree with 1.4.0 letter for letter. Changed in 2.0.0, mentioned once in passing (the #408 bullet under R3: "v1's initials GRANULARITY"), never classified. Classified LATE, on purpose: mechanisms.md#FACADE-CONTRACT promises 1.4-warning-free code keeps working except for release-log-classified fixes, and a classification three minors overdue is still owed, so the 2.3.0 log carries a bullet that says the behavior has held since 2.0.0. Four rules rather than one, because the Latin-alternation guard keys a rule to a single copied vocabulary and the 98 copy three (CONJUNCTIONS 66, BOUND_GIVEN_NAMES 19, PARTICLES 11) plus the `Ph. D.` merge (2). The connective rule cannot claim cluster (ii), and it takes BOTH of its exclusions — the case-sensitivity and the trailing whitespace lookahead — to keep it out. Of the 14, 7 carry a bare capital `E` and 7 a dotted `E.`/`e.`: the case-sensitivity excludes the bare capitals, and the alternation's trailing whitespace lookahead excludes the dotted ones whatever their case, a period not being whitespace. Since the fix in `31623d9` those names agree with 1.4.0 and diff only against the 2.x baselines, so neither half is keeping two live clusters apart today. What they do is make any FUTURE facade initials change on these shapes surface as UNEXPLAINED at 1.4.0 rather than be absorbed here as per-word grouping. Gate totals moved from 226 / 205 / 113 / 0 intentional to 352 / 247 / 155 / 14, zero unexplained, exactly the new rules' explained counts. -- RECOMPUTE: load `tools/differential/compare.py` by path; build the entries as main() does (`_load_entries`, `_load_shapes`, `_CORPUS_TIERS`, dedup by (name, order), drop order-bearing entries below their shape's minimum); patch `_worker_source` to append `_initials` to both rows; `_run_worker`; compare against the tree's facade and core. TWO RECIPES, and only the second of them needs that patch. (a) The CLASSIFIED TOTAL per baseline is read off the gate itself, no patch and no in-memory run: `compare.py --baseline X` prints a `## issue (N)` heading per rule, and the total is the sum of the headings of the rules whose `fields = ["_initials"]` — at 1.4.0 `fix(#385/#402)` 27, `fix(#360) los` 1 and the four `fix(initials-per-word)` rules 19 / 66 / 11 / 2, summing to 126; at 2.0.0 and at 2.1.0 `fix(#385/#402)` 27, `fix(#360) los` 1 and `fix(#462)` 14, summing to 42; at 2.2.0 `fix(#462)` alone, 14. What the gate does NOT print is the per-name detail — a heading lists at most ten of its names and no diff fields at all, and the UNEXPLAINED and radar blocks are empty on a green tree — so it answers how many and under which rule, never which names or which surface moved. (b) The PER-NAME SPLIT, and the pre-fix 140 / 28, need the in-memory run described above: load compare.py by path, run main()'s comparison loop or hook `diffing`, and keep the names whose diff is exactly `{"_initials"}`. READ THE PATCH STEP AS HISTORY, and it belongs to (b) alone: since `a748862` the shipped worker template emits `_initials` itself, so patching `_worker_source` is what a measurement BELOW that commit needs and re-applying it on this tree changes nothing. The pre-fix 140 / 28 must be taken on the branch's base, before `31623d9`, rather than on this tree. Split "initials only" from "roles and initials"; within the former, lowercase and strip `[.\s]` to separate per-word grouping (equal after stripping) from content changes. The guard's liveness is proved by mutation, not by the green run: with `not diff and` removed from the guard, a roles-and-initials name has its role diff REPLACED by `{_initials}`, which no existing rule declares, and the 1.4.0 run reports 117 of them as no longer classified — 79 unexplained plus 38 radar, from 0 and 0. The same mutation is where the first bullet's churn figures come from: the rules that lose at least one name are the rules strict subset semantics would have had to widen, counted against the rules that explain anything at that baseline (53 of 72, 43 of 66, 36 of 59 at 1.4.0 / 2.0.0 / 2.1.0; re-derived 2026-09-02, the 1.4.0 numerator having been recorded as 43 until then). - -### differential-ledger, the rule-order arc (2026-09-02, #382) - -The sixth ledger arc, and the one that closes the question mechanisms.md#LEDGER-RULE-SEPARATION has carried open since #375's reorder mutation: file order settles every contest, and nothing said which contests existed. Mechanics — the TOML shape, the required `why`, the trap that a rule key written below the block joins the exemption — are owned by tools/differential/README.md; these are the decisions. Every figure below was measured 2026-09-02 against the real 1.4.0 wheel or recomputed from the checked-in files, and each carries its recipe. - -Decisions that landed: - -- 2026-09-02 #382 — an order-decided contest must be DECLARED, and narrow-first is the declaration-free default. A pair is a contest when the later rule's `fields` are a strict subset of the earlier one's, some corpus name's `name_regex` reaches both, and some comparison order reaches both; where the EARLIER rule is the wider one it carries a `precedes_narrower` block naming the later rule and saying why it is the classifier of record — usually what it describes that the later one does not, and in the three regex-accident pairs (`fix(#296)`/jr, and the two `fix(cjk-glued-honorific-peel)`/`fix(suffix-routing)` pairs) that the later rule reaches the name through its REGEX rather than by describing it — both `fix(suffix-routing)` patterns are anchored at both ends and open on a bare `\S+` run, which swallows the trailing comma of `Smith,` and matches hangul as readily as Latin, while the prose they carry is scoped to a comma-less two-token Latin name. Only ONE of the three is false outright — `fix(#296)`/jr, where 1.4.0 read `'Smith, Jr.'` as title `'Jr.'`, first `'Smith'`, so the family→suffix movement the jr rule describes never happens on it. The other two are PARTIAL, and symmetrically so: on `김민준씨 Jr.` and on `田中さん II` alike the trailing token really does leave `family` for `suffix` (1.4.0 first `'田中さん'`, last `'II'` → tree last `'田中'`, suffix `'さん, II'`, exactly the movement the numeral rule shows on its own `John V`), and what neither `fix(suffix-routing)` rule can describe is the peel and the segmentation that move `given`. Recomputed 2026-09-02 against the 1.4.0 wheel; the exemptions themselves claim only the scope mismatch, so do not read a falsity claim into the two partial ones. `compare.py` refuses the run, before the worker spawns, where such a pair goes undeclared. Reordering is NOT the alternative fix and the failure message says so: it moves which rule classifies a name and breaks `_CROSS_RULE_WINNERS`. What the check buys is that nothing else in the suite can see the hazard at all: `_CORPUS_CLAIMS` measures each rule alone, the gate total is per-corpus, and `_CROSS_RULE_WINNERS` pins contested outcomes only for names somebody hand-added. -- 2026-09-02 #382 — the check covers NESTED pairs, and nesting is sufficient for an order-decided contest rather than necessary. Two rules whose `fields` merely INTERSECT both admit any diff inside that intersection, so `classify()` hands such a name to whichever is written first exactly as it does for a nested pair, and `order_contests` cannot see it. That class sits outside the check by the reasoning `order_contests`' docstring already states for EQUAL `fields` — neither rule is narrower, so "narrow-first" says nothing about the pair, `precedes_narrower` has no narrower rule to name, and `_CROSS_RULE_WINNERS` stays the instrument — which covers every pair where neither `fields` set contains the other, equal `fields` being its special case. Not oversight, and not free either: measured 2026-09-02 over the four ledgers, counting pairs that share a corpus name and whose `orders` are not disjoint, 11 are strictly nested wide-first (what the check refuses), 40 nest in either direction, 11 have equal `fields`, and 111 have any non-empty intersection — so 60 overlap without nesting or equality. Widening to the general predicate would demand 111 written justifications where the real number is eleven, the same kind of answer this entry already gives below about the 657 figure: a predicate whose roster nobody can write out one justification at a time is not a usable predicate, whichever condition inflated it. The blind spot is worked and filed as [#498](https://github.com/derek73/python-nameparser/issues/498): `fix(#271/#272/#298)` (`{family, given, middle}`) and `fix(cjk-delimited-nickname)` (`{family, given, nickname}`) intersect in `{family, given}` without nesting, three contract-tier corpus names (`マイケル・ジャクソン`, `威廉・莎士比亚`, `高橋・一郎`) diff exactly those two roles against the 1.4.0 wheel, swapping the two rules reattributes all three, none is in `_CROSS_RULE_WINNERS`, and `order_contests` lists the pair in neither arrangement. RECOMPUTE the five figures with `_rule_reach` per ledger over the `corpus*.jsonl` union, classifying each pair by whether `a.fields & b.fields`, `b.fields < a.fields`, `a.fields < b.fields` or `a.fields == b.fields`; #498's body carries the loop. Closed 2026-09-05 by #498, which did not widen this predicate: the class is now covered at RUN time, where a measured diff makes the population 14 rather than the 71 pairs of the equal-or-overlap class (the 60 counted here plus the 11 with equal `fields`), and every name in it is pinned — the three named here among them, so read that "none is in `_CROSS_RULE_WINNERS`" as the state this measurement found (the #498 completeness check below). -- 2026-09-02 #382 — an ESCAPE HATCH here, where #452's and #456's were declined, and on the terms #452 set. Both of those bans were free to state: measured at the time, 0 of the 179 rules across the three ledgers then on disk had #456's shape, and #452's fourteen over-declarations (3 of 67 EXPLAINING rules at 1.4.0, 5 of 58 at 2.0.0, 6 of 51 at 2.1.0 — a different and smaller population than #456's 179, which counts every rule) were all narrowed before the check landed, so neither ban had to argue with a rule that was correct as written. #452's entry states the price of that strictness — "the first rule that genuinely needs a wider declaration has to argue for a key the way `dormant` was argued for in #373". Read that as the PROCEDURE it sets, not as a hatch this key opens: `precedes_narrower` is not a wider `fields` declaration and does nothing for an over-declared rule, which still exits the run non-zero. What carries over is the standard of proof, and here it is measured rather than asserted: eleven pairs in `expected_since_1.4.0.toml` are wide-first and every one of them is correct where it sits, so a ban would have had eleven rules to reorder or eleven prose descriptions to falsify. The hatch is narrowed the way `dormant` is: it names ONE rule (a blanket opt-out would be inherited by every narrower rule added later, which is the widening the check exists to refuse), the `why` is required, and a declaration standing over a pair that is no longer contested is refused as loudly as an undeclared contest. -- 2026-09-02 #382 (decided in review) — the vacancy half REFUSES only on a full run and prints a NOTE under `--corpus`. The two checks are not symmetric under a narrowed name set, and the asymmetry is the whole reason: narrowing removes contests, so for the undeclared check `--corpus` is only ever more lenient (it can only under-report, never false-alarm — do not call that fail-closed, which this repo uses for the rosters that REFUSE on a missing entry), while for the vacancy check it INVERTS — a live declaration whose contested names all sit outside the subset reads as vacant. Shipped as a regression and caught in review: as first written, a `--corpus` run against the 1.4 ledger exited 1 and told the contributor to delete exemptions the full gate needs — measured, each of the six corpora run ALONE reports vacancies, 11 of the 11 for `corpus.jsonl`, `corpus_cjk.jsonl` and `corpus_shapes.jsonl`, and 8 / 7 / 5 for the other three — after which deleting them would have made the full run refuse with that many undeclared contests. Read the shape and not the digits: the number varies with the subset, and only zero would have been safe. The file already treats `--corpus` differently in two places and this is the third, so the three should be read together rather than made uniform: the corpus-floor roster is SKIPPED entirely under the flag, `over_declared_rules` still FAILS the run and appends a NOTE saying the union it computed is over a subset, and the vacancy check does not fail at all. The strengths differ because the error directions do — only the vacancy check inverts under narrowing. -- 2026-09-02 #382 — TWO name populations, deliberately, rather than one shared function. `main()` must check the corpus it ACTUALLY compares, because `--corpus` narrows it; the unit guard in tests/v2/test_ledger_guards.py must check every corpus on disk, so that a rule added by a later bundle is checked at pytest speed with no baseline wheel. Forcing one function would break `--corpus`. They agree by construction instead: `_entry_name` in tests/v2/_differential_fixtures.py says in its docstring that it mirrors `compare.py`'s `_load_entries`, and both read the same `corpus*.jsonl` glob. -- 2026-09-02 #382 — the recorded negative control is deliberately BLIND to `precedes_narrower`. `_ORDER_EXEMPTION_EFFECT` records the eleven pairs and their name counts from `order_contests`, which never reads a declaration; a control that consulted the mechanism it controls for would measure nothing, and would go green the moment the predicate stopped finding anything at all. Its assertion that not every ledger's list is empty is there for exactly that failure (mechanisms.md#RECORDED-ROSTERS). - -Found rather than decided, and worth as much: - -- **#382 option 3's premise is FALSE, and the crux name is in the corpus.** The issue proposed a mechanical narrow-first check — refuse the wide-first pair, or sort by specificity. `fields`-subset is a proxy for specificity, and it is the wrong one where a wider rule describes a COMPOUND behavior its component rule does not. `马丁·路德·金씨` divides on the nakaguro AND peels its glued hangul honorific: `fix(#272/#308) nakaguro division and a glued hangul honorific in one name` describes what happens to it, `fix(cjk-glued-honorific-peel) glued honorific peels into suffix` describes half of it, and the WIDER rule wins by position, correctly. Narrow-first would have reattributed the name to the rule describing half — #372's defect reintroduced by the check meant to prevent it. The name is contract tier (`corpus_rules.jsonl`) and appears nowhere in `_CROSS_RULE_WINNERS`, so it is exactly the crux #382 was filed over and exactly the name no existing guard was watching. -- **The predicate needed a THIRD key, found in review.** The first implementation read `name_regex` and `fields`. `_entry_matches` narrows by `orders` as well: two rules declaring disjoint `orders` never see the same comparison, so file order decides nothing between them however nested their `fields` are. Omitting it made the detector read different boundaries from the predicate it models — docs/design/AGENTS.md axis 2 — and would have demanded a written justification for a hazard that cannot occur. It changes no figure today and is kept for correctness, not for its yield: measured over the four ledgers, adding the `orders` test removes 2 of 1350 nested pairs and 0 of the 657 wide-first ones. The nearest live shape is in both 2.x ledgers, where `fix(#399) a maiden marker bounds the particle chain that swallowed it` (`orders = ["DEFAULT"]`) and `fix(#399)/feat(#395) a consumed maiden marker leaves the family-first fold no given name` (`["FAMILY_FIRST", "FAMILY_FIRST_GIVEN_LAST"]`) have nested `fields` and share a corpus name, `de la Cruz née Vega` — narrow-first today, so nothing reports it either way; invert that nesting and the omission would have demanded a `why` for a comparison that never happens. -- **Nine of the eleven pairs are LATENT, not live**, which is the honest statement of what a static predicate costs and what it buys. Measured against the 1.4.0 wheel, a pair is a live order-decided contest only where some co-matched name's ACTUAL diff is a subset of the narrower rule's `fields`. In nine of eleven the real diff needs a role the narrower rule does not declare, so that rule is ineligible for those names wherever it sits. Only two pairs are live: `fix(comma-family)`/`fix(comma-precomma-family)`, where `John Smith, Mr.` is the one such name, and the compound/peel pair below, where 15 of the 17 co-matched names have such a diff and nine of those are this pair's to decide — the other six go to rules written above both, so the two counts answer different questions and neither is derivable from the other. So the predicate OVER-REPORTS relative to the measured diffs, and the price of that is eleven reasons somebody had to write. What the nine buy is the hazard that would ACTIVATE if a rule's `fields` ever widened — written down before the three later bundles add rules, which is when it is cheap. -- **A whole error class in the first draft of the exemption prose: concluding a RULE's tier from its NAMES'.** Three separate `why` texts did it. Tier is a property of the corpus file a name comes from, and a rule's reach usually spans both tiers. Measured: `fix(cjk-glued-honorific-peel)` explains 17 names, 14 of them CONTRACT tier — say the tier and not the file, even though all 14 happen to sit in `corpus_cjk.jsonl` today: the tier is what the gate acts on and `_CORPUS_TIERS` is where to read it, while a name can join or leave a file without changing either — and deleting the rule sends 12 contract-tier names to UNEXPLAINED (two of the fourteen re-home to `fix(cjk-honorific-suffix)`; the other three of the seventeen are radar) — so the gate does demand that rule, whatever the tier of the names it is contested over. Only `fix(cjk-comma-compound)` is radar-only in the strong sense: 11 explained names, all radar, and its whole 23-name regex reach radar too. Worth recording as a caution and not only as a corrected fact — the review rounds on this prose found six false claims, then one, then two more of this class. Every one was in prose nothing can machine-check, and MOST were caught by re-running the wheel rather than by reading — but not all, which is the part worth keeping: of the first round's six, two came only from reading, one citing "a few rules below" for a quote 52 rules away and one opening "the one live pair of the eleven" while a second exemption in the same file declared itself live too. Distance-in-the-file and prose contradicting prose are the classes no wheel run can reach. -- **One exemption says "least wrong reading available", and that is a finding about the ledger's vocabulary.** `fix(cjk-comma-compound)` outranks `fix(cjk-glued-honorific-peel)` on nine names — 17 reach both regexes, 15 have a diff the peel rule's `{family, given, suffix}` admits, and 6 of those 15 are taken by rules written above both — and its label is untrue of three of the nine. `王先生, V.`, `田中さん, V.` and `김민준씨, V.` show no comma routing and no order flip; their whole diff is `{family, suffix}`, the glued peel alone, against `{family, given, suffix}` on the six genuine compounds. They land on the compound rule because its criterion is "the diff includes `family`", and `family` moves only because the peel took the honorific off it. The order must still stay: the peel rule's prose disclaims comma names, so promoting it would mislabel the six. The rule that would describe them is a family-side twin of `fix(cjk-comma-honorific-peel)`, which covers this shape for a POST-comma given name; writing one was weighed as [#496](https://github.com/derek73/python-nameparser/issues/496) and DECLINED (2026-09-02), and the Declined entry below carries the evidence, the #495 tension and the reopen trigger. Declining it discards no part of this finding: the compound rule's label is still wider than those three names are, which is what this bullet and the exemption's own `why` exist to say. -- **A guard can pin the winner of a contest and still let the recorded diff shape be wrong.** `test_the_recorded_rule_still_wins_each_contested_name` feeds `classify()` the RECORDED shape and never checks that shape against a real comparison, so a wrong shape that still routes to the same rule passes forever. This branch fixed one: `田中さん II` was recorded as diffing `{given, suffix}` and measures `{family, given, suffix}` against the 1.4.0 wheel, and the claim had been copied to four sites. The winner did not move, so every argument resting on it survived — which is why nothing noticed. Filed with the general shape as [#497](https://github.com/derek73/python-nameparser/issues/497). - -The measurement, and how to redo it. Eleven wide-first contests in `expected_since_1.4.0.toml`; 0 in each of the three 2.x ledgers. Six of the eleven are contested over at least one contract-tier name, five only over radar names — the division [#495](https://github.com/derek73/python-nameparser/issues/495) argues from, and it survives a name changing tier even though the two counts do not. The load-bearing contrast is between that eleven and what `fields`-subset alone says: dropped to nesting alone, with no corpus-reach and no `orders` condition, the 1.4 ledger holds 646 nested pairs of which 367 are wide-first, against the eleven contests it actually has — and across all four ledgers, 1350 nested pairs of which 657 are wide-first. Read the ratio and not the digits: the two answers differ by more than an order of magnitude and would still differ by one after any plausible drift, which is the argument — `fields`-subset alone is not a usable predicate, and it is the corpus-reach condition that makes the check something a person can answer eleven times. RECOMPUTE: load `tools/differential/compare.py` by path and call `order_contests(rules, names)` per ledger, with `names` the union of `_load_entries` over the `corpus*.jsonl` glob — 1116 distinct names today, 326 of them reached by a corpus `_CORPUS_TIERS` marks contract. READ THAT ROSTER, do not reconstruct it from a file's name or size: `corpus.jsonl` is the LARGEST corpus at 486 distinct names and it is RADAR, having been demoted by #468 as a v1-test-bank scrape, so the contract tier is the three SMALL files (`corpus_cjk.jsonl` 73, `corpus_rules.jsonl` 248, `corpus_shapes.jsonl` 35) and the split runs 326 / 790 rather than the other way about. Reading `corpus.jsonl` as contract gives exactly 786 / 330, which is the wrong answer this figure attracts — it was measured that way twice before a third recompute caught it, and the gate's own `corpora:` line is the cheap check, since it prints the contract files first; for the contrast, re-run the same combinations keeping only the strict-`fields`-subset test. `undeclared_contests` and `vacant_exemptions` both return empty over every ledger, which is what the unit guard asserts. Note WHICH population each caller reads: `main()` checks the entries the run loaded — 1120 at every baseline, of which 1113 compare at 1.4.0 once the shape-minimum skip runs, since that skip happens after the check — while the unit guard reads every corpus on disk. The arc moved no classification: the gate reports 352 / 247 / 155 / 14 intentional diffs at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, 0 unexplained and 0 radar-unclassified at all four, unchanged across the whole branch. - -Declined: - -- Reordering the eleven (2026-09-02) — the obvious way to make the check unnecessary, and wrong twice. Reordering moves which rule classifies a name, which is a behavior change to the ledger and breaks `_CROSS_RULE_WINNERS`; and no pair is even a reorder CANDIDATE, because in all eleven the narrower rule's own prose would be FALSE or MERELY PARTIAL of the co-matched names if it won, and either disqualifies it as classifier-of-record. Partiality is the commoner half and the arc's load-bearing distinction — compound versus component, which the `fix(#400/#274)`/`fix(#400)` exemption calls "the canonical compound-versus-component shape". Read the eleven `why` texts for it: `fix(#400)` "says nothing about a maiden marker"; a widened `fix(comma-family) lone post-comma piece routes to suffix/title` "would take the union and report only half of it"; `fix(cjk-glued-honorific-peel)` says "nothing about a division"; `feat(#273)`'s prose "has nothing to say about" the remainder rejoining as a name; and two of the three regex-accident pairs are partial rather than false, the `fix(suffix-routing)` jr and numeral rules genuinely naming the `{family, suffix}` half of `김민준씨 Jr.` and of `田中さん II` and describing neither the peel nor the segmentation that moves `given`. Falsity is the minority, and one of the eleven says so of itself: the exemption `fix(comma-family) a comma followed only by titles keeps the given/family split` carries over `fix(comma-precomma-family)` calls its own pair "the one where the narrower rule would be actively WRONG rather than merely partial", and adds that the claim "is about THIS name and is deliberately not generalised". `马丁·路德·金씨` is the clearest compound, and `王先生, V.` the reverse case, where the winner is already the least wrong reading available — and stays so, since the family-side twin that would describe it was weighed as #496 and declined (the next bullet). -- A family-side twin of `fix(cjk-comma-honorific-peel)` for a PRE-comma glued honorific (2026-09-02, [#496](https://github.com/derek73/python-nameparser/issues/496), closed as not planned) — the rule that would honestly describe `王先生, V.`, `田中さん, V.` and `김민준씨, V.`, whose diff is the glued peel alone while the label they carry says comma compound (the finding bullet above). Declined on two grounds. FIRST, no gate can demand it: all 17 names the compound/peel pair is contested across measure radar — but by two different warrants, and an audit that assumes one will find the count short. Sixteen sit in `corpus_cjk_tolerated.jsonl`, the file #488 created by demoting the composed comma/Latin-wrapper CJK forms to tolerated input, `김, 민준씨` — the name #382 was filed over — among them. The seventeenth, `Dr 田中さん, V.`, is in `corpus_issues.jsonl` alone: harvested and append-only, radar since #468, and untouched by #488. (`Dr 김민준씨, Jr.` is in both files, and reads radar from either.) Radar means an unmatched diff is reported rather than fatal for want of a RULE; it is not an unconditional never-fatal, since a `[[never]]` exclusion outranks the tier and stays fatal on a radar name — `_CORPUS_TIERS`' own note, and `main()`'s two-reasons comment, which routes an excluded name to `unexplained` rather than to `radar`. No exclusion refuses these, so nothing here can demand the twin. SECOND, [#495](https://github.com/derek73/python-nameparser/issues/495) points the other way over the same corner: it asks whether the radar-only rules already in the ledger still earn their place and records `fix(cjk-comma-compound)` as having zero contract-tier reach, so #496 proposed a SECOND rule for the very names #495 is weighing a first one away from. Fewer rules for this corner is the coherent direction, and that is what makes the decline decisive rather than merely permissive. REOPEN it if a pre-comma glued-honorific name ever reads contract — and watch BOTH routes, because the two files promote by different mechanisms. A `corpus_cjk_tolerated.jsonl` name is promoted by clearing `tolerated` on its case rows, which moves the text into `corpus_cjk.jsonl` (`_CORPUS_TIERS`, and build_cjk_corpus.py's split). `Dr 田中さん, V.` has no case row at all, so there is no flag to clear: it changes tier only by being CHOSEN — a new unmarked row, or a rules.md example — which puts the text in a contract corpus, and the (name, order) dedup loads contract files first and keeps that reading. Read the tier off `_CORPUS_TIERS` either way, never off a hand-built file→tier map (the roster caution above). -- Precise per-name contest detection at differential-run time (2026-09-02; its stated reason replaced 2026-09-03, [#497](https://github.com/derek73/python-nameparser/issues/497)) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. The decline stands, on COVERAGE alone: a run-time check sees one ledger — the one for the baseline that run was invoked with — and only once somebody invokes the tool, whereas the static predicate covers a rule from the moment it is written, including a rule nobody has ever run the wheel against, which is the whole point of #382. Nor can the unit suite simply adopt the precise check: `tests/v2/test_differential.py` is deliberately hermetic — its own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or `Popen` outright — so moving it to pytest speed means spawning a subprocess and, on a cold cache, reaching PyPI. WITHDRAWN, and named here so it is not re-argued: this bullet used to say the check would sit "behind a multi-minute run" and gave that as "the whole of the reason". No measurement was recorded beside it: the phrase entered on 2026-08-05 in `7767ba2` at a single site in `tools/differential/compare.py`, and was copied outward until eight sites carried it — four there, three in `tests/v2/test_differential.py`, and this bullet. Measured 2026-09-03 over the corpus population named above, a whole run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default; the worker pass is 0.10s and 0.29s of those, timed inside `main()`. All four baselines back to back total 1.97s, and a run with `uv`'s cache emptied first, so the pinned wheel is downloaded rather than reused, was still under a second — that last one is NETWORK- and environment-dependent and the RECOMPUTE below does not reach it, covering the warm run alone. Reproducing it means pointing `UV_CACHE_DIR` at an empty directory so the wheel is fetched rather than reused, and the number will differ on a different link. Read it as "the download is not the story either", not as a figure to check against. Read the class and not the digits: the worker pass is a fraction of a second and the whole gate is seconds at most, where the withdrawn phrase said minutes — so no argument here may be built on what the worker pass costs. RECOMPUTE: `time (uv run python tools/differential/compare.py --baseline 1.4.0 >/dev/null)`. For the worker's share, wrap `compare._run_worker` in a timer and then call `main()` — calling `_run_worker` directly on every loaded entry aborts at a 1.4.0 baseline, since the seven order-bearing shape-4/5 entries have to be dropped by the baseline-minimum skip first. Do NOT restate the decline as an error-direction argument either: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. This decline is unchanged and is not what #498 did: the #498 completeness check ADDS a run-time predicate over a class the static one covers not at all, rather than replacing the static one over the class it does cover — so the coverage argument above is the reason both exist (the #498 completeness check below). -- Giving the peel rule a predicate the compound rule fails (2026-09-02, #382 option 1) — narrowing `fix(cjk-comma-honorific-peel)`'s `name_regex` to the honorific-bearing shapes, whose stated effect in #382 was that the pair "becomes order-independent and the original contract holds again". Declined because that effect is UNREACHABLE, and measured rather than argued. The two rules ship a BYTE-IDENTICAL `name_regex` (verified 2026-09-02 by comparing the two strings), and `_rule_reach` computes each rule's names from its OWN pattern, so narrowing the peel rule's regex narrows the peel rule's reach and nothing else: the compound rule goes on reaching every one of the names. The general form, which no regex edit escapes — where the earlier rule's `fields` are a SUBSET of the later one's, order-independence requires the LATER rule to stop reaching the name, and an edit to the earlier rule's own regex cannot cause that. Here the peel rule's `{given, suffix}` is a strict subset of the compound rule's `{family, given, suffix, title}`, so every diff the peel rule admits the compound rule admits too. Implemented to check it: narrowing the peel regex to a `GLUED_HONORIFICS` alternation keeps all seven of the co-matched names whose 1.4.0 diff is exactly `{given, suffix}`, and swapping the two rules in the NARROWED ledger still reattributes all seven. So the pair keeps the arrangement the arc settled on, and that is the answer rather than a cost trade: it is narrow-first, the declaration-free default, so it appears in none of the eleven contests `order_contests` reports and owes no `precedes_narrower` block, and all seven names are pinned by name in `_CROSS_RULE_WINNERS` (23 corpus names reach both regexes; the seven are the ones #375's reorder mutation moves). The cost stands as a second reason and not the first: the narrowing would hand-copy more honorific vocabulary into the pattern and grow the `_HONORIFIC_SOURCES` sync-roster surface (mechanisms.md#CURATED-VOCABULARY-ALTERNATION's second half). Option 2's convention plus a check on the wide-first exceptions to it is what the arc took, and option 3 it REFUSED on a false premise (the finding above). One thing this does not settle, and the ledger comment on the peel rule overstates it: `fields` separates the two only for the UNION rows whose diff includes `family`; for those seven it is file order that decides. -- Scoping the check to contract-tier contests only (2026-09-02) — five of the eleven are contested only over radar names, so this would have cut the file's exemptions by nearly half. Declined because `_CROSS_RULE_WINNERS` already pins radar names — measured, MOST of the names it pins are radar-tier — so the repo would be inconsistent with itself about whether a radar contest matters. The majority is the whole of the argument and no digit belongs here: a count written at this spot was stale before the branch that wrote it merged, since the 2026-09-03 #497 entry below then deleted four of the roster's rows. RECOMPUTE: union the names in `_CROSS_RULE_WINNERS` (tests/v2/test_ledger_guards.py) and read each one's tier off `compare._CORPUS_TIERS`, via the first `corpus*.jsonl` file holding the name with contract files sorted first, as `main()` loads them. Whether the radar-only CJK comma rules still earn their place after #488's demotion is a real question and is filed as #495; it is a question about those rules, not about the check. -- A shared name-population function for both callers (2026-09-02, the spec's own first sketch) — see the two-populations decision above. `main()` and the unit guard must read different populations, so one function would break `--corpus`; they agree by a docstring that names the function it mirrors instead. -- Deleting the rules whose explained names are all radar (2026-09-02, [#495](https://github.com/derek73/python-nameparser/issues/495), closed as not planned) — filed off #488's demotion on the reasoning that a rule nothing can demand is a rule to remove, and refused on what deleting one actually does. Measured at baseline 1.4.0, 19 of the 72 explaining rules explain only radar names, between them 48 of the 352 intentional diffs — and that set is a SUPERSET of what the issue proposed deleting, not a different set. Three of the five rules #495 tabulated are in it — `fix(#296) a credential-only comma string reads a name and its postnominal`, `fix(cjk-comma-compound)`, and the roman-numeral `fix(suffix-routing)` — and they are exactly the three its option 2 named as having zero contract reach. The other two are outside it for the reason that disqualified them as candidates there too: they explain contract names, fourteen of them for `fix(cjk-glued-honorific-peel)` and `'Smith Jr.'` (corpus_rules.jsonl) for the jr `fix(suffix-routing)`. So the criterion reaches everything the issue wanted gone and more, and what refuses it is the fate of those 48 diffs. Measured by driving `classify()` over each one with its own rule removed: 25 report UNCLASSIFIED and 23 are claimed by ANOTHER rule (26 and 22 with all 19 deleted at once). For nearly half the population, deleting does not leave the diff watched-but-unexplained — it leaves it explained, by a rule whose comment is about something else. Nine of `fix(cjk-comma-compound)`'s eleven land on `fix(cjk-glued-honorific-peel)`; all seven of `fix(cjk-comma-honorific-peel)`'s land on `fix(cjk-comma-compound)`; `'Carod i'` lands on the roman-numeral `fix(suffix-routing)`, and `'Bob Jones, author'` on `fix(comma-precomma-family)`. That is the rule-broader-than-the-diff absorption this file already names twice — `fix(#424)` at decisions.md#H1, going on classifying a diff that shrank underneath it while the fields-only catch-all's heading stepped 14 -> 17 with nothing announcing either, and the fields-only arc's "a fields-only rule silently falsifies OTHER comments" — reached this time on purpose rather than by accident, which is not an improvement. Applied, it deletes `fix(#342)` and `fix(#397)`, the two NOT WANTED rules whose whole job is classifying a reading nobody wants so the gate stays green while the bug is open, and which carry the delete-when-fixed instruction the dormancy check enforces once the fix lands (the fields-only arc above). The cost there is NOT a red gate, and a first draft of this bullet said it was: both names sit in corpus_issues.jsonl, which is radar, and on a radar corpus an unmatched diff prints UNCLASSIFIED (radar) while the run keeps exiting 0 — UNEXPLAINED is the contract tier's word (tools/differential/README.md). Measured with both rules gone, the run reports `unexplained: 0; radar unclassified: 1` and exits 0. The one is `'Aishwarya Rai'`; `'Carod i'` is not reported at all, the numeral rule having taken it — so one of the two open bugs loses its marker not to silence but to a wrong label. Verified end to end against a ledger with `fix(#397)` removed: exit 0, 352 intentional diffs still, no UNCLASSIFIED, no EXPLAINED NOTHING, and `'Carod i'` printed under `fix(suffix-routing) a two-token name ending in a roman numeral`. It also deletes `feat(#273)`, `feat(#269) Arabic بن prefix chains onto family (non-Latin new-recognition)` and `fix(#360) ste moved into the never-given particles with mc` — records of new recognition behavior and of a vocabulary move — and the last two have to be named to the RULE rather than to the issue, as the ledger holds sibling rules under both strings. The `feat(#269)` sibling is the sharp one: `feat(#269) non-Latin titles/conjunctions recognized` does not qualify, explaining the contract-tier `'Хосе И Мария Сантос'` (corpus_rules.jsonl), so a delete by issue prefix takes both and sends that name UNEXPLAINED — measured, and a red gate. The criterion that does justify deleting a rule is that it explains NOTHING, which `dormant` already names and the dormancy check already enforces; a rule explaining eleven radar names is not dormant. What the radar tier is FOR answers the rest: decisions.md#cjk-comma-demotion puts it as giving the differential "somewhere to WATCH a name without promising it", and a rule explaining a radar name IS the watching. The five exemptions citing #495 are rewritten in the same diff rather than left standing: four of them put it as a LIVE question — the two under `fix(#296) a credential-only comma string`, the one on the `fix(cjk-comma-compound)` block, and the one on the `'田中さん II'` numeral contest — and the fifth, on the jr contest, called neither rule a candidate in the present tense. Each records the answer now and keeps every measurement it carried. One of those measurements was wrong and is corrected with them: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` explains eight names of which FIVE are contract, not six — the six having shipped in #499. It is not a mis-map: six is what tiering by FILE MEMBERSHIP gives from a CORRECT `_CORPUS_TIERS`, and the single name it gets wrong is `'John Smith, Dr.'`, the third RECOMPUTE trap below. #496's decline was argued inside the `fix(cjk-comma-compound)` one and stands without it: the twin rule was refused because nothing can demand it, which is untouched by what #495 answered. RECOMPUTE: do not join names to files by hand. `_CORPUS_TIERS` maps FILES, the report prints names with no file, and 24 of the 352 explained names sit in files of BOTH tiers, so a hand join has to reproduce `main()`'s contract-first dedup AND its baseline-minimum skip to get them right. Let the run do it: call `compare.main()` unchanged under two wrappers — one around `_run_worker`, whose `entries` argument is the final list with `tier` and `file` already stamped on every entry, and one around `classify()`, recording (name, order, diff, issue) for each of its 352 calls — then reclassify offline from those records against `rules` minus whichever rule is under test, having first confirmed the wrapped run reproduced the shipped one (352 intentional, 0 unexplained, 0 radar unclassified). Parse no printed repr by stripping quotes: three of the 352 names do not survive it — `"'Smitty' Dr. Jones"`, `"'Smitty' Jones Jr."`, and the spelling of محمد بن سلمان wrapped in RIGHT-TO-LEFT MARKs — and dropping them silently is what produced the wrong count of 20 on the first pass, the first of the three being `fix(#410)`'s only contract name. `ast.literal_eval` reads the printout correctly. The ten-name truncation is NOT the hazard it looks: contract files load first, so any contract name a rule explains sits in the printed prefix, and tiering from the first ten alone gives the same 19. THIRD, and the only one of the three that changes an answer on this tree: a name's tier, in any claim scoped to a BASELINE, is a property of the ENTRY that run compared and not of the files the string appears in. The two part where a string sits in files of both tiers AND its contract copy carries a shape tag, because the dedup key is `(name, order)` — a shape declaring an order leaves TWO entries rather than one contract winner, and the baseline-minimum skip can then drop the contract one. Measured, 2 of those 24 names carry a shape tag and exactly 1 bites at 1.4.0: `'John Smith, Dr.'` is in corpus_issues.jsonl (radar, no shape) and corpus_shapes.jsonl (contract, shape 4, FAMILY_FIRST, min_baseline 2.0.0); both entries survive the dedup, which the 2.0.0 run shows by comparing both, and at 1.4.0 the shape-4 one is among the seven skipped, leaving the radar entry as the only comparison. `'Beethoven, Ludwig van'`'s shape 2 is order-None with a 1.4.0 minimum, so the dedup collapses it and contract wins at every baseline. File-membership tiering gets exactly 1 of the 352 wrong, and that one is what `fix(comma-family)`'s six-versus-five turned on; `entry["tier"]` gets it right for free. The radar-only count of 19 is the same under either tiering. To exercise a ledger edit, point `_allowlist_for` at a scratch file rather than copying the tree — the corpus glob resolves from `HERE` and `_check_tree` refuses a nameparser outside `REPO_ROOT`, so a copy elsewhere cannot run at all. - -### differential-ledger, the recorded-shapes arc (2026-09-03, #497) - -The seventh ledger arc, and the one that makes a recorded INPUT measurable. The arc above closed on the finding that `test_the_recorded_rule_still_wins_each_contested_name` feeds `classify()` a diff shape nothing ever checks, so a shape that is wrong but still routes to the same rule passes forever; this is what was done about it, and about the second thing that search turned up — that the cost the same arc declined a check on had never been measured at all. Mechanics — where the shapes live, which name list each half of the check reads, what a `MOVED SHAPE` block may and may not claim — are owned by `tools/differential/compare.py` (`_RECORDED_DIFFS`' provenance note, `recorded_diff_mismatches`' docstring, and the two placement comments in `main()`), by `tests/v2/test_ledger_guards.py` (`_CROSS_RULE_WINNERS`' header and the note over its 2.x sections) and by tools/differential/README.md; these are the decisions. Every figure below was measured or recomputed 2026-09-03 on this branch; each one that can drift carries its recipe, and the two that cannot be re-derived from the tree at all — the sweep scan's own hit counts — say so where they appear. - -Decisions that landed: - -- 2026-09-03 [#497](https://github.com/derek73/python-nameparser/issues/497) — the recorded shapes live in the TOOL, as `compare._RECORDED_DIFFS`, and the RUN verifies them; `_CROSS_RULE_WINNERS` keeps only which rule wins a contested name and reads the shape from there. There because it is the one place both halves exist: `main()` has measured every name's real diff by the time it asks, where `validate_rules` runs before any corpus is read and the unit suite installs no wheel at all — `tests/v2/test_differential.py`'s own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or fake `Popen` under it. State it that way and not as "every test monkeypatches `_run_worker`", which is false: measured 2026-09-03 on this branch, of 171 `def test_*` functions in that module 20 mention `_run_worker` at all and six reach the real one with `Popen` faked beneath them by `_fake_popen`. RECOMPUTE by walking the module's `def test_*` bodies for the name. That short form is what compare.py's `_RECORDED_DIFFS` provenance note SAID when this entry was written; `f70a0ad`, the very next commit, withdrew it there, and the note now rejects the paraphrase rather than carrying it — so follow that pointer to the history and not to the text. The denominator has moved once already inside this arc: `f70a0ad`'s own message says 168, which is why the argument here is "most never mention it" and the digits carry a date. The hermeticity claim is unharmed and the loose paraphrase of it is what fails, which is this arc's subject arriving inside its own evidence. Read HERMETICITY as the reason and not speed. The rival placement is the unit suite, and the reason it is refused is that a check there would have to install a wheel — not that installing one is slow. The timing finding below retires every cost argument in this file and moves this decision not at all, which is the test such a placement argument has to pass and the reason to say out loud which ground carries it. `_CORPUS_FLOORS` is the precedent — tool-owned, corpus-shaped data that the guards read and monkeypatch — and this is the same direction. -- 2026-09-03 #497 — ONE roster, TWO checks, TWO PLACEMENTS — and, since `9360919`, a NOTE over the gap between them, recorded at the end of this entry. (Superseded in its first word 2026-09-05: TWO rosters since the watched-shapes arc below, `_WATCHED_DIFFS` beside `_RECORDED_DIFFS`, both placements unchanged; the departed-name refusal and the `NOT CHECKED` note read the union of the two, while the shape comparison runs once per roster, because the two halves carry different severities — a contest row's mismatch feeds the exit code on either tier, a watched row's only where its default-order entry is contract tier — and one call could not tell them apart.) The asymmetry of the two checks is the decision rather than where the code happened to fit. The ABSENT-NAME half — a recorded row naming a string no corpus holds any more, which nothing else would ever report — refuses PRE-worker, beside `vacant`, because it reads the ledger and the loaded names and nothing the worker produces. It takes the PLACEMENT #382 chose for its own pre-worker refusal and NOT #382's reason for it, which was the cost claim below — `compare.py`'s comment at that placement is one of the sites the timing finding retires. The ground it stands on was rewritten on this branch: a refusal raised after the wheel is installed and the corpus compared prints below the run's own published `baseline:` header, for a comparison it will never report. Not "disowning a comparison it has just published" — measured print order is `baseline:` at the tell, then the comparison loop, then `corpus: … intentional diffs:`, so at that position the header has printed and no line of the comparison has. The ORDER is the argument, and that corrected wording is what `compare.py` now carries at all three of its placement comments. Recorded because it is the second decision the false number was under, and the two came apart differently — the declined check below kept its conclusion on a new argument, and here the ORDER was right all along while the reason for it was not. Only re-deriving a claim says which of its supports was doing the work; a placement that reads unchanged is not thereby unaffected. The MOVED-SHAPE half is the opposite case — it needs the diffs, so it stays after the comparison — and there it may not RAISE. It PRINTS and feeds the exit code, like `over_declared_rules`, its structural sibling and the other post-worker check on recorded data: a raise at that point lands MID-report and takes dormancy, OVER-DECLARED, UNEXPLAINED and the radar block with it, which compare.py measures on a narrowed 1.4.0 run beside the check. A stale roster row must never hide an unexplained diff, that being the gate's primary output. The two pre-worker refusals may raise because nothing has printed yet; this one cannot, and the difference is PRINT ORDER rather than severity. The two halves also read opposite name lists, which is `recorded_diff_mismatches`' docstring's to state and is not restated here. What IS decided here is the THIRD thing this roster produces, added by `9360919` once the gap between those two lists was measured and found not empty: an order-bearing entry the baseline cannot honor sits in a corpus file and outside the compared set at once, so a row on such a name is checked by neither half and reported by neither. `main()` prints a `NOT CHECKED` note over it. A NOTE and not a refusal — the docstring's argument against refusing off the post-skip list stands, and it is an argument against REFUSING and not against SAYING, which are two decisions of which only the first was ever made. Informational, outside the exit code, and gated on neither `full_corpus` nor `args.corpus`, the intersection narrowing itself. Measured 2026-09-03 at 1.4.0 — both the only baseline where the skip fires and the only ledger with rows — the window is three names, none of which carries a roster row today, so the note prints on no run yet. The measurement that motivated it put a deliberately wrong shape on `'de la Cruz née Vega'` over the full corpus at 1.4.0: before `9360919` that exited 0 in 375 stdout lines naming the name in none of them, and it now exits 0 in 378 with the `NOT CHECKED` note naming it — re-measured 2026-09-03 by corrupting the row in memory around `main()`, which leaves the worktree alone. -- 2026-09-03 #497 — a SHAPE may be pinned where a COUNT may not, and that line is the whole of what keeps this roster affordable. A diff shape is one name under two parser versions, so adding a name to a corpus cannot move it; an explained-name count moves on every corpus edit. Shapes drift only when the PARSER changes those names — so the drift and the reason to re-read the roster are the same event, which is exactly what a recorded number usually fails to be. That is why #452's churn objection to `explains = N` (the fields-only arc above, 2026-08-28) does not reach this roster, and why pinning shapes here does not weaken it: the four ledgers carry 201 rules today, one `_CORPUS_CLAIMS` entry each, every one liable to move whenever a corpus does, against 31 recorded shapes that move only with the parser (51 since #498, 45 of them at 1.4.0). Read the two CAUSES and not the two totals — the argument is that one set is coupled to corpus edits and the other is not, and it survives either number moving. RECOMPUTE: `sum(len(v) for v in _CORPUS_CLAIMS.values())` in tests/v2/test_ledger_guards.py, and `sum(len(v) for v in _RECORDED_DIFFS.values())` in compare.py. -- 2026-09-03 #497 — four of the 35 rows the roster carried were DELETED rather than corrected. Two names across the two 2.x ledgers, so two rows each, and one of the two NAMES had a correctable shape — count rows or names deliberately, since the provenance note that holds the measurements speaks of the name. Deleted because neither name pinned a CONTEST, which is what the roster is for: at the shape each really produces exactly one rule admits it, so `classify()` was never adjudicating anything. `'Nguyen, Van'` diffs at no baseline at all, so no run ever asks `classify()` about it; `'Jane née and Jones Smith'` is a malformed harvest from a radar corpus whose right parse nobody can state, so a pin on it defends no boundary anyone would argue for — which is why the correctable one went too. compare.py's provenance note holds the per-name, per-baseline measurements behind both. The deletion emptied both 2.x sections, which stay present as explicit empty mappings — the completeness guards require it, and it is what makes emptiness sayable as a POSITION: a row is owed when someone argues a boundary, not before. What the emptiness is NOT — that these ledgers hold no contested diff — is measured and stated beside those sections, and whether the position should change now that those contests are measured rather than merely unexamined is [#501](https://github.com/derek73/python-nameparser/issues/501), open. (Superseded in its emptiness 2026-09-05: the two 2.x sections are no longer empty, the #501 adjudication below having pinned FIVE rows at 2.0.0 and one at 2.1.0, so `_CROSS_RULE_WINNERS` read 31 / 5 / 1 / 0 at that point — measured 2026-09-05 — and #501 is closed. It reads 45 / 5 / 1 / 0 since #498's fourteen pins, landed the same day; only the 1.4.0 figure moved. RECOMPUTE for either: `{k: len(v) for k, v in _CROSS_RULE_WINNERS.items()}` in tests/v2/test_ledger_guards.py. The criterion is not superseded and is what filled them: a row is owed when someone argues a boundary, and three of the six were found already argued, in the 2.0.0 ledger's own comment. Read the sentences above as the position emptiness stood on for as long as it stood.) Recorded with the deletion because it is the half a later sweep will get wrong: "only one rule admits it" is NOT on its own grounds to delete a row. A substantial minority of the surviving 1.4.0 rows are in that position and stay — the count and its recompute are beside them in test_ledger_guards.py, where six of the thirteen say so in their own comments and the other seven do not, which is why the count is recomputed rather than read off the roster — because the shapes they pin are shapes runs actually make, so a widened `fields` or a moved rule hands the name over and the test says so. Counting admitters is not the test at all: 11 of those 13 route to a DIFFERENT rule under some other shape, and mechanisms.md#RECORDED-ROSTERS now carries that measurement in place of the criterion it stated first. The four deleted rows could not do that work at any edit, which is the difference. -- 2026-09-03 #497 — the prose sweep is sorted by this file's own criterion (#451, "past-tense counts stay in a ledger comment; live ones do not") into three treatments, and the treatment follows from what the count is DOING rather than from how wrong it is. A count that IS the finding and reads past tense stays. A count standing beside the enumeration it counts loses the digit — the list is self-checking and the digit is the only part that can rot. A live exhaustiveness claim with nothing recomputing it is REGROUNDED on the property it was reaching for, so that the sentence stops depending on a number — saying a widening claims all but a handful of the whole population the assertion is over says what a fraction said and cannot go stale. One judgement goes with it and is the arc's own: a figure that could NOT be reproduced was not replaced with a fresh one. `_reaches_non_vocabulary`'s docstring now carries the shape of its finding, the instruction for measuring it, and no LIVE figure for the magnitude the old figures asserted — a replacement digit nobody can re-derive is the defect rather than the repair, and this file has already been burned by that shape (the #414 roster comment under the fields-only arc's `explains = N` bullet). The past-tense digits explaining why the live ones are gone stay, which is the first treatment applied inside the third, and a later sweep should not read them as survivors of the pass. - -Found rather than decided, and worth as much: - -- **The cost that decided a decline had never been measured, and the decline survived losing it.** "The multi-minute worker pass" entered 2026-08-05 in `7767ba2`, at ONE site, with no measurement beside it. The census of where it reached, the timing figures that replace it, their recompute and the `_run_worker` timing trap all live ONCE, in the rule-order arc's `Declined:` bullet on precise per-name contest detection above — the argument they falsify — and none of them is restated here; that census is the figure in this arc most likely to move, so pointing at its owner rather than copying it is the #451 rule applied to the very claim that demonstrates why. Two notes on reading it that belong here rather than there. Not every site spells the phrase — one says the pass "costs minutes" — so count the CLAIM and not the wording (`git grep -n -i minute 514a2a1`, this branch's base). And measured, the worker pass is a fraction of a second and a whole gate run is seconds at most, where the phrase said minutes: read that class and not the digits. Three things transfer past them, and a coda about removing it. The claim was never a measurement anyone got wrong — it was a plausible sentence about a subprocess that installs a wheel, and it propagated by COPYING, the mechanism the fields-only arc recorded for ledger comments and which reaches code comments identically. It was load-bearing and its conclusion still stands, rewritten on COVERAGE: a run-time check sees one ledger, for the baseline that run was invoked with, and only when somebody invokes the tool, where a static predicate covers a rule from the moment it is written. And that is how a false number stays invisible for a month — an argument with two supports loses one and nothing downstream of it changes, so no reader ever meets a contradiction. Removing it was itself hard to do cleanly: review of the commit that removed the phrase found four instances of the very defect that commit exists to remove, fixed in `40c82a7`. -- **A row pinning a race with one runner is never exercised as a contest, so its shape only ever had to agree with itself.** That is why nothing caught the four, and it is the general shape rather than a fact about two names. The four arrived with #452's work (`ee6aaa6`, PR #455) under two claims of measurement — the roster header's "The diff shapes are measured against the 1.4.0 wheel, not guessed" and the 2.1.0 block's own "measured against this baseline's own run rather than copied from the 2.0.0 rows" — and every one of the 31 rows at 1.4.0 agreed with the run on 2026-09-03, at all four baselines; the 45 rows there since #498 all agree on 2026-09-05, the gate exiting 0 with no `MOVED SHAPE` at any baseline. Do NOT read that as the 1.4.0 half having been sound: one of those 31 was guessed too, `田中さん II`, wrong on a genuinely CONTESTED row, found by hand a day before this branch opened (`e5e4955`, the finding the arc above filed as #497) and already corrected when the 31 were re-measured. So the one-runner mechanism explains why THESE four survived a check that would have caught a contested row, not why every wrong shape survived — the contested one needed somebody to sit down with the wheel, which is the cost the run now removes. The transferable test is what to ask of a recorded INPUT: not "is it checked" but "would its CONSUMER answer differently if it were wrong". mechanisms.md#RECORDED-ROSTERS forbids re-deriving the EXPECTATION from the inputs the check reads; this roster obeyed that to the letter and still went wrong, on the other side of the assertion, which is why that entry now carries the limit. -- **The scan that scoped the sweep was wrong three ways, and it is the scan rather than the three fixed lines that the next sweep will reuse.** Two of its 19 machine-readable hits were false positives — a version number, and a hypothesis rather than a measurement — so the real population was 17. Its file list omitted `expected_since_2.2.0.toml`, which held a further copy of a claim it was already treating in two other ledgers, so the sweep would have shipped two of three siblings corrected. And its predicate matched only lines BEGINNING with `#`, which made every docstring invisible to it: the stalest figures in the whole sweep survived the first pass for that reason alone, and a docstring pass over the two guard modules then found two more. Read the sweep as BOUNDED and not exhaustive — that docstring pass covered `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`, and compare.py's own docstrings were never swept. And a fourth thing, which is the finding about the finding: the scan's PREDICATE was not preserved. Its 19 hits and their 17 survivors are recorded in `8593f3a`'s commit message and cannot be re-derived from the tree, because nothing says what it matched beyond the leading `#`. That is the reusable artifact of a sweep — not the lines it fixed — so the next one has to write its own and has no earlier hit count to check itself against. Write the predicate down where the results go. -- **What the stale claims were, because the classes recur and the digits do not.** (i) A count of a population that had grown by half underneath it: a guard comment gave a Latin-half widening as claiming "644 of the 654 unclassified corpus names", where the same expression claims 994 of 1008 today (`len(_UNCLASSIFIED_NAMES)`, intersected with `_claimed("[A-Za-z]")`, in tests/v2/test_ledger_guards.py) — and since the argument was that the widening reaches essentially the whole population the assertion is over, it is written that way now and needs no recompute. (ii) A count of FILES: "all five corpora", in three places, where six `corpus*.jsonl` sit on disk and a run globs every one of them — the corpus count was never the argument, and the gate's own `corpora:` line is the check. (iii) A live count inside the very paragraph that names stale counts as the file's standing hazard — `fix(#274)`'s ledger comment walked "all 31 the regex claims" where that regex claims 32 today (`len(_claimed(rule["name_regex"]))`). (iv) A count over a self-checking enumeration: "Six such widenings" above a list of four, where nobody can now recover which two are missing and the list was always the evidence. The fifth is the one worth the space, and the docstring pass found it: `_carries` said the `isascii()` split covers "12 of the 16 entries", where at sweep time `MAIDEN_MARKERS` shipped 17 with 12 of them non-ASCII (measured 2026-09-03; RECOMPUTE with `len(MAIDEN_MARKERS)` and `sum(not m.isascii() for m in MAIDEN_MARKERS)` from `nameparser.config.maiden_markers`) — the NUMERATOR was right and stayed right while the denominator rotted. The repair names the predicate rather than counting its output, which is why the guard's own note past-tenses both figures and points the reader at the `isascii()` split itself. A count can go stale in one of its two halves, which is the shape a re-reader is least likely to catch, and it is a reason to prefer naming the predicate over counting its output. -- **The tier split inverts the ranking that #452's still-open half rests on.** Beside the fields-equals-union check that landed, #452 carries a second proposal: a ceiling on how much of the corpus one rule may REACH, measured there as a share of the whole corpus. #468's tiering makes that the wrong denominator, because a radar diff cannot fail the run — a rule reaching many radar names is not the hazard a rule reaching many contract names is. Measured over the corpus population and tier split the rule-order arc's RECOMPUTE above already owns and this bullet does not restate, the two comma rules reach 288 names, 25.8% of all of them, but 64 contract names, 19.6% of that tier; `fix(#271/#272/#298)` reaches 108, 9.7%, and 80 contract, 24.5%. So by total reach the comma rules are the outlier and by contract reach the CJK rule is, and a ceiling on total reach would rank the wrong rule as the biggest hazard. Read the INVERSION rather than the four percentages: it is a claim about which tier the denominator counts, and it survives any drift that does not reverse the two tiers' composition. Recorded here because the proposal is open and the measurements in it predate the tiering. RECOMPUTE: tier every corpus name off `compare._CORPUS_TIERS`, contract winning where a string sits in files of both tiers, then take each rule's `name_regex` reach split by tier. - -Declined: - -- `explains = N` per rule as ledger data (2026-09-03, again) — #452's own first sketch, declined there on churn, re-examined here only because this arc pins a different recorded value and owed an account of why the two differ, and declined again on the same grounds. The third decision above is the argument in full: a shape cannot move when a corpus does and a count can. The landed fields-equals-union check already catches the drift that motivated the sketch — a rule's explained set changing SHAPE underneath it — and needs no number to do it. The question belongs to [#452](https://github.com/derek73/python-nameparser/issues/452), which is OPEN and is exactly it; nothing here forecloses it, and its remaining reach-ceiling half should be re-measured on the tier finding above. -- The differential in CI (2026-09-03) — considered and dropped rather than deferred. It is a release gate that already runs locally on demand, and no sibling check — dormancy, over-declaration, the undeclared-contest refusal, this one — needed a CI job to be worth having. Worth recording as a DECLINE and not an omission precisely because the timing finding removes the only reason nobody proposed it: a job costing seconds is cheap, and cheapness was never the criterion. -- A general sweep of every number-carrying comment line (2026-09-03) — the sweep is scoped to the claim-shaped ones, which the scan put at 19 and which were 17 — the pair of figures that cannot be re-derived from the tree, since the scan's predicate was never preserved; they live in `8593f3a`'s commit message and the bullet above says so. The population it declines is larger by nearly two orders of magnitude: measured 2026-09-03, comment lines carrying a digit run to four figures across the differential tooling, its four ledgers and the two guard modules, against 17 claim-shaped ones (RECOMPUTE: count lines matching `^\s*#` that also contain a digit, over `tools/differential/*.py`, `tools/differential/*.toml`, `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`). The exact figure is deliberately not given: an earlier draft of this bullet carried one, and the NEXT commit on this branch added two comment lines and falsified it — inside the entry declining a sweep for exactly that. Most are issue references, version numbers and codepoints, none of which rot. Read the RATIO and not either digit: a hand sweep two orders of magnitude past the claims is how wrong claims get INTRODUCED rather than removed, and this arc demonstrated that at the smaller scale — the commit that removed the false cost claim needed a follow-up for four instances of the same defect inside itself, and the first prose sweep shipped having skipped three instances it had already identified (`8593f3a`). - -### differential-ledger, the watched-shapes arc (2026-09-05, #501) - -The eighth ledger arc, and the one that lets a diff shape be recorded WITHOUT a winner. The arc above moved the recorded shapes to where a run verifies them and, in the same commit (`962ae43`), bolted them to the winners with `set(winners) == set(shapes)` — never argued as a constraint, and the reason 51 corpus names whose only watcher was a classification rule could not be watched at the shape level: recording a shape meant pinning a winner, and a winner pin asserts an argument nobody had made. Mechanics — the two dicts and their contracts, the three print blocks, which name list each check reads — are owned by `tools/differential/compare.py` (`_WATCHED_DIFFS`' header, which carries the population and its recompute) and tools/differential/README.md's `MOVED SHAPE` section; these are the decisions. Every figure below was measured 2026-09-05 on this branch from real `main()` runs with `classify()` spied, and each carries its recipe; the one figure that goes stale by design, the quiet-tree count, is dated where it appears. Refs [#501](https://github.com/derek73/python-nameparser/issues/501), which stays open. - -Decisions that landed: - -- 2026-09-05 #501 — a shape MAY be recorded without a winner, in a SECOND roster (`compare._WATCHED_DIFFS`, keyed per ledger like `_RECORDED_DIFFS`, disjoint from it per ledger, both keyed by every ledger on disk — held by the guards at pytest speed and, since the PR's review round, by `main()` pre-worker, which refuses a ledger either dict has no section for) and NOT by relaxing the equality. Two grounds, and the first is the arc above's own placement decision turned around: the run cannot see `_CROSS_RULE_WINNERS` — `compare.py` cannot import the test module, and names that roster only in comments and in the advice it prints — so if the KIND of a row is to decide anything at run time, the kind must be DATA in the tool, and one dict with a relaxed guard leaves the run unable to tell a contest row from a standalone one, which means one repair text and one severity for both. The second is that the equality was doing work the relaxation gives up: a shape whose winner was deleted and forgotten is a deleted-winner slip, and under `<=` it is indistinguishable from a deliberate standalone row. So `set(winners) == set(shapes)` stays, both directions, and its docstring now says why the reverse direction is defended — a shape with no winner is not an orphan to tolerate but a row in the wrong dict. A name in both dicts is refused twice, by the guard at pytest speed and by `main()` pre-worker beside the departed-name refusal, because the tool may not assume the suite ran; the argument is the stronger claim, so such a name belongs in `_RECORDED_DIFFS` alone. The day a winner is argued for a watched name, the row MOVES and the pin goes beside it there; it never gains a partner where it is. -- 2026-09-05 #501 — SEVERITY follows the row's kind first and the tier second: a row with an argument behind it (a pinned winner) is fatal on both tiers; a row that is only a snapshot follows its tier, fatal on a contract name and printed under a non-fatal `MOVED SHAPE (radar)` block, parallel to `UNCLASSIFIED (radar)`, on a radar one. Three grounds, from the spec's review amendment A. A per-name shape is a STRONGER promise than a contract name gets — a contract name needs a RULE, which covers a shape class and batches across names, so a feature that legitimately moves thirty of these would produce thirty findings each needing its own "say why", the per-name burden #468 removed, reintroduced one level down on the very file #468 demoted for that reason (`corpus_issues.jsonl`, 49 of the 51; 48 of the 50 since the #501 adjudication below, and the burden argument is unmoved by either count). The `[[never]]` precedent cuts the other way — exclusions are fatal on both tiers because each is a per-name deliberate choice carrying a `why`, and these rows are measured, not reasoned: a sweep, not a choice. And when a snapshot row fires the only repair is to re-snapshot — a contest row has an argument to consult; `'Ph. D., Jr.'` or `'QC MP'` has no adjudicated parse, so "correct the row" means "record whatever it does now", which is mechanisms.md#RECORDED-ROSTERS's re-derivation hazard in a gate's clothing. Two limits on what this decides, stated so nobody reads more into it. It answers the NARROW tier question only — whether a snapshot on a radar name may fail the gate, which `shape_bad` being tier-blind had answered yes by default — and leaves the gate-and-changelog hypothesis where the spec left it: the framing that the ledger is a gate AND a changelog and `_CORPUS_TIERS` splits only the gate is a hypothesis, not codified, and the radar tier's explanation-relief has read 0 at every baseline because no feature work has offered it a case, not because it was declined. And it does not reverse #468; it applies it — "shown, never blocking" reaching a per-name snapshot on a radar name exactly as it reaches an unmatched diff there. It also turns the arc above's radar overlap from unreconciled history into a stated rule: measured 2026-09-03, 21 of the 31 contest rows at 1.4.0 sat on radar-tier names, every one carrying a winner, and they stay fatal because it is the argument and not the tier a contest row defends (RECOMPUTE in tools/differential/README.md's tier paragraph). -- 2026-09-05 #501 (found in the branch's code review) — the tier a watched row's severity follows is the DEFAULT-ORDER ENTRY's, not the name's. A recorded shape is read off the default-order comparison, so that comparison's entry is the one whose tier says whether the shape was promised; the first-loaded (contract-preferring) entry decides only for a name with no default-order entry, which is the name compared under a declared order alone. This is the rule-order arc's third RECOMPUTE trap applied to severity — "a name's tier, in any claim scoped to a BASELINE, is a property of the ENTRY that run compared and not of the files the string appears in", under its #495 decline above, whose measurement is not restated here. The live case is that arc's own: `'John Smith, Dr.'` is contract in `corpus_shapes.jsonl` only as shape 4 (`FAMILY_FIRST`) and radar in `corpus_issues.jsonl` under the default order, so a watched row on it would print and not fail, and the family-first promise would be untouched — it carries no row today, a test naming it. The dict header first said "a name both tiers hold reads contract", which was false of the code as written (`tier_of` is built over the post-skip entries, an order-None entry winning where one exists), and was corrected before the commit was reviewed again. -- 2026-09-05 #501 (found in the design-docs review) — the roster's DEFINING property is "no winner pinned", not "nothing else watches". The four #501 rows are pinned by an exact `Case(...)` literal in `tests/v2/cases.py` and two sit in contract corpora (`'김민준 박사님'` and `'선생님'`, in `corpus_cjk.jsonl` and `corpus_rules.jsonl` alike; the two `様.` strings are in `corpus_cjk_tolerated.jsonl` only — measured 2026-09-05 on this tree and at `e004ccf`, where the four sit in exactly the same files), so "nothing else watches" is the property of the sole-watched SUBPOPULATION and false of the dict as a whole, and a run whose message told a reader of a `'선생님'` row that nothing else watched it would send that reader past the case row that already adjudicates the parse. What every row has, and what the two checks enforce (the disjointness guard, the `both` refusal), is that no winner is pinned. The dict header, the fatal block's lead, the `(radar)` block's lead and the README say "no winner is pinned"; the population comment says which rows are also sole-watched, and said until this branch that a contested row's reader consults the case row before the roster — a line the header no longer needs and no longer carries, the four rows having moved to `_RECORDED_DIFFS` with winners pinned beside them. (The four moved to `_RECORDED_DIFFS` with the #501 adjudication below, 2026-09-05, so every row in `_WATCHED_DIFFS` is sole-watched today; the decision is about the roster's DEFINING property, which is unchanged — no row there pins a winner.) -- 2026-09-05 #501 — every sole-watched diff is recorded — 109 sole-watched rows plus the four #501 contests, 113 in all — not the exposed subset. (AMENDED 2026-09-05, later the same day: the #501 adjudication below took six rows to `_RECORDED_DIFFS` — the four contests and `'MD, PHD'`'s two, which this count read as sole-watched and which were not — so the roster is 107 rows over 50 names and is now exactly the sole-watched population. The decision is about WHICH diffs get a row and survives the totals moving.) Airtightness is a property of today's rule widths and not of the name: a rule whose `fields` happens to equal a name's diff admits no alternative today, and widening that rule later exposes the name with nothing saying so. And the 35 alternatives over the sole-watched rows that route to a DIFFERENT rule (the finding below; 44 with the four contests counted) are silent whatever the winner's width is, which is a stronger reason than airtightness: "every sole-watched diffing name" survives a rule edit, "the exposed subset" is a snapshot of the widths. - -Found rather than decided, and worth as much: - -- **The spec's 385 / 71 was GROSS of #452 and measured on a population the arc had already retracted; the net figure over the roster is the one to argue from.** `over_declared_rules` requires a rule's `fields` to EQUAL the union of the diffs it explains, so on a rule explaining only one name any shrink fires OVER-DECLARED and is not silent at all. Measured 2026-09-05 over the 113 rows `_WATCHED_DIFFS` holds: 390 alternative shapes classify to the same rule and 72 diffs are exposed by that count; net of #452 — keeping only the subsets `S` for which the union of the rule's OTHER explained diffs at that baseline, joined with `S`, still equals the rule's `fields` — 337 alternatives over 64 exposed diffs, and 44 further alternatives route to a different rule. Per baseline the gross / net / other-rule split reads 105 / 96 / 23 at 1.4.0, 158 / 137 / 20 at 2.0.0, 127 / 104 / 1 at 2.1.0 and 0 / 0 / 0 at 2.2.0. Over the 109 sole-watched rows alone — the roster minus the four #501 contests, which is the population the label "sole-watched" names — 367 gross over 68 exposed diffs, 314 net over 60, 35 other-rule, the whole difference sitting at 2.0.0 where the four contested rows are. The spec's first draft measured 385 / 71 gross, 330 / 62 net and 35 other-rule, with 123 / 112 / 23 at 1.4.0, over the population the next bullet retracts: 52 names and 113 diffs, holding the four names with 1.4.0 contest rows (45 rows there, `'MD, PHD'` alone contributing 14 alternatives) and not the four #501 rows — the same total by coincidence, so those figures read as the roster's and were not. The 8 diffs that leave the exposed count under the net test — `'Esq. van Gogh'` at all three baselines where it diffs, `'Dr. Do Van Johnson, MD'` at both 2.x, `'Aishwarya Rai'`, `'Jack M.A.'`, `'Jane van der Berg 旧姓 Jones'` — are exactly the diffs that sit ALONE on their rule (the 72 → 64 drop; the spec's population had 9, `'Carod i'` the ninth, its 71 → 62), but they do NOT account for all the alternatives #452 catches: they carry 24 of the 53 caught over the roster (26 of 55 over the spec's population), and the other 29 come from diffs whose rule explains other names that do not between them cover its `fields`, so the diff stays exposed while some of its alternatives are caught. Two figures are robust across all three populations, and they are what the argument rests on: the worst case, `'Ph. D., Jr.'` at 2.0.0 and 2.1.0, 30 alternatives against a five-field rule, all 30 surviving #452; and the 35 other-rule alternatives over the sole-watched rows, equally unpinned and equally silent, a handover nobody would see — the metric UNDERCOUNTS in that direction whatever it counts in the other. Both corrections leave the conclusion where it was. AMENDED 2026-09-05, later the same day: the #501 adjudication below moved six of these rows out of `_WATCHED_DIFFS`, so the population every figure in this bullet is over went from 113 rows to 107 and the roster figures above are a snapshot of the tree this arc's own commits left. Re-run by the RECOMPUTE below over the 107: 363 gross alternatives over 66 exposed diffs, 310 net of #452 over 58, 35 other-rule, with 105 / 96 / 23 at 1.4.0, 133 / 112 / 11 at 2.0.0, 125 / 102 / 1 at 2.1.0 and 0 / 0 / 0 at 2.2.0, and 8 diffs alone on their rule. Both of the figures the argument actually rests on reproduce unchanged, which is the point of having named them: `'Ph. D., Jr.'` at 2.0.0 and at 2.1.0 is still 30 alternatives against a five-field rule with all 30 surviving #452, and the other-rule count is still 35 — the two rows that left carried no other-rule alternative between them. AMENDED again since #498, which moved five more 1.4.0 rows to `_RECORDED_DIFFS`: the roster holds 102 rows, 36 of them at 1.4.0, and re-running the RECOMPUTE below over those 102 (measured 2026-09-05) gives 337 gross alternatives over 61 exposed diffs, 284 net of #452 over 53, 35 other-rule, with 79 / 70 / 23 at 1.4.0 and 133 / 112 / 11, 125 / 102 / 1 and 0 / 0 / 0 unmoved at the three 2.x baselines, and 8 diffs alone on their rule — only the 1.4.0 line moves, since only 1.4.0 rows left. Both figures the argument rests on reproduce a third time: `'Ph. D., Jr.'` at 2.0.0 and at 2.1.0 is still 30 alternatives against a five-field rule with all 30 surviving #452, and the other-rule count is still 35. RECOMPUTE: spy on `compare.classify` through a real `main()` run at each baseline, keep the default-order calls on the names keyed in `_WATCHED_DIFFS` for that ledger (the roster's own keys, not a scan of `tests/`; drop the four #501 names for the sole-watched figures), and for each enumerate the non-empty subsets of the winning rule's `fields`, sorting each by whether `classify()` routes it to the winner, to another rule, or to nothing; the #452-aware count applies the union test above, and a diff is "alone on its rule" when no other `(name, order)` call at that baseline classified to the same rule. -- **A "where is this named" population must include the tool.** The `tests/` scan cannot see `compare.py`, so four names carrying 1.4.0 contest rows — `'Bob Jones, author'`, `'Carod i'`, `'MD, PHD'`, `'van ma van'` — scored as sole-watched there, watched by exactly the mechanism the arc extends. The definition gained its last clause (not already keyed in `_RECORDED_DIFFS` for that ledger); three of the four return at a 2.x baseline where they have no contest row, and `'Carod i'` diffs under the default order at 1.4.0 ONLY, so it leaves the population entirely: 51 names, not the 52 the `tests/`-only scan gave, and 41 rows at 1.4.0 where the spec's first table said 45. (AMENDED 2026-09-05, later the same day: `'MD, PHD'` leaves entirely too once the #501 adjudication below pins its 2.x pair, since it then carries a contest row at every baseline it diffs at — so TWO of the four return at a 2.x baseline and two leave, and the population is 50 names against the same scan's 52. Re-measured by the recipe: 41 / 31 / 30 / 5 = 107 rows over 50 names, the scan's own 45 / 32 / 31 / 5 = 113 over 52 unchanged, and the roster is exactly the 50-name set.) One more clause the recipe carried and the definition did not until the design-docs review: explained by a ledger rule at that baseline — a diff no rule explains is already printed by every run as unclassified, so a rule is the only weak watcher a row is needed for; a definition clause and not a live count, since `radar unclassified` reads 0 at every baseline today and the two sets coincide. RECOMPUTE: the population recipe is in `_WATCHED_DIFFS`' header; print the names its last clause removes per ledger, and the four appear at 1.4.0 and none elsewhere. (Stale twice over by the end of 2026-09-05, and re-measured that day by exactly that recipe: the clause removes NINE at 1.4.0 — the four plus `'Bob Jones, compositeur'`, `'MD, DO, DDS'`, `'Smith, Ph. D. Jr. MD'`, `'Smith, Ph. D. MD'` and `'Smith, Ph.D. Jr.'`, the five #498 pinned — and `'MD, PHD'` again at 2.0.0 and at 2.1.0 since #501 pinned its 2.x pair, none at 2.2.0. Rows per ledger after the clause: 36 / 31 / 30 / 5.) -- **`corpus_shapes.jsonl`'s table row counted ENTRIES where the population counts NAMES.** A `(name, order)` pair is an entry, so a name compared under three orders is three, while the population is names diffing under the default order. Measured 2026-09-05 over the four baselines: 14 distinct names diff there, 11 under the default order, and the other three — `'de Mesnil Jean, Dr.'`, `'de la Cruz Juan Carlos, Dr.'`, `'de la Cruz née Vega'` — diff only under a declared order, the same trio that is the `NOT CHECKED` window at 1.4.0. The spec's row read "16 entries"; by the recipe here — distinct `(name, order)` pairs on which `classify()` was called across the four baselines — the count is 18, and the basis for 16 was not recorded, so 16 is not carried. Names and default-order names are the figures the population argument rests on, and both reproduce. RECOMPUTE with the same spy, restricted to that file's names, counting pairs, names, and names with `order is None`. -- **The per-file split is not a partition.** 49 of the 51 sole-watched names sit in `corpus_issues.jsonl` and 3 in `corpus.jsonl`, with `'dr Vincent van Gogh dr'` in both, so the per-file counts overlap by one and 49 + 3 reads 52 for a 51-name population. (AMENDED 2026-09-05, later the same day: 48 and 3 over the 50 the #501 adjudication below leaves, the same name overlapping, so 48 + 3 reads 51 for a 50-name population — the shape of the finding, which is that the split is not a partition, is what survives the digits.) RECOMPUTE from the population script's JSON against `_load_entries` over the two radar files. -- **A missing roster section was read as an empty one (found 2026-09-05, in the PR's review round).** `main()` read both shape rosters with `.get(ledger.name, {})`, so a ledger with no section checked nothing and said nothing: with the 1.4.0 key deleted from `_WATCHED_DIFFS` in memory, a full run at that baseline checked its 41 rows against nothing, printed the same 375 lines (differing only in the worker environment's path on the `baseline:` line, which differs between any two runs) and exited 0 — the guards would have caught the deletion at pytest speed, but the run may not assume the suite ran, which is `_CORPUS_TIERS`' own ground. Now refused pre-worker, beside the `both` refusal, naming the dict and the ledger; `_CORPUS_TIERS` refuses a VALUE outside `contract`/`radar` at load on the same principle, since every downstream tier read is an `!=` against one literal and a misspelling would land on whichever side it happened to fall. RECOMPUTE on a tree before the refusal: `del compare._WATCHED_DIFFS['expected_since_1.4.0.toml']` around `main()` at `--baseline 1.4.0`, and diff stdout and the exit code against an unmodified run, with two unmodified runs diffed as the noise control. -- **The tier rule a watched row follows was stated three ways and pinned by none (found 2026-09-05, in the PR's review round).** Three mutants of `tier_of` — order-None entries only, first-loaded entry always, `== "contract"` in place of `!= "radar"` — each passed the whole suite and all four gates byte-identically, because every test drove one bare-string corpus, on which the three readings coincide. What pins the rule now is two fixtures `_run_main` cannot build: the `'John Smith, Dr.'` shape (one string, contract under shape 4 in one file and radar under the default order in another, watched row moved on the default-order comparison — prints `MOVED SHAPE (radar)`, exit 0, which refuses first-loaded-always) and a declared-order-only name (a shape-4 entry with no default-order entry, watched row reporting `measured` None — a tier map over order-None entries alone has no key for it and dies mid-report). RECOMPUTE by applying each mutant to a copy of `compare.py` outside the worktree and running `tests/v2/test_differential.py`; before the two tests, all three pass it. - -Declined: - -- Relaxing `set(winners) == set(shapes)` to `<=` (2026-09-05, the spec's first section) — the first decision above in full: the run cannot read the winner roster, so the kind of a row has to be data, and the equality catches the deleted-winner slip the relaxation would legalize. The spec's own review amendment B withdrew it. -- Fatal-on-radar for snapshot rows (2026-09-05, the spec's default by inheritance from `shape_bad`) — the second decision above; the spec's own "What this costs" section accepted 52 radar names becoming able to fail a release, and amendment A refused that cost as the wrong one for a row nobody chose. Had the 51 — as measured 2026-09-05 before the #501 adjudication below, 50 after — been made fatal after all, that would have been a reversal of #468 for those names and would sit here as one. -- Pinning winners for the six #501 contests (2026-09-05) — stays #501's. The four 2.0.0 rows here and `'MD, PHD'`'s two record SHAPES, not winners: a shape move is now a finding on them, while a handover to another rule is not — file order still decides the winner, and only a winner pin sees it. #501's own objection to pinning a boundary nobody has argued stands, and the issue is scoped down by that comment, not closed. (REVERSED the same day, 2026-09-05, by the #501 adjudication below, which argued all six and moved the rows — read this bullet as what THIS ARC declined to do in its own commits and not as a standing position. What it got right is kept there: the rows recorded shapes and not winners for as long as nobody had argued the boundaries. What it got wrong is that three of the six had already been argued, in the 2.0.0 ledger's own comment, so the objection was about where the argument was written rather than about whether one existed.) -- A checked-in generator for the rows (2026-09-05) — the rows are a snapshot, and a generator that re-derives them on demand is the re-derivation hazard with a command line (mechanisms.md#RECORDED-ROSTERS). A scratch script and a recipe in the roster comment, as `_CORPUS_FLOORS` and `_CORPUS_CLAIMS` do. - -The measurement, and how to redo it. `_WATCHED_DIFFS` holds 41 / 36 / 31 / 5 rows at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, 113 in all; the 2.0.0 section is 32 sole-watched rows plus the four #501 contests (`'김민준 박사님'` and `'선생님'`, contract tier and fatal; `'田中さん 様.'` and `'田中さん, 様.'`, `corpus_cjk_tolerated.jsonl`, radar and printed — the first of that pair read contract when this was measured earlier the same day, and the period-class demotion under #cjk-comma-demotion above moved it hours later, which is the point the snapshot makes better than the prose could: a row's fatality is a property of the tier its NAME reads at, and no roster in `compare.py` records it). 51 distinct sole-watched names; every row a default-order shape, so `NOT CHECKED` names none of them. (AMENDED 2026-09-05, hours later: those four rows and `'MD, PHD'`'s two — at 2.0.0 and 2.1.0 — left for `_RECORDED_DIFFS` with the #501 adjudication below, so the roster now reads 41 / 31 / 30 / 5 = 107 over 50 names and the 2.0.0 section is 31 sole-watched rows and nothing else — since #498, 36 / 31 / 30 / 5 = 102 over the same 50, the five names it pinned at 1.4.0 having left that section alone. The figures in this paragraph were true of the tree this arc's commits left; run the RECOMPUTE at the end of it for today's.) Four mutation controls, in memory around `main()` with the worktree untouched, re-run 2026-09-05 against the amended data commit: `'QC MP'` corrupted at 1.4.0 (its only row — it does not diff at 2.0.0) prints `MOVED SHAPE (radar) expected_since_1.4.0.toml: 1 watched diff shape(s) disagree with this run` and exits 0, `'Smith Dr'` at 2.0.0 likewise; `'선생님'` corrupted at 2.0.0 prints `MOVED SHAPE expected_since_2.0.0.toml: 1 watched diff shape(s) disagree with this run` ending "This fails the run because the name is contract tier" and exits 1; `'Ph. D., Jr.'` corrupted at 2.1.0 prints the `(radar)` block and exits 0; and `'MD, PHD'` inserted into the 1.4.0 watched section beside its contest row is refused pre-worker, before any `baseline:` line, as "1 name(s) sit in both `_RECORDED_DIFFS['expected_since_1.4.0.toml']` and `_WATCHED_DIFFS['expected_since_1.4.0.toml']`", and fails `test_the_watched_roster_is_disjoint_and_names_every_ledger` naming it — the guard half wrapping `load_tool`, since that helper executes a fresh `compare` module per call and a mutation on an imported one never reaches it. A control whose corruption changed nothing would have meant an inert row; none did. The arc moved no classification: 352 / 247 / 155 / 14 intentional diffs, 0 unexplained and 0 radar-unclassified at every baseline, byte-identical summary lines before and after every commit. The quiet-tree claim that made this the moment to snapshot: measured 2026-09-05 at `4dd776a`, `git rev-list --count a965272..4dd776a -- nameparser/` is 5 commits since the tier split, moving 38 inserted and 15 deleted lines across two files, `_facade.py` (the #462 initials fix and its review rounds) and `_pipeline/_script_segment.py` (`f046d5d`, the tier-split day's own W3 demotion). PINNED TO THAT COMMIT deliberately, and NO LIVE HEAD FIGURE STANDS BESIDE IT — a first draft of this paragraph put one here as a contrast, and that number was already wrong at the commit that wrote it, which is precisely the defect the pin exists to avoid rather than an illustration of it. What can be said without a number that rots: the period-class bundle's commits after `4dd776a` touched `nameparser/_pipeline/_script_segment.py`'s comments and its W2/W3 citations — and the commit correcting this very sentence was one of them, after a draft that named two by hash had been made stale by the third — so the same range recomputed at HEAD is LARGER than the pinned one on all three counts, and it moves again with every further comment-only commit. Neither the count of such commits nor their hashes belongs here: `git log --oneline 4dd776a..HEAD -- nameparser/` lists them, and a figure that would have to be rewritten by each commit that changes it is not a figure prose can hold. So this is a POINT measurement of the tree the snapshot was taken on, and rereading it as a live one is the error to avoid — the figure goes stale by design, since the next feature is what the snapshot exists to make legible. RECOMPUTE the row counts with `{k: len(v) for k, v in compare._WATCHED_DIFFS.items()}`, the population with the script in the dict's header, the controls with `compare._WATCHED_DIFFS[ledger][name] = (...)` around `main()`, and the quiet tree with the command above plus `git diff --shortstat a965272..4dd776a -- nameparser/` — with the PINNED endpoint and not `HEAD`. A `HEAD` substitution answers a different question, and this paragraph deliberately does not state its answer. - -### differential-ledger, the #501 adjudication (2026-09-05) - -The ninth ledger arc, and the one that answers the question the arc above left open hours earlier in its own `Declined:` list — whether a contest that is MEASURED but unargued is owed a pinned winner. Six were measured across the two 2.x ledgers as this branch opened — five of the 247 diffs at 2.0.0 and one of the 155 at 2.1.0 moving a shape two or more rules admit — and all six are owed one: three of the six boundaries had ALREADY been argued, in a ledger comment, which is a place nothing in the tree can check, and one of those three arguments was false. Closing that one took a regex narrowing rather than a pin, and the narrowing then dissolved two of the six contests, which is this arc's sharpest finding about what a pin is for. Mechanics — the two rosters and their contracts, the severity rule, the population recipe, the six rows' own arguments — are owned by `tools/differential/compare.py` (`_RECORDED_DIFFS`' and `_WATCHED_DIFFS`' headers), by `tests/v2/test_ledger_guards.py` (the comment over `_CROSS_RULE_WINNERS`' two 2.x sections, and the per-row comments) and by tools/differential/README.md; these are the decisions. Every figure below was measured 2026-09-05 on this branch at `ddd8b2b`, from real `main()` runs at all four baselines with `classify()` and `_run_worker` spied, and each carries its recipe. Closes [#501](https://github.com/derek73/python-nameparser/issues/501). - -Decisions that landed: - -- 2026-09-05 #501 — a row is owed where the boundary is ARGUED ANYWHERE, not where it is argued in the guard module. The position the arc above recorded — "a row is owed when somebody ARGUES a boundary, not when one is merely measured" — is kept whole; what moves is WHERE an argument counts. Three of the six boundaries were already argued in the 2.0.0 ledger's own comment on `fix(#308/#312/#319/#320)`, which named `'田中さん 様.'`, `'田中さん, 様.'` and `'김민준 박사님'` one by one and said for each why the glued rule should win. That is a claim about which rule wins a name, and it is checked by NOTHING: `_CORPUS_CLAIMS` measures a rule's regex reach alone and cannot see a name changing hands inside it, the gate's totals are per-corpus, `order_contests` reports only wide-first nested pairs, and `classify()` is never asked the question unless a roster row asks it. `_CROSS_RULE_WINNERS` is the only thing in the tree that can check a ledger comment's adjudication, so scoping its entry criterion to arguments made in its own file left exactly the arguments it exists to check outside it. The proof is the third of the three, below. -- 2026-09-05 #501 — `'田中さん 様.'` and `'田中さん, 様.'` → `fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix`, at 2.0.0. The only one of the six that is a live two-runner contest after the narrowing: both names are admitted by the glued rule and by `fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix`, and the glued rule is written first (sorted indices 1 and 2; lines 71 and 149 of the ledger on this branch). Measured against the 2.0.0 wheel, both diff `{family, given, suffix}` — `'田中さん 様.'` reads first `'田中さん'`, last `'様.'` on the wheel and family `'田中'`, suffix `'さん, 様.'` on the tree; `'田中さん, 様.'` reads first `'様.'`, last `'田中さん'` on the wheel and the same family `'田中'`, suffix `'さん, 様.'` on the tree. Both names move through TWO mechanisms at once — the glued さん peels off 田中さん while the spaced 様 routes to `suffix` — with one rule per half, so neither rule describes the whole diff and the label is PARTIAL in #382's accepted sense. The prose test lands where the file order already does: the glued rule's own #320 clause names both strings outright — "#320 stops an ASCII period vetoing the honorific it finds there ('田中さん 様.', '田中さん, 様.')" — and no clause of the spaced rule mentions either, its examples being `'王小明 先生'`, `'김민준 씨'`, `'田中 太郎 様'`, `'田中 殿'`, `'김민준 씨.'` and `'김민준 양.'`. What the pin buys, measured: swapping the two rules in memory hands BOTH names to the spaced rule and touches nothing else — 2 of the 247 classifications move, `unexplained` stays 0 so the gate's summary line does not move, and neither rule's `_CORPUS_CLAIMS` reach can see a reorder at all. The guard fails on both names, first line "expected_since_2.0.0.toml: '田中さん 様.' diffing ['family', 'given', 'suffix'] is now explained by 'fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix', not 'fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix'." -- 2026-09-05 #501 — `'김민준 박사님'` → `fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix`, at 2.0.0, and the pin holds only because of the 2026-09-05 narrowing of the glued-peel `name_regex` in both ledgers (`a6a82d6`). Before the narrowing the glued rule reached the name on the 님 interior to 박사님 and, sitting first, took it. Measured against the 2.0.0 wheel the diff is `{family, given, suffix}`: the wheel reads first `'김민준'`, last `'박사님'`, and the tree reads given `'민준'`, family `'김'`, suffix `'박사님'` — the honorific WHOLE and standing as its own word, with nothing peeled off the name. The prose test is rules.md#W2's second sentence, "A part that is not name text — a post-nominal word standing on its own — is never the name's end: the split-off steps past it to the name word behind, and never dissects it", and that rule's witness line for this very string, `"김민준 박사님" → suffix="박사님"`. So the glued rule's title was FALSE of the name, on a contract-tier string (`corpus_cjk.jsonl`), held green by every guard in the suite from the date the comment's own parenthetical carries (2026-08-05) until this arc. What the pin buys, measured: restoring the pre-narrowing regex ALONE — one edit, no reorder, the glued rule still sitting ahead of the spaced one — hands the name straight back, 1 of the 247 classifications moving with `unexplained` still 0, and this row is the only thing that says so. The guard's first line is "expected_since_2.0.0.toml: '김민준 박사님' diffing ['family', 'given', 'suffix'] is now explained by 'fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix', not 'fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix'." -- 2026-09-05 #501 — `'선생님'` → `fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots`, at 2.0.0. Measured against the wheel the whole diff is `{family, given}` — the wheel reads first `'선생님'` and the tree reads family `'선생님'`, the 2.1 order flip on a hangul string and nothing else — which is exactly the shape that rule's comment claims for itself, "these are one diff shape -- pieces moving between given/middle/family on a native-script CJK name", and which rules.md#W2's own boundary line `"선생님" → family="선생님"` says is name text rather than a peel. The glued rule's claim on it was the same regex accident, the interior 님, and worse than a partial label: it declares `suffix` and NO `suffix` moves here at all. It never won, the order rule being written first (sorted index 0 against the glued rule's 1), and the narrowing has since taken its admission away. Recorded because the pair is the #498 class and nothing else can see it: the two rules' `fields` INTERSECT on `{family, given}` — `{family, given, suffix}` against `{_ambiguities, family, given, middle}` — with neither nested, so `undeclared_contests` looks past them in either arrangement and no `precedes_narrower` block is available to declare them. What the pin buys, measured: a reorder ALONE now moves nothing here (0 of the 247), so the row's whole work is against a RE-WIDENING — restore the wide regex and swap the two rules and 2 classifications move, `unexplained` staying 0, with the guard's first line "expected_since_2.0.0.toml: '선생님' diffing ['family', 'given'] is now explained by 'fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix', not 'fix(#271/#272/#298) native-script CJK: …'". -- 2026-09-05 #501 — `'MD, PHD'` → `fix(#296) a credential-only comma string reads a name and its postnominal`, at 2.0.0 AND at 2.1.0, as a row apiece rather than one shared row. Both #296 rules reach the string and declare EQUAL `fields` at each 2.x baseline, `["title", "suffix"]`, so `fields` cannot separate them at all and file order is the whole decision (sorted indices 25 and 26 at 2.0.0, 12 and 13 at 2.1.0). Measured against both wheels the diff is `{suffix, title}` — title `'PHD'` on the wheel becomes suffix `'PHD'` on the tree, family `'MD'` on both sides. The distinguishing clause is a property of the STRING and not of the rules' `fields`: the credential-only rule opens "'Jr., PhD' / 'MD, PHD': a string that is nothing BUT credentials", while the lone-post-comma rule is written for "A one-word name, a comma, one short word". `'MD, PHD'` holds no name, so the first rule's prose is true of it and the second's is not. Two rows and not one because the roster is keyed per ledger and a shape is baseline-relative — the 1.4.0 row on the same string and the same winner rests on a different measurement, `{family, given, suffix, title}` there, the comma-family move not yet having landed. What the pin buys, measured: swapping the two rules moves 1 of the 247 at 2.0.0 and 1 of the 155 at 2.1.0, `unexplained` staying 0 at both, and the guard fails at BOTH baselines, first line "expected_since_2.0.0.toml: 'MD, PHD' diffing ['suffix', 'title'] is now explained by 'fix(#296) a lone post-comma credential is a suffix', not 'fix(#296) a credential-only comma string reads a name and its postnominal'." and the same sentence under `expected_since_2.1.0.toml`. -- 2026-09-05 #501 — the glued-peel `name_regex` is NARROWED so it stops matching a listed honorific INTERIOR to a longer listed one, in BOTH ledgers that ship it (`expected_since_1.4.0.toml` and `expected_since_2.0.0.toml` carry it byte-identically). `(? fix(#271/#272/#298), the order flip with no comma routing …` — and carries `middle`, which neither name moves. The measurement is what settles the pair, not the table. What the pin buys, measured: moving the loser ahead hands both over and moves 9 of the 352 — the two, plus seven collateral names off `fix(cjk-comma-honorific-peel) glued honorific peels off a post-comma given name`; dropping `given` or `family` from the winner also hands both over — to `fix(cjk-comma-compound)`, this pair's loser — but at the price the nakaguro bullet above measures, since it is the same rule and the same two drops: 43 moved and 35 of them UNEXPLAINED. So here too the reorder is the edit whose cost stays inside this ledger's explained set. -- 2026-09-05 #498 — `'田中さん, Dr.'` → `fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots`, over `fix(cjk-glued-honorific-peel) glued honorific peels into suffix`, at 1.4.0: the SECOND unowned pair on one name, and the reason fifteen rows sit on fourteen names. `{family, given, middle}` against `{family, given, suffix}`, intersecting in `{family, given}` without nesting (sorted indices 0 and 34). Measured, `suffix` is `''` on both sides and the tree's family is the whole token `'田中さん'`: the glued さん is NOT peeled, where the peel this loser is named for would have taken family to `'田中'` and put さん into `suffix` — which is what does happen to `'田中さん, 様.'` in the next bullet, so the contrast between a mechanism that fired and one that did not sits inside this arc's own measurements. Its stated scope excludes the string anyway, the rule being written for a name with "no space before the honorific and no comma anywhere", and this name has a comma. What the pin buys, measured: moving the loser ahead hands the name over and moves 15 of the 352, much the widest of the seven pairs, the peel rule then also taking names off `fix(cjk-comma-compound)` and `fix(cjk-comma-honorific-peel)`. A `fields` narrowing does NOT buy this pair: dropping `given` or `family` from the winner sends the name to `fix(cjk-comma-compound)` and not to this loser, so a reorder is the only single edit that hands it over. -- 2026-09-05 #498 — `'田中さん, 様.'` → `fix(cjk-comma-compound) comma routing compounds with the CJK order flip`, over `fix(cjk-honorific-suffix) postnominal honorifics recognized, compounding with the CJK order flip`, at 1.4.0. `{family, given, title, suffix}` against `{family, given, middle, suffix}`, intersecting in `{family, given, suffix}` with `title` and `middle` keeping either from containing the other (sorted indices 33 and 35). Measured against the wheel: first `'様.'` → given `''`, family `'田中さん'` → `'田中'`, suffix `''` → `'さん, 様.'`, a diff of `{family, given, suffix}`. A genuine TWO-MECHANISM compound, and both halves are legible in the values — the glued さん peels off `田中さん` while the post-comma `様.` leaves `first` for `suffix`, the two arriving there together. The winner describes the comma half; the other half belongs to `fix(cjk-glued-honorific-peel)`, which is nested inside the winner and DECLARED over by a `precedes_narrower` block, so that half of this name's contest is justified where it stands and only the honorific-suffix pair is owed a row. That is the partial label #382 accepts, with every half of the compound owned between the two rules. The loser is the weaker candidate for the same half on its own terms: its prose is about a SPACED trailing honorific whose remainder takes an order flip, it disclaims the glued forms outright — "GLUED diffs (田中さん, 김민준씨, and the 김씨 that segmentation already reached) never match this rule" — and it says nothing about a comma, where the comma is the winner's subject; measured, no `middle` moves and no order flip occurs, `given` going empty rather than gaining a name. What the pin buys, measured: moving the loser ahead hands the name over and moves 2 of the 352 — this name, which comes off `fix(cjk-comma-compound)`, this bullet's winner, and `'田中さん 様.'`, which is the one that comes off the glued rule; a `fields` narrowing buys nothing here either, dropping `given`, `family` or `suffix` from the winner sending the name to the glued rule, so the reorder is again the only single edit. (The 2.0.0 section of `_CROSS_RULE_WINNERS` pins this same string to the GLUED rule. That is a different ledger with a different rule set, where the pair nests narrow-first and no declaration is owed — the #501 adjudication above — and it does not contradict this 1.4.0 verdict.) -- 2026-09-05 #498 — `'Bob Jones, compositeur'` → `fix(comma-family) a comma followed only by titles keeps the given/family split`, over `fix(comma-precomma-family) pre-comma run reads as family, not given`, at 1.4.0. EQUAL `fields`, both `{given, family}`, so file order is the whole decision (sorted indices 5 and 13). Measured against the wheel: first `'Bob Jones'` → given `'Bob'`, last `''` → family `'Jones'`, and title `'compositeur'` on BOTH sides — diff `{family, given}`. The winner's prose names the string and describes those values to the token, "the pre-comma name keeps its positional read -- first 'Bob Jones' -> given 'Bob', family 'Jones', the title staying where it was"; the loser's stated behavior, the pre-comma run reading as family, is half true at most here, `'Jones'` becoming the family and `'Bob'` not, the run not moving whole. The loser's own comment already concedes the sibling string `'Bob Jones, author'` to the winner on exactly that reasoning, and this row records the same for `compositeur`, which the winner's regex reaches by the same literal. It is also why a pin is spelled with the FULL issue string: three rules in this ledger begin `fix(comma-family)` (measured 2026-09-05 with tomllib over the ledger's `[[change]]` issues). What the pin buys, measured: moving the loser ahead moves 3 of the 352 — this name plus `'Bob Jones, author'` and `'John Smith, Mr.'`, and the three do not come off one rule: this name and `'Bob Jones, author'` come off the WINNER, and `'John Smith, Mr.'` off a third rule whose issue extends the winner's, `fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example`. The lone-post-comma rule, the third of the three rules whose issue begins `fix(comma-family)`, loses nothing here — which is the full-issue-string point made in measured form rather than by counting rules that share a prefix. Dropping `given` or `family` from the winner hands this name over on its own, moving 2 (this name and `'Bob Jones, author'`) with nothing left unexplained. -- 2026-09-05 #498 — `'MD, DO, DDS'` → `fix(#296) a dropped prenominal takes the name position it occupies`, over `fix(comma-family) lone post-comma piece routes to suffix/title, not first`, at 1.4.0. `{title, given, middle}` against `{given, title, suffix}`, intersecting in `{given, title}` — which is the measured diff — with neither containing the other (sorted indices 7 and 12). Measured against the wheel: title `'DO'` → `''` and first `''` → given `'DO'`, with family `'MD'` and suffix `'DDS'` byte-identical on both sides. The winner names the string and states its diff verbatim, "the rule's second name is 'MD, DO, DDS': title 'DO' -> given 'DO', the comma anchor's one corpus hit", which is the whole of what moved. The loser is about a one-word NAME followed by a comma and a lone credential; this string holds no name at all and has two post-comma pieces, and its direction is wrong as well — the loser routes a piece OUT of `first` where this name routes one IN. What the pin buys, measured: moving the loser ahead moves 8 of the 352 — this name, all six of the `fix(#325)` pair above, and `'Smith, Jr., PhD'`, since a move above index 7 necessarily passes `fix(#325)` at index 10 — so the reorder is the strictly larger edit here. The `fields` narrowing is smaller and still not an alternative: dropping `given` or `title` from the winner moves 2, handing `'MD, DO, DDS'` to this loser and sending `'Do Quang Minh'` — a radar-tier `corpus_issues.jsonl` name the winner is the sole explainer of — to NO rule, so radar unclassified goes from 0 to 1. Both edits cost something the pin does not; the pin is what records which rule is right without paying either. - -Found rather than decided, and worth as much: - -- **The class is measured in NAMES, not pairs, and the issue's framing had it in pairs.** 60 pairs overlap without nesting and 11 more have equal `fields`, each sharing a corpus name; measured with a diff in hand, 14 names have one that two such rules admit, over seven of those pairs. The static count over-reports for the reason #382's nine-of-eleven finding gave — most co-matched names' real diff needs a role the other rule does not declare — so the rule-order arc's objection to a widened STATIC predicate, that it would demand 111 justifications where the real number is eleven, does not reach a MEASURED one. That is the whole of why option 3 was cheaper than it read. -- **Seven of the fourteen sit in an EQUAL-`fields` pair — the clearest case of the class the docs name `_CROSS_RULE_WINNERS` the instrument for — and not one of the seven was pinned.** Two pairs, and in both `fields` separates the rules not at all and file order is the whole decision: `fix(#325) a split credential followed by another suffix after a one-word family comma reads as suffixes` ahead of `fix(comma-family) lone post-comma piece routes to suffix/title, not first` over six names, and `fix(comma-family) a comma followed only by titles keeps the given/family split` ahead of `fix(comma-precomma-family) pre-comma run reads as family, not given` over `'Bob Jones, compositeur'`. Equal `fields` is the special case the 2026-09-02 entry reasons FROM — the class it leaves to the instrument is every pair where neither set contains the other, and the other seven names sit in that wider half, over eight of the fifteen rows. The instrument was named there and checked by nothing until this arc, and its clearest cases were the unowned ones. -- **Fourteen names, SEVEN (winner, loser) pairs, fifteen (name, pair) rows — and a first draft counted five, by grouping the rows by WINNER.** `fix(#271/#272/#298)` heads three distinct pairs, against the delimited-nickname rule, the comma-compound rule and the glued-peel rule, which collapse to one under a by-winner count; and `'田中さん, Dr.'` is unowned against TWO losers, so it carries two of the fifteen rows and appears under two of the verdict bullets above. Seven pairs over fifteen rows is what the sweep reports. The fourteen is a count of NAMES, which is the unit the pins are written in — one row per name in each roster, whatever the number of losers behind it — and the three figures answer three different questions rather than disagreeing. - -Declined: - -- The wider narrow-first predicate (2026-09-05, the scope decision above) — 9 more names at 1.4.0 and 23 in all, declined as a pin the declaration-free default already justifies. -- A `precedes_narrower`-shaped per-pair declaration for non-nested pairs (2026-09-05, the issue's option 1) — the roster is per-name and already carries the argument, and a second key would let a PAIR be declared without any name's shape being measured. That is the #501 adjudication's own finding one level down: an argument nothing can check, given a key of its own. -- A static sibling of the check (2026-09-05) — there is none to write, for the reason the first decision above gives: without a diff the population is the 71-pair reach — 60 overlapping pairs and 11 with equal `fields` — and the roster is unwritable. -- Reordering any of the seven pairs (2026-09-05) — a reorder moves which rule classifies a name, which is a behavior change to the ledger and breaks `_CROSS_RULE_WINNERS`; every current winner is argued where it sits, and the seven verdict bullets above are that argument, name by name. A wrong winner would have been closed by a narrowing or a declared reorder and not by a pin; the adjudication found none. - -The measurement, and how to redo it. `_RECORDED_DIFFS` reads 45 / 5 / 1 / 0 = 51 rows at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0 and `_WATCHED_DIFFS` 36 / 31 / 30 / 5 = 102 over the same 50 distinct names — five names moved rosters at 1.4.0 and stayed watched at both 2.x baselines, where each has one admitter and no contest to own, so only the 1.4.0 row count fell and the population did not move. 32 of the 45 rows at 1.4.0 sit on radar-tier names and 13 have a single admitter at their recorded shape; none of the fourteen joins those 13, every one having two or more admitters at its recorded shape by the definition of the sweep that found it. THE POPULATION FIGURES, recomputed: drive `main()` at each baseline with `classify()` wrapped, keep the calls whose issue is not None, and for each list the rules satisfying `compare._entry_matches` at that call's measured shape and order — fewer than two admitters is no contest; for each admitter after the first, the winner's `fields` a strict subset of the loser's is narrow-first and justified, the loser's a strict subset of the winner's WITH the loser's issue in `compare._declared_over(winner)` is declared and justified, and anything else is owed a pin unless the name is a key of `_RECORDED_DIFFS` for that ledger. That gives 47 / 3 / 1 / 0 contested, 18 / 3 / 1 / 0 pinned and 14 / 0 / 0 / 0 owed on the tree this arc opened on, and 0 owed at every baseline since. Do NOT replay the corpus load by hand: the `(name, order)` dedup, the baseline-minimum skip and the tier stamp all happen inside `main()`, so a row's tier is read as `main()` reads it — the default-order entry's where there is one, the first-loaded entry's otherwise — by wrapping `_run_worker` for the entries it was handed. THE VERDICTS' values come off that same run — the old side from the worker's rows for the entries `main()` was handed, the new side from `HumanName` and `_tree_parse` as `main()` computes them — and only the facade is compared at a pre-2.0 baseline, `_surfaces_for` returning `{"facade"}` below 2.0.0, so every old and new value in the bullets above is `HumanName`'s. THE "WHAT THE PIN BUYS" FIGURES are measured the same way and their recipe sits beside the rows in tests/v2/test_ledger_guards.py: move the loser above the winner in a COPY of the run's own sorted rule list and re-classify the run's own 352 classified calls, counting the labels that change; the `fields` readings are that recipe with one role dropped from the winner instead of a rule moved, and nothing on disk moves for either. MUTATION CONTROL, in memory around `main()` at 1.4.0 with the worktree untouched: `'Smith, Ph. D. Jr.'` deleted from `_RECORDED_DIFFS['expected_since_1.4.0.toml']` — the contract-tier row of the six-name equal-`fields` pair — makes the run exit 1 and print `UNPINNED CONTEST expected_since_1.4.0.toml: 1 contested pair(s), one per name and losing rule, that no rule outranks by nesting or declaration and no row pins`, naming the name under its winner as `'Smith, Ph. D. Jr.' [contract] ['suffix', 'title']` with the reason "equal `fields`, file order the whole decision". Unmutated, no `UNPINNED CONTEST` block prints at any of the four baselines; a corruption that changed nothing would have meant an inert check, and this one did not. The arc moved no classification: 352 / 247 / 155 / 14 intentional diffs, 0 unexplained and 0 radar unclassified, exit 0, summary lines byte-identical at every commit of the branch — a pin moves no classification, and neither does a check that fires on nothing. RECOMPUTE the row counts with `{k: len(v) for k, v in compare._RECORDED_DIFFS.items()}` and the same over `_WATCHED_DIFFS`; the radar and one-runner splits by taking each row's tier as above and classifying it at its recorded shape against `_sorted_rules`. - -### differential-ledger, a pin names one rule (2026-09-05, #508) - -The eleventh ledger arc, and a small one: it changes how a `_CROSS_RULE_WINNERS` pin is read, not what any pin says. The winner guard compared `classify()`'s answer to the pin with `startswith`, chosen without a recorded reason, so a pin was satisfied by any rule whose issue extended it. The #498 completeness check, earlier the same day, had borrowed that guard's meaning outright — a `_RECORDED_DIFFS` key counts as "a winner is pinned" because the two rosters' keys are equal — and its review found the case where the borrowed meaning was hollow. Mechanics are owned by `_pinned_rule`'s docstring in tests/v2/test_ledger_guards.py; these are the decisions. Every figure was measured 2026-09-05 on master at `449a7bc`. - -Decisions that landed: - -- 2026-09-05 #508 — **a pin names exactly one rule.** The guard resolves each pin to one rule's issue — exact where a rule's issue equals the pin, else the single rule whose issue starts with it — and refuses a pin that resolves to none or to several; the winner comparison is then EQUALITY against the resolved issue. Prefix pins stay: a family prefix that reaches one rule names it as well as the full string does, and 17 of the 51 rows are written that way. Measured over the 51 rows before the change: 31 full issue strings reaching only themselves; 17 short prefixes reaching one rule; 2 full strings that are also a strict prefix of a sibling's — `'Bob Jones, author'` and `'Bob Jones, compositeur'`, pinned to the all-titles rule whose issue `... keeps the given/family split, the C1 example` extends; 1 short prefix reaching three rules — `'Andrews, M.D.'` on the bare `fix(comma-family)`. All three ambiguous rows are in `expected_since_1.4.0.toml`, the 2.x rosters have none, and each was latent: `classify()` returned the intended rule. `'Andrews, M.D.'` is spelled out to the lone-post-comma rule that wins it; the Bob Jones rows are unchanged, exact comparison being what their comment already rested on. A second guard, `test_every_pinned_winner_names_exactly_one_rule`, refuses an ambiguous pin on its own at pytest speed, so the failure names the pin rather than a classification. RECOMPUTE: for each ledger and each pin, count the `[[change]]` issues that `startswith` the pin and whether one equals it (the loop is in the #508 issue body). - -Found rather than decided: - -- **The #498 arc's "a key in `_RECORDED_DIFFS` means a winner is pinned" was true of 48 rows and hollow for 3.** Nothing was wrong today, because `classify()` happened to return the intended rule for all three. What the prefix comparison could not do was FAIL when it stopped: the `'Bob Jones, compositeur'` roster comment rested its pin on the C1 rule's regex being unable to reach the string, an argument no test executed — measured 2026-09-05, forcing `classify()` to answer the C1 string for that name passed the old guard and fails the new one. - -Declined: - -- Exact match everywhere (2026-09-05) — rewriting the 18 short pins to full strings and ending the family-pin idiom. The idiom is harmless where the prefix reaches one rule, which is 17 of the 18, and the resolver makes the eighteenth loud; a reworded rule breaking its pins is the same outcome under either reading. -- Prefix plus an allowlist (2026-09-05, #508's option 2) — a second list to keep honest, for three rows the resolver handles without one. -- A word-boundary prefix (2026-09-05, #508's option 3) — answers the two Bob Jones rows and not `'Andrews, M.D.'`, and adds a tokenising rule to a string comparison. -- Refusing two rules in one ledger from sharing a prefix (2026-09-05) — a property of the ledger, not of a pin; the ledger's rule families are deliberate (`fix(comma-family)`, `fix(#296)`), and the resolver already makes the only case that matters loud. - -### differential-ledger, retired for a snapshot (2026-10-13, #664) - -The ledgers' last arc. They were built to show that 2.0 changed nothing against 1.4 except fixes the release log lists, and they did; as a standing check every later behavior change had to be classified once per ledger, and the guards policing the ledgers outgrew the ledgers. - -Decisions that landed: - -- 2026-10-13 #664 — **a committed snapshot replaces the classified gate.** `tools/differential/snapshot.jsonl` records every corpus name's reading (the seven role fields, ambiguity kinds, `initials()`, and the facade where it differs), one line per (name, order); `tests/v2/test_snapshot.py` fails on drift and prints the moves grouped by changed field. A change that moves readings re-records, and the PR diff of the snapshot is the list of names it moved. Nothing is classified: what a reviewer reads in the diff is what a ledger rule used to say in a regex. The six `expected_since_*.toml` ledgers and `tests/v2/test_ledger_guards.py` are deleted, and with them the contract/radar tiers, so the CJK corpus is one file again and the rules corpus harvests tolerated rules' examples too. -- 2026-10-13 #664 — **`compare.py` stays, unclassified.** It keeps the released-wheel runner and its checks against comparing one tree with itself (the version tell, the tree-side path check, the stripped `PYTHONPATH`; mechanisms.md#VERSION-TELL, #GENERATED-SCRIPT-OUTSIDE-THE-WORKTREE), prints the same grouped report as the snapshot and exits 0. It answers "how did release X read these names" for a release tagged before the snapshot existed; after that, `snapshot.py --since ` is the cheaper answer. -- 2026-10-13 #664 — **the CLDR agreement ratchet stays** (#654). It answers a different question, agreement with CLDR's stated fields rather than stability. - -Measured 2026-10-12, before the change, over the 40 first-parent commits ending at 04cc0814: 5,236 of 25,226 changed lines (21%) were ledger, `compare.py` or harness-test edits, and 21 of the 40 commits touched them. The share is uneven rather than steady: across the 29 of those commits that touch `nameparser/` it runs from 0% to 82%, median 11%, above 30% on 11 of them (#655: 236 of 448). Run against the code before and after #655, the snapshot reports five moved names, against that PR's 236 lines of ledger and guard edits; merging master afterwards, it reported #663's one new corpus name where #663 had added the same ten lines to each of six ledgers. RECOMPUTE (shares): `git log --first-parent -40 --format=%h 04cc0814`, and for each commit sum `git show --numstat` over every path except `*.jsonl` and `cldr_agreement.json`, the harness share over paths matching `expected_since|compare\.py|ledger_guards|test_differential|_differential_fixtures`; a commit "touches `nameparser/`" when any path starts with it. - -Accepted costs: - -- No forced explanation per moved name. A reviewer can approve a large snapshot diff without reading it; the grouping by changed fields is the mitigation. -- No bound on what a change may reach. `_CORPUS_CLAIMS`, the dormancy checks and the contest pins existed only to police ledgers; mechanisms.md#LEDGER-RULE-SEPARATION and #CROSS-RULE-OUTCOME-PINS are retired with them. -- The 1.4 compatibility promise is a question asked when needed (`compare.py --baseline 1.4.0`), not a gate at every release. The facade is deleted in 3.0. - -Declined: - -- Trimming the ledgers rather than retiring them (2026-10-13). The radar tier (#468, 2026-09-01) was that attempt: it stopped new corpora from failing the run, but every contract name still needed a rule in every ledger, and the cost grew by one ledger per release. - -### comma-suffix-arc — #291/#296/#316 (2026-07-26 → 2026-08-01) - -#291 was filed 2026-07-26 out of the 2.0 vocabulary cleanup, with -its decline-by-measurement evidence in the issue body; "z domu" folded into it 2026-07-27 (see Excluded, MAIDEN_MARKERS). Intent for #291 and #296 is settled by the approved bundle spec; #296 (with #325) SHIPPED 2026-08-23 as the bundle's first three commits, with one deviation from the audit table recorded at C1 ('md' keeps dual membership — bare 'Md' is Muhammad); #291 was UNSHIPPED then, carried as a deviates: marker on C1 (#291 closed 2026-08-24; the marker came off with #564 in 2.4). The arc's bookkeeping: #291/#296 moved milestone v2.1 → v2.2 on 2026-08-01 (with #289/#293); the 2026-07-30 design doc was amended in place 2026-08-01 (A1–A6); its durable content is HERE rather than in the gitignored doc; #316 (trailing titles) was filed the same day, carrying the esq cleanup as a Related section. - -The approved 19-word TITLES∩suffix audit table (2026-07-30/08-01, intent for the unshipped bundle): jr, junior, phd, md, do and se DROP from TITLES (se: Structural Engineer is the PE/SE post-nominal pair, no prenominal convention — v1 residue); dr drops from SUFFIX_WORDS and sra from the suffix sets (v1 residue); sr, lt, cpl, cpt, cpo, csm, sgm, ra and vc KEEP dual membership (position decides); ms and sa keep dual membership AND join the ambiguous set (the periods gate). Derek's framing thesis for the whole table: v1's sets encoded where the v1 PARSER needed words to be, not where words can occur. Amendment A6: once SUFFIX_PHRASES ships, the glued peel's token-level post-nominal test cannot step over a phrase segment ("김민준씨, LEED AP" will not peel) — #291 owns deciding that, with a case row either way. - -- 2026-08-01 (plan amendment 1, Derek's call: "ship + repair the merge") — dropping dr/ms from the suffix vocabulary flips segment() to FAMILY_COMMA and degrades "John Smith, Dr." to family="John Smith" (mechanisms.md#VOCABULARY-FEEDS-STRUCTURE is this shape); the approved repair, ordered FIRST in the bundle so no commit introduces a regression a later one heals: assign segment 0 positionally when segment 1 is nothing but titles. - -Declined: - -- Splitting the dead multi-word suffix entries into single-word entries (2026-07-26, the evidence in #291's body) — measured: - "Smith, A.P." has the suffix steal the given initials; "John - Leed" and "Mary Nicet" lose family names; and the period-gate escape is equivalent to removal because nobody writes "L.E.E.D.". This decline is why the seven removable entries were REMOVED in 2.0 rather than split, and it is the missing history behind C1's - #291 marker. - - Amended 2026-08-23 (#291 closed working-as-designed): the decline stands as measured — splitting into single words does cost `Smith, A.P.`, `John Leed` and `Mary Nicet` — but the inference drawn from it did not. "A multi-word *entry* is inert" was read as "a multi-word *credential* is unreachable", and that is false: the run predicate `is_wholly_suffix` reassembles adjacent suffix tokens, so `parse("John Smith, MD PhD").suffix` has been `'MD PhD'` since 1.4.0, and `psm i`/`psm ii` — two of the seven entries called dead — parse today from their component words. The third option neither the issue nor the spec considered is to leave the shipped vocabulary alone and let callers add the component words to a `Lexicon`, which is what was chosen. - - Also declined with it: `SUFFIX_PHRASES` and segment-level matching (unnecessary once the run predicate is measured), and amendment A6's glued-peel question (no phrase-matching unit, so `_is_post_nominal`'s token-level test stays correct by construction). -- The trailing-abbreviation structural fallback — with a measurement LIMIT rather than a measurement: the differential corpora structurally cannot evidence it, because they hold only names someone wrote down, and an unrecognized abbreviation is by definition outside the vocabulary. A green run there proves nothing (the reusable harness fact is in mechanisms.md's field notes). -- SUFFIX_PHRASES matching in assignment — measured cost: it renders suffix="LEED, AP", because the suffix view comma-joins suffix words unless they carry the stable "joined" tag, which grouping applies within a piece and — since #436/#437, 2026-09-06 — post_rules applies between two post-nominals the writer did not comma-separate. The general form: multi-word vocabulary must merge where the render tag is applied, not where the role is assigned. - -### api-churn-declines — do not re-propose without new cause - -Declined (2026-07-06, the post-1.3.0 modernization sweep; each on churn-outweighs-benefit grounds): the hn.C → constants rename; capitalization-knob consolidation; the logger rename. (Lenient matching, declined in the same sweep, is recorded under comparison-surface.) - -### initials-repertoire — is_initial across scripts (#320) - -- The principle: decided per script from CLDR's LOCALE-AUTHORED name patterns rather than from the script's typology — ja and ko author 29 and 32 patterns and use an initial in none of them, while ru uses one in 7 of 39 (`_policy._NO_INITIALS`' comment carries the counting recipe and the trap: `initialPattern` alone reports a period convention for locales that have none). Corrected 2026-09-10, #cjk-full-stops: this read "alphabetic-vs-CJK, not Latin-vs-non-Latin — Han, kana and hangul characters are morphemes or syllables, so a single one cannot stand in for a name word", which is the phonological restatement that comment forbids in as many words — Devanagari is an abugida and Arabic an abjad, neither has letters in that sense and both abbreviate, so a rule keyed on typology would put them here and their initials are real. _NO_INITIALS is ENUMERATED per script rather than derived from the script ranges, so a new Script member forces a decision instead of silently deciding Thai has no initials. - -Declined: - -- Narrowing is_initial to [A-Za-z] — measured: "Й." regresses to the #267 Ukrainian-conjunction misreading (the exact regression 2.1.0's release log claims to prevent), and the stable "initial" tag silently strips from every Cyrillic, Greek, Arabic and Hebrew initial, invisibly to the fields. - -### comparison-surface — why value equality died, and what replaced it - -- 2026-07 (1.3.0 arc, #223/#224; executed in the v2 core — measured: HumanName == HumanName is False, matches() and comparison_key() live, the facade cites #223) — the trilemma: case-insensitive equality, equality-with-plain-strings, and hashability are mutually inconsistent (a hash cannot equal both hash("John Smith") and hash("john smith")), so 1.2.0's design was unfixable in place, not merely disliked. Additional costs of the old surface: equality ran through str(self) and so depended on mutable string_format, and maiden was invisible to == (absent from the default format). -- 2026-07-06 — the cross-constants asymmetry, pinned: a str operand is reparsed with self.C; a HumanName operand is compared as already parsed. - -Declined: - -- Component-based __eq__/__hash__ — workable, considered, rejected: it still privileged one equality semantic for a domain that has none, and kept the mutate-while-hashed hazard. -- Lenient matching (initials-compatible, Bob/Robert) — declined WITHOUT an issue, deliberately: matches() is exact-components by design. Someone will propose could_match() within a year; this entry is the resolved-as-no they should find. - -### R2 — the all-particles family_base divergence - -- 2026-08-18 #404/#385 — R2's DISCRIMINATOR REVERSED. It said a family written wholly out of particle vocabulary has a base where one of those words "is itself borne as an ordinary surname", and that only the words that are never anyone's name stay particles. That is a vocabulary test, and it is the wrong one: a particle earns its name by joining forward to the word it modifies, so a particle with nothing to join is not doing a particle's work whatever it is borne as. POSITION decides, and the borne-as-surname question does not arise. - The invariant that replaces it, and the reason this is worth a rule rather than three bug fixes: a non-empty family always has a non-empty base. A particle needs a base to attach to. Measured, 43 of 751 corpus names x 3 orders violated it — "Del Toro" under FAMILY_FIRST reported family 'Del' with no base, losing the surname from the base and from the initials. - Marked rather than untagged (mechanisms.md#MARK-DONT-STRIP): `particle` is stable API and means "from the particle vocabulary wherever it lands", which stays true; UNJOINED_TAG records what was decided. That also leaves #405's ambiguity emitter a predicate to key on, which stripping would have destroyed for exactly the never-given half where the fork is most interesting. - Blast radius: 44 of 2,253 parses move and every one is VIEWS-only — the seven role fields are byte-identical everywhere, so the differential harness cannot see this change at all (it compares roles only). Tests are the whole verification, and the invariant is one of them. Amended 2026-09-02 (#484): the harness now compares `initials()` under the `_initials` pseudo-field for names whose roles agree, so it CAN see this change — the 27 names are classified by the `fix(#385/#402)` rule at 1.4.0, 2.0.0 and 2.1.0. Tests are no longer the whole verification. - -- Recorded 2026-08-16, intent UNVERIFIED: the v1-era design held that a family name cannot be only particles ("Anh Do" — Do is a surname AND a particle), so last_base was guarded non-empty. The facade still guards (HumanName("Anh Do").last_base == "Do"); the v2 core does not (parse("Anh Do").family_base == "", family_particles == "Do"), and the surname vanishes from initials (parse("Anh Do").initials() == "A."). -- 2026-08-16 #385 RESOLVED by the collision criterion, not on its own terms: the issue's option 3 ("guard the view only when a word is vocabulary-ambiguous") is what decisions.md#vocabulary-collisions produces when applied here. - "Do" is borne as an ordinary surname, so it is ambiguous - vocabulary and anchors the base; "van der" is never anyone's name, so an all-particle family there genuinely has no base. The two rows of the issue's table were never one case. This is the keystone's clearest payoff: #385 was filed as a leaf with three options and no way to choose between them, and the criterion picks one without arguing about family_base at all. -- ANSWERED 2026-08-18 by the entry above, and recorded rather than left implicit: "Do" does NOT remain in family_particles once it is the base. The two views partition the family — parse("Anh Do").family_particles is "" and family_base is "Do" — which is asserted as a partition over the whole case table, not just as an example. Was: "Still open inside the resolution: whether Do remains in family_particles once it is also the base. Recorded here rather than left to the implementing PR to decide by accident." -- 2026-08-16 (pre-merge coherence pass) — the resolution moves R3 too, and R3 now carries its own marker. Initials read the BASE family word, so anchoring "Do" changes parse("Anh Do").initials() from "A." to "A. D." while - "Juan van der" stays "J." (no borne name, no base, and initials - of a bare particle run would be nonsense). The general lesson, worth more than this instance: a deviates: marker gets written on the rule whose STATEMENT changed, but a rule can change another rule's OUTPUT without touching its statement, and nothing looks for that — the runner asserts per example line, so an unmarked downstream rule stays green precisely because its own examples avoid the affected input. When adding a marker, walk the changed rule's `interacts:` targets and ask whether any of THEIR examples move. - -- 2026-08-29 #461 — TRIED AND BACKED OUT; the question is back on the issue. The unjoined mark readmits the words of an all-particle part to `initials()`, and #461 narrowed that readmission to the `particle` tag alone, on the authority of R3's "A CONJUNCTION never initials, so a base that is one contributes nothing even then". It was reverted before merge, in the same PR. What the narrowing got wrong is best said in terms of the MARK rather than of the fields. The mark is a statement about a whole PART — none of its words is doing the work its tag names — and #461 honored that statement for some of the part's words while keeping one of them out. Under `Lexicon.default().add(particles={'y'})`, `parse("Juan de y")` is such a part: `family_base` is `de y`, both words carry the mark, and #461 initialed `J. d.`, admitting the `de` as the name word the mark makes it and refusing the `y`. R2's reasoning does not split that way. A particle with nothing left to join is not acting as a particle, which is the whole reason the mark exists; a conjunction inside that same part has nothing left to join either, for the same reason and at the same moment, so it is a name word of the part and initials with the rest. Scope that carefully, because its general form is far wider than the claim and would be wrong: this is about a part the mark has ALREADY turned into name words, not about any conjunction that happens to join nothing. Outside such a part the skip stands whatever the conjunction is or is not joining, and none of those moved or may move — `parse("Juan y Garcia")` initials `J. G.` with the `y` a middle name word by P3, `parse("Juan Velasquez y Garcia")` initials `J. V. G.` over base `Velasquez y Garcia`, `parse("Jon Dough and")` initials `J. D.` over base `Dough and`. Base and initials differ in every one of those, legitimately, and R3's own `"Juan de y" → initials="J."` line is a fourth, its `family_base` being `y` under the default vocabulary. So "the base holds a word the initials do not" is NOT the criterion and was not the finding; the mark is. What is therefore in question is R3's "even then" clause, which carries the carve-out into the one part where the joining has stopped — not the code that failed to implement it. The clause is left standing and #461 now asks whether it belongs there; nothing else in R3 moved, and the paragraph #461 added under it is gone. R4's conjunction sentence rests on that clause by name ("the carve-out R3 states for initials") and is untouched here, so whoever settles #461 settles R4's cross-reference with it. Note what the backout cannot record: a `deviates:` marker hangs on an example LINE and this shape has none to hang on — `particles` and `conjunctions` are disjoint in the default vocabulary and in every locale pack, so no input string the doc runner parses reaches it. rules.md itself therefore carries no trace, and this bullet with mechanisms.md#RENDER-HONORS-THE-PARSE is where the gap is written down. It is not the only gap under R3's conjunction sentence, and the other is older, wider and markable: "A CONJUNCTION never initials" is unqualified, while a conjunction in the GIVEN group has always initialed — `parse("John and Jane Smith")` gives `J. a. J. S.` and `parse("Duke of Edinburgh")` `D. o. E.`, 25 of the 1094 corpus names, all reachable from the default vocabulary. Recorded here because it was found here; it is not #461's to fix. RESOLVED into the normative document on 2026-08-29, in the review of PR #463: rules.md#R3 now scopes its carve-out to the middle and base family words and declares the given group unsettled, in prose and WITHOUT a `deviates:` marker. The marker was considered and rejected on its own definition — it states the INTENDED value, and there is none to state. R3 counts name words while P3 makes a connective and its neighbours ONE name word, so `John and Jane Smith` has four candidate answers and no entry anywhere picks one: `J. a. J. S.` today, `J. J. S.` reading the carve-out as written, `J. S.` reading P3's join as a single name word, and `J a J. S.` at 1.4.0 (measured on the released wheel, which joins the run rather than skipping the conjunction). Marking it would put an invented value in a normative document and hold the parser to it, since the runner asserts the today-value strictly. The marker's `#N` slot has no owner either — the sentence above disclaims #461 for this gap, and filing its own issue is a maintainer's call rather than a docs commit's. What "markable" meant two sentences up is the MECHANICAL property that an input string exists to hang a marker on, unlike the #461 shape where none does; it was never a claim that the intended value is known. Neither direction is visible to the gate: 0 of the 1094 corpus names move either way, and the counts hold at 229 / 194 / 102 with 0 unexplained at 1.4.0 / 2.0.0 / 2.1.0. Both views of the contested token are pinned rather than left to prose — `test_repair_keeps_a_conjunction_lowercase_in_a_particle_part` holds R4's ungated conjunct, gating which had passed the entire suite until #461's test existed, and `test_initials_readmits_a_conjunction_in_a_particle_part` holds today's initials answer the way a `deviates:` marker would, so re-deciding #461 fails the suite until that pin moves with it. - -- 2026-09-20 #461 — AMENDS the 2026-08-29 bullet above, which said of four readings that "none of those moved or may move". Two of the four moved and two did not, and the half that survived is the half the criterion was always about. WHAT MOVED: `parse("Juan y Garcia")` initials "J. y. G." where that bullet recorded "J. G.", and R3's own `"Juan de y"` example line reads "J. y." where it read "J.". WHAT DID NOT: `parse("Juan Velasquez y Garcia")` is still "J. V. G." over base "Velasquez y Garcia", and `parse("Jon Dough and")` is still "J. D." over base "Dough and" (all four measured 2026-09-20 on this branch). The criterion that separates them is the one the bullet above named, and it is still not a word count: the question is asked of the WHOLE PART, and a part holding another name word for the connective to join — the part's working particles set aside — is a part where the connective is doing a connective's work and contributes nothing. "Velasquez y Garcia" and "Dough and" each hold such a word, so the skip stands there; the family "y", the family "de y" and the middle "y" of "Juan y Garcia" hold none, so the connective is its part's own name word and initials with it. - So that bullet was RIGHT ABOUT THE MECHANISM and wrong only about the clause it preserved. Its finding — that a mark honored for some of a part's words and not for one of them is honoring nothing — is exactly what rules.md#R3 now states, for all three groups at once, and it is why the fact is computed ONCE over the whole part rather than re-derived per word by each view. What changed is the clause it left standing: R3's "A CONJUNCTION never initials, so a base that is one contributes nothing even then" is conditional now, and "none of those may move" was the reading of the day rather than a decision. decisions.md#R3's 2026-09-20 bullet carries the rule as taken. The pin that bullet describes did its job: `test_initials_readmits_a_conjunction_in_a_particle_part` was written to fail when #461 was re-decided, and it failed. - -Declined: - -- 2026-08-18 — the GROUPING half of #404: a particle run that joins nothing does not chain, so "Jong van der" would split into middle 'van' plus family 'der'. Measured and rejected, though NOT for the reason first recorded here. The first draft said the split makes the family "no longer all-particle so the base fix stops firing" — false, and `der` and `la` are both shipped particles, so a family of either IS all-particle and the rule fires on it ("Juan Smith der" gives base 'der'). What the split actually costs is the SCOPE of the base and a stray particle relocated: grouping can decline to merge but cannot keep the words apart, because roles re-assemble them and two adjacent same-role pieces are one part at the field level. "Juan Smith van der" becomes middle 'Smith van', family 'der' — a base of 'der' rather than 'van der', and a middle name nobody wrote. Keeping the run whole in one part is what gives the base its full extent. The split reading needs the leftover distribution to know these are separate units, which is mechanisms.md#UNIT-PARTITION's problem. - -### R3 — initials honor the fold - -R3's earlier history is under `decisions.md#R2`, which this entry does not repeat: the #461 backout and the unsettled given-group conjunction question are recorded there because they were decided alongside R2's all-particle clause. This entry is R3's own, and R3's `history:` points here. - -- 2026-08-30 #408 — DONE: `initials()` orders folded words first, as every other view of the same parse already did. O3's fold and P6's attachment both TAG rather than move a token (mechanisms.md#FOLDED_TAG — spans cannot reorder), and `_types._text_for` partitions on that tag for every role it renders, so the `family` field reads `van der` where the tokens sit `der ... van`. `initials()` walked `tokens_for(role)` in written order and never asked, so the two views disagreed about one parse: `parse("der, y van")` gave family `van der` and initials `y. d. v.`, and it gives `y. v. d.` now. This is mechanisms.md#RENDER-HONORS-THE-PARSE's second shape, filed as its worked instance, so the design is not re-argued here — only what the measurement added to it. - WHAT THE MEASUREMENT ADDED, and it reframes the issue from a consistency fix to a parity restoration. v1 had this feature as `CONSTANTS.middle_name_as_last` (`_config_shim` maps it to `Policy.middle_as_family`) and had NO general `name_order`, so only the two default-order policies have a v1 reference at all. Measured on the released 1.4.0 wheel over the 1094-name deduped corpus, from a script run outside the worktree that asserts `nameparser.__version__` first: at the default policy 1 name moves and agrees with 1.4.0 neither before nor after; under `middle_as_family` 71 move, of which **54 come into exact 1.4.0 agreement and NOT ONE moves away from it**. Worked pair: `Doe, Dr. John A.` gives `J. A. D.` at 1.4.0, gave `J. D. A.` here, and gives `J. A. D.` again; `Brundridge, Contessa A` likewise `C. A. B.` / `C. B. A.` / `C. A. B.`. The facade had been giving the folded answer all along — `HumanName.initials()` reads `first_list`/`middle_list`/`last_list`, and `_facade._list_for` prepends the carriers exactly as `_text_for` does — so the core view was simultaneously out of step with the field beside it, with the facade above it, and with 1.4.0 behind it. A view that stopped honoring a record had been reproducing a v1 bug v1 did not have. - THE 17 THAT AGREE WITH 1.4.0 NEITHER WAY, recorded because the obvious explanation is wrong — and the FIRST REPLACEMENT for it was wrong the same way, corrected by this branch's docs review before merge. It is not that their parse already differs: only 2 of the 17 parse differently from 1.4.0 under this policy, `Smith, John V.` (#432's dotted numeral) and `남궁민수 지훈` (the unspaced-hangul surname split and the script order, rules.md#W1). The other 15 have SEVEN BYTE-IDENTICAL FIELDS at 1.4.0 and here — `der, y van` INCLUDED, which an earlier draft of this bullet filed with the two above by carrying over a measurement taken at the OTHER policy, where its parse does differ. Of those 15, 14 differ from 1.4.0 only in v1's initials GRANULARITY — v1 initials one group per `*_list` element, so a run its parse joined into one element (`Velasquez y Garcia`, `abdul salam`) yields `V G.` where 2.x, one token per word, yields `V. G.` — and strip the periods and spaces and all 14 come into 1.4.0's letter ORDER after this change, none having been in it before (`JVGQ` → `JQVG`). The 15th is `der, y van`, and it is worth naming rather than lumping: its fields match 1.4.0 exactly and its initials do not, because 1.4.0 contributes NOTHING for a family that is all particles (`van der` → `y.`) where 2.x contributes its words. That is R3's own adjudicated clause and decisions.md#R2's territory — R3 carries `"Juan van der" → initials="J. v. d."` against 1.4.0's `J.` — so this one name is a divergence this document has already decided in favor of, not an unexplained residue. So the parity claim is stronger than "54 of 71 and the rest unexplained": 68 of the 71 come into 1.4.0's ordering, 54 of them exactly, one more is R2's decided divergence, and 2 have a parse difference that predates this change. - BLAST RADIUS by policy, over the same corpus × three `name_order` values × `middle_as_family` off/on (6564 parses, 660 move). Default order: 1 / 71. FAMILY_FIRST: 1 / 294. FAMILY_FIRST_GIVEN_LAST: 1 / 292. STATE THE POPULATION BESIDE THE COUNT, per this file's own rule: the shape an order change can be observed in is a contributing folded token standing AFTER a contributing unfolded one of the same role, and at the default order the corpus holds exactly 1 such name without `middle_as_family` and 72 with it. So the movers are 1 of 1 and 71 of 72 — a near-total hit rate over a corpus that carries one instance of P6's half, not a blast radius near zero. (The one non-mover is `Chang, Andy C I`, where the folded `C` and the family `Chang` share a letter and the reorder is invisible in initials.) Nine corpus names carry the tag at all at the default order; the other eight have a single contributing family token, which makes the partition the identity. The `middle_as_family` half is where the reach is, which follows from the producers: O3 folds every middle word, while P6's attachment moves initials only where the attached run is READMITTED by the unjoined mark — a family that is all particles — so it reaches one corpus name in each order and the same one. RECOMPUTE, and the COMPARATOR is the half that has to be stated, because the obvious phrasing of it measures nothing: parse the `tools/differential/corpus*.jsonl` files deduped under each of the six policies and compare `initials()` against THE PRE-CHANGE RENDERING — the same contributing tokens left in WRITTEN order. What this sentence said until 2026-08-30 was "against a folded-first partition of `tokens_for(role)`", which is what `initials()` now IS: that recipe compares the shipped implementation with itself and yields 0 differing parses, the inert-measurement shape (mechanisms.md's Verification shapes section) written into the very sentence meant to prevent it, and the only recompute path the repository offered for docs/release_log.rst's counts as well. CONTRIBUTING is the other half of the recipe and it comes FIRST: apply this view's own skip filter — outside the given group a token tagged `particle` or `conjunction` contributes nothing unless it also carries UNJOINED_TAG — and compare the two orders only over what survives it. Leave the filter out and every skipped word reads as a difference instead: 1546 rather than 660 against written order, and 1018 rather than 0 against the folded partition. Re-measured this way on 2026-08-30 and reproducing the counts above: 660 of 6564 parses, 1 / 71 at the default order, 1 / 294 and 1 / 292 under the two family-first orders. - THE VERIFICATION LIMIT, and it is decisions.md#R4's, restated because a plausible reading of "one name moves at the default policy" is that the ledger owes it a rule. It does not, and could not: `tools/differential/compare.py` compares the seven role fields plus `_ambiguities`, and `initials()` is not among them — the same blindness recorded for `capitalized()` under R4. Measured rather than reasoned: the gate was run at all three baselines on the pre-change tree (exported with `git archive HEAD` into a scratch directory, so nothing in the working tree moved) and on the changed one, and the two outputs are byte-identical apart from the tree path and the baseline-worker temp path — 1094 corpus names, 229 / 194 / 102 intentional diffs, `unexplained: 0`, and 67 / 58 / 51 headings with identical per-heading counts, at 1.4.0 / 2.0.0 / 2.1.0. NO LEDGER RULE WAS ADDED, and none was needed. What stands in its place is `rules.md#R3`'s example line and `tests/v2/test_render.py`, both mutation-checked. Amended 2026-09-02 (#484): the blindness this paragraph rests on is retired — the gate compares `initials()` for roles-identical names now, so an initials-only change DOES owe a ledger rule. #408 still owes none, and for a different reason than the one recorded here: its movement is 1.4.0 parity, and the one corpus name it moves at the default order (`der, y van`) diffs on ROLES at all three baselines, under `fix(#379)`, so the pseudo-field never enters. The semantics are in `decisions.md#differential-ledger, the initials view`. - WHY R3 GAINED A SENTENCE rather than leaning on R1. R1 already says folded family words "render before the rest of the family wherever they stood in the string" — but its subject is "Every FIELD", and `initials()` is not a field: R3 calls it "this view", and it takes a format spec and two separators rather than being read as an attribute. So R1 does not reach it by its own words, and #408 is precisely what leaving that to inference costs. R3's new clause states the order in R3's own voice, carries the O3 example line (`"Hassan, Mohamad Ahmad Ali"` under `middle_as_family` → `initials="M. A. A. H."`, reusing R1's own input so `corpus_rules.jsonl` is unchanged at 241 names), and R1 and R3 now name each other in `interacts:` — the field says which order, the view says it follows the field. P6's half gets no example line, for the same reason R1 carries none: the only input that witnesses it is `der, y van`, whose PARSE is itself contested, so pinning its initials normatively would pin a value another open question can move. It is pinned in the unit test instead, which is where today's-behavior pins belong. - THE PARTITION IS PER ROLE, not scoped to FAMILY, and this is a deliberate choice rather than a copied line. Both producers re-role to FAMILY, so FAMILY is the only role a parse can put the tag on today and the GIVEN and MIDDLE arms are unreachable. They are there because `_text_for` partitions for every role it renders and the two views must not diverge again — the same uniformity `_post_rules`' own UNJOINED_TAG loop takes for its three roles, "for uniformity with the rule, not because all three are observable". A producer that ever folded into another part would otherwise reopen #408 there with nothing to fail. `test_initials_folds_in_every_role_it_renders` pins it from a hand-built name, since no input string can. Mutation-checked four ways — scoping the partition to FAMILY, and skipping it for GIVEN, for MIDDLE, for FAMILY — and each fails that test. It took THREE drafts of the fixture to make that true, and the two misses are the same miss one role apart. The first carried two GIVEN tokens and ONE MIDDLE: a one-element partition is the identity, so the MIDDLE arm was asserted and unpinned, and skipping the partition for MIDDLE passed the entire suite. The second fixed MIDDLE and carried NO FAMILY token at all, in a test named for every role it renders — so skipping the partition for FAMILY passed this test, and was caught only by its siblings and by R3's example line, neither of which is about per-role application. A zero-element group is the identity too, and reads even less like a gap than a one-element one, which is why the second miss survived a review that had just named the first. Two tokens per role is what closes it, and the general form is carried as mechanisms.md#TWO-ELEMENT-GROUPS rather than left here: a test written to pin a partition, a sort or a dedup needs at least two elements in every group it claims to cover, or the claim rides on a no-op. -- 2026-09-01 #462 — DONE, the facade's twin: `HumanName.initials()` dropped a dotted or bare-capital `E`/`Y` from the middle and family groups because `_process_initial` tested conjunction membership alone, where 1.4.0's `is_conjunction` was "in the set AND NOT `is_an_initial`". `Scott E. Werner` gave `S. W.` from 2.0.0 through 2.2.0 and gives `S. E. W.` again; `John E Smith` (bare capital) and `Juan Y. Garcia` likewise; `parse().initials()` never had the bug, since `_classify` tags `E.` an initial and the render reads the tag (mechanisms.md#RENDER-HONORS-THE-PARSE). The issue said the bare form was "still correctly dropped", which is true of lowercase `e` and false of bare capital `E` — 7 of the 14 corpus names that move are bare capitals. Restored with `_render._INITIAL`, v1's own `initial` shape, because the facade may import `_render` and not `_pipeline`; scoped to `_process_initial`, its only caller, so a future reader of `_is_conjunction` does not inherit a decision made for initials. Found by #484's pseudo-field at the 1.4.0 baseline, where it had sat as 14 unreported diffs; the gate could not see it before because the seven fields never moved. -- 2026-09-13 #528 — DONE: `HumanName.initials()` reads the parse's tags. The facade's initials view was the last derived view still re-deciding the CONNECTIVE question after the parse had answered it — scoped to that question deliberately, since the PARTICLE question is still re-decided from the lexicon in two places by decision, `_cap_word`'s particle conjunct (`decisions.md#R4`'s NOT DONE bullet) and `_is_particle` here, R4 drawing that boundary per question rather than per view. #462's fix above is why the connective half was still being re-decided: restoring v1's `is_conjunction` ("in the set AND NOT `is_an_initial`") restored a SHAPE test standing in for a tag, which agreed with the parse only while the parser agreed with the shape. #383/#479 ended that the same day it landed — rules.md#P3 now reads a one-case single-letter connective from vocabulary rather than from case, and no shape over the raw word can see it — so the two views of one parse disagreed: `HumanName("john e smith").initials()` gave "j. s." against the core's "j. e. s.", and `HumanName("JUAN Y GARCIA").initials()` gave "J. Y. G." against the core's "J. G.". Both give the core's answer now. This is mechanisms.md#RENDER-HONORS-THE-PARSE applied to the last view that had not taken it, and #458's principle reaching the facade layer. - THE SHAPE IS R4's, not a new one. A word backed by a token asks `"conjunction" in tok.tags`; a token carrying `UNCLASSIFIED_TAG` — text spliced in by `hn.middle = ...` through `ParsedName.replace()`, or restored by the v1 pickle path in `__setstate__` — was read by no parse, so the vocabulary answers, through `_render._reads_as_conjunction`, the same helper `_cap_word` calls for the same tokens. One question and not two: whether a word is a connective is a fact the word can answer alone, while whether a particle is acting as a particle is a fact about the whole part, which is R4's own boundary and is why `_is_particle` stays a live vocabulary lookup here. `_is_conjunction` lost its only caller and is gone, which retires the last sentence of the 2026-09-01 entry above — there is no future reader of it to inherit a decision made for initials. - A STATED PRIVATE-METHOD BREAK, accepted rather than worked around. `_process_initial` keeps v1's `(name_part, firstname=False)` signature for direct callers and gains an optional `tokens=`, and `_initials_lists` always passes it — so a v1-shaped subclass that overrides `_process_initial` with the two-argument signature now raises `TypeError` the first time `initials()` runs. The alternative considered was a string wrapper kept over a token core, which would leave such an override silently INEFFECTIVE instead: it would be called, and its answer discarded by the token path beside it. A loud break on a private hook beats a quiet one, and it is stated in the method's own comment and pinned by a test. FORWARDING IS STILL THE ONLY WAY TO RECEIVE THE FIX: an override must ACCEPT `tokens` **and forward it**, `def _process_initial(self, name_part, firstname=False, tokens=None): return super()._process_initial(name_part, firstname, tokens=tokens)`, which gives `Forwards("john e smith").initials()` the fix's own answer, "j. e. s.". Widening the signature alone does not receive the fix either — `**kwargs`, or an explicit `tokens=None` that the super() call drops. Revised 2026-09-14, before this paragraph ever reached master: the token path now passes the group's own text as `name_part` (`" ".join(tok.text for tok in group)`) rather than an empty placeholder, precisely so an override that ignores the `tokens` keyword falls onto the STRING path and behaves as it did before this upgrade, rather than going quietly blank. Measured 2026-09-14: `WidensOnly("john e smith").initials()` is "j. s.", the pre-#528 answer, where the forwarding override above and the library itself both give "j. e. s." — the STRING path's vocabulary fallback reads the middle "e" as the connective and drops it, where the token path's parse-backed reading (restored by #528) does not. So "accept a `tokens` keyword" is still the wrong instruction on its own; the forwarding is what receives the fix, not merely what avoids a `TypeError`. This is the only break in #528; on the public v1 surface nothing moves but the initials of six corpus names: `john e smith`, `john e jones`, `jones, john e`, `jose e maria santos`, `JUAN GARCIA Y LOPEZ` and `JUAN Y GARCIA`, the six the ONE CORPUS NAME paragraph below counts. - - A SECOND STATED BREAK, FOUND AND FIXED IN REVIEW (2026-09-14): before #528, `_initials_lists` read `self.first_list`/`self.middle_list`/`self.last_list` — public properties a v1 subclass may override — and the move to `_list_tokens_for(member)` above dropped that: an overridden property was silently ignored by `initials()` alone, while `last_base` (via `_split_last`, which reads `self.last_list`), `surnames` (`self.middle_list + self.last_list`) and `given_names` (`self.first_list + self.middle_list`, which never reaches `last_list` at all) kept honoring it. Fixed by checking, per member, whether the class overrides that property (`getattr(type(self), f"{member}_list") is not getattr(HumanName, f"{member}_list")` — three class-attribute identity checks, nothing on the parse path) and, where it is overridden, taking the pre-#528 STRING path over the override's own strings instead of the token walk. Measured 2026-09-14 against `git show 338daf7:nameparser/_facade.py` (before #528): a three-property `Sub` (`first_list` uppercasing every element of `super().first_list`; `middle_list` dropping every element of `super().middle_list` that starts with `"X"`; `last_list` always returning `["Zorro"]`) on `"john Xavier smith"` gives "J. Z." both there and here — reading only the `last_list` override into the recipe predicts "j. X. Z." instead (measured 2026-09-14 with `last_list` alone overridden on this tree), since the upper-cased first and the filtered-out `Xavier` both feed the result too — where the unfixed tree gave "j. X. s."; a one-property `Sub` (only `last_list` overridden, same `["Zorro"]`) on `"john e smith"` gives "j. e. Z." — the un-overridden first/middle keep the (post-#528) token path's own answer, and only the overridden last takes the pre-#528 fallback. Honoring the override, rather than leaving it silently ignored, is why "This is the only break in #528" above still holds: a second silent break on a PUBLIC surface would have gone unnoticed by every existing test, unlike the private-method break above, which is loud by design. Pinned at `tests/v2/test_facade.py::test_initials_honor_an_overridden_list_property`. - ACCEPTED COST, MEASURED (2026-09-13, this branch): for a word backed by a token the connective answer is fixed at parse time, as `capitalize()`'s already was. `C = Constants(); h = HumanName("juan y garcia", constants=C)` gives "j. g."; `C.conjunctions.remove("y")` leaves it "j. g." with no re-parse, where before #528 it gave "j. y. g." immediately; `h.full_name = "juan y garcia"` re-parses and gives "j. y. g.". Unpinned before this change and pinned by it, at `tests/v2/test_facade.py::test_initials_freeze_the_connective_answer_at_parse_time`. - ACCEPTED COST, THE SECOND ONE, and it is the no-parse paths rather than the configuration, THREE of them, not one: state restoration (pickle, `copy.copy`, `copy.deepcopy`) and the keyword constructor all skip the tokenizer, so every word they carry takes the vocabulary fallback instead of a parse's tag. `__setstate__` stamps every restored token `UNCLASSIFIED_TAG`, because a v1 pickle carries the `*_list` STRINGS and no tags, so a restored name is spliced text throughout; `copy.copy` and `copy.deepcopy` use the same `__getstate__`/`__setstate__` state hooks, so a copy takes the identical fallback (measured 2026-09-14). The THIRD producer, unrecorded until this review round: `HumanName(first=..., middle=..., last=...)` routes straight through the field-setter properties in `__init__` and never calls `_apply_full_name`, so its tokens are `UNCLASSIFIED_TAG` for the identical reason a restored pickle's are — not a new bug, a third route to a cost already accepted. `HumanName(first="john", middle="e", last="smith").initials()` gives "j. s." where `HumanName("john e smith").initials()` gives "j. e. s."; `HumanName(first="JUAN", middle="Y", last="GARCIA").initials()` gives "J. Y. G." where `HumanName("JUAN Y GARCIA").initials()` gives "J. G." (measured 2026-09-14). A round-tripped, copied or deep-copied `JUAN Y GARCIA` therefore gives "J. Y. G." where the live parse gives "J. G.", and a round-tripped, copied or deep-copied `john e smith` gives "j. s." where the live parse gives "j. e. s." (the pickle round trip measured 2026-09-13, copy and deepcopy measured 2026-09-14, all three giving the identical two values). `capitalize()` has disagreed with the live parse on exactly those two names for exactly that reason since the tag was introduced — "Juan Y Garcia" against "Juan y Garcia", "John e Smith" against "John E Smith", all four measured the same day — and the keyword constructor gives the identical disagreement, `HumanName(first="john", middle="e", last="smith").capitalize()` leaving "John e Smith" against the live parse's "John E Smith" on both this tree and 338daf7 (measured 2026-09-14) — so this is the existing shape reaching one more view rather than a new one, and the remedy for the state-restoration pair is the one `__setstate__`'s own comment gives: the restored state is a v1 blob, not a parse; the keyword constructor has no state to restore an argument from — the caller supplied fields, not a string, so there was never a tokenizer run to have tagged them. RECOMPUTE ACROSS THE CORPUS, ALL THREE PATHS TOGETHER: rebuild every one of the deduped `tools/differential/corpus*.jsonl` glob's 1174 distinct names from its own parse's seven keyword-constructible fields (`first`, `middle`, `last`, `title`, `suffix`, `nickname`, `maiden`) and diff `initials()` against the live parse — 1 of 1174 differs before #528 (`Ph. D., John`, the pre-existing merge the ONE CORPUS NAME paragraph below names, unrelated to this mechanism) and 7 now, the added six being the same six `initials()` movers named throughout this entry. Pinned at `tests/v2/test_facade.py::test_initials_of_an_unpickled_or_copied_name_ask_the_vocabulary_too`. - THE POPULATION, MEASURED 2026-09-14: the deduped `tools/differential/corpus*.jsonl` glob holds two line shapes — `{"name": ...}` objects in `corpus.jsonl` and `corpus_shapes.jsonl`, bare JSON strings in the other four — so reading both and deduping is what gives its 1174 distinct names; a dict-only read stops at 528. Over that glob, exactly 6 give a different `initials()` after a pickle round trip than the live parse, up from 0 before #528 — the same six the STATED BREAK paragraph above names: `john e smith`, `john e jones`, `jones, john e`, `jose e maria santos`, `JUAN GARCIA Y LOPEZ` and `JUAN Y GARCIA`. `copy.copy` and `copy.deepcopy` move the identical six over the same glob, confirming the state hooks are one mechanism rather than a pickle-specific accident. - ONE CORPUS NAME STILL DIVERGES and it is not this rule's. Measured over the deduped `tools/differential/corpus*.jsonl` glob (1174 distinct names) before and after: seven names had the two views disagreeing, six moved here, and `Ph. D., John` remains — the facade merges "Ph." and "D." into ONE list element (v1's `fix_phd`) and renders "P D" with the separator and no inner delimiter, giving "J. P D." against the core's "J. P. D.". That is `fix(initials-per-word) the Ph. D. merge`, a 2.0.0 rendering change ledgered at 1.4.0 since #484, and #528 preserves the element boundaries exactly so it neither moves nor is absorbed. RECOMPUTE by parsing every name of the glob on both surfaces and diffing `initials()`; the before half needs the pre-#528 `_facade.py` and `_render.py` on the path, which `git show` writes into a scratch copy of the package — never a checkout in a shared worktree. - DIFFERENTIAL. The facade's `_initials` is compared at every baseline, 1.4.0 included, and the core's from 2.0.0; the pseudo-field enters a name's diff only where every role and every ambiguity kind agrees. So at 1.4.0 FOUR of the six take a new rule (`fix(#528) the facade's initials follow the parse's connective tags`) and two do not — `jose e maria santos` and `JUAN GARCIA Y LOPEZ` move roles against that baseline, and #383/#479's role rule explains them. At 2.0.0 through 2.2.0 nothing new: the e-names' `_ambiguities` diff keeps `_initials` out of their diff, and `JUAN Y GARCIA`'s single `_initials` row survives with the facade half of its two causes closed, which `fix(#462)`'s dated paragraph in `expected_since_2.0.0.toml` now says. At 2.3.0 the existing rule stands and both surfaces move together. All five gates re-run at 0 unexplained on 2026-09-13. - -- 2026-09-20 #461 — DONE, and the clause is settled for every group at once. THE RULE AS TAKEN: a connective contributes nothing where it is JOINING — a part holding another name word for it to join, the part's working particles set aside — and where its part holds nothing else it is joining nothing, so it initials like any other name word and agrees with the base. That picks ONE of the four candidate answers the 2026-08-29 bullet under `decisions.md#R2` listed for "John and Jane Smith" and makes the other three unreachable: `parse("John and Jane Smith").initials()` is "J. J. S.", not the "J. a. J. S." of 2.0 through 2.3, not P3's one-name-word "J. S.", and not 1.4.0's run-together "J a J. S.". THE CRITERION IS NEVER A WORD COUNT, which is what keeps `parse("Jon Dough and")` at "J. D." over base "Dough and" and `parse("Juan Velasquez y Garcia")` at "J. V. G." — a part of two words where one is the connective is still a part where it joins, and a part of three is no different. - BOTH VIEWS, ONE MARK. The fact is about a PART, so it is decided once where the parts are settled and recorded on the token (`UNJOINED_CONJUNCTION_TAG`, mechanisms.md#MARK-DONT-STRIP), and the core view and the v1 facade both read it. That is the direct answer to what the first #461 attempt found: a mark honored for some of a part's words and not for one of them is honoring nothing. A SEPARATE marker rather than a widening of `UNJOINED_TAG`, because that one is also read by `family_particles`/`family_base` and by `_cap_word`, and none of those three may move. `Parser.revise` recomputes both marks, the same reason `_remarked` recomputes the first. - BLAST RADIUS, MEASURED 2026-09-20 against the parent commit `46651750` over the deduped corpus-union-cases set under eight configurations, both processes asserting their own `nameparser.__file__`. RE-MEASURED 2026-09-20 at the end of the second review round, the population having grown again with its own rows. Over the population AS IT STANDS — 1568 non-empty names — the default order moves 12 names' roles, 3 more names' reports with the roles fixed, and 56 names' `initials()`, 22 gaining a letter and 34 losing one; `HumanName.initials()` moves on THE SAME 56 names with THE SAME values. Family-first moves 11 / 3 / 34 and `FAMILY_FIRST_GIVEN_LAST` 12 / 3 / 64. The twelve role movers at the default order are "Carod i Rovira, Josep", "Doe, Jane nee Puig i Soler", "Henry i of England", "Jane Doe nee Puig i Ma", "Jane Doe nee Puig i Soler", "Josep Carod i Rovira", "Josep Carod i Rovira III", "Josep Carod i de Rovira", "Josep Lluis Carod i Rovira", "john smith i jr", "maier, amy i, jr." and "rovira, i" — seven of the twelve are rows this bundle added, which is why the restricted figure below is the comparable one. Restricted to the 1543 names that stood in the corpus at the parent, the same sweep reads 2 / 3 / 46 at the default order, 18 gaining and 28 losing, with 1 / 3 / 24 family-first and 2 / 3 / 52 given-last — unmoved by the second review, whose report change touches only "JOHN QUINCY SMITH I", a name the round itself added. (The 1558-name sweep that stood here read 6 / 3 / 50 with 19 gaining and 31 losing, and 5 / 3 / 28 and 6 / 3 / 58 for the two family-first orders; it is a dated snapshot over a smaller population.) RECOMPUTE by parsing every name of the deduped `tools/differential/corpus*.jsonl` glob plus `tests/v2/cases.py`'s texts under those eight configurations in two processes, one on this tree and one on the parent's, and diffing the seven role fields, the ambiguity kinds and `initials()` on both surfaces. The gate cannot answer this on its own: its `_initials` pseudo-field sees only names whose roles stayed put, so a count taken from the classified summary is a FLOOR. - TWO OF THE GAINS ARE 1.4.0 PARITY RESTORED, and that is what retires two ledger rules: `JUAN Y GARCIA` gives "J. Y. G." and `محمد و علي` gives "م. و. ع.", which is what 1.4.0 gave for both. The two ranges differ and an earlier draft of this bullet ran them together, so state them apart (measured on all five wheels 2026-09-20): `محمد و علي` gave "م. ع." at 2.0.0, 2.1.0, 2.2.0 AND 2.3.0, while `JUAN Y GARCIA` gave "J. G." at 2.0.0, 2.1.0 and 2.2.0 and went back to 1.4.0's "J. Y. G." at 2.3.0 — the reason being the one the 2026-09-13 #383/#479 entry under P3 records, and the 2.4.0 release note says the same. The readmission is a parity RESTORATION on those two names rather than a deviation, and the given-group half in the other direction is the deviation — see rules.md#R3's Accepted clause, which states both. - THE FRAME BUDGET WENT DOWN: `tools/perf/call_count.py` reads `py3.11 parse=406.00 facade=443.00` where the parent read `412.00 / 449.00`. The two marks now come off ONE walk over the roles instead of two generator expressions, and on 3.11 a genexp costs a frame for its creation and one per resume, so the reference name's two non-empty parts pay exactly six. `_CALL_BASELINE[(3,11)]` is `{"parse": 410, "facade": 447}` with `_BAND = 0.02`, so the bands are [401.8, 418.2] and [438.06, 455.94] and both figures land inside: NO baseline edit and no `decisions.md#parse-cost` decision is owed. PEP 709 inlined comprehensions and not generator expressions, so 3.12 through 3.15 should drop by the same six. - THE NO-PARSE PATHS KEEP THE OLDER READING, the same degradation the 2026-09-13 #528 bullet above accepted and for the same reason: a token the parse never saw carries no mark, so the view falls back to the vocabulary. Measured 2026-09-20, `HumanName("JUAN GARCIA Y LOPEZ").initials()` is "J. G. L." live and "J. G. Y. L." after `pickle`, `copy.copy` or `copy.deepcopy`; a name built from keyword fields carries no tags either, and so does one whose `*_list` property a subclass overrides. The remedy is the v1 one — assign `full_name` again — since the facade has no `revise`. - HOW FAR IT REACHES, over a STATED population rather than an example, because this bundle MULTIPLIES it and the earlier wording did not say by how much: over the deduped `tools/differential/corpus*.jsonl` glob plus `tests/v2/cases.py`'s texts — 1,568 non-empty names — the v1 `initials()` of a pickled, copied or deep-copied name differs from the live parse's on 34 names here against 6 at the parent 46651750, measured 2026-09-20 in two processes each asserting its own `nameparser.__file__`. #461's lone-connective half is what multiplies it: every part whose only word is the connective initials live and drops after the round trip, so `HumanName("Juan de y").initials()` is "J. y." live and "J." after `copy.deepcopy`. RECOMPUTE by building that population and comparing `initials()` before and after each of the three state paths. A larger figure measured over a generated grid says nothing this does not — the population is what the number is about. -- 2026-10-02 #585 — DONE: an initial is the shortest start of its word that composes to the composed word's first character, written in the form the writer used. `initials()` on both surfaces had taken a token's first CODE POINT (`t.text[0]` in `_render.initials`, `word[0]` and `w[0]` in `_facade`), which for a name typed decomposed (NFD) is a base letter without the mark that completes it: `émile zola` initialled `e. z.` against `é. z.`, and — the reach the issue did not name, found by the corpus walk's control — decomposed katakana lost its dakuten, `マイケル ジャクソン` initialling `マ. シ.` against `マ. ジ.`. `_render._first_letter` is the one helper both surfaces call. One shape has no such start: a mark composing with nothing that canonical order puts between the letter and a mark that does (`o` + U+0331 + U+0301, whose composed spelling is `ó` + U+0331), where the initial is the composed character decomposed, `o` + U+0301 — found by the docs review, 2026-10-02. That fallback is for a start LONGER than one code point: a first code point that is a whole character Unicode excludes from composition (`क़`, U+0958, which NFC writes `क` + a nukta) is kept as written, its nukta with it, as before this change; the second review found the first draft of the fallback stripping it (`ज़ाकिर` initialling `ज.`). And the fallback writes the character wholly decomposed even where the writer composed part of it (`ẹ` + U+0331 + U+0302 gives `e` + U+0323 + U+0302) — the twin holds, the written form does not, accepted as contrived. WHY NOT "THE FIRST LETTER AND THE COMBINING MARKS AFTER IT", the shape #542 used for case repair and the issue's own option 1, measured before writing it: decomposed hangul jamo are LETTERS (category Lo), not marks, so it would initial a decomposed `김` as `ᄀ` where the composed name gives `김`; and a Bengali vowel sign IS a mark (Mc), so it would move a COMPOSED `কিরণ` from `ক.` to `কি.`, which is a question about Indic syllables nobody has decided, and a change the gate would see on every composed Indic name. Composing the prefix answers both: a composed word gives `text[0]` exactly as before, so no composed name's initials move. DECLINED: composing the initial to NFC (the issue's option 2, and what outside PR #586 did, on the core surface only). It fixes the dropped mark but hands back a composed initial for a decomposed name, where #542 settled that the render views keep the form they were given (decisions.md#R4, 2026-10-02). VERIFICATION, which the gate DOES see, unlike #542's: `_initials` is a compared pseudo-field (#484), and the two NFD example lines this change adds to rules.md#R3 enter the rules corpus and diff at every baseline, classified by a `fix(#585)` `_initials` rule in each of the five ledgers; the CJK rule's corpus claim grows 138 → 139 at 1.4.0 and 2.0.0 with the katakana row, which lies in its script span. `tests/v2/test_properties.py::test_a_decomposed_name_renders_as_its_composed_twin` joins #542's walk rather than opening its own and records 44 disagreements — 22 texts, each on both surfaces — over the 139 texts compared with `text[0]` restored (this change's corpus, its two R3 rows included; measured 2026-10-02), with a live control beside it. - -### R4 — case repair reads the unjoined mark - -- 2026-08-29 #407 — DECIDED: case repair consults the mark #404 introduced, and consults it for the WHOLE PART rather than for a particle standing alone. Until now three views read it and the fourth did not, which is how `parse("ANH DO")` came to report family_base 'DO' and initials 'A. D.' while `capitalized()` returned 'Anh do' — the same word treated as an ordinary name word by three readers and as a particle by the fourth. The scope question is the only real choice here, and standing-alone loses on the first Vietnamese name that tests it: `anh van do` has family `van do`, two particle-vocabulary words and neither of them alone, so a rule keyed on aloneness would capitalize `Anh Do` and leave `Anh van do`, making the same surname read differently for having a given-name word in front of it. rules.md#R2 states the test as "Every word", not "standing alone", and this is the case that shows why it is stated that way. One predicate, four views. -- 2026-08-29 — this is a DELIBERATE DEVIATION from 1.4.0, not a parity restoration, said plainly because the surrounding branch work is mostly restorations and a reader will otherwise assume this is one. Measured on the released 1.4.0 wheel: `HumanName('ANH DO').capitalize()` leaves `'Anh do'` and `HumanName('anh van do').capitalize()` leaves `'Anh van do'`; both are `'Anh Do'` and `'Anh Van Do'` on this branch. There was nothing to restore — v1 lowercased on vocabulary membership alone and had no notion of a particle with nothing to join, the mark being a 2.x invention. -- 2026-08-29 — the ACCEPTED COST, agreed before the change rather than discovered after it: a degenerate family that is nothing but particles capitalizes too, so `juan van der` renders `Juan Van Der` where 1.4.0 gave `Juan van der`. This is the same trade rules.md#R2 already took for `family_base` — those words ARE the base, so they are name words, and a parser that called them name words for sorting and particles for display would be the incoherent option. Blast radius, measured by running both predicates in one process, and stated over BOTH populations because the neighbouring bullets in this entry measure over 1094 and a bare "4 and 23" would read as measured there too. Over the 1090 names the corpora carried before this change: 4 names move under a plain `capitalized()` — `ANH DO`, `vai la`, `van ma van`, and `der, y van`, which becomes `y Van Der` and keeps its lowercase conjunction — and 23 under `force=True`, the extra 19 being mixed-case inputs the gate holds back (rules.md#R5). Over today's 1094 it is 5 and 24: this change's own R4 example rows added `anh van do`, and that name is itself a mover. -- 2026-08-29 — THE VERIFICATION LIMIT, which matters more than the change. `tools/differential/compare.py` compares the seven role fields plus `_ambiguities` and nothing else, so `capitalized()` is not a compared field and NO ledger run at any baseline can see this deviation: the corpus counts and the three intentional-diff totals hold across it precisely because the harness is blind to it, and a green run is evidence about roles, not about case. mechanisms.md#FACADE-CONTRACT promises that v1-visible changes are "release-log-classified fixes, each verified in the differential ledger". The first half is kept — docs/release_log.rst carries the 2.2.0 bullet — and the SECOND HALF CANNOT BE, for this change or for any other that moves only case. Standing in its place: rules.md#R4's own example rows (`ANH DO`, `anh van do`, and the `juan de la vega` boundary), rules.md#R5's, and the v1-facade tests in tests/test_capitalization.py. The ones that pin the fix are mutation-checked against deletion of the tag consult; the `juan de la vega` control passes that mutation BY DESIGN, which is what makes it a control rather than a witness. Read the silence as a SCOPE and not as a gap: the harness compares parses, case repair is a render view, and nothing is owed on the tracker for it — the next bullet is why that is a decision rather than a deferral. Amended 2026-09-02 (#484): the premise moved and the conclusion did not. `compare.py` compares the seven role fields plus `_ambiguities` AND `_initials`, so "nothing else" is now one pseudo-field short and a render view is no longer invisible by construction — but `initials()` is the only one added, deliberately (`decisions.md#differential-ledger, the initials view`), so `capitalized()` is still not a compared field, and the next bullet's "the compared surfaces stay the seven roles plus `_ambiguities`" should be read as naming the surfaces case repair is absent from. Do NOT read #484 as the precedent that answers the next bullet: it MEETS that bullet's two objections rather than waiving them. `_initials` is a diff-of-diffs gated on the fields agreeing — precisely the instrument the next bullet declines for case — and the column did arrive pre-failed, by 140 entries at 1.4.0 as measured on this branch's base — 126 of them classified by six new 1.4.0 rules on the shipped tree, and the other 14 by `fix(#462)` at the three 2.x baselines once the facade fix moved them there. What earned it its place is a demonstrated ESCAPE: #408 moved initials for a whole minor with the gate green, and the 98 per-word names had been moving since 2.0.0, unseen and unclassified. So the next bullet's case for no `_capitalized` column stands until a case change escapes the same way, and that is the test a future proposal has to meet. -- 2026-08-29 — WHY CASE IS NOT A COMPARED SURFACE, settled here rather than left open, because "add a column" is the obvious answer to the bullet above and it is the wrong instrument. What the harness is FOR is parsing behavior — tools/differential/README.md says to run it "when touching parsing behavior, and before cutting a release" — and what earns it its two-process cost is that a role is EMERGENT: the eight stages fold over one name and each can move what the next sees, so the reach of a vocabulary or rule edit is not derivable by reading the diff and has to be measured over a corpus instead. Case repair is the opposite kind of thing. `_render._cap_word` is a pure function of `(word, role, tags, lexicon)` with four branches — the particle/conjunction lowercasing, the exceptions map, the Mac/Mc convention, and plain `capitalize` — sitting behind R5's single gate, so an edit to it is legible: you can see what you touched. The measurement agrees with the reasoning. Over the 1094 corpus names the 1.4.0 and 2.1.0 wheels ALREADY disagree on `capitalize(force=True)` for 139 of them (recompute by running `HumanName(n).capitalize(force=True)` under each wheel over the corpus files deduped, and diffing the two maps; the count was the same over the 1090 names the corpora carried before this change). Nearly every one is an echo of a parse diff the field comparison already classifies — a token that changed role capitalizes differently — so a column would re-report the ledger's own explained diffs in a vocabulary its rules cannot match, and would arrive pre-failed by 139 entries. Isolating the residue means comparing case only where the fields agree, a diff-of-diffs, to reach a class that a unit test reaches directly. So the compared surfaces stay the seven roles plus `_ambiguities`, by design and not by omission, and case is pinned where case belongs: the `rules.md#R4` and `#R5` example rows, and tests/test_capitalization.py. -- 2026-08-29 — DEBT DISCHARGED, the obligation the last bullet of decisions.md#R5 left this commit; recorded here so the pointer resolves rather than dangling. The FALSEHOOD: R4 promised repair "vocabulary exceptions (McDonald) included" flatly, which `str(parse('Juan Mcdonald').capitalized())` — `'Juan Mcdonald'` — falsifies, the gate refusing before any vocabulary is consulted. The statement now scopes that promise to where repair acts at all and hands the where to R5. A SECOND error in the same eight words turned up while rewriting them, and is worth recording because it had survived unread for as long as the first: McDonald is not a vocabulary exception. The shipped exceptions are five entries — II, III, IV, M.D., Ph.D. (recompute with `Lexicon.default().capitalization_exceptions_map`) — and `Mc`/`Mac` is a separate convention applied to any name shaped like one, so the old parenthetical illustrated the promise with the one mechanism it was not about. The statement now names an exception the vocabulary actually records (Ph.D.) and the Mac/Mc convention separately. The AMBIGUITY: "an already-correct name comes back unchanged" is true read as idempotence and reads as false under bearer-correctness, and the repair was to say WHICH is meant — a name already written the way repair would write it comes back unchanged — plus one clause refusing the inference a reader might otherwise draw, that a spelling its bearer chose is not spared for having been chosen. That clause is deliberately a REFUSAL and not a carve-out: sparing deliberately single-cased names is behavior the parser does not have and that R5's rationale declines on the record, so `str(parse('bell hooks').capitalized())` is still `'Bell Hooks'`. R4's boundary moved too: the old `"Juan McDonald" → capitalized="Juan McDonald"` row was inert — it passes with R5's gate deleted — so it became a `capitalized_forced=` row, which asserts what repair APPLIES rather than what the gate keeps and fails when the Mac/Mc convention is removed (measured by neutering it in one process: `'Juan Mcdonald'`). The boundary is now `juan de la vega`, a working particle run R4's new clause deliberately does not reach. - -- 2026-08-29 — the CONJUNCTION CARVE-OUT, stated in rules.md#R4 rather than left to a code comment, and the reason the comment gave was false. `_cap_word`'s conjunction conjunct is deliberately NOT gated on the unjoined mark, and the behavior is right: R3 already carries the carve-out for initials ("A CONJUNCTION never initials, so a base that is one contributes nothing even then"), and R4 was the half of the pair that carried it nowhere. What the comment claimed instead was an impossibility — `particles` and `conjunctions` are disjoint, so no conjunction token can ever carry the mark — and disjointness is a property of the SHIPPED vocabulary while both sets are public, configurable API. Measured: under `Lexicon.default().add(particles={'y'})`, `anh y van` parses to an all-particle family whose `y` carries `conjunction`, `particle` and the mark at once, and repairs to `Anh y Van`; gating the conjunct too, in one process, gives `Anh Y Van`. So the clause decides something and is not a no-op. It gets NO example row, and that is a limit of the document rather than a choice: the runner parses with the default vocabulary, over which `particles ∩ conjunctions` is empty in the default lexicon and in all four locale packs, so no input string can witness it. `der, y van` is the nearest thing and does not witness it either — the `y` there is a GIVEN word standing beside the all-particle family, not inside it (tests/test_capitalization.py says so at the row). - -- 2026-08-29 — the REPLACE/REVISE BOUNDARY, raised by the review of this branch as a gap needing a systematic fix and corrected by Derek to what it actually is: the documented boundary, with a supported path across it. `_cap_word` keys the particle TEST on lexicon membership but the REPAIR on the unjoined mark, and the mark is a parse product — so `parse('de la').capitalized(force=True)` is `'De La'` while `base.replace(family='de la').capitalized(force=True)` is `'de la'`, `replace()` splicing raw text into a field without classifying it. `Parser.revise()` is the method that classifies it, and its docstring already promised exactly this: each value gets a full sub-parse "so the stable tags survive and the tag-driven views ... behave as if the text had been parsed". Measured: `Parser().revise(base, family='de la').capitalized(force=True)` is `'De La'`. So there is nothing to fix — rules.md#R4 carries it as an Accepted boundary naming `revise()`, not as a deviation, and `revise()`'s enumeration of the tag-driven views gains `capitalized()`, which #407 made the fourth. (#458 would additionally make `replace()` agree, by keying the test on the `particle` tag rather than re-deriving membership from the word — the same shape it already names for `conjunction`/`initial`. That is a convenience, not the resolution.) It is a limit and not a regression either way: 2.1.0 gives `'de la'` on the spliced path too, so only the parsed path moved. - -- 2026-08-29 #458 — DONE: case repair's conjunction test reads the `conjunction` TAG instead of re-deriving the word's class at render time, so a view honors the decision the parse recorded (mechanisms.md#VOCAB-TAGS) rather than taking it again. The two copies of the question were not the same question, which is the reason this was worth doing over a corpus that cannot see it: `_classify.py` asks `is_initial(token.text)` — the shape test ANDed with a repertoire test since #320 — while `_render.py` asked a bare pattern, shape only, over each WORD of a token's text rather than the token. It does NOT retire the hand-sync obligation the module carried in a comment ("keep in sync with `nameparser/_pipeline/_vocab.py` by hand"), and an earlier draft of this bullet said it did: the tag read has nothing to keep in sync, but `_render._INITIAL` was deleted and then restored in the same PR, the fallback in the bullet below giving it a reader again. The obligation is narrower and sharper than before — the two copies no longer answer the same question about the same token, so what has to hold is that the fallback answers as `_vocab` would — and test_regex_sync is what holds it. MEASURED, re-derived on this branch rather than taken from the issue: 0 of the 1094 corpus names move under `capitalized()` or `capitalized(force=True)`, and 0 move with each name also re-run uppercased and lowercased (6564 name/variant/lexicon rows = 1094 x 3 spellings x 2 lexicons, the second lexicon being `Lexicon.default().add(particles={'y'})`, and 13128 `capitalized` calls over them); the R5 force-versus-case counts recorded above hold — 62 upper / 16 lower through the facade, 63 upper / 18 lower through the core. Movement EXISTS outside that population, and the issue's "no output moves" is a claim about the corpus only: the old predicate re-decided per word, so `juan e-f smith` capitalized to `Juan e-F Smith` — the Italian conjunction inside a hyphenated middle name — and now gives `Juan E-F Smith`, agreeing with the uppercase spelling it always gave. An earlier draft of this change moved untagged tokens the same way — a family `replace()` splices in as `de y` repairing to `de Y` — and that was a REGRESSION, caught in review before it shipped and fixed in the bullet below; read this bullet with that one. NOT DONE, and the parenthetical in the boundary bullet above should be read as still open: the PARTICLE conjunct still keys on lexicon membership, so `replace()` does not agree there. Making it read the `particle` tag is a second change with its own blast radius — it moves a boundary rules.md#R4 states in prose — and #458 as filed does not ask for it. - -- 2026-08-29 (#458 review) — THE FALLBACK FOR TEXT THE PARSE NEVER SAW, and why it is scoped to one of the two questions `_cap_word` asks. Reading the tag and nothing else regressed every ASSIGNED field holding a conjunction, because a value `replace()` splices in is never classified and so carries no tag to read. Measured on the released 1.4.0 and 2.1.0 wheels, all agreeing and all broken by the tag-only reading: `h.last = 'velasquez y garcia'` then `capitalize(force=True)` gives `John Velasquez y Garcia`, `h.middle = 'e.'` gives `John E. Smith`, `h.last = 'smith-y'` gives `John Smith-y`, `h.first = 'y'` gives `y Smith`. That is the canonical Spanish surname and the shape `y` is in the vocabulary FOR, so an Accepted clause was not available: it had to be fixed. RULE: a token carrying `UNCLASSIFIED_TAG` was never read, so there is no decision to honor and the view falls back to the vocabulary. Note what the discriminator is NOT, in two directions, the second of which took a second review round to find. It is not UNTAGGEDNESS: an ordinary parsed name word carries no tags either (`velasquez` parses to `tags=[]`), so that would have swept in the whole parsed corpus. And it is not the absent SPAN, which is what the first fix used. Span-less means SYNTHETIC, a wider set than unclassified: `Parser.revise()` builds span-less tokens too, from a full sub-parse whose tags it keeps deliberately, and its docstring promises the tag-driven views "behave as if the text had been parsed" — so keying the fallback on the span overrode exactly the tags `revise()` exists to preserve. Measured on the branch that did it: `revise(name, middle='e-f').capitalized(force=True)` gave `e-F` where `parse('john e-f smith')` gave `E-F`, and under `Parser(lexicon=Lexicon())` a revised family `de la vega` initialed `j. v.` where the parse initialed `j. d. l. v.`. So the mark is stamped by the two producers of genuinely unclassified text — `ParsedName.replace()`, and the facade's v1 pickle load, which rebuilds a name from `*_list` strings and no tags — and a HAND-BUILT span-less token is not marked, so it takes the tag path like everything else in the library. That is the tag-driven semantics every other view already has, and it is a change from the span reading in exactly one place: a hand-built span-less token whose text is CONJUNCTION vocabulary now capitalizes rather than lowercasing (measured: a hand-built family `velasquez y garcia` repairs to `Velasquez Y Garcia` untagged, `Velasquez y Garcia` with either the `conjunction` tag or the mark). The PARTICLE conjunct is unaffected either way — it keys on lexicon membership rather than on the mark, which is the open item this bullet's neighbours record — so a hand-built `de la Vega` is unchanged. SCOPED TO THE PER-WORD QUESTION, and this is the durable half. *Is this word a conjunction rather than an initial* is a property of the word and its spelling: an unread token has no ambiguity the parse would have resolved differently, so the vocabulary gives the answer the parser would have given, v1's initial carve-out included — which is why `_render._INITIAL` came back and why test_regex_sync's two-copy assertion matters MORE now, the fallback being right only while it answers as `_vocab` would. *Is this particle acting as a particle here* is a property of the whole PART, and answering it needs a reading on every word of the part, which is exactly what a spliced field lacks. `_cap_word` is handed the whole token's tags — it is not blind to the part because it walks one word, and an earlier draft of this bullet said it was — but with no tag on any word of the part there is nothing to re-derive the part answer FROM, so the fallback here is the word question alone and the particle half falls through to plain particle treatment. Measured cost of the wide reading IN CASE REPAIR, so nobody re-derives it: `replace(family='de la')` gives `de la` today and a parse gives `De La`; widening case repair's fallback would make them agree and would thereby REVERSE the replace/revise boundary recorded two bullets above, with `revise()` as its documented crossing. Do not widen it without reopening the decision. Verified after the fix: `replace(family='velasquez y garcia').capitalized(force=True)` is `Velasquez y Garcia`, `replace(family='de la').capitalized(force=True)` is `de la`, and the parsed and `revise()` paths are unchanged. - -- 2026-08-29 (#458 review, then the review of PR #463) — the SAME FALLBACK in `initials()`: WRITTEN, MEASURED AND DROPPED before merge. The defect it aimed at is real and older than #458 — `replace(family='velasquez y garcia').initials()` gives `j. v. y. g.` where the parse, the facade and 1.4.0 all give `j. v. g.` (the facade and 1.4.0 measured through the TWIN of that splice — `h = HumanName('juan smith')` then `h.last = 'velasquez y garcia'`, the assigned construction this bullet is about, keeping `first`), and `replace(family='de la vega')` gives `j. d. l. v.` for `j. v.` — and it is unfixed again. What killed the fix is the parameter list: `initials()` takes no lexicon, so the fallback had to reach for `Lexicon.default()` and GUESS. `capitalized(lexicon=...)` is handed the caller's vocabulary and so guesses nothing, which is why its fallback stays. Measured cost of the guess, and the reason a merely-imperfect fallback was not good enough: under `Parser(lexicon=Lexicon.default().add(particles={'y'}))`, `parse('Juan de y').initials()` is `J. d. y.` and `parse('Juan Perez').replace(family='de y').initials()` was `J. d. y.` before the fallback and `J.` after it — an entire field's initials gone, because the default vocabulary reads `de y` as particles-plus-conjunction where the caller's reads it as an all-particle part. Guessing the wrong vocabulary for a name parsed under a custom one is worse than answering from tags alone, and `initials()` cannot be told better. The missing crossing is the whole of it: `Parser` carries `capitalized`, `matches` and `revise` as its custom-lexicon crossings and no `initials`, so there is no supported way to hand this view the parse's vocabulary. #464 is filed for that; when it lands, the fallback becomes answerable rather than a guess and this decision is worth reopening. ACCEPTED until then, and stated so a reader does not re-derive it as a bug: `replace(family='de la vega').initials()` is `j. d. l. v.` where the facade and a parse both give `j. v.`. The reach is the v2 core only — `HumanName.initials_list()` computes from the field STRINGS with its own lexicon (`_facade._initials_lists`) and gives `['j', 'v', 'g']` at 1.4.0, 2.1.0 and here, under that same assignment. That is why the facade cannot be used as evidence that the core is fine. Do NOT read `_initials_lists` as a v1 parity REFERENCE, though, which is how an earlier wording of this bullet labelled it: #462 is open on a parity break in that exact path, and it is a break inside the facade rather than upstream of it. `HumanName('Scott E. Werner')` partitions to `first='Scott'`, `middle='E.'`, `last='Werner'` at 1.4.0, at 2.1.0 and here — byte-identical fields — and `initials()` on them is `S. E. W.` at 1.4.0 and `S. W.` at 2.1.0 and here, the facade reading the middle `E` as the conjunction; assigning those three fields by hand reproduces each version's answer, so no parse difference is involved. The parity label was therefore doing work its own measurement cannot support, and only the per-name measurements are load-bearing here. MEASURE THIS BULLET WITH THE ASSIGNED CONSTRUCTION, and name the construction beside each value: the review of PR #463 measured `HumanName('juan velasquez y garcia')` — the PARSED form — against sentences about the assigned one, and 'corrected' two true claims into false ones. The two constructions agree from 2.0 on and DISAGREE at 1.4.0, where the parsed form gives `['j', 'v g']` and `j. v g.`. The `last` string is byte-identical either way; `initials_list()` reads `last_list`, which the 1.4.0 parse leaves as `['velasquez y garcia']` — the conjunction run joined into one element — where assignment re-splits it to `['velasquez', 'y', 'garcia']`. So `computes from the field STRINGS` is exact for 2.x and true of the ASSIGNED form only at 1.4.0. Two things learned in the attempt are worth keeping though the code is gone. `_types._remarked` is not the bug and needs no change: it recomputes the mark from TAGS after every edit, which is right for a record of what the parse decided and simply silent about text nobody classified. And any future fallback must be scoped per ROLE, not per name — a role the parse classified whole is decided by its tags however the other roles were built — a distinction that passes every other test in the suite and needs a hand-built name to witness. - -- 2026-08-29 — WHY THE BOUNDARY WENT UNNOTICED UNTIL #407, which is where a future reader should look for it. For an ALL-PARTICLE part the other three tag-driven views give the same answer through `replace()` and `revise()` alike: measured over `de la`, `van der`, `do`, `de` and `van de la`, all five agree on `family_particles=''`, on a `family_base` holding the whole part, and on initials from every word. They converge because an UNTAGGED part and a MARKED all-particle part reach the same place by different routes — untagged, no word is recognized as a particle; marked, none is ACTING as one — and all three views only ask which words are particles. Case repair is the one view that asks a second question, since it must also decide whether to lowercase, so it is where the two routes first come apart. The mirror case confirms the reading: on a MIXED part the convergence is the other way round — `de la vega` and `van der berg` diverge in all three views between `replace()` and `revise()` (`replace()` reports particles `''` and base `'de la vega'` where `revise()` reports `'de la'` and `'vega'`) and AGREE on case repair, R4's all-particle clause not reaching them. So before #407 the distinction was invisible on exactly the shape the clause is about, and visible only on shapes the clause does not govern. - -- 2026-09-13 #459 (landed 2026-09-22, PR #521) — DECIDED: a credential acronym the exceptions map does not carry is an initialism, so a single-case word the parse put in the SUFFIX role from `suffix_acronyms` repairs to its all-caps spelling rather than a title-cased one (the clause in `_render._cap_word`, after the exceptions-map lookup and before the Mac/Mc rule — the order is load-bearing, since `mcse` is in the acronym set and matches the Mac/Mc shape, and reads `MCSE` only because the acronym clause is asked first; rules.md#R4 states the precedence and pins it with a row). The scope is narrow on purpose. The exceptions map is consulted first, so its five entries — the spellings `str.capitalize()` gets wrong, `md` → M.D. and `phd` → Ph.D. because they are punctuated, and `ii`/`iii`/`iv` → II/III/IV because a numeral is written all-caps and they are `suffix_words` the acronym clause would never see — keep theirs and the clause never touches them. The repair is gated on the SUFFIX role, so a word that is in the acronym vocabulary but parsed as an ordinary name word (`anh van do` → `Anh Van Do`) still repairs as that name word -- the gate is the whole reason the fix is safe on surnames that share a spelling with a credential. Because it is the ROLE and not a parse tag, the gate also reaches a suffix spliced in as raw text: `parse("john smith").replace(suffix="mba").capitalized()` gives `MBA` like the parsed name does, which is the contrast R4's Accepted paragraph draws against the all-particle clause. The same PR removed `ph` from `suffix_acronyms`; see `Excluded (SUFFIX_ACRONYMS — ph ...)`, beside the esq block, for why. The wider design the issue proposed (letter masks, `md` leaving the map, the given-role half of `QC MP`) stays on the rescoped #459; this clause is the narrow part #459 already accepts, not a re-litigation of those. -Reach, population first, measured 2026-09-22 on the PR branch merged with master at 23e52dc6, over the deduped corpus glob (1338 distinct names): 178 names carry acronym vocabulary in a suffix token and 30 of those are written in a single case, and 22 move on the default `capitalized()` path, 119 under `force=True`. Recompute by exec'ing the pre-change `_cap_word` -- `git show 23e52dc6:nameparser/_render.py`, the master this branch merged, which carries #397/#461's generation gate but not this clause -- into `nameparser._render`'s namespace and swapping it in for the changed one in a single process, then diffing `str(capitalized(parse(n), None, force=...))` over the glob. The digits have moved twice and neither move is the clause's: the PR's first head (2026-09-11, `ph` still in the set, 1143 names) measured 5 default and 71 forced; the commit that dropped `ph` took the 13 `PH. D.` forced movers back to `Ph. D.` and left 111 in the population, 5 default and 58 forced; and the merge brought in the corpus rows of PR #530 (#289/#516), PR #532 (#531) and PR #534 (#533), whose single-case trailing `MA` credentials (`DOE, JOHN MA`, `jane doe nee smith ma`, `JOHN SMITH, MA`, the CJK `田中 太郎, MA` family; their mixed-case twins hold still under R5) account for 15 of the 17 further default movers; the other two are `STEVEN HARDMAN, MD, DO, DDS` and `john smith mcse`, the row this PR's own rules.md amendment adds (`McSe` under the pre-change function, which is the precedence claim above measured). The ratio is the stable claim: every single-case name with an acronym suffix the map does not carry moves, and nothing else does. -Accepted costs, deferred to the rescoped #459 rather than relitigated here: the all-caps default reaches words conventionally written mixed-case -- `bsc`/`msc` read `BSC`/`MSC` under `force=True`, and `Dr. med. univ. Margit Popp, MSc` is a corpus name that reads `MSC` under `force=True` only, its mixed case holding it back on the default path under R5 -- which the letter-mask design #459 defers is meant to recover; `ii`/`iii`/`iv` are `suffix_words` rather than acronyms and need the exceptions map precisely because the clause would not see them there; and because the ambiguous five (`ba`, `do`, `ed`, `jd`, `ma`) are in `suffix_acronyms`, the clause moves WHICH parse triggers the repair rather than preventing it -- `john smith ed` → `John Smith ED`, `john smith ba` → `BA`, and `smith, ms.` → `MS.` on the default path. That is the same #342/#454-class cost #459 already accepts, and the alternative (reading classify's `vocab:suffix` tag) was measured and costs `jd` → `Jd`, because `jd` carries `vocab:suffix-ambiguous` and not `vocab:suffix`, so the role gate is the right instrument. - -- 2026-09-20 #461/#397 — THE GROUNDING MOVED, AND THEN A DEFECT THE GROUNDING EXPOSED. R4's sentence rested on R3 by name ("the carve-out R3 states for initials"), and R3's carve-out is conditional now while R4's is not, so the cross-reference is CUT and replaced by this rule's own reason: a connective that initials because it joins nothing is still not written the way a name is written. Measured 2026-09-20, plain and forced, over `juan y`, `john and jane smith`, `duke of edinburgh`, `juan de y`, `juan y garcia`, `JUAN Y GARCIA`, `josep carod i rovira`, `JOSEP CAROD I ROVIRA` and — under `add(particles={"y"})` — `Anh y Van`: all nine are byte-identical to the same call at the parent, re-measured after the repair below and still byte-identical, and `tests/test_capitalization.py` is green. - WHAT THE PLAIN CALL DOES, stated over a population rather than over nine names, because "capitalized() does not move" was drafted for this bullet and is false as written. Swept 2026-09-20 over the 1558 non-empty corpus-union-cases names under eight configurations against the parent commit, `capitalized()` moves on SIX distinct names and every one of them is a name whose ROLES moved under the join — it renders different fields rather than treating a word differently. The correct statement is therefore: where only `initials()` moves, `capitalized()` is byte-identical; where the roles move, it follows them. - WHAT THE FORCED CALL DID, AND THE DEFECT THAT WAS FOUND WRITING THIS RECORD. The sweep above was run for the PLAIN claim and turned up a second answer: `capitalized(force=True)` moved on ten further names whose roles, initials and plain repair all stayed put, every one a lower-case `i` — `parse("Carod i")` forced gave "Carod i" where 1.4.0 and 2.3.0 gave "Carod I", and so did "John Quincy Smith i", "Josep Carod i", "Lluis Carod i", "Josep Lluis Carod i III", "Josep Lluis Carod i V", "Carod y de Rovira i", "Rovira, Josep Carod i Jr.", "Carod i Rovira" and "Josep i Rovira". Nine of those were a DEFECT and are REPAIRED here; the remaining two are the rule. - WHY IT WAS A DEFECT, and the line R4 now draws. In those names the parse read the letter as the GENERATION it also spells and roled the token SUFFIX — but classify had tagged it `conjunction`, and repair's conjunction conjunct read the tag without asking the role, so a generation was lower-cased as a connective. Derek's principle decides it: a connective with nothing to join is not acting as a connective. Repair now asks the ROLE the parse decided rather than the tag alone, and a suffix-roled token is repaired as the suffix it was read as. `parse("John Quincy Smith i")` forced gives "John Quincy Smith I" again on BOTH surfaces, the facade repairing through the same helper. - WHAT REMAINS AND IS NOT A DEFECT: `Carod i Rovira` and `Josep i Rovira`, where the letter is a lone MIDDLE word the parse tagged `conjunction` — three-word names, so P3's carve-out leaves the letter a name word and nothing joined. Forced repair keeps it lower-case, which is R4's own rule and exactly the answer the same shape has always had for `y`: `Carod y Rovira` and `Josep y Rovira` give "Carod y Rovira" and "Josep y Rovira" too. Those two names are the whole of what #397 moves on this view. - AND THEN THE REPAIR ITSELF WAS TOO WIDE, found by the second review round (2026-09-20). The guard shipped as the suffix ROLE alone, and the role says only where a word LANDED. A plain connective can land in the suffix field without being generational vocabulary at all — the third part of a comma form is read as the suffix run whatever its words are — so `parse("Smith, John, and").capitalized(force=True)` gave 'John Smith And' where 1.4.0, 2.0.0 through 2.3.0 and the parent 46651750 all give 'John Smith and', and the same for `, y`, `, e`, `, und`, `, of` and `Doe, Jane, and Jr.`. A field SPLICED in went the same way and against R4's own Accepted paragraph, which says a field nobody read asks the vocabulary: `HumanName(first="John", last="Smith", suffix="de y")` repaired forced to 'John Smith De, Y' against 'John Smith De, y' on both wheels. THE GUARD IS NOW THE PAIR — the suffix role AND the `vocab:suffix` tag — because the clause turns on the GENERATION and the role alone cannot see it; classify's tag is the vocabulary half of the decision and reading both re-derives neither. Invisible to everything already watching: no name of the off-switch grid carries the shape (its oracle is a letter of the class, and none of these words is one) and no corpus name carries it either, which is why INV7b was written over a grid of its own. ACCEPTED, and the one reading this moves rather than restores: a field spliced to `suffix="i"` now repairs to 'John Smith i' where the parent gave 'John Smith I'. `i` is connective vocabulary on this branch and was not at the parent, so the spliced field follows `y` now — which is R4's Accepted paragraph working as written, the PARSED name being unaffected because it has a reading and the reading is the generation. - MEASURED, and the numbers are the fix's own control. Over the same eight configurations the forced-only set is ten names before the fix and two after; the invariant `tests/v2/test_properties.py::test_a_letter_that_did_not_join_repairs_as_the_off_switch_does` — a letter that became no connective of this name repairs as the off-switch parse repairs it, plain and forced, on both surfaces — fails on 7,860 repairs at `e540d4c5` (6,843 core, 1,017 v1) and on 0 here, and removing the generation conjunct outright fails it on 8,054. (11,341 / 10,424 / 917 was the same control over the full cross product before the grid was trimmed by shape and then grown by the second review; it is a dated snapshot and the test's docstring is the live figure.) The differential still cannot see any of it, case being no compared surface (the 2026-08-29 bullets above settle why), which is why this needed an invariant rather than a ledger rule. - -- 2026-09-23 #459 — DECIDED (Derek, 2026-09-23): the exceptions map holds case MASKS, and nothing else. A value is its key's own letters and digits, each in the case it takes, and case repair lays it over the word as the writer punctuated it, keeping every other character where the writer put it — so one `phd` entry repairs `phd` to `PhD`, `ph.d.` to `Ph.D.` and `PH.D.` to `Ph.D.`, and the map can hold a conventionally mixed-case acronym (`BSc`, `MSc`) without a dotted spelling per source. A replacement string is substitution, not capitalization: it is how `md` became `M.D.`, how `iii.` lost its period, and why the map never grew past the five entries it was imported with in 2011. The shipped map is `phd` → `PhD`, `bsc` → `BSc`, `msc` → `MSc`; `md`, `ii`, `iii` and `iv` left it, a suffix `md` now repairing by the acronym clause and a suffix numeral by the numeral clause (both below). Lookup is unchanged — the edge-stripped key, then the period-free one — and the map stays ROLE-FREE: an entry is the caller saying how a word is written wherever it stands (`phd smith` gives `PhD Smith`). rules.md#R4 now states the invariant every clause serves — repair changes case and nothing else — and `tests/v2/test_properties.py::test_case_repair_changes_case_and_nothing_else` holds it over every corpus name and case-table text, plain and forced, on both surfaces, under the shipped lexicon and one with the map emptied, comparing under `str.casefold()` (the Unicode-boundary bullet below says why that comparator). `md` → `MD` and `phd` → `PhD` are DELIBERATE DEVIATIONS from every release, 1.4.0 through 2.3.0, all of which gave the dotted forms (measured 2026-09-23 on all five wheels). There is no contributor intent behind the period insertion to preserve: the map, its `md` entry and `capitalize()` itself arrived together in the 2011 svn import (45a15394), authored by Derek. - -- 2026-09-23 #459 — THE CHECK, and two places it departs from the approved spec, both by measurement. A value that does not spell its key's letters and digits is a `ValueError` at `Lexicon` construction (`_lexicon._normpairs`). Both sides are folded through `_normalize` and compared as ALPHANUMERICS (`_lexicon._alnum`), so `('2nd', '2ND')` is accepted and `('2nd', '3ND')` raises — a comparison of letters alone would have let a digit key take any other digit. The message names the entry and offers the key at hand recased — `capitalization_exceptions=(('jr', 'JR'),)` for a `jr` key — a tuple of pairs, because the field is annotated with what it stores and a dict literal there would be an arg-type error in the pasted code; where `.upper()` changes the alphanumeric count (`straße` → `STRASSE`) it offers the key unchanged instead, an identity mask, so the offered code always constructs. Every entry point reaches it: construction, `dataclasses.replace()`, `|`, unpickling (`__setstate__` re-runs `__post_init__`), and the v1 shim's snapshot; `tests/v2/test_lexicon.py` sweeps all five. FIRST departure: the spec compared the value against the KEY as "already normalized, period-free and lowercase" — it is not period-free, `_normalize` stripping only EDGE full stops, so `Ph.D.` is stored `ph.d` and, compared raw, a key written with an interior period could never validate. SECOND: the spec's "the two letter sequences are always equal in length" is false. The key is NFC-composed and the word is not, so a word written in decomposed hangul (`씨` as two jamo) finds a one-letter key. The applier, `_render._apply_mask`, declines there (it returns None) and `_cap_word` falls through to its next clause — a caseless script has no case to recase, so nothing is lost — pinned by `tests/v2/test_render.py::test_the_mask_keeps_every_non_letter_and_declines_a_miscount`. Without the decline the naive applier raised `IndexError` inside `capitalized()` (measured in review on 2026-09-23, on the prototype and again by the spec review). - -- 2026-09-23 #459 — A DECIDED EXCEPTION to the shim's never-raise rule (AGENTS.md: "The shim TRANSLATES; it never raises on a config v1 accepted"). A v1 `Constants` whose `capitalization_exceptions` carries a value that does not spell its key — `{'jr': 'Junior'}` — raises the `ValueError` above at `Constants._snapshot()`, i.e. at the first parse. Taken rather than translated because there is nothing to translate TO: 1.4.0 substituted such a value for the word, 2.4's repair only recases, and so the only v1-faithful reading is the one this change removes. No such value was found in the repository, the docs, the issue tracker or any test when the design was approved; the hypothesis fuzzer drew arbitrary pairs and now draws recasings of its key. A value that merely carries PUNCTUATION is not one of these: it is a legal mask (the joins bullet below), so a v1 `Constants` unpickled from 1.4.0, still carrying `md` → `M.D.` and `phd` → `Ph.D.`, loads and parses — measured 2026-09-23, `john smith md` giving `John Smith MD` under it. Pinned by `tests/test_capitalization.py::test_a_mismatched_exception_value_raises_at_the_first_parse`; recorded in AGENTS.md's shim bullet and cross-referenced from #3-0-reevaluations. - -- 2026-09-23 #459 (review) — A MASK'S PUNCTUATION MARKS WHERE ITS LETTERS ARE JOINED, and is never written into the word. Letters written side by side in a value are one RUN, and any character between them that is not a letter — a full stop, a space, a digit — ends the run. That matters for exactly one case, the LONE INITIAL: a single letter the writer split off beside a full stop (a `FULL_STOPS` member, not the ASCII period alone, #322) is written in capitals whatever the mask says there, but only where the mask writes that letter inside a run of two or more — the writer split a chunk the mask keeps together, so each split piece is an initial. So `p.h.d.` under `PhD` gives `P.H.D.` (the mask is ONE three-letter run), `b.sc.` under `BSc` gives `B.Sc.`, `ph.d.` gives `Ph.D.`, and `d.phil.` under a caller's `DPhil` gives `D.Phil.`; while `h.c.` under a caller's own `h.c` mask, which spells each letter alone, stays `h.c.`, `a.bc` under `abc` gives `A.bc`, `ab.c` gives `ab.C`, `2b.` under `2b` stays `2b.`, and `x2b.c` under `x2bc` gives `x2B.C`. Everywhere else a value's punctuation changes nothing: `('md', 'M.D.')` and `('md', 'MD')` repair every word alike (measured on `md`, `m.d.`, `md smith` and `m.d`). The rule took three drafts, and the two wrong ones are the record worth keeping. The FIRST laid the mask letter by letter with no initial rule and REGRESSED what substitution had got right: `b.s.c.` repaired to `B.S.c.` and `p.h.d.` to `P.h.D.`, where the parent 4d0680e6 gave `B.S.C.` and `Ph.D.`. The SECOND capitalized every lone letter beside a stop, which overrode a caller's own mask — measured in review on 2026-09-23, `_apply_mask('h.c.', 'h.c')` gave `'H.C.'` and `dr. h.c. hans meier` forced gave `Dr. H.C. Hans Meier`, where the caller had written the case they wanted. The third reads the mask's runs, as above, and is rules.md#R4's `p.h.d.` line. CASING goes through the whole word — `word.lower()`/`word.upper()` — wherever both keep the word's length, so context-sensitive letters come out as the language writes them (`ΚΟΣ` under `Κος` gives `Κος`, the final sigma intact), and per character otherwise (`straße` under `STRAßE` gives `STRAßE`). KNOWN LIMIT: a mask letter reads as upper when it is not lower, so a titlecase letter (`Dž`, Unicode category Lt) reads as upper and `dž` under `Dž` gives `DŽ`; no per-character titlecase mapping is attempted. Pinned in tests/v2/test_render.py by `test_a_split_initial_is_capitalized_only_where_the_mask_keeps_it_joined`, `test_a_masks_punctuation_marks_its_joins_and_is_never_written` and `test_a_mask_cases_through_the_whole_word_for_context_sensitive_letters`. - -- 2026-09-23 #459 (review) — DECLINED, after being written: a construction-time `UserWarning` for a punctuated value, and a shim translation of the two pre-2.4 shipped pairs. Both were implemented during review and then dropped, because both rested on a premise the code falsified — that a value's punctuation is IGNORED. The warning fired on a value whose punctuation differed from its key's (`'M.D.'` for `md`, `'h.c'` for `hc`) and told the caller to write the value without it (`Write the value as 'hc'`); but `_render._letter_run_ge2` READS a mask's punctuation, so following the advice changed the output — `('hc', 'h.c')` keeps `dr. h.c. hans meier` at `h.c.` under force, where `('hc', 'hc')` gives `H.C.`. The translation rewrote `md` → `M.D.` and `phd` → `Ph.D.`, byte-equal, to their unpunctuated masks in `Constants._snapshot()`; it changed no output (those pairs repair every word alike, above) and existed only so a v1 `Constants` still carrying them — one unpickled from 1.4.0, for instance — would not trip the warning, so with the warning gone it had nothing left to do. A punctuated value is simply a legal mask, accepted silently at every entry point (`tests/v2/test_lexicon.py::test_a_value_may_carry_punctuation_that_marks_its_joins`). The lesson generalizes: "this input is silently ignored" is a MECHANISM claim about what the code reads, and it earns the same one-command check as any other before a diagnostic is built on it. - -- 2026-09-23 #459 (review) — THE INVARIANT'S UNICODE BOUNDARY, and why rules.md#R4 words it as it does. The approved spec stated the invariant as "no character is added, removed or reordered"; rules.md#R4 says "the repaired word is the word as written, recased", because two inherited cases break the stronger reading and neither is this change's to fix. Python's casing tables uppercase `ß` to `SS`, so a clause that uppercases can lengthen a word: `john smith ß.x.` reads a by-shape credential and repairs to `John Smith SS.X.`, casefold-equal and one character longer (the parent 4d0680e6 gave `Ss.x.`, longer too). And the roman shape matches under `re.I`, which admits the dotless `ı`, so `john smith ıı` reads suffix and repairs to `John Smith II` — not even casefold-equal to its input, `'ıı'.casefold()` being `ıı` (the parent gave `Iı`). The first comes from the casing tables and the second from the parser's own pattern; neither is fixed. `tests/v2/test_properties.py::test_case_repair_changes_case_and_nothing_else` compares under `str.casefold()` because of the first, and that comparator has a boundary on each side: it CANNOT see a repair that trades `ss` for `ß` or back, the two folding alike — a recorded blind spot — and it WOULD report the second, which it does not today only because no corpus name or case-table text spells a numeral with `ı`. A length check beside it was considered and declined: it would flag the `ß` boundary this bullet records as accepted. - -- 2026-09-23 #459 item 4 — DECIDED (Derek, 2026-09-23): a word the parse put in a NAME role repairs as a name word, whatever vocabulary it is in. `qc mp` gives `Qc MP`, `ed smith` gives `Ed Smith`, `john smith ed` gives `John Smith ED`: the parse decides the role and reports the fork (`suffix-or-name`), and repair follows the role (mechanisms.md#RENDER-HONORS-THE-PARSE). Closes the item with no code. What the decision also reaches, found while planning and NOT in the approved spec, which said `iv smith` would keep `IV Smith` "through the numeral rule": the numeral clause is suffix-gated — gating it is what keeps the given name `Vi` from reading `VI` — so the four words that left the map lose, in any role but the suffix, the role-free capitals the map gave them. Measured 2026-09-23: `iv smith` gives `Iv Smith` and `john iii smith` gives `John Iii Smith`, where every release from 1.4.0 through 2.3.0 gave `IV Smith` and `John III Smith`; `II Van Johnson` gives `Ii van Johnson` under force; and `md` read as a TITLE — the parse's reading of a leading `MD` — repairs as ordinary title text, so `MD DDS` gives `Md Dds` and `Md Abdul Karim`, the usual abbreviation of Mohammed and a shape-tagged corpus row, gives `Md Abdul Karim` under force where every release gave `M.D. Abdul Karim`. DECIDED (Derek, 2026-09-23, when the plan surfaced it): a numeral in a name role repairs as a name word, the same rule read to its end. DECLINED, the alternative put to him with it: keeping `ii`, `iii` and `iv` in the map as masks, which would have restored the three numerals in a name role while leaving a title `md` at `Md`. - -- 2026-09-23 — PERIOD INSERTION IS OUT of `capitalized()` for good, and PARKED rather than declined: if anyone ever asks for a canonical dotted spelling (`md` → `M.D.`), it belongs in a normalization view — Derek's working name for it is `repair()` — which may re-spell a word, not in a case repair, which may not. Not a commitment; no issue filed. - -- 2026-09-23 #459 — NUMERALS: a SUFFIX-roled word whose text, edge periods stripped, fullmatches the roman shape repairs to capitals. It reaches `i` through `x`, which is why `ii`/`iii`/`iv` left the map: `vi` through `ix` read as suffix by that shape, carry no vocabulary tag, and title-cased to `Vi`, `Vii`, `Viii`, `Ix` (measured on 4d0680e6, and `Vi`, `Vii` on every release wheel 1.4.0 through 2.3.0); and `john smith iii.` now keeps its period, `III.`, where every release gave `III`. The shape is `_vocab._ROMAN`, v1's `roman_numeral`, and `_render` holds a HAND COPY pinned against config by `tests/v2/test_regex_sync.py`: the approved spec said the clause would import the predicate, but tests/v2/test_layering.py's ALLOWED table gives `_render.py` `_types` and `_lexicon` and nothing else, and `_render._INITIAL` is the precedent for exactly this. The copy re-derives no parse decision — the role is the parse's, read as recorded; the pattern answers only how a numeral is written. THE PARSE LIMIT beside it, recorded and not fixed here: the roman shape stops at `x`, so `john smith xi` reads family `xi` and repairs to `John Smith Xi`, and after a family comma `john smith, vi` reads given `vi` and repairs to `Vi John Smith` — both rules.md#R4 boundary lines, and both for #459's close-out. The #397 generation guard is unchanged and still decides first: `Carod i` forced gives `Carod I`, the numeral clause now being what writes the capital. - -- 2026-09-23 #459 — A SUFFIX READ BY ITS DOTTED SHAPE is an acronym to repair: the acronym clause fires on a SUFFIX-roled token whose period-free text is listed OR which carries `SHAPE_ACRONYM_TAG` (#516's by-shape reading, rules.md#S3), so `john smith x.y.z.` repairs to `John Smith X.Y.Z.` where the parent gave `X.y.z.`. The constant moved from `_pipeline/_state.py` to `_types.py`, which `_render` may import, and `_state` imports it back — still one spelling, which is what the constant exists for. The role gate holds: `Jack X.Y.Z.` keeps its surname and repairs as a name word, `Jack X.y.z.` under force. ACCEPTED: an unlisted dotted credential written in mixed case reads all-caps wherever repair acts, since a by-shape member has no listed casing to honor — `John Smith B.Tech.` gives `John Smith B.TECH.` under force, where every release gave `B.tech.`, no better. And since the suffix-free gate (decisions.md#R5, 2026-09-23) this reaches the DEFAULT path too wherever the name words are one-case: `john smith B.Tech.` gives `John Smith B.TECH.` where every release returned it untouched, while the corpus spelling `John Smith B.Tech.` stays untouched there, its name words being mixed. A caller who writes such a credential adds a mask: `('btech', 'BTech')` gives `B.Tech.` on either spelling (measured 2026-09-23). - -- 2026-09-23 #478 — DECIDED: inside `_cap_text`, a part of a HYPHENATED token that is connective vocabulary keeps its lowercase when a WORDED part stands on each side of it within the token (split on `-`; a part counts only if it holds a word, and an empty part does not count and is skipped, so a trailing hyphen supplies no neighbour while a doubled hyphen changes nothing: `juan garcia--y-lopez` gives `Juan Garcia--y-Lopez`). An edge part repairs as ordinary name text. `jose ortega-y-gasset` and `JOSE ORTEGA-Y-GASSET` both give `Jose Ortega-y-Gasset`, and `maria silva-e-sousa` gives `Maria Silva-e-Sousa`. It reads POSITION and VOCABULARY, as the particle arm already does per sub-word (`maria da-silva` → `Maria da-Silva`, unchanged) — a question about a sub-word the parse never asked, the parse having read the hyphenated word as one untagged token, which is the same standing mechanisms.md#RENDER-HONORS-THE-PARSE records for the particle arm — and it does not re-derive the initial-versus-conjunction class from a word's case, which is what #458 removed. The edges are pinned on both sides: `juan e-f smith` keeps `Juan E-F Smith` and `juan y-garcia` gives `Juan Y-Garcia`, and in a three-part word `juan y-garcia-lopez` gives `Juan Y-Garcia-Lopez` and `juan garcia-lopez-y` gives `Juan Garcia-Lopez-Y`. A SINGLE LETTER MARKED WITH A PERIOD is an initial there, as the parse reads it in spaced text, never the connective (the guard is `not re.fullmatch(r"\w\.", part)`): the first draft lowered the `e.` in `j.-e.-p. dupont`, giving `J.-e.-P. Dupont` where every release from 1.4.0 through 2.3.0 gave `J.-E.-P. Dupont`, a regression found in review and pinned since as a rules.md#R4 boundary line. A MULTI-letter word marked with a period is not an initial and stays reachable, as its spaced reading is (`hans smith und. jones` tags `und.` a conjunction), so `hans smith-und.-jones` gives `Hans Smith-und.-Jones`. This reverses, for the interior position only, what #458's tag read did to the shape: 2.2.0 and 2.3.0 gave `Ortega-Y-Gasset` for both spellings — the side effect #478 reported, which the 2026-08-29 #458 bullet above does not record — and #458's edge answer stands. Parity, measured on the released wheels: 1.4.0, 2.0.0 and 2.1.0 re-decided per word and gave `Jose Ortega-y-Gasset` lowered but `Jose Ortega-Y-Gasset` upper, so the all-caps half is a parity break with them on #479's one-case precedent; they also gave `Juan y-Garcia` at the edge, which neither 2.2.0, 2.3.0 nor this tree does. KNOWN NON-REACH, with its witness: the Māori corpus name `Te Awanui-a-Rangi Black` still repairs to `Te Awanui-A-Rangi Black` under force, because `a` is not connective vocabulary and must not become one (a spaced `a` is an initial or a name elsewhere; P3's Excluded block for the connectives says so). ACCEPTED: `mary-e-smith` reads `Mary-e-Smith` — no evidence the shape exists, and the hyphen is the writer joining the parts. Every role, since a hyphenated given name is the same text shape; a middle-role row is pinned beside the family ones (`smith, jose ortega-y-gasset` forced gives `Jose Ortega-y-Gasset Smith`). - -- 2026-09-23 #459/#478 — MEASURED, with the recipe; none of these numbers is a live count. Population: the deduped `tools/differential/corpus*.jsonl` glob, both line shapes (an object with `name`, a bare string), as this change leaves it — 1375 distinct names, the new rules.md example lines among them (1371 before the two 2026-09-24 decisions below added four; re-measured 2026-09-24). Comparator: the parent 4d0680e6 — `git archive 4d0680e6` into a scratch directory, both trees imported BY PATH in separate processes over the SAME corpus directory, each asserting `nameparser.__file__` — comparing every token's `capitalized()` and `capitalized(force=True)` text on the core, and the rendered string on the facade, and attributing each moved token PER SUB-WORD: the token is split into its `_WORD` sub-words (`(\w|\.)+`), and the FIRST sub-word whose repair changed decides the bucket, tried in this order — its period-free key is a shipped mask key (`phd`, `bsc`, `msc`, and since 2026-09-24 the fifteen below) or one that left the map (`md`, `ii`, `iii`, `iv`); else the token is a suffix carrying `SHAPE_ACRONYM_TAG` or the sub-word fits the roman shape; else the sub-word is an interior connective part of a hyphenated token; else, on the default path only, the token's repair used to equal its input, which is R5's admission — with a bucket for anything else, asserted empty. The per-sub-word split is load-bearing: keyed on the token as written, the corpus name `John Doe, MD-PhD-` (key `md-phd`) falls to the hyphen bucket instead, giving 15 and 82, 3 and 5 over that day's 1371 names and 11 and 77, 0 and 1 over the parent's corpus (measured in review on 2026-09-23). Mask, or a word that left the map: 16 names on the default path and 84 under force over the 1375 (15 and 83 over the 1371 names before the 2026-09-24 decisions). Dotted shape and numerals: 3 and 16 over both. Hyphenated link: 4 and 5 over the 1375 (3 and 4 over the 1371), every one of them a rules.md#R4 example line — no corpus name written before this change carries a connective part in a hyphenated word, at the interior or an edge (0 of 1340 at 4d0680e6). R5's gate is R5's bullet. The same comparison over the parent's own corpus directory, 1340 names and none of this change's example lines, gives 11 and 78, 1 and 14, and 0 and 0 — the same before and after the two 2026-09-24 decisions, which move no name written before them: the difference is this change's own rules.md lines, and those are the counts to read as reach into names written for other reasons. The core and the facade move on the same number of names in both views. No role field moves on any name, which is also all the differential gate can say: its compared surface is the seven role fields, `_ambiguities` and `_initials`, never case (the 2026-08-29 bullet above). - -- 2026-09-23 — mechanisms.md#RENDER-HONORS-THE-PARSE CORRECTED, found in this change's re-read of that entry: it still said case repair carries the connective carve-out "ON R3's authority", quoting R4's pre-#461 text ("the carve-out R3 states for initials"), and that `.initials()` gives `j. a. j. s.` for `john and jane smith`. Both had been stale since #461 (2026-09-20), which cut R4's cross-reference to R3 in favor of R4's own reason and stopped a connective with a name word beside it from initialing. The entry now states R4's own reason, dates the cut, and names where the two views part today (`Juan de y`: initials `J. y.`, forced repair `Juan de y`), re-measured 2026-09-23. - -- 2026-09-24 #459/#492/#478 (branch review) — FOUR FINDINGS FROM THE REVIEW ROUND, recorded together because each is a small correction to a bullet already above rather than a new decision. (a) THE UNICODE BOUNDARY IS NOT SCOPED TO THE TWO SHAPE CLAUSES, AND NOT TO ONE WAY A MASK LETTER COMES OUT UPPER EITHER. The 2026-09-23 bullet above states the `ß` and dotless-`ı` boundary against `_cap_word`'s acronym/numeral clauses, but the MASK path can lengthen a word through `ß` exactly the same way. `_apply_mask` decides per letter whether it is written upper -- by the mask's own case, or by the split-off-initial force -- and then takes one casing step: the letter at that position in the whole-word `word.upper()`/`word.lower()` when both keep the word's length, otherwise a per-character `c.upper()`/`c.lower()`. `'ß'.upper()` is `'SS'`, so wherever the word holds a `ß` the whole-word form changes the length and every letter goes through `c.upper()`, which lengthens a `ß` written upper to `SS` whichever of the two reasons made it upper. Through the force: an identity mask validates fine (`('a.ß', 'aß')` spells the key's own letters unchanged), so `john a.ß smith` forced gives middle `A.SS` -- casefold-equal to its input and one character longer, the same trade the shape clauses make. Through the mask's own case, no full stop or split letter involved: `('straße', 'STRAẞE')`, the value spelling `straße`'s own letters unchanged with the actual German capital `ẞ` (not the digraph `'STRASSE'`, which the validator refuses as a VALUE for failing to spell `ß` at all), validates and `john straße` forced gives family `STRASSE`. The validator's refusal of `('straße', 'STRASSE')` is a different question -- whether the VALUE spells the KEY -- and says nothing about what the applier can produce from a mask that does validate. This same boundary is where `capitalized()`'s own "Idempotent" docstring claim narrows: forcing `A.SS` (that first pass's own output) through the SAME lexicon a second time gives `A.ss`, not `A.SS` back, because the second pass folds the word to `a.ss` and the exceptions map's key is still `a.ß` -- a different letter sequence, not merely a different case of one -- so the lookup that found the entry on the first pass misses on the second and the word falls through to plain title-casing. Pinned by `tests/v2/test_render.py::test_a_masks_upper_fallback_can_lengthen_a_word_through_ss` and `tests/v2/test_render.py::test_a_lengthening_mask_is_not_a_fixpoint_under_a_second_forced_pass`. The same boundary reaches plain title-casing with no mask involved, by two distinct mechanisms: LENGTHENING, where a letter's own upper() is a different letter SEQUENCE that stays \w -- 'ʼn' (one letter) upper-cases to the two cased characters 'ʼN', so `parse("ʼna smith").capitalized(force=True)` gives given `ʼNa`, and forcing THAT output a second time gives `ʼna` ('N' is no longer the word's first character, so `str.capitalize()` lower-cases it on the second pass) -- and SPLITTING, where a letter's own upper() is a base letter plus a COMBINING MARK that `_WORD` does not match at all -- 'ǰ' upper-cases to 'J' + a combining caron, so `parse("ǰo smith").capitalized(force=True)` gives given `J̌o`, and forcing THAT output a second time gives `J̌O` (`_cap_text` reads the output as two separate words, 'J' and 'o', the combining mark matching neither, and 'o' capitalized alone is 'O'). (b) MASK VALUES ARE STORED NFC-COMPOSED. `_normpairs` compared a value against its key NFC-composed but stored the value AS WRITTEN, so a value written in decomposed form spelled one composed letter as a base letter plus a non-alpha combining mark -- changing which letters `_letter_run_ge2` reads as split off beside a full stop from what the composed spelling of the identical value would read. `john smith pé.x.` forced gave `Pé.X.` under a composed mask and `Pé.x.` under the decomposed spelling of the SAME value. Fixed by NFC-composing the value at the same point the key is composed; pinned by `tests/v2/test_lexicon.py::test_a_decomposed_value_is_compared_and_stored_nfc_composed` and `tests/v2/test_render.py::test_a_decomposed_mask_value_reads_the_same_split_as_composed`. (c) THE HYPHEN CLAUSE'S INITIAL TEST IS NOW A PINNED MODULE CONSTANT. `_cap_text`'s inline `re.fullmatch(r"\w\.", part)` was a third hand copy of the initial shape's period half, invisible to `tests/v2/test_regex_sync.py`'s hand-copy sweep because it was never hoisted to a module-level pattern. Hoisted to `_render._DOTTED_INITIAL` and registered as `_INITIAL`'s period alternative; pinned by `tests/v2/test_regex_sync.py::test_dotted_initial_is_the_period_alternative_of_initial`. (d) THE V1 RAISE NOW CARRIES A V1-SPELLED HINT, VIA A TYPED EXCEPTION. The 2026-09-23 bullet above records that a mismatched-mask value raises `ValueError` at a v1 `Constants`' first parse; the message it raised was `Lexicon`'s own, offering a `Lexicon(capitalization_exceptions=...)` constructor call to a caller looking at a `TupleManager` assignment through `Constants`. `_lexicon._normpairs` now raises a dedicated `_MaskValueError(ValueError)` for exactly that one shape, built in the `UnicodeDecodeError` idiom: `__init__(self, key, normalized_key, value)` stores the three offending inputs and computes `self.offered` (from `_offered_mask`) and the message from them, rather than taking a pre-built message. `__reduce__` returns `(type(self), (self.key, self.normalized_key, self.value), self.__dict__)` -- reconstructing from the three fields and then restoring `self.__dict__` (so an attribute added after construction, e.g. `add_note()`, survives too) -- since the inherited default (`type(self)(*self.args)`, `self.args` holding only the message) cannot reconstruct an instance that needs three fields, not one, and raises `TypeError` trying; a worker in a `ProcessPoolExecutor` pickles an exception it raises to deliver it to the caller, so an unpicklable one there surfaces as `BrokenProcessPool` instead. `_config_shim._build_snapshot` wraps ONLY the `Lexicon(...)` construction in `try`/`except _MaskValueError` and re-raises a message built fresh from `e.key`/`e.value`/`e.offered` -- not from `str(e)`, which is the 2.0 exception's own message offering the 2.0 `capitalization_exceptions=((...),)` constructor spelling no v1 caller can paste -- with `constants.capitalization_exceptions[e.key] = e.offered`. Every OTHER `ValueError` `Lexicon(...)` can raise passes through unchanged, by TYPE rather than by a message-substring test. Of those, the three SUBSET checks (`particles_ambiguous` inside `particles`, `suffix_acronyms_ambiguous` inside `suffix_acronyms`, `honorific_tails` inside `suffix_words`) cannot fire from here: each is built as an intersection or union with its OWN base, so the normalized image of that relation is still a subset whatever the raw spelling. THE CONTRADICTION CHECK AND THE GATE-BYPASS CHECK ARE A DIFFERENT SHAPE, AND DO REACH `Lexicon(...)` -- this bullet claimed otherwise ("cannot arise ... observing no raise") until the confirmation round measured the opposite. Both compare two sets built by SEPARATE set arithmetic in `_build_snapshot`, on the RAW SetManager entries, before Lexicon's own per-field normalization ever runs: `SetManager` strips edge periods but not edge whitespace, while `Lexicon`'s normalization strips both, so an entry carrying edge whitespace can slip past the shim's own subtraction or union and still collide once Lexicon normalizes it independently. Measured: `c.suffix_not_acronyms.add('ba ')` then a parse raises Lexicon's own gate-bypass check (`an ambiguous suffix acronym must not also be a suffix word; in both: ba`), and `c.bound_first_names.add("'t ")` then a parse raises the contradiction check -- both pass through the `except _MaskValueError` clause above unchanged, being a different exception type. Pre-existing at HEAD and at master alike, not introduced by this session and not fixed here -- left for the orchestrator to file as its own issue. What DOES reach `Lexicon(...)` here unchanged, always, is an entry normalizing to empty, in `capitalization_exceptions` or in ANY set field -- measured: `c.titles.add("...")` then a parse raises Lexicon's own `titles entry '' normalizes to empty`, unaltered -- since nothing above translates that away, and a non-str value's `TypeError`, a raise v1 also had, later, at `capitalize()`. Declined: a first draft pre-validated `capitalization_exceptions` in isolation, a second call to `_normpairs` ahead of `Lexicon(...)` gated on a message-substring test; measured to make a multi-word key's dead-entry warning fire twice on the shim path, and to rest on a filter that was untested (deleting it passed the whole suite) and wrong (an unrelated ValueError then got the v1 hint appended regardless). Pinned by `tests/test_capitalization.py::test_a_mismatched_exception_value_raises_at_the_first_parse` (the hint, and that it carries no 2.0 constructor spelling), `tests/v2/test_config_shim.py::test_an_unrelated_capitalization_exceptions_valueerror_has_no_v1_hint` (the pass-through), `tests/v2/test_config_shim.py::test_a_multiword_capitalization_exceptions_key_warns_exactly_once` (the single warning), and `tests/v2/test_lexicon.py::test_mask_value_error_survives_pickle_and_copy` (the pickle/copy round-trip). - -- 2026-09-24 #459 — DECIDED (Derek, 2026-09-24): the shipped map gains a mask for each listed acronym whose conventional spelling is mixed case, taken together with decisions.md#R5's keep-as-written bullet of the same day, which is the half that surfaced the need. The reviewer who measured the R5 regression named nine keys — `edd`, `dsc`, `psyd`, `meng`, `dmin`, `thd`, `drph`, `pharmd`, `mdiv` — and a sweep of `SUFFIX_ACRONYMS` for further entries whose usual spelling is unambiguously mixed found ten more: `bt`, `chfc`, `cpht`, `diplac`, `kt`, `phc`, `rph` and `thm`, and `lac` and `ded`. Of those nineteen, four are left out (below), so SHIPPED is fifteen — seven of the reviewer's nine and eight of the sweep's ten — every key a `suffix_acronyms` member: `bt` → `Bt` (Baronet), `chfc` → `ChFC` (Chartered Financial Consultant), `cpht` → `CPhT` (Certified Pharmacy Technician), `diplac` → `DiplAc` (Diplomate in Acupuncture, written `Dipl. Ac.`), `dmin` → `DMin` (Doctor of Ministry), `drph` → `DrPH` (Doctor of Public Health), `dsc` → `DSc` (Doctor of Science), `kt` → `Kt` (Knight Bachelor), `mdiv` → `MDiv` (Master of Divinity), `pharmd` → `PharmD` (Doctor of Pharmacy), `phc` → `PhC` (Pharmaceutical Chemist), `psyd` → `PsyD` (Doctor of Psychology), `rph` → `RPh` (Registered Pharmacist), `thd` → `ThD` (Doctor of Theology), `thm` → `ThM` (Master of Theology). The four LEFT OUT are two of the reviewer's nine, `meng` and `edd`, and two of the sweep's ten, `lac` and `ded` (the Excluded block below): a mask applies wherever its word stands (the 2026-09-23 masks bullet above), so a key borne as a name re-spells a person — the #vocabulary-collisions C-i test read with the mask's reach, which is every position. Candidates considered and not taken for want of one settled spelling: `djur` (`D.Jur.` beside the German `Dr. jur.`), and `litk`, `litl`, `litp`, `mscmsm`, whose credentials could not be identified. WHY BOTH HALVES: each serves a population the other cannot. Keeping a suffix as written serves the writer who spelled it (`john smith EdD`), but a name written wholly in one case carries no spelling to keep, its suffix being one-case too, so only a mask gives `john smith psyd` → `John Smith PsyD` (`PSYD` by the acronym clause without it; `Psyd` on every release 1.4.0 through 2.3.0, measured 2026-09-24); and a mask cannot reach an unlisted credential or a caller's own spelling, which the keep rule does. No corpus name written before this change carries any of the fifteen keys (measured 2026-09-24 over the parent's 1340 names and this tree's 1375), so the MEASURED bullet above moves only by the rules.md line `john smith psyd`. Recompute the list: `sorted(Lexicon.default().capitalization_exceptions)`. Pinned on both surfaces by `tests/v2/test_render.py::test_a_shipped_mask_spells_its_suffix_in_either_single_case`, which derives its rows from the shipped pairs and so pins the property rather than the list, and the exclusion's fork by `test_a_listed_acronym_that_is_a_name_word_gets_no_mask`, whose recorded negative control is the `meng`/`edd` masks added back. - -- 2026-09-24 — SUPERSEDED IN PART: the 2026-09-23 bullet above headed "A SUFFIX READ BY ITS DOTTED SHAPE" accepted that `john smith B.Tech.` gives `John Smith B.TECH.` on the DEFAULT path once the suffix-free gate admitted the name. That half no longer holds: decisions.md#R5's keep-as-written bullet (2026-09-24) keeps a suffix written in more than one case, so the default path gives `John Smith B.Tech.`, the credential as written — every release 1.4.0 through 2.3.0 returned the whole name untouched, `john smith B.Tech.` (measured 2026-09-24). The acceptance stands under force only — `John Smith B.TECH.` with `force=True`, where every release gave `B.tech.` — and the mask remedy it names is unchanged. The rules.md#R5 example line proposed for this spelling (`juan garcia B.Tech.`) was not added: the differential gate reported it UNEXPLAINED at every baseline, the by-shape suffix reading being a role change no ledger rule claims for that name, so it is pinned as a unit test instead (`tests/v2/test_render.py::test_a_suffix_written_in_more_than_one_case_is_kept_as_written`). - -- 2026-09-24 #478 — DECIDED (Derek, 2026-09-24): the hyphen clause keeps its reading in a name written wholly in one case, and the case where that disagrees with rules.md#P3 is a recorded boundary, not a change. The two collide on a marked letter. Spaced, a letter the vocabulary marks as reading both ways reads as an initial in a one-case name (P3), so `maria silva e sousa` repairs to `Maria Silva E Sousa`; hyphenated, the interior part reads as the connective whatever the name's case, so `maria silva-e-sousa` repairs to `Maria Silva-e-Sousa`. Derek's reasoning: hyphenating `Silva-e-Sousa` is the writer joining the surname on purpose, so the interior word is a connective by that act, whatever the one-case rule would say of the spaced form. THE COST, measured 2026-09-24: a one-case name whose hyphenated bare initials happen to spell a connective — `J-E-P DUPONT` gives `J-e-P Dupont`, where every release 1.4.0 through 2.3.0 and the parent 4d0680e6 gave `J-E-P Dupont`, and `JOHN A-Y-B SMITH` gives `John A-y-B Smith` (every release: `John A-Y-B Smith`). The period-marked spelling is unaffected (`j.-e.-p. dupont` keeps `J.-E.-P. Dupont`, the 2026-09-23 #478 bullet). `conjunctions_ambiguous`, P3's knob, does NOT reach a hyphenated word: under `Lexicon.default().add(conjunctions_ambiguous={"y"})` the spaced `JOSE ORTEGA Y GASSET` repairs to `Jose Ortega Y Gasset` (default lexicon: `Jose Ortega y Gasset`) while `JOSE ORTEGA-Y-GASSET` stays `Jose Ortega-y-Gasset` under both. rules.md#R4's hyphen sentence said the hyphens join the name "as the spaced connective would", which claimed an equality the one-case case breaks; it now says the hyphens are the writer's join and states the split, with an Accepted paragraph and the `J-E-P DUPONT` boundary line. Pinned by `tests/v2/test_render.py::test_the_hyphen_is_the_writers_join_even_in_a_one_case_name`. -- 2026-10-02 #541 — THE "LEFT FOR THE ORCHESTRATOR TO FILE" FINDING of the 2026-09-24 branch-review bullet (d) above is #541, and it was wider than that bullet saw: besides the two raises it names, an entry carrying whitespace v1 never matched SILENTLY ACTIVATED in every set field the shim copies through and in `capitalization_exceptions` keys — measured 2026-10-02 against 1.4.0 (run outside the worktree, `PYTHONSAFEPATH=1`): `titles ' dean '` gave title `dean` on `dean john smith` where 1.4.0 gave first `dean`, and likewise `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a `' zzc '` key; the raise (`suffix_not_acronyms 'ma '`) and the activation (`titles ' dean '`) both reproduce on the 2.0.0, 2.2.0 and 2.3.0 wheels. DECIDED (Derek): `Constants._snapshot()` drops every entry `_config_shim._v1_matchable` rejects — empty after `lc()`, or not equal to its own single-spaced re-join — from all nine set fields and the keys, BEFORE its set algebra, and names them in one `UserWarning` whose remedy runs (`tests/v2/test_config_shim.py::test_the_offered_remedy_runs_and_silences_the_warning`). The test is exact because a v1 piece comes from a whitespace split re-joined only with single spaces; it generalizes the filter `given_name_titles` already had, and `tests/v2/test_config_shim.py::_UNFILTERED_OUTCOME` records what each swept row does with it off. One exception, decided when the 1.4 pickle test caught it: every release from at least 0.5.8 through 1.4.0 SHIPPED two such entries in `TITLES` (`'actor '`, `'television '`), so a restored 1.4 pickle carries them and its user never wrote them; they are dropped WITHOUT the warning (`_V14_SHIPPED_UNMATCHABLE`, held to the pickle's own unmatchable set by `test_the_1_4_shipped_unmatchable_roster_is_exactly_the_pickles`), keeping that pickle warning-free. The exemption is keyed by entry, not by provenance, so a user who writes `'actor '` or `'television '` themselves is not warned either — accepted, since the two strings are 1.x's own typos. The drop is a reading change for that pickle: `Actor John Smith` gave title `Actor` on 2.0.0 through 2.3.0 and gives first `Actor` now, as 1.4.0 did. DECLINED: stripping whitespace in `SetManager` (activates every v1-inert entry above), and a silent drop for user entries (Derek: the entry is certainly a typo and certainly dead, so say so). ACCEPTED WIDENING, deliberately left: a key written with capitals or edge periods (`'McDonald'`, `'phd.'`) never matched on 1.4.0, whose lookup is `lc(word)` against keys stored as written, but has matched since 2.0 through `Lexicon`'s fold; dropping it would break a working 2.x config to restore an inertness nobody wanted. Also left: a set entry `Lexicon` folds to empty that `lc()` does not, a lone non-ASCII full stop (`'。'`), still raises at the first parse in every field `Lexicon` receives directly — `titles`, `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a `capitalization_exceptions` key; `first_name_titles` drops it quietly (`if t`), and `non_first_name_prefixes` and `suffix_acronyms_ambiguous` never pass it to `Lexicon` at all — v1-matchable, so outside this rule. -- 2026-10-02 #582 — SUPERSEDES the #541 bullet's "Also left" sentence above. DECIDED (Derek): an entry `Lexicon._normalize` folds to empty is dropped by `_config_shim._v1_matchable` with the #541 warning, in every set field and key. The case is a lone CJK full stop (`'。'`, `'．'`, `'｡'`, or a run of them), which 2.3.0's `FULL_STOPS` fold (#322/#323) empties while v1's `lc()` strips only `.`. Measured 2026-10-02 on released wheels run outside the checkout with `PYTHONSAFEPATH=1`: 1.4.0 through 2.2.0 accept `c.titles.add('。')` and read `。 john smith` with title `。`, while 2.3.0 raises `ValueError` at the first parse, as did this tree before the change, in `titles`, `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a key. ACCEPTED DEPARTURE from 1.4.0, the only one this filter makes: such an entry used to act on a token that is nothing but that full stop. Since 2.3.0 `Lexicon` can neither hold the entry nor match the token, its lookup fold emptying both, so reproducing v1 is not on offer and the choice was drop or raise. The warning's wording moved with it, from "nameparser 1.x never matched" to "match no name word". `given_name_titles`' `if t` filter was deleted as unreachable: `_title_key` is empty only when every word folds away, and then the whole entry folds to empty and was already dropped. With no shim-reachable plain `ValueError` left from `_normpairs`, `test_an_unrelated_capitalization_exceptions_valueerror_has_no_v1_hint` turns the filter off to keep the except-by-type clause pinned. THE SAME FOLD, ONE CHARACTER IN: an entry with a CJK full stop at its EDGE (not only full stops) passed the filter, and the set algebra compared it raw while `Lexicon` folded it, so `c.suffix_not_acronyms.add('ma。')` (beside the ambiguous `ma`) raised the gate-bypass check and a bound `'zed。'` beside a never-given particle `zed` raised the contradiction check, both from 2.3.0 on; measured accepted on 1.4.0 and 2.2.0 by the review of this change. `_build_snapshot` now compares `_normalize`d spellings in exactly the two computations `Lexicon` re-checks after its own fold — dropping a `suffix_not_acronyms` entry whose fold is an ambiguous acronym's, and taking into `particles_ambiguous` every particle whose fold is a bound given name's — so such an entry reads exactly as its folded spelling (`test_an_edge_full_stop_collision_reads_as_its_folded_spelling`; the fold-off control is `_UNFOLDED_OUTCOME`). That reading is the `dean。` widening below, not v1's: 1.4.0 matched `ma。` only on a `ma。` token. DECLINED, after being written and reviewed: folding EVERY kept entry before the set algebra. `non_first_name_prefixes` and `suffix_acronyms_ambiguous` reach `Lexicon` only through raw set algebra, so folding them is a NEW match, not the 2.3.0 one: `prefixes zz` with `non_first_name_prefixes 'zz。'` moved `zz smith` from first `zz` to last `zz smith`, an NFD/NFC pair across those fields moved the same way, and `suffix_acronyms_ambiguous 'zq。'` beside `suffix_acronyms zq` moved `john zq` from suffix to last — the particle readings the same on 1.4.0, 2.2.0, 2.3.0 and the pre-fold tree, the `zq` one on 2.2.0, 2.3.0 and the pre-fold tree (1.4.0 gives last `zq` there by its own two-word reading, not by matching the entry). They are pinned at those readings by `test_an_edge_full_stop_entry_outside_the_two_checks_reads_as_before`. ACCEPTED WIDENING, left as it is (Derek): since 2.3.0 a `titles` entry `'dean。'` also matches a bare `dean`, where 1.4.0 matched only `dean。`. The same fold makes it, and dropping the entry would lose the `dean。` match too, so it sits beside the capitalized-key widening above. -- 2026-10-02 #542 — A DECOMPOSED WORD IS REPAIRED AS ITS COMPOSED TWIN, and the repaired text keeps the form it was typed in: no clause composes its OUTPUT. Two mechanisms do it. The word splitter and the mask's neighbour tests read a combining mark (Unicode category M) as part of the letter before it; the three clauses that ask a regex about a whole word's SHAPE — Mac/Mc and the two initial tests — ask it of the word's composed spelling. They differ only for a mark with no precomposed form, where composing leaves the word as written and so agrees with the parse: under a caller-added conjunction `q́`, `ivan q́. petrov` parses `q́.` as the conjunction, and the hyphen clause keeps `Petrov-q́.-Sidorov` lowercase as it does (measured by the docs review, 2026-10-02). A name typed NFD (macOS file names, some databases) had been split at each mark by `_WORD`'s `(\w|\.)+`, `\w` matching no mark, so `josé garcía` repaired to `José GarcíA` on every release from 1.4.0. Python's `re` has no `\p{M}`, and a hand-written class of the combining blocks would be a copy of Unicode data that drifts (it would already miss Adlam, a cased script whose marks sit outside them), so the test is `unicodedata.category(ch)[0] == "M"`, asked by `_render._sub_words` after each `_WORD` match and by `_render._beside` for a letter's neighbours. DECLINED: NFC-composing before repair, the issue's option 2. It fixes the split but hands back text in a form the writer did not use, which is a change beyond case under rules.md#R4 — `tests/v2/test_properties.py::test_case_repair_changes_case_and_nothing_else` compares with `casefold()`, which does not normalize, and would report it. THE ISSUE'S CLAIM THAT THE MASK NEEDED NO CHANGE ONCE THE WORD REACHED IT WHOLE WAS MEASURED FALSE, and the Mac/Mc clause needed one too; both were found by fixing the splitter alone and comparing again. `_apply_mask`'s split-off-initial override and `_letter_run_ge2` read a letter's neighbour by raw index, which in NFD is the mark rather than the full stop or the letter past it: with the splitter alone, under a caller's `('éx', 'éx')` mask `john smith é.x.` forced gave `É.X.` composed and `é.X.` decomposed — a shape that AGREED before the fix (`É.X.` both ways) only because the old splitter cut the word at its mark before the mask was looked up — and under `('éex', 'éeX')` `john smith ée.x` gave `ée.X` against `éE.X` (`ÉE.X` before the fix). Both read past marks now. The Mac/Mc clause's `\w{2,}` fails at the mark, so with the splitter alone `macée` gave `Macée` decomposed against `MacÉe` composed (`MacéE` before the fix, the split having made the last `e` a word of its own); the clause is now DECIDED on the composed spelling and applied to the word as written. The two initial-shape tests, `_DOTTED_INITIAL` in the hyphen clause and `_INITIAL` in the unclassified-text fallback, matched the raw word and now match its composed spelling, so a decomposed `й.` — `й` is the one default conjunction that decomposes — is the initial its composed twin is: `ivan petrov-й.-sidorov` repaired to `Ivan Petrov-й.-Sidorov` decomposed against `Ivan Petrov-Й.-Sidorov` composed, and a middle spliced in as decomposed `й.` kept `й.` against `Й.`, at 97af1f02 and at this change's first commit alike (found by the docs review, 2026-10-02). AMENDS the 2026-09-24 branch-review bullet (a): its SPLITTING half is retired. `parse("ǰo smith").capitalized(force=True)` gives `J̌o`, and a second forced pass now gives `J̌o` back rather than `J̌O`, the mark staying with its letter (measured 2026-10-02); the LENGTHENING half (`ß`, `ʼn`) stands. A mark with no letter before it heads no word: `str.capitalize()` on a word beginning with one would upper-case the mark and leave the letter after it lowercase. Out of reach and left there, two of them: `initials()`, a rules.md#R3 view rather than case repair, takes a token's first character, which for a decomposed initial letter is the base letter without its accent (`parse("émile zola")` initials `é. z.` composed and `e. z.` decomposed, measured 2026-10-02; fixed the same day under #585, decisions.md#R3); and unspaced hangul, whose NFD parse differs from its NFC parse because segmentation matches the census list as written (docs/usage.rst, "Decomposed text"), so repair follows a different parse; hangul jamo are letters, not marks, and nothing here touches them. VERIFICATION: the differential gate cannot see this — `capitalized()` is not a compared surface (this entry, 2026-08-29); the two NFD rows this change adds to the rules corpus reach the gate through the role fields, `_ambiguities` and `_initials` only. Standing in its place: rules.md#R4's two NFD example lines, `tests/v2/test_properties.py::test_a_decomposed_name_repairs_as_its_composed_twin` (renamed `test_a_decomposed_name_renders_as_its_composed_twin` by #585, which joined initials to the walk), which decomposes every corpus and case-table text that decomposes, compares its repair with the composed text's on both surfaces plain and forced, and records as its control 144 disagreements over 137 texts with 97af1f02's `_render.py` over this change's corpus (measured 2026-10-02), and `tests/v2/test_render.py::test_a_decomposed_word_repairs_as_its_composed_twin` and `test_a_spliced_decomposed_initial_reads_as_its_composed_twin` for the shapes no corpus name reaches, whose two mask rows were mutation-checked by restoring raw-index neighbours. -- 2026-10-04 #604 — DECIDED (Derek, 2026-10-04): a mask outranks the particle lowercase. Adding the Irish particles moved case repair the wrong way: `SEÁN Ó MURCHÚ` repaired to `Seán ó Murchú`, where 2.3.0 gave `Seán Ó Murchú` (Ó having been a middle initial there), and `ní` and `ua` the same, because repair writes every particle in lowercase while Irish writes these three capitalized. The map is already "how a word is written wherever it stands" (the role-free note in `_cap_word`), so the particle arm now steps aside for a word the map carries, and the map gains `ó`→`Ó`, `ní`→`Ní`, `ua`→`Ua`. Each mask is its word's own title case, so a bare word in any other role repairs as it would without one. The one reach past particles is the dotted spelling of the two-letter masks: each writes its word as one run, so R4's split-off-letter clause capitalizes both letters, and `u.a. smith` repairs to `U.A. Smith` where it gave `U.a. Smith`, `n.í. smith` to `N.Í. Smith` where it gave `N.í. Smith` (measured 2026-10-04, with and without the three masks; `ó` has no dotted twin), which is right for two initials. Measured before deciding, no shipped exceptions key was a particle (`set(Lexicon.default().capitalization_exceptions_map) & Lexicon.default().particles` was empty), so in the SHIPPED vocabulary the precedence moved nothing but the three new entries. A CALLER's map is another matter, and this is a deliberate change there too (Derek, 2026-10-04, after the design and code reviews both found it): `c.capitalization_exceptions['van'] = 'Van'` makes `ludwig van beethoven` repair to `Ludwig Van Beethoven`, where 1.4.0 through 2.3.0 gave `Ludwig van Beethoven`, the particle having won in v1's `cap_word` and every release since. Kept because an entry is how the caller says a word is written wherever it stands, and a caller who maps `van` to `Van` plausibly means it; the release bullet says so, since the differential gate cannot see `capitalized()` (#R3, above). Pinned by test_a_caller_particle_mask_outranks_the_particle_lowercase. Not extended to `mac`, which repairs to `mac` as before (`Liam mac Murchú`): `mac` is an ambiguous particle shared with Scottish and other spellings, and nothing in #604 weighed it. The Malay `binti`/`binte` carry no mask and keep their lowercase (`Ina binti Navalamar`, where 2.3.0 gave `Ina Binti Navalamar`). - -Excluded (CAPITALIZATION_EXCEPTIONS — meng, edd, lac, ded, left out of the 2026-09-24 masks, #459): - -- `meng` (MEng) and `edd` (EdD) are conventionally mixed-case credentials and listed acronyms, and must not get a mask while the map stays role-free: a mask applies in every role, and both are borne as given names or surnames. Measured 2026-09-24 with each mask added to a private lexicon: `MENG LI` gives `MEng Li`, `meng wang` gives `MEng Wang`, `edd smith` gives `EdD Smith` and `EDD KIMBER` gives `EdD Kimber`, where the shipped map and every release give `Meng Li`, `Meng Wang`, `Edd Smith` and `Edd Kimber`. The same holds for two sweep candidates, `lac` (LAc, Licensed Acupuncturist; `Lac` is a Vietnamese given name — the trailing word in native order, `Nguyễn Văn Lạc` — and a French surname; corrected 2026-09-25, #540, from "a Vietnamese and French surname") and `ded` (DEd; `Ded` is an Albanian given name, `ded gjo luli` giving `DEd Gjo Luli` under the mask). A writer who spells one mixed-case keeps it (decisions.md#R5, 2026-09-24); a caller who wants the mask adds it to a private lexicon. A NOTE on `SUFFIX_ACRONYMS` itself, which this map does not decide. The two name-word collisions there come from two bulk additions, not one: `lac` (with `edd` and `ded`) from the Wikipedia post-nominals import, `af5bdab` (2019-12-11, #93), and `meng` from `3e14ea20` (2026-07-01, the German/Dutch title and degree batch, first shipped in 1.3.0). Both read as a suffix after a given name, and lose a family name to it (superseded 2026-09-25 by #540: both words are marked ambiguous, `wang meng` reads family `meng` and renders `Wang Meng` — see #suffix-acronym-collisions). Measured 2026-09-24 on the released wheels, the parent 4d0680e6 and this tree: `wang meng`, `li meng` and `zhang meng` read family `meng` on 1.4.0 and suffix `meng`, with no family, on every release from 2.0.0; and since #521 on master the suffix renders `Wang MENG` (parent and tree alike), where 2.3.0 gave `Wang Meng`. `nguyen van lac` reads suffix `lac` with family `van` on every release from 1.4.0, and since #521 renders `Nguyen Van LAC` where 2.3.0 gave `Nguyen Van Lac`. This is a #vocabulary-collisions C-i question about the acronym set, not about this map. - -Excluded (CAPITALIZATION_EXCEPTIONS — md, ii, iii, iv, removed 2026-09-23, #459): - -- `md`, `ii`, `iii` and `iv` are OUT of the exceptions map and must not be put back by a sweep that finds one of them title-cased somewhere. As a SUFFIX each is written in capitals with no entry — `md` by the acronym clause, the numerals by the numeral clause — and in any other role each repairs as the parse read it by decision (the item-4 bullet above), so an entry would only bring back the role-free capitals that bullet retired: `iv smith` → `IV Smith`, `Md Abdul Karim` → `MD Abdul Karim`. A caller who wants them back adds them to a private lexicon, as masks. - -### R5 — the case-repair gate - -- 2026-08-29 (#407 arc) — EXTRACTION, not a decision: the parser is untouched. The two halves of R5 have separate provenance, and conflating them is easy enough that the first draft of this entry did. The REFUSAL — repair skips any name already carrying more than one case — is older than the git history: `git log -S "name == name.upper() or name == name.lower()" --reverse` bottoms out at 45a1539 (2011-02-03), the initial import from svn, where `capitalize()` already opens with that guard and a bare `return`. (A path-filtered search answers 280895b instead, the same-day commit that moved the module into `nameparser/`; the code did not change there.) The OVERRIDE is bf1e0a5, 2016-06-02, which did not add the refusal but wrapped it — `if not (name == name.upper() or ...)` became `if not force and not (...)` — and shipped in 0.4.0 (June 2, 2016; its own release-log line is docs/release_log.rst under that heading). So R5's statement as a whole holds from 0.4.0 on. rules.md had never said any of it, though bf1e0a5's diff shows the API docstring already did: "It will not adjust the case of names entered in mixed case" was there before that commit edited around it. The gap was rules.md's alone. Evidence, measured on the released 1.4.0 wheel — the last v1 release and one of the differential baselines, so a natural thing to measure against and not a release that introduced anything here — and re-measured on this branch today, facade and core agreeing: `HumanName('Shirley Maclaine').capitalize()` leaves `'Shirley Maclaine'` — mixed case, wrong, and kept — while the same name under `force=True` gives `'Shirley MacLaine'`; `HumanName('Juan McDonald').capitalize()` leaves `'Juan McDonald'`. rules.md's own preamble classifies behavior in this position as "pinned-but-undocumented — an extraction gap to close, not a specification", which is why the fix lands in the document rather than in `_render.py`. -- 2026-08-29 — why the gap survived, and what the new boundary buys. R4 promised that "an already-correct name comes back unchanged", which is idempotence: true of everything the gate returns, and silent about the gate. The two readings were indistinguishable through R4's boundary, because `"Juan McDonald"` comes back unchanged under `force=True` as well — an already-correct input cannot witness a rule whose subject is what gets kept REGARDLESS of correctness. `Shirley Maclaine` separates them, so R5's block pins it twice: kept as written on a plain `capitalized=` line, and repaired on the forced line. Under idempotence the plain line would have to fail. The BOUNDARY sits on the forced line, which took a correction to get right — the first draft marked the refusal, and the refusal is R5's effect, not its absence. R5's own stated exception is repair-regardless, so that is the non-firing case (rules.md's preamble: the boundary is "usually the rule's OWN stated exception"). -- 2026-08-29 — WHY R5'S STATEMENT WAS REWORDED RATHER THAN GIVEN A THIRD EXAMPLE, recorded at length because the wrong turn was taken twice and the reasoning is reusable. R5 first said a mixed-case name is returned untouched "whether or not its casing is correct". That phrasing reads as a disjunction with two branches, and invites the question of which example witnesses the already-correct one. Nothing can: for a mixed-case name that repair would not alter, kept and repaired are the SAME STRING by construction, so no example line distinguishes a parser with the gate from one without it. The unwitnessable branch is a property of the claim, not a gap in our choice of names — and the bullet above had already said so ("an already-correct input cannot witness a rule whose subject is what gets kept REGARDLESS of correctness") before two review rounds pushed past it. Two rows were tried and both withdrawn. `"Juan McDonald" → capitalized="Juan McDonald"` was INERT (mechanisms.md's inert-measurement class): it passes with R5's gate deleted, which is the whole failure shape that class names. `"Vincent Van Gogh" → capitalized="Vincent Van Gogh"` did discriminate — measured by comparing the shipped call against gate-off behavior, which the forced call reproduces exactly, since deleting the gate is what `force` already does: shipped `'Vincent Van Gogh'`, gate-off `'Vincent van Gogh'`, the particle rule lowercasing `Van` the moment repair runs — but it was REDUNDANT, not complementary. It asserts the same proposition as the `Shirley Maclaine` row above it: repair would change this name, and the gate keeps it anyway. The two differ only in which repair rule would have fired, and which rule repair applies is R4's subject, not R5's. And the frame it was chosen under was itself wrong, which is the more useful half of the lesson (reframed 2026-08-29 on Derek's correction, while R4 was being reworked). The row was picked to be bearer-correct — a name whose casing the bearer would endorse — and that frame drags the document into per-name arguments about whose spelling is right: this one needed `Vincent van Gogh` to be the correct spelling, which P6's own examples contradict, and the tussenvoegsel convention behind them is amended later on this branch anyway. CORRECTNESS DOES NOT ENTER INTO IT. Mixed case is the writer making an explicit choice, and repair defers to that choice rather than judging it — a name kept is not a name endorsed. Read that way the withdrawn row needed no claim about Dutch orthography at all, and the reason it needed one is exactly that the frame was wrong. The fix was to stop claiming the unwitnessable branch: the statement now says a mixed-case name is kept and that whether its casing is right does not enter into the decision, which `Shirley Maclaine` — casing wrong, kept — witnesses whole. GENERAL LESSON, and the reason this is long: when an example cannot be found for half a rule, suspect the STATEMENT before suspecting the example set. A phrasing that promises more branches than the behavior has will absorb inert examples indefinitely, each one looking like progress. Caution for the commit that reworks R4 and will choose its own mixed-case rows: avoid a name whose family base is wholly particle vocabulary (`Anh Van Do`, base `Van Do`), because the #407 work changes how those capitalize and would neuter such a row silently; `gogh` and `vega` are in neither `particles` nor `particles_ambiguous`, `van` and `do` are in both. -- 2026-08-29 — the override has TWO routes, and the rule states neither, by design. Per-call is the obvious one. The second is a v1 Constants attribute the facade still honors, `force_mixed_case_capitalization` (nameparser/_facade.py resolves it when no per-call value is given; docs/release_log.rst records it as "still honored through the facade"). Measured today: with that attribute set True, `HumanName('Shirley Maclaine')` then a bare `.capitalize()` — no argument at all — LEAVES `'Shirley MacLaine'` (the v1 call mutates in place and returns None, so the name is read back with `str()`). An earlier draft of R5 said "only an explicit request to repair regardless overrides that", which that measurement falsifies; the statement now says repair was asked for anyway, without saying by what route (it read `repair regardless of how the name is cased was asked for` until 2026-08-29, when that phrasing turned out to have a second reading -- see the override bullet below), because rules.md is implementation-free by its own preamble. -- 2026-08-29 — `capitalized_forced` is a test-side pseudo-field, not a parser field. The route the EXAMPLES use is the per-call argument to `capitalized()`, and an argument is not a policy, locale or extras gate — the only three things rules.md's grammar admits in an example's annotation slot — so it cannot ride that slot, and the doc runner grew a resolver branch instead. (The facade attribute above is a second route to the same behavior, not a second thing to assert; the core takes the argument only.) The R5 block's example lines are asserted by the suite like any others. They do enlarge one file: `corpus_rules.jsonl` is generated from this document's examples, and `Shirley Maclaine` had not been among them, so regenerating adds it (235 lines, from 234). Whether that moves the DIFFERENTIAL population is a separate question with a per-name answer, and the general answer is that a rules.md example CAN move it: the harness dedupes across corpus files, so a new example name costs a population slot exactly when no other corpus already carries that same string. `Shirley Maclaine` was already in `corpus.jsonl`, so it costs nothing and the population holds at 1090 — measured, not assumed. The row withdrawn above is the counter-case, and worth keeping for it: `Vincent Van Gogh` appears in no other corpus (`corpus.jsonl` and `corpus_issues.jsonl` carry `Vincent van Gogh`, a DIFFERENT string that does not dedupe against it), and while it was in the block the population read 1091. Neither name needed a ledger rule, which is a measurement rather than a consequence of the above: the gate exits 0 with 0 unexplained at all three baselines either way. -- 2026-08-29 — WHAT THE OVERRIDE DOES AND DOES NOT PROMISE, from Derek's framing of R5 and then measured, because the framing implies a property that is ALMOST true and the gap is the useful part. The framing first, and it supersedes the correctness talk elsewhere in this entry: mixed case is the writer making an explicit choice, and repair defers to that choice instead of judging it. Nothing is being called correct or incorrect — a name kept is a name whose writer said something about it, and a name repaired is one whose writer did not. The property that seems to follow is that asking for repair REGARDLESS should ignore the given casing entirely, so one name repairs to one string however it was written. MEASURED over the 1094 corpus names and it does NOT hold. On the v2 core — `parse(n).capitalized(force=True)` against `parse(n.upper()).capitalized()`, rendering all seven roles — the two differ for 63 names, and the lowercase direction for 18. Naming the surface matters here, and BOTH counts move with the surface, which an earlier draft of this bullet got half right: through `HumanName` and `str()`, which is what tests/test_capitalization.py uses, the counts are 62 and 16 — and the two directions are dropped for DIFFERENT reasons, which an earlier draft of this bullet ran together under the first. UPPER: the facade's default render spec omits the maiden name, so `str()` cannot see `Jane van der Berg née y Jones` (the whole of the upper difference, and it is a conjunction inside the maiden name — `maiden` is `y Jones` forced against `Y Jones` uppercased and every other role is byte-identical). LOWER: the two the facade drops are `John van der J. V` and `abdul V Smith`, and the maiden field is EMPTY in both directions for both, so the omitted role explains neither. What hides them is that `str()` CONCATENATES adjacent roles, so a token that crosses a role BOUNDARY and moves nothing else is invisible in the joined string: `John van der J. V` is family `van der J. V` forced against family `van der J.` plus suffix `V` lowercased, and `abdul V Smith` is given `Abdul V` forced against given `Abdul` plus middle `V` lowercased — same seven roles rendered, same string joined. Re-derived 2026-08-29 on this branch, and named per name so the next reader can check the arithmetic: core 63/18, facade 62/16. Two things about those numbers are the opposite of what one would guess. UPPERCASE IS THE WORSE DIRECTION, not the clean one. And the misses are not merely the pipeline's case-sensitivity leaking in: of the 63, only 25 move a role at all, and the other 38 parse byte-identically and diverge inside the repair (25 and 37 through the facade, the missing name being one of the byte-identical ones). The mechanism is v1's initial carve-out — a conjunction is not lowercased where it is written initial-shaped, and initial-shaped means one CAPITAL letter — so uppercasing turns every one-letter conjunction into an initial (`Velasquez y Garcia, Dr. Juan Q.` forced keeps `y`; the same name uppercased then repaired gives `Y`, with a byte-identical partition either side -- the comma form is the one to cite here, the space-written `Dr. Juan Q. Velasquez y Garcia` being a member of the 25 whose roles DO move), and lowercasing turns a middle initial `E` into the Italian conjunction. 1.4.0 does the same (`JUAN Y GARCIA` capitalizes to `Juan Y Garcia`), so this is inherited, and it is recorded here rather than fixed here. Recompute both directions by running the two forms over the corpus files deduped and diffing. -- 2026-08-29 — and therefore NOT stated in rules.md, which is a deliberate choice rather than an oversight. The document's examples are keyed on input STRINGS, so any statement of the property invites exactly the test that falsifies it — re-case the input, expect the same output — and the counterexamples are already in the corpora. The property is true of the repair given a parse, and rules.md speaks input-to-output; a rule stating it would be over-broad in the one direction a reader would check. What R5's statement says is enough for the promise that IS kept: a mixed-case name is kept unless repair was asked for anyway. That clause was REWORDED for this, and the reword is the whole point rather than a tidy-up. It read `unless repair regardless of how the name is cased was asked for`, which carries two readings -- the intended one, that the request overrides the keeping, and a second one, that the repair disregards the input's casing, which is this property in nearly this bullet's own words. A reader taking the second reading would run the re-casing test predicted above, land on `Velasquez y Garcia, Dr. Juan Q.` (in the corpus today), and conclude the RULE is wrong when only the phrasing was. Nine words, and they asserted the thing the paragraph exists to deny. The property is pinned in tests/test_capitalization.py instead, over names carrying no single-letter word whose class case decides, with `juan y garcia` beside it as the recorded exception. R5's example block gains `"SHIRLEY MACLAINE" → capitalized="Shirley MacLaine"` from this work, and it earns its place on its own ground rather than as half of a convergence pair: it is the only row in the block that fails when the gate is narrowed to lowercase-only, every other row passing that mutation. Measured three ways — gate deleted (passes, so it does not witness the gate's existence), gate narrowed to accept only all-lowercase (FAILS, and alone in the block), Mac/Mc convention deleted (fails, with the other two rows). Until it was added, R5 stated that repair acts on a name written entirely in one case and witnessed only the lowercase half of it. That lowercase half is still `"juan mcdonald"`, which is byte for byte an R4 row as well, and the duplication is deliberate rather than an editing slip: the two rules make different claims about the same line — R4 that the repair honors the Mac/Mc convention, R5 that an all-lowercase name is acted on at all — and dropping it from R5 would leave the gate's lowercase half unwitnessed inside the rule that states the gate. Five other rows already sit under two rules apiece for the same reason (P5/P6 twice, P5/O5, N3/M4, W1/W3). -- 2026-08-29 — DEBT this extraction leaves, named so the next commit inherits an obligation rather than a rediscovery. Pulling the gate out into R5 leaves R4 carrying ONE falsehood and ONE ambiguity — different defects wanting different repairs, and `interacts: R5` carries neither, the field being advisory. FALSE: R4 promises repair "vocabulary exceptions (McDonald) included", but `str(parse('Juan Mcdonald').capitalized())` is `'Juan Mcdonald'` — the gate refuses before any vocabulary is consulted, and only `str(parse('Juan Mcdonald').capitalized(force=True))`, `'Juan McDonald'`, reaches the exception. R4 needs its promise scoped to names the gate admits. AMBIGUOUS, not false: R4's "an already-correct name comes back unchanged" means correct by the repair's own conventions, i.e. idempotence, and under that meaning it is true; a reader hears correct as the bearer writes it, and under THAT meaning `str(parse('bell hooks').capitalized())` — `'Bell Hooks'` — looks like a counterexample. It is not one, because `bell hooks` is not already-correct in R4's sense. What R4 owes is a disambiguation of "correct", NOT a narrowing to spare deliberately single-cased names: that would be new behavior, and R5's own rationale declines it on the ground that single case leaves the repair nothing to read. Also for that commit, and inert as things stand: R4's boundary row `"Juan McDonald" → capitalized="Juan McDonald"` passes with R5's gate deleted, exactly like the R5 row that was withdrawn above; rewriting it to `capitalized_forced=` makes it discriminate for R4's own subject but still witnesses nothing about the already-correct question. This commit adds R5 and touches R4 only on its pointer line, leaving both defects as found rather than half-fixed by a commit whose subject is something else. -- 2026-09-13 #383/#479 — AMENDMENT to the 2026-08-29 override bullet above: the MECHANISM it names is gone, and with it the direction it called surprising. That bullet explained the re-casing gap by v1's initial carve-out — "uppercasing turns every one-letter conjunction into an initial" — and cited `Velasquez y Garcia, Dr. Juan Q.` forced (keeping "y") against the same name uppercased then repaired (giving "Y"). Measured on this branch today, all three spellings agree: forced, uppercased-then-repaired and lowercased-then-repaired each give `Dr. Juan Q. Velasquez y Garcia`, and the space-written `Dr. Juan Q. Velasquez y Garcia` does the same. A one-case name no longer reads a bare capital connective as an initial on shape; the vocabulary decides, and "y" is outside the marked subset (decisions.md#P3, 2026-09-13). Nothing about R5's gate changed — only what repair finds on the other side of it. - THE DIRECTION REVERSED, which is the part worth carrying: uppercase was the worse direction and is now the better one. Recipe, one comparator, both sides — over the deduped `tools/differential/corpus*.jsonl` glob, compare `parse(n).capitalized(force=True)` against `parse(n.upper()).capitalized()` and against `parse(n.lower()).capitalized()`, rendering all seven role fields, and count the names that differ in each direction; run it once on this tree and once against a released wheel installed in a throwaway environment (the tree shadows the wheel otherwise). Measured 2026-09-13 over 1174 names: the 2.3.0 wheel gives 69 upper and 23 lower, this tree gives 11 upper and 18 lower. On the facade surface — `HumanName(n).capitalize(force=True)` read back with `str()`, against the same call on the re-cased spellings — 2.3.0 gives 68 and 20, this tree 10 and 15; the surface still matters for the reasons the 2026-08-29 bullet names (the default render spec omits the maiden name, and `str()` concatenates adjacent roles). The uppercase count collapsed; the lowercase one did not, its remaining members being what the carve-out never explained. The 63/18 and 62/16 figures in the bullet above are a dated snapshot over a 1094-name corpus and are not reproducible today in either direction — do not treat them as this recipe's before-column. - rules.md#R5 needs no matching amendment: it never carried the claim. The 2026-08-29 bullet that follows the override bullet above says so in its own words ("and therefore NOT stated in rules.md"), and R5's Rationale, statement and example lines were re-read on this branch to confirm it. What DID move in the suite is the mirrored pin in tests/test_capitalization.py, where `JUAN Y GARCIA` had been pinned as inherited 1.4.0 behavior ("Juan Y Garcia") and now agrees with its lowercase spelling ("Juan y Garcia"), with a mixed-case control beside it. - -- 2026-09-23 #492 — DECIDED (Derek, 2026-09-23): the gate reads every token but the SUFFIX-roled ones. R5 says mixed case is evidence the writer cased the NAME deliberately, and a suffix written the way a suffix is written — `III`, `PhD`, `Jr.` — says nothing about how the writer cased the name words. So `juan garcia III` repairs to `Juan Garcia III` where every release, 1.4.0 through 2.3.0, returned it untouched, and `juan garcia PhD`, `JUAN GARCIA Jr.` and `dr. juan garcia III` repair the same way. v1's test for it, `test_capitalization_exception_for_already_capitalized_III`, was a known failure since 2012 — added 2012-10-07 (e2df6937) as a plain failing test against the Google Code tracker's issue 22, marked xfail 2026-09-01 (d02dc41e) — and this change retires the marker and the `_KNOWN_FAILURE` suffix the test carried. Every token not roled SUFFIX stays in the gate text, NICKNAME and MAIDEN included: `jane doe nee SMITH III` and `juan garcia (Bob)` are left alone (pinned in `tests/v2/test_render.py::test_the_gate_leaves_the_suffixes_out`, where a mutant filtering those roles too survived the suite otherwise). A name whose non-suffix tokens are caseless is one-case exactly as the whole name was, so only the suffix repairs (`김민준 phd` → `민준 김 PhD`); a name with NO non-suffix token is one the parse never produces (rules.md#H4 — measured, `Jr.` is a title, `PhD` a given name, `MD, PhD` family and suffix), so only a synthetic `ParsedName` reaches the empty text, which is one-case, and every suffix repairs to its own fixpoint. Force is unchanged. WHY "USE FORCE" IS NOT THE ANSWER: `force=True` is lossy on correctly cased names — `Danny DeVito` → `Danny Devito`, `Shia LaBeouf` → `Shia Labeouf`, `Anne-Marie DuPont` → `Anne-Marie Dupont`, `Bill deBlasio` → `Bill Deblasio` (re-measured 2026-09-23) — so a batch caller cannot force the lowercase rows without damaging the cased ones beside them. - -- 2026-09-23 #492 — THE R5 / PARSER SPLIT, decided and recorded rather than left for a reader to discover. The parser's own one-case readings — rules.md#P3's single-letter connective and rules.md#S2's case signal at the suffix slot — still count a cased suffix, as Derek's 2026-09-19 comment on #492 and decisions.md#S2's 2026-09-14 bullet require (the judged token stays in the span: `Jack MA` is mixed because of `MA`); R5's gate does not. So one name can be mixed to the parser and one-case to repair. `john e jones III` is mixed to P3, `III` included, so its `e` reads as a connective and it repairs to `John e Jones III`, while `john e jones iii` is one-case, reads `e` as an initial and repairs to `John E Jones III`; the corpus name `john e jones, III` moves the first way on the default path, where every release left it untouched. The gate follows the parse's suffix call wherever the parse makes one, the ambiguous credential class included, so two shapes outside the corpus move on the default path as well: `jack MA` gives `Jack MA` — S2's lean read `MA` as the credential, and the gate then leaves it out — and `MD, PhD` gives `Md PhD`, an input that is nothing but vocabulary (rules.md#H4's shape) read as family `MD` and suffix `PhD`, where 2.2.0 and 2.3.0 left `MD PhD` untouched; the family `MD` repairs as a name word, by the item-4 decision under R4. All four strings are pinned by `tests/v2/test_render.py::test_the_gate_and_the_parser_read_a_cased_suffix_differently`. - -- 2026-09-23 #492 — TITLES STAY EVIDENCE, and the reason is a limit of title REPAIR rather than a claim about titles. Title repair is still plain `str.capitalize()`, and widening the gate to titles would surface its defects on the default path. Measured under `force=True`, where they already show (re-measured 2026-09-23): `Lt.Gov. juan e garcia` → `Lt.gov. Juan e Garcia`, `The Lord of the Universe` → `the Lord of the Universe`, `Dr. med. univ. Margit Popp` → `Dr. Med. Univ. Margit Popp`. So `Dr. juan garcia` is left alone (rules.md#R5's boundary line). This may move when title repair is trusted. - -- 2026-09-23 #492 — MEASURED, population first. Recipe: for each name in the deduped corpus glob, the gate as shipped before #492 (the joined text of EVERY token) declines it AND the joined text of its non-suffix tokens is one-case; those are exactly the names the new gate admits, and the default path returns their forced repair (since 2026-09-24, less any suffix written in more than one case, which it keeps as written — the bullet below). The movers are the admitted names whose default-path string changes, counted by the R4 MEASURED bullet's comparator. At 4d0680e6, over its 1340 names, the population was 17 and 12 of them changed their default-path string — `juan garcia III`, `dr. ben alex johnson III`, `john e jones, III`, `john smith MA` and eight `pennie von bergen wessels` spellings; the other five (`威廉·莎士比亚, PhD`, `田中さん, PhD`, `田中さん, Ph. D.`, `김민준씨 Jr.`, `1 & 2, 3 4 5, Mr.`) are admitted and repair to themselves. The approved spec said "17 move": 17 is the population, 12 the movers. Over the corpus this change leaves (1375 names, R5's new example lines among them) the population is 22 and the movers 17, the five new ones being those lines (20 and 15 over 1371 names before the 2026-09-24 decision below added `john smith EdD` and `juan garcia Iii`; re-measured 2026-09-24, the parent-corpus 17 and 12 unchanged). No forced repair moves by this rule, as none can: the gate is never consulted under force. - -- 2026-09-24 #492 — DECIDED (Derek, 2026-09-24): where repair was not forced, a SUFFIX token written in more than one case is kept as written; one written in a single case is repaired like any other token, and force repairs both, unchanged. It is the gate's own one-case test asked of each suffix on its own (`_render._in_one_case`, one predicate for both), and it is the 2026-09-23 #492 reasoning read to its end: a suffix is left out of the gate because a cased one says nothing about how the writer cased the NAME, and for the same reason a mixed-case one is the writer's spelling of the credential, which repair has no evidence to overrule. THE REGRESSION THAT SURFACED IT, measured by the branch's code reviewer and re-measured 2026-09-24: once the gate stopped counting the suffix, a one-case name with a conventionally mixed-case listed acronym was admitted and the acronym clause wrote the acronym in capitals — `john smith EdD` → `John Smith EDD`, `JANE DOE, DSc` → `Jane Doe DSC`, `juan garcia PsyD` → `Juan Garcia PSYD` — where the parent 4d0680e6 and every release 1.4.0 through 2.3.0 returned those names untouched. They now give `John Smith EdD`, `Jane Doe DSc` and `Juan Garcia PsyD`. The accepted `B.Tech.` → `B.TECH.` on the default path (decisions.md#R4, 2026-09-23) was the same shape, and its default-path half is superseded (decisions.md#R4, 2026-09-24). THE COST, accepted: "as written" keeps a garbled spelling with the deliberate one, so `juan garcia Iii` gives `Juan Garcia Iii` on the default path (`Juan Garcia III` forced) and `john smith Mba` gives `John Smith Mba`, where the branch before this decision gave `III` and `MBA`; every release returned both names untouched. A rules.md#R5 boundary line pins `Iii`. Why not keep only a spelling the vocabulary does not know how to repair: that would re-derive, per word, a judgment about which mixed spellings are deliberate, which is the evidence R5 says repair cannot read. The masks added the same day (decisions.md#R4) are the other half: they serve the one-case names this rule cannot reach (`john smith psyd` → `PsyD`). No corpus name written before this decision moves by it (measured 2026-09-24 against the branch before it, over this tree's 1375 names: the only movers are the new rules.md lines). Pinned by `tests/v2/test_render.py::test_a_suffix_written_in_more_than_one_case_is_kept_as_written`, core and facade; a mutant dropping the per-token test fails it, `test_the_gate_leaves_the_suffixes_out` and two rules.md#R5 lines, and one dropping its `force` guard fails it and the forced `EdD` line. - -### parse-cost — what a parse is allowed to cost - -Every number below is a py3.11 measurement of 2026-08-31, recomputable with `uv run python tools/perf/call_count.py` (add `--modules`, `--stages`, or `--against `). That harness is in the tree BECAUSE of this entry: its first draft took figures across several sessions with throwaway scripts on whichever interpreter was to hand, and published a module table that spliced one interpreter's before-column onto another's after-column. Four of its eight module numbers were wrong, its four stage numbers were 2x (2000-parse totals labelled per 1000, which its own arithmetic contradicted — four stages summing past the whole parse), and it blamed the wrong PR. Quote nothing here without the interpreter beside it. - -- 2026-08-31 #475 — the benchmark bounds FUNCTION CALLS per parse, not seconds, as a per-interpreter BAND of ±2%. A loose 5s wall-clock backstop stays over BOTH entry points for what frame counts cannot see: a compiled regex that starts backtracking emits no `call` event, and neither does C-level work. (A quadratic inside a comprehension or generator IS visible — `call` fires once per generator resume.) - WHY THE CLOCK HAD TO GO. The 1.0s bound failed four times across #466 and #474, always on CI's 3.12 job, always between 1.01s and 1.08s, with master re-running green each time. Three local methods disagreed with CI and with each other: uninstrumented ~1% branch-over-parent (inside the parent's own spread), coverage-instrumented indistinguishable, and the whole benchmark file under `--cov` made the branch FASTER. The same harness measured `origin/master` at 94-96ms and again at 89-92ms twenty minutes later. A check that cannot separate a 1% change from a busy runner does not fail safe; it fails expensively. - WHY A BAND AND NOT A CEILING, which is the correction the review forced. A ceiling with headroom is ANOTHER threshold that happens to break — the failure this replaced. The first draft set 470 against a 408 baseline, and measured, that catches nothing smaller than +61 calls on 3.11 and +84 on 3.12: it would have missed the very regression it was built for on four of five interpreters. The cycle's +67 arrived across a dozen PRs at roughly five calls each, and no ceiling loose enough to be safe can see five. A ±2% band trips at +5, verified by injection. A DROP trips it too, which is intended: an unexplained fall is as much a signal as a rise. - PER INTERPRETER, because the count is not machine-independent — 410 on 3.11, 388 on 3.12, 406 on 3.13-3.15 for `parse`. PEP 709 inlined the comprehension frames 3.11 counts and 3.13 added others back. The first draft claimed "the same number on any machine" and took its baseline on 3.11 by luck: had it been taken on 3.12, the 3.11 job would have gone red with nothing in the file explaining why. A version with no row fails loudly with its own number rather than passing unguarded. - WHAT THE OLD BOUND HAD CAUGHT, which is the substantive half. Measured `--against v2.1.0`: `parse` 343 → 410 and `HumanName` 380 → 447, **+67 calls per parse**; by stage, per 1000 parses, `group` 13.5 → 22.2ms, `classify` 11.6 → 13.5, `assign` 11.1 → 13.0, `post_rules` 5.2 → 8.0. The bound was not too tight. The parser had grown into it, and the clock could not say so with enough confidence to act on. - WHERE THE CALLS WENT, per parse, v2.1.0 → here: `_group.py` 94 → 84, `_pieces.py` 0 → 51, `_post_rules.py` 16 → 33, `_vocab.py` 26 → 32, `_assign.py` 21 → 18, `_classify.py` 14 → 16. - AND WHICH PR SPENT THEM, which the first draft got backwards. It named #439's predicate extraction as "the largest contributor" and built a trade-off on it. Measured across its own merge boundary, #439 cost **zero**: 404 → 404, `_group` shedding 51 exactly as `_pieces` gained them. A pure relocation, and `mechanisms.md#ONE-PREDICATE-PER-QUESTION` says as much — the sharing between `group` and `assign` predated the module. The real largest single contributor is **#424 at +23** (381 → 404), the PR that made group's chain and walk mirror assign's; then #404 at +12, with #361, #367 and #434 at +8 each. So the entry's old conclusion — that recovering the cost means reintroducing the drift #439 removed — deterred work on a PR that spent nothing while +23 went unexamined. Recovering it, if ever wanted, starts at #424's peel walks. - ONE REPAIR WAS TRIED AND REVERTED: collapsing `post_rules`' three separate role-index scans into one pass. It saves **6 calls per parse** on 3.11 (the first draft said one, which is not even reachable — `_idx` is a one-line list comprehension, so each removed call frees two frames). Reverted because it was measured against a noisy timing harness that showed nothing; at 6 calls it is 9% of the cycle's growth and worth reconsidering against the band, which can now see it. - RAISING OR LOWERING A ROW IS A DECISION, not a maintenance chore. Append here with the interpreter and the harness invocation, as above. - -- 2026-09-26 #546 -- the stages copy their state with `_state.copy_with` instead of `dataclasses.replace`, and every row moves DOWN. `dataclasses.replace` walks `fields()` and calls `__init__` on each copy, three frames on 3.11 and 3.12 and four from 3.13 where `copy_with` is one. One parse of the reference name makes 18 copies: six of the state, one each from tokenize, segment, classify, group, assign and post_rules, and twelve of single tokens, six each from classify and assign (`extract_delimited` returns the state unchanged when there is no delimiter, and `script_segment` returns early on ASCII input). That is the whole of the drop, 18 x 2 = 36 frames on 3.11 and 3.12 and 18 x 3 = 54 from 3.13 (recompute: wrap `copy_with` with a counter in the eight stage modules and parse the reference name). A field copy builds what `replace` builds for a dataclass that is decorated itself rather than inheriting the decoration, keeps the generated `__init__`, and has no `__post_init__` and no `init=False` field. `_copyable_fields` checks exactly those four, and `_COPY_FIELDS` runs it over `WorkToken`, `PendingAmbiguity` and `ParseState` at import, so a class that stops qualifying fails there and `copy_with` copies nothing else. `test_the_guard_refuses_a_class_a_field_copy_would_get_wrong` records, for each refused shape, what `replace` builds and what an unguarded copy would build instead. To mypy, `copy_with` is `from dataclasses import replace as copy_with`, which keeps the dataclass plugin's keyword and type checks at every call site; an assignment (`copy_with = dataclasses.replace`) would not, since the plugin keys on the callee's full name (measured: a misspelled field and a wrong-typed value both pass through the assignment and both fail through the import). Measured 2026-09-26 with `uv run python tools/perf/call_count.py --against e0f1a2f`, each row on its own interpreter, parse/facade: 3.11 406/443 → 370/407, 3.12 384/421 → 348/385, 3.13, 3.14 and 3.15 402/439 → 348/385. The rows drop by 40 and 58 rather than 36 and 54 because e0f1a2f already read 4 under every row, inside the band, and the new rows are set to what the harness reads now. `_LINK_BASELINE`'s 64-link clause reads 2587 → 2301 on 3.11. By stage (`--stages`, py3.11, ms per 1000 parses of the reference name): group 19.6 → 17.3, classify 13.4 → 9.8, assign 11.1 → 8.0, tokenize 8.1 → 6.6, post_rules 7.9 → 6.5, segment 2.6 → 1.7. BEHAVIOR IDENTICAL: the differential gate's report at all five baselines matches e0f1a2f's line for line apart from the path header. -- 2026-09-29 #561 -- SUPERSEDED IN PART: the 2026-09-26 bullet above, whose four conditions ("decorated itself", "keeps the generated `__init__`", no `__post_init__`, no `init=False` field) were not sufficient. `replace` builds through `obj.__class__(...)`, and a field copy skips every hook that call runs; four shapes met all four conditions and still parted: a validating `__new__`, a validating `__setattr__` on a non-frozen dataclass and a validating metaclass `__call__`, each giving `ValueError` from `replace` where an unguarded field copy built `value=-1`, and an `InitVar` with no default, which `replace` demands (a `ValueError` through 3.12, a `TypeError` from 3.13) and a field copy never sees. #561's first tightening (80554266) found only the first three, and by swapping the class's OWN `__dataclass_params__` for an inherited lookup it also let through an `__init__` borrowed from another dataclass under that dataclass's name, which the 2026-09-26 guard had refused; review caught both. So the guard is now a list of conditions a class must MEET (`_state._copy_refusals`, one reason string per unmet condition): decorated as a dataclass itself, frozen, an `__init__` generated for it (compiled from `` and named `.__init__`) that takes exactly its `init` fields, the default metaclass, no `__new__` above `object`, no `__post_init__`, no `init=False` field. It is a TRIPWIRE for a realistic edit to the three pipeline classes and says so, not a proof against any class: a forged `__qualname__` on a borrowed `__init__` in a class that is also decorated itself, or a `__class__` property, still passes, and `copy_with` copies nothing but `WorkToken`, `PendingAmbiguity` and `ParseState`, which meet every condition on 3.11 to 3.15. `_UNGUARDED_EFFECT` records for each row what `replace` builds, what an unguarded copy builds, and which conditions refuse it; `test_every_guard_condition_alone_refuses_a_recorded_shape` asserts that each condition is the only refusal of some row, so dropping one fails a test rather than contradicting this paragraph (checked by removing the `__new__` condition in a scratch copy: two failures). Call counts unchanged, the guard running at import only. -- 2026-10-10 #624 -- every row moves DOWN by 4: `parse` 308 → 304 on 3.11 and 289 → 285 on 3.12-3.15, `facade` 345 → 341 and 326 → 322, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 8e6e524a (#631's merge) and of the change, the import asserted (8e6e524a read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 51 → 49 and `_group.py` 25 → 23 -- group's leading-position scan, a generator asking `is_leading_title` per piece, gave way to `leading_titles`, which it already called, and `chain_lead`, which asks `is_prefix_piece` only of the titles. -- 2026-10-10 #625 -- every row moves UP by 3: `parse` 304 → 307 on 3.11 and 285 → 288 on 3.12-3.15, `facade` 341 → 344 and 322 → 325, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 505d6dec (#632's merge) and of the change, the import asserted (505d6dec read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 49 → 52, `_group.py` 23 → 23. The reference name holds a particle run (`de la`), so it pays the merge's call and one more `merge_pieces` and `joined_tags`, the chain then merging `de la` with `Vega` where it had merged the three words at once; the first cut asked the title run and the lead in the merge and again in the read, +11, and group now asks the title run once and hands both on. Every name of three or more words read at the trailing site pays the call, +1 (`John Q. Smith` 202 → 203 on 3.11, warm); a name with a particle run pays the 3 (`Juan de la Vega` 230 → 233). -- 2026-10-06 #617 -- every row moves DOWN by 10: `parse` 374 → 364 on 3.11 and 353 → 343 on 3.12-3.15, `facade` 411 → 401 and 390 → 380, each measured on its own interpreter with `PYTHONPATH= pythonX.Y tools/perf/call_count.py` against 83f4e914 (every row had sat inside its band before, the 3.12-3.15 ones at +5 over a 348 baseline). `--modules` puts all ten in group and the piece layer (`_group.py` + `_pieces.py` 142 → 132, every other module unchanged): P3's connective joins moved out of `_group_segment` into `_pieces.join_connectives`, which asks `is_conj_piece`, `is_title_piece` and `is_prefix_piece` directly where the loops had asked through one-line closures over them, two frames per test becoming one. The link pin moves with it, 2,678 → 2,479 for the 64-link name on 3.11 (`_LINK_BASELINE`), and there for two reasons, counted per function in #617's review: the closures (`conj` 133 frames, `title` and `prefix` one each) and the single-letter test, which had joined each connective's text through a generator (65 frames, one per link, more than the pin's whole band), less the one `join_connectives` frame. A deliberate move, not drift: the change was a refactor (decisions.md#P3, 2026-10-06). The reference rows hold no comma; a comma part holding a connective now pays the shared loop (`John Smith, Mr. and Mrs.` 262 → 269 on 3.11, warm), and one holding none skips it and costs what it did (`Smith, John` 161), the review having found the first cut charging every family-comma parse two frames per word of the part. THEN 39 MORE on every row, in the branch's /simplify pass: group ran the rootname count (about five frames a piece, its only reader the carve-out) and the shared loop over every segment of three or more pieces, connective or not, and over the corpora and the case table under three orders 0 of the 3,153 group calls with no connective left free changed a piece. Group now skips both where its frozen-set walk, which already reads every piece's `conjunction` tag, finds no connective free, as `_comma._reading_pieces` does: `parse` 364 → 325 on 3.11 and 343 → 304 on 3.12-3.15, `facade` 401 → 362 and 380 → 341, `_group.py` + `_pieces.py` 132 → 93, measured on each interpreter as above; the link pin does not move, every link in it joining. Per name on 3.11, warm: the Vega name 436 → 386, `Smith, John Quincy Adams Bob` 326 → 297, `Smith, John` and `John Smith, Mr. and Mrs.` unchanged at 161 and 269. No output moved over the #617 grids (decisions.md#P3). -- 2026-10-07 -- every row moves DOWN by 8 more: `_group_segment`'s last three one-line predicate closures, `prefix` over `is_prefix_piece`, `suffix` over `is_suffix_piece` and `marker` over `_is_maiden_marker_piece`, are inlined as direct calls, finishing what #617 did for `title`, `conj` and `merge`. Each closure was a frame of its own ahead of the predicate's, and the chain's two inner scans ask `prefix` once per piece they pass, so the cost grows with particle pieces (frame-budget question 1 in AGENTS.md). Measured with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y tools/perf/call_count.py`, each row on its own interpreter, parse/facade against 0c54662b: 3.11 325/362 → 317/354, 3.12, 3.13, 3.14 and 3.15 304/341 → 296/333. `--modules` on 3.11 puts all eight in `_group.py` (34 → 26), every other module unchanged. The whole of the drop is the closures, counted per function on 0c54662b (a profile hook counting `call` events whose code is named `prefix`, `suffix` or `marker` in `_group.py`): the reference name asked `prefix` 7 times and `suffix` once, `Juan` + `de la Vega ` × 8 + `Smith` asked 40 and 9 and reads 937 → 888 on 3.11 (919 → 870 on 3.12-3.15), and the 64-link name asked `prefix` once, so `_LINK_BASELINE` moves 2,479 → 2,478 on 3.11 and gains rows for 3.12-3.15 at 2,459 (2,460 on 0c54662b), each measured on its own interpreter and the first the pin has had beyond 3.11; `marker` is reached only by P5's bound-join decline and the reference name never asks it. BEHAVIOR IDENTICAL: the full suite passes, and the differential gate's report at all five baselines matches 0c54662b's line for line apart from the path header. -- 2026-10-08 #620 -- no row moves: `parse` 308 on 3.11 and 289 on 3.12-3.15, `facade` 345 and 326, measured as the #614 bullet below on 4278693d (#621's merge) and on the change alike. The 3.12-3.15 pin read 290/327 until this date: #614's /simplify took one frame off those interpreters (289/326 on 4278693d, measured 2026-10-08) and the pin stayed, inside its band, so the #614 bullet's 290 is the pin's number, not that tree's. `--modules` on 3.11 unchanged. The merged copy and its per-word exclusion test are gone, and `_chain_units` asks `chain_run_end` per particle unit and `_weighed` per word behind one; the chain itself calls `chain_run_end` with the particle test inline where it had called `is_prefix_piece` per piece, which pays for the rest. decisions.md#S2, this date. -- 2026-10-07 #614 -- every row moves DOWN again, on top of the closure inlining above: `parse` 317 → 308 on 3.11 and 296 → 290 on 3.12-3.15, `facade` 354 → 345 and 333 → 327, each measured on its own interpreter with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y /tools/perf/call_count.py` against adf6da88 (#619's merge), the import asserted. `--modules` on 3.11: `_pieces.py` 59 → 51, `_group.py` 26 → 25, `_assign.py` 9 → 10, every other module unchanged (`_vocab.py` read 37 until #614's /simplify inlined the one roman test the counting view asks per word, 2026-10-07). S2's trailing run is read once between P3's joins and the particle chain, so the chain's stop and its re-ask, the reserve's two views and assign's own reading collapse into one, and the counting view it reads through is one call on every segment that reaches the read, and its unit walk (`_counted_as_one`) runs only on a segment holding a particle past its first piece; the link pin reads 2,478 → 2,482, inside its band, and stays. A segment of fewer than three pieces never reaches the read and costs what it did (`John Smith` 152, `Smith, John` 161, `John Smith, PhD` 206 on 3.11, warm). decisions.md#S2, this date. - -### removed-v1-surface - -- empty_attribute_default: removed in 2.0 (#255; deprecated in 1.4 per the bridge discipline). Origin #44 (2016): a DB-NULL convenience whose first answer — `name.title or None` — became the migration path. The in-band-signaling bug that sealed it (#254): the 2016 `.replace('None','')` scrub could not tell interpolated None from name text, so "Nonez Smith" rendered - "z Smith" — the fix shape, worth keeping as a one-liner, is - SUBSTITUTE BEFORE FORMAT, never scrub after. Two removal cautions shaped the implementation: tombstone-not-deletion (a plain deleted attribute would make assignment silently accepted-and-ignored, the #241 failure family) and readers-not-just-writers (#44's own thread advised asserting against the attribute, so reads existed downstream). The tombstone and pickle-tolerant load die with the shim. (An earlier 3-0-reevaluations bullet said "left untyped, typeable in 3.0" — promoted from a memory that was already three days stale when written; nothing is left to type.) -- bytes input: removed in 2.0 (#245; decode-first DeprecationWarnings shipped 1.3.0 per the bridge discipline); the shipped TypeError carries the decode hint. -- no_vowels: removed in 2.0 (#268, filed 2026-07-07, closed 2026-07-28) — never consulted by any parser version, ASCII-only; the facade carries no replacement because there was nothing to replace. - -### cjk-full-stops — which characters are a period on CJK text (2026-09-10, #322/#323) - -ONE GAP SEEN FROM TWO SIDES, which is why the two issues were answered together. #322 asked WHICH CHARACTERS count as a full stop: the parser knew the ASCII period and nothing else, so `김민준 씨．` — written with the fullwidth stop a Japanese or Chinese IME produces by default — missed the suffix entry that `김민준 씨.` reached, and so did the ideographic `씨。` and its halfwidth `씨｡`. #323 asked WHERE a full stop is read at all: the one site that knew the ASCII period was the lexicon's edge strip, so a stop glued to a name word was invisible to a vocabulary lookup and to nothing else. Either answer alone leaves the other half broken — a fullwidth stop the lookup folds would still have taken its token out of the script reading — so the bundle is one arc with four code commits and this record. - -- **FULL_STOPS names four characters; five sites read them.** `nameparser/_lexicon.py::FULL_STOPS` is `".．。｡"`: the ASCII period, FULLWIDTH FULL STOP U+FF0E, IDEOGRAPHIC FULL STOP U+3002 and HALFWIDTH IDEOGRAPHIC FULL STOP U+FF61. Four sites import the string, so "which characters are a period" is answered once: `_lexicon._normalize` (the lookup AND storage fold), `_vocab._normalized_for_script` (the script-classification fold), and in `_script_segment` the surname site's head match and the honorific peel's tail match. The FIFTH site reads no stop set at all. `_pieces.is_leading_title` — rules.md#H2's opening-abbreviation shape — refuses a period-marked word whose text carries a character of a script with no initials, through `_vocab.in_initialless_script` (renamed from `_in_initialless_script`; the same veto `is_initial` has carried since #320), so `_PERIOD_ABBREV` stays an ASCII-period pattern and the four-character set is not copied there. Not every period test reads this set, and the ones that do not are named so the claim is checkable: the interior-period shapes — the dotted acronym `M.A.` and the split `Ph. D.` (`_vocab`) — and FOUR ASCII-only edge tests on Latin shapes across THREE modules — the initial `J.` (`_vocab`), the bracketed credential `(Mgr.)` and the word-internal apostrophe rule (`_extract`), and the trailing middle-initial carve-out `V.` (`_assign`) — stay ASCII. What that does NOT mean is that the wider stops leave Latin text alone. The TESTS stay ASCII; the vocabulary FOLD does not, so a Latin word wearing a wide stop reaches vocabulary an ASCII-only carve-out would have kept it out of — `V。` folds to `v` and does not end in the ASCII period that `_reads_as_a_trailing_suffix`'s carve-out tests (`text.endswith(".")`, `_assign.py` ~line 479, whose docstring calls the period "the whole carve-out"), so the carve-out never fires for `V。` and it is roman five, suffix, where `V.` stays a middle initial through that same carve-out — not through `is_initial`, which is not the decider despite reading true on both spellings: `is_initial("V")` is True, yet `Smith, John V` (no period at all) still reads suffix `V`, so being initial-shaped never by itself keeps a word out of suffix. `Jr。` needs no carve-out argument and simply reaches its entry. An earlier wording of this passage called `Ｍ．Ｂ．Ａ．` and `V。` "a fullwidth-Latin question this set does not raise" (corrected 2026-09-10): that was right for `V。` alone. `V。` is an edge stop on a bare word, and the fold raises the question at every such edge, moving the parse as above. `Ｍ．Ｂ．Ａ．` does NOT move — every one of its periods is interior, none at an edge for `_lexicon._normalize`'s edge strip to reach, so the fold never touches it at all, and the dotted-acronym reading in `_vocab.suffix_as_written` (`n.replace(".", "")`) is ASCII-only regardless of the fold: `_normalize("Ｍ．Ｂ．Ａ．")` is `ｍ．ｂ．ａ`, which matches no acronym form the ASCII-only test recognizes. MEASURED 2026-09-10: `parse("John Smith Ｍ．Ｂ．Ａ．")` and `parse("John Smith, Ｍ．Ｂ．Ａ．")` are identical on `d37b8ec` and on this tree — `Ｍ．Ｂ．Ａ．` stays exactly as it was. The set raises the question at every EDGE fold, and the four ASCII-only tests are the places it is not raised rather than a boundary on the set. MEASURED 2026-09-10 on this tree: `Andrew Perkins (Mgr。)` reads nickname `Mgr。` where `Andrew Perkins (Mgr.)` reads title `Mgr.`; `Smith, John V。` reads suffix `V。` where `Smith, John V.` reads middle `V.`; and `John Smith, Jr。` reads suffix `Jr。` where `d37b8ec` read given `Jr。`, family `John Smith`. The last two are case rows now (`latin_roman_numeral_with_an_ideographic_full_stop`, `latin_suffix_with_an_ideographic_full_stop`), and `is_suffix_strict("Jr。", Lexicon.default())` and `_normalize("Dr。") == "dr"` pin the fold's Latin reach at unit level. Nothing rewrites token text at any of the five: the stop stays on the token it arrived with, and every parsed part is still an exact slice of the input (rules.md's T Background). -- **NFC composes at LOOKUP, and that is what makes the fold symmetric.** `_normalize` now composes NFC for a non-ASCII word, at match time and at storage both. NFD hangul decomposes to jamo and missed every vocabulary entry, so `NFD("김민준, 씨.")` read title `씨.`, family `김민준` and now reads family `김민준`, suffix `씨.`. MEASURED 2026-09-09, re-measured 2026-09-10: no shipped entry changes under NFC — 0 of the 1735 strings in the twelve set-valued fields of `Lexicon.default()` (the standing pin `tests/v2/test_lexicon.py::test_every_shipped_entry_is_already_nfc` walks those twelve plus both halves of the five `capitalization_exceptions` pairs, 1745 strings, and asserts the same). RECOMPUTE: fold every string member of those fields through `unicodedata.normalize("NFC", w)` and count `!= w`. -- **The reach is not one script, and the Latin part of it was nobody's ask.** 310 of those 1735 entries are non-ASCII and 158 DECOMPOSE under NFD — and they divide by script in a way worth having in front of you, measured 2026-09-10: 117 HANGUL (every hangul syllable has a canonical decomposition, which is #322's own half — `씨`, `님`, `박사님`, `선생님`), 18 LATIN (`señor`, `née`, `attaché`, `fräulein`, `chargé`, `rozená` …), 13 ARABIC (`أبو`, `الأستاذة` …), 6 BENGALI and 4 CYRILLIC (`урождённая`, `й`). So NFD Latin input matches now too, which no issue asked for — measured against `d37b8ec`: `NFD("Señor Juan Garcia")` read given `Señor`, middle `Juan` and now reads title `Señor`, given `Juan`; `NFD("Attaché John Smith")` moves the same way; `NFD("Maria Kowalska née Nowak")` went from middle `Kowalska née`, family `Nowak` to family `Kowalska`, maiden `Nowak`. RECOMPUTE: count the entries `w` of those twelve fields where `unicodedata.normalize("NFD", w) != w`, and key the count by the first letter's Unicode name. The argument outlives the digits and is the SHAPE of the division rather than its size: the fix was reached for on a hangul gap, and every other composable script in the vocabulary rode along with it, so a decomposed export from a macOS-origin system was missing entries in five scripts and not in the one corner the issue named. -- **The stop set is script-agnostic at the lookup fold too, and the same unasked-for Latin reach exists for the three wider stops.** The edge strip in `_lexicon._normalize` takes any of the four `FULL_STOPS` off any word before a vocabulary lookup or a particle match, Latin and Cyrillic word forms included, so a Latin or Cyrillic word wearing the fullwidth, ideographic or halfwidth stop now reaches vocabulary and particle membership that only the ASCII period reached before — the same unasked-for reach the bullet above records for NFD, on a different axis of the fold. MEASURED 2026-09-10, against `d37b8ec`: `Dr。 John Smith` reads title `Dr。` where it read given `Dr。`; `Проф。 Иванов` reads title `Проф。` where it read given `Проф。`; `Beethoven, Ludwig van．` reads family `van． Beethoven` where it read family `Beethoven` alone. RECOMPUTE: parse the three strings on this tree and on `d37b8ec` and read `title` alongside the name fields. -- **NFKC was DECLINED, not deferred, and the proposal it declines is #322's own.** The issue asked for "an NFKC fold for vocabulary lookup only" on the argument that it "would address all three" of its mechanisms — one change with one risk profile instead of three patches. Measured, it addresses two and a half: `unicodedata.normalize("NFKC", "．")` is `"."` and `NFKC("｡")` is `"。"`, but `NFKC("。")` is `"。"` — the ideographic stop is not a compatibility character and no normalization folds it to a period, so `씨。`, one of the three spellings the issue opens with, would still have missed the entry and the character would still have had to be listed. And NFKC at the storage fold would REWRITE stored vocabulary (compatibility folds change ligatures, widths and superscripts), which is the one thing `_normalize`'s `lower()`-not-`casefold()` decision already refused. Listing four characters answers the question NFKC only half answers, and answers it without touching what a caller authored. RECOMPUTE: `unicodedata.normalize("NFKC", c)` for each of the four. -- **#323 turned out broader than it was filed.** The issue named the honorific peel. Making the classification fold read through an edge stop moved three readers of the `None` it used to return, and only one of them was the peel's neighbour test: the SURNAME SITE stepped past the family name onto the given name (`양. 지훈` cut 지훈 in half — given `양.`, middle `지`, family `훈` — and now reads given `지훈`, family `양.`), the ORDER RULE fell back to positional (`양 지훈.` lost family-first and now keeps it; `田中 太郎.` likewise), and the segmenter's neighbour precondition missed a writer-drawn boundary (`山田太郎 田中.` consulted a pluggable segmenter on `山田太郎` as if it stood alone, and is blocked now). The issue weighed two candidate fixes and chose neither: teaching the surname site to consult `is_suffix_strict` beside `effective_script`, or making the classification tolerant of a trailing period, which it called the broader one and expected to interact with #322. The broader one shipped, and the narrow one would have reached the filed name alone — `김.` is no suffix vocabulary, so `김. 민준` sits outside it entirely. -- **The surname-site amendment was found by measurement during execution, not designed.** With the classification fold alone, `김. 민준` read given `김`, middle `.`, family `민준`: the token classified as hangul, the site matched `김` against its own head, and what was left over — the stop — BECAME the remainder. So the site matches on `text.rstrip(FULL_STOPS)` and cuts the core, a head being a prefix, which makes the offset that cuts the core cut the text; the stop rides with the remainder (`김민준.` divides as `김` + `민준.`, `김. 민준` as family `김.` plus given `민준`). `rstrip`, not `strip`: a LEADING stop would break the prefix argument. The reading is recorded in `ebf64db`'s comment at the site. -- **Two readings JOINED the bundle, approved in session on 2026-09-09.** Neither was filed. FIRST, the glued-period honorific (rules.md#W3): a stop on the honorific's own word stood between the listed tail and the token's end, so nothing peeled and `田中さん.` and `김민준씨.` read as titles. The peel now matches the tail on the core and cuts BEFORE it, so both divide where their stop-less spellings do and the stop rides with the honorific. The move was free because W3 had recorded that reading as measured on 2026-09-05 and pinned by nothing, its own words being that neither string is a case row or a corpus line, so no row pinned those two readings; both are case rows now, tolerated. SECOND, H2's opening-abbreviation shape on a CJK word: `田中.` read title and now reads family `田中.`, `田中. 太郎` reads family `田中.`, given `太郎`. Han is what WITNESSES this one, and hangul cannot: hangul segmentation runs first, so `김민준.` never reaches the shape test and divides as `김` + `민준.` by the surname site instead. Latin and Cyrillic are untouched (`Smith. John` still reads title `Smith.`, `Проф. Иванов` title `Проф.`), and the veto is contains-any, so `Kim김. Smith` is refused a title too — deliberately: a word carrying a script with no abbreviations is not wearing an abbreviation's period. -- **Recorded limits, all five measured 2026-09-10 on this tree — and the segmenter one is a limit the review round CLOSED, kept in this bullet because this is where the claim stood. The halfwidth-kana one was CLOSED on 2026-10-03 by #594; see the CLOSED note that ends the halfwidth passage below, and decisions.md#W4.** HALFWIDTH KANA sits outside the script table, so it reads by the LATIN shapes at both period sites and not just at the initial one: `_vocab.is_initial("ﾗ.")` is `True` where `is_initial("ラ.")` is `False`, and H2's opening-abbreviation shape MISROUTES a longer halfwidth opener — `parse("ﾀﾅｶ. John")` gives title `ﾀﾅｶ.` where `parse("タナカ. John")` gives given `タナカ.`. Both because `_policy._SCRIPT_RANGES` excludes the halfwidth kana block U+FF65–FF9F on purpose — legacy bank and CSV data writes it, and the table's comment calls that a separate normalization problem, #272's separator handling touching only the halfwidth DOT U+FF65 and none of the kana behind it — and `_NO_INITIALS`, being a tuple of Script members, silently inherits the exclusion. That is #322's second mechanism, restated here because it is the one the bundle did NOT close. An earlier wording of this bullet said "nothing misroutes today, every kana honorific being multi-character", which had the direction backwards: the misroute is at the SHAPE site rather than the peel, and being multi-character is what carries a halfwidth opener INTO H2's shape rather than what keeps it out. NOTHING PINS EITHER READING — no case row, no corpus line, for the halfwidth block or for `ﾀﾅｶ. John` — so both can move without the suite or the differential saying so. It stays a recorded limit rather than a fix: what to do with halfwidth kana is a normalization question (widen the script table, or fold widths at classification) and not a full-stop one, and the bundle answered the full-stop one. CLOSED 2026-10-03 by #594, which widened the table (decisions.md#W4 carries why not the fold): `ﾀﾅｶ. John` reads given `ﾀﾅｶ.` and `is_initial("ﾗ.")` is `False`, and both readings are pinned now — the case rows `ja_halfwidth_katakana_opener_with_a_period_is_a_name` and `ja_halfwidth_katakana_lone_name_with_a_period_is_not_a_title`, and `test_is_initial_script_repertoire`. RECOMPUTE: parse `ﾀﾅｶ. John` and `タナカ. John` and read `title` beside the name fields; `is_initial` alone reports only the shorter half. A PLUGGABLE SEGMENTER receives the CORE, the token with its trailing stops removed — the same string the vocabulary match reads, the surname site handing it `core` rather than `text` (corrected 2026-09-10, in the review round after this entry was first written: the site handed over the raw token, which this bullet recorded as a limit, and the fix costs nothing because a head is a prefix — every offset reported against the core cuts `text` in the same place, so the stops ride with the last piece and no answer of the segmenter's can make a stop a piece of its own). The shipped `ja` pack's `_wholly_japanese` therefore ACCEPTS `山田太郎.` now, where the raw token failed its repertoire test: measured 2026-09-10 with `namedivider-python` installed (`uv run --extra ja`), `parser_for(locales.JA, segmenter=locales.ja_segmenter())` reads `山田太郎.` as family 山田, given `太郎.` with a SEGMENTATION report scoring 0.44, where `d37b8ec` read title `山田太郎.` and the un-corrected branch read family `山田太郎.`; the already-divided `山田 太郎.` reads family 山田, given `太郎.` on this tree and given 山, middle 田, family `太郎.` at `d37b8ec`. Pinned by `test_a_consulted_segmenter_receives_the_core_without_the_stop`. A LEADING stop keeps the token out of the SURNAME site only, because the classification fold that gates it rstrips as it does: `.김민준` classifies as no script, never becomes a surname site and stays one whole word, reading given `.김민준` exactly as `d37b8ec` reads it (corrected 2026-09-10 in the same round: the fold stripped both edges when this entry was written, which moved the ROLE to family and — with a segmenter configured — offered the token to it raw, an answer of offset 1 dividing it into the stop and the name; the fold now matches the sites, and hiding such a token is the no-split rules.md#W1 already accepts). It does NOT keep the token out of the HONORIFIC PEEL, which consults no script at all — the tail alone is its license (rules.md#W2's Background) — so a leading stop does not defeat it: `.김민준씨` peels to `.김민준` and `씨` exactly as `김민준씨` does, pinned by `test_the_peel_reads_the_trailing_stop_only`. A BRACKETED CJK CREDENTIAL degrades: rules.md#S1's escape in `_extract` calls a clause suffix-shaped when it ends in an ASCII period, so `(김민준.) John Smith` is unwrapped and the H2 veto then leaves `김민준.` as name text the surname site divides — given 김, middle `민준. John`, family Smith, where 2.2.0 and `d37b8ec` read title `김민준.` — pinned as the tolerated `ko_name_with_a_period_in_a_bracketed_credential`. And a caller-authored NFD surname entry is composed at ingest, so raw NFD input then finds no entry and the name goes unsplit; pinned by `test_an_nfd_authored_entry_is_stored_composed`. -- **Pickle compatibility, decided 2026-09-10 (Derek): a release-note line, not a forgiving load.** `Lexicon.__setstate__` re-runs `_normalize` over the loaded state and rejects any entry the fold changes, so a `Lexicon` pickled by 2.1.x or 2.2.x whose caller-added entries the widened fold now touches (a non-ASCII entry written with a wide stop such as `씨。`, or authored in NFD) raises `ValueError: incompatible Lexicon pickle: entries are not normalized (titles: 씨。); this state was not written by this version of nameparser` — a message that is wrong about provenance for exactly these pickles, which a released version did write. Measured 2026-09-10 (recompute: take `Lexicon.default().add(titles={"씨"}).__getstate__()`, add `씨。` to its `titles`, and load it into `Lexicon.__new__(Lexicon)`). The shipped vocabulary is unaffected (zero of 1735 strings change under NFC, and no shipped entry carries a wide stop). Two remedies were weighed: a release-note line with the fix a caller applies (rebuild the `Lexicon` from its source rather than unpickle it), or a load path that accepts state whose only drift is one fold pass away, re-folds it and warns naming the entries. The note was chosen: the load guard was designed to refuse rewritten data (the guarded-raise design #3-0-reevaluations records as right regardless of the in-a-minor friction) rather than become a fourth place caller data is corrected without a word, the population is caller-added CJK entries pickled across a minor, and a rebuild is the documented way to carry a `Lexicon` across versions. The 2.3.0 release note carries the line; `tests/v2/test_lexicon.py::test_unpickling_rejects_unnormalized_entries` is the pin of the refusal itself. -- **CORRECTIONS this entry makes to entries above it.** #indic-honorifics' "Abbreviation marks and periods" bullet said `_normalize` "lowercases and strips edge whitespace and ASCII periods only — no NFC, no NFKC, no casefold anywhere in that module"; it now composes NFC for a non-ASCII word and strips all four stops. The VISARGA ARGUMENT still holds and holds for the same reason: ঃ U+0983 has no canonical decomposition, so NFC leaves it exactly where it was — recompute with that bullet's own one-liner, `Lexicon.default().add(titles={"মোঃ"}).titles & {"মোঃ"}`, still non-empty. #W1's 2026-07-29 §1a bullet said "segmentation MATCHING stays raw" and called the classification fold "the one deliberate exception"; matching is now folded too, so what stays raw is SEGMENTATION — the surname site's membership test and the peel's tail slice, which index the token's own text — and the folds are two, not one. #cjk-comma-demotion's 2026-09-05 five-parse bullet is amended in place: `田中さん 太郎.` no longer flips the order (the period is invisible to the classification, so it reads family `田中さん`, given `太郎.` as its no-period twin does), and `田中さん.` and `김민준씨.` no longer read as titles, so that bullet's summary — that the ONLY one of the five where a period changes nothing is the one where it sits on a word the split-off steps past — is superseded: the period changes nothing in four of the five now, and `田中さん 様.` is joined rather than alone. #initials-repertoire's principle bullet restated the veto phonologically ("morphemes or syllables"), which `_policy._NO_INITIALS`' own comment forbids in as many words — Devanagari is an abugida and Arabic an abjad, neither has letters in that sense and both abbreviate — so the bullet now states the CLDR criterion the constant actually uses. -- **What moved, measured 2026-09-10 and re-measured the same day after the review round.** SEVENTEEN names in `tools/differential/corpus*.jsonl` read differently against `d37b8ec`, and all seventeen are the bundle's own new rows, every one of them on the RADAR tier in `corpus_cjk_tolerated.jsonl`. Twelve are the CJK movers this bundle's landing commits wrote; the thirteenth, the wholly-katakana `マイケル.`, joined in the whole-branch review that produced this entry, reading given `マイケル.` where `d37b8ec` reads title `マイケル.` — the katakana arm of the same H2 veto, pinned as `ja_katakana_lone_name_with_a_period_is_not_a_title` in `tests/v2/cases.py`. The last FOUR joined in the review round after that, as pins on readings nothing held: three stop-bearing FAMILY_COMMA spellings (`김민준씨., J.씨`, `田中さん., V.`, `이, J.씨.`) and the bracketed `(김민준.) John Smith`, the recorded degradation in the limits bullet above. A leading-stop shape moves the same way but sits in no corpus at all: `.김민준씨.` — a stop no script ever writes, glued before a honorific-bearing word — read one whole given `.김민준씨.` at `d37b8ec` and now reads given `.김민준`, suffix `씨.`, pinned only by the stage tests for its two edges taken separately (`test_the_peel_reads_the_trailing_stop_only` for the leading stop, `test_peels_a_listed_tail_through_a_trailing_full_stop` for the trailing one) rather than by any corpus row or a combined case of its own. The three comma rows agree with their stop-less twins in every field once the riding stop is removed from the suffix (`씨., J.씨` vs `씨, J.씨`; `さん.` vs `さん`; `씨.` vs `씨`) — each differs from its twin in exactly that one field, by exactly the stop that rides — which is the peel's own "one name, two spellings" argument holding as far as a raw field comparison can show it; the bracketed row does not agree with anything and is not meant to. `田中. 太郎` stays there with the rest, though it is what WITNESSES rules.md#H2's Accepted clause added here: `tools/differential/compare.py`'s tier comment records the 2026-09-05 precedent for exactly this collision — where a demoted file holds a text a rules.md example line names, the EXAMPLE LINE moves into the tolerated rule and the row is not promoted, because marking the row alone leaves the name enforced and documented as demoted. So the clause's CJK reading is carried in W3's tolerated example block, H2's own block keeps the Latin control `Smith. John`, and all four #323 shape rows stay `tolerated`. THE POPULATION THAT COULD HAVE MOVED AND DID NOT is two different sets depending on which criterion is read, and both are worth having in front of you. Under the SHAPE this bundle is about — an edge full stop glued to a word carrying a classified character — the corpora hold 22 names: the seventeen movers and FIVE non-movers (`田中さん 様.`, `田中さん, 様.`, `김민준 씨.`, `김민준 양.`, `김민준, 씨.`), all five already in `corpus_cjk_tolerated.jsonl` before this bundle. Two of the twenty-two need the detector to be written carefully, which is why the recipe is spelled out below: the stop on `김민준씨., J.씨` and `田中さん., V.` sits before a COMMA, so a detector splitting on whitespace alone sees `김민준씨.,` and finds no edge stop (an 18 recorded earlier in the day came from such a detector, over a corpus four names smaller), and the stop on `(김민준.)` is inside a bracketed clause until rules.md#S1's escape unwraps it. Under the looser reading of an ASCII PERIOD ANYWHERE in a CJK-bearing name, the corpora held 20 distinct names on 21 rows before this bundle (19 rows in the tolerated file and 2 in `corpus_issues.jsonl`, `Dr 김민준씨, Jr.` being the one name on two of them) — unmoved by the katakana row, which is a name this bundle's own work wrote, not a pre-bundle count. The looser set is wider because it sweeps in periods sitting on LATIN tokens inside a CJK name — `毛 泽东 Dr.` and `田中さん, V.` — which is precisely what the veto is scoped not to touch, it reading the word and not the name. NEITHER the five nor the twenty moves — the seventeen movers are the whole of what did, and every one of them is a row this bundle wrote. An earlier wording of this bullet said "the twenty-one CJK names carrying an ASCII period": that counted rows as names and quoted the looser criterion beside the shape's argument. RECOMPUTE: parse the union of the corpus files on both trees and diff the seven fields and the ambiguity kinds; for the SHAPE population, take each name's tokens (`_pipeline._tokenize`, not a whitespace split) and ask whether any token's `strip(FULL_STOPS)` is shorter and still carries a classified character, then add the bracketed clause by hand; for the looser one, count distinct names and rows separately. The gate exits 0 at all four baselines with `radar unclassified: 0`; one ledger rule carries them, `fix(#322/#323)`, ELEVEN members at 1.4.0 and 2.0.0 and SEVENTEEN at 2.1.0 and 2.2.0 — the three hangul names the older ledgers hand to their native-script CJK rule are one part of the difference, 2.1.0 being the release that shipped that behavior, and the three FAMILY_COMMA rows are the other: at 2.0.0 none of the three diffs at all, and at 1.4.0 the two that do (`田中さん., V.`, `이, J.씨.`) are already claimed by the broad `fix(cjk-comma-compound)` and `fix(cjk-comma-honorific-peel)` rules that ledger carries, so adding them to this rule would only take a name off a rule that describes it — the same narrow-first reasoning that keeps the three hangul names out. -- **The invariant the rules imply, run over two inputs (AGENTS.md axis 11), measured 2026-09-10.** If an edge full stop is punctuation on a CJK word, then a CJK name written with one must divide exactly as the same name written without one. Written as a check over the corpora — for every corpus name carrying a classified character and an edge stop on some word, parse it, parse the spelling with every word's edge stops removed, and compare the seven fields with the stops removed from the FIRST parse's values too — 30 of the 34 such names agree (re-measured 2026-09-10 after the review round's four new rows; it read 28 of 30 before them, the katakana row マイケル. having joined the population and agreed). FOUR do not, and only three of those are about the parser. Two are the standing boundary, and neither is a CJK word: `毛 泽东 Dr.` (the period makes `Dr.` title vocabulary the trailing walk claims, where bare `Dr` is a name word) and `田中さん V.` (`V.` is a dotted numeral where bare `V` is the generational suffix) — both LATIN words wearing a Latin period inside a CJK name, which is exactly what the veto is scoped not to touch, it reading the word and not the name. The third is new and is the bracketed degradation the limits bullet records: `(김민준.) John Smith` reads given 김 / middle `민준. John` / family Smith where `(김민준) John Smith` reads nickname 김민준 — the period is what makes rules.md#S1 call the clause a credential rather than a nickname, so the two spellings are genuinely different inputs to `_extract` and the invariant should not hold across them. The FOURTH is the comparison's own artifact and not a finding: `이, J.씨.` and `이, J.씨` divide identically (given `J.`, family 이, suffix `씨.` and `씨`), but stripping the first parse's VALUES takes the period off the initial `J.`, where in the bare input that period is interior and survives. A normalization that runs over field values cannot tell a word-final stop in a value from a word-interior one in the name; that is worth knowing before reading the next round's count. RECOMPUTE with that description; it consults no rule statement, which is the point of the axis. -- **Parse cost, measured 2026-09-10 on py3.11** (`tools/perf/call_count.py --against d37b8ec`, and a script counting frames the way its `calls_for` does for the other names). The reference name never reaches the veto — `Dr.` is listed, so the title piece is taken before the shape test — and `parse=409.00 facade=446.00` is unchanged at both trees. On an H2-SHAPED OPENER the veto costs three to five frames: facade 304 → 308 for `Xyz. John Smith`, 385 → 390 for `Xyz. van Berg MA`, 231 → 232 for `Smith. John` (parse-only: 267 → 271, 348 → 353, 194 → 195). The number to quote is these and not "zero"; decisions.md#parse-cost's budget is about the reference name and this is off it. The largest movement measured is not the veto at all: `김민준씨.` goes facade 209 → 397, the peel now firing where the parse used to bail to a title reading, which is a whole segmentation path running rather than an added test. - -### script-table-placement — a recorded reversal - -- 2026-07-29 (script-ranges relocation) — REVERSES #271's documented choice ("tables deliberately live in _vocab"): the tables moved to _policy because packs cannot import the pipeline, evidenced by two hand copies with sync tests and a blocked 16–30× compiled-regex optimization (a module-level re.Pattern in a pack would flip its classification, hence the closure-held-pattern convention). Rider: U+3006 〆 joined the HAN span on a justification deliberately beyond UAX #24 — it is Script=Common but appears solely in Japanese surnames (〆木, 〆谷, 〆野). - -### review-agent-single-source — the review axes live in one file - -- 2026-08-31 (#473) — `.claude/agents/design-docs-reviewer.md` restated AGENTS.md's review axes and, being untracked, drifted three ways against the file it implements: axis 10 never arrived, axis 4's outward half never arrived, and and the corpus-population caution kept a wording AGENTS.md itself records as corrected. That third one carried a second defect neither copy could show: BOTH quoted a corpus measurement that decisions.md#P6 had already superseded, so the endorsed wording was stale too. AGENTS.md's caution is rewritten in this change to carry the lesson without a standing count. The drift was one-directional — every correction landed in the tracked, reviewed, diffable file and none reached the copy. `.claude/agents/` and `.claude/commands/` are now tracked, and the agent points at AGENTS.md rather than restating it. The general form: when a duplicate carries no independent content, delete it; a duplicate is only worth keeping when it says something its source does not. - -Declined: a test pinning the agent's axis headings against AGENTS.md's list, which is what #473 proposed. Heading comparison would have caught the missing axis 10 and NEITHER of the other two drifts — both changed body prose and left headings intact — so the test detects a third of the defect while licensing the arrangement that produced all of it. Same shape as the `STABLE_TAGS` twin flagged in `nameparser/_types.py`, and the same remedy applies there if it is ever revisited. - -### two-input-invariants — checking a rule against something that is not the code - -- 2026-08-31 (#473; earned by #461 and #466) — AGENTS.md axis 11. Axes 1–10 compare a document to another document, to its own examples, or to the implementation, so all ten can only find code that disagrees with a rule; none can find a rule that is wrong. #461 was backed out and #466 closed unmerged for exactly that reason, a month apart, each after a review round that found real defects and missed the premise beneath them. An invariant relating two DIFFERENT inputs consults no rule statement and is not blind that way. - -- The worked invariant: for a name written `Family Given Particle`, the family-comma writing under the default order and the comma-less writing under FAMILY_FIRST must parse alike. Measured 2026-08-31 on 9f1d46e, comparing the non-empty role fields over `nameparser.config.particles.PARTICLES` × six families × six givens: it holds on all 36 pairs for 67 of the 70 particles, and fails on all 36 for `do`, `mc` and `vd` — precisely the words that are also suffix vocabulary, which rules.md#P6's Accepted consequence already names as the contested memberships. The invariant found that set from behavior alone, having read no rule. - -- Recompute: for each P in PARTICLES compare the non-empty role fields of `Parser().parse(f"{F}, {G} {P}")` against `Parser(policy=Policy(name_order=FAMILY_FIRST)).parse(f"{F} {G} {P}")` across any spread of families and givens that are not THEMSELVES particle vocabulary. That restriction is load-bearing, not hygiene: with F or G drawn from PARTICLES the correspondence fails for every particle, because the spread word joins the particle run and the invariant is then measuring a different shape (`Doe, Van de` gives given "Van" middle "de"; `Doe Van de` under FAMILY_FIRST gives given "Van de"). Measured, that is 0 of 70 rather than 67 of 70, so a reviewer who picks a particle word as a family — a natural choice when probing particle behavior — would report P6 broken. The argument survives the digits moving: what carries it is that the failures partition onto the particle-and-suffix words, not how many pairs there are. - -Declined: #473's "0 of 216 → 216 of 216" framing — but not re-argued here, because decisions.md#P6 already records that number's provenance (a script slicing `sorted(particles)[:14]`, which excluded exactly do, mc and vd), already names it axis 2 in its purest form, and already closes "Recompute over the WHOLE vocabulary, never a sample." Cite that entry for why the figure cannot be trusted; a second account of it in this file is the duplication decisions.md#review-agent-single-source was written to stop. #473's companion "72 of 216 — the ambiguous-particle half" measures a DIFFERENT partition of the vocabulary from the one above and must not be read as agreeing with it: the ambiguous particles are the 37 that are not never-given, and decisions.md#P6 records that #466 failed because a never-given test at the new site would have excluded all 37, `van` among them — not because of the three particle-and-suffix words. Two partitions, two different failures. - -Resolved: #469's invariant landed as `tests/v2/test_order_correspondence.py` — a generated parametrized sweep over a small deliberately-forked vocabulary (never-given/ambiguous/multi-token particles, an ASCII and a non-ASCII family, doubled middles), plus a second, non-parametrized test sweeping the entire shipped particle vocabulary (`nameparser.config.particles.PARTICLES`) in the minimal pair and asserting the disagreeing set is exactly `{do, mc, vd}` — the negative control this entry's Recompute bullet describes, now executable rather than a recipe to rerun by hand. - -### 3-0-reevaluations — decisions shaped by the v1 shim - -Promoted 2026-08-15 from session memory (Derek's 2026-07-30 ask; promotion approved 2026-08-15). Discipline when appending: mark each entry (A) "would decide differently without the shim" — real 3.0 work — or (B) "cited 1.4 parity but stands on its own", recorded so nobody re-litigates it. Append here whenever a design choice cites 1.4 parity or the shim as a load-bearing reason. Standing discipline for the removals themselves: every removal warns in a RELEASED version first — the rule established by the 1.3.0 eq/hash work (#223/#224), the reason the v1.4 milestone existed, and the reason FACADE-CONTRACT's "warning-free on 1.4" anchor works at all. 3.0's shim removals follow the same bridge. The CONVERSE rule, declined-with-reasoning twice in the migration design (2026-07-11): a bridge warning must WAIT when the replacement does not yet exist — 1.4 warned on neither shared-CONSTANTS mutation nor the subclass hooks, because users had no actionable response until 2.0, and #262's contract docs shipped in the same release (warn-while-documenting is a mixed message). - -- (A) v1 field vocabulary at the facade boundary: CJK semantics squeeze into first/last through HumanName while the core speaks given/family. 3.0 could drop the aliasing entirely. -- (A) Render's v1-inherited string_format defaults, including space-joined output for interpunct transcriptions (#298: str(HumanName("威廉·莎士比亚")) == "威廉 莎士比亚"; users reinstate the dot via custom format). 3.0 render defaults are a clean slate. -- (A) The differential harness baseline is 1.4-on-PyPI: post-shim, the migration promise it verifies dissolves; the successor baseline is presumably last-2.x. Machinery survives; corpus contracts change. -- (A) A .pyi stub for Policy (wide __init__, narrow attributes) — deferred from #334, see the differential-ledger Declined entry. -- (A) nameparser.config removal scope: the 3.0 schedule says - "nameparser.config in its entirety" while actually enumerating - only the five shim exports; whether the DATA modules keep the package as their home or move under the core is open, and Lexicon's public field docs cross-reference nameparser.config.particles et al. (see the maintainer note in nameparser/config/__init__.py). -- (A) `force_mixed_case_capitalization` as a second route into R5's override (2026-08-29, recorded while extracting R5). The core takes `force` per call and nothing else; the attribute reaches the same behavior only because the facade resolves it when no per-call value is given, which is v1 parity and nothing more. Measured: with it True, a bare `.capitalize()` on `HumanName('Shirley Maclaine')` leaves it `'Shirley MacLaine'` (mutator: read the result back with `str()`). Without the shim there is one route, and R5's statement could say so plainly instead of being written to cover both. Recorded as a removal CANDIDATE — nothing is decided here, and the bridge discipline above applies whenever it is. -- (B) Pickle-guard layout breaks landing in minors (2.1's __setstate__ breaks): the guarded-raise design is right regardless; only the in-a-minor friction is shim-era. -- (B) Positional-read-when-unlicensed as the safe direction (script_orders fallbacks, #298 dot-suppression granularity): coincides with 1.4 parity but stands alone — family-first is the marked case needing affirmative evidence. -- (B) The deprecation-bridge shape (#293/#354) is the template for every remaining 2.x→3.0 shim (the bridge DISCIPLINE — warn in a released version first — predates it, from #223/#224; this entry is the module-__getattr__ mechanics): PEP 562 module __getattr__ PLUS __all__ (star imports never reach __getattr__ — measured: `from ...prefixes import *` bound nothing and leaked the helper); warn per read-location rather than per process (a write-back let a vendored dependency's first read consume the only warning); and the TYPE_CHECKING split, because a module __getattr__ silently disables mypy's attr-defined checking for the whole module (measured on a py.typed package). -- (A) Parking lot from the 2026-07-06 config-model discussion, 3.0-shaped by design: scoped config override via contextvars (`with nameparser.config.use(c):`, the decimal.localcontext pattern — fixes test pollution and config wars while keeping the shared default), and a requests-Session-style Parser(constants) API (rejected then as a full rewrite; the shared CONSTANTS survives 2.x because it is the only config channel that reaches parses inside code users don't own — #262's contract framing). -- (B) The 1.3.0-era legacy-pickle property-key skip (suffixes_prefixes_titles in old blobs) rides in the shim's __setstate__ and dies with it. -- (B) The FAMILY_COMMA doctrine (rule W3): inherited from v1's lastname-comma but correct on its own terms — an explicit comma is stronger evidence than script. -- (A) 2026-08-17 — PARTICLE VOCABULARY AS LOCALE PACKS. Raised while reversing the order-independence decision above: every argument in the leading-particle thread is a language judgement wearing a position heuristic. `de` and `do` lead surnames in Vietnamese, `von` is German, `dos` Portuguese, `das` both Portuguese and a borne Bengali surname — and the per-word comments in nameparser/config/particles.py cite exactly these traditions as their justification. Locale packs already exist (ru, tr_az, zh, ja) and already carry vocabulary and rotations, so particles are the obvious next tenant: a caller who knows the tradition could opt into a tuned set instead of the parser splitting the difference globally. Not 2.x work — it needs the packs to carry vocabulary OVERRIDES rather than additions, and it would give C-i's positional qualifier a per-locale answer rather than one global one. -- (A) 2026-08-18 — LOCALE CONTROL OF P3's SINGLE-LETTER CARVE-OUT, and the sharper requirement it puts on the entry above: a pack would have to carry per-word POLICY, not just membership. Which single letters a tradition wants joined differs by language — Spanish writes "y" and would want "e" read as an initial, Portuguese wants the reverse, Dutch would want every single letter read as an initial. One global rule serves all three today: a single ALPHABETIC character in a name of fewer than four pieces stays a name word, and a bare Latin capital never joins (#383). It is v1's Google Code issue 11 fix ("john e smith") generalized to a vocabulary v1 did not have — the set now holds six single-letter conjunctions (y, e, и, і, й, و), and the carve-out reaches all of them because it counts characters, while the capital veto reaches only the Latin ones because it tests a Latin shape. That asymmetry is the visible seam and it is why #383 is open. A pack that could only ADD or REPLACE vocabulary could not express any of it; the join threshold and the initial veto would have to be per-word, per-pack values. Measured while documenting P3: "&" is single-character but not a letter, so it joins at any length ("Juan & Garcia" is one part while "Juan y Garcia" is three). -- (A) 2026-09-13 — SHOULD A LOCALE PACK BE ABLE TO REMOVE BASE VOCABULARY? `Locale.lexicon` is UNIONED onto the base and never removes — the field's own comment says so — which makes `conjunctions_ambiguous` (decisions.md#P3, 2026-09-13) expressible by a pack in ONE DIRECTION ONLY: an `nl` pack can add "y", and a `pt` pack cannot remove "e", so a Portuguese caller has to write `Parser(lexicon=Lexicon.default().remove(conjunctions_ambiguous={"e"}))` by hand. Accepted for 2.x and parked here rather than filed, because 3.0 is when a wider set of locale packs gets designed and this is part of that work (Derek, 2026-09-13). It is the 2026-08-17 PARTICLE VOCABULARY AS LOCALE PACKS bullet's "overrides rather than additions" requirement meeting a second tenant, which is what makes it a shape question rather than one pack's inconvenience. -- (A) 2026-09-13 — THE DIFFERENTIAL HARNESS CANNOT SEE A LOCALE PACK'S EFFECT AT ALL. `compare.py` builds one `Parser` per name ORDER and nothing else, and a `Case.locale` row projects into the corpora as a bare name string — no corpus row carries a locale key, verified over every `tools/differential/corpus*.jsonl` file on 2026-09-13 — so every corpus name is parsed under the DEFAULT `Lexicon` on both sides. A pack's effect on a name can therefore never diff, at any baseline. That is fine while packs only ADD vocabulary the default already carries most of, and stops being fine the moment the bullet above is answered yes. The shape of the fix is a `locale` carried per corpus ENTRY the way `shape` already carries an order, resolved at the baselines that ship the pack and skipped below them. Parked with the bullet above rather than filed, for the same reason: the two are one piece of work (Derek, 2026-09-13). -- (A) 2026-09-20 #397 — A SECOND INSTANCE OF THE SAME PER-LOCALE QUESTION, cross-referenced rather than parked again. Catalan/Polish `i` joins `conjunctions_ambiguous` beside `e`, so the marked subset now carries two letters and the 2026-08-18 bullet above — which counted six single-letter conjunctions and named the per-word-policy requirement — describes a set of seven, `i` included. Nothing in that bullet's argument moves: the subset IS the per-word policy the bullet asked for, made configurable without packs, and the pack-side half is still what the 2026-09-13 (A) bullet above asks, since a `ca` or `pl` caller still has to write `Lexicon.default().remove(conjunctions_ambiguous={"i"})` by hand. See `decisions.md#P3`'s 2026-09-20 bullet. -- (B) 2026-09-23 #459 — THE SHIM RAISES on a v1 config it once accepted: a `capitalization_exceptions` value that does not spell its key's letters and digits raises `ValueError` at the first parse (decisions.md#R4, 2026-09-23). Cited against the shim's never-raise rule and taken anyway, so it is recorded here: the raise stands on its own — a case mask has no reading for a value that re-spells its word, and the core `Lexicon` raises the same way with no shim involved — and only where it LANDS for a v1 caller, at the first parse rather than at construction, is shim-era. 3.0 has nothing left to decide. -- (B) 2026-10-02 #541 — THE SHIM DROPS, with a warning, every v1 `Constants` entry v1 could never match (decisions.md#R4, 2026-10-02). Cited from 1.4 parity, and it dies with the shim: the core `Lexicon` normalizes edge whitespace away and has no v1 reading to reproduce, so 3.0 has nothing left to decide. diff --git a/docs/design/decisions/3-0-reevaluations.md b/docs/design/decisions/3-0-reevaluations.md new file mode 100644 index 00000000..88385573 --- /dev/null +++ b/docs/design/decisions/3-0-reevaluations.md @@ -0,0 +1,25 @@ +# 3-0-reevaluations — decisions shaped by the v1 shim + +Promoted 2026-08-15 from session memory (Derek's 2026-07-30 ask; promotion approved 2026-08-15). Discipline when appending: mark each entry (A) "would decide differently without the shim" — real 3.0 work — or (B) "cited 1.4 parity but stands on its own", recorded so nobody re-litigates it. Append here whenever a design choice cites 1.4 parity or the shim as a load-bearing reason. Standing discipline for the removals themselves: every removal warns in a RELEASED version first — the rule established by the 1.3.0 eq/hash work (#223/#224), the reason the v1.4 milestone existed, and the reason FACADE-CONTRACT's "warning-free on 1.4" anchor works at all. 3.0's shim removals follow the same bridge. The CONVERSE rule, declined-with-reasoning twice in the migration design (2026-07-11): a bridge warning must WAIT when the replacement does not yet exist — 1.4 warned on neither shared-CONSTANTS mutation nor the subclass hooks, because users had no actionable response until 2.0, and #262's contract docs shipped in the same release (warn-while-documenting is a mixed message). + +- (A) v1 field vocabulary at the facade boundary: CJK semantics squeeze into first/last through HumanName while the core speaks given/family. 3.0 could drop the aliasing entirely. +- (A) Render's v1-inherited string_format defaults, including space-joined output for interpunct transcriptions (#298: str(HumanName("威廉·莎士比亚")) == "威廉 莎士比亚"; users reinstate the dot via custom format). 3.0 render defaults are a clean slate. +- (A) The differential harness baseline is 1.4-on-PyPI: post-shim, the migration promise it verifies dissolves; the successor baseline is presumably last-2.x. Machinery survives; corpus contracts change. +- (A) A .pyi stub for Policy (wide __init__, narrow attributes) — deferred from #334, see the differential-ledger Declined entry. +- (A) nameparser.config removal scope: the 3.0 schedule says + "nameparser.config in its entirety" while actually enumerating + only the five shim exports; whether the DATA modules keep the package as their home or move under the core is open, and Lexicon's public field docs cross-reference nameparser.config.particles et al. (see the maintainer note in nameparser/config/__init__.py). +- (A) `force_mixed_case_capitalization` as a second route into R5's override (2026-08-29, recorded while extracting R5). The core takes `force` per call and nothing else; the attribute reaches the same behavior only because the facade resolves it when no per-call value is given, which is v1 parity and nothing more. Measured: with it True, a bare `.capitalize()` on `HumanName('Shirley Maclaine')` leaves it `'Shirley MacLaine'` (mutator: read the result back with `str()`). Without the shim there is one route, and R5's statement could say so plainly instead of being written to cover both. Recorded as a removal CANDIDATE — nothing is decided here, and the bridge discipline above applies whenever it is. +- (B) Pickle-guard layout breaks landing in minors (2.1's __setstate__ breaks): the guarded-raise design is right regardless; only the in-a-minor friction is shim-era. +- (B) Positional-read-when-unlicensed as the safe direction (script_orders fallbacks, #298 dot-suppression granularity): coincides with 1.4 parity but stands alone — family-first is the marked case needing affirmative evidence. +- (B) The deprecation-bridge shape (#293/#354) is the template for every remaining 2.x→3.0 shim (the bridge DISCIPLINE — warn in a released version first — predates it, from #223/#224; this entry is the module-__getattr__ mechanics): PEP 562 module __getattr__ PLUS __all__ (star imports never reach __getattr__ — measured: `from ...prefixes import *` bound nothing and leaked the helper); warn per read-location rather than per process (a write-back let a vendored dependency's first read consume the only warning); and the TYPE_CHECKING split, because a module __getattr__ silently disables mypy's attr-defined checking for the whole module (measured on a py.typed package). +- (A) Parking lot from the 2026-07-06 config-model discussion, 3.0-shaped by design: scoped config override via contextvars (`with nameparser.config.use(c):`, the decimal.localcontext pattern — fixes test pollution and config wars while keeping the shared default), and a requests-Session-style Parser(constants) API (rejected then as a full rewrite; the shared CONSTANTS survives 2.x because it is the only config channel that reaches parses inside code users don't own — #262's contract framing). +- (B) The 1.3.0-era legacy-pickle property-key skip (suffixes_prefixes_titles in old blobs) rides in the shim's __setstate__ and dies with it. +- (B) The FAMILY_COMMA doctrine (rule W3): inherited from v1's lastname-comma but correct on its own terms — an explicit comma is stronger evidence than script. +- (A) 2026-08-17 — PARTICLE VOCABULARY AS LOCALE PACKS. Raised while reversing the order-independence decision above: every argument in the leading-particle thread is a language judgement wearing a position heuristic. `de` and `do` lead surnames in Vietnamese, `von` is German, `dos` Portuguese, `das` both Portuguese and a borne Bengali surname — and the per-word comments in nameparser/config/particles.py cite exactly these traditions as their justification. Locale packs already exist (ru, tr_az, zh, ja) and already carry vocabulary and rotations, so particles are the obvious next tenant: a caller who knows the tradition could opt into a tuned set instead of the parser splitting the difference globally. Not 2.x work — it needs the packs to carry vocabulary OVERRIDES rather than additions, and it would give C-i's positional qualifier a per-locale answer rather than one global one. +- (A) 2026-08-18 — LOCALE CONTROL OF P3's SINGLE-LETTER CARVE-OUT, and the sharper requirement it puts on the entry above: a pack would have to carry per-word POLICY, not just membership. Which single letters a tradition wants joined differs by language — Spanish writes "y" and would want "e" read as an initial, Portuguese wants the reverse, Dutch would want every single letter read as an initial. One global rule serves all three today: a single ALPHABETIC character in a name of fewer than four pieces stays a name word, and a bare Latin capital never joins (#383). It is v1's Google Code issue 11 fix ("john e smith") generalized to a vocabulary v1 did not have — the set now holds six single-letter conjunctions (y, e, и, і, й, و), and the carve-out reaches all of them because it counts characters, while the capital veto reaches only the Latin ones because it tests a Latin shape. That asymmetry is the visible seam and it is why #383 is open. A pack that could only ADD or REPLACE vocabulary could not express any of it; the join threshold and the initial veto would have to be per-word, per-pack values. Measured while documenting P3: "&" is single-character but not a letter, so it joins at any length ("Juan & Garcia" is one part while "Juan y Garcia" is three). +- (A) 2026-09-13 — SHOULD A LOCALE PACK BE ABLE TO REMOVE BASE VOCABULARY? `Locale.lexicon` is UNIONED onto the base and never removes — the field's own comment says so — which makes `conjunctions_ambiguous` (decisions/P3, 2026-09-13) expressible by a pack in ONE DIRECTION ONLY: an `nl` pack can add "y", and a `pt` pack cannot remove "e", so a Portuguese caller has to write `Parser(lexicon=Lexicon.default().remove(conjunctions_ambiguous={"e"}))` by hand. Accepted for 2.x and parked here rather than filed, because 3.0 is when a wider set of locale packs gets designed and this is part of that work (Derek, 2026-09-13). It is the 2026-08-17 PARTICLE VOCABULARY AS LOCALE PACKS bullet's "overrides rather than additions" requirement meeting a second tenant, which is what makes it a shape question rather than one pack's inconvenience. +- (A) 2026-09-13 — THE DIFFERENTIAL HARNESS CANNOT SEE A LOCALE PACK'S EFFECT AT ALL. `compare.py` builds one `Parser` per name ORDER and nothing else, and a `Case.locale` row projects into the corpora as a bare name string — no corpus row carries a locale key, verified over every `tools/differential/corpus*.jsonl` file on 2026-09-13 — so every corpus name is parsed under the DEFAULT `Lexicon` on both sides. A pack's effect on a name can therefore never diff, at any baseline. That is fine while packs only ADD vocabulary the default already carries most of, and stops being fine the moment the bullet above is answered yes. The shape of the fix is a `locale` carried per corpus ENTRY the way `shape` already carries an order, resolved at the baselines that ship the pack and skipped below them. Parked with the bullet above rather than filed, for the same reason: the two are one piece of work (Derek, 2026-09-13). +- (A) 2026-09-20 #397 — A SECOND INSTANCE OF THE SAME PER-LOCALE QUESTION, cross-referenced rather than parked again. Catalan/Polish `i` joins `conjunctions_ambiguous` beside `e`, so the marked subset now carries two letters and the 2026-08-18 bullet above — which counted six single-letter conjunctions and named the per-word-policy requirement — describes a set of seven, `i` included. Nothing in that bullet's argument moves: the subset IS the per-word policy the bullet asked for, made configurable without packs, and the pack-side half is still what the 2026-09-13 (A) bullet above asks, since a `ca` or `pl` caller still has to write `Lexicon.default().remove(conjunctions_ambiguous={"i"})` by hand. See `decisions/P3`'s 2026-09-20 bullet. +- (B) 2026-09-23 #459 — THE SHIM RAISES on a v1 config it once accepted: a `capitalization_exceptions` value that does not spell its key's letters and digits raises `ValueError` at the first parse (decisions/R4, 2026-09-23). Cited against the shim's never-raise rule and taken anyway, so it is recorded here: the raise stands on its own — a case mask has no reading for a value that re-spells its word, and the core `Lexicon` raises the same way with no shim involved — and only where it LANDS for a v1 caller, at the first parse rather than at construction, is shim-era. 3.0 has nothing left to decide. +- (B) 2026-10-02 #541 — THE SHIM DROPS, with a warning, every v1 `Constants` entry v1 could never match (decisions/R4, 2026-10-02). Cited from 1.4 parity, and it dies with the shim: the core `Lexicon` normalizes edge whitespace away and has no v1 reading to reproduce, so 3.0 has nothing left to decide. diff --git a/docs/design/decisions/A1-ambiguity-report.md b/docs/design/decisions/A1-ambiguity-report.md new file mode 100644 index 00000000..2f51241b --- /dev/null +++ b/docs/design/decisions/A1-ambiguity-report.md @@ -0,0 +1,15 @@ +# A1 — the ambiguity report + +- 2026-09-20 #397 (second review) — A REPORT MUST NOT CONTRADICT THE READING BESIDE IT, and the shape could not arise before this cycle. `classify`'s `conjunction-or-initial` fork offers exactly two readings and its detail says which it took ("it is read as an initial"). Where the rest of the parse then roles that letter SUFFIX, the fork resolved to NEITHER branch: `JOHN QUINCY SMITH I` carried the connective-or-initial beside a `suffix-or-name` saying the same token reads as a generational suffix, and `john smith i`, `HENRY I`, `SMITH, JOHN I`, `JOSEP CAROD I ROVIRA III` and `abdul e i` were the same. `i` is the first word that is both a MARKED connective and suffix vocabulary, so no parse before #397 could reach it — which is why the rule is written down now rather than having been. + DECIDED: the report is WITHDRAWN where the role is SUFFIX or TITLE, in `_pipeline/_assemble.py`, the one place that sees final roles and the ambiguity list together — and beside a withdrawal that was already there, for a report whose referents did not survive assembly. That is the narrow reading of mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE rather than an exception to it: classify still owns the fork and still takes it where it is taken; what goes is a report whose subject the parse went on to read as something else. Emitting later was the alternative and was rejected as more invasive for the same answer — the fork's inputs (the case class, the own-words span) live in classify and moving the emitter moves them. + MEASURED: over the review's 43,925-name grid under eight configurations, 9,806 rows on 1,637 distinct names lose exactly one `conjunction-or-initial` and nothing else — no report of any other kind moves and none is gained. On the deduped corpus-union-cases population the only name affected is `JOHN QUINCY SMITH I`, a row this round added, so the ledger's three-name `fix(#397) the Catalan link reports a connective-or-initial in a one-case name` rule is untouched: all three of its names role the letter MIDDLE and all three still report. The invariant is `tests/v2/test_properties.py::test_no_report_contradicts_the_reading_beside_it`, which states it twice — no connective-or-initial on a token roled SUFFIX or TITLE, and no token named by two reports whose details assert different readings — and fails on 8,204 rows at `dc3bdf9c`, 0 here. +- 2026-10-08 #626 — A REPORT NAMES THE FIELD ITS WORD LANDS IN. Six of assign's reports worded the field from the role the word held at assign (`read as a given name`), and rules in post_rules still move a word after that: H1 moves the name behind a title to the family, P1's family-first fold takes the particle run and one unit and lays the rest out by the order, P6's attachment takes a trailing name word into the family at the end of a name read family-first, and under opt-in policies the patronymic rotations (O1, O2) and `middle_as_family`'s fold (O3) move one too. So `Attorney General of Minnesota` reported 'General of Minnesota' "read as a given name by convention" with the unit in the family (since 2.3.0, #491's join clause); `Kim Min Do` under FAMILY_FIRST told the caller 'Do' was "read as a middle name" with 'Do' in the family, and `de Kim Ma` under FAMILY_FIRST said 'Ma' was a middle name with 'Ma' the given name (both this cycle, since #289 made the Title-case word a name word); `, Ma Dr.` said 'Ma' after the comma was "read as the given name" with 'Ma' in the family (this cycle, since #289 added that comma report; 2.3.0 reads the same fields and reports nothing); and `Van Ivan Petrovich` under the East Slavic patronymic rule said the leading 'Van' was "read as a given name" after O1 rotated it into the family (at 2.3.0 too). #614's second review had fixed one instance of the class (`Freiherr von Berg Ma`) by moving its report to the stage that knows the outcome. + DECIDED: the DECISION stays at its site and the WORDING of the field moves to assemble, the one place that knows final roles (the same reasoning as the 2026-09-20 withdrawal above). An emitter whose detail names a field a later rule could change writes the sentence up to the field and hands the rest as `PendingAmbiguity.field_tail` (`""` where the field ends the sentence); assemble joins them around `FIELD_READING` of the first referent's final role. All six such emitters in assign use it -- H4's all-suffix name, title-or-name's join clause, O5's lone word, the trailing peel's picks, the leading ambiguous particle and the comma report on the word after a family comma -- including those right on every measured input, so no later rule can make them wrong. A report in a name emptied for want of content is withdrawn rather than shipped half-worded (no emitter reaches that today). A placeholder in the detail was declined: token text can carry any characters, and a name containing the placeholder would corrupt its own report. Reports worded where the field is settled keep their literal wording: a comma's credential reading, the suffix run's and the numeral fork's, H4's "the name" and P6's own "joins the family". The comma report's given-side wording moves from "the given name" to "a given name" with the rest. + MISSED, then caught in review: the first draft converted five emitters and its grid had no empty-head comma with a trailing title, the shape AGENTS.md's comma sweep already asks for; four of the review's five readers found the sixth emitter on `, Ma Dr.`. The draft also named only H1 and P6 as movers; P1's fold was found by stepping the stages, and the review of the fix found the opt-in movers O1 to O3, which no grid over the three orders reaches -- reverting the leading-particle emitter passed the whole suite until rows under those policies pinned it. + MEASURED 2026-10-08, each report's field phrase compared against its tokens' final roles. RECIPE: texts are the 1,536 names of `tools/differential/corpus*.jsonl` and 347 `tests/v2/cases.py` texts (the table at 0c54662b, the parent of #614's first commit, less the corpus names; today's table adds rows this entry's review wrote for the shapes the grid missed, so it reads more wrong claims on master, 6 more as measured on the review), plus every head in {'John', 'John Smith', 'John Q. Smith', 'Mary Ann Smith', 'John van Smith', 'Juan de la Vega', 'John van', 'anh van', 'Jan Freiherr von Berg', 'Freiherr von Berg', 'Freiherr St', 'St', 'Dr. St', 'Freiherr', 'Dr.', 'Mr. and Mrs.', 'Attorney General of', 'abdul salam', 'Juan y Garcia', 'Jane Doe nee van der Berg', 'Dr. John Smith', 'JOHN SMITH', 'john smith', 'Smith', 'de Mesnil', 'Dr. de Mesnil', 'Kim Min', 'Kim Min Jun', 'Ortega y Gasset', 'van', 'Do', ''} followed by one or two words (repetition allowed) from the list [PhD, MA, Ma, DO, Do, do, vd, mc, Jr, III, V, V., VI, X.Y.Z., Ph. D., Esq., Prof., Sir, Ed, van, and, Jones, ba, John, Smith, Minnesota, de, St, Freiherr], space-joined and stripped; 20,000 draws after Python's `random.seed(626)`, each `random.choice(heads)`, then `random.randint(3, 4)` words each `random.choice(words)`, space-joined and stripped, both lists in the order written here; then, unstripped, `{head}, {word}`, `Smith, {head} {word}` with any double space replaced by one, and `{word}, {head}` for every head and word, and `, {w1} {w2}` and `(Bob), {w1} {w2}` for every ordered pair of words; the empty text dropped and duplicates removed -- 53,517 texts; each parsed under the three orders, so the opt-in movers are outside these figures; a claim is a match of `(reads? as|joins) ()` for the phrases `_FIELD_CLAIMS` in tests/v2/test_cases.py lists plus "the given name", and it is wrong when its tokens' roles are not among the fields the phrase names. On master (26cdb891) 2,725 of 89,092 claims were wrong, in three emitters: title-or-name's join clause (1,575), the trailing peel's picks (1,114) and the comma report (36); on the tree, 0. The comparator is master, which words the field at assign; the tree words it at assemble. Fingerprint against master over the differential corpora, master's case-table names and a seeded comma grid (13,090 names) under six policies (the three orders, `lenient_comma_suffixes=False`, a `/` suffix delimiter and `CapsSuffixes.OFF`): no role, kind, initials or rendering moves; the details that change are 6,395 comma reports on 1,086 names reworded from "the given name" to "a given name", 85 on 15 names now naming the family they land in, and 40 title-or-name join reports on 10 names now naming the family. Pinned by `test_a_report_names_the_field_its_word_lands_in` in tests/v2/test_cases.py, over every case row under all three orders except the locale rows (a row with its own policy only as declared), whose recorded control is 14 failures on master, and by `test_the_field_sweep_sees_the_claims_it_checks`, which records how many claims that sweep checks; the opt-in movers by the rows `Van Ivan Petrovich` (East Slavic), `Van Ali Veli oglu` (Turkic) and `Do Bishop Do` (FAMILY_FIRST with `middle_as_family`), which the sweep reads as declared. + +- 2026-10-10 #629 — A REPORT ON WORDS IN TWO FIELDS NAMES NEITHER. Segment's `comma-structure` report on a part past the second comma said the part was "consumed as suffix best-effort", true through 2.3.0, when the whole part read as suffix. Since #603 (rules/C2) a title word there reads as a title, so `John Smith, Jr., Dr. Bart` was told 'Dr.' was a suffix with 'Dr.' in the title (this cycle, never shipped). + DECIDED: the report names no field: "segment 'Dr. Bart' beyond the recognized comma structures; read best-effort". It says what is certain, that the part is outside the recognized comma structures, and the fields say where each word went. The input is malformed, so the report is not worth machinery: the fix is one string, and a part that reads wholly as suffix loses the word "suffix" from its detail, which is not contract (kinds are). + Declined: naming every field the part's words land in, worded at assemble from the final roles like #626's `field_tail` ("consumed as title and suffix best-effort"). It was built (0d5fea40, PR #630) and reviewed: it needed a second tail on `PendingAmbiguity`, an assemble branch and an equality check in the field guard, and the branch withdrew the report from a name emptied of content (`", , -"`), against rules/A2's "ambiguities born from its punctuation survive on the empty result", which no grid caught because every grid word had letters. More code to say more about garbage input; the user-facing gain did not pay for it. + MEASURED 2026-10-10, tree against master bc243869 over the names of `tools/differential/corpus*.jsonl` plus every `tests/v2/cases.py` text, each under the three orders, comparing every report's kind, detail and referents and the seven fields: only the detail of `comma-structure` reports moves, and no report appears or goes. + diff --git a/docs/design/decisions/A2-empty-name.md b/docs/design/decisions/A2-empty-name.md new file mode 100644 index 00000000..b809f6ba --- /dev/null +++ b/docs/design/decisions/A2-empty-name.md @@ -0,0 +1,4 @@ +# A2 — the empty name + +- 2026-07 (v2 core, PR #288) — v1 kept parse(".") as first="."; 2.0 empties content-free input instead. The born-empty-ambiguity survival was a review fix in the rc1 arc: an unbalanced-delimiter report must outlive the emptying, or malformed input becomes indistinguishable from blank input. + diff --git a/docs/design/decisions/C1-suffix-comma-decision.md b/docs/design/decisions/C1-suffix-comma-decision.md new file mode 100644 index 00000000..6dce8958 --- /dev/null +++ b/docs/design/decisions/C1-suffix-comma-decision.md @@ -0,0 +1,83 @@ +# C1 — the suffix-comma decision + +- 2026-07 (v2 core, PR #288) — v1 parity: only the second segment decides (v1 parser.py:1318), and ">1 word before the first comma" is v1's guard, which is why "Smith, PhD" keeps the listing form. The strict/lenient knob's blast radius on default vocabulary is exactly the single-letter Roman numerals i and v — the only initial-shaped members of the shipped suffix words — which is what makes rules/C1's "V." examples the canonical pair. +- 2026-07 (plan deviation #3, recorded) — the decision is definitionally vocabulary-dependent: there is no way to recognize a credential run without consulting the suffix word lists, so the structural stage reads vocabulary through one predicate. +- 2026-07-12/13 (v2 Policy work, PR #288) — the lenient token test is the default and `lenient_comma_suffixes=False` restores the strict one. (An earlier entry here credited #291/#296 — git author dates place both the field and its wiring in the Policy commits, and the #291/#296 design doc never mentions the knob.) +- 2026-08 #319 — the wholly-suffix predicate was lifted into the vocabulary layer so the comma decision and the honorific peel's segment test cannot drift apart. +- 2026-08-23 #296/#325 — the listing form's post-comma part is read for what it is. Three commits of the approved #291/#296 bundle (spec 2026-07-30, plan 2026-08-01; see the comma-suffix arc below), in the plan's order. (1) A comma followed only by titles fixed no family boundary, so the pre-comma name keeps its positional read: `John Smith, Mr.` reads given 'John', family 'Smith', title 'Mr.' where 2.0 had merged the family ('Bob Jones, author' moves with it at every baseline). "Nothing but titles" is title-shaped AND not suffix vocabulary — the plan's helper said only the first, and would have read `Smith, PhD Jr.` as the title run 'PhD Jr.' and `Smith, Mr. Jr.` as 'Mr. Jr.' (both master's suffix 'Jr.'; the gate and a measurement against master caught them). (2) The TITLES/suffix overlap audit and the ordering, one commit because each alone is wrong (the plan's F1): reading the credential run first would make `Smith, Dr.` a suffix while 'dr' was SUFFIX_WORDS residue, and the audit alone leaves `Smith, Jr.` a title by the period-abbreviation inference. The audit table's dispositions ship as approved — 'do', 'jr', 'junior', 'phd', 'se' out of TITLES; 'dr' and 'sra' out of the suffix sets; 'ms' and 'sa' into the ambiguous set — with ONE deviation on evidence the table did not have: 'md' keeps dual membership, because bare 'Md' before a name is the Bengali and South Asian Muslim abbreviation of Muhammad (#343/#345's corpus rows `Md Abdul Karim`, `Md. Abdul Karim`; the drop read the first as given 'Md', middle 'Abdul') while 'MD' after the name is the degree — position decides, as for 'sr'. The ordering is wider than the plan's lone piece: a post-comma part that is nothing but suffix pieces is the credential run, whole, which is what #325 needed (`Smith, Ph. D. Jr.` had put the split credential in the given name once a second suffix made it no longer lone — a 1.4.0 regression; v1 read suffix 'Ph. D.', title 'Jr.'). Vocabulary decides which words qualify and position breaks the tie for the genuine duals: `Smith, Sr.` is Senior where `Sr. Garcia` is Señor, `Smith, Esq.` is the postnominal where H2 still reads `Esq. Smith` as a title (H2 carries that as Accepted). `Smith, Dr. Jr.` keeps the 2.0 deviation's other case — a title, then the last piece as the suffix (v1 read first 'Jr.'; 2.0 through master suffix 'Jr.') — and `Smith, John Jr.` its given name. Consequences the audit carries, all measured and pinned: a leading 'PhD' or 'Jr' is a name word; `Do Quang Minh` reads given 'Do' with the particle-or-given fork, and `Dr. Do Van Johnson, MD` given 'Do', family 'Van Johnson' (the #367 rule for that name is replaced; TITLES ∩ ambiguous is {freiherr, st} now, which keeps the chained emitter reachable); a trailing bare 'Dr' is a name word, joining every other title-only word there (#316 is the open question); `田中さん, PhD` reads suffix 'さん, PhD'; the #100 prefix-join tests use 'lt', a genuine dual, as their token. #291 (multi-word credentials) is the bundle's fourth commit and its own PR. +- 2026-08-23 (review of PR #428, four reviewers) — the repair's condition is "no name word", not "all titles": `John Smith, Mr. Jr.` (a title and a postnominal) had kept the merge the rationale argues against, and reads given 'John', family 'Smith', suffix 'Jr.' now. Its guard counts NAME pieces: the positional read peels a trailing suffix first, so `Smith Jr., Mr.` — two pieces, one name — read positionally lost its family (the code review); it keeps family 'Smith', suffix 'Jr.' at every baseline. The positional read records its ORDER, and post_rules' leading-piece scan keys on "assign records no order after a family comma" rather than on the structure, so P1's family-first fold reaches the pre-comma name as it does without the comma — `de Mesnil Jean, Dr.` under a family-first order had read family 'de' (the test review). The credential run does NOT step over a delimiter core the policy names (`Smith, RN - CRNA` reads given 'RN' under `extra_suffix_delimiters` as without it): the code review asked for it, a draft did it, and tests/test_suffixes.py's "the delimiter must not affect parsing at all" outside the suffix-comma form is v1 parity pinned on purpose — C1 carries the limit as Accepted. A second deviation from the audit table: `ms` and `sa` do NOT join the ambiguous set — the gate is position-blind and the collision is not; gated, `John Smith, MS` lost its suffix-comma route and read title 'MS', and `Smith, Ms.` passed the gate on its one period and read as a credential anyway. Both are genuine duals, and position decides: `Smith, Ms.` reads suffix 'Ms.' (title at every baseline; C1 carries it as Accepted, with `Smith, Ms. Jane` keeping the title) and `John Smith, MS` keeps suffix 'MS'. And the gate itself now means what S2 says: an ambiguous acronym counts when written with ITS periods, one after each letter (`M.A.`, `J.D.`), not on any period — `Smith, Ed.` had become a credential on a name's trailing period, and `Jack Ma.` had read suffix 'Ma.' since 2.0 (S2 carries the boundary; 1.4.0 read family 'Ma.'). Two limits recorded rather than fixed: `田中さん, Dr.` keeps the honorific glued, joining master's `田中さん, Mr.` — the honorific peel runs in script_segment on the suffix-comma and no-comma structures, before group or assign can say the comma fixed nothing (C1 Accepted); and group's chain emitter still reports no PARTICLE_OR_GIVEN on the family-comma path, so `Freiherr von Berg Ma, Mr.` reads as its no-comma twin but without the report — group runs before assign decides which read applies, and plumbing a deferred report through the state is more than the shape is worth today. +- 2026-08-23 #429 — one comma segment is one suffix entry, and the two stages disagreed about which segments those are. group asked by segment INDEX (`tail_start`: 1 after a suffix comma, 2 after a family comma), which is right wherever assign reads the segment as suffixes for the same structural reason — but not after a family comma, where segment 1 is a name slot assign re-reads by CONTENT with `_segment_holds_no_name` and routes piece by piece (#296/#325). So the roles were right and the render was not: `Smith, MD PhD` gave suffix 'MD, PhD' with a comma the writer never typed, and `Smith, CBE MC`, `Smith, BSc MBA` and `Smith, PhD Jr.` the same, while the full-name `John Smith, MD PhD` has rendered 'MD PhD' since 1.4.0. The predicate moved down into group beside the piece predicates it already reads — the same direction `_leading_titles` moved in #424, and the only direction available, since assign imports group. (#436/#437 took group's call away again on 2026-09-06, leaving assign the one caller; the predicate itself stays in the piece layer.) + SCOPE, stated as the condition actually reads rather than as the first draft described it: any family comma whose following segment holds no name word, NOT only a one-word family. `John Smith, Jr. III` renders suffix 'Jr. III' (1.4.0's reading), and a title-led segment is in too — `Smith, Dr. MD PhD` renders 'MD PhD'. The differential cannot witness the widening: of 1069 corpus names, 303 carry a structure comma, the predicate is true for 36, only 9 of those have two or more pieces in the segment, and none of the 9 combines a multi-word pre-comma name with two suffix pieces. 0 unexplained is therefore evidence about the corpus, not about the class — the case table carries it instead. + THE REGRESSION THE REVIEW FOUND, and the reason this entry is long: the first draft let ANY piece open an entry, as the tail block always had. That is safe on a tail segment, where assign routes every piece to SUFFIX — which is what `tail` means — and wrong off it, where a title piece routes to TITLE. Two failures, one cause. The `joined` tag is role-BLIND and the facade heals it for every role, so a title opening an entry collapsed `title_list` from ['Rev.', 'Dr.'] to ['Rev. Dr.'] on `Smith, Rev. Dr.`; and where the pre-comma name left a suffix, the tag glued the next suffix BACKWARD across the writer's own comma — `Smith Jr., Mr. Jr.` rendered suffix 'Jr. Jr.', the exact inverse of the bug being fixed. Neither is visible to the differential, which compares strings, nor to the case table as it stood, which asserts the title STRING (space-joined either way). The rule that replaces it separates two joins that had been one: WITHIN a piece the tag renders a merged piece as one unit, whatever role it holds — that half still stands; BETWEEN pieces it continued an entry, and only a piece that rendered into the same run could do that. The entry was sticky across a piece that was not in it, so an interleaved title did not split the run it sat in (`Smith, MD Dr. PhD` renders 'MD PhD'), and a delimiter core closed it, which was the one thing that separated two entries. That between-piece half is GONE since 2026-09-06: #436/#437 replaced it with a rule over the commas the writer typed, the bullet below. Both readings it was built for are unchanged — the stickiness holds by construction now rather than by code. + Scoped deliberately, and the scoping is the other decision: `tail` still gates delimiter-core dropping, which is v1 `expand_suffix_delimiter` parity (#206, commit 021823e — three code comments say #191, which is the German/Dutch vocabulary PR, and are corrected here) and a TAIL rule. The core set is threaded to the block as `seg_cores` so the widened condition cannot drop cores where `tail` says not to. Worth knowing before anyone tries to pin that: for every realistic delimiter the two spellings are indistinguishable by construction, because the core must first pass the very predicate that gates the block — the only reachable class is a policy whose delimiter core is itself suffix or title vocabulary (`Policy(extra_suffix_delimiters={" jr "})`, `Smith, MD jr`), which the case table does not enshrine. + Cost, measured rather than optimised away: `_segment_holds_no_name` now runs TWICE per family-comma parse, once here and once in assign, on the same pieces — verified pure recomputation, not two questions (408 of 506 comma-bearing test literals reach both sites, 0 disagreements, and off-tail `seg_cores` is empty so `kept` never filters, which is why assign sees exactly the pieces group tested). Removing the second call is worth 0.6–1.4 µs, 1.2–2.2% of a family-comma parse and 0% of every other parse (NO_COMMA and SUFFIX_COMMA make no call at all). Caching it needs a derived ParseState field the two sites would otherwise not share, so the number is recorded instead. The order is cheapest-first per 7009fcf: two bool loads and an int compare gate the call. + R1 gained the separator sentence with this change. That the invariant it states — one comma segment is one suffix entry — was written down in no rule is a large part of why two stages disagreed about it for years: C1 governs routing, R1 governed word ORDER and never mentioned separation, and the only statement of the rule was a code comment. + Left alone, measured and pre-existing: the NO-COMMA path comma-joins a space-separated run too (`John Smith MD PhD` renders 'MD, PhD' at 1.4.0, 2.0.0, 2.1.0 and master alike). The fix requires FAMILY_COMMA and does not reach it; that path has no writer-drawn boundary and is a separate question, recorded here and NOT yet filed. It is load-bearing for one thing: `str()` of a fixed parse is a no-comma string, so re-parsing it re-inserts the comma — round-tripping is not stable on these shapes, and an earlier draft of the release bullet claimed it was. CLOSED 2026-09-06 by #436/#437, the bullet below — every sentence above stands as what that tree measured, and all three of its open clauses are now spent: the question WAS filed, as #436 and #437; the boundary moved from segment shape to the comma the writer typed, which reaches the no-comma path because it never asks about segments at all; and round-tripping is stable on these shapes, pinned by `test_a_suffix_run_survives_a_render_and_a_reparse` in tests/v2/test_render.py. + The name `Smith, MD PhD` entered the corpora as this fix's C1 example — a FIRING one, carrying no `· boundary` annotation, which marks the non-firing example a rule must have — so its ledger entries arrive with it. A fourth written for the 1.4.0 ledger EXPLAINED NOTHING and was dropped: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name there, its fields covering all three that move. That is #373's dormant-rule diagnosis doing its job, and #426 is the precedent for dropping rather than keeping a shadowed rule. +- 2026-08-25 #430 / #432 — one boundary, fixed as one: inside a comma segment a ONE-CHARACTER suffix word — the only kind a reader could take for an initial — is read by WHAT STANDS BEFORE IT. Behind another suffix it describes that suffix and continues the run, period included — `Smith, PSM I` and `Smith, PSM I.` are suffix 'PSM I'/'PSM I.' where they read given 'PSM' plus suffix 'I', an initial in that slot being no shape anyone writes. Behind a name word the period is decisive, marking an abbreviation and so name material — `Smith, John V.` is middle 'V.' where it read suffix 'V.', which is what `Smith, John B.` always did; the only thing that made V differ is that V is also suffix vocabulary. The bare form is the boundary and does not move: `Smith, John V` stays suffix 'V' (v1 parity, #144), which is why the test is the PERIOD and not the numeral. SCOPE, stated as the code reads rather than as the intent was argued: the class is one-character SUFFIX VOCABULARY, not "a roman numeral". It sweeps in `2` (`Smith, John 2.` is middle '2.'), misses `X` and `IX` (roman, but not vocabulary — `Smith, PSM X` is unchanged), and never touched `II`/`III`/`IV` at all, which are two characters and so were never initial-shaped and never vetoed. Anyone adding a one-character suffix in a script that has initials extends it. And the strict knob is consulted: under `lenient_comma_suffixes=False` the initial veto stands and the run ends at the numeral, which the first draft silently overrode by reading no policy at all. + #430's cause was rules/S2's initial veto in `is_suffix_piece`, which keeps a numeral out of a suffix reading — right after a name, wrong inside a credential run — so the segment did not look like a run at all and fell to the given-and-suffix walk. Two wrong answers came out of that one cause, and a fix verified on `Smith, PSM I` alone would have left the other: where the run's first word is TITLES vocabulary too (`Smith, MD I`, and `Smith, Jr. I` through the period-abbreviation inference) the leading-title peel took it instead, giving title 'MD' plus given 'I'. + THREE readers, one reading, which is the part worth copying. The gate that says a segment is a credential run, assign's router that assigns its roles, and group's one-entry join that renders it must agree PIECE FOR PIECE, and #429 shipped the inverse of its own fix by deriving that agreement twice. `segment_suffix_reading` returns the per-piece verdicts and all three read it (mechanisms.md#ONE-PREDICATE-PER-QUESTION); the intermediate state where only the gate and the router had been converted rendered `Smith, PSM I` as 'PSM, I', the render splitting a run whose roles were already right — the same tell as #429, one predicate along. TWO readers since 2026-09-06: #436/#437 took the render join out of group, and `segment_suffix_reading`'s remaining readers are assign's gate and assign's router, one call site between them. The predicate is not moved — one call site is no reason to relocate what two sites will contest again — and what is worth carrying away is unchanged, since the reason the three had to agree is the reason the two still do. + Declined, and the v1 suite caught it: asking `is_trailing_numeral_suffix` for #432's half. It answers the period question — `_ROMAN` is anchored and admits no period — but it also refuses a numeral standing behind an initial-shaped piece, which is a NO-COMMA rule and the opposite of this path's parity: `Chang, Andy C I` is first 'Andy', middle 'C', suffix 'I', and asking that predicate here made the numeral a middle. decisions.md's #401/#421 entry records the same fork declining to transfer to this walk under LENIENT. The half that applies is the period alone. + Blast radius, and the corpus IS blind to the moving classes: of the 1069 names on master, seven carry a one-character numeral behind a word inside a comma segment — `Chang, Andy C I`, `Doe, Rev. John V, Jr.`, `Maier, Amy I, Jr.`, `Maier, Amy Lauren I`, `Smith, John I`, `Smith, John V`, `Berg, abdul V` — every one of them BARE, and NONE moves. Zero pre-existing corpus names move at all; the only movers are the three this change adds as C1 examples. So the gate's 0-unexplained is evidence about the corpus, not about the change, and the case table is the whole coverage. `Berg, abdul V` cannot reach the walk at all: the bound-given join makes `abdul V` one piece, which is why the LENIENT mirror #401/#421 declined does not collide here. The moved names entered the corpora as C1's new examples, and carry ledger entries at all three baselines except #430's at 1.4.0, where `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name and a second rule would be dormant. + Two things the /simplify round settled by measurement rather than argument. The walk asks `is_suffix_piece(pieces[m - 1])` — a VOCABULARY test — where the reading carries the verdict it just recorded, and unifying them looks obviously right until you run it: the walk assigns `PSM` to GIVEN in `Smith, PSM I.`, so a reading-style flag says "behind a name" and sends the numeral to middle, losing the strict parity restored two commits earlier. The two sites ask different questions on purpose, and the vocabulary answer is the load-bearing one. The cost is a reading the walk cannot reach: `Smith, John PSM I I.` moves from middle 'I' + suffix 'PSM, I.' to middle 'I I.' + suffix 'PSM', two consecutive one-character numerals behind a credential — accepted, both readings being defensible for a shape nobody writes, and recorded here rather than chased at the price of two correct pins. + The walk is also strict-blind, pre-existing and unchanged: it calls is_suffix_lenient unconditionally, so `Smith, John V` reads suffix 'V' under `lenient_comma_suffixes=False` as under the default. The reading consults the knob and the walk does not, which is now conspicuous where it was merely unexamined. Not this change's to fix. + Not done, and recorded rather than left implicit: neither half emits an ambiguity, though the no-comma `John Smith V` reports `suffix-or-name` for the same fork. The comma path has never reported it (decisions/C1 records PARTICLE_OR_GIVEN suppressed there for a related reason), and adding an emitter needs a trigger, a stage-ownership entry and a case row — more than this fix, and #432 raises it as a secondary observation rather than a requirement. +- 2026-09-06 #436/#437 — the entry boundary is the comma the writer typed, not the shape of the segments. Two consecutive post-nominals are one entry iff they sit in the same comma bucket and nothing between them parts the run; `comma_bucket` is the function `segment` builds segments with, so "same part" here is an identity with segment's answer rather than a resemblance to it (mechanisms.md#ONE-PREDICATE-PER-QUESTION). What PARTS a run is a name word — given, middle or family — or a dropped delimiter core (#206). What does NOT part it is a token whose role renders into some other field altogether: the pass names all three of them beside itself as `_RENDERS_ELSEWHERE` — title, nickname and maiden — and such a token is not standing in the run at all, which is what keeps `Smith, MD Dr. PhD` one entry (#429's stickiness, now by construction) and `Smith, MD "Doc" PhD` and `Smith, MD (nee Jones) PhD` with it. Nor does a dropped token that is not a core part it: that is the bare maiden marker of `Smith, MD nee Jones PhD`, dropped with no role at all, so the role test cannot see it. Which is why the dropped arm reads the core SET rather than treating every dropped index as a boundary — a core is the one dropped token the writer typed AS a separator — and the set is `delimiter_cores` in the vocabulary layer, the same derivation off `Policy.extra_suffix_delimiters` that group's drop site reads, imported rather than repeated so the drop site and this pass cannot disagree about what a core is. That is this file's own ONE-PREDICATE-PER-QUESTION again, one level down from the readers #430 counted. FOUND, and it is the reason this is a fix rather than a change: R1 stated the rule in words on 2026-08-23 and the NO-COMMA path had violated it since the sentence was written — #429 added the sentence and recorded the violation without fixing it — its bullet above opens "Left alone, measured and pre-existing" and says of the no-comma path that it "is a separate question". `John Smith MD PhD` rendered suffix 'MD, PhD' at 1.4.0, 2.0.0, 2.1.0 and 2.2.0 alike, a comma nobody typed, and it is now R1's own example line. This bullet is R1's recorded history as well as C1's — `rules/R1`'s `history:` field points at `decisions/C1` and at nothing else, so the separator rule's provenance has always lived under the suffix-comma decision, and a reader arriving from R1 arrives here. + WHAT GROUP LOST, and it is the whole between-piece half: `tail` by index, #429's `segment_suffix_reading` verdict ORed into it, the per-piece gate, and the stickiness across an interleaved title. Three shape-derived answers to one question was the defect, and the review of #429 had already found that shape wrong in both directions at once. The stickiness needs no code now: `Smith, MD Dr. PhD` has no comma between MD and PhD and the title between them renders into another field, so they join, while `Smith Jr., Mr. Jr.` has the writer's own comma and they do not — both still pinned. What stays in group is core DROPPING keyed on `tail` through `seg_cores` (v1 `expand_suffix_delimiter` parity, #206, a TAIL rule) and the within-piece tag that heals the `Ph. D.` merge, which is role-blind and right for every role. `segment_suffix_reading` is left where it is with ONE call site, in assign; the #430 entry above records three readers, and the render join was the third. + AMENDED FROM THE SPEC TWICE, both times by measurement, and this is the half worth reading twice. The approved predicate parted an entry only at a DROPPED delimiter core. Measured 2026-09-06, the two corpus names the spec names as that class — `Smith, MD - PhD - FACS` and `John Doe, MD - PhD - FACS` — drop no core at all under the default policy the corpora compare under, the dash being no delimiter there: the dashes survive as ordinary given/middle words between PhD and FACS, and a rule reading `dropped` alone joins them where 'PhD, FACS' is pinned. `cases.py`'s `suffix_delimiter_detection` had the same shape pinned already. So the predicate asks what STANDS between the two rather than only what was dropped, and both halves are pinned, one row each, because a dropped-core test alone passes on the rule that broke the corpus. Reading the dropped arm against the core set also retires the spec's open question of how the pass tells a core from a maiden marker: no `WorkToken` tag and no new `ParseState` field, the set being the one group already derives. + THEN AMENDED AGAIN, in review of the FIRST DRAFT of this bundle's code commit — caught before that commit was rewritten, so the commit as it shipped already names all three transparent roles — and the second find is worth as much as the decision. That first draft's predicate made TITLE the only transparent role and treated every dropped index as a boundary, and it was wrong in both directions at once. Measured 2026-09-06 against the parent tree — parse the strings on a pristine checkout of the parent, then against a copy of the pass with `_RENDERS_ELSEWHERE` cut to the title alone and the dropped arm reading any drop as a boundary — three comma-path shapes that render 'MD PhD' at the parent went to 'MD, PhD': `Smith, MD "Doc" PhD`, `Smith, MD (nee Jones) PhD` and `Smith, MD nee Jones PhD`, a nickname, a parenthesised maiden clause and a bare maiden marker, one regression apiece. And the no-comma twin of the first, `John Smith MD "Doc" PhD`, rendered 'MD, PhD' at the parent and stayed there under that predicate — the very bug being fixed, left unfixed on a nickname-carrying shape. The differential witnessed none of the four, so the gate was green over all of it and the code review is what caught it; the general form is that a transparency written from the roles a corpus happens to hold reaches only the roles that corpus happens to hold. Four stage tests pin the four shapes now, and the pass renders 'MD PhD' for all four. + MEASURED 2026-09-06, before and after in the same worktree, the gate exiting 0 at every baseline with `unexplained: 0` and `radar unclassified: 0` throughout: intentional diffs 352 → 360 at 1.4.0, 247 → 254 at 2.0.0, 155 → 166 at 2.1.0 and 14 → 28 at 2.2.0, Latin-only 247 → 255, 151 → 158, 151 → 158 and 14 → 24. The corpus itself grows by exactly one name, the R1 example `John Smith MD PhD` entering `corpus_rules.jsonl` — 1113 → 1114 at 1.4.0 and 1120 → 1121 at the other three — and that is the +1 above the mover count at every baseline. Recipe: `tools/differential/compare.py --baseline B` at each of the four, whole output kept rather than the summary line; the movers themselves from the spec's Recompute block extended to read `state.dropped`, the token roles and the core set, which the bare comma-bucket version cannot do — it reports 15, the two delimiter-core names being the two it joins wrongly. Exactly ONE name changes RULE, at two baselines and no more: `abdul Smith Jr V` at 2.0.0 and 2.1.0, off `fix(#401) the bound-given reserve counts the trailing numeral assign reads as the suffix` and onto the compound rule, its widened diff no longer being a subset of `fix(#401)`'s `fields`. Checked by spying `compare.classify` and diffing the WHOLE (name → rule) map per baseline, not the gate's printed sections, which list each rule's first ten names and would hide a swap inside a larger one. Six new rules across the four ledgers at fourteen placements: the Latin alternation in all four, the bound-given compound at 2.0.0 and 2.1.0, and four one-name CJK rules at 2.1.0 and 2.2.0. The Latin alternation sits FIRST in each ledger on purpose — moved LAST in a scratch copy of the ledger with `compare._allowlist_for` redirected at the copy, the run reports three undeclared order-decided contests at 1.4.0, three at 2.0.0, three at 2.1.0 and none at 2.2.0. + VIEWS, over the 1117 corpus names — the 1116 that were there plus the R1 example: exactly 13 of the 1116 pre-existing names move, and the new example `John Smith MD PhD` makes 14; each of them moves in `suffix_list` and in the v2 dictionary and token-tag list and nowhere else. No other `*_list` view and no v2 role's token list moves on any name in the corpora. Recipe: for every corpus name dump the facade's eleven `*_list` properties plus `initials_list()` (a method, not a property) and, from `parse(name)`, `as_dict()`, `tokens_for(role)` for each role and every token's tags, before and after, and diff key by key. The tags are dumped beside the views because the change IS a tag change, and a view that renders the same over a token whose tags moved is what a view-only snapshot cannot see. Every mover goes from a comma-joined render to a space-joined one; none goes the other way, and none has a comma in the original between the two words. + RE-PINNED, nine `cases.py` rows and not the two the spec expected: `title_plus_one_word_two_suffixes`, `family_segment_multiple_suffixes`, `suffix_run`, `family_comma_segment_zero_is_not_the_run`, `family_comma_numeral_behind_a_suffix_is_not_an_initial` and the four CJK honorific rows. Eight assertions outside the case table moved with them, four of those in the v1-parity suite (`tests/test_capitalization.py`, `tests/test_suffixes.py`), and they are a deliberate v1 deviation rather than a parity break: v1 inserted a comma into a run the writer had spaced, its own comments calling that "not ideal but at least its in the right bucket", and the release-log bullet names the deviation. Four rules.md example lines outside R1 — P2's `John Smith Mc V`, P5's `abdul Smith Jr Ma`, C1's `Smith, John PhD I.` and W3's `田中さん 様.` — carried the comma too; all four names are differential corpus entries, the last of them in `corpus_cjk_tolerated.jsonl`, so the movement was observed by the gate and not only by the doc test that executes them. `family_comma_segment_zero_is_not_the_run` keeps its NAME and its point — segment 0 is the family segment even when wholly credential-shaped, which is what the mutation matrix found and what the row still guards — and only its separator moves. The comma-joined values the nine carried were the shape machinery's artifact and never a reading anyone chose. + ROUND-TRIPPING is stable on the shapes #429 fixed, which is what that entry recorded as the one load-bearing consequence of leaving this open: `str()` of a fixed parse is a no-comma string, and the no-comma path put the comma back. `test_a_suffix_run_survives_a_render_and_a_reparse` in tests/v2/test_render.py asserts `parse(str(parse(s))).suffix == parse(s).suffix` over the three writings. NOT stable in general, and the general claim is the one to resist — measured 2026-09-06 by evaluating that same equality over every corpus name, at the parent tree and here: 35 of 1116 names failed at the parent and 23 of 1117 fail here, thirteen fixed and ONE newly failing. The one is `Smith, John PhD I.`, and it is this same artifact seen from the other side: it round-tripped only because the render emitted 'John Smith PhD, I.' with a comma the writer never typed, which the no-comma path read back as a run. The render writes 'John Smith PhD I.' now, and that string reads middle 'Smith PhD', family 'I.' at the parent and here alike — a bare trailing initial behind a name is name material, #430's own boundary — so the two spellings genuinely disagree and the comma had been hiding it. The 23 that remain are the standing class: `str()` is a rendering, not a canonical form, and no rule in this file promises it re-parses. + NOT FIXED, and named because this change makes it reachable on more inputs: `revise(n, suffix=n.suffix)` is not the identity on a space-joined run. `Parser.revise` classifies each value by a full sub-parse of that value alone — its own docstring records the limit, the value being "classified ON ITS OWN" — and a bare 'MD PhD' carries no comma for the entry rule to route by, so the two words come back as two entries and the field re-renders 'MD, PhD'. Pre-existing, and the same shape as the `str()` limit above rather than a new one. Measured 2026-09-06 over every corpus name, comparing `revise(parse(n), suffix=parse(n).suffix).suffix` against `parse(n).suffix` at the parent tree and here: 24 of the parent's 1116 names fail and 38 of the 1117 here do, none of the parent's 24 recovering, and the 14 added are exactly the 14 movers — the 13 pre-existing names plus the R1 example. Not this change's to fix: the entry rule reads the commas in a whole name, and what `revise` hands its sub-parse is a field, so closing it means deciding what a field value's separators mean, which is a `revise` question. Answered, and FIXED, by #511 — the next bullet. + DECLINED, all four with the evidence: marking the boundary at the core-drop site (#437's MARK-DONT-STRIP shape) — `dropped` already holds the fact with its span, and a second recording of it is the duplication that mechanism exists to prevent, one level up; making the `"joined"` tag role-aware — within a piece it is role-blind and correct for every role, `Smith, Ph. D. Smith` giving `first_list == ['Ph. D.']`, and only the between-piece half was ever a suffix concept; a render-time span scan instead of the recorded tag — `_facade.__setstate__` and `ParsedName.replace()` synthesise span-less tokens, so an unpickled name has nothing to scan and the tag IS the entry structure a pickle carries; and fixing the no-comma path inside group's block as a third branch beside `tail` and `reading`, which is the shape #429 took. +- 2026-09-06 #511 — a suffix value handed to `revise()` derives its entries from its own commas, by the rule a whole name uses. `Parser.revise` sub-parsed each value and forced every harvested token to the named role AFTER the sub-parse had run R1's entry pass, which keys on Role.SUFFIX; a bare 'MD PhD' reads there as a title and a family name, so the pass joined nothing and the field rendered 'MD, PhD'. The fix runs the same pass again: `revise` now sub-parses to a ParseState, forces the role on every non-dropped token, and calls `suffix_entries` — the pass, lifted unedited out of post_rules' tail into a function the two callers share (mechanisms.md#ONE-PREDICATE-PER-QUESTION) — over the forced state, then assembles and harvests as before. So a comma in the value parts two credentials and a space joins them. TWO SPELLINGS of the lifted pass, and the reason is the call budget: a first draft had post_rules call the state-in/state-out wrapper, and the second ParseState build cost three more calls per parse against the band tests/v2/test_benchmark.py holds (py3.11, 2026-09-06, tools/perf/call_count.py: 450 calls/name before the move, 451 with the in-place worker `_mark_suffix_entries` that post_rules now calls, 454 with the draft; the facade band tops at 455.9), so the worker writes in place as every other post rule does and the wrapper exists for the one caller with no token list of its own. NO `_run` HELPER for the same reason: a draft routed `parse()` through a private state builder shared with `revise`, and that cost one frame per parse on the hot path (py3.11, 2026-09-06: 415 calls/name against 414 without it, in a 402-418 band), so the four-line construction is spelled twice and `parse()` is untouched. MEASURED 2026-09-06 over the 1117 distinct names in `tools/differential/corpus*.jsonl`, comparing `revise(p, suffix=p.suffix).suffix` against `p.suffix` under the default parser for every name with a non-empty suffix (368 of the 1117): 38 differed before and 1 after; the differential gate is byte-identical at all four baselines, `revise` not being on the compare path. RECOMPUTE: parse each corpus name, skip an empty suffix, revise the parse with its own suffix, count the names whose suffix moved (the script is in the #511 issue body). THE ONE LEFT is '김민준씨, J.씨', and it is not entry structure: the whole-name parse keeps 'J.씨' one glued suffix token, suffix '씨, J.씨', while the sub-parse of the bare value '씨, J.씨' peels the honorific off the initial, so the revised field renders '씨, J. 씨' — right entries, the spurious comma of before ('씨, J., 씨') gone, one word read differently by the value's own parse than by the whole name's. That is the "classified ON ITS OWN" limit `revise`'s docstring has always recorded, and CJK honorific peeling is a W-rule question this change does not move; pinned in `test_revise_reads_a_glued_honorific_on_its_own`. SUPERSEDES the phd-merge acceptance of 'Ph., D.' on this path (its bullet says how). STALE TAGS, tried and backed out: a draft cleared the sub-parse's own "joined" with the forcing and re-derived it, because the pass only ADDS the tag and `revise(n, family="Jones MD PhD")` carries the sub-parse's between-piece suffix mark on 'PhD' onto a FAMILY token. Measured 2026-09-06 against `330ee55`, the clear also destroyed every WITHIN-piece mark on a non-suffix value — 'D.' of `revise(n, family="John Ph. D. Smith")` lost the merge mark the #436 bullet's DECLINED list calls role-blind and correct for every role — while on a ParsedName only the suffix string view reads "joined" (`_text_for`'s suffix_join gate) — the facade's `_list_for` heals it for every role, but no path puts a revised name into a HumanName, the v1 setters going through `replace()`, so wiring those setters onto `revise` is the change that would show the stale mark, as `last_list == ['Jones', 'MD PhD']` — `initials()` and every field string being identical at both trees for every shape measured. So the tags are kept minus FOLDED_TAG as before; the between-piece mark on a forced non-suffix role is a tag-only oddity that predates this change and stays, and for a suffix value nothing depends on the sub-parse's marks, every pair it joined sharing a bucket with no parting token and the pass setting it again; pinned in `test_revise_keeps_the_sub_parses_within_piece_mark`. Dropped tokens keep their role and tags through the forcing, because the pass filters them by index and assemble omits them. ONE MORE LIMIT, pinned in `test_revise_leaves_a_policy_delimiter_unparted_without_a_tail_segment`, and it is the "classified ON ITS OWN" limit again rather than a rule of revise's: a delimiter the policy names through `extra_suffix_delimiters` is dropped, and so parts entries, only on a segment after a comma that the reading of the words makes a tail, and a value with no comma of its own has none — under `Policy(extra_suffix_delimiters=frozenset({" - "}))` the whole name 'Doe, John, MD PhD - FACS' renders 'MD PhD, FACS' while `revise(n, suffix="MD PhD - FACS")` renders 'MD PhD - FACS', the dash surviving as a token and the forced role making it a suffix word; measured 2026-09-06, a value whose own words read with a tail segment does part ('John Doe, MD - FACS' revises to 'John Doe, MD, FACS') and one after a suffix comma does not ('MD, PhD - FACS' stays), which is how the value's words read as a name deciding it. The round-trip is unaffected, the whole-name view having already rendered that boundary as a comma. DECLINED: a list-valued `revise(p, suffix=["MD PhD", "FACS"])`, which widens the API and leaves the string path where it was; documenting the limit with pins alone, the docstring already recording it and #511's measurement showing it reachable on every space-joined run #436 produced; and a text-only read of the delimiter cores inside `revise`, a second rule for a case no round-trip reaches. Out of scope and left as is: `ParsedName.replace(suffix="MD, PhD")` renders 'MD,, PhD', `replace` whitespace-splitting by contract and the facade setters riding on it for v1 parity. +- 2026-09-27 (Derek), #544 — THE NAME-WORD COUNT READS A RUN. The single-token rule this section records for the ambiguous class generalizes to a post-comma part of two or more words, every one suffix vocabulary or a class member (listed or by shape), at least one a member and none a single-letter roman numeral: behind two or more name words the part is the credential run, and the flip reports once over the whole part (`John Smith, Ed Ma`, `Jane Doe, MS LAc`). A title/suffix dual opening the part counts as suffix vocabulary there, and a run whose every member is listed and leans credential is left to the family-comma path, which already reads it whole. The forks and their measurements are the #544 entry under S2. +- 2026-09-28 (Derek), #544 — A PART READ WHOLLY AS SUFFIXES REPORTS NO NAME READING OF ITS WORDS. group's particle chain runs over every comma segment, and its two emitters — `particle-or-given` when a particle behind a word of both the title and the particle vocabulary chains (since 2.0.0, de264af1) and `suffix-or-name` when the chain takes an ambiguous acronym into the name (#289/#516, 59d8f38a, in no release) — reported inside a TAIL segment, which assign reads wholly as suffixes. Each such report named a reading the parse never made, against rules/A1's "A report names the reading the parse took". A family comma's tail was already silent, since group hands the chain no report list anywhere after a family comma; the suffix comma's tails were not. Measured on the released wheels: `John Smith, Jr., Freiherr von Richthofen` reports `particle-or-given` on 'von', a token in the suffix role, at 2.0.0, 2.1.0, 2.2.0 and 2.3.0; `John Smith, Jr., PhD van Ma` and `John Smith, Jr., PhD Do Ma` report it on 'van' and 'Do' at 2.0.0 and 2.1.0 only. The `suffix-or-name` half reached `John Smith, Jr., PhD Do Ma`, `John Smith, MA, PhD Do Ma` and `John Smith, Jr., PhD van Ma` on master (e10e83b4), and the run rule of the bullet above made it reachable behind ONE comma: `John Smith, PhD Do Ma` carried C1's flip and a second report on 'Ma'. FIXED by scope, not by a new test at the emitter: group passes the chain no report list in a tail segment either, so both emitters go quiet there together, and rules/C2 states the boundary for any part consumed wholly as suffixes. (Mechanism SUPERSEDED by #636, decisions/C1, 2026-10-11: group makes no chain in a tail segment at all, and the part after a suffix comma had reached none since #613, so the report list is withheld only after a family comma.) The maiden channel is a separate parameter and is untouched: a tail segment's reader is NONE, so the maiden walk reports nothing there to begin with. No mechanisms.md entry: this is AMBIGUITY-AT-THE-DECISION-SITE's own contract (a report fires only where the parse chose between live readings) applied to a stage whose reading a later stage overrides for the whole segment. MEASURED 2026-09-28, the tree against the same tree with `None if family_comma else ambiguities` restored in `group()` (the comparator), each parse recorded as its seven fields plus `(kind, [(token text, token role)])` per report, under all three name orders: 0 of the 1441 differential-corpus names move, so the gate has nothing to classify; over a comma grid — the prefixes `John Smith, `, `John Smith, Jr., ` and `Smith, John, ` times every run of one to three words drawn with repetition from {PhD, MA, Ma, Do, van, de, Jr, MEng, Ed, y, i}, each text as written, lowercased and uppercased, deduplicated to 11,049 texts — 360 parses (120 texts, every one of them mixed case) lose one `suffix-or-name` report apiece, every removed report on a token in the suffix role, 0 reports added, 0 field moves. 21 of the 120 texts carry one comma and are the run rule's reach; the other 99 carry two and moved the same way (297 parses) when the same one-line change was applied to master e10e83b4; the `Smith, John, ` prefix moves nothing, being a family comma. The grid holds no word of both the title and the particle vocabulary, so the `particle-or-given` half is witnessed by the case row alone. Pinned by the case rows `a_credential_run_after_the_comma_reports_no_chain_fork` and `a_part_past_the_second_reports_no_particle_fork`; the rows the chain still reports on outside a tail are `the_chain_reports_the_acronym_it_takes` and `titled_particle_chain_survives_a_title_that_is_also_a_particle`. +- 2026-10-01 (Derek), #562 (mechanism and accepted reports superseded by #613, below: the decision now binds the part before any chain runs, so the pair reports nothing) — A PARTICLE CHAIN UNSETTLES A RUN THE CAPITALS SETTLED, AND THE COUNT READS IT. The 2026-09-27 bullet above left a run whose every member is listed and leans credential to the family-comma path, "which already reads it whole". That promise fails wherever two particles stand side by side in the part: group chains them into one particle run (P2), assign reads the part as name text, and P6 attaches the chain to the family — `John Smith, PhD DO DO` read given 'PhD', family 'DO DO John Smith', and `John Smith, DO DO DO` given 'DO', middle 'DO DO'. rules/S2 already said the capitals do not decide a member chained behind another particle ("the run attaches whatever the capitals say (P6)"), so C1's shortcut was resting on a premise S2 denies. Of the two fixes #562 weighed, the one taken narrows the shortcut and leaves S2 as written: a part holding two particles side by side is read by the count, which two name words before the comma flip to the credential run, reported (`suffix-or-name`). The other — letting C1's evidence or S2's credential-in-front company outrank the chain — would have contradicted S2's sentence and P6's `Doe, John van DO` example, so it needed S2 amended rather than a gap filled. The test is ANY two adjacent particles, not a member behind one: `vd` is a particle and an unambiguous suffix word, so `John Smith, PhD vd DO` and `John Smith, MA vd vd` chained and misread the same way, the second with no member behind a particle at all. Segment runs before classify, so it asks classify's own predicate (`_normalize(text) in lexicon.particles`) and only while the run is still settled. The test does not ask whether the family-comma path would actually have misread the part, which it could not without reading ahead to group: where that path did read the part whole — for example a pair opening the part with an unambiguous particle-and-suffix word (`John Smith, vd DO`, `John Smith, VD DO`), a credential that is also a title in front (`John Smith, MD DO DO`), or a credential closed by a period in front (`John Smith, Esq. DO DO`, `John Smith, Jr. DO DO`), the list being by example rather than a census — the count flips the part to the same fields and reports the call, as every flip at this comma does (rules/C1's "A decision either way at this comma is reported"). Those reports are ACCEPTED (Derek, 2026-10-01: none of these is a name anyone would write on purpose, so a report is the right signal): with the capitals no longer settling the run, the call is the count's, and the report says so; `tests/v2/cases.py` pins `John Smith, vd DO`. 1.4.0 read every one of these names as the fix does; 2.0.0 and 2.1.0 read `John Smith, PhD DO DO` as title 'PhD', given 'DO DO', and 2.2.0 and 2.3.0 as the issue describes. MEASURED 2026-10-01 against master 0eadedeb, py3.11, `nameparser.__file__` asserted on each side, each parse compared as its seven fields plus its sorted ambiguity kinds: 0 of the 1453 differential-corpus names move (the two names this change adds are the gate's only movers, under a new fix(#562) rule in the four 2.x ledgers); over tests/v2/test_properties.py's settled grid (5,580 texts) 15 move, 10 of them role moves, and the other 5 (`John Smith, MD DO DO`, `MS`, `Esq.`, `Sr`, `Ms` in front) keep their fields and gain the flip's report; over a wider grid — the prefixes `John Smith, `, `Smith, ` and `Doe, John ` times every run of one to three words drawn with repetition from {PhD, MA, Ma, DO, Do, do, vd, van, Jr, MD, Ms}, 4,389 texts — 30 move, 17 of them role moves, every mover a `John Smith, ` text now reading given 'John', family 'Smith' and the whole part as suffix, and every one reporting `suffix-or-name`. Recompute: check out the parent into a separate worktree, parse each grid in both trees under `PYTHONSAFEPATH=1` with the tree's root first on `sys.path`, and diff. The settled grid's own pin moves with it: tests/v2/test_properties.py's `_SETTLED_COUNT` reads 2,994 where it read 3,024, the ten `_SETTLED_EXCEPTIONS` that pinned #562 are gone, and its two recorded negative controls read 1,142 and 215 — the second had already moved from 751 to 215 with #563, before this change. LEFT OPEN, as #562 asked: `Smith, PhD DO DO` (ONE name word before the comma, so the count keeps the listing form, and P6 attaches the chain: given 'PhD', family 'DO DO Smith'), where no rule states S2's credential-in-front company against a chain; and `John Smith, PhD van der`, whose particles are no members and never reach the run test, reading given 'PhD', family 'van der John Smith' as before. +- 2026-10-01 (Derek) — A CAPITALS-SETTLED RUN OPENED BY A WORD OF THE CLASS REPORTS, AND RULES.MD#C1 NOW SAYS SO. C1 said a run whose every class word is written in capitals "reads whole in silence". That held only behind another credential: a class word OPENING the part is the first word after the comma, whose decision C1 reports either way, so `Doe, MA PhD`, `John Smith, MA MA` and `Smith, MA PhD` read wholly as suffixes and report `suffix-or-name` once, on that word, while `John Smith, PhD MA` and `Smith, PhD MA` are silent. The behavior predates #562 and is kept: nobody repeats `MA` at the end of their name on purpose, so the report is the right signal. Statement corrected, no parse moved. MEASURED 2026-10-01 on master b39c370c, the prefixes `John Smith, ` and `Smith, ` times every run of two or three words drawn with repetition from {MA, BA, ED, DO, JD, PhD, MD, Jr, Esq.} holding at least one class word: all 900 runs opening with a class word report, 895 of them read whole (14 of those hold a #562 particle pair, whose count flip reports over the whole part rather than on the first word) and the other five being `Smith, MA DO DO` and its like, the one-name-word chain #562 left open; of the 560 opening with another credential, 554 read whole in silence, and the other six hold a #562 particle pair — four reading whole and reporting (`John Smith, PhD DO DO`), two being `Smith, PhD DO DO` and `Smith, Jr DO DO`. Recompute: parse that grid and bucket by whether the first word is a class word, whether `ambiguities` is empty, and whether the suffix is the whole part. The same day (Derek) rules/S2's first-slot precedence sentence gained the pointer to that exception, so the next reader does not rediscover it: S2 says the count decides the first slot after a family comma before case, true for one word (`John Smith, MA` flips by the count) and false for a capitals-settled run, where C1's shortcut reads the capitals first and the comma keeps its family reading. And S2's list of reporting slots lost "the segments beyond it": no `suffix-or-name` report lands past the second comma (`Smith, John, MA` and `John Smith, Jr., MA` are silent). It once did: at #530's merge cc78c960 `John Smith, Jr., PhD Do Ma` reported `suffix-or-name` on 'Ma' beside its `comma-structure` flag, from group's chain emitter, which the 2026-09-28 #544 bullet above silenced in a tail. S2 now points at C2 for what such a part does report, which is not only `comma-structure` — an unclosed delimiter there adds `unbalanced-delimiter` (`John Smith, Jr., (Bob`), and a maiden clause standing in it is read as one (`Smith, John, Jr nee Jones MA`, maiden 'Jones MA'). Derek, 2026-10-01: the silence is right. +- 2026-10-01 (Derek), #554 — A WORD OF BOTH THE PARTICLE AND THE SUFFIX VOCABULARY COUNTS IN THE RUN, AND STILL SPEAKS FOR NOTHING. After #544 the run test (`_segment.py`) counts `vd` and `mc` as suffix words, while S2's company (`_pieces._anchors`) refuses to let them speak for the word behind them, an exclusion added within PR #552 after that PR's own anchor pass had brought back a split of `Smith vd Ma, John` into family 'Smith Ma', suffix 'vd'. That split is not only an intermediate state of #552: 2.2.0 and 2.3.0 shipped it (2.0.0 and 2.1.0 read family 'Smith vd Ma'), #530 removed it in this cycle, and #552 kept it removed. The issue framed that as two answers to one question and offered (b): end the run at such a particle, so that one "has a live name reading" predicate decides both. DECIDED (a), the shipped behavior, and no parse moves. They are two questions. Run membership asks what the word IS, and `vd` is unambiguous suffix vocabulary; company asks whether the word SPEAKS FOR the one behind it, and a particle cannot, P2 joining it forward to exactly that word. Both answers follow from the word's vocabulary, and they differ by design rather than by drift; S2's "end the run" is the company's run, not the part C1 counts. The clause reaches exactly `vd` and `mc`: `do`, also in both vocabularies, is in the ambiguous half and so is already a word of C1's class, reporting where these two do not (`John Smith, Jr do` against `John Smith, Jr vd`). Recompute the set: `L = Parser().lexicon; L.particles & (L.suffix_acronyms | L.suffix_words) - L.suffix_acronyms_ambiguous`, and drop the subtraction to see `do` with them. + The evidence is the writing that carries no other signal. Neither reading of `vd Ma` after a comma is realistic, a particle in front of a Chinese family name being as rare as `vd` or `mc` as a credential, so the mixed-case writing is an arbitrary edge case and decides nothing. Written in one case, where S2 has no capitals to read and the vocabulary is all there is, the part reads as the credential run, and that is the reading a reader would take: measured 2026-10-01 on master (fc682e36), `JOHN SMITH, VD MA` and `john smith, vd ma` read given 'JOHN'/'john', family 'SMITH'/'smith', suffix 'VD MA'/'vd ma' and report `suffix-or-name`, as `John Smith, vd Ma` does, and `John Smith, mc Ma` reads the same in every casing. It is also 1.4.0's reading, verified against the released wheel the same day: 1.4.0 read suffix 'vd Ma' and 'VD MA', and 2.0.0 through 2.3.0 read the whole name as the family. (b) would add a particle exception to C1, and to #563's paired-initials test besides, in order to give `JOHN SMITH, VD MA` family 'JOHN SMITH VD MA' with no given name: the worse reading, on exactly the input where nothing else speaks. + #563's site needs nothing of its own. `García Márquez, vd G.J.` reads as `García Márquez, PhD G.J.` does, in every casing: given 'García', family 'Márquez', suffix 'vd G.J.', and neither reports. If a surname split by a credential run in silence is a defect, it is the run's, not the particle's. Open: #573 — uniform-case `vd`/`mc` before a comma and after a family comma (decided 2026-10-02, the P6 section's entry), and the silent mixed-case `Doe, Jane PhD vd Ma` (now #578). Recompute: `parse(s)` with `as_dict()` and `[a.kind.value for a in parse(s).ambiguities]` over each string above and its `.upper()` and `.lower()`. +- 2026-10-01 (Derek), #575 (where the count runs superseded by #613, below) — THE COUNT BEFORE THE COMMA COUNTS A PARTICLE SURNAME ONCE. Every count rules/C1 makes of the part before the comma — v1's "more than one word" for an unambiguous credential, the ambiguous class's name-word count (#289, #544), and assign's two-name-word test for the positional read after a comma followed by no name word (#296/#325) — counted tokens or pieces, so `De La Cruz` was three words and `van der Berg` two (the ambiguous `van` stands as its own piece, P1's fork). This cycle's count had therefore read `De La Cruz, Ed` as a credential comma with no given name and split `van der Berg, MA` into given 'van', family 'der Berg' — an unreleased regression against 2.3.0, which read both as the listing form — and 1.4.0 through 2.3.0 all split `van der Berg, PhD`. DECIDED: a particle run and the one name word it attaches to are one word in all three counts, so a particle surname reads as a one-word surname does. + THE COMMA SETTLES P1's FORK, and that is the decision rather than a side effect. Standing alone, `Van Buren` is given 'Van', family 'Buren' with a `particle-or-given` report, so counting it as one surname makes the comma's reading disagree with the standalone one. Derek's call, after weighing the narrow alternative (count only a part P1 already reads as all surname, i.e. one led by a never-given particle like `de`): the listing form puts a surname before the comma, so the comma is evidence the standalone parse does not have, and the readings it gives are the ones a person takes — `Van Buren, Ed` given 'Ed', family 'Van Buren'; `van der Berg, MA` family 'van der Berg', suffix 'MA' (one name word, so C1 reads the case, and capitals in a mixed-case name make the credential). `Van Johnson, Dr.` moves with it, family 'Van Johnson', title 'Dr.', where 2.2.0 and 2.3.0 read given 'Van' and `tests/v2/pipeline/test_assign.py` had pinned the positional fork; that test now uses `Van Johnson Smith, Dr.`, which still has two name words. + THE PARTICLE REACHES ONE WORD, as P1's fold does, not to the end of the part as P2's chain does: `de Mesnil Jean, Dr.` under a family-first order is family 'de Mesnil', given 'Jean', two name words. The cost, and it is real for an ambiguous particle: a `van`-led part with a word after the surname counts two and keeps the positional read with P1's fork, so `van Buren John, Ed` reads given 'van', middle 'Buren', family 'John' and `van der Berg Smith, PhD` given 'van', family 'der Berg Smith', both unchanged from master. Under the default order a never-given particle loses nothing (P1's fold makes `de Mesnil Jean` all surname anyway). `_vocab.unit_ends` carries both reaches behind a `chain` flag, post_rules' fold reading the full chain as before. + ONLY PARTICLE CHAINS, not mechanisms.md#UNIT-PARTITION's full set — stated there as the Contract's one exception, and in P3's statement, whose "ONE name word wherever another rule counts them" now names C1's count as the exception. A bound given-name pair builds a GIVEN name, and P5 gives up a family word where the name has no other (`abdul Salam` alone is given 'abdul', family 'Salam'), so `abdul Salam, Ed` keeps the credential reading. Whether P3 joins a single-letter connective depends on the words of the whole name — `Carod i Rovira, Josep` joins as four words where `Ortega y Gasset` alone, three, does not — and the count is part of deciding what the whole name is, so segment could reach P3's answer only by copying P3; `Ortega y Gasset, Ed` keeps the credential reading. Approved as "particle chains and connective joins", narrowed to particles when P3's exception surfaced in implementation (the first draft joined `John e Smith, III` into one family name, caught by `tests/test_conjunctions.py`). + A WORD THAT IS ALSO A TITLE IS NO PARTICLE WHERE IT OPENS THE PART. `TITLES ∩ particles` is `{freiherr, st}` and `bound_given_names ∩ particles` is `{abu, أبو, ابو}` (recompute: `L = Parser().lexicon; L.particles & L.titles`, `L.particles & L.bound_given_names`). The first draft counted every particle, and the docs review found it reading `Freiherr von Berg, PhD` as family 'Freiherr von Berg' and `St John, PhD` as family 'St John', where every release reads the title. The first fix then excluded both overlaps in EVERY position, and the review of that fix found two errors in it. Position: inside a surname a title-particle chains as P1 and P2 chain it, and the blanket exclusion brought #575's own defect back for `de St Pierre, Ed`, `De St. Croix, Ed` and `de Abu Bakar, Ed` (family only, no given name, where 2.3.0 read given 'Ed'). And direction: 2.0 through 2.3 read `Abu Bakar, Ed` as given 'Ed', family 'Abu Bakar', so excluding the bound-given particle gave released behavior up rather than keeping it — and Abu Bakar is a common Malay surname. DECIDED: a title-particle is no particle only where it LEADS the part, and a bound given-name particle is a particle. So `Freiherr von Berg, PhD` and `St John, PhD` keep the title, `de St Pierre, Ed` reads given 'Ed', and `Abu Bakar, Ed` reads given 'Ed', family 'Abu Bakar' while `Abu Bakar, PhD` reads family 'Abu Bakar', suffix 'PhD' where 1.4.0 through 2.3.0 read given 'Abu'. The agreement test compares each token in both positions; it cannot see which position a caller passes, so case rows pin each count's flag: `de St Pierre, Ed` segment's, and `St John, Dr.` and `von St Johann, PhD` assign's, one per direction (the review of this round found assign's flag fixed to either constant passing every test, and both mutants now fail). `Freiherr von Berg, Ed` reads family 'Freiherr von Berg', given 'Ed' — 2.0.0's through 2.3.0's reading too, so no change against a release; only this cycle's count had read title 'Freiherr', family 'von Berg', suffix 'Ed'. The surname counts once and the title is no name word, so the count reads the listing form, which keeps a leading title in the family as it already did for `Prof. Cruz, Ed` (superseded for listed titles by #H6, 2026-10-17: the count is unchanged, and the listed title then peels). For `Prof.` that is a cost (rules/C1's Accepted line); for a German rank it is the right reading (Derek, 2026-10-01): since 1919 a former noble title is part of the legal surname, written between the given name and the particle (Karl-Theodor Freiherr von und zu Guttenberg), so `Freiherr von Berg` is the family name. That is also why `freiherr` is in PARTICLES at all — added with the German prefixes in 3e14ea20 (#18) without a stated reason, and the reason is this one. `Freiherr von Berg, PhD` keeps title 'Freiherr', the unambiguous credential's count taking the title as a word; both are readings of one name, and neither is changed here. A title in front is otherwise outside the fix: v1's count for an unambiguous credential counts the title as a word, so `Dr. van der Berg, PhD` reads as `Dr. van der Berg` does alone, given 'van'. + `De La Cruz, M.J. K.L.` reads given 'M.J.', suffix 'K.L.', reporting twice, as `Cruz, M.J. K.L.` does (Derek, 2026-10-01): with one name word before the comma the paired-initials count no longer reaches it, and C1's two-dotted-groups sentence now says "behind two or more name words". #563 had read it as a flipped credential run with no given name. + SEGMENT RUNS BEFORE CLASSIFY, so its count builds the two facts it reads (particle, suffix) from the vocabulary (`_vocab.surname_unit_tags`), while assign derives them from classify's tags (`_vocab.surname_unit_facts`) over every token of the part, so a suffix word stops a particle in both: the first draft's assign count dropped suffix pieces before walking and read `van Jr. Berg, Mr.` as family 'van Berg' (the code review). `test_classify.test_surname_unit_tags_agree_with_classify` sweeps every single-word vocabulary entry in three casings; its first run caught `JD.CPA` and `Msc.Ed.`, which classify tags as suffixes through the period-joined derivation, now mirrored, and that pair is the recorded negative control (`_SURNAME_UNIT_CONTROL`, asserted with the mirror patched out). + BLAST RADIUS, measured 2026-10-01: the gate exits 0 at all five baselines, and of the 1459 names in master's corpora exactly one moves, `De La Cruz, M.J. K.L.` (above, decided). Comparator: `parse(n).as_dict()`, the ambiguity kinds and `initials()` for every name in every `tools/differential/corpus*.jsonl` at `git archive origin/master`, under all three name orders, on master's tree against this one. The population that could move is small and the corpus is evidence about itself, not the rule: two corpus names have a part before the comma that is one particle surname of several words, with the part after the comma holding a credential — that one and `De La Cruz, M.J. PhD`, which keeps given 'M.J.'. Recompute: corpus names whose part before the first comma has more than one token and `_vocab.surname_unit_count` 1 (24 at master, nearly all `de la Vega, Juan`-type listings), then keep those whose second segment holds a word `_vocab.ambiguous_class_candidate` admits (a listed or dotted member of the credential class) — 2 at master. A filter on any suffix word keeps 17, the `de la Vega, Juan … III` listings among them. Against 2.3.0, `fix(#575)` classifies one name, `van der Berg, PhD`; the other particle-surname example names diff there only by this cycle's #289 count and report. Outside the corpora the move against 2.3.0 is a class, not a list: a leading ambiguous particle and one word, before a comma followed by an unambiguous credential or a title alone, now reads as one surname (`Abu Bakar, PhD`, `bin Laden, PhD`, `Mac Donald, PhD`, `van Gogh, Jr.`, `Van Johnson, Dr.`), where 2.3.0 read the particle as the given name; `Freiherr von Berg, Ed` and `Abu Bakar, Ed` move only against this cycle's master. The boundary case rows added in review carry no shape tag on purpose: their diffs against the older baselines come from earlier changes (#296's positional read, #289's count), so admitting them to the contract corpus would have stretched unrelated ledger rules over them; the case table asserts them either way. + COST, measured 2026-10-01 by `sys.setprofile` call counts (mean of 50, after one warm-up) against `git archive origin/master`: `Smith, John` 183 → 183, `John Smith, MA` 252 → 252, `John Smith, PhD` 209 → 217, `Dr. Juan de la Vega III` (the benchmark reference) 366 → 366. The +8 is the unambiguous credential path building its token list and asking each token whether it is a particle; a part with no particle never builds the units, and a frame-free approximation of `_normalize` was declined as a second spelling of it. +- 2026-10-01 (Derek), #564 — AN UNLISTED ALL-CAPS WORD IS A CREDENTIAL IN THE PART AFTER A COMMA BEHIND TWO NAME WORDS, BY DEFAULT. The decision, its two-letter rule and the mixed-run membership are recorded under decisions/S2 (#564), where the caps shape's switch has always been decided; this bullet is C1's pointer to it. +- 2026-10-03 (Derek), #549 — A DECLARED DELIMITER PARTS A TRAILING SUFFIX PART AS A COMMA WOULD, AND THIS REVERSES #538'S SKIP READING (M2, 2026-09-26). The question #549 asked was whether a core a connective join swallows should be skipped (the text read as if the core were absent) or should end the run it stands in; Derek's ground for the second was the one a caller has when declaring one: a delimiter set to ` - ` is meant to behave like a comma around suffixes, and the simplest parser that honours that is one where it IS a comma there. So `group` cuts a tail segment at its cores before grouping anything, groups each part as the comma twin's own segment would be grouped, and drops the cores, which post_rules' entry pass still reads back as boundaries. No join, maiden walk or link search can reach across a core because none of them ever sees one, and the machinery that let them step over it is gone: the `cores` parameter of `_group_segment`, `_maiden_take`, `_run_neighbours` and `_link_joins_inside_the_clause`, `_maiden_take`'s `seen` index list, the `skip` arguments of `peel_walk` and `trailing_start`, and the #206 drop block that ran after the joins. One call per parse fewer (`tools/perf/call_count.py`, py3.11, 2026-10-03: parse 369 against 370 at 061f02da). + WHAT MOVES, against 1.4.0 and against 2.3.0. A connective beside a core no longer joins across it: `HumanName("Smith, John, PhD - and MD", suffix_delimiter=" - ").suffix` is 'PhD, and MD', and `... Puig - y Soler` 'Puig, y Soler' -- 1.4.0's answers (its `expand_suffix_delimiter` split the part's text before anything joined), where 2.0.0, 2.1.0, 2.2.0 and 2.3.0 each gave 'PhD - and MD' and 'Puig - y Soler' (all measured 2026-10-03 with the released wheels). The generational link moves back too: 2.3.0 read `Smith, John, Puig - i Soler` as 'Puig, i Soler' and `Smith, John, PhD née Puig - i Soler` as maiden 'Puig', suffix 'PhD, i Soler', `i` being no connective there, and #397/#538 had moved them to 'Puig - i Soler' and maiden 'Puig i Soler' before either shipped; both read as 2.3.0 did again. An interior core still parts: `... PhD née Puig Dr. i - y Soler` gives suffix 'PhD i, y Soler'. A MAIDEN MARKER BESIDE A CORE TAKES NO CLAUSE, and this is the larger cost by count: with the core straight before the marker (`Smith, John, MD - née Jones Smith`) or straight after it (`Smith, John, PhD née - Jones`), the comma twin's marker has no name word ahead of it in its part, or nothing behind it, and M2 declines, so the parse reads suffix 'MD, née Jones Smith' and 'PhD née, Jones' with no maiden, where 2.0.0 through 2.3.0 read maiden 'Jones Smith' and 'Jones' (measured 2026-10-03; 1.4.0 had no maiden markers and gave the suffix reading). It is the one rules/C1 records as Accepted. At the default policy nothing moves, `extra_suffix_delimiters` being empty: the gate exits 0 at all five baselines. + THE DECLINED READING, TAKEN. #538 declined the boundary reading because it moves `Smith, John, PhD née Puig - i Soler` from maiden 'Puig i Soler' to maiden 'Puig' with suffix 'PhD, i Soler', "a birth-name link pushed into the credentials". That is what the same text with a comma typed in the core's place reads, and it is what 2.3.0 read. The marker beside a core (WHAT MOVES, above) is the same trade at a higher price, a maiden name lost rather than a link moved, and it is taken on the same ground; the review of this change found it, the first draft of this entry having weighed only the link shape. The comma reading is accepted as the cost of one rule for a declared separator, where the skip reading kept two (a lone core dropped like a comma, a core beside a connective stepped over like a word) and #549 was the seam between them. + INVARIANT, pinned by `tests/v2/test_properties.py::test_a_delimiter_core_in_a_tail_reads_as_its_comma_twin`: every ROLE field of a name with a ` - ` core in a part after the suffix comma equals that of the same text with a comma in its place, over 129 generated texts; 102 disagreed at 061f02da, 0 after. It replaces #538's `test_a_delimiter_core_reads_as_if_it_were_not_written`, whose invariant this decision reverses. Scoped to TRAILING parts on purpose: before the first comma, or in the given part of the listing form, the core is a word (C1's Accepted entry, v1 parity), and a comma typed there changes the name's structure rather than parting a suffix run, so the twin is a different parse. The ambiguity reports are not part of it: C2 reports `comma-structure` once per extra comma part, so a typed comma can add a report the core does not (`Smith, John, Puig - Puig` against `Smith, John, Puig, Puig`). + MEASURED 2026-10-03, branch against 061f02da. POPULATION: each head in {`Smith, John,`, `Smith, John, Jr.,`, `John Smith,`, `John Smith`} followed by every 2-, 3- and 4-word product of {PhD, MD, Puig, i, y, -, Jr., Soler, Mr., née, and} that contains `-`, 19,972 texts, plus the 1,505 distinct names of `tools/differential/corpus*.jsonl` at 061f02da. Under `extra_suffix_delimiters=(" - ",)` 2,624 of the generated parses move (suffix alone 2,384, suffix and maiden 240 -- every one of the 240 a maiden name lost to a marker beside a core: 100 with `née -`, 108 with `- née`, 32 with both); under `(" / ",)`, with every `-` word rewritten to `/`, the same 2,624 (2,384 and 240), the move being a property of the core's position rather than its spelling; and under `(" and ",)` over the texts as written, where `and` is now the core and the dash a word, 2,528 (2,468 and 60). No other field and no ambiguity report moves under any of the three. In the corpus, one name moves under each of two policies: `Smith, John, PhD née Puig - i Soler` under ` - ` (above), and `Doe, Jane, and Jr.` under ` and `, suffix 'and Jr.' to 'Jr.', a leading core dropped as any lone core is. COMMA-TWIN AGREEMENT over the generated texts under ` - `, comparing the seven role fields of each against the text with ` - ` replaced by `, ` under the default policy and keeping only texts whose core is neither first nor last after the head nor beside another core: `Smith, John,` 752 of 2,100 disagreed before and 0 after, `Smith, John, Jr.,` 752 of 2,100 and 0; `John Smith,` 1,956 of 2,100 both before and after, the head where the typed comma moves the name's structure. (A first draft counted 973 of 3,310 for the `Jr.,` head, its filter reading `Jr.,` as part of the body; the review of this change caught it.) RECOMPUTE: parse each text under each policy at both trees (pin `nameparser.__file__` on each side, AGENTS.md's two-tree gotcha), compare the seven role fields and the sorted ambiguity kinds, and count the texts whose record differs; for the twin, compare the role fields alone. + SUPERSEDES, under M2: the 2026-09-26 #538 entry (its stepping, its invariant and its Not-repaired list, which this closes); the 2026-09-22 #397 follow-up's bound argument (a core can no longer stand between a marker and the clause's first word, the segment being cut at it first); and the #418 reorder entry's closing repair, which screened a tail's cores out of the marker pass so that `Smith, John, PhD née - Jones` read maiden 'Jones' -- it reads suffix 'PhD née, Jones' now, as above. Entries there that describe the walk stepping over a tail's cores (the #424 and #533 bullets) describe machinery this change deletes; they are dated and left as written. Not changed: a core OUTSIDE a tail segment is still a word (C1's Accepted entry), and a core inside a token (`RN/CRNA` under `/`) still reads by `splits_into_suffixes` and keeps the token whole. + +- 2026-10-04 (Derek), #603 (decided at group's head since #613, below) — A CREDENTIAL OPENING THE PART AFTER THE COMMA MAKES IT THE POSTNOMINAL PART, WHATEVER FOLLOWS; AND A TITLE WORD PAST THE SECOND COMMA IS A TITLE. One word that is neither suffix nor title vocabulary used to flip the whole comma reading: `John Smith, PhD Jones` read family 'John Smith', given 'PhD', where `John Smith, MD FACS` and `Eric H. Holder, Jr. Attorney General` were already read as a name and its postnominals. Now the first suffix word of the part, with only titles in front of it, opens the part when it starts S2's run (`_pieces.starts_a_credential_run`, #602's own predicate, so the two runs cannot disagree about what starts one), and every word after it reads as that run reads: a title word as a title, any other as a suffix. The mechanism is the one #296 built for a part holding no name word, extended rather than added beside: `segment` still decides FAMILY_COMMA, and `segment_suffix_reading` returns a reading instead of None, so assign's no-name path reads the part before the comma positionally where it holds two name words. Every word the opened part takes that would otherwise have made it the given part is reported (`suffix-or-name`), as #602's run reports what it absorbs. + THE ONE-WORD PATH, DECIDED (Derek): the whole part is suffixes. `Doe, PhD Jones` reads family 'Doe', suffix 'PhD Jones', no given name, as `Smith, PhD` already reads. DECLINED: reading the credential as a suffix and the next name word as the given name (`Smith, PhD Jones` → given 'Jones'), proposed by analogy with a title at the head of the given part. Asked whether that is a real format, it is not -- nobody writes `Family, Credential Given` on purpose -- and the one realistic input it served, a misplaced generation (`Smith, Jr. John` for `Smith Jr., John`), is garbage in, garbage out. One reading for both comma paths was the deciding ground: the one-word path disagreeing with the full-name path is the shape #429, #430 and #432 each found. Accepted cost: `Smith, Jr. John` loses 'John' to the suffix, where 2.3 read given 'John' with 'Jr.' misfiled as a title. + A TITLE/SUFFIX DUAL OPENS NOTHING (Derek). A word in both vocabularies (`md`, `ms`, `sr`, the ranks `lt`, `cpl` and the rest) is the listing form's title in front of a given name, `Smith, Ms Jane`, and a credential behind it opens nothing either (`Smith, Ms PhD Jones` keeps given 'PhD', as `Smith, MD PhD Ma` keeps given 'Ma' under #544). DECLINED after building it: a dual opening the part behind two or more words before the comma. Derek first chose it for `John Smith, MD Jones`; the reviews found what the count cannot see -- a surname of two words (`García Márquez, Ms Gabriela`, `Smith Jones, Sr. Maria`), a title in front of a one-word surname (`Dr. Smith, Ms Jane`, the count taking the title as a word), and spaced initials (`García Márquez, Ms G. J.`) each lost the given name -- and with the word explained (a dual is a word in both lists, decided by position), Derek chose that duals never open. So `John Smith, MD Jones` and the `MD - PhD - FACS` delimiter rows keep their 2.3 reading. An ambiguous-class member opens nothing either (`John Smith, Ma Jones` keeps given 'Ma'), nor does a suffix word S2's run start refuses (`John Smith, vd Jones`). + STRICT MODE: the veto refuses an initial-shaped word the suffix READING, not the run. `Smith, PSM I.` under `lenient_comma_suffixes=False` reads suffix 'PSM I.' (2.3: given 'PSM', suffix 'I.'), reporting the absorbed 'I.', which matches #602's no-comma run (`John Smith PhD V.` reports; its given-part run does not, a disagreement #602 left and this does not touch). + THE SPELLING GAP THIS FOUND, FIXED HERE. `starts_a_credential_run` refused any piece longer than one token, so the split credential `Ph. D.` group merges started neither #602's run nor this one while `PhD` started both: `Smith, John Ph. D. Jones` read middle 'Jones' where `Smith, John PhD Jones` read suffix 'PhD Jones'. The merged piece now starts the run, the "suffix" piece tag being the Ph./D. merge's alone; `trailing_candidates` had already admitted it, so its pre-check and the predicate now agree. The no-comma spelling `John Smith Ph. D. Jones` still reads family 'Jones': the trailing peel's walk never holds the merged piece, which is a separate path and is left (AGENTS.md's spelling sweep). + C2, WIDENED (Derek): a word of the title vocabulary in a part past the second comma that is not also suffix vocabulary is a title -- the part stands behind a comma, the line H5 draws for reading a title from the end of a name -- and a piece holding a suffix word stays a suffix, a dual and a connective join that took one (`PhD - and MD`) alike. `Eric H. Holder, Jr., Attorney General` reads title 'Attorney General', and a part of title and suffix words is recognized rather than flagged. The flag's test is asked word by word against the title vocabulary, a period-joined word through `period_joined_vocab` as classify tags it (`John Smith, Jr., Lt.Gov.`, which the code review found still flagged), so `Secretary of State` reads its title but keeps `comma-structure`: knowing that group's title chain takes the connective would be `segment` modelling `group`, the sign #611 is about. Every release through 2.3.0 read these as suffixes, 1.4.0 included (`Eric H. Holder, Jr., Attorney General` suffix 'Jr., Attorney General', `Smith, John, Prof.` suffix 'Prof.', measured with the released wheels 2026-10-04); a first draft of this entry said 1.4.0 read them as titles, inferring it from the 1.4.0 run reporting no unexplained diff, when the reason was the broad `fix(comma-family)` rule there claiming them, which #603's two rules now precede by declaration. Accepted consequence: a noble rank there reads as a title, `John Smith, Jr., Freiherr von Richthofen` → title 'Freiherr', suffix 'Jr., von Richthofen'. + WHAT MOVES, measured 2026-10-04 against master at 97d36e0d, after the dual decision above. The differential corpora at 97d36e0d (1,519 name and order entries over 1,515 distinct names): four names, every one intended -- the dash-delimited `Steven Hardman, RN - CRNA` now reads given 'Steven', family 'Hardman', suffix 'RN - CRNA', and `John, Smith, Dr.`, `Andrew Perkins, Jr., Col. (Ret)` and `1 & 2, 3 4 5, Mr.` read their tail title. A generated grid: every name of `tools/differential/corpus*.jsonl` at 97d36e0d, bare-string and object lines alike, plus 120,000 one-to-seven-word texts drawn with `random.Random(601)` from the 54 words {Jane, Doe, Smith, John, nee, née, geb., z domu, PhD, MA, Ma, MD, Jr., Jr, III, V, VI, i, y, de, de la, van, vd, mc, do, Do, DO, abd, abdul, Dr., Prof., King., Le, Attorney, General, Jones, `,`, ba, R.A.I., (nee, Smith), Ó, binti, Ph., D., Chief, Justice, Ms, Sr., RN, -, G.J., MJ, Col.}, plus each of the heads {Jane Doe, Doe,, Smith, John, John Smith, Dr., Mai Le, Jane van der Berg} followed by every three-word product of {nee Smith, PhD, MA, Prof., de, Jr., Jones, VI, do, i Soler, nee}; 98,630 texts, each parsed under the default policy, `name_order` family-first and a ` - ` delimiter, the seven role fields, the ambiguity kinds and details and `initials()` compared against 97d36e0d. 589 texts move: 581 have a part after a comma whose first non-title word is a suffix word that is not also a title, or a title word past the second comma, or both; one, `Jr. i Ó VI Attorney, Ph.`, is the two-name-piece count reading a connective-joined piece beside a suffix word as the family (title 'Ph.', family 'Jr. Ó VI Attorney', suffix 'i', where master read title 'Jr. Ph.', given 'i'); the other seven were read by hand -- a leading period-shape title (`geb.`) in front of the opening credential, and #602's given-part run started by `Ph. D.` (`, abdul Ph. D. Do binti Smith`). Frames per parse through `Parser().parse`, py3.11: the reference band and plain names unchanged (`Dr. Juan Q. Xavier de la Vega III` 449, `Smith, John` 179, `John Smith, Jr.` 210, `Smith, John, Jr.` 223, `Smith, Ms Jane` 256); the positional count gets cheaper, leaving in C for a single piece and building its name set in a loop rather than a comprehension (`Smith, Jr.` 163 → 156, `Smith, MD PhD` 260 → 253, `John Smith, Dr.` 261 → 260); and the opened part pays for what it reads (`John Smith, PhD Jones` 338 → 361). Measured after the /simplify round, which also made the count ask for two name PIECES as well as two units (`Vega y Lopez Jr., PhD Jones` had lost its family). +- 2026-10-04 #611 — C1, S2 AND P6 GREW BY PATCHING, AND EACH SHOWS THE DECISIVE SIGN; THE RETHINKS ARE FILED, NOT DONE. The question was docs/design/AGENTS.md's, asked of the three longest rules after M2 (#601): does a guard model what another stage would do? C1: `segment` predicts group's particle chain (the #562 pair test) and assign's capitals reading (the skip #562 broke), mirrors classify's tags in its unit counts, and decides the structure in two stages, assign re-deciding #296's positional read with a second count -- the copy #603 extended. S2: the words-to-spare count runs over pieces group merges, so the chain re-asks the peel and rolls back (`John van Mc`), P5 compares two views, group emits S2's report, and the peel runs up to five times a parse. P6: assign holds back exactly the particle tail post_rules will attach (`particle_tail(..., sticky_from)`). Decided: #603 shipped as a targeted change inside today's machinery; the rethinks are #613 (C1, read the comma once after classify and bind, P6 folded in) and #614 (S2, read the trailing run once over units and bind), C1 first because S2's unit count should be shared with it, neither for 2.4. +- 2026-10-06 (Derek), #613 — C1 IS DECIDED ONCE, AFTER CLASSIFY, AND THE PART IT DECIDES IS BOUND BEFORE ANY JOIN. #611 found the decisive sign in C1: segment decided the comma structure before any word was tagged, so it modelled what later stages would do -- group's particle chain unsettling a run the capitals settled (#562's pair), assign's reading of a capitals-settled run -- and assign re-decided #296's positional read with a second name-word count. The decision now runs at the head of group (`_comma.decide`), once classify has tagged every word: the part after the first comma is read ONCE by `segment_suffix_reading`, the reading a part holding no name word gets, with the ambiguous words the count licenses passed in, and a postnominal part's SUFFIX and TITLE roles are bound there, so no later stage can read it differently (mechanisms.md#READ-WITHOUT-THEN-BIND, its second instance after M2's #601). segment hands every comma form over as FAMILY_COMMA and keeps C2's flag; script_segment, which runs between, asks its own narrower question (`_postnominal_behind_a_whole_name`, the vocabulary-only suffix test behind two surname units, its peeled tail excluded). Deleted with it: segment's decision block and #564's contrast (moved into `_comma`), classify's comma-run caps branch, assign's comma reading, positional count and second report, `_vocab.run_word_fold`, and the settled-run mirror test in `test_properties.py`, whose whole job was to hold segment's model of group equal to group -- with one decider there is no model left to hold. + DECIDED (Derek, 2026-10-06), each measured before the code was written: (1) a CONNECTIVE surname counts its words before the comma, as #575 decided for every surname but a particle chain -- `Ortega y Gasset, Dr.` reads title 'Dr.', given 'Ortega', middle 'y', family 'Gasset' (b547c247: family 'Ortega y Gasset'), and PR #615's Vega rows, which had pinned the connective as one unit, reverse; (2) nothing after the decision unsettles what it bound, so #562's pair report goes silent (`John Smith, PhD vd DO` keeps suffix 'PhD vd DO' and loses `suffix-or-name`) and a part of credentials holding particles side by side is the credential run behind any name (`De La Cruz, DO DO DO` reads suffix 'DO DO DO', b547c247 given 'DO', middle 'DO DO'); (3) no particle run, connective join or bound given name reaches into a part read as postnominal -- `John Smith, Ph. D. and Mary Jones` reads suffix 'Ph. D. and Mary Jones' (b547c247: family 'John Smith', given 'Ph. D. and Mary', middle 'Jones'), `De La Cruz, MA DO Mr.` reads suffix 'MA DO', title 'Mr.' (b547c247: given 'MA', middle 'DO Mr.', the chain having taken the title), and the one join kept inside the part is P3's connective binding title words into one title (`John Smith, Mr. and Mrs.` keeps title 'Mr. and Mrs.', `_comma._title_chains`; the first draft lost it, and the corpus now carries the name); (4) the positional read of a name before a suffix comma reports as the comma-less name does (`Freiherr von Berg, DO` gains the chain's `particle-or-given`, as `Freiherr von Berg` reports), and a part before the comma that is one name word with a suffix in it is the listing form's family rather than a positional read: `Smith Jr., Esq.` reads family 'Smith', suffix 'Jr., Esq.', and `Smith Jr., V` given 'V', family 'Smith', suffix 'Jr.', as `Smith Jr., John` reads (b547c247: given 'Smith' with `given-or-family`, and given 'Smith', suffix 'Jr., V'). + ACCEPTED WITH THEM, found in measurement or review rather than decided ahead. A connective between a credential and a title joins nothing, since it takes its kind from the credential in front: `John Smith, PhD and Dr.` reads suffix 'PhD and', title 'Dr.' with a report (b547c247: given 'PhD and Dr.'), garbage either way, pinned as the cost of (3). The split credential written around another word is not merged, merging being for adjacent halves: `John Smith, Ph. Ma D.` reads title 'Ph.', given 'Ma', middle 'D.', family 'John Smith' (b547c247: suffix 'Ph. Ma D.'), garbage too. A LISTED member written with a period reads as it does without one behind a whole name, the count licensing it before the lone-piece route's period inference can (`John Smith, Ma.` suffix 'Ma.' as `John Smith, Ma`; b547c247 title 'Ma.'), and `John Smith, PhD Ma.` reports the count's flip over the part where b547c247 reported the anchored pick, as `John Smith, PhD Ma` always did. The garbage `Ph. D., Jr.` reads family 'Ph. D.' (b547c247: title 'Ph.', family 'D.'), its watched row re-recorded in compare.py. + THE CJK RADAR TIER, settled in review and then by Derek. script_segment runs before group, so it cannot read decide's answer, and it needs one: whether the part after the comma is the postnominal part of a whole name decides whether the name before it divides and where a glued honorific peels from. The first draft asked a hand copy of C1 there, by the vocabulary alone, and the copy drifted from decide in four review rounds -- the ambiguous class's count (`김민준 박, MA` read as a credential run behind a two-word name while the name stayed whole), decide's numeral refusal and policy-selected suffix test (`김민준 박, MA V` divided behind a family comma, family '김 민준 박'), a suffix before the comma counted as a name word (`김민준 Jr., PhD` divided, the division then making decide agree), and the split `Ph. D.` (`김민준 Ph. D., PhD`) -- each drift dividing a name decide read as a family comma, or leaving whole one it read as a suffix comma. DECIDED (Derek, 2026-10-06, the /simplify pass): script_segment asks `_comma.decide` itself, on a classified copy of the name as written, and only where the answer can change something (the peel once its two candidate sites differ, the division once there is a token to divide), so no Latin or accented-Latin comma name pays for it. The division now follows decide in both directions (`test_the_division_never_outruns_the_comma_decision`: on 3320d700, the last tree with the copy, 0 of its 300 parses divided behind a family comma and 44 stayed whole behind a suffix comma; now 0 and 0). Against b547c247 that restores every reading the copy had moved (`Smith 김민준씨, XYZ`, `John 田中さん, XYZ`, `김민준 박, X.Y.Z.` divide and peel as master read them), and moves one class master read otherwise: a title and one CJK name word before a credential the capitals license, which decide reads as a suffix comma -- `Dr 김민준, MA` reads title 'Dr', given '민준', family '김', as `Dr 김민준` does (b547c247: family '김민준', the division declined and H1 making the one word the family), and the pre-existing tolerated row `Dr. 김민준씨, MA` moves with it; so does a glued honorific in a part decide reads as postnominal, `John Smith, Dr. J.씨` reading title 'Dr.', given 'John', family 'Smith', suffix 'J.씨' as `John Smith, J.씨` always did (b547c247 peeled the 씨 from the part and read given 'J.', family 'John Smith'). Measured on the /simplify pass's 78,282-parse fingerprint: 1,106 parses move, every one a CJK name before the comma or a `J.씨` after it. `마틴 킹, MA` reads as master did; its report counts three name words, decide reading the divided name. `_vocab.surname_unit_count`, which only the copy asked, is deleted. + DELIMITER CORES are dropped only behind a whole name, which is C1 Accepted's v1 parity (`Smith, MD - PhD - FACS` keeps its cores): the first draft dropped them behind a one-word family too. + WHAT MOVES, measured 2026-10-06 against b547c247 (PR #615's merge), py3.11, with the review round's fixes in. The harness: every differential corpus name at b547c247 plus a comma grid -- for each head in `Smith`, `John Smith`, `van der Berg`, `García Márquez`, `De La Cruz`, `Dr. Smith`, `John Q. Smith`, `SMITH`, `JOHN SMITH`, `Smith Jr.`, `Vega y Lopez`, `LLOYD WEBBER`, `abdul Salam`, `john smith` and `Freiherr von Berg`, the forms `H, A`, `H, A, Jr.` and `H, John A` for every A in `PhD MD Jr. III V V. I MA Ma ma Ed DO Do vd de X.Y.Z. XYZ LEED AP G.J. MJ Ms Mr. Dr. Prof. Jones John Q. Ph. D. Esq. RN - nee i y Sr. and MEng 씨`, `H, A1 A2` for every ordered pair of those, `H, C1 C2 C3` and `H, C1, C2` over `PhD MA Ma DO vd G.J. MJ Ms Mr. Jones V XYZ Jr. Ph. D.` -- 80,423 distinct texts, each parsed under the default policy, `lenient_comma_suffixes=False` and a family-first order; comparator the role fields, the ambiguity kinds and details, and `initials()`. 3,631 texts move, 1,773 with a role. Under the default policy the 1,739 role movers fall in the classes above and nowhere else, sorted by the first of these tests a text meets, in this order (the classifier is shape-only, so a later class can hold texts an earlier one would have taken): `Ph.` and `D.` in the part with a word between them, 31 (ACCEPTED); a connective `y` or `i` in the part before the comma, 936 (1); a part before the comma ending in a suffix word, 456 (4); a connective `and`, `y` or `i` in the part after it, 65 (3); a particle (`do vd mc de van`, any case) in the part after it, 250 (2, 3); and `Ph. D., Jr.`. The report-only movers, compared IN ORDER: 1,177 gain `particle-or-given`, every one the `Freiherr von Berg` head (4); 166 lose `suffix-or-name` (2's pairs) and 4 lose `given-or-family` (`Smith Jr., Jr., Mr.` and its kin, 4); 182 keep the same reports in a new order, `comma-structure` now written before the comma's `suffix-or-name` because segment emits the former and the decision runs after it (report order is no contract, and nothing pins it); 266 change a kind or a detail, the shapes being the first-word report taking the place of the count's flip report on a part a class word opens (`García Márquez, DO DO`), the absorbed-word report no longer naming a particle the chain had joined to the word (`De La Cruz, PhD DO D.` reports 'D.' where it reported 'DO D.'), and the count's flip taking the place of an anchored pick (`John Smith, PhD Ma.`); 63 move under a non-default policy alone, 34 of them in a role. The corpora at b547c247 (1,526 distinct names): four move -- `John Smith, PhD DO DO` and `John Smith, PhD vd DO` lose (2)'s report, `Ph. D., Jr.` as above, and `마틴 킹, MA` in its report detail alone. The gate exits 0 at all five baselines; intentional diffs 631 → 632 at 1.4.0 and 285 → 286 at 2.3.0, the corpus having gained `Ortega y Gasset, Dr.` and `John Smith, Mr. and Mrs.` as C1 examples, and radar unclassified gains `Ph. D., Jr.` at both. + COST, `sys.setprofile` call counts after one warm-up, b547c247 → the tree with this entry's P6 fold and review fixes in (the P6 bullet under P6 gives the fold's own share): `John Smith` 151 → 151, `Smith, John` 179 → 160, `Smith, John Q.` 246 → 230, `Smith, Jr.` 156 → 156, `Smith, John, Jr.` 223 → 204, `John Smith, MA` 249 → 253, `John Smith, Dr.` 260 → 212, `Smith, MD PhD` 253 → 185, `John Smith, PhD Jones` 361 → 262, `Doe, Jane nee Smith` 309 → 293, the reference parse `Dr. Juan Q. Xavier de la Vega III` 449 → 449. A no-comma name never enters `decide`, decide returns before C1's counts where the part holds a name word and nothing of the ambiguous class (guarded by `test_a_listing_comma_with_nothing_to_decide_skips_the_counts`), and group filters its segments for bound roles only when `decide` bound something. The paired-initials speaker scan came back quadratic in the move (`is_suffix_piece` 120 → 1,620 calls for 4× the input on `'John Smith, ' + 'MD '*k + 'G.J. '*k`) while #563's guard, which counted `_normalize`, passed; the scan now finds the speaker once, and the guard counts what the scan calls. + THE PR REVIEW, after the two rounds above. An EMPTY part before the comma had decide return early, leaving the part to the listing walk: `, PhD` read given 'PhD', `, Jr.` title 'Jr.', `, MD PhD` title 'MD', family 'PhD', where b547c247 and 2.3.0 read suffix -- decide now reads the part behind an empty head, which names no whole name, so the comma stays the family comma with the part bound (the heads of every grid above were non-empty, so none saw it). And three more costs the frame guards could not see, each now a clock row in `_PREFIXED_SHAPES` (base 1600, broken 9.9-11.9 against fixed 4.0-4.3): `_title_chains` rebuilding the growing title from a slice at each join, the paired-initials test copying the shape set per pair and scanning a list of the class members, and the core filter scanning a list of the cores. And one render view: decide cleared its caps-shape marks whenever C1's run test declined, so a part #603's opener bound anyway kept its all-caps word without them, and case repair wrote `John Smith, PhD Jones XYZ` as '... PhD Jones Xyz' (rules/R4); the marks now stay on every bound caps word that carries the name's contrast, roles and reports unmoved over the harness (`test_a_bound_caps_word_keeps_its_capitals_in_case_repair`). + SUPERSEDES: the 2026-10-01 #562 bullet's mechanism and its accepted reports (its pair report and the settled-run test); the 2026-10-04 #603 bullet's "segment decides" wording; and the #575 bullet's account of WHY segment builds its unit facts from vocabulary -- `_comma._whole_name` counts over classify's tags, and `name_word_count`, which reads the vocabulary, is asked by decide and by script_segment, the agreement sweep `test_surname_unit_tags_agree_with_classify` keeping the vocabulary view equal to the tags. +- 2026-10-06 #617 — CROSS-REFERENCE: the title join decision (3) above makes inside a part read as postnominal is no longer `_comma._title_chains` but group's own loop, `_pieces.join_connectives`, and it now merges a connective run first (`Smith, Chancellor of the Exchequer MP` reads suffix 'MP') and freezes the generational class; decided and measured in decisions/P3's 2026-10-06 #617 entry, which amends (3). +- 2026-10-11 (Derek), #636 — NO PARTICLE CHAIN IS MADE IN A PART THAT HOLDS NO NAME WORD. A part past the second comma, after either comma, is read wholly as suffixes, its title words as titles (C2, #603), but group still ran the particle chain over it, which builds a name unit no reading uses (the part after a suffix comma, which C2 covers too, had reached no chain since #613, `_comma.decide` binding it and emptying its segment before any join; a suffix-comma grid moves nothing here -- `{head} {w1 … wk}` for k from 1 to 3 over the heads `John Smith, Jr.`, `John Smith, PhD`, `, PhD`, `John Smith, MD Ma`, `John Smith, Jr. MA`, `Smith, PhD`, `, Jr.`, `John Smith, X.Y.Z.` and the words `van der de la von und St Freiherr do mc vd du Dr. Dr Prof. Prof Attorney Sir Lt.Gov. Mr. MA Ma PhD X.Y.Z. VI V Jr. Berg Jones`, 606,216 parses under the three orders, a sample reaching both comma structures): a particle past the part's first word took the words behind it, a title word among them, so `John Smith, Jr., van der Berg Prof.` read suffix 'Jr., van der Berg Prof.' where `John Smith, Jr., Prof.` and `John Smith, Jr., van Berg Prof.` (whose leading `van` chains nothing, P4) read title 'Prof.'. Now group makes no chain in such a part, and asks no chain bound there, so C2 reads every title word in it, bare (`Prof`) or mid-part (`van der Prof. Berg`) alike. DECIDED (Derek): no chain there, rather than the issue's proposal, #623's bound (`trailing_run_start`) at this site; that reads trailing titles only (H5), narrower than C2's rule, and adds a call where removing the chain takes one away. The chain's two report emitters had been handed no report list in these parts for the same reason (rules/C2's no-name-reading clause, 2026-09-28); with no chain there is nothing to suppress, and the handoff now suppresses reports only after a family comma (removing the tail half moved nothing over the suite, #624's comma-tail grid's 292,950 parses, every corpus name and case text, and the suffix-comma grid above). + MEASURED 2026-10-11 against 3b2b6f0d (#635's merge), py3.11, `nameparser.__file__` asserted on each side. #624's comma-tail grid (its recipe, decisions/P2, 2026-10-10; 97,650 texts under the three orders), with every corpus name and case text at 3b2b6f0d: 18,318 parses on 6,106 texts move a role, every one in the grid and none of 3b2b6f0d's corpus names or case texts, and no parse moves a report alone. 15,330 put a title word that stood behind a particle run into the title field (`,, 'Bob' van der Attorney` → title 'Attorney', suffix 'van der'; master suffix 'van der Attorney'), and 2,988 take a particle back out of a title the chain had joined it to, where the chain merged it with a P3-joined title and kept the tag (`,, 'Bob' van der Mr. and Mrs.` → title 'Mr. and Mrs.', suffix 'van der'; master title 'der Mr. and Mrs.', suffix 'van'). The S2 stress grid, #624's title-particle grid, #623's given-part grid and #625's particle grid: 0. The five gates exit 0, a `fix(#636)` rule in each 2.x ledger; at 1.4.0 the broad `fix(comma-family) lone post-comma piece routes to suffix/title` rule explains the new example. Frames: the reference rows and family-comma names do not move; a tail part with a particle costs less (`John Smith, Jr., Freiherr von Richthofen` 440 → 428, `John Smith, Jr., van der Berg Prof.` 462 → 452, py3.11 warm). Recorded negative control: with the chain run in these parts again, 9 tests fail -- the three `fix(#636)` rows on both surfaces, the two new rules/C2 examples, and `a_part_past_the_second_reports_no_particle_fork`, whose chain report the handoff no longer suppresses, so that row guards the change too -- and with it run only after a family comma, the review's rows `Smith, John, van der Berg Prof.` and `,, van der Berg Prof.` (both surfaces), which it added after that mutant passed the suite. + THE REVIEW (2026-10-11), measured, all garbage-neutral or better, no reading worse over the review's generated tail-part sweeps under thirteen configurations (fields, reports, `initials()`, `capitalized()` and the facade's views compared; the populations were the reviewer's and are not recorded here, so no count is quoted): a word in both the title and particle vocabularies inside a tail part now reads as a title as a plain title there already did (`John Smith, Jr., Berg St Smith` → title 'St'); a particle between two titles there becomes a suffix entry of its own (`John Smith, Jr., St von Mr. and Mrs.` → title 'St Mr. and Mrs.', suffix 'Jr., von'). Two gaps outside this change, already on master: P5's bound join still runs in a tail part after a suffix comma (`John Smith, Jr., abdul Dr. Smith` keeps 'Dr.' in the suffix where `John Smith, Jr., Berg Dr. Smith` reads title 'Dr.'), and rules/C2's no-name-reading clause is contradicted for the part after a suffix comma by #603's per-word report (`John Smith, PhD van der Berg` reports suffix-or-name on each word). + Open: #640, #641. (Both settled 2026-10-11: #640 by #637's entry, decisions/P5; #641 below.) +- 2026-10-11 (Derek), #641 — C2'S NO-NAME-READING CLAUSE MEANS A WORD READ AS A NAME, AND C1'S PER-WORD REPORT IS THE PART'S OWN. rules/C2 said a part the parse consumes wholly as suffixes "raises no report about reading a word of it as a name", whether it is the part after a suffix comma or a part beyond the second; but the part after a suffix comma reports each word its opening credential takes (`John Smith, PhD van der Berg` → suffix-or-name on 'van', 'der' and 'Berg', "follows a credential, so it reads as part of the suffix run; it may be a name word"), the report rules/S2 states ("A name word the run absorbs is reported `suffix-or-name`"), which C1's credential-opening branch reaches by reading the words after it as that run reads them, and which C1's example pins (`John Smith, PhD Jones` → `suffix-or-name`). The rule text and the code had disagreed since #603 added the report (in `assign`; #613 later moved it into `_comma.decide`), and no test held either side's wording. DECIDED (Derek): the report stays -- a word behind a credential in a part the comma made postnominal is the fork it reports in `John Smith PhD Jones` -- and C2's sentence is narrowed to what it meant: no report that a word of such a part was READ AS a name (a particle chained onto a name word, an acronym taken into the name). The part's own reports are listed by their rules: C1's report of the decision at its comma (the flip, or the first word after the comma), S2's report on each word a credential in front of it reads as a suffix although it may be a name word (a name word the run absorbs, a member the company decides), and C2's flag. Measured by the review over about 1.2 million parses of suffix-comma and tail parts: those are the only reports such a part carries. A text change only: no reading or report moves (the suite and the five gates unchanged), and the one code comment quoting the old sentence quotes the new. + diff --git a/docs/design/decisions/D1-segmenterless-activation-warning.md b/docs/design/decisions/D1-segmenterless-activation-warning.md new file mode 100644 index 00000000..0eabaa0c --- /dev/null +++ b/docs/design/decisions/D1-segmenterless-activation-warning.md @@ -0,0 +1,6 @@ +# D1 — the segmenterless-activation warning + +- 2026-08 #337 — WARN rather than raise, deliberately: registering the JA pack without the extra must stay safe (the inert registration is itself a pinned property), and a warning is filterable by the caller who wants exactly that inertness. The Japanese install hint is conditional — it appears only when a Japanese script is among the dead scripts; a hangul-only gap gets the generic remedies. +- 2026-08 #337 — the warning exists because parser_for(locales.JA) without segmenter= used to build a parser that silently behaved like a working one minus the feature; it re-emits from the parser_for frame so the reported location is the caller's own call. +- 2026-08-07 #339 — diagnostics that hand the reader code must hand code that type-checks: the message's offered deactivation used a bare tuple literal for segment_scripts — an arg-type error under mypy in a py.typed package — and became frozenset(), pinned by a test asserting the offered spelling. The known-bad spelling is in the denylist test. The structural fact behind the recurrence: autodoc renders docstrings into the API reference, so docs/*.rst is NOT the boundary of "the docs" — a guide and the reference can teach different spellings on the same rendered page, invisibly to any .rst-only sweep. + diff --git a/docs/design/decisions/D2-construction-raises-parse-never-does.md b/docs/design/decisions/D2-construction-raises-parse-never-does.md new file mode 100644 index 00000000..2f0539e1 --- /dev/null +++ b/docs/design/decisions/D2-construction-raises-parse-never-does.md @@ -0,0 +1,9 @@ +# D2 — construction raises, parse never does + +- 2026-07 (v2 core, PR #288) — every raise in the locale-apply path is a plain TypeError/ValueError so the wrap-with-locale-code rewrap cannot break on exotic exception signatures. + +Declined (ambiguity kinds for script-resolved names, 2026-07-27): + +- A "han-script" zh-vs-ja kind — applying the ZH pack IS the disambiguation; per-name flags after an explicit opt-in are noise. +- ORDER emission for script-resolved names — native-script order is convention, not a guess; the reserved kind stays unemitted. SEGMENTATION fires only on a genuine multi-split vocabulary fork (夏侯惇, 남궁민수). + diff --git a/docs/design/decisions/H1-title-and-one-name-word.md b/docs/design/decisions/H1-title-and-one-name-word.md new file mode 100644 index 00000000..80de1a0a --- /dev/null +++ b/docs/design/decisions/H1-title-and-one-name-word.md @@ -0,0 +1,20 @@ +# H1 — a title and one name word + +- 2026-08-25 (#410) — H1's "and nothing else" counted a suffix, a nickname and a maiden name as further name words, so a title-plus-surname name reported no family the moment any of them stood beside it: `Dr. Smith` reads family 'Smith' and `Dr. Smith née Jones` read given 'Smith' with the family empty. #410 reports the maiden flavor; measured, all three roles suppress the rule identically and the fix is one term in one guard, so the term went rather than the maiden role being special-cased. What decided the width is H1's own rationale — a title addresses by surname — which says nothing about what stands BESIDE the name. The rationale is stated for H1 rather than as a doc-wide principle about what a suffix is, deliberately: N3 counts a suffix the other way (`'Smitty' Jones Jr.` reads given 'Jones', family ''), and which of the two readings is right for a nickname-led name is not decided here. #399 is what made this urgent rather than latent: stopping the particle chain at the marker routed the canonical title-and-particle shape (`Freiherr von Richthofen geb. Albrecht`) into H1 for the first time. N3 moves with it, without N3 changing: `'Smitty' Dr. Jones` declines N3's one-piece count as it always did, and H1 now names the family behind it, so it reads family 'Jones' where it read given 'Jones' through 2.1. Five corpus names change reading at every baseline, of which four arrive as new diffs and are classified with the fix. The fifth, `Freiherr von Richthofen V`, was already classified under the `fix(#424)` rule: #410 narrows its diff from {given, family, suffix} to {family, suffix}, and `classify()` takes the first rule whose declared `fields` are a SUPERSET of the observed diff, so a shrinking diff kept matching and no run ever named it. A real movement behind a green gate, and the second of that kind here: the fields-only `fix(suffix-routing)` catch-all was absorbing three names at 1.4.0 the same way, its heading growing with nothing to announce it. Both are one shape — a rule broader than the diff it explains — and that is the lesson worth keeping rather than either fix. It is also, being a title, one name word and a suffix, the suffix flavor of this very shape, which two ledger comments had claimed no corpus carries. At 1.4.0 three of the four were being absorbed by the fields-only `fix(suffix-routing)` rule (its heading went 14 -> 17) — invisible to `_CORPUS_CLAIMS`, which records the whole corpus for a rule with no `name_regex`, so the guard could not see them arrive. They have their own rule now, as #372's names got one off the same catch-all. The v1 suite shipped the nickname reading as a strict xfail (`tests/test_nicknames.py::test_nickname_and_last_name_with_title`), which passes now. Known gap between the statement and the guard: H1 says "exactly one name word" and "that word", but the guard never counts units. It tests which ROLES are unoccupied — no middle, no family — and then retags every given token, so a name whose given run is two name WORDS (units, not tokens) reaches it and moves both: `Freiherr de V Jr` reads family 'de V'. The distinction matters for reading the claim that follows: a multi-token given run is ordinary — the rules.md example `Freiherr von Richthofen V` has one, its two tokens being a single particle unit — and it is a run of two UNITS that no real input produces. A 58,338-name sweep found no corpus name and no rules.md example with one — but it is recorded because decisions.md elsewhere leans on H1's "exactly one name word" as a settled scope when narrowing P5's licence, and that reading of H1 is stronger than the code behind it. Not fixed here: the widening this entry records is about which roles disqualify the rule, not about how it counts. + +- 2026-09-08 [#489](https://github.com/derek73/python-nameparser/issues/489) (the #316/#489 bundle) — a title RUN addresses as its LAST title does. `Her Majesty Queen Elizabeth` read family `Elizabeth` because the `given_name_titles` lookup was keyed on the whole run, and `her majesty queen` is no shipped entry. A run is written as several titles and is addressed the way its final one is, so the run's folded key is now matched whole OR by its last word. **This AMENDS the 2026-08-22 #369 entry under #P5**, which keyed both sites on the whole run so they could never disagree; the amendment keys both on the LAST word and the invariant is untouched — H1's retag and P5's licence still cannot read one run two ways, through one predicate beside `_title_key`. What the amendment reopens is that entry's EXAMPLE and not its reasoning: `'mr sir'` was #369's own instance of a run neither site treats as a given-name title, and it now IS one to both, `sir` being the last word. + The whole-run arm is kept for a caller's multi-word phrase entry — `lt col` is stored and matched as one key — and its order carries nothing: the `or` short-circuits but decides nothing, since for a ONE-WORD run the key IS its last word. Every shipped given-name title is a single word, asserted in tests/v2/test_lexicon.py rather than counted here, so under the default vocabulary that arm can only ever match a one-word run, which the last-word arm reads the same way. Recompute the single-word property with `L = Parser().lexicon; sorted(t for t in L.given_name_titles if " " in t)`, which gives `[]` on 2026-09-08; the set holds 47 entries that day, and the argument survives the digit moving. + Measured over the corpus glob as it stood at the fix: ONE name moves, `Her Majesty Queen Elizabeth`. TWO after the docs commit that follows, which puts `Dr. Sir John` in corpus_rules.jsonl as an H1 example — re-measured there with the predicate's last-word arm dropped to a whole-run key, so the diff read forwards is this amendment's. Probes that move with it: `Her Majesty Queen Elizabeth II`, `Reverend Mother Teresa` (`mother` is a given-name title), `Dr. Sir John`, `Mr Sir John`, `Xyz. Sir John` — H2's unlisted word in the run does not change the run's last word — and, through P5's licence, `Sir Sheikh abdul rahman`, which reads given `abdul rahman` with an EMPTY FAMILY, the Accepted outcome H1 already records for `Sir John`. + VOCABULARY SCOPE, recorded so the boundary is not read as a gap this bundle left. `prince`, `princess`, `lord` and `lady` are not given-name titles, so `His Excellency Lord Duncan` still reads family `Duncan` and `Her Royal Highness Princess Anne` family `Anne`; `king`, `queen` and `mother` are. Whether the first four should join is a vocabulary question with its own frequency argument — it would move `Prince Harry` and `Lady Gaga` — and is deliberately out of this bundle, filed as [#519](https://github.com/derek73/python-nameparser/issues/519). Recompute the membership with `sorted(Parser().lexicon.given_name_titles)`. + Frame delta measured at zero on both entry points: each site's read sits inside the branch it serves, and the reference name `Dr. Juan0000 de la Vega III` enters neither — H1's guard wants an unoccupied family, P5's a bound given-name word. + TRAILING REACH, added 2026-09-09 in review of the docs commit. H1's site asks which ROLES are unoccupied and never where the title stands, so a run that #H5 chained in BEHIND the one name word decides that word's field exactly as a run in front of it does. Measured on this tree: `Smith Sir.` reads given `Smith` with an empty family and `Smith Queen.` reads given `Smith`, both runs ending in a given-name title, while `Smith Dr.` reads family `Smith`. No code moved for this — the reading has been the shipped one since the trailing walk landed — but the STATEMENT said "a title followed by exactly one name word" and described only the front slot, so what the review changed is the statement, an example line (`"Smith Sir." → given="Smith"`, which puts the name in corpus_rules.jsonl), and the `interacts:` lines, H1 gaining H3 and H5 and H5 gaining H1. The trailing half is where H1 and H5 meet: H5 decides which words leave the name, H1 decides the field of the one left standing. + +- 2026-09-09 (the #316/#489 bundle, review round 2) — WHICH run addresses, where a name carries a title run at BOTH ends. The TRAILING REACH paragraph above is right about the trailing slot and was incomplete about the front one: H1 collected every `Role.TITLE` token into ONE run, and with #316(a) shipped that run can span the whole name. `Sir John Prof.` keyed `sir prof`, whose last word `prof` is no given-name title, so it read title `Sir Prof.`, family `John` where `Sir John` reads given `John` — the title written behind the name changing what the title in front of it addressed by. `Queen Elizabeth Prof.` lost its given name the same way, and `Dr. Smith Sir.` moved in the other direction, the trailing `sir` making the leading `dr` address by given name and turning family `Smith` into given `Smith`. + THE RULE: the run standing BEFORE the one name word is the run that addresses; a run standing behind it decides only when none stands before. The split is at the NAME WORD, and the first fix split at the first token of ANOTHER ROLE instead, which a nickname written in front of the titles satisfies: `'Smitty' Sir Jones Prof.` then found no leading run and read family `Jones` where `'Smitty' Sir Jones` reads given, and `Dr. 'Smitty' Sir John` lost the run it had — the same defect one nickname away, caught by the design review's two-input invariant. H1's own rationale is what decides it, saying in as many words that what stands beside the name word does not decide this reading; so a run written around a nickname is ONE run and `Dr. 'Smitty' Sir John` reads given `John` as `Dr. Sir John` does. That is #H5's transparency principle applied to the one-word name — `X Prof.` is `X` plus a title, so adding the title cannot change how the words in front of it are read — and it leaves the trailing slot's own reading untouched, `Smith Sir.` still reading given `Smith` because no run stands in front of it. rules/H1's statement carries the ordering and gains two examples, `"Sir John Prof." → given="John"` and `"Dr. Smith Sir." → family="Smith"`; #H5's one-name-word clause is conditioned the same way. + FOUND BY THE PR REVIEW'S P5 FINDING, and that is the part worth keeping. The reserve fix under #P5 was correct on its own terms and left `Sir abdul rahman Prof.` reading family `abdul rahman`: the join fired for the right reason and the field was still wrong. What the review then asked was which run each site reads, and the answer was that they read DIFFERENT runs — group's licence has always read only the pieces ahead of the bound word (`range(fk)`), while H1 read every title token — so the 2026-08-22 #369 invariant under #P5, that H1 and P5 cannot read one run two ways, had been broken by the composite keying without any site's own tests noticing. Both now read the leading run, through the same predicate, and `nameparser/_pipeline/_group.py` records that beside the licence. + MEASURED 2026-09-09, over every distinct name in `tools/differential/corpus*.jsonl`, parsed on the fix's parent (`e489dc1`) and on the fix and diffed across the seven fields and the ambiguity kinds: over the corpus AS THE PARENT HAD IT, not one name moves; over the corpus as this commit leaves it, exactly TWO do and they are the two rules.md examples this entry adds (`Sir John Prof.`, `Dr. Smith Sir.`). The composite shape needs a title run at BOTH ends and no corpus name had one before those two, which is why a defect this plain sat behind four green gates. `Dr. King Sir.` is untouched and stays #H4's: its leading run is `dr king`, whose last word `king` IS a given-name title, so the word the peel left standing reads given `Sir.` with `title-or-name` reported. The end-to-end invariant is `tests/v2/test_parser.py::test_the_p5_licence_and_h1_read_a_title_run_the_same_way`, whose trailing-title half now requires `f"{title} abdul rahman Prof."` and `f"{title} John Prof."` to give the same given and family as the spellings without the trailing title; restoring the whole-`titles` key fails six of its eleven rows, which is the mutation that proves it discriminates. It carries the nickname spellings as well, and those are the only rows that separate the two splits above. Frame delta zero on both entry points: the run is built inside H1's guard, after the role counts have short-circuited, and the reference name `Dr. Juan0000 de la Vega III` has a family and never enters it. + +- 2026-09-10 [#519](https://github.com/derek73/python-nameparser/issues/519) — `prince` and `princess` join the given-name titles; `lord` and `lady` do not. This RESOLVES the VOCABULARY SCOPE paragraph of the 2026-09-08 #489 bullet above, which is left as written: it named the four as not given-name titles and filed the question, and two of the four have now moved. The criterion is rules.md H Background's — membership follows how the title addresses — and `prince` and `princess` address by the given name (`Prince Harry`, `Princess Anne`), with no surname reading of the word behind it. The clause is for the two words and not for a royal class: TITLES holds `emperor`, `tsar`, `sultan`, `pharaoh`, `archduke`, `maharani` and their pairs, none of them weighed here, and `Emperor Akihito` still reads family `Akihito`. `lord` and `lady` split the same way as each other, by the bearer's rank, which the text does not carry: the given-name form is a courtesy style for children of the senior ranks alone — younger sons of dukes and marquesses (`Lord Peter`), daughters of dukes, marquesses and earls (`Lady Diana`) — and every peer and every wife takes the title or surname (`Lord Byron`, `Lady Thatcher`). [Debrett's "The trouble with titles"](https://debretts.com/the-trouble-with-titles/) and [Bratman's peerage primer](https://www.dbratman.net/nobility.html), both read 2026-09-10, agree on that boundary, Debrett's naming `Lady Jane Debrett` for a baronet's wife as the typical error. The issue gave `lord` the surname reading outright, hedged with a "mostly"; the hedge is this class. The set has no way to say "sometimes" — the `venerable` reasoning under #indic-honorifics — so Derek's call was to leave both out, decided rather than deferred, and the issue's proposed frequency test on `lady` was not run. Both are pinned as negative controls (`Lord Byron`, `Lady Gaga`) beside `His Excellency Lord Duncan`. + Both read sites move, through the one predicate: H1's fold reads `Prince Harry` given `Harry` and `Her Royal Highness Princess Anne` given `Anne` (the #489 bundle's negative control, flipped), and P5's licence reads `Prince abdul Rahman` given `abdul Rahman` with an empty family, as `Sir abdul Rahman` does. What membership does NOT decide is whether the word is a title at all: `Prince Fielder` reads given `Fielder` now and read family `Fielder` before, wrong both ways, and that is [#348](https://github.com/derek73/python-nameparser/issues/348)'s collision. + Measured 2026-09-10 over every name in `tools/differential/corpus*.jsonl` (1308 rows), parsed on this tree and on the same tree with a Lexicon whose `given_name_titles` lacks the two words, diffed across the seven fields and the ambiguity kinds: exactly TWO names move, `Prince Charles` and `Prince Fielder`, both radar-tier in corpus_issues.jsonl, both `{family, given}`. THREE after the docs commit, which puts `Her Royal Highness Princess Anne` in corpus_rules.jsonl as an H1 example. `Prince of Wales Jr` does not move — the joined run addresses by `wales`. Recompute by parsing the corpus glob twice with the two Lexicons and diffing. TITLES membership is unchanged, TITLES being the union. + diff --git a/docs/design/decisions/H2-leading-abbreviation-title.md b/docs/design/decisions/H2-leading-abbreviation-title.md new file mode 100644 index 00000000..69f0fba0 --- /dev/null +++ b/docs/design/decisions/H2-leading-abbreviation-title.md @@ -0,0 +1,10 @@ +# H2 — the leading-abbreviation title + +- 2026-06-30 (leading-period-title design; v2 core, PR #288) — the shape test is v1 parity (period_abbreviation): two-plus letters then a period, bare initials exempt. Its site is the head of the part CARRYING THE GIVEN NAME — the whole name, or the post-comma part under a family comma — not "the head of the name"; that scope correction is PR #315 (2026-08-01, docs-only), verified against 1.4.0 from PyPI, so the parity claim is real and the narrower description never was. The extraction litmus (2026-08-15): the spec drafted this rule as + "recognized by vocabulary, not by written shape" and the live + parser falsified that framing — the shape heuristic is real, and what the abugida gap (#343/#344) shows is its LIMIT, not its absence. Recorded as the rule's Accepted consequence — and the 2026-08-15 landing initially re-narrowed the scope to "name- opening", correcting one error while preserving another; the eighth review round fixed it. + +Decided 2026-09-08 (was Open: [#316](https://github.com/derek73/python-nameparser/issues/316), what a trailing title-vocabulary word should do while the comma paths disagreed): the trailing slot reads VOCABULARY, not shape (#H5), and the leading inference here stays unconditional — #316's open question 2, a symmetric leading rule that would read "Esq. Smith" as a suffix, is DECLINED. #109 shipped the leading inference on purpose, the family comma already carries the credential case ("Smith, Esq." → suffix, this rule's own Accepted clause), and reversing S2's "a suffix never opens the string" for period-marked words for the sake of one word is the wrong trade. The two comma paths agree now: "John Smith Prof." and "Smith, Prof." both read title Prof. See #H5. + +- 2026-09-10 — the shape declines a word carrying a script with no initials (#323); the argument, the halfwidth-kana limit and the measurements are in #cjk-full-stops. That limit was closed on 2026-10-03 by #594, which classifies halfwidth katakana (decisions/W4). + diff --git a/docs/design/decisions/H3-title-runs-floor.md b/docs/design/decisions/H3-title-runs-floor.md new file mode 100644 index 00000000..6cf949c2 --- /dev/null +++ b/docs/design/decisions/H3-title-runs-floor.md @@ -0,0 +1,10 @@ +# H3 — the title run's floor + +- 2026-09-08 (the #316/#489 bundle) — the leading title run leaves a name word standing, and a post-nominal is not one. Derek's question is what opened it, verbatim: "Why does `Dr King Jr` need to parse `king` differently than `Dr King`? Jr is a recognized suffix, so it could not count as a following name." The mechanism behind the old reading is an ORDER: the leading peel runs before the trailing suffix peel and its only floor was "a title needs a following piece", so it took `Dr King` whole and left `Jr` to be the name — title `Dr King`, family `Jr`, no suffix at all. The floor added here is a second one, asked of the run's last word: where everything behind the run is a suffix piece and that word is not itself suffix vocabulary, the run gives it back. `Dr King Jr` now reads title `Dr`, family `King`, suffix `Jr` — what v1 wanted (#v1-xfail-triage), what rules/S2's descriptive note predicted, and what the comma spelling `King, Dr Jr` has always given. +- Measured over the differential corpus glob as it stood at the fix — 1123 distinct names, 1263 rows, before this bundle's own rules.md examples entered corpus_rules.jsonl and made it 1136 and 1284 — tree against the same tree with the floor removed: exactly one name changes, `Dr King Jr`. FOUR after the docs commit, which puts `Dr. King MD`, `Dr Jr` and `Sir Jr` in corpus_rules.jsonl as examples of the floor and its edge (THREE from 2026-09-09, the /simplify round having taken `Sir Jr` out of rules.md — see #H5 — where `Dr Jr` pins the same reading) — re-measured there, and `Prince of Wales Jr` enters with them and does NOT move, which is the one-word gate witnessed in the corpus. The case row `Dr. King MD` moves the same way. Recompute by parsing every name in `tools/differential/corpus*.jsonl` twice, once on the tree and once with `_pieces.leading_titles` monkeypatched back to the bare "a title needs a following piece" loop, and diffing the seven name fields plus `ambiguities`; 1.4.0 gives the degenerate reading for both names, so the change diffs at every baseline. +- The word given back must be a NAME CANDIDATE, and that is what leaves the degenerate inputs alone. All-suffix inputs are untouched because the run's last word is itself suffix vocabulary — `MD DDS` keeps title `MD`, family `DDS`, and `Jr. Ph. D.` keeps title `Jr.`, suffix `Ph. D.` — and all-title inputs are untouched because there is no rest for the floor to read: `Dr.`, `Marquess of Bath` and `Coach` are unchanged. +- ONE WORD is the gate, and it is a Task 1 review outcome rather than the drafting's shape. A joined title unit given back would lose the title entirely, which is worse than the reading it replaces: `Prince of Wales Jr` keeps title `Prince of Wales`, family `Jr` rather than becoming given `Prince of Wales` with no title at all. `Lord Chancellor Jr`, whose run is two separate words, does move — title `Lord`, family `Chancellor`, suffix `Jr`. +- The edge accepted: where the run's whole content is the word given back, no title is left to make the reading H1's, so the word stands as the NAME rather than as the family. `Dr Jr` reads given `Dr`, suffix `Jr` and `Sir Jr` given `Sir`, suffix `Jr`, both reporting `title-or-name` by H4's lone-title-word convention (2026-09-09: `Sir Jr` is no longer written down anywhere but this sentence — once the floor empties the run no branch reads `vocab:given-title`, so the two readings are one and `Dr Jr` carries it). That is the residue rules/S2's note is now scoped to. +- Two residuals the floor does not reach, recorded because they look like misses and are not. A run whose LAST word is itself suffix vocabulary is not given back, so `Dr King MD PhD` still reads title `Dr King MD`, family `PhD` — the floor asks about one word, not about the run's contents. And the floor asks `is_suffix_piece`, which vetoes a bare initial-shaped numeral, so `Dr King V` keeps the whole run as the title and reads given `V` (`king` being a given-name title and the run's last word, which is #H1's 2026-09-08 amendment), where `Dr Smith V` reads family `Smith`, suffix `V`. That numeral fork is #401/#421's territory and is deliberately outside this floor. +- The floor lands in `leading_titles` rather than in assign because that predicate is the ONE answer to where the leading run ends (mechanisms.md#ONE-PREDICATE-PER-QUESTION): assign sets the roles from it and group's chain guard reads the same count, so a floor in assign would have given the two sites different runs. One cost measured and one shape forced by it: the branch's entry gate is two inline tag reads rather than `is_suffix_piece`, because `leading_titles` runs four times per parse and asking the authoritative predicate first cost 8 calls per parse of the reference name against a band with room for two (#parse-cost). The tag reads are the cheapest NECESSARY condition for the next piece to be a suffix piece at all, so an ordinary titled name leaves the branch without entering a frame; `is_suffix_piece` stays the predicate that ANSWERS. Frame delta measured at zero on both entry points. + diff --git a/docs/design/decisions/H4-input-that-is-nothing-but-vocabulary-still-has-to-name-somebody.md b/docs/design/decisions/H4-input-that-is-nothing-but-vocabulary-still-has-to-name-somebody.md new file mode 100644 index 00000000..6281c6ff --- /dev/null +++ b/docs/design/decisions/H4-input-that-is-nothing-but-vocabulary-still-has-to-name-somebody.md @@ -0,0 +1,13 @@ +# H4 — an input that is nothing but vocabulary still has to name somebody + +- 2026-09-07 #491 — the reading is unchanged and the silence is what ends. Handed a string the title peel eats down to one last word which is itself title vocabulary, the parser reads that word as the name; decisions/v1-xfail-triage recorded it in the fourth of its NOT FIXED entries — "this is a name parser, not a title parser" — and said in the same breath that what is actually wrong is the guess being silent. It now reports `title-or-name`, with `detail` naming the word that was made into the name. + Six corpus names gain it when this is written: the Queen's Bench string, `Lord Chancellor`, `Dr. King`, `The Rt Hon`, `His Holiness` and `His Holiness the Dalai Lama`. Three read contract-tier and three radar, and the tier split is an artifact of the documentation rather than of the shape — the three contract ones are contract because this bundle made them rules.md examples, which puts them in corpus_rules.jsonl. (2026-09-08, the #316/#489 bundle: the peel half's population was TEN, and the count moved for two different reasons. `Dr King Jr` and `Dr. King MD` gained the report because the title run's floor now leaves a name word standing where the run used to swallow it (#H3), and `Dr Jr` gained it as the same floor's residue; those three also ENTERED the corpus in that commit, as rules.md examples of the floor. `Sir Jr` gained the report too and reads exactly as `Dr Jr` does, but it is NO corpus member and is counted nowhere here: it entered corpus_rules.jsonl in the same commit as a rules.md example and left again on 2026-09-09, when the /simplify round dropped the example as a duplicate of `Dr Jr`'s (#H5). So the peel half is NINE from that round, six contract and three radar. Recompute rather than trusting any of these numbers: parse every name in the `tools/differential/corpus*.jsonl` glob and collect the ones whose `ambiguities` carry the kind, keeping the tier the FIRST file holding the name declares with the contract files sorted first, as `main()` loads them — the README's table is where the tiers are, and sorting the glob plainly instead reads `Dr. King`, `Dr King Jr` and the Queen's Bench string as radar and gives three and six.) `Dr. King` is the one worth arguing about and it is deliberate: `king` is in the titles vocabulary for the addressing forms, which the triage entry above decided and did not reopen, so `Dr. King` IS an input whose last standing word is title vocabulary and the rule claims it. Reporting there is honest rather than noisy — the reading came from a convention, not from anything the input says — and a caller who wants only the exotic cases has `detail` to filter on. + ONE emitter, and it is at assign's lone-name-word site rather than at H1's retag, which is where the drafting put it. H1 is not the site: under a declared family-first order the assignment places the word in the family directly and H1 never runs, so an emitter there would report under one order and not the other for a reading that is the same either way. Measured under the default order, FAMILY_FIRST and FAMILY_FIRST_GIVEN_LAST, the six report identically. That placement is also why the `detail` names no field, unlike O5's: under the default order H1 retags the word after assign, so a field named at the emitter would be the one the word was placed in and not the one it ends in — and the fork the kind reports is title-versus-name, which no field answers either way. + What is silent, all measured 2026-09-08. A lone title word: `Dr.`, `Sir`, `King` and the chained `Prince of Wales` are a title run with nothing behind it, the peel takes the whole string, no word is left standing to be read as a name, and nothing was chosen — mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE's "a branch that runs but changes nothing is not a decision". A title with an ordinary word behind it: `Dr. Smith` and `King Charles` leave a word standing, but not a title-vocabulary one, so H1 alone explains them. And a title followed by post-nominal vocabulary: `Dr King Jr` and `Dr. King MD` peel `Dr King` and `Dr. King` WHOLE, leaving a credential that the bare-suffix carve-out makes the name — a different convention, and its report is scoped to inputs no title stands in (spelled `n == 0` when this was written, one clause of the `field_undecided` predicate since the 2026-09-08 consolidation recorded under O5), so a title in front of the run takes the input out of it and leaves the reading H1's. **Corrected 2026-09-08 (the #316/#489 bundle):** the two example names no longer read that way and the sentence is wrong about them, though the SCOPE it describes is unchanged and still true of the suffix half. The title run's floor gives back a one-word last piece where everything behind the run is post-nominal (#H3), so `Dr King Jr` reads title `Dr`, family `King`, suffix `Jr` and `Dr. King MD` title `Dr.`, family `King`, suffix `MD`. Both now leave a NAME word standing, and both report through this rule's TITLE half rather than being silent — `king` being title vocabulary is what makes them the peel shape. What is still true, and is what the sentence was written to say, is that the SUFFIX half is reached only where no title was peeled first: `MD DDS` has a title peeled and reports nothing, `DDS` being no title. rules/H4's Accepted clause is rewritten to match, and `Dr. King MD` is an example line there now. The peeled titles are never tested for anything: H2 makes an unlisted abbreviation a title by SHAPE, and `Xyz. Smith` is not this input. What the rule turns on is the word left standing. + A refinement of Derek's, made after the population was measured and widening the rule past the all-titles shape it was drafted for: a lone name word that is a JOIN (P3) carrying title vocabulary reports `title-or-name` too, the fork there being whether the title word inside the unit is a title at all rather than which field the unit takes. `John of Prince` and `Smith and Prince` are the measured inputs; no corpus name reaches that branch, because a join LED by a title word is chained into a title run by H3 (`Prince of Wales`), so the two are pinned as case rows rather than as rules.md examples. It sits on O5's branch and takes precedence there, which is why O5's statement says a title silences THIS kind and not every report at the site. + The suffix half is the same argument on the other vocabulary and needed no new kind. An input whose every word is post-nominal vocabulary reads its first word as a name — assign's "everything suffix-shaped after titles: first one is the name" carve-out — and that is the doubt SUFFIX_OR_NAME already names, so it reports that. `Rinpoche` and `QC MP` are the corpus names; `PhD`, `MBA` and `III` are the same shape. `Jr.` alone is NOT the shape at all: H2's opening-abbreviation rule reads it as a title before the suffix vocabulary is consulted, which is that rule's stated precedence and is recorded here because the expectation going in was that `Jr.` alone read as a name. The guard carries two exclusions of its own. A maiden name beside the credential says the input is not post-nominal vocabulary and nothing else, so `abd née Jones` is out for the reason M4 keeps it out of O5's report. And the report is scoped to names no script order placed, so a lone glued CJK honorific — さん, 씨, 선생님 — reports nothing: that is the same shape read through the glued-honorific rules (W2, #271/#308) and the script's own order, and whether those readings should report is left to the arc that revisits them rather than settled here. + An asymmetry on the boundary between this rule and O5, noted and deliberately not fixed. `MA` and `Ma` alone report `given-or-family`; `PhD` alone reports `suffix-or-name`. Both are a bare credential with nothing beside it, and what separates them is which gate reads them: `ma` is an AMBIGUOUS acronym, so S2's gate declines to peel it and the word stands as the one name word, which is O5's branch; `phd` is unambiguous, so it peels to suffix, leaves no name piece, and reaches the bare-suffix carve-out, which is this rule's. Two conventions, two kinds, and the reading each name gets is the same either way — the caller sees a report in both cases and only the kind differs. Fixing it would mean one of the two gates changing what it reads, which moves fields for a bundle that moves none. + Nothing moves but the report, and the rules go in the three 2.x ledgers only — two of them at first and a third after the 2026-09-08 round below, which is where the ten names and the join clause's own rule come from. None is a copy of a wordlist and all three are declared in `_NOT_A_VOCABULARY_COPY`: a member copying TITLES would reach `Dr. Smith`, `Rabbi Cohen` and every other title-plus-surname name, and a member copying the suffix sets would reach `John Smith MD` and `Smith Jr.`, none of which moves. What selects these names is a SHAPE — the peel leaving one unit, and vocabulary standing in it — which no wordlist expresses. The population is EIGHT distinct names when this is written, TWELVE after the #316/#489 bundle and ELEVEN from 2026-09-09, when the /simplify round on the bundle's PR took `Sir Jr` out of rules.md and so out of corpus_rules.jsonl (#H5); the ten this entry says twice above names two OTHER quantities — the ten corpus ROWS the eight names occupy, `Dr. King` and the Queen's Bench string each appearing in two corpus files, and the peel half's ten after the bundle, itself nine from that same round. Recompute all three by parsing every name in the `tools/differential/corpus*.jsonl` glob, collecting the ones whose `ambiguities` carry the kind, and counting rows and distinct names separately (2026-09-09: eight names in ten rows at a0b93f0, eleven names in fourteen rows on this branch, nine of the eleven the peel half and two the join half). + +- 2026-09-08 #518 review round — the join clause is HOISTED beside the peel clause instead of sitting under O5's. It had been written inside O5's `n == 0` leg, so a leading title, a maiden marker or a vocabulary claim on the word silenced it — and every one of those decides which FIELD the unit takes, which is not what this clause asks. Measured before and after on this branch: `Lord Chancellor née Jones` and `Dr. King née Jones` (maiden), `Dr. Smith and Prince` and `Mr. John and King` (a peeled title), `van and Prince` and `J. and Prince` (a claimed word) were all silent and all now report `title-or-name`, with no role moving on any of them. The maiden pair takes the peel half and the other four the join half. TWO CORPUS NAMES MOVE, which the drafting expected to be zero: `Attorney General of Minnesota` and `Deputy Secretary of State`, both a title peeled in front of a joined unit that stands last — and a title needs a following piece, so H3 cannot chain the join into a title run the way it chains `Prince of Wales`. "No corpus name reaches the branch" was true only of the guarded version, and the claim is corrected in rules/H4 as well. They take a third `feat(#491)` ledger rule of their own rather than joining the all-titles alternation, whose issue line describes an all-titles input, which neither of these is. Also this round: the emitter comment's carve-out list no longer names "a group-flagged credential" (no such condition is in the code, and `phd_split` pins the opposite), and the `Order` NamedTuple is `EffectiveOrder` with field `order`, `roles` having been the name of two different things three lines apart. + diff --git a/docs/design/decisions/H5-trailing-title-run.md b/docs/design/decisions/H5-trailing-title-run.md new file mode 100644 index 00000000..44e77187 --- /dev/null +++ b/docs/design/decisions/H5-trailing-title-run.md @@ -0,0 +1,30 @@ +# H5 — the trailing title run + +- 2026-09-08 (the #316/#489 bundle; closes #316(a)) — a trailing run of period-marked title words is a title. `John Smith Prof.` read family `Prof.` and lost the surname while `Smith, Prof.` read title `Prof.`; the two comma paths disagreed and #316 is that question. They agree now. +- **The doctrine, stated rather than left to be inferred.** A period-marked word is claimed by SHAPE at the front and by VOCABULARY at the back. At the front an unlisted abbreviation is a title (#H2) and the shape outranks the vocabulary, which is why `Esq. Smith` reads title though `esq` is post-nominal vocabulary. At the back there is no shape rule, so only a listed title word chains into the title and `John Smith Xyz.` keeps family `Xyz.`. A BARE title word at the back is a name word — `John Smith Sir` and `Mary Jane King` are untouched — TITLES holding ordinary surnames being exactly why, which is the Excluded (TITLES) block in decisions/vocabulary-exclusions ("the trailing-position rule that must NOT be adopted"). The asymmetry is honest rather than an oversight: #109 shipped the leading inference on purpose, and no blanket trailing vocabulary rule is safe while king, judge and bishop are titles. +- **Q3 declined: H2 stays unconditional.** #316's open question 2 asked for the symmetric leading rule, `Esq. Smith` → suffix. Declined: it reverses rules/S2's "a suffix never opens the string" for period-marked words for the sake of one word, and the family comma already carries the credential case (`Smith, Esq.` → suffix). Recorded at #H2, where the `Open: #316` hook was. +- **Q2 decided: a RUN, mirroring #H3.** Successive single words that wear the abbreviation shape AND are title vocabulary chain into the title from the end, floor one name piece, so `John Smith Prof. Dr.` reads title `Prof. Dr.` rather than making `Prof.` a name word. Not chosen: one word only, which would have left `Prof.` standing as a name word in exactly the input the rule is about. The title field joins in INPUT order and the rendering needed no change — the title view joins TITLE tokens in token order already, so `Dr. John Smith Prof.` reads title `Dr. Prof.` for free. +- **The predicate refuses `is_leading_title` and that refusal is the rule.** The walk uses `is_title_piece`, the vocabulary read, shared with the leading run so the two cannot disagree about what a title WORD is while disagreeing, deliberately, about what a title SHAPE is. `is_leading_title` carries H2's unlisted-abbreviation inference; with it, `John Smith Xyz.` would lose its family name to a title. That is the plan's mutation check for this commit. +- **No fork is reported**, which is #316's open question 4. Under the input-is-a-name premise a period-marked title word is not a reading a reader would hesitate over; where the doubt is real it is the word left STANDING that carries it, and #H4 already reports that. No `AmbiguityKind` is added by this bundle and `tests/v2/test_contracts.py::_AMBIGUITY_TRIGGERS` is unchanged. +- **A1, the peel order, decided by Derek: the first peel is PROVISIONAL and a trailing title is TRANSPARENT to the suffix reading.** `X Prof. Y` reads exactly as `X Y` reads, plus the title. SCOPED 2026-09-09, in review of the docs commit: that is a claim about inputs where a name word still stands on both sides of the chain — two or more name words. Where the chain leaves ONE name word there is no second reading for it to be transparent to, and #H1 decides the field instead: `Smith Prof.` reads family `Smith` where `Smith` alone reads given `Smith` and reports `given-or-family`, and `Smith Sir.` reads given `Smith` with an empty family, `sir` being a given-name title. All three measured on this tree; rules/H5's statement carries the scope and `_assign.py`'s excerpt of it was updated in the same edit. The problem is order: with ONE peel, `Prof.` standing behind `Jr.` stopped the suffix peel before `Jr.`, and the title walk then removed the very word that had been blocking it — `John Smith Jr. Prof.` read family `Jr.`, a generational suffix promoted to the family name, which is worse output than the reading it replaced. So the pieces the walk takes are spliced out and ONE peel runs over what stands, in original order, and that second answer alone places a piece or reports a fork. Measured: `John Smith Jr. Prof.` reads suffix `Jr.`; `John Prof. MA` reads the family `MA` that `John MA` reads, S2's reserve keeping a bare ambiguous acronym the family of a two-word name where a second peel laid over a first read family `John`, suffix `MA`; and `John Smith V Prof. VI` reads what `John Smith V VI` reads — middle `Smith V`, family `VI`, nothing reported — where two peels each reporting their own last piece reported twice. The choice is CORPUS-NEUTRAL: both variants move the same eight names, no more and no fewer, so it is a question about output quality alone. +- **A2, the family-comma segment-1 path, decided: it gets the walk.** The spec's condition was "if the comma segment gate already routes those, say so and leave it". Measured, it does not: `Smith, John Prof.` read middle `Prof.` at 1.4.0, 2.0.0, 2.1.0, 2.2.0 and at this branch's parent — all five measured — while `Smith, John Prof. Dr.` read middle `Prof.`, suffix `Dr.` through 2.1.0 and middle `Prof. Dr.` from 2.2.0, `dr` having left the suffix vocabulary in #296. The eleven comma rows the spec calls "already routed" are the `Smith, Prof.` shape, where segment 1 holds NO name word and `segment_suffix_reading` reads it piece by piece — a different gate and a different mechanism. So the segment-1 walk was genuinely missing; with it, `Smith, John Prof.` reads title `Prof.`, given `John`, family `Smith`. It moved NO corpus name at the fix; `Smith, John Prof.` is a rules/H5 example, so it ENTERS the corpus in the docs commit that follows and is one of the seventeen in the population bullet below. Case rows pin it either way. The segment's walk follows the same transparency principle with its own lenient loop, which is why the candidates are the pieces that walk would not read as a suffix rather than the strict suffix test alone (`Smith, John Prof. Jr.` must reach past the post-nominal, `Smith, John Prof. V` past the numeral the lenient tail test claims, #144). +- **What A2 does NOT reach, and why that is right.** `Smith, John Prof. MA` still reads middle `Prof. MA`: after a family comma a bare ambiguous acronym is a MIDDLE name and has been since 2.0 (the `Smith, Ed` cost S2 already accepted), so `MA` stands at the end of the segment, is not a period-marked title word, and stops the walk before it starts. The walk reads from the end; it does not hunt. +- **Measured population: FIVE corpus names at the fix, SEVENTEEN after the docs commit that follows it and the two review rounds on it.** The five are the four planted no-comma rows in corpus_issues.jsonl (`John Smith Dr.`, `John Smith Mr.`, `John Smith Prof.`, `John Smith Rev.`) plus `Andrew Perkins (Mgr.)`, which the drafting did not expect. Eleven of the other twelve are rules.md examples entering corpus_rules.jsonl, eight of them this rule's own — `Smith Prof.`, `Dr. John Smith Prof.`, `John Smith Prof. Dr.`, `John Smith Prof. Jr.`, `John Smith Jr. Prof.`, `John Prof. MA`, `Smith, John Prof.` and `Mary Jane King.`, the last of those added by the first 2026-09-09 review of that commit — plus three of #H1's, which reach this walk because the run each puts behind the name word is what the walk takes: `Smith Sir.`, and `Sir John Prof.` and `Dr. Smith Sir.` from the second review round the same day. The twelfth is `毛 泽东 Dr.`, which entered corpus_cjk_tolerated.jsonl in the FIRST review round and is the one member of the population in a native script: the walk runs before the positional read, so the pieces the script test sees are all Han and the family-first order stands. It is a corpus name the walk moves like any other, and the reason it took its own ledger rule rather than a seventeenth alternative is a guard about alternations, not a difference in the argument (the ledger comment beside it says which). So the growth is documentation rather than reach, and both numbers come off the same recipe — parse every corpus name twice, once on the tree and once with the walk disabled, and diff — run on the two trees. **The disabling stub is NOT the same on the two trees, corrected 2026-09-09 in review of the /simplify round below.** At the fix (`5568fdc`) `trailing_titles` returned the count the chain TAKES and both call sites were in assign, so a stub returning 0 took nothing and disabled the walk. Since the /simplify round it returns the count it LEAVES STANDING, so 0 now takes EVERYTHING and `John Smith` raises IndexError before any measurement happens; the disabling stub on this tree is `lambda rest, *a: len(rest)`, and it has to be bound over both `_pieces.trailing_titles`, which is what `tail_reading` calls, and the name `_assign` imported for the segment-1 walk, patching either alone leaving the other walk live. Re-run that way the two figures hold: FIVE corpus names at `5568fdc`, SEVENTEEN on this branch (2026-09-09). That fifth is a genuine hit rather than a misfire: rules/S1 drops the brackets and reads the content exactly as if written bare, so `(Mgr.)` is a trailing period-marked title word and reads as one; its own test stays green. Recompute by parsing every name in `tools/differential/corpus*.jsonl` on the tree and on the branch's third commit's parent and diffing the seven name fields plus `ambiguities`. Baselines differ per name — `John Smith Dr.` read suffix `Dr.` at 1.4.0, 2.0.0 and 2.1.0 (`dr` left the suffix vocabulary in 2.2, #296) and family `Dr.` at 2.2.0 — so the ledger entries are per-baseline. +- **The inline frame-free gates at the two walk sites were REMOVED in review**, and this is the ONE-PREDICATE-PER-QUESTION half of the entry. Each site had a cheap inline test written to match the walk's own first condition; that is a second implementation of the question, and the measurement that justified it did not survive re-running. The band test runs early in a session where the facade sits at 453, so the "one frame of headroom" claim did not reproduce. With the gates gone the walk costs +1 frame on each entry point, inside the plan's target of two and inside `test_facade_cost_stays_within_its_band`. The walk's own cheapness is where the saving lives instead: the abbreviation-shape test is a compiled regex (a C call, no Python frame) and runs BEFORE the vocabulary call, and almost no name ends in a period-marked word, so the ordinary parse pays one match and stops. +- **A pre-existing detail mismatch, recorded and NOT fixed.** #H4's join shape reports `title-or-name` with a `detail` that says the unit was "read as a given name by convention", while under the default order H1 retags the unit to the family — so `Dr. John of Prince` reports that text with the unit in `family`. This rule adds a second input with the same mismatch, `John of Prince Prof.`, and fixes neither: the wording predates this bundle, the fork the kind reports is title-versus-name which no field answers either way, and #H4 already records why the detail names no field for the peel shape. +- 2026-09-09 (review round 2) — **A1's transparency holds for the ONE-name-word case too, and the scope written a day earlier was the defect rather than the boundary.** A1 was scoped that morning to inputs where a name word still stands on both sides of the chain, handing the one-word case to #H1 unconditionally: "the title behind that word decides its field as a title in front of it would". #H1 was then reading BOTH ends of the name as one title run, so `Sir John Prof.` read family `John` where `Sir John` reads given `John` — the trailing title changing what the leading one addressed by, which is exactly the non-transparency A1 denies. The clause is now conditioned on there being no run in FRONT of the word: where one stands it addresses and the chained trailing title only joins the title field, so `X Prof.` reads as `X` plus the title for one name word as for two. Where no run stands in front the trailing one still decides, unchanged — `Smith Prof.` reads family `Smith`, `Smith Sir.` given `Smith`. Nothing in the walk moved; the fix is entirely in which run H1 keys, and #H1's 2026-09-09 entry carries it, the measurement and the mutation. The #P5 clause below reads the same way afterwards and gains its second half: a bound given-name word behind a trailing title joins as it joins with the title absent AND lands in the same field. Said of the COMMA-LESS writing, and the review that found the H1 defect measured why the qualifier is needed: after a family comma the reserve reads no peel at all, so the join fires over the trailing title word and takes it into the given name — `Berg, abdul Prof.` reads given `abdul Prof.` where `Berg, John Prof.` reads title `Prof.`. That is PARITY (1.4.0 reads it the same, measured) and a gap in the segment path rather than a boundary of the rule, so it is recorded rather than fixed here: the clause carries the scope, and the case row `title_word_trailing_behind_a_bound_pair_after_a_comma` pins it without making it normative. Tracked as part of #316. (FIXED by #637, decisions/P5, 2026-10-11: the join declines the trailing title word, and the row pins the title.) +- 2026-09-09 (the /simplify round on the bundle's PR) — **the trailing peel and the title chain are ONE reading run to a FIXED POINT, and A1's "ONE peel runs over what stands" was a truncation of it.** Two contrived readings contradicted clauses this section states, both found by an altitude review reading the code against the rule rather than by a test. (1) THE SUBTRACTION MODEL. #P5's reserve modelled assign's reading as the peel's count minus the chain's take, on each of the two views the join compares. That arithmetic equals assign's answer only while the chain's take does not change what the peel would do, and at #S2's bare-ambiguous reserve it does: `abdul rahman MA` reads given `abdul`, family `rahman`, suffix `MA` — the join declining because peeling the acronym unjoined and not joined is a suffix reading the join would change — while `abdul rahman MA Prof.` read given `abdul rahman`, family `MA`, the subtraction having counted `MA` a name word to spare. The Accepted clause below, "a bound given-name word behind one joins exactly as it joins with the title absent", was false as stated. (2) THE TRUNCATED FIXED POINT. The re-peel ran ONCE, so a second title was read half way: `John Prof. MA` reads title `Prof.`, given `John`, family `MA`, and `John Prof. MA Prof.` read title `Prof.`, given `John`, family `Prof.`, suffix `MA` — the re-peel un-peeled the acronym and re-exposed the FIRST title, which nothing then took. Both measured on the branch tip, 2026-09-09. +- **The fix is one function, `_pieces.tail_reading`:** peel, chain, splice the chained pieces out, peel again over what is left, until the chain takes nothing. Assign calls it in place of its inline sequence and all its reports still come from the final peel; the reserve calls it on both views and compares the suffix lists it returns, and both subtractions are deleted. `abdul rahman MA Prof.` now reads given `abdul`, family `rahman`, suffix `MA` under title `Prof.`, and `John Prof. MA Prof.` reads title `Prof. Prof.`, given `John`, family `MA`. One further clause had to be said in the code rather than left to the arithmetic: the join must not take a piece the chain takes (#P5 — "a trailing title word the run takes is no word to spare"), which the subtraction had enforced by accident, by leaving that piece in the tail it compared. Without it `Sir abdul Prof.` joins the title into the given name; with it the reading is title `Sir Prof.`, given `abdul`, as before. `trailing_titles` returns the count it LEAVES STANDING in the same edit, the way `peel_trailing` counts, so the two compose at the call site without arithmetic at all. +- **Cost, and what paid it.** The shared helper is one frame on the common path, where the inline sequence was none. It is paid by `_effective_order` taking the segment's pieces and the name-piece indices — the shape every piece-layer predicate takes — instead of a list built for it, which on 3.11 is a comprehension frame on every parse; the reference name stays at 416 parse / 453 facade. The helper returns a bare 3-tuple rather than a NamedTuple for the same budget: a NamedTuple's `__new__` is itself a frame, measured at +1 here. +- **How big each defect's class is, measured as the rule states it.** The two readings named above are the shapes that made the defects visible; the class both sit in is the TRANSPARENCY class A1 is stated over, and the two defects reach different parts of it — an earlier wording of this bullet flattened them into one width and named the wrong shape for it. Recipe: take the alphabet `John Smith abdul rahman Prof. Dr. Sir MA Jr. V Xyz. King de née`, form every 3- and 4-word arrangement ending in `Prof.` or `Dr.` whose spelling WITHOUT that last word still fills two of given/middle/family (3,210 of them — the two-or-more-name-words scope the A1 clause is stated for), and compare the seven fields other than `title` against that shorter spelling. rules/H5 says they must be equal. On the branch tip 786 differ; on this commit 698 do, none of them new, and all 698 are the P2/M2 residue the rule already accepts as a boundary — a particle chain or a maiden marker took the trailing word before the H5 chain could see it (`John de Prof.`, `Mary née Prof.`). The 88 that close PARTITION by defect, and that partition is what makes them two blast radii rather than one. 56 of them carry a SINGLE period-marked title word, and every one of those carries a bound given-name word as well — the subtraction model's reach, `abdul rahman MA Prof.` itself and `abdul John MA Prof.` beside it, where the arithmetic counted the acronym a name word to spare. The other 32 carry TWO, which is the truncated fixed point's reach: `John Prof. MA Prof.` and `John Dr. MA Dr.`, where the single re-peel re-exposed a title nothing then took. The two-title members of the 3,210 number 784, which is the population that could have witnessed the fixed point at all: 164 of them differed on the branch tip and 132 still do, every one of those 132 the same P2/M2 residue the 698 are, so the defect accounted for exactly the 32 that closed and no more. Over the same sweep's 82,712 inputs (both comma-less and family-comma spellings, 2 to 4 words) exactly 156 parses move, splitting the same way — 96 carry two period-marked title words, 60 carry one, and all 60 of those are bound-given — and not one mover is a comma spelling, the family-comma segment path reaching neither defect. Every figure in this bullet re-measured 2026-09-09 on this tree against `7ac2218`, the round's parent. + +- **Nothing else moved, and it was measured rather than argued.** Snapshot all seven fields, the ambiguity kinds and the recorded `order` for every name in `tools/differential/corpus*.jsonl` plus every string literal in `tests/v2/cases.py` (`ast.walk` over that file, which sweeps up the notes and the case ids too — harmless, they parse like anything else), on the branch tip and on the fix, and diff: 2682 strings before and 2681 after, 2674 of them on both trees, and NONE of those 2674 reads differently. The seven that differ are the case rows this round added and dropped, not parses that moved. The two readings above are inputs no corpus holds — as is every other member of the class the bullet below measures — and each takes a case row (`title_word_trailing_run_is_read_to_a_fixed_point`, `title_word_trailing_behind_a_bound_pair_at_the_peel_reserve`). The same round dropped the `Sir Jr` example from rules/S2 and its case row: it reads by the same branches as `Dr Jr`, the same roles and the same reported kind, the run being empty by then and no branch reading `vocab:given-title` — so the pair pinned one reading twice. Its leaving corpus_rules.jsonl moves two ledger rosters: the jr suffix-routing rule 7 → 6 corpus names in the 1.4.0 ledger, and the #489 peel-floor rule 4 → 3 in all four. + +- 2026-09-26 #535 — CROSS-REFERENCE: the maiden half of rules/H5's "the chain reads PIECES" Accepted row now reads the title — a maiden clause is no longer a join that puts the trailing word out of the chain's reach, so `Mary Smith née Jones Prof.` reads title 'Prof.' — decided in the 2026-09-26 #535 entry under `### M2`. +- 2026-10-07 #614 — the accepted cost 'the chain reads PIECES, so a join that ran earlier puts the word out of reach' is retired for the particle chain on a main segment: the chain is read with S2's run, once, after P3's connective joins and before the particle chain is made, so a particle chain stops in front of a trailing title -- `John van der Berg Prof.` reads title 'Prof.', family 'van der Berg' (every release from 1.4.0: family 'van der Berg Prof.'), the example rules/H5 carried for the cost now pinning the reading. It stands for a connective's join, made before the read, so a title it joined into a longer unit is no single word (`John Smith Prof. and Dr.` keeps family 'Prof. and Dr.', the case row that pins the one-word gate), and after a family comma, whose given part is read once all its joins are made (`Smith, John van der Berg Prof.` keeps middle 'van der Berg Prof.'; SUPERSEDED IN PART by #623, below: there the chain now stops in front of the title). decisions/S2, this date. +- 2026-10-10 (Derek), #623 — AFTER A FAMILY COMMA THE PARTICLE CHAIN STOPS IN FRONT OF A TRAILING TITLE TOO. Assign's walk over the given part already reads the trailing title run, with its own suffix reading (#144's lenient tail with two parts, #602's run, P6's attachment), so `Smith, John Prof.` reads title 'Prof.'. What it lacked was a chain that left the title standing: group bounded the chain there by `trailing_start`, the suffix peel alone, so the chain took the title into the particle's name before the walk met it, and `Smith, John van der Berg Prof.` read middle 'van der Berg Prof.' where `John van der Berg Prof.` reads title 'Prof.' since #614 (1.4.0 and 2.3.0 read middle 'van der Berg Prof.' for the comma spelling). At that site the bound is now `_pieces.trailing_run_start`, where the run `tail_reading` takes begins, its titles included, so the chain stops in front of the title and the walk reads it: title 'Prof.', middle 'van der Berg'. Nothing new is counted -- only the chain's stop moves -- so the issue's question of which words-to-spare floor the given part uses does not arise, and the comma's own readings of a numeral or a bare acronym there (`Smith, John van der Berg VI` middle 'van der Berg VI') do not move. P6 (the issue's open question) reads after the title run, as though the title were absent: `Smith, Jan van Prof.` → title 'Prof.', family 'van Smith', as `Smith, Jan van` reads. The bound is asked only where a particle stands past the chain's lead, by an inline tag scan, since the chain otherwise merges nothing -- which also drops the `trailing_start` call master made for every family-comma part. + DECIDED (Derek): the narrower bound. + Declined: the issue's proposal, reading the given part's run once ahead of the chain and splitting it off as the main segment's is (#614). It would have assign place the given part off group's read, a second reader beside the walk's own predicates, for a class the bound alone closes: the transparency measurement below finds no text the narrower fix leaves broken that the read would mend. + MEASURED 2026-10-10 against 4d5631c2 (#634's merge), py3.11, `nameparser.__file__` asserted on each side. A given-part grid -- `{family}, {head} {middle} {tail}`, and the same with `, Jr.` appended, for family in `Smith`, `van der Berg`, `Ortega y Gasset`, (empty), `(Bob)`; head in `John`, `Dr. John`, `Freiherr`, `abdul`, `Freiherr St`, `J.`; middle in (none), `van`, `van der`, `de la`, `van Berg`, `van der Berg`, `Mc Mc`, `von und zu`, `Berg`, `Berg van`, `do`, `St`, `y Lopez`; tail in (none), `Prof.`, `Dr.`, `Prof. Dr.`, `Prof`, `Sir`, `Prof. PhD`, `PhD Prof.`, `Jr. Prof.`, `Prof. Jr.`, `MA`, `Ma`, `VI`, `V`, `X.Y.Z.`, `Ph. D.`, `Esq.`, `née Jones`, `Prof. Jones`, `Xyz.`, `King.`, `Prof. V`, `MA Prof.`, `Lt.Gov.` -- 18,480 texts under the three orders and `lenient_comma_suffixes=False`, comparing the seven fields and every report: 17,340 of the 73,920 parses move a role (4,335 texts under each configuration), and no parse moves a report alone. All but 500 parses (125 texts) put a trailing period-marked title into the title field. H5's transparency over the grid's default-order texts whose given part ends in `Prof.`, `Dr.`, `King.` or `Prof. Dr.` (5,390; the parse with the title against the parse without it, plus the title): broken on 392, against 3,048 on master, and on no text master kept. By cause, 352 of the 392 are H1's move of the name behind a title into an empty family (an empty or nickname-only head), and 40 are P5's lenient post-comma join taking the title into a bound given name -- 24 after a named family (`Smith, abdul Dr.` reads given 'abdul Dr.') and 16 behind an empty head (`, abdul Prof.`), the gap rules/H5's Accepted P5 paragraph records (fixed by #637, decisions/P5, 2026-10-11). The 500 put a title in front of a numeral with a third comma part (`(Bob), Freiherr Mc Mc Prof. V, Jr.`): the bound reads 'V' as a numeral and stops before 'Prof.', where the walk keeps 'V' a name word with a third part (#144), so 'Prof.' lands in the middle 'Prof. V' rather than the given 'Mc Mc Prof.' -- a name word either way, garbage. The S2 stress grid's comma half (#614's recipe): 56 moves, all a trailing title behind a particle after a family comma (`Smith, Jan Freiherr von Berg Prof.`); #624's title-particle grid: 24, the same class (`Smith, John Freiherr Dr.` → title 'Dr.', family 'Freiherr Smith'). #625's particle grid, every corpus name and case text, and #624's comma-tail grid: 0. The five gates exit 0 with the new `fix(#623)` rule in every ledger. Frames (3.11, warm): the reference rows do not move; a family-comma name with a particle pays 2 with no trailing run (`Smith, John van der Berg` 282 → 284), 4 with a suffix (`Smith, John van der Berg Jr.`), and the moved shapes 38 to 73 in a sample (`Smith, John van der Berg Prof. PhD` +38, `Smith, John van der Berg Prof.` 310 → 360, `Smith, Jan van Prof.` +73), with an unmoved particle name ending in a suffix and a title 20 to 22 (`Smith, John van der Berg PhD Prof.` 382 → 402, `Smith, John van der Berg Jr. Prof.` 382 → 404), rare shapes no guard sees; one without a particle pays less (`Smith, John Quincy` 221 → 213, `Doe, John MA` 254 → 227), which `test_a_given_part_with_no_particle_skips_the_chain_bound` holds (the bound skipped, 'Smith, John Prof.' 250 frames with it asked against 207). Recorded negative control: with the old bound, 8 tests fail -- the two `fix(#623)` rows on both surfaces, the three new rules/H5 examples and the field-sweep count. + THE REVIEW (2026-10-10). The bound is `tail_reading`'s run, read as the comma-less name reads it, and the walk reads the given part by its own predicates, so where the two disagree on a word behind a title the chain stops for a title the walk does not take, or cannot cross one to reach a particle P6 takes: a title in front of a shape-only numeral (`Smith, John Mc Prof. VI` → suffix 'Mc', middle 'Prof. VI'; master middle 'Mc Prof. VI'; the walk never reads 'VI' as a suffix after a comma), in front of a numeral with a third comma part (the 500 above), in the middle of the given part in front of a particle (`Smith, John van Prof. Mc` → title 'Prof.', middle 'van', family 'Mc Smith'), and a period-marked title-particle ending it, which P6 attaches (`Smith, John van MA St.` → suffix 'MA', family 'St. Smith'). Every such shape is garbage, most read as master already reads the text without the title (`Smith, John Mc VI` suffix 'Mc'), and none is a reading a well-formed name loses: over about 1.6 million generated rows under every order, the opt-in policies, one-case writing and caller lexicons, and over a 134,720-pair transparency grid (`Smith, ` and `, ` heads; one to three words from `John`, `van`, `der`, `von und zu`, `Berg`, `Ma`, `MA`, `V`, `VI`, `Jr.`, `PhD`, `Ph. D.`, `Mc`, `St.`, `St`, `Freiherr`, `X.Y.Z.`, `Esq.`, `abdul`, `Prof.`; a trailing ` Prof.`, with and without `, Jr.` after it; the three orders and `lenient_comma_suffixes=False`; transparent where given, middle, family, nickname and maiden equal the title-less text's and the title gains exactly ` Prof.`; master broken on 53,384 pairs, the tree on 9,734), no text transparent to its title on master is broken now. (The first review named `Smith, John Ph. D. St. Prof.` as one, its oracle reading the direction backwards: master gives suffix 'Ph. D. St. Prof.' against the title-less 'family St. Smith', and the tree is transparent.) Recorded, not fixed (Derek): garbage, and narrowing the bound would add a condition for it. The review also found the bound's particle gate held by no test, two of its narrowings undoing readings master gives -- a scan from the name's start rather than the chain's lead (`Smith, Freiherr von Berg Prof.`) and a test of single-token particles alone (`Smith, John von und zu Berg Prof.`) -- now rows of their own, with an empty-head row (`, Jan van Prof.`, AGENTS.md's comma sweep), and the frame saving now held as above. Each of the three mutations fails a test. The re-ask after the chain (`left < tail`) still decides readings, though its documented case (`Freiherr von Berg Ma`) takes the read-once path since #614: removed, it moves 400 parses of the given-part grid on master and here alike, a title-particle head before a particle run and a dotted acronym (`Smith, Freiherr van Berg X.Y.Z.` → given 'van Berg X.Y.Z.', as 1.4.0 reads it; without it suffix 'X.Y.Z.'). It decides a main segment's run with a name piece inside it too (`Freiherr van Berg Dr. X.Y.Z.` → family 'van Berg Dr. X.Y.Z.', as master and 2.3.0 read it; without it given 'van Berg Dr.', family 'X.Y.Z.', 1.4.0's reading), a garbage shape no row holds. The first review reported it changing nothing over a grid with no such shape, and #638 was filed on that; the fix review found it live and held by no test, and the row `the_chain_runs_again_where_its_merges_free_the_acronym` (parity) holds it now. Left for follow-ups, two gaps outside this change: a part past the second comma reads a trailing title behind a particle into the suffix (`John Smith, Jr., van der Berg Prof.`, where `John Smith, Jr., Prof.` reads title 'Prof.'; #636), and P5's post-comma join taking a title (the 40 above; #637). + diff --git a/docs/design/decisions/H6-title-before-a-family-comma.md b/docs/design/decisions/H6-title-before-a-family-comma.md new file mode 100644 index 00000000..1a9cf959 --- /dev/null +++ b/docs/design/decisions/H6-title-before-a-family-comma.md @@ -0,0 +1,9 @@ +# H6 — a title before a family comma + +- 2026-10-17 (Derek), #701 — A TITLE OPENING THE PART BEFORE A FAMILY COMMA IS THE TITLE. `Dr. Smith, John` read family 'Dr. Smith', and every release did the same: 1.4.0 from PyPI gives last 'Dr. Smith' and 'Prof. Brühl' (suffix 'Dr.') for `Prof. Dr. Brühl, Ada Cornelia` (measured 2026-10-17, `PYTHONSAFEPATH=1`, wheel path asserted). The reading was written down on purpose as Accepted clauses of rules/H2 and #H3 ("before a family comma the pre-comma text is wholly the family name") and assumed by C1's Freiherr clause ('Prof. Cruz, Ed'); all three are rewritten here. The listing form moves the surname to the front and leaves the title where it was written, and CLDR's index renderings write exactly that in several locales (`prof. dr. von Selimović Hodžić, Izeta Amra`, `г-н Вернер, Иван Асен`). + DECIDED (Derek, the five points of the design proposal): (1) VOCABULARY ONLY. H2's shape does not fire before a family comma, so `Xyz. Smith, John` keeps family 'Xyz. Smith' — the slot is a surname's slot too, as the trailing slot is (#H5), and every CLDR case is a listed title. (2) The run leaves the family its last piece (`Dr., John`, `Greve, Anna`). (3) A word that is a particle too stops the run, bare or period-marked, so `St Pierre, Ed` and the 1919 legal-surname reading of `Freiherr von Berg, Ed` (C1's Accepted clause) stand, and a title in front of them still peels. (4) The peeled titles join the title field in input order, and the comma decision (`_comma.decide`) is untouched — the count before the comma already treated a title as no name word. (5) ACCEPTED COST: a bare surname-like title opening a two-word surname is the title, `King Smith, John` → title 'King', as `Greve Anna` already reads (#H1). + ADDED IN THE PROTOTYPE, not in the proposal: #H3's floor. The first version peeled `MD` from `MD PhD Jr., John`, leaving family 'PhD' — which a case row (`family_comma_segment_zero_is_not_the_run`) caught. A title addresses somebody, and a post-nominal is not somebody, so the run stops where nothing but post-nominals stands behind it: `Dr. Jr., John` keeps family 'Dr.'. The second version stopped in front of ANY piece of the suffix vocabulary, and the design-docs review found it too broad: a title/suffix dual standing SECOND in the run blocked it and then read as a suffix inside the family (`Rev. Sr. Smith, Mary` → family 'Rev. Smith', suffix 'Sr.', where the comma-less `Rev. Sr. Mary Smith` reads title 'Rev. Sr.'), and so did `Mc` and `vd`, which the family loop's own P2 clause reads as a surname's head (`Dr. Mc Donald, John`). Its check compared `T Smith, John` with `T John Smith` over every title word, bare and period-marked, alone, behind `Dr.` and ahead of it (4,788 pairs over the 798 title words before #705's merge): 36 disagreed, 12 the decided particle class and 24 the twelve duals in second position. The floor asks the H3 question instead — is everything behind this title post-nominal? — so the duals peel wherever a name word follows them, as they do in front of a name with no comma. + Declined: excluding the title/suffix duals from the run (it would keep `Ms Smith, Jane` whole); peeling by H2's shape too (the trailing doctrine, #H5, says only vocabulary claims a word in a surname's slot). + MEASURED 2026-10-17 against master `eb2c1fa5`, py3.11. Snapshot (`uv run python tools/differential/snapshot.py --since master`, before this change's own rows entered the corpora): 20 entries move, all one shape — a title leaves the family for the title field — and 18 of their names are held only by corpus_cldr.jsonl. Several of those are names C1 reads as a family comma though the part after it is a credential CLDR's vocabulary lacks (`hr. Hans Henrik Sven Åge Møller, MF`); the title still comes out right there, and the family stays as wrong as before. CLDR agreement: 29 fields now agree and 28 more get closer to CLDR's value, with none regressing. Merging master after #705 listed `bapak` adds one more of the same shape, `Bapak Adinata, Dwi Putro bin MP` (title 'Bapak' out of family 'bin Bapak Adinata'), two more CLDR fields agreeing and one closer. Cost (`sys.setprofile` call events, one warm-up): `Dr. Juan Q. Xavier de la Vega III` 366 → 366 (the benchmark's `_REFERENCE` row, 307, is unmoved too), `Smith, John` 161 → 161, `Smith Jones, John` 203 → 203 (an inline tag test ahead of `is_title_piece` keeps a part no title opens free of frames), `Dr. Smith, John` 200 → 201 (one `is_title_piece`; the floor's `is_suffix_piece` on 'Smith' replaces the family loop's, which no longer asks the first family piece), `Prof. Dr. Brühl, Ada` 279 → 280. The family loop does not re-ask a piece at or past the floor, so a long post-nominal run costs one frame more than on master, not one per member: `Dr. Smith Jr. PhD MD, John` 306 → 307 (310 before the /simplify round), `Dr. Smith` plus twenty `PhD`, `, John` 903 → 904 (924 before). Those two changes moved no reading on a grid of 88,740 family-comma heads (one to four words from `Dr.`, `Prof`, `St`, `Freiherr`, `Sr.`, `Ms`, `Smith`, `van`, `Vd`, `PhD`, `Jr.`, `MA`, `V`, `de`, `Rev.`, `Mc`, `née`, before `, John`) under the default and `FAMILY_FIRST` orders, comparing fields, ambiguity kinds and `initials()`. + THE PR REVIEW (2026-10-17), three decisions. (a) `gov` JOINS TITLES (Derek). `Lt. Gov. Smith, John` read title 'Lt.', family 'Gov. Smith' — a compound split across two fields, where master kept it whole in the family and the comma-less `Lt. Gov. John Smith` reads title 'Lt. Gov.' by H2's shape. The run is vocabulary-only, so the fix is the vocabulary: `gov` is the abbreviation of the listed `governor`. Counts (`uv run python tools/name_usage.py Gov`, 2026-10-17): 236 US Census 2010 surname bearers, rank 80,419, 91.53% Asian/Pacific Islander; SSA given names under 5 a year; Wikidata 3 family-name bearers, all Israeli. The claim acts only in front, so `John Gov` keeps family 'Gov', and a surname written first loses it (`Gov Sokha` → title 'Gov'), the graf-class trade (#salutation-titles) at a smaller incidence. No corpus name moved. The other unlisted second words the review found (`insp`, `sen`, `supt`, `cmdr`, `atty`, `msgr`) still split; each is a vocabulary question on its own. (b) `Dr. PhD Smith, John` KEEPS the branch reading, family 'PhD Smith' (Derek): the first piece the run leaves is family whatever it is, as piece 0 is with no title (`PhD Smith, John`, read that way by 1.4.0 too); master and 1.4.0 gave `Dr. PhD Smith, John` family 'Dr. Smith', suffix 'PhD'. Garbage input, so no row. (c) `Graf Lambsdorff, Otto` reads title 'Graf' while `Freiherr von Berg, Ed` keeps the rank, and the clause first written for it gave the 1919 legal-surname argument as the reason — which holds for every German rank alike. The reason is the particle stop: `freiherr` is the one rank in `particles`. rules/H6's Accepted clause now says so, and the vocabulary question (should `graf`, `fürst`, `prinz` and the rest read as `freiherr` does, with and without a comma) is [#709](https://github.com/derek73/python-nameparser/issues/709). + diff --git a/docs/design/decisions/M1-delimited-maiden-names.md b/docs/design/decisions/M1-delimited-maiden-names.md new file mode 100644 index 00000000..f113f8b5 --- /dev/null +++ b/docs/design/decisions/M1-delimited-maiden-names.md @@ -0,0 +1,22 @@ +# M1 — delimited maiden names + +- 2026-07-03 (maiden-bucket design, landed via the v2 core, PR + #288) — the maiden reading of a delimiter pair is opt-in because + enclosure conventions genuinely vary; there is no default pair. +- 2026-07-18 (cc7063f; Derek's proposal out of a docs-review pain point — routing parens to maiden used to require editing both buckets in tandem, and the nickname default's contents were not discoverable) — bucket overlap is canonicalized before parsing: a pair listed for maiden is dropped from the effective nickname set (maiden wins), and the public DEFAULT_NICKNAME_DELIMITERS constant landed with it. The v1 facade restores v1's nickname-wins reading by pre-subtracting on its side. Weighed and ruled the same day: maiden-wins applies THROUGH apply_patch too — a pack's maiden pair silently removes a user's unrelated explicit nickname pair; warn-on-removal was considered (a silent-failure review flagged the site) and rejected, the silence ruled intended. +- 2026-08-04 #329 (PR #331) — the marker word inside a delimited clause is dropped from a multi-word clause during grouping; the extraction itself keeps the whole enclosed span, so nothing is lost when there is no marker. Three decisions argued separately: the clause-size guard exists because Nee is an attested surname (Irish Ní/Nee, a Chinese romanization) — load-bearing, not defensive, and mutation-proven; clauses are independent — "(Nee) (Jones)" reads maiden "Nee Jones", each clause dropping or keeping its own marker (the first implementation leaked across clauses, a real defect); and the contentless "(née —)" alone now parses to every field empty and bool() False (an explicit alternative keeping maiden "née —" was rejected; pinned by the maiden_marker_delimited_content_free case). +- M1 and M2 are disjoint by construction, which is the no-conflict guarantee: M2's walk covers joining structure that role-bearing words never enter, while M1's drop reaches only extracted content. Verified independently at review over 7,775 records: 967 diffs, every one in the maiden field. + +Declined: + +- Neighbour-scoping the drop (drop a marker whose next word also reads maiden) — implemented, reviewed and rejected: it also fires on the bare-marker path, eating the surname out of "Jane Smith nee Nee Jones" (maiden "Jones" instead of "Nee Jones"), and it leaks across adjacent clauses. It is the obvious implementation, which is why this entry exists. + +- 2026-08-05 #329/#335 — marker auto-detection inside a nickname-delimited clause was deferred to #335 on a corpus measurement: 山田 花子（旧姓 佐藤） is in the CJK differential corpus so the #329 change was gate-visible, while "Jane Smith (née Jones)" is in no corpus — shipping auto-detection in 2.1 would have let a real Latin-affecting change ride under a "0 Latin-only" gate report. + +- 2026-08-26 #335 — correction to the 2026-08-05 entry above, which reasoned from "the #329 change was gate-visible". It was not. Under the default policy the corpora run, no name reached the drop pass at all, so 山田 花子（旧姓 佐藤） moving between 2.0.0 and 2.1.0 was the East Asian order flip and not #329. Nor is #329 gate-visible now that M3 routes marker-led clauses to maiden without configuration: a ledger rule narrows by which FIELDS move and never by what they hold, so reverting the drop leaves the same six names moving the same fields and all three gates green (measured by mutation 2026-08-26). The {nickname, maiden} pair is the 2.1.0 ledger's; the other two baselines classify the CJK name differently, which is why those ledgers are split and must stay so. The deferral #329 argued for was still the right call — it was about a Latin-affecting change riding under a gate report, and that half held — but the premise is corrected here rather than left to be re-used. tools/differential/README.md carries the same correction beside the table it belongs to. +- 2026-08-26 #335 (M3) — the opt-in this section opens with narrowed, and the boundary is not where the 2026-07-03 entry drew it. A clause whose content announces itself no longer needs its pair declared, so what configuring maiden_delimiters buys is now markerless content and one-word clauses. See #M3 for the whole reasoning; M1 still governs every clause whose pair is configured, and M3 defers to it for the marker drop itself. + +- 2026-08-26 #434 — correction to the 2026-08-04 #329 entry above, on the two statements a phrase entry falsified, and a decision that entry's reasoning does not reach. Its "dropped from a multi-word clause" is no longer the condition: "(z domu)" under a configured pair is a two-word clause whose marker is NOT dropped, because both words ARE the marker. The condition has always been "the clause holds a word past its marker" and reads that way in the code; with one-word markers the two were indistinguishable, which is why the entry could say the wrong one for two years without cost. Nor is there a "clause-size guard" any more, load-bearing or otherwise — the containment test now spans the whole marker run, so its subject is the token AFTER the run rather than the second token of the clause. What survives unchanged is the REASON that guard was written: Nee is an attested surname (Irish Ní/Nee, a Chinese romanization), so a clause of nothing but its marker keeps its words, and that argument is mutation-proven still. Recorded here rather than only under #M2, because a future PR reading M1's own history pointer would otherwise find "the clause-size guard" described as load-bearing and restore something that reintroduces the phrase defect while passing every doc test. + + The decision the rewrite forces, stated so a later PR has something to weigh against. Under a configured maiden pair, "Maria Kowalska (z domu)" emits maiden "z domu" — the marker itself as the whole maiden name. That is not what the phrase argument under #M2 says about those words: "z domu" is borne as no name in any position, which is exactly why shipping it as a phrase is safe. So the value is a name nobody has. It is emitted anyway, and deliberately: M1's carve-out is that a clause holding nothing past its marker keeps its WORDS, and the parser has no way to know a caller's configured pair did not enclose an unfamiliar surname — the same ignorance that makes "(Nee)" keep its word. The alternative readings, an empty maiden or a nickname, both require the parser to judge its own vocabulary's words unnameable, which nothing else in the library does. Cheap to revisit: the shape is one rules.md example and one case row away from a decision either way, and no corpus name has it. + diff --git a/docs/design/decisions/M2-maiden-marker-rule.md b/docs/design/decisions/M2-maiden-marker-rule.md new file mode 100644 index 00000000..3f4ac9f3 --- /dev/null +++ b/docs/design/decisions/M2-maiden-marker-rule.md @@ -0,0 +1,78 @@ +# M2 — the maiden-marker rule + +- 2026-07-03 (maiden-bucket design; #274 filed 2026-07-07, landed in the v2 core, PR #288) — the marker takes everything after it up to a trailing suffix, greedily: "née Jones Smith" is a two-word maiden name, matching how the marker is actually used in running text. The marker itself is dropped as structural, like a delimiter character. + +- 2026-08-19 #399 (PR #409) — a maiden marker bounds a particle join reaching it from the left, where the marker is actually taken as a marker. rules.md carried the opposite as an Accepted limitation ("a particle chain swallows a marker in its path, the join (P2) running first"), but that was a consequence of stage order rather than a judgment: the chain merges pieces before the marker is consumed, and the consumer looks for a lone marker piece which by then no longer exists. So `Ursula von der Leyen geb. Albrecht` lost the maiden name that `Ursula Leyen geb. Albrecht` kept. + + The gate on the stop is the whole substance of the rule and was added in review, not in the first cut. Sharing one marker-piece predicate between the stop and the consumer settles what a marker piece IS; it does NOT settle whether the consumer takes, because the consumer also needs a non-suffix piece after the marker. Stopping without that left the marker as a piece of its own, and a lone trailing piece takes a role field: `Jane van der Berg née` read middle 'van der Berg', family 'née' — the marker became the surname and the real surname was demoted, which is the stranding the stop exists to prevent, one field over. Three consequences of the ungated form, each measured and each fixed by the gate: `Jane van der Nee` moved a Dutch bearer of the attested surname Nee out of the family (M1 already names Nee as a surname); `Ursula von der Leyen geb.`, an ordinary truncated record, reported the marker as the surname; and `St St née` stopped reporting its particle-or-given fork, because group's emitter is guarded on the chain having merged something and the stop made it merge nothing for a different reason than the guard assumes — an A1 violation of the shape #405 tracks, and worse than #405's, since it REMOVES a report callers already see. The gate tests the consumer's own condition rather than restating it, which is what makes the two halves agree. M2 states the gate ("a marker left as a word bounds nothing"); the trailing-marker rows pin it. + + Weighed against moving the maiden handler ahead of the chain, and the recorded reason for preferring the stop has been corrected twice. It is NOT that the reorder disturbs names whose maiden side carries its own particle chain: `Anna Müller geb. von der Berg` is identical before #399, after it, and under a reconstructed reorder, and mechanically cannot differ — no suffix and no conjunction on the maiden side means merged and unmerged pieces hand the consumer the same words. The reason is scope: the reorder changes what the consumer receives for every name rather than adding one condition, and two review sweeps found no test and no corpus name that separates the two options, so it would ship unmeasurable. A review sweep also found the reorder would reach maiden names the stop does not (the P3/P5 join-swallow below), and would drop P3's rootname count when the maiden words leave early. That second effect was recorded here as a COST and #411 has since shown it is the opposite: P3's count has the same defect P5's reserve had, counting words a later pass removes, so `juan y garcia nee jones` loses its family name entirely. Correcting the ledger rather than leaving it: the reorder's real price is scope — it changes what the consumer receives for every name — not that consequence. It stays available, and #418 tracks the P3 count on its own. + + Two things fell out rather than being decided separately: the same issue's open question about the leftover given slot (the marker used to compete for it, giving `de la Cruz née Vega` given "née" under a family-first order; consumed and dropped, it never reaches the placement — though a marker the consumer declines still does, which is M2's "just a word" reading and not a leak), and the wholly-particle family a stop can leave, which #404's R2 reading already covers — "Jane de la née Jones" reports family "de la" with a non-empty base. A 305,364-parse sweep at review found no empty-base or partition violation either side of the change. + + Two limits stay, now recorded in M2 rather than left implicit: the bound reaches only a marker standing as a word of its own, so P3's connective join and P5's bound-given join each still absorb a marker before the bound can see it — the deleted Accepted clause covered a class of which this fixed one instance and left two. (P5's was closed by #411 on 2026-08-21, and P3's the same day by the #412 / #417 / #418 entry below, which takes the reorder this entry set aside.) Tests carry the whole verification: of the marker-bearing differential corpus names none has a particle ahead of the marker, so the gate is blind to this class and its clean exit says nothing about the change. A synthetic cross-script row was written to reach the CJK corpus builder — the only route a case row has into any corpus — and then removed: it used that corpus's provenance as a back door, and it pinned nothing the Latin rows did not. The honest route is [#413](https://github.com/derek73/python-nameparser/issues/413), where the harvester's pattern turns out to miss every name written in backticks, which is how this tracker writes them. `Ursula von der Leyen geb. Albrecht` sat in #399's own title the entire time. + +- 2026-08-21 #412 / #417 / #418 (PR #420) — the marker pass runs before every join in the grouping stage. The three issues were one shape: a join rule computed something — P3's word count (#418), the chain's stop condition (#417), the connective join's range (#412) — against a piece list the marker pass would rewrite a few lines later. #411 had fixed the same shape in P5's reserve by hand, excluding the departing words and declining a join onto the marker, and #418 argued the general fix was a guard at the merge primitive: no join may span a marker piece. Stated unconditionally that guard is the ungated stop the 2026-08-19 entry above measured and rejected (it strands `Jane van der Berg née`), so it would have to be gated on "a marker the consumer will take", evaluated live at every merge — #417's shared predicate, plus a per-merge scan the `"and " * n` benchmark shape would have to tolerate, plus the count exclusions kept in step. The reorder set aside above gets the property by construction: the consumer runs first, what it takes is gone before any count or join looks, and the stop, the exclusion and the shared-predicate question all disappear, a net deletion. P5's decline-onto-a-marker test stays, because it is about what a name word is, not about counting — measured: removing it re-read `Berg, abdul née PhD` as given 'abdul née'. It tests the LONE marker piece, which is the only marker the consumer can have declined on its own; a declined marker the particle chain has already taken travels with the chain, and P5 joins the chain — `Abd van der Berg née Jr Jones` reads given 'Abd van der Berg née' where the old stop had kept the marker out of it. Review found this; rules/P5 now says "standing as a word of its own" for that reason. + + The recorded ground for setting the reorder aside was that nothing separated it from the stop. That stopped being true when #413's harvest landed, two hours after #418 was filed: eleven corpus names flip P3's carve-out under an appended maiden clause (eight losing the family — #418 counted seven and six, before the harvest), and seven show the connective join swallowing the marker — `johnny y`, `Juan de y`, `e and e` with the marker behind the connective, and `Claus SchroederVP of`, `Jon Dough and`, `Jon Dough and of`, `jones, john e` with the marker ahead of it — where #412 recorded no corpus name behind it at all (review counted the seven; the first draft of this entry had three). Measured over every corpus name, every corpus name with ` nee jones` / ` née Jones` / ` née Jr Jones` appended (comma names included) and every case row, 4,180 inputs: 55 differ, three of them inputs this change itself adds (the P3 example, its appended forms, the new case row), and each of the rest either a shape of the three issues or the clause ceasing to change how the rest of the name reads — `The Hon nee jones` reads given 'The', family 'Hon' because `The Hon` does; `der, y van nee jones` reads family 'van der' because `der, y van` does. No ambiguity report moves. The suffix-comma tails, the ordering cost the #411 entry left unmeasured, were byte-identical in every shape tried — under the default policy, which review showed was the wrong population. With `extra_suffix_delimiters` configured, a delimiter core after a marker in a tail became a word the pass could take (`Smith, John, PhD née - Jones` read maiden '- Jones'), and a core the pass left standing alone stopped being dropped (`Smith, John, - née Jones` read suffix '-', maiden 'Jones'): the tail's core drop used to run before the pass and now ran after it. Closed before merge by having the pass read a tail segment with its cores screened out and leave them for the drop — the old order restored for cores — with both shapes pinned at the piece level. Two corpus names move at the 2.0.0 and 2.1.0 baselines, the #412 pair; at 1.4.0 fix(#274) already classifies them. A third, `Juan y Garcia née Jones`, is the example this change adds to P3 and so to the rules corpus, and moves at all three — the one #418 shape the gate can see, the other eight showing it only under an appended clause. + + The one reading the order costs: the consumer sees unjoined pieces, so "up to any trailing suffix" now means the first suffix WORD after the marker, and a connective beside it cannot un-suffix it first. `Jane née Jr y Jones` read maiden 'Jr y Jones' and now declines (family 'Jr y Jones'). The shape is a suffix homograph ANYWHERE in the maiden name with a connective beside it, not only in first position — review measured `Jane Smith née Jones Jr y Smith` going from maiden 'Jones Jr y Smith' to maiden 'Jones', family 'Jr y Smith', middle 'Smith': the maiden walk stops at the homograph, and the connective then builds a family name out of what is left, demoting the real one. The first draft of this entry priced it as the first-word shape, from a sweep that appended only ` née Jr Jones`. Accepted rather than built around, and the one place a merge-time guard would have read differently. + + #417's question — whether `van der Berg née` is a family name — is answered by consistency rather than decided: a marker the consumer declines is M2's "just a word" and rides inside whatever join reaches it, which is the reading `Jane van der Berg née` already had. The fields of `Jane van der Berg née Jr Jones` do not change at all (middle 'van der Berg née Jr', family 'Jones'); only the pieces do, which is why its pin is at the piece level. The invariant test that hid the shape now lists it. + + Review added two pins the first cut had left to prose. A corpus-wide assertion that appending a maiden clause moves no other field (tests/v2/test_parser.py), with #410's lone-residual shape as its one boundary until #410 closed it on 2026-08-25 — `Dr. Jane née Smith` reads family 'Jane' now, as `Dr. Jane` does, so the assertion covers every corpus name that parses to anything, and the only skip left is M2's own (nothing at all before the marker). And a field-level row for the chain-carried marker. One guard is deliberately left without a pin: `_is_maiden_marker_piece`'s lone-piece half has no default-vocabulary input that reaches it now that the pass runs first (dropping it leaves the suite and a 337k-name review sweep identical), and it stays as M2's own definition — "standing as a word of its own" — rather than as a guard a test holds. + +Open (M2): +[#317](https://github.com/derek73/python-nameparser/issues/317) +the fullwidth-colon marker (旧姓：佐藤 arrives as one word; the head-peel question). + +- 2026-08-22 #424 — the walk stops before the trailing numeral assign reads as the suffix. "Up to any trailing suffix" was asked with the suffix-piece test, whose initial veto does not see a bare `V`: `John née Jones Smith V` read maiden 'Jones Smith V' where `John Smith V` reads suffix 'V'. The walk now also stops at `_trailing_start` — the S2 peel read from the marker on, over the pieces as they stand, stepping over a tail segment's delimiter cores as the walk does — but at the NUMERAL only. The bare-acronym fork counts pieces, and the walk removes the very pieces it counted: `John née Jones Smith Ma` peeled over the pieces as they stand reads the acronym as a credential with words to spare, and once 'Jones Smith' has left the name it is the family of a two-piece name — maiden 'Jones Smith', family 'Ma', which is worse than the maiden 'Jones Smith Ma' it had always read. The numeral fork reads the piece before the numeral, and the take removes that very piece: afterwards assign sees the piece before the MARKER there, and where that is initial-shaped the fork does not fire — the first draft stopped the walk all the same, and `J. née Jones Smith V` read maiden 'Jones Smith', family 'V' (both reviews found it: 27 of the design-docs review's 144 marker shapes, every one a head ending in an initial; the pool is unrecorded). The walk runs the peel again over the view the take would leave — the pieces before the marker and the numeral on, built as P5's reserve builds its view — so it stops only where the numeral reads as the suffix both as written and as left: `J. née Jones Smith V` keeps maiden 'Jones Smith V', as `J. V` reads the V as a name, and `Dr. née Jones Smith V` keeps maiden 'Jones Smith V', since assign peels the title first and a numeral alone is no fork (the first re-ask checked the preceding piece only and handed that V to the given name — the code review). As written too: `Jane née Jones J. V` keeps maiden 'Jones J. V' though the take would leave `Jane V`, the words as written being what the marker takes; the second docs review found the statement saying only the second half. The acronym is left to assign, recorded as Accepted. The peel is read from the marker rather than after it, so `Jane Smith née V` declines as `Jane Smith née PhD` does — nothing after the marker but a suffix, the marker stays a word, suffix 'V' — which is 1.4.0's reading (the first draft read maiden 'V', and the 1.4.0 gate is what showed it); and `Jane née Jones J. V` keeps maiden 'Jones J. V' (the piece before is initial-shaped). 1.4.0 had no maiden support, so the examples fall under the #274 rule at that baseline where its fields reach, and `J. née Jones Smith V` (v1 read suffix 'V') under an accepted rule of its own; against 2.0.0 and 2.1.0 the examples that move are classified — `John née Jones Smith V` in maiden and suffix, `Jane Smith née V` in the five fields the marker's survival touches — and `J. née Jones Smith V` is 2.x parity. + +- 2026-08-25 #410 — the maiden name no longer makes H1 decline; the rule counts name words and a marker-announced name is not one. The decision and its width are recorded at decisions/H1, since the term it removed was never maiden-specific. + +- 2026-08-26 #434 — a maiden marker may be a PHRASE, and the Polish "z domu" ships as the first one. The rule had assumed a marker is one word everywhere it was written down, and the library's own advice made that assumption load-bearing: the dead-entry warning told a caller with a multi-word entry to "split it into separate entries", which for this field produces two wrong readings rather than one inert entry. With z and domu stored separately, "Maria Kowalska z domu Nowak" reads maiden "domu Nowak" — the marker's own second word inside the value #434 reports — and "Anna z Nowak" loses its family name outright, because a bare preposition is then a marker and M2 hands a marker every word after it. So the warning was not merely unhelpful for this field; following it was the defect. A phrase entry matched whole is the only shape that fixes both: z is an ordinary Polish preposition and domu an ordinary noun, and neither ever claims anything alone. + + A run predicate — identify a run of marker words, then look the joined run up — was rejected, and the reason is the difference between the two exceptions this makes. given_name_titles is the STORAGE precedent and NOT the matching one. Its entries are stored space-joined and folded per word (_lexicon._title_key, since whole-phrase normalization would leave 'lt. col' with its interior period), and markers copy exactly that. But its MATCHING identifies the run per word first — 'lt' and 'col' are each title vocabulary, so the parse has already read both as titles before anything is joined — and z and domu are markers in no vocabulary at all, so no run can be identified before the lookup. Markers therefore need genuine multi-token lookahead, longest first, which is a different mechanism wearing the same storage rule. Worth knowing that until this change NO shipped vocabulary set held a single multi-word entry, given_name_titles included: the exception existed for caller-configured vocabulary only, and "z domu" is the first phrase the library itself ships. + + That the phrase is safe where its words are not is the C-i question answered one level up, and the contrast worth drawing is with the neighbour in the Excluded (MAIDEN_MARKERS) block in decisions/vocabulary-exclusions. #vocabulary-collisions C-i asks whether a word is borne as an ordinary name in the position the vocabulary claim acts on; roz fails it — Roz is a diminutive of Rosalind and stands exactly where M2's claim acts, the word after a given name — and MAIDEN_MARKERS has no ambiguous subset to express a partial answer in, so the only remedies available there were ship and do not ship, and it was removed. z fails C-i at least as badly. What changes is that the claim need not be made about z at all: the phrase is the unit that claims, and "z domu" is borne as no name in any position. C-i is a test on the claiming unit, and a phrase entry is a way of choosing a different unit rather than a way of passing the test with the old one. This is not a general escape — a phrase whose words are common and whose pairing is also common would fail C-i as a phrase — but it is the answer whenever the collision is with a word that only a longer phrase makes unambiguous. + + Markers are consulted at FOUR sites, which is one more than the title-run key's three, and _title_key's own docstring names the failure mode ("a divergence between them fails silently: the entry simply stops matching"). So one predicate answers "does a marker start here, and where does it end" — _pipeline._vocab.maiden_marker_run, in the vocabulary layer because it is a question about token TEXTS, per mechanisms.md#ONE-PREDICATE-PER-QUESTION's rule that the destination follows the layer. The four are classify's tagging, extract's clause test, and group's TWO — the piece test M2's take is built on and the drop inside an extracted clause (#329), which is the pair that makes four where titles have three. classify calls the predicate in a sequence pass, tagging a run's head "vocab:maiden-marker" as before and its continuations "vocab:maiden-marker-cont"; extract calls it over a clause's whitespace words; both of group's sites read those tags rather than re-deriving the run, which is the same mechanism's recorded-answer half, available because group runs after classify. tests/v2/pipeline/test_group.py::test_every_marker_site_ends_the_run_in_the_same_place is the contract, modelled on the P5/H1 title-run test: it collects each site's answer to "how many words is the marker" over nine spellings and asserts they cannot disagree, and each of the four sites has been mutated to confirm it fails. + + One divergence between those sites is deliberately PRESERVED and must survive any future unification. extract splits a clause on WHITESPACE while the tokenizer splits punctuation off, so "née," is marker-led to classify and not to M3. That is what keeps group's role filter reachable, and cases.py::marker_glued_to_punctuation_keeps_the_clause_a_nickname is the row that pins it — 'Jane "née, Janie" Smith (née Jones)' reads nickname 'née Janie', maiden 'Jones'. Sharing the predicate did not merge the two questions, because each site hands it a different sequence of words; confirmed by mutation, where making _maiden_marked strip commas leaves that row (and its facade twin) the only failures. SUPERSEDED IN PART 2026-10-12 by #653: inside a clause the tokenizer now keeps a comma on its word, so 'née,' is no marker to either site and that row reads nickname 'née, Janie'. The divergence survives through rules/T2's nakaguro, which tokenize divides at and the whitespace split does not, and cases.py::marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname is the row that pins it now ('佐藤 花子 "旧姓・田中" (旧姓 鈴木)' reads nickname '旧姓 田中', maiden '鈴木'). + + The commit order carries a finding about this repo's tests worth stating on its own. Taken in the plan's order — predicate, then storage plus the warning carve-out, then the sites — the middle commit would have exempted maiden_markers from the dead-entry warning before anything could match a phrase. That is not merely untidy: pyproject sets filterwarnings = ["error"], so a test that configures a phrase entry FAILS until the exemption lands, which means the carve-out cannot be deferred past the sites either. The two constraints leave one arrangement — the predicate alone, then storage and the four sites together, then the shipped entry — and the middle commit is fused for that reason rather than by preference. Shipping "z domu" last is what keeps any bisected revision from holding a phrase entry nothing can match. + + Review found one real defect, and it is the one this whole design exists to prevent. A marker run is TAGGED over the whole span-sorted token stream and CONSUMED over one segment, and the two populations are not the same: segment's role-None filter keeps only tokens extract has not already given a role, then buckets what is left by the commas before them. So a phrase written across a bracketed clause — "Anna z (domu) Nowak" — had both its words tagged while the piece walk could see only the first, and M2 took a proper PREFIX of the phrase for the whole marker: family 'Anna', maiden 'Nowak', a bare Polish preposition eating the name, and "Maria z (domu) Kowalska Nowak" losing its family name outright. Reachable on shipped vocabulary under the default policy, and it falsified three of this branch's own claims at once, in three files. The fix is a rule about the TAG rather than a guard on the consumer: classify refuses to tag a run whose tokens are not structurally contiguous — same role, so a clause edge ends a run, and the same comma bucket, computed the way segment computes it. Refusing rather than truncating, because truncating hands M2 the same wrong prefix one word shorter. The lookahead is bounded at the boundary before the predicate is asked, so a two-word entry refused at a clause edge still leaves a one-word entry starting there free to match, and an all-single-word vocabulary skips the walk entirely. rules/M2 now states the qualifier — a phrase is recognized only whole AND only where its words stand together — which it did not, and that gap is part of why the defect was invisible. + + The defensive complement was declined and the reason is worth recording: having _marker_run_pieces re-derive the run from token indices and refuse a mismatch would be a guard that, with the tagging rule in place, can never fire — untestable code standing in for an invariant one stage already guarantees. What it would have bought is written into that function's docstring instead, which now names segment's role-None filter as the second reason `seen` can skip a token; the sentence that said the continuation "is always in `seen`" without naming that reason is what made the gap invisible to review. + + The contract test should have caught this and did not, and the hole is worth naming because it is a property of how the contract was FORMULATED. It varied the marker's SPELLING across nine forms and held its PLACEMENT fixed, so every run it built was structurally contiguous by construction and no probe ever asked what the sites do when a run straddles a boundary — which is precisely where two sites walking different token populations can disagree. The placement axis is now its own test over eight placements, asserting two things no spelling can reach: that every tagged run has one role and one comma bucket, and that group drops a run whole or declines it. Mutation-checked in three pieces, which is what shows the two halves of the rule are both load-bearing: removing the contiguity bound outright fails both new case rows and all three straddling placements; removing only the ROLE half fails the two clause placements and both rows; removing only the COMMA half fails exactly one placement and NO case row, because a comma-split run happens to be declined downstream for an unrelated reason today. That last one is the argument for a tag-level contract test in its own right — the only witness to half the rule is an assertion about tags, not about fields. + + Perf, and how it was measured, because the first attempt at it was wrong in a way worth recording. Timing four source trees in separate processes and comparing them across sessions gave figures whose spread was the size of the effect: merged master alone measured 0.579s, 0.593s and 0.607s on one machine inside an hour. Every number below is a median of five INTERLEAVED rounds — every variant timed once per round, in order, so drift lands on all of them alike — over 8000 parses of a four-name mix (plain, titled-with-initials, phrase-marked, comma-and-particle). Against master: the run pass alone +6.8%, a head-word fast path inside the predicate +1.4%, the contiguity rule on top of that +1.6%, and after the simplify round -0.3% and -1.3% on two separate five-round runs. So the shipped mechanism has no cost this benchmark can resolve. That is not a free lunch and should not be read as one: classify no longer does the per-token `n in lexicon.maiden_markers` membership test master did — the marker decision moved into one pass that answers for every token at once — so what the marker pass adds and what it removed are within noise of each other. The reviewer reproduced the +7.2% and +1.6% stages independently on other hardware, which is the corroboration a percentage of this size needs. + + Four structural choices got it there, and each is a smaller version of the same move: ask the question once and pass the answer along. The fast path lives INSIDE the predicate rather than at its callers, so it stays an implementation detail and not a second answer. maiden_marker_head takes the FOLD rather than the text (suffix_as_written's shape, "n is _normalize(text), passed in so callers normalize once"), and classify folds once per token for the marker pass and the vocabulary tags together, which took the common token from two folds to one — fewer than before this work. maiden_marker_run folds each word once and keys from a prefix, where _title_key(words[:n]) per candidate length re-folds the whole prefix and made a ONE-word hit cost more than a two-word one; sixteen of the seventeen shipped entries are one word. And the marker pass returns its decision as index -> tag for classify's single token-building loop to write, rather than rebuilding every marker token a second time with dataclasses.replace, which is the most expensive call in a parse. The comma-bucket sweep the contiguity rule needs is deferred until a marker head is actually found, so a single-word vocabulary and a phrase vocabulary over a name holding no marker both skip it entirely. + + One limit is recorded rather than closed. The contiguity test is role plus comma bucket, and role is one-directional: a role change is always a clause edge, but two ADJACENT clauses of the same role are indistinguishable, so 'Jane (z) (domu) Jones' does tag a run across two separate nickname clauses. Nothing reads it — the piece walk never sees role-bearing tokens and the clause drop is scoped to one span — and the parse is identical with and without the phrase mechanism. The clean fix is a per-token structural UNIT id derived from state.segments and state.extracted, which would replace role-plus-bucket with one comparison and close the adjacent-clause case as a side effect. Deliberately not taken here: it changes a mechanism four correctness reviewers had just cleared, inside a pass whose stated business was not to change behavior. It is the shape to reach for if this test ever needs to grow a third term. + +- 2026-09-19 (Derek), #533 — THE CLAUSE ENDS AT A TRAILING CREDENTIAL, AND THE DOUBLE CHECK THE 2026-08-22 BULLET DEFERRED NOW EXISTS FOR THE ACRONYM TOO. That bullet stopped the walk before the trailing numeral and left the bare-acronym fork alone, with its reason stated: the acronym fork COUNTS pieces and the walk removes the very pieces it counted, so a peel read over the pieces as they stand said "credential with words to spare" about `John née Jones Smith Ma` and would have left maiden 'Jones Smith', family 'Ma'. The reason was right and is unchanged; what it argued for was a deferral, and the answer is the one the numeral got — ask the peel TWICE, once over the pieces as written and once over the name the take would leave, and stop only where both read the word as the credential. "Left to assign" meant "read in silence", assign never getting the question at all, the clause's tokens carrying the maiden role before pieces exist for it: the end of a maiden clause was the last trailing position in the library where a member of the ambiguous credential class was read without a report. WHICH RULE reads the name left standing is the part the numeral's version did not have to answer, because a numeral reads the same way everywhere and a credential does not. With no comma it is the trailing peel; after a family comma it is #531's slot, where the comma has already settled the count and the writing decides alone; before that comma the words are the family the comma named and a stop would hand one of them to `family` rather than to `suffix`; past a second comma the segment is read as credentials whole and no trailing rule is consulted. The last two READ AS NONE, which is a statement and not a default — the first prototype reported on `Smith, John, Jr nee Jones MA`, where the walk had asked nothing. A FIFTH CASE, found by the design-docs review of this commit and not by the plan, and THE PR REVIEW THEN SETTLED IT THE OTHER WAY — see the review bullet below, which is where the reasoning now lives. The shape: after a family comma the reader may take the word and nothing may then report it. `Jane Doe, Jr nee Smith MA` is a family comma whose segment 1 is `Jr nee Smith MA`, so the given-slot reader is asked and would take the member, but the take leaves that segment as `Jr MA`, post-nominals only, which the credential-run gate reads whole without ever reaching the emitter — and the released word therefore lands in `given`, not in `suffix`. This commit read it as maiden 'Smith', suffix 'Jr MA', silently; the review round declines the stop instead, so the clause keeps 'Smith MA' and reports, which is 2f57ff21's reading. The KEPT direction always reported here, the clause's own emitter being what raises it: `Jane Doe, Jr nee Smith Ma` and `Jane Doe, Jr nee MA` both say so, and now so does `Jane Doe, Jr nee Smith MA`. Pinned by a unit test rather than by a rules.md example line: the name differs from the tree at all five baselines and most of the diff is not this change's (1.4.0 reads title 'Jr', given 'nee', family 'Jane Doe'; 2.0.0 and 2.1.0 title 'Jr', family 'Jane Doe'; 2.2.0 and 2.3.0 already read given 'Jane', family 'Doe', suffix 'Jr'), so an example line would have put five ledger rules into this docs commit to attribute four other changes' readings. THE SECOND CHECK ASKS WHETHER THE READER TAKES THAT WORD, not whether it takes something, and the distinction is not academic: the numeral's existing re-ask tests whether the view's trailing run starts at the view's end, exactly right for a fork that reads one piece, and under it `JOHN NEE JONES SMITH MA PHD` leaves `JOHN MA PHD`, whose peel takes 'PHD' and then DECLINES 'MA' for want of words to spare — so the weaker question answers yes while the member becomes the FAMILY name, which is the 2026-08-22 disaster one word further on. Measured both ways on the prototype; the shipped check compares the view's run start against the member's own index in that view, and `JOHN NEE JONES SMITH MA PHD` keeps maiden 'JONES SMITH MA' with suffix 'PHD' and reports. AFTER A FAMILY COMMA THE CHECK NEEDS #531'S FLOOR, NOT ONLY ITS LEAN: `Doe, Jane nee Smith MA do` leaves `Jane MA do`, where 'MA' leans credential but the 'do' behind it does not read as a suffix (P6 keeps it, and `Doe, Jane MA do` reads middle 'MA', family 'do Doe'), so #531's slot would read 'MA' as a MIDDLE name — and a check that asked only about 'MA' released it from the clause into the current name's middle, a word crossing from one person's name to another's. With the floor the clause keeps 'MA do' whole and reports the 'do'. THE FLOOR ON THE FIRST WORD IS A CLAMP, NOT A VETO, and the first draft got two names wrong by making it one: stated as "a member that is the ONLY word after the marker stays the maiden name" and implemented as "cancel the stop when nothing would be left", it cancels too much wherever the peel consumed that first word AND words behind it — `Doe, J. nee MA ba` peels 'ba' and then 'MA', so the first piece the peel took IS the only maiden word, and the veto handed 'ba' back to the clause, giving maiden 'MA ba' where `Doe, J. ba` reads suffix 'ba'. Clamped to the piece after the marker instead, that name keeps maiden 'MA' and reads suffix 'ba', reporting both. The floor itself is deliberate and diverges from the `née V` / `née PhD` precedent, where the marker declines and stays an ordinary word: certain suffix vocabulary declines the marker, while an ambiguous word is kept by the clause it ends, because that class is the one carrying no evidence of which it is — its members are borne surnames as well as credentials, and nobody writes a credential straight after the marker. 1.4.0 read `Jane Doe nee MA` as last 'nee', suffix 'MA' (measured on the wheel), so this half restores nothing and does not try to. THE `do` PAIR is #531's reading, unchanged and now SHARED as a predicate rather than spelled twice (`credential_at_the_given_slot` in `_pipeline/_pieces.py`; the drift would have been silent, each site's own tests going on passing): `Doe, Jane nee Smith do` keeps maiden 'Smith do', `Doe, Jane nee Smith Do` keeps 'Smith Do', `Doe, Jane nee Smith DO` gives maiden 'Smith' with suffix 'DO', and the comma-less `Jane Doe nee Smith do` reads suffix 'do' — right for the reason the whole change is, since `John Doe do` reads suffix 'do' today, P6 does not run without a comma, and a clause must not change how a word outside it reads. ACCEPTED COSTS, each measured on this tree: `JANE DOE NEE YO-YO MA` reads maiden 'YO-YO', suffix 'MA', a two-word birth surname losing its last word in a one-case record — and `JANE YO-YO MA` reads suffix 'MA' too, so the cost is the one-case reading's rather than this rule's, and in mixed case the writing saves it (`Jane Doe nee Yo-Yo Ma` keeps maiden 'Yo-Yo Ma' and reports); `Jane Doe nee Smith X.Y.Z.` reads suffix where 1.4.0 read last 'X.Y.Z.', the same trade #531 took at its own slot and for agreement with `John Doe X.Y.Z.` rather than for parity; (two costs listed here were WITHDRAWN by the PR review and are no longer costs at all — `Doe, Dr. nee Smith MA` and `Berg, abdul nee Jones MA` both keep their maiden words and report; the bullet below carries the reason); `Jane Smith nee Jones and MA` reads maiden 'Jones and' with suffix 'MA', the take running before the joins (#420) so the connective is a piece of its own when the walk stops; and a report on a declined word adds noise without moving a field, which is #530's stated rule and the trade #531 took at its slot. NOT TRANSPARENT HERE: a trailing title. `Jane Doe nee Smith MA Prof.` is unchanged and silent while `Jane Doe nee Smith Prof. MA` gives maiden 'Smith Prof.' with suffix 'MA' and reports, because the walk reads the trailing credential run and the H5 title chain is read by the rule the walk does not run. Accepted for #533, pinned as a pair of boundary example lines under M2; moving the walk onto the title-aware reading would change the numeral half too, which the 2026-08-22 bullet settled with its own measurements, and the pair belongs to one decision about what "trailing" means inside a clause — a follow-up is drafted for Derek rather than filed here. THE `one_case` PLUMBING AT THESE SITES IS NOW LIVE, and the 2026-09-18 bullet under `### S2` headed THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT stands as what was true then: `numeral_only` answered off the peeled numeral and the numeral fork is decided before the peel reads a lean, so the fact reached only the bare-acronym fork, which that reading discarded. The acronym fork is asked now. Re-measured 2026-09-19 with the same wrapper shape over the same population that entry's recipe names — AND THE PAIR IS THE FINDING: over the corpus as it stood the day before this change, dropping the argument inside the walk moved 0 parses; over the corpus WITH this change's own rows it moves 36 of 10,752 (1,792 names), on six distinct names (`Doe, Jane nee Smith DO`, `Doe, Jane nee Smith Ma`, `Jane Doe nee Smith Ma`, `Jane Doe nee Smith Ma JD`, `Jane Doe nee Yo-Yo Ma`, `John née Jones Smith MA`). The population, the six policies and the denominator are recorded at the call site in `_pipeline/_group.py`, as the 2026-09-18 entry records its own; both were re-run on 2026-09-19 after the review found the figures first written there (1,790 names, 10,740 and 21,480 parses) did not reproduce from the recipe as stated — they had been taken two corpus names earlier in the same day's work. The FINDING was unharmed, 36 parses on the same six names. The plumbing was live either way and the corpus could not show it, which is the blindness mechanisms.md's corpus field note asks to be measured before any "N names move" is written down. + +- 2026-09-19 (Derek), #533 review — A WORD THE CLAUSE GIVES UP READS AS A POST-NOMINAL OR THE CLAUSE KEEPS IT, AND THAT INVARIANT REPLACES THE ACCEPTED SILENT MOVERS ABOVE. The PR review measured what the released words actually do, and three of them do not land in `suffix`: `Doe, Prof. nee Smith A.B.` read given 'A.B.' (66 distinct names in the review's own sweep, the same shape reaching `ba`, `X.Y.Z.`, caps-on `XYZ` and the lower-case spellings); `Berg, Jane van der nee Smith DO` read family 'van der DO Berg' and `Jane Doe nee Smith DO DO` read family 'DO DO', a word crossing from the BIRTH name into the current one, which is #424's failure from the other side; and `Berg, abdul nee Jones MA` read given 'abdul MA'. Two of those had been written up as ACCEPTED COSTS in the bullet above, on the argument that the clause form AGREES with the clause-less one. The agreement is real and it is not the question: the clause-less `Berg, abdul MA` puts the word in the given name because nobody promised otherwise, while a CLAUSE promised that the word was either the birth name or a credential, and a third answer is the promise broken in silence. So the direction is the parent's, conservative: THE STOP IS RIGHT ONLY WHERE THE RELEASED WORD ENDS THE PARSE SUFFIX-ROLED, and where the take cannot promise that, the clause keeps the word (and reports it, the emitter asking whether a trailing rule reads these words at all rather than whether the view check then passed). Two view checks carry it, each modelling a LATER decision rather than predicting one: the given-slot reader requires the view to still hold a name word AHEAD of the member, since a trailing slot needs a part to be the end of; and a release is withdrawn where a join reached below the marker pass would absorb the word — P2's chain, for a particle-tagged member behind a particle piece or beside another released one, and P5's bound-given join, for the word after the bound one. Both over-decline rather than predict, which is the direction the invariant asks for. MEASURED, and the pair of numbers is the argument: over the review's 142,518-name corpus under six policies, 855,108 parses, the invariant holds 0 violations at 2f57ff21 and 0 here, against 1,446 in 40 shapes at the commit this round started from; and a token-level diff against 2f57ff21 finds 128,424 moved parses on 24,958 names with ZERO of the moved tokens unnamed by a `suffix-or-name` report, against 4,100 silent moved tokens in 59 shapes before. `tests/v2/test_properties.py` carries the invariant as a property over a generated grid rich enough to hold every shape above; the grid fails on 790 of its 8,466 parses at d97d3eb7 and passes at 2f57ff21, which is what makes it a test rather than a restatement. TWO ACCEPTED SILENCES REMAIN AND ARE NOT SILENCES OF THIS KIND: a member with a name word BEHIND it was never asked about, and a delimited clause is not a fork at all (below). ONE ROUGH EDGE IS RECORDED RATHER THAN FIXED, and it PREDATES this change: with `extra_suffix_delimiters=('-',)`, `Jane Doe nee Smith - MA` gives maiden 'Smith -' — a clause ending on a bare delimiter token. 2f57ff21 does the same thing one word later (`Jane Doe nee Smith - PhD` gives maiden 'Smith -' there too), so what #533 changes is only which words reach the edge. Walking the stop back over it is not the two-line change it looks like: the delimiter-core set the walk already steps over is EMPTY off a tail segment, which is exactly where this name's `-` sits, so the fix would have to widen what counts as a core on every path and would move the parent's reading as well. Left for its own issue. + +- 2026-09-19 (Derek), #533 review — THE DELIMITERS TRUMP ANY OTHER READING, and that is a settled position rather than an un-asked fork. `Jane Doe (nee Smith Ma)` and `Jane Doe (nee Smith MA)` both keep the WHOLE span as the maiden name and report nothing, in every release including this one; the marker INSIDE the delimiters is what tells the parser the span is a maiden clause, even where the pair is not configured in `maiden_delimiters` (M3). The writer drew the boundary, so no fork is called and there is nothing to report — which is a different kind of quiet from the silences M2 and `AmbiguityKind.SUFFIX_OR_NAME` document, where a fork WAS available and the parser declined to ask it. The boundary cuts both ways: `Jane Doe (nee Smith) MA` gives suffix 'MA' and reports, the member standing outside the span being an ordinary trailing credential. Behavior is unchanged from 2f57ff21 and was not touched; what the review added is the three rules.md example lines, the case rows, and the statement in M2 and in the `SUFFIX_OR_NAME` docstring, because an undocumented settled position reads as an oversight to the next person who measures it. + +- 2026-09-20 (Derek), #397 review — A LINK INSIDE A MAIDEN CLAUSE STAYS IN THE BIRTH NAME, AND THE SPEC'S Q3 WAS WRONG ABOUT THIS BY MEASUREMENT. The walk has ended the birth name at the first suffix WORD after the marker since the #420 reorder, and the Catalan link is also the roman numeral, so `Jane Doe nee Puig i Soler` read maiden 'Puig' and left 'i Soler' outside the clause — middle 'Doe i', family 'Soler' at master 46651750, and the same truncation at every 2.x release. THE TRUNCATION PREDATES #397. What #397 added is the harm: with the link joining, those leaked words merged into the CURRENT surname, and 0fbcaa0b read family 'Doe i Soler' — a word of one person's birth name inside the surname they carry now, which is the class of failure #424 and #533 exist to prevent, arrived at from a third direction. The comma form leaks into the given part instead (`Doe, Jane nee Puig i Soler` read given 'Jane i Soler'), and a Catalan head loses the distinction between its own link and the clause's (`Maria Carod i Rovira nee Puig i Soler` read family 'Carod i Rovira i Soler'). THE SPEC'S Q3 said this interaction was unchanged; it was written from the rule and not from a parse, and the four names above are what a measurement said instead. Derek decided on 2026-09-20 to fix it in #397's own PR rather than file it, on the ground that the PR is what turned a truncation into a leak. + THE FIX IS P3'S OWN CLASS TEST, ASKED WITH THE CLAUSE'S BOUNDS. The walk does not stop at a suffix piece that is a connective PLACED TO JOIN inside the clause — a name word of the clause on its left and one on its right, which is the condition the 41de3df6 join already asks through `_name_word_beside`, called here rather than restated (mechanisms.md#ONE-PREDICATE-PER-QUESTION). Two bounds make it the CLAUSE's question and not the segment's, and each is a control: the left bound is the first piece after the whole marker run, so the marker is no name word (`Jane Doe nee i Soler` declines as before, family 'nee i Soler'); the right bound is where assign's trailing run begins over the pieces as written, so a generation or credential ending the clause is outside it (`Jane Doe nee Puig i III` keeps suffix 'i III', `Jane Doe nee Puig i MA` suffix 'i MA' — and the MA one needs the BOUND, carrying no `vocab:suffix` tag for the piece test to refuse it by). The right bound is `trailing_start`'s own answer read off the peel pair already in hand, not a second call. #533's trailing-credential stop and #424's numeral stop are byte-identical across the change, measured. + MEASURED, and the pair of numbers is the argument. A new property grid — 4,590 clause texts (five heads including a Catalan one with its own link, all seventeen shipped markers, six clause bodies, five trailing runs, comma and no-comma) against the default policy, both family-first orders and strict comma, 15,810 rows, of which 10,540 hold a link the `y` spelling puts between two birth-name words — carries two invariants in `tests/v2/test_properties.py`. THE y TWIN (`y` is the same connective with the ambiguity removed, so it is an oracle and not a second subject): the `i` parse's seven fields equal the `y` parse's, letter for letter apart from the link. It fails on ALL 10,540 guarded rows at 0fbcaa0b AND at the parent 46651750 — no clause shape holding a joining link read as its twin did — and on 0 here. NO BIRTH-NAME WORD IN THE CURRENT NAME, delimited by the twin's own maiden tokens: 28,985 tokens at 0fbcaa0b against 22,185 at 46651750, and 0 here. The GAP between those two is exactly what this branch added, and it is why the y twin alone would not have told the story: the twin fails identically at both commits while the harm doubled between them. + THE GUARD IS NARROWER THAN "the twin keeps the link", by measurement rather than by preference. `Jane Doe nee Puig y` keeps its `y` in maiden 'Puig y' while `Jane Doe nee Puig i` reads maiden 'Puig' with suffix 'i' — deliberately, the link joining nothing there — so a guard asking only whether the twin kept the letter would demand the two agree where the rules say they must not. `Puig y III` and `y Soler` are the same shape from the other two sides, and the grid keeps all three as the bodies the guard has to refuse. + INV6's off-switch exemption is NOT widened, and the reason is the one AGENTS.md gives for not widening an exemption a grid cannot reach: measured 2026-09-20, 1,680 rows of that grid carry both a class letter and a marker and 0 of them put the letter after the marker, its generator placing the connective among the name's own words. A clause link has its own grid instead. + TWO SECOND-ORDER MOVEMENTS, both recorded rather than repaired. The two one-case spellings part company, which is P3's own Accepted clause reaching a taken marker rather than a declined one: classify leaves a letter after a marker to the mixed-case rule, so `JANE DOE NEE PUIG I SOLER` never moved (the capital is an initial, and the initial veto kept the walk going all along) while `jane doe nee puig i soler` goes from family 'doe i soler' to maiden 'puig i soler'. And a report is GAINED where a longer clause ends on an ambiguous credential: `Jane Doe nee Puig i Ma` keeps maiden 'Puig i Ma' and says `suffix-or-name` where 0fbcaa0b read family 'Doe i Ma' in silence — the clause's own emitter, reaching a word the truncation had put out of its reach. A THIRD, found by the second review and the mirror of that one: a report is LOST where the clause takes a class member it used to end at. `Jane Doe nee MA i Soler` reads maiden 'MA i Soler' in silence where the parent 46651750 read maiden 'MA' with a `suffix-or-name` on it. That FOLLOWS from the link no longer ending the clause and is not a second decision: the report is the walk's own, raised on the LAST word the clause kept where a trailing rule was asked about it, and with the link joining, 'MA' has a name word behind it and is no longer that word — which is exactly what `Jane Doe nee MA Smith` has always done with the same acronym. The `y` twin is the oracle and it AGREES, in fields and in reports: `Jane Doe nee MA y Soler` reads maiden 'MA y Soler' in silence at the parent and here. Measured 2026-09-20 over the review grid, 24 names lose the report this way and all 24 agree with their `y` twin on every field and every report. + +- 2026-09-22 #397 follow-up — A DELIMITER CORE PAST THE CLAUSE'S FIRST WORD PASSES FOR THE NAME WORD BESIDE A LINK, RECORDED AS A DEVIATION RATHER THAN REPAIRED. The link exception above wants "a connective standing between two name words of the clause", and a separator the caller declared through `Policy.extra_suffix_delimiters` is structure, not a name word — so a link with one beside it joins nothing and should end the clause like any other suffix word. Between the marker and the clause's first word that already holds, the bound refusing the core before either piece test is asked. PAST that first word it does not: the core is an ordinary index to the run walk, which steps over connectives and nothing else, so it stands in for the name word on the link's left and the clause runs on past a title it would otherwise stop at. MEASURED 2026-09-22 under `extra_suffix_delimiters=(" - ",)`: `Smith, John, PhD née Puig Mr. - i Soler` reads maiden 'Puig Mr. i Soler' where its separator-less twin `Smith, John, PhD née Puig Mr. i Soler` stops at 'Puig Mr.'. THE POPULATION is the branch's own sweep, recorded with the code it describes (2026-09-21, corpus ∪ cases.py ∪ the property grids ∪ a 50,925-name generated set with cores, under thirteen core-bearing policies): the predicate is asked about a core in 51,072 of 900,023 calls, the answer differs from a core-skipping reading in 8,094 parses over 1,278 texts, and 1,824 of those move `maiden` on 288 texts — none of the 288 reachable at the default policy, `extra_suffix_delimiters` being empty there. NOT REPAIRED HERE: the fix threads the core set through three call sites into the run walk and moves the parent's reading as well, which makes it its own change rather than a rider on a review round. Open: #538. PINNED TWICE MEANWHILE. rules/M2 carries the shape as a `deviates: #538` example under an `extra_suffix_delimiters-dash` annotation — the first entry `tests/v2/rules_doc.py`'s registry has had for that field, named after the Policy field and carrying the delimiter in the suffix because the field's value is a set rather than a flag. And `tests/v2/pipeline/test_group.py::test_a_core_beside_a_link_wrongly_passes_for_a_word_until_538` holds the pair at the piece level, named so nobody reads it as the contract. ONE COST OF THE DOC EXAMPLE, worth knowing before the repair lands: its string enters `corpus_rules.jsonl`, where the differential gate parses it with the DEFAULT facade — no delimiter declared, so the dash is an ordinary name word and #538's reading is off the path entirely. It moves there for the 2026-09-20 link fix instead, and is classified as that at all five baselines: added to the `fix(#397) a link inside a maiden clause stays in the birth name` alternation at the four 2.x ledgers (suffix 'PhD i Soler' → 'PhD', maiden 'Puig Mr. -' → 'Puig Mr. - i Soler', identical at each), and given its own rule at 1.4.0, where v1 had no maiden markers and read the whole suffix-comma tail as one suffix. +- 2026-09-26 #538 — A DELIMITER CORE IS STEPPED OVER BY THE LINK'S NEIGHBOUR SEARCH, AS A CONNECTIVE IS (Derek, 2026-09-26). The 2026-09-22 follow-up above recorded the deviation; this resolves it. `_run_neighbours` treats a lone core like a connective, so the word on a link's side is the one past the core and a maiden clause reads exactly as the same text written without the core: `Smith, John, PhD née Puig Mr. - i Soler` under `extra_suffix_delimiters=(" - ",)` now stops at maiden 'Puig Mr.' as its separator-less twin does. INVARIANT, pinned by `tests/v2/test_properties.py::test_a_delimiter_core_reads_as_if_it_were_not_written` over 81 generated texts, maiden field: 6 disagreed at e0f1a2fa, 0 after. Declined: THE BOUNDARY READING, which the 2026-09-22 entry's wording implied ("a link with one beside it joins nothing") — a core ending the neighbour search on its side. It fixes the reported shape equally, and it moves `Smith, John, PhD née Puig - i Soler` and `… Puig i - Soler` from maiden 'Puig i Soler' to maiden 'Puig', suffix 'PhD i Soler' — a birth-name link pushed into the credentials, where the skip reading leaves both as they read. Default-unreachable either way. THE SAME STEPPING APPLIES OUTSIDE A CLAUSE: the `frozen` loop's own `_run_neighbours` call also takes `cores`, so a connective beside a core looks past it; where a credential or nothing stands beyond, it stays a lone suffix word rather than joining, and the core it stood beside -- now a lone piece with nothing joined to it -- is dropped as #206 drops any lone core, the same way it already drops one between two ordinary post-nominals: `Smith, John, PhD - i Soler` 'PhD - i Soler' -> 'PhD, i Soler', `Smith, John, PhD - i - MD` 'PhD - i - MD' -> 'PhD, i, MD', `Smith, John, - i Puig` '- i Puig' -> 'i Puig', `Smith, John, Puig i -` 'Puig i -' -> 'Puig i'. MEASURED 2026-09-26: 191 of 8,097 generated core-bearing tail texts (the three heads `Smith, John,`, `Smith, John, Jr.,` and `John Smith,`, each followed by every 2-4-word product of {PhD, MD, Puig, i, y, -, Jr., Soler, Mr.} containing a core) move `suffix` and no other field, none reachable at the default policy; the comparison is each text parsed under `extra_suffix_delimiters=(" - ",)` with `_run_neighbours` handed its cores and with it handed none. Not repaired here: a core the join MERGES into a joined piece -- interior (`… Puig Dr. i - y Soler`, still 'PhD i - y Soler') or at the edge a link joins across, a name word standing beyond it rather than a credential or nothing (`Smith, John, Puig - i Soler` 'Puig - i Soler', its separator-less twin 'Puig i Soler'; `Smith, John, Puig i - Soler` 'Puig i - Soler', measured 2026-09-26) -- is not dropped from the suffix text; the join is right, the surviving separator is the gap. Also in that family: an ORDINARY connective's join still takes a core as its neighbour, the stepping above being for generational vocabulary alone (rules/P3) -- `Smith, John, PhD - and MD` keeps suffix 'PhD - and MD' (measured 2026-09-26), pre-existing and unchanged here. Open: #549, which carries both the surviving core and this one. +- 2026-10-03 #549 — SUPERSEDED: the skip reading the #538 entry above settles is reversed, and a declared delimiter in a trailing part now parts it as a comma would, so `Smith, John, PhD née Puig - i Soler` reads maiden 'Puig' with suffix 'PhD, i Soler', the reading #538 declined and the one 2.3.0 gave, and a marker with a core straight before or after it takes no clause (`Smith, John, PhD née - Jones` reads suffix 'PhD née, Jones', reversing the #418 entry's repair). The decision, its measurement and the invariant that replaces #538's are under C1's 2026-10-03 entry. +- 2026-09-26 #535 — THE MAIDEN WALK READS THE TRAILING TITLE CHAIN, AND EVERY STOP ASKS ONE RELEASE QUESTION OF THE SPAN IT GIVES UP (Derek, 2026-09-26). This resolves the NOT TRANSPARENT HERE clause of the 2026-09-19 #533 entry above, which recorded `Jane Doe nee Smith MA Prof.` and `Jane Doe nee Smith Prof. MA` disagreeing and deferred the pair as one decision about what "trailing" means inside a clause. BOTH HALVES WERE TAKEN TOGETHER: a trailing title ends the clause (`Jane Doe nee Smith Prof.` reads title 'Prof.', maiden 'Smith', where 2.3.0 read maiden 'Smith Prof.'), and a credential or numeral standing in front of that title gets the stop it gets with the title absent (`… Smith MA Prof.` gives suffix 'MA', `… Smith V Prof.` suffix 'V'). Only the first half would have left the pair disagreeing in the other direction; H5's transparency is a statement about the peel and the chain read together to their fixed point, so the walk now reads the end of the clause the way assign reads the end of the name — `tail_reading`, the shared predicate mechanisms.md#ONE-PREDICATE-PER-QUESTION names — and over the forms the transparency property test below exercises, the two spellings give one answer. They split where something in or ahead of the clause stands to take the title — for example a particle chain (ahead of the clause or inside it), a bound-given join, a name left with no name word, or, after a family comma, a title word in front of a class member with a title behind it, which the given part's chain stops short of exactly as it does with no marker (`Doe, Jane nee Smith Rev. MA Prof.` reads maiden 'Smith Rev.', title 'Prof.', suffix 'MA', where `… Rev. Prof. MA` reads title 'Rev. Prof.', maiden 'Smith', suffix 'MA'; bare `Doe, Jane Rev. MA Prof.` reads middle 'Rev.') — and the Accepted pairs recorded here and in rules/M2 are examples of that, not a complete list. `tests/v2/test_properties.py::test_a_trailing_title_is_transparent_to_the_maiden_clause` holds that as an invariant over the two inputs, over heads and runs where the credential is given up and nothing ahead would take the title, with its negative control at e0f1a2fa in its docstring. ACCEPTED, THE TRANSPARENCY BOUNDARY: where the clause KEEPS the credential the spellings differ, because a clause is one contiguous run and a title inside the kept text cannot leave without the words behind it — `Doe nee Smith ba Prof.` reads title 'Prof.', maiden 'Smith ba', and `Doe nee Smith Prof. ba` maiden 'Smith Prof. ba'; `abdul nee Smith MA Prof.` reads title 'Prof.', family 'abdul', maiden 'Smith MA', and `abdul nee Smith Prof. MA` given 'abdul', maiden 'Smith Prof. MA' (2.3.0 kept every word in all four). The report differs with them: `Doe, Dr. nee Smith MA` and `… Prof. MA` report `suffix-or-name` on the kept MA, `… MA Prof.` reports nothing, the kept member no longer ENDING the clause, which is what M2's report is asked of. rules/M2 carries the first pair as an Accepted example. ACCEPTED, FURTHER SPLITS, measured 2026-09-26. Where something ahead of the clause would take the title, the spellings differ even with the credential given up in both, because the first-suffix-word stop ends the clause before the title check runs and the title check alone declines: `Jane van der Berg nee Smith PhD Prof.` reads title 'Prof.', maiden 'Smith', suffix 'PhD', and `… Prof. PhD` maiden 'Smith Prof.', suffix 'PhD' — H5's accepted particle-chain boundary inherited, the bare `Jane van der Berg Smith PhD Prof.` / `… Prof. PhD` splitting the same way — and `Berg, abdul nee Smith PhD Prof.` (bound-given join) and `Doe, Dr. nee Smith PhD Prof.` (no name word left) split likewise; the parent d9d80492 and 2.3.0 read all of these as the tree does, so only the claim is new. And a released particle with a title behind it is withdrawn: `Jane Doe nee Smith DO Prof.` reads title 'Prof.', maiden 'Smith DO', where `… Prof. DO` reads title 'Prof.', maiden 'Smith', suffix 'DO' (2.3.0 maiden 'Smith DO Prof.' and 'Smith Prof. DO'); `do` likewise. A particle INSIDE the clause does it from the other side, its chain able to take the credential behind it: `Jane Doe nee Smith do MA Prof.` reads title 'Prof.', maiden 'Smith do MA', and `… do Prof. MA` title 'Prof.', maiden 'Smith do', suffix 'MA' (2.3.0 kept every word in both). rules/M2 carries a pair of each shape as Accepted examples. ACCEPTED, H5'S REACH: `Jane Doe nee Smith King.` reads title 'King.', maiden 'Smith' (2.3.0 maiden 'Smith King.'), H5's accepted `Mary Jane King.` cost now reaching the last word of a birth name as it reaches the last word of a current one; rules/M2 carries it as an Accepted example. READER SCOPE IS UNCHANGED: the chain is consulted only where a trailing rule reads the clause's words (no comma, the part before a suffix comma, the given part after a family comma); before a family comma and in a tail segment the walk reads the peel alone and the clause keeps the title (`Doe nee Smith Prof., Jane` keeps maiden 'Smith Prof.'). THE FIRST-WORD FLOOR MOVED INTO THE CHAIN. `trailing_titles` and `tail_reading` take a `floor` (1 everywhere else), and the walk passes the position just past the marker's first word, so the chain never takes that word: `Jane Doe nee King.` keeps its maiden name (TITLES holds borne surnames and the marker announced one), and `Jane Doe nee Prof. Dr.` keeps 'Prof.' and gives up 'Dr.'. The first draft applied the floor as a clamp on the title stop after the chain had run, and that is measurably wrong rather than merely inelegant: a chain allowed to take the first word has already spliced it out of the count the re-peel reads, so `Jane Doe nee King. ba` kept 'ba' in the clause where `Jane Doe nee Smith ba` gives it up. `test_a_title_first_word_counts_as_a_word` is the invariant (a title-vocabulary first word against an ordinary one, the credential behind it read alike) and its docstring records the clamp's failure count. ONE RELEASE CHECK FOR THREE STOPS. The numeral, the credential and the title stop each ask `_release_reads_off` whether the name the take would leave reads what they give up as titles or suffixes with no join below the take absorbing it — rules/M2's invariant, "a word the clause gives up reads as a post-nominal or the clause keeps it", which the title stop now answers to as well. It is asked of the whole SPAN a stop gives up, not of the stop's own word: a first draft that checked the word alone broke the invariant on 513 of a 5,198-parse sweep (measured 2026-09-26 on that draft, which is not in the tree to recompute), because a stop gives up every word behind it — `DOE NEE SMITH PROF. MA` stopped at the title and handed the MA to the family, where the name left standing reads MA as a name; it keeps maiden 'SMITH PROF. MA' now. The question is asked per reader, the way that reader reads: the TRAILING reader is assign's own `tail_reading` over the view; the GIVEN_SLOT reader has its count settled by the comma, so it is the writing alone with a name word ahead. The name-word-ahead half is what keeps `Dr. nee Jones Smith Prof.` whole — the take would leave `Dr. Prof.`, whose Prof. would be the family name. THE JOINS. A released span holding a title with a particle ahead of it declines, because P2's chain runs on over a trailing title (rules/H5's Accepted `John van der Berg Prof.`): `Jane van der Berg nee Smith Prof.` keeps maiden 'Smith Prof.', and a released particle with a title behind it is withdrawn for the same reason (`Jane Doe nee Smith MA do Prof.` keeps maiden 'Smith MA do'). The bound-given (P5) half of `_join_takes_the_member` is asked for the GIVEN_SLOT reader alone: P5 is `BoundJoin.LENIENT` only after a family comma, and before one its STRICT reserve already refuses a join that would change a suffix reading, so asking there over-declined — `abdul nee Smith V` kept 'V' in the clause, where the scoped check lets it go to suffix as 2.3.0 did. The consequence is recorded by a case row rather than prevented: `abdul nee Smith Dr.` reads title 'Dr.', family 'abdul', maiden 'Smith', which is how `abdul Dr.` reads bare (2.3.0 read given 'abdul', maiden 'Smith Dr.'). THREE NUMERAL-STOP READINGS MOVE WITH IT, decided in rather than deferred, since each is the same invariant broken at the stop this change was already rewriting. The numeral stop never asked the join question: `Berg, abdul nee Smith V` read given 'abdul V' at 2.2.0 and 2.3.0 (given 'abdul nee', suffix 'V' at 2.0.0 and 2.1.0, #411's reserve differing between those pairs before the walk runs) and reads maiden 'Smith V' now. After a family comma the given slot reads a lone numeral as a suffix only where the given part is the LAST comma part — assign's own two-segment condition from #144 — and the walk now asks it too (`tail_follows`): `Doe, Jane nee Smith V, PhD` read middle 'V' at 2.2.0, 2.3.0 and e0f1a2fa, an M2 violation predating this change that its first draft had extended to `… V Prof., PhD`, and it reads maiden 'Smith V' now, as 2.0.0 read it. The withdrawal reaches through a title behind the numeral, so `Doe, Jane nee Smith Prof. V, PhD` keeps maiden 'Smith Prof. V' where `Doe, Jane nee Smith V Prof., PhD` gives up the title — the same asymmetry the bare given slot has with no marker in it (`Doe, Jane Prof. V, PhD` against `Doe, Jane V Prof., PhD`). And the numeral stop reads FROM the marker, unlike the other two, so a numeral straight after the marker is not held to the first-word floor, and it may decline the clause; examples, not a rule over every shape: `Dr. nee V` and `Doe, J. nee V` keep maiden 'V' as 2.3.0 did, though `Doe, J. V` reads suffix 'V', and `Doe nee V, Jane` and `Smith, John, PhD nee V` read no clause at all (family 'Doe nee V', suffix 'PhD nee V', as at 2.3.0) — `Jane Doe nee V Prof.` read maiden 'V Prof.' at 2.3.0 and reads family 'nee', suffix 'V', title 'Prof.' now, as `Jane Doe nee V` reads plus the title. After a family comma with another comma part behind the given one, the same #144 condition that keeps `Doe, Jane nee Smith V, PhD` whole now keeps a lone numeral in the clause as well: `Doe, Jane nee V, PhD` read given 'Jane', middle 'nee V', suffix 'PhD' at 2.3.0 and at the parent d9d80492 and reads maiden 'V', suffix 'PhD' now, as 2.0.0 and 2.1.0 read it, `Doe, Jane nee V, Jr.` likewise — the marker had been read as a name word there at 2.2.0 and 2.3.0, and a case row records it. A LINK THE WALK STOPS AT ASKS THE RELEASE QUESTION TOO. Reading the end of the clause through the title chain makes the link exception refuse a link it used to join — in `Jane Doe nee Smith i DO Prof.` the DO is the peel's once the title is chained — and a stop at a link gives up the words behind it. Unchecked, that put 'Doe i' in the middle name and 'DO Prof.' in the family (e0f1a2fa read maiden 'Smith i DO Prof.'), and `Berg, abdul nee Smith i V Prof., MD` read middle 'V'. So a link the exception refuses asks `_release_reads_off` of the run it would give up — only where the exception, bounded by the peel over the words as WRITTEN, would have joined it (the refusal is the title chain's), where a trailing rule reads the clause, and where the link is not the first word after the marker (a first-word stop declines the clause and gives nothing up); where that fails the clause keeps the link and walks on. Scoped that narrowly because a first version asked it of every refused link and kept links the name left standing reads off: `Doe, Jane nee Smith i V` kept maiden 'Smith i' where every release gives up suffix 'i V' (23,472 of a 411,936-parse link grid moved clean readings, measured 2026-09-26). With it, the given-part model reads the lenient numeral in both of assign's passes — the literal last piece first, then the last one standing once the chain has taken the titles behind it — and only where no comma part follows (`Doe, Jane nee Smith i V Prof.` gives up 'i V' and the title, as bare `Doe, Jane i V Prof.` reads them; `… i V Prof., PhD` keeps maiden 'Smith i V'). The same model moves the credential stop where a lenient numeral follows the credential: `Doe, Jane nee Smith MA V` now gives up 'MA V' (suffix 'MA V', maiden 'Smith') as bare `Doe, Jane MA V` reads it, where d9d80492 and 2.3.0 read maiden 'Smith MA', suffix 'V'; `… MA V, PhD` keeps maiden 'Smith MA V'. Over that link grid (links i, y, e, and, i y, y i, and i × 8 heads × bodies × `, MD` tail × 3 cases × 3 name orders), against the tree before the link check: 0 new violations, 6,195 fixed, and the clean readings that move now match the bare given part's (`DOE, JANE NEE SMITH MA I` gives up 'MA I', as `DOE, JANE MA I` reads suffix 'MA I'). `Jane Doe nee Smith i DO Prof.` now reads title 'Prof.', maiden 'Smith i DO' and reports the kept DO, and `Berg, abdul nee Smith i V Prof., MD` maiden 'Smith i V Prof.', suffix 'MD'. A RELEASED TITLE THAT IS ALSO A PARTICLE IS KEPT AFTER A FAMILY COMMA, because P6 attaches a particle trailing the given part to the family: released by the title stop, 'St.' in `Doe, Jane nee Smith St.` reads family 'St. Doe', so `_join_takes_the_member` declines it and the name reads maiden 'Smith St.', as e0f1a2fa and 2.3.0 read it, and so does `Doe, Jane nee Smith MA St.`. Titles only — a credential that is also a particle ('DO') is the given slot's own lean, which the credential stop has already asked (#533). A FOURTH MEASUREMENT, against e0f1a2fa rather than d9d80492: the M2 invariant over a fuzz of twelve heads ({`Jane Doe`, `Doe, Jane`, `Doe, Prof.`, `Jane Doe, PhD`, `Doe, J.`, `Berg, Jane van der`, `Jane van der Berg`, `Berg, abdul`, `abdul Berg`, `J. Doe`, `Prof. Jane Doe`, `Dr.`}), ` nee Smith ` and every one-to-three-word sequence over {MA, Ma, V, Prof., St., King., PhD, ba, do, DO, Jr., i, van, y} holding at least one of Prof., St., King., then nothing or `, MD`, as written, upper- and lower-cased (107,352 parses, measured 2026-09-26 on the narrowed tree; the intermediate tree above read 2,228): 588 texts that violated it at e0f1a2fa no longer do, and 12 newly do, all `Dr. nee Smith i MA|V` followed by `Prof.`, `King.` or `St.`, with or without `, MD`. Each is the title-carrying twin of `Dr. nee Smith i MA` / `Dr. nee Smith i V`, which read family 'i' at e0f1a2fa already: the bare-title head leaves no name word, the unguarded stop's class (#548), which the title now reaches because it leaves the clause. ACCEPTED: `Jane Doe (nee Smith Prof.)` reads title 'Prof.', maiden 'Smith' — bracket content ending in a period is suffix-shaped (S1), so the brackets are dropped and the clause is read bare, outside the reach of the delimiter precedence; rules/M2 carries it as an Accepted example. MEASURED 2026-09-26, the tree against its parent d9d80492, over three generated grids, and these are dated snapshots. Recipe: grid A is each head in {`Doe, Jane`, `Doe, J.`, `Doe, Prof.`, `Berg, abdul`, `Doe, Jane van`, `Jane Doe`, `J. Doe`, `Dr.`, `abdul`, `Jane van der Berg`, `John`}, then ` nee `, then every sequence of one to three words (repetition allowed) from {Smith, Jones, Prof., Dr., Sir, MA, DO, PhD, V, III, i, do, van, Ma, M.A., King., Rev., ba}, then either nothing or `, PhD`, each text as written, upper-cased and lower-cased, duplicates removed — 327,936 texts; grid B, aimed at the bound-given join, is the same construction over heads {`Dr. abdul`, `abdul rahman`, `Dr. abdul rahman`, `Berg, Dr. abdul`, `Berg, abdul rahman`, `abu`, `Berg, abu`, `Mr. abu`, `Dr.`, `Berg, Dr.`, `abdul`, `Berg, abdul`}, words {Smith, Prof., MA, V, do, van, PhD, Ma, ba, III, Jr., M.A.} and tails {nothing, `, PhD`, `, Jr., MD`}, plus these fourteen texts, as written only: `Jane Doe nee Ph. D. Prof.`, `Jane Doe nee Ph. D. Smith Prof.`, `Jane Doe z domu King. ba`, `Jane Doe z domu Smith Prof.`, `Jane Doe nee King. Prof. ba`, `Jane Doe nee King. Prof.`, `Doe, Jane nee Smith V,`, `Doe, Jane nee Smith V, ` (with the trailing space), `Doe, Jane, nee Smith V`, `Doe, Jane nee Smith V Prof., PhD`, `Doe, Jane nee Smith Prof. V, PhD`, `Doe, Jane nee Smith MA, PhD`, `Doe, Jane nee Smith V (Jr.)`, `Doe, Jane nee Smith V "Bo"` — 173,057 texts. The invariant tested is M2's: in a parse with a non-empty maiden field, every token written after the ` nee ` marker is roled maiden, title or suffix (so the two `z domu` texts are parsed and not checked). Grid C, a four-word probe of the title chain after a family comma, is each head in {`Doe, Jane`, `Jane Doe`, `Doe, J.`, `John Smith`, `Doe, Jane Mary`}, then ` nee `, then every sequence of four words (repetition allowed) from {Smith, Jones, Prof., Dr., King., MA, PhD, V, Jr., ba, do}, then either nothing or `, PhD`, as written only — 146,410 texts. Per grid, the counts are: texts that newly violate the invariant at the tree, texts that violated it at the parent and no longer do, texts that violate it at both. Grid A: 0, 2,012 and 17,854; grid C: 0, 1,368 and 15,534; grid B: 0, 1,222 and 11,149 (all re-measured 2026-09-26 on the tree with the link and particle-title checks below, as narrowed; an intermediate tree whose link check fired on every refused link read grid A 0, 3,772 and 16,094, its extra 1,760 fixes being links the as-written reading refuses too, which are #548's class), one of those last being the check itself counting the quoted nickname in `Doe, Jane nee Smith V "Bo"`. In grids A and B, every other remaining violation has the clause ending immediately before a word of the unambiguous suffix vocabulary written the way the plain suffix-word stop takes it (PhD, III, M.A., Jr., and a lower-case i or v; never a bare capital, which reads as an initial), which is where that stop ends a clause without asking a release question. DEFERRED: the walk's plain suffix-word stop — the one ending the clause at the first suffix word, as distinct from the trailing numeral, credential and title stops — asks no release question at all, and where the words behind it cannot read as post-nominals they land in a name part: on degenerate heads the stop word itself does (`Dr. nee Smith PhD Prof.` reads family 'PhD', `Doe nee Smith Jr. Prof., Jane` family 'Doe Prof.'), and with an ordinary head the words behind it do (`Doe, Jane nee Smith PhD Smith` reads middle 'Smith'). Unchanged here, and what "the clause keeps it" should mean where the kept words would follow a credential the clause itself ended at is its own question; rules/M2 carries `Doe nee Smith Jr. Prof., Jane` as an Accepted example meanwhile. The trailing numeral's stop has the same gap before a family comma, where it is made over the peel alone with no release question: `Doe nee Smith V, Jane` reads family 'Doe V', maiden 'Smith' (so did 2.2.0, 2.3.0 and the parent d9d80492; 2.0.0 and 2.1.0 read maiden 'Smith V'), and rules/M2 carries it beside the other. Open: #548. COST, measured 2026-09-26 on CPython 3.11.16 as profiler call events in one `Parser.parse` after a warm-up parse, parent → tree: `John Smith` 171 unchanged, as are `Jane Doe Prof.` and `John Smith MA`; `Jane Doe nee Smith` 244 → 246; `Jane Doe nee Smith MA` 377 → 388; `Jane Doe nee Smith Prof.` 276 → 372, the one shape that now runs the chain and a release check it never ran. Recompute: count `sys.setprofile` call events around the second of two `Parser().parse` calls, with the tree and d9d80492 each first on `sys.path`. +- 2026-09-27 (Derek), #544 — S2'S COMPANY CLAUSE DOES NOT REACH ACROSS A CLAUSE, recorded as an Accepted boundary rather than repaired. `Jane Doe Jr. nee Smith Ma` keeps maiden 'Smith Ma' and reports it, the clause's own words standing between 'Jr.' and 'Ma', where the clause-less `Jane Doe Jr. Ma` reads suffix 'Jr. Ma'. tests/v2/test_properties.py's clause-agreement walk pins the pairs that differ for exactly this reason as its `anchored_head` class — 810 of its 20,412 pairs, recorded 2026-09-27, 0 with the anchor off. Inside the clause the company reads as it does anywhere: `Jane Doe nee Smith PhD MEng` ends the clause at 'PhD' and reads suffix 'PhD MEng', maiden 'Smith'. + +- 2026-10-04 (Derek), #601 — A MARKER COUNTS ONLY BEHIND A SURNAME, AND IT TAKES THE WORDS AFTER IT UP TO THE TRAILING RUN THE NAME WOULD END WITH IF THE CLAUSE WERE NOT WRITTEN. This SUPERSEDES the release machinery of the #533 and #535 entries above (the release question, the per-stop checks, the anchored-head boundary of the #544 entry) and the given-part and suffix-comma reach the 2026-07-03 rule had; the #399, #420, #434 and delimiter entries stand. The trigger was #548, `Dr. nee Smith PhD Prof.`, where 2.3.0 read family 'PhD', maiden 'Smith': a clause whose head was nothing but a title, and a release check asking whether each word it gave up could still read as a post-nominal. #548 was closed into this issue and #602 (S2's run) because both halves of its answer were rules rather than repairs. THE HEAD RULE (Derek): a marker is one only in the name before any comma or the surname part before a family comma, behind at least one name word past the leading title run, and not straight behind an unambiguous suffix word or a connective. Anywhere else it is an ordinary word — in the given part after a family comma (`Doe, Jane nee Smith` reads middle 'nee Smith', 1.4.0's reading) and in a part after a suffix comma (`Smith, John, PhD née Jones` reads suffix 'PhD née Jones', also 1.4.0's). Derek's framing was that a maiden marker always follows a family name, and that a marker following something the parser has decided is a title, suffix, particle or connective is not a marker. THE PARTICLE HALF WAS DROPPED BY MEASUREMENT: prototype variant 1 refused a marker behind particle vocabulary and so declined `Anh Do née Tran` and `Mai Le née Nguyen`, Do, Le and Van being particles that are also the commonest Vietnamese surnames; a particle counts as the surname (`Jane van nee Smith` reads family 'van', maiden 'Smith'). THE BOUND (Derek): the clause takes its words up to the run of post-nominals and titles that the trailing rules read at the end of the name with the clause removed, and the words it gives up take the roles that reading gives them — the TRAILING rule's reading of the clause-free name, not the full parse of it, so a head the full parse would read differently (H1's lone title, a P5 join) cannot change where the clause ends. The first word after the marker is always taken, the marker having announced a name — a lone roman numeral included, so `Jane Smith née V` reads maiden 'V' where the clause-free `Jane Smith V` reads suffix 'V', accepted rather than given a numeral exception (rules/M2's Accepted block) — unless it is an unambiguous suffix word, where the marker declines (`Jane Smith nee PhD` reads family 'nee', suffix 'PhD', as before). Before a family comma the clause keeps the 2026-07-03 stop at the first suffix word, since no trailing rule reads that part's end. VARIANTS, prototyped 2026-10-03 behind an environment switch on one tree (the numbers are in #601's comments): v2/v3 grew the clause until the existing release check accepted it and reached 0 invariant violations but inherited the check's caution about particles (`Anh Do geb. de la Vega PhD Prof.` kept 'PhD Prof.' in the maiden name); v4 read the run once over the name as written without binding it, and is the recorded negative control in tests/v2/test_properties.py (350 of the clause grid's 4,482 parses fail under it, measured 2026-10-04); v5/v6 bound that reading as roles but still counted the clause's words among the words to spare, so `John nee Smith ba` read 'ba' as a credential where `John ba` reads a name; v7, the clause-free reading bound as roles, was adopted. Against #535's grids v7 left 0 violations of the M2 invariant where the tree then had 15,586 (grid A, 327,936 texts) and 8,713 (grid B, 173,057), measured 2026-10-03 with an invariant check written to #535's recipe. WHAT THE PROTOTYPE MISSED, found implementing it: the numeral fork. A trailing roman numeral is a suffix only by its shape against the word before it, and v7 read the clause's last word without that test, so `John née Jones Smith VI` kept 'VI' in the maiden name; the take now asks `is_trailing_numeral_suffix` of the last piece, and cases.py's `a_numeral_the_clause_free_name_reads_ends_the_clause` pins it. And the head's title run is measured over the words before the marker, a lone title excepted: measured over the whole segment, `Lord Chancellor née Jones` counted 'Chancellor' as a title and refused the marker. TRIAGE, 2026-10-04: the prototype's 68 failing case rows (40 facade failures among them) were re-pinned as `fix(#601)` with their group named in the note, bar the head-rule bug above; tests/v2/test_parser.py's corpus-append invariant became two-sided (appending ` née X` moves no other field, OR the take records why it declined), with a named set for the suffixes that themselves hold a marker. DEGENERATE INPUT, NO READING DESIGNED FOR (Derek, 2026-10-04: garbage in, garbage out). Two names that read worse by eye moved, and they are recorded here so the move is not mistaken for a regression, not because either reading is wanted: `Berg, Jane van der nee Smith DO` reads middle 'van der nee Smith', family 'Berg' where 2.3.0 read family 'van der Berg', maiden 'Smith DO' — the marker in a given part being a word, the particles no longer trail the given name (P6); and `Jane Doe, PhD née Smith` reads given 'PhD', middle 'née Smith', family 'Jane Doe' where 2.3.0 read given 'Jane', family 'Doe', suffix 'PhD', maiden 'Smith', which 2.3.0 reached only through the clause taking the marker (the comma part opening with a credential is #603's shape). Measured 2026-10-04 against 2.3.0. COST: the take is one function and no release helper survives; the #533 review's comma-head grids could no longer reach a take at all and were replaced by counting heads (tests/v2/test_properties.py's `_maiden_clause_grid`, 4,482 parses). +- 2026-10-04 #610 — THE MAIDEN TAKE'S CANDIDATE WORDS ARE ONE FUNCTION BESIDE THE PEEL, AND SHARING IT FIXED A GAP THE HAND COPY HAD. `_maiden_take` chose the clause words to read its clause-free view over with a filter of its own (`_clause_tail_word` plus a numeral clause), a hand copy of the peel's and the chain's admissions whose contract — a superset of what `tail_reading` takes — nothing stated or tested. It is now `_pieces.trailing_candidates`, written beside `peel_trailing`, and tests/v2/pipeline/test_pieces.py pins the superset at the take itself, on the unjoined pieces the take is handed, over a generated grid (37,044 clauses through `group`). The grid found the copy wrong twice. The numeral fork reads a shape-only roman numeral (`VI`, in no wordlist) as the WALK's last piece, and two things leave it last without standing at the end of the name: a period-marked title behind it, which the chain takes first, and the merged `Ph. D.`, which group flags and the walk never holds. The copy admitted the numeral only as the clause's last word, so `Jane Doe nee Smith VI Prof.` kept maiden 'Smith VI' where `John Smith VI Prof.` reads suffix 'VI', and `Jane Doe nee Smith VI Ph. D.` the same way. Both now end the clause at the numeral, as rules/M2's statement already said they should. Recorded negative control: the loop as #601 shipped it misses 432 of that grid's clauses, every one this shape. The test's first version read plain names through all of `group`, on pieces the joins had already merged, which review caught: there the old loop missed 288 names, and a first draft of the fix, covering the title half only, still missed 111 — the `Ph. D.` half, which that grid is what found. Against master 8a8459a8 over the 95,813-text equivalence grid the change moved 10 readings, all of this class. The same issue lifted P6's particle-tail walk into `_pieces.particle_tail`, read off roles, so post_rules' attachment and assign's given-part credential run (#602) ask one walk; that half moved none of the 95,813 readings. + diff --git a/docs/design/decisions/M3-marker-led-clause.md b/docs/design/decisions/M3-marker-led-clause.md new file mode 100644 index 00000000..65478fe9 --- /dev/null +++ b/docs/design/decisions/M3-marker-led-clause.md @@ -0,0 +1,11 @@ +# M3 — the marker-led clause + +- 2026-08-26 #335 (M3, PR on fix/335-maiden-marked-clause) — a bracketed clause whose content opens with a recognized marker word and carries a word after it reads as the maiden name, whichever bucket the enclosing pair sits in. The parse already had the information: "Jane Smith née Jones" gave maiden "Jones" while "Jane Smith (née Jones)" gave nickname "née Jones", because ( ) is a nickname pair by default and nothing looked inside it. The clause says "maiden" out loud, so the caller should not have to say it in Policy. +- 2026-08-26 #335 — the MECHANISM, and the issue proposed the other one. Masking the delimiters and letting M2's bare-marker rule consume the content would avoid a second implementation, and it was rejected on measurement: the two readings agree on a trailing clause and diverge on an interior one. "Jane (née Jones) Smith" reads given "Jane", family "Smith", maiden "Jones" under extraction; masking turns it into the bare "Jane née Jones Smith", which reads maiden "Jones Smith" and family "" (measured 2026-08-26), because M2's take runs to the end of the name. The closing delimiter is a boundary the writer typed and masking throws it away. Extraction was also the SMALLER change — extract already emits Role.MAIDEN regions and group already drops the marker inside one (#329), so this is a role swap at the existing emit site plus one predicate, in the branch S1 has already declined. That placement is what keeps S1 first, but be exact about the mechanism, because the obvious mutation does not test it: hoisting the swap above the _suffix_shaped call changes nothing at all (measured 2026-08-26), since the S1 branch never reads `role` -- it masks the two delimiter spans and hands the content to the token stream, and the role it would have carried is never consulted. What inverts the precedence is giving M3 the clause first, an early `extracted.append((Role.MAIDEN, inner))` ahead of the S1 test: that reads "Jane Smith (née Jr.)" as family "Smith", maiden "Jr." instead of family "née", suffix "Jr.", and M3's own example line for it fails. So the precedence is pinned, and by that example rather than by the branch shape. +- 2026-08-26 #335 — the condition is a word AFTER the marker, not the marker alone. The reason is M1's: a one-word clause keeps its word because Nee is an attested surname (Irish Ní/Nee, and a Chinese romanization), and a marker with nothing after it marks nothing. Be precise about what pins it, because the obvious answer is wrong: M1's own boundary line carries the maiden-parens annotation, so it asserts the CONFIGURED reading and passes either way — built and measured 2026-08-26, a len(words) > 0 predicate still gives it maiden "Nee". What the loosened predicate actually breaks, measured by mutation 2026-08-26, is M3's own boundary example ("Jane Smith (née)" → nickname "née") and tests/v2/cases.py::maiden_marked_clause_one_word_stays_a_nickname in both runners — three pytest failures — AND the 2.1.0 differential gate, which goes red with three unexplained names: "Jane Smith (Nee)", "Jane Smith (Nee) (Jones)" and "Jane Smith (née)". The gate is the stronger signal and the one to look at first. The middle name is why: loosened, it reads maiden "Nee" and nickname "Jones", splitting one bracket pair into the maiden field and the next into the nickname field, which is the direct contradiction of M1's "two enclosures read as one maiden name". Anyone re-litigating the second-word test should check those, not M1's line. So the real division of labour is NOT markerless versus marker-led, which is how it was first written into rules.md and had to be corrected at review: it is one word versus a marker plus a word. "Jane Smith (Nee)" is marker-led and still reads nickname "Nee" by default, maiden "Nee" only with the pair configured (both measured 2026-08-26). +- 2026-08-26 #335 — accepted consequence: the word taken after the marker is not tested for being a name word, so M3 does not stop at a suffix word where M2's bare take does. "Jane Smith (née V)" reads maiden "V" while the bare "Jane Smith née V" reads suffix "V" and family "née" (both measured). The reason is structural rather than an oversight: M3 hands the WHOLE clause to Role.MAIDEN and the bracket is its right boundary, so the trailing-suffix rule never sees the word as trailing the name at all. Pinned as M3's Accepted line in rules.md, which puts "Jane Smith (née V)" in the differential corpus and classified in all three ledgers. +- 2026-08-26 #335 — rules/M3 lists implemented: nameparser/_pipeline/_extract.py alone, and that is deliberate rather than an omission. M3 decides a REGION'S ROLE; the marker drop it describes ("the marker itself dropped, as M1 drops it") is M1's mechanism in _group.py, which cites M1 and is governed by it. Adding _group.py to M3's list would mean inventing an M3 citation there for behavior M1 already owns — and test_doc_citations requires implemented: to equal the set of modules that cite the rule, so the two would have to move together. Do not re-open this. +- 2026-08-26 #335 — the durable finding, and the one worth reading before touching either side: _maiden_marked asks a WHITESPACE-SPLIT, _normalize'd question of the clause's first word, while tokenize and classify ask a token-level one. _normalize strips a trailing period but not a comma, so 'née,' is not a marker to M3 while the tokenizer splits the comma off and still tags the token vocab:maiden-marker. That divergence is the only thing keeping the "role is not Role.MAIDEN" branch of _group.group's clause-scoped drop pass reachable at all: without it, a marker-led NICKNAME clause could no longer exist, since M3 would have converted every one to MAIDEN before group ran. Anyone "unifying" the two tests would orphan that filter silently. The pin is tests/v2/cases.py::marker_glued_to_punctuation_keeps_the_clause_a_nickname, and it needs BOTH of its clauses — the drop pass is gated on the name holding a maiden region at all, so a marker-glued nickname clause standing alone leaves the branch unexercised (measured by mutation, 2026-08-26). The row that used to pin the filter, maiden_marker_delimited_beside_a_nickname_clause, lost the job to M3: both its clauses are marker-led, so both are maiden now and there is no contrast left in it. SUPERSEDED IN PART 2026-10-12 by #653: a clause's comma now stays on its word, so 'née,' is no marker to the tokenizer either and the comma row no longer reaches the branch; the divergence lives on through rules/T2's nakaguro, and the pin is cases.py::marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname (see #N1's #653 bullet). +- 2026-08-26 #335 — M3 makes the marker vocabulary act in a THIRD position. M2's claim acts on the word after a name word; M1's on the content of a configured pair; M3's now on the first word of any bracketed clause, under the default policy. #vocabulary-collisions C-i is keyed on the position a claim acts on, so extending where a rule acts re-asks C-i of the whole set it reads, and nothing in the process prompts that — the rule was designed, reviewed and approved without the set being re-examined. Review afterwards found one entry that fails C-i, the Czech abbreviation roz, and its disposition is the Excluded (MAIDEN_MARKERS) block in decisions/vocabulary-exclusions, which is keyed to the vocabulary set rather than to any rule: it was removed. Two things worth separating, since the finding surfaced during this work and is easy to file under it — the defect was M2's and predates #335 (bare "Rosalind Roz Smith" lost its family name on master), and M3's own boundary already excluded the common one-word "(Roz)" spelling. What belongs to M3 is only the widening, and the lesson is the general one: re-ask C-i of a vocabulary set whenever a rule extends where its claim acts. +- 2026-08-26 #335 — what configuring maiden_delimiters still buys, since M3 removes the commonest reason to reach for it: markerless clauses, and one-word clauses. "Cherice J. (Johnson) Williams" is a real US convention and a corpus name, and nothing in that clause says maiden, so it stays a nickname by default — only a caller who knows their data can say otherwise. The knob is a declaration about the DATA; M3 is a reading of what a clause declares about itself. + diff --git a/docs/design/decisions/M4-maiden-marker-makes-the-lone-name-word-the-family.md b/docs/design/decisions/M4-maiden-marker-makes-the-lone-name-word-the-family.md new file mode 100644 index 00000000..8353df52 --- /dev/null +++ b/docs/design/decisions/M4-maiden-marker-makes-the-lone-name-word-the-family.md @@ -0,0 +1,14 @@ +# M4 — a maiden marker makes the lone name word the family + +- 2026-08-27 #445 (M4, PR on fix/445-maiden-marked-lone-name) — Derek's rationale, in his terms: normally if there is only one name we assume it is a given name because we have to pick one, but maiden markers mark a previous surname before marriage, so it would not make sense to include one if there was no other surname for it to clarify — and given that, when a marker leaves only one name word we switch the assumption and read that word as the family name. `parse("Smith née Jones")` reported given 'Smith' with an EMPTY family through 2.1 and reads family 'Smith', maiden 'Jones' since. The rule is written in post_rules as H1's sibling and placed directly under it, so the interaction is decidable by reading rather than by running: where H1 fired there is no GIVEN left for M4 to move, and where H1 declined because the title addresses by given name ("Sir John née Jones") the `not titles` guard keeps M4 off the same word. A titled name is H1's at both outcomes, which is what keeps H1's given-name-title carve-out intact. +- 2026-08-27 #445 — the third time in the 2.2 bundle that a name carrying a maiden clause reported no family, and the three share a SYMPTOM rather than a cause. State that carefully, because the first draft of this entry claimed one cause for all three and was wrong. #410 (H1): a title plus one name word reported no family the moment a suffix, nickname or maiden name stood beside it. #411 (P5's reserve): the bound given-name join counted the marker and the maiden name as words it could spend, and spent the family name. Those two ARE one cause -- a count that included words which are no part of the name -- and the grep that would find a fourth is a rule counting name words without first asking which of them are name words. #445 is not that: it changes no count anywhere (post_rules gains a new block, assign gains only a comment), and `parse("Smith")` with no clause beside it at all already read given 'Smith' with an empty family, so nothing was being swallowed. Its cause is the one the next entry gives: O4 decides nothing at one name word, and the reading came from a convention nobody had written down. The site that grep would miss is `_name_positions`'s `count == 1` branch, silent for every order and every name -- which is exactly what O5 now records. +- 2026-08-27 #445 — O4 was SILENT at exactly one name word, which is why the shipped reading had no rule to point at and why O5 now exists. O4 reads a name by comparing where its words stand ("the first name word is the given name, the last is the family name"); with one word the first IS the last, so there is nothing to compare and the rule decides nothing. O5 records the reading as the convention it is — a guess fixed in advance so that the same input always reads the same way, not a determination about the word — and names H1, N3 and M4 as the rules that DO decide such a name. Written that way deliberately: a rule asserting "a lone name word is the given name" as a fact would have made #445 a contradiction of the documented behavior instead of an exception to a documented guess, and would make #449 one too. +- 2026-08-27 #445 — the two carve-outs are not inventions, and both rest on mechanisms.md#TWO-LAYER-ASSIGN: the positional layer never overrides a vocabulary claim, and M4 changes only what POSITION decided. A word the vocabulary claims as a given name keeps that reading ('abd née Jones' — `vocab:bound-given`), and so does a word written as an initial ('J. née Jones Smith V' — the `initial` shape tag; amended 2026-09-13 under P3: the tag is no longer purely a shape claim). Both are read off tags classify already recorded rather than off a predicate of M4's own, and both witnesses are real corpus names rather than constructed cases, so each carve-out has something in the differential that would notice its loss. Mutation-checked on a scratch copy: dropping the `initial` carve-out fails that name's case row, its facade twin, M4's own boundary example and the pre-existing assertion in test_the_chain_and_the_walk_stop_where_the_peel_begins; dropping `vocab:bound-given` fails the other name's two runners, M4's boundary, P5's own doc example, O5's own `abd née Jones` line and the corpus-wide maiden-clause property on 'abdul' -- six in all, the O5 line having been added by the review round, and the list is exhaustive as re-run against the final tree. +- 2026-08-27 #445 — the two halves of Derek's answer differ in their relationship to 1.4.0, and a reader should not have to re-derive it. The bracketed spelling RESTORES v1: `Smith (née Jones)` read family 'Smith', nickname 'née Jones' on 1.4.0, 2.0.0 and 2.1.0 alike, and now reads family 'Smith', maiden 'Jones' — the clause changes hands and the family name stays put, so the 1.4.0 gate's diff on that name SHRANK to {nickname, maiden}. The interior spelling is a NEW reading: `Jane née Jones Smith` read first 'Jane', middle 'née Jones', last 'Smith' on 1.4.0 and reads family 'Jane', maiden 'Jones Smith' now, the real surname being inside the maiden value by M2's greedy take. That the marker stands inside the name changes nothing for M4, which counts what the take LEAVES rather than where the marker stood — one name word left that way is one name word, which is the widest half of the decision and the half no earlier version agrees with. +- 2026-08-27 #445 — a precedence claim corrected while drafting, and it is exactly the class this bundle keeps producing. The first draft of M4 said a name carrying a nickname is N3's rather than this rule's. Measured false: `'Smitty' Jones Jr. née Smith` reads family 'Jones', because N3's count does not set a suffix aside, so N3 declines and M4 fires. A nickname holds nothing off — where N3 has already named the family M4 finds nothing left to move, and where N3 declined M4 names it. The general lesson is the one #410 and #411 already taught from the other side: a rule that counts name words and a rule that counts something else will disagree at the edges, and the disagreement is only visible if the interaction is measured rather than asserted. +- 2026-08-27 #445 — what the ledger's `# revisit when #445 lands` markers bought, and what they got wrong. Four blocks carried them, and each predicted that a "keep the family" fix would leave the diff a subset of the fields its rule already declared, so the rule would go on explaining the name and the change would be absorbed with the gate green. That was wrong in a way the rule's own shape settles: M4 MOVES the one name word rather than adding one, so `given` empties as `family` fills, the diff outgrows every declaration, and eight names arrived UNEXPLAINED at all three baselines. Two counts run through this and they are different sets, so name which is which: NINE corpus names change reading, of which eight arrive as new diffs — the ninth, 'Smith (née Jones)', has its 1.4.0 diff SHRINK instead, its family agreeing with v1 again. The four markers between them named NINE names to re-measure, of which seven moved: the two that did not are M4's carve-outs, and the two movers the markers could not name were not corpus names until this change's own examples added them. The markers earned their keep all the same — they named the right rule, the right field and very nearly the right set — so they are corrected in place rather than deleted, and the correction says what actually happened. THREE of the four moved in the opposite direction from their own prediction, and getting that number right took two passes. The fix(#335) rule for 'Smith (née Jones)' appears in all three ledgers, and every copy feared absorbing a WIDENED diff while the diff in fact SHRANK to {nickname, maiden}. The first pass narrowed only the 1.4.0 copy, on a measurement of that wheel, and left the other two at four fields on the reasoning that 2.0.0 and 2.1.0 'read `given`, and the baseline cannot move'. They did not: only the BARE spelling ever read `given`, and this rule holds the bracketed one, which reads family 'Smith', nickname 'née Jones' on every released version. So two rules stood over a two-field diff declaring four, and classify's subset test would have explained a `given`/`family` regression on that name in silence -- the absorption those very notes existed to prevent, surviving the round that answered them. A third reviewer found it by instrumenting the comparator to dump real diff-field sets, and that is the lesson worth keeping: ledger prose cannot be checked by reading it, and the round that corrects a marker is exactly where a wrong premise gets written down with confidence. +- 2026-08-27 #445 — the corpus-wide property test is the repair worth copying. tests/v2/test_parser.py's `test_a_maiden_clause_changes_nothing_else` asserts over the corpus that appending " née Jones" adds a maiden name and moves no other field, and M4 falsifies it for fourteen names. Skipping them would have bought a green suite and lost the check; instead the test computes M4's guard from the base parse's tokens and asserts the flip — family takes what `given` held, every other field standing still — so the class that used to be a hole is now the strongest witness the rule has, fourteen names against the six rows cases.py carries. It is also what executes M4's nickname precedence, `'Smitty' Jones Jr.` being one of the fourteen. +- 2026-08-27 #445 (review round) — M4 is keyed on the maiden NAME, not on a marker, and the statement was corrected to say so after two reviewers measured the same defect independently. The guard tests for a token in the MAIDEN role, and M1's caller-configured pair produces one with no marker anywhere: under `Policy(maiden_delimiters=frozenset({("(", ")")}))`, `Smith (Jones)` reads family 'Smith', maiden 'Jones', and so do M1's own boundary examples `Smith (Nee)` and `Smith (z domu)`. The BEHAVIOUR is right — the rationale transfers to a declared pair without a word of change, since what announces a former surname is the clause, not the vocabulary that marked it — so the fix was to the rule, which had said "a recognized maiden marker that takes its name and leaves exactly one name word" and now says "a maiden name standing beside exactly one name word". That covers M1, M2 and M3 uniformly, M1 is in M4's `interacts:` and M4 in M1's, and the configured path has an example line of its own. The general lesson is the one this bundle keeps teaching from a new angle: a statement written from the case that prompted the change describes that case, not the code, and only measuring the OTHER paths into the same guard finds the difference. +- 2026-08-27 #445 (review round) — a known gap recorded rather than closed, the way H1 carries its own. M4's guard counts GIVEN tokens; H1 counts nothing at all (it tests which roles are unoccupied). So a name word another rule has joined counts as several here and M4 declines where H1 fires: `Dr. Dean of Chemistry` reads family 'Dean of Chemistry' while `Dean of Chemistry née Jones` keeps given 'Dean of Chemistry', the connective join (P3) having left three GIVEN tokens. rules/P3 says the joined part is one name word wherever another rule counts them, so the two siblings genuinely disagree and this is a gap rather than a boundary. Not closed here: widening the count to units moves zero corpus names (measured), and a behaviour change nobody has approved does not belong in a branch whose blast radius was settled. The sibling claim in the code comment and in this entry is narrowed to match — sibling except in what it counts. +- 2026-08-27 #445 — the relationship to #449, briefly, because the two look like the same question and are not. #449 asks whether a lone name word with nothing to decide it should REPORT an ambiguity; #445 removes a class of names from that population by giving them something that decides. So #445 reduces #449's reach rather than competing with it, and #449's measurement — a naive "one name word, nothing decided it" condition fires on 19 of 1,085 corpus names, only three or four of them the case it is about — is re-taken after this lands. O5 cites #449 in prose and deliberately carries no `deviates:` marker: that marker asserts an intended output the runner then checks strictly, and #449 had not decided one. It has since — 2026-09-07, decisions/O5 — and the answer arrived as ordinary example lines rather than as a marker, the report being shipped behavior and not a deviation. The re-measurement this bullet promises is taken there: the naive condition is wider than either number suggests, firing on 143 of the 1123 corpus names, and O5's own population is 27. + diff --git a/docs/design/decisions/N1-default-delimiter-pairs.md b/docs/design/decisions/N1-default-delimiter-pairs.md new file mode 100644 index 00000000..bc2458bc --- /dev/null +++ b/docs/design/decisions/N1-default-delimiter-pairs.md @@ -0,0 +1,6 @@ +# N1 — the default delimiter pairs + +- 2026-10-03 #597 — THE HALFWIDTH CORNER BRACKETS ARE A DEFAULT NICKNAME PAIR. `｢｣` (U+FF62/U+FF63) join `DEFAULT_NICKNAME_DELIMITERS` beside `「」`, and the v1 facade gains the matching `halfwidth_corner_brackets` key, so `HumanName` reads them as `parse()` does. They are the same punctuation as `「」` in JIS X 0201's narrower encoding, the legacy data halfwidth katakana comes from (#594), where they were the only corner brackets available, and they appear in no non-CJK text. Before this every release kept the clause in the name, brackets and all — a middle where it stood spaced (`山田 ｢ﾀﾛｰ｣ ﾀﾛｳ`), part of one given word where it was glued (`山田｢ﾀﾛｰ｣太郎`) — and since #594 the bracket also cost the spaced name its order: the halfwidth kana now classify, but the unclassified bracket kept `山田 ｢ﾀﾛｰ｣ ﾀﾛｳ` from being wholly East Asian script, so it read given 山田 (in released versions the unclassified kana did that on their own). A v1 `Constants` restored from a pickle keeps the keys it was saved with and so does not gain the pair, the same policy #273's pairs followed: restored configuration is never widened behind the caller's back. A pair is not script-gated, so `John ｢Jack｣ Smith` reads nickname Jack as `John 「Jack」 Smith` does. Nothing else needs a halfwidth twin: Unicode has no halfwidth form of `『』` or of the fullwidth parentheses. MEASURED 2026-10-03: no corpus line held a halfwidth bracket before this change, so the three `fix(#597)` case rows are the whole population, and they move at every baseline. RECOMPUTE: parse `山田 ｢ﾀﾛｰ｣ ﾀﾛｳ`, `山田｢ﾀﾛｰ｣太郎` and `John ｢Jack｣ Smith` here and on each baseline from PyPI and read `nickname` beside the name fields. + +- 2026-10-12 #653 — A DELIMITED CLAUSE KEEPS ITS COMMA. `Jane (Janey, Jo) Smith` reads nickname 'Janey, Jo' again, as 1.4.0 did, where 2.0.0 through 2.4.0 gave 'Janey Jo'. The comma was lost in tokenize, not in the view: `_tokenize_region` serves the main stream and the extracted regions alike, and treated `COMMA_CHARS` as a separator in both, although only the main stream records a segmentation offset for it -- inside a region it divided the clause into words and recorded nothing, so the view's space join had no comma left to keep. It shipped that way with the first tokenizer (7e341a95, 2026-07-12) and no entry chose it; `;`, `/` and `&` were always kept. Now a comma separates only where it records an offset, so inside a clause it is word text on the word it touches (`Janey,`), and the view's space join restores the writer's spacing (`Janey,Jo`, `Janey , Jo` and `,Jo` all read as 1.4.0 read them). WHAT ELSE MOVED, all by the same token change: a marker glued to a comma is no marker inside a clause, as it already was in the main stream (`Jane Smith née, Jones` reads no maiden), so under `maiden_delimiters` `Jane (née, Jones) Smith` keeps maiden 'née, Jones' where the dropped comma had let the drop pass take 'née'; and a phrase marker divided by a comma inside a clause no longer matches, which the M Background already stated (`Maria Kowalska (z, domu Nowak)` keeps maiden 'z, domu Nowak' under `maiden_delimiters`; rules/M1 now pins it). A punctuation-only clause reads as its punctuation (`Jane (,) Smith` → nickname ','), as `(;)` and `(-)` already did and as 1.4.0 did — and a name holding one now has a nickname for rules/N3 to count, so `Jones (,)` reads family 'Jones' with no ambiguity report where 2.x read given 'Jones', reporting GIVEN_OR_FAMILY since #449 in 2.3 (1.4.0: last 'Jones', nickname ','; the same for `(,) Jones` and `Jones ","`). A COMMA AFTER A MARKER UNMARKS IT (rules/M3, decided in review 2026-10-12): once the clause kept its comma, a spaced one after the marker (`Jane (née , Jones) Smith`) passed M3's whitespace test, since ',' was a word after the marker, and the drop pass then took the marker and left maiden ', Jones'; `(née , ) Smith` gave maiden ','. Rather than read the spaced comma as glued `née,` is not, M3 and M1's drop now ask the one predicate `_vocab.comma_follows_marker` of the word after the marker run, so a comma standing alone or glued to either word unmarks it: `Jane (née , Jones) Smith` reads nickname 'née , Jones', which is 1.4.0's reading, where 2.2 through 2.4 read maiden 'Jones'; under `maiden_delimiters` the clause keeps its marker, `(née ,Jones)` reading maiden 'née ,Jones'. Declined: dropping the comma with the marker, which restores 2.4's maiden 'Jones' for the spaced comma while the glued `(née, Jones)` stays a nickname, so the two spellings would disagree. A trailing comma rides as written (`(née Jones,)` → maiden 'Jones,'). The two sites hand the predicate different strings — extract a whitespace word, group a token — so it reads past any leading character that is neither a comma nor a letter or digit: a second review found `Jane (née \u200f, Jones) Smith`, visually the spaced-comma name, read maiden 'née , Jones' while it tested only the first character, M3 seeing the right-to-left mark tokenize drops and group the bare comma (the same for an emoji or a nakaguro before the comma). cases.py's marker_followed_by_a_comma_behind_a_bidi_mark_is_no_marker pins it. ACCEPTED: a whitespace run inside a clause still collapses to one space (1.4.0 kept 'Janey,Jo Bo' verbatim), and the v1 `nickname_list` stays one element per word as it has been since 2.0 (1.4.0 gave one per clause), the comma now riding on its word (`['Janey,', 'Jo']`) as `;` always did. THE DROP-PASS WITNESS MOVED: `marker_glued_to_punctuation_keeps_the_clause_a_nickname` had been the only case row reaching the `role is not Role.MAIDEN` branch of `_group.group`'s drop pass, and it reached it through the dropped comma; `marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname` (`佐藤 花子 "旧姓・田中" (旧姓 鈴木)`) reaches it through rules/T2's nakaguro instead, and is now the one row that fails with the branch removed (measured 2026-10-12). MEASURED 2026-10-12: before the rules.md examples landed no corpus name held a comma inside a clause, and the gate at 2.4.0 read 0 diffs with the fix applied; with them, the two examples are the 2 intentional diffs at each of 2.0.0 through 2.4.0 under the `fix(#653)` rule, and parity at 1.4.0. The witness row joins `corpus_cjk.jsonl` and diffs at 1.4.0, 2.0.0 and 2.1.0 only, classified under the #335 rules there (a new literal-keyed `fix(#335/#272)` rule at 1.4.0, one more literal in the existing #335 rules at 2.0.0 and 2.1.0); every baseline's gate exits 0. + diff --git a/docs/design/decisions/N2-same-character-quotes-and-shared-character-conventions.md b/docs/design/decisions/N2-same-character-quotes-and-shared-character-conventions.md new file mode 100644 index 00000000..a0737750 --- /dev/null +++ b/docs/design/decisions/N2-same-character-quotes-and-shared-character-conventions.md @@ -0,0 +1,9 @@ +# N2 — same-character quotes and shared-character conventions + +- 2026-07-18 #273 (7ee6e3a) — the typographic delimiter pairs shipped with a per-pair full-text scan; three independent review agents broke it the same day: with two shared-character conventions genuinely present, the earlier-sorted pair stole the close and the legitimate match dropped silently, zero ambiguity. +- 2026-07-18 (05fe693, the same day) — the interleaved leftmost-match restructure replaced it (an altitude review called it the right depth); the author's own first fix — post-hoc offset filtering of unbalanced candidates alone — was superseded within the day and survives only as the bulk-recorded dangler filter rules/N2 now names. The word-internal carve-out covers ONLY the straight apostrophe because it is the one delimiter character that occurs mid- and end-of-word in real names; Derek's distinguishing test: with double quotes the same shape ("Mari\" Aube\"") is genuinely ambiguous, while Mari' Aube' is not. A cached delimiter-charset prescreen landed in the same commit, measured: no-delimiter names parse faster than the pre-#273 baseline. + +Declined: + +- The offset-filter as the whole fix — it repaired the reported symptom (spurious unbalanced flags) while leaving the stolen- close mis-extraction in place. + diff --git a/docs/design/decisions/N3-lone-word-nickname-rule.md b/docs/design/decisions/N3-lone-word-nickname-rule.md new file mode 100644 index 00000000..406b0f84 --- /dev/null +++ b/docs/design/decisions/N3-lone-word-nickname-rule.md @@ -0,0 +1,4 @@ +# N3 — the lone-word nickname rule + +- 2026-07 (v2 core, PR #288; recorded plan deviation #2 of the core plan) — v1's rule counted pieces before grouping; the v2 port fires only when the nickname accompanies exactly ONE piece in total — a title counts against it, so "'Smitty' Dr. Jones" reaches H1 with a title and one name word left standing — through 2.1 that meant given="Jones" with the family empty, and since #410 (2026-08-25) H1 names the family, so it reads family="Jones". The count is unchanged; what moved is what happens after it declines. The rule lives in assignment because that is where the piece count is settled. (An earlier wording here said "one non-title piece", predicting the opposite output; the coherence review measured the truth.) + diff --git a/docs/design/decisions/O1-east-slavic-rotation.md b/docs/design/decisions/O1-east-slavic-rotation.md new file mode 100644 index 00000000..4d071f68 --- /dev/null +++ b/docs/design/decisions/O1-east-slavic-rotation.md @@ -0,0 +1,17 @@ +# O1 — East Slavic rotation + +- (v1 era, PR #154) — why patronymic handling is OPT-IN at all: unconditional detection breaks ordinary Latin names whose endings collide (Martin, Franklin, Benjamin), the finding that forced v1's Russian work behind a flag and set the opt-in shape v2 inherited. +- 2026-07-12 (landed in the v2 core, PR #288) — v1 parity pinned live: the rotation reconstructs token position from assigned roles, which is faithful to v1 only under the default given-first order. + +- 2026-08-15 — the rotation × non-default name_order interaction question rode #270, which closed 2026-07-28 with the order constants and no recorded answer for the rotations. +- 2026-09-07 #384 — OPTION 1: the rotations are scoped to the DEFAULT order, and a gate on the read order is what makes that true. The principle is decisions/P1's 2026-09-07 entry and rules.md's Name order Background — a name order is a property of the data source, the caller declares it, the declaration governs what no vocabulary has claimed, and a declared family-first order outranks the shape. This rotation exists to RESTORE the default reading from a family-first listing, which is the whole of its job, so a caller who has DECLARED a family-first order has already supplied what it would have inferred and position decides instead. + The first draft of this entry said the scoping was by CONSTRUCTION and there was nothing to implement, on the reasoning that the rotation asks whether the word standing in the FAMILY position carries a patronymic ending and only the default order puts a name's last word there. That is false, and measuring it is what found it: under a family-first order the family slot holds the FIRST word, and a patronymic-DERIVED surname standing first carries the same ending, so such a name reached the rotation and was rotated out of the reading the caller declared. Measured 2026-09-07 before the gate, `Мицкевич Адам Юзеф` under `parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST)))` read family `Адам`, given `Юзеф`, middle `Мицкевич`, where plain FAMILY_FIRST reads family `Мицкевич`, given `Адам`, middle `Юзеф`; `Рабинович Иван Лев` read family `Иван` the same way. After the gate both read as plain FAMILY_FIRST does. The condition is on `state.order` — the order assign actually READ — rather than `policy.name_order`, for the reason decisions/P1's fold gives: a script_orders entry can override the policy, and the roles the rotation reads are the ones assign made. No corpus name moves at any of the four baselines, and sweeping the corpora under each pack with FAMILY_FIRST declared shows no difference either: they hold no three-word name whose FIRST word carries an East Slavic patronymic ending, and no four-word name whose FIRST word is a standalone Turkic marker (both counts 0 over the `tools/differential/corpus*.jsonl` union, measured 2026-09-07 by applying `_post_rules`'s own four patronymic patterns to the first word). The general lesson is docs/design/AGENTS.md's axis 10, re-run every capability claim — an option chosen because it needs no code is exactly where the no-code claim goes unmeasured. Two of the gate's three conditions have no possible witness and are recorded here instead: `read_order is None` cannot co-occur with a rotation shape, since assign leaves the order unrecorded only where it positioned at most one name word, and `state.order` differs from `policy.name_order` only through `script_orders`, whose scripts are East Asian and so cannot match the patronymic regexes. Unwitnessed by any test, on the precedent decisions/P6's 2026-08-30 #467 entry sets. + Measured 2026-09-07 with `parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST)))`. The family-first listing reads the same either way: `Сидоров Иван Петрович` gives family `Сидоров`, given `Иван`, middle `Петрович` under the default order by rotation, and the identical parse under FAMILY_FIRST by position. The divergence is on natural-order input — `Иван Петрович Сидоров` gives family `Сидоров` by rotation under the default order, and family `Иван`, given `Петрович`, middle `Сидоров` under FAMILY_FIRST. That row is ACCEPTED, not a defect: the caller said this source is written family-first and is being taken at their word. rules/O1 carries it as an Accepted consequence. + O2 is the same decision on the Turkic marker and its 2026-09-07 entry says so. The shapes differ, which is why the two rules state their own consequences rather than sharing one: under FAMILY_FIRST `Ali Ahmad Vali oglu` reads family `Ali` exactly as the rotation would, and it is under FAMILY_FIRST_GIVEN_LAST that it reads given `oglu`, middle `Ahmad Vali` — the marker in a name field, which is O2's existing Accepted consequence reached by a second route. Both readings are pinned in tests/v2/test_locales.py beside O1's row (measured 2026-09-07). + Where the row is pinned, and why not in rules.md. The example grammar admits ONE annotation, and `tests/v2/rules_doc.py`'s `resolve_annotation` resolves it to a policy, a locale or a gated locale — never a pair. Composite keys exist for other pairings (`family-first+middle_as_family`, `[ja+segmenter]`), but no registered annotation combines a pack with an order, so the two cannot be written on the same example line. Registering a combined policy would be a test change made inside a docs decision. #470 asked whether an example line should declare its format and was closed not-planned on 2026-09-02 — "no notation is needed. The Accepted clause is already the right home for this" — so prose in the rule is the decided home here, not a workaround. The parse is a `tests/v2/test_locales.py` row beside the pack's other rows, and rules/O1 states it in prose. + +Declined: + +- Option 2, suppressing the rotation and re-deriving the order from the patronymic trace under a declared family-first order — the declaration already carries the information this would recompute, and recomputing it means overriding the caller with a string-level guess, which is the principle above inverted. +- Option 3, reporting an ambiguity wherever a patronymic trace disagrees with the declared order — the report would fire on natural-order input inside a family-first source, and the caller cannot tell that from a mis-declared record, which is the only thing the report would be for. It is NOT true that it would fire on every correct family-first parse: a family-first listing CARRYING THE TRACE reads the same under both orders — the rotation fires only where the last word carries the ending, and it then makes the first word the family exactly as position does — so the trace and the position agree there and nothing is reported. Scoped deliberately, the unscoped claim being false: `Сидоров Иван Лев` is a family-first listing with no trace, and it reads family `Лев` under the ru pack's default order against family `Сидоров` under FAMILY_FIRST (measured 2026-09-07). What is left is a signal on the one shape where the declaration is doing its job, indistinguishable from the shape where the declaration is wrong. A report that cannot be filtered on is the failure decisions/P6's 2026-08-29 entry records for a different rule; here it is avoidable by not adding it. + diff --git a/docs/design/decisions/O2-turkic-rotation.md b/docs/design/decisions/O2-turkic-rotation.md new file mode 100644 index 00000000..8af2b170 --- /dev/null +++ b/docs/design/decisions/O2-turkic-rotation.md @@ -0,0 +1,8 @@ +# O2 — Turkic rotation + +- 2026-07-02 (Turkic design; landed in the v2 core, PR #288) — shape fixed at exactly four name words (1 given + 2 middle + 1 marker), v1 parity; other shapes keep their positional reading even when that leaves the marker in a name field (see the rule's Accepted consequence). + +- 2026-07-02 (design, orthography coverage) — dotless ı does not case-fold to i, so qızı is a literal alternative in the pattern (re.I cannot bridge it; all-caps QIZI falls to the ASCII alternative); the patterns are NFC literals and NFD input does not match. Scoping decline: suffix-attached Kazakh/Uzbek patronymics (Әбішұлы) are bound-suffix morphology, not a standalone marker, and deliberately out. +- 2026-08-15 — same rotation/name_order interaction status as O1. +- 2026-09-07 #384 — OPTION 1, the same answer as decisions/O1's entry of the same date and on the same principle: the rotation restores the default reading from a family-first listing, so a declared family-first order supplies that reading instead and the rotation stands down. It stands down by the same gate on the read order, for the reason O1's entry records — the marker test reads the word in the FAMILY position, which under a family-first order is the FIRST word, so a name opening with the marker reached it. Measured 2026-09-07 before the gate, `oglu Ahmad Vali Ali` with Turkic handling and FAMILY_FIRST read family `Ahmad`; after it, family `oglu`, which is what plain FAMILY_FIRST reads and this rule's own 2026-07-02 Accepted consequence — the marker in a name field. The natural-order twin `Ali Ahmad Vali oglu` is decisions/O1's entry of the same date, which measures it under both family-first orders and pins it in tests/v2/test_locales.py. Options 2 and 3 are Declined for O1's reasons. + diff --git a/docs/design/decisions/O4-positional-assignment-and-declared-order.md b/docs/design/decisions/O4-positional-assignment-and-declared-order.md new file mode 100644 index 00000000..d30e08aa --- /dev/null +++ b/docs/design/decisions/O4-positional-assignment-and-declared-order.md @@ -0,0 +1,15 @@ +# O4 — positional assignment and declared order + +- 2026-08-07 #83 — romanized Chinese order is answered by DECLARATION, not detection: pinyin carries no signal and diaspora makes locale no guide (the issue's own 2019/2021 conclusions); Policy(name_order=FAMILY_FIRST) is the answer the thread wanted. +- 2026-08-07 #146 — Vietnamese needs the THIRD order constant: measured on "Nguyễn Thị Minh Khai", FAMILY_FIRST strands given="Thị" while FAMILY_FIRST_GIVEN_LAST reads given="Khai", middle="Thị Minh", family="Nguyễn". This is why three order constants exist rather than two. + +- Provenance: the three-constants argument originates in #270's own body (2026-07-07): "A boolean family_name_first flag was considered and rejected: Vietnamese order is + [family][middle][given]." The 2026-08-07 #146 measurement below + is the verification; #270 is the origin. + +Declined: + +- Free-form order tuples (('last','middle','first')-style, #270's original draft shape) — the shipped design is exported order constants with tuple rejection at construction (rule D2's + [bad-name-order] example is the pinned message). +- middle_as_family as the way to suppress the middle slot for Vietnamese (2026-08-07 #146) — measured: it merges the middles into the family, giving family="Thị Minh Nguyễn" for "Nguyễn Thị Minh Khai" under family-first-given-last — a plausible-looking wrong answer. There is no policy field that suppresses middle, and name_order rejects a two-role tuple ("name_order must be one of the exported orders"); given_names (given + middle) is the view that stays correct wherever the internal boundary falls. + diff --git a/docs/design/decisions/O5-lone-name-words-convention-and-reporting-it.md b/docs/design/decisions/O5-lone-name-words-convention-and-reporting-it.md new file mode 100644 index 00000000..95c5036c --- /dev/null +++ b/docs/design/decisions/O5-lone-name-words-convention-and-reporting-it.md @@ -0,0 +1,15 @@ +# O5 — the lone name word's convention, and reporting it + +- 2026-09-07 #449 — the convention is REPORTED, as `given-or-family`, with `detail` naming the field it chose. rules/O5 has said since #445 that the one-word reading is a guess fixed in advance rather than a determination; what #449 asked is what the library should SAY about it, and the answer is the one A1 gives everywhere else — a caller can only act on doubt that is reported. `detail` names the field rather than the kind naming it, because the field follows the read order and the kind does not: that is the PARTICLE_OR_GIVEN precedent, recorded at that kind's docstring. + What the measurement settled was the POPULATION, not the principle. The naive condition — one name piece placed, nothing else looked at — fires on 143 of the 1123 distinct corpus names, because a title peel, a nickname, a maiden clause, a trailing credential run or a script order can each leave exactly one piece standing where the convention decided nothing. The 2026-08-27 #445 entry below records "19 of 1,085" for a differently written naive condition on a smaller corpus; both numbers are naive and neither is O5's. Guarded on O5's own carve-out list the population is 27 corpus names, 12 contract and 15 radar, and the other 116 report no `given-or-family`, 8 of them reporting `title-or-name` instead (H4's two branches are ahead of O5's at the same line). A first draft wrote 110, which is the subtraction done wrong. Recompute by wrapping `_name_positions` with a recorder that fires at `count == 1`, parsing the deduped `corpus*.jsonl` glob through it, and comparing that set against the names whose parse reports the kind; measured 2026-09-08 on this branch. The guard itself is the code, which is where it should be read. + Two conditions are in the guard that O5's statement had not named, and both are in it now. A script whose own convention settles the order decided the reading, so W4 outranks this the way H1 does — that is what keeps 毛泽东, 高橋みなみ, 씨 and the rest of the script-ordered names out. Asking that question needed a shape change: `_effective_order` used to return the roles triple alone, and a caller comparing that triple against the declared `name_order` cannot tell a script rule DECIDING from a declaration standing unopposed — under a declared family-first order a Han name's W4 entry and the declaration agree. It now returns an `Order(roles, by_script)` and the emitter reads the flag. The second condition is content: a piece with no letter or digit in it is no name word, A2 empties such an input, and a convention report on an empty name would describe a reading nobody got — so `parse("(")` keeps its unbalanced-delimiter report and gains nothing. + Five CJK-bearing names DO report, and none of them is a script rule deciding. Three are a script rule DECLINING: マイケル (W4 gives katakana no order of its own, a wholly-katakana name being read as a transcription), 王·Smith (T3's interpunct suppresses the script_orders lookup whole) and 田中、太郎 (the comma sits inside the token, so the text resolves to no single script). Two are a glued honorific peeled off a Latin stem — Andersonさん and Anderson선생님 — where what is left is Latin and has no script order at all. Recorded because "no CJK name reports" was the expectation going in, and it is wrong in a way that is right: where the script rules decline, what is left IS the one-word convention. + Four conditions were drafted into the guard and none of them shipped, but they were dropped for TWO different reasons, and a first draft of this entry flattened both into "inert". Each was measured the same way: restore that one clause on the O5 branch of a `git archive` copy of this commit, reparse the deduped `corpus*.jsonl` glob, and count which of the 27 lose the report. Two are genuinely INERT, silencing ZERO names (measured 2026-09-08, and again after the 2026-09-08 review round below). `len(state.segments) == 1` is inert OVER THE CORPUS rather than by construction, and a first draft of this line gave the construction reason, which is false: a family comma that names a family does read segment 0 wholly as the family and never positionally, but a comma followed by NO name word hands segment 0 back to the positional read, and `Smith V, Dr.` reaches this very site that way. What silences it is not the clause but the title in segment 1 — after the review round below, which is what tells the read that a title stands there. `not flagged`: a split credential decides the field no more than a whole one does. The general shape is the one this file keeps recording — a guard written from the cases that prompted it carries terms the code already excludes — and the instrument that finds them is mutation, not reading. + The other two were dropped DELIBERATELY, and the same mutation prices each. `not suffix_pieces` silences NINE of the 27 — `Smith Jr.`, `'Smitty' Jones Jr.`, `Carod i`, `Donald mc`, `Jack M.A.`, `John V`, `Mohamad X`, `Andersonさん`, `Anderson선생님` — and `not has_nickname` silences ONE, `'Smitty' Jones Jr.`. Both were dropped on one principle: a suffix or a nickname standing beside the lone name word decides nothing about that word's FIELD, which is the only question this report is about. `Smith Jr.` and `'Smitty' Jones Jr.` are S2's peel taking the suffix and the convention placing what is left, so they report. A first draft justified the nickname clause with "N3 returns before this site", and that is false as written: `_assign.py`'s N3 branch tests `len(pieces) == 1 and len(rest) == 1 and has_nickname`, so it returns only where the segment holds ONE piece, and `'Smitty' Jones Jr.` holds two and reaches this line. What N3 takes is the nickname-plus-one-word shape, not every name a nickname stands in. + Not reported, deliberately: `abdul`, where bound-given vocabulary claimed the word; `de`, where a lone particle's reading is P4's; and `J.`, where the initial's shape is the claim. Those are M4's `_NEVER_FLIPPED` pair plus the particle, read off the tags classify already recorded rather than off a predicate of this emitter's own — literally so since 2026-09-08: `_NEVER_FLIPPED` moved to `_state` beside `WorkToken.tags` (post_rules imports _assign, so _assign cannot reach the other way) and `_WORD_ALREADY_CLAIMED` is that frozenset unioned with `particle`, so the two sets cannot drift. Also not reported: `Sir John`, `'Smitty' Jones` and `abd née Jones`, which O5's example block lists because the convention is what H1, N3 and M4 each left behind on them. The rule's SCOPE and the report's scope differ there, and the difference is stated in the rule rather than left to be discovered: the report is narrowed to an input nothing else stands in, because a report on a name where a title, a nickname or a maiden name is visible in the output tells the caller less than the output already does. + The report is order-independent, which is not the same as the FIELD being order-independent: `Andrew` reports under the default order with `detail` naming given and under FAMILY_FIRST with `detail` naming family. Measured over the whole corpus under the default order, FAMILY_FIRST and FAMILY_FIRST_GIVEN_LAST, the set of names reporting this kind is the same 27 in all three; the only names whose reported kinds move with the order at all are four pre-existing `particle-or-given` reports. + Nothing moves but the report. Every role is identical before and after — measured by parsing all 1123 corpus names under all three orders against a `git archive` of the parent commit, 3369 parses, zero role differences — and the three 2.x ledgers classify the 27 names on `_ambiguities` alone; `_ambiguities` cannot enter a diff below baseline 2.0, so `expected_since_1.4.0.toml` is untouched and its 368 intentional diffs are unchanged. SIX ledger rules rather than one, because an alternation carrying a script-classified member is claimed by the honorific pin in tests/v2/test_ledger_guards.py, which would demand it be a hand copy of GLUED_HONORIFICS: the twenty-two Latin-and-Arabic names ride one alternation, declared in `_NOT_A_VOCABULARY_COPY`, and the five CJK-bearing names take a literal-anchored rule each, which is also where their arguments differ. The three ledger copies are NOT byte-identical, which the drafting expected them to be: in `expected_since_2.0.0.toml` alone the two glued-honorific rules declare `["_ambiguities", "given", "suffix"]`, because at that baseline #308's peel has not shipped and its diff arrives in the same rule as this report, and the gate's over-declared check refuses a rule declaring a role no diff it explains moves. + +- 2026-09-08 #518 review round (the entry above ran into a second day; its own measurements are dated 2026-09-08 under a 2026-09-07 header) — a title standing after a family comma now silences the report. `John V, Dr.` is the shape: a comma with no name word after it hands segment 0 back to the positional read, the leading-title peel counts only what stands in segment 0, so `n` is 0 and the report named `given` for a word H1 then wrote to `family` — the caller was told the convention chose a field the output does not show. `_assign_main` takes a keyword-only `titled`, which the family-comma call fills from the gate's own reading (a False in it is a title, not a suffix), and the field-deciding emitters require `not titled`. Measured on this branch: `John V, Dr.`, `Smith V, Prince` and `Smith V, Dr.` keep every role and lose the `given-or-family` report, keeping the roman numeral's; `John V, Sir` does too, and it is the one worth naming — a given-name title leaves the word in `given`, which is the field the report would have named, so the report and the reading agreeing is what a silenced clause looks like when the rule that decided it happens to agree. Zero corpus names move: no corpus name reaches the site under a comma at all, which is also why the `len(state.segments) == 1` clause above measures inert. + Consolidated the same day, 2026-09-08, and BEHAVIOR IDENTICAL. Segment 1 is now assigned BEFORE segment 0's positional read, so the title standing after the comma already carries a `Role.TITLE` when that read reaches its emitters; the keyword is gone and `n == 0`, `not titled`, the maiden scan and `not resolved.by_script` are one `field_undecided` predicate that both field-deciding emitters read. The token scan subsumes `n == 0` because nothing before assign sets `Role.TITLE` and assign sets it in exactly two places, the leading-title peel and the post-comma reading. Measured against a `git archive` of the previous commit: every distinct name in the deduped `corpus*.jsonl` glob plus the shapes named in this entry and in the #491 entries — 1136 names, 5680 parses — reads the same seven roles and reports the same kinds, details and token texts under the default order, FAMILY_FIRST and FAMILY_FIRST_GIVEN_LAST. One shape constraint came out of the measurement rather than the reading: the two token scans sit UNDER the peel count rather than beside it, because run on every parse they cost 10 calls/name and put the facade 457 over the 456 ceiling of its band (decisions/parse-cost). + diff --git a/docs/design/decisions/P1-lone-particle-fold.md b/docs/design/decisions/P1-lone-particle-fold.md new file mode 100644 index 00000000..2ad1ee09 --- /dev/null +++ b/docs/design/decisions/P1-lone-particle-fold.md @@ -0,0 +1,56 @@ +# P1 — lone particle fold + +A lone PIECE is the whole test — deliberately narrower than "a never-given particle is never reported as the given name," which would be false (the over-broad invariant shipped to five sites before #361's review falsified it with three counter-examples). The counter-example set has since shrunk, and its shrinkage is the section's history: "Sir de Mesnil" fell to #367 (titles became transparent); "Juan de la Vega" under family-first — the whole chain in the given position — was called working-as-intended by +#359, contested by #368, and is working-as-intended again as of +the 2026-08-16 entries below. The survivor is the degenerate bare +"de". + +- 2026-08 #359 — the opening site is read from joining structure (pieces), not from assigned roles, so the fold holds under every name_order. Before this, the role-only read let "de Mesnil" split under a family-first order. +- 2026-08 #367 — titles are transparent to the fold: "Sir de Mesnil" now reads like "de Mesnil". Fixed by removing the title→particle chain in grouping, not by touching this rule. +- 2026-08-16 (order-precedence keystone; #364, #365, #368) — the stage split is the decision, and the three issues are one question seen from three angles. GROUPING is vocabulary's job and is order-independent: a particle joins forward through consecutive particles and stops at the first non-particle, and no name_order moves that stopping point. ASSIGNMENT is name_order's job: groups take roles by the declared order. The bugs existed because the implementation runs the two in the opposite dependency — `assign` hands out positions from `_effective_order` and `post_rules` then inspects a fixed list of ROLES, so P1's fold sites and P2's chain had coverage that varied with the declared order by accident. Consequences, each recorded in its own right below: the fold takes only the particle's own group (#364); the middle position needs no third site once grouping is order-independent (#365); and #368 reverses. +- 2026-08-16 #364 — the fold takes the particle RUN and the ONE name word it attaches to, not every remaining word. + "de Mesnil Jean" is family="de Mesnil" plus given="Jean". Run, + not particle: "de la Vega" is two particles onto one word, and an earlier wording here said "the particle", which its own example contradicted (rule-vs-decision-record review). Nothing ever argued for + "takes everything"; it was the shape of v1's + handle_non_first_name_prefix, not a decision. Measured before deciding, over every name in the three differential corpora with NO string prefilter: exactly ONE family holds words beyond its particle's group — "de Mesnil Garcia". + #364's own body warns the change "breaks the v1 parity + tools/differential protects" and that "each ledger would need re-examining"; measured, it is one name and one ledger entry. A shape filter (no comma, leading never-given particle, three or more words) returns THREE candidates, of which two do not move: + "de la Vega" is one group start to finish, and "de Mesnil Jr." + has only two name words because Jr. is a suffix. The pre-merge fact-check caught this stated as "the filter gives one name", which it does not. Measurement trap, recorded because the fact-check fell into it twice: the particle group runs through ANY particle, not only the never-given ones — "de la Vega" chains never-given "de" through AMBIGUOUS "la" onto "Vega". A detector that walks only the never-given run splits the group after "de la" and reports 50 false movers. +- 2026-08-16 #368 REVERSED — shipped behavior is correct. + "Juan de la Vega" under FAMILY_FIRST groups [Juan][de la Vega] + and assigns family="Juan", given="de la Vega". The earlier decision ("the particle wins ... whatever order was declared") was made before the grouping/assignment split was stated and cannot survive it: a mid-name chain HAS a head word and is positioned like any other group. What NON_GIVEN_NAME_PARTICLES guarantees is that the bare word never reads as a given name, not that no name part may begin with one. The asymmetry with the leading case is P4's, not an exception invented here: a leading particle chains nothing, so without the fold pure position makes the BARE particle the given name. That is measurable today on the ambiguous half, where no fold fires — + "van Mesnil Juan" gives given="van", middle="Mesnil", + family="Juan". given="de" is the reading the vocabulary exists to forbid, and the fold is what prevents it. The all-orders agreement in P1 is deliberate and is W4's shape: a wholly-hangul name reads family="김" under every declared order because the script carries a signal the order does not override, and a leading never-given particle is the Latin-script analogue. decisions/O4 already draws the line — "Words no vocabulary has claimed read by position" — so name_order governs the unclaimed remainder, which is most inputs. + +- 2026-08-17 — SUPERSEDES the order-independence half of the 2026-08-16 order-precedence keystone entry above (#364, #365, #368). That entry says "no name_order moves that stopping point"; it may. Declaring FAMILY_FIRST or FAMILY_FIRST_GIVEN_LAST is precisely an assertion that what follows the family is NOT more surname, and a particle run's stopping point is exactly the question of where surname material ends — so the declaration is evidence about it, not merely about which slot the result lands in. The grouping/assignment split survives everywhere else; what is narrowed is the claim that the stopping point is one of the things order cannot touch. + Concretely: under GIVEN_FIRST a leading particle run keeps joining to the end of the name. (This entry originally said that makes "de Mesnil Jean" read "exactly like pennie von bergen wessels — the same shape". Measured while implementing #395, that is false and is corrected here rather than left standing: pennie is the GIVEN name there, von is ambiguous so this fold can never fire on it, and the surname reading comes from P2's mid-name chain. What the real name does support is the weaker claim the argument needs — that a particle followed by several words can be all surname, "von Bergen Wessels" being one.) Under a family-first order the run stops after its first non-particle name word, because the caller has said the rest is given or middle. + Accepted, and it is the cost that decided #364 three separate ways before this: "de la Vega Juan" under the DEFAULT order stays wholly a surname. A caller who means family "de la Vega" plus given "Juan" writes the comma. The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language. + Why the question kept thrashing, worth recording so a fourth attempt does not start from scratch: "how much does a leading particle run take" is a LANGUAGE judgement being forced through a POSITION heuristic, in a parser that has correctly refused to detect language (decisions/O4, rules.md's Not-in-scope). Every argument in the thread — `de`/`do` as Vietnamese surnames, `von` as German, `dos` as Portuguese — is really about which tradition the name belongs to. The order declaration is the one place a CALLER supplies that information instead of the parser guessing, which is why keying on it is better than any position rule we tried. + Implementation consequence: the stopping point may now read name_order, which the superseded sentence forbade. Whether that lives in grouping (order-aware) or as a split in assignment is open — PR #391 showed assignment cannot split an existing piece today, so choosing assignment means giving it token-level slicing. + Supersedes nothing else: #368's reversal, the trailing-orphan rule P6, and the claim that a leading never-given particle takes the FAMILY rather than a given name all stand. +- 2026-08-18 #395 — WHERE THE STOP LIVES, settling what the entry above left open. Not grouping and not assignment: the fold in post_rules, which already retags tokens and is the only site that can express the stop without moving a piece boundary. Grouping was PR #394 (four regressions) and assignment was PR #391, where the finding was that assign cannot split an existing piece — and the piece is the problem: "de la Cruz Juan Carlos" groups as [de][la Cruz Juan Carlos], the ambiguous particle having chained forward, so any rule that stops "after the first name word" must cut INSIDE a piece. post_rules can, because roles are per token and nothing downstream reads pieces (measured: only _assign, which runs before it, and post_rules itself). Grouping therefore stays order-independent, which keeps the 2026-08-16 keystone intact everywhere except the reach of this one fold. + The order is READ, not re-derived: assign now records the order it actually used on ParseState.order, and the fold keys on that. policy.name_order would have been wrong — a script_orders entry can put the family first under a given-first policy, and the roles assign already wrote would then disagree with the roles the fold computes. The same reasoning already appears one function away, where the PARTICLE_OR_GIVEN emitter reads the role off the token rather than assuming given. + A unit, not a token: a particle chain (P2), a conjunction join (P3) and a bound given-name pair (P5) each count once. All three are read off the TAGS rather than the pieces, but for two different reasons, and the difference matters to anyone who tries to simplify this. The conjunction join grouping DID build and the prefix chain then swallowed: the name reaches the fold as [de][la Vega y Santos Juan], the ambiguous particle having chained forward over the join, so the JOIN's boundary is gone. The leading particle keeps its own piece, and must — the fold's site test wants a lone piece, so a one-piece name would not fold at all. The bound-given join grouping never built at all: P5 joins only where the bound word is the first non-title piece, and at a fold site the first piece is the particle, so "ibn Awf abdul Rahman" arrives as four separate pieces. Restoring piece boundaries in group would fix the first and silently split the second. It also makes P5's new implemented: pointer stronger than a citation — this is the only place that join happens in this shape. This is what keeps "de la Vega y Santos Juan" from stopping between Vega and Santos — the case rules/P3 was amended for on 2026-08-17, now executable. + Measured, with the denominator this log requires: of 751 corpus names, SEVEN reach P1's leading fold (DE MESNIL, De Groot, de Mesnil, de Mesnil Garcia, de Mesnil Jr., de la Vega, dos Santos) and exactly ONE of those has anything past the run to lose, so one moves (de Mesnil Garcia) under each family-first order and none under the default. The single mover is a fact about the corpus holding no family-first listing with leftovers, not about the blast radius. The differential harness cannot see any of it — it runs the corpus under the default policy against 1.4.0, which has no name_order — so its exit 0 is evidence for the accepted cost and for nothing else. The verification that counts is the two-leftover shape: mutation-checked, discarding name_order from the leftover placement fails three tests across three layers — the FAMILY_FIRST_GIVEN_LAST case row (one of the two, the FAMILY_FIRST row being blind to it by construction), test_post_rules.py's test_the_two_family_first_orders_differ_at_two_leftovers, and rules.md's P1 example for that order. An earlier draft of this sentence said "exactly one, and no other test in the suite": true when measured, false one commit later, because two of the three guards were added after the measurement and it was not re-run. Recording the failure mode as well as the number, since it is this log's recurring one. +- 2026-09-07 #471 DECLINED — the greedy reach under the default order is the READING, not a gap in one, and what decides it is a principle rather than a limit: a name order is a property of the DATA SOURCE, not of a string. The caller declares it to match how their records are written: the declaration governs what no vocabulary has claimed and a declared family-first order outranks the shape; rules.md's Name order Background states it. decisions/O1's 2026-09-07 entry applies the same principle to the patronymic rotations. Applied here: a never-given particle marks where a surname BEGINS, multi-word surnames are ordinary — Spanish double surnames above all — and under the default order nothing has said this surname ends before the string does. So `de la Torre Vega` is a surname-only string read right, and `de la Cruz Juan Carlos` is a family-first listing read under the wrong order. The remedy is the declaration or the comma: FAMILY_FIRST already gives family `de la Cruz`, given `Juan`, middle `Carlos` (#395, the 2026-08-18 entry above), and the family comma fixes the surname whatever order is declared. `de la Family Family Given` and `de la Family Given Given` are undecidable under EITHER order, which is what makes the comma the answer rather than a better heuristic. + This SUPERSEDES the REASON the 2026-08-17 entry above gives for the same outcome — "The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language." That was right and weaker. The order is not a language judgement at all: it is a declaration about the data, and a caller who has one supplies it whether or not a parser could ever have guessed it. The outcome that entry reached, and everything else in it, stands — and so does its "Why the question kept thrashing" paragraph: the REACH question is a language judgement, which is why the order declaration is where the caller supplies it. What is superseded is only the claim that the parser's blindness to language is the reason. + It also answers the 2026-08-16 #364 entry's "Nothing ever argued for 'takes everything'; it was the shape of v1's handle_non_first_name_prefix, not a decision." This entry argues for it. The v1 provenance is unchanged and the argument is new, which is the only thing that had been missing. + Measured 2026-09-07 on the tree, and on the 1.4.0, 2.0.0, 2.1.0 and 2.2.0 wheels, which agree on every default-order row: `de la Torre Vega` gives family `de la Torre Vega`; `de la Cruz Juan Carlos` gives family `de la Cruz Juan Carlos`; `de Mesnil Jean` gives family `de Mesnil Jean`; and the comma form `de la Torre Vega, Juan` gives family `de la Torre Vega`, given `Juan` under both orders. Under FAMILY_FIRST `de la Torre Vega` reads family `de la Torre`, given `Vega` — the cost the declaration accepts, and the reason a surname-only record in a family-first source still wants its comma. Recompute with `Parser().parse(name)` and `Parser(policy=Policy(name_order=FAMILY_FIRST)).parse(name)`; for the wheels, `HumanName(name).last` from a directory outside the checkout so the tree cannot shadow them. + Nothing moves. No parse changes; the `parity` row `leading_never_given_particle_two_leftovers` in tests/v2/cases.py and the guard `test_the_family_first_fold_is_not_explained_under_the_default_order` in tests/v2/test_ledger_guards.py stay as the pins of the decided behavior; no release-note bullet is written, there being no behavior to note and no by-design precedent in docs/release_log.rst. rules/P1 gains `de la Torre Vega` as an example, which puts the name in the contract corpus corpus_rules.jsonl. Its parity is over the SEVEN roles, not the `initials` view: at the 1.4.0 baseline only, the name falls into the already-recorded class `fix(initials-per-word) a particle chain inside a name part` beside its sibling `de la Cruz Juan Carlos` — the v1→v2 per-word initialling shipped since 2.0.0, not a regression — so the 1.4.0 intentional count goes 367 → 368 and that rule's recorded claim goes 107 → 108 names with a new digest. The three 2.x baselines are unmoved at 263 / 175 / 37 (296 / 210 / 72 after #449 and #491 landed the same day; #471 itself moved none), and `unexplained: 0` at all four. + The `Open:` pointer to #471 that stood at the end of decisions/P6's 2026-08-30 #467 entry is removed with this entry, which is where the question is now answered. P1's own `Open:` block below is #360's and is untouched. + Deliberately NOT decided here: reporting an ambiguity on the shapes neither order settles — the honest-output flag — would be an AmbiguityKind and belongs with the ambiguity bundle (#449/#491/#348) if anywhere. Declining the reach change does not decline that. Two thirds of that bundle shipped 2026-09-07 as decisions/O5 and decisions/H4; #348 was left OUT by choice, being a question about the TITLES vocabulary rather than about a decision site, and the shapes this sentence is about are neither one — no rule fixes them by convention, so there is no branch to report at. + +Declined: + +- A strict xfail asserting "de Mesnil" → family under FAMILY_FIRST (#359 review) — #359 deliberately left those semantics open, and a strict xfail decides the question by the back door. +- Keying the leading-particle exception on "the first piece that is not a title" — the obvious implementation, and wrong: st, do and freiherr are titles AND ambiguous particles, so it collapsed + "St John Smith" into one given name and broke test_add_title + (which adds "te", also a particle). The shipped predicate is + "not a title or a prefix". +- 2026-08-16 — deleting P4 so a leading particle chains and is then positioned, which is the only way to make "de Mesnil Jean" vary by declared order. It avoids given="de" (the group would be + [de Mesnil]) but breaks "de la Vega": measured, a single group is + positioned by the declared order — "Cher" reads given under GIVEN_FIRST — so "de la Vega" would read given="de la Vega" unless a further rule forced a particle-headed group into the family. Add that rule and [de Mesnil][Jean] yields the #364 reading anyway, so the deletion buys nothing and costs P4. + +Decided 2026-08-18 (was Open: [#360](https://github.com/derek73/python-nameparser/issues/360), which particles count as never-given): #360 applied the criterion at decisions/vocabulary-collisions and moved `mc` and `ste` into the set (from the may-be-given half), adding the Spanish/Portuguese articles; later vocabulary work extends the set under the same criterion (#604's Irish particles). + diff --git a/docs/design/decisions/P2-particles-join-forward.md b/docs/design/decisions/P2-particles-join-forward.md new file mode 100644 index 00000000..9ff164d6 --- /dev/null +++ b/docs/design/decisions/P2-particles-join-forward.md @@ -0,0 +1,17 @@ +# P2 — particles join forward + +- 2026-08 #367 — REMOVED a chain, it did not create one: before, the title displaced the particle out of the leading position, so + "Sir de Mesnil" grouped [Sir][de Mesnil] — a chain — and + reported given="de Mesnil" with no family at all. After, "de" is the leading name piece, and a leading particle chains nothing (rule P4): the family reading comes from P1's fold, which is why P1's fold needed no change (its interacts: points here). Grouped today: [Sir] [de] [Mesnil]. + +- Group distribution is load-bearing, not incidental: the v1 Portuguese tests (test_portuguese_dos, test_portuguese_prefixes) require multiple particle groups to land across middle and family, while #132 wanted the combined double-surname reading — the same shape with opposing wants, which is why the combined reading lives in the surnames VIEW and the split in the fields. + +- 2026-08-22 #424 — the chain stops where assign's trailing peel begins. The chain ran "until a trailing suffix begins" and asked with the suffix-piece test, which vetoes a bare `V` as an initial — the #401 question at a third site — so `John van der Berg V` read family 'van der Berg V' (1.4.0 read it so too: shipped since 1.x), and `John van der Berg Ma` read family 'van der Berg Ma' where 1.4.0 read suffix 'Ma' — a 2.0 regression on S2's other fork. Both now stop where `_trailing_start` says the run begins: the S2 peel (`_peel_trailing` over `_peel_walk`, shared with assign and P5's reserve since #425) read over the pieces as they stand from the first name piece, once per segment, kept as a length from the end that the chain's merges ahead of it do not move (the benchmark's `particles` shape is the guard). The chain takes both forks — its merges leave the acronym at least the three pieces the fork counted, so assign peels it after as the walk did before — where the maiden walk takes only the numeral (the M2 entry says why). The fork reads the piece before the numeral as it stands, so `John van der J. V` keeps family 'van der J. V', as `John J. V` reads the V as a name; and a numeral with a suffix behind it is not last in the walk, so `John van der Berg V Jr` keeps family 'van der Berg V'. No differential corpus name had either shape; the rules examples carry them — the numeral against every baseline, the acronym against 2.0.0 and 2.1.0. Two more sites were asking assign's question with a test of their own, both found by the reviews of this change. The walk's starting point and P4's leading-particle scan asked group's `title()`, which does not see H2's unlisted abbreviations — assign peels those as titles all the same — so `Xyz. van Johnson` chained where `Dr. van Johnson` did not (#367 had keyed the exception on the first piece of the name, by group's test), and `Xyz. van Berg MA` read family 'van Berg MA' on every 2.x tree — the chain had taken the acronym, as it took every acronym; the draft that stopped before it left assign two pieces where the fork had counted three, and MA became the family. Both ask assign's own test now — `_is_leading_title` and the period-abbreviation pattern moved down into group, with `_leading_titles` as the one definition of where the name begins: the chain's walk, the bound join's first name piece (the second docs review found that one still on group's test — `Xyz. abdul John Smith` joined nothing where `Dr. abdul John Smith` read given 'abdul John') and, through `_is_leading_title`, the scan and the PARTICLE_OR_GIVEN report — so `Xyz. van …` reads as `Dr. van …` does. `Esq. van Gogh`, a corpus name, moves with it (1.4.0 read family 'van Gogh' for both, as #367's rule records; its own rule at every baseline, and one for the P4 example). A particle of the unambiguous suffix vocabulary too (vd, mc) is a suffix piece to the peel: where it opens the trailing run the chain stops before it as before any suffix word and the peel takes it — `John Smith Mc V` reads suffix 'Mc, V' where master and 1.4.0 read family 'Mc V', a class the second docs review found unrecorded (2,554 of its 117,306 constructed names, an eighteen-word pool that puts the words in every position) — and where it continues a prefix run, the run takes it as a particle, so `John van Mc` keeps family 'van Mc', every baseline's reading and the one P6 chose for the shape after a comma. Both are Accepted under P2. The fork's count was the last thing the chain asked on its own authority, and the reviews found it wrong behind a title-and-particle word: #367 stops the leading-particle scan at `Freiherr`, `St`, `Do` — the name's own first piece — where assign's title peel steps over them, so the chain takes the name's first word, and the acronym the fork counted with three pieces met assign with two: `Freiherr von Berg MA` read given 'von Berg', family 'MA' (1.4.0's reading, as it happens; master read family 'von Berg MA'). So the chain asks the peel again over the pieces it leaves — `chain(tail)`, then `_trailing_start` over what it built, and a second run without the stop where the verdict changed, the snapshot being one copy per segment with a trailing run — and takes what assign will not peel: family 'von Berg MA'. The numeral cannot flip, a chain group never being initial-shaped, so `Freiherr von Richthofen V` keeps suffix 'V', and its chain, the one name piece left, reads as `Dr. Smith V` reads — where 1.4.0 and master read family 'von Richthofen V'; the code review asked for that boundary to be on record, and P2 carries it. #410 (2026-08-25) then moved the name half of both: a title with one name word behind it names the family whatever stands beside it, so the pair reads family 'von Richthofen' and family 'Smith' today, where each read `given` with no family when this entry was written. The boundary is unchanged — the chain still stops before the numeral, and the two names still read alike; only the field the one name piece lands in moved. This name is the fifth corpus name #410 moves, and the only one already classified, its ledger rule's `fields` spanning the narrower diff. +- 2026-10-02 (Derek), #573 — the part before a family comma: a particle that is suffix vocabulary too heads the name word behind it in any case. Decided with P6's third exception and S2's never-given lead; the record is the P6 section's entry of the same date. +- 2026-10-07 #614 — CROSS-REFERENCE: the chain's stop is the trailing run S2 reads once, after P3's joins and before the chain is made, counting the chain's own run -- a particle run, the words both particles and suffix vocabulary behind it included, and the plain words after it -- as the one name word it will make; the re-ask after the chain's own merges (the 'Freiherr von Berg Ma' rollback above) runs only where the run was not read there. decisions/S2, this date; rules/P2's statement amended. +- 2026-10-10 (Derek), #624 — WHERE THE NAME STARTS IS DECIDED ONCE, AND OF TWO TITLE-PARTICLES THE SECOND LEADS. `TITLES ∩ particles_ambiguous` is `{freiherr, st}`. Such a word in the leading titles is the chain's leading position (P4), so the particle behind it is inside a name (`Freiherr von Berg` → family 'von Berg'). Group's chain found that position with a scan of its own that stopped at the FIRST such word, and chained the second as a surname particle, while assign read both as titles and the trailing read's unit count (`_chain_units`, #620) opened a unit inside the titles to follow the chain: `Freiherr St John Smith MA` read title 'Freiherr', family 'St John Smith', with a title-or-name report saying it was "read as a family name by convention", where `Dr. St John Smith` and `Sir St John Smith` read title 'Dr. St' / 'Sir St', given 'John', family 'Smith'. + DECIDED (Derek: consistency, one shared answer, no special case for garbage): `_pieces.chain_lead` gives the leading position once -- the LAST title in assign's own title run (`leading_titles`) that is also a particle, else the first piece past the run -- and group's chain and the read's count both take it. The run is read as WRITTEN, before H3's give-back (Derek, the review's option (a)): a title the run hands back to the name because only suffix words follow it is still a title to the chain, so the particle behind it is the name's leading piece and chains nothing (P4). `leading_titles` stops at a piece that is itself a leading title only where it gave that piece back or at the segment's last piece (a title needing a following piece), so `chain_lead` recognises the give-back from `n` alone, the last piece ruled out by its position, behind the same inline tag test, and no second title walk is needed. No particle stands between the lead and the run's end, so the chain opens no unit inside the titles: the read's `at` and its `q > k + 1` gate are gone, and group's chain skips every piece up to the lead (`k <= leading`, equivalent to the old `k == leading` wherever the lead was the first title-particle, since no particle stood before it). `Freiherr St John Smith MA` → title 'Freiherr St', given 'John', family 'Smith', suffix 'MA'; `Freiherr St van Berg MA` → title 'Freiherr St', family 'van Berg'. + NOT DONE: the issue also proposed retiring assign's re-read fallback (`read = None` where group's joins left no name word past the titles). Measured, it still does work this change does not touch: disabled, sixteen or seventeen more tests fail depending on how it is disabled, most of them as an IndexError, so it guards totality and not only a reading -- among them the case row `a_title_read_before_the_chain_can_strand_the_name` (`Freiherr von Berg Dr. and Ed. Prof.`, where a CONNECTIVE join makes the name a title), the locale non-interference tests and two case-repair property tests. It stays. + MEASURED 2026-10-10 against 8e6e524a (#631's merge), py3.11, `nameparser.__file__` asserted on each side. A title-particle head grid -- every sequence of one to four words over `Freiherr St Dr. Sir Xyz. von van de do Mc John Smith MA PhD VI Prof. née Jr. and`, plus `Smith, ` and `, Jr.` over `Freiherr St Dr. von John`, 137,810 names under the three orders, comparing the seven fields and every report: 5,964 role moves, no report-only move. 5,868 have two title-particles in the leading title run (a period-shaped word such as `Jr.` and a connective title such as `and Freiherr` counting as titles there), all now titles -- where H3 then gives the last back, the word behind it is the post-nominal H3 saw rather than a word that title chained (`Dr. Freiherr St Mc` → title 'Dr. Freiherr', family 'St', suffix 'Mc'; master family 'St Mc'). 96 are H3's give-back before a run of dual particles, read as H3 states (`Dr. Mc Mc` → given 'Dr.', suffix 'Mc Mc'; master given 'Dr.', middle 'Mc', family 'Mc', the read having counted the second 'Mc' as bound into a run the chain never built). The first draft read the give-back as the chain's lead and gave `Dr. Mc Mc` title 'Dr.', family 'Mc Mc', chaining the name's leading particle behind a plain title against P4, and with a caller lexicon (`esq` added as an ambiguous particle) `Dr. esq vd` chained 'esq' and reported it; the review caught both, and option (a) is what this entry records. A comma-tail grid, `{head} {w1 … wk}` for k from 1 to 3 over the heads `John Smith, Jr.,`, `Smith, John,`, `Smith, Jr.,`, `,,`, `Smith,,`, `John Smith, PhD, Jr.,` and the words `Dr. Dr Attorney`, `Secretary of State` (one word of the list), `Lt.Gov. Freiherr St von`, `van der` (one word), `Bart Jones Jr. PhD MA Ma X.Y.Z. nee VI V 'Bob' (Bob) Prof.`, `Mr. and Mrs.` (one word), `I y` -- 97,650 texts, three orders: 1,368 parses on 456 texts move, every one a part past the second comma led by two title-particles, now all titles as C2 reads a title word there (`,, Freiherr Freiherr Bart` → title 'Freiherr Freiherr', suffix 'Bart'; master title 'Freiherr', suffix 'Freiherr Bart'). The S2 stress grid (#614's recipe): 0 moves. Every corpus name and case text at 8e6e524a under the three orders: only the case rows #624 rewrites move. The five differential gates exit 0. Cost: every frame baseline moves DOWN by 4 (decisions/parse-cost, this date). + ROWS: five rows pinned the old decision and now read the new one (`fix(#624)`, each note giving the earlier reading): `Freiherr St van Berg MA`, `Freiherr Freiherr Prof do`, `St St VI`, `Freiherr St MA`, `Freiherr St Prof.`; `Freiherr St John Smith MA`, the issue's example, is added, and `Dr. Mc Mc` pins the give-back half (the review found that half unpinned, a revert passing the suite). The marker row that held group's particle-or-given emitter on `St St née` moves to `St van née`, the same structure with a chained particle that is not a title (parity), since `St St née` no longer reaches that emitter. + diff --git a/docs/design/decisions/P3-connectives.md b/docs/design/decisions/P3-connectives.md new file mode 100644 index 00000000..5bdb80aa --- /dev/null +++ b/docs/design/decisions/P3-connectives.md @@ -0,0 +1,64 @@ +# P3 — connectives + +- 2026-07-30 #267 — the four-word single-letter asymmetry: a bare Latin capital connective is vetoed as an initial while a Cyrillic capital joins. #267's closure ("v2.0 behaves this way by default … verify it was the right call") showed only the Cyrillic half; the Latin-capital veto was never separately adjudicated, which rules/P3 records as an Accepted consequence. +- 2026-08-17 — a conjunction-joined run counts as ONE name word in every rule that counts them. Raised while bounding the leading particle run (#395): a run that stops "after the first name word" would otherwise be able to stop INSIDE a part P3 built, splitting "de la Vega y Santos Juan" between "Vega" and "Santos". This is not new behavior — H1 already counts this way, measured: "Mr. Jack and Jill" gives family="Jack and Jill" while "Mr. Jack Jill" gives given="Jack", family="Jill". What is new is that rules.md says so, which is what binds #395's implementation. Recorded because the first draft of the clause asserted that no rule could show it today and shipped an Accepted note to that effect; the shipped H1 pair was simply never run. Detector failure of the usual shape — reasoning about which rules count instead of parsing a name and looking. + +- 2026-08-21 #418 (PR #420) — the three-word carve-out counts the name that remains once the maiden name leaves. The count was taken while the marker and the maiden name were still pieces, so `juan y garcia nee jones` counted five, 'y' joined, and the family was empty once the clause left — given 'juan y garcia', family ''. Nine corpus names show it under an appended clause. Closed by order, not by an exclusion: the marker pass now runs before the count (decisions/M2, 2026-08-21), and the three-piece gate ahead of the count, which had the same exposure, is closed the same way. rules/P3 now says the count is of the name's own words. The question #397 and #411 left open — whether a word's membership in one vocabulary should suppress it from a count gating a rule keyed on another — is untouched: this was never a membership question, the words are simply not in the name. + +- 2026-09-13 #383 + #479 — BLESSED for the mixed-case half, and DECIDED BY VOCABULARY for the one-case half, which is the half the question never reached. A single-letter connective joins only on positive evidence. Where a name is written in more than one case the writing is that evidence and nothing moves: a bare LATIN capital reads as an initial ("Jose E Maria Santos" → middle "E Maria"), which is #383 answered with "bless". The shape is still LATIN and the phrase "Latin-capital veto" does not retire with this change — `_INITIAL` is ASCII-only, so a Cyrillic capital is never initial-shaped, joins, and #267's reading above is untouched. An earlier draft of this work claimed the phrase retires; it was overstated, and what became evidence-based is the one-case half alone. + WHERE THE WRITING SAYS NOTHING. A name written wholly in ONE case — all upper and all lower alike — carries no case evidence about any letter in it. The reading there comes from a new marker subset `Lexicon.conjunctions_ambiguous` ⊂ `conjunctions`, default `{e}`: a member reads as an INITIAL and reports the new `AmbiguityKind.CONJUNCTION_OR_INITIAL`, a non-member JOINS even written as a bare capital and reports nothing. That is the third instance of the ambiguous-subset pattern, after `particles_ambiguous` and `suffix_acronyms_ambiguous`, so no mechanisms.md entry is owed. + ROW 1 OF #479 IS THE DEFECT, NOT ROW 4 (Derek, 2026-09-12). The one-case lowercase reading is the one that was wrong, not the one-case uppercase reading that looked wrong beside it. + WHY `e` AND NOT `y`. A bare E initial is common (Edward, Elizabeth) and an "e" between two surnames is rare outside couple listings; a bare Y initial is rare and "y" between two surnames is the commonest Hispanic compound and this library's oldest fixture ("Velasquez y Garcia"). The 2026-08-18 LOCALE CONTROL bullet under #3-0-reevaluations already said that which letters a tradition wants joined is per-word POLICY rather than membership; this subset is that policy made configurable, which is the smaller half of what that bullet asked for and ships without locale packs. `i` is not here because it is not conjunction vocabulary at all yet; it ships in the subset when #397 adds it, a bare I initial being as common as a bare E. + MEASURED (2026-09-13, this branch, over the deduped `tools/differential/corpus*.jsonl` glob — 1174 distinct names). SEVENTEEN names are written in one case and carry a cased single-letter connective: that is the WHOLE population this fork can reach, and ten of them move something against 2.3.0 (a role, `initials()`, or the report). Recompute the population by keeping every corpus name whose space-joined text equals its own `upper()` or `lower()` and which holds a one-character token that is in `conjunctions` after `_lexicon._normalize` and whose `upper()` and `lower()` differ; recompute the movers by parsing each of them here and under a released wheel and diffing the seven role fields, `initials()` and the ambiguity kinds. Of the EIGHT such names the corpora already held before this PR added its own examples, not one moves a role — the three-word carve-out or an existing initial reading covers every one of them. The two role moves are both new example names, and they break 1.4.0 parity in OPPOSITE directions: `jose e maria santos` read given "jose e maria" at every release from 1.4.0 through 2.3.0 and now reads given "jose", middle "e maria"; `JUAN GARCIA Y LOPEZ` read middle "GARCIA Y" and now reads family "GARCIA Y LOPEZ" (measured on the released 1.4.0 and 2.3.0 wheels from a throwaway environment, 2026-09-13 — this tree shadows the wheel otherwise). The ledgers at all five baselines classify them. + THE RULE IS PER TOKEN AT CLASSIFY TIME AND REACHES EVERY NAME LENGTH (Derek, 2026-09-13), the three-word shape included: `john e smith` reads middle "e" tagged as an initial, so `parse(...).initials()` gives "j. e. s." where 2.3.0 gave "j. s." and `capitalized()` gives "John E Smith" where 2.3.0 gave "John e Smith". P3's three-word carve-out is untouched — it already says a single letter in a short name is more likely an initial, and this is the same judgement extended to the letter's case — so no ROLE moves at three words and the visible change is in the two derived views. + BOTH QUESTIONS ARE ASKED OF THE NAME'S OWN WORDS — the case class AND the fork, not just the three-word count. Tokens standing before the first maiden-marker run head and not already carrying a role from a delimited clause are the name; the rest are not. Found by measurement rather than by design: appending " née Jones" made an all-caps name mixed-case and flipped the fork, failing the repo's invariant that a maiden clause changes nothing else, and a delimited `"Y"` was being conjunction-tagged (so R4 lowercased it) while a delimited `"E"` reported. ACCEPTED edge, and it is WIDER than an early draft of this entry said (measured 2026-09-13, in review): a maiden marker's run and every word AFTER it leave the own-words span the moment classify tags the marker, and they stay outside whether or not the marker pass later DECLINES the marker and leaves it a word. So a declined marker does two things, not one. It still does not count toward the case class, which is visible only when the declined marker word is the only differently-cased token (`JUAN Y GARCIA née` joins where `JUAN Y GARCIA née Jones` keeps "Y" a name word) — that much the early draft had. But it also leaves any connective standing after it to the MIXED-CASE rule rather than to the one-case fork, so a bare Latin capital there is an initial and its lowercase spelling joins. That is the one shape in which a name's OWN words read differently in its two one-case spellings: `JUAN NÉE JR Y LOPEZ` reads given "JUAN", middle "NÉE JR Y" with its "Y" tagged `initial`, while `juan née jr y lopez` reads given "juan", middle "née", family "jr y lopez" with its "y" tagged `conjunction`. SCOPE THAT PHRASE TO THE OWN WORDS, because a clause's words are not covered by it and a re-review caught the over-reach: a letter inside a delimited or maiden clause is read as it always was, which still means a different TAG in the two spellings, and case repair shows it — measured 2026-09-13, `JANE VAN DER BERG NÉE Y JONES` capitalizes to `Jane van der Berg (Y Jones)` where `jane van der berg née y jones` gives `(y Jones)`, and `JUAN "Y" GARCIA LOPEZ` keeps nickname `"Y"` where the lowercase spelling keeps `"y"`. The FIELDS agree in both pairs; what differs is the repaired case of a clause's word, which is that clause's rules doing their job rather than this one's. ACCEPTED rather than repaired, on two grounds: classify cannot know what group will decline, and a clause's words are read by the clause's own rules — which is the same doctrine the own-words span states, applied to the one shape where it bites. rules/P3's Accepted clause carries the same wording and the same example pair. (2026-10-11, release sweep: #602's credential run now reads `JR Y LOPEZ` as suffixes in the capitals spelling, so P3 carries the pair `DR. NÉE JUAN Y LOPEZ` / `dr. née juan y lopez` instead, the marker declined behind a title the same way: middle "JUAN Y" with "Y" an initial, against family "juan y lopez".) + REPORT SCOPE IS SUBSET MEMBERS ONLY. Both readings are live only there; "y" in a one-case name is not in doubt, and reporting it would be noise on every lowercase Hispanic record. Matches how `particles_ambiguous` and `suffix_acronyms_ambiguous` report, and matches AGENTS.md's "a kind is worth adding only if a reader would hesitate too". + CASELESS LETTERS NEVER ENTER THE FORK. Arabic و has no case, so `token.upper() != token.lower()` is false and today's reading stands. A caseless-script input counts as "one case" under the helper, harmlessly, because the fork also requires a cased token. + NO SWITCH. The subset is the knob: remove "e" to restore the joining reading, add "y" for a Dutch-style "every single letter is an initial". That is also this shape's answer to #516's switch question. + ACCEPTED, AND RECORDED RATHER THAN FIXED: a locale pack cannot express "remove e". `Locale.lexicon` is unioned onto the base and never removes — its own field comment says "a pack never removes base vocabulary" — so an `nl` pack CAN add "y" while a `pt` caller has to write `Parser(lexicon=Lexicon.default().remove(conjunctions_ambiguous={"e"}))` by hand. The first (A) bullet added to #3-0-reevaluations on this date asks whether packs should be able to remove vocabulary at all. + ACCEPTED, AND DEFERRED TO A FOLLOW-UP ISSUE (Derek, 2026-09-13): the facade and the core now disagree about the same letter. `HumanName.initials()` does not follow the parse's tags — `_facade._process_initial` re-derives "is this the connective" from vocabulary plus initial shape on the part's raw text — so `HumanName("john e smith").initials()` stays "j. s." while `parse("john e smith").initials()` gives "j. e. s.", and `JUAN Y GARCIA` splits the other way, facade "J. Y. G." against core "J. G.". Both measured here 2026-09-13 and pinned by a contrastive test rather than left to prose. `_facade.py` is untouched in this PR; the fix has R4's shape — consult the PARSED token wherever the part maps to one and fall back to the vocabulary only where it does not — and it is a follow-up issue, not yet filed, so no number is cited here. `capitalize()` is not split: it follows the parse on both surfaces, so `HumanName("john e smith").capitalize()` gives "John E Smith" too. + 2026-09-13 #528 — AMENDS the paragraph above: CLOSED THE SAME DAY. `_facade.py` was touched after all — #528 shipped the fix the paragraph predicted, with the R4 shape it predicted: the facade consults the PARSED token wherever the part maps to one and falls back to the vocabulary only where it does not. Both views give "j. e. s." and "J. G." now (measured 2026-09-13 after the fix); the contrastive test the paragraph above describes became the agreement test `tests/v2/test_render.py::test_facade_initials_follow_the_one_case_fork`. `decisions/R3`'s 2026-09-13 entry carries the fix, its accepted costs and the one corpus name where the two views still differ. `capitalize()` was never split: it followed the parse on both surfaces throughout the window the paragraph above describes, giving "John E Smith" for that whole window — not "all along", since 2.3.0, before #383/#479 landed, gave "John e Smith" for the same name. + TWO OTHER CAUSES SHARE THE `_initials` FIELD IN THE LEDGERS AND NEITHER IS THIS PR'S, recorded because a reader meeting them under these names will reach for this entry. The Arabic `محمد و علي` diffs on `_initials` at 1.4.0 only (`م. و. ع.` → `م. ع.`), measured byte-identical either side of the fork: it is a pre-existing #269 consequence — a recognized non-Latin connective contributes no initial — surfaced by the row entering the contract corpus, and `expected_since_1.4.0.toml` ledgers it as `feat(#269)`. `JUAN Y GARCIA` diffs on `_initials` at 2.0.0 through 2.2.0 for a DIFFERENT reason than it does at 2.3.0: at those three baselines `fix(#462)` already admits the name (the facade moved `J. G.` → `J. Y. G.` there), and this PR's core move `J. Y. G.` → `J. G.` gets its own rule at 2.3.0 alone. Two causes, one field, documented in a dated paragraph on `fix(#462)` in those three ledgers rather than as a competing rule. + Out of scope, each its own issue: #492 (whether a cased suffix token counts as case evidence — `is_one_case` is written so R5 and #492 can share it later, but render does not import it here); #478 (hyphenated connective repair — its spaced-form claim now depends on "y" staying OUT of the subset); #461 (R3's clause); #289 and #516 (the same case-class fact read at the suffix and post-comma slots); the render-side conjunction fallback for spliced raw text, which stays vocabulary-keyed as rules/R4 states. + - 2026-09-15 #289/#516 — AMENDS the out-of-scope list above: #289 and #516 are no longer out of scope, having shipped in 2.4 on the fact this entry's own fork computes. The fact is now a `ParseState` field rather than a classify local, taken over the SAME own-words span this entry defines, and `_pieces.own_words` is where that span lives for both readers. #492 stays out of scope and stays deferred. See #S2. + - 2026-09-23 #492/#478 — AMENDS the out-of-scope list above again: both have landed, in RENDER only. #492: R5's gate leaves the suffixes out (decisions/R5, 2026-09-23); this entry's fork is untouched and still counts a cased suffix, which is the R5/parser split that entry records, and render still does not import `is_one_case`. #478: a connective part inside a hyphenated word keeps its lowercase (decisions/R4, 2026-09-23); the spaced-form claim it rested on is unchanged, and still depends on `y` staying out of the subset. + +- 2026-09-13 #383/#479 — `conjunctions_ambiguous` is deliberately NOT registered in `_SUBSET_FIELDS`, so an orphan marker entry raises nothing. An orphan there is INERT rather than harmful: the classify fork tests membership in `conjunctions` before it reads the subset, and the emitter tests it again, so a marker entry whose base word is gone is never consulted by either. AGENTS.md's invariants rule guards harm, not no-ops, and the precedent is this file's #given-name-titles Declined entry, where two attempts at a check each cost a working configuration to forbid a condition that costs nothing. What holds the SHIPPED constant to the subset relation is an import-time assert in `nameparser/config/conjunctions.py` (which also holds it to cased single letters, the only entries the fork can read), and the v1 shim's intersection, which is a provable no-op. So `remove(conjunctions={"e"})` simply works, leaving a stale marker entry behind that does nothing. Pinned by `test_removing_a_conjunction_leaves_its_ambiguous_marker_alone` in tests/v2/test_lexicon.py and by the orphan behavioral pin in tests/v2/pipeline/test_classify.py, both of which cite this entry. + +- 2026-09-13 #383/#479 — AMENDS the 2026-08-27 #445 entry under M4 below, which calls `initial` "the `initial` shape tag": since this change the tag is no longer purely a shape claim, because a marked single-letter connective in a one-case name carries it without being initial-SHAPED. rules/M4's statement now says "read as an initial" for that reason, and M4's carve-out itself is unchanged — it still reads the tag classify recorded rather than a predicate of its own. + +- 2026-09-20 #397 — DECIDED AND SHIPPED: `i` joins `conjunctions` and `conjunctions_ambiguous` together, which is what the 2026-09-13 entry above said would happen and which half of it the conjunctions Open block in decisions/vocabulary-exclusions called contested. `parse("Josep Carod i Rovira")` gives family "Carod i Rovira" where every release from 1.4.0 through 2.3.0 gave middle "Carod i" with family "Rovira"; `parse("Josep Lluis Carod i Rovira")` gives middle "Lluis" with that same family; and `parse("Carod i Rovira, Josep")` gives family "Carod i Rovira" where they read family "Carod Rovira" and took the link into `suffix` as a generation marker (all three measured 2026-09-20 on the released 1.4.0, 2.0.0, 2.1.0, 2.2.0 and 2.3.0 wheels from a throwaway environment, this tree shadowing a wheel otherwise). + THE OFF SWITCH is `Lexicon.default().remove(conjunctions={"i"})`, and on the v1 surface deleting `i` from `C.conjunctions`; both restore all three readings above byte for byte, measured 2026-09-20. That is a PROPERTY rather than an anecdote, and it is this change's durable statement, and it takes THREE exemptions rather than two once case repair is included — the earlier wording of this sentence named two and the release note named none. A name's FIELDS equal its reading with the letter out of the vocabulary unless (1) a link joined between two name words or (2) the two readings disagree about the letter being an initial; its REPAIR, plain and forced, equals the off switch's unless one of those two holds or (3) the parse placed the letter among the NAME words as a connective, where R4 keeps it lower-case and the off switch — for which it is no connective at all — title-cases it: `parse("Dr. John i Smith").capitalized(force=True)` keeps 'i' against the off switch's 'Dr. John I Smith', and `Carod i Rovira` and `Josep i Rovira` are the same shape. Measured 2026-09-20 over the shipped grid's 28,448 class-letter rows: 1,462 take the join exemption, 9,296 the initial-reading one, and 2,228 more reach the third, which is INV7's alone. `tests/v2/test_properties.py::test_a_link_that_joins_nothing_changes_no_field` runs it over a 16,576-text grid paired with the configurations that can read each text differently — 66,752 rows, of which 28,448 carry a class letter and are parsed twice — and carries the recorded control: 1,480 failing parses at `c8550b64`, the commit the first review was written against, 154 at `dc3bdf9c`, and 0 here. (Quoted over the full cross product of texts × lexicons × policies the earlier numbers were 2,360 and a 12,880-name grid times eight parsers; the shipped grid is trimmed by SHAPE and grew again with the second review, so the test's own docstring is the live figure and this is a dated snapshot.) + THE COUNT REACHES THE CLASS (Derek, 2026-09-20), which ANSWERS the C-ii question the 2026-08-21 #418 bullet above left open — "whether a word's membership in one vocabulary should suppress it from a count gating a rule keyed on another". It does not: a single-letter connective counts as a name word toward P3's three-word carve-out whatever else the vocabulary says it is, and `i` is also the roman numeral. The alternative was rejected on its own arithmetic rather than on taste — counting the link as the generation it also spells lets a connective raise the bar for ITS OWN join, so "Josep Carod i Rovira", the shortest linked name and the commonest one, would be the only shape that failed to link while the four-word spellings joined. The arithmetic runs the other way too, which is why the count is scoped to where the word is PLACED TO JOIN: a connective joining nothing would otherwise LOWER the bar for an unrelated one, and "Carod y Rovira i" is that shape — it counted the trailing generation and let the "y" join. It reads given "Carod", middle "y", family "Rovira", suffix "i" here, the same FIELDS the parent gave — its initials moved, but under R3's rule rather than this one. + ACCEPTED CONSEQUENCE OF REACHING THE CLASS, and the one row the design's reading tables show it touching: under `Lexicon.default().add(particles={"y"})`, `Juan Velasquez y Garcia` goes from given "Juan" / middle "Velasquez" / family "y Garcia" to given "Juan" / family "Velasquez y Garcia" — the custom lexicon now AGREES with the default one, which is the right direction — and its `family_base` becomes "Velasquez Garcia", a pre-existing R2 limit (`family_base` drops a particle wherever it stands, not only leading) surfaced rather than created here. Measured 2026-09-20 on this branch. + WIDER THAN ONE ROW, and the second review measured how much: the design's reading tables show one name because they hold one of this shape, not because one moves. Over the review's 43,925-name generated grid, `add(particles={"y"})` moves a word the parent roled SUFFIX into a name part on 130 names BEYOND what the default lexicon already moves there — "ABDUL Y ROVIRA MA" goes from given "ABDUL" / family "Y ROVIRA" / suffix "MA" to given "ABDUL Y ROVIRA" / family "MA" — and 71 of those 130 still report `suffix-or-name` on the moved word, so the reading is marked rather than silent. Four more lose a title, all four the same name in its four spellings ("Jr. y Soler Puig MA" and its casings). The direction is the one the row above describes: the custom lexicon converges on the default one, the particle joining what the connective already joined. Measured 2026-09-20. + THE BOTH-SIDES SCOPE, AND THE REVIEW THAT FIXED IT. A connective that is also generational vocabulary joins only where a NAME WORD stands on each side of it — a word the rest of the parse reads as a name word rather than as a credential, the trailing numeral or a title, looked for past any run of connectives standing between. The FIRST implementation tested POSITION (an interior piece) rather than the neighbours' CLASS, and swallowed a credential: "Josep Lluis Carod i III" read family "Carod i III" where the "III" is the generation. An independent review found it. It is recorded here because the invariant that ought to have caught it cannot: INV1 inspects the part the join PRODUCED, where the absorbed credential is itself the name word standing on the right, so that name SATISFIES it — the general lesson is a field note in mechanisms.md of this date, and the instrument is `tests/v2/test_properties.py::test_a_trailing_credential_never_joins_into_a_name_part`, which asks the off-switch parse instead and carries its own control (639 failing parses at `c8550b64` over the shipped grid, 105 at `dc3bdf9c`, 0 here; 960 over the full cross product, a dated snapshot). + THE BOUND THE CONDITION READS, AND THE SECOND REVIEW THAT FIXED IT (2026-09-20). "A name word" is looked for between assign's leading-title run and the start of its trailing suffix run, and that second bound was read by `_pieces.trailing_start`, the peel over the pieces as WRITTEN. A trailing TITLE hides the suffix run from that peel — it takes nothing and answers `len(pieces)` — so the condition was told a credential was a name word and the join swallowed it: `John Quincy Adams i MA Prof.` read family 'Adams i MA' with no report and an `initials()` of "J. Q. A. M.", where `John Quincy Adams i MA`, the same name one title shorter, reads family 'Adams' with suffix 'i MA' and reports the acronym. `John Quincy i MA Prof.`, `Josep Lluis i MA Prof.` and their one-case spellings are the rest of it. THE FIX IS H5's OWN READING, `_pieces.trailing_start_past_titles`, which runs the peel and the title chain to the fixed point `tail_reading` already computes for assign — so the join and the placement agree about where the name ends, which is what H5's sentence says they must. Asked once per segment and only where such a connective was found, so no ordinary name pays it; frame-neutral on `tools/perf/call_count.py` (parse=406.00, facade=443.00). Measured over the review's 43,925-name grid under eight configurations: 27 rows move, on 9 distinct names, every one of them onto the reading the off-switch parser gives. + AND THE CONDITION IS ASKED ONCE PER RUN, not once per member, which is a THIRD fix in the same clause and was a defect the second review left standing (found in its own report, fixed 2026-09-20). `_name_word_beside` walked the run of connectives beside a link to find the first name word past it, and both callers ask it per member, so a name holding a run of n links walked it n times and the stage went quadratic in the run's length: `"Josep " + "i " * n + "Rovira"` grew 3.8x per doubling against the 2.0x every other shape holds — 59ms at n=800 and 214ms at 1,600, where the parent 46651750 reads 8.3ms — and `"Jane Doe nee Puig " + "i " * n + "Soler"` the same, the clause's own caller. Every member of one contiguous run has the SAME nearest name word on each side, so `_group._run_neighbours` answers it once for the whole segment in two linear passes, built on the same gate as the two bounds. All four shapes now read 1.97-2.03x per doubling to n=1,600. THE PRICE IS A CONSTANT ON SHORT NAMES and is recorded rather than hidden: one call plus `_is_conj_piece` for the pieces the walk never reached, so `Josep Carod i Rovira` goes 304 -> 307 frames and `Jane Doe nee Puig i Soler` 307 -> 312, against 6,741 -> 2,652 for that second name with a run of 64. A name with no generational connective is untouched (`John Smith` 163, `Smith, John` 185, `Juan Garcia y Lopez` 285, `Jane Doe nee Smith` 233, all unmoved). Byte-identical over 1,748,904 parses — 145,742 names, the corpus, the case table, all three property grids and a run-heavy generated set, under three lexicons and four policies — same digest on both trees. Guarded twice, because one instrument cannot reach both callers: `tests/v2/test_benchmark.py`'s thirteenth `_SHAPES` row (`link_run`, `"i Und "`) times the join's caller, and `test_a_clause_link_run_does_not_cost_quadratically` counts frames for the clause's, which no repeated unit can express. + NOT FIXED AND RECORDED INSTEAD, because it is neither this rule's nor this branch's: the PARTICLE CHAIN reads the same bound the same blind way, so a trailing title lets it run to the end of the segment. `de la ma prof.`, `de la e ma prof.`, `van der berg ma prof.` and `Josep de Carod y Rovira MA Prof.` all put the title inside the family at the parent 46651750 and at 1.4.0, with no letter of this class in them at all. What the branch does is send MORE names down it — once `i` is marked, a one-case `de la i ma` stops ending in a suffix piece and the chain reaches past it — so `de la i ma prof.` and `de la i rovira prof.` now read as their `e` twins always have. 25 names of the review grid are in this family and none of them is a `_name_word_beside` case. It is the #535 boundary and belongs to its own change; `tests/v2/test_properties.py`'s INV1-strengthened carries a LINK-FREE CONTROL so the invariant can tell the two apart, and says so where it is asked. + THE CLASS TEST IS NOT A LENGTH TEST, corrected by the second review (2026-09-20). The stage's own filter carried a `len(tok.text) != 1` beside the two tag tests, narrowing this clause to ONE-LETTER connectives where the rule names a class. Caller-reachable: under `Lexicon.default().add(conjunctions={"og"}, suffix_words={"og"})`, `John Quincy Smith og` read family 'Smith og' with the filter and reads family 'Smith' with suffix 'og' without it — the rule's own answer. DROPPED, measured byte-identical over every oracle in use (fields, reports, `initials()`, `capitalized()` plain and forced, every token's role) across 359,053 parses: the 351,400-parse review grid under eight lexicon/policy/locale configurations, the 2,175-parse sweep of `tests/v2/cases.py` under three orders and the 5,478-parse sweep of the differential corpora under six. It has to be — `i` is the only member of the class in the default vocabulary and in every locale pack, and it is one letter. The THREE-WORD carve-out stays single-letter, because that is what its own sentence says. + THE SCOPE'S ALTERNATIVES, MEASURED 2026-09-20 rather than argued, by building each alternative from this tree and sweeping the 1558 non-empty corpus-union-cases names under eight configurations (default, `lenient_comma_suffixes=False`, both family-first orders, and the four locale packs). ABSENT — no both-sides condition at all — differs on five names, and every one of them is a row this bundle added: "John Quincy Smith i" would read family "Smith i", "Josep Lluis Carod i III" family "Carod i III", "Josep Lluis Carod i V" family "Carod i V", "Carod y de Rovira i" given "Carod y de" with family "Rovira i", and "Rovira, Josep Carod i Jr." middle "Carod i Jr.". Nothing that stood in the corpus at the parent separates absent from narrow, so those rows ARE the evidence and pruning one retires it. WIDE — the same condition with the generational-vocabulary test dropped, so it reaches every single-character connective — differs on five as well, and THREE of those were in the corpus already: "Mr. & Mrs. John Smith" would read title "Mr." with given "&" and middle "Mrs. John", "1 & 2, 3 4 5, Mr." would lose "2" out of the family, and "Jane née Jr y Jones" would read middle "née Jr y" with family "Jones". So the class test is not decoration — widening the rule past the link-and-generation class breaks "Mr. & Mrs." — and narrow is both what the design asked for and what measurement supports. (A "0 differences over 1545 names" figure was measured for this scope before the review supplied the class test and before those rows existed; it is superseded here.) + ACCEPTED, THE TWO-WORD SHAPE: "Carod i" keeps given "Carod" with suffix "i", and "Josep Carod i" keeps family "Carod" with suffix "i", because a connective with nothing to its right is connecting nothing. 2.0.0 through 2.3.0 read BOTH of those the same way; 1.4.0 read the two-word one differently, giving family "i" with given "Carod" (measured on the wheels 2026-09-20, correcting an earlier draft of this bullet that claimed 1.4.0 parity for it). That does not move the accepted cost, because the 1.4.0 difference is not this rule's: the two-word shape's routing left 1.4.0's reading in 2.0, which is why the case row carries `fix(suffix-routing)` rather than a connective classification. What this rule decides is that the link does not join there, and 2.0 through 2.3 agree with it. The ledger rule that called this NOT WANTED is rewritten in place rather than deleted — it still explains the name, and the standing is decided now rather than unwanted. + ACCEPTED, A CONNECTIVE RUN: "Henry i of England" reads given "Henry i of England" with a `given-or-family` report, where the parent read given "Henry" with family "i of England". The "i" and the "of" merge as a run and the run joins the whole name into one piece — the same thing "Duke of Edinburgh" already did. + ACCEPTED, THE LOWER-CASE REGNAL NUMERAL, and it is the shape that makes the join FIRE with no report at all on input nobody would call Catalan: a mixed-case name with a lowercase `i` between two name words joins like any other link, so "John Paul i Smith" reads family "Paul i Smith" where the parent read middle "Paul i", and "john i Smith Jones" reads given "john i Smith" where it read middle "i Smith". Nothing marks the reading, because nothing is ambiguous by this rule's lights — the letter IS connective vocabulary and it IS joining two name words — and a report there would fire on every Catalan name too (rules/A1's hesitating-reader test). The frequency is the argument for accepting it: measured 2026-09-20 over the deduped corpus-union-cases population of 1,568 non-empty names under the three orders, the join fires on SIX names and five of them are the Catalan rows this bundle added; the sixth is "Henry i of England", the paragraph above. A regnal numeral written lowercase between two name words is the shape that would pay, and the corpus holds one. + ACCEPTED, THE PARTICLE ON THE RIGHT. A particle is a name word for the both-sides test, so the link joins across one: "Josep de Carod i Rovira" reads family "de Carod i Rovira", which is the Catalan reading, while "Josep Carod i de Rovira" reads middle "Carod i de" with family "Rovira" where the parent read middle "Carod i" with family "de Rovira" — the join takes the particle as its right-hand neighbour and the particle chain then follows the joined piece instead of the tail word. Recorded rather than repaired: which side a particle standing between a link and a surname belongs to is P2's question and not this one. + ACCEPTED, AND THE HONEST STATEMENT OF THE ONE-CASE HALF, written out because this branch's own first commit message overclaimed that one-case names keep their reading. `i` is in the MARKED subset, so in a name written wholly in one case it reads as an INITIAL and reports `conjunction-or-initial`, exactly as `e` has since #383/#479. For "JOSEP CAROD I ROVIRA" and "josep carod i rovira" the fields are indeed unchanged and only the report is new. But a one-case name reports wherever a bare `i`/`I` stands among the NAME'S OWN WORDS, which is a great many more names than the Catalan ones — "JOHN I SMITH" and "john i smith" one report each, "JOHN SMITH I" and "john smith i" two, "HENRY I" and "henry i" three. A letter inside a maiden clause is read by the clause's rules and stays silent, exactly as `e` does ("JANE DOE NEE I JONES" reports nothing), which is the own-words scope the 2026-09-13 #383/#479 entry above defines. And in ALL-LOWER names the initial reading MOVES FIELDS wherever the parent read the lower-case letter as the generation. Each such name now reads as its ALL-CAPS twin already did: "rovira, i" gives given "i" where it gave suffix "i" ("ROVIRA, I" already gave given "I"); "john smith i jr" gives middle "smith" with family "i" where it gave family "smith" with suffix "i jr" ("JOHN SMITH I JR" already did); "maier, amy i, jr." gives middle "i" with suffix "jr." where it gave suffix "i, jr." (the corpus name "Maier, Amy I, Jr." reads middle "I" and does not move); and "josep de carod i rovira" gives family "de carod i rovira" where it gave middle "de carod i" with family "rovira". None of those four is a corpus name; all are measured 2026-09-20 on this tree and against the parent's. + NO CHANGE TO THE EXCLUDED BLOCK, recorded as a decision rather than left implicit (Derek, 2026-09-20). It now lists "y" against TWO marked letters rather than one, and that is still right: the argument that put "y" outside is about "y" — the commonest Hispanic compound and this library's oldest fixture — while "i" matches "e" exactly, a bare I initial being as common as a bare E. + +- 2026-09-21 #397 follow-up — AMENDS the "AND THE CONDITION IS ASKED ONCE PER RUN" paragraph above, whose frame figures stand as measured at `6048eb5d`. BOTH SIDES ARE NOW ONE CALL. A connective is placed to join only where a name word stands on EACH side of it, so neither caller ever wanted one side's answer: `_name_word_beside(k, step, ...)` was asked twice and the two answers ANDed at the call site, in the `frozen` loop and again in `_link_joins_inside_the_clause`. It is `_group._between_name_words(k, lo, hi, ...)` now and answers for both — one Python frame per generational connective instead of two, with `step` and its ternary gone. Nothing else moved: the left arm still short-circuits the right, and the arrays, the bounds and the two piece tests are the same tests in the same order. Byte-identical over 3,931,700 parses — 157,188 names (the corpus, cases.py, the property grids and a run-heavy generated set) under six lexicons and four policies, plus the facade surface and the #528 no-parse paths — same sha256 for all twenty-six digest rows and the dumps compare equal. + THE OLD NAME STILL STANDS WHERE IT WAS WRITTEN, and this sentence is the grep trail: `_name_word_beside` is the spelling in decisions/M2's 2026-09-20 #397 bullet and in the "NOT FIXED AND RECORDED INSTEAD" paragraph above, and dated entries are not edited, so neither is corrected — a search for either name reaches the whole arc. One place the old name was NOT a citation and is renamed: `tests/v2/test_properties.py`'s per-side mirror, now `_name_word_on_the_side`. That helper is a deliberately independent second implementation, reading spans and off-switch roles where the parser reads pieces and tags, and carrying the implementation's own name made it read as a call into the thing it checks rather than as a model of it. + THE FIGURES, AND THE INTERPRETER, because the pair above spliced two and this one does not. Re-measured 2026-09-21 on py3.11 through `tests/v2/test_benchmark.py`'s own `_frames_for` shape, `b9ed1429` → `9fd84463` → this tree: `Josep Carod i Rovira` 311 → 314 → 313, `Josep Lluis Carod i III` 377 → 381 → 380, `Jane Doe nee Puig i Soler` 315 → 320 → 319, `John Quincy Adams i MA Prof.` 438 → 443 → 442, and the clause guard's own pair 1,125 → 875 → 859 at sixteen links and 6,741 → 2,651 → 2,587 at sixty-four (3.01x, re-pinned in the test). `tools/perf/call_count.py` is unmoved at parse=406.00 facade=443.00 — its reference name carries no link — and so are `John Smith` 172, `Smith, John` 203, `Juan Garcia y Lopez` 291, `John and Jane Smith` 293, `Jane Doe nee Smith` 245 and `Jane Doe nee Smith PhD` 340, none of which reaches the predicate. WHICH OF THE EARLIER FIGURES REPRODUCE, stated exactly where this sentence first said only that one half "very nearly" did: re-measured 2026-09-22 on py3.11, the short-name pairs (304 → 307, 307 → 312) do not reproduce at all, and of the run-of-64 pair (6,741 → 2,652) the LEFT half does and the right does not — `b9ed1429` reads 6,741 on the nose, while `6048eb5d`, the tree its 2,652 was taken on, reads 2,651 here, the same figure `9fd84463` reads. So the splice runs through a single arrow rather than between two of them, which is exactly the spliced table `tools/perf/call_count.py`'s docstring was written about. Timings are unmoved: `"Josep " + "i " * n + "Rovira"` and `"Jane Doe nee Puig " + "i " * n + "Soler"` both read 1.96-2.02x per doubling from n=200 to n=1,600, 11.36ms and 9.39ms at the top end against 11.58ms and 9.58ms at `9fd84463`. + AND THE FOLD BOUGHT A TEST, which is the part worth keeping: while the predicate was called once per side, a mutation of the suffix or the title test hit BOTH sides at once and the left-hand rows killed it, so the right-hand halves were never separately covered — their own rows (`Josep Lluis Carod i Jr.`, `i Mr.`) stand at the END of the name, where `hi` refuses them before either piece test is asked. Folded, the two sides mutate independently and both right-hand tests survived the whole suite. `test_a_credential_or_honorific_mid_name_on_the_right_too` is the pair that kills them, and every arm of the folded predicate now dies by a named test (recorded in its docstring). + +- 2026-09-24 #478 — CROSS-REFERENCE, no change to this rule: case repair's hyphen clause is the one place a marked letter in a name written wholly in one case is read as the connective rather than an initial — `maria silva-e-sousa` repairs to `Maria Silva-e-Sousa` beside the spaced `Maria Silva E Sousa`, and `conjunctions_ambiguous` does not reach the hyphenated form. Decided there, with its cost (`J-E-P DUPONT` → `J-e-P Dupont`): decisions/R4's 2026-09-24 #478 bullet. + +- 2026-09-26 #538 — CROSS-REFERENCE: rules/P3's separator sentence (a declared delimiter is read past as a connective is) is decided and measured in the 2026-09-26 #538 entry under `### M2`. +- 2026-10-03 #549 — CROSS-REFERENCE: that separator sentence is reversed; it now says a declared separator ends the search as a comma would, a connective never joining across one, and the decision is C1's 2026-10-03 entry. + +- 2026-10-06 #617 — the comma decision joins title words with GROUP'S connective loop, not a copy of it. `_comma.decide` reads the part after the first comma before group joins anything (#613), and the one join the part can hold — a connective binding title words into one title, `John Smith, Mr. and Mrs.` — was made by `_comma._title_chains`, a second copy of the P3 loop in `_group_segment`. The two had already parted three ways: the copy merged no connective run before joining, skipped a single-letter connective unconditionally where group skips it only under the rootname count's carve-out, and derived the title kind alone. Both now call `_pieces.join_connectives` (mechanisms.md#ONE-PREDICATE-PER-QUESTION), which takes the carve-out's answer from its caller (`letter_stays`: group's `total < 4`, always true for the comma decision's own reading, and reaching a LONE single-letter connective only, one beside another connective merging into their run first), a `titles_only` mode for the comma part (a join is made only into a title and takes no prefix kind, any other connective left a word for the reading to place, which is what the copy did), and the `frozen` token set group already passed itself. The merge and the piece predicates it needs moved with it (`merge_pieces`, `joined_tags`, `is_conj_piece`, `is_prefix_piece`), and `_comma._reading_pieces` builds the reading's input, calling the shared loop only where the part holds a connective left free — nearly no part does, and the first cut, calling it always, cost every family-comma parse two frames per word of the part, one `is_conj_piece` call in each of the loop's two passes (`Smith, John` +2, `Smith, John` with sixteen more given words +34, measured inside decide in #617's review); `test_a_comma_part_with_no_free_connective_skips_the_join` holds the gate, with that control recorded. + DECIDED (Derek, 2026-10-06): (1) A CONNECTIVE RUN merges in the comma part as it does in the name — P3's own "connective runs included" — so a title holding `of the` reads whole there too, and a single letter beside another connective joins as a member of their run. `John Smith, Minister of the Interior and Mrs.` reads given 'John', family 'Smith', title 'Minister of the Interior and Mrs.' (83f4e914: family 'John Smith', the copy having joined 'Minister of the' alone and left 'Interior' a name word); `Smith, Chancellor of the Exchequer MP` reads suffix 'MP' (83f4e914: given 'MP'); `John Smith, Mr. y and Mrs.` reads title 'Mr. y and Mrs.' behind given 'John' (83f4e914: family 'John Smith'). The garbage case the issue named, `Prof. Smith, and and Dr.`, reads title 'Prof. and and Dr.', family 'Smith' (83f4e914: title 'and and Dr.', family 'Prof. Smith'). Every one of these reads as 2.3.0 read it, checked against the released wheel, and so did b547c247: what moves on realistic input was #613's copy regressing names it never meant to touch, between releases, so no release note follows. A LONE single-letter connective is unchanged: the comma decision's reading joins no title across one, so a part of titles it separates is no titles-alone part and the listing form reads it, where P3's own count decides the join — `Juan Garcia Lopez, Sr. y Sra.` keeps given 'y' and title 'Sr. Sra.', while `Smith, Sr. y Sra. Juan Maria` joins title 'Sr. y Sra.' behind given 'Juan', both as 2.3.0 and 83f4e914 read them. A case row pins the first. No rules.md text states it, because it is no rule of the comma's: the first review round added a C1 clause saying a lone letter after a comma always stays a word, and the second struck it with the `Juan Maria` reading, P3 deciding in the listing part as everywhere else. (2) EVERY CONNECTIVE OF THE GENERATIONAL CLASS IS FROZEN in the part the comma decision reads, P3's "joins only where a name word stands on each side of it" applied to a part read for credentials, which holds no name word — group freezes the ones it finds without a name word beside them, the same rule. Frozen means no join or run of its own; a neighbour's join may still take one in, as in group (`Smith, Dr. and i` reads title 'Dr. and i'). The default class is `i`, which as one letter the copy never joined and which reaches the freeze only through a run: without it the shared run merge took `Rovira, Dr. i i` into title 'Dr. i i', which `test_a_link_that_joins_nothing_changes_no_field` caught. A generational word of more letters reaches it in a plain join too, which only caller vocabulary can spell, and there the copy DID join it: under `Lexicon.default().add(conjunctions={"og"}, suffix_words={"og"})`, `Smith, Mr. og Mrs.` reads title 'Mr. Mrs.', suffix 'og' (83f4e914 and 2.3.0: title 'Mr. og Mrs.'; b547c247 as now), and as a plain connective `og` still joins. + MEASURED 2026-10-06 against 83f4e914, py3.11, over that tree's population — each branch-added case row and the new rules.md example are fixes against it by construction and not counted here. Two session harnesses, recorded as recipes: each parses its names on both trees and compares the seven role fields, every ambiguity's kind and detail, and `initials()` (the fingerprint adds `capitalized()` and each ambiguity's token spans). The FINGERPRINT (78,282 parses): every corpus name and case row, plus 6,000 seeded draws (`random.seed(7)`) of a front from `'', Smith, John Smith, John Q. Smith, Dr. Smith, de la Vega, SMITH, LLOYD WEBBER, Smith Jr., Vega y Lopez, 김민준 박, 김민준, Dr 김민준` and one to four words behind its comma from the 40 words `Smith John Jane de van vd VD mc do DO MA Ma PhD Ph. D. MD Jr. Jr V V. VI III G.J. XY XYZ X.Y.Z. Dr. Mr. and Prof. Sir i Und RN/CRNA / Jones 김민준 박 J.씨 씨`, three in ten with a third part (each draw in that order: `choice(fronts)`, `randint(1, 4)`, `choice(words)` per word, `random() < .3`, `choice(words)`), plus every two-word pair from that list behind `John Smith, `, `, ` and `Doe, Jane `, under six policies (default, both family-first orders, strict comma suffixes, `/` as a suffix delimiter, caps suffixes off); 30 parses differ, every one a generated name with two connectives side by side after the comma, and no corpus name and no case row moves. The CONNECTIVE GRID (485,300 parses): every three-word string over the 30 words `and of the y e i Und & Mr. Mrs. Dr. Prof. Secretary State Sir PhD MA MD Jr. III V de van der la John Smith Jones J. Ma`, bare and behind `John Smith, `, plus 40,000 seeded draws (`random.seed(617)`) of a front from `'', Smith, John Smith, Dr. Smith, Prof. Smith, de la Vega, Vega y Lopez, SMITH` and one to five of those words, written both as `front, words` and `words front`, three in ten with a third part (drawn in the fingerprint's order, `randint(1, 5)`), under four policies (default, family-first, strict comma suffixes, `&` as a suffix delimiter), and 2,700 parses under the `og` lexicon, every pair of those words in the three templates `John Quincy Smith og {a} {b}`, `John Smith, og {a} {b}` and `Smith, {a} og {b}`; 5,911 differ, 5,590 with two connectives side by side after the comma counting `og` as one, the other 321 the `og` freeze; 4,131 move a role and 1,780 only reports, and in every one of those 1,780 each report the branch adds names a run of two or more connectives, reported once as one run (`John Smith, III and and PhD` reports `'and and' follows a credential` where it reported each `and`). The five differential gates exit 0 and no diff moves but the new example's; that one new rules.md example reads as every 2.x baseline reads it and moves three 1.4.0 rules' corpus reach by one each. Frame cost falls by 10 a parse on every interpreter's reference row, and by 39 more once the /simplify pass gated group's call as the comma's is gated (decisions/parse-cost, this date); per name on 3.11, warm, against 83f4e914, `Smith, John`, `John Smith, PhD` and `John Smith, PhD MD MBA RN` cost what they did (161, 206, 294), a long given part less (`Smith, John Quincy Adams Bob` 332 → 297), and a comma part holding a connective pays the shared loop (`John Smith, Mr. and Mrs.` 262 → 269). + AMENDS decisions/C1's 2026-10-06 #613 entry, whose decision (3) names `_comma._title_chains` as where the title join is made and describes the join as P3's connective binding title words: it is `_comma._reading_pieces` calling `_pieces.join_connectives` now, and the join merges a connective run first and freezes the generational class, so (3)'s account holds of the join's PRODUCT, one title, and no longer of its reach — rules/C1 is amended to match. The `_PREFIXED_SHAPES` `title_chain` row, recorded there as the clock guard on the copy's slice rebuild, guards the shared loop on the same input (`'Mr. and ' * n` behind `John Smith, `), its control re-taken on the moved code (tests/v2/test_benchmark.py records it); the `_SHAPES` `conjunctions` row guards group's calls as before. +- 2026-10-07 #614 — CROSS-REFERENCE: P3's joins are made before S2's trailing run is read, so the run counts a joined part as one name word, as this rule says every count does (`Juan y Garcia X.Y.Z.` keeps family 'X.Y.Z.'), and a salutation keeps its connective (`Mr. and Mrs.` reads title 'Mr. and Mrs.'). #614's first design read before these joins too and broke both; #614's review reversed it (decisions/S2, this date). + +- Provenance: the single-letter-connective guard is v1's fix for Google Code issue 11 ("john e smith", 2013, commit 33676c9) — the "#11" citations that circulated pointed at a GitHub accident, not the real source. Recorded so the archaeology stays done. diff --git a/docs/design/decisions/P5-bound-given-names.md b/docs/design/decisions/P5-bound-given-names.md new file mode 100644 index 00000000..2c6496e0 --- /dev/null +++ b/docs/design/decisions/P5-bound-given-names.md @@ -0,0 +1,47 @@ +# P5 — bound given names + +- 2026-06-30 (first-name-prefix-join design; v1-era, carried into the v2 port) — the join is vocabulary-driven and deliberately tiny. + +- 2026-08-18 — `abd` ADDED, and the reserve count corrected to let it work. It is the transliteration for the spellings that write the article as its own word ("Abd Allah", "Abd al-Rahman"), which abdul/abdel/abdal do not match; the Arabic-script عبد has covered the same word since #269, so only the Latin side was short. Not a C-ii collision, though it looks like one: C-ii settles a PRECEDENCE by frequency where neither reading is a name, and here one reading is a name and neither had to be given up. It is C-i's positional qualifier (2026-08-17) applied to a bound-given word — the naming use is LEADING, the credential use trailing, which is the `de` row of that table ('a TRAILING surname — no clash') rather than the `vd` row. `abd` is also the postnominal ABD ("All But Dissertation") in SUFFIX_ACRONYMS, and all three credential spellings still parse ("Jane Smith ABD", "Jane Smith, ABD", "Jane Smith A.B.D."). + +- 2026-08-21 #411 (PR #416) — the reserve counts the name the join will actually leave behind, and the join never absorbs a marker. P5 has always said the join "needs a name word to spare, so two name words alone do not join"; the count was taken while the maiden marker and the maiden name were still pieces, because group's marker pass runs later in the same stage. `abdul Berg née Jones` counted four words, joined, and left no family name at all — given 'abdul Berg', family ''. That spelling is the pre-existing witness: it reads the same way at 2.0.0 and 2.1.0. `abd` does NOT — it reads correctly at 2.1.0 and acquired the defect only when #400 made it a bound given-name word this cycle, so for that spelling this is a regression #400 introduced and this closes. #399's chain stop widened the class again by putting the particle spellings in front of the reserve for the first time. + + Excluding the departing words is not sufficient on its own, and review is what established that. Where the maiden walk stops at an inner suffix the excluded span is short, enough words survive it to clear the threshold, and the join fires and takes the MARKER — after which nothing leaves and the count that authorised the join was reasoning about a name that never came to be. So the join now declines outright when the piece it would absorb is a marker, which is what P5 meant all along: it joins the bound word to a name word, and a marker is not one. Tested directly rather than through the span, because the two differ exactly where it matters — a marker with nothing but a suffix after it has no span at all, yet the join would still have absorbed it (`Berg, abdul née PhD` read given 'abdul née'). + + What sharing `_maiden_span` between the count and the marker pass buys is one definition of what a maiden span IS. It does not make the two agree about what leaves: they run at different times on different piece lists, and the join between them could invalidate the span the count was taken against. Forbidding the join to absorb a marker is what closes that, not the sharing — the sharing only keeps the two from disagreeing about the span itself. + + Two consequences. It closes the P5 half of the join-swallow #412 records, so that issue narrows to P3's connective join (closed the same day by the #412 / #417 / #418 entry under M2, which also retires this entry's by-hand exclusion). And where the bound word is also suffix vocabulary, a declining join after a family comma leaves the S2 suffix reading and the name has no given name at all — `Berg, abd née Jones` reads family 'Berg', suffix 'abd', maiden 'Jones', which is exactly how `Berg, abd` alone has always parsed. `abd` is the only shipped word in both sets. + + Two alternatives to the exclusion, recorded so "reorder" is not read as one option with one price. The rejected one moves the maiden handler ahead of the PARTICLE CHAIN, which changes what the consumer receives for every name — that is the scope objection, and it stands. A narrower one was not weighed at the time: move the marker pass to run after P2/P3's joins but BEFORE P5, inside _group_segment rather than in group(). Under that arrangement the reserve counts the surviving name directly, so the span prediction, the exclusion and the marker guard all disappear, and P2/P3 are untouched. Its real cost is ordering against the tail-segment delimiter-core drop, which today runs between the joins and the marker pass and would be reversed for comma tails — unmeasured, which is why this records the option rather than taking it. + + No corpus name produces a #411-attributable diff — every baseline already reads these names the way the fix does — so the ledger rule records the side effect on `van der Berg, abdul née Jones` rather than the defect. That name became an M2 example in the #399 fix and reached the corpus when #414's rules-doc corpus harvested it the following day. + What made it look impossible: adding the word alone changed nothing on the common three-word shape. The reserve count in the join gate excluded suffix-vocabulary pieces, and `abd` IS one, so its own piece did not count toward the three it needed — the rule declined silently. Counting the claimed piece as a name piece fixes it, and is right on its own terms: the reserve asks whether enough OTHER words are left to spare. Measured, the vocabulary-only alternatives both cost something the count fix does not — moving `abd` into SUFFIX_ACRONYMS_AMBIGUOUS loses "Jane Smith, ABD", and dropping it from SUFFIX_ACRONYMS loses the postnominal entirely. + Two limits, both measured and both accepted. The three-token "abd al rahman" still does not join — `al` is a particle and chains forward, which is the half of the original exclusion that survives. And position decides at the two ENDS only: in the given slot of a family-comma name the credential still wins, so "Smith, Abd" reports suffix 'Abd' with no given name where "Smith, Abdul" reports the given name, and no ambiguity is emitted for it. + Same shape as #397 (Catalan `i`, where dual membership shrinks the rootname count P3's carve-out tests), and the second instance the SAME DAY, #397 having been filed two and a half hours earlier. The general question — whether a word's membership in one set should suppress it from a count gating a rule keyed on another — is open there. + +- 2026-08-21 #418 (PR #420) — #411's by-hand exclusion of the departing words is gone, subsumed by the marker pass running before the reserve is counted (decisions/M2, 2026-08-21). Its other half stays: the join still declines when the piece it would absorb is a lone marker, because the only marker left by then is one the consumer declined, and P5 says a marker is not a name word — measured, removing the test re-read `Berg, abdul née PhD` as given 'abdul née'. "Lone" is load-bearing and review is what made it explicit: a declined marker the particle chain has already taken is a word of the chain, and the join takes the chain — `Abd van der Berg née Jr Jones` reads given 'Abd van der Berg née', where the old chain stop had kept the marker outside it. P5's statement now says "standing as a word of its own". The narrower reorder recorded above (marker pass after P2/P3's joins, before P5) was overtaken by the full one ahead of every join, which closes #412 and #418 as well, and the tail-segment ordering it priced as unmeasured was measured: byte-identical under the default policy, a real regression with a suffix delimiter configured, found in review and closed before merge (the M2 entry has the shapes). One row re-pinned: `abd née Jones Jr Smith Berg` reads given 'abd Jr', as `abd Jr Smith Berg` already does — P5 absorbing a suffix piece when the reserve allows is pre-existing and independent of markers, noted rather than fixed here. + +- 2026-08-22 #369 — a given-name title licenses the join with one word to spare. #367 regressed `Sheik Abu Bakar` from given 'Abu Bakar' to given 'Abu', family 'Bakar', and its release note recorded why the old reading had been an accident: `abu` is a particle as well as a bound word, and the title used to displace it out of the leading position so the particle CHAIN took 'Bakar'. `Sheik abdul salam` showed that a bound word alone never joined behind a title — the reserve wanted three name words and a title is not one. The question the issue put was whether `Sheik abdul salam` is a two-word given name with no surname, or given 'abdul' plus family 'salam'. Decided on the signal the title already carries: `sheik` is a GIVEN-NAME title, and H1 reads that membership as the assertion that the one word after it is the given name, family empty. P5 now reads it the same way — behind a given-name title the family comma's LENIENT reserve applies, 'abdul salam' joins, and `family=''` is the Accepted outcome H1 records for "Sir John". Not "after any title": `Dr. abdul salam` keeps given 'abdul', family 'salam', because `dr` addresses by family and the reserve exists for exactly that word. Keyed on the whole title run through `_title_key`, exactly as post_rules keys H1, so the two rules cannot read one run two ways: 'mr sir' is not a given-name title to either. Only STRICT relaxes — the family comma's DISABLED segment has no given name to join, and its post-comma segment is LENIENT already. + + AMENDED 2026-09-08 (#489, the #316/#489 bundle): the key is no longer the whole run. Both sites now match the run's folded key WHOLE or by its LAST WORD, and the invariant this entry rests on — that H1 and P5 cannot read one run two ways — is what the amendment preserves rather than what it costs, both sites still reading through one predicate beside `_title_key`. What moves is this entry's own example: `'mr sir'` IS a given-name-title run to both sites now, `sir` being its last word, so `mr sir abdul rahman` joins where the sentence above says it does not. The leaf test that pinned the old reading, test_group's `test_the_title_run_is_one_key_as_h1_reads_it`, is renamed `test_the_title_run_is_read_as_h1_reads_it`; it runs the GROUP stage alone, so what it pins is P5's side — that `mr sir abdul rahman` joins — and not the agreement between the two sites. The agreement rests on the shared predicate, `_run_addresses_by_given` beside `_title_key`, which is the one place either site asks the question, and it is ASSERTED end to end by `tests/v2/test_parser.py::test_the_p5_licence_and_h1_read_a_title_run_the_same_way`: eleven title runs, each parsed in both spellings, requiring `parse(f'{title} John').family == ''` and `parse(f'{title} abdul rahman').family == ''` to agree run by run. Nothing else in this entry changes: `Dr. abdul salam` still keeps given `abdul`, family `salam`, `dr` addressing by family. See #H1 for the measurement and the vocabulary scope. + + Measured at all three baselines: `Sheik abdul salam` moves (given, family), and with it the `Sheik abdul salam Jr` rules example — the same move with the suffix standing — and no other corpus name's fields do; `Sheik Abu Bakar` returns to the fields it had at 1.4.0, 2.0.0 and 2.1.0 — a restoration, which the ledgers record as a non-event — with one v2-surface change: the PARTICLE_OR_GIVEN report it carried through 2.1 is gone, because the join that takes 'Bakar' is P5's rather than the chain's, and a bound word read as the bound word is not a fork. The Arabic-script spelling comes along: `الشيخ عبد الله` read given 'عبد', family 'الله' and now reads given 'عبد الله', which is the name. The previous #369 ledger rules at all three baselines classified the regression "because the cause is known and recorded, NOT because the reading is wanted"; they are rewritten for the fix, and the 1.4.0 one is what keeps `Sheik abdul salam` explained at all: it landed on the fields-only suffix-routing catch-all before #451 deleted that rule, and with no fields-only rule left in any ledger the name now arrives UNEXPLAINED without it — measured by driving `classify()` over the 1.4.0 ledger minus this rule, which returns None for the name and for the `Sheik abdul salam Jr` example. A stronger reason to keep the rule than the one this entry first gave: the alternative is a red gate, not a mislabel. + + Two things review added before merge. The licence lifts the reserve for two WORDS, so the piece it lets the join take must be a single word: a particle chain is one piece, but it is the family name P2 built, and without that clause `Sir abdul van der Berg` read given 'abdul van der Berg', family '' — where the untitled name keeps family 'van der Berg', and so does `Sir John van der Berg`. H1's own scope is "exactly one name word", and the licence now matches it. `Sheik abdul al Rahman` therefore stays given 'abdul', family 'al Rahman', the same limit the `abd` entry above records for the three-token spelling. `Sir abdul van der Berg` is a rules.md example and so sits in the rules corpus, where it reads byte-identical to every baseline — the gate witnesses the clause; `Sheik abdul al Rahman` is in no corpus, and the group test is its only witness. And P5 now states the precedence `Sheik Abu Bakar` only exercised: a bound word that is also a particle is read as the bound word, the join outranking P4's leading-position reading, with no fork reported. That is the reading `Abu Bakar Salim` has always had — P5's join and P4's "the words stay separate" were never reconciled in writing — and the two rules now point at each other. + + Three more from the PR review. The licence takes a name WORD, not a suffix: lowered to LENIENT, the reserve would let the join absorb a suffix piece — `Sheik abdul Jr Smith` read given 'abdul Jr' — which is the shape #421 records for the post-comma LENIENT path and which the STRICT reserve had kept away from titled names; the licence now declines a suffix piece on its own, and #421 will make that decline general. One shape it cannot decline belongs to #401: `Sir abdul V` reads given 'abdul V', because the reserve's initial veto counts a bare `V` as a name word, and the lowered threshold now lets that count fire where STRICT did not — recorded in #401 rather than fixed here. And the join is order-blind, so under a family-first order the licensed pair lands in the FAMILY with no given name: `Sheik abdul salam` reads family 'abdul salam' under both `FAMILY_FIRST` and `FAMILY_FIRST_GIVEN_LAST` (was family 'abdul', given 'salam'), as `Sir John` reads family 'John' there — H1's exemption never runs under a family-first order, so the two rules still agree. Accepted rather than scoped to the given-first read: the effective order is settled in assign, after group, and the outcome is the one H1 already gives the same shape. Two corpus names move under each family-first order, `Sheik abdul salam` and `Sheik Abu Bakar`, measured by hand — the gate reads the default order only. + +- 2026-08-22 #401 / #421 — the reserve and the join ask the same question, and it is the one assign answers. Two defects, one cause: P5 says the join "needs a name word to spare" and joins "the word after it", and neither site knew what a name word was. The reserve counted the pieces that were not suffix pieces, and `_is_suffix_piece` vetoes on the initial tag — right for assign's middle-initial question, wrong for the reserve's: a bare `V` carries both `vocab:suffix` and `initial`, so `abdul Smith V` counted three name words, joined, and assign then read the `V` as the suffix it always was — given 'abdul Smith', family '' (#401; shipped since 1.x — 1.4.0 read first 'abdul Smith', last 'V' — every bound word, only the single-letter numerals V, I, X). The join never asked at all: with a word to spare it took a suffix piece as "the word after" — `abdul Jr Smith Berg` read given 'abdul Jr' (#421; 1.4.0 parity) — and where the suffix was the split credential, merge() unioned the piece tags, the joined piece became a suffix piece, and assign sent the bound word to the suffix field: `abdul Ph. D. Smith Berg` read suffix 'abdul Ph. D.', a 2.0 regression (v1's fix_phd extracted the credential before parsing, so position never mattered). + + Fixed with one predicate, `reads_as_suffix`: a suffix piece, or the trailing roman numeral assign's fork takes — and that fork is now `_vocab.is_trailing_numeral_suffix`, lifted out of assign so the two stages share its definition instead of the reserve carrying a copy (the `_ROMAN` pattern moved with it; the regex-sync pin follows it). The reserve counts a piece out when it reads as a suffix, and the join declines the piece it would take when it does — the same decline it already made for a marker. The mirror is assign's fork condition for condition: the numeral is the LAST piece and not the first name piece — not "the piece before it is not a title", which the first draft wrote and the design-docs review falsified, `jr` being title vocabulary as well as a suffix, so `abdul Smith Jr V` lost its family exactly as before — and the piece before it is read as the join would leave it, the joined pair's first token being the bound word (`abdul J. V`, the design-docs review's other find, otherwise suppressed the fork for the reserve alone and read given 'abdul J.', family ''; the reserve now declines, and assign, seeing the unjoined pieces, reads given 'abdul', middle 'J.', family 'V', exactly as it reads `John J. V`). Two more the code review found: assign drops the group-flagged credential pieces (the Ph. D. merge) from its walk at any position before the trailing peel, so a numeral can be last in that walk without being the last piece — `abdul Smith V Ph. D.` lost its family until "last" and "the piece before" were read over the pieces assign keeps — pinned at the piece level only, because a trailing split credential is the shape the 1.4.0 ledger's `[[never]]` entry refuses to explain any diff on, and v1's reading of that name carried the #401 family loss; the example stayed out of the rules corpus rather than the never-entry being narrowed for one bound-word spelling; and the post-comma walk has no numeral fork at all (a trailing numeral there is read by a lenient last-of-two rule, or as a middle initial with a third part), so a mirror applied under LENIENT declined `Berg, abdul V` and `Berg, abdul V, jr` against every baseline with nothing classifying it — the numeral branch is scoped to STRICT, and after a family comma the reserve counts suffix pieces alone, as it always did. So `abdul Smith V Jr` still joins and assign reads family 'V', and `abdul V Smith` still joins an initial, which is a name word. What is NOT mirrored: S2's other positional fork, the bare ambiguous acronym. Where the join leaves two pieces no family is lost — `abdul Smith Ma` reads family 'Ma' where `John Smith Ma` reads suffix 'Ma', assign keeping the acronym as the family of a two-piece name — but with a suffix word between, the join leaves three, assign peels the acronym and then the suffix, and the family is gone: `abdul Smith Jr Ma` reads given 'abdul Smith', family '', suffix 'Jr, Ma' where `John Smith Jr Ma` keeps family 'Smith'. Pre-existing and byte-identical before this fix, and the whole of what remains of #401's shape on the main walk: a review sweep of the STRICT path found 132 family-losing shapes, every one ending in a bare ambiguous acronym. Left for #425 rather than widened into here, and framed there as sharing assign's peel over a post-join VIEW rather than mirroring a second fork by hand: with a view the count is the view's length, the remap and the first-piece carve-out fall away, and the other two sites that stop at a suffix with the same initial veto — P2's chain and M2's walk, which take a trailing numeral the same way (#424) — would ask the question once. The walk was also computed once rather than per piece after review measured the per-piece rebuild as quadratic (13x growth for 4x the input against the benchmark's 6x bound, on a shape no benchmark unit led with); `bound_given` is that unit now. Readings: `abdul Smith V` → given 'abdul', family 'Smith', suffix 'V', the SUFFIX_OR_NAME report unchanged; `abdul Jr Smith Berg` → given 'abdul', middle 'Jr Smith', where `John Jr Smith Berg` puts it; `abdul Ph. D. Smith Berg` → given 'abdul', middle 'Smith', suffix 'Ph. D.' — the SUFFIX field is 1.4.0's again, the given is not: v1's fix_phd extracted the credential before parsing and the join then took 'Smith', reading given 'abdul Smith' — which the issue reports correctly, and then mislabels its own proposed reading as "1.4.0's reading restored". The decline is chosen over reaching past the credential for the next name word, because P5 joins "the word after it": a join that skipped a suffix would read a shape no one writes as if the suffix were not there, and the marker decline already sets the precedent that the join takes the word after it or nothing. A judgement, recorded as one; `Berg, abdul Jr Smith` → suffix 'Jr'; and `Sir abdul V`, the shape #369's licence had widened #401 onto, reads suffix 'V'. #369's licence-local suffix clause is retired, subsumed by the general decline. `bound_given_join_sees_only_the_surviving_name` re-pins to given 'abd', middle 'Jr Smith', as the #418 entry anticipated. No differential corpus name has either shape — measured: the gate was byte-identical at all three baselines before the rules.md examples went in — so the examples carry the shapes into the rules corpus, where they move against every baseline and are classified there. The 1.4.0 `fix(#401)` rule lists three fields where the 2.x rules list two, because 1.4.0 did not even read the V as a suffix; a draft with two fields handed the name to the fields-only suffix-routing catch-all, whose capped printout hid it, and this entry briefly called #401 a 2.0 regression on that inert measurement — the design-docs review caught it. EXPLAINED NOTHING is not byte-identical. + +- 2026-08-22 #425 — the reserve runs assign's peel over the view the join would leave. #423 had the reserve re-derive assign's trailing peel by hand — the flagged-credential drop, the walk's start, the numeral fork's conditions, the preceding piece remapped to "as the join would leave it" — with each seam held by a pin, and the bare-ambiguous-acronym fork not mirrored at all: `abdul Smith Jr Ma` read given 'abdul Smith', family '', suffix 'Jr, Ma' where `John Smith Jr Ma` keeps family 'Smith', the whole residue of #401's shape on the main walk (every family-losing bound-word shape in the sweep below contains one). The altitude review's framing is taken: the peel is one function, `_peel_trailing` over the list `_peel_walk` builds (the walk's input defined once too, after the /simplify pass found it spelled at three sites), housed in group beside the piece predicates because assign imports those and group cannot import assign; assign calls it and reports what it returns (the numeral fork's piece, the acronym picks, in the order it always reported them — byte-identical over the suite and the three gates); and the reserve builds the VIEW merge() would leave — the pair one piece, its tags the union — and calls the same function. `BoundJoin`'s "the value IS the threshold" trick survives restated as post-join name pieces: STRICT 2, LENIENT 1. The remap and the `k == first_name_k` carve-out are gone, the joined piece being two tokens and never a suffix piece. +- 2026-10-07 #614 — THE RESERVE READS NO VIEW: S2's trailing run is read once, after P3's joins and before the bound-given join is tried, and split off, so the words to spare are the name pieces it left, and the join can move no suffix reading because nothing in group reads the run again (rules/P5 amended; decisions/S2, this date). SUPERSEDES the two-view comparison, which survives only where the run is not read there (a run with a name piece inside it), and with it two pinned readings: `abdul J. V` and `Sir abdul J. V` now join, given 'abdul J.' with family 'V' -- unjoined the initial in front suppresses the numeral fork, and that is the one reading taken now, where the joined view had read the V as a suffix and declined; 1.4.0 read the same, 2.0 and 2.1 suffix 'V', 2.2 and 2.3 middle 'J.' -- and `Sheik abdul Ma` reports the pick the read weighed (`suffix-or-name`), superseding the #425 bullet's application of the #369 precedent to the pair's suffix fork: the read decides 'Ma' before the join, so the fork is a fork. #369's own precedent, a bound word that is also a particle read as the bound word with no PARTICLE_OR_GIVEN report, is untouched (`Sheik Abu Bakar` reports nothing). Accepted by Derek as a class with the rest of #614's moves. + + One principle came out of writing the ledger that the view alone did not state. `abdul Smith Ma` had read given 'abdul Smith', family 'Ma' since 2.0, and the #423 entry called that the lossless shape; the 1.4.0 gate's catch-all count said otherwise — 1.4.0 reads first 'abdul', last 'Smith', suffix 'Ma', as `John Smith Ma` reads, and the fields-only catch-all had absorbed the diff since the example went in (the count is the tell, per the #401 entry). Unjoined, the acronym has words to spare and peels as a credential; joined, the view is two pieces and the fork keeps it as the family — the join was CHANGING what assign reads. So the rule is stated as P5 now has it: the join joins two name words into one and changes nothing else, which the code asks directly: the pieces the peel takes are the same either way (a count proxy, "the view leaves exactly one name word fewer", was the first spelling; the altitude review pointed out it held only because the peel is a tail walk, and the comparison states the rule). That comparison subsumes the unjoined-piece check #423 added for `abdul V` (unjoined the peel takes the V, joined nothing) and restores `abdul Smith Ma` to 1.4.0's reading; `abdul Smith Berg Ma` keeps its join, the acronym peeling either way. Every #423 pin holds. Measured: the 1.4.0 gate is byte-identical at the code change, before the examples went in — the three `Ma`-final examples are v1 parity, the catch-all back at 15 — and `abdul Smith Jr Ma` and `abdul Smith Ma` move against 2.0.0 and 2.1.0, classified there; no hand corpus has the shapes. The population the corpus cannot see, from the design-docs review's sweep of 67,860 constructed main-walk shapes (three bound words — abdul, abd, abu — × twelve words — Smith, Berg, Ed, J., Jr, M.A., Ma, PhD, V, X, 'Ph. D.', 'van der', none a title word — × tails of one to four; no comma shapes; a different twelve-word pool gives different counts, so the pool is the claim): before this change 2,160 lost the family and every one contained a bare ambiguous acronym (864 with further suffixes behind it, so "ending in" was an artefact of #423's shorter sweep); after it none does, and all 67,860 match the family and suffix their ordinary-given twin reads. 5,166 move against master in all — 3,006 where the acronym was the family and is now the suffix (`Sir abdul Smith Ma`), 2,160 where the family is restored. The title-word class below was outside that population and moves after a family comma too (`Berg, abdul Sir`); so was the licence class at the end of this entry. + + Three things the principle does not reach, recorded so nobody applies it there. The post-comma LENIENT path is untouched (AMENDED by #637, below: its decline now reads H5's trailing run where a title word follows the bound word), and NOT for the reason first written here ("its walk has no peel" — it reads a trailing numeral by the lenient last-of-two rule, as the #423 entry says): the family is fixed there and the pair is the given whatever follows, so the principle is not applied, and `Berg, abdul V` keeps given 'abdul V' though the V would be the suffix unjoined — 1.4.0 parity, and exactly the regression #423 measured when a mirror was applied there; P5 now carries it as Accepted. The vocabulary decline is the UNAMBIGUOUS vocabulary: a bare ambiguous acronym the peel does not take is a name word, so `abdul Ma Smith` reads given 'abdul Ma' (1.4.0 read middle 'Ma' — a 2.x difference no corpus witnesses, now an Accepted example). And the principle is about the suffix reading, not the report: `Sheik abdul Ma` reports no SUFFIX_OR_NAME where `Sheik John Ma` does, the #369 precedent — a bound word read as the bound word is not a fork. One more class moves, found by the code review and then measured against 1.4.0: the 2.x reserve had excluded TITLE pieces from its count — an unrecorded deviation from v1, whose join counted suffixes out and nothing else — and the shared peel, following assign, reads a mid-name title word as the name word it is. `Berg, abdul Sir` (given 'abdul', middle 'Sir' since 2.0) and `abdul Smith Sir` (declined since 2.0) read given 'abdul Sir' and given 'abdul Smith', family 'Sir' — 1.4.0's readings, both; `abdul Sir Smith Berg` never stopped reading given 'abdul Sir'. No corpus name has the shapes that move, and no rules example carries one — `abdul Sir Smith Berg`, P5's example of the reading, is byte-identical at every baseline, so the class is pinned in tests alone (the release log says so; the late review of this entry found the sentence here implying otherwise). The same review found the clause had taken `do`, `freiherr` and `st` from P6 — recorded under P6. What the review actually caught was the tag artefact that exclusion had been hiding: the conjunction merge derives a `title` piece tag for a chain such as "Sheikh and Ahmad", merge()'s union handed it to the pair, and assign peeled the pair as a leading title — `abdul Sheikh and Ahmad Bakar Smith` read title 'abdul Sheikh and Ahmad' on 2.0 and 2.1 (v1 read it as the given), and `abdul Sheikh and Ahmad Bakar`, which the exclusion had kept unjoined, would have too once it went. The bound join drops the tag, and the view drops it the same way — the pair is a given name whatever tag the word carried. One more class the comparison moves, found by the code review's classification of every residual master-to-branch diff (12 of 17,640 constructed names — a fourteen-word pool, bare and behind `Sir`, tails of two and three, 6 × (14² + 14³); the review did not record the pool — one shape; 360 of 360 over {Sir, Sheikh, Dame, Sheik, King, Father} × {abdul, abd, abu, umm} × {J., V, X, A., I} × {V, X, I}): behind a given-name title, a numeral after an initial-shaped piece. `Sir abdul J. V` read given 'abdul J.', suffix 'V' since #369 — the fork suppressed unjoined, fired joined, and the licence's threshold of one let the join through — and the comparison declines it: given 'abdul', middle 'J.', family 'V', exactly as `Sir John J. V`. Pinned at both levels; it was the only shape a looser comparison let through under mutation at the test review. #424 — P2's chain and M2's walk taking a trailing numeral — is the remaining asker of the same question and now has `_peel_trailing` to ask it of. +- 2026-10-11 (Derek), #637 and #640 — THE BOUND JOIN DECLINES A TITLE WORD A TRAILING RULE READS AS A TITLE. P5 joins a bound given-name word to the word after it, and declined a maiden marker and a suffix piece; a title word it took, so where a trailing rule would have read that word as a title the joined piece was no title word to read. After a family comma the join reads no peel (the LENIENT reserve, #423), so `Berg, abdul Prof.` read given 'abdul Prof.' where `Berg, John Prof.` reads title 'Prof.' (#637, the gap the 2026-09-09 A1 entry recorded); in a part past the second comma after a suffix comma, where the join runs STRICT, `John Smith, Jr., abdul Dr. Smith` read suffix 'Jr., abdul Dr. Smith' where `John Smith, Jr., Berg Dr. Smith` reads title 'Dr.' (#640). Now the join declines a title word a trailing rule reads as a title: after a family comma, one H5's trailing run takes from the given part (`tail_reading`, asked only when a title word follows the bound word), and in a part past the second comma any title word, which C2 reads as a title. `Berg, abdul Prof.` → title 'Prof.', given 'abdul'; `John Smith, Jr., abdul Dr. Smith` → title 'Dr.'. + DECIDED (Derek): this narrower clause (a′) rather than declining every title word (a). P5 states that a title word standing in the name is a name word and joins like one (H3), and (a) undid it: it failed six tests, a P5 rules example and the two tests named for that clause among them, and over the bound-given grid below moved 4,044 parses, 3,000 of them a join split with no title read (`(Bob), Abu Bishop` → given 'Abu', middle 'Bishop') beside the 1,044 that (a′) moves (six tests measured against 4455962f's suite). (a′) leaves every join on a title word inside the name standing -- `abdul Sir Smith Berg`, `Berg, abdul Prof. Ali`, `Smith, abdul Bishop`. + MEASURED 2026-10-11 against 4455962f (#639's merge), py3.11, `nameparser.__file__` asserted on each side. A bound-given grid -- `{head} {bound} {next} {rest}` for head in (none), `Dr.`, `Sir`, `Smith,`, `van der Berg,`, `,`, `(Bob),`, `Smith, John,`, `John Smith, Jr.,`, `John Smith, PhD,`, `John`, `Mr. and Mrs.`; bound in `abdul`, `Abu`, `abd al`; next in `Dr.`, `Prof.`, `Prof`, `Bishop`, `King`, `King.`, `Sir`, `St`, `Freiherr`, `Salam`, `Rahman`, `Lt.Gov.`, `Mr.`, `van`, `MA`, `J.`; rest in (none), `Smith`, `Ali Smith`, `Dr.`, `Smith Dr.`, `Jr.`, `MA`, `Smith, Jr.`, `Prof. Smith` -- 5,184 texts under the three orders, comparing the seven fields and every report: 1,044 parses move, every one a title word into the title field and no report moved alone -- 660 in a part past the second comma (`John Smith, Jr., Abu Bishop Ali Smith` → title 'Bishop', as C2 reads `John Smith, Jr., Bishop`), the rest after a family comma or an empty head (`(Bob), Abu King.` → title 'King.', given 'Abu'). #623's given-part grid (decisions/H5): 280 -- 260 the same class (`(Bob), abdul Dr.` → title 'Dr.', family 'abdul', H1's move behind a title), and 20 that split the join with no title read (`Smith, abdul Prof. V, Jr.` → given 'abdul', middle 'Prof. V'), the class below. Every corpus name and case text at 4455962f: only `Berg, abdul Prof.`, the row that had pinned the gap. The S2 stress grid, #624's title-particle grid and #625's particle grid: 0. The five gates exit 0, with `fix(#637)` and `fix(#640)` rules in each 2.x ledger; at 1.4.0 the broad `fix(comma-family) lone post-comma piece routes to suffix/title` rule explains both new examples. Frames: the reference rows do not move; a comma-less name with a bound given word pays nothing, the site asked before the title test (`abdul salam ahmed salem` 241), a family-comma one with no title behind it one for the title test (`Berg, abdul salam` 208 → 209), and one with a title word behind it after a family comma about ten to twenty for the H5 read (`Berg, abdul Bishop` 208 → 218, `Berg, abdul Prof.` 207 → 228), rare shapes no guard sees. Recorded negative controls (with the review's rows): with no title declined, 11 tests fail (the four #637/#640 rows on both surfaces, H5's example and two P5 examples); with the past-the-second-comma half alone removed, 5 (its two rows on both surfaces and one P5 example). + THE REVIEW (2026-10-11). The decline after a family comma asks H5's run as the comma-less name reads it (`tail_reading`), and the given part's own walk reads some words behind a title otherwise, the disagreement #623's entry (decisions/H5) records for the chain's bound: there the join is declined and no title is read. A shape-only numeral VI to X, or V before a third comma part, which the walk keeps a name word (`Smith, abdul Prof. VI` → given 'abdul', middle 'Prof. VI', where master and 1.4.0 read given 'abdul Prof.', middle 'VI'); a period-marked title that is also a particle, which P6 then attaches with its report (`Smith, abdul St.` → family 'St. Smith', particle-or-given); and a bound word that is also a particle before one (`Smith, Abu St.` → title 'St.', where `Smith, Zed St.` reads family 'St. Smith'). Every such shape is garbage; the first two read as the same name with an ordinary given word does. Recorded, not narrowed, as #623's disagreement was (Derek, #623), and rules/P5 states it as an Accepted clause; the review's sweep found nothing reading worse than master on a real name and no comma-less name moving. It also found two mechanisms no test held, now rows: the bare-title half past the second comma (`John Smith, Jr., abdul Sir Smith`) and the whole trailing run after a family comma rather than the last word (`Berg, abdul Prof. Dr.`). diff --git a/docs/design/decisions/P6-trailing-orphan-particle.md b/docs/design/decisions/P6-trailing-orphan-particle.md new file mode 100644 index 00000000..0c2f24f2 --- /dev/null +++ b/docs/design/decisions/P6-trailing-orphan-particle.md @@ -0,0 +1,69 @@ +# P6 — the trailing orphan particle + +- 2026-08-23 — `Jong, vd` is C1's now, not P6's: C1's listing-form clause (#296's bundle) reads a post-comma part that is nothing but suffix words as the credential run, and `vd` is suffix vocabulary, so the row reads family 'Jong', suffix 'vd' by statement (it read so before, by the lone-piece deviation). The #380 scope note below calling it deliberately unresolved is superseded for that one row; the P6 question — a tussenvoegsel with no given word left — is unchanged. +- 2026-08-18 #379/#380 IMPLEMENTED — the rule shipped as stated, both `deviates:` markers out. Keyed on the token's VOCABULARY rather than its assigned role, which is what gives the S2 precedence its effect: assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma. The words-to-spare guard is a PIECE test, not a count — every trailing piece that is wholly particles attaches, and the run must leave a GIVEN word ahead of it, so "Nguyen, Van" keeps its only given word. Not "a name word": see the defect below, where that wider reading passed on family text P1 had just produced. MIDDLE was in the test briefly and came out — 0 hits over 740,552 instrumented guard sites, and structurally unreachable, since the only rule that leaves a MIDDLE with no GIVEN ahead of it is gated on state.order, which assign never records on the family-comma path P6 runs on. + Measured: ONE of 751 corpus names moves ("Vega, Juan de la"), at all three baselines, and it needed a new ledger rule in each. The LEDGER rule's regex — not the parse rule, which has no regex and fires on the whole particle vocabulary through the tag — is a PARTIAL copy of PARTICLES, deliberately: the words that actually end a Dutch, Iberian or German listing, not all 70. The ledger guards forced that narrowing: the first draft used the neighbours' bare comma regex, which claimed 215 names and reached two protected exclusions. A member `des` had to come out as well, and for a reason worth keeping straight: it is not in PARTICLES at all, so it could copy nothing, and the text it reached was the opening of "Designated Judge David A. Ezra" rather than a person's name. `_reaches_non_vocabulary("des", PARTICLES)` prints the FRAGMENT it matched, `['Des']`, not the name containing it — which is how the retracted wording arose in the first place. + Two defects found in review, both from the rule's neighbours rather than from its own logic. P1's fold runs EARLIER in the same function and retags all of segment 1 to FAMILY, so a words-to-spare test written as "some name role remains" passed on family text P1 had just produced — and the attachment then hoisted the particle in front of a base it never preceded ("Smith, de Mesnil van" → 'van Smith de Mesnil'). 1.4 gives 'de Mesnil van Smith' for that input and 2.1 gives 'Smith de Mesnil van', so neither the defect nor the fix restores 1.4 — what the fix restores is 2.1, the pre-comma-family reading having moved back in 2.0. The test has to name GIVEN, which is what the rule says. And a post-nominal sits BEHIND the tussenvoegsel in this listing, so the backward walk stopped on it and the attachment never fired: "Berg, Jan van Jr." and "Berg, Jan van, Jr." parsed two ways on nothing but a comma. The walk now steps over a trailing piece holding no name, unless it is itself particle vocabulary — `vd` arrives suffix-roled and IS the run. + Fallout worth its own record: "Vega, Juan de la" was the input tests/test_initials.py used to build an ALL-PARTICLE middle name, and this rule takes that shape away from THAT input. It does not remove the shape — "Vega, Santa de la", "Berg, Van der" and "Nguyen, Van Le" all still produce one in the default order, because the trailing run swallows the given word too and the attachment then declines. The test is repointed at the last of those rather than deleted. Whether such a middle should exist at all is #402 — for a never-given particle it is #365's defect, and for an ambiguous one the parser reports it as a middle name and then declines to initial it, which is a miscount either way. + +- 2026-08-16 (order-precedence keystone; #379, #380, #365) — a particle ending the name has nothing to link forward to, and no particle is a name by itself, so it attaches to the family name standing beside it and renders BEFORE it. The distinction from a chain is what makes this a rule rather than an exception: a chained group has a head word and can be positioned; an orphan has no head, so position has nothing to work with. +- Scope: the COMMA form only, deliberately. "Jong, Anke de" is unambiguous — the comma has already named the family. Without the comma the written shape is not settled: "Jong Anke de" may be a misformatted listing (arguably a missing comma under a declared family-first order) and "Jong de" may be a given name beside a particle. Those keep their positional reading and are not tracked as deviations. +- The words-to-spare guard is load-bearing, not incidental. #379's own subject is "van", which is in the AMBIGUOUS half — so a rule keyed to never-given particles alone would not fix the issue it was filed for, while a rule with no guard breaks Vietnamese + "Nguyen, Van" (given="Van") by eating the only given word. The + guard is S2's shape, reused: consume only when the name has words to spare. +- Rendering before the family has precedent in rules/R1 — folded family words under O3 already "render before the rest of the family wherever they stood in the string". The surnames view renders backwards today ("Vega de la", "Jong de") and flips with this rule. +- Measured under this rule's OWN comma scope, over 782 corpus names (245 of them comma-bearing): exactly one moves, + "Vega, Juan de la" → family="de la Vega". A pre-merge fact-check + corrected an earlier count of three here — "Smith van der" and + "Sander van" have no comma, so the rule as scoped does not reach + them; they were measured before the comma scoping was chosen and carried forward unfiltered. +- What that number means is the opposite of reassuring. The corpus holds 245 comma names, TWO of which end in a particle and one of which clears the words-to-spare guard ("Nguyen, Van" is the other, and the guard is what keeps it a given name), so it is very nearly blind to the shape this rule governs — Dutch and Flemish listings are barely represented. Treat the one as evidence about the corpus, not about the blast radius, and run tools/differential at all three baselines in the implementing PR regardless of how small this looks. +- 2026-08-16 (pre-merge coherence pass) — P6 and S2 CONTEST + "Berg, Jan vd": today S2 wins and reports suffix="vd", while + P6's marker asserts family="vd Berg". P6 wins, and the rule says so in its statement rather than leaving the pair to file order. The reason is C-ii's, not a new judgement: vd as the British Volunteer Decoration is rarer than vd as van der, and a trailing abbreviation AFTER A FAMILY COMMA is the tussenvoegsel position specifically. Scope check on the precedence, so it cannot creep: it reaches only words that are both particle and suffix vocabulary, in the trailing-orphan position, under a family comma, with a given word to spare. "John Smith, PhD" and "Smith, Jr." are untouched (not particles), and "Jong, vd" is untouched (no given word remains), which is why that row stays out of scope rather than becoming a counter-example. Recorded because the pair was declared in `interacts:` and left unresolved — `interacts:` is advisory by design and pins nothing, so a declared interaction is a prompt to state the outcome, not a substitute for stating it. +- 2026-08-16 — P6 is the first rule in rules.md that nothing implements. Legitimate per the preamble (the document is normative, and a gap is a tracked deviation), but it left the rule pointing at nothing, so the shape got a pointer rather than an exception: `tracked: #379, #380` in place of `implemented:`, with exactly one of the two required of every rule. An unimplemented rule can no longer sit here untracked, and a shipped rule cannot keep a stale tracking pointer after its issues close. Mutation-tested three ways before being believed (drop the pointer, carry both, malformed refs); each fails. +- The RATIONALE first shipped here was wrong and is corrected: it read "no particle is a name by itself", which is true only of the never-given half — decisions/vocabulary-collisions says the opposite in as many words ("most particles are short words that double as names"). The error mattered rather than merely reading badly: the words-to-spare guard exists BECAUSE the rule reaches ambiguous particles (#379's own subject is "van"), so the rationale undercut its own guard. Caught in review, after a coherence pass that interrogated rule-vs-rule pairs and never checked rule-vs-decision-record — which is the gap to close next time, not a one-off. + +Decided 2026-08-19 (was Open: [#380](https://github.com/derek73/python-nameparser/issues/380), "Berg, Jan vd" under this rule): the attachment reads `vd` as *van der* and joins it to the family the comma named (family "vd Berg"); the vd reading itself is decisions/vocabulary-collisions (C-ii). The no-given-word case "Jong, vd" is still deliberately unresolved — see the scope note. + +- 2026-08-22 (late review of #425's docs, PR #426) — the bound join runs before P6 sees the name, and P6 now says so. P5's new title-word clause took `do`, `freiherr` and `st` — title AND particle vocabulary, until #296's audit took `do` out of TITLES, since when the join takes `do` as the particle it is — away from the attachment: `Berg, abdul do` had read family 'do Berg' since #379 and reads given 'abdul do' now. That is 1.4.0's reading, and it is what every other particle already did: `Berg, abdul van` read given 'abdul van' at 1.4.0, on 2.0 and 2.1, and on master before #426 — the old reserve's title exclusion had held only the title-word trio back. The precedence is the stage order (P5 joins in group; P6 attaches in post_rules, over what is left -- in assign since #613, after the given part is read, still over what P5 left) rather than a judgement about tussenvoegsels behind bound words, and both rules carry it as Accepted with `Berg, abdul van` as the example, parity at every baseline. The precedence has one exception, found by the review of #424's docs when the Accepted clause claimed the attachment "never sees" the particle: a trailing particle of the unambiguous suffix vocabulary too — `vd`, `mc`; `do` is ambiguous, and a name word to the join — is a suffix piece to the join, which never takes one (#421), so the attachment does see it, and `Berg, abdul vd` reads family 'vd Berg' where 1.4.0 read given 'abdul vd'. Both rules say so, with the example; at 1.4.0 the fix(comma-precomma-family) rule already reaches it (a rule written for it was shadowed, and dropped). + +- 2026-08-29 #405 (the A1 deviation, reversed) — the attachment reports the fork it decides, and the Accepted clause saying it does not is deleted rather than re-tracked. What that clause rested on was assign's own scoping comment: after a comma that fixed the family, a leading given-position particle is "not meaningfully ambiguous". True of the FAMILY — the comma names which piece that is — and not of the particle standing behind the given name, which P6 decides on its own a stage later, out of the same vocabulary that makes `Van Johnson` report `particle-or-given` with no comma in sight. The old state was the shape A1 forbids: `Van Johnson` reported the fork, `Nguyen, Thi Van` decided the identical fork silently and lost a given name doing it. + TWO kinds, keyed on WHAT THE ATTACHMENT OVERRODE and not on what the words are, because `mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE` asks the emitted fork to be the one taken. Where assign had read the run as a post-nominal, the overridden reading is the post-nominal — `suffix-or-name`, which is exactly the S2 precedence this rule already claims in writing. Otherwise, where the run holds an ambiguous particle (`van`, `Van`, `la`), the overridden reading is that word as a name word — `particle-or-given`. One kind for both would assert "particle or given" about a word the parse had read as a credential and never weighed as a name, naming a branch nobody considered. + The key is the READING and not the vocabulary, and the docs review caught a first draft that said otherwise ("a word that is also suffix vocabulary declines a post-nominal"). That wording is wrong for both of the words it most obviously describes. `do` is the third member of the particle ∩ suffix-acronym overlap and reports `particle-or-given`: it sits in the AMBIGUOUS acronym half, which had already left it a name word, so no credential reading was ever taken for the attachment to override. And a run of several words is read whole — `Berg, Jan de vd` reports NOTHING though `vd` is suffix vocabulary, because behind `de` the run was read as name words, where the lone `Berg, Jan vd` reports `suffix-or-name`. Both are pinned in tests/v2/cases.py; neither is in any corpus, so nothing else would have caught the wrong statement. + The two arms are ORDERED rather than asserted disjoint, and that is a correction to the plan, which recorded them as disjoint by construction ("a suffix-vocabulary particle arrives suffix-roled and IS the whole run"). That premise is a property of the SHIPPED vocabulary, not of the rule: `vd` and `mc` are the only words in both the particle and the unambiguous suffix sets and neither is ambiguous particle vocabulary, so no shipped input reaches both arms — measured, 0 both-arm runs over 29,400 generated names (every ordered pair of the 70-word particle vocabulary in six family-comma shapes: bare, behind a given word, behind a title, before `Jr.`, before `MD`, and with the pair split across the comma). But a caller's `Lexicon` may put one word in both, and the `assert` the plan asked for DID fire on `Berg, Jan zz` under a Lexicon adding `zz` to `particles_ambiguous` and `suffix_acronyms`. `rules/A1` says parsing never fails on any input and names exactly one exception, a caller's segmenter; an `AssertionError` out of a stage would be a second. So the code decides instead: the suffix arm wins, because assign had read the word as a post-nominal and the name-word reading was never on the table for the attachment to decline. Pinned in tests/v2/pipeline/test_post_rules.py. + The never-given half stays SILENT, and that is the substantive half of the decision. `Jong, Piet de` has no live alternative: `de` is not a name in any reading and is not suffix vocabulary either, so the attachment chose between one reading and nothing, which is not a fork. Reporting it would be "the code got here" keying, the failure this project already has on record (a fork reported for every ambiguous particle at once on `Dr. Van Jr.` — the count is left out on purpose, since decisions/vocabulary-collisions says to recompute before quoting it). + Measured over the 1094-name differential corpus: P6's attachment fires on 9 names, of which 5 take the ambiguous arm (`Beethoven, Ludwig van`, `Berg, Jan van der`, `Nguyen, Thi Van`, `Vega, Juan de la`, `der, y van`), 2 the suffix arm (`Berg, Jan vd`, `Berg, abdul vd`) and 2 stay silent (`Jong, Anke de`, `Jong, Piet de`). Seven names gain an ambiguity kind and no role field moves that was not already moving. Read that seven as evidence about the corpus rather than about the blast radius, the caution the 2026-08-16 entry above already gives for this rule: 304 of the 1094 names carry a comma and only 13 end in a particle, 11 of those being rows the #379/#380 arc added itself (this change added four corpus names and not one of them carries a comma, so the 11 is unchanged by it). The reach is a SHAPE — every family-comma listing ending in an ambiguous particle, which is to say ordinary Dutch, Flemish and unaccented Vietnamese listings — and a caller holding such records sees a report on all of them, not on seven. The LEDGER shape that follows from that is not the one the plan predicted, and the difference is worth recording because it will recur for any report added to a rule whose fields already move. A new `fields = ["_ambiguities"]` rule explains none of the seven: `classify` matches on a rule's `fields` being a SUPERSET of the whole diff, and against 2.0.0 and 2.1.0 these names diff in `middle`/`family` (or `family`/`suffix`) from #379 and #380 as well. So the repair is to WIDEN those two existing rules to `{middle, family, _ambiguities}` and `{family, suffix, _ambiguities}` in both ledgers, not to add a rule. Each widened rule still passes the #452 equality check, since the union over the names it explains is exactly what it declares — the two silent `Jong` names contribute `{middle, family}` and the five ambiguous ones contribute `_ambiguities` on top. 1.4.0 needs nothing at all, having no ambiguity surface to diff. Gate before and after: byte-identical output at all three baselines apart from the provenance header — 1094 names, 229/194/102 intentional, 0 unexplained, and no per-heading count moved. The seven names were already in the diff; only their field sets grew. + The most visible consequence, and it is not the Vietnamese one: `Beethoven, Ludwig van` — the textbook-correct Dutch listing, read exactly right — now carries a report. That is honest rather than regrettable. The parser cannot separate it from `Nguyen, Thi Van`; the two are the same string shape over the same vocabulary, and a report on one is a report on the other. A caller who wants only the doubtful cases has nothing here to filter on, because there is nothing to filter on in the input either. + +- 2026-08-30 #467 (the no-comma site, and the given position given back) — a declared family-first order names the family as a comma does, so P6's attachment fires without a comma too. It is keyed on TWO things, and review showed the second is not implied by the first: the run must stand in a MIDDLE — the one position that means nothing for a particle, middles being further given names — and it must END the name. Together they name the order without asking it: the default order ends with the FAMILY and FAMILY_FIRST_GIVEN_LAST with the GIVEN name, so only FAMILY_FIRST can put a name's last word in a middle. This DECIDES the parenthesis the 2026-08-16 Scope bullet left open ("arguably a missing comma under a declared family-first order") and narrows that bullet to the default order. + P1 LOSES ITS GIVEN-POSITION SITE in the same change, and that half is a correction rather than a narrowing. That slot holds what the caller DECLARED to be the given name; the never-given vocabulary supplies a reading where position leaves the question open and vetoes none that position has already given. `Ménil de` under either family-first order now reports given `de`. + THE PREMISE THIS REPLACES WAS FALSE, and it had been stated in three places (rules/P6's rationale, `_lexicon.py`, `config/particles.py`): that a never-given particle "cannot be a name at all". Measured, all 33 members report given == themselves when parsed bare. This is the SECOND correction of the same over-claim in this entry — the 2026-08-16 note above records dropping "no particle is a name by itself" for the identical reason. The conclusions drawn from it were mostly right; the reason was not, and it is load-bearing, because stating it as a property of the VOCABULARY invites the same inference for a particle ending the name, where there is nothing to join forward to and the writing is all there is. + NO VOCABULARY TEST AT THE NEW SITE, and that is the substance of the design. The comma path needs one because a comma cannot separate the Dutch reading from the Vietnamese; the declared order can, and does both at once — `Beethoven Ludwig van` under FAMILY_FIRST gives family 'van Beethoven' though `van` is ambiguous vocabulary, and `Nguyen Thi Van` under FAMILY_FIRST_GIVEN_LAST keeps given 'Van' though it is the same word in the same set. A never-given test here would have excluded 37 ambiguous particles — von, di, da, del, le, and `van` itself, the flagship word of the listing the rule is named for. An earlier attempt (#466) carried exactly that test and failed on the Dutch case because of it. + NO RE-LAYOUT EITHER, and this is a property of ONE order rather than a simplification: under FAMILY_FIRST the roles run family, given, middle, middle…, so dropping a trailing MIDDLE leaves every other piece's role untouched. It is false of FAMILY_FIRST_GIVEN_LAST (family, middle…, given) and of the default order, which is the second thing the ends-the-name test secures. #466 removed a piece and re-laid the leftover out, which lost a given name outright on `van Berg Jan de` and promoted a post-nominal into the given slot on `Berg Jan Jr. de`. Both defects are structurally unreachable here. The one guard that remains is the base: attaching to an all-particle family renders one particle in front of another, and R2 reads those words as ordinary name words, which no rule reorders. + THE INVARIANT THIS RESTS ON, and the reason to trust it over the argument it replaced: for a name of the form `Family Given [Particle]`, the family-comma writing under the default order and the comma-less writing under FAMILY_FIRST parse identically. Measured over ALL 70 particles × 3 families × 3 givens = 630 names: 0 of 630 agreed before this change, **603 of 630** after. The invariant draws the never-given/ambiguous line by itself instead of having it asserted, which is why no vocabulary test appears at the site. + The 27 failures are the three words that are BOTH particle and suffix vocabulary — `vd`, `mc`, `do` — and they are a real limit, not noise: P6's precedence over S2 is stated inside the comma paragraph and justified by the comma ("a trailing abbreviation after a family comma is the tussenvoegsel far more often"), so the comma form gives family 'vd Berg' where the comma-less family-first form gives suffix 'vd'. The rule now says so where it states the correspondence. + A FIRST DRAFT OF THIS ENTRY CLAIMED 216 of 216, and the number came from a script that sliced the particle vocabulary — `sorted(particles)[:14]` — which excluded exactly those three words. That is AGENTS.md's axis 2 ("your detector is a second, unreviewed implementation") in its purest form: the detector agreed with the design because it inherited the design's blind spot. Recompute over the WHOLE vocabulary, never a sample. + Scope the invariant to that SHAPE as well: 219 of the 304 comma-bearing corpus names parse differently with the comma simply removed, because a comma also delimits titles and suffixes. (An earlier draft said "181 of 247", which reproduces under no definition of "comma name" — the 304 matches the count the 2026-08-29 entry above already records.) + An asymmetry falls out of it rather than being carved: shape 2 places the given name BEFORE the middles, so it corresponds to FAMILY_FIRST only. There is no comma format that puts the given name last, so FAMILY_FIRST_GIVEN_LAST has no twin to inherit an equivalence from. That is why the trailing particle is one order's business (#469). + ONE EXAMPLE LINE WAS DELETED RATHER THAN UPDATED: `"Nguyen Thi Van" family-first → middle="Van"`. The name is written in FAMILY_FIRST_GIVEN_LAST; its FAMILY_FIRST reading is wrong by construction, so the line pinned a value nobody wants and nothing should depend on. Recorded here because the general question — whether a rules.md example should declare its format, and whether the doc test should refuse to run one outside it — is #470 and is NOT decided by this change. + REACH: 30 of 6606 parses move — this tree's 1101-name corpus × three `name_order` values × `middle_as_family` off and on — over twelve names, NONE under the default order. Recompute by parsing the `tools/differential/corpus*.jsonl` files of this tree, deduped, under each of the six policies, and comparing the seven role fields against a checkout of the parent commit reading the SAME corpus files; the comparator is the pre-change code, never a construction of this rule. Dated snapshot, measured 2026-08-30. Note the corpus grew from master's 1094 by this change's own example lines, which is why the count must name the tree it was taken on. + The differential gate cannot see any of it: `compare.py` parses every name as `HumanName(name)` and sweeps no policy, so no non-default `name_order` behavior has ever been compared across versions. Output is byte-identical at all three baselines apart from the added corpus names, none of which diffs. + WHAT REVIEW FOUND, because the corpus could not. Four agents on PR #472 found the site firing outside the order it was reasoned from, in two directions, and the differential corpus reported neither: 52 default-order names (a conjunction stops a particle's forward chain and leaves it standing in a middle — `Maria Luisa y de la Cruz` gave family 'de la Cruz' where master gives 'la Cruz') and 366 FAMILY_FIRST_GIVEN_LAST folds of a particle that does not end the name (`de Anke van y` gave family 'van de Anke' with the given name still behind it). Of 1099 corpus names only 6 hold a conjunction-then-particle adjacency and none in this shape, so the REACH recipe honestly reported 0 default-order movers throughout. The count was right and the population was blind; "none under the default order" was a property of the corpus, not of the code. Both are now `· boundary` examples in rules.md so the doc runner holds them. + The repair is one clause — the run must END the name — and it subsumes the order test a first repair added: measured over 542,592 generated parses, an explicit `state.order[0] is Role.FAMILY` never decides anything the ends-the-name test has not already decided, so it is not in the code. What IS kept is `state.order is not None`, which is what keeps this site off the family-comma path (0 co-firings with P6's comma site over ~892,000 parses). + A stale index list was repaired with it: O1 and O2 retag between roles without recomputing, and this site was the first consumer of `families` after them (33,840 stale observations instrumented). No shipped input reaches a wrong answer through it — no `_TURKIC`/`_EAST_SLAVIC`-matching word is particle vocabulary — so the fix is unwitnessed by any test and is recorded here instead. +- 2026-09-18 #531 (cross-amendment) — THE PRECEDENCE OVER S2 GAINS ONE EXCEPTION, AND THE RUN WALK GAINS ONE STAND-DOWN CONDITION. An all-caps member of the AMBIGUOUS credential class, in a name written in more than one case, reads as the credential and this attachment stands down: `Doe, John DO` is suffix `DO` where `Doe, John Do` and `SMITH, JOHN DO` still attach. The argument, the accepted cost (`SMITH, JOHN DO` paired with `NASCIMENTO, EDSON ARANTES DO`) and the two carve-outs are in decisions/S2's 2026-09-18 `do` bullet; what belongs here is the CONDITION, which is narrowed to the ambiguous tag rather than to the suffix role: a suffix-roled single-token piece carrying that tag is not part of the run, while `vd` and `mc` — unambiguous suffix vocabulary, also particles, also suffix-roled — stay inside it and are unaffected. Keying on the role instead would have stood the attachment down for them too, and keying on nothing would have let it silently override the slot's reading (`Doe, John DO` read family `DO Doe` with the condition absent, verified). The 2026-08-18 entry's "assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma" is unchanged for `vd` and `mc` and now has this one exception for `do`. + +- 2026-10-02 (Derek), #573 — A WORD OF BOTH THE PARTICLE AND THE UNAMBIGUOUS SUFFIX VOCABULARY READS ALIKE IN EVERY CASE AT THE THREE SITES WHERE THE UNIFORM CASE SPLIT IT. Resolves the shapes the C1 section's 2026-10-01 #554 entry left open. `vd` and `mc` split in names written wholly in one case, in three shapes, each split one stage acting on a reading the next never took. The mixed-case spellings were not all right either: `Smith vd Ma, John` read family 'Smith Ma', suffix 'vd' in 2.2.0 and 2.3.0, #530 having fixed it only in this cycle, and the fixes below move mixed-case names as well as uniform ones (blast radius, below). + BEFORE A FAMILY COMMA (P2). `SMITH VD MA, JOHN` read family 'SMITH MA', suffix 'VD' (2.2.0, 2.3.0 and this cycle's tree; 2.0.0 and 2.1.0 read family 'SMITH VD MA'). In one case the S2 trailing reading takes `VD MA`, so the particle chain stops before `VD`; the part before a family comma then takes strict suffix words one piece at a time, and the lone `VD` was peeled from between two family words. DECIDED in that per-piece read: a particle that is suffix vocabulary too, with a name piece behind it in the part, heads that name and stays family (P2's join), so `VD MA` reads family whatever the case. Not decided in the chain: a comma followed by no name word reads that same part positionally, trailing credential included (`Berg de MA, Prof.` keeps suffix 'MA' and its report), and the chain runs before that choice is made. A first draft skipped the chain's stop for the whole family part and moved exactly that shape; a widened grid caught it. Declined: reading `VD MA` as credentials, family 'SMITH', which the comma's naming of the family argues against. `SMITH VD, JOHN` and `SMITH VD JR, JOHN` keep the suffix: nothing, or a suffix word, stands behind the particle. + BETWEEN CREDENTIALS AFTER A FAMILY COMMA (P6). `DOE, JANE PHD VD MA` read family 'VD DOE', suffix 'PHD, MA': assign had read `PHD VD MA` whole, and P6's attachment, which outranks the suffix reading for such a word by design, reached into the run and pulled `VD` out. DECIDED: P6 gains a third exception — a lone such word with a post-nominal read in front of it and another behind, a title between it and the one in front transparent (H5), keeps the post-nominal reading in any case, reported as S2's credential fork. The test is the roles assign gave, asking whether it read the run whole, so a member read as the credential in front speaks as a degree does (`DOE, JANE MA VD PHD` stays whole); a first draft asked S2's company query instead, which counts only an unambiguous credential in front, and missed that shape and the titled one (`DOE, JANE PHD PROF. VD MA`). A generation counts on either side, and that is a decision rather than a side effect: P6's tussenvoegsel stands right behind the given name, so any post-nominal between them says the word is not one, and `Berg, Jan PhD vd Jr.` reads suffix 'PhD vd Jr.' where 2.2.0 and 2.3.0 read family 'vd Berg', suffix 'PhD, Jr.' — the shape `Berg, Jan van Jr.` P6's rationale names has no post-nominal in front and still attaches. No title is skipped BEHIND the word: a title there leaves it a name in assign (`DOE, JANE PHD VD PROF. MA` reads middle 'VD PROF.', as before), so a skip there was dead code, measured never to fire over the grid below, and was removed. Boundaries: with nothing behind it the word ends the name and attaches (`Doe, Jane PhD vd` → family 'vd Doe', 2.2.0's and 2.3.0's reading); two such words side by side are one particle run, which P6 takes whole as its first exception says (`DOE, JANE PHD VD MC MA` → family 'VD MC DOE', unchanged); a plain particle between credentials was never read as a post-nominal and attaches as before (`Doe, Jane PhD de PhD` → family 'de Doe'). Declined: middle 'VD MA', 2.0.0's and 2.1.0's reading — under #554 run membership asks what the word IS. + ONE NEVER-GIVEN PARTICLE OPENING THE GIVEN PART (S2, P1). `SMITH, VD MA` read family 'SMITH VD', suffix 'MA'. S2's given-part slot reads a member ending the given part as the credential because "the words to spare are there by construction", the first word after the comma being the given name; a never-given particle is not one — P1's fold takes it forward into the family — so `MA` was read as a credential with no given name left, and the fold took `VD` alone. DECIDED: where the given part opens with a never-given particle and nothing but particles stands before the member, chained (`VD DE LA`) or not, the member is the word the particle attaches to and reads as a name: family 'SMITH VD MA', the reading of 2.0.0 through 2.3.0 and of `Smith, vd Ma`. The particle test is P1's own predicate (`_pieces.is_lone_never_given_particle`), so the slot cannot drift from the fold it predicts. Plain never-given particles share the shape (`smith, de ma`). Boundaries: capitals in a mixed-case name still make the member the credential, as they do with no words to spare anywhere (`Smith, de MA` keeps suffix 'MA'); an ambiguous particle can be a given name (`SMITH, VAN MA` keeps given 'VAN', suffix 'MA'); and a name word between the particle and the member is the word it attaches to, leaving the member a word to spare (`SMITH, VD BERG MA` keeps suffix 'MA'). Declined: family 'SMITH', suffix 'VD MA' (as `SMITH, PHD MA` reads) — under #554 `VD` speaks for nothing, so only the missing given name could have made `MA` a credential, and it is the particle's surname. + NOT DECIDED HERE: the mixed-case `Doe, Jane PhD vd Ma` (middle 'vd Ma', silent), the issue's fourth bullet. Its reading is unchanged; whether the family-comma path should report the chain's fork there is a separate question. Open: #578 (decided the same day, the S2 section's #578 entry: silent, as it reads). Nor the positional read of a part before a comma followed by no name word, where the case still splits the word: `Smith vd Ma, Prof.` reads given 'Smith', family 'vd Ma', and `SMITH VD MA, PROF.` family 'SMITH', suffix 'VD MA', on this tree and on `e56dd296` alike — that part reads as the comma-less name does, and the comma-less name splits the same way. + BLAST RADIUS, measured 2026-10-02 with `e56dd296` (the pre-change master) as the comparator. The gate's output is byte-identical at all five baselines apart from the tree path: no corpus name holds these shapes, the rules.md examples aside. Over a generated grid, 5,676 of 409,498 parses move, every one in one of two classes — 3,534 where a family gains the words behind its particle, 2,142 where `vd` or `mc` leaves the family for the credential run — and no list of ambiguity kinds changes; 1,310 of the movers are written in mixed case. Recompute: from the pool `vd mc de van do Ma MA PhD Jr John Berg V la Prof.`, build every `<1 or 2 words>, `, `<3 words>, <1 word>`, `Smith , John`, `Doe, Jane <2-4 words>` and `Smith, <2-4 words>`; deduplicate, then parse each as written, `.upper()` and `.lower()`, keyed by the resulting text; keep the entries whose `as_dict()` or sorted ambiguity kinds differ between the trees, and class each by whether the new family holds every old family word and more, or the old family lost only `vd`/`mc` while the suffix kept every word. Cost: the given-slot test was QUADRATIC as first written, rescanning the particle run once per member — the second review measured 29,670 → 165,591 frames on `SMITH, VD ` + `DE `×k + `MA `×k at k = 256 against `e56dd296`, where an earlier wall-clock check at k = 100 and 400 had read 3.18× and passed for linear. The run's end is now kept once per title set: 1.90–1.99× per doubling of k from 16 to 256, and `_REREAD_SHAPES["lead"]` in tests/v2/test_benchmark.py guards it (generator frames at 8 units against 32: 3.38× kept, 7.53× rescanning, bound 6.0). The frame-count reference band did not move. +- 2026-10-06 #613 — THE COMMA SITE IS DECIDED IN THE PASS THAT READS THE GIVEN PART. #611 found P6 showing the decisive sign smaller: assign held back from #602's run exactly the particle tail post_rules would attach a stage later, by asking `particle_tail` over roles that meant "not yet read" there and "suffix" in post_rules. The attachment and its third exception (`_inside_a_credential_run`) moved into assign, right after the given-part walk, and `particle_tail` is asked ONCE, at the moment the walk reaches #602's run: the pieces in front of the run hold the walk's roles and the run's words none, the state in which its answer equals the one over the finished roles (argued piece by piece in its docstring). post_rules ran P1's leading fold before P6, so assign asks P1's condition itself (`is_lone_never_given_particle` on the given piece), which is what keeps `Smith, de Mesnil van` the fold's. H1 and M4, the other rules ahead of the old site, are gated on there being no family, which a family comma always has. The no-comma site (#467) stays in post_rules, since it runs over the order P1's family-first redistribution leaves. NOTHING MOVES, measured 2026-10-06 against the commit before it: the P6 grid (103,786 distinct texts: each of the heads `Berg, Jan`, `Berg, Jan Piet`, `BERG, JAN`, `Berg, Dr. Jan`, `Berg, de`, `Berg, van`, `Berg, abdul`, `Smith, de Mesnil`, `de la, Jan` and `Nguyen, Thi`, followed by every sequence of zero to three of `van de der vd VD mc do DO Do PhD Jr. MA Ma Prof. V la Piet`, with and without a trailing `, Jr.` -- under the default policy, `lenient_comma_suffixes=False`, a family-first order and `middle_as_family`; comparator every token's role and folded or unjoined tag, every ambiguity's kind, detail and token IN ORDER, and `initials()`) and the #613 harness's 80,423 texts are identical, the gate's output byte-identical at all five baselines but for the baseline path, and the suite unchanged. ONE SHAPE DID MOVE, and the PR review found it on a grid with an EMPTY part before the comma, a head no measurement above held: there no family is named, H1 and M4 are not gated off, and they had made the given word the family before the old site attached -- `, Mr. Jones vd` read title 'Mr.', family 'Jones', suffix 'vd', and assign attaching first read given 'Jones', family 'vd' (177 texts on that grid). Reading P6 strictly instead (no family named, no attachment) moved 1,302 texts against master, `, Jan van` among them, so the attachment behind an empty family part stays in post_rules, after H1 and M4, through the same helper; that grid now differs from b547c247 only in C1's decided classes. Frames drop on every family-comma name, post_rules' block having paid a second `particle_tail` and a generator-driven search for the given word on every one, tail or not, where assign now attaches only behind a non-empty tail: `Smith, John` 169 → 160, `Smith, John, Jr.` 213 → 204, `Doe, Jane nee Smith` 302 → 293, a no-comma name unchanged. +- 2026-10-17 #700 — THE ELIDED PARTICLE AND THE HYPHEN-ATTACHED ARABIC ARTICLE ARE PARTICLE VOCABULARY, NEVER-GIVEN, AND THE RULE IS UNCHANGED. Library catalogs file `Paul d'Ivoi` as `Ivoi, Paul d'` and `Ḥusnī az-Zaʿīm` as `Zaʿīm, Ḥusnī az-`, the particle split off with its mark still on it; both read as a middle name (1.4.0 and 2.4 alike) because neither written form was in `PARTICLES`. P6 asks only the particle tag, so the fix is the vocabulary alone: no code moved. Three decisions, each taken by Derek on the #700 session's options. ELIDED SET: `d'`, `l'`, `dell'`, each also in its U+2019 spelling, and `de’` and `’t` beside the existing `de'` and `'t` (`’t` added in the PR review, a gap the test review found), since nothing folds the typographic apostrophe to the ASCII one; particles.py asserts at import that every elided entry has both spellings. ARTICLES: a CLOSED list — `al-`, `ul-`, the article assimilated to each of the fourteen sun letters in ASCII and DMG spellings (`ar-`, `az-`, `aš-`, `ash-`, `aṭ-`, …; lām's being `al-` itself), and the Egyptian `el-` with its assimilations in ASCII spellings only (`es-`, `esh-`, `ed-`, …; no `eš-` or `eṣ-`) — all never-given, where the bare `al` stays ambiguous (Al Gore): written apart with its hyphen, the word is the article and nothing else. RENDERING: the attached family joins with a space like every particle, `d' Ivoi` and `al- Rashid`. + Declined: a shape rule (a vowel, consonants, a hyphen) in place of the list. It would need a new predicate and the bare-word collision check the issue named, for a population the list already covers; the corpora attest seven article spellings (`al- az- aš- ar- es- as- aṭ-`), all in it — recompute: every whitespace- or comma-bounded token of `tools/differential/corpus*.jsonl` matching `\w{1,4}-` at a name's end. + Not decided here: the GLUED forms (`Paul d'Ivoi`, `Harun al-Rashid`) stay one word with no particle split, out of #700's scope by its own statement. Hyphen-minus only: U+2010 and the non-breaking hyphen are not listed. The Scottish Gaelic `a’` (`Mac a’ Ghobhainn`, in corpus_cldr.jsonl) is mid-name and not an index-form particle, and was not added. Nor are the elisions no corpus holds (`dall'`, `nell'`, `degl'`): the list is closed, so such a spelling reads as a middle name until it is listed. + RENDERING FOLLOWED THE VOCABULARY, found in review: R4's `capitalized()` split a token at its mark and asked the particle test of the stem, so `IVOI, PAUL D'` repaired to `Paul D' Ivoi` — 30 of the 32 first-listed spellings, all but `al-` and `de’`, whose stems are particles themselves. The whole token is now asked the particle arm's question first, and lowercased only where that arm would lowercase it; wherever the arm declines (an all-particle part, the given or suffix role, a recorded casing) the token goes to the per-word walk as before — a first version sent every whole-token particle down the arm's other clauses instead, which moved `smith, john, ed-` from 'ED-' to 'Ed-' and a configured `van-der` from 'Van-Der' to 'Van-der' (the fix commit's own review). The one shipped word outside #700 it reaches is `'t`, whose stem is `t`: `van 't Hoff` repairs to 'van 't Hoff', where 1.4.0 through 2.4.2 wrote 'van 'T Hoff' — R4's reading, and the right Dutch one. A casing configured for a stem (`capitalization_exceptions` `al` → `AL`) no longer reaches the marked spelling, which is now a particle in its own right: `al-` is lowercased. The snapshot cannot see a render view (decisions/R4); measured instead by `capitalized()` and `capitalized(force=True)` against `00dd2759`, over every name in `de6c045d`'s `tools/differential/corpus*.jsonl` as written, lowercased and uppercased (28 names move, every move lowercasing one token ending in `'` or `-`), and over every particle in eight positions under all three orders (adding `'T` → `'t`, which no corpus name held until R4's `van 't hoff` example joined corpus_rules.jsonl). The boundary `Ivoi, d'` was taken out of rules/P6: P6 does not attach there, and the reading it gets instead (family `Ivoi d'`, as `Jong, de` has read since 2.0) is the never-given vocabulary's family-comma reading, which P6's statement does not describe — a P6 example would have pinned another rule's output. It moves from given `d'` (1.4.0 through 2.4.2, the word being no particle there), and so does every new entry standing alone after a comma (`Smith, al-`); it stays pinned as a case row. + Measured 2026-10-17 against the parent commit `b4a004c3`: 39 corpus names move, every one the same shape — a trailing `d'` or article leaves the middle name and joins the family (38 in corpus_gnd.jsonl, `Orval, Pierre d'` in corpus_conventions.jsonl), and no ambiguity kind changes. Recompute: `uv run python tools/differential/snapshot.py --since b4a004c3`. That is evidence about the corpus as much as the change: these are the comma-bearing names ending in one of the new forms, and GND's index forms are where they occur. + diff --git a/docs/design/decisions/P7-one-letter-particle-against-the-initial-it-spells.md b/docs/design/decisions/P7-one-letter-particle-against-the-initial-it-spells.md new file mode 100644 index 00000000..c6e3c762 --- /dev/null +++ b/docs/design/decisions/P7-one-letter-particle-against-the-initial-it-spells.md @@ -0,0 +1,11 @@ +# P7 — a one-letter particle against the initial it spells + +- 2026-10-04 #604 — the PERIOD decides, stated as a shape rather than as a property of one word. `ó` is the first one-letter particle the vocabulary has held, and the particle match folds edge periods away, so `ó` in PARTICLES alone also claimed `Ó.` — the initial of Óscar, Ólafur, Ólöf. Measured on the branch before the veto: `Juan Ó. Pérez` and `J. Ó. Pérez` read family `Ó. Pérez`, `Pérez, Juan Ó.` read family `Ó. Pérez` too (P6 attaching the trailing "particle" behind the comma), and both the core's and the facade's `initials()` dropped the Ó. With the veto every one reads middle `Ó.` (given `Ó.` for `Ó. Pérez`) and initials keep it. A one-letter word written with its period is an initial, whatever particle the letter spells; The same line already stands for a single-letter connective, pinned in the code: 'e.' in 'john e. smith' is tagged an initial, not the Spanish conjunction (classify's initial veto on the conjunction tag). rules/P3 does not state it — P3 decides that fork by case, not by the period — and the only rules.md statement of the period line is R4's hyphen clause, scoped to a hyphenated word in case repair. +- WHERE the veto lives, and why it is one helper asked at four sites: `_lexicon._spells_an_initial`. Besides classify's tags, the particle vocabulary is read directly by `_vocab.surname_unit_tags` (the comma unit count, #575), `_render._cap_word`'s case repair, and the facade's `_is_particle` (the v1 last-name split and initials), and a site that kept bare membership read `Ó.` as a particle where classify does not: `Ó. Pérez, Ed` became family `Ó. Pérez`, given `Ed`, where `Q. Pérez, Ed` reads given `Q.`, family `Pérez`, suffix `Ed`. _render and the facade may not import the pipeline (tests/v2/test_layering.py), so the helper sits in the one module all of them already import. Every site asks it AFTER its membership hit, so a word no particle set holds pays no frame for it (AGENTS.md's frame budget: the classify call is per token). A fifth reader, segment's settled-run test in the comma credential walk (#562, deleted with segment's decision in #613), was NOT a site: an initial ends that walk before the particle question is asked, so a veto there changed nothing and was left out. Measured 2026-10-04 while the draft still asked the helper there (the call has since been removed, so that instrument no longer exists): with the helper answering False at that site alone, over every two-to-four-word arrangement of `Ó. ó. de vd Pérez Juan PhD MD VD DO do Do MA Ma Jr.` holding `Ó.` or `ó.`, in every comma position, under both `lenient_comma_suffixes` settings, no reading moved. To re-check it on this tree, wrap `_segment._normalize` so a one-letter period-written word never matches the particle set — the second review did, over 49,999 texts from `Ó. de vd DO do Pérez Juan MA PhD Ed Jr. MD VD van Q.` written as-is, upper and lower, under both settings: 0 moved, against 48 for a positive control that vetoes every particle at that site. The guard is the two-input invariant in tests/v2/test_properties.py — a name holding `Ó.` reads as the same name holding `Q.` in every view — with `_P7_SITE_EFFECT` recording, per site, how many grid texts break with that site's veto removed. +- 2026-10-04 (design review; Derek) — THE PERIOD IS THE ASCII ONE, matching `_vocab.is_initial`'s shape. The first draft took any of `FULL_STOPS`, so `Ó。` was vetoed although is_initial calls it no initial; now it is the particle, as `Q。` is no initial. The second half of is_initial — the scripts that write no initials (#320, `_policy._NO_INITIALS`) — is NOT reproduced: `_lexicon` may not import `_policy` (tests/v2/test_layering.py), and `_render` and the facade, which must ask the same question, can reach neither. Accepted as the residue: a CALLER's one-character particle in Han, Hangul or kana, written with a period, is vetoed where is_initial would call it no initial (`Lexicon.default().add(particles={'之'})` reads `王 之. 明` with `之.` outside the particle). No shipped particle is such a character. A hand copy of the script check, pinned by a sync test, was the alternative and was declined for a case only a caller can construct. +- Accepted: a bare `Juan Ó Pérez`, an Óscar initial written without its period, reads family `Ó Pérez`. Accented initials normally carry the period, and the bare letter is exactly how the particle is written. + +Declined: + +- A per-entry "matches only when written bare" flag (the issue's first framing, a new Lexicon field or set) — the period convention is a fact about INITIALS, not about `ó`, so it is stated once for every one-letter particle, including any a caller configures. A field would be one more vocabulary axis to translate through the v1 shim and to sweep in every guard roster, for a question no entry answers differently. + diff --git a/docs/design/decisions/R2-all-particles-family-base-divergence.md b/docs/design/decisions/R2-all-particles-family-base-divergence.md new file mode 100644 index 00000000..488dc236 --- /dev/null +++ b/docs/design/decisions/R2-all-particles-family-base-divergence.md @@ -0,0 +1,25 @@ +# R2 — the all-particles family_base divergence + +- 2026-08-18 #404/#385 — R2's DISCRIMINATOR REVERSED. It said a family written wholly out of particle vocabulary has a base where one of those words "is itself borne as an ordinary surname", and that only the words that are never anyone's name stay particles. That is a vocabulary test, and it is the wrong one: a particle earns its name by joining forward to the word it modifies, so a particle with nothing to join is not doing a particle's work whatever it is borne as. POSITION decides, and the borne-as-surname question does not arise. + The invariant that replaces it, and the reason this is worth a rule rather than three bug fixes: a non-empty family always has a non-empty base. A particle needs a base to attach to. Measured, 43 of 751 corpus names x 3 orders violated it — "Del Toro" under FAMILY_FIRST reported family 'Del' with no base, losing the surname from the base and from the initials. + Marked rather than untagged (mechanisms.md#MARK-DONT-STRIP): `particle` is stable API and means "from the particle vocabulary wherever it lands", which stays true; UNJOINED_TAG records what was decided. That also leaves #405's ambiguity emitter a predicate to key on, which stripping would have destroyed for exactly the never-given half where the fork is most interesting. + Blast radius: 44 of 2,253 parses move and every one is VIEWS-only — the seven role fields are byte-identical everywhere, so the differential harness cannot see this change at all (it compares roles only). Tests are the whole verification, and the invariant is one of them. Amended 2026-09-02 (#484): the harness now compares `initials()` under the `_initials` pseudo-field for names whose roles agree, so it CAN see this change — the 27 names are classified by the `fix(#385/#402)` rule at 1.4.0, 2.0.0 and 2.1.0. Tests are no longer the whole verification. + +- Recorded 2026-08-16, intent UNVERIFIED: the v1-era design held that a family name cannot be only particles ("Anh Do" — Do is a surname AND a particle), so last_base was guarded non-empty. The facade still guards (HumanName("Anh Do").last_base == "Do"); the v2 core does not (parse("Anh Do").family_base == "", family_particles == "Do"), and the surname vanishes from initials (parse("Anh Do").initials() == "A."). +- 2026-08-16 #385 RESOLVED by the collision criterion, not on its own terms: the issue's option 3 ("guard the view only when a word is vocabulary-ambiguous") is what decisions/vocabulary-collisions produces when applied here. + "Do" is borne as an ordinary surname, so it is ambiguous + vocabulary and anchors the base; "van der" is never anyone's name, so an all-particle family there genuinely has no base. The two rows of the issue's table were never one case. This is the keystone's clearest payoff: #385 was filed as a leaf with three options and no way to choose between them, and the criterion picks one without arguing about family_base at all. +- ANSWERED 2026-08-18 by the entry above, and recorded rather than left implicit: "Do" does NOT remain in family_particles once it is the base. The two views partition the family — parse("Anh Do").family_particles is "" and family_base is "Do" — which is asserted as a partition over the whole case table, not just as an example. Was: "Still open inside the resolution: whether Do remains in family_particles once it is also the base. Recorded here rather than left to the implementing PR to decide by accident." +- 2026-08-16 (pre-merge coherence pass) — the resolution moves R3 too, and R3 now carries its own marker. Initials read the BASE family word, so anchoring "Do" changes parse("Anh Do").initials() from "A." to "A. D." while + "Juan van der" stays "J." (no borne name, no base, and initials + of a bare particle run would be nonsense). The general lesson, worth more than this instance: a deviates: marker gets written on the rule whose STATEMENT changed, but a rule can change another rule's OUTPUT without touching its statement, and nothing looks for that — the runner asserts per example line, so an unmarked downstream rule stays green precisely because its own examples avoid the affected input. When adding a marker, walk the changed rule's `interacts:` targets and ask whether any of THEIR examples move. + +- 2026-08-29 #461 — TRIED AND BACKED OUT; the question is back on the issue. The unjoined mark readmits the words of an all-particle part to `initials()`, and #461 narrowed that readmission to the `particle` tag alone, on the authority of R3's "A CONJUNCTION never initials, so a base that is one contributes nothing even then". It was reverted before merge, in the same PR. What the narrowing got wrong is best said in terms of the MARK rather than of the fields. The mark is a statement about a whole PART — none of its words is doing the work its tag names — and #461 honored that statement for some of the part's words while keeping one of them out. Under `Lexicon.default().add(particles={'y'})`, `parse("Juan de y")` is such a part: `family_base` is `de y`, both words carry the mark, and #461 initialed `J. d.`, admitting the `de` as the name word the mark makes it and refusing the `y`. R2's reasoning does not split that way. A particle with nothing left to join is not acting as a particle, which is the whole reason the mark exists; a conjunction inside that same part has nothing left to join either, for the same reason and at the same moment, so it is a name word of the part and initials with the rest. Scope that carefully, because its general form is far wider than the claim and would be wrong: this is about a part the mark has ALREADY turned into name words, not about any conjunction that happens to join nothing. Outside such a part the skip stands whatever the conjunction is or is not joining, and none of those moved or may move — `parse("Juan y Garcia")` initials `J. G.` with the `y` a middle name word by P3, `parse("Juan Velasquez y Garcia")` initials `J. V. G.` over base `Velasquez y Garcia`, `parse("Jon Dough and")` initials `J. D.` over base `Dough and`. Base and initials differ in every one of those, legitimately, and R3's own `"Juan de y" → initials="J."` line is a fourth, its `family_base` being `y` under the default vocabulary. So "the base holds a word the initials do not" is NOT the criterion and was not the finding; the mark is. What is therefore in question is R3's "even then" clause, which carries the carve-out into the one part where the joining has stopped — not the code that failed to implement it. The clause is left standing and #461 now asks whether it belongs there; nothing else in R3 moved, and the paragraph #461 added under it is gone. R4's conjunction sentence rests on that clause by name ("the carve-out R3 states for initials") and is untouched here, so whoever settles #461 settles R4's cross-reference with it. Note what the backout cannot record: a `deviates:` marker hangs on an example LINE and this shape has none to hang on — `particles` and `conjunctions` are disjoint in the default vocabulary and in every locale pack, so no input string the doc runner parses reaches it. rules.md itself therefore carries no trace, and this bullet with mechanisms.md#RENDER-HONORS-THE-PARSE is where the gap is written down. It is not the only gap under R3's conjunction sentence, and the other is older, wider and markable: "A CONJUNCTION never initials" is unqualified, while a conjunction in the GIVEN group has always initialed — `parse("John and Jane Smith")` gives `J. a. J. S.` and `parse("Duke of Edinburgh")` `D. o. E.`, 25 of the 1094 corpus names, all reachable from the default vocabulary. Recorded here because it was found here; it is not #461's to fix. RESOLVED into the normative document on 2026-08-29, in the review of PR #463: rules/R3 now scopes its carve-out to the middle and base family words and declares the given group unsettled, in prose and WITHOUT a `deviates:` marker. The marker was considered and rejected on its own definition — it states the INTENDED value, and there is none to state. R3 counts name words while P3 makes a connective and its neighbours ONE name word, so `John and Jane Smith` has four candidate answers and no entry anywhere picks one: `J. a. J. S.` today, `J. J. S.` reading the carve-out as written, `J. S.` reading P3's join as a single name word, and `J a J. S.` at 1.4.0 (measured on the released wheel, which joins the run rather than skipping the conjunction). Marking it would put an invented value in a normative document and hold the parser to it, since the runner asserts the today-value strictly. The marker's `#N` slot has no owner either — the sentence above disclaims #461 for this gap, and filing its own issue is a maintainer's call rather than a docs commit's. What "markable" meant two sentences up is the MECHANICAL property that an input string exists to hang a marker on, unlike the #461 shape where none does; it was never a claim that the intended value is known. Neither direction is visible to the gate: 0 of the 1094 corpus names move either way, and the counts hold at 229 / 194 / 102 with 0 unexplained at 1.4.0 / 2.0.0 / 2.1.0. Both views of the contested token are pinned rather than left to prose — `test_repair_keeps_a_conjunction_lowercase_in_a_particle_part` holds R4's ungated conjunct, gating which had passed the entire suite until #461's test existed, and `test_initials_readmits_a_conjunction_in_a_particle_part` holds today's initials answer the way a `deviates:` marker would, so re-deciding #461 fails the suite until that pin moves with it. + +- 2026-09-20 #461 — AMENDS the 2026-08-29 bullet above, which said of four readings that "none of those moved or may move". Two of the four moved and two did not, and the half that survived is the half the criterion was always about. WHAT MOVED: `parse("Juan y Garcia")` initials "J. y. G." where that bullet recorded "J. G.", and R3's own `"Juan de y"` example line reads "J. y." where it read "J.". WHAT DID NOT: `parse("Juan Velasquez y Garcia")` is still "J. V. G." over base "Velasquez y Garcia", and `parse("Jon Dough and")` is still "J. D." over base "Dough and" (all four measured 2026-09-20 on this branch). The criterion that separates them is the one the bullet above named, and it is still not a word count: the question is asked of the WHOLE PART, and a part holding another name word for the connective to join — the part's working particles set aside — is a part where the connective is doing a connective's work and contributes nothing. "Velasquez y Garcia" and "Dough and" each hold such a word, so the skip stands there; the family "y", the family "de y" and the middle "y" of "Juan y Garcia" hold none, so the connective is its part's own name word and initials with it. + So that bullet was RIGHT ABOUT THE MECHANISM and wrong only about the clause it preserved. Its finding — that a mark honored for some of a part's words and not for one of them is honoring nothing — is exactly what rules/R3 now states, for all three groups at once, and it is why the fact is computed ONCE over the whole part rather than re-derived per word by each view. What changed is the clause it left standing: R3's "A CONJUNCTION never initials, so a base that is one contributes nothing even then" is conditional now, and "none of those may move" was the reading of the day rather than a decision. decisions/R3's 2026-09-20 bullet carries the rule as taken. The pin that bullet describes did its job: `test_initials_readmits_a_conjunction_in_a_particle_part` was written to fail when #461 was re-decided, and it failed. + +Declined: + +- 2026-08-18 — the GROUPING half of #404: a particle run that joins nothing does not chain, so "Jong van der" would split into middle 'van' plus family 'der'. Measured and rejected, though NOT for the reason first recorded here. The first draft said the split makes the family "no longer all-particle so the base fix stops firing" — false, and `der` and `la` are both shipped particles, so a family of either IS all-particle and the rule fires on it ("Juan Smith der" gives base 'der'). What the split actually costs is the SCOPE of the base and a stray particle relocated: grouping can decline to merge but cannot keep the words apart, because roles re-assemble them and two adjacent same-role pieces are one part at the field level. "Juan Smith van der" becomes middle 'Smith van', family 'der' — a base of 'der' rather than 'van der', and a middle name nobody wrote. Keeping the run whole in one part is what gives the base its full extent. The split reading needs the leftover distribution to know these are separate units, which is mechanisms.md#UNIT-PARTITION's problem. + diff --git a/docs/design/decisions/R3-initials-honor-the-fold.md b/docs/design/decisions/R3-initials-honor-the-fold.md new file mode 100644 index 00000000..fc8fd7c6 --- /dev/null +++ b/docs/design/decisions/R3-initials-honor-the-fold.md @@ -0,0 +1,32 @@ +# R3 — initials honor the fold + +R3's earlier history is under `decisions/R2`, which this entry does not repeat: the #461 backout and the unsettled given-group conjunction question are recorded there because they were decided alongside R2's all-particle clause. This entry is R3's own, and R3's `history:` points here. + +- 2026-08-30 #408 — DONE: `initials()` orders folded words first, as every other view of the same parse already did. O3's fold and P6's attachment both TAG rather than move a token (mechanisms.md#FOLDED_TAG — spans cannot reorder), and `_types._text_for` partitions on that tag for every role it renders, so the `family` field reads `van der` where the tokens sit `der ... van`. `initials()` walked `tokens_for(role)` in written order and never asked, so the two views disagreed about one parse: `parse("der, y van")` gave family `van der` and initials `y. d. v.`, and it gives `y. v. d.` now. This is mechanisms.md#RENDER-HONORS-THE-PARSE's second shape, filed as its worked instance, so the design is not re-argued here — only what the measurement added to it. + WHAT THE MEASUREMENT ADDED, and it reframes the issue from a consistency fix to a parity restoration. v1 had this feature as `CONSTANTS.middle_name_as_last` (`_config_shim` maps it to `Policy.middle_as_family`) and had NO general `name_order`, so only the two default-order policies have a v1 reference at all. Measured on the released 1.4.0 wheel over the 1094-name deduped corpus, from a script run outside the worktree that asserts `nameparser.__version__` first: at the default policy 1 name moves and agrees with 1.4.0 neither before nor after; under `middle_as_family` 71 move, of which **54 come into exact 1.4.0 agreement and NOT ONE moves away from it**. Worked pair: `Doe, Dr. John A.` gives `J. A. D.` at 1.4.0, gave `J. D. A.` here, and gives `J. A. D.` again; `Brundridge, Contessa A` likewise `C. A. B.` / `C. B. A.` / `C. A. B.`. The facade had been giving the folded answer all along — `HumanName.initials()` reads `first_list`/`middle_list`/`last_list`, and `_facade._list_for` prepends the carriers exactly as `_text_for` does — so the core view was simultaneously out of step with the field beside it, with the facade above it, and with 1.4.0 behind it. A view that stopped honoring a record had been reproducing a v1 bug v1 did not have. + THE 17 THAT AGREE WITH 1.4.0 NEITHER WAY, recorded because the obvious explanation is wrong — and the FIRST REPLACEMENT for it was wrong the same way, corrected by this branch's docs review before merge. It is not that their parse already differs: only 2 of the 17 parse differently from 1.4.0 under this policy, `Smith, John V.` (#432's dotted numeral) and `남궁민수 지훈` (the unspaced-hangul surname split and the script order, rules/W1). The other 15 have SEVEN BYTE-IDENTICAL FIELDS at 1.4.0 and here — `der, y van` INCLUDED, which an earlier draft of this bullet filed with the two above by carrying over a measurement taken at the OTHER policy, where its parse does differ. Of those 15, 14 differ from 1.4.0 only in v1's initials GRANULARITY — v1 initials one group per `*_list` element, so a run its parse joined into one element (`Velasquez y Garcia`, `abdul salam`) yields `V G.` where 2.x, one token per word, yields `V. G.` — and strip the periods and spaces and all 14 come into 1.4.0's letter ORDER after this change, none having been in it before (`JVGQ` → `JQVG`). The 15th is `der, y van`, and it is worth naming rather than lumping: its fields match 1.4.0 exactly and its initials do not, because 1.4.0 contributes NOTHING for a family that is all particles (`van der` → `y.`) where 2.x contributes its words. That is R3's own adjudicated clause and decisions/R2's territory — R3 carries `"Juan van der" → initials="J. v. d."` against 1.4.0's `J.` — so this one name is a divergence this document has already decided in favor of, not an unexplained residue. So the parity claim is stronger than "54 of 71 and the rest unexplained": 68 of the 71 come into 1.4.0's ordering, 54 of them exactly, one more is R2's decided divergence, and 2 have a parse difference that predates this change. + BLAST RADIUS by policy, over the same corpus × three `name_order` values × `middle_as_family` off/on (6564 parses, 660 move). Default order: 1 / 71. FAMILY_FIRST: 1 / 294. FAMILY_FIRST_GIVEN_LAST: 1 / 292. STATE THE POPULATION BESIDE THE COUNT, per this file's own rule: the shape an order change can be observed in is a contributing folded token standing AFTER a contributing unfolded one of the same role, and at the default order the corpus holds exactly 1 such name without `middle_as_family` and 72 with it. So the movers are 1 of 1 and 71 of 72 — a near-total hit rate over a corpus that carries one instance of P6's half, not a blast radius near zero. (The one non-mover is `Chang, Andy C I`, where the folded `C` and the family `Chang` share a letter and the reorder is invisible in initials.) Nine corpus names carry the tag at all at the default order; the other eight have a single contributing family token, which makes the partition the identity. The `middle_as_family` half is where the reach is, which follows from the producers: O3 folds every middle word, while P6's attachment moves initials only where the attached run is READMITTED by the unjoined mark — a family that is all particles — so it reaches one corpus name in each order and the same one. RECOMPUTE, and the COMPARATOR is the half that has to be stated, because the obvious phrasing of it measures nothing: parse the `tools/differential/corpus*.jsonl` files deduped under each of the six policies and compare `initials()` against THE PRE-CHANGE RENDERING — the same contributing tokens left in WRITTEN order. What this sentence said until 2026-08-30 was "against a folded-first partition of `tokens_for(role)`", which is what `initials()` now IS: that recipe compares the shipped implementation with itself and yields 0 differing parses, the inert-measurement shape (mechanisms.md's Verification shapes section) written into the very sentence meant to prevent it, and the only recompute path the repository offered for docs/release_log.rst's counts as well. CONTRIBUTING is the other half of the recipe and it comes FIRST: apply this view's own skip filter — outside the given group a token tagged `particle` or `conjunction` contributes nothing unless it also carries UNJOINED_TAG — and compare the two orders only over what survives it. Leave the filter out and every skipped word reads as a difference instead: 1546 rather than 660 against written order, and 1018 rather than 0 against the folded partition. Re-measured this way on 2026-08-30 and reproducing the counts above: 660 of 6564 parses, 1 / 71 at the default order, 1 / 294 and 1 / 292 under the two family-first orders. + THE VERIFICATION LIMIT, and it is decisions/R4's, restated because a plausible reading of "one name moves at the default policy" is that the ledger owes it a rule. It does not, and could not: `tools/differential/compare.py` compares the seven role fields plus `_ambiguities`, and `initials()` is not among them — the same blindness recorded for `capitalized()` under R4. Measured rather than reasoned: the gate was run at all three baselines on the pre-change tree (exported with `git archive HEAD` into a scratch directory, so nothing in the working tree moved) and on the changed one, and the two outputs are byte-identical apart from the tree path and the baseline-worker temp path — 1094 corpus names, 229 / 194 / 102 intentional diffs, `unexplained: 0`, and 67 / 58 / 51 headings with identical per-heading counts, at 1.4.0 / 2.0.0 / 2.1.0. NO LEDGER RULE WAS ADDED, and none was needed. What stands in its place is `rules/R3`'s example line and `tests/v2/test_render.py`, both mutation-checked. Amended 2026-09-02 (#484): the blindness this paragraph rests on is retired — the gate compares `initials()` for roles-identical names now, so an initials-only change DOES owe a ledger rule. #408 still owes none, and for a different reason than the one recorded here: its movement is 1.4.0 parity, and the one corpus name it moves at the default order (`der, y van`) diffs on ROLES at all three baselines, under `fix(#379)`, so the pseudo-field never enters. The semantics are in `decisions/differential-ledger, the initials view`. + WHY R3 GAINED A SENTENCE rather than leaning on R1. R1 already says folded family words "render before the rest of the family wherever they stood in the string" — but its subject is "Every FIELD", and `initials()` is not a field: R3 calls it "this view", and it takes a format spec and two separators rather than being read as an attribute. So R1 does not reach it by its own words, and #408 is precisely what leaving that to inference costs. R3's new clause states the order in R3's own voice, carries the O3 example line (`"Hassan, Mohamad Ahmad Ali"` under `middle_as_family` → `initials="M. A. A. H."`, reusing R1's own input so `corpus_rules.jsonl` is unchanged at 241 names), and R1 and R3 now name each other in `interacts:` — the field says which order, the view says it follows the field. P6's half gets no example line, for the same reason R1 carries none: the only input that witnesses it is `der, y van`, whose PARSE is itself contested, so pinning its initials normatively would pin a value another open question can move. It is pinned in the unit test instead, which is where today's-behavior pins belong. + THE PARTITION IS PER ROLE, not scoped to FAMILY, and this is a deliberate choice rather than a copied line. Both producers re-role to FAMILY, so FAMILY is the only role a parse can put the tag on today and the GIVEN and MIDDLE arms are unreachable. They are there because `_text_for` partitions for every role it renders and the two views must not diverge again — the same uniformity `_post_rules`' own UNJOINED_TAG loop takes for its three roles, "for uniformity with the rule, not because all three are observable". A producer that ever folded into another part would otherwise reopen #408 there with nothing to fail. `test_initials_folds_in_every_role_it_renders` pins it from a hand-built name, since no input string can. Mutation-checked four ways — scoping the partition to FAMILY, and skipping it for GIVEN, for MIDDLE, for FAMILY — and each fails that test. It took THREE drafts of the fixture to make that true, and the two misses are the same miss one role apart. The first carried two GIVEN tokens and ONE MIDDLE: a one-element partition is the identity, so the MIDDLE arm was asserted and unpinned, and skipping the partition for MIDDLE passed the entire suite. The second fixed MIDDLE and carried NO FAMILY token at all, in a test named for every role it renders — so skipping the partition for FAMILY passed this test, and was caught only by its siblings and by R3's example line, neither of which is about per-role application. A zero-element group is the identity too, and reads even less like a gap than a one-element one, which is why the second miss survived a review that had just named the first. Two tokens per role is what closes it, and the general form is carried as mechanisms.md#TWO-ELEMENT-GROUPS rather than left here: a test written to pin a partition, a sort or a dedup needs at least two elements in every group it claims to cover, or the claim rides on a no-op. +- 2026-09-01 #462 — DONE, the facade's twin: `HumanName.initials()` dropped a dotted or bare-capital `E`/`Y` from the middle and family groups because `_process_initial` tested conjunction membership alone, where 1.4.0's `is_conjunction` was "in the set AND NOT `is_an_initial`". `Scott E. Werner` gave `S. W.` from 2.0.0 through 2.2.0 and gives `S. E. W.` again; `John E Smith` (bare capital) and `Juan Y. Garcia` likewise; `parse().initials()` never had the bug, since `_classify` tags `E.` an initial and the render reads the tag (mechanisms.md#RENDER-HONORS-THE-PARSE). The issue said the bare form was "still correctly dropped", which is true of lowercase `e` and false of bare capital `E` — 7 of the 14 corpus names that move are bare capitals. Restored with `_render._INITIAL`, v1's own `initial` shape, because the facade may import `_render` and not `_pipeline`; scoped to `_process_initial`, its only caller, so a future reader of `_is_conjunction` does not inherit a decision made for initials. Found by #484's pseudo-field at the 1.4.0 baseline, where it had sat as 14 unreported diffs; the gate could not see it before because the seven fields never moved. +- 2026-09-13 #528 — DONE: `HumanName.initials()` reads the parse's tags. The facade's initials view was the last derived view still re-deciding the CONNECTIVE question after the parse had answered it — scoped to that question deliberately, since the PARTICLE question is still re-decided from the lexicon in two places by decision, `_cap_word`'s particle conjunct (`decisions/R4`'s NOT DONE bullet) and `_is_particle` here, R4 drawing that boundary per question rather than per view. #462's fix above is why the connective half was still being re-decided: restoring v1's `is_conjunction` ("in the set AND NOT `is_an_initial`") restored a SHAPE test standing in for a tag, which agreed with the parse only while the parser agreed with the shape. #383/#479 ended that the same day it landed — rules/P3 now reads a one-case single-letter connective from vocabulary rather than from case, and no shape over the raw word can see it — so the two views of one parse disagreed: `HumanName("john e smith").initials()` gave "j. s." against the core's "j. e. s.", and `HumanName("JUAN Y GARCIA").initials()` gave "J. Y. G." against the core's "J. G.". Both give the core's answer now. This is mechanisms.md#RENDER-HONORS-THE-PARSE applied to the last view that had not taken it, and #458's principle reaching the facade layer. + THE SHAPE IS R4's, not a new one. A word backed by a token asks `"conjunction" in tok.tags`; a token carrying `UNCLASSIFIED_TAG` — text spliced in by `hn.middle = ...` through `ParsedName.replace()`, or restored by the v1 pickle path in `__setstate__` — was read by no parse, so the vocabulary answers, through `_render._reads_as_conjunction`, the same helper `_cap_word` calls for the same tokens. One question and not two: whether a word is a connective is a fact the word can answer alone, while whether a particle is acting as a particle is a fact about the whole part, which is R4's own boundary and is why `_is_particle` stays a live vocabulary lookup here. `_is_conjunction` lost its only caller and is gone, which retires the last sentence of the 2026-09-01 entry above — there is no future reader of it to inherit a decision made for initials. + A STATED PRIVATE-METHOD BREAK, accepted rather than worked around. `_process_initial` keeps v1's `(name_part, firstname=False)` signature for direct callers and gains an optional `tokens=`, and `_initials_lists` always passes it — so a v1-shaped subclass that overrides `_process_initial` with the two-argument signature now raises `TypeError` the first time `initials()` runs. The alternative considered was a string wrapper kept over a token core, which would leave such an override silently INEFFECTIVE instead: it would be called, and its answer discarded by the token path beside it. A loud break on a private hook beats a quiet one, and it is stated in the method's own comment and pinned by a test. FORWARDING IS STILL THE ONLY WAY TO RECEIVE THE FIX: an override must ACCEPT `tokens` **and forward it**, `def _process_initial(self, name_part, firstname=False, tokens=None): return super()._process_initial(name_part, firstname, tokens=tokens)`, which gives `Forwards("john e smith").initials()` the fix's own answer, "j. e. s.". Widening the signature alone does not receive the fix either — `**kwargs`, or an explicit `tokens=None` that the super() call drops. Revised 2026-09-14, before this paragraph ever reached master: the token path now passes the group's own text as `name_part` (`" ".join(tok.text for tok in group)`) rather than an empty placeholder, precisely so an override that ignores the `tokens` keyword falls onto the STRING path and behaves as it did before this upgrade, rather than going quietly blank. Measured 2026-09-14: `WidensOnly("john e smith").initials()` is "j. s.", the pre-#528 answer, where the forwarding override above and the library itself both give "j. e. s." — the STRING path's vocabulary fallback reads the middle "e" as the connective and drops it, where the token path's parse-backed reading (restored by #528) does not. So "accept a `tokens` keyword" is still the wrong instruction on its own; the forwarding is what receives the fix, not merely what avoids a `TypeError`. This is the only break in #528; on the public v1 surface nothing moves but the initials of six corpus names: `john e smith`, `john e jones`, `jones, john e`, `jose e maria santos`, `JUAN GARCIA Y LOPEZ` and `JUAN Y GARCIA`, the six the ONE CORPUS NAME paragraph below counts. + + A SECOND STATED BREAK, FOUND AND FIXED IN REVIEW (2026-09-14): before #528, `_initials_lists` read `self.first_list`/`self.middle_list`/`self.last_list` — public properties a v1 subclass may override — and the move to `_list_tokens_for(member)` above dropped that: an overridden property was silently ignored by `initials()` alone, while `last_base` (via `_split_last`, which reads `self.last_list`), `surnames` (`self.middle_list + self.last_list`) and `given_names` (`self.first_list + self.middle_list`, which never reaches `last_list` at all) kept honoring it. Fixed by checking, per member, whether the class overrides that property (`getattr(type(self), f"{member}_list") is not getattr(HumanName, f"{member}_list")` — three class-attribute identity checks, nothing on the parse path) and, where it is overridden, taking the pre-#528 STRING path over the override's own strings instead of the token walk. Measured 2026-09-14 against `git show 338daf7:nameparser/_facade.py` (before #528): a three-property `Sub` (`first_list` uppercasing every element of `super().first_list`; `middle_list` dropping every element of `super().middle_list` that starts with `"X"`; `last_list` always returning `["Zorro"]`) on `"john Xavier smith"` gives "J. Z." both there and here — reading only the `last_list` override into the recipe predicts "j. X. Z." instead (measured 2026-09-14 with `last_list` alone overridden on this tree), since the upper-cased first and the filtered-out `Xavier` both feed the result too — where the unfixed tree gave "j. X. s."; a one-property `Sub` (only `last_list` overridden, same `["Zorro"]`) on `"john e smith"` gives "j. e. Z." — the un-overridden first/middle keep the (post-#528) token path's own answer, and only the overridden last takes the pre-#528 fallback. Honoring the override, rather than leaving it silently ignored, is why "This is the only break in #528" above still holds: a second silent break on a PUBLIC surface would have gone unnoticed by every existing test, unlike the private-method break above, which is loud by design. Pinned at `tests/v2/test_facade.py::test_initials_honor_an_overridden_list_property`. + ACCEPTED COST, MEASURED (2026-09-13, this branch): for a word backed by a token the connective answer is fixed at parse time, as `capitalize()`'s already was. `C = Constants(); h = HumanName("juan y garcia", constants=C)` gives "j. g."; `C.conjunctions.remove("y")` leaves it "j. g." with no re-parse, where before #528 it gave "j. y. g." immediately; `h.full_name = "juan y garcia"` re-parses and gives "j. y. g.". Unpinned before this change and pinned by it, at `tests/v2/test_facade.py::test_initials_freeze_the_connective_answer_at_parse_time`. + ACCEPTED COST, THE SECOND ONE, and it is the no-parse paths rather than the configuration, THREE of them, not one: state restoration (pickle, `copy.copy`, `copy.deepcopy`) and the keyword constructor all skip the tokenizer, so every word they carry takes the vocabulary fallback instead of a parse's tag. `__setstate__` stamps every restored token `UNCLASSIFIED_TAG`, because a v1 pickle carries the `*_list` STRINGS and no tags, so a restored name is spliced text throughout; `copy.copy` and `copy.deepcopy` use the same `__getstate__`/`__setstate__` state hooks, so a copy takes the identical fallback (measured 2026-09-14). The THIRD producer, unrecorded until this review round: `HumanName(first=..., middle=..., last=...)` routes straight through the field-setter properties in `__init__` and never calls `_apply_full_name`, so its tokens are `UNCLASSIFIED_TAG` for the identical reason a restored pickle's are — not a new bug, a third route to a cost already accepted. `HumanName(first="john", middle="e", last="smith").initials()` gives "j. s." where `HumanName("john e smith").initials()` gives "j. e. s."; `HumanName(first="JUAN", middle="Y", last="GARCIA").initials()` gives "J. Y. G." where `HumanName("JUAN Y GARCIA").initials()` gives "J. G." (measured 2026-09-14). A round-tripped, copied or deep-copied `JUAN Y GARCIA` therefore gives "J. Y. G." where the live parse gives "J. G.", and a round-tripped, copied or deep-copied `john e smith` gives "j. s." where the live parse gives "j. e. s." (the pickle round trip measured 2026-09-13, copy and deepcopy measured 2026-09-14, all three giving the identical two values). `capitalize()` has disagreed with the live parse on exactly those two names for exactly that reason since the tag was introduced — "Juan Y Garcia" against "Juan y Garcia", "John e Smith" against "John E Smith", all four measured the same day — and the keyword constructor gives the identical disagreement, `HumanName(first="john", middle="e", last="smith").capitalize()` leaving "John e Smith" against the live parse's "John E Smith" on both this tree and 338daf7 (measured 2026-09-14) — so this is the existing shape reaching one more view rather than a new one, and the remedy for the state-restoration pair is the one `__setstate__`'s own comment gives: the restored state is a v1 blob, not a parse; the keyword constructor has no state to restore an argument from — the caller supplied fields, not a string, so there was never a tokenizer run to have tagged them. RECOMPUTE ACROSS THE CORPUS, ALL THREE PATHS TOGETHER: rebuild every one of the deduped `tools/differential/corpus*.jsonl` glob's 1174 distinct names from its own parse's seven keyword-constructible fields (`first`, `middle`, `last`, `title`, `suffix`, `nickname`, `maiden`) and diff `initials()` against the live parse — 1 of 1174 differs before #528 (`Ph. D., John`, the pre-existing merge the ONE CORPUS NAME paragraph below names, unrelated to this mechanism) and 7 now, the added six being the same six `initials()` movers named throughout this entry. Pinned at `tests/v2/test_facade.py::test_initials_of_an_unpickled_or_copied_name_ask_the_vocabulary_too`. + THE POPULATION, MEASURED 2026-09-14: the deduped `tools/differential/corpus*.jsonl` glob holds two line shapes — `{"name": ...}` objects in `corpus.jsonl` and `corpus_shapes.jsonl`, bare JSON strings in the other four — so reading both and deduping is what gives its 1174 distinct names; a dict-only read stops at 528. Over that glob, exactly 6 give a different `initials()` after a pickle round trip than the live parse, up from 0 before #528 — the same six the STATED BREAK paragraph above names: `john e smith`, `john e jones`, `jones, john e`, `jose e maria santos`, `JUAN GARCIA Y LOPEZ` and `JUAN Y GARCIA`. `copy.copy` and `copy.deepcopy` move the identical six over the same glob, confirming the state hooks are one mechanism rather than a pickle-specific accident. + ONE CORPUS NAME STILL DIVERGES and it is not this rule's. Measured over the deduped `tools/differential/corpus*.jsonl` glob (1174 distinct names) before and after: seven names had the two views disagreeing, six moved here, and `Ph. D., John` remains — the facade merges "Ph." and "D." into ONE list element (v1's `fix_phd`) and renders "P D" with the separator and no inner delimiter, giving "J. P D." against the core's "J. P. D.". That is `fix(initials-per-word) the Ph. D. merge`, a 2.0.0 rendering change ledgered at 1.4.0 since #484, and #528 preserves the element boundaries exactly so it neither moves nor is absorbed. RECOMPUTE by parsing every name of the glob on both surfaces and diffing `initials()`; the before half needs the pre-#528 `_facade.py` and `_render.py` on the path, which `git show` writes into a scratch copy of the package — never a checkout in a shared worktree. + DIFFERENTIAL. The facade's `_initials` is compared at every baseline, 1.4.0 included, and the core's from 2.0.0; the pseudo-field enters a name's diff only where every role and every ambiguity kind agrees. So at 1.4.0 FOUR of the six take a new rule (`fix(#528) the facade's initials follow the parse's connective tags`) and two do not — `jose e maria santos` and `JUAN GARCIA Y LOPEZ` move roles against that baseline, and #383/#479's role rule explains them. At 2.0.0 through 2.2.0 nothing new: the e-names' `_ambiguities` diff keeps `_initials` out of their diff, and `JUAN Y GARCIA`'s single `_initials` row survives with the facade half of its two causes closed, which `fix(#462)`'s dated paragraph in `expected_since_2.0.0.toml` now says. At 2.3.0 the existing rule stands and both surfaces move together. All five gates re-run at 0 unexplained on 2026-09-13. + +- 2026-09-20 #461 — DONE, and the clause is settled for every group at once. THE RULE AS TAKEN: a connective contributes nothing where it is JOINING — a part holding another name word for it to join, the part's working particles set aside — and where its part holds nothing else it is joining nothing, so it initials like any other name word and agrees with the base. That picks ONE of the four candidate answers the 2026-08-29 bullet under `decisions/R2` listed for "John and Jane Smith" and makes the other three unreachable: `parse("John and Jane Smith").initials()` is "J. J. S.", not the "J. a. J. S." of 2.0 through 2.3, not P3's one-name-word "J. S.", and not 1.4.0's run-together "J a J. S.". THE CRITERION IS NEVER A WORD COUNT, which is what keeps `parse("Jon Dough and")` at "J. D." over base "Dough and" and `parse("Juan Velasquez y Garcia")` at "J. V. G." — a part of two words where one is the connective is still a part where it joins, and a part of three is no different. + BOTH VIEWS, ONE MARK. The fact is about a PART, so it is decided once where the parts are settled and recorded on the token (`UNJOINED_CONJUNCTION_TAG`, mechanisms.md#MARK-DONT-STRIP), and the core view and the v1 facade both read it. That is the direct answer to what the first #461 attempt found: a mark honored for some of a part's words and not for one of them is honoring nothing. A SEPARATE marker rather than a widening of `UNJOINED_TAG`, because that one is also read by `family_particles`/`family_base` and by `_cap_word`, and none of those three may move. `Parser.revise` recomputes both marks, the same reason `_remarked` recomputes the first. + BLAST RADIUS, MEASURED 2026-09-20 against the parent commit `46651750` over the deduped corpus-union-cases set under eight configurations, both processes asserting their own `nameparser.__file__`. RE-MEASURED 2026-09-20 at the end of the second review round, the population having grown again with its own rows. Over the population AS IT STANDS — 1568 non-empty names — the default order moves 12 names' roles, 3 more names' reports with the roles fixed, and 56 names' `initials()`, 22 gaining a letter and 34 losing one; `HumanName.initials()` moves on THE SAME 56 names with THE SAME values. Family-first moves 11 / 3 / 34 and `FAMILY_FIRST_GIVEN_LAST` 12 / 3 / 64. The twelve role movers at the default order are "Carod i Rovira, Josep", "Doe, Jane nee Puig i Soler", "Henry i of England", "Jane Doe nee Puig i Ma", "Jane Doe nee Puig i Soler", "Josep Carod i Rovira", "Josep Carod i Rovira III", "Josep Carod i de Rovira", "Josep Lluis Carod i Rovira", "john smith i jr", "maier, amy i, jr." and "rovira, i" — seven of the twelve are rows this bundle added, which is why the restricted figure below is the comparable one. Restricted to the 1543 names that stood in the corpus at the parent, the same sweep reads 2 / 3 / 46 at the default order, 18 gaining and 28 losing, with 1 / 3 / 24 family-first and 2 / 3 / 52 given-last — unmoved by the second review, whose report change touches only "JOHN QUINCY SMITH I", a name the round itself added. (The 1558-name sweep that stood here read 6 / 3 / 50 with 19 gaining and 31 losing, and 5 / 3 / 28 and 6 / 3 / 58 for the two family-first orders; it is a dated snapshot over a smaller population.) RECOMPUTE by parsing every name of the deduped `tools/differential/corpus*.jsonl` glob plus `tests/v2/cases.py`'s texts under those eight configurations in two processes, one on this tree and one on the parent's, and diffing the seven role fields, the ambiguity kinds and `initials()` on both surfaces. The gate cannot answer this on its own: its `_initials` pseudo-field sees only names whose roles stayed put, so a count taken from the classified summary is a FLOOR. + TWO OF THE GAINS ARE 1.4.0 PARITY RESTORED, and that is what retires two ledger rules: `JUAN Y GARCIA` gives "J. Y. G." and `محمد و علي` gives "م. و. ع.", which is what 1.4.0 gave for both. The two ranges differ and an earlier draft of this bullet ran them together, so state them apart (measured on all five wheels 2026-09-20): `محمد و علي` gave "م. ع." at 2.0.0, 2.1.0, 2.2.0 AND 2.3.0, while `JUAN Y GARCIA` gave "J. G." at 2.0.0, 2.1.0 and 2.2.0 and went back to 1.4.0's "J. Y. G." at 2.3.0 — the reason being the one the 2026-09-13 #383/#479 entry under P3 records, and the 2.4.0 release note says the same. The readmission is a parity RESTORATION on those two names rather than a deviation, and the given-group half in the other direction is the deviation — see rules/R3's Accepted clause, which states both. + THE FRAME BUDGET WENT DOWN: `tools/perf/call_count.py` reads `py3.11 parse=406.00 facade=443.00` where the parent read `412.00 / 449.00`. The two marks now come off ONE walk over the roles instead of two generator expressions, and on 3.11 a genexp costs a frame for its creation and one per resume, so the reference name's two non-empty parts pay exactly six. `_CALL_BASELINE[(3,11)]` is `{"parse": 410, "facade": 447}` with `_BAND = 0.02`, so the bands are [401.8, 418.2] and [438.06, 455.94] and both figures land inside: NO baseline edit and no `decisions/parse-cost` decision is owed. PEP 709 inlined comprehensions and not generator expressions, so 3.12 through 3.15 should drop by the same six. + THE NO-PARSE PATHS KEEP THE OLDER READING, the same degradation the 2026-09-13 #528 bullet above accepted and for the same reason: a token the parse never saw carries no mark, so the view falls back to the vocabulary. Measured 2026-09-20, `HumanName("JUAN GARCIA Y LOPEZ").initials()` is "J. G. L." live and "J. G. Y. L." after `pickle`, `copy.copy` or `copy.deepcopy`; a name built from keyword fields carries no tags either, and so does one whose `*_list` property a subclass overrides. The remedy is the v1 one — assign `full_name` again — since the facade has no `revise`. + HOW FAR IT REACHES, over a STATED population rather than an example, because this bundle MULTIPLIES it and the earlier wording did not say by how much: over the deduped `tools/differential/corpus*.jsonl` glob plus `tests/v2/cases.py`'s texts — 1,568 non-empty names — the v1 `initials()` of a pickled, copied or deep-copied name differs from the live parse's on 34 names here against 6 at the parent 46651750, measured 2026-09-20 in two processes each asserting its own `nameparser.__file__`. #461's lone-connective half is what multiplies it: every part whose only word is the connective initials live and drops after the round trip, so `HumanName("Juan de y").initials()` is "J. y." live and "J." after `copy.deepcopy`. RECOMPUTE by building that population and comparing `initials()` before and after each of the three state paths. A larger figure measured over a generated grid says nothing this does not — the population is what the number is about. +- 2026-10-02 #585 — DONE: an initial is the shortest start of its word that composes to the composed word's first character, written in the form the writer used. `initials()` on both surfaces had taken a token's first CODE POINT (`t.text[0]` in `_render.initials`, `word[0]` and `w[0]` in `_facade`), which for a name typed decomposed (NFD) is a base letter without the mark that completes it: `émile zola` initialled `e. z.` against `é. z.`, and — the reach the issue did not name, found by the corpus walk's control — decomposed katakana lost its dakuten, `マイケル ジャクソン` initialling `マ. シ.` against `マ. ジ.`. `_render._first_letter` is the one helper both surfaces call. One shape has no such start: a mark composing with nothing that canonical order puts between the letter and a mark that does (`o` + U+0331 + U+0301, whose composed spelling is `ó` + U+0331), where the initial is the composed character decomposed, `o` + U+0301 — found by the docs review, 2026-10-02. That fallback is for a start LONGER than one code point: a first code point that is a whole character Unicode excludes from composition (`क़`, U+0958, which NFC writes `क` + a nukta) is kept as written, its nukta with it, as before this change; the second review found the first draft of the fallback stripping it (`ज़ाकिर` initialling `ज.`). And the fallback writes the character wholly decomposed even where the writer composed part of it (`ẹ` + U+0331 + U+0302 gives `e` + U+0323 + U+0302) — the twin holds, the written form does not, accepted as contrived. WHY NOT "THE FIRST LETTER AND THE COMBINING MARKS AFTER IT", the shape #542 used for case repair and the issue's own option 1, measured before writing it: decomposed hangul jamo are LETTERS (category Lo), not marks, so it would initial a decomposed `김` as `ᄀ` where the composed name gives `김`; and a Bengali vowel sign IS a mark (Mc), so it would move a COMPOSED `কিরণ` from `ক.` to `কি.`, which is a question about Indic syllables nobody has decided, and a change the gate would see on every composed Indic name. Composing the prefix answers both: a composed word gives `text[0]` exactly as before, so no composed name's initials move. DECLINED: composing the initial to NFC (the issue's option 2, and what outside PR #586 did, on the core surface only). It fixes the dropped mark but hands back a composed initial for a decomposed name, where #542 settled that the render views keep the form they were given (decisions/R4, 2026-10-02). VERIFICATION, which the gate DOES see, unlike #542's: `_initials` is a compared pseudo-field (#484), and the two NFD example lines this change adds to rules/R3 enter the rules corpus and diff at every baseline, classified by a `fix(#585)` `_initials` rule in each of the five ledgers; the CJK rule's corpus claim grows 138 → 139 at 1.4.0 and 2.0.0 with the katakana row, which lies in its script span. `tests/v2/test_properties.py::test_a_decomposed_name_renders_as_its_composed_twin` joins #542's walk rather than opening its own and records 44 disagreements — 22 texts, each on both surfaces — over the 139 texts compared with `text[0]` restored (this change's corpus, its two R3 rows included; measured 2026-10-02), with a live control beside it. + diff --git a/docs/design/decisions/R4-case-repair-reads-the-unjoined-mark.md b/docs/design/decisions/R4-case-repair-reads-the-unjoined-mark.md new file mode 100644 index 00000000..dd91ddba --- /dev/null +++ b/docs/design/decisions/R4-case-repair-reads-the-unjoined-mark.md @@ -0,0 +1,70 @@ +# R4 — case repair reads the unjoined mark + +- 2026-08-29 #407 — DECIDED: case repair consults the mark #404 introduced, and consults it for the WHOLE PART rather than for a particle standing alone. Until now three views read it and the fourth did not, which is how `parse("ANH DO")` came to report family_base 'DO' and initials 'A. D.' while `capitalized()` returned 'Anh do' — the same word treated as an ordinary name word by three readers and as a particle by the fourth. The scope question is the only real choice here, and standing-alone loses on the first Vietnamese name that tests it: `anh van do` has family `van do`, two particle-vocabulary words and neither of them alone, so a rule keyed on aloneness would capitalize `Anh Do` and leave `Anh van do`, making the same surname read differently for having a given-name word in front of it. rules/R2 states the test as "Every word", not "standing alone", and this is the case that shows why it is stated that way. One predicate, four views. +- 2026-08-29 — this is a DELIBERATE DEVIATION from 1.4.0, not a parity restoration, said plainly because the surrounding branch work is mostly restorations and a reader will otherwise assume this is one. Measured on the released 1.4.0 wheel: `HumanName('ANH DO').capitalize()` leaves `'Anh do'` and `HumanName('anh van do').capitalize()` leaves `'Anh van do'`; both are `'Anh Do'` and `'Anh Van Do'` on this branch. There was nothing to restore — v1 lowercased on vocabulary membership alone and had no notion of a particle with nothing to join, the mark being a 2.x invention. +- 2026-08-29 — the ACCEPTED COST, agreed before the change rather than discovered after it: a degenerate family that is nothing but particles capitalizes too, so `juan van der` renders `Juan Van Der` where 1.4.0 gave `Juan van der`. This is the same trade rules/R2 already took for `family_base` — those words ARE the base, so they are name words, and a parser that called them name words for sorting and particles for display would be the incoherent option. Blast radius, measured by running both predicates in one process, and stated over BOTH populations because the neighbouring bullets in this entry measure over 1094 and a bare "4 and 23" would read as measured there too. Over the 1090 names the corpora carried before this change: 4 names move under a plain `capitalized()` — `ANH DO`, `vai la`, `van ma van`, and `der, y van`, which becomes `y Van Der` and keeps its lowercase conjunction — and 23 under `force=True`, the extra 19 being mixed-case inputs the gate holds back (rules/R5). Over today's 1094 it is 5 and 24: this change's own R4 example rows added `anh van do`, and that name is itself a mover. +- 2026-08-29 — THE VERIFICATION LIMIT, which matters more than the change. `tools/differential/compare.py` compares the seven role fields plus `_ambiguities` and nothing else, so `capitalized()` is not a compared field and NO ledger run at any baseline can see this deviation: the corpus counts and the three intentional-diff totals hold across it precisely because the harness is blind to it, and a green run is evidence about roles, not about case. mechanisms.md#FACADE-CONTRACT promises that v1-visible changes are "release-log-classified fixes, each verified in the differential ledger". The first half is kept — docs/release_log.rst carries the 2.2.0 bullet — and the SECOND HALF CANNOT BE, for this change or for any other that moves only case. Standing in its place: rules/R4's own example rows (`ANH DO`, `anh van do`, and the `juan de la vega` boundary), rules/R5's, and the v1-facade tests in tests/test_capitalization.py. The ones that pin the fix are mutation-checked against deletion of the tag consult; the `juan de la vega` control passes that mutation BY DESIGN, which is what makes it a control rather than a witness. Read the silence as a SCOPE and not as a gap: the harness compares parses, case repair is a render view, and nothing is owed on the tracker for it — the next bullet is why that is a decision rather than a deferral. Amended 2026-09-02 (#484): the premise moved and the conclusion did not. `compare.py` compares the seven role fields plus `_ambiguities` AND `_initials`, so "nothing else" is now one pseudo-field short and a render view is no longer invisible by construction — but `initials()` is the only one added, deliberately (`decisions/differential-ledger, the initials view`), so `capitalized()` is still not a compared field, and the next bullet's "the compared surfaces stay the seven roles plus `_ambiguities`" should be read as naming the surfaces case repair is absent from. Do NOT read #484 as the precedent that answers the next bullet: it MEETS that bullet's two objections rather than waiving them. `_initials` is a diff-of-diffs gated on the fields agreeing — precisely the instrument the next bullet declines for case — and the column did arrive pre-failed, by 140 entries at 1.4.0 as measured on this branch's base — 126 of them classified by six new 1.4.0 rules on the shipped tree, and the other 14 by `fix(#462)` at the three 2.x baselines once the facade fix moved them there. What earned it its place is a demonstrated ESCAPE: #408 moved initials for a whole minor with the gate green, and the 98 per-word names had been moving since 2.0.0, unseen and unclassified. So the next bullet's case for no `_capitalized` column stands until a case change escapes the same way, and that is the test a future proposal has to meet. +- 2026-08-29 — WHY CASE IS NOT A COMPARED SURFACE, settled here rather than left open, because "add a column" is the obvious answer to the bullet above and it is the wrong instrument. What the harness is FOR is parsing behavior — tools/differential/README.md says to run it "when touching parsing behavior, and before cutting a release" — and what earns it its two-process cost is that a role is EMERGENT: the eight stages fold over one name and each can move what the next sees, so the reach of a vocabulary or rule edit is not derivable by reading the diff and has to be measured over a corpus instead. Case repair is the opposite kind of thing. `_render._cap_word` is a pure function of `(word, role, tags, lexicon)` with four branches — the particle/conjunction lowercasing, the exceptions map, the Mac/Mc convention, and plain `capitalize` — sitting behind R5's single gate, so an edit to it is legible: you can see what you touched. The measurement agrees with the reasoning. Over the 1094 corpus names the 1.4.0 and 2.1.0 wheels ALREADY disagree on `capitalize(force=True)` for 139 of them (recompute by running `HumanName(n).capitalize(force=True)` under each wheel over the corpus files deduped, and diffing the two maps; the count was the same over the 1090 names the corpora carried before this change). Nearly every one is an echo of a parse diff the field comparison already classifies — a token that changed role capitalizes differently — so a column would re-report the ledger's own explained diffs in a vocabulary its rules cannot match, and would arrive pre-failed by 139 entries. Isolating the residue means comparing case only where the fields agree, a diff-of-diffs, to reach a class that a unit test reaches directly. So the compared surfaces stay the seven roles plus `_ambiguities`, by design and not by omission, and case is pinned where case belongs: the `rules/R4` and `#R5` example rows, and tests/test_capitalization.py. +- 2026-08-29 — DEBT DISCHARGED, the obligation the last bullet of decisions/R5 left this commit; recorded here so the pointer resolves rather than dangling. The FALSEHOOD: R4 promised repair "vocabulary exceptions (McDonald) included" flatly, which `str(parse('Juan Mcdonald').capitalized())` — `'Juan Mcdonald'` — falsifies, the gate refusing before any vocabulary is consulted. The statement now scopes that promise to where repair acts at all and hands the where to R5. A SECOND error in the same eight words turned up while rewriting them, and is worth recording because it had survived unread for as long as the first: McDonald is not a vocabulary exception. The shipped exceptions are five entries — II, III, IV, M.D., Ph.D. (recompute with `Lexicon.default().capitalization_exceptions_map`) — and `Mc`/`Mac` is a separate convention applied to any name shaped like one, so the old parenthetical illustrated the promise with the one mechanism it was not about. The statement now names an exception the vocabulary actually records (Ph.D.) and the Mac/Mc convention separately. The AMBIGUITY: "an already-correct name comes back unchanged" is true read as idempotence and reads as false under bearer-correctness, and the repair was to say WHICH is meant — a name already written the way repair would write it comes back unchanged — plus one clause refusing the inference a reader might otherwise draw, that a spelling its bearer chose is not spared for having been chosen. That clause is deliberately a REFUSAL and not a carve-out: sparing deliberately single-cased names is behavior the parser does not have and that R5's rationale declines on the record, so `str(parse('bell hooks').capitalized())` is still `'Bell Hooks'`. R4's boundary moved too: the old `"Juan McDonald" → capitalized="Juan McDonald"` row was inert — it passes with R5's gate deleted — so it became a `capitalized_forced=` row, which asserts what repair APPLIES rather than what the gate keeps and fails when the Mac/Mc convention is removed (measured by neutering it in one process: `'Juan Mcdonald'`). The boundary is now `juan de la vega`, a working particle run R4's new clause deliberately does not reach. + +- 2026-08-29 — the CONJUNCTION CARVE-OUT, stated in rules/R4 rather than left to a code comment, and the reason the comment gave was false. `_cap_word`'s conjunction conjunct is deliberately NOT gated on the unjoined mark, and the behavior is right: R3 already carries the carve-out for initials ("A CONJUNCTION never initials, so a base that is one contributes nothing even then"), and R4 was the half of the pair that carried it nowhere. What the comment claimed instead was an impossibility — `particles` and `conjunctions` are disjoint, so no conjunction token can ever carry the mark — and disjointness is a property of the SHIPPED vocabulary while both sets are public, configurable API. Measured: under `Lexicon.default().add(particles={'y'})`, `anh y van` parses to an all-particle family whose `y` carries `conjunction`, `particle` and the mark at once, and repairs to `Anh y Van`; gating the conjunct too, in one process, gives `Anh Y Van`. So the clause decides something and is not a no-op. It gets NO example row, and that is a limit of the document rather than a choice: the runner parses with the default vocabulary, over which `particles ∩ conjunctions` is empty in the default lexicon and in all four locale packs, so no input string can witness it. `der, y van` is the nearest thing and does not witness it either — the `y` there is a GIVEN word standing beside the all-particle family, not inside it (tests/test_capitalization.py says so at the row). + +- 2026-08-29 — the REPLACE/REVISE BOUNDARY, raised by the review of this branch as a gap needing a systematic fix and corrected by Derek to what it actually is: the documented boundary, with a supported path across it. `_cap_word` keys the particle TEST on lexicon membership but the REPAIR on the unjoined mark, and the mark is a parse product — so `parse('de la').capitalized(force=True)` is `'De La'` while `base.replace(family='de la').capitalized(force=True)` is `'de la'`, `replace()` splicing raw text into a field without classifying it. `Parser.revise()` is the method that classifies it, and its docstring already promised exactly this: each value gets a full sub-parse "so the stable tags survive and the tag-driven views ... behave as if the text had been parsed". Measured: `Parser().revise(base, family='de la').capitalized(force=True)` is `'De La'`. So there is nothing to fix — rules/R4 carries it as an Accepted boundary naming `revise()`, not as a deviation, and `revise()`'s enumeration of the tag-driven views gains `capitalized()`, which #407 made the fourth. (#458 would additionally make `replace()` agree, by keying the test on the `particle` tag rather than re-deriving membership from the word — the same shape it already names for `conjunction`/`initial`. That is a convenience, not the resolution.) It is a limit and not a regression either way: 2.1.0 gives `'de la'` on the spliced path too, so only the parsed path moved. + +- 2026-08-29 #458 — DONE: case repair's conjunction test reads the `conjunction` TAG instead of re-deriving the word's class at render time, so a view honors the decision the parse recorded (mechanisms.md#VOCAB-TAGS) rather than taking it again. The two copies of the question were not the same question, which is the reason this was worth doing over a corpus that cannot see it: `_classify.py` asks `is_initial(token.text)` — the shape test ANDed with a repertoire test since #320 — while `_render.py` asked a bare pattern, shape only, over each WORD of a token's text rather than the token. It does NOT retire the hand-sync obligation the module carried in a comment ("keep in sync with `nameparser/_pipeline/_vocab.py` by hand"), and an earlier draft of this bullet said it did: the tag read has nothing to keep in sync, but `_render._INITIAL` was deleted and then restored in the same PR, the fallback in the bullet below giving it a reader again. The obligation is narrower and sharper than before — the two copies no longer answer the same question about the same token, so what has to hold is that the fallback answers as `_vocab` would — and test_regex_sync is what holds it. MEASURED, re-derived on this branch rather than taken from the issue: 0 of the 1094 corpus names move under `capitalized()` or `capitalized(force=True)`, and 0 move with each name also re-run uppercased and lowercased (6564 name/variant/lexicon rows = 1094 x 3 spellings x 2 lexicons, the second lexicon being `Lexicon.default().add(particles={'y'})`, and 13128 `capitalized` calls over them); the R5 force-versus-case counts recorded above hold — 62 upper / 16 lower through the facade, 63 upper / 18 lower through the core. Movement EXISTS outside that population, and the issue's "no output moves" is a claim about the corpus only: the old predicate re-decided per word, so `juan e-f smith` capitalized to `Juan e-F Smith` — the Italian conjunction inside a hyphenated middle name — and now gives `Juan E-F Smith`, agreeing with the uppercase spelling it always gave. An earlier draft of this change moved untagged tokens the same way — a family `replace()` splices in as `de y` repairing to `de Y` — and that was a REGRESSION, caught in review before it shipped and fixed in the bullet below; read this bullet with that one. NOT DONE, and the parenthetical in the boundary bullet above should be read as still open: the PARTICLE conjunct still keys on lexicon membership, so `replace()` does not agree there. Making it read the `particle` tag is a second change with its own blast radius — it moves a boundary rules/R4 states in prose — and #458 as filed does not ask for it. + +- 2026-08-29 (#458 review) — THE FALLBACK FOR TEXT THE PARSE NEVER SAW, and why it is scoped to one of the two questions `_cap_word` asks. Reading the tag and nothing else regressed every ASSIGNED field holding a conjunction, because a value `replace()` splices in is never classified and so carries no tag to read. Measured on the released 1.4.0 and 2.1.0 wheels, all agreeing and all broken by the tag-only reading: `h.last = 'velasquez y garcia'` then `capitalize(force=True)` gives `John Velasquez y Garcia`, `h.middle = 'e.'` gives `John E. Smith`, `h.last = 'smith-y'` gives `John Smith-y`, `h.first = 'y'` gives `y Smith`. That is the canonical Spanish surname and the shape `y` is in the vocabulary FOR, so an Accepted clause was not available: it had to be fixed. RULE: a token carrying `UNCLASSIFIED_TAG` was never read, so there is no decision to honor and the view falls back to the vocabulary. Note what the discriminator is NOT, in two directions, the second of which took a second review round to find. It is not UNTAGGEDNESS: an ordinary parsed name word carries no tags either (`velasquez` parses to `tags=[]`), so that would have swept in the whole parsed corpus. And it is not the absent SPAN, which is what the first fix used. Span-less means SYNTHETIC, a wider set than unclassified: `Parser.revise()` builds span-less tokens too, from a full sub-parse whose tags it keeps deliberately, and its docstring promises the tag-driven views "behave as if the text had been parsed" — so keying the fallback on the span overrode exactly the tags `revise()` exists to preserve. Measured on the branch that did it: `revise(name, middle='e-f').capitalized(force=True)` gave `e-F` where `parse('john e-f smith')` gave `E-F`, and under `Parser(lexicon=Lexicon())` a revised family `de la vega` initialed `j. v.` where the parse initialed `j. d. l. v.`. So the mark is stamped by the two producers of genuinely unclassified text — `ParsedName.replace()`, and the facade's v1 pickle load, which rebuilds a name from `*_list` strings and no tags — and a HAND-BUILT span-less token is not marked, so it takes the tag path like everything else in the library. That is the tag-driven semantics every other view already has, and it is a change from the span reading in exactly one place: a hand-built span-less token whose text is CONJUNCTION vocabulary now capitalizes rather than lowercasing (measured: a hand-built family `velasquez y garcia` repairs to `Velasquez Y Garcia` untagged, `Velasquez y Garcia` with either the `conjunction` tag or the mark). The PARTICLE conjunct is unaffected either way — it keys on lexicon membership rather than on the mark, which is the open item this bullet's neighbours record — so a hand-built `de la Vega` is unchanged. SCOPED TO THE PER-WORD QUESTION, and this is the durable half. *Is this word a conjunction rather than an initial* is a property of the word and its spelling: an unread token has no ambiguity the parse would have resolved differently, so the vocabulary gives the answer the parser would have given, v1's initial carve-out included — which is why `_render._INITIAL` came back and why test_regex_sync's two-copy assertion matters MORE now, the fallback being right only while it answers as `_vocab` would. *Is this particle acting as a particle here* is a property of the whole PART, and answering it needs a reading on every word of the part, which is exactly what a spliced field lacks. `_cap_word` is handed the whole token's tags — it is not blind to the part because it walks one word, and an earlier draft of this bullet said it was — but with no tag on any word of the part there is nothing to re-derive the part answer FROM, so the fallback here is the word question alone and the particle half falls through to plain particle treatment. Measured cost of the wide reading IN CASE REPAIR, so nobody re-derives it: `replace(family='de la')` gives `de la` today and a parse gives `De La`; widening case repair's fallback would make them agree and would thereby REVERSE the replace/revise boundary recorded two bullets above, with `revise()` as its documented crossing. Do not widen it without reopening the decision. Verified after the fix: `replace(family='velasquez y garcia').capitalized(force=True)` is `Velasquez y Garcia`, `replace(family='de la').capitalized(force=True)` is `de la`, and the parsed and `revise()` paths are unchanged. + +- 2026-08-29 (#458 review, then the review of PR #463) — the SAME FALLBACK in `initials()`: WRITTEN, MEASURED AND DROPPED before merge. The defect it aimed at is real and older than #458 — `replace(family='velasquez y garcia').initials()` gives `j. v. y. g.` where the parse, the facade and 1.4.0 all give `j. v. g.` (the facade and 1.4.0 measured through the TWIN of that splice — `h = HumanName('juan smith')` then `h.last = 'velasquez y garcia'`, the assigned construction this bullet is about, keeping `first`), and `replace(family='de la vega')` gives `j. d. l. v.` for `j. v.` — and it is unfixed again. What killed the fix is the parameter list: `initials()` takes no lexicon, so the fallback had to reach for `Lexicon.default()` and GUESS. `capitalized(lexicon=...)` is handed the caller's vocabulary and so guesses nothing, which is why its fallback stays. Measured cost of the guess, and the reason a merely-imperfect fallback was not good enough: under `Parser(lexicon=Lexicon.default().add(particles={'y'}))`, `parse('Juan de y').initials()` is `J. d. y.` and `parse('Juan Perez').replace(family='de y').initials()` was `J. d. y.` before the fallback and `J.` after it — an entire field's initials gone, because the default vocabulary reads `de y` as particles-plus-conjunction where the caller's reads it as an all-particle part. Guessing the wrong vocabulary for a name parsed under a custom one is worse than answering from tags alone, and `initials()` cannot be told better. The missing crossing is the whole of it: `Parser` carries `capitalized`, `matches` and `revise` as its custom-lexicon crossings and no `initials`, so there is no supported way to hand this view the parse's vocabulary. #464 is filed for that; when it lands, the fallback becomes answerable rather than a guess and this decision is worth reopening. ACCEPTED until then, and stated so a reader does not re-derive it as a bug: `replace(family='de la vega').initials()` is `j. d. l. v.` where the facade and a parse both give `j. v.`. The reach is the v2 core only — `HumanName.initials_list()` computes from the field STRINGS with its own lexicon (`_facade._initials_lists`) and gives `['j', 'v', 'g']` at 1.4.0, 2.1.0 and here, under that same assignment. That is why the facade cannot be used as evidence that the core is fine. Do NOT read `_initials_lists` as a v1 parity REFERENCE, though, which is how an earlier wording of this bullet labelled it: #462 is open on a parity break in that exact path, and it is a break inside the facade rather than upstream of it. `HumanName('Scott E. Werner')` partitions to `first='Scott'`, `middle='E.'`, `last='Werner'` at 1.4.0, at 2.1.0 and here — byte-identical fields — and `initials()` on them is `S. E. W.` at 1.4.0 and `S. W.` at 2.1.0 and here, the facade reading the middle `E` as the conjunction; assigning those three fields by hand reproduces each version's answer, so no parse difference is involved. The parity label was therefore doing work its own measurement cannot support, and only the per-name measurements are load-bearing here. MEASURE THIS BULLET WITH THE ASSIGNED CONSTRUCTION, and name the construction beside each value: the review of PR #463 measured `HumanName('juan velasquez y garcia')` — the PARSED form — against sentences about the assigned one, and 'corrected' two true claims into false ones. The two constructions agree from 2.0 on and DISAGREE at 1.4.0, where the parsed form gives `['j', 'v g']` and `j. v g.`. The `last` string is byte-identical either way; `initials_list()` reads `last_list`, which the 1.4.0 parse leaves as `['velasquez y garcia']` — the conjunction run joined into one element — where assignment re-splits it to `['velasquez', 'y', 'garcia']`. So `computes from the field STRINGS` is exact for 2.x and true of the ASSIGNED form only at 1.4.0. Two things learned in the attempt are worth keeping though the code is gone. `_types._remarked` is not the bug and needs no change: it recomputes the mark from TAGS after every edit, which is right for a record of what the parse decided and simply silent about text nobody classified. And any future fallback must be scoped per ROLE, not per name — a role the parse classified whole is decided by its tags however the other roles were built — a distinction that passes every other test in the suite and needs a hand-built name to witness. + +- 2026-08-29 — WHY THE BOUNDARY WENT UNNOTICED UNTIL #407, which is where a future reader should look for it. For an ALL-PARTICLE part the other three tag-driven views give the same answer through `replace()` and `revise()` alike: measured over `de la`, `van der`, `do`, `de` and `van de la`, all five agree on `family_particles=''`, on a `family_base` holding the whole part, and on initials from every word. They converge because an UNTAGGED part and a MARKED all-particle part reach the same place by different routes — untagged, no word is recognized as a particle; marked, none is ACTING as one — and all three views only ask which words are particles. Case repair is the one view that asks a second question, since it must also decide whether to lowercase, so it is where the two routes first come apart. The mirror case confirms the reading: on a MIXED part the convergence is the other way round — `de la vega` and `van der berg` diverge in all three views between `replace()` and `revise()` (`replace()` reports particles `''` and base `'de la vega'` where `revise()` reports `'de la'` and `'vega'`) and AGREE on case repair, R4's all-particle clause not reaching them. So before #407 the distinction was invisible on exactly the shape the clause is about, and visible only on shapes the clause does not govern. + +- 2026-09-13 #459 (landed 2026-09-22, PR #521) — DECIDED: a credential acronym the exceptions map does not carry is an initialism, so a single-case word the parse put in the SUFFIX role from `suffix_acronyms` repairs to its all-caps spelling rather than a title-cased one (the clause in `_render._cap_word`, after the exceptions-map lookup and before the Mac/Mc rule — the order is load-bearing, since `mcse` is in the acronym set and matches the Mac/Mc shape, and reads `MCSE` only because the acronym clause is asked first; rules/R4 states the precedence and pins it with a row). The scope is narrow on purpose. The exceptions map is consulted first, so its five entries — the spellings `str.capitalize()` gets wrong, `md` → M.D. and `phd` → Ph.D. because they are punctuated, and `ii`/`iii`/`iv` → II/III/IV because a numeral is written all-caps and they are `suffix_words` the acronym clause would never see — keep theirs and the clause never touches them. The repair is gated on the SUFFIX role, so a word that is in the acronym vocabulary but parsed as an ordinary name word (`anh van do` → `Anh Van Do`) still repairs as that name word -- the gate is the whole reason the fix is safe on surnames that share a spelling with a credential. Because it is the ROLE and not a parse tag, the gate also reaches a suffix spliced in as raw text: `parse("john smith").replace(suffix="mba").capitalized()` gives `MBA` like the parsed name does, which is the contrast R4's Accepted paragraph draws against the all-particle clause. The same PR removed `ph` from `suffix_acronyms`; see `Excluded (SUFFIX_ACRONYMS — ph ...)`, beside the esq block, for why. The wider design the issue proposed (letter masks, `md` leaving the map, the given-role half of `QC MP`) stays on the rescoped #459; this clause is the narrow part #459 already accepts, not a re-litigation of those. +Reach, population first, measured 2026-09-22 on the PR branch merged with master at 23e52dc6, over the deduped corpus glob (1338 distinct names): 178 names carry acronym vocabulary in a suffix token and 30 of those are written in a single case, and 22 move on the default `capitalized()` path, 119 under `force=True`. Recompute by exec'ing the pre-change `_cap_word` -- `git show 23e52dc6:nameparser/_render.py`, the master this branch merged, which carries #397/#461's generation gate but not this clause -- into `nameparser._render`'s namespace and swapping it in for the changed one in a single process, then diffing `str(capitalized(parse(n), None, force=...))` over the glob. The digits have moved twice and neither move is the clause's: the PR's first head (2026-09-11, `ph` still in the set, 1143 names) measured 5 default and 71 forced; the commit that dropped `ph` took the 13 `PH. D.` forced movers back to `Ph. D.` and left 111 in the population, 5 default and 58 forced; and the merge brought in the corpus rows of PR #530 (#289/#516), PR #532 (#531) and PR #534 (#533), whose single-case trailing `MA` credentials (`DOE, JOHN MA`, `jane doe nee smith ma`, `JOHN SMITH, MA`, the CJK `田中 太郎, MA` family; their mixed-case twins hold still under R5) account for 15 of the 17 further default movers; the other two are `STEVEN HARDMAN, MD, DO, DDS` and `john smith mcse`, the row this PR's own rules.md amendment adds (`McSe` under the pre-change function, which is the precedence claim above measured). The ratio is the stable claim: every single-case name with an acronym suffix the map does not carry moves, and nothing else does. +Accepted costs, deferred to the rescoped #459 rather than relitigated here: the all-caps default reaches words conventionally written mixed-case -- `bsc`/`msc` read `BSC`/`MSC` under `force=True`, and `Dr. med. univ. Margit Popp, MSc` is a corpus name that reads `MSC` under `force=True` only, its mixed case holding it back on the default path under R5 -- which the letter-mask design #459 defers is meant to recover; `ii`/`iii`/`iv` are `suffix_words` rather than acronyms and need the exceptions map precisely because the clause would not see them there; and because the ambiguous five (`ba`, `do`, `ed`, `jd`, `ma`) are in `suffix_acronyms`, the clause moves WHICH parse triggers the repair rather than preventing it -- `john smith ed` → `John Smith ED`, `john smith ba` → `BA`, and `smith, ms.` → `MS.` on the default path. That is the same #342/#454-class cost #459 already accepts, and the alternative (reading classify's `vocab:suffix` tag) was measured and costs `jd` → `Jd`, because `jd` carries `vocab:suffix-ambiguous` and not `vocab:suffix`, so the role gate is the right instrument. + +- 2026-09-20 #461/#397 — THE GROUNDING MOVED, AND THEN A DEFECT THE GROUNDING EXPOSED. R4's sentence rested on R3 by name ("the carve-out R3 states for initials"), and R3's carve-out is conditional now while R4's is not, so the cross-reference is CUT and replaced by this rule's own reason: a connective that initials because it joins nothing is still not written the way a name is written. Measured 2026-09-20, plain and forced, over `juan y`, `john and jane smith`, `duke of edinburgh`, `juan de y`, `juan y garcia`, `JUAN Y GARCIA`, `josep carod i rovira`, `JOSEP CAROD I ROVIRA` and — under `add(particles={"y"})` — `Anh y Van`: all nine are byte-identical to the same call at the parent, re-measured after the repair below and still byte-identical, and `tests/test_capitalization.py` is green. + WHAT THE PLAIN CALL DOES, stated over a population rather than over nine names, because "capitalized() does not move" was drafted for this bullet and is false as written. Swept 2026-09-20 over the 1558 non-empty corpus-union-cases names under eight configurations against the parent commit, `capitalized()` moves on SIX distinct names and every one of them is a name whose ROLES moved under the join — it renders different fields rather than treating a word differently. The correct statement is therefore: where only `initials()` moves, `capitalized()` is byte-identical; where the roles move, it follows them. + WHAT THE FORCED CALL DID, AND THE DEFECT THAT WAS FOUND WRITING THIS RECORD. The sweep above was run for the PLAIN claim and turned up a second answer: `capitalized(force=True)` moved on ten further names whose roles, initials and plain repair all stayed put, every one a lower-case `i` — `parse("Carod i")` forced gave "Carod i" where 1.4.0 and 2.3.0 gave "Carod I", and so did "John Quincy Smith i", "Josep Carod i", "Lluis Carod i", "Josep Lluis Carod i III", "Josep Lluis Carod i V", "Carod y de Rovira i", "Rovira, Josep Carod i Jr.", "Carod i Rovira" and "Josep i Rovira". Nine of those were a DEFECT and are REPAIRED here; the remaining two are the rule. + WHY IT WAS A DEFECT, and the line R4 now draws. In those names the parse read the letter as the GENERATION it also spells and roled the token SUFFIX — but classify had tagged it `conjunction`, and repair's conjunction conjunct read the tag without asking the role, so a generation was lower-cased as a connective. Derek's principle decides it: a connective with nothing to join is not acting as a connective. Repair now asks the ROLE the parse decided rather than the tag alone, and a suffix-roled token is repaired as the suffix it was read as. `parse("John Quincy Smith i")` forced gives "John Quincy Smith I" again on BOTH surfaces, the facade repairing through the same helper. + WHAT REMAINS AND IS NOT A DEFECT: `Carod i Rovira` and `Josep i Rovira`, where the letter is a lone MIDDLE word the parse tagged `conjunction` — three-word names, so P3's carve-out leaves the letter a name word and nothing joined. Forced repair keeps it lower-case, which is R4's own rule and exactly the answer the same shape has always had for `y`: `Carod y Rovira` and `Josep y Rovira` give "Carod y Rovira" and "Josep y Rovira" too. Those two names are the whole of what #397 moves on this view. + AND THEN THE REPAIR ITSELF WAS TOO WIDE, found by the second review round (2026-09-20). The guard shipped as the suffix ROLE alone, and the role says only where a word LANDED. A plain connective can land in the suffix field without being generational vocabulary at all — the third part of a comma form is read as the suffix run whatever its words are — so `parse("Smith, John, and").capitalized(force=True)` gave 'John Smith And' where 1.4.0, 2.0.0 through 2.3.0 and the parent 46651750 all give 'John Smith and', and the same for `, y`, `, e`, `, und`, `, of` and `Doe, Jane, and Jr.`. A field SPLICED in went the same way and against R4's own Accepted paragraph, which says a field nobody read asks the vocabulary: `HumanName(first="John", last="Smith", suffix="de y")` repaired forced to 'John Smith De, Y' against 'John Smith De, y' on both wheels. THE GUARD IS NOW THE PAIR — the suffix role AND the `vocab:suffix` tag — because the clause turns on the GENERATION and the role alone cannot see it; classify's tag is the vocabulary half of the decision and reading both re-derives neither. Invisible to everything already watching: no name of the off-switch grid carries the shape (its oracle is a letter of the class, and none of these words is one) and no corpus name carries it either, which is why INV7b was written over a grid of its own. ACCEPTED, and the one reading this moves rather than restores: a field spliced to `suffix="i"` now repairs to 'John Smith i' where the parent gave 'John Smith I'. `i` is connective vocabulary on this branch and was not at the parent, so the spliced field follows `y` now — which is R4's Accepted paragraph working as written, the PARSED name being unaffected because it has a reading and the reading is the generation. + MEASURED, and the numbers are the fix's own control. Over the same eight configurations the forced-only set is ten names before the fix and two after; the invariant `tests/v2/test_properties.py::test_a_letter_that_did_not_join_repairs_as_the_off_switch_does` — a letter that became no connective of this name repairs as the off-switch parse repairs it, plain and forced, on both surfaces — fails on 7,860 repairs at `e540d4c5` (6,843 core, 1,017 v1) and on 0 here, and removing the generation conjunct outright fails it on 8,054. (11,341 / 10,424 / 917 was the same control over the full cross product before the grid was trimmed by shape and then grown by the second review; it is a dated snapshot and the test's docstring is the live figure.) The differential still cannot see any of it, case being no compared surface (the 2026-08-29 bullets above settle why), which is why this needed an invariant rather than a ledger rule. + +- 2026-09-23 #459 — DECIDED (Derek, 2026-09-23): the exceptions map holds case MASKS, and nothing else. A value is its key's own letters and digits, each in the case it takes, and case repair lays it over the word as the writer punctuated it, keeping every other character where the writer put it — so one `phd` entry repairs `phd` to `PhD`, `ph.d.` to `Ph.D.` and `PH.D.` to `Ph.D.`, and the map can hold a conventionally mixed-case acronym (`BSc`, `MSc`) without a dotted spelling per source. A replacement string is substitution, not capitalization: it is how `md` became `M.D.`, how `iii.` lost its period, and why the map never grew past the five entries it was imported with in 2011. The shipped map is `phd` → `PhD`, `bsc` → `BSc`, `msc` → `MSc`; `md`, `ii`, `iii` and `iv` left it, a suffix `md` now repairing by the acronym clause and a suffix numeral by the numeral clause (both below). Lookup is unchanged — the edge-stripped key, then the period-free one — and the map stays ROLE-FREE: an entry is the caller saying how a word is written wherever it stands (`phd smith` gives `PhD Smith`). rules/R4 now states the invariant every clause serves — repair changes case and nothing else — and `tests/v2/test_properties.py::test_case_repair_changes_case_and_nothing_else` holds it over every corpus name and case-table text, plain and forced, on both surfaces, under the shipped lexicon and one with the map emptied, comparing under `str.casefold()` (the Unicode-boundary bullet below says why that comparator). `md` → `MD` and `phd` → `PhD` are DELIBERATE DEVIATIONS from every release, 1.4.0 through 2.3.0, all of which gave the dotted forms (measured 2026-09-23 on all five wheels). There is no contributor intent behind the period insertion to preserve: the map, its `md` entry and `capitalize()` itself arrived together in the 2011 svn import (45a15394), authored by Derek. + +- 2026-09-23 #459 — THE CHECK, and two places it departs from the approved spec, both by measurement. A value that does not spell its key's letters and digits is a `ValueError` at `Lexicon` construction (`_lexicon._normpairs`). Both sides are folded through `_normalize` and compared as ALPHANUMERICS (`_lexicon._alnum`), so `('2nd', '2ND')` is accepted and `('2nd', '3ND')` raises — a comparison of letters alone would have let a digit key take any other digit. The message names the entry and offers the key at hand recased — `capitalization_exceptions=(('jr', 'JR'),)` for a `jr` key — a tuple of pairs, because the field is annotated with what it stores and a dict literal there would be an arg-type error in the pasted code; where `.upper()` changes the alphanumeric count (`straße` → `STRASSE`) it offers the key unchanged instead, an identity mask, so the offered code always constructs. Every entry point reaches it: construction, `dataclasses.replace()`, `|`, unpickling (`__setstate__` re-runs `__post_init__`), and the v1 shim's snapshot; `tests/v2/test_lexicon.py` sweeps all five. FIRST departure: the spec compared the value against the KEY as "already normalized, period-free and lowercase" — it is not period-free, `_normalize` stripping only EDGE full stops, so `Ph.D.` is stored `ph.d` and, compared raw, a key written with an interior period could never validate. SECOND: the spec's "the two letter sequences are always equal in length" is false. The key is NFC-composed and the word is not, so a word written in decomposed hangul (`씨` as two jamo) finds a one-letter key. The applier, `_render._apply_mask`, declines there (it returns None) and `_cap_word` falls through to its next clause — a caseless script has no case to recase, so nothing is lost — pinned by `tests/v2/test_render.py::test_the_mask_keeps_every_non_letter_and_declines_a_miscount`. Without the decline the naive applier raised `IndexError` inside `capitalized()` (measured in review on 2026-09-23, on the prototype and again by the spec review). + +- 2026-09-23 #459 — A DECIDED EXCEPTION to the shim's never-raise rule (AGENTS.md: "The shim TRANSLATES; it never raises on a config v1 accepted"). A v1 `Constants` whose `capitalization_exceptions` carries a value that does not spell its key — `{'jr': 'Junior'}` — raises the `ValueError` above at `Constants._snapshot()`, i.e. at the first parse. Taken rather than translated because there is nothing to translate TO: 1.4.0 substituted such a value for the word, 2.4's repair only recases, and so the only v1-faithful reading is the one this change removes. No such value was found in the repository, the docs, the issue tracker or any test when the design was approved; the hypothesis fuzzer drew arbitrary pairs and now draws recasings of its key. A value that merely carries PUNCTUATION is not one of these: it is a legal mask (the joins bullet below), so a v1 `Constants` unpickled from 1.4.0, still carrying `md` → `M.D.` and `phd` → `Ph.D.`, loads and parses — measured 2026-09-23, `john smith md` giving `John Smith MD` under it. Pinned by `tests/test_capitalization.py::test_a_mismatched_exception_value_raises_at_the_first_parse`; recorded in AGENTS.md's shim bullet and cross-referenced from #3-0-reevaluations. + +- 2026-09-23 #459 (review) — A MASK'S PUNCTUATION MARKS WHERE ITS LETTERS ARE JOINED, and is never written into the word. Letters written side by side in a value are one RUN, and any character between them that is not a letter — a full stop, a space, a digit — ends the run. That matters for exactly one case, the LONE INITIAL: a single letter the writer split off beside a full stop (a `FULL_STOPS` member, not the ASCII period alone, #322) is written in capitals whatever the mask says there, but only where the mask writes that letter inside a run of two or more — the writer split a chunk the mask keeps together, so each split piece is an initial. So `p.h.d.` under `PhD` gives `P.H.D.` (the mask is ONE three-letter run), `b.sc.` under `BSc` gives `B.Sc.`, `ph.d.` gives `Ph.D.`, and `d.phil.` under a caller's `DPhil` gives `D.Phil.`; while `h.c.` under a caller's own `h.c` mask, which spells each letter alone, stays `h.c.`, `a.bc` under `abc` gives `A.bc`, `ab.c` gives `ab.C`, `2b.` under `2b` stays `2b.`, and `x2b.c` under `x2bc` gives `x2B.C`. Everywhere else a value's punctuation changes nothing: `('md', 'M.D.')` and `('md', 'MD')` repair every word alike (measured on `md`, `m.d.`, `md smith` and `m.d`). The rule took three drafts, and the two wrong ones are the record worth keeping. The FIRST laid the mask letter by letter with no initial rule and REGRESSED what substitution had got right: `b.s.c.` repaired to `B.S.c.` and `p.h.d.` to `P.h.D.`, where the parent 4d0680e6 gave `B.S.C.` and `Ph.D.`. The SECOND capitalized every lone letter beside a stop, which overrode a caller's own mask — measured in review on 2026-09-23, `_apply_mask('h.c.', 'h.c')` gave `'H.C.'` and `dr. h.c. hans meier` forced gave `Dr. H.C. Hans Meier`, where the caller had written the case they wanted. The third reads the mask's runs, as above, and is rules/R4's `p.h.d.` line. CASING goes through the whole word — `word.lower()`/`word.upper()` — wherever both keep the word's length, so context-sensitive letters come out as the language writes them (`ΚΟΣ` under `Κος` gives `Κος`, the final sigma intact), and per character otherwise (`straße` under `STRAßE` gives `STRAßE`). KNOWN LIMIT: a mask letter reads as upper when it is not lower, so a titlecase letter (`Dž`, Unicode category Lt) reads as upper and `dž` under `Dž` gives `DŽ`; no per-character titlecase mapping is attempted. Pinned in tests/v2/test_render.py by `test_a_split_initial_is_capitalized_only_where_the_mask_keeps_it_joined`, `test_a_masks_punctuation_marks_its_joins_and_is_never_written` and `test_a_mask_cases_through_the_whole_word_for_context_sensitive_letters`. + +- 2026-09-23 #459 (review) — DECLINED, after being written: a construction-time `UserWarning` for a punctuated value, and a shim translation of the two pre-2.4 shipped pairs. Both were implemented during review and then dropped, because both rested on a premise the code falsified — that a value's punctuation is IGNORED. The warning fired on a value whose punctuation differed from its key's (`'M.D.'` for `md`, `'h.c'` for `hc`) and told the caller to write the value without it (`Write the value as 'hc'`); but `_render._letter_run_ge2` READS a mask's punctuation, so following the advice changed the output — `('hc', 'h.c')` keeps `dr. h.c. hans meier` at `h.c.` under force, where `('hc', 'hc')` gives `H.C.`. The translation rewrote `md` → `M.D.` and `phd` → `Ph.D.`, byte-equal, to their unpunctuated masks in `Constants._snapshot()`; it changed no output (those pairs repair every word alike, above) and existed only so a v1 `Constants` still carrying them — one unpickled from 1.4.0, for instance — would not trip the warning, so with the warning gone it had nothing left to do. A punctuated value is simply a legal mask, accepted silently at every entry point (`tests/v2/test_lexicon.py::test_a_value_may_carry_punctuation_that_marks_its_joins`). The lesson generalizes: "this input is silently ignored" is a MECHANISM claim about what the code reads, and it earns the same one-command check as any other before a diagnostic is built on it. + +- 2026-09-23 #459 (review) — THE INVARIANT'S UNICODE BOUNDARY, and why rules/R4 words it as it does. The approved spec stated the invariant as "no character is added, removed or reordered"; rules/R4 says "the repaired word is the word as written, recased", because two inherited cases break the stronger reading and neither is this change's to fix. Python's casing tables uppercase `ß` to `SS`, so a clause that uppercases can lengthen a word: `john smith ß.x.` reads a by-shape credential and repairs to `John Smith SS.X.`, casefold-equal and one character longer (the parent 4d0680e6 gave `Ss.x.`, longer too). And the roman shape matches under `re.I`, which admits the dotless `ı`, so `john smith ıı` reads suffix and repairs to `John Smith II` — not even casefold-equal to its input, `'ıı'.casefold()` being `ıı` (the parent gave `Iı`). The first comes from the casing tables and the second from the parser's own pattern; neither is fixed. `tests/v2/test_properties.py::test_case_repair_changes_case_and_nothing_else` compares under `str.casefold()` because of the first, and that comparator has a boundary on each side: it CANNOT see a repair that trades `ss` for `ß` or back, the two folding alike — a recorded blind spot — and it WOULD report the second, which it does not today only because no corpus name or case-table text spells a numeral with `ı`. A length check beside it was considered and declined: it would flag the `ß` boundary this bullet records as accepted. + +- 2026-09-23 #459 item 4 — DECIDED (Derek, 2026-09-23): a word the parse put in a NAME role repairs as a name word, whatever vocabulary it is in. `qc mp` gives `Qc MP`, `ed smith` gives `Ed Smith`, `john smith ed` gives `John Smith ED`: the parse decides the role and reports the fork (`suffix-or-name`), and repair follows the role (mechanisms.md#RENDER-HONORS-THE-PARSE). Closes the item with no code. What the decision also reaches, found while planning and NOT in the approved spec, which said `iv smith` would keep `IV Smith` "through the numeral rule": the numeral clause is suffix-gated — gating it is what keeps the given name `Vi` from reading `VI` — so the four words that left the map lose, in any role but the suffix, the role-free capitals the map gave them. Measured 2026-09-23: `iv smith` gives `Iv Smith` and `john iii smith` gives `John Iii Smith`, where every release from 1.4.0 through 2.3.0 gave `IV Smith` and `John III Smith`; `II Van Johnson` gives `Ii van Johnson` under force; and `md` read as a TITLE — the parse's reading of a leading `MD` — repairs as ordinary title text, so `MD DDS` gives `Md Dds` and `Md Abdul Karim`, the usual abbreviation of Mohammed and a shape-tagged corpus row, gives `Md Abdul Karim` under force where every release gave `M.D. Abdul Karim`. DECIDED (Derek, 2026-09-23, when the plan surfaced it): a numeral in a name role repairs as a name word, the same rule read to its end. DECLINED, the alternative put to him with it: keeping `ii`, `iii` and `iv` in the map as masks, which would have restored the three numerals in a name role while leaving a title `md` at `Md`. + +- 2026-09-23 — PERIOD INSERTION IS OUT of `capitalized()` for good, and PARKED rather than declined: if anyone ever asks for a canonical dotted spelling (`md` → `M.D.`), it belongs in a normalization view — Derek's working name for it is `repair()` — which may re-spell a word, not in a case repair, which may not. Not a commitment; no issue filed. + +- 2026-09-23 #459 — NUMERALS: a SUFFIX-roled word whose text, edge periods stripped, fullmatches the roman shape repairs to capitals. It reaches `i` through `x`, which is why `ii`/`iii`/`iv` left the map: `vi` through `ix` read as suffix by that shape, carry no vocabulary tag, and title-cased to `Vi`, `Vii`, `Viii`, `Ix` (measured on 4d0680e6, and `Vi`, `Vii` on every release wheel 1.4.0 through 2.3.0); and `john smith iii.` now keeps its period, `III.`, where every release gave `III`. The shape is `_vocab._ROMAN`, v1's `roman_numeral`, and `_render` holds a HAND COPY pinned against config by `tests/v2/test_regex_sync.py`: the approved spec said the clause would import the predicate, but tests/v2/test_layering.py's ALLOWED table gives `_render.py` `_types` and `_lexicon` and nothing else, and `_render._INITIAL` is the precedent for exactly this. The copy re-derives no parse decision — the role is the parse's, read as recorded; the pattern answers only how a numeral is written. THE PARSE LIMIT beside it, recorded and not fixed here: the roman shape stops at `x`, so `john smith xi` reads family `xi` and repairs to `John Smith Xi`, and after a family comma `john smith, vi` reads given `vi` and repairs to `Vi John Smith` — both rules/R4 boundary lines, and both for #459's close-out. The #397 generation guard is unchanged and still decides first: `Carod i` forced gives `Carod I`, the numeral clause now being what writes the capital. + +- 2026-09-23 #459 — A SUFFIX READ BY ITS DOTTED SHAPE is an acronym to repair: the acronym clause fires on a SUFFIX-roled token whose period-free text is listed OR which carries `SHAPE_ACRONYM_TAG` (#516's by-shape reading, rules/S3), so `john smith x.y.z.` repairs to `John Smith X.Y.Z.` where the parent gave `X.y.z.`. The constant moved from `_pipeline/_state.py` to `_types.py`, which `_render` may import, and `_state` imports it back — still one spelling, which is what the constant exists for. The role gate holds: `Jack X.Y.Z.` keeps its surname and repairs as a name word, `Jack X.y.z.` under force. ACCEPTED: an unlisted dotted credential written in mixed case reads all-caps wherever repair acts, since a by-shape member has no listed casing to honor — `John Smith B.Tech.` gives `John Smith B.TECH.` under force, where every release gave `B.tech.`, no better. And since the suffix-free gate (decisions/R5, 2026-09-23) this reaches the DEFAULT path too wherever the name words are one-case: `john smith B.Tech.` gives `John Smith B.TECH.` where every release returned it untouched, while the corpus spelling `John Smith B.Tech.` stays untouched there, its name words being mixed. A caller who writes such a credential adds a mask: `('btech', 'BTech')` gives `B.Tech.` on either spelling (measured 2026-09-23). + +- 2026-09-23 #478 — DECIDED: inside `_cap_text`, a part of a HYPHENATED token that is connective vocabulary keeps its lowercase when a WORDED part stands on each side of it within the token (split on `-`; a part counts only if it holds a word, and an empty part does not count and is skipped, so a trailing hyphen supplies no neighbour while a doubled hyphen changes nothing: `juan garcia--y-lopez` gives `Juan Garcia--y-Lopez`). An edge part repairs as ordinary name text. `jose ortega-y-gasset` and `JOSE ORTEGA-Y-GASSET` both give `Jose Ortega-y-Gasset`, and `maria silva-e-sousa` gives `Maria Silva-e-Sousa`. It reads POSITION and VOCABULARY, as the particle arm already does per sub-word (`maria da-silva` → `Maria da-Silva`, unchanged) — a question about a sub-word the parse never asked, the parse having read the hyphenated word as one untagged token, which is the same standing mechanisms.md#RENDER-HONORS-THE-PARSE records for the particle arm — and it does not re-derive the initial-versus-conjunction class from a word's case, which is what #458 removed. The edges are pinned on both sides: `juan e-f smith` keeps `Juan E-F Smith` and `juan y-garcia` gives `Juan Y-Garcia`, and in a three-part word `juan y-garcia-lopez` gives `Juan Y-Garcia-Lopez` and `juan garcia-lopez-y` gives `Juan Garcia-Lopez-Y`. A SINGLE LETTER MARKED WITH A PERIOD is an initial there, as the parse reads it in spaced text, never the connective (the guard is `not re.fullmatch(r"\w\.", part)`): the first draft lowered the `e.` in `j.-e.-p. dupont`, giving `J.-e.-P. Dupont` where every release from 1.4.0 through 2.3.0 gave `J.-E.-P. Dupont`, a regression found in review and pinned since as a rules/R4 boundary line. A MULTI-letter word marked with a period is not an initial and stays reachable, as its spaced reading is (`hans smith und. jones` tags `und.` a conjunction), so `hans smith-und.-jones` gives `Hans Smith-und.-Jones`. This reverses, for the interior position only, what #458's tag read did to the shape: 2.2.0 and 2.3.0 gave `Ortega-Y-Gasset` for both spellings — the side effect #478 reported, which the 2026-08-29 #458 bullet above does not record — and #458's edge answer stands. Parity, measured on the released wheels: 1.4.0, 2.0.0 and 2.1.0 re-decided per word and gave `Jose Ortega-y-Gasset` lowered but `Jose Ortega-Y-Gasset` upper, so the all-caps half is a parity break with them on #479's one-case precedent; they also gave `Juan y-Garcia` at the edge, which neither 2.2.0, 2.3.0 nor this tree does. KNOWN NON-REACH, with its witness: the Māori corpus name `Te Awanui-a-Rangi Black` still repairs to `Te Awanui-A-Rangi Black` under force, because `a` is not connective vocabulary and must not become one (a spaced `a` is an initial or a name elsewhere; the connectives' Excluded block in decisions/vocabulary-exclusions says so). ACCEPTED: `mary-e-smith` reads `Mary-e-Smith` — no evidence the shape exists, and the hyphen is the writer joining the parts. Every role, since a hyphenated given name is the same text shape; a middle-role row is pinned beside the family ones (`smith, jose ortega-y-gasset` forced gives `Jose Ortega-y-Gasset Smith`). + +- 2026-09-23 #459/#478 — MEASURED, with the recipe; none of these numbers is a live count. Population: the deduped `tools/differential/corpus*.jsonl` glob, both line shapes (an object with `name`, a bare string), as this change leaves it — 1375 distinct names, the new rules.md example lines among them (1371 before the two 2026-09-24 decisions below added four; re-measured 2026-09-24). Comparator: the parent 4d0680e6 — `git archive 4d0680e6` into a scratch directory, both trees imported BY PATH in separate processes over the SAME corpus directory, each asserting `nameparser.__file__` — comparing every token's `capitalized()` and `capitalized(force=True)` text on the core, and the rendered string on the facade, and attributing each moved token PER SUB-WORD: the token is split into its `_WORD` sub-words (`(\w|\.)+`), and the FIRST sub-word whose repair changed decides the bucket, tried in this order — its period-free key is a shipped mask key (`phd`, `bsc`, `msc`, and since 2026-09-24 the fifteen below) or one that left the map (`md`, `ii`, `iii`, `iv`); else the token is a suffix carrying `SHAPE_ACRONYM_TAG` or the sub-word fits the roman shape; else the sub-word is an interior connective part of a hyphenated token; else, on the default path only, the token's repair used to equal its input, which is R5's admission — with a bucket for anything else, asserted empty. The per-sub-word split is load-bearing: keyed on the token as written, the corpus name `John Doe, MD-PhD-` (key `md-phd`) falls to the hyphen bucket instead, giving 15 and 82, 3 and 5 over that day's 1371 names and 11 and 77, 0 and 1 over the parent's corpus (measured in review on 2026-09-23). Mask, or a word that left the map: 16 names on the default path and 84 under force over the 1375 (15 and 83 over the 1371 names before the 2026-09-24 decisions). Dotted shape and numerals: 3 and 16 over both. Hyphenated link: 4 and 5 over the 1375 (3 and 4 over the 1371), every one of them a rules/R4 example line — no corpus name written before this change carries a connective part in a hyphenated word, at the interior or an edge (0 of 1340 at 4d0680e6). R5's gate is R5's bullet. The same comparison over the parent's own corpus directory, 1340 names and none of this change's example lines, gives 11 and 78, 1 and 14, and 0 and 0 — the same before and after the two 2026-09-24 decisions, which move no name written before them: the difference is this change's own rules.md lines, and those are the counts to read as reach into names written for other reasons. The core and the facade move on the same number of names in both views. No role field moves on any name, which is also all the differential gate can say: its compared surface is the seven role fields, `_ambiguities` and `_initials`, never case (the 2026-08-29 bullet above). + +- 2026-09-23 — mechanisms.md#RENDER-HONORS-THE-PARSE CORRECTED, found in this change's re-read of that entry: it still said case repair carries the connective carve-out "ON R3's authority", quoting R4's pre-#461 text ("the carve-out R3 states for initials"), and that `.initials()` gives `j. a. j. s.` for `john and jane smith`. Both had been stale since #461 (2026-09-20), which cut R4's cross-reference to R3 in favor of R4's own reason and stopped a connective with a name word beside it from initialing. The entry now states R4's own reason, dates the cut, and names where the two views part today (`Juan de y`: initials `J. y.`, forced repair `Juan de y`), re-measured 2026-09-23. + +- 2026-09-24 #459/#492/#478 (branch review) — FOUR FINDINGS FROM THE REVIEW ROUND, recorded together because each is a small correction to a bullet already above rather than a new decision. (a) THE UNICODE BOUNDARY IS NOT SCOPED TO THE TWO SHAPE CLAUSES, AND NOT TO ONE WAY A MASK LETTER COMES OUT UPPER EITHER. The 2026-09-23 bullet above states the `ß` and dotless-`ı` boundary against `_cap_word`'s acronym/numeral clauses, but the MASK path can lengthen a word through `ß` exactly the same way. `_apply_mask` decides per letter whether it is written upper -- by the mask's own case, or by the split-off-initial force -- and then takes one casing step: the letter at that position in the whole-word `word.upper()`/`word.lower()` when both keep the word's length, otherwise a per-character `c.upper()`/`c.lower()`. `'ß'.upper()` is `'SS'`, so wherever the word holds a `ß` the whole-word form changes the length and every letter goes through `c.upper()`, which lengthens a `ß` written upper to `SS` whichever of the two reasons made it upper. Through the force: an identity mask validates fine (`('a.ß', 'aß')` spells the key's own letters unchanged), so `john a.ß smith` forced gives middle `A.SS` -- casefold-equal to its input and one character longer, the same trade the shape clauses make. Through the mask's own case, no full stop or split letter involved: `('straße', 'STRAẞE')`, the value spelling `straße`'s own letters unchanged with the actual German capital `ẞ` (not the digraph `'STRASSE'`, which the validator refuses as a VALUE for failing to spell `ß` at all), validates and `john straße` forced gives family `STRASSE`. The validator's refusal of `('straße', 'STRASSE')` is a different question -- whether the VALUE spells the KEY -- and says nothing about what the applier can produce from a mask that does validate. This same boundary is where `capitalized()`'s own "Idempotent" docstring claim narrows: forcing `A.SS` (that first pass's own output) through the SAME lexicon a second time gives `A.ss`, not `A.SS` back, because the second pass folds the word to `a.ss` and the exceptions map's key is still `a.ß` -- a different letter sequence, not merely a different case of one -- so the lookup that found the entry on the first pass misses on the second and the word falls through to plain title-casing. Pinned by `tests/v2/test_render.py::test_a_masks_upper_fallback_can_lengthen_a_word_through_ss` and `tests/v2/test_render.py::test_a_lengthening_mask_is_not_a_fixpoint_under_a_second_forced_pass`. The same boundary reaches plain title-casing with no mask involved, by two distinct mechanisms: LENGTHENING, where a letter's own upper() is a different letter SEQUENCE that stays \w -- 'ʼn' (one letter) upper-cases to the two cased characters 'ʼN', so `parse("ʼna smith").capitalized(force=True)` gives given `ʼNa`, and forcing THAT output a second time gives `ʼna` ('N' is no longer the word's first character, so `str.capitalize()` lower-cases it on the second pass) -- and SPLITTING, where a letter's own upper() is a base letter plus a COMBINING MARK that `_WORD` does not match at all -- 'ǰ' upper-cases to 'J' + a combining caron, so `parse("ǰo smith").capitalized(force=True)` gives given `J̌o`, and forcing THAT output a second time gives `J̌O` (`_cap_text` reads the output as two separate words, 'J' and 'o', the combining mark matching neither, and 'o' capitalized alone is 'O'). (b) MASK VALUES ARE STORED NFC-COMPOSED. `_normpairs` compared a value against its key NFC-composed but stored the value AS WRITTEN, so a value written in decomposed form spelled one composed letter as a base letter plus a non-alpha combining mark -- changing which letters `_letter_run_ge2` reads as split off beside a full stop from what the composed spelling of the identical value would read. `john smith pé.x.` forced gave `Pé.X.` under a composed mask and `Pé.x.` under the decomposed spelling of the SAME value. Fixed by NFC-composing the value at the same point the key is composed; pinned by `tests/v2/test_lexicon.py::test_a_decomposed_value_is_compared_and_stored_nfc_composed` and `tests/v2/test_render.py::test_a_decomposed_mask_value_reads_the_same_split_as_composed`. (c) THE HYPHEN CLAUSE'S INITIAL TEST IS NOW A PINNED MODULE CONSTANT. `_cap_text`'s inline `re.fullmatch(r"\w\.", part)` was a third hand copy of the initial shape's period half, invisible to `tests/v2/test_regex_sync.py`'s hand-copy sweep because it was never hoisted to a module-level pattern. Hoisted to `_render._DOTTED_INITIAL` and registered as `_INITIAL`'s period alternative; pinned by `tests/v2/test_regex_sync.py::test_dotted_initial_is_the_period_alternative_of_initial`. (d) THE V1 RAISE NOW CARRIES A V1-SPELLED HINT, VIA A TYPED EXCEPTION. The 2026-09-23 bullet above records that a mismatched-mask value raises `ValueError` at a v1 `Constants`' first parse; the message it raised was `Lexicon`'s own, offering a `Lexicon(capitalization_exceptions=...)` constructor call to a caller looking at a `TupleManager` assignment through `Constants`. `_lexicon._normpairs` now raises a dedicated `_MaskValueError(ValueError)` for exactly that one shape, built in the `UnicodeDecodeError` idiom: `__init__(self, key, normalized_key, value)` stores the three offending inputs and computes `self.offered` (from `_offered_mask`) and the message from them, rather than taking a pre-built message. `__reduce__` returns `(type(self), (self.key, self.normalized_key, self.value), self.__dict__)` -- reconstructing from the three fields and then restoring `self.__dict__` (so an attribute added after construction, e.g. `add_note()`, survives too) -- since the inherited default (`type(self)(*self.args)`, `self.args` holding only the message) cannot reconstruct an instance that needs three fields, not one, and raises `TypeError` trying; a worker in a `ProcessPoolExecutor` pickles an exception it raises to deliver it to the caller, so an unpicklable one there surfaces as `BrokenProcessPool` instead. `_config_shim._build_snapshot` wraps ONLY the `Lexicon(...)` construction in `try`/`except _MaskValueError` and re-raises a message built fresh from `e.key`/`e.value`/`e.offered` -- not from `str(e)`, which is the 2.0 exception's own message offering the 2.0 `capitalization_exceptions=((...),)` constructor spelling no v1 caller can paste -- with `constants.capitalization_exceptions[e.key] = e.offered`. Every OTHER `ValueError` `Lexicon(...)` can raise passes through unchanged, by TYPE rather than by a message-substring test. Of those, the three SUBSET checks (`particles_ambiguous` inside `particles`, `suffix_acronyms_ambiguous` inside `suffix_acronyms`, `honorific_tails` inside `suffix_words`) cannot fire from here: each is built as an intersection or union with its OWN base, so the normalized image of that relation is still a subset whatever the raw spelling. THE CONTRADICTION CHECK AND THE GATE-BYPASS CHECK ARE A DIFFERENT SHAPE, AND DO REACH `Lexicon(...)` -- this bullet claimed otherwise ("cannot arise ... observing no raise") until the confirmation round measured the opposite. Both compare two sets built by SEPARATE set arithmetic in `_build_snapshot`, on the RAW SetManager entries, before Lexicon's own per-field normalization ever runs: `SetManager` strips edge periods but not edge whitespace, while `Lexicon`'s normalization strips both, so an entry carrying edge whitespace can slip past the shim's own subtraction or union and still collide once Lexicon normalizes it independently. Measured: `c.suffix_not_acronyms.add('ba ')` then a parse raises Lexicon's own gate-bypass check (`an ambiguous suffix acronym must not also be a suffix word; in both: ba`), and `c.bound_first_names.add("'t ")` then a parse raises the contradiction check -- both pass through the `except _MaskValueError` clause above unchanged, being a different exception type. Pre-existing at HEAD and at master alike, not introduced by this session and not fixed here -- left for the orchestrator to file as its own issue. What DOES reach `Lexicon(...)` here unchanged, always, is an entry normalizing to empty, in `capitalization_exceptions` or in ANY set field -- measured: `c.titles.add("...")` then a parse raises Lexicon's own `titles entry '' normalizes to empty`, unaltered -- since nothing above translates that away, and a non-str value's `TypeError`, a raise v1 also had, later, at `capitalize()`. Declined: a first draft pre-validated `capitalization_exceptions` in isolation, a second call to `_normpairs` ahead of `Lexicon(...)` gated on a message-substring test; measured to make a multi-word key's dead-entry warning fire twice on the shim path, and to rest on a filter that was untested (deleting it passed the whole suite) and wrong (an unrelated ValueError then got the v1 hint appended regardless). Pinned by `tests/test_capitalization.py::test_a_mismatched_exception_value_raises_at_the_first_parse` (the hint, and that it carries no 2.0 constructor spelling), `tests/v2/test_config_shim.py::test_an_unrelated_capitalization_exceptions_valueerror_has_no_v1_hint` (the pass-through), `tests/v2/test_config_shim.py::test_a_multiword_capitalization_exceptions_key_warns_exactly_once` (the single warning), and `tests/v2/test_lexicon.py::test_mask_value_error_survives_pickle_and_copy` (the pickle/copy round-trip). + +- 2026-09-24 #459 — DECIDED (Derek, 2026-09-24): the shipped map gains a mask for each listed acronym whose conventional spelling is mixed case, taken together with decisions/R5's keep-as-written bullet of the same day, which is the half that surfaced the need. The reviewer who measured the R5 regression named nine keys — `edd`, `dsc`, `psyd`, `meng`, `dmin`, `thd`, `drph`, `pharmd`, `mdiv` — and a sweep of `SUFFIX_ACRONYMS` for further entries whose usual spelling is unambiguously mixed found ten more: `bt`, `chfc`, `cpht`, `diplac`, `kt`, `phc`, `rph` and `thm`, and `lac` and `ded`. Of those nineteen, four are left out (below), so SHIPPED is fifteen — seven of the reviewer's nine and eight of the sweep's ten — every key a `suffix_acronyms` member: `bt` → `Bt` (Baronet), `chfc` → `ChFC` (Chartered Financial Consultant), `cpht` → `CPhT` (Certified Pharmacy Technician), `diplac` → `DiplAc` (Diplomate in Acupuncture, written `Dipl. Ac.`), `dmin` → `DMin` (Doctor of Ministry), `drph` → `DrPH` (Doctor of Public Health), `dsc` → `DSc` (Doctor of Science), `kt` → `Kt` (Knight Bachelor), `mdiv` → `MDiv` (Master of Divinity), `pharmd` → `PharmD` (Doctor of Pharmacy), `phc` → `PhC` (Pharmaceutical Chemist), `psyd` → `PsyD` (Doctor of Psychology), `rph` → `RPh` (Registered Pharmacist), `thd` → `ThD` (Doctor of Theology), `thm` → `ThM` (Master of Theology). The four LEFT OUT are two of the reviewer's nine, `meng` and `edd`, and two of the sweep's ten, `lac` and `ded` (the CAPITALIZATION_EXCEPTIONS block for meng, edd, lac and ded in decisions/vocabulary-exclusions): a mask applies wherever its word stands (the 2026-09-23 masks bullet above), so a key borne as a name re-spells a person — the #vocabulary-collisions C-i test read with the mask's reach, which is every position. Candidates considered and not taken for want of one settled spelling: `djur` (`D.Jur.` beside the German `Dr. jur.`), and `litk`, `litl`, `litp`, `mscmsm`, whose credentials could not be identified. WHY BOTH HALVES: each serves a population the other cannot. Keeping a suffix as written serves the writer who spelled it (`john smith EdD`), but a name written wholly in one case carries no spelling to keep, its suffix being one-case too, so only a mask gives `john smith psyd` → `John Smith PsyD` (`PSYD` by the acronym clause without it; `Psyd` on every release 1.4.0 through 2.3.0, measured 2026-09-24); and a mask cannot reach an unlisted credential or a caller's own spelling, which the keep rule does. No corpus name written before this change carries any of the fifteen keys (measured 2026-09-24 over the parent's 1340 names and this tree's 1375), so the MEASURED bullet above moves only by the rules.md line `john smith psyd`. Recompute the list: `sorted(Lexicon.default().capitalization_exceptions)`. Pinned on both surfaces by `tests/v2/test_render.py::test_a_shipped_mask_spells_its_suffix_in_either_single_case`, which derives its rows from the shipped pairs and so pins the property rather than the list, and the exclusion's fork by `test_a_listed_acronym_that_is_a_name_word_gets_no_mask`, whose recorded negative control is the `meng`/`edd` masks added back. + +- 2026-09-24 — SUPERSEDED IN PART: the 2026-09-23 bullet above headed "A SUFFIX READ BY ITS DOTTED SHAPE" accepted that `john smith B.Tech.` gives `John Smith B.TECH.` on the DEFAULT path once the suffix-free gate admitted the name. That half no longer holds: decisions/R5's keep-as-written bullet (2026-09-24) keeps a suffix written in more than one case, so the default path gives `John Smith B.Tech.`, the credential as written — every release 1.4.0 through 2.3.0 returned the whole name untouched, `john smith B.Tech.` (measured 2026-09-24). The acceptance stands under force only — `John Smith B.TECH.` with `force=True`, where every release gave `B.tech.` — and the mask remedy it names is unchanged. The rules/R5 example line proposed for this spelling (`juan garcia B.Tech.`) was not added: the differential gate reported it UNEXPLAINED at every baseline, the by-shape suffix reading being a role change no ledger rule claims for that name, so it is pinned as a unit test instead (`tests/v2/test_render.py::test_a_suffix_written_in_more_than_one_case_is_kept_as_written`). + +- 2026-09-24 #478 — DECIDED (Derek, 2026-09-24): the hyphen clause keeps its reading in a name written wholly in one case, and the case where that disagrees with rules/P3 is a recorded boundary, not a change. The two collide on a marked letter. Spaced, a letter the vocabulary marks as reading both ways reads as an initial in a one-case name (P3), so `maria silva e sousa` repairs to `Maria Silva E Sousa`; hyphenated, the interior part reads as the connective whatever the name's case, so `maria silva-e-sousa` repairs to `Maria Silva-e-Sousa`. Derek's reasoning: hyphenating `Silva-e-Sousa` is the writer joining the surname on purpose, so the interior word is a connective by that act, whatever the one-case rule would say of the spaced form. THE COST, measured 2026-09-24: a one-case name whose hyphenated bare initials happen to spell a connective — `J-E-P DUPONT` gives `J-e-P Dupont`, where every release 1.4.0 through 2.3.0 and the parent 4d0680e6 gave `J-E-P Dupont`, and `JOHN A-Y-B SMITH` gives `John A-y-B Smith` (every release: `John A-Y-B Smith`). The period-marked spelling is unaffected (`j.-e.-p. dupont` keeps `J.-E.-P. Dupont`, the 2026-09-23 #478 bullet). `conjunctions_ambiguous`, P3's knob, does NOT reach a hyphenated word: under `Lexicon.default().add(conjunctions_ambiguous={"y"})` the spaced `JOSE ORTEGA Y GASSET` repairs to `Jose Ortega Y Gasset` (default lexicon: `Jose Ortega y Gasset`) while `JOSE ORTEGA-Y-GASSET` stays `Jose Ortega-y-Gasset` under both. rules/R4's hyphen sentence said the hyphens join the name "as the spaced connective would", which claimed an equality the one-case case breaks; it now says the hyphens are the writer's join and states the split, with an Accepted paragraph and the `J-E-P DUPONT` boundary line. Pinned by `tests/v2/test_render.py::test_the_hyphen_is_the_writers_join_even_in_a_one_case_name`. +- 2026-10-02 #541 — THE "LEFT FOR THE ORCHESTRATOR TO FILE" FINDING of the 2026-09-24 branch-review bullet (d) above is #541, and it was wider than that bullet saw: besides the two raises it names, an entry carrying whitespace v1 never matched SILENTLY ACTIVATED in every set field the shim copies through and in `capitalization_exceptions` keys — measured 2026-10-02 against 1.4.0 (run outside the worktree, `PYTHONSAFEPATH=1`): `titles ' dean '` gave title `dean` on `dean john smith` where 1.4.0 gave first `dean`, and likewise `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a `' zzc '` key; the raise (`suffix_not_acronyms 'ma '`) and the activation (`titles ' dean '`) both reproduce on the 2.0.0, 2.2.0 and 2.3.0 wheels. DECIDED (Derek): `Constants._snapshot()` drops every entry `_config_shim._v1_matchable` rejects — empty after `lc()`, or not equal to its own single-spaced re-join — from all nine set fields and the keys, BEFORE its set algebra, and names them in one `UserWarning` whose remedy runs (`tests/v2/test_config_shim.py::test_the_offered_remedy_runs_and_silences_the_warning`). The test is exact because a v1 piece comes from a whitespace split re-joined only with single spaces; it generalizes the filter `given_name_titles` already had, and `tests/v2/test_config_shim.py::_UNFILTERED_OUTCOME` records what each swept row does with it off. One exception, decided when the 1.4 pickle test caught it: every release from at least 0.5.8 through 1.4.0 SHIPPED two such entries in `TITLES` (`'actor '`, `'television '`), so a restored 1.4 pickle carries them and its user never wrote them; they are dropped WITHOUT the warning (`_V14_SHIPPED_UNMATCHABLE`, held to the pickle's own unmatchable set by `test_the_1_4_shipped_unmatchable_roster_is_exactly_the_pickles`), keeping that pickle warning-free. The exemption is keyed by entry, not by provenance, so a user who writes `'actor '` or `'television '` themselves is not warned either — accepted, since the two strings are 1.x's own typos. The drop is a reading change for that pickle: `Actor John Smith` gave title `Actor` on 2.0.0 through 2.3.0 and gives first `Actor` now, as 1.4.0 did. DECLINED: stripping whitespace in `SetManager` (activates every v1-inert entry above), and a silent drop for user entries (Derek: the entry is certainly a typo and certainly dead, so say so). ACCEPTED WIDENING, deliberately left: a key written with capitals or edge periods (`'McDonald'`, `'phd.'`) never matched on 1.4.0, whose lookup is `lc(word)` against keys stored as written, but has matched since 2.0 through `Lexicon`'s fold; dropping it would break a working 2.x config to restore an inertness nobody wanted. Also left: a set entry `Lexicon` folds to empty that `lc()` does not, a lone non-ASCII full stop (`'。'`), still raises at the first parse in every field `Lexicon` receives directly — `titles`, `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a `capitalization_exceptions` key; `first_name_titles` drops it quietly (`if t`), and `non_first_name_prefixes` and `suffix_acronyms_ambiguous` never pass it to `Lexicon` at all — v1-matchable, so outside this rule. +- 2026-10-02 #582 — SUPERSEDES the #541 bullet's "Also left" sentence above. DECIDED (Derek): an entry `Lexicon._normalize` folds to empty is dropped by `_config_shim._v1_matchable` with the #541 warning, in every set field and key. The case is a lone CJK full stop (`'。'`, `'．'`, `'｡'`, or a run of them), which 2.3.0's `FULL_STOPS` fold (#322/#323) empties while v1's `lc()` strips only `.`. Measured 2026-10-02 on released wheels run outside the checkout with `PYTHONSAFEPATH=1`: 1.4.0 through 2.2.0 accept `c.titles.add('。')` and read `。 john smith` with title `。`, while 2.3.0 raises `ValueError` at the first parse, as did this tree before the change, in `titles`, `prefixes`, `suffix_acronyms`, `suffix_not_acronyms`, `conjunctions`, `bound_first_names` and a key. ACCEPTED DEPARTURE from 1.4.0, the only one this filter makes: such an entry used to act on a token that is nothing but that full stop. Since 2.3.0 `Lexicon` can neither hold the entry nor match the token, its lookup fold emptying both, so reproducing v1 is not on offer and the choice was drop or raise. The warning's wording moved with it, from "nameparser 1.x never matched" to "match no name word". `given_name_titles`' `if t` filter was deleted as unreachable: `_title_key` is empty only when every word folds away, and then the whole entry folds to empty and was already dropped. With no shim-reachable plain `ValueError` left from `_normpairs`, `test_an_unrelated_capitalization_exceptions_valueerror_has_no_v1_hint` turns the filter off to keep the except-by-type clause pinned. THE SAME FOLD, ONE CHARACTER IN: an entry with a CJK full stop at its EDGE (not only full stops) passed the filter, and the set algebra compared it raw while `Lexicon` folded it, so `c.suffix_not_acronyms.add('ma。')` (beside the ambiguous `ma`) raised the gate-bypass check and a bound `'zed。'` beside a never-given particle `zed` raised the contradiction check, both from 2.3.0 on; measured accepted on 1.4.0 and 2.2.0 by the review of this change. `_build_snapshot` now compares `_normalize`d spellings in exactly the two computations `Lexicon` re-checks after its own fold — dropping a `suffix_not_acronyms` entry whose fold is an ambiguous acronym's, and taking into `particles_ambiguous` every particle whose fold is a bound given name's — so such an entry reads exactly as its folded spelling (`test_an_edge_full_stop_collision_reads_as_its_folded_spelling`; the fold-off control is `_UNFOLDED_OUTCOME`). That reading is the `dean。` widening below, not v1's: 1.4.0 matched `ma。` only on a `ma。` token. DECLINED, after being written and reviewed: folding EVERY kept entry before the set algebra. `non_first_name_prefixes` and `suffix_acronyms_ambiguous` reach `Lexicon` only through raw set algebra, so folding them is a NEW match, not the 2.3.0 one: `prefixes zz` with `non_first_name_prefixes 'zz。'` moved `zz smith` from first `zz` to last `zz smith`, an NFD/NFC pair across those fields moved the same way, and `suffix_acronyms_ambiguous 'zq。'` beside `suffix_acronyms zq` moved `john zq` from suffix to last — the particle readings the same on 1.4.0, 2.2.0, 2.3.0 and the pre-fold tree, the `zq` one on 2.2.0, 2.3.0 and the pre-fold tree (1.4.0 gives last `zq` there by its own two-word reading, not by matching the entry). They are pinned at those readings by `test_an_edge_full_stop_entry_outside_the_two_checks_reads_as_before`. ACCEPTED WIDENING, left as it is (Derek): since 2.3.0 a `titles` entry `'dean。'` also matches a bare `dean`, where 1.4.0 matched only `dean。`. The same fold makes it, and dropping the entry would lose the `dean。` match too, so it sits beside the capitalized-key widening above. +- 2026-10-02 #542 — A DECOMPOSED WORD IS REPAIRED AS ITS COMPOSED TWIN, and the repaired text keeps the form it was typed in: no clause composes its OUTPUT. Two mechanisms do it. The word splitter and the mask's neighbour tests read a combining mark (Unicode category M) as part of the letter before it; the three clauses that ask a regex about a whole word's SHAPE — Mac/Mc and the two initial tests — ask it of the word's composed spelling. They differ only for a mark with no precomposed form, where composing leaves the word as written and so agrees with the parse: under a caller-added conjunction `q́`, `ivan q́. petrov` parses `q́.` as the conjunction, and the hyphen clause keeps `Petrov-q́.-Sidorov` lowercase as it does (measured by the docs review, 2026-10-02). A name typed NFD (macOS file names, some databases) had been split at each mark by `_WORD`'s `(\w|\.)+`, `\w` matching no mark, so `josé garcía` repaired to `José GarcíA` on every release from 1.4.0. Python's `re` has no `\p{M}`, and a hand-written class of the combining blocks would be a copy of Unicode data that drifts (it would already miss Adlam, a cased script whose marks sit outside them), so the test is `unicodedata.category(ch)[0] == "M"`, asked by `_render._sub_words` after each `_WORD` match and by `_render._beside` for a letter's neighbours. DECLINED: NFC-composing before repair, the issue's option 2. It fixes the split but hands back text in a form the writer did not use, which is a change beyond case under rules/R4 — `tests/v2/test_properties.py::test_case_repair_changes_case_and_nothing_else` compares with `casefold()`, which does not normalize, and would report it. THE ISSUE'S CLAIM THAT THE MASK NEEDED NO CHANGE ONCE THE WORD REACHED IT WHOLE WAS MEASURED FALSE, and the Mac/Mc clause needed one too; both were found by fixing the splitter alone and comparing again. `_apply_mask`'s split-off-initial override and `_letter_run_ge2` read a letter's neighbour by raw index, which in NFD is the mark rather than the full stop or the letter past it: with the splitter alone, under a caller's `('éx', 'éx')` mask `john smith é.x.` forced gave `É.X.` composed and `é.X.` decomposed — a shape that AGREED before the fix (`É.X.` both ways) only because the old splitter cut the word at its mark before the mask was looked up — and under `('éex', 'éeX')` `john smith ée.x` gave `ée.X` against `éE.X` (`ÉE.X` before the fix). Both read past marks now. The Mac/Mc clause's `\w{2,}` fails at the mark, so with the splitter alone `macée` gave `Macée` decomposed against `MacÉe` composed (`MacéE` before the fix, the split having made the last `e` a word of its own); the clause is now DECIDED on the composed spelling and applied to the word as written. The two initial-shape tests, `_DOTTED_INITIAL` in the hyphen clause and `_INITIAL` in the unclassified-text fallback, matched the raw word and now match its composed spelling, so a decomposed `й.` — `й` is the one default conjunction that decomposes — is the initial its composed twin is: `ivan petrov-й.-sidorov` repaired to `Ivan Petrov-й.-Sidorov` decomposed against `Ivan Petrov-Й.-Sidorov` composed, and a middle spliced in as decomposed `й.` kept `й.` against `Й.`, at 97af1f02 and at this change's first commit alike (found by the docs review, 2026-10-02). AMENDS the 2026-09-24 branch-review bullet (a): its SPLITTING half is retired. `parse("ǰo smith").capitalized(force=True)` gives `J̌o`, and a second forced pass now gives `J̌o` back rather than `J̌O`, the mark staying with its letter (measured 2026-10-02); the LENGTHENING half (`ß`, `ʼn`) stands. A mark with no letter before it heads no word: `str.capitalize()` on a word beginning with one would upper-case the mark and leave the letter after it lowercase. Out of reach and left there, two of them: `initials()`, a rules/R3 view rather than case repair, takes a token's first character, which for a decomposed initial letter is the base letter without its accent (`parse("émile zola")` initials `é. z.` composed and `e. z.` decomposed, measured 2026-10-02; fixed the same day under #585, decisions/R3); and unspaced hangul, whose NFD parse differs from its NFC parse because segmentation matches the census list as written (docs/usage.rst, "Decomposed text"), so repair follows a different parse; hangul jamo are letters, not marks, and nothing here touches them. VERIFICATION: the differential gate cannot see this — `capitalized()` is not a compared surface (this entry, 2026-08-29); the two NFD rows this change adds to the rules corpus reach the gate through the role fields, `_ambiguities` and `_initials` only. Standing in its place: rules/R4's two NFD example lines, `tests/v2/test_properties.py::test_a_decomposed_name_repairs_as_its_composed_twin` (renamed `test_a_decomposed_name_renders_as_its_composed_twin` by #585, which joined initials to the walk), which decomposes every corpus and case-table text that decomposes, compares its repair with the composed text's on both surfaces plain and forced, and records as its control 144 disagreements over 137 texts with 97af1f02's `_render.py` over this change's corpus (measured 2026-10-02), and `tests/v2/test_render.py::test_a_decomposed_word_repairs_as_its_composed_twin` and `test_a_spliced_decomposed_initial_reads_as_its_composed_twin` for the shapes no corpus name reaches, whose two mask rows were mutation-checked by restoring raw-index neighbours. +- 2026-10-04 #604 — DECIDED (Derek, 2026-10-04): a mask outranks the particle lowercase. Adding the Irish particles moved case repair the wrong way: `SEÁN Ó MURCHÚ` repaired to `Seán ó Murchú`, where 2.3.0 gave `Seán Ó Murchú` (Ó having been a middle initial there), and `ní` and `ua` the same, because repair writes every particle in lowercase while Irish writes these three capitalized. The map is already "how a word is written wherever it stands" (the role-free note in `_cap_word`), so the particle arm now steps aside for a word the map carries, and the map gains `ó`→`Ó`, `ní`→`Ní`, `ua`→`Ua`. Each mask is its word's own title case, so a bare word in any other role repairs as it would without one. The one reach past particles is the dotted spelling of the two-letter masks: each writes its word as one run, so R4's split-off-letter clause capitalizes both letters, and `u.a. smith` repairs to `U.A. Smith` where it gave `U.a. Smith`, `n.í. smith` to `N.Í. Smith` where it gave `N.í. Smith` (measured 2026-10-04, with and without the three masks; `ó` has no dotted twin), which is right for two initials. Measured before deciding, no shipped exceptions key was a particle (`set(Lexicon.default().capitalization_exceptions_map) & Lexicon.default().particles` was empty), so in the SHIPPED vocabulary the precedence moved nothing but the three new entries. A CALLER's map is another matter, and this is a deliberate change there too (Derek, 2026-10-04, after the design and code reviews both found it): `c.capitalization_exceptions['van'] = 'Van'` makes `ludwig van beethoven` repair to `Ludwig Van Beethoven`, where 1.4.0 through 2.3.0 gave `Ludwig van Beethoven`, the particle having won in v1's `cap_word` and every release since. Kept because an entry is how the caller says a word is written wherever it stands, and a caller who maps `van` to `Van` plausibly means it; the release bullet says so, since the differential gate cannot see `capitalized()` (#R3, above). Pinned by test_a_caller_particle_mask_outranks_the_particle_lowercase. Not extended to `mac`, which repairs to `mac` as before (`Liam mac Murchú`): `mac` is an ambiguous particle shared with Scottish and other spellings, and nothing in #604 weighed it. The Malay `binti`/`binte` carry no mask and keep their lowercase (`Ina binti Navalamar`, where 2.3.0 gave `Ina Binti Navalamar`). diff --git a/docs/design/decisions/R5-case-repair-gate.md b/docs/design/decisions/R5-case-repair-gate.md new file mode 100644 index 00000000..05263d27 --- /dev/null +++ b/docs/design/decisions/R5-case-repair-gate.md @@ -0,0 +1,24 @@ +# R5 — the case-repair gate + +- 2026-08-29 (#407 arc) — EXTRACTION, not a decision: the parser is untouched. The two halves of R5 have separate provenance, and conflating them is easy enough that the first draft of this entry did. The REFUSAL — repair skips any name already carrying more than one case — is older than the git history: `git log -S "name == name.upper() or name == name.lower()" --reverse` bottoms out at 45a1539 (2011-02-03), the initial import from svn, where `capitalize()` already opens with that guard and a bare `return`. (A path-filtered search answers 280895b instead, the same-day commit that moved the module into `nameparser/`; the code did not change there.) The OVERRIDE is bf1e0a5, 2016-06-02, which did not add the refusal but wrapped it — `if not (name == name.upper() or ...)` became `if not force and not (...)` — and shipped in 0.4.0 (June 2, 2016; its own release-log line is docs/release_log.rst under that heading). So R5's statement as a whole holds from 0.4.0 on. rules.md had never said any of it, though bf1e0a5's diff shows the API docstring already did: "It will not adjust the case of names entered in mixed case" was there before that commit edited around it. The gap was rules.md's alone. Evidence, measured on the released 1.4.0 wheel — the last v1 release and one of the differential baselines, so a natural thing to measure against and not a release that introduced anything here — and re-measured on this branch today, facade and core agreeing: `HumanName('Shirley Maclaine').capitalize()` leaves `'Shirley Maclaine'` — mixed case, wrong, and kept — while the same name under `force=True` gives `'Shirley MacLaine'`; `HumanName('Juan McDonald').capitalize()` leaves `'Juan McDonald'`. rules.md's own preamble classifies behavior in this position as "pinned-but-undocumented — an extraction gap to close, not a specification", which is why the fix lands in the document rather than in `_render.py`. +- 2026-08-29 — why the gap survived, and what the new boundary buys. R4 promised that "an already-correct name comes back unchanged", which is idempotence: true of everything the gate returns, and silent about the gate. The two readings were indistinguishable through R4's boundary, because `"Juan McDonald"` comes back unchanged under `force=True` as well — an already-correct input cannot witness a rule whose subject is what gets kept REGARDLESS of correctness. `Shirley Maclaine` separates them, so R5's block pins it twice: kept as written on a plain `capitalized=` line, and repaired on the forced line. Under idempotence the plain line would have to fail. The BOUNDARY sits on the forced line, which took a correction to get right — the first draft marked the refusal, and the refusal is R5's effect, not its absence. R5's own stated exception is repair-regardless, so that is the non-firing case (rules.md's preamble: the boundary is "usually the rule's OWN stated exception"). +- 2026-08-29 — WHY R5'S STATEMENT WAS REWORDED RATHER THAN GIVEN A THIRD EXAMPLE, recorded at length because the wrong turn was taken twice and the reasoning is reusable. R5 first said a mixed-case name is returned untouched "whether or not its casing is correct". That phrasing reads as a disjunction with two branches, and invites the question of which example witnesses the already-correct one. Nothing can: for a mixed-case name that repair would not alter, kept and repaired are the SAME STRING by construction, so no example line distinguishes a parser with the gate from one without it. The unwitnessable branch is a property of the claim, not a gap in our choice of names — and the bullet above had already said so ("an already-correct input cannot witness a rule whose subject is what gets kept REGARDLESS of correctness") before two review rounds pushed past it. Two rows were tried and both withdrawn. `"Juan McDonald" → capitalized="Juan McDonald"` was INERT (mechanisms.md's inert-measurement class): it passes with R5's gate deleted, which is the whole failure shape that class names. `"Vincent Van Gogh" → capitalized="Vincent Van Gogh"` did discriminate — measured by comparing the shipped call against gate-off behavior, which the forced call reproduces exactly, since deleting the gate is what `force` already does: shipped `'Vincent Van Gogh'`, gate-off `'Vincent van Gogh'`, the particle rule lowercasing `Van` the moment repair runs — but it was REDUNDANT, not complementary. It asserts the same proposition as the `Shirley Maclaine` row above it: repair would change this name, and the gate keeps it anyway. The two differ only in which repair rule would have fired, and which rule repair applies is R4's subject, not R5's. And the frame it was chosen under was itself wrong, which is the more useful half of the lesson (reframed 2026-08-29 on Derek's correction, while R4 was being reworked). The row was picked to be bearer-correct — a name whose casing the bearer would endorse — and that frame drags the document into per-name arguments about whose spelling is right: this one needed `Vincent van Gogh` to be the correct spelling, which P6's own examples contradict, and the tussenvoegsel convention behind them is amended later on this branch anyway. CORRECTNESS DOES NOT ENTER INTO IT. Mixed case is the writer making an explicit choice, and repair defers to that choice rather than judging it — a name kept is not a name endorsed. Read that way the withdrawn row needed no claim about Dutch orthography at all, and the reason it needed one is exactly that the frame was wrong. The fix was to stop claiming the unwitnessable branch: the statement now says a mixed-case name is kept and that whether its casing is right does not enter into the decision, which `Shirley Maclaine` — casing wrong, kept — witnesses whole. GENERAL LESSON, and the reason this is long: when an example cannot be found for half a rule, suspect the STATEMENT before suspecting the example set. A phrasing that promises more branches than the behavior has will absorb inert examples indefinitely, each one looking like progress. Caution for the commit that reworks R4 and will choose its own mixed-case rows: avoid a name whose family base is wholly particle vocabulary (`Anh Van Do`, base `Van Do`), because the #407 work changes how those capitalize and would neuter such a row silently; `gogh` and `vega` are in neither `particles` nor `particles_ambiguous`, `van` and `do` are in both. +- 2026-08-29 — the override has TWO routes, and the rule states neither, by design. Per-call is the obvious one. The second is a v1 Constants attribute the facade still honors, `force_mixed_case_capitalization` (nameparser/_facade.py resolves it when no per-call value is given; docs/release_log.rst records it as "still honored through the facade"). Measured today: with that attribute set True, `HumanName('Shirley Maclaine')` then a bare `.capitalize()` — no argument at all — LEAVES `'Shirley MacLaine'` (the v1 call mutates in place and returns None, so the name is read back with `str()`). An earlier draft of R5 said "only an explicit request to repair regardless overrides that", which that measurement falsifies; the statement now says repair was asked for anyway, without saying by what route (it read `repair regardless of how the name is cased was asked for` until 2026-08-29, when that phrasing turned out to have a second reading -- see the override bullet below), because rules.md is implementation-free by its own preamble. +- 2026-08-29 — `capitalized_forced` is a test-side pseudo-field, not a parser field. The route the EXAMPLES use is the per-call argument to `capitalized()`, and an argument is not a policy, locale or extras gate — the only three things rules.md's grammar admits in an example's annotation slot — so it cannot ride that slot, and the doc runner grew a resolver branch instead. (The facade attribute above is a second route to the same behavior, not a second thing to assert; the core takes the argument only.) The R5 block's example lines are asserted by the suite like any others. They do enlarge one file: `corpus_rules.jsonl` is generated from this document's examples, and `Shirley Maclaine` had not been among them, so regenerating adds it (235 lines, from 234). Whether that moves the DIFFERENTIAL population is a separate question with a per-name answer, and the general answer is that a rules.md example CAN move it: the harness dedupes across corpus files, so a new example name costs a population slot exactly when no other corpus already carries that same string. `Shirley Maclaine` was already in `corpus.jsonl`, so it costs nothing and the population holds at 1090 — measured, not assumed. The row withdrawn above is the counter-case, and worth keeping for it: `Vincent Van Gogh` appears in no other corpus (`corpus.jsonl` and `corpus_issues.jsonl` carry `Vincent van Gogh`, a DIFFERENT string that does not dedupe against it), and while it was in the block the population read 1091. Neither name needed a ledger rule, which is a measurement rather than a consequence of the above: the gate exits 0 with 0 unexplained at all three baselines either way. +- 2026-08-29 — WHAT THE OVERRIDE DOES AND DOES NOT PROMISE, from Derek's framing of R5 and then measured, because the framing implies a property that is ALMOST true and the gap is the useful part. The framing first, and it supersedes the correctness talk elsewhere in this entry: mixed case is the writer making an explicit choice, and repair defers to that choice instead of judging it. Nothing is being called correct or incorrect — a name kept is a name whose writer said something about it, and a name repaired is one whose writer did not. The property that seems to follow is that asking for repair REGARDLESS should ignore the given casing entirely, so one name repairs to one string however it was written. MEASURED over the 1094 corpus names and it does NOT hold. On the v2 core — `parse(n).capitalized(force=True)` against `parse(n.upper()).capitalized()`, rendering all seven roles — the two differ for 63 names, and the lowercase direction for 18. Naming the surface matters here, and BOTH counts move with the surface, which an earlier draft of this bullet got half right: through `HumanName` and `str()`, which is what tests/test_capitalization.py uses, the counts are 62 and 16 — and the two directions are dropped for DIFFERENT reasons, which an earlier draft of this bullet ran together under the first. UPPER: the facade's default render spec omits the maiden name, so `str()` cannot see `Jane van der Berg née y Jones` (the whole of the upper difference, and it is a conjunction inside the maiden name — `maiden` is `y Jones` forced against `Y Jones` uppercased and every other role is byte-identical). LOWER: the two the facade drops are `John van der J. V` and `abdul V Smith`, and the maiden field is EMPTY in both directions for both, so the omitted role explains neither. What hides them is that `str()` CONCATENATES adjacent roles, so a token that crosses a role BOUNDARY and moves nothing else is invisible in the joined string: `John van der J. V` is family `van der J. V` forced against family `van der J.` plus suffix `V` lowercased, and `abdul V Smith` is given `Abdul V` forced against given `Abdul` plus middle `V` lowercased — same seven roles rendered, same string joined. Re-derived 2026-08-29 on this branch, and named per name so the next reader can check the arithmetic: core 63/18, facade 62/16. Two things about those numbers are the opposite of what one would guess. UPPERCASE IS THE WORSE DIRECTION, not the clean one. And the misses are not merely the pipeline's case-sensitivity leaking in: of the 63, only 25 move a role at all, and the other 38 parse byte-identically and diverge inside the repair (25 and 37 through the facade, the missing name being one of the byte-identical ones). The mechanism is v1's initial carve-out — a conjunction is not lowercased where it is written initial-shaped, and initial-shaped means one CAPITAL letter — so uppercasing turns every one-letter conjunction into an initial (`Velasquez y Garcia, Dr. Juan Q.` forced keeps `y`; the same name uppercased then repaired gives `Y`, with a byte-identical partition either side -- the comma form is the one to cite here, the space-written `Dr. Juan Q. Velasquez y Garcia` being a member of the 25 whose roles DO move), and lowercasing turns a middle initial `E` into the Italian conjunction. 1.4.0 does the same (`JUAN Y GARCIA` capitalizes to `Juan Y Garcia`), so this is inherited, and it is recorded here rather than fixed here. Recompute both directions by running the two forms over the corpus files deduped and diffing. +- 2026-08-29 — and therefore NOT stated in rules.md, which is a deliberate choice rather than an oversight. The document's examples are keyed on input STRINGS, so any statement of the property invites exactly the test that falsifies it — re-case the input, expect the same output — and the counterexamples are already in the corpora. The property is true of the repair given a parse, and rules.md speaks input-to-output; a rule stating it would be over-broad in the one direction a reader would check. What R5's statement says is enough for the promise that IS kept: a mixed-case name is kept unless repair was asked for anyway. That clause was REWORDED for this, and the reword is the whole point rather than a tidy-up. It read `unless repair regardless of how the name is cased was asked for`, which carries two readings -- the intended one, that the request overrides the keeping, and a second one, that the repair disregards the input's casing, which is this property in nearly this bullet's own words. A reader taking the second reading would run the re-casing test predicted above, land on `Velasquez y Garcia, Dr. Juan Q.` (in the corpus today), and conclude the RULE is wrong when only the phrasing was. Nine words, and they asserted the thing the paragraph exists to deny. The property is pinned in tests/test_capitalization.py instead, over names carrying no single-letter word whose class case decides, with `juan y garcia` beside it as the recorded exception. R5's example block gains `"SHIRLEY MACLAINE" → capitalized="Shirley MacLaine"` from this work, and it earns its place on its own ground rather than as half of a convergence pair: it is the only row in the block that fails when the gate is narrowed to lowercase-only, every other row passing that mutation. Measured three ways — gate deleted (passes, so it does not witness the gate's existence), gate narrowed to accept only all-lowercase (FAILS, and alone in the block), Mac/Mc convention deleted (fails, with the other two rows). Until it was added, R5 stated that repair acts on a name written entirely in one case and witnessed only the lowercase half of it. That lowercase half is still `"juan mcdonald"`, which is byte for byte an R4 row as well, and the duplication is deliberate rather than an editing slip: the two rules make different claims about the same line — R4 that the repair honors the Mac/Mc convention, R5 that an all-lowercase name is acted on at all — and dropping it from R5 would leave the gate's lowercase half unwitnessed inside the rule that states the gate. Five other rows already sit under two rules apiece for the same reason (P5/P6 twice, P5/O5, N3/M4, W1/W3). +- 2026-08-29 — DEBT this extraction leaves, named so the next commit inherits an obligation rather than a rediscovery. Pulling the gate out into R5 leaves R4 carrying ONE falsehood and ONE ambiguity — different defects wanting different repairs, and `interacts: R5` carries neither, the field being advisory. FALSE: R4 promises repair "vocabulary exceptions (McDonald) included", but `str(parse('Juan Mcdonald').capitalized())` is `'Juan Mcdonald'` — the gate refuses before any vocabulary is consulted, and only `str(parse('Juan Mcdonald').capitalized(force=True))`, `'Juan McDonald'`, reaches the exception. R4 needs its promise scoped to names the gate admits. AMBIGUOUS, not false: R4's "an already-correct name comes back unchanged" means correct by the repair's own conventions, i.e. idempotence, and under that meaning it is true; a reader hears correct as the bearer writes it, and under THAT meaning `str(parse('bell hooks').capitalized())` — `'Bell Hooks'` — looks like a counterexample. It is not one, because `bell hooks` is not already-correct in R4's sense. What R4 owes is a disambiguation of "correct", NOT a narrowing to spare deliberately single-cased names: that would be new behavior, and R5's own rationale declines it on the ground that single case leaves the repair nothing to read. Also for that commit, and inert as things stand: R4's boundary row `"Juan McDonald" → capitalized="Juan McDonald"` passes with R5's gate deleted, exactly like the R5 row that was withdrawn above; rewriting it to `capitalized_forced=` makes it discriminate for R4's own subject but still witnesses nothing about the already-correct question. This commit adds R5 and touches R4 only on its pointer line, leaving both defects as found rather than half-fixed by a commit whose subject is something else. +- 2026-09-13 #383/#479 — AMENDMENT to the 2026-08-29 override bullet above: the MECHANISM it names is gone, and with it the direction it called surprising. That bullet explained the re-casing gap by v1's initial carve-out — "uppercasing turns every one-letter conjunction into an initial" — and cited `Velasquez y Garcia, Dr. Juan Q.` forced (keeping "y") against the same name uppercased then repaired (giving "Y"). Measured on this branch today, all three spellings agree: forced, uppercased-then-repaired and lowercased-then-repaired each give `Dr. Juan Q. Velasquez y Garcia`, and the space-written `Dr. Juan Q. Velasquez y Garcia` does the same. A one-case name no longer reads a bare capital connective as an initial on shape; the vocabulary decides, and "y" is outside the marked subset (decisions/P3, 2026-09-13). Nothing about R5's gate changed — only what repair finds on the other side of it. + THE DIRECTION REVERSED, which is the part worth carrying: uppercase was the worse direction and is now the better one. Recipe, one comparator, both sides — over the deduped `tools/differential/corpus*.jsonl` glob, compare `parse(n).capitalized(force=True)` against `parse(n.upper()).capitalized()` and against `parse(n.lower()).capitalized()`, rendering all seven role fields, and count the names that differ in each direction; run it once on this tree and once against a released wheel installed in a throwaway environment (the tree shadows the wheel otherwise). Measured 2026-09-13 over 1174 names: the 2.3.0 wheel gives 69 upper and 23 lower, this tree gives 11 upper and 18 lower. On the facade surface — `HumanName(n).capitalize(force=True)` read back with `str()`, against the same call on the re-cased spellings — 2.3.0 gives 68 and 20, this tree 10 and 15; the surface still matters for the reasons the 2026-08-29 bullet names (the default render spec omits the maiden name, and `str()` concatenates adjacent roles). The uppercase count collapsed; the lowercase one did not, its remaining members being what the carve-out never explained. The 63/18 and 62/16 figures in the bullet above are a dated snapshot over a 1094-name corpus and are not reproducible today in either direction — do not treat them as this recipe's before-column. + rules/R5 needs no matching amendment: it never carried the claim. The 2026-08-29 bullet that follows the override bullet above says so in its own words ("and therefore NOT stated in rules.md"), and R5's Rationale, statement and example lines were re-read on this branch to confirm it. What DID move in the suite is the mirrored pin in tests/test_capitalization.py, where `JUAN Y GARCIA` had been pinned as inherited 1.4.0 behavior ("Juan Y Garcia") and now agrees with its lowercase spelling ("Juan y Garcia"), with a mixed-case control beside it. + +- 2026-09-23 #492 — DECIDED (Derek, 2026-09-23): the gate reads every token but the SUFFIX-roled ones. R5 says mixed case is evidence the writer cased the NAME deliberately, and a suffix written the way a suffix is written — `III`, `PhD`, `Jr.` — says nothing about how the writer cased the name words. So `juan garcia III` repairs to `Juan Garcia III` where every release, 1.4.0 through 2.3.0, returned it untouched, and `juan garcia PhD`, `JUAN GARCIA Jr.` and `dr. juan garcia III` repair the same way. v1's test for it, `test_capitalization_exception_for_already_capitalized_III`, was a known failure since 2012 — added 2012-10-07 (e2df6937) as a plain failing test against the Google Code tracker's issue 22, marked xfail 2026-09-01 (d02dc41e) — and this change retires the marker and the `_KNOWN_FAILURE` suffix the test carried. Every token not roled SUFFIX stays in the gate text, NICKNAME and MAIDEN included: `jane doe nee SMITH III` and `juan garcia (Bob)` are left alone (pinned in `tests/v2/test_render.py::test_the_gate_leaves_the_suffixes_out`, where a mutant filtering those roles too survived the suite otherwise). A name whose non-suffix tokens are caseless is one-case exactly as the whole name was, so only the suffix repairs (`김민준 phd` → `민준 김 PhD`); a name with NO non-suffix token is one the parse never produces (rules/H4 — measured, `Jr.` is a title, `PhD` a given name, `MD, PhD` family and suffix), so only a synthetic `ParsedName` reaches the empty text, which is one-case, and every suffix repairs to its own fixpoint. Force is unchanged. WHY "USE FORCE" IS NOT THE ANSWER: `force=True` is lossy on correctly cased names — `Danny DeVito` → `Danny Devito`, `Shia LaBeouf` → `Shia Labeouf`, `Anne-Marie DuPont` → `Anne-Marie Dupont`, `Bill deBlasio` → `Bill Deblasio` (re-measured 2026-09-23) — so a batch caller cannot force the lowercase rows without damaging the cased ones beside them. + +- 2026-09-23 #492 — THE R5 / PARSER SPLIT, decided and recorded rather than left for a reader to discover. The parser's own one-case readings — rules/P3's single-letter connective and rules/S2's case signal at the suffix slot — still count a cased suffix, as Derek's 2026-09-19 comment on #492 and decisions/S2's 2026-09-14 bullet require (the judged token stays in the span: `Jack MA` is mixed because of `MA`); R5's gate does not. So one name can be mixed to the parser and one-case to repair. `john e jones III` is mixed to P3, `III` included, so its `e` reads as a connective and it repairs to `John e Jones III`, while `john e jones iii` is one-case, reads `e` as an initial and repairs to `John E Jones III`; the corpus name `john e jones, III` moves the first way on the default path, where every release left it untouched. The gate follows the parse's suffix call wherever the parse makes one, the ambiguous credential class included, so two shapes outside the corpus move on the default path as well: `jack MA` gives `Jack MA` — S2's lean read `MA` as the credential, and the gate then leaves it out — and `MD, PhD` gives `Md PhD`, an input that is nothing but vocabulary (rules/H4's shape) read as family `MD` and suffix `PhD`, where 2.2.0 and 2.3.0 left `MD PhD` untouched; the family `MD` repairs as a name word, by the item-4 decision under R4. All four strings are pinned by `tests/v2/test_render.py::test_the_gate_and_the_parser_read_a_cased_suffix_differently`. + +- 2026-09-23 #492 — TITLES STAY EVIDENCE, and the reason is a limit of title REPAIR rather than a claim about titles. Title repair is still plain `str.capitalize()`, and widening the gate to titles would surface its defects on the default path. Measured under `force=True`, where they already show (re-measured 2026-09-23): `Lt.Gov. juan e garcia` → `Lt.gov. Juan e Garcia`, `The Lord of the Universe` → `the Lord of the Universe`, `Dr. med. univ. Margit Popp` → `Dr. Med. Univ. Margit Popp`. So `Dr. juan garcia` is left alone (rules/R5's boundary line). This may move when title repair is trusted. + +- 2026-09-23 #492 — MEASURED, population first. Recipe: for each name in the deduped corpus glob, the gate as shipped before #492 (the joined text of EVERY token) declines it AND the joined text of its non-suffix tokens is one-case; those are exactly the names the new gate admits, and the default path returns their forced repair (since 2026-09-24, less any suffix written in more than one case, which it keeps as written — the bullet below). The movers are the admitted names whose default-path string changes, counted by the R4 MEASURED bullet's comparator. At 4d0680e6, over its 1340 names, the population was 17 and 12 of them changed their default-path string — `juan garcia III`, `dr. ben alex johnson III`, `john e jones, III`, `john smith MA` and eight `pennie von bergen wessels` spellings; the other five (`威廉·莎士比亚, PhD`, `田中さん, PhD`, `田中さん, Ph. D.`, `김민준씨 Jr.`, `1 & 2, 3 4 5, Mr.`) are admitted and repair to themselves. The approved spec said "17 move": 17 is the population, 12 the movers. Over the corpus this change leaves (1375 names, R5's new example lines among them) the population is 22 and the movers 17, the five new ones being those lines (20 and 15 over 1371 names before the 2026-09-24 decision below added `john smith EdD` and `juan garcia Iii`; re-measured 2026-09-24, the parent-corpus 17 and 12 unchanged). No forced repair moves by this rule, as none can: the gate is never consulted under force. + +- 2026-09-24 #492 — DECIDED (Derek, 2026-09-24): where repair was not forced, a SUFFIX token written in more than one case is kept as written; one written in a single case is repaired like any other token, and force repairs both, unchanged. It is the gate's own one-case test asked of each suffix on its own (`_render._in_one_case`, one predicate for both), and it is the 2026-09-23 #492 reasoning read to its end: a suffix is left out of the gate because a cased one says nothing about how the writer cased the NAME, and for the same reason a mixed-case one is the writer's spelling of the credential, which repair has no evidence to overrule. THE REGRESSION THAT SURFACED IT, measured by the branch's code reviewer and re-measured 2026-09-24: once the gate stopped counting the suffix, a one-case name with a conventionally mixed-case listed acronym was admitted and the acronym clause wrote the acronym in capitals — `john smith EdD` → `John Smith EDD`, `JANE DOE, DSc` → `Jane Doe DSC`, `juan garcia PsyD` → `Juan Garcia PSYD` — where the parent 4d0680e6 and every release 1.4.0 through 2.3.0 returned those names untouched. They now give `John Smith EdD`, `Jane Doe DSc` and `Juan Garcia PsyD`. The accepted `B.Tech.` → `B.TECH.` on the default path (decisions/R4, 2026-09-23) was the same shape, and its default-path half is superseded (decisions/R4, 2026-09-24). THE COST, accepted: "as written" keeps a garbled spelling with the deliberate one, so `juan garcia Iii` gives `Juan Garcia Iii` on the default path (`Juan Garcia III` forced) and `john smith Mba` gives `John Smith Mba`, where the branch before this decision gave `III` and `MBA`; every release returned both names untouched. A rules/R5 boundary line pins `Iii`. Why not keep only a spelling the vocabulary does not know how to repair: that would re-derive, per word, a judgment about which mixed spellings are deliberate, which is the evidence R5 says repair cannot read. The masks added the same day (decisions/R4) are the other half: they serve the one-case names this rule cannot reach (`john smith psyd` → `PsyD`). No corpus name written before this decision moves by it (measured 2026-09-24 against the branch before it, over this tree's 1375 names: the only movers are the new rules.md lines). Pinned by `tests/v2/test_render.py::test_a_suffix_written_in_more_than_one_case_is_kept_as_written`, core and facade; a mutant dropping the per-token test fails it, `test_the_gate_leaves_the_suffixes_out` and two rules/R5 lines, and one dropping its `force` guard fails it and the forced `EdD` line. + diff --git a/docs/design/decisions/README.md b/docs/design/decisions/README.md new file mode 100644 index 00000000..608d032e --- /dev/null +++ b/docs/design/decisions/README.md @@ -0,0 +1,123 @@ +# Decisions + +The why behind the rules: a lightweight ADR log, keyed by rule ID (`P2 — `), by mechanism slug, or by a short slug for cross-cutting and tooling decisions. Each key is one file, named for its key and name and listed in the index below, so every `decisions/P2` reference in code or in the rules resolves to one file. A new entry under an existing key goes in that key's file; a new key is a new file and an index line. + +The gitignored planning specs are the working medium and die with their branches; this folder is the record. A landing design PR distills its spec's durable residue into its entry files (the checklist is AGENTS.md's "Landing a design"). + +Entry conventions: + +- **Dated decision entries**, append-only in spirit: a reversed decision is not edited, a later entry supersedes it. Each entry cites its issue or PR. The date is the DECISION's, never the release's (add "shipped in X" alongside if useful), and git author dates outrank remembered ones — two review rounds corrected exactly these two errors. +- **Harvested entries** (content landed from a session's report) keep their provenance: a "measurements are that session's, spot-checked at landing" framing line, the contributor's measured-vs-remembered markings where they matter, and — the hard-won one — a flagged uncertainty is resolved by reading the source artifact, never by inference from the neighboring argument. +- **`Declined:`** — proposals rejected WITH the evidence that killed them. Resolved-as-no is a decision; without a home for it, the next person re-derives the rejected proposal and its measurement. +- **`Excluded:`** — standing prohibitions with indefinite lifetime, keyed by vocabulary set: entries that must stay OUT of a wordlist, each with its reason. Distinct from Declined because the failure mode differs — nobody re-derives a rejected proposal, but someone sweeping a wordlist ships the excluded entry as a bug. +- **`Open:`** — unresolved questions as issue links with one-line handles. The ISSUE is canonical; this block never restates it. Two keyings: under a rule ID for questions about the rule, and — like `Excluded:` — keyed to a VOCABULARY SET for contested memberships, the category neither `deviates:` nor `Excluded:` covers: the rule is right and a word's set membership is the question (the shape: `rai` in the suffix acronyms and `swami` absent from the given-name titles, both since answered — see decisions/suffix-acronym-collisions and decisions/indic-honorifics). The blocks keyed by vocabulary rather than by a rule live in decisions/vocabulary-exclusions, headed by the set (or sets) they bind, so a wordlist editor meets the `Excluded:` and `Open:` blocks together; add a new block there, beside any block for the same set. An entry may still record its own keep-outs inline (decisions/generational-words lists SUFFIX_WORDS ones), so search the folder for the set's name too. +- **Weighing entries** for contested questions: the options considered, each option's intended effect, and the accepted costs of the option chosen. The costs accepted here are the artifacts the rule lists under its `Accepted:` consequences; the two link by rule ID. + +## Index + +## Titles & honorifics (H) + +- [H1 — a title and one name word](H1-title-and-one-name-word.md) +- [H2 — the leading-abbreviation title](H2-leading-abbreviation-title.md) +- [H3 — the title run's floor](H3-title-runs-floor.md) +- [H4 — an input that is nothing but vocabulary still has to name somebody](H4-input-that-is-nothing-but-vocabulary-still-has-to-name-somebody.md) +- [H5 — the trailing title run](H5-trailing-title-run.md) +- [H6 — a title before a family comma](H6-title-before-a-family-comma.md) + +## Particles & surname prefixes (P) + +- [P1 — lone particle fold](P1-lone-particle-fold.md) +- [P2 — particles join forward](P2-particles-join-forward.md) +- [P3 — connectives](P3-connectives.md) +- [P5 — bound given names](P5-bound-given-names.md) +- [P6 — the trailing orphan particle](P6-trailing-orphan-particle.md) +- [P7 — a one-letter particle against the initial it spells](P7-one-letter-particle-against-the-initial-it-spells.md) + +## Suffixes: generational & credentials (S) + +- [S2 — the case signal at the suffix slot](S2-case-signal-at-the-suffix-slot.md) + +## Nicknames & quoted names (N) + +- [N1 — the default delimiter pairs](N1-default-delimiter-pairs.md) +- [N2 — same-character quotes and shared-character conventions](N2-same-character-quotes-and-shared-character-conventions.md) +- [N3 — the lone-word nickname rule](N3-lone-word-nickname-rule.md) + +## Maiden names (M) + +- [M1 — delimited maiden names](M1-delimited-maiden-names.md) +- [M2 — the maiden-marker rule](M2-maiden-marker-rule.md) +- [M3 — the marker-led clause](M3-marker-led-clause.md) +- [M4 — a maiden marker makes the lone name word the family](M4-maiden-marker-makes-the-lone-name-word-the-family.md) + +## Commas & structure (C) + +- [C1 — the suffix-comma decision](C1-suffix-comma-decision.md) + +## Name order (O) + +- [O1 — East Slavic rotation](O1-east-slavic-rotation.md) +- [O2 — Turkic rotation](O2-turkic-rotation.md) +- [O4 — positional assignment and declared order](O4-positional-assignment-and-declared-order.md) +- [O5 — the lone name word's convention, and reporting it](O5-lone-name-words-convention-and-reporting-it.md) + +## Scripts & writing systems (W) + +- [W1 — unspaced CJK division](W1-unspaced-cjk-division.md) +- [W2 — the glued-honorific peel](W2-glued-honorific-peel.md) +- [W3 — the writer's divisions are respected](W3-writers-divisions-are-respected.md) +- [W4 — script-scoped order](W4-script-scoped-order.md) + +## Tokens, initials & punctuation (T) + +- [T1 — separators, not joiners](T1-separators-not-joiners.md) +- [T3 — the interpunct's flank guard](T3-interpuncts-flank-guard.md) + +## Ambiguity & tie-breaking (A) + +- [A1 — the ambiguity report](A1-ambiguity-report.md) +- [A2 — the empty name](A2-empty-name.md) + +## Rendering & views (R) + +- [R2 — the all-particles family_base divergence](R2-all-particles-family-base-divergence.md) +- [R3 — initials honor the fold](R3-initials-honor-the-fold.md) +- [R4 — case repair reads the unjoined mark](R4-case-repair-reads-the-unjoined-mark.md) +- [R5 — the case-repair gate](R5-case-repair-gate.md) + +## Construction & configuration diagnostics (D) + +- [D1 — the segmenterless-activation warning](D1-segmenterless-activation-warning.md) +- [D2 — construction raises, parse never does](D2-construction-raises-parse-never-does.md) + +## Named topics + +- [3-0-reevaluations — decisions shaped by the v1 shim](3-0-reevaluations.md) +- [api-churn-declines — do not re-propose without new cause](api-churn-declines.md) +- [cjk-comma-demotion — the script shapes are pure, the wrappers are tolerated (2026-09-01, #469)](cjk-comma-demotion.md) +- [cjk-full-stops — which characters are a period on CJK text (2026-09-10, #322/#323)](cjk-full-stops.md) +- [comma-suffix-arc — #291/#296/#316 (2026-07-26 → 2026-08-01)](comma-suffix-arc.md) +- [comparison-surface — why value equality died, and what replaced it](comparison-surface.md) +- [deviates-registry — packs stay pure data (option C)](deviates-registry.md) +- [differential-ledger — the differential ledger's arcs](differential-ledger.md) +- [generational-words — junior and senior in other languages (2026-10-14, #661)](generational-words.md) +- [given-name-titles — deliberately unvalidated](given-name-titles.md) +- [indic-honorifics — the renunciate class and the Indic honorific vocabulary (2026-09-06, #346/#344/#343)](indic-honorifics.md) +- [initials-repertoire — is_initial across scripts (#320)](initials-repertoire.md) +- [legacy-rule-numbers — the old docstring numbering](legacy-rule-numbers.md) +- [ma-do — ambiguous acronyms by decision](ma-do.md) +- [nfd-shapes — shape tests read letters, and a decomposed letter is one letter](nfd-shapes.md) +- [normalization-fold — lower(), not casefold(); comparison folds harder](normalization-fold.md) +- [parse-cost — what a parse is allowed to cost](parse-cost.md) +- [phd-merge — the "Ph. D." split](phd-merge.md) +- [removed-v1-surface](removed-v1-surface.md) +- [render-default — the default view's format](render-default.md) +- [review-agent-single-source — the review axes live in one file](review-agent-single-source.md) +- [salutation-titles — salutations, offices and ranks for the languages TITLES had none for (2026-10-04, #606)](salutation-titles.md) +- [script-table-placement — a recorded reversal](script-table-placement.md) +- [suffix-acronym-collisions — the trailing-position collision class, decided (2026-09-07, #342/#454)](suffix-acronym-collisions.md) +- [suffix-field-composition — three kinds of thing in one field](suffix-field-composition.md) +- [two-input-invariants — checking a rule against something that is not the code](two-input-invariants.md) +- [v1-xfail-triage — the eight inherited aspirations, four retired and four kept (three of those eight since fixed, 2026-09-08 and 2026-09-23)](v1-xfail-triage.md) +- [vocabulary-collisions — when a word earns the ambiguous marking](vocabulary-collisions.md) +- [vocabulary-exclusions — words kept out of a vocabulary set, and contested memberships](vocabulary-exclusions.md) diff --git a/docs/design/decisions/S2-case-signal-at-the-suffix-slot.md b/docs/design/decisions/S2-case-signal-at-the-suffix-slot.md new file mode 100644 index 00000000..1c44ffca --- /dev/null +++ b/docs/design/decisions/S2-case-signal-at-the-suffix-slot.md @@ -0,0 +1,108 @@ +# S2 — the case signal at the suffix slot + +- 2026-09-14 #289 + #516 (this PR) — A BARE AMBIGUOUS ACRONYM IS READ BY THE EVIDENCE THE WRITING CARRIES, and written case is evidence the words-to-spare count does not have. In a name written in more than one case, a member of `suffix_acronyms_ambiguous` written in CAPITALS leans credential and is taken as a suffix with nothing to spare (`Jack MA` → given 'Jack', suffix 'MA', as `Jack MD` has always read); one written in any other cased form that is not wholly lower leans SURNAME and stays a name word with plenty to spare (`John Smith Ma` → middle 'Smith', family 'Ma'). An all-lower member carries no lean, a name written wholly in one case carries none at all, and both take the positional rule unchanged — `JOHN SMITH MA` suffix, `ANH DO` family, `john smith ma` suffix. Switch-free and scoped to the listed set: `ba do ed jd ma`. + THE PREDICATE KEEPS THE JUDGED TOKEN IN THE SPAN, and that is what gives the right answers here. `JOHN SMITH Ma` is MIXED because of `Ma`, and a Title-case word against all-caps neighbours is exactly the surname signal this design wants; excluding the judged token would make the name one-case and hand `Ma` to the positional rule. Same for `Jack MA` read the other way — included, the name is mixed and the capitals lean credential; excluded, `own == ['Jack']` is one case and the peel keeps the family. The general question — whether a cased suffix token counts as case evidence — is #492's and stays deferred; this entry records that the INCLUDING reading is what this design rests on. +- 2026-09-14 (Derek) — THE LEAN REACHES THE POST-COMMA GIVEN SLOT: `Smith, MA` → family 'Smith', suffix 'MA', while `Smith, Ma` and `Smith, ma` stay given. Positive evidence outranks position there, so the credential reading fires with ONE word before the comma where the positional rule would not. + AND THE COUNT AT A COMMA IS OF NAME WORDS. For this class, "words to spare" in `X, Y` form means the part before the comma holds two or more NAME words: `Smith Jr., MA` has two tokens and one name word, and a token count hands its family to `given` (measured on the prototype). rules/C1 already said both things — "more than one word precedes the comma" in its opening clause and "a part before the comma with more than one name word" later — so the doc edit is a reconciliation inside C1 rather than a new claim. +- 2026-09-15 (spec review) — THE COUNT REACHES THE LISTED SET TOO — one uniform rule for the whole ambiguous class rather than two. `John Smith, Ed` → given John, family Smith, suffix Ed; `john smith, ma` and `JOHN SMITH, MA` move the same way, each being a one-case name that takes the positional rule, which now reaches the comma structure. All three are 1.4.0 RESTORATIONS — measured on the wheel, v1 reads all three as suffixes — so today's tree is what deviates and this restores parity rather than opening distance from it. Corpus population: ZERO (verified by grepping `tools/differential/corpus*.jsonl` for all three; `Royce, Ed` is the corpus's only member of the sibling one-word shape), so no corpus mover count moves and `Davis Royce, Ed` is the case row that carries the decision. +- 2026-09-17, corrected — A CASELESS SCRIPT IS INERT ONLY AS A LEAN, NOT AS THE NAME-WORD COUNT. `is_one_case` answers True trivially for text with no case to write a contrast in, so `ambiguous_lean` returns `None` for `毛泽东, MA`, `마틴 킹, MA` and `田中 太郎, MA` alike — the LEAN really is inert by construction, as first written. But the comma count above is of NAME words "whatever case the name is written in", which is orthogonal to case and reaches a caseless script the same as any other: `마틴 킹, MA` and `田中 太郎, MA` each hold TWO pre-comma name words, so the structure flips and both read suffix 'MA' — a PARITY RESTORATION, not a fresh deviation (1.4.0 read both as suffix on the wheel), with the Hangul/Han surname split then running positionally over what the comma leaves (`마틴 킹` → given '틴', middle '킹', family '마'; `田中 太郎` → given '太郎', family '田中'), unrelated to this design and #271/#272's pre-existing behavior. `毛泽东, MA` does not move — one pre-comma token, the count never reaches two. Pinned `a_caseless_script_wrote_no_contrast_hangul` and `a_caseless_script_wrote_no_contrast_japanese` (tests/v2/cases.py), both `tolerated=True`. +- 2026-09-17, corrected — THE COMMA-PATH REPORT TRACKS THE FORK CONSULTED, NOT THE LEAN. A first pass gated the family-comma report on whether `ambiguous_lean` fired, which wrongly silenced `Smith, ma` (all-lower, no lean) and `毛泽东, MA` (caseless, no lean) — the TRAILING slot has never gated its own report on the lean either (a bare ambiguous acronym reported before #289 too, pick declined or not), because the report is about a fork the parse CONSULTED, and the fork is consulted whenever a class member sits in the slot, lean or no lean. Fixed: the guard is class MEMBERSHIP on the first post-comma piece (`vocab:suffix-ambiguous` or `shape:acronym`, never the lean) — `Smith, ma` reports `"'ma' after the comma is also an ordinary name word; read as the given name"` (since #626, 2026-10-08: "read as a given name", the field worded from where the word lands), `毛泽东, MA` reports the same wording for `'MA'`, and `Smith, MA PhD` (a two-piece post-comma part) reports through its FIRST piece regardless of what follows it, exactly as a one-piece part would. The structure decision reports in `segment`, where that branch is taken, and the family-comma reading in `assign`, so one DECISION never reports twice — a second ambiguous token elsewhere is a second fork and reports on its own (`Smith MA, Ed` carries two `suffix-or-name` reports, one per fork, plus a `given-or-family` for the lone remaining word). +- 2026-09-14 (Derek) — #516's DOTTED HALF IS A SWITCH, `Policy.unlisted_dotted_suffixes`, DEFAULT ON: an unlisted token of two or more period-separated chunks, any chunk length (`X.Y.Z.`, `B.Tech.`, `Q.W.E.R.T.`), joins the ambiguous class BY SHAPE; whole-token vocabulary still wins (`M.A.`, `Ph.D.`, `A.B.C.` — `abc` IS a suffix acronym). The positional rule then decides and reports either way: `John Smith X.Y.Z.` → suffix, `Jack X.Y.Z.` → family, `Smith, A.B.` → given, `John Smith, A.B.` → suffix. Case is irrelevant — the periods are the evidence — so `john smith x.y.z.` reads as its mixed-case twin. Switch OFF: name material everywhere as 2.3 read it, and the fork is STILL reported, the parser having chosen the name reading over a credential one. +- 2026-09-15 (Derek) — THE ROMAN-CHUNK ACCIDENT RETIRES, NARROWLY. rules/S3's chunk rule survives except where every chunk the vocabulary matches is a SINGLE ASCII CHARACTER — the accident exactly. Measured, `{w for w in (L.suffix_acronyms | L.suffix_words) if len(w) == 1}` is `{i, v, 2}` plus the glued CJK honorific tails `{様, 殿, 氏, 군, 님, 씨, 양}`, so "single ASCII character" names the first three and only those — CHARACTER because `2` is a digit and is in the set, ASCII because `씨` is the one that must KEEP its chunk claim (`J.씨`). So `John Smith R.A.I.` and `John Smith J.u.n.i.o.r.` reach the shape class and read suffix by POSITION with the same fields plus the report, `Jack X.Y.I.` reads family where it read suffix (1.4.0's reading), and `Msc.Ed.`, `JD.CPA`, `Lt.Gov.`, `J.씨` keep their chunk-derived readings. The WIDE retirement — any chunk match yielding to the shape — was MEASURED AND REJECTED: it moves three more corpus names, `Doe, John Msc.Ed.` losing a real credential to `middle` and the two `김민준씨, J.씨` rows re-routing their honorific peel, for nothing this design wants. +- 2026-09-17/18, #516 review rounds — THE SHAPE VERDICT NEEDS TWO FURTHER GATES, both found by measuring after the shape verdict shipped. (i) ALPHABETIC: `period_joined_vocab`'s "shape" branch now requires every chunk to be `.isalpha()` — a bare digit chunk is not an acronym letter by any reading, so `Smith, 1.4`, `John Smith 1.4` and `John Smith, 1.4` stay unclaimed (no report, no role move), where an unguarded shape verdict had wrongly admitted them; the delimited `Bridge (1.4)` control only proved digits reach extract's escape, saying nothing about the same digits undelimited, which is the gap this closes. (ii) INITIALLESS-SCRIPT: the same test `is_title_shaped` already makes for H2's own leading-abbreviation inference (#323) — `text.isascii() or not in_initialless_script(text)` — now gates the shape verdict too, so `John Smith 田.中.`, `김 민준 이.박.` and `John Smith たな.か.` stay family: a script with no period abbreviations at all has nothing for interior periods to abbreviate, so a CJK word glued into period-separated characters is not spelling an acronym either. `John Smith 田.中.` is pinned `tolerated=True` rather than `shape=` — a Latin period convention glued onto CJK characters is a composed form no writing system produces, and the case table's own validator refuses a Latin shape tag on CJK text; `이.박.` and `たな.か.` are covered by direct `period_joined_vocab` assertions instead of a second corpus row each. +- 2026-09-17, #516 review round — `SMITH, J.R.` READS BY SHAPE FOR A REASON UNRELATED TO WHY IT DECLINES THE LISTED WORD. `_normalize` keeps interior periods ON PURPOSE (v1 parity, pinned 2026-07-17: `'J.R.'` must not collapse to the listed suffix word `'jr'`), so `ambiguous_class_member` correctly answers False for `Smith, J.R.`, `Smith, E.S.Q.` and `Smith, E.T., Jr.` for a reason that has nothing to do with #516. What #516 adds is a SEPARATE route: each chunk (`'j'`/`'r'`, `'e'`/`'s'`/`'q'`, `'e'`/`'t'`) is an unlisted single-letter chunk nothing in the vocabulary claims, so the token joins the class BY SHAPE instead, exactly as `'K.D.'` does in `J.A. K.D.`. The READING is unchanged in all three — one pre-comma name word is never enough to flip the structure — but each now gains `suffix-or-name`, the class having been considered and declined rather than never reached at all. + `is_wholly_suffix` NEVER READS THE BY-SHAPE CLASS. A first version admitted a by-shape member there unconditionally, reasoning it was needed for `John Smith, A.B.`'s structure flip; proved wrong by mutation — that flip already goes through `_segment.py`'s own `pre_comma_names >= 2` disjunct alone, and nothing else in the suite depended on the admission except this predicate's own two unit tests. Left in, the admission combined with C1's LEGACY token-count disjunct (which counts WORDS, not NAME words, and predates the name-word-count decision above) to flip `Smith Jr., A.B.` to given `'Smith'`, suffix `'Jr., A.B.'` with a self-contradicting "holds 1 name words" report. Fixed by dropping the admission outright: the by-shape class reaches the comma form ONLY through `_vocab.ambiguous_class_candidate`, never through `is_wholly_suffix`. `Smith Jr., A.B.` now reads given `'A.B.'`, family `'Smith'`, suffix `'Jr.'` — the same shape the LISTED class reads for `Smith Jr., Ma` and `Smith Jr., ma` (given `'Ma'`/`'ma'`, family `'Smith'`, suffix `'Jr.'`), not for `Smith Jr., MA` (given `''`, family `'Smith'`, suffix `'Jr., MA'` — the caps lean has already moved that one into the suffix run, measured on this tree; at 2.3.0 it read given `'MA'`, family `'Smith'`, suffix `'Jr.'`, the very shape `Smith Jr., A.B.` reads now) — with the report; classification `parity`, not `fix(#516)`: FULL PARITY on the 1.4.0 wheel (first A.B., last Smith, suffix Jr.), only the report being new. +- 2026-09-14 (Derek), corrected 2026-09-18 — #516's ALL-CAPS HALF IS A SWITCH TOO, `Policy.unlisted_caps_suffixes`, DEFAULT OFF, and the asymmetry is the decision. An unlisted all-caps word of two or more letters, with no period ANYWHERE (`isalpha()` rejects a period anywhere, not merely an interior one, so a single TRAILING period, H2's own shape, is unaffected either way), in a name `one_case` says is written in more than one case, joins the class by shape (`_vocab.caps_shape_candidate`). + UNLISTED MEANS IN NO WORDLIST AT ALL, not merely "no whole-token suffix vocabulary" — found only by measuring after a narrower, tag-based guard shipped and reversed: `titles`, `given_name_titles`, `particles`, `particles_ambiguous`, `conjunctions`, `conjunctions_ambiguous`, `bound_given_names`, `suffix_acronyms`, `suffix_acronyms_ambiguous`, `suffix_words` and `maiden_markers` — ELEVEN lists, checked by direct membership rather than by tag, since `segment` calls the shape test before `classify` has tagged anything. Measured with the narrower guard alone (`John Smith DE`, `John Smith NEE` wrongly moved to suffix), and stated here as a recipe rather than a fixed count, since a raw per-list membership tally and a de-duplicated one (a word appearing in two lists, `abu`/`أبو`/`ابو` across `bound_given_names` and `particles_ambiguous`, is one leak, not two) give different totals: `len([w for w in getattr(lexicon, name) if w.isascii() and len(w) >= 2 and w.isalpha()])` per excluded list, run over the shipped `Lexicon.default()`, gives on the order of dozens for `particles` and `particles_ambiguous` and single digits for `conjunctions` and `bound_given_names` — every one of them a wordlist entry the caps shape test's earlier, narrower guard did not check at all. The one figure that reproduces exactly, because ASCII case is what the test needs and every other maiden marker is a non-Latin or non-ASCII spelling the shape test's own `isupper()`/`isalpha()` gate excludes on its own: FOUR maiden markers are ASCII (`geb`, `geboren`, `geborene`, `nee`), out of `len(lexicon.maiden_markers)` total (17, measured 2026-09-18) — `'NEE'`/`'GEB'` are two of those four, capitalized, which is what the narrower guard let through. ACCEPTED GAP, recorded rather than chased: an all-caps German title spelled with `SS` for `ß` (`GROSSFÜRST`) is unlisted by this same lookup too, for a reason predating this switch entirely — `_normalize` uses `lower()`, never `casefold()`, specifically so it does not corrupt `großfürst` to the misspelling `grossfürst`, and classify's own existing `vocab:title` check has never recognized this spelling either. Left for whoever next generalizes the ß-fold question. + THE RUN IS THE CAPS CLASS'S ALONE. The multi-token shape `'LEED AP'` needs a RUN test (`John Smith, LEED AP` → suffix `'LEED AP'`), but the listed and dotted halves are single-token classes by construction — the listed set is inherently bare words and the dotted half never needed more than one token — so testing the run over either of them asks a question this design never posed. An un-narrowed run test measured three reversals before this was caught: `John Smith, Ed Ma` (both listed, Title-case, which #289's OWN lean reads as NAME), `John Smith, ma do` (both listed, lower) and `John Smith, X.Y.Z. A.B.` (both dotted) all wrongly moved to suffix as a run. Narrowed to the caps branch alone; `Smith, LEED AP` still declines and reports, and all three reversals are now pinned identical OFF/ON. + `JACK VI` AND `JOHN SMITH VI` DO NOT MOVE, AND NOT FOR THE REASON A FIRST DRAFT GAVE. `VI` passes the caps shape test and carries both `SHAPE_ACRONYM_TAG` and `vocab:suffix-ambiguous` with the switch on — the predicate reaches it, contrary to what an early write-up of this entry said. What actually holds it unchanged is rules/S2's OWN roman-numeral fork (`_pieces.is_trailing_numeral_suffix`), which claims a bare `'VI'` piece and decrements the peel's walk BEFORE the bare-ambiguous/shape branches beneath it are ever reached — a mechanism this switch does not touch and never held `VI` back through non-membership. + THE COST IS A TWO-WORD CONTRAPOSITIVE, not a flat one. Off by default because French and Korean records write the SURNAME in capitals — `Jean DUPONT`, `Minjun KIM` and `Jean Pierre DUPONT` all read family at 1.4.0, at 2.3 and at this default — and `isupper()` is script-agnostic, so the same decline holds in ANY script with a case contrast (`Иван ИВАНОВ`, `Jean ÉCOLE`, measured). Turning it on does not cost every candidate alike: at TWO words there are no words to spare, so `Jean DUPONT` and `Minjun KIM` report the fork and KEEP their family — the class is considered and declined, the mirror of the listed lean's own words-to-spare rule. At THREE words there ARE, and `Jean Pierre DUPONT` gives family `'Pierre'`, suffix `'DUPONT'` — the one genuine cost, and a swallowed family name is the worse of the two failures #vocabulary-collisions C-i already argues from. Default-off is 1.4.0 parity for the whole class regardless; on, nothing but the report changes at two words, and only the three-word shape actually moves. Off, NOTHING is reported either. +- 2026-09-14 (Derek), scope corrected 2026-09-18 — NO NEW AmbiguityKind. Every decision at these slots emits the existing `AmbiguityKind.SUFFIX_OR_NAME` — no new kind — including the comma path, which is the first report of the comma's OWN decision (listing or credential run) in the library (`COMMA_STRUCTURE` already reports on the comma path, but as a shape the parse could not recognize, not a fork it called); the comma-quiet policy in `_types.py` and rules/C1 gain an exception scoped to the ambiguous class. Switch B off emits nothing. This is narrower than an earlier wording of this entry claimed: an attachment decided AFTER a family comma is a separate fork, P6's, and it has reported on its own since 2.3 — measured on this tree and at master `1f78bef` (2.3.0), `parse("Berg, Jan vd")` gives given `'Jan'`, family `'vd Berg'`, `ambiguities=['suffix-or-name']`, emitted from `_pipeline/_post_rules.py`'s attachment fork, and `AmbiguityKind.SUFFIX_OR_NAME`'s own docstring in `_types.py` already lists that family-comma case. + THREE READING SITES, NOT TWO. The trailing suffix slot (`_pieces.peel_trailing`/`listed_lean`) and the post-comma given slot (`_pieces.segment_suffix_reading`) are the two the decision names first; `_vocab.is_wholly_suffix` over a `groups[2:]` TAIL segment is the third, and part of the blast radius rests on it: `Steven Hardman, MD, DO, DDS` loses its `comma-structure` flag only because the lean makes segment 2 (`DO`) wholly suffix. That is the one place this arc QUIETS a report rather than adding one, and it has a case row of its own because a quieted flag is invisible to a reader sweeping for new ones. +- 2026-09-14, corrected 2026-09-17 and 2026-09-18 — WHERE THE FACT LIVES, AND WHY THERE. The comma forms need the class decision at SEGMENT time, two stages before the own-words span is computable from marker TAGS. Four options were measured on a runtime probe (baseline 410.00 frames): segment computing it with a direct marker-head walk, +15; the same re-running the whole marker pass, +16; the fact staying in classify while classify RE-DECIDES `structure`, +0 but refused ON ORDERING — `structure` is read between segment and classify by `_script_segment.py`, which returns early on FAMILY_COMMA, so a flip decided in classify would leave the surname-split gate un-run on a name that turns out to have a suffix comma (measured: `Smith 김민준씨, MA` reads given 'Smith', middle '김', family '민준', suffix '씨, MA' on this tree — the figure first written here, given 'MA' with family 'Smith 김민준씨', was the reading before the flip moved to segment, re-measured 2026-09-18 — and a DIFFERENT wrong split again with the flip decided before script_segment — two stages disagreeing about one parse is what mechanisms.md#PIPELINE-STAGE-CONTRACTS exists to forbid); the marker pass and the fact MOVING to segment, +8, refused because killing the duplicated fold means carrying it on the state through CJK splits, a new bug surface in the one stage that changes token COUNT. SHIPPED: one shared helper, `_pieces.own_words(tokens, comma_offsets, markers, marker_tags=None)`, called by classify always (passing its own tag map) and by segment only when a comma form could turn on the fact. Every figure below is a TOTAL against `1f78befd` (the merge base this whole arc branched from), re-measured 2026-09-18 rather than accumulated per-commit: a `sys.setprofile` harness shaped like `tools/perf/call_count.py`'s `calls_for` (one warm-up parse, mean of 50, module-level `nameparser.parse`, run as a SCRIPT FILE outside either checkout so neither run's `sys.path[0]` masks the other — a `-c` one-liner instead would let the invoking shell's cwd shadow the older snapshot and read the current tree twice), against a `git archive 1f78befd nameparser` extraction on one side and this tree's own editable install on the other, same resolved interpreter both sides. Reference name (`tools/perf/call_count.py`'s own): 411 → 412 (+1). A comma-less, non-trigger control, `John Smith`: 175 → 176 (+1) — segment's lazy `own_words` call never fires without a comma, so this is classify's own always-on share of the cost. Comma names: a non-trigger, `Smith, John`: 203 → 207 (+4); `John Smith, Jr.`: 229 → 233 (+4); `Smith, John, Jr.`: 253 → 257 (+4) — every non-trigger comma name pays the SAME +4 total, dominated by classify's shared span cut plus the cheap case-free candidacy tests the three reading sites each add, not by the lazy fact itself (which these names never force). A genuine trigger, `Smith, MA`: 203 → 225 (+22) — the lazy fact plus its three readers' own work once a real candidate is found. (An earlier draft of this paragraph reported "+1 to +2 on non-trigger comma names" and "+19 to +20" on `Smith, MA`; those were PER-COMMIT deltas from the fix-round commentary above, accumulated inexactly rather than measured as one total, and are superseded by the totals here.) `own_words` shares its exact span-cutting walk with classify's own cut through a second shared function, `_vocab.tag_marker_runs` (moved out of `_classify.py` so `_pieces.py`'s own layering rule — it may import `_state` and `_vocab` only — is not broken): both call sites run the identical walk over the identical inputs, including `comma_offsets`, so a marker phrase like `z domu` is bucketed the same way at a comma either side. A corpus probe over the 1174 corpus entries confirms 0 cut differences and 0 verdict differences between the two call shapes (an earlier, text-only approximation that predated sharing `tag_marker_runs` differed on the CUT for 2 names — `Anna z Nowak`, `Anna z (domu) Nowak`, where `z` opens the phrase entry `z domu` but no run completes — while agreeing on the BOOLEAN for all 1174 even then). WITH SWITCH B ON, a genuine caps candidate pays more, recorded rather than chased to zero since nothing gates it — re-measured 2026-09-18 with `Parser().parse` on both sides of the on/off comparison (the convention `tests/v2/test_policy.py`'s own test uses), against this tree's own default: `Smith, John` 206 → 213 (+7), `Smith, XYZ` 205 → 245 (+40). Recompute (`tools/perf/call_count.py`'s own `calls_for` counts only its fixed reference name, so a chosen comma name needs the same shape spelled inline): `uv run python -c "import sys; from nameparser import Parser, Policy +def calls_for(fn, name, n=50): + fn(name); calls=0 + def counter(frame,event,arg): + nonlocal calls + if event=='call': calls+=1 + sys.setprofile(counter) + try: + for _ in range(n): fn(name) + finally: sys.setprofile(None) + return calls/n +off=Parser(); on=Parser(policy=Policy(unlisted_caps_suffixes=True)) +for n in ('Smith, John','Smith, XYZ'): print(n, calls_for(off.parse, n), calls_for(on.parse, n))"` (both cheaper than an unconsolidated spelling would be, after the caps predicate was consolidated into one shared function, `_vocab.caps_shape_candidate`, called from classify, `ambiguous_class_candidate` and `_segment.py`'s run test alike — the usual "a shared call costs the default a frame" objection does not apply, because every caller's own first conjunct is `policy.unlisted_caps_suffixes` itself, False by default; `tests/v2/test_policy.py`'s own dated comment carries the same two figures and the recipe for `tools/perf/call_count.py` reference readers). The band never moved at either policy — `test_benchmark.py` gates only the comma-less reference name, which no comma-path cost can reach — and `_CALL_BASELINE` is untouched throughout. + NO MECHANISMS ENTRY IS OWED, and here is why: the promoted fact is STATE-OFFSET-CHANNELS' "a pre-token fact is recorded on the ParseState and consulted later" applied to a name-level boolean rather than to offsets, and `ParseState.order` is the precedent that entry and ONE-PREDICATE-PER-QUESTION both already name. That entry's own sentence — "recorded as offsets … or by presence alone where the fact is name-level" — already covers it; the one edit made there is to name `one_case` as the third channel. +- 2026-09-15 (Derek) — ACCEPTED, NOT REPAIRED, TWO SHAPES. (i) `Freiherr von Berg MA` MOVES: it was listed as unreachable by the peel and as staying fix(#424)'s, and measured, the caps lean makes it reachable, because the lean ends P2's chain with no words to spare where the count would not. Today title 'Freiherr', family 'von Berg MA'; designed title 'Freiherr', family 'von Berg', suffix 'MA'; 1.4.0 first 'von Berg', last 'MA'. Three readings exist and the designed one ships; it moves a rules/P2 example line and re-scopes fix(#424)'s ledger rule, whose `fields` gains `suffix`. (ii) `abdul Smith Jr Ma` reads `Jr` as a MIDDLE NAME: the surname lean breaks the peel AT `Ma`, so the unambiguous `Jr` in front of it is never reached. Today given 'abdul', family 'Smith', suffix 'Jr Ma'; designed given 'abdul Smith', middle Jr, family 'Ma'. One case row rather than real data, but the shape is real — a name-leaning ambiguous acronym BLOCKS the peel of a genuine suffix behind it — and the walk stopping at the declined pick is the rule, not a bug the peel owes a repair. +- OUT OF SCOPE, each its own issue or entry: no tokenizer split (`J.R.R.`, `A.B.` stay single tokens); the trailing-period-only shape is NOT the signal (`John Smith Xyz.` → family, and AGENTS.md's trailing-abbreviation paragraph stands for bare words and sentence-final stops); no new `AmbiguityKind`; no locale packs; #492 stays deferred. NEITHER SWITCH REACHES THE v1 SHIM, and that is a consequence stated rather than repaired: `lenient_comma_suffixes` has no v1 manager either and `_build_snapshot()` simply does not translate it. Switch A is DEFAULT ON, so `HumanName` changes with it and a v1 caller cannot turn it off — the deliberate reading of the bridge, where the facade tracks the parser's defaults and 2.x behavior changes reach it (the #528 precedent). +- 2026-09-18 — RULES.MD STATEMENTS NEVER NAME A POLICY FIELD, AND THIS ARC MADE THAT LOAD-BEARING. rules.md's own preamble binds statements to be implementation-free ("no stage names, no function names, no regexes"), and no existing statement names a `Policy` field either — C1 says "strict mode" and leaves `strict-comma-suffixes` to the annotation slot. S2's and S3's switch clauses follow that precedent rather than setting a new one: the prose describes `unlisted_dotted_suffixes` and `unlisted_caps_suffixes` as caller configuration and points at the example lines, and the ANNOTATIONS carry the exact field names (`unlisted_dotted_suffixes-off`, `unlisted_caps_suffixes-on`, registered in `tests/v2/rules_doc.py`'s `POLICIES` beside `strict-comma-suffixes`). Recorded here rather than as a silent edit to the preamble or to two rules' prose: if a future design wants a field named directly in a statement, that is a preamble amendment for someone to propose, not a convention this entry changes on its own authority. +- 2026-09-18, corrected — `BUILD_RULES_CORPUS.PY` HARVESTING A POLICY-ANNOTATED-ONLY EXAMPLE IS LONG-STANDING, NOT NEW, AND THIS ARC ADDS TWO MORE INSTANCES OF IT. Its own docstring says "Policy annotations are deliberately ignored... a family-first-scoped example is simply one more name to diff" — an earlier draft of this bullet claimed that sentence was written when every annotated example had a default-policy sibling; measured false, at the commit that wrote it and on this tree alike. Recompute: parse every rule's examples with `tests/v2/rules_doc.py`'s own `parse_rules_doc`, group by `text` across the WHOLE document, and keep every text whose every appearance's annotation resolves (`resolve_annotation`) to kind `"policy"` rather than `"locale"`/`"gated_locale"` and never appears bare. That gives **19** pre-existing examples (`Mesnil de`, `Smith, de Mesnil Jean`, `de la Vega y Santos Juan`, `ibn Awf abdul Rahman`, `Nguyen Thi Van`, `Jane Smith (Nee)`, `Jane Smith (Nee) (Jones)`, `Garcia Juan Carlos`, `Nguyễn Thị Minh Khai`, `Del Toro`, `Anh Van Do`, `Beethoven Ludwig van`, `Berg Jan de Jr.`, `Garcia`, `Ménil Christophe de`, `Ménil de`, `Smith (Jones)`, `de Anke van y`, `van Berg Jan de`) plus this arc's own **2** (`Jean DUPONT`, `Jean Pierre DUPONT`, both `unlisted_caps_suffixes-on`, rules/S2) — **21** total on this tree. So the shape was accepted the day the docstring was written, not discovered here; what this arc adds is two more members, checked here for the reason the advice below names: `Jean Pierre DUPONT`'s DEFAULT reading (given Jean, middle Pierre, family DUPONT — measured; the annotated ON reading, given Jean, family Pierre, suffix DUPONT, is the only one rules.md states) never appeared in the doc before, and it produces no diff at any of the five baselines either way. Recorded per the same over-collection principle that keeps `(` and `.,` in the corpus (`tools/differential/README.md`'s provenance table): an annotated example's TEXT is a default-policy corpus name regardless of its annotation, full stop, and every one of the 21 — not only this PR's two — is a standing instance a reader should check for a default-policy diff on before assuming it free. +- 2026-09-18 — THE BLAST RADIUS IS THE GATE'S OWN COUNT, NOT A PREDICTION. Measured at 2.3.0 (`uv run python tools/differential/compare.py`, the default baseline, the previous release), and RE-MEASURED after the review round later the same day and again after the verification round that followed it, which is why several sets of figures carrying one date would otherwise sit here: corpus 1234 names, intentional diffs 51, unexplained 0, radar unclassified 7 — SUFFIX_OR_NAME rows the gate reports but no rule yet claims, named so a release note is never quantified from the intentional count alone. The round added TEN corpus names, not the eight first written here (corrected in the verification round of 2026-09-18): `Doe, John X.Y.Z.`, `Dr. 김민준씨, MA`, `John Smith, MD, Ma`, `John Smith, MD, R.A.I.`, `John de Ma`, `Jo김민준씨, DO`, `Kim김민준씨, MA`, `STEVEN HARDMAN, MD, DO, DDS`, `anh van mc`, `毛泽东 MA` — case rows and rules.md example lines both. The verification round itself then added THREE more (`Anh van Do`, `Doe, John MA`, `John Smith, MD, XYZ`, all rules.md boundary examples), taking the gate's count at this baseline from 1231 to 1234. Recompute the delta rather than trusting either figure: load every `tools/differential/corpus*.jsonl` through compare.py's own `_load_entries`, dedup by name in file order, and diff that list against the same list built from `git show :tools/differential/corpus*.jsonl`. That by-name count runs four below the gate's at the 2.x baselines, the gate deduping by (name, order) and four entries carrying a declared order; the DELTAS agree, which is what a "the round added N" sentence is claiming. The round also added a FOURTH rule, `fix(#289/#516) the glued CJK honorific peel reads the case lean` (3 composed CJK names: `Dr. 김민준씨, MA`, `Jo김민준씨, DO`, `Kim김민준씨, MA` — radar-tier by demotion and classified anyway, since a radar diff this arc intended is one a release note may be written from). The four rules carry 20 + 9 + 9 + 3 = 41 of the 51, `John de Ma` having joined the first. The pre-round reading of this paragraph, kept for the shape of the argument rather than the digits: `fix(#289) a written case contrast decides a bare ambiguous acronym` (19 names, incl. `Davis Royce, Ed`, `Freiherr von Berg MA`, `Jack MA`, `Jack Wei Ma`, `John Prof. MA`, `John Smith Ma`, `John Smith, Ed`, `John Smith, MA`, `John Smith, Ma`, `John van der Berg Ma`); `fix(#516) an unlisted dotted acronym is read by position` (9: `Jack X.Y.I.`, `John Smith Q.W.E.R.T.`, `John Smith X.Y.Z.`, `John Smith, A.B.`, `john smith x.y.z.`, `John Smith B.Tech.`, `John Smith C.H.A.`, `John Smith E.S.Q.`, `Smith, E.S.Q.`); `fix(#289/#516) the ambiguous credential class reports at slots that were silent` (9: `John Smith J.u.n.i.o.r.`, `Royce, Ed`, `Smith, A.B.`, `Smith, Ma`, `J.A. K.D.`, `Jack X.Y.Z.`, `John Smith R.A.I.`, `Smith Jr., A.B.`, `Steven Hardman, MD, DO, DDS`). This is LARGER than the plan's original "11 role + 4 report-only" prediction at the default policy — entirely by corpus growth (this arc's own commits admitted the case-row texts above as contract-tier names) and by two mechanisms the runtime prototype that produced that figure predated (`ambiguous_class_candidate`'s existence and the family-comma report's shape-tag branch), never by a wider rule. Recompute with the command above and read the count the gate returns; the numbers here are a snapshot dated 2026-09-18, not a promise. +- 2026-09-18, WHOLE-PR REVIEW ROUND — WHAT IT MOVED, MEASURED BEFORE AND AFTER. Ten findings; the five that changed behavior are here with the shape each was found by, since the shape is what generalizes. (1) A LOST REPORT. Where the prefix chain absorbed an acronym the S2 peel had DECLINED, nobody emitted the flag S2 promises for either reading: `assign` reports from `peel_trailing`'s picks and a pick reaches it only as a LONE piece, so the token it would have reported on no longer existed. Before: `John van der Berg Ma`, `John de Ma`, `Dr. John van Smith Ma`, `John van Smith Ma Jr.` and `John Smith Mc Ma` gave their family names with `ambiguities` empty; after, each names `suffix-or-name` once and no role moves. The emitter is at the chain's own merge and declines where the PARTICLE run took the word (`anh van do`, `anh van mc` — P6's fork, not S2's, and 1.4.0 read both silently). Recompute: parse the five names and read `ambiguities`. (2) TWO STAGES ASKING ONE QUESTION WITH DIFFERENT INPUTS. `_script_segment` asked `is_wholly_suffix` of the post-comma run WITHOUT the `one_case` fact `segment` had recorded one stage earlier, so `Kim김민준씨, MA` read family `Kim김민준씨`, suffix `MA` (씨 unpeeled) while `Kim김민준씨, PhD` read family `Kim김민준`, suffix `씨, PhD` — one name, two credential spellings, two divisions. After: both peel. `Jo김민준씨, DO` and `Dr. 김민준씨, MA` move with them; `김민준씨, MA` does NOT, being wholly one case (Hangul is caseless and `MA` contrasts with nothing), which is the doctrine holding rather than an exception. rules/W3's claim that the two stages differ "in what else they require, not in what they ask of the words" was false until this and is true now. (3) A REPORT ABOUT THE PARSER'S OWN NEW READING. rules/S3's narrow retirement moved `R.A.I.`, `X.Y.I.` and `J.u.n.i.o.r.` from the vocabulary verdict into the shape class, and `segment`'s tail-segment test reads `is_wholly_suffix`, which is blind to that class — so `John Smith, MD, R.A.I.`, `John Smith, Ph.D., R.A.I.` and `John Smith, MD, J.u.n.i.o.r.` gained a `comma-structure` flag 2.3 did not raise. After: quiet, at the default. The test is narrowed to the BY-SHAPE half deliberately — a LISTED member reaches that reading through the case lean, and `STEVEN HARDMAN, MD, DO, DDS` (one case, nothing leans) keeps its flag as the lean's own recorded negative control; `John Smith, MD, Ma` keeps it too. With `unlisted_dotted_suffixes=False` the flag stands, the token being name material there. C2 is amended to state all of this. (4) THE WORDLIST ROSTER. `_vocab.caps_shape_candidate`'s exclusion named ELEVEN of `_lexicon._VOCAB_FIELDS`' thirteen: `Lexicon.default().add(surnames={"dupont"})` with the caps switch on still read `Jean Pierre DUPONT` as given `Jean`, family `Pierre`, suffix `DUPONT` — a caller listing a word as a SURNAME and getting a credential, which is this switch's own worst failure arriving through the one wordlist that says "family name". After: family `DUPONT`, no report. The roster is derived from the field tuple, with a per-field test; `honorific_tails`, the other omission, was already excluded transitively (⊆ `suffix_words` by Lexicon invariant) and joins for completeness. (5) A LISTED MEMBER READ BY SHAPE. A caller may list a DOTTED entry in `suffix_acronyms_ambiguous` ('a.b'), and the whole token then matches that set while `suffix_as_written`'s period-free acronym lookup ('ab') misses it — so the chunk view reached classify's shape branch, `SHAPE_ACRONYM_TAG` rode along, and `_pieces.listed_lean` (which declines wherever that tag is) silenced the caller's own case lean: `Jack A.B.` read family where `Jack MA` reads suffix. After: suffix `A.B.`, the lean applying. Nothing default moves, the shipped ambiguous vocabulary carrying no periods. Also in the round and behavior-neutral: `Policy.unlisted_dotted_suffixes` and `unlisted_caps_suffixes` moved to the END of `Policy` and `PolicyPatch` — inserted mid-class, they had re-bound every positional argument from `strip_emoji` on, which is API for a dataclass that is not `kw_only`; the bool-validation roster is derived from `dataclasses.fields`, the hand-written twin in the tests having named neither new flag; `ambiguous_class_candidate` lost an optional `one_case` parameter no production caller ever passed, and with it a caps branch that answered False for every name the library has parsed; and `one_case` is REQUIRED on the four piece predicates that own it. +- 2026-09-18 — DIGIT-DOTTED TOKENS MOVE, SILENTLY, AND IT IS ACCEPTED. The narrow retirement's set of single-ASCII-character suffix entries is `{i, v, 2}` — the roman numerals AND the lone digit — so a dotted token whose only chunk match is that digit loses the chunk claim, and the SHAPE class cannot take it either: `period_joined_vocab`'s shape verdict wants every chunk `isalpha()`, which no digit is. Measured against 1f78bef: `John Smith 1.4.2` suffix `1.4.2` → family `1.4.2`; `Smith, 1.4.2` family `Smith`/suffix `1.4.2` → given `1.4.2`/family `Smith`; `John Smith, 1.4.2` given `John`/family `Smith`/suffix `1.4.2` → given `1.4.2`/family `John Smith`. No report at any policy, and the switch does not reach it either way. ACCEPTED: a version string read as a credential was the same accident the retirement removes, and the three readings are pinned as case rows. `John Smith 1.4` was already family at 1f78bef and is unaffected — two chunks, neither a match. docs/release_log.rst's wording is corrected from "a single-character roman numeral" to the digit-inclusive form, which is what the code tests. +- 2026-09-18 — THE COMMA REPORT'S REACH IS THE FIRST POST-COMMA PIECE, AND THE GIVEN SEGMENT'S TRAILING SLOT IS AN OPEN FOLLOW-UP. `assign`'s family-comma emitter reads `state.pieces[1][0]` — the first piece after the comma — so a class member standing at the END of the given part is never asked about. PRE-EXISTING and unchanged by this arc: `Doe, John MA` reads middle `MA` and `Doe, John X.Y.Z.` middle `X.Y.Z.`, both silent, at 1f78bef and on this tree alike. What this arc adds is names that MOVED into that silence: `Doe, John J.u.n.i.o.r.` and `Doe, John X.Y.I.` went suffix → middle, and `John Smith nee Jones R.A.I.` suffix → maiden (the maiden walk absorbs it). `Freiherr von Berg X.Y.I.` was listed here too and does not belong: corrected in the verification round of 2026-09-18, it moved suffix → family but GAINED a report rather than losing one. At 1f78bef and on the 2.3.0 wheel it read title `Freiherr`, family `von Berg`, suffix `X.Y.I.`, reporting `particle-or-given` alone; on this tree it reads title `Freiherr`, family `von Berg X.Y.I.` and reports `particle-or-given` AND `suffix-or-name`, the chain emitter taking it by its shape tag. The role moved into a slot that does not report and the report came from the chain instead, so the name is evidence for the emitter rather than for the silence. The universal claims are narrowed to what ships rather than the emitters widened: `nameparser/_types.py`'s `SUFFIX_OR_NAME` member now names the slots it covers and the one it does not, and rules/S3's "and the same at a comma" now says which parts of a comma form it means. Widening it is a MAINTAINER DECISION and is left open, filed as [#531](https://github.com/derek73/python-nameparser/issues/531): the slot is `_assign`'s given-part walk, the fork is genuine there, and whether a middle initial's neighbourhood should start reporting is a judgement about noise (rules/A1's "a kind is worth adding only if a reader would hesitate too" applies to a SLOT the same way). And the open question is a 1.4.0 PARITY question as well as a noise one, which the verification round of 2026-09-18 established and the entry had backwards: `Doe, John MA` reads suffix `MA` on the 1.4.0 wheel and middle `MA` at 2.0.0 through 2.3.0 and here, so the silence at this slot is a 2.0-era reading that has never matched v1 — "every release" was the wrong span and it is now "every 2.x release" wherever this is stated (`nameparser/_types.py`'s `SUFFIX_OR_NAME` member, docs/release_log.rst). The dotted sibling does not divide that way: `Doe, John X.Y.Z.` is middle `X.Y.Z.` at 1.4.0 too. So a maintainer widening the slot is choosing between reporting the fork and restoring v1's ROLE, which are different repairs, and only the bare-acronym half has a parity claim behind it. Recorded under 3-0-reevaluations' standing rule that a design citing 1.4 parity as load-bearing says so where the design lives. Two more report movements recorded here rather than repaired: `van MA` LOSES its `particle-or-given` (1f78bef reported both kinds; the caps lean now takes `MA` to suffix, leaving `van` the only name piece, and assign's leading-particle emitter wants more pieces following), and `John Smith Ma Ma` reports ONCE where 1f78bef reported twice (the first `Ma` is a middle name now, not a second declined pick). And a reach of the default-on dotted switch worth naming: `John Smith www.example.com` reads suffix `www.example.com` with a report, where 1f78bef read family — a URL is three alphabetic period-separated chunks and the shape rule cannot tell it from an acronym. +- 2026-09-18 — A KNOWN ROUGH EDGE UNDER `lenient_comma_suffixes=False`, FOR DEREK'S ATTENTION. Measured: `Smith V., Ma` gives given `Smith`, family `V.`, suffix `Ma`, and `Smith V, Ma` gives given `Smith`, suffix `V, Ma`. The cause is the strict knob rather than the credential class: under strict, the initial-shaped `V.` fails the suffix test, so `name_word_count` sees TWO name words before the comma, flips the structure, and `Smith V.` is then read positionally. The control that shows it is the knob's and not this arc's: `Smith V., PhD` reads given `Smith`, family `V.`, suffix `PhD` under the same knob on master. Consistent, then, and left as it is — recorded with case rows so a future change to either the knob or the count has to notice it. +- 2026-09-18 — THE PREDICTION THAT DID NOT SURVIVE ITS OWN MEASUREMENT, kept because the failure mode is this log's recurring one. A review round called THREE `one_case` plumbing sites in `_group.py` inert and proposed removing them. Measured with a runtime wrapper over 1,642 names under six policies — the default, both family-first orders, strict commas, and each 2.4 switch flipped — 9,852 parses: the maiden walk's PAIR is inert, 0 parses move, and it is inert BY CONSTRUCTION rather than by corpus luck (`numeral_only` answers off `peeled.numeral`, and the numeral fork is decided before the peel reads a lean). The chain-tail measure is not: dropping it there moves 18, on `John van der Berg Ma`, `John de Ma` and `Freiherr von Berg MA` under every one of the six. Both figures are at the call sites. The population is the DISTINCT union, in file order, of every `tools/differential/corpus*.jsonl` entry, every `tests/v2/cases.py` text, and `tests/test_variations.TEST_NAMES` with the three comma permutations `test_variations_of_TEST_NAMES` builds (no-comma, family-comma, and suffix-comma where the name has a suffix), empty strings dropped; the wrapper patches the call sites in a copy of the module source and rebinds `_pipeline.STAGES`, which binds stage functions at import and would otherwise leave the mutation inert. This entry first carried 2,338 names and 14,028 parses, which that recipe does not reproduce by any reading of it — the distinct union is 1,642, summing the three sources' distinct counts gives 2,202, and not deduping at all gives 2,690 — so both figures were re-measured on this tree in the verification round of 2026-09-18 and restated here and at the call sites. The FINDING was unharmed: 0 and 18 reproduce exactly, on the same three names, under all six policies, and the control that drops all four sites at once moves those same 18, which is what shows the wrapper can report a difference at all. The argument was right for two sites out of three and would have taken a third with it. +- 2026-09-18 — THE COMMA HARNESS AND THE FRAME BAND, AFTER THE ROUND. Unchanged at the default: 412/449 on the reference name (`uv run python tools/perf/call_count.py`), and the module-`parse` totals above hold exactly — `John Smith` 176, `Smith, John` 207, `John Smith, Jr.` 233, `Smith, John, Jr.` 257, `Smith, MA` 225. WITH SWITCH B ON the two figures recorded above each drop by one: `Smith, John` 213 → 212 and `Smith, XYZ` 245 → 244, `Parser().parse` on both sides as before. The cause is the dead caps branch leaving `ambiguous_class_candidate`: `segment`'s single-token call used to reach a `caps_shape_candidate` call that could only ever answer False, and now does not. Recompute with the recipe above. +- 2026-09-18, VERIFICATION ROUND — THE REPORT INVENTORY AN UPGRADER SEES, 2.3.0 → TREE, AND ITS RECIPE. docs/release_log.rst's comma-report bullet claimed TWO reports go away and named `Steven Hardman, MD, DO, DDS` and `John Smith, MD, R.A.I.`. The second is false and was never measured against the wheel: `John Smith, MD, R.A.I.` reports NOTHING at 2.3.0, having carried a `comma-structure` flag only inside this arc's own development, where rules/S3's narrow retirement raised it and the C2 amendment quieted it again. Over the whole corpus exactly ONE report is lost, and one more is SWAPPED. Measured: LOST — `comma-structure` ×1 (`Steven Hardman, MD, DO, DDS`), `given-or-family` ×2 (`Jack X.Y.I.`, and `e and e`, which is PR #527's conjunction change and not this arc's). GAINED — `suffix-or-name` ×37, `conjunction-or-initial` ×9 (#527's again), `given-or-family` ×2 (`Jack MA`, `Jack MA.`). So `Jack X.Y.I.` is the only name whose report CHANGED KIND on this arc's account, `given-or-family` → `suffix-or-name`, and the bullet now names it as the swap rather than counting it among the losses. Recompute: collect the corpus glob through compare.py's `_load_entries` deduped by name, parse each name on the tree and on the 2.3.0 wheel (a PEP 723 `uv run --no-project` script pinning `nameparser==2.3.0`, per AGENTS.md's warning that a same-tree comparison reports parity), and take the per-name multiset difference of the `ambiguities` kind values in both directions. The kind counts are multisets, not name counts — `e and e` contributes two `conjunction-or-initial` gains on its own — which is why a bullet quantified from the gate's per-rule name lists would not answer this question. +- 2026-09-18 (Derek), #531 — CAPITALS DECIDE FOR `do`, AND THE PARTICLE RULE KEEPS EVERY OTHER SPELLING. `do` is the one member of the ambiguous credential class (`ba`, `do`, `ed`, `jd`, `ma` — recompute with `sorted(Lexicon.default().suffix_acronyms_ambiguous)`) that is also particle vocabulary, so the trailing slot of the given part and P6's attachment want the same word. Where the writing carries a positive credential lean — an all-caps member in a name written in more than one case — the credential reading wins and P6 stands down: `Doe, John DO` reads suffix `DO` and reports `suffix-or-name`. Every other spelling attaches exactly as it did before, with P6's own `particle-or-given` and no second report: `Doe, John do`, `Doe, John Do`, `DOE, JOHN DO`, `doe, john do` all read family `do Doe` in their own case. THE PAIRING IS THE ARGUMENT, and the accepted cost and the reason are the two halves of it. In ONE CASE the rule cannot tell `NASCIMENTO, EDSON ARANTES DO` from `SMITH, JOHN DO` and reads both as the particle — right about the Brazilian record, wrong about the osteopath, whose `DO` 1.4.0 read as a suffix; both report `particle-or-given`, which is how a caller finds the second. In MIXED CASE it can, and does: `Nascimento, Edson Arantes do` keeps family `do Nascimento` while `Doe, John DO` gives suffix `DO`. Accepted rather than repaired: the one-case reading is the commoner of the two collisions, and the Nascimento record is the name a wrong answer would damage. The other four members are not particles and are unaffected. This is the exception rules/P6's "this attachment outranks the suffix reading (S2)" clause now names. Implementing it takes TWO carve-outs rather than one, which is worth recording because the second is in the REPORT and not the reading: the reading carve-out takes a particle-tagged member only on a POSITIVE credential lean, since `Doe, John do` leans nothing and the positional reading would otherwise take it — the wrong ANSWER, not merely a stray report; and the emitter beside it skips a particle-tagged member it did not itself take, since an emitter gated on tag membership alone reported both kinds on every `do` row. Where P6 owns the fork, P6 reports it, which is mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE read strictly. ONE SHAPE THE PAIRING DOES NOT REACH — a shape, not a spelling, and the difference is the review finding: ANY particle or particle chain standing in front of the caps `DO` takes it out of this slot, because grouping's prefix chain merges the two into ONE particle piece before the slot exists and there is then no lone member for the capitals to lean on. It is a fact about this WORD being particle vocabulary itself, not about particles generally — the caps `MA` in `Doe, John van MA` is not swallowed, reads as the credential, and leaves `van` to attach behind it. `Doe, John van DO` reads family `van DO Doe`, and so do the `de`, `von`, `del`, `la` and `van der` spellings of it (measured 2026-09-19 — parse them and read `family`). Pre-existing and unmoved by #531; `Doe, John van DO` is the case row and rules/P6's example line, and P6's statement carries the carve-out. The mirror shape is silent rather than merely unreached and is recorded on `AmbiguityKind.SUFFIX_OR_NAME`'s fourth boundary: a particle BEHIND the member ends the given part where this walk reads it, so `Doe, John MA do` keeps middle `MA` beside family `do Doe` though the capitals would otherwise have taken it, the attachment that moves `do` running a stage later. +- 2026-09-18 (Derek), #531 — THE COMMA REPORT'S REACH NOW INCLUDES THE GIVEN SEGMENT'S TRAILING SLOT, AND THE OPEN FOLLOW-UP IS CLOSED. The 2026-09-18 bullet above headed THE COMMA REPORT'S REACH IS THE FIRST POST-COMMA PIECE recorded this slot as an open maintainer decision and named the two questions it turned on — whether a middle initial's neighbourhood should start reporting (a noise judgement, rules/A1) and whether the silence was a 1.4 parity gap. Both are answered here, and that bullet stands as it landed. The slot now reads and reports: `Doe, John MA` gives suffix `MA` and `Doe, John Ma` keeps middle `Ma`, each saying which way it went. Derek chose to restore the ROLE and report both ways — one rule for both spellings — over a report-only change and over a capitals-only one, because the issue exists in the first place because two spellings of one name disagree. The noise question was settled by MEASURING THE DISAGREEMENT rather than by argument: over a generated sweep of 78 pairs — five listed members and two by-shape tokens in three cased spellings each, plus five controls in one spelling apiece, so 26 words against three name shapes — 48 pairs disagreed about whether the word was a credential or a name, and every one of the 48 disagreed in the same direction, the comma form declining what the comma-less form took. After this change 3 disagree and all three are the lower-case `do` rows P6 owns. The sweep and its allowlist are `tests/v2/test_properties.py`; a slot that answers differently from the same name written without a comma is not a quiet slot, it is an inconsistent one. Recorded under `3-0-reevaluations`' standing rule because v1 parity is LOAD-BEARING for one half of this and explicitly NOT for the other: `Doe, John MA` reads suffix `MA` on the 1.4.0 wheel, so the bare-acronym half RESTORES v1's role and the report is all that is new there — its 1.4.0 ledger rule neither retired nor narrowed to `_ambiguities` (nothing below baseline 2.0 can diff on that pseudo-field) but was RE-POINTED to the four names where the writing declines the credential, which keep the 2.0-era middle name against v1. `Doe, John X.Y.Z.` reads middle `X.Y.Z.` at 1.4.0 too, so the dotted half LEAVES v1 and carries a 1.4.0 ledger rule of its own; it moves to match the comma-less `John Doe X.Y.Z.` and rules/S3's shape rule, not to restore anything. BLAST RADIUS, stated the way this log's own rule asks: twenty corpus names move, and only TWO of them were in any corpus before this branch (`Doe, John MA` and `Doe, John X.Y.Z.`, both admitted by #530's own arc) — the other eighteen are this change's own case rows, so what the differential measures on pre-existing data is two names, and the population the rule reaches is a SHAPE (every family-comma listing whose given part ends in a member of this class) that the corpora barely sample. ACCEPTED COSTS, all measured on the differential corpora: `SMITH, JOHN DO` keeps family `DO SMITH` where 1.4.0 read suffix `DO`, paired with the Nascimento record in the bullet above; one-case and caseless names take the credential with no case evidence at all, so `DOE, MARY JO MA`, `doe, john ma`, `田中, 太郎 MA` and `김, 민준 MA` all read a suffix, which is what 1.4.0 read for each of the four — "caseless is inert" holds for the LEAN and not for the outcome, since `is_one_case` answers True for a script with no case and the positional reading then decides; `Doe, John van MA` loses its middle to the family, reading family `van Doe`, suffix `MA` with two reports where it read middle `van MA` in silence, which is `Berg, Jan van Jr.`'s reading arriving through a shape it could not reach before (Derek accepted it 2026-09-18 as P6 working correctly rather than as a cascade to carve out); `Doe, John Prof. MA` gains a TITLE role `Prof.` never had, H5's transparency reaching it once `MA` leaves the walk, landing it on the same answer as `Doe, John MA Prof.`; `Smith, LEED AP` moves under the default-off caps switch alone, to given `LEED`, family `Smith`, suffix `AP` with two reports, so no default reading is at stake; and a genuine middle name that is also a class member is now a credential wherever it ends the given part and is not Title-cased — `DOE, JOHN ED` reads suffix `ED` — which is the same cost the comma-less form has carried since 2.0. NOT REPAIRED HERE and left open: the maiden walk claims `Smith MA` whole in `Doe, Jane nee Smith MA` before this slot exists, so the rule cannot reach it and the name stays silent with maiden `Smith MA` — the same gap #530's close-out saw from the other side with `John Smith nee Jones R.A.I.`, and it is out of scope for #531. +- 2026-09-19 (Derek), #533 — THE CLASS'S LAST SILENT TRAILING POSITION IS CLOSED, AND #531'S READING NOW LIVES IN ONE PLACE. The slot list in rules/S2 gains the trailing slot of a maiden marker's clause, and #S3's enumeration gains it with the by-shape spellings; the rule that governs what the clause does with those words is M2's, and the entry under `### M2` above is where its reasoning lives. The gap the bullet above left open as out of scope for #531 is the one this closes, from both sides: `Doe, Jane nee Smith MA` now gives maiden 'Smith' with suffix 'MA' and reports, and `John Smith nee Jones R.A.I.` gives suffix 'R.A.I.' again — a RESTORATION rather than a change, since 2.3.0 read it that way (measured on the wheel) and this unreleased cycle moved it into the maiden name when #516 took dotted tokens out of the certain-suffix class, where no corpus file held the name and no gate could see it. Two things belong here rather than under M2. FIRST, `credential_at_the_given_slot` in `_pipeline/_pieces.py` is now where #531's reading of a class member ending the given part is written, with two callers — assign's walk over that part, and the maiden walk's second check over the name a take would leave. Spelling it twice is precisely the "condition written to match it" mechanisms.md#ONE-PREDICATE-PER-QUESTION names, and the drift would have been silent, since each site's own tests would have gone on passing. It is a text-and-tags question, which is what puts it in `_pieces` rather than beside either caller — the destination follows the LAYER, not the topic. It costs one frame PER MEMBER asked at that slot: measured 2026-09-19 against 2f57ff21 per `Parser.parse`, `Doe, John MA` goes 310 → 311 and `Doe, John MA Ma MA`, which asks four times, 439 → 443, while a name with no member there never reaches it and pays nothing. The reference band does not move (412/449 on `uv run python tools/perf/call_count.py`), no test pins 310, and Derek took the trade rather than keep two conditions in step across two stages with no test that asks them both. The maiden path got one frame CHEAPER in the same pass, the numeral reading calling the peel pair it had wrapped rather than the wrapper: `Jane Doe nee Smith` 249 → 248. SECOND, THE ONE-CASE HEAD IS AN ACCEPTED EXCEPTION AND IT IS THE M2 INSTANCE OF #492'S DEFERRED QUESTION about whether a cased suffix token counts as case evidence. `DOE, JANE nee Smith Ma` reads suffix 'Ma' where the clause-less `DOE, JANE Ma` keeps middle 'Ma', because the own-words span stops at the marker — rules/P3 puts "a maiden marker's run and every word after it" outside the name's own words from the moment the marker is tagged — so the member's own Title-casing is not in the span `one_case` is computed over and the clause HIDES the contrast. That is THE PREDICATE KEEPS THE JUDGED TOKEN IN THE SPAN, the 2026-09-14 #289/#516 entry at the head of this section, failing structurally rather than by oversight: at this slot the judged token is never in the span, and that entry's own answer — include it — cannot be had here. Widening the span for this one question would change `one_case` for the whole name, and three sites read it, so the exception is accepted instead. Measured 2026-09-19: 114 of 2016 generated pairs — every listed member and both by-shape spellings, in three cased spellings, against sixteen heads and six clause bodies of NAME WORDS ONLY — against 186 allowlisted and 984 disagreeing outside the class before the change, and 0 outside it after. The PR review widened that grid to three markers (`nee`, `née`, `geb.`) and three policies (the default and each 2.4 switch), and the class is INDIFFERENT to both: 1,026 of 18,144, exactly 9x the original in both columns, and still 0 disagreeing outside it. `tests/v2/test_properties.py` carries the sweep with the class defined STRUCTURALLY (`one_case` true of the clause form and false of the clause-less one) rather than as a name list. The recorded control beside it is now the SET and not only its size, as a digest over the members: a count cannot notice a swap, one pair leaving and another arriving, which is the same blindness the count was added to close one level up. +- 2026-09-23 #492 — THE DEFERRED QUESTION the 2026-09-14 predicate paragraph and the 2026-09-19 #533 bullet above name — whether a cased suffix token counts as case evidence — IS ANSWERED FOR RENDER ONLY. R5's gate now leaves the suffixes out (decisions/R5, 2026-09-23); the parse-time fact this section rests on keeps counting the judged token exactly as written above, so `Jack MA` still leans credential and `DOE, JANE nee Smith Ma` still reads suffix `Ma`. The split is decided and recorded under R5, with the shapes it makes visible there (`jack MA` at this slot, `john e jones III` at P3's). +- 2026-09-27 (Derek), #544 — A CREDENTIAL RUN KEEPS ITS AMBIGUOUS MEMBERS: C1'S NAME-WORD COUNT READS A RUN AS IT READS ONE WORD, AN UNAMBIGUOUS CREDENTIAL IN FRONT OF A MEMBER SPEAKS FOR IT AT EVERY TRAILING SLOT, AND A LISTED MEMBER'S CHUNKED DOTTED SPELLING PASSES THE PERIOD GATE. Three readers never looked past the member itself: C1 flipped a part to the suffix comma by the name-word count only when it was ONE token, and the trailing readers — the no-comma peel, the comma part read wholly as credentials, the given part's trailing slot — met the member first, took its name lean and stopped, so the credential in front of it was never reached. #540 marked `meng` and `lac`, whose conventional spellings lean name, which turned the one-case-row cost this section's 2026-09-15 item (ii) accepted into the ordinary spelling of two degrees. DECIDED, ten forks, (9) and (10) on 2026-09-28. (1) THE WHOLE RUN AT C1: two or more words after the comma, every one suffix vocabulary or a class candidate, at least one a candidate, behind two or more name words, is the credential run — all-member runs (`John Smith, Ed Ma`) and a leading member (`John Smith, MEng PhD`) included; offered and declined were an anchored-only rule and the issue's own "last word ambiguous" wording, which contradicted its `Ed Ma` consequence. (2) ALL TRAILING PATHS, ANCHORED IN FRONT: a listed member with an unambiguous credential in front of it, through other members, reads as the credential on the no-comma peel, in the comma part read wholly as credentials and at the given slot; a credential BEHIND it anchors nothing (`Wang Ma PhD` keeps family 'Ma'). (3) THE CHUNKED DOTTED GATE: a listed member written in two or more period-closed letter chunks (`M.Eng.`, `L.Ac.`) passes as `M.A.` does; one trailing period (`Ma.`, `Ed.`) still does not. (4) SINGLE-LETTER ROMAN NUMERALS neither anchor nor count toward a C1 run, S3's single-character retirement being the precedent; they peel as before, and the anchor's half of the exclusion is a matter of their shape, boundary (d) below. (5) NO NEW REPORT WHERE THE WRITING SETTLED IT: a run whose every member is LISTED and leans credential — S2's lean, capitals in a mixed-case name — is not flipped, the family-comma path already reading it as the credential run with the fields and the report it always had (`John Smith, PhD MA`) — narrowed 2026-10-01 by the #562 entry under C1: a part holding two particles side by side is read by the count instead, and reports; the alternative, flipping such a run and suppressing only the flip's report, removes reports the family-comma path makes today, `John Smith, MA Jr` losing the one it makes for 'MA'. Because the lean is the listed set's, a by-shape member in capitals never settles a run: `John Smith, X.Y.Z. MA` flips to suffix 'X.Y.Z. MA' and reports. (6) THE ANCHOR DOES NOT REACH ACROSS A MAIDEN CLAUSE: `Jane Doe Jr. nee Smith Ma` keeps maiden 'Smith Ma' where `Jane Doe Jr. Ma` reads suffix 'Jr. Ma' — rules/M2's Accepted boundary, and the M2 bullet of this date. (7) AN ANCHOR OVERRIDES P6 AT THE GIVEN SLOT: `doe, jane v phd do` reads suffix 'v phd do' where it read family 'do doe'; the bullet below amends #531's pairing. (8) THE DUAL EXCLUSION IS SCOPED TO THE GIVEN PART'S LEADING TITLE RUN, NOT THE C1 RUN: after a full name the dual counts as the suffix vocabulary it is, as the legacy disjunct already counted it (`John Smith, MS MA`, `John Smith, MD`), and that is what reads the issue's headline `Jane Doe, MS LAc` as suffix 'MS LAc'. (9) A MEMBER THE COMPANY DECIDES REPORTS WHERE IT WAS DECIDED, in a family comma's part read wholly as credentials as at the peel: each member whose own writing declined and which the credential in front made the credential reports `suffix-or-name` at that reading. `Smith, PhD Ma`, `Smith, PhD MEng` and `Smith, Ph. D. MEng` report the member; `Smith, Dr. PhD LAc` and `John Quincy Smith, Prof. PhD LAc` report 'LAc'; `john smith, v phd ma` reports 'ma'; `John Smith, PhD Ma Prof.`, which C1 does not flip for its trailing title, reports 'Ma' where its twin `John Smith, PhD Ma` reports C1's flip once over the part. A member its own capitals made the credential stays silent (`Smith, PhD MA`), and the part's first piece, which nothing stands in front of, is the first-slot report's word and never this one's. (10) A DUAL IN THE GIVEN PART'S LEADING TITLE RUN SILENCES THE WHOLE PART: refinement (b) below reads such a dual as a title that speaks for nothing, and in a part whose leading title run holds one no credential behind the dual speaks for a member of the ambiguous class either, so the company never reads such a part wholly as credentials: the titles give it its reading, the dual a title and the next word the given name, and behind that given name the given slot reads as it reads any (`Smith, MD PhD Jr Ma` reads suffix 'Jr Ma', as `Smith, MD Jane Jr Ma` does). A part with no member to speak for, or only members whose capitals decide them, reads as it did: `Smith, MD PhD` suffix 'MD PhD', `Smith, Ms MD MA` suffix 'Ms MD MA'. `Smith, Ms MD Ma`, `Smith, MD MS Ma` and `SMITH, MD MS BA` read title 'Ms MD' / 'MD MS', given 'Ma' / 'BA'; `Smith, MD PhD Ma` reads title 'MD', given 'PhD', middle 'Ma', the given slot reporting 'Ma'; e10e83b4 read each of them so. The run is the walk's leading title run, so a dual behind a plain title stands in it too (`Smith, Prof. MD Ma` and `smith, prof. md ma` read title 'Prof. MD', given 'Ma'; `Smith, Dr. MD PhD Ma` keeps given 'PhD'), while a plain title alone silences nothing (`Smith, Dr. PhD LAc` reads title 'Dr.', suffix 'PhD LAc'). Behind a real given name the company reads as ever (`Doe, Jane MD Ma` and `Doe, Jane Ms Ma` read suffix 'MD Ma' and 'Ms Ma'), and past the title run a dual speaks like any suffix word (`Smith, PhD Ms Ma` reads suffix 'PhD Ms Ma'); `Smith, Ms Ma`, `Nguyen, Sr Ba` and `Smith, MD Do` are unchanged. + WHAT MAY ANCHOR, four boundaries of the company clause, each a rule. (a) A CONNECTIVE (P3) anchors nothing and ends the run: the generational `i` is also Catalan's conjunction, and `Jane Doe nee Puig i Ma` keeps its link, maiden 'Puig i Ma'. (b) A TITLE/SUFFIX DUAL IN THE GIVEN PART'S LEADING TITLE RUN after a one-word family comma is a title there and anchors nothing — and, by (10), no credential behind it speaks for a member in that part — so the given slot's pass starts past the leading title run: `Smith, Ms Ma`, `Nguyen, Sr Ba` and `Smith, MD Do` keep their given names, and `Smith, MD MA Ma` keeps middle 'Ma', no unambiguous credential standing in its run once the title is set aside. Everywhere else a dual anchors like any suffix word (`John Smith MD MEng`, and (8)). (c) THE WALK'S OWN LEADING PIECE NEVER ANCHORS. In a name with no comma before the run, the leading piece is the name word H4's carve-out keeps whatever its vocabulary, and letting it anchor spends that name: in `Om Ma` and `PhD Ma` 'Ma' stays a name word under every name order (family 'Ma' in the default order, given 'Ma' under both family-first orders), where an anchoring `PhD` would leave given 'PhD', suffix 'Ma' and no family at all. One name word in front frees the same word — `John Om Ma` reads suffix 'Om Ma'. A family comma's given part has no such position, its first piece being read by the comma part's own credential reading, so `Smith, PhD MEng` reads family 'Smith', suffix 'PhD MEng'. (d) A SINGLE-LETTER ROMAN NUMERAL, IN ANY CASE (`V`, `v`, `I.`), anchors nothing for its SHAPE — one letter is how a middle initial is written, and S3 retired single-character vocabulary matches for the same reason (`John Smith PhD V Ma` keeps family 'Ma') — not because a generation is no credential. The rule is stated by that shape and the code tests exactly it (`_vocab.is_single_letter_numeral`); "initial-shaped" was the wrong word for it, `is_initial_shaped` answering False for a lower-case `v` that the exclusion covers too. And the exclusion is not about generations: a multi-letter numeral or generational word anchors as any suffix word does (`John Smith PhD III Ma` reads suffix 'PhD III Ma', `abdul Smith Jr Ma` suffix 'Jr Ma'). + ACCEPTED COSTS, each a reading this design chooses. `John Smith, Ms Ma` → suffix 'Ms Ma', reported, as `John Smith, Ms` alone already reads suffix (8). `Om Jr Ma` → given 'Om', suffix 'Jr Ma' and no family name: 'Jr' is not the leading piece, so it anchors 'Ma', and this section's standing Accepted clause already consumes an unambiguous suffix that leaves no family (`Om Jr` → given 'Om', suffix 'Jr'); every release from 2.0.0 made that same role assignment. `doe, jane v phd do` reports `suffix-or-name` where it reported `particle-or-given`, the fork reported being the credential pick rather than P6's attachment (7). And a one-word family comma whose given part OPENS with a capitalised `MA` in a mixed-case name, with an unambiguous credential behind it, reads the whole part as a credential run, family 'Smith' and no given name, where the parent kept given 'MA': `Smith, MA PhD Ma` read given 'MA', middle 'Ma', suffix 'PhD' and reads suffix 'MA PhD Ma'. 'MA' leans credential on its capitals, so it is read as the credential it is written as, and what follows it is then the company's (or, for `M.Eng.`, the chunked gate's). Measured 2026-09-28 on the grid below: 138 parses (46 texts, three orders each) go from given 'MA' to no given name, 72 of them with no `M.Eng.` in the run; 12 of the 138 are the parses the attribution below leaves unattributed, the rest falling in the anchor and `M.Eng.` classes. Each reports 'MA' at the first slot, and a member behind the credential reports as the company's pick (9). + ACCEPTED LIMITS, each keeping the member a name word, rules/S2's Accepted block and case rows: the merged `Ph. D.` is outside the no-comma walk (`John Smith Ph. D. MEng` family 'MEng'; the comma and given-slot spellings do anchor it, `Smith, Ph. D. MEng` and `Doe, Jane Ph. D. MEng` reading suffix 'Ph. D. MEng'), a title between the credential and the member breaks the run (`John Smith PhD Prof. Ma`), and a particle member P2 has chained before the peel is out of reach (`John Smith PhD Do Do` family 'Do Do'; `Smith, PhD Do Ma` given 'PhD', middle 'Do Ma'). + THE TWO-INPUT CHECK. tests/v2/test_properties.py's M2 clause-agreement walk asks every parse it takes both directions of the company clause: a listed member behind a qualifying credential reads as one, and a member read as a credential because of the word in front has that word in the suffix too. Recorded negative controls, from that test's docstring: 48 failing parses with the anchor off, 30 with the walk's leading piece allowed to anchor, 0 on this tree; re-run 2026-09-28 after (9) and (10), unmoved. The walk holds no comma head that opens its part with a dual, so (10) is pinned by case rows and a unit test instead (`a_dual_opening_the_given_part_turns_the_anchor_off`, `a_dual_opening_the_given_part_silences_a_later_degree`, test_pieces' `test_a_dual_in_the_leading_title_run_turns_the_anchor_off`). + REVERSED BY THIS ENTRY rather than edited: this section's 2026-09-15 ACCEPTED item (ii) (the bullet after this entry says so); the three "wrongly moved" reversals the 2026-09-14 paragraph THE RUN IS THE CAPS CLASS'S ALONE records — `John Smith, Ed Ma`, `John Smith, ma do` and `John Smith, X.Y.Z. A.B.` are credential runs now by C1's own count under every policy that keeps their words in the class (with `unlisted_dotted_suffixes` off, `X.Y.Z.` and `A.B.` are no class members and the third keeps the listing form), while that paragraph's narrowing of the CAPS run test stands; and #540's two pending questions (the `suffix-acronym-collisions` bullet of this date). + MEASURED 2026-09-28, the #544 tree against its parent e10e83b4, py3.11, after (9) and (10); the 2026-09-27 figures this paragraph first carried were taken before them and are superseded. Recompute: every run of one to three words from {PhD, MD, Jr, MA, Ma, MEng, M.Eng., LAc, Ed, Do, ba, MS, V, Prof.} behind each of `Wang {}`, `John Smith {}`, `John Quincy Smith {}`, `Dr. John Smith {}`, `John Smith, {}`, `John Quincy Smith, {}`, `Smith, {}`, `Doe, Jane {}`, `Jane Doe nee Smith {}`, `Doe, Jane nee Smith {}` and `Jan van der Berg {}`, each written as given, lower-cased and upper-cased and deduped, parsed under the three name orders at otherwise default policy, the seven role fields and the ambiguity kinds compared against the same grid parsed by e10e83b4. THE DETECTOR, which both the invariants and the attribution read, looks only at the run's own words, compared case-free: {phd, md, jr, ms, m.eng.} are the unambiguous credentials, {ma, meng, lac, ed, do, ba} the members; a member is ANCHORED where an unambiguous credential stands in front of it in the run with nothing but members between, except that `ms` or `md` opening a comma part anchors nothing; and a run word is IN A NAME FIELD where it appears among the space-split words of given, middle, family or maiden more often than the run's unconstrained occurrences of it account for. The two invariants count parses with an anchored member in a name field, and with an unambiguous credential in one. THE ATTRIBUTION puts each moved parse in ONE class, the first that holds, in this order: THE ANCHOR, where the tree's two violation counts are each no higher than the parent's and one is lower; THE CHUNKED `M.Eng.`, where the run holds `M.Eng.`; A LONE `v`, where re-running the first test with `v` counted as an unambiguous credential makes it hold; THE C1 RUN, a comma shape with two or more name words before the comma whose run is only credentials and members, at least one a member; and anything left unattributed. 254,496 parses over 84,832 texts; 89,825 move (29,943 texts, 19,123 of them with a field move): 39,261 the anchor, 14,676 the C1 run, 35,696 the chunked `M.Eng.` (32,472 of those only losing the report it raised as a pick), 180 a lone `v` (`john smith, v phd ma`, suffix 'v phd ma'), and 12 unattributed, the `Smith, MA PhD ba` parses above. Clean-to-clean — no run word in a name field on either side — 23,454 move, every one `M.Eng.`'s dropped report, and none elsewhere. The two invariants, 0 NEW violations of either: an anchored member in a name field, 29,199 → 1,530 (the residue is a particle member P2 has chained, a dual inside a leading title run, `smith, prof. md ma`, and the member behind a credential in a part a dual opens, `Smith, MD PhD Ma`, which (10) keeps a name); an unambiguous credential in a name field, 36,675 → 4,734 (the residue is an interior single-letter `V`, a non-final `Prof.`, the `PhD` a comma part keeps as its given name when a `V` or a title follows it, and the same `PhD` behind a dual that opens the part, (10) again). Against the tree before (9) and (10), the same grid moves 5,076 parses: 4,536 (1,512 texts, every one a comma shape whose part reads wholly as credentials) gain a `suffix-or-name` report and move nothing else, and 540 (180 texts, every one `Smith, ` then `MD` or `MS` opening the run, in each case form) return to e10e83b4's reading, fields and reports alike, from a silent suffix reading of the whole part. The differential corpora at the parent (1,418 names, three orders) move four names, every one intended: `John Smith, PhD MEng`, `john smith, phd meng`, `Wang M.Eng.`, `abdul Smith Jr Ma`. Frames per parse through `Parser().parse`, py3.11: the reference band is unmoved (370/407, `uv run python tools/perf/call_count.py`), and so is every ordinary name measured (`Smith, John Quincy` 260, `Doe, John MA` 283, `Doe, Jane nee Smith PhD` 338) except a comma name with two or more words on each side of the comma, which enters the C1 run test and pays `run_word_fold`'s one call before its first name word ends it (`Doe Smith, Jane Q.` 317 → 318, `Garcia Lopez, Maria Jose` 330 → 331, measured 2026-09-28); the new questions cost where they are asked — `John Smith Ma` 244 → 250, the anchor pass that finds nothing; `John Smith, MA Jr` 355 → 382, the case fact forced to see the capitals settle the run; `John Smith PhD MEng` 286 → 326; `Smith, MD PhD Ma` 349 → 362, the given slot's pass that (10) leaves with nothing to find — and `John Smith, PhD MEng` gets cheaper, 358 → 314, as does `Smith, PhD Ma`, 286 → 271. + NO MECHANISMS ENTRY IS OWED: the anchor pass is ONE-PREDICATE-PER-QUESTION's `_pieces.credential_anchors`, which `segment_suffix_reading` reads inline for the frame budget under a keep-in-step note, and its linearity is the answer #531's trailing floor and #397's `_run_neighbours` already give — one forward pass per question, never a look-behind per member. tests/v2/test_benchmark.py's `credential_run` shape guards it: the per-member look-behind measured 15.8× for 4× the input at base 800, against 4.13–4.21× on this tree at every base (that module's own record). + ADDENDUM 2026-09-28 (Derek). (e) A PARTICLE ANCHORS NOTHING, a fifth boundary beside the four WHAT MAY ANCHOR lists: a word of both the particle and the unambiguous suffix vocabulary (`vd` and `mc` in the shipped lexicon; `do` sits in the ambiguous half, so it is a member and never an anchor) heads the family name behind it (P2), and anchoring there split the family around a suffix — `Smith vd Ma, John` read family 'Smith Ma', suffix 'vd'; `Jan vd Ma` suffix 'vd Ma' and no family; `Smith Mc Ma, John` and `D. Mc Ba Ed, Smith` the same way. Each reads as e10e83b4 reads it again (family 'Smith vd Ma', 'vd Ma', 'Smith Mc Ma', 'D. Mc Ba Ed'). The exclusion is of the word in front only: a particle MEMBER behind a credential is still spoken for (`doe, jane v phd do` keeps suffix 'v phd do'), and a C1 run holding the particle is C1's reading rather than the anchor's (`Jan Berg, vd Ma` reads suffix 'vd Ma', as `Jan Berg, vd` reads suffix 'vd'). Measured over the grid of MEASURED above with `vd` and `Mc` added to its words (127,578 texts, 382,734 parses), against the tree before this addendum: 9,426 parses (3,142 texts) move, every one holding `vd` or `Mc` directly in front of a member; 8,940 return to e10e83b4's reading exactly, and the other 486 all hold `M.Eng.`, whose own change is (3) — 406 of them take e10e83b4's fields and differ only in the pick `M.Eng.` no longer reports, and 80 take the fields e10e83b4 gives the same text with `PhD` in `M.Eng.`'s place (`Jane Doe nee Smith M.Eng. vd Ma` reads middle 'Doe M.Eng.', family 'vd Ma', as `Jane Doe nee Smith PhD vd Ma` reads middle 'Doe PhD'). The differential corpora move 0 of 1441 names, and the grid of MEASURED without the two words moves 0 parses, so every figure there stands. THE PASS IS ASKED ONLY WHERE A SUFFIX PIECE IS IN REACH: `_pieces.anchor_in_reach` walks back through the lone members in front of a declined member on tags alone and answers False wherever the pass must — the first piece past the members is no suffix piece, or there is none — and it is asked only before the pass exists, one lookup answering after that, so a run of members stays linear. An ordinary name ending in a declined member stops paying for the pass (frames, py3.11, before → after, e10e83b4 in brackets): `John Smith Ma` 250 → 246 (244), `Smith, John Ma` 285 → 279 (275), `Doe, Jane MA do` 374 → 369 (363), `Doe, John Q. Ma` 328 → 320 (316), `Jan vd Ma` 308 → 243 (238); a name the pass does read pays the test on top, `John Smith PhD MEng` 326 → 329 and `Doe, Jane PhD MEng` 359 → 362. On the no-comma peel the reach starts past the walk's leading piece, which never anchors — so a by-shape member, which reaches the pass only at the peel's second position (a lean of None with a word to spare is consumed first), finds nothing in reach, and the by-shape guard that stood there was deleted as unreachable. `segment_suffix_reading` calls `credential_anchors` (`first_kept=False`, a comma part's first piece being a run member like any other) in place of the inline copy NO MECHANISMS describes, and the keep-in-step note goes with it. The pass is built only where the reach test finds a suffix piece in front of a declined member, which no member opening the part has, and a part whose name word stands ahead of any member returns before one is asked: a plain comma name pays nothing (`Smith, John` 182, `Doe, John MA` 283, `Smith, J. Q.` 246, `Smith, Ed` and `Smith, Ma` 213, `Smith, Ed John` 270, all as before the call replaced the copy; built on every declined member instead, the last three paid 215, 215 and 273), a title in front pays the reach test alone (`Smith, Dr. Ma` 253 → 254), and a part the company decides pays three frames over the copy (`Smith, PhD MEng` 273 → 276, `Smith, PhD Ma` 271 → 274; `Smith, PhD Ma John` 330 → 335, its pass built for 'Ma' before 'John' ends the part). THE TWO-INPUT CHECK's second control reads 42, not 30, with no change to the property: the walk's leading piece is now held out of the pass twice on the no-comma peel, by `credential_anchors` and by the reach test, either alone suffices, and with both removed the by-shape heads the deleted guard held back (`PhD X.Y.Z.`) fail beside the listed ones; the first control is unmoved at 48. And the 2026-09-28 C2 bullet's "a TAIL segment, which assign reads wholly as suffixes" holds outside a maiden clause standing in it — `Jane Doe, PhD, Jr nee van Ma` keeps maiden 'van Ma' — which rules/C2's statement now says. The reach test's one-way exactness is held by tests/v2/pipeline/test_pieces.py's `test_anchor_in_reach_never_hides_an_anchor`, over the case table's texts and runs of one to three words behind two heads; its recorded negative control, the reach test reading a `vocab:suffix` token as no suffix piece, fails at 878 member positions. +- 2026-09-27 (#544) — THE 2026-09-15 ACCEPTED ITEM (ii) IS REVERSED, and the bullet stands as it landed. `abdul Smith Jr Ma` reads given 'abdul', family 'Smith', suffix 'Jr Ma' — 2.3.0's reading — because the unambiguous 'Jr' in front of the Title-case 'Ma' anchors it (the entry above): the peel takes both, and P5's reserve, now seeing the family the join would take, declines the join. Item (i) stands. The case row is now `a_credential_in_front_anchors_a_declined_pick`, and rules/S2's Accepted block names the shapes that still keep the company out of reach. +- 2026-09-27 (Derek), #544 — THE #531 PAIRING GAINS A SECOND EXCEPTION, and CAPITALS DECIDE FOR `do` above stands as it landed. That bullet let only a positive credential lean override P6's attachment at the given slot; an unambiguous credential IN FRONT of the member now overrides it as well, the degree being a second and stronger signal: `doe, jane v phd do` reads suffix 'v phd do' and reports `suffix-or-name` where it read family 'do doe' and reported P6's fork. The one-case record the pairing protects has nothing in front of its particle, so `NASCIMENTO, EDSON ARANTES DO` still reads family 'DO NASCIMENTO' and reports `particle-or-given`. +- 2026-09-30 (Derek), #563 — PAIRED INITIALS ARE THE EXCEPTION TO C1'S NAME-WORD COUNT, AND A FLIP ON THE DOTTED SHAPE ALONE IS SILENT. The 2026-09-14 (Derek) entry above, #516's DOTTED HALF IS A SWITCH, read `John Smith, A.B.` as suffix 'A.B.' on the count of two name words before the comma. The count cannot tell a given name and a family name from ONE surname written in two words, and the dotted shape's own most common member is a person's initials, so the unreleased tree read `García Márquez, G.J.` as given 'García', family 'Márquez', suffix 'G.J.', `De La Cruz, M.J.` with no given name at all, and `van der Berg, A.J.` as given 'van' — where 1.4.0 through 2.3.0 read each as initials. A particle check would have caught only the last two; `García Márquez` and `Lloyd Webber` look exactly like `John Smith`. What the parser CAN see is the token, so the line is drawn there: two single letters joined by a period, with or without one after the second (`_vocab.is_paired_initials`; `García Márquez, G.J` reads as `G.J.` does), read as the given name after the comma and report the fork, and only an UNAMBIGUOUS suffix word IN FRONT of them that is not also title vocabulary, or a second word the class admits by dotted shape in the same part, makes them the credential run (`John Smith, PhD X.Y.`, `John Smith, X.Y. P.Q.`). Three letters or more (`X.Y.Z.`) and any longer chunk (`B.Tech.`) read by the count as before. Three choices made with it (Derek, 2026-09-30): (i) FRONT ONLY — a word behind the pair does not speak, so `De La Cruz, M.J. PhD` keeps given 'M.J.' and `John Smith, A.B. PhD` reads given 'A.B.', the direction rules/S2's company clause already takes; a LISTED dotted word behind does not speak either (`John Smith, A.B. Ph.D.` → given 'A.B.'), the second-word exception being for words the class admits by shape; (ii) SILENT FLIP — a flip resting only on words the class admits by dotted shape reports nothing, `B.Tech.` included, because the only such word a reader takes for a name is a pair of initials, and the comma flips a pair only where a word beside it has already said otherwise; a LISTED member in the same part still reports the flip (`John Smith, PhD MEng`, `John Smith, X.Y.Z. MA`); (iii) THE COMMA SLOT ONLY — the trailing slot (`John Smith R.T.`), the given part's trailing slot after a family comma (`Doe, John R.T.`) and the maiden clause keep #516's reading and its report; #563 leaves the no-comma question open. A generational word in front speaks like any unambiguous suffix word (`García Márquez, Jr. G.J.` → suffix 'Jr. G.J.'), as S2's company clause has it — unless it is also a title, so `García Márquez, Sr G.J.` reads title 'Sr', given 'G.J.'. ACCEPTED: `García Márquez, G.J.R.` reads suffix 'G.J.R.' — initials are conventionally written apart, as separate words this rule never reads (`García Márquez, G. J. R.` gives given 'G.', middle 'J. R.'), and three letters run together are far likelier a credential. REVERSED BY THIS ENTRY rather than edited, for the dotted half at the first slot after a comma only: the #516 SWITCH entry's `John Smith, A.B.` → suffix and its "reports either way"; the 2026-09-14 NO NEW AmbiguityKind entry's "Every decision at these slots emits" SUFFIX_OR_NAME; and rules/C1's "A decision either way at this comma is reported". ALSO FIXED: the trailing slot's report described a by-shape pick in the listed member's words ("written without periods is both a post-nominal"), false of `X.Y.Z.` on both counts; it now says the word is shaped like a post-nominal but listed in no vocabulary. + REVIEW ROUND, same day. The docs review found the first version let a word of both the title and the suffix vocabulary speak for the pair, since C1 counts such a word opening the part as a suffix word: `García Márquez, Ms G.J.` read given 'García', family 'Márquez', suffix 'Ms G.J.' with NO report — #563's own defect, made silent — where 2.3.0 read title 'Ms', given 'G.J.'. The twelve title/suffix duals (`ms`, `md`, `sr`, `lt`, `sa`, `ra`, `vc`, `cpl`, `cpo`, `cpt`, `csm`, `sgm`) all reached it. Fixed by letting only a suffix word that is not title vocabulary speak: in front of a pair, a dual is the title of the given part the pair opens, as S2 already reads a dual in that part's title run, and `Ms G.J.`, `MD G.J.`, `Lt G.J.` now read title plus given as every release did. The title lookup runs only once a pair is met. The code review's frame count found the single-token path asking `ambiguous_class_member` a second time, +2 frames on every reporting comma name (`John Smith, MA`, 252 → 254, profiler call events); for a word past the candidate test LISTED is exactly "no period" (`ambiguous_class_member` declines any period), which is now asked inline. Measured after both fixes against master, same counter: `Smith, John` 183 and `John Smith, MA` 252 on both trees, `John Smith, X.Y.Z.` 295 → 286 (the report it no longer builds), and `García Márquez, G.J.` 338 → 350, the family-comma reading and its report costing more than the flip did. + SECOND REVIEW ROUND, same day, on the first round's fix commit. (a) TWO PAIRS (Derek): `De La Cruz, M.J. K.L.` read family 'De La Cruz', suffix 'M.J. K.L.' — no given name — in silence, the silent-flip rationale ("only paired initials are taken for a name") failing on exactly the case where each word speaking for a pair is itself a pair. The reading stands, two dotted groups not being how anyone writes a person's initials, but that flip now REPORTS: when the only words speaking for a pair are other pairs, every shape word in the part being one. `John Smith, X.Y. P.Q.` gains the report with it; `John Smith, X.Y.Z. G.J.` and `John Smith, PhD G.J. K.L.`, where something else speaks, stay silent. The same rule reports `García Márquez, Ms G.J. K.L.`, whose honorific the run takes, where the first round's fix took it in silence. (b) A CLASS MEMBER IN FRONT SPEAKS FOR NOTHING (Derek): `García Márquez, Ed G.J.`, `Ma G.J.` and `MA G.J.` flipped on the member in front, where S2's company lets only an unambiguous credential speak for a member. Now the comma keeps the family, given 'Ed' — and 'G.J.' then ends the given part, the slot (iii) left alone, so it reads suffix 'G.J.' as `Doe, John R.T.` does, both forks reported; 1.4.0 and 2.3.0 gave middle 'G.J.'. Resolving that slot is the no-comma question #563 leaves open. (c) WORDING, no behavior: C1's sentence counting a dual opening a part as a suffix word now defers to the pair's titles; the pair's report reaches it only where it opens the part, `Smith, Ms G.J.` having always been silent (the comma report's reach is the first post-comma piece, 2026-09-18); and the silence is stated as "no listed word takes part", which covers `García Márquez, PhD G.J.`, where "dotted-shape words alone" did not. + SIMPLIFY ROUND, same day, behavior-identical (0 diffs over 21,604 parses: every corpus name, every quoted string in tests/v2/cases.py and 115 composed `pre, post` probes, under four policies, comparing fields and ambiguity details against the pre-round commit c125f69b; the same harness finds 252 diffs against master). One finding was a cost, not a style point: the speaker test scanned every word in front of EACH pair for a non-title, so `John Smith, MD MD ... G.J. G.J. ...` cost duals × pairs `_normalize` calls (163 at 8 of each, 1,387 at 32, py3.11). Only the first pair's scan can change the answer, since every later pair has the same words in front and more, so it is asked once: 107 and 395. `tests/v2/test_benchmark.py::test_the_paired_initials_title_scan_does_not_cost_quadratically` guards the ratio and fails at c125f69b. The run loop also asks LISTED as "no period", as the single-token test does, and `flip_reports` is set once after the run decision rather than piecemeal. + MEASURED 2026-09-30 against master b98b26e3, every name in this branch's `tools/differential/corpus*.jsonl` parsed on both trees with `nameparser.__file__` asserted on each side: 11 of 1453 distinct names differ, every one of them a name this change's rules.md examples and case rows put in the corpus (the two-pair names `De La Cruz, M.J. K.L.` and `John Smith, X.Y. P.Q.` are not among them: after the second round they read and report exactly as master does). THE POPULATION THAT COULD MOVE is the shape's, and the corpus barely holds it: over master's 1441 distinct names, 13 have a pair opening the part after the first comma, and `John Smith, A.B.` is the only one behind two or more NAME words with an unlisted, non-CJK pair (`Smith Jr., A.B.` has one name word, `Kenneth Clarke Q.C., M.P.` and `Virginia G. Essandoh, J.D.` hold listed acronyms, the rest one word) — so it is the only mover over that corpus, and the count is evidence about the corpus rather than about the rule's reach. Recompute: check out the parent into a separate worktree, parse every corpus name in each tree under `PYTHONSAFEPATH=1` with the tree's root first on `sys.path`, and diff `as_dict()` plus the sorted ambiguity kinds; for the population, take each name whose text after its first comma opens with a token matching `[^\W\d_]\.[^\W\d_]\.?` whole. +- 2026-10-01 (Derek), #564 — THE ALL-CAPS HALF READS THE COMMA POSITION BY DEFAULT, AND THE SWITCH HAS THREE SETTINGS. Supersedes the default of the 2026-09-14 entry above (its reasoning stands for the positions it was argued over). That entry turned the whole caps half off because French and Korean records write the SURNAME in capitals; but the convention writes them at the end of a name (`Jean DUPONT`) or before a comma (`DUPONT, Jean`), never after a comma behind a full name, so the reason for the off default never reached the comma position and that position was switched off with it. The corpus held three names of exactly that shape — `Ahmad Jayadi, CHA`, `John Smith, RAI`, `The Rt Hon Kenneth Clarke QC MP, HMG` — all credentials, all read as the given name at 2.3.0. + DECIDED, with Derek's calls marked. (1) `Policy.unlisted_caps_suffixes` becomes a `CapsSuffixes` StrEnum, OFF / AFTER_COMMA / EVERYWHERE, default AFTER_COMMA (Derek: an enum over two booleans or a mixed `bool | "comma"`, so every state is named and no meaningless combination exists). The field was new in 2.4 and unreleased, so no deprecation is owed; a bool raises a TypeError naming both replacements. The third value exists because redefining "off" would have left no setting that keeps a given name written in capitals (Derek's question, `García Márquez, GABRIEL`). (2) AFTER_COMMA reads only the part right after the first comma, behind two or more name words — the comma's own structure decision, which segment makes from the text before classify runs. The given part's last word after a family comma (`Doe, John XYZ`) and a maiden clause's last word are trailing slots and stay behind EVERYWHERE, as the comma-less end of a name does. (3) A lone two-letter word is never admitted at the comma, in any setting (Derek): it is the undotted form of #563's paired initials, and two words before the comma may be one surname, so `García Márquez, MJ` keeps given 'MJ'. Inside a longer part a two-letter word is read exactly as #563 reads the dotted pair — measured 2026-10-01 on the shipped tree by swapping `MJ` ↔ `M.J.` in runs of one to three words from {MJ, PhD, XYZ, CPA, Jr., MA, LEED, III} after seven name prefixes whose own words carry the contrast (`García Márquez`, `Smith Jones`, `Mr García Márquez`, `de García Márquez`, `García Márquez née Smith`, `John st Smith`, `García y López`): the comma's structure decision differed on 0 of 1295. After five all-caps prefixes it differed on 325 of 925, by design, `MJ` being no class member without the contrast while the dotted shape is case-free (`GARCÍA MÁRQUEZ, CPA MJ` keeps given 'CPA' where `CPA M.J.` makes the run). Recompute: run segment and read `ParseState.structure` for each pair. The equivalence is the comma's decision only: at the given part's last word after a family comma a dotted `M.J.` is a credential by shape (S3) where `MJ` is one only under EVERYWHERE, so `García Márquez, MA M.J.` reads suffix 'M.J.' and `MA MJ` middle 'MJ' — so `García Márquez, MJ PhD` keeps given 'MJ' as `De La Cruz, M.J. PhD` does, and C1 says so by pointing at #563's own sentences rather than restating them. Two restatements were wrong: the first draft's "three letters or more, or a run holding such a word" (both reviews), and its replacement, "with only a credential behind it ... while a credential in front makes the run", which #563's vocabulary rules contradict (`John Smith, MA MJ` reads given 'MA'; `García Márquez, MJ XYZ` makes the run). (3b) An unlisted all-caps word is a by-shape member of C1's run (#544), beside listed credentials, not only in a run of all-caps words (Derek, after review): before, `John Smith, XYZ` read suffix while `John Smith, PhD XYZ` read given 'PhD' and `John Smith, XYZ Jr.` given 'XYZ' — more evidence for a credential producing a name reading, already true under the old `True` and made a default by this change. The contrast has to be the NAME's, and it is carried only by one of the name's own words before the comma (`own_words`, so no maiden clause or delimited content) holding a capital and whose last LETTER is lowercase (`_vocab.written_as_a_name`), with no period and not claimed by a wordlist as a title, particle, connective, credential, generation, maiden marker or honorific (`_vocab.claimed_as_non_name`) (Derek). Every earlier draft asked for "a lowercase letter somewhere, less some exclusions", and each leaked a class the next review found: a mixed-case credential (`LLOYD WEBBER, ANDREW PhD`), a maiden clause or title (`LLOYD WEBBER née Smith, ANDREW PhD`, `Mr LLOYD WEBBER, ANDREW`), a generation, connective or shape title (`LLOYD WEBBER Jr., ANDREW`, `GARCÍA y LÓPEZ, ANDREW`, `Insp. LLOYD WEBBER, ANDREW`), and glued particles and unlisted lowercase words (`GISCARD d'ESTAING, VALÉRY`, `HAFEZ al-ASSAD, BASHAR`, `LLOYD ap RHYS, DAFYDD`) — no closed list separates an all-caps record from a mixed-case name. A Title-case draft (`str.istitle()`) closed those but read too narrowly, missing interior capitals and elisions (`DiCaprio`, `IJzerman`, `al-Rashid`, `d'Estaing`, `McDonald` — every one a mixed-case name that kept a given 'XYZ') and taking a lone capital as Title case (`de GAULLE C, CHARLES` lost its given name). Derek weighed "contains a capital" (an all-caps record with any lowercase word then always carries the contrast) and "not all capitals but contains a capital" (which takes `d'ESTAING` and `al-ASSAD`, the lowercase standing before the capitals) and first chose a capital directly followed by a lowercase letter, with a leading `Mc`/`Mac` skipped. Its review found the skip was one member of a class: any capitalized prefix glued to a surname written in capitals supplies such a pair (`LLOYD FitzGERALD, RONALD`, `PAOLO DeVITO, MARCO`, `JEAN LaFLEUR, PIERRE`, `DICK VanDYKE, JOHN`, `PAUL DuBOIS, JEAN`, `JEAN St-PIERRE, MARC`, and `LLOYD SMITH-McDONALD, RONALD`, the skip applying only at a word's start), as does `ß`, which has no capital (`MÜLLER WEIß, HANS`) — each lost its given name — while titlecase digraphs failed it (`Džokić Ljubić, XYZ`) and it cost a frame per letter. The shipped test, Derek's: a capital in the word and its last letter lowercase. A surname written in capitals ends in a capital whatever is glued in front of it, so every one of those records keeps its given name with no prefix list, and `McDonald` and `Džokić` pass. "Last letter" is the last letter that carries CASE, not the last character: the word is composed (NFC) first; every non-letter and every caseless letter is passed over (`Jones'`, `Smith2`, `Smith)`, `Jonesʼ` with U+02BC, the transliteration letters in `Wafāʾ`, a trailing `李`); and so is a lowercase letter whose capital is not one character (`ß` → `SS`, `ĸ` with none, the `fi` ligature), which is no evidence of case — `WEIß` is written in capitals, `Weiß` is not. Two cuts got this wrong: the first tested the last character, and its review found every name typed with decomposed accents and ending in an accented letter losing the contrast (`José André, XYZ`, `Lê Thị Hà, XYZ`, `René Noé, XYZ`, `Chloé Zoé, MBA XYZ` read given where their composed spellings read suffix), with `Jones'`, `Smith2` and `Smith)`; the second skipped non-letters but stopped at a caseless letter, so `Asmāʾ Wafāʾ, XYZ` and `JOHN Jonesʼ, XYZ` kept given 'XYZ' (its review). MEASURED 2026-10-02 against `a16927c6`, the capital-then-lowercase commit: the seven capitalized-surname prefixes (`FitzGERALD`, `DeVITO`, `LaFLEUR`, `VanDYKE`, `DuBOIS`, `St-PIERRE`, `SMITH-McDONALD`), `MÜLLER WEIß` and `KUJAĸ de PETERSEN` keep their given names, `Džokić Ljubić` gains the contrast, `JOHN O'NEILL's` and `JEAN-pierre DUPONT` gain one (accepted below), and nothing else moves; the decomposed-accent names, `Asmāʾ Wafāʾ` and `JOHN Jonesʼ` read as `a16927c6` reads them. OFF on this tree equals `unlisted_caps_suffixes=False` on `7394078a` (origin/master at the time) on all 4286 inputs. RECIPE, the comparator being those two commits (`git archive` each): the inputs are every name in `tools/differential/corpus*.jsonl`, every match of the regex `"([^"\n]{2,60})"` in `tests/v2/cases.py` holding a comma (a text match, not a literal parser, so fragments of case notes are included, harmlessly; extracting the literals exactly with `tokenize` gave 4273 inputs and the same conclusions), and each of these prefixes joined by ", " to each of the parts `XYZ`, `ANDREW`, `ANDREW PhD`, `PhD XYZ`, `MJ`, `XYZ Jr.` — `John Smith`, `DiCaprio LaBeouf`, `IJzerman IJsselmeer`, `al-Rashid al-Hassan`, `d'Estaing d'Orléans`, `McDonald MacLeod`, `Džokić Ljubić`, `Asmāʾ Wafāʾ`, `John Smith-Jones`, `john smith`, `García y López`, `LLOYD WEBBER`, `GISCARD d'ESTAING`, `HAFEZ al-ASSAD`, `LLOYD McDONALD`, `LLOYD FitzGERALD`, `PAOLO DeVITO`, `JEAN LaFLEUR`, `DICK VanDYKE`, `PAUL DuBOIS`, `JEAN St-PIERRE`, `LLOYD SMITH-McDONALD`, `MÜLLER WEIß`, `KUJAĸ de PETERSEN`, `LLOYD ap RHYS`, `de GAULLE C`, `Mr LLOYD WEBBER`, `LLOYD WEBBER née Smith`, `JOHN O'NEILL's`, `JOHN SMITHs`, `JEAN-pierre DUPONT`, `JOHN Jones'`, `JOHN Jonesʼ`, `JOHN Smith)`, `JOHN Smith2`, and the NFD forms of `José André`, `Lê Thị Hà`, `René Noé`, `Chloé Zoé`. Parse each under OFF, AFTER_COMMA and EVERYWHERE and compare `as_dict()`, the ambiguity kinds and `capitalized(force=True)`; the population is a fact of the tree it runs on (4286 here), since the corpora and the case table grow. ACCEPTED with it (Derek): a name written wholly in lowercase reads the listing form, `john smith, XYZ` keeping given 'XYZ', an unlisted title written in mixed case (`Doña GARCÍA LÓPEZ, MARÍA`) still carries the contrast, and so do two writings in an otherwise all-caps record that end in a lowercase letter: a glued possessive or plural (`JOHN O'NEILL's, XYZ`, `JOHN SMITHs, XYZ` read suffix `XYZ`) and a hyphenated part written in mixed case or lowercase (`JOHN SMITH-Jones, XYZ`, `JEAN-pierre DUPONT, XYZ`). ACCEPTED with (4): in a mixed-case name a capitalized given name followed by a generation goes the same way as one alone, `García Márquez, JUAN Jr.` reading suffix 'JUAN Jr.', reported; and behind a particle-led surname of two name words the flip leaves no given name at all, `De La Cruz García, MARÍA` reading family 'De La Cruz García', suffix 'MARÍA', because P1 reads a never-given particle's part as all surname. (4) ACCEPTED (Derek): a given name written in capitals behind a two-word surname without particles reads as the credential, `García Márquez, GABRIEL` giving given 'García', family 'Márquez', suffix 'GABRIEL', reported; capitalizing the given and not the family is not a convention, and OFF is the escape hatch. #575 was the prerequisite: before it, the count read `Van Buren, MARTIN` and `De La Cruz, MARIA` as two or three name words and this default would have taken their given names too. + WHAT MOVES, measured 2026-10-01 with the gate at all five baselines. The three corpus names above, plus the C1 examples `John Smith, XYZ` and `John Smith, LEED AP` — the latter closing a `deviates: #291` marker rules.md had carried under a closed issue — classified by `fix(#564)` in every ledger; the radar-unclassified count is what it was before the change at every baseline. `John Smith, RAI` and `Ahmad Jayadi, CHA` read suffix again by their capitals, as they did by vocabulary before #342 removed both words: parity at 1.4.0, only the comma's report at 2.0 through 2.2, so the #342 rule's `fields` lose `given` (the OVER-DECLARED check) and the watched shape for `John Smith, RAI` is re-recorded at those four baselines. `Smith, XYZ` keeps given 'XYZ' and, at the default, reports nothing; EVERYWHERE still reports the declined fork there, as it did. + CASE REPAIR FOLLOWS THE READING. The comma decision is segment's, made from the text, while case repair keeps a word in capitals only where classify wrote the shape tag (rules/R4); the first draft tagged only under EVERYWHERE, so `parse("John Smith, XYZ").capitalized(force=True)` rendered 'John Smith Xyz' at the default and 'John Smith XYZ' under EVERYWHERE (found by the docs review, axis 5). Classify now tags a caps-shaped word in the part a suffix comma opened under AFTER_COMMA too, so both settings render that part alike, and rules/R4 names the caps shape beside the dotted one; `test_render`'s forced-repair test fails with the tagging removed (an R4 example could not witness it, R5 leaving a mixed-case name unrepaired unless forced). A caps word in a third or later comma part is still tagged only under EVERYWHERE, so `John Smith, MD, XYZ` renders 'Xyz' at the default, as master does. + COST, `sys.setprofile` call counts (mean of 50, after one warm-up) against `git archive origin/master`: `Smith, John` 183 → 183, `Smith, JOHN` 183 → 183, `Smith, XYZ` 182 → 182, `John Smith, PhD` 217 → 217, `John Smith, MA` 252 → 252, `John Smith, Ph. D.` 254 → 254, `John Smith, CPA` 217 → 217, `John Smith, MD PhD` 260 → 260, `John Smith, MBA CPA` 261 → 261, `Dr. Juan de la Vega III` 366 → 366, `John Smith, XYZ` 251 → 266, and an all-caps record with a capitalized given name and a credential, `LLOYD WEBBER, ANDREW PhD`, 338 → 364, a constant +26 that does not grow with the name's length (`'LLOYD'*64 + ' WEBBER, ANDREW PhD'` 653 → 679; the second draft's per-character contrast test had made that +350). Past the all-caps check the contrast is linear in the words before the comma, about four frames a word (the generator, the case test's call, and the own-words walk's fold and marker test; `'GAULLE ' + 'ap '*k + 'GAULLE, CHARLES'` is +7, +26, +38, +54 over master at k = 0, 1, 4, 8), and constant in a word's letters (`'de ' + 'GAULLE'*k + ', CHARLES'` 253 → 275, 343 → 365 and 631 → 653 at k = 1, 16, 64), where the capital-then-lowercase draft grew a frame per letter; all paid only once a caps word is in hand. The comma test is on by default, so three C-level checks go before any call: two or more words before the comma (the first draft lacked it, and `Smith, JOHN`, a common record format, paid +25 for a flip it can never make — the code review), the first word in capitals, and the lone-two-letter length. Every caller of the caps predicate asks first, in C, what it would decline anyway — alphabetic capitals, not a listed suffix word — so a listed credential never pays for the call (the review of the first fix found `John Smith, CPA` +4 and `John Smith, Ph. D.` +5 before these). The 2026-09-14 entry's recompute recipe predates the enum: its `on` is `Policy(unlisted_caps_suffixes=CapsSuffixes.EVERYWHERE)` now, and its `off=Parser()` is `Policy(unlisted_caps_suffixes=CapsSuffixes.OFF)`, `Parser()` being AFTER_COMMA. +- 2026-10-02 (Derek), #573 — a never-given particle opening the given part after a family comma leaves no word to spare; the member behind it is the surname it heads, unless mixed-case capitals say credential. The record is the P6 section's entry of the same date. +- 2026-10-02 (Derek), #578 — A PARTICLE BETWEEN A CREDENTIAL AND A TITLE-CASE WORD STAYS SILENT, AS IT READS. `Doe, Jane PhD vd Ma` reads given 'Jane', middle 'vd Ma', family 'Doe', suffix 'PhD', reporting nothing. No new rule decides that: S2's given-part slot already says a particle in front of a member takes the member out of the slot, and where the word behind reads as a name the two are one name, asked nothing and reporting nothing — `vd` reads as `van` does (`Doe, Jane PhD van Ma`, the same reading), and S2's own boundary example is `Doe, John van Ma`. The writing is malformed whichever reading is meant: a Title-case `Ma` behind a lowercase `vd` behind a `PhD` is neither a Dutch listing nor a credential run as anyone writes one. A reader leaning anywhere leans suffix (Derek: both words follow a known credential, and the writer likely did not know how suffixes are capitalized), but that is a guess about a writer's mistake, not a convention the writing carries. Declined: (a) reading `vd Ma` as suffixes, which needs code to make a credential speak across a particle — against P2 (a particle joins the word behind it) and #554 (`vd` speaks for nothing behind it); (b) reporting the fork, which needs an emitter on the family-comma path for a decision the particle chain makes before assign knows which read the part gets (AGENTS.md: prefer leaving a fork silent and documenting the omission over machinery for input nobody reads with confidence). Where the writing does speak the parser already follows it: `Doe, Jane PhD vd MA` reads suffix 'PhD vd MA' and reports (#573). Pinned by the case row `a_particle_after_a_credential_heads_a_title_case_name`. +- 2026-10-04 (Derek), #602 — A CREDENTIAL AFTER THE NAME CORE STARTS A RUN TO THE END OF ITS PART. An unambiguous Latin suffix word (the Latin restriction retracted by #679, below) of two or more letters standing after two name words with no comma (a lone particle not counting toward the two; qualified by #627, below), or after the given word in the part after a family comma, makes every later word of that part a suffix, except a title word, which reads as a title: `John Smith PhD Jones` → suffix 'PhD Jones', `Eric H. Holder Jr. Attorney General` → suffix 'Jr.', title 'Attorney General'. A name word the run absorbs is reported `suffix-or-name`, a reader hesitating over `Jones` there. Before this the three comma shapes disagreed — middle 'Smith PhD' with no comma, middle 'Jones' and suffix 'PhD' after a family comma, and suffix 'Jr., Jones' after a suffix comma (C2's tail) — and the no-comma reading made a credential a middle name, the one reading nobody intends. The suffix-comma path already read it this way, and the comma spelling of the Holder name always did. + WHAT STARTS THE RUN is one predicate, `_pieces.starts_a_credential_run`, read by the trailing peel (`credential_run`, applied to every finished peel, so assign, P5's reserve, P2's chain stop and the maiden walk see one answer) and by the given-part walk. EXCLUDED, each with its reason: a TITLE WORD — 746 title words are in no suffix set (measured 2026-10-04: `len(L.titles - L.suffix_acronyms - L.suffix_words)` for the default lexicon), many of them surnames (`King`, `Bishop`, `Judge`), so `Mary Jane King Smith` keeps family 'Smith'; a title is read from the end of a name only behind a comma or with a period (H5, Derek). The AMBIGUOUS CLASS, which carries no evidence of which it is. A SINGLE LETTER, in any case: a lowercase `i` or `v` carries no `initial` tag, and found by the v1 off-switch test, `Josep Carod i Rovira` with `i` removed from the conjunctions read suffix 'i Rovira'. A CONNECTIVE. A word that is also PARTICLE or BOUND GIVEN-NAME vocabulary — found by the suite, not foreseen: `abd` is in the credential list (ABD) and heads `Abd Allah`, and `Mohamed Ali Abd Allah` read suffix 'Abd Allah' until it was excluded. The NON-LATIN HONORIFIC words, an honorific followed by a name word being a different question from a credential followed by one (retracted by #679, below). A word that is also a title (`MD`, `Ms`, `Sr.`) does start the run: position decides, and after the name core it is the suffix. + DECLINED: lifting the credential out — `John Smith PhD Jones` as given 'John', middle 'Smith', family 'Jones', suffix 'PhD', the family-comma path's old reading. It keeps every name word in a name field, but it reads a credential as something written in the middle of a name, which is the reading the rule rejects. + REVERSED BY THIS ENTRY rather than edited: the #578 bullet above (`Doe, Jane PhD vd Ma` now reads suffix 'PhD vd Ma', the credential starting the given part's run), and two of the three shapes #544's ACCEPTED LIMITS kept out of the company's reach — a title standing between the credential and the member, and a member a particle chain had taken. The case rows `a_title_between_degree_and_member_keeps_it_a_name` and `a_member_the_particle_chain_took_is_out_of_reach` keep their ids and now pin the run. The third, the merged `Ph. D.`, stays out of reach: that piece is outside the walk the run is read over. + MEASURED 2026-10-04, every name in `tools/differential/corpus*.jsonl` parsed on this tree and on aa401d80 with `nameparser.__file__` asserted on each side: 7 of 1,514 move, four of them this entry's own new rules/S2 examples. The three already in the corpus: `Jane van der Berg née Jr Jones` (suffix 'Jr Jones', where the marker's declined clause had left middle 'van der Berg née Jr'), and `John Doe, MD - PhD - FACS` and `Smith, MD - PhD - FACS`, whose dash inside the given part, a word there under every policy (#549), is now in the run: suffix 'PhD - FACS' where they read middle '-', suffix 'PhD, FACS'. The comma-part question — whether a part OPENING with a credential makes the comma a suffix comma (`John Smith, PhD Jones` still reads family 'John Smith') — is #603's. + +- 2026-10-07 (Derek), #614 — S2'S TRAILING RUN IS READ ONCE, BETWEEN P3'S CONNECTIVE JOINS AND THE PARTICLE CHAIN, AND BINDS. The issue's diagnosis held on the day: the words-to-spare count is taken over pieces and group's joins make pieces, so group re-asked the peel before and after its own joins -- the particle chain's stop and its re-ask and rollback, the bound-given reserve's two views (P5) -- and assign then read the run again. Now the run of a main segment (no comma, or the part before a suffix comma) of three or more pieces is read once, after the Ph. D. merge, the maiden take and P3's connective joins, which do not depend on the run and make the units it counts, and before the particle chain and the bound-given join, which do; the run is split off, the chain and the join group the name in front of it, and the run is put back unchanged (`_pieces.read_trailing_run`). The chain's stop is the end of what is left and the reserve counts the name pieces the read left; assign places and reports the run off the read (`ParseState.tail_reads`, `_assign._placed`). The read COUNTS a particle run and the plain name words after it as the one unit the chain will make of them (`_counting_view`, a copy; rules/P3's own count, mechanisms.md#UNIT-PARTITION), the words both particles and suffix vocabulary straight behind a particle among them, while anything the read weighs stays a word of its own -- suffix vocabulary, the ambiguous class and its shape-only members, roman numerals by shape, initials, period-marked titles. + STALE IN THE ISSUE, measured before designing: segment no longer decides the comma or emits S2 reports (#613 moved C1 to `_comma.decide`); assign predicts P1's fold but no longer P6, which it decides since #613; the two comma restart inputs (`John Smith, PhD vd DO`, `PhD DO DO`) no longer reach the chain, decide binding that part first; `credential_run` already applied in one place; and #613's `name_word_count` reads particle and suffix facts at P1's reach and ignores connective and bound-given joins, so S2 could not share it as it stood. The issue's 'licensed by neighbour', read literally, moves about 25 inputs (`JOHN SMITH MA`, six S3 example lines) and still five read as scoped to the duals, so it was not taken; the counting view's particle run is the one part of it the design needed. + THE FIRST DESIGN AND THE REVIEW THAT REVERSED IT. Derek decided first that the read WINS OVER EVERY JOIN, connectives included -- prototyped both ways in a scratch copy -- and that implementation passed the suite, the gates and the fingerprint. #614's PR review then found it broke what the corpus never tests: a salutation alone read its connective as the family name (`Mr. and Mrs.` gave title 'Mr. Mrs.', family 'and' -- the H5 chain took 'Mrs.' before P3 could join), a joined leading salutation counted as three words (`Mr. and Mrs. Jr` gave family 'and', `Mr. and Mrs. Do` a spurious report), and the read counted a particle chain's or a connective join's words one by one where rules/P3 says a joined part is one name word wherever another rule counts (`Freiherr von Berg X.Y.Z.` and `Juan y Garcia X.Y.Z.` took the acronym as a credential against every release, while `Freiherr Berg X.Y.Z.` kept it the family). DECIDED (Derek, 2026-10-07): connectives join first, and the read counts the chain's units -- which returns every one of those to master's reading, and most of the first design's 'tail beats connectives' class with them. A second-round finding the counting view itself made: folding the plain words after a particle into its unit swallowed a shape-only roman numeral (`Juan de la Vega VI` read family 'de la Vega VI'), the #610 class AGENTS.md's spelling sweep records, so a word the numeral fork reads by shape stays out of the unit. + DECIDED BEFORE THE REVERSAL AND STANDING: (1) the read wins over the particle chain and the bound-given join; (2) the consequences below, accepted as a class. + WHAT MOVES. One realistic reading: a trailing period-marked title behind a particle chain is the title, `John van der Berg Prof.` reading title 'Prof.', family 'van der Berg' (every release 1.4.0-2.3.0 and 0c54662b: family 'van der Berg Prof.'), which retires the cost rules/H5 recorded as accepted; the six heads of the stress grid below that end in a particle chain move the same way with `Dr.` or `Prof.` behind them. The garbage `Freiherr von Berg Dr. and Ed. Prof.` reads title 'Freiherr von Berg Dr. and Ed.', family 'Prof.' (0c54662b family 'von Berg Dr. and Ed. Prof.'): the title is read off before the chain, which then joins a piece carrying the joined title and reads as a leading title, so assign reads the name again. P5 no longer re-reads the joined view: `abdul J. V` and `Sir abdul J. V` read given 'abdul J.', family 'V' -- 1.4.0's reading restored; 2.0 and 2.1 read suffix 'V', 2.2 and 2.3 kept 'J.' a middle name -- the initial in front suppressing the numeral fork in the one reading taken, three name words standing and the join leaving two; and `Sheik abdul Ma` reports the pick the read weighed (decisions/P5, this date). Reports: a pick the particle chain takes into the name is reported by the chain, in its own words, and group takes it out of the read's picks so assign does not report it again (#614's /simplify: assign first skipped a pick it found already reported, inferring group's decision from group's output), so `Jan vd Ma`, `John de Ma`, `John van der Berg Ma` and `Nguyen Van Lac` report as on 0c54662b. The second review found why it must be the chain: assign words a pick by the role it holds before post_rules, and under a leading title-particle that is the given role the fold then moves, so with assign reporting `Freiherr von Berg Ma` told the caller 'read as a given name' of a word in the family. The chain also reports a member the read never weighed, one it takes short of the run, its scan ending at the next particle (`Freiherr von Berg Ed Do` reports 'Ed'). One report moves for the better: a pick the run keeps is reported as the suffix it reads as, `Juan y Garcia Mc Do MD van Ed` reporting 'Ed' 'read as a suffix' where 0c54662b said the particle chain took it into the name, of a word in the suffix. + THE SECOND REVIEW (bfa798fb, 2026-10-07) found four defects in the counting view and the report hand-off, every one on a shape the corpus, the gates and the first stress grid held no instance of, and each now has a case row that dies without its fix. (1) The read started at the end of the leading titles counted in PIECES, applied to the view: a title that is also a particle opens a unit inside the titles, so the index landed past the name and `Freiherr St van Berg MA` read family 'St van Berg MA' (0c54662b suffix 'MA'); the view now returns where the name starts in its own indices -- the first unit opened at or before the titles' end, since a unit opened inside them makes a name of them (SUPERSEDED IN PART by #624, decisions/P2, 2026-10-10: the chain now opens no unit inside the titles) even where it ends short (`Freiherr Freiherr Prof do` lost its report on 'do' until that second half). (2) Assign reported the chain's picks, in the wrong field's words, above. (3) The view made units INSIDE #602's credential run, which hid a title word from the run's title test: `John Smith MD van Secretary Jones` lost title 'Secretary' to the suffix. Past a run starter a unit is now the bare particle run, as the chain's own merge leaves it. (4) A `title` piece-tag exclusion in `_counted_as_one` made garbage differ from 0c54662b (`Freiherr von Mr. and Mrs. X.Y.Z.`) and guarded nothing (3) did not; removed rather than pinned. + NOT THE SAME READ AS M2'S: the maiden take reads its own clause-free view of the words before any join, word by word, as rules/M2 states, and its Accepted clause already covers what that does where a particle chain would take the run; S2's read counts the chain's units after P3's joins. Two reads of one name answer different questions -- where the clause ends, where the name's run starts -- and are kept apart rather than made one here. + RESIDUALS, the paths that read as before #614: a segment of fewer than three pieces (no join reaches it); a run with a name piece inside it (an H5 title in front of a name word the peel declined, `John Dr. G.J.`, `John van V Dr. V`), which is not split -- group re-asks and assign reads for itself; and a name group's joins leave without a name word in front of the run, which assign reads again so it keeps one -- the same procedure as before #614, not always the same answer, the Freiherr garbage above being the measured case. + MEASURED 2026-10-07 against 0c54662b (#618's merge), py3.11, `nameparser.__file__` asserted on each side; #619, merged between, moved no reading. The two #617 harnesses as decisions/P3's 2026-10-06 entry records them (the fingerprint, 78,324 parses; the connective grid, 485,300), and an S2 stress grid built for this entry: the 22 heads `John`, `John Smith`, `John Q. Smith`, `Mary Ann Smith`, `John van Smith`, `Juan de la Vega`, `John van`, `anh van`, `Jan Freiherr von Berg`, `Freiherr von Berg`, `abdul salam`, `abdul rahman al-said`, `Juan y Garcia`, `Josep Carod i Rovira`, `Jane Doe nee van der Berg`, `Dr. John Smith`, `JOHN SMITH`, `john smith`, `Smith`, `de Mesnil`, `Kim Min Jun`, `Ortega y Gasset`, each followed by every one- and two-word tail over the 35 words `PhD MA Ma DO Do do vd VD mc Mc MD Jr Jr. Sr III V V. VI I X.Y.Z. XYZ Ph. D. Esq. Prof. Dr. Sir MBA RN G.J. Ed van and Jones Ed. ba` (`Ph. D.` one word of the list); then 25,000 draws seeded `random.seed(614)`, each `choice(heads)`, `randint(3, 5)` and that many `choice(words)`, joined by spaces; then every head before a comma with every two-word pair behind it, and `Smith, ` and ` , John` for every word; deduplicated, under four policies (default, both family-first orders, strict comma suffixes) -- 81,138 names, 324,552 parses, comparing the seven roles, every ambiguity's kind and detail, `initials()` and `capitalized()`. Fingerprint: 6 parses move, `John van der Berg Prof.` under six policies, and nothing else. Connective grid: 1,248 move, no corpus or case name among them: 1,212 the title class, and 36 garbage shapes holding a title or a credential run behind a leading particle (`de der Dr. MA PhD Vega y Lopez` reads suffix 'PhD Vega y Lopez', #602's run, where 0c54662b kept every word the family), 8 of them moving a report's grouping alone (below). Stress grid: 8,472, no corpus or case name; 48 move reports alone, 20 the suffix wording above and 28 the grouping of an absorbed word's report where the run the view stops unit-making at is not the one the peel reads (`anh van Do Sr DO Jones MBA` reports 'DO' and 'Jones' where 0c54662b reported 'DO Jones'); restricted to a head and one tail word under the default policy, 12 parses, the six particle-chain heads with `Dr.` and `Prof.`, all the title class. (At bfa798fb, before the second review, these were 30, 2,936 and 20,532, the slice 33.) Whole tail readings per corpus name (`tail_reading` calls, 1,536 names): 1,125 read once, 56 twice, none three times, against 1,095, 55 and 30 on 0c54662b; peel passes, which count H5's fixed point's own re-peels, 1,215 names at one against 583. Five differential gates exit 0 with the one move classified `fix(#614)` at every baseline. + COST, `tools/perf/call_count.py` on each interpreter against adf6da88 (#619's merge): `parse` 317 -> 308 on 3.11 and 296 -> 290 on 3.12-3.15, `facade` 354 -> 345 and 333 -> 327 (decisions/parse-cost, this date). Per name on 3.11, warm: the Vega name 376 -> 368, and every segment of fewer than three pieces unchanged, never reaching the read (`John Smith` 152, `Smith, John` 161, `John Smith, PhD` 206). +- 2026-10-08 (Derek), #620 — S2'S READ COUNTS THE CHAIN'S UNITS OVER THE PIECES AS WRITTEN, RATHER THAN MERGING A COPY OF THEM. The #614 read above counted through a merged copy of the pieces, a second statement of P2's chain with its own stop list, and its reviews found three defects that were that copy's: an index applied across the two layouts (`Freiherr St van Berg MA`), a unit formed inside #602's run hiding a title (`John Smith MD van Secretary Jones`), and a shape-only numeral folded into a unit (`Juan de la Vega VI`), each patched in the copy. Now `_pieces._chain_units` marks each piece by the unit the chain will put it in -- OPENS, JOINED (a name word the chain joins to the run in front of it) or BOUND (a particle inside the run, a word the run took; retired by #625, this section, 2026-10-10, group merging the run first) -- asking `chain_run_end`, the chain's run end, which `_group`'s chain now calls too. The peel's words to spare (`second_unit`) and #602's run start count in those marks and the peel stops at any piece the chain joins into a run, which is a name word whichever mark it carries; everything else reads the pieces as written, so nothing is hidden, and the copy, its view index and its credential-run gate are gone. + WHAT THE COUNT KEEPS. A word the reading weighs -- a suffix word of either kind, the ambiguous class, a word in it by shape, an initial, a roman numeral by shape, a period-marked title word (`_weighed`) -- keeps a unit of its own though the chain may join it, as a position always counted it; only the plain words behind a particle run are JOINED, and a unit with nothing past its opener moves no name start into the titles (SUPERSEDED IN PART by #624, decisions/P2, 2026-10-10: no unit opens inside the titles, so the gate is gone). The first prototype marked every word the chain reaches JOINED, by the chain's own rule alone, and the review found it wrong in two shapes the earlier count read right: `Freiherr von Berg MA X.Y.Z.` kept 'MA X.Y.Z.' in the family (a member the chain would join counted as no word in front of the next), and `St St VI` and `Freiherr St MA` read title 'St St', 'Freiherr St' with the numeral or acronym the whole family (a unit of words the read then takes, which the chain never builds). Both masters read all three as they read now. (#624 since reads `St St VI` and `Freiherr St MA` as title 'St St' / 'Freiherr St' again, by a different route: the chain and the count now share one leading position, the last title-particle, so neither builds the unit -- decisions/P2, 2026-10-10.) The copy's exclusion list had been carrying the count positions always kept -- a member in front counts -- and the marks keep it without the copy. The numeral fork still reads the piece in front of the numeral as written -- reading the unit's first word broke rules/P2's `John van der J. V` -- so `John van B and Smith X` reads family 'van B and Smith X' again, the reading #614's copy had moved. + ALSO COUNTED IN UNITS: #602's run start, a particle opening a chain run counting once and its bound particle not at all (`John der la Jr. Prof. Smith` keeps its run, `van la Smith Secretary Jr. Dr. Smith` starts none, as on master). And `credential_run`, on the read's path, gathers adjacent particles into the one absorbed piece the chain makes of them (`Dr. John Smith Esq. RN Mc Mc` reports 'Mc Mc'; SUPERSEDED by #625, this section, 2026-10-10: group merges the run before the read, which absorbs it as one piece); M2's clause-free view reports word by word as it always has. + DECLINED, measured: a units test in `tail_reading`'s resume condition. A splice lowers a count only by taking out titles that open units of their own, and with fewer than two units in front of it every piece there is in the first piece's chain run, which the titles behind it join; fuzzed over 372,330 names, the test changed nothing. + CLASSIFICATION, found writing this entry's rows: eight case rows #614 added as `parity` did not match 1.4.0 -- they had been compared against master -- and are relabelled with the change each reading came from, bisected (#424, #289, #296, #516, #602 twice; R2 and P2 for two that read so since the 2.0 pipeline). + MEASURED 2026-10-08 against 4278693d (#621's merge), py3.11, `nameparser.__file__` asserted, with the #614 harnesses and recipe above: fingerprint 0 moves; connective grid 0; S2 stress grid 12, three garbage names under four policies, none in the head-plus-one-word slice -- `Freiherr von Berg Dr. X.Y.Z. VD` reads title 'Freiherr Dr.', suffix 'X.Y.Z. VD' (H5's trailing title, the class #614 moved; master family 'von Berg Dr. X.Y.Z.'), `Jan Freiherr von Berg VD V and ba I` reads family 'VD V and ba I' as adf6da88 did (master suffix 'I'), and `de Mesnil Ma do mc and Ph. D.` reports the 'do' the peel took beside adf6da88's 'Ma'. Five gates exit 0. Frames unchanged, 308 on 3.11 and 289 on 3.12-3.15 (decisions/parse-cost). + /SIMPLIFY (2026-10-08), measured. Each pass of `tail_reading` had rescanned for the second unit from the front, a C-level quadratic no frame guard saw: `Freiherr von` + `Berg `*n + `MA Dr. `*n read 5.77x for 4x the input against master's 4.08x; the lookup is now made once per read and handed to every pass (4.03x), and `test_the_second_unit_is_found_once_per_read` counts it. `_weighed` is pinned to the reading by `test_a_word_the_read_takes_is_never_one_the_count_joined` (no piece marked JOINED is ever in the run the read takes; 320 grid names fail with the test answering False). The gathering of an absorbed particle run reads the BOUND marks, and #602's run start one condition. Grids byte-identical, frames unchanged. Left for a follow-up, each moving readings: merging the tail's particle runs in group before the read, which would retire the BOUND stop and the gathering (80 grid names move, `Dr. mc mc`, and the chain's particle report needs re-keying; done by #625, this section, 2026-10-10, where #624 had already moved those 80); and the name start decided once, the item below. + LEFT OPEN, by Derek's scoping: reading a family comma's given part the same way, and deciding the name's start once for a title-particle head (#620's related paths, which change readings). The second was settled by #624 (decisions/P2, 2026-10-10), the first by #623 with a narrower bound rather than a second read (decisions/H5, 2026-10-10). +- 2026-10-10 (Derek), #627 — A LONE PARTICLE WITH THE RUN RIGHT BEHIND IT COUNTS AS A NAME WORD. #602's run start needs two name words before the credential, and "a lone particle is not a name word for the count", for `de Mesnil`, where the particle and the word behind it are one surname. Since #620 the count is taken in the chain's units, so a particle that opens a unit counts once (`van der`, `von Berg`) and a P3-joined phrase is one piece (`von und zu`); the lone particle stayed uncounted only where nothing joins it, which is exactly where a credential stands behind it. So `John von PhD Jones` read middle 'von PhD', family 'Jones' -- the chain later taking the credential as the particle's surname -- while `John van der PhD Jones` and `John von und zu PhD Jones` read family 'van der' / 'von und zu', suffix 'PhD Jones', and the comma spelling `Smith, John von PhD Jones` already read suffix 'PhD Jones'. The issue's question was which way to make them agree. + DECIDED (Derek): the particle stops at the credential, so it is a name word of its own and the rest is suffix: `John von PhD Jones` → family 'von', suffix 'PhD Jones'. The input is malformed (a particle is not written before a post-nominal), so the decision is whichever reading the shared count already gives the other forms, not a special case for this one: the existing clause gains its own limit, a lone particle counting when the run starts right behind it, in `run_start`'s one loop. The caller without the chain's units (`trailing_start`) shares the expression so the two cannot disagree, but no reading shows that half: it changes the position it returns for `Smith, John von PhD Jones` (2 where master gave 4), so group's chain takes its stopped path and asks `trailing_start` again, reaching the same reading at 16 more frames (339 → 355 on py3.11; a rare shape no frame guard sees, decided for the shared expression over a per-caller copy); a mutant ignoring the flag there passed the suite with the grid byte-identical (#627's reviews). The 1.4.0 and 2.3.0 wheels read middle 'von PhD', as they read `von und zu PhD`; the joined forms changed this cycle with #602. + Declined: counting every unit, a lone particle included. It moved 2,836 S2-grid parses rather than 2,432, the extra ones all `de Mesnil `, where it counted the surname as two words, and it split the two callers: a leading `van` counted before another particle (`van la Smith Jr. Jones` → suffix 'Jr. Jones') but not before a name word (`van Smith Jr. Jones`), and the case row `van la Smith Secretary Jr. Dr. Smith` caught it. + MEASURED 2026-10-10 against 195c4081 (#630's merge), py3.11, `nameparser.__file__` asserted on each side. S2 stress grid (the #614 entry's recipe above, 324,552 parses): 2,432 role moves, no report-only move; every move is a lone particle -- in that grid `van`, or a dual `do`/`mc`/`vd` in any case -- with an unambiguous credential right behind it becoming the family (under the family-first orders the given name, the family staying first), the run taking the rest. The grid's comma shapes do not move; the review found the same move in shapes the grid lacks, as intended: other particles (`von`, `de`, `du`, `la`, `bin`), and a head before a suffix comma (`John von PhD Jones, Jr.` → family 'von', suffix 'PhD Jones, Jr.'). Fingerprint of every `tools/differential/corpus*.jsonl` name and `tests/v2/cases.py` text at 195c4081 under the three orders (1,925 texts, the seven fields and every report): 0 moves; this change adds one text, its own example, which moves. The five differential gates exit 0. Held by the case row `John von PhD Jones` (`fix(#627)`) against the row `van la Smith Secretary Jr. Dr. Smith`, whose lone particle has a name word behind it and still does not count, and the row `Freiherr von vd PhD Jones`, where the particle standing right before the credential is one the run took and the flag is not asked of it (a mutant asking it of every piece passed the suite until that row). +- 2026-10-10 (Derek), #625 — THE TAIL'S PARTICLE RUNS ARE MERGED IN GROUP BEFORE S2'S READ, SO THE READ NEEDS NO BOUND MARK AND NO GATHERING. Since #614 split the trailing run off ahead of the chain, the read sees particles the chain has not merged yet, and #620 re-created the chain's particle-to-particle merge in the reader: `_chain_units` marked a particle standing straight behind another BOUND, the peel stopped at it, and `credential_run` gathered an absorbed run of them into one report. Now group merges each run of two or more adjacent particles past the chain's leading position (`_pieces.merge_particle_runs`, at the trailing site, after P3's joins and before the read) into the piece the chain would open its run with, so the read counts, stops at and absorbs the run as one piece, and the BOUND mark, the peel's stop for it and the gathering are gone. The merged piece keeps `prefix` and drops `title`: a P3 join carries the title tag in from a word in both vocabularies (`St und Berg`), and on the merged run it read the run as a leading title. The title run and the lead are read once, before the merge, and handed to the read, which had asked both again. The chain's PARTICLE_OR_GIVEN report keyed on its run claiming something past the particle (`j > k + 1`), which a merged run has done already though the chain then claims nothing more (`Freiherr von der`); it now keys on that or on the run being one the merge made (`premerged`) -- not on the piece holding several tokens, which a P3-joined phrase does too, and which moved 11,358 reports on the grid below (`Dr. St Do und`). The rule text is unchanged: rules/S2 already counts a particle run, the duals straight behind a particle included, as the one name word the chain makes. What grows is code, not concepts: the read loses a mark and two branches and re-creates nothing of the chain, group gains the merge (about 45 more lines than it removes across `_pieces.py` and `_group.py`, docstrings included). + DECIDED (Derek): retiring the two reader-side mechanisms is worth the moves -- every one is garbage input, now read consistently, and none is a released reading restored. + MEASURED 2026-10-10 against 505d6dec (#632's merge), py3.11, `nameparser.__file__` asserted on each side. A particle-run grid -- `{head} {run} {tail}` for every head in (none), `Dr.`, `Freiherr`, `St`, `John`, `Dr. John`, `Freiherr von`, `John Smith`, `Sir`, `Freiherr St`; every run of one to three words over `van der de la von und mc Mc vd do Do St du`; every tail in (none), `Berg`, `MA`, `PhD`, `Ma`, `Jr.`, `Berg MA`, `Prof.`, `Berg Ph. D.`, `V`, `VI`, `X.Y.Z.`, `née Smith`, `Berg Jr. PhD`, `Berg van`, `, Jr.`, `, John` (a comma tail written against the run) -- 395,148 texts, 1,185,444 parses under the three orders, comparing the seven fields and every report (kind, detail, tokens): 774 role moves (258 texts under each order), no report-only move. Every move holds `St und`, the joined piece carrying the title tag, which master merged into the particle run in front with the tag kept and read as a title (`Freiherr Do St und Berg MA` → title 'Freiherr Do St und Berg', family 'MA'); now title 'Freiherr', family 'Do St und Berg', suffix 'MA', the name part read alike with and without the trailing 'MA'. Not a released reading: 1.4.0 reads first 'Do St und Berg', last 'MA', and 2.3.0 family 'Do St und Berg MA', and over the 258 moved texts under the default order master's seven fields equal 2.3.0's on 152, the tree's on none. The S2 stress grid (#614's recipe, above): no role move, and one text's report under its four policies -- `de Mesnil Ma do mc and Ph. D.` no longer reports 'do': P3 joins `mc and Ph. D.` into one suffix-tagged piece, the merge folds 'do' into it, and the peel takes the piece whole rather than picking 'do', the report adf6da88 did not make either and #620 added (its entry above). The #624 title-particle head grid (decisions/P2, 2026-10-10): 0. Every corpus name and case text, and #624's comma-tail grid, under the three orders: 0. The five differential gates exit 0, their reports identical line for line to the tree before. The 2026-10-08 prototype's 80 moves (`Dr. mc mc`) are not in the population: #624 already reads `Dr. Mc Mc` as H3 states. Recorded negative controls (2026-10-10): with the merge off, 30 tests in tests/v2 fail (32 over the whole suite, measured at the review's fix), `test_an_absorbed_particle_run_reports_as_the_one_word_the_chain_makes` among them; with the report keyed on `j > k + 1` alone, the case rows `a_particle_takes_the_dual_behind_it_at_the_head` and `a_particle_run_before_the_run_stays_one_name_word`; with the merge starting at the segment's second piece rather than past the lead, 13; with `title` kept, only `test_a_merged_particle_run_is_a_particle_and_no_title`, the unit test added for it. Cost: +3 frames on the reference row, which holds `de la` (decisions/parse-cost, this date). + THE REVIEW (2026-10-10) found the change reaching further, every shape garbage, none given a row (Derek): no move goes the other way, and nothing moved over comma shapes (an empty or nickname-only head, a family comma's given part), connective-separated runs, runs beside a maiden marker or holding a nickname, bound given names, one-case names, or caller lexicons adding title or suffix words as particles. (1) The `St und` class is any P3 join whose left word is in both the title and particle vocabularies, `und`, `and` or `of` (`y` moves nothing: master already reads `Freiherr Do St y Berg MA` as now), a caller's title-particle included (`Freiherr Do Freiherr and Berg MA`: master title 'Freiherr Do Freiherr and Berg', family 'MA'; now title 'Freiherr', family 'Do Freiherr and Berg', suffix 'MA'). (2) A title-particle head, a particle run, then a title and a dotted word (`Freiherr van der Dr. G.J. MA`, 88 texts on its grid under each order): the run now counts as one name word, so 'G.J.' has none to spare, the read takes nothing and group's own chain takes 'Dr. G.J.' -- family 'van der Dr. G.J.', suffix 'MA', as the one-particle `Freiherr van Dr. G.J. MA` reads (master title 'Freiherr Dr.', family 'van der', suffix 'G.J. MA'). (3) H3's give-back behind a leading title where every word is suffix vocabulary, the last two a run of duals (`Dr. vd Jr. mc mc`, about 1,800 parses over two grids): the merged run is a particle piece and no suffix word, so the title stays -- title 'Dr.', family 'vd', suffix 'Jr. mc mc', where master gave 'Dr.' back as the given name with 'vd' still standing; `Dr. vd Jr. mc` still gives it back. The review also found two runs in one name unreached by any test (an off-by-one in the merge's bookkeeping passed the suite), now the rows `two_particle_runs_merge_each` and `a_second_run_of_duals_is_the_family` (parity) and a half of the merge's unit test, which also holds that a lone particle is no merged run; and the read's title run taken before the merge held only by the frame bands, now `test_the_read_takes_the_title_run_as_written_before_the_merge` (`Dr. mc Jr mc mc`, which with the run asked again after the merge, by the read or by group, reads family 'mc Jr mc mc'; the fix's own review found the group-side half unheld by a read-level test, so it pins the parse too). +- 2026-10-14 (Derek), #679 — ANY UNAMBIGUOUS SUFFIX WORD OF TWO OR MORE LETTERS STARTS THE RUN, WHATEVER ITS SCRIPT; THE HONORIFIC EXCEPTION IS RETRACTED. #602 excluded "the NON-LATIN HONORIFIC words, an honorific followed by a name word being a different question from a credential followed by one", and drew that line by script, the vocabulary having no honorific marker. #661 then added Cyrillic and Devanagari GENERATIONAL words the script line refused too, splitting Serbian by the script it is written in (#generational-words). Measured 2026-10-14 over every non-Latin suffix word (51) in six shapes under three orders (918 parses), the exception protected no correct reading: an honorific followed by a name word read as a MIDDLE name, in silence (`राम शर्मा साहब मोहन` gave middle 'शर्मा साहब', family 'मोहन'; `משה כהן ז"ל דוד` middle 'כהן ז"ל'), and dropping it reads the honorific as the suffix it is with the word behind it absorbed and reported, as `John Smith PhD Jones` reads. 360 of the 576 honorific parses move and 285 of the 342 generational ones, the same 120 and 95 under each order; the one-character honorifics (씨 様 氏 殿 군 님 양, and जी, whose vowel sign is no letter) never move, S2's two-letter minimum keeping them out already. No realistic name with a name word after an honorific was found, and no corpus name outside rules.md's own examples moves; of the three S2 examples that had pinned the refusal, two now pin the run, and the mixed-script `Taro Tanaka さん Jones` gives way to `राम शर्मा साहब मोहन`. Recompute: the words are master's `Lexicon.default().suffix_words` (after #681) whose letters are not all LATIN by Unicode name; each goes between same-script name words in the shapes `{a} {b} {w} {c}`, `{b}, {a} {w} {c}`, `{a} {b}, {w} {c}`, `{a} {b} {w} {c} {c}`, `{a} {w} {c}` and `{b} {a} {w} {c}`, under each of the three orders, keyed by the order's NAME (the first count keyed by its first role and lost an order, two orders both beginning with FAMILY); compare `as_dict()` and the ambiguity kinds with `_pieces.starts_a_credential_run`'s script test in and out. Generational words are not a class of their own here: a suffix word is a suffix word, which is what drops `_pieces._all_latin` (#681, the same day) and the need for any honorific or generational marker as #679's roster adds non-Latin words. + diff --git a/docs/design/decisions/T1-separators-not-joiners.md b/docs/design/decisions/T1-separators-not-joiners.md new file mode 100644 index 00000000..b5c11bfa --- /dev/null +++ b/docs/design/decisions/T1-separators-not-joiners.md @@ -0,0 +1,4 @@ +# T1 — separators, not joiners + +- 2026-07 (v2 core, PR #288) — v1's squash_emoji/squash_bidi REMOVED the character and joined its neighbors. (v1.3.0 had no bidi handling at all: squash_bidi entered late v1 via #266, 2026-07-07, on the emoji precedent's shape.) ('A😀B' → 'AB'); v2 makes an ignorable character a separator ('A😀B' → 'A', 'B'). The unavoidable consequence of every part being an exact positioned piece of the input: with no rewriting stage, nothing can splice two half-words together. + diff --git a/docs/design/decisions/T3-interpuncts-flank-guard.md b/docs/design/decisions/T3-interpuncts-flank-guard.md new file mode 100644 index 00000000..fa06f0b3 --- /dev/null +++ b/docs/design/decisions/T3-interpuncts-flank-guard.md @@ -0,0 +1,13 @@ +# T3 — the interpunct's flank guard + +- 2026-07-30 #298 — codepoint scope, chosen over a blanket + "dot = transcription" rule that would have flipped 高橋・一郎, a + correct and deliberately pinned #272 reading: U+00B7 records the transcription fact and suppresses division; U+30FB/U+FF65 stay pure separators whose pieces the script license reads (rule W4's Accepted pair demonstrates both). Cross-convention input reads by the codepoint it was actually typed with — a chosen limitation, recorded in #298's comments. + +Declined: + +- Ambiguity emission on a half-flanked interpunct ("王·Smith") — proposed in the 2.1 PR review, declined because the dot decides silently under the no-emission decision; the undivided-dot-stays-in-the-word behavior went to docs instead. + + +- 2026-07-30 #298 — U+00B7 divides only between classified-script characters because it is also the Catalan punt volat, interior to legitimate names (Gal·la). The nakaguro (T2) needs no such guard: its codepoints are CJK-only and appear in no other script's names, which is what licenses an unconditional rule. + diff --git a/docs/design/decisions/W1-unspaced-cjk-division.md b/docs/design/decisions/W1-unspaced-cjk-division.md new file mode 100644 index 00000000..62f16f2e --- /dev/null +++ b/docs/design/decisions/W1-unspaced-cjk-division.md @@ -0,0 +1,13 @@ +# W1 — unspaced CJK division + +- 2026-07-27 #271 (decision; shipped in 2.1.0 via PR #294) — Korean division ships as a default: the census surname list is closed, hangul is self-selecting (a hangul entry can only match hangul text), and being unsplit is recoverable while a wrong split is not — which is also why an unrecognized name stays whole. The filed proposal (#271, 2026-07-07) asked for OPT-IN segmentation for Korean too, "like all localization"; default-on is the later refinement, and the census/self-selecting argument above is what justified promoting Korean past the blanket opt-in stance. +- 2026-07-29 #272 (the ja amendment; shipped in 2.1.0 via PR #297) — Han division is opt-in per language pack because Han text does not identify its language (高橋一郎 under a Chinese list divides wrongly); a pluggable segmenter takes what the vocabulary declines, so pack + segmenter compose mechanically: a listed surname is a dictionary certainty and wins. + +- 2026-07-29 (ja amendment §1a) — script CLASSIFICATION NFC-normalizes its input (NFD katakana carries combining marks outside the block; NFD hangul decomposes to jamo, which would miss the order rule for macOS-origin names) while SEGMENTATION stays raw — chosen over offset-mapping complexity because NFD then degrades to no-split, never to a wrong split. The read-only folds are the deliberate exceptions to "nothing rewrites the text" (rules.md T Background), and neither ever enters a token. (Corrected 2026-09-10, #cjk-full-stops: this read "segmentation MATCHING stays raw" and "the one deliberate exception". Vocabulary matching composes NFC too since #322 — `_lexicon._normalize` folds every lookup and every stored entry the same way — so the folds are two, and what stays raw is segmentation proper: the surname site's membership test and the peel's tail slice, which index the token's own text. The no-split degradation is what still holds there.) + +Declined: + +- JMnedict as bundled segmentation data (#272's body, 2026-07-07) — the pip packaging (jamdict-data) was last compiled 2021-04; JMnedict carries no frequency data, so it cannot resolve the ambiguous 2+2/3+1 splits that motivate a segmenter at all; and bundling raised licensing questions the core avoids by delegating to namedivider — whose code and GBDT model are MIT, whose surname data carries Myoji-Yurai terms permitting exactly this name-dividing use, and whose CC-BY-SA BERT model is unused (the 2026-07-29 amendment corrected an earlier flat-"MIT" description; so does this entry). The kind of decline someone re-proposes in two years. + +- 2026-08 — zh and ja packs are corpus ALTERNATIVES, one per corpus; stacking them (parser_for(ZH, JA, segmenter=...)) is for genuinely mixed data that accepts the trade: a listed Chinese surname wins before the segmenter is consulted, so 高橋一郎 still divides 高 + 橋一郎 under ZH+JA exactly as under ZH alone. Kana gating resolves through the same script function order uses, so kana-licensed composites (高橋みなみ) gate in under JA while pure-katakana tokens never do (amendment 2026-07-27: activation is per script because the ambiguity is per script). + diff --git a/docs/design/decisions/W2-glued-honorific-peel.md b/docs/design/decisions/W2-glued-honorific-peel.md new file mode 100644 index 00000000..3373ef72 --- /dev/null +++ b/docs/design/decisions/W2-glued-honorific-peel.md @@ -0,0 +1,7 @@ +# W2 — the glued-honorific peel + +- 2026-08 #308 — an entry peels only where it can never end a name: 씨/님/さん/様/先生 peel; 양/군/氏/博士/殿 stay spaced-only; 君 is in neither set while its kana spelling くん peels. The vocabulary carries the license, so no structural or per-script gate stands over the peel. +- 2026-08 #312 — the peel crosses the family comma and the 间隔号: both answer where a name DIVIDES into surname and given, a question the peel never asks. Provenance matters here: neither gate was argued for the peel specifically — both came with the placement (#312's own framing) — so the crossing was a repair of inherited gates, not a designed-in property. +- 2026-08 #319 — a wholly suffix-shaped second run is declined as a peel site (the "田中さん, V." shape), but only when the name's own run offers a site, since a glued honorific is itself part of what makes a run read as suffix-shaped. +- 2026-10-17 #671 — SPACED AND AFTER-COMMA ONLY, no glued peel (Derek, 2026-10-17). 議員/议员, 国会議員/國會議員, 太平紳士/太平绅士, 医学博士/醫學博士, 二世 and the job titles 社長/部長/課長 and the simplified 社长/部长/课长 (Derek, 2026-10-17: the characters are shared and the offices are real in both languages; no borne-name data either way, #707) join `SUFFIX_WORDS`; none joins `GLUED_HONORIFICS`. The spaced bar is only that a word standing alone is no name, so they ship; the glued bar ("can it END a name?") needs real-name counts that this change did not get — the Wikidata suffix query timed out and the native-script Wikidata corpus holds none of the words — and 部長 fails it on argument alone (長 is a borne given name, 阿部長). A follow-up can vet 医学博士, 議員 and 太平紳士 one at a time. Consequence measured on `parser_for(locales.ZH)`: after a listed surname each word divides off, `王二世` family 王, suffix 二世 (it read given 二世) and `李部長` family 李, suffix 部長, because the segmenter's division is not a peel and the split-off 二世 is then a whole-token suffix word, the way 王博士 already reads; `陳怡君二世` still reads given 怡君二世 (glued, nothing divides it; superseded for the glued question by the #707 entry below). ジュニア was already shipped by #661, and #679 declined 二世 as CLDR's generation word ("regnal, and glued"); here it ships spaced-only as the Chinese/Japanese numeral, which does not reverse that. Excluded as the issue stated: 小 (a prefix and a common given-name character) and 冯/馮 for "von" (a top-30 surname). +- 2026-10-17 #707 — the #671 words meet the glued bar, all but 部長/部长 (Derek, 2026-10-17). 議員/议员, 国会議員/國會議員, 太平紳士/太平绅士, 医学博士/醫學博士, 二世, 社長/課長 and 社长/课长 join `GLUED_HONORIFICS`; 部長/部长 stay spaced-only on the #671 argument, since 長 is a borne given name (Wikidata holds it as a Chinese given name, Q96240389; the Japanese reading rests on the argument) and 阿部長 can be 阿部 + 長 — the measurement below found no such bearer, and a zero that is a lower bound does not refute the argument. The measurement is `tools/name_usage.py`'s ends-with row, new in #707: a local index of every CJK label on eleven name-item classes (family name, given name and their Japanese, Chinese, Korean, male, female, unisex and undifferentiated-kanji subclasses; 215,798 index entries on 128,695 items, 215,696 distinct item-label pairs, fetched 2026-10-17 by the first index, whose character class stopped at the Basic Multilingual Plane; the rebuilt index that takes supplementary-plane Han too (215,820 entries, same day) gave the same answer for every word, 殿 and the boundary pairs below), matched locally, with the matching items' bearers counted live. Two questions per word: does any name item's label END in it, and does any person bear a family name ending in the word's head and a given name equal to its tail (the 阿部 + 長 shape)? Neither found anything for any spelling, with one exception that is a data error: the only item ending in 二世 is Q18339792, the given name Odo, whose Chinese label carries a regnal numeral (厄德二世), and its 109 bearers are Western people named Odo. Of the boundary shapes, only 二+世, 社+長, 社+长, 部+長 and 部+长 had a family name and a given-name item to pair at all, and each counted 0 people. What makes the zeros evidence: the index agreed exactly with a live `STRENDS` query on all 20 class × word cells it was cross-checked on (five subclasses × 殿, 子, 郎, 田, the largest 403 items); 子 found 439 items and 2,436 given-name bearers, 2,081 of them Japanese; and 田 + 太郎 counted 22 people across the boundary. What limits them: Wikidata's coverage, not the index's — 殿 ends 13 name items where a surname dictionary lists about 90, and Japanese family-name items number 4,919. Two shortcuts were measured and rejected before the index: `STRENDS` over every label of a name class timed out at WDQS's 60s with no person join, and WDQS full-text search found 4 family names ending in 殿, matching only where its tokenizer split the character out. Consequence: the snapshot moved four CLDR renderings, each losing its glued word to `suffix` (`王雅婷婷婷二世議員`, both spellings of `…小惠儀太平紳士`, `馮·陳·怡君·達印·二世醫學博士`), and `陳怡君二世`, which the case table and rules.md W2 pin; cldr_agreement records one suffix field newly agreeing (`二世 醫學博士`, where the peel takes 醫學博士 and the 二世 left behind is a whole-token suffix word) and none regressing. The peel still takes one word: `王雅婷婷婷二世議員` peels 議員 and keeps 二世 in the name. Recompute: `uv run python tools/name_usage.py 議員 议员 国会議員 國會議員 太平紳士 太平绅士 医学博士 醫學博士 二世 社長 部長 課長 社长 部长 课长`; the first run builds the index (33 minutes on 2026-10-17, resumable), and `--refresh-index` rebuilds it. ACCEPTED COST, found by the design-docs review and kept (Derek, 2026-10-17): the vetting asked whether a NAME ends in each word and never whether a longer, unlisted OFFICE word does, and common ones do — 衆議院議員 and 参議院議員, 副社長 and 代表取締役社長, 元議員 and 市议员, 副課長, 獣医学博士. The peel matches at the end of the token, spaced or glued, so each splits at the listed tail: `岸田文雄衆議院議員` reads family 岸田文雄衆議院, suffix 議員, and `田中 副社長` given 副, suffix 社長, where before the word stayed whole in the name. Neither reading is right; the compound's head is what the peel leaves behind. No corpus held a compound office word when the review found this, which is why the snapshot was silent; the two W2 examples now put `岸田文雄衆議院議員` and `田中 副社長` in corpus_rules.jsonl. Older entries carry the same cost unrecorded until now (`김민준의학박사` reads given 민준의학, suffix 박사). Listing a compound whole would peel it whole, longest first. diff --git a/docs/design/decisions/W3-writers-divisions-are-respected.md b/docs/design/decisions/W3-writers-divisions-are-respected.md new file mode 100644 index 00000000..cb9c754f --- /dev/null +++ b/docs/design/decisions/W3-writers-divisions-are-respected.md @@ -0,0 +1,6 @@ +# W3 — the writer's divisions are respected + +- 2026-08 (segmenter amendment 2026-07-29) — a segmenter answers where an UNDIVIDED name divides, so it is consulted only when the gated token is the name part's only script-written one: "山田 太郎" was divided by its writer and must not have its family divided again. The peeled tail is exempt (that boundary was manufactured, not written); a SPACED honorific is not — at that position it cannot be told from a spaced name element. The trade was measured before deciding: counting spaced honorifics keeps four real surnames whole (佐藤 氏, 田中 様, 鈴木 先生, 中村 教授) and costs the one division 山田太郎 様. +- 2026-08 #312 — under FAMILY_COMMA the whole stage stood down until the peel moved in front of the gate: the comma doctrine is about the input's structure, not the split's source, so it covers vocabulary and segmenter identically, while an honorific is no part of the name whichever side of the comma it glues to. +- 2026-07 — the stage runs AFTER comma segmentation on purpose: running before would make the comma structure depend on the split ("김민준, Jr." pre-split would read suffix-comma on vocabulary alone; as written it stays the listing form). + diff --git a/docs/design/decisions/W4-script-scoped-order.md b/docs/design/decisions/W4-script-scoped-order.md new file mode 100644 index 00000000..8c4feca1 --- /dev/null +++ b/docs/design/decisions/W4-script-scoped-order.md @@ -0,0 +1,7 @@ +# W4 — script-scoped order + +- 2026-07-27 (script-scoped order amendment) — the family-first override is keyed to the SCRIPT of the written name, never to a guessed language: wholly-Han, wholly-Hangul, and kana-licensed Japanese read family-first because zh, ko and ja all write family-first in native script; wholly-katakana names are predominantly transcriptions and keep the declared order. Latin transliterations are never touched. +- 2026-07-29 #272 — the kana license: Han∪kana with at least one kana cannot be Chinese and is not a transcription, so 高橋みなみ reads family-first though it is written in two scripts. +- 2026-10-03 #594 — HALFWIDTH KATAKANA IS KATAKANA. The script table classifies the halfwidth kana block U+FF65–FF9F as `Script.KATAKANA`, so `山田 ﾀﾛｳ` reads family 山田 as `山田 エミ` does; every release through 2.3.0 left the block unclassified and read it given 山田, family ﾀﾛｳ, the positional reading. Legacy data written for JIS X 0201 systems, which had no kanji, spells names this way — bank, payroll and CSV exports. DECIDED by widening the table rather than by folding: nothing rewrites the text before parsing (rules.md's T Background), so normalization at tokenize was never available, and an NFKC fold at classification — the other place the issue offered — reaches far past the kana (`ＪＯＨＮ` to `JOHN`, `㈱` to `(株)`), and it would move a wholly-katakana name across W4's line: a voicing mark with no precomposed form to join (`ｱﾞ`, `ﾝﾞ`) folds to the COMBINING U+3099, which sits in the HIRAGANA block, so the folded token reads as kana-licensed Japanese and `ｱﾞｲ ﾀﾛｳ` would read family-first where the unfolded one keeps the declared order (measured 2026-10-03 by parsing the NFKC spelling `ア゙イ タロウ`, which reads family `ア゙イ`; a two-input sweep of halfwidth names against their NFC(NFKC) spellings differed only where such a mark stood; RECOMPUTE: parse each halfwidth name and `unicodedata.normalize("NFC", unicodedata.normalize("NFKC", name))` and compare the fields). The interpunct flank guard is NOT the hazard, though an earlier wording of this bullet said it was: it reads raw text and asks only whether a character is classified at all, which hiragana and katakana both are. The same measurement shows a pre-existing gap the fold would have widened rather than created: full-width katakana typed with an uncomposable combining U+3099 already takes the kana license today. A block range is how the table already decides (U+30FB and the voicing marks are in by block), and it takes the voicing marks U+FF9E/U+FF9F in because they are spacing characters following their base. The span starts at U+FF65, the halfwidth nakaguro, which tokenize already treats as a separator (rules/T2), and stops short of U+FF61–FF64, whose U+FF61 is a full stop. Everything keyed on the table follows: the kana license, the second-East-Asian-word count that keeps a segmenter from being asked (`高橋一郎 ﾀﾛｳ`), the 间隔号's flank guard (`ﾀﾛｳ·ﾔﾏﾀﾞ` divides), and `_NO_INITIALS`, which closes the limit #cjk-full-stops recorded (`ﾀﾅｶ. John` read title `ﾀﾅｶ.`). THE RATIONALE FOR WHOLLY-KATAKANA NAMES IS AMENDED, NOT THE BEHAVIOR: a wholly-halfwidth name keeps the declared order exactly as a wholly-katakana one does (`ﾔﾏﾀﾞ ﾀﾛｳ` reads given ﾔﾏﾀﾞ), but the 2026-07-27 reason — that such names are predominantly transcriptions — is weaker here, since a system with no kanji wrote Japanese names in katakana too, family-first. What stands is the narrower claim: the script cannot tell a Japanese reading from a transcription, so it settles no order, and a caller who knows their katakana is Japanese adds `(Script.KATAKANA, FAMILY_FIRST)` to `script_orders` (docs/usage.rst's Boundaries section carries the recipe). `name_order=FAMILY_FIRST` works too, at the cost of reversing Latin names as well. MEASURED 2026-10-03: the differential gate saw nothing before this change because no corpus line held halfwidth kana; the eight case rows #594 added are its whole population, five of them contract movers (`山田 ﾀﾛｳ`, `山田 ﾀﾞｲｽｹ`, `山田 ﾍﾟｰﾀｰ`, the unspaced `山田ﾀﾛｳ`, `ﾀﾛｳ·ﾔﾏﾀﾞ`), two tolerated radar movers (`ﾀﾅｶ. John`, `ﾏｲｹﾙ.`) and one parity row (`ﾔﾏﾀﾞ ﾀﾛｳ`). RECOMPUTE: parse the eight on this tree and on 2.3.0 from PyPI and read `title` beside the name fields. +- 2026-10-03 #596 — A VOICING MARK TAKES THE SCRIPT OF THE KANA BEFORE IT. The four kana voicing marks U+3099–309C are in the hiragana BLOCK, which is how the script table decides, but none belongs to either syllabary: a mark voices the kana it follows. NFC composes one only where a precomposed voiced kana exists (カ + U+3099 is ガ), so `ア゙イ`, `ン゙` and every spacing ゛/゜ kept a hiragana-block character inside a katakana word, the word took the kana license, and a wholly-katakana name read family-first — the gap the #594 entry above measured, present since 2.1.0. DECIDED by deleting, from the classification copy only, every mark standing after a KANA, so the word reads as its base's script; token text is untouched. A mark after anything else — hangul, Han, Latin, or nothing — voices no kana and keeps the table's answer, as before. That scope is a review finding, not a nicety: the first cut deleted every mark with ANY character before it, which handed `김゙민준` the hangul script, and the surname division, which cuts the raw text, then split the mark off its base — given 김, family `゙민준`, unreported, where 2.3.0 left the word whole and reported given-or-family; `parser_for(locales.ZH)` cut `王゙小明` the same way. A case row watches `김゙민준` — marked tolerated, the reading of a mark after a non-kana being unpromised (rules.md's W Background) — so the radar tier, not the contract, sees it move. Declined: taking the marks out of the hiragana range instead. Measured 2026-10-03 on patched copies of this tree, it fixes the katakana names as well but leaves a word holding a mark in NO script, so `あ゙い たろう` loses family-first, `山田 ア゙イ` loses the kana license and reads family `ア゙イ`, and the unspaced `山田ア゙イ` reads given; the two parity case rows #596 added pin those readings. Declined too: narrowing to the two combining marks, which leaves `ア゛イ タロウ` family-first. NFD-typed text was the issue's worry and is not affected by either option, since classification composes first: NFD `やまだ がくと`, `ジョン ガイ` and `김 민준` parse identically under all of them. A mark with nothing before it keeps the table's answer, so `゙アイ タロウ` still reads family-first; that is degenerate input, recorded in rules.md's W Background as unpromised rather than given a branch. ONE ARGUMENT IN THE #594 ENTRY ABOVE NO LONGER HOLDS: an NFKC fold would no longer move a wholly-katakana name across W4's line, since the combining mark it produces (`ｱﾞ` to `ア゙`) now reads as the katakana it follows — #594's own RECOMPUTE, run 2026-10-03 on this tree over `ｱﾞｲ ﾀﾛｳ`, `ﾝﾞ ﾀﾛｳ`, `山田 ｱﾞｲ`, `ﾏｲｹﾙ ｱﾞｲ`, `ｱﾟｲ ﾀﾛｳ`, `山田 ﾀﾛｳ` and `ﾔﾏﾀﾞ ﾀﾛｳ` under all three orders, finds no role differing. The range decision stands on its other ground, the fold's reach past the kana (`ＪＯＨＮ`, `㈱`). The interpunct clause of rules/W4 was corrected in the same change: a name the 间隔号 divides keeps the DECLARED order, not its source order — under `name_order=FAMILY_FIRST` `威廉·莎士比亚` reads family 威廉 — and an example pins it. MEASURED 2026-10-03: no corpus line held an uncomposable mark before this change, so the case rows #596 added are the whole population. At 2.1.0, 2.2.0 and 2.3.0 the three `fix(#596)` rows move; at 2.0.0, 2.1.0 and 2.2.0 `김゙민준` moves in its report alone, #449's given-or-family being new in 2.3, and is parity at 1.4.0, which has no report. At 1.4.0 and 2.0.0, which had no kana license, the three movers are parity, while `あ゙い たろう` and `山田 ア゙イ` move as every native-script CJK name does there, under the existing `fix(#271/#272/#298)` rule. RECOMPUTE: parse `ア゙イ タロウ`, `ア゛イ タロウ`, `マイケル ア゙イ`, `あ゙い たろう`, `山田 ア゙イ` and `김゙민준` here and on each baseline from PyPI. + diff --git a/docs/design/decisions/api-churn-declines.md b/docs/design/decisions/api-churn-declines.md new file mode 100644 index 00000000..880dd490 --- /dev/null +++ b/docs/design/decisions/api-churn-declines.md @@ -0,0 +1,4 @@ +# api-churn-declines — do not re-propose without new cause + +Declined (2026-07-06, the post-1.3.0 modernization sweep; each on churn-outweighs-benefit grounds): the hn.C → constants rename; capitalization-knob consolidation; the logger rename. (Lenient matching, declined in the same sweep, is recorded under comparison-surface.) + diff --git a/docs/design/decisions/cjk-comma-demotion.md b/docs/design/decisions/cjk-comma-demotion.md new file mode 100644 index 00000000..c93f9257 --- /dev/null +++ b/docs/design/decisions/cjk-comma-demotion.md @@ -0,0 +1,35 @@ +# cjk-comma-demotion — the script shapes are pure, the wrappers are tolerated (2026-09-01, #469) + +Closes #469, and continues the corpus-tier arc below rather than standing apart from it: the tier split gave the differential somewhere to WATCH a name without promising it, and this is the first doctrine narrowing to spend that. No parser behavior moves anywhere in it. Counts are this session's and every one is recomputable from the checked-in tree — `wc -l` over tools/differential/, `uv run python tools/differential/build_shapes_corpus.py --coverage`, and one `uv run python tools/differential/compare.py --baseline X` run per baseline, read off its `corpora:` and `corpus:` lines. + +- 2026-09-01 #469 — THE LINE IS FORMAT PURITY, and not comma class. Shapes 6 ("Family Given [Honorific]") and 7 (the interpunct/katakana transcription in source order) admit wholly classified-script text and refuse a comma and every ASCII letter; each composed form around a CJK name — the family-comma listing W3 reads, the honorific-comma probe zoo, a leading Latin title, a trailing credential comma — is TOLERATED input instead: parsed best-effort, contract-exempt, changeable without notice. What the inventory is FOR is the whole argument. It tabulates written ARRANGEMENTS, and a comma-wrapped or Latin-wrapped CJK name is not an arrangement any of the three writing systems produces — it is a listing convention from elsewhere with a CJK name inside it. So the admission test is a property of the FORMAT, and a rule reading such input describes rather than promises. + Supersedes an in-session fork of this same day's design, recorded because it is a line the next reader would redraw the same wrong way. The first cut kept trailing CREDENTIAL commas contract while demoting the family comma, on the theory that the two comma classes differ in kind — a credential comma being a Latin convention wrapped around anything, where a family comma makes a claim about the name's own division. The shape lens replaced it: both are wrappers, neither is a form the script writes, and a doctrine promising one while tolerating the other has to keep explaining a distinction its own admission test cannot state. Purity states the line in one sentence and needs no comma taxonomy at all. + +- 2026-09-01 #469 — SCOPES the FAMILY_COMMA bullet under 3-0-reevaluations below, which reads "inherited from v1's lastname-comma but correct on its own terms — an explicit comma is stronger evidence than script". It stands unchanged for LATIN script, which is the script it was argued about. For CJK script it is narrowed to a description: an explicit comma still beats script and the parser still reads it that way, but every input carrying one is a form no East Asian writing system produces, so what the parser makes of it is current behavior and not a promise. The bullet is not edited — it was right about what it was reasoning over, and this entry is the scope it never had to state while Latin was the only script in the room. + +- 2026-09-01 #469 — the 2026-08-23 (PR #428) snapshot under decisions/C1 below attributes `田中さん, Dr.` keeping its honorific glued to "(C1 Accepted)". That clause now lives in rules/W3's `tolerated:` note: the BEHAVIOR is unchanged and only its home moved, W3 being where the CJK comma forms are set down now that they are described rather than promised. The snapshot stays as written — it was true when it was taken, and this is the note that says where to read the clause instead. + +- CLDR, PULLED VERBATIM 2026-09-01 — #469 asked for the citation and asked that it be a pull rather than a recollection, so the failures are recorded with the findings. The paths named in the plan (`https://raw.githubusercontent.com/unicode-org/cldr/main/common/personNames/ko.xml`, `.../zh.xml`, `.../ja.xml`) all 404: there is no `common/personNames/` directory in the repository, and the data is a `` element inside `common/main/.xml`. Read there instead, from unicode-org/cldr at `main`, commit `8e4fb0fa1e287c4612b512cf2ef77e890467da28`; the grammar from https://www.unicode.org/reports/tr35/tr35-personNames.html. The grammar first, verbatim, because it decides whether absence is evidence: "A namePattern is composed of a sequence of field IDs, each enclosed in curly braces, and separated by zero or more literal characters (eg, space or comma + space)." A comma between surname and given is therefore expressible, and its absence below is a choice the locale data makes rather than a limit of the format. The `order="surnameFirst" usage="referring" formality="formal"` patterns, verbatim, long then medium then short: + ko {surname} {given} {given2} {credentials} · {surname} {given} {credentials} · {surname} {given} + zh {generation}{surname} {given} {given2} {credentials}{title} · {generation}{surname} {given} {given2-initial}{credentials} · {surname} {given} {given2} + ja {surname} {given2} {given}{title} · {surname} {given}{title} · {surname} {given}{title} + Counted rather than eyeballed: each of the three files holds 42 `namePattern` elements, and not one of the 126 contains a comma of any width (U+002C, U+FF0C, U+3001) or a semicolon. The delimiter between surname and given is a single space throughout. + THE ONE COMMA IN REACH, recorded because the record must not overclaim. It is in the locale-neutral root, whose sorting pattern reads verbatim `{surname} {surname2}, {title} {given} {given2} {credentials}`. `sorting` is a list-ordering format and not a referring one — UTS #35 verbatim: "Used to format names for a sorted list. example: “Brown, William” [medium, informal]" — and ko and zh override all six of their sorting slots comma-free (`{surname} {given}` and kin), while ja overrides five and leaves `order="sorting" length="medium" usage="referring" formality="formal"` as `↑↑↑`, inheriting root's comma-bearing pattern. So the honest reading, and it is the stronger one for being stated with its exception: no native pattern in any of the three writes a comma between surname and given, and the sole comma any of them can reach is INHERITED from the locale-neutral default, in the format whose entire job is sorted lists. That is the "listing convention carried in from elsewhere" the W Background already named, arriving from CLDR rather than from this project's own reasoning — the pull CONFIRMS the writing-system rationale, and complicates it only by showing the exact seam where a comma does get in. + Two findings worth keeping past the comma question. zh writes `{surname}{title}` with no delimiter at all (and ja `{surname} {given}{title}`), which corroborates rules/W2's glued honorific from the formatter's side, independently of the peel's own vocabulary argument. And ja declares `・` — a foreign name's spaces become the nakaguro — which is shape 7's divider reached from the formatting direction rather than the parsing one. + +- 2026-09-01 — the SHIPPED 2.2.0 release notes are left untouched, deliberately, and this bullet is the record of that choice. docs/release_log.rst describes the comma-crossing as a fix in two 2.1.0 entries ("Fix a comma or a 间隔号 stopping the glued-honorific peel", and #319's `田中さん, V.` entry). Both were true statements about what those releases did, made while the crossing was contract and #312 had argued it. Demoting the rule does not make a released note false about its release, and editing shipped notes to agree with a later doctrine would cost the log the one property it exists for. The demotion is 2.3's news and belongs in 2.3's notes; the log is a history, not a mirror of the current contract. + +- COUNTS, 2026-09-01, with the recompute beside each. 29 case rows carry `tolerated=True` over 26 distinct texts — the flag is per ROW and the corpus is per TEXT, which is why the two numbers differ (recompute: read `CASES` and count `c.tolerated`). The 26th text arrived with the W3 demotion itself: the given-side listing `지훈, 남궁민수` was a rules.md example with no case row, and would have left the harness altogether when the rules corpus stopped carrying W3, so it was given a tolerated row rather than dropped. `corpus_cjk.jsonl` went 98 → 73 and the radar-tier `corpus_cjk_tolerated.jsonl` was created at 25, then 26; the identity is the check worth keeping, since 73 + 25 = 98 exactly says the split MOVED names and dropped none, and the 99th is that one rescued row. `corpus_rules.jsonl` went 252 → 248: six comma-bearing example texts left (`田中さん, Dr.`, `田中さん, PhD`, `남궁민수, 지훈`, `지훈, 남궁민수` when the builder skipped W3 whole, plus `김, 민준씨` and `田中さん, V.` from W2's swap) and two pure ones arrived (`김민준씨`, `田中さん 様.`). `corpus_shapes.jsonl` stands at 37, with the new shapes covered 5 names (shape 6) and 2 (shape 7). And the stop condition held: intentional diffs are UNMOVED at 226/205/113/0 across 1.4.0/2.0.0/2.1.0/2.2.0, with unexplained 0, radar unclassified 0 and exit 0 at each — 1113 names compare at 1.4.0 (7 skipped, shapes 4/5) and 1120 above it. That the counts do not move is the substance and not a formality: a classified diff counts identically on either tier, so demoting a name changes which file it loads from and nothing about what the gate makes of it. + +- 2026-09-05 — THE SWEEP'S CRITERION WAS NARROWER THAN THE DOCTRINE IT IMPLEMENTED, and three rows sat in the gap. The 2026-09-01 sweep asked, over the CJK-bearing case rows, for a comma or an ASCII letter. A trailing ASCII period is neither, so `田中さん 様.`, `김민준 씨.` and `김민준 양.` — the three `fix(#320)` probes — stayed contract; and `田中さん 様.` was then chosen as rules/W2's replacement example for its "ignores surrounding punctuation" clause on the strength of passing that same test, so the miss propagated into the normative document the same day. A period standing after a CJK honorific is a listing artifact no writing system produces, which is this entry's own class in its own words: the admission test judges the FORMAT, and `様.` is no more an arrangement Japanese writes than `田中さん, PhD` is. The three rows carry `tolerated=True` now, and nothing about the parse moves — the rows still pin #320's mechanism at HEAD, and the texts are still compared and classified at every baseline, from the radar file instead of the contract one. CORRECTED IN REVIEW, 2026-09-05: a first draft of this bullet called all three "the spaced twin of a comma row that WAS demoted", and only two are. `田中さん, 様.` and `김민준, 씨.` are both case rows and both in `corpus_cjk_tolerated.jsonl`; `김민준, 양.` is no row at all — enumerating the CASES texts containing `양` returns `김지양`, `양 미선`, `김민준 양`, `김민준 양.` and `양 지훈`, and nothing with a comma. `김민준 양.` is demoted on the class argument alone, which is what carries the other two as well: an existing demoted twin is corroboration, never the warrant, or the class would end wherever the case table happens to stop. (Recompute: read `CASES` and collect the distinct texts containing the character.) + +- 2026-09-05 — THE CRITERION IS RESTATED AS THE DOCTRINE rather than as a sample of it. A SHAPE-TAGGED row (shapes 6/7, `Case.__post_init__` in tests/v2/cases.py) now admits no ASCII character but the space between two name words, so a period, a digit and a bracket are refused beside the comma and the Latin letter the check already refused, and the next composed form does not need a fresh criterion to catch it. The gate is a property of the shape TAG and not of CJK text, which is the half that keeps it honest: an UNTAGGED row may carry ASCII, and several deliberately do. The four mixed-script rows commit `d562164` reviewed one by one — `John 王`, `王·Smith`, `Andersonさん`, `Anderson선생님` — keep the contract tier on the reasons recorded there, untouched by this. + +- 2026-09-05 — `山田 太郎 (マイケル・ジャクソン)` IS KEPT contract: it is the one text the wider sweep surfaces and this entry declines to demote. Its ASCII parentheses are not a convention carried in from elsewhere — they are one SPELLING of a delimiter Japanese writing does produce, and the parser reads the spellings identically. Measured 2026-09-05 under the default facade: `(マイケル・ジャクソン)`, the fullwidth `（マイケル・ジャクソン）` and the corner-bracketed `「マイケル・ジャクソン」` all give family 山田, given 太郎, nickname `マイケル ジャクソン` (recompute: parse the three; ASCII square brackets are NOT in that set and give family `ジャクソン]`, which is what makes this a delimiter roster rather than a rule about brackets). What the row pins is the nakaguro INSIDE a delimited nickname (`fix(#272)`), and which of the three delimiters spells it is incidental to that. + +- 2026-09-05 — W2's PUNCTUATION HALF MOVED to W3 with its example; ITS SECOND HALF IS NORMATIVE AND STAYED, and this bullet is the within-branch correction of a first draft that took both. The clause read "The split-off ignores surrounding punctuation, but never treats a part that is not name text as the name's end". The FIRST half had no pure witness and could not be given one — native CJK writing puts no punctuation around a name for a split-off to ignore, so every input it describes is a listing form, which is W3's ground — and it moved there as description, rewritten to what is measured rather than to what it asserted (the bullet below). The SECOND half is a different claim and the reason given for cutting it does not reach it: it is what the parser does with NO comma and no period anywhere in the input, and contract-tier pure names depend on it. Measured 2026-09-05 by forcing `_is_post_nominal` to False in memory — the predicate the site scan `_peel_site` reads in nameparser/_pipeline/_script_segment.py — and re-parsing (family, given, suffix, title): `田中さん 様` ships `('田中', '', 'さん, 様', '')` and gives `('田中さん', '', '様', '')` without the step; `김민준 박사님` ships `('김', '민준', '박사님', '')` and gives `('김', '민준', '박사, 님', '')`; `선생님` ships `('선생님', '', '', '')` and gives `('선', '생', '님', '')`. The three break DIFFERENT clauses of the half, and a first draft of this bullet said all three were dissection. The restored sentence makes two claims — the post-nominal word is never the name's END, and it is never DISSECTED — and `田中さん 様` fails only the first: `様` comes out WHOLE as the suffix, undissected, but it is what the split-off takes as its SITE, so the peel lands on the post-nominal word and `田中さん` behind it is left unpeeled, where shipped the scan steps past `様` and peels `さん` off `田中さん`. `김민준 박사님` and `선생님` are the dissection failures, `박사님` coming apart into `박사, 님` and `선생님` into `선, 생, 님`. Between them the three witness both clauses, and neither clause is witnessed twice over. Two of the three, `김민준 박사님` and `선생님`, are `corpus_cjk.jsonl` names on the CONTRACT tier, and `ko_honorific_glued_doctor_spaced` is the #308/#271 regression row for the first; that row pins `suffix="박사님"`, which is what `parse('김민준 박사님').suffix` returns under the default facade, measured 2026-09-05. Under W2's remaining FIRST statement alone — a listed honorific glued to the end of the name's last name word splits off once — `박사님` ends in the listed tail `님` and reads as peelable, giving `박사` + `님`. So a W2 cut back to that one sentence would have BEGUN contradicting the regression row its own example line pins, not stopped: the second half is what keeps rules.md agreeing with the shipped parse, and that is the argument for restoring it. Restored in W2's register as "A part that is not name text — a post-nominal word standing on its own — is never the name's end: the split-off steps past it to the name word behind, and never dissects it." — witnessed by `김민준 박사님` → `suffix="박사님"` and the boundary line `선생님` → `family="선생님"`, both texts `corpus_cjk.jsonl` already held, so the two example lines arrive in `corpus_rules.jsonl` without adding a name to the pool. `田中さん 様` would have been the third witness and is NOT added: it is not a case row, and a new contract name needs a 1.4.0 classification this bundle does not have. `_script_segment.py` carries the restored sentence as a second W2 citation, on `_peel_site` where the scan-back lives. RECOMPUTE: rebind `_is_post_nominal` to `lambda state, i: False` on the module and re-parse the three. + +- 2026-09-05, FOUND while rewriting the clause — A PERIOD ON THE NAME'S OWN LAST WORD IS NOT IGNORED; IT IS DECISIVE, and the moved half asserted the opposite. The half said the split-off "ignores surrounding punctuation", and W3's first descriptive rewrite of it kept that shape, attributing the crossing to the punctuation. Five parses, 2026-09-05 under the default facade, say otherwise — counted by INPUT STRING, the convention the rest of this bullet uses when it says "the five inputs" and "parse the five strings". A first draft counted four by counting the SENTENCES below instead, the last of which carries two inputs (`田中さん.` and `김민준씨.`). Six, if the no-period twin `田中さん 太郎` quoted inside the third sentence as its control counts as a parse of its own rather than as part of that item. `田中さん 様` → family `田中`, suffix `さん, 様`: the crossing happens with no punctuation in the input at all, so the vocabulary licenses it and not the period. `田中さん 様.` → family `田中`, suffix `さん, 様.`: the period rides along and moves nothing, `様.` reading post-nominal-strict exactly as `様` does. `田中さん 太郎.` → family `太郎.`, given `田中さん`: a period on a NAME word, and no crossing — and not ignored either, since the same input without it is family `田中さん`, given `太郎`, the order reading flipped by the period alone. `田中さん.` → title `田中さん.`, and `김민준씨.` → title `김민준씨.`: a period glued to the honorific's own word stands between the honorific and that word's end, no listed tail matches, the peel declines and the whole text reads as a title — the parse a period changes most, not least. Of the five inputs, the ONLY one where a period changes nothing is the one where it sits on a word the split-off was going to step past regardless. So W3 now says the step past a post-nominal word is W2's and needs no punctuation, that a period on a separate post-nominal word rides into the suffix, and that a period glued to the honorific's own word decides against the split-off. AMENDED 2026-09-10 by #cjk-full-stops, and it is the summary that goes rather than the measurements: every parse above keeps its date and is what the parser did on 2026-09-05 (the two suffix values render space-joined since #436/#437, the next day, `さん 様` for `さん, 様`). Of the four period-bearing inputs, ALL FOUR now divide exactly as their no-period twins — `田中さん 太郎.` reads family `田中さん`, given `太郎.`, the classification fold making the period invisible to the order rule, and `田中さん.` and `김민준씨.` peel their honorific with the stop riding on it — so the period changes no division at all here, and the third clause of the sentence before this one is superseded: a period glued to the honorific's own word rides WITH the honorific now. What survives is the first two clauses and the finding this bullet is titled for, inverted in its consequence rather than in its content: the period was decisive because a site was reading it, and the answer was to stop three more sites reading it. RECOMPUTE: parse the five strings, each also without its period, and read `title` alongside the name fields — a rewrite that reads only `family` and `suffix` sees two empty results and can call them a null effect. + +- COUNTS, 2026-09-05, with the recompute beside each. `corpus_cjk.jsonl` went 73 → 70 and `corpus_cjk_tolerated.jsonl` 26 → 29 — the same identity as the first demotion, 99 names either side, so the split MOVED three texts and dropped none. `corpus_rules.jsonl` went 248 → 247 → 249: W2 gave up the `田中さん 様.` line, then took the two the review round restored the second half with (`김민준 박사님`, `선생님`). Both arrivals are texts `corpus_cjk.jsonl` already held, so the file grew and the compared POOL did not — the same shape as the shapes-corpus growth of 2026-09-01, and the reason a witness had to be picked from names already inside. 32 case rows carry `tolerated=True` over 29 distinct texts (recompute: read `CASES` and count `c.tolerated`, per row and per text). `_CORPUS_FLOORS["corpus_cjk.jsonl"]` is lowered 70 → 67 deliberately, and the tolerated file's floor of 22 still stands under 29. The stop condition held again: intentional diffs UNMOVED at 352 / 247 / 155 / 14 across 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, unexplained 0, radar unclassified 0 and exit 0 at each. RECOMPUTE THE SWEEP, which is the part worth having rather than the digits: it is the CJK-bearing (`_has_cjk`) rows that do NOT declare `tolerated`, carrying any ASCII character other than a space, grouped by the kind of character that trips them and joined to the corpus files holding each text. Over the tree before this bundle it returns eight distinct texts — the three period rows, the parenthesized nickname, and the four mixed-script rows — and over this one, five: the period class is gone and the letter class is untouched, which is the shape of the answer to look for rather than the count. + diff --git a/docs/design/decisions/cjk-full-stops.md b/docs/design/decisions/cjk-full-stops.md new file mode 100644 index 00000000..5fe151d0 --- /dev/null +++ b/docs/design/decisions/cjk-full-stops.md @@ -0,0 +1,19 @@ +# cjk-full-stops — which characters are a period on CJK text (2026-09-10, #322/#323) + +ONE GAP SEEN FROM TWO SIDES, which is why the two issues were answered together. #322 asked WHICH CHARACTERS count as a full stop: the parser knew the ASCII period and nothing else, so `김민준 씨．` — written with the fullwidth stop a Japanese or Chinese IME produces by default — missed the suffix entry that `김민준 씨.` reached, and so did the ideographic `씨。` and its halfwidth `씨｡`. #323 asked WHERE a full stop is read at all: the one site that knew the ASCII period was the lexicon's edge strip, so a stop glued to a name word was invisible to a vocabulary lookup and to nothing else. Either answer alone leaves the other half broken — a fullwidth stop the lookup folds would still have taken its token out of the script reading — so the bundle is one arc with four code commits and this record. + +- **FULL_STOPS names four characters; five sites read them.** `nameparser/_lexicon.py::FULL_STOPS` is `".．。｡"`: the ASCII period, FULLWIDTH FULL STOP U+FF0E, IDEOGRAPHIC FULL STOP U+3002 and HALFWIDTH IDEOGRAPHIC FULL STOP U+FF61. Four sites import the string, so "which characters are a period" is answered once: `_lexicon._normalize` (the lookup AND storage fold), `_vocab._normalized_for_script` (the script-classification fold), and in `_script_segment` the surname site's head match and the honorific peel's tail match. The FIFTH site reads no stop set at all. `_pieces.is_leading_title` — rules/H2's opening-abbreviation shape — refuses a period-marked word whose text carries a character of a script with no initials, through `_vocab.in_initialless_script` (renamed from `_in_initialless_script`; the same veto `is_initial` has carried since #320), so `_PERIOD_ABBREV` stays an ASCII-period pattern and the four-character set is not copied there. Not every period test reads this set, and the ones that do not are named so the claim is checkable: the interior-period shapes — the dotted acronym `M.A.` and the split `Ph. D.` (`_vocab`) — and FOUR ASCII-only edge tests on Latin shapes across THREE modules — the initial `J.` (`_vocab`), the bracketed credential `(Mgr.)` and the word-internal apostrophe rule (`_extract`), and the trailing middle-initial carve-out `V.` (`_assign`) — stay ASCII. What that does NOT mean is that the wider stops leave Latin text alone. The TESTS stay ASCII; the vocabulary FOLD does not, so a Latin word wearing a wide stop reaches vocabulary an ASCII-only carve-out would have kept it out of — `V。` folds to `v` and does not end in the ASCII period that `_reads_as_a_trailing_suffix`'s carve-out tests (`text.endswith(".")`, `_assign.py` ~line 479, whose docstring calls the period "the whole carve-out"), so the carve-out never fires for `V。` and it is roman five, suffix, where `V.` stays a middle initial through that same carve-out — not through `is_initial`, which is not the decider despite reading true on both spellings: `is_initial("V")` is True, yet `Smith, John V` (no period at all) still reads suffix `V`, so being initial-shaped never by itself keeps a word out of suffix. `Jr。` needs no carve-out argument and simply reaches its entry. An earlier wording of this passage called `Ｍ．Ｂ．Ａ．` and `V。` "a fullwidth-Latin question this set does not raise" (corrected 2026-09-10): that was right for `V。` alone. `V。` is an edge stop on a bare word, and the fold raises the question at every such edge, moving the parse as above. `Ｍ．Ｂ．Ａ．` does NOT move — every one of its periods is interior, none at an edge for `_lexicon._normalize`'s edge strip to reach, so the fold never touches it at all, and the dotted-acronym reading in `_vocab.suffix_as_written` (`n.replace(".", "")`) is ASCII-only regardless of the fold: `_normalize("Ｍ．Ｂ．Ａ．")` is `ｍ．ｂ．ａ`, which matches no acronym form the ASCII-only test recognizes. MEASURED 2026-09-10: `parse("John Smith Ｍ．Ｂ．Ａ．")` and `parse("John Smith, Ｍ．Ｂ．Ａ．")` are identical on `d37b8ec` and on this tree — `Ｍ．Ｂ．Ａ．` stays exactly as it was. The set raises the question at every EDGE fold, and the four ASCII-only tests are the places it is not raised rather than a boundary on the set. MEASURED 2026-09-10 on this tree: `Andrew Perkins (Mgr。)` reads nickname `Mgr。` where `Andrew Perkins (Mgr.)` reads title `Mgr.`; `Smith, John V。` reads suffix `V。` where `Smith, John V.` reads middle `V.`; and `John Smith, Jr。` reads suffix `Jr。` where `d37b8ec` read given `Jr。`, family `John Smith`. The last two are case rows now (`latin_roman_numeral_with_an_ideographic_full_stop`, `latin_suffix_with_an_ideographic_full_stop`), and `is_suffix_strict("Jr。", Lexicon.default())` and `_normalize("Dr。") == "dr"` pin the fold's Latin reach at unit level. Nothing rewrites token text at any of the five: the stop stays on the token it arrived with, and every parsed part is still an exact slice of the input (rules.md's T Background). +- **NFC composes at LOOKUP, and that is what makes the fold symmetric.** `_normalize` now composes NFC for a non-ASCII word, at match time and at storage both. NFD hangul decomposes to jamo and missed every vocabulary entry, so `NFD("김민준, 씨.")` read title `씨.`, family `김민준` and now reads family `김민준`, suffix `씨.`. MEASURED 2026-09-09, re-measured 2026-09-10: no shipped entry changes under NFC — 0 of the 1735 strings in the twelve set-valued fields of `Lexicon.default()` (the standing pin `tests/v2/test_lexicon.py::test_every_shipped_entry_is_already_nfc` walks those twelve plus both halves of the five `capitalization_exceptions` pairs, 1745 strings, and asserts the same). RECOMPUTE: fold every string member of those fields through `unicodedata.normalize("NFC", w)` and count `!= w`. +- **The reach is not one script, and the Latin part of it was nobody's ask.** 310 of those 1735 entries are non-ASCII and 158 DECOMPOSE under NFD — and they divide by script in a way worth having in front of you, measured 2026-09-10: 117 HANGUL (every hangul syllable has a canonical decomposition, which is #322's own half — `씨`, `님`, `박사님`, `선생님`), 18 LATIN (`señor`, `née`, `attaché`, `fräulein`, `chargé`, `rozená` …), 13 ARABIC (`أبو`, `الأستاذة` …), 6 BENGALI and 4 CYRILLIC (`урождённая`, `й`). So NFD Latin input matches now too, which no issue asked for — measured against `d37b8ec`: `NFD("Señor Juan Garcia")` read given `Señor`, middle `Juan` and now reads title `Señor`, given `Juan`; `NFD("Attaché John Smith")` moves the same way; `NFD("Maria Kowalska née Nowak")` went from middle `Kowalska née`, family `Nowak` to family `Kowalska`, maiden `Nowak`. RECOMPUTE: count the entries `w` of those twelve fields where `unicodedata.normalize("NFD", w) != w`, and key the count by the first letter's Unicode name. The argument outlives the digits and is the SHAPE of the division rather than its size: the fix was reached for on a hangul gap, and every other composable script in the vocabulary rode along with it, so a decomposed export from a macOS-origin system was missing entries in five scripts and not in the one corner the issue named. +- **The stop set is script-agnostic at the lookup fold too, and the same unasked-for Latin reach exists for the three wider stops.** The edge strip in `_lexicon._normalize` takes any of the four `FULL_STOPS` off any word before a vocabulary lookup or a particle match, Latin and Cyrillic word forms included, so a Latin or Cyrillic word wearing the fullwidth, ideographic or halfwidth stop now reaches vocabulary and particle membership that only the ASCII period reached before — the same unasked-for reach the bullet above records for NFD, on a different axis of the fold. MEASURED 2026-09-10, against `d37b8ec`: `Dr。 John Smith` reads title `Dr。` where it read given `Dr。`; `Проф。 Иванов` reads title `Проф。` where it read given `Проф。`; `Beethoven, Ludwig van．` reads family `van． Beethoven` where it read family `Beethoven` alone. RECOMPUTE: parse the three strings on this tree and on `d37b8ec` and read `title` alongside the name fields. +- **NFKC was DECLINED, not deferred, and the proposal it declines is #322's own.** The issue asked for "an NFKC fold for vocabulary lookup only" on the argument that it "would address all three" of its mechanisms — one change with one risk profile instead of three patches. Measured, it addresses two and a half: `unicodedata.normalize("NFKC", "．")` is `"."` and `NFKC("｡")` is `"。"`, but `NFKC("。")` is `"。"` — the ideographic stop is not a compatibility character and no normalization folds it to a period, so `씨。`, one of the three spellings the issue opens with, would still have missed the entry and the character would still have had to be listed. And NFKC at the storage fold would REWRITE stored vocabulary (compatibility folds change ligatures, widths and superscripts), which is the one thing `_normalize`'s `lower()`-not-`casefold()` decision already refused. Listing four characters answers the question NFKC only half answers, and answers it without touching what a caller authored. RECOMPUTE: `unicodedata.normalize("NFKC", c)` for each of the four. +- **#323 turned out broader than it was filed.** The issue named the honorific peel. Making the classification fold read through an edge stop moved three readers of the `None` it used to return, and only one of them was the peel's neighbour test: the SURNAME SITE stepped past the family name onto the given name (`양. 지훈` cut 지훈 in half — given `양.`, middle `지`, family `훈` — and now reads given `지훈`, family `양.`), the ORDER RULE fell back to positional (`양 지훈.` lost family-first and now keeps it; `田中 太郎.` likewise), and the segmenter's neighbour precondition missed a writer-drawn boundary (`山田太郎 田中.` consulted a pluggable segmenter on `山田太郎` as if it stood alone, and is blocked now). The issue weighed two candidate fixes and chose neither: teaching the surname site to consult `is_suffix_strict` beside `effective_script`, or making the classification tolerant of a trailing period, which it called the broader one and expected to interact with #322. The broader one shipped, and the narrow one would have reached the filed name alone — `김.` is no suffix vocabulary, so `김. 민준` sits outside it entirely. +- **The surname-site amendment was found by measurement during execution, not designed.** With the classification fold alone, `김. 민준` read given `김`, middle `.`, family `민준`: the token classified as hangul, the site matched `김` against its own head, and what was left over — the stop — BECAME the remainder. So the site matches on `text.rstrip(FULL_STOPS)` and cuts the core, a head being a prefix, which makes the offset that cuts the core cut the text; the stop rides with the remainder (`김민준.` divides as `김` + `민준.`, `김. 민준` as family `김.` plus given `민준`). `rstrip`, not `strip`: a LEADING stop would break the prefix argument. The reading is recorded in `ebf64db`'s comment at the site. +- **Two readings JOINED the bundle, approved in session on 2026-09-09.** Neither was filed. FIRST, the glued-period honorific (rules/W3): a stop on the honorific's own word stood between the listed tail and the token's end, so nothing peeled and `田中さん.` and `김민준씨.` read as titles. The peel now matches the tail on the core and cuts BEFORE it, so both divide where their stop-less spellings do and the stop rides with the honorific. The move was free because W3 had recorded that reading as measured on 2026-09-05 and pinned by nothing, its own words being that neither string is a case row or a corpus line, so no row pinned those two readings; both are case rows now, tolerated. SECOND, H2's opening-abbreviation shape on a CJK word: `田中.` read title and now reads family `田中.`, `田中. 太郎` reads family `田中.`, given `太郎`. Han is what WITNESSES this one, and hangul cannot: hangul segmentation runs first, so `김민준.` never reaches the shape test and divides as `김` + `민준.` by the surname site instead. Latin and Cyrillic are untouched (`Smith. John` still reads title `Smith.`, `Проф. Иванов` title `Проф.`), and the veto is contains-any, so `Kim김. Smith` is refused a title too — deliberately: a word carrying a script with no abbreviations is not wearing an abbreviation's period. +- **Recorded limits, all five measured 2026-09-10 on this tree — and the segmenter one is a limit the review round CLOSED, kept in this bullet because this is where the claim stood. The halfwidth-kana one was CLOSED on 2026-10-03 by #594; see the CLOSED note that ends the halfwidth passage below, and decisions/W4.** HALFWIDTH KANA sits outside the script table, so it reads by the LATIN shapes at both period sites and not just at the initial one: `_vocab.is_initial("ﾗ.")` is `True` where `is_initial("ラ.")` is `False`, and H2's opening-abbreviation shape MISROUTES a longer halfwidth opener — `parse("ﾀﾅｶ. John")` gives title `ﾀﾅｶ.` where `parse("タナカ. John")` gives given `タナカ.`. Both because `_policy._SCRIPT_RANGES` excludes the halfwidth kana block U+FF65–FF9F on purpose — legacy bank and CSV data writes it, and the table's comment calls that a separate normalization problem, #272's separator handling touching only the halfwidth DOT U+FF65 and none of the kana behind it — and `_NO_INITIALS`, being a tuple of Script members, silently inherits the exclusion. That is #322's second mechanism, restated here because it is the one the bundle did NOT close. An earlier wording of this bullet said "nothing misroutes today, every kana honorific being multi-character", which had the direction backwards: the misroute is at the SHAPE site rather than the peel, and being multi-character is what carries a halfwidth opener INTO H2's shape rather than what keeps it out. NOTHING PINS EITHER READING — no case row, no corpus line, for the halfwidth block or for `ﾀﾅｶ. John` — so both can move without the suite or the differential saying so. It stays a recorded limit rather than a fix: what to do with halfwidth kana is a normalization question (widen the script table, or fold widths at classification) and not a full-stop one, and the bundle answered the full-stop one. CLOSED 2026-10-03 by #594, which widened the table (decisions/W4 carries why not the fold): `ﾀﾅｶ. John` reads given `ﾀﾅｶ.` and `is_initial("ﾗ.")` is `False`, and both readings are pinned now — the case rows `ja_halfwidth_katakana_opener_with_a_period_is_a_name` and `ja_halfwidth_katakana_lone_name_with_a_period_is_not_a_title`, and `test_is_initial_script_repertoire`. RECOMPUTE: parse `ﾀﾅｶ. John` and `タナカ. John` and read `title` beside the name fields; `is_initial` alone reports only the shorter half. A PLUGGABLE SEGMENTER receives the CORE, the token with its trailing stops removed — the same string the vocabulary match reads, the surname site handing it `core` rather than `text` (corrected 2026-09-10, in the review round after this entry was first written: the site handed over the raw token, which this bullet recorded as a limit, and the fix costs nothing because a head is a prefix — every offset reported against the core cuts `text` in the same place, so the stops ride with the last piece and no answer of the segmenter's can make a stop a piece of its own). The shipped `ja` pack's `_wholly_japanese` therefore ACCEPTS `山田太郎.` now, where the raw token failed its repertoire test: measured 2026-09-10 with `namedivider-python` installed (`uv run --extra ja`), `parser_for(locales.JA, segmenter=locales.ja_segmenter())` reads `山田太郎.` as family 山田, given `太郎.` with a SEGMENTATION report scoring 0.44, where `d37b8ec` read title `山田太郎.` and the un-corrected branch read family `山田太郎.`; the already-divided `山田 太郎.` reads family 山田, given `太郎.` on this tree and given 山, middle 田, family `太郎.` at `d37b8ec`. Pinned by `test_a_consulted_segmenter_receives_the_core_without_the_stop`. A LEADING stop keeps the token out of the SURNAME site only, because the classification fold that gates it rstrips as it does: `.김민준` classifies as no script, never becomes a surname site and stays one whole word, reading given `.김민준` exactly as `d37b8ec` reads it (corrected 2026-09-10 in the same round: the fold stripped both edges when this entry was written, which moved the ROLE to family and — with a segmenter configured — offered the token to it raw, an answer of offset 1 dividing it into the stop and the name; the fold now matches the sites, and hiding such a token is the no-split rules/W1 already accepts). It does NOT keep the token out of the HONORIFIC PEEL, which consults no script at all — the tail alone is its license (rules/W2's Background) — so a leading stop does not defeat it: `.김민준씨` peels to `.김민준` and `씨` exactly as `김민준씨` does, pinned by `test_the_peel_reads_the_trailing_stop_only`. A BRACKETED CJK CREDENTIAL degrades: rules/S1's escape in `_extract` calls a clause suffix-shaped when it ends in an ASCII period, so `(김민준.) John Smith` is unwrapped and the H2 veto then leaves `김민준.` as name text the surname site divides — given 김, middle `민준. John`, family Smith, where 2.2.0 and `d37b8ec` read title `김민준.` — pinned as the tolerated `ko_name_with_a_period_in_a_bracketed_credential`. And a caller-authored NFD surname entry is composed at ingest, so raw NFD input then finds no entry and the name goes unsplit; pinned by `test_an_nfd_authored_entry_is_stored_composed`. +- **Pickle compatibility, decided 2026-09-10 (Derek): a release-note line, not a forgiving load.** `Lexicon.__setstate__` re-runs `_normalize` over the loaded state and rejects any entry the fold changes, so a `Lexicon` pickled by 2.1.x or 2.2.x whose caller-added entries the widened fold now touches (a non-ASCII entry written with a wide stop such as `씨。`, or authored in NFD) raises `ValueError: incompatible Lexicon pickle: entries are not normalized (titles: 씨。); this state was not written by this version of nameparser` — a message that is wrong about provenance for exactly these pickles, which a released version did write. Measured 2026-09-10 (recompute: take `Lexicon.default().add(titles={"씨"}).__getstate__()`, add `씨。` to its `titles`, and load it into `Lexicon.__new__(Lexicon)`). The shipped vocabulary is unaffected (zero of 1735 strings change under NFC, and no shipped entry carries a wide stop). Two remedies were weighed: a release-note line with the fix a caller applies (rebuild the `Lexicon` from its source rather than unpickle it), or a load path that accepts state whose only drift is one fold pass away, re-folds it and warns naming the entries. The note was chosen: the load guard was designed to refuse rewritten data (the guarded-raise design #3-0-reevaluations records as right regardless of the in-a-minor friction) rather than become a fourth place caller data is corrected without a word, the population is caller-added CJK entries pickled across a minor, and a rebuild is the documented way to carry a `Lexicon` across versions. The 2.3.0 release note carries the line; `tests/v2/test_lexicon.py::test_unpickling_rejects_unnormalized_entries` is the pin of the refusal itself. +- **CORRECTIONS this entry makes to entries above it.** #indic-honorifics' "Abbreviation marks and periods" bullet said `_normalize` "lowercases and strips edge whitespace and ASCII periods only — no NFC, no NFKC, no casefold anywhere in that module"; it now composes NFC for a non-ASCII word and strips all four stops. The VISARGA ARGUMENT still holds and holds for the same reason: ঃ U+0983 has no canonical decomposition, so NFC leaves it exactly where it was — recompute with that bullet's own one-liner, `Lexicon.default().add(titles={"মোঃ"}).titles & {"মোঃ"}`, still non-empty. #W1's 2026-07-29 §1a bullet said "segmentation MATCHING stays raw" and called the classification fold "the one deliberate exception"; matching is now folded too, so what stays raw is SEGMENTATION — the surname site's membership test and the peel's tail slice, which index the token's own text — and the folds are two, not one. #cjk-comma-demotion's 2026-09-05 five-parse bullet is amended in place: `田中さん 太郎.` no longer flips the order (the period is invisible to the classification, so it reads family `田中さん`, given `太郎.` as its no-period twin does), and `田中さん.` and `김민준씨.` no longer read as titles, so that bullet's summary — that the ONLY one of the five where a period changes nothing is the one where it sits on a word the split-off steps past — is superseded: the period changes nothing in four of the five now, and `田中さん 様.` is joined rather than alone. #initials-repertoire's principle bullet restated the veto phonologically ("morphemes or syllables"), which `_policy._NO_INITIALS`' own comment forbids in as many words — Devanagari is an abugida and Arabic an abjad, neither has letters in that sense and both abbreviate — so the bullet now states the CLDR criterion the constant actually uses. +- **What moved, measured 2026-09-10 and re-measured the same day after the review round.** SEVENTEEN names in `tools/differential/corpus*.jsonl` read differently against `d37b8ec`, and all seventeen are the bundle's own new rows, every one of them on the RADAR tier in `corpus_cjk_tolerated.jsonl`. Twelve are the CJK movers this bundle's landing commits wrote; the thirteenth, the wholly-katakana `マイケル.`, joined in the whole-branch review that produced this entry, reading given `マイケル.` where `d37b8ec` reads title `マイケル.` — the katakana arm of the same H2 veto, pinned as `ja_katakana_lone_name_with_a_period_is_not_a_title` in `tests/v2/cases.py`. The last FOUR joined in the review round after that, as pins on readings nothing held: three stop-bearing FAMILY_COMMA spellings (`김민준씨., J.씨`, `田中さん., V.`, `이, J.씨.`) and the bracketed `(김민준.) John Smith`, the recorded degradation in the limits bullet above. A leading-stop shape moves the same way but sits in no corpus at all: `.김민준씨.` — a stop no script ever writes, glued before a honorific-bearing word — read one whole given `.김민준씨.` at `d37b8ec` and now reads given `.김민준`, suffix `씨.`, pinned only by the stage tests for its two edges taken separately (`test_the_peel_reads_the_trailing_stop_only` for the leading stop, `test_peels_a_listed_tail_through_a_trailing_full_stop` for the trailing one) rather than by any corpus row or a combined case of its own. The three comma rows agree with their stop-less twins in every field once the riding stop is removed from the suffix (`씨., J.씨` vs `씨, J.씨`; `さん.` vs `さん`; `씨.` vs `씨`) — each differs from its twin in exactly that one field, by exactly the stop that rides — which is the peel's own "one name, two spellings" argument holding as far as a raw field comparison can show it; the bracketed row does not agree with anything and is not meant to. `田中. 太郎` stays there with the rest, though it is what WITNESSES rules/H2's Accepted clause added here: `tools/differential/compare.py`'s tier comment records the 2026-09-05 precedent for exactly this collision — where a demoted file holds a text a rules.md example line names, the EXAMPLE LINE moves into the tolerated rule and the row is not promoted, because marking the row alone leaves the name enforced and documented as demoted. So the clause's CJK reading is carried in W3's tolerated example block, H2's own block keeps the Latin control `Smith. John`, and all four #323 shape rows stay `tolerated`. THE POPULATION THAT COULD HAVE MOVED AND DID NOT is two different sets depending on which criterion is read, and both are worth having in front of you. Under the SHAPE this bundle is about — an edge full stop glued to a word carrying a classified character — the corpora hold 22 names: the seventeen movers and FIVE non-movers (`田中さん 様.`, `田中さん, 様.`, `김민준 씨.`, `김민준 양.`, `김민준, 씨.`), all five already in `corpus_cjk_tolerated.jsonl` before this bundle. Two of the twenty-two need the detector to be written carefully, which is why the recipe is spelled out below: the stop on `김민준씨., J.씨` and `田中さん., V.` sits before a COMMA, so a detector splitting on whitespace alone sees `김민준씨.,` and finds no edge stop (an 18 recorded earlier in the day came from such a detector, over a corpus four names smaller), and the stop on `(김민준.)` is inside a bracketed clause until rules/S1's escape unwraps it. Under the looser reading of an ASCII PERIOD ANYWHERE in a CJK-bearing name, the corpora held 20 distinct names on 21 rows before this bundle (19 rows in the tolerated file and 2 in `corpus_issues.jsonl`, `Dr 김민준씨, Jr.` being the one name on two of them) — unmoved by the katakana row, which is a name this bundle's own work wrote, not a pre-bundle count. The looser set is wider because it sweeps in periods sitting on LATIN tokens inside a CJK name — `毛 泽东 Dr.` and `田中さん, V.` — which is precisely what the veto is scoped not to touch, it reading the word and not the name. NEITHER the five nor the twenty moves — the seventeen movers are the whole of what did, and every one of them is a row this bundle wrote. An earlier wording of this bullet said "the twenty-one CJK names carrying an ASCII period": that counted rows as names and quoted the looser criterion beside the shape's argument. RECOMPUTE: parse the union of the corpus files on both trees and diff the seven fields and the ambiguity kinds; for the SHAPE population, take each name's tokens (`_pipeline._tokenize`, not a whitespace split) and ask whether any token's `strip(FULL_STOPS)` is shorter and still carries a classified character, then add the bracketed clause by hand; for the looser one, count distinct names and rows separately. The gate exits 0 at all four baselines with `radar unclassified: 0`; one ledger rule carries them, `fix(#322/#323)`, ELEVEN members at 1.4.0 and 2.0.0 and SEVENTEEN at 2.1.0 and 2.2.0 — the three hangul names the older ledgers hand to their native-script CJK rule are one part of the difference, 2.1.0 being the release that shipped that behavior, and the three FAMILY_COMMA rows are the other: at 2.0.0 none of the three diffs at all, and at 1.4.0 the two that do (`田中さん., V.`, `이, J.씨.`) are already claimed by the broad `fix(cjk-comma-compound)` and `fix(cjk-comma-honorific-peel)` rules that ledger carries, so adding them to this rule would only take a name off a rule that describes it — the same narrow-first reasoning that keeps the three hangul names out. +- **The invariant the rules imply, run over two inputs (AGENTS.md axis 11), measured 2026-09-10.** If an edge full stop is punctuation on a CJK word, then a CJK name written with one must divide exactly as the same name written without one. Written as a check over the corpora — for every corpus name carrying a classified character and an edge stop on some word, parse it, parse the spelling with every word's edge stops removed, and compare the seven fields with the stops removed from the FIRST parse's values too — 30 of the 34 such names agree (re-measured 2026-09-10 after the review round's four new rows; it read 28 of 30 before them, the katakana row マイケル. having joined the population and agreed). FOUR do not, and only three of those are about the parser. Two are the standing boundary, and neither is a CJK word: `毛 泽东 Dr.` (the period makes `Dr.` title vocabulary the trailing walk claims, where bare `Dr` is a name word) and `田中さん V.` (`V.` is a dotted numeral where bare `V` is the generational suffix) — both LATIN words wearing a Latin period inside a CJK name, which is exactly what the veto is scoped not to touch, it reading the word and not the name. The third is new and is the bracketed degradation the limits bullet records: `(김민준.) John Smith` reads given 김 / middle `민준. John` / family Smith where `(김민준) John Smith` reads nickname 김민준 — the period is what makes rules/S1 call the clause a credential rather than a nickname, so the two spellings are genuinely different inputs to `_extract` and the invariant should not hold across them. The FOURTH is the comparison's own artifact and not a finding: `이, J.씨.` and `이, J.씨` divide identically (given `J.`, family 이, suffix `씨.` and `씨`), but stripping the first parse's VALUES takes the period off the initial `J.`, where in the bare input that period is interior and survives. A normalization that runs over field values cannot tell a word-final stop in a value from a word-interior one in the name; that is worth knowing before reading the next round's count. RECOMPUTE with that description; it consults no rule statement, which is the point of the axis. +- **Parse cost, measured 2026-09-10 on py3.11** (`tools/perf/call_count.py --against d37b8ec`, and a script counting frames the way its `calls_for` does for the other names). The reference name never reaches the veto — `Dr.` is listed, so the title piece is taken before the shape test — and `parse=409.00 facade=446.00` is unchanged at both trees. On an H2-SHAPED OPENER the veto costs three to five frames: facade 304 → 308 for `Xyz. John Smith`, 385 → 390 for `Xyz. van Berg MA`, 231 → 232 for `Smith. John` (parse-only: 267 → 271, 348 → 353, 194 → 195). The number to quote is these and not "zero"; decisions/parse-cost's budget is about the reference name and this is off it. The largest movement measured is not the veto at all: `김민준씨.` goes facade 209 → 397, the peel now firing where the parse used to bail to a title reading, which is a whole segmentation path running rather than an added test. + diff --git a/docs/design/decisions/comma-suffix-arc.md b/docs/design/decisions/comma-suffix-arc.md new file mode 100644 index 00000000..5cb1ed23 --- /dev/null +++ b/docs/design/decisions/comma-suffix-arc.md @@ -0,0 +1,22 @@ +# comma-suffix-arc — #291/#296/#316 (2026-07-26 → 2026-08-01) + +#291 was filed 2026-07-26 out of the 2.0 vocabulary cleanup, with +its decline-by-measurement evidence in the issue body; "z domu" folded into it 2026-07-27 (see Excluded, MAIDEN_MARKERS). Intent for #291 and #296 is settled by the approved bundle spec; #296 (with #325) SHIPPED 2026-08-23 as the bundle's first three commits, with one deviation from the audit table recorded at C1 ('md' keeps dual membership — bare 'Md' is Muhammad); #291 was UNSHIPPED then, carried as a deviates: marker on C1 (#291 closed 2026-08-24; the marker came off with #564 in 2.4). The arc's bookkeeping: #291/#296 moved milestone v2.1 → v2.2 on 2026-08-01 (with #289/#293); the 2026-07-30 design doc was amended in place 2026-08-01 (A1–A6); its durable content is HERE rather than in the gitignored doc; #316 (trailing titles) was filed the same day, carrying the esq cleanup as a Related section. + +The approved 19-word TITLES∩suffix audit table (2026-07-30/08-01, intent for the unshipped bundle): jr, junior, phd, md, do and se DROP from TITLES (se: Structural Engineer is the PE/SE post-nominal pair, no prenominal convention — v1 residue); dr drops from SUFFIX_WORDS and sra from the suffix sets (v1 residue); sr, lt, cpl, cpt, cpo, csm, sgm, ra and vc KEEP dual membership (position decides); ms and sa keep dual membership AND join the ambiguous set (the periods gate). Derek's framing thesis for the whole table: v1's sets encoded where the v1 PARSER needed words to be, not where words can occur. Amendment A6: once SUFFIX_PHRASES ships, the glued peel's token-level post-nominal test cannot step over a phrase segment ("김민준씨, LEED AP" will not peel) — #291 owns deciding that, with a case row either way. + +- 2026-08-01 (plan amendment 1, Derek's call: "ship + repair the merge") — dropping dr/ms from the suffix vocabulary flips segment() to FAMILY_COMMA and degrades "John Smith, Dr." to family="John Smith" (mechanisms.md#VOCABULARY-FEEDS-STRUCTURE is this shape); the approved repair, ordered FIRST in the bundle so no commit introduces a regression a later one heals: assign segment 0 positionally when segment 1 is nothing but titles. + +Declined: + +- Splitting the dead multi-word suffix entries into single-word entries (2026-07-26, the evidence in #291's body) — measured: + "Smith, A.P." has the suffix steal the given initials; "John + Leed" and "Mary Nicet" lose family names; and the period-gate escape is equivalent to removal because nobody writes "L.E.E.D.". This decline is why the seven removable entries were REMOVED in 2.0 rather than split, and it is the missing history behind C1's + #291 marker. + + Amended 2026-08-23 (#291 closed working-as-designed): the decline stands as measured — splitting into single words does cost `Smith, A.P.`, `John Leed` and `Mary Nicet` — but the inference drawn from it did not. "A multi-word *entry* is inert" was read as "a multi-word *credential* is unreachable", and that is false: the run predicate `is_wholly_suffix` reassembles adjacent suffix tokens, so `parse("John Smith, MD PhD").suffix` has been `'MD PhD'` since 1.4.0, and `psm i`/`psm ii` — two of the seven entries called dead — parse today from their component words. The third option neither the issue nor the spec considered is to leave the shipped vocabulary alone and let callers add the component words to a `Lexicon`, which is what was chosen. + + Also declined with it: `SUFFIX_PHRASES` and segment-level matching (unnecessary once the run predicate is measured), and amendment A6's glued-peel question (no phrase-matching unit, so `_is_post_nominal`'s token-level test stays correct by construction). +- The trailing-abbreviation structural fallback — with a measurement LIMIT rather than a measurement: the differential corpora structurally cannot evidence it, because they hold only names someone wrote down, and an unrecognized abbreviation is by definition outside the vocabulary. A green run there proves nothing (the reusable harness fact is in mechanisms.md's field notes). +- SUFFIX_PHRASES matching in assignment — measured cost: it renders suffix="LEED, AP", because the suffix view comma-joins suffix words unless they carry the stable "joined" tag, which grouping applies within a piece and — since #436/#437, 2026-09-06 — post_rules applies between two post-nominals the writer did not comma-separate. The general form: multi-word vocabulary must merge where the render tag is applied, not where the role is assigned. + diff --git a/docs/design/decisions/comparison-surface.md b/docs/design/decisions/comparison-surface.md new file mode 100644 index 00000000..0e4f6f45 --- /dev/null +++ b/docs/design/decisions/comparison-surface.md @@ -0,0 +1,10 @@ +# comparison-surface — why value equality died, and what replaced it + +- 2026-07 (1.3.0 arc, #223/#224; executed in the v2 core — measured: HumanName == HumanName is False, matches() and comparison_key() live, the facade cites #223) — the trilemma: case-insensitive equality, equality-with-plain-strings, and hashability are mutually inconsistent (a hash cannot equal both hash("John Smith") and hash("john smith")), so 1.2.0's design was unfixable in place, not merely disliked. Additional costs of the old surface: equality ran through str(self) and so depended on mutable string_format, and maiden was invisible to == (absent from the default format). +- 2026-07-06 — the cross-constants asymmetry, pinned: a str operand is reparsed with self.C; a HumanName operand is compared as already parsed. + +Declined: + +- Component-based __eq__/__hash__ — workable, considered, rejected: it still privileged one equality semantic for a domain that has none, and kept the mutate-while-hashed hazard. +- Lenient matching (initials-compatible, Bob/Robert) — declined WITHOUT an issue, deliberately: matches() is exact-components by design. Someone will propose could_match() within a year; this entry is the resolved-as-no they should find. + diff --git a/docs/design/decisions/deviates-registry.md b/docs/design/decisions/deviates-registry.md new file mode 100644 index 00000000..f9a54849 --- /dev/null +++ b/docs/design/decisions/deviates-registry.md @@ -0,0 +1,4 @@ +# deviates-registry — packs stay pure data (option C) + +- 2026-07-18 (d4aaafa; the DEVIATES design note) — a `deviates` field ON Locale was rejected: callables break value equality and pickle-by-reference, and DEVIATES is acceptance metadata, not runtime behavior (revisit only if a runtime consumer like explain() appears). The REGISTRY became the contract instead: test_registry_is_the_pack_contract fails structurally unless every registered pack ships DEVIATES and its rotator list. This is why LOCALE-PACKS-PURE-DATA can promise "no code paths of their own" and mean it. + diff --git a/docs/design/decisions/differential-ledger.md b/docs/design/decisions/differential-ledger.md new file mode 100644 index 00000000..5428fe6c --- /dev/null +++ b/docs/design/decisions/differential-ledger.md @@ -0,0 +1,304 @@ +# differential-ledger — the differential ledger's arcs + +## differential-ledger — tooling decisions (2.1.0 release arc) + +Harvested 2026-08-15 from the release-arc session; measurements are that session's, spot-checked at landing. + +- 2026-07-24 (rc1 arc, predating this section's window) — the TWO-CORPUS design: corpus_issues.jsonl (198 tracker-harvested names, 166 absent from the v1-test-bank corpus) exists because v1 test banks are structurally blind to anything 2.0 added; compare.py globs corpus*.jsonl and fails loudly on none. Its first catch was the leading-credential case ("Ph. D. John Smith" → suffix). + +- 2026-08-05 #332 — ledger field vocabulary is Role's names, not the facade's: canonicalizing to first/last would have put an eighth place naming roles differently from Role inside the durable record, and the facade vocabulary is removed at 3.0. + +Declined: + +- Policy annotations widened to input unions (2026-08-05 #334) — five documented spellings fail mypy and every one has a type-clean equivalent; widening would make every READER see a union, and reading is the commoner operation. A .pyi stub typing __init__ wide and attributes narrow was deferred as a 3.0 candidate (needs a parallel signature list with its own sync test). +- Closing the differential gate's default-policy ceiling (2026-08-04 #332) — a per-row policy in the corpus needs defined behavior for baselines that cannot construct newer policy fields; tests/v2/cases.py already covers opt-in paths per row, so the ceiling is documented instead. +- `_check_tree` as is_relative_to(REPO_ROOT) (2026-08-05 #332) — accepts .venv/, build/ and dist/ copies inside the repo; the invariant is "is the source package", so the predicate is is_relative_to(REPO_ROOT / "nameparser"). +- Empty-string probe as the ledger over-match guard (2026-08-05 + #332) — `.`, `.+`, `\b`, `[\s\S]` all decline "" and still match + every corpus name; replaced by the sentinel set (mechanisms.md#SENTINEL-SET-OVER-MATCH-CHECK). +- "Lists every role" checked against all eight fields entries (2026-08-05 #332) — a seven-role list passed while omitting _ambiguities, which below baseline 2.0 cannot enter a diff at all; the check is against V2_FIELDS. +- A seed ledger rule with name_regex and no fields (2026-08-05 + #332) — it matched every CJK-bearing name and would have + classified all 89 diffs on the first pass, exiting 0 having distinguished nothing. +- 2026-08-07 #333 — the canonical-rule selector (keyed on literal + #271/#272 slug substrings) was deleted rather than repaired, + option A of three: split "every hand copy equals the table" (the sweep's job) from "the table did not change shape without a decision" (the span-bearing test's job), so a selector break can never take the decision gate out as collateral. This retired the slug taboo an earlier mechanisms.md entry carried. +- The B7 sanctioned-extra span (pre-#332 arc) — added, then removed as a pure loss when review measured that every B7-divided name already matches through its guaranteed classified flanks, so the span's only effect was absorbing punt-volat Latin regressions ("Gal·la Marcet" probes on both sides, recorded in PR #305's history). The cleanest "sanctioned extras must be earned" precedent the ledger has. +- An issue for rotating DEFAULT_BASELINE (2026-08-07) — it recurs every release; it lives in the AGENTS.md release checklist beside the VERSION bump instead (#333 must land first). + +Excluded (ledger name_regex patterns — one over-matching rule shadows the whole ledger, since name_regex rules sort first): "", +"(?:)", ".", ".+", "\b", "[\s\S]". Enforced by the sentinel-set +check. + +## differential-ledger, the dormancy arc (2026-08, #328–#376) + +The second ledger arc; measurements are that session's, the machinery verified present at landing. + +Decisions that landed: + +- 2026-08 #328 — `[[never]]` exclusions: ledger vocabulary for + "this shape must never be explained". classify() consults + exclusions before rules, which makes them MONOTONE — an entry only ever removes a name, never moves one between rules — so an exclusion's blast radius is exactly the names it captures, independent of rule order. +- 2026-08-12 #373 — `dormant = ""`: a rule explaining nothing fails the run in both directions, with three diagnoses (reverted / shadowed by X / refused by a [[never]] entry). Always on, not behind --strict: a check nobody passes a flag to is a check that doesn't exist. +- 2026-08 #373 — `--baseline 2.0.0` joined the release checklist: that ledger's rules had no dynamic coverage at all; measured clean at 90/0, so the gap was closed rather than documented. + +Declined: + +- Ambiguity reporting on multi-matching rules (#372/#373) — measured: 28% of claimed name×role pairs already have ≥2 matching rules (732 of 2619; 432 are one pair of rules alone). A report firing on 28% of what it inspects is wallpaper. The actionable slice shipped as the "shadowed by " diagnosis, which speaks only on FULL shadowing. +- A specificity floor for fields-only rules (#372/#373) — when this was proposed, exactly one fields-only rule existed across the ledgers, naming 3 of 7 roles; a six-of-seven floor matched nothing, and nothing would reveal it vacuous. Still declined: #451 banned the SHAPE instead (the fields-only arc below), which is a different proposal and leaves no ledger with a rule for a floor to measure. +- Specificity reordering of the rule sort (#328) — measured across all 751 names, on the ledger as it then stood: width-then-regex-length moved five rule populations and sent 17 names into the generic fields-only rule, draining the CJK-specific ones. No reading of the sort produces the "exactly one label changes" originally claimed; that figure was corrected on the PR. +- Unanchoring the honorific-suffix rule to reach glued forms (#376) — it would make a future suffix regression on 김지양 classify as a recognized honorific. A confidently wrong label is worse than a catch-all's honest breadth, and the gate reads the same either way. + +## differential-ledger, the fields-only arc (2026-08-27, #451) + +The third ledger arc, and the one that ends a shape rather than adding vocabulary. Measurements are this session's, at the counts each commit records. + +Decisions that landed: + +- 2026-08-27 #451 — `validate_rules` REJECTS a rule carrying `fields` and no `name_regex`. Such a rule claims every name whose diff fits those fields, and `_CORPUS_CLAIMS` records a regexless rule's reach as the WHOLE corpus — already at its maximum, so arrivals never move it and `test_every_rule_claims_the_recorded_share_of_the_corpus` passes whether it explains 4 names or 40. The one rule with the shape drifted from the four names its prose described to 25 before #372 cut it back, and then to SEVENTEEN after — decisions/H1 records that 14 -> 17 step and #410 taking three back — standing at fourteen across six unrelated behaviour families when this bundle opened, every guard green throughout, absorbing two open v2.2 bugs (#342, #397) as intentional 2.0 changes while AGENTS.md names the 1.4 summary as the release log's source for Behavior Changes. Free to state only because the fourteen were given rules first; a role-count floor was the earlier proposal and stays declined (#372/#373, vacuous). +- 2026-08-27 #451 — a rule whose declared `fields` EXCEED every diff it explains is a defect in its own right, not untidiness. Measured on the catch-all: it declared `{given, family, suffix}`, and by the time this bundle's first three commits had given its `{given, family}` names rules of their own, nothing it still explained moved `given`. Note WHEN the excess appeared, because it is the whole mechanism — the declaration was earned when the rule was written and became excess as its explained set SHRANK under it, which is exactly `fix(#424)` at decisions/H1, where #410 narrowed a diff from three roles to two and the rule kept matching by superset with no run naming it. A rule does not have to widen to become over-declared; the diffs beneath it only have to leave. The two together are why #452 was filed — nothing counts what a rule actually explains, only whether it explains anything. +- 2026-08-27 #451 — a ledger rule may classify a reading NOBODY WANTS, provided its `issue` says so. `fix(#342)` and `fix(#397)` carry NOT WANTED in the issue text, the cause-not-consent sentence decisions/P5 established for the #369 regression, and a delete-when-fixed instruction the dormancy check enforces once the fix lands. +- 2026-08-27 #451 — one vocabulary per alternation, FORCED rather than chosen. `_LatinCopy` carries one `vocabulary` and `test_latin_alternations_mean_something_the_vocabulary_ships` asserts exactly one roster key per alternation, so `jr` (SUFFIX_WORDS) and `mc`/`mp` (SUFFIX_ACRONYMS) cannot share one. That constraint forces one of the three splits, and forces it only given the roster as written: `_LatinCopy.vocabulary` is a hand-supplied frozenset, so declaring `SUFFIX_WORDS | SUFFIX_ACRONYMS` would let `(jr\.?|v|mp|mc)` pass the member checks — measured. Declining to write a union vocabulary is a judgement, that a rule's alternation should name one wordlist a reader can go and check, and it is recorded here as one rather than as a constraint. The other two are forced separately and are recorded below under the declined single-rule tail: `m\.?a\.?` cannot be an alternation member at all, and the numeral alternation copies no wordlist. Four rules, three independent reasons, none of them taste. +- 2026-08-27 #451 — a ledger rule's character class must not reach a codepoint the IMPLEMENTATION does not act on, and is pinned to that. A SUBSET, not a copy: `test_the_emoji_boundary_rule_copies_the_dividing_ranges` asserts `claimed - divides == set()` and nothing in the other direction, deliberately — `_EMOJI_RANGES` has FOUR blocks and the rule copies the two astral ones, leaving the BMP pair out because no corpus name reaches them through its token anchor. Reading it as equality would instruct the next author to pre-claim the BMP half, which is the #372 shape this same section leans on. The emoji rule's first draft spanned `U+1F300-U+1FAFF` while the two ASTRAL blocks of `_tokenize.py`'s `_EMOJI_RANGES` are narrower than that span: it claimed 1072 codepoints where the parser leaves the token whole, so its own prose ("an emoji inside a token divides it") was false for every one of them and a `{given, family}` diff with another cause would have classified as intended. `test_the_emoji_boundary_rule_copies_the_dividing_ranges` holds it, and fails on the draft it replaced. +- 2026-08-27 #451 — `_sorted_rules` is KEPT although the ban makes it the identity on every ledger that loads. Verified against all three: the ban removes the only shape that could occupy its second tier, so file order settles every tie there is and the function reorders nothing. Kept because it is what makes the ban safe to state for any reader that does not call `validate_rules` first — a future tool, a REPL, a test fixture. Deleting it would move the guarantee from the code into a convention, which is the trade #451 was filed to undo. Recorded as a decision rather than as a consequence precisely so a later dead-code sweep finds the reasoning before the `return`. +- 2026-08-27 #451 — past-tense counts stay in a ledger comment; live ones do not. "It claimed 25 until #372" is a fixed fact. "Four corpus names, and the prose fits all four" was written when the rule explained four and still read that way at fourteen, because nothing recomputes it. Where a count is owned by another comment, point at the owner rather than restating it — two copies means one gets updated and the other does not. +- 2026-08-28 #452 — a rule's declared `fields` must EQUAL the union of the diffs it explains, checked by `compare.py` at the end of every run and failing it like an unexplained diff. The statement is exact rather than heuristic, which is what makes it cheap: `classify()` already requires `declared >= union` for the rule to match the names it matches, so the only possible error is the other direction, and the union is simultaneously the check and the repair. Narrowing to it cannot orphan a name, since every name a rule explains contributed to it. Measured before landing: 3 of 67 explaining rules over-declared at 1.4.0, 5 of 58 at 2.0.0, 6 of 51 at 2.1.0 — all fourteen narrowed first, so the check was silent the day it arrived. +- 2026-08-28 #452 — over-declaration is BASELINE-RELATIVE, and each ledger is measured on its own run rather than copied. `fix(#296) a lone post-comma credential is a suffix` declared `{family, given, suffix, title}` in all three ledgers: exactly exercised at 1.4.0, where v1 reads the pre-comma word as `first` and all four roles move, and over-declared at both 2.x baselines, where the same behaviour moves only `{suffix, title}` — which is all it declares in those two files now. A reader comparing the three sees one rule with TWO different field lists and should read that as correct rather than as drift. Two and not three, measured: the 2.x pair narrowed to the same set, so the split is 1.4.0 against both 2.x ledgers, not one list per file. +- 2026-08-28 #452 — NO escape hatch, decided rather than deferred. `dormant` already covers the explains-nothing case in both directions, a rule with no `fields` has nothing to over-declare, and the ledger's own doctrine — "a rule that pre-claims shapes it has never seen is the #372 failure mode" — makes strictness the existing principle. Accepted cost, stated so it is not rediscovered as a surprise: the first rule that genuinely needs a wider declaration has to argue for a key the way `dormant` was argued for in #373, rather than reaching for one that already exists. +- 2026-08-29 #456 — a rule carrying `name_regex` and no `fields` is REJECTED, the symmetric twin of #451's ban. It narrows by name and by nothing else, so on any name its regex reaches it claims every diff shape there is — measured, 255 shapes from baseline 2.0 on and 127 below it. #452 is what made it urgent rather than merely untidy: `over_declared_rules` skips a rule with no `fields`, correctly, since one declaring no roles cannot over-declare them — so deleting the line is the cheapest way to silence an OVER-DECLARED failure AND the most permissive thing that can be done to the rule, which is the #372 failure mode reached by following a gate error message. Free to enforce on #451's own terms: measured, 0 of 179 rules across the three ledgers had the shape. After it, every rule carries both keys, and the three rejections in `validate_rules` read as one rule — narrow by name and by role, or it is not a rule. NO escape hatch for a "genuinely unbounded" rule, declined until one appears, same call and same reason as #452's. + +Found rather than decided, and worth as much: + +- **A fields-only rule silently falsifies OTHER comments.** The 2.x ledgers' idiom "such a name should arrive UNEXPLAINED and be read once" holds there because no rule above the one making the promise claims that diff shape. A fields-only rule is the extreme way to break it, not the only way: measured while fixing #452, a `{family, maiden}` diff on `fix(#367)`'s own probe goes to `fix(#274)` in the 1.4.0 ledger, which has no fields-only rule anywhere. Note the shape, not the role: `{given, family, maiden}` on the same probe is claimed by nothing, because `fix(#274)`'s `fields` stop at maiden/middle/family. That is the point — the question is only ever answerable per shape. The reusable check is "does anything above me claim this shape", which is `classify()` driven directly — NOT "is there a catch-all". In the 1.4.0 ledger it was false for anything of the catch-all's shape, and nothing said so. Measured while the rule still lived, by driving `compare.classify()` directly over the seven `_MUST_NOT_MATCH` probes of the two rules #451 ported, at diff shape `{given, family}`: ZERO arrived unexplained. Four went to the catch-all ('Mcintyre Smith Jr.', 'Los Santos', 'John Sir de Mesnil', 'Sir Smith'), two to `fix(comma-precomma-family)` ('McDonald, Ronald', 'Smith, Sir de Vaux'), and one ('Ste Marie') to the neighbouring `fix(#360) ste` rule that exists to claim it. Driven through `classify()` rather than read off a gate run, deliberately: six of the seven are in no corpus, so a run would never reach them — which is also why the promise they carried could go false unnoticed. The blast radius of the shape was never just its own heading. +- **Copy-paste is how a ledger comment goes false without being edited.** All three ledgers carried the same `fix(#379)` comment; four of its sentences were true only in the 1.4.0 file they were written for, including one citing `fix(suffix-routing)`, which exists in no 2.x ledger, and one saying a name "falls to the fields-only catch-all" twenty lines after the same file says no fields-only rule exists there. Corrected in `8237b54`. A cross-ledger claim needs re-deriving per file, not copying. + +Declined: + +- A `[[never]]` for #342 and #397 (2026-08-27) — honest, and it reddens the gate: an excluded name reports UNEXPLAINED and exits non-zero, so every release would block until both bugs are fixed. Both are on v2.2, so the window is real. The cause-not-consent rule says the same thing in the ledger and keeps the gate green. +- One replacement rule with a `(jr|v|mp|x|m\.?a\.?|mc)` tail (2026-08-27) — measured, not argued: `m\.?a\.?` matches the corpus fragment `'M.A.'`, which `_normalize` leaves as `'m.a'` and is not a `SUFFIX_ACRONYMS` entry, so the member guard rejects it — the same finding the `ambiguous-surname-acronym` rule already records. `x` is in no vocabulary set at all, reaching `suffix` through the trailing-roman-numeral fork, so it could not be a member either. +- A literal alternation of all 14 absorbed names (2026-08-27), on #413's literal-anchored precedent — kept as the fallback if a reading turned out contested, and not needed: every family had an existing rule in another ledger, a decision to cite, or an open issue. +- Extending #451 to the trailing-numeral ambiguity (2026-08-27) — `'Mohamad X'` loses its family to `_ROMAN`, and by C-i's "under uncertainty, default to AMBIGUOUS" a trailing `X` IS borne as an ordinary name in that position (Malcolm X). Numerals reach `suffix` through a pattern rather than a vocabulary set, so there is no ambiguous SUBSET for `x` to join — but the fork does not go unreported: measured, `parse('Mohamad X')` and `parse('John V')` both carry `AmbiguityKind.SUFFIX_OR_NAME`, which `_assign` raises from the fork itself because "V/X/I are ordinary middle initials, so taking it as a suffix is a call, not a fact". The contrast worth keeping is with the acronym half: `parse('Aishwarya Rai')` loses its family and reports NOTHING, because `rai` is unambiguous vocabulary. So the numeral fork already flags its own uncertainty and the vocabulary path does not — which is an argument about `SUFFIX_ACRONYMS_AMBIGUOUS`, not about `_ROMAN`, and it belongs to #342 rather than here. `'Malcolm X'` is in no corpus (`'Mohamad X'` is, and is what this bundle classified); no open issue on the numeral half. +- `explains = N` per rule (2026-08-28, #452's own first sketch) — catches a rule explaining fewer NAMES even while its fields stay exercised, which the field check does not. Declined on maintenance cost rather than merit: 179 numbers across the three ledgers, every one of them liable to move whenever a corpus does rather than only when a rule does. Measured on the precedent — #414's harvest moved 17 of the 30 `_CORPUS_CLAIMS` entries that existed then and added 14 more, parsing the roster at `7a10689^` and `7a10689`. (That roster's own comment said "Twenty of these moved" and was wrong; measured three ways — that commit, the whole of PR #415, and the backtick-harvest commit after it — the figure is 17 every time, and the comment now says so with the recipe.) The field check needs no number and cannot go stale. This is the design to revisit if a name-count shrink ever bites. +- Extending it to `'Donald mc'` (2026-08-27) — `mc` is `SUFFIX_ACRONYMS` AND `PARTICLES`, the collision C-ii decided for `vd`, and C-ii reaches this shape only through P6's attachment, which is scoped to the comma form. The authority is rules/S2's Accepted — "an unambiguous suffix is consumed even when that leaves no family name at all" — and NOT P6's scope note, which was this entry's first citation and does not support it. P6 promises the comma-less shapes "keep their positional reading", and for the words that are both particle and suffix vocabulary it does not hold: measured, `'Jong de'` and `'Smith do'` keep theirs (family 'de', family 'do') while `'Donald mc'` and `'Smith vd'` lose the family entirely to `suffix`. So `'Donald mc'` is out of #451's scope on S2's statement, and P6's own Accepted clause is falsified by two of the three words it sweeps in — its only example, `"Jong Anke de"`, is `de`-shaped, which is why no test catches it. Recorded here as a rules.md defect rather than a ledger one, and FIXED 2026-09-07 under #454: P6's Accepted clause now states the exception, naming mc and vd as the two words the positional reading does not hold for. The measurement above is unchanged and is what the repaired clause says. See #suffix-acronym-collisions. + +## differential-ledger, the corpus-tier arc (2026-09-01, #468/#469) + +The fourth ledger arc, and the first to change what the gate is FOR rather than what a rule may say. Mechanics — the tier table, the promotion recipe, the shape key, the `corpora:` line — are owned by tools/differential/README.md and not restated here; these are the decisions behind them. Counts are this session's, each recomputable by running `uv run python tools/differential/compare.py` (add `--baseline 1.4.0` for the skip figures) over the checked-in corpora and reading its `corpora:` and `corpus:` lines. + +Decisions that landed: + +- 2026-09-01 #468 — every corpus carries a TIER, and only the CONTRACT tier can fail the run. `corpus.jsonl` (scraped from v1's test banks) and `corpus_issues.jsonl` (harvested from the tracker) became RADAR: their diffs still classify against the ledger, so release notes keep their grouping, but an unmatched one prints under `UNCLASSIFIED (radar)` and the run still exits 0. `corpus_cjk.jsonl`, `corpus_rules.jsonl` and `corpus_shapes.jsonl` keep the full discipline. The reason is a VOCABULARY gap and not a volume one: the gate demanded a rule-precision explanation for names nobody chose, and the honest explanation for many of them was "this string exercises nothing" — which no ledger key can say: a rule means "this diff is intended and here is why", an exclusion means "this shape must never be explained", and neither is the sentence wanted. A rule invented to say it would be the #372 shape by construction, since the only thing it could narrow on is which file the name happens to sit in. `_CORPUS_TIERS` in compare.py is the roster and is fail-closed like `_CORPUS_FLOORS`: a corpus added without a tier is a hard error, so the split forces a decision when a corpus arrives rather than defaulting one. The tier rule is about UNMATCHED diffs, and two recorded things outrank it on a radar name — a `[[never]]` exclusion (two bullets below) and a contest row in `_RECORDED_DIFFS` (the watched-shapes arc, 2026-09-05, which states the rule and its ground); a watched row that fails does so on an entry whose tier reads contract — the default-order entry when the name has one, the first-loaded otherwise — which is this rule applied and not an exception to it. +- 2026-09-01 #468 — nothing is DELETED to quiet the gate; promotion is the answer to "then how does a radar name ever bind again". A name enters the contract by being CHOSEN — a tests/v2/cases.py row plus a shape tag, which puts it in `corpus_shapes.jsonl` at the next regeneration. Deleting the string was the alternative and is worse in both directions: it loses the radar signal, and it makes the gate's own population editable by whoever finds the gate inconvenient, which is the one edit a differential gate must not make cheap. A meaningless radar string costs one parse and one report line against a comparison of 1120 names today (1113 when this bullet was written, hours earlier the same day; #486's bullet below carries the arithmetic). +- 2026-09-01 (decided in review) — a `[[never]]` exclusion OUTRANKS the tier: a diff it refuses reports UNEXPLAINED and exits 1 on either tier. Not a carve-out but the tier's own principle applied — the split governs names nobody looked at, and an exclusion is chosen exactly as a rule is, someone having written its `why` and its `examples`. It is also what the shipped ledgers already promise, and the measurement is what settled it: the 1.4.0 ledger's trailing-`Ph. D.` exclusion, whose `why` says "a diff here is a regression", reaches exactly three corpus names (`John Smith Ph. D.`, `John Smith, Ph. D.`, `John Smith, Ph.D.`) and every one of them lives in `corpus.jsonl` or `corpus_issues.jsonl` alone. Reading the tier as outranking the exclusion would have retired that entry outright, silently, in the commit that split the corpora. Recompute by matching each `[[never]]` entry's `name_regex` over the corpus files and reading which files the matches come from. +- 2026-09-01 (decided in review) — a ledger RULE may narrow by comparison order, through a new `orders` key; it is optional, and its absence is the order-blind behavior every rule written before shape-tagged entries has. Forced by the first order-scoped rule anyone wrote: `feat(#395)`'s fold on `de la Cruz Juan Carlos` moves `{family, given, middle}` under the two family-first orders, and the most plausible regression of that very change — the fold leaking into the DEFAULT order — moves those same three roles on that same string, which `corpus_rules.jsonl` carries as a bare string and therefore compares. Order-blind, the rule describing the feature would have absorbed the regression of the feature and called it intentional: #372's failure mode reached from a new direction, and on the one name best placed to hide it. The legal set is BORROWED from shapes.py rather than hand-copied (`_legal_orders`), the same call build_cjk_corpus.py makes for the script table — an order no shape declares is an order no comparison runs under, so a rule scoped to a typo could only ever be dormant. Amended 2026-09-01 (decided in review): the legal set gains a `DEFAULT` SENTINEL, and the two legacy rules that needed it were scoped. Found when the reviewer traced the absorption running the OTHER way from the one above — the order-blind `fix(#399) a maiden marker bounds the particle chain that swallowed it` rule reaches `de la Cruz née Vega`, which `corpus_shapes.jsonl` compares under both family-first orders, and it sorts AHEAD of the two rules written for that name, so a family-first-only `{family, maiden}` regression there would have classified as intentional. The fix a rule wants is `orders`, and no rule could say "default order only": the members are constant NAMES, the default order is the absence of one, and TOML has no null inside an array. So `"DEFAULT"` names it. `fix(#399) …that swallowed it` took `["DEFAULT"]` in both 2.x ledgers and `fix(#296) … 'John Smith, Dr.' keeps its split and its title` took `["DEFAULT", "FAMILY_FIRST"]`, those being the comparisons each one's diffs actually come from. Order-blind rules stay legal — every ledger written before shape tags is full of them — so the absorption is made VISIBLE rather than illegal: a run prints an `ORDER-BLIND` block for each (issue, name, order) where a rule with no `orders` key explained an order-bearing diff, informational and outside the exit code. Recompute by running the gate at each of the four baselines and reading that block; it is empty at all four today. +- 2026-09-01 (decided in review) — EXCLUSIONS stay order-blind, and `validate_exclusions` rejects an `orders` key as unknown. The dormancy arc's monotone-refusal reasoning is the whole argument: an exclusion only ever removes a name from classification, so the widest thing an over-wide one can do is make a name report UNEXPLAINED and fail the run. There is no silent direction to protect against, and the narrowing a rule needs is the one an exclusion must not have — "never explain this, but only under FAMILY_FIRST" is a promise with a hole in it. +- 2026-09-01 #469 — "some names are only supported under one order" is answered by the SHAPE rather than by a ledger exception. shapes.py records, per input shape, the `name_order` the shape is an input FOR and the oldest baseline whose worker can honor that order; `compare.py` parses a tagged entry under that order on both sides, compares it on the v2 surface alone (the facade is the v1-compat surface, and a family-first name is not a v1 contract), and leaves out an entry whose run predates its `min_baseline` — reported as `skipped N names tagged shape(s) [...]` and counted in the `corpora:` line, so a shrunken comparison is never silent. Structural, and that is the point: nothing has to be written into any ledger to say a family-first name is not a default-order promise. Today's arithmetic, all of it recomputable from the checked-in files and one gate run: 13 shape entries, of which 4 dedupe into names already compared under the default order, for 1113 comparisons; shapes 4 and 5 declare `min_baseline` 2.0.0, so 7 entries are skipped at `--baseline 1.4.0` and 1106 names compare there. +- 2026-09-01 #486 (later the same day, so read this bullet as superseding the arithmetic in the one above rather than the two figures disagreeing) — the shape 1-3 variation matrix filled the given-first half of the inventory, which the feature work that authored shapes 4 and 5 had left at whatever the pipeline PRs happened to tag. 14 existing case rows gained a tag and 3 rows were authored for slots no row instantiated, taking `corpus_shapes.jsonl` from 13 entries to 30 and shapes 1/2/3 from 1/4/1 names to 9/11/3. The arithmetic moves with it and the recipe is unchanged — run `uv run python tools/differential/compare.py` and read its `corpora:` and `corpus:` lines, adding `--baseline 1.4.0` for the skip figures: 30 shape entries, of which 14 dedupe into names already compared under the default order, for 1120 comparisons; shapes 4 and 5 still declare `min_baseline` 2.0.0, so 7 entries are skipped at `--baseline 1.4.0` and 1113 names compare there. The dedupe count is the half worth reading, because it is what the tier promotion looks like from this side: 8 of the newly tagged names were in a RADAR corpus only, so the tag moved them into the contract tier without adding a comparison, and 2 more were already contract through the rules corpus. None of the 17 names #486 tagged or authored needed a ledger rule — the seven of them that were in no corpus at all diff at no baseline, and intentional stayed 226/205/113/0 across the change. Not a claim about all 30 entries: the seven shape 4/5 entries are classified by ledger rules today, and scoping exactly those rules is what the `orders` bullet three above is about. +- 2026-09-01 #469 — the CJK arrangement is deliberately OUTSIDE the shape inventory. Whether an unspaced CJK name is a third family-first shape is the open question on #469, and `corpus_cjk.jsonl` — contract-tier already, generated from the case table already — covers that ground meanwhile. Tagging it in would have to ANSWER the question first, since a shape row cannot be written without a `name_order` and a `min_baseline` for it, and the inventory is a table of arrangements rather than where that argument belongs. + +- 2026-09-01 #469 (later the same day, so read this bullet as SUPERSEDING the one above rather than the two disagreeing) — the CJK arrangement is IN the inventory after all, as shapes 6 and 7, and the bullet above is right about why it could not be until the question was answered: a shape row cannot be written without settling what it admits. What settled it was splitting the question the bullet treats as one. There is no single "unspaced CJK name" shape — there is a pure family-first arrangement (6) and a source-order transcription listing (7), and the composed forms are not arrangements at all. Both take `order=None`, which is the part that had looked impossible: the family-first reading is SCRIPT-carried rather than declared, so a pure shape 6/7 string already parses correctly under the DEFAULT policy and there is nothing for `order` to name. `min_baseline` is then documentary rather than a skip trigger, an asymmetry with shapes 4/5 that tools/differential/shapes.py's docstring states so it reads as designed. The doctrine half is its own entry (cjk-comma-demotion above); what belongs here is that `corpus_cjk.jsonl` no longer "covers that ground meanwhile" — the ground is split between it and a radar file now. +- 2026-09-01 #469 — the `tolerated` flag on a case row is the demotion's VEHICLE, and it is recorded here rather than as a mechanisms.md entry. It is an explicit declaration, reviewed one row at a time exactly as a `shape=` tag is, mutually exclusive with one, and restricted to rows whose text bears a classified codepoint; `build_cjk_corpus.py` partitions on it, and a text marked on one row and not another is a hard error, the flag being per-text in effect. Why not a mechanisms entry: that catalog is keyed by RECURRING problem shape and its contract statements are citable verbatim from code comments, and this is one arc's device with no second instance and no comment needing to cite it — the tier mechanics it rides on are already owned by tools/differential/README.md, so an entry would restate a source rather than say something the source does not (the #473 lesson under decisions/review-agent-single-source). If a second arc ever needs "record the behavior, promise nothing", that is the point to promote it and this bullet is the first instance to cite. + +## differential-ledger, the initials view (2026-09-01, #484/#462) + +The fifth ledger arc: the gate gains a column it never had. `initials()` is a derived render view, so a change in it with every role identical was invisible to the seven-field comparison by construction — the `middle_as_family_fold_order` row's `notes` in `tests/v2/cases.py` say as much, and #408 moved initials for a whole minor with the gate green. Mechanics (which rows carry `_initials`, the roles-identical guard, the one-field rule constraint) are owned by tools/differential/README.md; these are the decisions. + +- 2026-09-01 #484 — `_initials` is a pseudo-field on BOTH surfaces, the facade's at every baseline (1.4.0 has `initials()`) and the core's from 2.0 on, and it enters a name's diff ONLY when the seven roles and `_ambiguities` agree on every compared surface. The alternative — strict subset semantics, as `_ambiguities` has — was measured and rejected: names whose roles AND initials both move are claimed by 53 of the 72 explaining rules at 1.4.0, 43 of 66 at 2.0.0 and 36 of 59 at 2.1.0 (recompute below), every one of which would have had to grow `_initials`, and a rule declaring it absorbs any initials change on its names anyway, so the churn bought no discrimination. A role move drags its initials with it; the pseudo-field is a render-layer instrument, not a second copy of the role comparison. Consequences a rule author must know: `fields = ["_initials"]` is a rule of its own kind and lists nothing else — validate_rules refuses the mix as dead; and a count taken from the `_initials` summary is a FLOOR on initials movement, not the population (AGENTS.md carries the 83 / 28 / 55 measurement). `family_base` was considered as a second derived view and NOT added (YAGNI; adding one is the same two rows per surface, recorded here rather than on the issue). +- WHAT IT FOUND, measured 2026-09-01 by patching compare.py's own worker to emit `initials()` and comparing against the tree — and measured on THIS BRANCH'S BASE, before `31623d9` fixed #462, which is the tree the parenthetical describes: there the tree matched 2.2.0 exactly, 0 diffs of any kind. Three clusters of initials-only names, 140 at 1.4.0 and 28 at each of 2.0.0 and 2.1.0. On the SHIPPED tree the same recipe gives 126 at 1.4.0 and 42 / 42 / 14 at 2.0.0 / 2.1.0 / 2.2.0 (re-measured 2026-09-02; the gate itself is the recipe now, see RECOMPUTE). The two readings reconcile through cluster (ii) alone — 126 + 14 = 140, 28 + 14 = 42, 0 + 14 = 14 — the 14 #462 names having moved from the 1.4.0 side to the 2.x side when the fix landed. (i) The R2 readmission, 27 names at 1.4.0, 2.0.0 and 2.1.0 — `Anh Do` → `A. D.`, `Juan van der` → `J. v. d.` — shipped in 2.2.0 with a release note and the R2 entry above but no ledger rule, because none could see it; now `fix(#385/#402)`, a literal list of the 27 (the shape "a part of nothing but particles" is no property of the raw string), plus `de los Santos` under its own `fix(#360)` rule since ITS initials moved when `los` became a particle, not with R2. (ii) The facade dropping a middle `E`/`Y` initial, 14 names at 1.4.0 only, all but one radar tier — the population beside the count, per docs/design/AGENTS.md's population rule, is the names carrying such a letter in the MIDDLE or FAMILY group, which is the only position the bug reached, and there are 14 of them: 14 of 14 in that position moved. The `fix(#462)` rule's regex (`(?:^|[\s,])(?:[EY]|[EeYy]\.)(?=[\s,]|$)`) reaches 18 corpus names, four more than the population, and the other four (`E Anne D`, `E Jones`, `E Maria`, `Y. L.`) carry the letter in the GIVEN group, which has always initialed every word it holds, so nothing moved. The discriminator is the GROUP and not the position: `E Anne D,Leonardo` leads with the E too, but its comma re-roles the run into the family group, and it is one of the 14 that move: #462, a 1.4 parity break shipped in 2.0.0 and recorded nowhere. Fixed in the same PR (R3's bullet below); the 14 then move against the three 2.x baselines instead and carry a `fix(#462)` rule in each. (iii) Per-word grouping, 98 names at 1.4.0 only: 1.4.0 initialed one group per `*_list` element and rendered a multi-word element's letters with the separator and no delimiter — `Juan Velasquez y Garcia` → `J. V G.` at 1.4.0 and `J. V. G.` here, `Abdul Salam Hassan` → `A S. H.` and `A. S. H.` — and the 2.0 facade initials each word. Measure with the FULL name, not with the run alone: `Velasquez y Garcia` and `Abdul Salam` as whole inputs give `V. G.` and `A. S.` at 1.4.0 and here alike, the run being the family and initialed per word either way, so the bare forms this bullet illustrated with until 2026-09-02 reproduced nothing. Strip periods and spaces and all 98 agree with 1.4.0 letter for letter. Changed in 2.0.0, mentioned once in passing (the #408 bullet under R3: "v1's initials GRANULARITY"), never classified. Classified LATE, on purpose: mechanisms.md#FACADE-CONTRACT promises 1.4-warning-free code keeps working except for release-log-classified fixes, and a classification three minors overdue is still owed, so the 2.3.0 log carries a bullet that says the behavior has held since 2.0.0. Four rules rather than one, because the Latin-alternation guard keys a rule to a single copied vocabulary and the 98 copy three (CONJUNCTIONS 66, BOUND_GIVEN_NAMES 19, PARTICLES 11) plus the `Ph. D.` merge (2). The connective rule cannot claim cluster (ii), and it takes BOTH of its exclusions — the case-sensitivity and the trailing whitespace lookahead — to keep it out. Of the 14, 7 carry a bare capital `E` and 7 a dotted `E.`/`e.`: the case-sensitivity excludes the bare capitals, and the alternation's trailing whitespace lookahead excludes the dotted ones whatever their case, a period not being whitespace. Since the fix in `31623d9` those names agree with 1.4.0 and diff only against the 2.x baselines, so neither half is keeping two live clusters apart today. What they do is make any FUTURE facade initials change on these shapes surface as UNEXPLAINED at 1.4.0 rather than be absorbed here as per-word grouping. Gate totals moved from 226 / 205 / 113 / 0 intentional to 352 / 247 / 155 / 14, zero unexplained, exactly the new rules' explained counts. +- RECOMPUTE: load `tools/differential/compare.py` by path; build the entries as main() does (`_load_entries`, `_load_shapes`, `_CORPUS_TIERS`, dedup by (name, order), drop order-bearing entries below their shape's minimum); patch `_worker_source` to append `_initials` to both rows; `_run_worker`; compare against the tree's facade and core. TWO RECIPES, and only the second of them needs that patch. (a) The CLASSIFIED TOTAL per baseline is read off the gate itself, no patch and no in-memory run: `compare.py --baseline X` prints a `## issue (N)` heading per rule, and the total is the sum of the headings of the rules whose `fields = ["_initials"]` — at 1.4.0 `fix(#385/#402)` 27, `fix(#360) los` 1 and the four `fix(initials-per-word)` rules 19 / 66 / 11 / 2, summing to 126; at 2.0.0 and at 2.1.0 `fix(#385/#402)` 27, `fix(#360) los` 1 and `fix(#462)` 14, summing to 42; at 2.2.0 `fix(#462)` alone, 14. What the gate does NOT print is the per-name detail — a heading lists at most ten of its names and no diff fields at all, and the UNEXPLAINED and radar blocks are empty on a green tree — so it answers how many and under which rule, never which names or which surface moved. (b) The PER-NAME SPLIT, and the pre-fix 140 / 28, need the in-memory run described above: load compare.py by path, run main()'s comparison loop or hook `diffing`, and keep the names whose diff is exactly `{"_initials"}`. READ THE PATCH STEP AS HISTORY, and it belongs to (b) alone: since `a748862` the shipped worker template emits `_initials` itself, so patching `_worker_source` is what a measurement BELOW that commit needs and re-applying it on this tree changes nothing. The pre-fix 140 / 28 must be taken on the branch's base, before `31623d9`, rather than on this tree. Split "initials only" from "roles and initials"; within the former, lowercase and strip `[.\s]` to separate per-word grouping (equal after stripping) from content changes. The guard's liveness is proved by mutation, not by the green run: with `not diff and` removed from the guard, a roles-and-initials name has its role diff REPLACED by `{_initials}`, which no existing rule declares, and the 1.4.0 run reports 117 of them as no longer classified — 79 unexplained plus 38 radar, from 0 and 0. The same mutation is where the first bullet's churn figures come from: the rules that lose at least one name are the rules strict subset semantics would have had to widen, counted against the rules that explain anything at that baseline (53 of 72, 43 of 66, 36 of 59 at 1.4.0 / 2.0.0 / 2.1.0; re-derived 2026-09-02, the 1.4.0 numerator having been recorded as 43 until then). + +## differential-ledger, the rule-order arc (2026-09-02, #382) + +The sixth ledger arc, and the one that closes the question mechanisms.md#LEDGER-RULE-SEPARATION has carried open since #375's reorder mutation: file order settles every contest, and nothing said which contests existed. Mechanics — the TOML shape, the required `why`, the trap that a rule key written below the block joins the exemption — are owned by tools/differential/README.md; these are the decisions. Every figure below was measured 2026-09-02 against the real 1.4.0 wheel or recomputed from the checked-in files, and each carries its recipe. + +Decisions that landed: + +- 2026-09-02 #382 — an order-decided contest must be DECLARED, and narrow-first is the declaration-free default. A pair is a contest when the later rule's `fields` are a strict subset of the earlier one's, some corpus name's `name_regex` reaches both, and some comparison order reaches both; where the EARLIER rule is the wider one it carries a `precedes_narrower` block naming the later rule and saying why it is the classifier of record — usually what it describes that the later one does not, and in the three regex-accident pairs (`fix(#296)`/jr, and the two `fix(cjk-glued-honorific-peel)`/`fix(suffix-routing)` pairs) that the later rule reaches the name through its REGEX rather than by describing it — both `fix(suffix-routing)` patterns are anchored at both ends and open on a bare `\S+` run, which swallows the trailing comma of `Smith,` and matches hangul as readily as Latin, while the prose they carry is scoped to a comma-less two-token Latin name. Only ONE of the three is false outright — `fix(#296)`/jr, where 1.4.0 read `'Smith, Jr.'` as title `'Jr.'`, first `'Smith'`, so the family→suffix movement the jr rule describes never happens on it. The other two are PARTIAL, and symmetrically so: on `김민준씨 Jr.` and on `田中さん II` alike the trailing token really does leave `family` for `suffix` (1.4.0 first `'田中さん'`, last `'II'` → tree last `'田中'`, suffix `'さん, II'`, exactly the movement the numeral rule shows on its own `John V`), and what neither `fix(suffix-routing)` rule can describe is the peel and the segmentation that move `given`. Recomputed 2026-09-02 against the 1.4.0 wheel; the exemptions themselves claim only the scope mismatch, so do not read a falsity claim into the two partial ones. `compare.py` refuses the run, before the worker spawns, where such a pair goes undeclared. Reordering is NOT the alternative fix and the failure message says so: it moves which rule classifies a name and breaks `_CROSS_RULE_WINNERS`. What the check buys is that nothing else in the suite can see the hazard at all: `_CORPUS_CLAIMS` measures each rule alone, the gate total is per-corpus, and `_CROSS_RULE_WINNERS` pins contested outcomes only for names somebody hand-added. +- 2026-09-02 #382 — the check covers NESTED pairs, and nesting is sufficient for an order-decided contest rather than necessary. Two rules whose `fields` merely INTERSECT both admit any diff inside that intersection, so `classify()` hands such a name to whichever is written first exactly as it does for a nested pair, and `order_contests` cannot see it. That class sits outside the check by the reasoning `order_contests`' docstring already states for EQUAL `fields` — neither rule is narrower, so "narrow-first" says nothing about the pair, `precedes_narrower` has no narrower rule to name, and `_CROSS_RULE_WINNERS` stays the instrument — which covers every pair where neither `fields` set contains the other, equal `fields` being its special case. Not oversight, and not free either: measured 2026-09-02 over the four ledgers, counting pairs that share a corpus name and whose `orders` are not disjoint, 11 are strictly nested wide-first (what the check refuses), 40 nest in either direction, 11 have equal `fields`, and 111 have any non-empty intersection — so 60 overlap without nesting or equality. Widening to the general predicate would demand 111 written justifications where the real number is eleven, the same kind of answer this entry already gives below about the 657 figure: a predicate whose roster nobody can write out one justification at a time is not a usable predicate, whichever condition inflated it. The blind spot is worked and filed as [#498](https://github.com/derek73/python-nameparser/issues/498): `fix(#271/#272/#298)` (`{family, given, middle}`) and `fix(cjk-delimited-nickname)` (`{family, given, nickname}`) intersect in `{family, given}` without nesting, three contract-tier corpus names (`マイケル・ジャクソン`, `威廉・莎士比亚`, `高橋・一郎`) diff exactly those two roles against the 1.4.0 wheel, swapping the two rules reattributes all three, none is in `_CROSS_RULE_WINNERS`, and `order_contests` lists the pair in neither arrangement. RECOMPUTE the five figures with `_rule_reach` per ledger over the `corpus*.jsonl` union, classifying each pair by whether `a.fields & b.fields`, `b.fields < a.fields`, `a.fields < b.fields` or `a.fields == b.fields`; #498's body carries the loop. Closed 2026-09-05 by #498, which did not widen this predicate: the class is now covered at RUN time, where a measured diff makes the population 14 rather than the 71 pairs of the equal-or-overlap class (the 60 counted here plus the 11 with equal `fields`), and every name in it is pinned — the three named here among them, so read that "none is in `_CROSS_RULE_WINNERS`" as the state this measurement found (the #498 completeness check below). +- 2026-09-02 #382 — an ESCAPE HATCH here, where #452's and #456's were declined, and on the terms #452 set. Both of those bans were free to state: measured at the time, 0 of the 179 rules across the three ledgers then on disk had #456's shape, and #452's fourteen over-declarations (3 of 67 EXPLAINING rules at 1.4.0, 5 of 58 at 2.0.0, 6 of 51 at 2.1.0 — a different and smaller population than #456's 179, which counts every rule) were all narrowed before the check landed, so neither ban had to argue with a rule that was correct as written. #452's entry states the price of that strictness — "the first rule that genuinely needs a wider declaration has to argue for a key the way `dormant` was argued for in #373". Read that as the PROCEDURE it sets, not as a hatch this key opens: `precedes_narrower` is not a wider `fields` declaration and does nothing for an over-declared rule, which still exits the run non-zero. What carries over is the standard of proof, and here it is measured rather than asserted: eleven pairs in `expected_since_1.4.0.toml` are wide-first and every one of them is correct where it sits, so a ban would have had eleven rules to reorder or eleven prose descriptions to falsify. The hatch is narrowed the way `dormant` is: it names ONE rule (a blanket opt-out would be inherited by every narrower rule added later, which is the widening the check exists to refuse), the `why` is required, and a declaration standing over a pair that is no longer contested is refused as loudly as an undeclared contest. +- 2026-09-02 #382 (decided in review) — the vacancy half REFUSES only on a full run and prints a NOTE under `--corpus`. The two checks are not symmetric under a narrowed name set, and the asymmetry is the whole reason: narrowing removes contests, so for the undeclared check `--corpus` is only ever more lenient (it can only under-report, never false-alarm — do not call that fail-closed, which this repo uses for the rosters that REFUSE on a missing entry), while for the vacancy check it INVERTS — a live declaration whose contested names all sit outside the subset reads as vacant. Shipped as a regression and caught in review: as first written, a `--corpus` run against the 1.4 ledger exited 1 and told the contributor to delete exemptions the full gate needs — measured, each of the six corpora run ALONE reports vacancies, 11 of the 11 for `corpus.jsonl`, `corpus_cjk.jsonl` and `corpus_shapes.jsonl`, and 8 / 7 / 5 for the other three — after which deleting them would have made the full run refuse with that many undeclared contests. Read the shape and not the digits: the number varies with the subset, and only zero would have been safe. The file already treats `--corpus` differently in two places and this is the third, so the three should be read together rather than made uniform: the corpus-floor roster is SKIPPED entirely under the flag, `over_declared_rules` still FAILS the run and appends a NOTE saying the union it computed is over a subset, and the vacancy check does not fail at all. The strengths differ because the error directions do — only the vacancy check inverts under narrowing. +- 2026-09-02 #382 — TWO name populations, deliberately, rather than one shared function. `main()` must check the corpus it ACTUALLY compares, because `--corpus` narrows it; the unit guard in tests/v2/test_ledger_guards.py must check every corpus on disk, so that a rule added by a later bundle is checked at pytest speed with no baseline wheel. Forcing one function would break `--corpus`. They agree by construction instead: `_entry_name` in tests/v2/_differential_fixtures.py says in its docstring that it mirrors `compare.py`'s `_load_entries`, and both read the same `corpus*.jsonl` glob. +- 2026-09-02 #382 — the recorded negative control is deliberately BLIND to `precedes_narrower`. `_ORDER_EXEMPTION_EFFECT` records the eleven pairs and their name counts from `order_contests`, which never reads a declaration; a control that consulted the mechanism it controls for would measure nothing, and would go green the moment the predicate stopped finding anything at all. Its assertion that not every ledger's list is empty is there for exactly that failure (mechanisms.md#RECORDED-ROSTERS). + +Found rather than decided, and worth as much: + +- **#382 option 3's premise is FALSE, and the crux name is in the corpus.** The issue proposed a mechanical narrow-first check — refuse the wide-first pair, or sort by specificity. `fields`-subset is a proxy for specificity, and it is the wrong one where a wider rule describes a COMPOUND behavior its component rule does not. `马丁·路德·金씨` divides on the nakaguro AND peels its glued hangul honorific: `fix(#272/#308) nakaguro division and a glued hangul honorific in one name` describes what happens to it, `fix(cjk-glued-honorific-peel) glued honorific peels into suffix` describes half of it, and the WIDER rule wins by position, correctly. Narrow-first would have reattributed the name to the rule describing half — #372's defect reintroduced by the check meant to prevent it. The name is contract tier (`corpus_rules.jsonl`) and appears nowhere in `_CROSS_RULE_WINNERS`, so it is exactly the crux #382 was filed over and exactly the name no existing guard was watching. +- **The predicate needed a THIRD key, found in review.** The first implementation read `name_regex` and `fields`. `_entry_matches` narrows by `orders` as well: two rules declaring disjoint `orders` never see the same comparison, so file order decides nothing between them however nested their `fields` are. Omitting it made the detector read different boundaries from the predicate it models — docs/design/AGENTS.md axis 2 — and would have demanded a written justification for a hazard that cannot occur. It changes no figure today and is kept for correctness, not for its yield: measured over the four ledgers, adding the `orders` test removes 2 of 1350 nested pairs and 0 of the 657 wide-first ones. The nearest live shape is in both 2.x ledgers, where `fix(#399) a maiden marker bounds the particle chain that swallowed it` (`orders = ["DEFAULT"]`) and `fix(#399)/feat(#395) a consumed maiden marker leaves the family-first fold no given name` (`["FAMILY_FIRST", "FAMILY_FIRST_GIVEN_LAST"]`) have nested `fields` and share a corpus name, `de la Cruz née Vega` — narrow-first today, so nothing reports it either way; invert that nesting and the omission would have demanded a `why` for a comparison that never happens. +- **Nine of the eleven pairs are LATENT, not live**, which is the honest statement of what a static predicate costs and what it buys. Measured against the 1.4.0 wheel, a pair is a live order-decided contest only where some co-matched name's ACTUAL diff is a subset of the narrower rule's `fields`. In nine of eleven the real diff needs a role the narrower rule does not declare, so that rule is ineligible for those names wherever it sits. Only two pairs are live: `fix(comma-family)`/`fix(comma-precomma-family)`, where `John Smith, Mr.` is the one such name, and the compound/peel pair below, where 15 of the 17 co-matched names have such a diff and nine of those are this pair's to decide — the other six go to rules written above both, so the two counts answer different questions and neither is derivable from the other. So the predicate OVER-REPORTS relative to the measured diffs, and the price of that is eleven reasons somebody had to write. What the nine buy is the hazard that would ACTIVATE if a rule's `fields` ever widened — written down before the three later bundles add rules, which is when it is cheap. +- **A whole error class in the first draft of the exemption prose: concluding a RULE's tier from its NAMES'.** Three separate `why` texts did it. Tier is a property of the corpus file a name comes from, and a rule's reach usually spans both tiers. Measured: `fix(cjk-glued-honorific-peel)` explains 17 names, 14 of them CONTRACT tier — say the tier and not the file, even though all 14 happen to sit in `corpus_cjk.jsonl` today: the tier is what the gate acts on and `_CORPUS_TIERS` is where to read it, while a name can join or leave a file without changing either — and deleting the rule sends 12 contract-tier names to UNEXPLAINED (two of the fourteen re-home to `fix(cjk-honorific-suffix)`; the other three of the seventeen are radar) — so the gate does demand that rule, whatever the tier of the names it is contested over. Only `fix(cjk-comma-compound)` is radar-only in the strong sense: 11 explained names, all radar, and its whole 23-name regex reach radar too. Worth recording as a caution and not only as a corrected fact — the review rounds on this prose found six false claims, then one, then two more of this class. Every one was in prose nothing can machine-check, and MOST were caught by re-running the wheel rather than by reading — but not all, which is the part worth keeping: of the first round's six, two came only from reading, one citing "a few rules below" for a quote 52 rules away and one opening "the one live pair of the eleven" while a second exemption in the same file declared itself live too. Distance-in-the-file and prose contradicting prose are the classes no wheel run can reach. +- **One exemption says "least wrong reading available", and that is a finding about the ledger's vocabulary.** `fix(cjk-comma-compound)` outranks `fix(cjk-glued-honorific-peel)` on nine names — 17 reach both regexes, 15 have a diff the peel rule's `{family, given, suffix}` admits, and 6 of those 15 are taken by rules written above both — and its label is untrue of three of the nine. `王先生, V.`, `田中さん, V.` and `김민준씨, V.` show no comma routing and no order flip; their whole diff is `{family, suffix}`, the glued peel alone, against `{family, given, suffix}` on the six genuine compounds. They land on the compound rule because its criterion is "the diff includes `family`", and `family` moves only because the peel took the honorific off it. The order must still stay: the peel rule's prose disclaims comma names, so promoting it would mislabel the six. The rule that would describe them is a family-side twin of `fix(cjk-comma-honorific-peel)`, which covers this shape for a POST-comma given name; writing one was weighed as [#496](https://github.com/derek73/python-nameparser/issues/496) and DECLINED (2026-09-02), and the Declined entry below carries the evidence, the #495 tension and the reopen trigger. Declining it discards no part of this finding: the compound rule's label is still wider than those three names are, which is what this bullet and the exemption's own `why` exist to say. +- **A guard can pin the winner of a contest and still let the recorded diff shape be wrong.** `test_the_recorded_rule_still_wins_each_contested_name` feeds `classify()` the RECORDED shape and never checks that shape against a real comparison, so a wrong shape that still routes to the same rule passes forever. This branch fixed one: `田中さん II` was recorded as diffing `{given, suffix}` and measures `{family, given, suffix}` against the 1.4.0 wheel, and the claim had been copied to four sites. The winner did not move, so every argument resting on it survived — which is why nothing noticed. Filed with the general shape as [#497](https://github.com/derek73/python-nameparser/issues/497). + +The measurement, and how to redo it. Eleven wide-first contests in `expected_since_1.4.0.toml`; 0 in each of the three 2.x ledgers. Six of the eleven are contested over at least one contract-tier name, five only over radar names — the division [#495](https://github.com/derek73/python-nameparser/issues/495) argues from, and it survives a name changing tier even though the two counts do not. The load-bearing contrast is between that eleven and what `fields`-subset alone says: dropped to nesting alone, with no corpus-reach and no `orders` condition, the 1.4 ledger holds 646 nested pairs of which 367 are wide-first, against the eleven contests it actually has — and across all four ledgers, 1350 nested pairs of which 657 are wide-first. Read the ratio and not the digits: the two answers differ by more than an order of magnitude and would still differ by one after any plausible drift, which is the argument — `fields`-subset alone is not a usable predicate, and it is the corpus-reach condition that makes the check something a person can answer eleven times. RECOMPUTE: load `tools/differential/compare.py` by path and call `order_contests(rules, names)` per ledger, with `names` the union of `_load_entries` over the `corpus*.jsonl` glob — 1116 distinct names today, 326 of them reached by a corpus `_CORPUS_TIERS` marks contract. READ THAT ROSTER, do not reconstruct it from a file's name or size: `corpus.jsonl` is the LARGEST corpus at 486 distinct names and it is RADAR, having been demoted by #468 as a v1-test-bank scrape, so the contract tier is the three SMALL files (`corpus_cjk.jsonl` 73, `corpus_rules.jsonl` 248, `corpus_shapes.jsonl` 35) and the split runs 326 / 790 rather than the other way about. Reading `corpus.jsonl` as contract gives exactly 786 / 330, which is the wrong answer this figure attracts — it was measured that way twice before a third recompute caught it, and the gate's own `corpora:` line is the cheap check, since it prints the contract files first; for the contrast, re-run the same combinations keeping only the strict-`fields`-subset test. `undeclared_contests` and `vacant_exemptions` both return empty over every ledger, which is what the unit guard asserts. Note WHICH population each caller reads: `main()` checks the entries the run loaded — 1120 at every baseline, of which 1113 compare at 1.4.0 once the shape-minimum skip runs, since that skip happens after the check — while the unit guard reads every corpus on disk. The arc moved no classification: the gate reports 352 / 247 / 155 / 14 intentional diffs at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, 0 unexplained and 0 radar-unclassified at all four, unchanged across the whole branch. + +Declined: + +- Reordering the eleven (2026-09-02) — the obvious way to make the check unnecessary, and wrong twice. Reordering moves which rule classifies a name, which is a behavior change to the ledger and breaks `_CROSS_RULE_WINNERS`; and no pair is even a reorder CANDIDATE, because in all eleven the narrower rule's own prose would be FALSE or MERELY PARTIAL of the co-matched names if it won, and either disqualifies it as classifier-of-record. Partiality is the commoner half and the arc's load-bearing distinction — compound versus component, which the `fix(#400/#274)`/`fix(#400)` exemption calls "the canonical compound-versus-component shape". Read the eleven `why` texts for it: `fix(#400)` "says nothing about a maiden marker"; a widened `fix(comma-family) lone post-comma piece routes to suffix/title` "would take the union and report only half of it"; `fix(cjk-glued-honorific-peel)` says "nothing about a division"; `feat(#273)`'s prose "has nothing to say about" the remainder rejoining as a name; and two of the three regex-accident pairs are partial rather than false, the `fix(suffix-routing)` jr and numeral rules genuinely naming the `{family, suffix}` half of `김민준씨 Jr.` and of `田中さん II` and describing neither the peel nor the segmentation that moves `given`. Falsity is the minority, and one of the eleven says so of itself: the exemption `fix(comma-family) a comma followed only by titles keeps the given/family split` carries over `fix(comma-precomma-family)` calls its own pair "the one where the narrower rule would be actively WRONG rather than merely partial", and adds that the claim "is about THIS name and is deliberately not generalised". `马丁·路德·金씨` is the clearest compound, and `王先生, V.` the reverse case, where the winner is already the least wrong reading available — and stays so, since the family-side twin that would describe it was weighed as #496 and declined (the next bullet). +- A family-side twin of `fix(cjk-comma-honorific-peel)` for a PRE-comma glued honorific (2026-09-02, [#496](https://github.com/derek73/python-nameparser/issues/496), closed as not planned) — the rule that would honestly describe `王先生, V.`, `田中さん, V.` and `김민준씨, V.`, whose diff is the glued peel alone while the label they carry says comma compound (the finding bullet above). Declined on two grounds. FIRST, no gate can demand it: all 17 names the compound/peel pair is contested across measure radar — but by two different warrants, and an audit that assumes one will find the count short. Sixteen sit in `corpus_cjk_tolerated.jsonl`, the file #488 created by demoting the composed comma/Latin-wrapper CJK forms to tolerated input, `김, 민준씨` — the name #382 was filed over — among them. The seventeenth, `Dr 田中さん, V.`, is in `corpus_issues.jsonl` alone: harvested and append-only, radar since #468, and untouched by #488. (`Dr 김민준씨, Jr.` is in both files, and reads radar from either.) Radar means an unmatched diff is reported rather than fatal for want of a RULE; it is not an unconditional never-fatal, since a `[[never]]` exclusion outranks the tier and stays fatal on a radar name — `_CORPUS_TIERS`' own note, and `main()`'s two-reasons comment, which routes an excluded name to `unexplained` rather than to `radar`. No exclusion refuses these, so nothing here can demand the twin. SECOND, [#495](https://github.com/derek73/python-nameparser/issues/495) points the other way over the same corner: it asks whether the radar-only rules already in the ledger still earn their place and records `fix(cjk-comma-compound)` as having zero contract-tier reach, so #496 proposed a SECOND rule for the very names #495 is weighing a first one away from. Fewer rules for this corner is the coherent direction, and that is what makes the decline decisive rather than merely permissive. REOPEN it if a pre-comma glued-honorific name ever reads contract — and watch BOTH routes, because the two files promote by different mechanisms. A `corpus_cjk_tolerated.jsonl` name is promoted by clearing `tolerated` on its case rows, which moves the text into `corpus_cjk.jsonl` (`_CORPUS_TIERS`, and build_cjk_corpus.py's split). `Dr 田中さん, V.` has no case row at all, so there is no flag to clear: it changes tier only by being CHOSEN — a new unmarked row, or a rules.md example — which puts the text in a contract corpus, and the (name, order) dedup loads contract files first and keeps that reading. Read the tier off `_CORPUS_TIERS` either way, never off a hand-built file→tier map (the roster caution above). +- Precise per-name contest detection at differential-run time (2026-09-02; its stated reason replaced 2026-09-03, [#497](https://github.com/derek73/python-nameparser/issues/497)) — it would replace the static predicate's over-reporting with the measured nine-of-eleven split, and it needs the pinned-wheel worker pass to do it. The decline stands, on COVERAGE alone: a run-time check sees one ledger — the one for the baseline that run was invoked with — and only once somebody invokes the tool, whereas the static predicate covers a rule from the moment it is written, including a rule nobody has ever run the wheel against, which is the whole point of #382. Nor can the unit suite simply adopt the precise check: `tests/v2/test_differential.py` is deliberately hermetic — its own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or `Popen` outright — so moving it to pytest speed means spawning a subprocess and, on a cold cache, reaching PyPI. WITHDRAWN, and named here so it is not re-argued: this bullet used to say the check would sit "behind a multi-minute run" and gave that as "the whole of the reason". No measurement was recorded beside it: the phrase entered on 2026-08-05 in `7767ba2` at a single site in `tools/differential/compare.py`, and was copied outward until eight sites carried it — four there, three in `tests/v2/test_differential.py`, and this bullet. Measured 2026-09-03 over the corpus population named above, a whole run is 0.33s wall at baseline 1.4.0 and 0.57s at 2.2.0, the default; the worker pass is 0.10s and 0.29s of those, timed inside `main()`. All four baselines back to back total 1.97s, and a run with `uv`'s cache emptied first, so the pinned wheel is downloaded rather than reused, was still under a second — that last one is NETWORK- and environment-dependent and the RECOMPUTE below does not reach it, covering the warm run alone. Reproducing it means pointing `UV_CACHE_DIR` at an empty directory so the wheel is fetched rather than reused, and the number will differ on a different link. Read it as "the download is not the story either", not as a figure to check against. Read the class and not the digits: the worker pass is a fraction of a second and the whole gate is seconds at most, where the withdrawn phrase said minutes — so no argument here may be built on what the worker pass costs. RECOMPUTE: `time (uv run python tools/differential/compare.py --baseline 1.4.0 >/dev/null)`. For the worker's share, wrap `compare._run_worker` in a timer and then call `main()` — calling `_run_worker` directly on every loaded entry aborts at a 1.4.0 baseline, since the seven order-bearing shape-4/5 entries have to be dropped by the baseline-minimum skip first. Do NOT restate the decline as an error-direction argument either: within the NESTED pairs computing real diffs can only remove them, but the predicate is not a superset of the contests, since nesting is sufficient for one and not necessary (the bullet above, and #498). What the static check buys is coverage of a rule nobody has run the wheel against, not a guarantee of refusing everything it should. This decline is unchanged and is not what #498 did: the #498 completeness check ADDS a run-time predicate over a class the static one covers not at all, rather than replacing the static one over the class it does cover — so the coverage argument above is the reason both exist (the #498 completeness check below). +- Giving the peel rule a predicate the compound rule fails (2026-09-02, #382 option 1) — narrowing `fix(cjk-comma-honorific-peel)`'s `name_regex` to the honorific-bearing shapes, whose stated effect in #382 was that the pair "becomes order-independent and the original contract holds again". Declined because that effect is UNREACHABLE, and measured rather than argued. The two rules ship a BYTE-IDENTICAL `name_regex` (verified 2026-09-02 by comparing the two strings), and `_rule_reach` computes each rule's names from its OWN pattern, so narrowing the peel rule's regex narrows the peel rule's reach and nothing else: the compound rule goes on reaching every one of the names. The general form, which no regex edit escapes — where the earlier rule's `fields` are a SUBSET of the later one's, order-independence requires the LATER rule to stop reaching the name, and an edit to the earlier rule's own regex cannot cause that. Here the peel rule's `{given, suffix}` is a strict subset of the compound rule's `{family, given, suffix, title}`, so every diff the peel rule admits the compound rule admits too. Implemented to check it: narrowing the peel regex to a `GLUED_HONORIFICS` alternation keeps all seven of the co-matched names whose 1.4.0 diff is exactly `{given, suffix}`, and swapping the two rules in the NARROWED ledger still reattributes all seven. So the pair keeps the arrangement the arc settled on, and that is the answer rather than a cost trade: it is narrow-first, the declaration-free default, so it appears in none of the eleven contests `order_contests` reports and owes no `precedes_narrower` block, and all seven names are pinned by name in `_CROSS_RULE_WINNERS` (23 corpus names reach both regexes; the seven are the ones #375's reorder mutation moves). The cost stands as a second reason and not the first: the narrowing would hand-copy more honorific vocabulary into the pattern and grow the `_HONORIFIC_SOURCES` sync-roster surface (mechanisms.md#CURATED-VOCABULARY-ALTERNATION's second half). Option 2's convention plus a check on the wide-first exceptions to it is what the arc took, and option 3 it REFUSED on a false premise (the finding above). One thing this does not settle, and the ledger comment on the peel rule overstates it: `fields` separates the two only for the UNION rows whose diff includes `family`; for those seven it is file order that decides. +- Scoping the check to contract-tier contests only (2026-09-02) — five of the eleven are contested only over radar names, so this would have cut the file's exemptions by nearly half. Declined because `_CROSS_RULE_WINNERS` already pins radar names — measured, MOST of the names it pins are radar-tier — so the repo would be inconsistent with itself about whether a radar contest matters. The majority is the whole of the argument and no digit belongs here: a count written at this spot was stale before the branch that wrote it merged, since the 2026-09-03 #497 entry below then deleted four of the roster's rows. RECOMPUTE: union the names in `_CROSS_RULE_WINNERS` (tests/v2/test_ledger_guards.py) and read each one's tier off `compare._CORPUS_TIERS`, via the first `corpus*.jsonl` file holding the name with contract files sorted first, as `main()` loads them. Whether the radar-only CJK comma rules still earn their place after #488's demotion is a real question and is filed as #495; it is a question about those rules, not about the check. +- A shared name-population function for both callers (2026-09-02, the spec's own first sketch) — see the two-populations decision above. `main()` and the unit guard must read different populations, so one function would break `--corpus`; they agree by a docstring that names the function it mirrors instead. +- Deleting the rules whose explained names are all radar (2026-09-02, [#495](https://github.com/derek73/python-nameparser/issues/495), closed as not planned) — filed off #488's demotion on the reasoning that a rule nothing can demand is a rule to remove, and refused on what deleting one actually does. Measured at baseline 1.4.0, 19 of the 72 explaining rules explain only radar names, between them 48 of the 352 intentional diffs — and that set is a SUPERSET of what the issue proposed deleting, not a different set. Three of the five rules #495 tabulated are in it — `fix(#296) a credential-only comma string reads a name and its postnominal`, `fix(cjk-comma-compound)`, and the roman-numeral `fix(suffix-routing)` — and they are exactly the three its option 2 named as having zero contract reach. The other two are outside it for the reason that disqualified them as candidates there too: they explain contract names, fourteen of them for `fix(cjk-glued-honorific-peel)` and `'Smith Jr.'` (corpus_rules.jsonl) for the jr `fix(suffix-routing)`. So the criterion reaches everything the issue wanted gone and more, and what refuses it is the fate of those 48 diffs. Measured by driving `classify()` over each one with its own rule removed: 25 report UNCLASSIFIED and 23 are claimed by ANOTHER rule (26 and 22 with all 19 deleted at once). For nearly half the population, deleting does not leave the diff watched-but-unexplained — it leaves it explained, by a rule whose comment is about something else. Nine of `fix(cjk-comma-compound)`'s eleven land on `fix(cjk-glued-honorific-peel)`; all seven of `fix(cjk-comma-honorific-peel)`'s land on `fix(cjk-comma-compound)`; `'Carod i'` lands on the roman-numeral `fix(suffix-routing)`, and `'Bob Jones, author'` on `fix(comma-precomma-family)`. That is the rule-broader-than-the-diff absorption this file already names twice — `fix(#424)` at decisions/H1, going on classifying a diff that shrank underneath it while the fields-only catch-all's heading stepped 14 -> 17 with nothing announcing either, and the fields-only arc's "a fields-only rule silently falsifies OTHER comments" — reached this time on purpose rather than by accident, which is not an improvement. Applied, it deletes `fix(#342)` and `fix(#397)`, the two NOT WANTED rules whose whole job is classifying a reading nobody wants so the gate stays green while the bug is open, and which carry the delete-when-fixed instruction the dormancy check enforces once the fix lands (the fields-only arc above). The cost there is NOT a red gate, and a first draft of this bullet said it was: both names sit in corpus_issues.jsonl, which is radar, and on a radar corpus an unmatched diff prints UNCLASSIFIED (radar) while the run keeps exiting 0 — UNEXPLAINED is the contract tier's word (tools/differential/README.md). Measured with both rules gone, the run reports `unexplained: 0; radar unclassified: 1` and exits 0. The one is `'Aishwarya Rai'`; `'Carod i'` is not reported at all, the numeral rule having taken it — so one of the two open bugs loses its marker not to silence but to a wrong label. Verified end to end against a ledger with `fix(#397)` removed: exit 0, 352 intentional diffs still, no UNCLASSIFIED, no EXPLAINED NOTHING, and `'Carod i'` printed under `fix(suffix-routing) a two-token name ending in a roman numeral`. It also deletes `feat(#273)`, `feat(#269) Arabic بن prefix chains onto family (non-Latin new-recognition)` and `fix(#360) ste moved into the never-given particles with mc` — records of new recognition behavior and of a vocabulary move — and the last two have to be named to the RULE rather than to the issue, as the ledger holds sibling rules under both strings. The `feat(#269)` sibling is the sharp one: `feat(#269) non-Latin titles/conjunctions recognized` does not qualify, explaining the contract-tier `'Хосе И Мария Сантос'` (corpus_rules.jsonl), so a delete by issue prefix takes both and sends that name UNEXPLAINED — measured, and a red gate. The criterion that does justify deleting a rule is that it explains NOTHING, which `dormant` already names and the dormancy check already enforces; a rule explaining eleven radar names is not dormant. What the radar tier is FOR answers the rest: decisions/cjk-comma-demotion puts it as giving the differential "somewhere to WATCH a name without promising it", and a rule explaining a radar name IS the watching. The five exemptions citing #495 are rewritten in the same diff rather than left standing: four of them put it as a LIVE question — the two under `fix(#296) a credential-only comma string`, the one on the `fix(cjk-comma-compound)` block, and the one on the `'田中さん II'` numeral contest — and the fifth, on the jr contest, called neither rule a candidate in the present tense. Each records the answer now and keeps every measurement it carried. One of those measurements was wrong and is corrected with them: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` explains eight names of which FIVE are contract, not six — the six having shipped in #499. It is not a mis-map: six is what tiering by FILE MEMBERSHIP gives from a CORRECT `_CORPUS_TIERS`, and the single name it gets wrong is `'John Smith, Dr.'`, the third RECOMPUTE trap below. #496's decline was argued inside the `fix(cjk-comma-compound)` one and stands without it: the twin rule was refused because nothing can demand it, which is untouched by what #495 answered. RECOMPUTE: do not join names to files by hand. `_CORPUS_TIERS` maps FILES, the report prints names with no file, and 24 of the 352 explained names sit in files of BOTH tiers, so a hand join has to reproduce `main()`'s contract-first dedup AND its baseline-minimum skip to get them right. Let the run do it: call `compare.main()` unchanged under two wrappers — one around `_run_worker`, whose `entries` argument is the final list with `tier` and `file` already stamped on every entry, and one around `classify()`, recording (name, order, diff, issue) for each of its 352 calls — then reclassify offline from those records against `rules` minus whichever rule is under test, having first confirmed the wrapped run reproduced the shipped one (352 intentional, 0 unexplained, 0 radar unclassified). Parse no printed repr by stripping quotes: three of the 352 names do not survive it — `"'Smitty' Dr. Jones"`, `"'Smitty' Jones Jr."`, and the spelling of محمد بن سلمان wrapped in RIGHT-TO-LEFT MARKs — and dropping them silently is what produced the wrong count of 20 on the first pass, the first of the three being `fix(#410)`'s only contract name. `ast.literal_eval` reads the printout correctly. The ten-name truncation is NOT the hazard it looks: contract files load first, so any contract name a rule explains sits in the printed prefix, and tiering from the first ten alone gives the same 19. THIRD, and the only one of the three that changes an answer on this tree: a name's tier, in any claim scoped to a BASELINE, is a property of the ENTRY that run compared and not of the files the string appears in. The two part where a string sits in files of both tiers AND its contract copy carries a shape tag, because the dedup key is `(name, order)` — a shape declaring an order leaves TWO entries rather than one contract winner, and the baseline-minimum skip can then drop the contract one. Measured, 2 of those 24 names carry a shape tag and exactly 1 bites at 1.4.0: `'John Smith, Dr.'` is in corpus_issues.jsonl (radar, no shape) and corpus_shapes.jsonl (contract, shape 4, FAMILY_FIRST, min_baseline 2.0.0); both entries survive the dedup, which the 2.0.0 run shows by comparing both, and at 1.4.0 the shape-4 one is among the seven skipped, leaving the radar entry as the only comparison. `'Beethoven, Ludwig van'`'s shape 2 is order-None with a 1.4.0 minimum, so the dedup collapses it and contract wins at every baseline. File-membership tiering gets exactly 1 of the 352 wrong, and that one is what `fix(comma-family)`'s six-versus-five turned on; `entry["tier"]` gets it right for free. The radar-only count of 19 is the same under either tiering. To exercise a ledger edit, point `_allowlist_for` at a scratch file rather than copying the tree — the corpus glob resolves from `HERE` and `_check_tree` refuses a nameparser outside `REPO_ROOT`, so a copy elsewhere cannot run at all. + +## differential-ledger, the recorded-shapes arc (2026-09-03, #497) + +The seventh ledger arc, and the one that makes a recorded INPUT measurable. The arc above closed on the finding that `test_the_recorded_rule_still_wins_each_contested_name` feeds `classify()` a diff shape nothing ever checks, so a shape that is wrong but still routes to the same rule passes forever; this is what was done about it, and about the second thing that search turned up — that the cost the same arc declined a check on had never been measured at all. Mechanics — where the shapes live, which name list each half of the check reads, what a `MOVED SHAPE` block may and may not claim — are owned by `tools/differential/compare.py` (`_RECORDED_DIFFS`' provenance note, `recorded_diff_mismatches`' docstring, and the two placement comments in `main()`), by `tests/v2/test_ledger_guards.py` (`_CROSS_RULE_WINNERS`' header and the note over its 2.x sections) and by tools/differential/README.md; these are the decisions. Every figure below was measured or recomputed 2026-09-03 on this branch; each one that can drift carries its recipe, and the two that cannot be re-derived from the tree at all — the sweep scan's own hit counts — say so where they appear. + +Decisions that landed: + +- 2026-09-03 [#497](https://github.com/derek73/python-nameparser/issues/497) — the recorded shapes live in the TOOL, as `compare._RECORDED_DIFFS`, and the RUN verifies them; `_CROSS_RULE_WINNERS` keeps only which rule wins a contested name and reads the shape from there. There because it is the one place both halves exist: `main()` has measured every name's real diff by the time it asks, where `validate_rules` runs before any corpus is read and the unit suite installs no wheel at all — `tests/v2/test_differential.py`'s own header says "nothing here spawns `uv` or the network", and the tests that need a baseline fake `_run_worker` or fake `Popen` under it. State it that way and not as "every test monkeypatches `_run_worker`", which is false: measured 2026-09-03 on this branch, of 171 `def test_*` functions in that module 20 mention `_run_worker` at all and six reach the real one with `Popen` faked beneath them by `_fake_popen`. RECOMPUTE by walking the module's `def test_*` bodies for the name. That short form is what compare.py's `_RECORDED_DIFFS` provenance note SAID when this entry was written; `f70a0ad`, the very next commit, withdrew it there, and the note now rejects the paraphrase rather than carrying it — so follow that pointer to the history and not to the text. The denominator has moved once already inside this arc: `f70a0ad`'s own message says 168, which is why the argument here is "most never mention it" and the digits carry a date. The hermeticity claim is unharmed and the loose paraphrase of it is what fails, which is this arc's subject arriving inside its own evidence. Read HERMETICITY as the reason and not speed. The rival placement is the unit suite, and the reason it is refused is that a check there would have to install a wheel — not that installing one is slow. The timing finding below retires every cost argument in this file and moves this decision not at all, which is the test such a placement argument has to pass and the reason to say out loud which ground carries it. `_CORPUS_FLOORS` is the precedent — tool-owned, corpus-shaped data that the guards read and monkeypatch — and this is the same direction. +- 2026-09-03 #497 — ONE roster, TWO checks, TWO PLACEMENTS — and, since `9360919`, a NOTE over the gap between them, recorded at the end of this entry. (Superseded in its first word 2026-09-05: TWO rosters since the watched-shapes arc below, `_WATCHED_DIFFS` beside `_RECORDED_DIFFS`, both placements unchanged; the departed-name refusal and the `NOT CHECKED` note read the union of the two, while the shape comparison runs once per roster, because the two halves carry different severities — a contest row's mismatch feeds the exit code on either tier, a watched row's only where its default-order entry is contract tier — and one call could not tell them apart.) The asymmetry of the two checks is the decision rather than where the code happened to fit. The ABSENT-NAME half — a recorded row naming a string no corpus holds any more, which nothing else would ever report — refuses PRE-worker, beside `vacant`, because it reads the ledger and the loaded names and nothing the worker produces. It takes the PLACEMENT #382 chose for its own pre-worker refusal and NOT #382's reason for it, which was the cost claim below — `compare.py`'s comment at that placement is one of the sites the timing finding retires. The ground it stands on was rewritten on this branch: a refusal raised after the wheel is installed and the corpus compared prints below the run's own published `baseline:` header, for a comparison it will never report. Not "disowning a comparison it has just published" — measured print order is `baseline:` at the tell, then the comparison loop, then `corpus: … intentional diffs:`, so at that position the header has printed and no line of the comparison has. The ORDER is the argument, and that corrected wording is what `compare.py` now carries at all three of its placement comments. Recorded because it is the second decision the false number was under, and the two came apart differently — the declined check below kept its conclusion on a new argument, and here the ORDER was right all along while the reason for it was not. Only re-deriving a claim says which of its supports was doing the work; a placement that reads unchanged is not thereby unaffected. The MOVED-SHAPE half is the opposite case — it needs the diffs, so it stays after the comparison — and there it may not RAISE. It PRINTS and feeds the exit code, like `over_declared_rules`, its structural sibling and the other post-worker check on recorded data: a raise at that point lands MID-report and takes dormancy, OVER-DECLARED, UNEXPLAINED and the radar block with it, which compare.py measures on a narrowed 1.4.0 run beside the check. A stale roster row must never hide an unexplained diff, that being the gate's primary output. The two pre-worker refusals may raise because nothing has printed yet; this one cannot, and the difference is PRINT ORDER rather than severity. The two halves also read opposite name lists, which is `recorded_diff_mismatches`' docstring's to state and is not restated here. What IS decided here is the THIRD thing this roster produces, added by `9360919` once the gap between those two lists was measured and found not empty: an order-bearing entry the baseline cannot honor sits in a corpus file and outside the compared set at once, so a row on such a name is checked by neither half and reported by neither. `main()` prints a `NOT CHECKED` note over it. A NOTE and not a refusal — the docstring's argument against refusing off the post-skip list stands, and it is an argument against REFUSING and not against SAYING, which are two decisions of which only the first was ever made. Informational, outside the exit code, and gated on neither `full_corpus` nor `args.corpus`, the intersection narrowing itself. Measured 2026-09-03 at 1.4.0 — both the only baseline where the skip fires and the only ledger with rows — the window is three names, none of which carries a roster row today, so the note prints on no run yet. The measurement that motivated it put a deliberately wrong shape on `'de la Cruz née Vega'` over the full corpus at 1.4.0: before `9360919` that exited 0 in 375 stdout lines naming the name in none of them, and it now exits 0 in 378 with the `NOT CHECKED` note naming it — re-measured 2026-09-03 by corrupting the row in memory around `main()`, which leaves the worktree alone. +- 2026-09-03 #497 — a SHAPE may be pinned where a COUNT may not, and that line is the whole of what keeps this roster affordable. A diff shape is one name under two parser versions, so adding a name to a corpus cannot move it; an explained-name count moves on every corpus edit. Shapes drift only when the PARSER changes those names — so the drift and the reason to re-read the roster are the same event, which is exactly what a recorded number usually fails to be. That is why #452's churn objection to `explains = N` (the fields-only arc above, 2026-08-28) does not reach this roster, and why pinning shapes here does not weaken it: the four ledgers carry 201 rules today, one `_CORPUS_CLAIMS` entry each, every one liable to move whenever a corpus does, against 31 recorded shapes that move only with the parser (51 since #498, 45 of them at 1.4.0). Read the two CAUSES and not the two totals — the argument is that one set is coupled to corpus edits and the other is not, and it survives either number moving. RECOMPUTE: `sum(len(v) for v in _CORPUS_CLAIMS.values())` in tests/v2/test_ledger_guards.py, and `sum(len(v) for v in _RECORDED_DIFFS.values())` in compare.py. +- 2026-09-03 #497 — four of the 35 rows the roster carried were DELETED rather than corrected. Two names across the two 2.x ledgers, so two rows each, and one of the two NAMES had a correctable shape — count rows or names deliberately, since the provenance note that holds the measurements speaks of the name. Deleted because neither name pinned a CONTEST, which is what the roster is for: at the shape each really produces exactly one rule admits it, so `classify()` was never adjudicating anything. `'Nguyen, Van'` diffs at no baseline at all, so no run ever asks `classify()` about it; `'Jane née and Jones Smith'` is a malformed harvest from a radar corpus whose right parse nobody can state, so a pin on it defends no boundary anyone would argue for — which is why the correctable one went too. compare.py's provenance note holds the per-name, per-baseline measurements behind both. The deletion emptied both 2.x sections, which stay present as explicit empty mappings — the completeness guards require it, and it is what makes emptiness sayable as a POSITION: a row is owed when someone argues a boundary, not before. What the emptiness is NOT — that these ledgers hold no contested diff — is measured and stated beside those sections, and whether the position should change now that those contests are measured rather than merely unexamined is [#501](https://github.com/derek73/python-nameparser/issues/501), open. (Superseded in its emptiness 2026-09-05: the two 2.x sections are no longer empty, the #501 adjudication below having pinned FIVE rows at 2.0.0 and one at 2.1.0, so `_CROSS_RULE_WINNERS` read 31 / 5 / 1 / 0 at that point — measured 2026-09-05 — and #501 is closed. It reads 45 / 5 / 1 / 0 since #498's fourteen pins, landed the same day; only the 1.4.0 figure moved. RECOMPUTE for either: `{k: len(v) for k, v in _CROSS_RULE_WINNERS.items()}` in tests/v2/test_ledger_guards.py. The criterion is not superseded and is what filled them: a row is owed when someone argues a boundary, and three of the six were found already argued, in the 2.0.0 ledger's own comment. Read the sentences above as the position emptiness stood on for as long as it stood.) Recorded with the deletion because it is the half a later sweep will get wrong: "only one rule admits it" is NOT on its own grounds to delete a row. A substantial minority of the surviving 1.4.0 rows are in that position and stay — the count and its recompute are beside them in test_ledger_guards.py, where six of the thirteen say so in their own comments and the other seven do not, which is why the count is recomputed rather than read off the roster — because the shapes they pin are shapes runs actually make, so a widened `fields` or a moved rule hands the name over and the test says so. Counting admitters is not the test at all: 11 of those 13 route to a DIFFERENT rule under some other shape, and mechanisms.md#RECORDED-ROSTERS now carries that measurement in place of the criterion it stated first. The four deleted rows could not do that work at any edit, which is the difference. +- 2026-09-03 #497 — the prose sweep is sorted by this file's own criterion (#451, "past-tense counts stay in a ledger comment; live ones do not") into three treatments, and the treatment follows from what the count is DOING rather than from how wrong it is. A count that IS the finding and reads past tense stays. A count standing beside the enumeration it counts loses the digit — the list is self-checking and the digit is the only part that can rot. A live exhaustiveness claim with nothing recomputing it is REGROUNDED on the property it was reaching for, so that the sentence stops depending on a number — saying a widening claims all but a handful of the whole population the assertion is over says what a fraction said and cannot go stale. One judgement goes with it and is the arc's own: a figure that could NOT be reproduced was not replaced with a fresh one. `_reaches_non_vocabulary`'s docstring now carries the shape of its finding, the instruction for measuring it, and no LIVE figure for the magnitude the old figures asserted — a replacement digit nobody can re-derive is the defect rather than the repair, and this file has already been burned by that shape (the #414 roster comment under the fields-only arc's `explains = N` bullet). The past-tense digits explaining why the live ones are gone stay, which is the first treatment applied inside the third, and a later sweep should not read them as survivors of the pass. + +Found rather than decided, and worth as much: + +- **The cost that decided a decline had never been measured, and the decline survived losing it.** "The multi-minute worker pass" entered 2026-08-05 in `7767ba2`, at ONE site, with no measurement beside it. The census of where it reached, the timing figures that replace it, their recompute and the `_run_worker` timing trap all live ONCE, in the rule-order arc's `Declined:` bullet on precise per-name contest detection above — the argument they falsify — and none of them is restated here; that census is the figure in this arc most likely to move, so pointing at its owner rather than copying it is the #451 rule applied to the very claim that demonstrates why. Two notes on reading it that belong here rather than there. Not every site spells the phrase — one says the pass "costs minutes" — so count the CLAIM and not the wording (`git grep -n -i minute 514a2a1`, this branch's base). And measured, the worker pass is a fraction of a second and a whole gate run is seconds at most, where the phrase said minutes: read that class and not the digits. Three things transfer past them, and a coda about removing it. The claim was never a measurement anyone got wrong — it was a plausible sentence about a subprocess that installs a wheel, and it propagated by COPYING, the mechanism the fields-only arc recorded for ledger comments and which reaches code comments identically. It was load-bearing and its conclusion still stands, rewritten on COVERAGE: a run-time check sees one ledger, for the baseline that run was invoked with, and only when somebody invokes the tool, where a static predicate covers a rule from the moment it is written. And that is how a false number stays invisible for a month — an argument with two supports loses one and nothing downstream of it changes, so no reader ever meets a contradiction. Removing it was itself hard to do cleanly: review of the commit that removed the phrase found four instances of the very defect that commit exists to remove, fixed in `40c82a7`. +- **A row pinning a race with one runner is never exercised as a contest, so its shape only ever had to agree with itself.** That is why nothing caught the four, and it is the general shape rather than a fact about two names. The four arrived with #452's work (`ee6aaa6`, PR #455) under two claims of measurement — the roster header's "The diff shapes are measured against the 1.4.0 wheel, not guessed" and the 2.1.0 block's own "measured against this baseline's own run rather than copied from the 2.0.0 rows" — and every one of the 31 rows at 1.4.0 agreed with the run on 2026-09-03, at all four baselines; the 45 rows there since #498 all agree on 2026-09-05, the gate exiting 0 with no `MOVED SHAPE` at any baseline. Do NOT read that as the 1.4.0 half having been sound: one of those 31 was guessed too, `田中さん II`, wrong on a genuinely CONTESTED row, found by hand a day before this branch opened (`e5e4955`, the finding the arc above filed as #497) and already corrected when the 31 were re-measured. So the one-runner mechanism explains why THESE four survived a check that would have caught a contested row, not why every wrong shape survived — the contested one needed somebody to sit down with the wheel, which is the cost the run now removes. The transferable test is what to ask of a recorded INPUT: not "is it checked" but "would its CONSUMER answer differently if it were wrong". mechanisms.md#RECORDED-ROSTERS forbids re-deriving the EXPECTATION from the inputs the check reads; this roster obeyed that to the letter and still went wrong, on the other side of the assertion, which is why that entry now carries the limit. +- **The scan that scoped the sweep was wrong three ways, and it is the scan rather than the three fixed lines that the next sweep will reuse.** Two of its 19 machine-readable hits were false positives — a version number, and a hypothesis rather than a measurement — so the real population was 17. Its file list omitted `expected_since_2.2.0.toml`, which held a further copy of a claim it was already treating in two other ledgers, so the sweep would have shipped two of three siblings corrected. And its predicate matched only lines BEGINNING with `#`, which made every docstring invisible to it: the stalest figures in the whole sweep survived the first pass for that reason alone, and a docstring pass over the two guard modules then found two more. Read the sweep as BOUNDED and not exhaustive — that docstring pass covered `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`, and compare.py's own docstrings were never swept. And a fourth thing, which is the finding about the finding: the scan's PREDICATE was not preserved. Its 19 hits and their 17 survivors are recorded in `8593f3a`'s commit message and cannot be re-derived from the tree, because nothing says what it matched beyond the leading `#`. That is the reusable artifact of a sweep — not the lines it fixed — so the next one has to write its own and has no earlier hit count to check itself against. Write the predicate down where the results go. +- **What the stale claims were, because the classes recur and the digits do not.** (i) A count of a population that had grown by half underneath it: a guard comment gave a Latin-half widening as claiming "644 of the 654 unclassified corpus names", where the same expression claims 994 of 1008 today (`len(_UNCLASSIFIED_NAMES)`, intersected with `_claimed("[A-Za-z]")`, in tests/v2/test_ledger_guards.py) — and since the argument was that the widening reaches essentially the whole population the assertion is over, it is written that way now and needs no recompute. (ii) A count of FILES: "all five corpora", in three places, where six `corpus*.jsonl` sit on disk and a run globs every one of them — the corpus count was never the argument, and the gate's own `corpora:` line is the check. (iii) A live count inside the very paragraph that names stale counts as the file's standing hazard — `fix(#274)`'s ledger comment walked "all 31 the regex claims" where that regex claims 32 today (`len(_claimed(rule["name_regex"]))`). (iv) A count over a self-checking enumeration: "Six such widenings" above a list of four, where nobody can now recover which two are missing and the list was always the evidence. The fifth is the one worth the space, and the docstring pass found it: `_carries` said the `isascii()` split covers "12 of the 16 entries", where at sweep time `MAIDEN_MARKERS` shipped 17 with 12 of them non-ASCII (measured 2026-09-03; RECOMPUTE with `len(MAIDEN_MARKERS)` and `sum(not m.isascii() for m in MAIDEN_MARKERS)` from `nameparser.config.maiden_markers`) — the NUMERATOR was right and stayed right while the denominator rotted. The repair names the predicate rather than counting its output, which is why the guard's own note past-tenses both figures and points the reader at the `isascii()` split itself. A count can go stale in one of its two halves, which is the shape a re-reader is least likely to catch, and it is a reason to prefer naming the predicate over counting its output. +- **The tier split inverts the ranking that #452's still-open half rests on.** Beside the fields-equals-union check that landed, #452 carries a second proposal: a ceiling on how much of the corpus one rule may REACH, measured there as a share of the whole corpus. #468's tiering makes that the wrong denominator, because a radar diff cannot fail the run — a rule reaching many radar names is not the hazard a rule reaching many contract names is. Measured over the corpus population and tier split the rule-order arc's RECOMPUTE above already owns and this bullet does not restate, the two comma rules reach 288 names, 25.8% of all of them, but 64 contract names, 19.6% of that tier; `fix(#271/#272/#298)` reaches 108, 9.7%, and 80 contract, 24.5%. So by total reach the comma rules are the outlier and by contract reach the CJK rule is, and a ceiling on total reach would rank the wrong rule as the biggest hazard. Read the INVERSION rather than the four percentages: it is a claim about which tier the denominator counts, and it survives any drift that does not reverse the two tiers' composition. Recorded here because the proposal is open and the measurements in it predate the tiering. RECOMPUTE: tier every corpus name off `compare._CORPUS_TIERS`, contract winning where a string sits in files of both tiers, then take each rule's `name_regex` reach split by tier. + +Declined: + +- `explains = N` per rule as ledger data (2026-09-03, again) — #452's own first sketch, declined there on churn, re-examined here only because this arc pins a different recorded value and owed an account of why the two differ, and declined again on the same grounds. The third decision above is the argument in full: a shape cannot move when a corpus does and a count can. The landed fields-equals-union check already catches the drift that motivated the sketch — a rule's explained set changing SHAPE underneath it — and needs no number to do it. The question belongs to [#452](https://github.com/derek73/python-nameparser/issues/452), which is OPEN and is exactly it; nothing here forecloses it, and its remaining reach-ceiling half should be re-measured on the tier finding above. +- The differential in CI (2026-09-03) — considered and dropped rather than deferred. It is a release gate that already runs locally on demand, and no sibling check — dormancy, over-declaration, the undeclared-contest refusal, this one — needed a CI job to be worth having. Worth recording as a DECLINE and not an omission precisely because the timing finding removes the only reason nobody proposed it: a job costing seconds is cheap, and cheapness was never the criterion. +- A general sweep of every number-carrying comment line (2026-09-03) — the sweep is scoped to the claim-shaped ones, which the scan put at 19 and which were 17 — the pair of figures that cannot be re-derived from the tree, since the scan's predicate was never preserved; they live in `8593f3a`'s commit message and the bullet above says so. The population it declines is larger by nearly two orders of magnitude: measured 2026-09-03, comment lines carrying a digit run to four figures across the differential tooling, its four ledgers and the two guard modules, against 17 claim-shaped ones (RECOMPUTE: count lines matching `^\s*#` that also contain a digit, over `tools/differential/*.py`, `tools/differential/*.toml`, `tests/v2/test_ledger_guards.py` and `tests/v2/test_differential.py`). The exact figure is deliberately not given: an earlier draft of this bullet carried one, and the NEXT commit on this branch added two comment lines and falsified it — inside the entry declining a sweep for exactly that. Most are issue references, version numbers and codepoints, none of which rot. Read the RATIO and not either digit: a hand sweep two orders of magnitude past the claims is how wrong claims get INTRODUCED rather than removed, and this arc demonstrated that at the smaller scale — the commit that removed the false cost claim needed a follow-up for four instances of the same defect inside itself, and the first prose sweep shipped having skipped three instances it had already identified (`8593f3a`). + +## differential-ledger, the watched-shapes arc (2026-09-05, #501) + +The eighth ledger arc, and the one that lets a diff shape be recorded WITHOUT a winner. The arc above moved the recorded shapes to where a run verifies them and, in the same commit (`962ae43`), bolted them to the winners with `set(winners) == set(shapes)` — never argued as a constraint, and the reason 51 corpus names whose only watcher was a classification rule could not be watched at the shape level: recording a shape meant pinning a winner, and a winner pin asserts an argument nobody had made. Mechanics — the two dicts and their contracts, the three print blocks, which name list each check reads — are owned by `tools/differential/compare.py` (`_WATCHED_DIFFS`' header, which carries the population and its recompute) and tools/differential/README.md's `MOVED SHAPE` section; these are the decisions. Every figure below was measured 2026-09-05 on this branch from real `main()` runs with `classify()` spied, and each carries its recipe; the one figure that goes stale by design, the quiet-tree count, is dated where it appears. Refs [#501](https://github.com/derek73/python-nameparser/issues/501), which stays open. + +Decisions that landed: + +- 2026-09-05 #501 — a shape MAY be recorded without a winner, in a SECOND roster (`compare._WATCHED_DIFFS`, keyed per ledger like `_RECORDED_DIFFS`, disjoint from it per ledger, both keyed by every ledger on disk — held by the guards at pytest speed and, since the PR's review round, by `main()` pre-worker, which refuses a ledger either dict has no section for) and NOT by relaxing the equality. Two grounds, and the first is the arc above's own placement decision turned around: the run cannot see `_CROSS_RULE_WINNERS` — `compare.py` cannot import the test module, and names that roster only in comments and in the advice it prints — so if the KIND of a row is to decide anything at run time, the kind must be DATA in the tool, and one dict with a relaxed guard leaves the run unable to tell a contest row from a standalone one, which means one repair text and one severity for both. The second is that the equality was doing work the relaxation gives up: a shape whose winner was deleted and forgotten is a deleted-winner slip, and under `<=` it is indistinguishable from a deliberate standalone row. So `set(winners) == set(shapes)` stays, both directions, and its docstring now says why the reverse direction is defended — a shape with no winner is not an orphan to tolerate but a row in the wrong dict. A name in both dicts is refused twice, by the guard at pytest speed and by `main()` pre-worker beside the departed-name refusal, because the tool may not assume the suite ran; the argument is the stronger claim, so such a name belongs in `_RECORDED_DIFFS` alone. The day a winner is argued for a watched name, the row MOVES and the pin goes beside it there; it never gains a partner where it is. +- 2026-09-05 #501 — SEVERITY follows the row's kind first and the tier second: a row with an argument behind it (a pinned winner) is fatal on both tiers; a row that is only a snapshot follows its tier, fatal on a contract name and printed under a non-fatal `MOVED SHAPE (radar)` block, parallel to `UNCLASSIFIED (radar)`, on a radar one. Three grounds, from the spec's review amendment A. A per-name shape is a STRONGER promise than a contract name gets — a contract name needs a RULE, which covers a shape class and batches across names, so a feature that legitimately moves thirty of these would produce thirty findings each needing its own "say why", the per-name burden #468 removed, reintroduced one level down on the very file #468 demoted for that reason (`corpus_issues.jsonl`, 49 of the 51; 48 of the 50 since the #501 adjudication below, and the burden argument is unmoved by either count). The `[[never]]` precedent cuts the other way — exclusions are fatal on both tiers because each is a per-name deliberate choice carrying a `why`, and these rows are measured, not reasoned: a sweep, not a choice. And when a snapshot row fires the only repair is to re-snapshot — a contest row has an argument to consult; `'Ph. D., Jr.'` or `'QC MP'` has no adjudicated parse, so "correct the row" means "record whatever it does now", which is mechanisms.md#RECORDED-ROSTERS's re-derivation hazard in a gate's clothing. Two limits on what this decides, stated so nobody reads more into it. It answers the NARROW tier question only — whether a snapshot on a radar name may fail the gate, which `shape_bad` being tier-blind had answered yes by default — and leaves the gate-and-changelog hypothesis where the spec left it: the framing that the ledger is a gate AND a changelog and `_CORPUS_TIERS` splits only the gate is a hypothesis, not codified, and the radar tier's explanation-relief has read 0 at every baseline because no feature work has offered it a case, not because it was declined. And it does not reverse #468; it applies it — "shown, never blocking" reaching a per-name snapshot on a radar name exactly as it reaches an unmatched diff there. It also turns the arc above's radar overlap from unreconciled history into a stated rule: measured 2026-09-03, 21 of the 31 contest rows at 1.4.0 sat on radar-tier names, every one carrying a winner, and they stay fatal because it is the argument and not the tier a contest row defends (RECOMPUTE in tools/differential/README.md's tier paragraph). +- 2026-09-05 #501 (found in the branch's code review) — the tier a watched row's severity follows is the DEFAULT-ORDER ENTRY's, not the name's. A recorded shape is read off the default-order comparison, so that comparison's entry is the one whose tier says whether the shape was promised; the first-loaded (contract-preferring) entry decides only for a name with no default-order entry, which is the name compared under a declared order alone. This is the rule-order arc's third RECOMPUTE trap applied to severity — "a name's tier, in any claim scoped to a BASELINE, is a property of the ENTRY that run compared and not of the files the string appears in", under its #495 decline above, whose measurement is not restated here. The live case is that arc's own: `'John Smith, Dr.'` is contract in `corpus_shapes.jsonl` only as shape 4 (`FAMILY_FIRST`) and radar in `corpus_issues.jsonl` under the default order, so a watched row on it would print and not fail, and the family-first promise would be untouched — it carries no row today, a test naming it. The dict header first said "a name both tiers hold reads contract", which was false of the code as written (`tier_of` is built over the post-skip entries, an order-None entry winning where one exists), and was corrected before the commit was reviewed again. +- 2026-09-05 #501 (found in the design-docs review) — the roster's DEFINING property is "no winner pinned", not "nothing else watches". The four #501 rows are pinned by an exact `Case(...)` literal in `tests/v2/cases.py` and two sit in contract corpora (`'김민준 박사님'` and `'선생님'`, in `corpus_cjk.jsonl` and `corpus_rules.jsonl` alike; the two `様.` strings are in `corpus_cjk_tolerated.jsonl` only — measured 2026-09-05 on this tree and at `e004ccf`, where the four sit in exactly the same files), so "nothing else watches" is the property of the sole-watched SUBPOPULATION and false of the dict as a whole, and a run whose message told a reader of a `'선생님'` row that nothing else watched it would send that reader past the case row that already adjudicates the parse. What every row has, and what the two checks enforce (the disjointness guard, the `both` refusal), is that no winner is pinned. The dict header, the fatal block's lead, the `(radar)` block's lead and the README say "no winner is pinned"; the population comment says which rows are also sole-watched, and said until this branch that a contested row's reader consults the case row before the roster — a line the header no longer needs and no longer carries, the four rows having moved to `_RECORDED_DIFFS` with winners pinned beside them. (The four moved to `_RECORDED_DIFFS` with the #501 adjudication below, 2026-09-05, so every row in `_WATCHED_DIFFS` is sole-watched today; the decision is about the roster's DEFINING property, which is unchanged — no row there pins a winner.) +- 2026-09-05 #501 — every sole-watched diff is recorded — 109 sole-watched rows plus the four #501 contests, 113 in all — not the exposed subset. (AMENDED 2026-09-05, later the same day: the #501 adjudication below took six rows to `_RECORDED_DIFFS` — the four contests and `'MD, PHD'`'s two, which this count read as sole-watched and which were not — so the roster is 107 rows over 50 names and is now exactly the sole-watched population. The decision is about WHICH diffs get a row and survives the totals moving.) Airtightness is a property of today's rule widths and not of the name: a rule whose `fields` happens to equal a name's diff admits no alternative today, and widening that rule later exposes the name with nothing saying so. And the 35 alternatives over the sole-watched rows that route to a DIFFERENT rule (the finding below; 44 with the four contests counted) are silent whatever the winner's width is, which is a stronger reason than airtightness: "every sole-watched diffing name" survives a rule edit, "the exposed subset" is a snapshot of the widths. + +Found rather than decided, and worth as much: + +- **The spec's 385 / 71 was GROSS of #452 and measured on a population the arc had already retracted; the net figure over the roster is the one to argue from.** `over_declared_rules` requires a rule's `fields` to EQUAL the union of the diffs it explains, so on a rule explaining only one name any shrink fires OVER-DECLARED and is not silent at all. Measured 2026-09-05 over the 113 rows `_WATCHED_DIFFS` holds: 390 alternative shapes classify to the same rule and 72 diffs are exposed by that count; net of #452 — keeping only the subsets `S` for which the union of the rule's OTHER explained diffs at that baseline, joined with `S`, still equals the rule's `fields` — 337 alternatives over 64 exposed diffs, and 44 further alternatives route to a different rule. Per baseline the gross / net / other-rule split reads 105 / 96 / 23 at 1.4.0, 158 / 137 / 20 at 2.0.0, 127 / 104 / 1 at 2.1.0 and 0 / 0 / 0 at 2.2.0. Over the 109 sole-watched rows alone — the roster minus the four #501 contests, which is the population the label "sole-watched" names — 367 gross over 68 exposed diffs, 314 net over 60, 35 other-rule, the whole difference sitting at 2.0.0 where the four contested rows are. The spec's first draft measured 385 / 71 gross, 330 / 62 net and 35 other-rule, with 123 / 112 / 23 at 1.4.0, over the population the next bullet retracts: 52 names and 113 diffs, holding the four names with 1.4.0 contest rows (45 rows there, `'MD, PHD'` alone contributing 14 alternatives) and not the four #501 rows — the same total by coincidence, so those figures read as the roster's and were not. The 8 diffs that leave the exposed count under the net test — `'Esq. van Gogh'` at all three baselines where it diffs, `'Dr. Do Van Johnson, MD'` at both 2.x, `'Aishwarya Rai'`, `'Jack M.A.'`, `'Jane van der Berg 旧姓 Jones'` — are exactly the diffs that sit ALONE on their rule (the 72 → 64 drop; the spec's population had 9, `'Carod i'` the ninth, its 71 → 62), but they do NOT account for all the alternatives #452 catches: they carry 24 of the 53 caught over the roster (26 of 55 over the spec's population), and the other 29 come from diffs whose rule explains other names that do not between them cover its `fields`, so the diff stays exposed while some of its alternatives are caught. Two figures are robust across all three populations, and they are what the argument rests on: the worst case, `'Ph. D., Jr.'` at 2.0.0 and 2.1.0, 30 alternatives against a five-field rule, all 30 surviving #452; and the 35 other-rule alternatives over the sole-watched rows, equally unpinned and equally silent, a handover nobody would see — the metric UNDERCOUNTS in that direction whatever it counts in the other. Both corrections leave the conclusion where it was. AMENDED 2026-09-05, later the same day: the #501 adjudication below moved six of these rows out of `_WATCHED_DIFFS`, so the population every figure in this bullet is over went from 113 rows to 107 and the roster figures above are a snapshot of the tree this arc's own commits left. Re-run by the RECOMPUTE below over the 107: 363 gross alternatives over 66 exposed diffs, 310 net of #452 over 58, 35 other-rule, with 105 / 96 / 23 at 1.4.0, 133 / 112 / 11 at 2.0.0, 125 / 102 / 1 at 2.1.0 and 0 / 0 / 0 at 2.2.0, and 8 diffs alone on their rule. Both of the figures the argument actually rests on reproduce unchanged, which is the point of having named them: `'Ph. D., Jr.'` at 2.0.0 and at 2.1.0 is still 30 alternatives against a five-field rule with all 30 surviving #452, and the other-rule count is still 35 — the two rows that left carried no other-rule alternative between them. AMENDED again since #498, which moved five more 1.4.0 rows to `_RECORDED_DIFFS`: the roster holds 102 rows, 36 of them at 1.4.0, and re-running the RECOMPUTE below over those 102 (measured 2026-09-05) gives 337 gross alternatives over 61 exposed diffs, 284 net of #452 over 53, 35 other-rule, with 79 / 70 / 23 at 1.4.0 and 133 / 112 / 11, 125 / 102 / 1 and 0 / 0 / 0 unmoved at the three 2.x baselines, and 8 diffs alone on their rule — only the 1.4.0 line moves, since only 1.4.0 rows left. Both figures the argument rests on reproduce a third time: `'Ph. D., Jr.'` at 2.0.0 and at 2.1.0 is still 30 alternatives against a five-field rule with all 30 surviving #452, and the other-rule count is still 35. RECOMPUTE: spy on `compare.classify` through a real `main()` run at each baseline, keep the default-order calls on the names keyed in `_WATCHED_DIFFS` for that ledger (the roster's own keys, not a scan of `tests/`; drop the four #501 names for the sole-watched figures), and for each enumerate the non-empty subsets of the winning rule's `fields`, sorting each by whether `classify()` routes it to the winner, to another rule, or to nothing; the #452-aware count applies the union test above, and a diff is "alone on its rule" when no other `(name, order)` call at that baseline classified to the same rule. +- **A "where is this named" population must include the tool.** The `tests/` scan cannot see `compare.py`, so four names carrying 1.4.0 contest rows — `'Bob Jones, author'`, `'Carod i'`, `'MD, PHD'`, `'van ma van'` — scored as sole-watched there, watched by exactly the mechanism the arc extends. The definition gained its last clause (not already keyed in `_RECORDED_DIFFS` for that ledger); three of the four return at a 2.x baseline where they have no contest row, and `'Carod i'` diffs under the default order at 1.4.0 ONLY, so it leaves the population entirely: 51 names, not the 52 the `tests/`-only scan gave, and 41 rows at 1.4.0 where the spec's first table said 45. (AMENDED 2026-09-05, later the same day: `'MD, PHD'` leaves entirely too once the #501 adjudication below pins its 2.x pair, since it then carries a contest row at every baseline it diffs at — so TWO of the four return at a 2.x baseline and two leave, and the population is 50 names against the same scan's 52. Re-measured by the recipe: 41 / 31 / 30 / 5 = 107 rows over 50 names, the scan's own 45 / 32 / 31 / 5 = 113 over 52 unchanged, and the roster is exactly the 50-name set.) One more clause the recipe carried and the definition did not until the design-docs review: explained by a ledger rule at that baseline — a diff no rule explains is already printed by every run as unclassified, so a rule is the only weak watcher a row is needed for; a definition clause and not a live count, since `radar unclassified` reads 0 at every baseline today and the two sets coincide. RECOMPUTE: the population recipe is in `_WATCHED_DIFFS`' header; print the names its last clause removes per ledger, and the four appear at 1.4.0 and none elsewhere. (Stale twice over by the end of 2026-09-05, and re-measured that day by exactly that recipe: the clause removes NINE at 1.4.0 — the four plus `'Bob Jones, compositeur'`, `'MD, DO, DDS'`, `'Smith, Ph. D. Jr. MD'`, `'Smith, Ph. D. MD'` and `'Smith, Ph.D. Jr.'`, the five #498 pinned — and `'MD, PHD'` again at 2.0.0 and at 2.1.0 since #501 pinned its 2.x pair, none at 2.2.0. Rows per ledger after the clause: 36 / 31 / 30 / 5.) +- **`corpus_shapes.jsonl`'s table row counted ENTRIES where the population counts NAMES.** A `(name, order)` pair is an entry, so a name compared under three orders is three, while the population is names diffing under the default order. Measured 2026-09-05 over the four baselines: 14 distinct names diff there, 11 under the default order, and the other three — `'de Mesnil Jean, Dr.'`, `'de la Cruz Juan Carlos, Dr.'`, `'de la Cruz née Vega'` — diff only under a declared order, the same trio that is the `NOT CHECKED` window at 1.4.0. The spec's row read "16 entries"; by the recipe here — distinct `(name, order)` pairs on which `classify()` was called across the four baselines — the count is 18, and the basis for 16 was not recorded, so 16 is not carried. Names and default-order names are the figures the population argument rests on, and both reproduce. RECOMPUTE with the same spy, restricted to that file's names, counting pairs, names, and names with `order is None`. +- **The per-file split is not a partition.** 49 of the 51 sole-watched names sit in `corpus_issues.jsonl` and 3 in `corpus.jsonl`, with `'dr Vincent van Gogh dr'` in both, so the per-file counts overlap by one and 49 + 3 reads 52 for a 51-name population. (AMENDED 2026-09-05, later the same day: 48 and 3 over the 50 the #501 adjudication below leaves, the same name overlapping, so 48 + 3 reads 51 for a 50-name population — the shape of the finding, which is that the split is not a partition, is what survives the digits.) RECOMPUTE from the population script's JSON against `_load_entries` over the two radar files. +- **A missing roster section was read as an empty one (found 2026-09-05, in the PR's review round).** `main()` read both shape rosters with `.get(ledger.name, {})`, so a ledger with no section checked nothing and said nothing: with the 1.4.0 key deleted from `_WATCHED_DIFFS` in memory, a full run at that baseline checked its 41 rows against nothing, printed the same 375 lines (differing only in the worker environment's path on the `baseline:` line, which differs between any two runs) and exited 0 — the guards would have caught the deletion at pytest speed, but the run may not assume the suite ran, which is `_CORPUS_TIERS`' own ground. Now refused pre-worker, beside the `both` refusal, naming the dict and the ledger; `_CORPUS_TIERS` refuses a VALUE outside `contract`/`radar` at load on the same principle, since every downstream tier read is an `!=` against one literal and a misspelling would land on whichever side it happened to fall. RECOMPUTE on a tree before the refusal: `del compare._WATCHED_DIFFS['expected_since_1.4.0.toml']` around `main()` at `--baseline 1.4.0`, and diff stdout and the exit code against an unmodified run, with two unmodified runs diffed as the noise control. +- **The tier rule a watched row follows was stated three ways and pinned by none (found 2026-09-05, in the PR's review round).** Three mutants of `tier_of` — order-None entries only, first-loaded entry always, `== "contract"` in place of `!= "radar"` — each passed the whole suite and all four gates byte-identically, because every test drove one bare-string corpus, on which the three readings coincide. What pins the rule now is two fixtures `_run_main` cannot build: the `'John Smith, Dr.'` shape (one string, contract under shape 4 in one file and radar under the default order in another, watched row moved on the default-order comparison — prints `MOVED SHAPE (radar)`, exit 0, which refuses first-loaded-always) and a declared-order-only name (a shape-4 entry with no default-order entry, watched row reporting `measured` None — a tier map over order-None entries alone has no key for it and dies mid-report). RECOMPUTE by applying each mutant to a copy of `compare.py` outside the worktree and running `tests/v2/test_differential.py`; before the two tests, all three pass it. + +Declined: + +- Relaxing `set(winners) == set(shapes)` to `<=` (2026-09-05, the spec's first section) — the first decision above in full: the run cannot read the winner roster, so the kind of a row has to be data, and the equality catches the deleted-winner slip the relaxation would legalize. The spec's own review amendment B withdrew it. +- Fatal-on-radar for snapshot rows (2026-09-05, the spec's default by inheritance from `shape_bad`) — the second decision above; the spec's own "What this costs" section accepted 52 radar names becoming able to fail a release, and amendment A refused that cost as the wrong one for a row nobody chose. Had the 51 — as measured 2026-09-05 before the #501 adjudication below, 50 after — been made fatal after all, that would have been a reversal of #468 for those names and would sit here as one. +- Pinning winners for the six #501 contests (2026-09-05) — stays #501's. The four 2.0.0 rows here and `'MD, PHD'`'s two record SHAPES, not winners: a shape move is now a finding on them, while a handover to another rule is not — file order still decides the winner, and only a winner pin sees it. #501's own objection to pinning a boundary nobody has argued stands, and the issue is scoped down by that comment, not closed. (REVERSED the same day, 2026-09-05, by the #501 adjudication below, which argued all six and moved the rows — read this bullet as what THIS ARC declined to do in its own commits and not as a standing position. What it got right is kept there: the rows recorded shapes and not winners for as long as nobody had argued the boundaries. What it got wrong is that three of the six had already been argued, in the 2.0.0 ledger's own comment, so the objection was about where the argument was written rather than about whether one existed.) +- A checked-in generator for the rows (2026-09-05) — the rows are a snapshot, and a generator that re-derives them on demand is the re-derivation hazard with a command line (mechanisms.md#RECORDED-ROSTERS). A scratch script and a recipe in the roster comment, as `_CORPUS_FLOORS` and `_CORPUS_CLAIMS` do. + +The measurement, and how to redo it. `_WATCHED_DIFFS` holds 41 / 36 / 31 / 5 rows at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0, 113 in all; the 2.0.0 section is 32 sole-watched rows plus the four #501 contests (`'김민준 박사님'` and `'선생님'`, contract tier and fatal; `'田中さん 様.'` and `'田中さん, 様.'`, `corpus_cjk_tolerated.jsonl`, radar and printed — the first of that pair read contract when this was measured earlier the same day, and the period-class demotion under #cjk-comma-demotion above moved it hours later, which is the point the snapshot makes better than the prose could: a row's fatality is a property of the tier its NAME reads at, and no roster in `compare.py` records it). 51 distinct sole-watched names; every row a default-order shape, so `NOT CHECKED` names none of them. (AMENDED 2026-09-05, hours later: those four rows and `'MD, PHD'`'s two — at 2.0.0 and 2.1.0 — left for `_RECORDED_DIFFS` with the #501 adjudication below, so the roster now reads 41 / 31 / 30 / 5 = 107 over 50 names and the 2.0.0 section is 31 sole-watched rows and nothing else — since #498, 36 / 31 / 30 / 5 = 102 over the same 50, the five names it pinned at 1.4.0 having left that section alone. The figures in this paragraph were true of the tree this arc's commits left; run the RECOMPUTE at the end of it for today's.) Four mutation controls, in memory around `main()` with the worktree untouched, re-run 2026-09-05 against the amended data commit: `'QC MP'` corrupted at 1.4.0 (its only row — it does not diff at 2.0.0) prints `MOVED SHAPE (radar) expected_since_1.4.0.toml: 1 watched diff shape(s) disagree with this run` and exits 0, `'Smith Dr'` at 2.0.0 likewise; `'선생님'` corrupted at 2.0.0 prints `MOVED SHAPE expected_since_2.0.0.toml: 1 watched diff shape(s) disagree with this run` ending "This fails the run because the name is contract tier" and exits 1; `'Ph. D., Jr.'` corrupted at 2.1.0 prints the `(radar)` block and exits 0; and `'MD, PHD'` inserted into the 1.4.0 watched section beside its contest row is refused pre-worker, before any `baseline:` line, as "1 name(s) sit in both `_RECORDED_DIFFS['expected_since_1.4.0.toml']` and `_WATCHED_DIFFS['expected_since_1.4.0.toml']`", and fails `test_the_watched_roster_is_disjoint_and_names_every_ledger` naming it — the guard half wrapping `load_tool`, since that helper executes a fresh `compare` module per call and a mutation on an imported one never reaches it. A control whose corruption changed nothing would have meant an inert row; none did. The arc moved no classification: 352 / 247 / 155 / 14 intentional diffs, 0 unexplained and 0 radar-unclassified at every baseline, byte-identical summary lines before and after every commit. The quiet-tree claim that made this the moment to snapshot: measured 2026-09-05 at `4dd776a`, `git rev-list --count a965272..4dd776a -- nameparser/` is 5 commits since the tier split, moving 38 inserted and 15 deleted lines across two files, `_facade.py` (the #462 initials fix and its review rounds) and `_pipeline/_script_segment.py` (`f046d5d`, the tier-split day's own W3 demotion). PINNED TO THAT COMMIT deliberately, and NO LIVE HEAD FIGURE STANDS BESIDE IT — a first draft of this paragraph put one here as a contrast, and that number was already wrong at the commit that wrote it, which is precisely the defect the pin exists to avoid rather than an illustration of it. What can be said without a number that rots: the period-class bundle's commits after `4dd776a` touched `nameparser/_pipeline/_script_segment.py`'s comments and its W2/W3 citations — and the commit correcting this very sentence was one of them, after a draft that named two by hash had been made stale by the third — so the same range recomputed at HEAD is LARGER than the pinned one on all three counts, and it moves again with every further comment-only commit. Neither the count of such commits nor their hashes belongs here: `git log --oneline 4dd776a..HEAD -- nameparser/` lists them, and a figure that would have to be rewritten by each commit that changes it is not a figure prose can hold. So this is a POINT measurement of the tree the snapshot was taken on, and rereading it as a live one is the error to avoid — the figure goes stale by design, since the next feature is what the snapshot exists to make legible. RECOMPUTE the row counts with `{k: len(v) for k, v in compare._WATCHED_DIFFS.items()}`, the population with the script in the dict's header, the controls with `compare._WATCHED_DIFFS[ledger][name] = (...)` around `main()`, and the quiet tree with the command above plus `git diff --shortstat a965272..4dd776a -- nameparser/` — with the PINNED endpoint and not `HEAD`. A `HEAD` substitution answers a different question, and this paragraph deliberately does not state its answer. + +## differential-ledger, the #501 adjudication (2026-09-05) + +The ninth ledger arc, and the one that answers the question the arc above left open hours earlier in its own `Declined:` list — whether a contest that is MEASURED but unargued is owed a pinned winner. Six were measured across the two 2.x ledgers as this branch opened — five of the 247 diffs at 2.0.0 and one of the 155 at 2.1.0 moving a shape two or more rules admit — and all six are owed one: three of the six boundaries had ALREADY been argued, in a ledger comment, which is a place nothing in the tree can check, and one of those three arguments was false. Closing that one took a regex narrowing rather than a pin, and the narrowing then dissolved two of the six contests, which is this arc's sharpest finding about what a pin is for. Mechanics — the two rosters and their contracts, the severity rule, the population recipe, the six rows' own arguments — are owned by `tools/differential/compare.py` (`_RECORDED_DIFFS`' and `_WATCHED_DIFFS`' headers), by `tests/v2/test_ledger_guards.py` (the comment over `_CROSS_RULE_WINNERS`' two 2.x sections, and the per-row comments) and by tools/differential/README.md; these are the decisions. Every figure below was measured 2026-09-05 on this branch at `ddd8b2b`, from real `main()` runs at all four baselines with `classify()` and `_run_worker` spied, and each carries its recipe. Closes [#501](https://github.com/derek73/python-nameparser/issues/501). + +Decisions that landed: + +- 2026-09-05 #501 — a row is owed where the boundary is ARGUED ANYWHERE, not where it is argued in the guard module. The position the arc above recorded — "a row is owed when somebody ARGUES a boundary, not when one is merely measured" — is kept whole; what moves is WHERE an argument counts. Three of the six boundaries were already argued in the 2.0.0 ledger's own comment on `fix(#308/#312/#319/#320)`, which named `'田中さん 様.'`, `'田中さん, 様.'` and `'김민준 박사님'` one by one and said for each why the glued rule should win. That is a claim about which rule wins a name, and it is checked by NOTHING: `_CORPUS_CLAIMS` measures a rule's regex reach alone and cannot see a name changing hands inside it, the gate's totals are per-corpus, `order_contests` reports only wide-first nested pairs, and `classify()` is never asked the question unless a roster row asks it. `_CROSS_RULE_WINNERS` is the only thing in the tree that can check a ledger comment's adjudication, so scoping its entry criterion to arguments made in its own file left exactly the arguments it exists to check outside it. The proof is the third of the three, below. +- 2026-09-05 #501 — `'田中さん 様.'` and `'田中さん, 様.'` → `fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix`, at 2.0.0. The only one of the six that is a live two-runner contest after the narrowing: both names are admitted by the glued rule and by `fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix`, and the glued rule is written first (sorted indices 1 and 2; lines 71 and 149 of the ledger on this branch). Measured against the 2.0.0 wheel, both diff `{family, given, suffix}` — `'田中さん 様.'` reads first `'田中さん'`, last `'様.'` on the wheel and family `'田中'`, suffix `'さん, 様.'` on the tree; `'田中さん, 様.'` reads first `'様.'`, last `'田中さん'` on the wheel and the same family `'田中'`, suffix `'さん, 様.'` on the tree. Both names move through TWO mechanisms at once — the glued さん peels off 田中さん while the spaced 様 routes to `suffix` — with one rule per half, so neither rule describes the whole diff and the label is PARTIAL in #382's accepted sense. The prose test lands where the file order already does: the glued rule's own #320 clause names both strings outright — "#320 stops an ASCII period vetoing the honorific it finds there ('田中さん 様.', '田中さん, 様.')" — and no clause of the spaced rule mentions either, its examples being `'王小明 先生'`, `'김민준 씨'`, `'田中 太郎 様'`, `'田中 殿'`, `'김민준 씨.'` and `'김민준 양.'`. What the pin buys, measured: swapping the two rules in memory hands BOTH names to the spaced rule and touches nothing else — 2 of the 247 classifications move, `unexplained` stays 0 so the gate's summary line does not move, and neither rule's `_CORPUS_CLAIMS` reach can see a reorder at all. The guard fails on both names, first line "expected_since_2.0.0.toml: '田中さん 様.' diffing ['family', 'given', 'suffix'] is now explained by 'fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix', not 'fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix'." +- 2026-09-05 #501 — `'김민준 박사님'` → `fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix`, at 2.0.0, and the pin holds only because of the 2026-09-05 narrowing of the glued-peel `name_regex` in both ledgers (`a6a82d6`). Before the narrowing the glued rule reached the name on the 님 interior to 박사님 and, sitting first, took it. Measured against the 2.0.0 wheel the diff is `{family, given, suffix}`: the wheel reads first `'김민준'`, last `'박사님'`, and the tree reads given `'민준'`, family `'김'`, suffix `'박사님'` — the honorific WHOLE and standing as its own word, with nothing peeled off the name. The prose test is rules/W2's second sentence, "A part that is not name text — a post-nominal word standing on its own — is never the name's end: the split-off steps past it to the name word behind, and never dissects it", and that rule's witness line for this very string, `"김민준 박사님" → suffix="박사님"`. So the glued rule's title was FALSE of the name, on a contract-tier string (`corpus_cjk.jsonl`), held green by every guard in the suite from the date the comment's own parenthetical carries (2026-08-05) until this arc. What the pin buys, measured: restoring the pre-narrowing regex ALONE — one edit, no reorder, the glued rule still sitting ahead of the spaced one — hands the name straight back, 1 of the 247 classifications moving with `unexplained` still 0, and this row is the only thing that says so. The guard's first line is "expected_since_2.0.0.toml: '김민준 박사님' diffing ['family', 'given', 'suffix'] is now explained by 'fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix', not 'fix(#307/#308/#320) spaced CJK postnominal honorific routed to suffix'." +- 2026-09-05 #501 — `'선생님'` → `fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots`, at 2.0.0. Measured against the wheel the whole diff is `{family, given}` — the wheel reads first `'선생님'` and the tree reads family `'선생님'`, the 2.1 order flip on a hangul string and nothing else — which is exactly the shape that rule's comment claims for itself, "these are one diff shape -- pieces moving between given/middle/family on a native-script CJK name", and which rules/W2's own boundary line `"선생님" → family="선생님"` says is name text rather than a peel. The glued rule's claim on it was the same regex accident, the interior 님, and worse than a partial label: it declares `suffix` and NO `suffix` moves here at all. It never won, the order rule being written first (sorted index 0 against the glued rule's 1), and the narrowing has since taken its admission away. Recorded because the pair is the #498 class and nothing else can see it: the two rules' `fields` INTERSECT on `{family, given}` — `{family, given, suffix}` against `{_ambiguities, family, given, middle}` — with neither nested, so `undeclared_contests` looks past them in either arrangement and no `precedes_narrower` block is available to declare them. What the pin buys, measured: a reorder ALONE now moves nothing here (0 of the 247), so the row's whole work is against a RE-WIDENING — restore the wide regex and swap the two rules and 2 classifications move, `unexplained` staying 0, with the guard's first line "expected_since_2.0.0.toml: '선생님' diffing ['family', 'given'] is now explained by 'fix(#308/#312/#319/#320) glued CJK honorific peeled off the name into suffix', not 'fix(#271/#272/#298) native-script CJK: …'". +- 2026-09-05 #501 — `'MD, PHD'` → `fix(#296) a credential-only comma string reads a name and its postnominal`, at 2.0.0 AND at 2.1.0, as a row apiece rather than one shared row. Both #296 rules reach the string and declare EQUAL `fields` at each 2.x baseline, `["title", "suffix"]`, so `fields` cannot separate them at all and file order is the whole decision (sorted indices 25 and 26 at 2.0.0, 12 and 13 at 2.1.0). Measured against both wheels the diff is `{suffix, title}` — title `'PHD'` on the wheel becomes suffix `'PHD'` on the tree, family `'MD'` on both sides. The distinguishing clause is a property of the STRING and not of the rules' `fields`: the credential-only rule opens "'Jr., PhD' / 'MD, PHD': a string that is nothing BUT credentials", while the lone-post-comma rule is written for "A one-word name, a comma, one short word". `'MD, PHD'` holds no name, so the first rule's prose is true of it and the second's is not. Two rows and not one because the roster is keyed per ledger and a shape is baseline-relative — the 1.4.0 row on the same string and the same winner rests on a different measurement, `{family, given, suffix, title}` there, the comma-family move not yet having landed. What the pin buys, measured: swapping the two rules moves 1 of the 247 at 2.0.0 and 1 of the 155 at 2.1.0, `unexplained` staying 0 at both, and the guard fails at BOTH baselines, first line "expected_since_2.0.0.toml: 'MD, PHD' diffing ['suffix', 'title'] is now explained by 'fix(#296) a lone post-comma credential is a suffix', not 'fix(#296) a credential-only comma string reads a name and its postnominal'." and the same sentence under `expected_since_2.1.0.toml`. +- 2026-09-05 #501 — the glued-peel `name_regex` is NARROWED so it stops matching a listed honorific INTERIOR to a longer listed one, in BOTH ledgers that ship it (`expected_since_1.4.0.toml` and `expected_since_2.0.0.toml` carry it byte-identically). `(? fix(#271/#272/#298), the order flip with no comma routing …` — and carries `middle`, which neither name moves. The measurement is what settles the pair, not the table. What the pin buys, measured: moving the loser ahead hands both over and moves 9 of the 352 — the two, plus seven collateral names off `fix(cjk-comma-honorific-peel) glued honorific peels off a post-comma given name`; dropping `given` or `family` from the winner also hands both over — to `fix(cjk-comma-compound)`, this pair's loser — but at the price the nakaguro bullet above measures, since it is the same rule and the same two drops: 43 moved and 35 of them UNEXPLAINED. So here too the reorder is the edit whose cost stays inside this ledger's explained set. +- 2026-09-05 #498 — `'田中さん, Dr.'` → `fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots`, over `fix(cjk-glued-honorific-peel) glued honorific peels into suffix`, at 1.4.0: the SECOND unowned pair on one name, and the reason fifteen rows sit on fourteen names. `{family, given, middle}` against `{family, given, suffix}`, intersecting in `{family, given}` without nesting (sorted indices 0 and 34). Measured, `suffix` is `''` on both sides and the tree's family is the whole token `'田中さん'`: the glued さん is NOT peeled, where the peel this loser is named for would have taken family to `'田中'` and put さん into `suffix` — which is what does happen to `'田中さん, 様.'` in the next bullet, so the contrast between a mechanism that fired and one that did not sits inside this arc's own measurements. Its stated scope excludes the string anyway, the rule being written for a name with "no space before the honorific and no comma anywhere", and this name has a comma. What the pin buys, measured: moving the loser ahead hands the name over and moves 15 of the 352, much the widest of the seven pairs, the peel rule then also taking names off `fix(cjk-comma-compound)` and `fix(cjk-comma-honorific-peel)`. A `fields` narrowing does NOT buy this pair: dropping `given` or `family` from the winner sends the name to `fix(cjk-comma-compound)` and not to this loser, so a reorder is the only single edit that hands it over. +- 2026-09-05 #498 — `'田中さん, 様.'` → `fix(cjk-comma-compound) comma routing compounds with the CJK order flip`, over `fix(cjk-honorific-suffix) postnominal honorifics recognized, compounding with the CJK order flip`, at 1.4.0. `{family, given, title, suffix}` against `{family, given, middle, suffix}`, intersecting in `{family, given, suffix}` with `title` and `middle` keeping either from containing the other (sorted indices 33 and 35). Measured against the wheel: first `'様.'` → given `''`, family `'田中さん'` → `'田中'`, suffix `''` → `'さん, 様.'`, a diff of `{family, given, suffix}`. A genuine TWO-MECHANISM compound, and both halves are legible in the values — the glued さん peels off `田中さん` while the post-comma `様.` leaves `first` for `suffix`, the two arriving there together. The winner describes the comma half; the other half belongs to `fix(cjk-glued-honorific-peel)`, which is nested inside the winner and DECLARED over by a `precedes_narrower` block, so that half of this name's contest is justified where it stands and only the honorific-suffix pair is owed a row. That is the partial label #382 accepts, with every half of the compound owned between the two rules. The loser is the weaker candidate for the same half on its own terms: its prose is about a SPACED trailing honorific whose remainder takes an order flip, it disclaims the glued forms outright — "GLUED diffs (田中さん, 김민준씨, and the 김씨 that segmentation already reached) never match this rule" — and it says nothing about a comma, where the comma is the winner's subject; measured, no `middle` moves and no order flip occurs, `given` going empty rather than gaining a name. What the pin buys, measured: moving the loser ahead hands the name over and moves 2 of the 352 — this name, which comes off `fix(cjk-comma-compound)`, this bullet's winner, and `'田中さん 様.'`, which is the one that comes off the glued rule; a `fields` narrowing buys nothing here either, dropping `given`, `family` or `suffix` from the winner sending the name to the glued rule, so the reorder is again the only single edit. (The 2.0.0 section of `_CROSS_RULE_WINNERS` pins this same string to the GLUED rule. That is a different ledger with a different rule set, where the pair nests narrow-first and no declaration is owed — the #501 adjudication above — and it does not contradict this 1.4.0 verdict.) +- 2026-09-05 #498 — `'Bob Jones, compositeur'` → `fix(comma-family) a comma followed only by titles keeps the given/family split`, over `fix(comma-precomma-family) pre-comma run reads as family, not given`, at 1.4.0. EQUAL `fields`, both `{given, family}`, so file order is the whole decision (sorted indices 5 and 13). Measured against the wheel: first `'Bob Jones'` → given `'Bob'`, last `''` → family `'Jones'`, and title `'compositeur'` on BOTH sides — diff `{family, given}`. The winner's prose names the string and describes those values to the token, "the pre-comma name keeps its positional read -- first 'Bob Jones' -> given 'Bob', family 'Jones', the title staying where it was"; the loser's stated behavior, the pre-comma run reading as family, is half true at most here, `'Jones'` becoming the family and `'Bob'` not, the run not moving whole. The loser's own comment already concedes the sibling string `'Bob Jones, author'` to the winner on exactly that reasoning, and this row records the same for `compositeur`, which the winner's regex reaches by the same literal. It is also why a pin is spelled with the FULL issue string: three rules in this ledger begin `fix(comma-family)` (measured 2026-09-05 with tomllib over the ledger's `[[change]]` issues). What the pin buys, measured: moving the loser ahead moves 3 of the 352 — this name plus `'Bob Jones, author'` and `'John Smith, Mr.'`, and the three do not come off one rule: this name and `'Bob Jones, author'` come off the WINNER, and `'John Smith, Mr.'` off a third rule whose issue extends the winner's, `fix(comma-family) a comma followed only by titles keeps the given/family split, the C1 example`. The lone-post-comma rule, the third of the three rules whose issue begins `fix(comma-family)`, loses nothing here — which is the full-issue-string point made in measured form rather than by counting rules that share a prefix. Dropping `given` or `family` from the winner hands this name over on its own, moving 2 (this name and `'Bob Jones, author'`) with nothing left unexplained. +- 2026-09-05 #498 — `'MD, DO, DDS'` → `fix(#296) a dropped prenominal takes the name position it occupies`, over `fix(comma-family) lone post-comma piece routes to suffix/title, not first`, at 1.4.0. `{title, given, middle}` against `{given, title, suffix}`, intersecting in `{given, title}` — which is the measured diff — with neither containing the other (sorted indices 7 and 12). Measured against the wheel: title `'DO'` → `''` and first `''` → given `'DO'`, with family `'MD'` and suffix `'DDS'` byte-identical on both sides. The winner names the string and states its diff verbatim, "the rule's second name is 'MD, DO, DDS': title 'DO' -> given 'DO', the comma anchor's one corpus hit", which is the whole of what moved. The loser is about a one-word NAME followed by a comma and a lone credential; this string holds no name at all and has two post-comma pieces, and its direction is wrong as well — the loser routes a piece OUT of `first` where this name routes one IN. What the pin buys, measured: moving the loser ahead moves 8 of the 352 — this name, all six of the `fix(#325)` pair above, and `'Smith, Jr., PhD'`, since a move above index 7 necessarily passes `fix(#325)` at index 10 — so the reorder is the strictly larger edit here. The `fields` narrowing is smaller and still not an alternative: dropping `given` or `title` from the winner moves 2, handing `'MD, DO, DDS'` to this loser and sending `'Do Quang Minh'` — a radar-tier `corpus_issues.jsonl` name the winner is the sole explainer of — to NO rule, so radar unclassified goes from 0 to 1. Both edits cost something the pin does not; the pin is what records which rule is right without paying either. + +Found rather than decided, and worth as much: + +- **The class is measured in NAMES, not pairs, and the issue's framing had it in pairs.** 60 pairs overlap without nesting and 11 more have equal `fields`, each sharing a corpus name; measured with a diff in hand, 14 names have one that two such rules admit, over seven of those pairs. The static count over-reports for the reason #382's nine-of-eleven finding gave — most co-matched names' real diff needs a role the other rule does not declare — so the rule-order arc's objection to a widened STATIC predicate, that it would demand 111 justifications where the real number is eleven, does not reach a MEASURED one. That is the whole of why option 3 was cheaper than it read. +- **Seven of the fourteen sit in an EQUAL-`fields` pair — the clearest case of the class the docs name `_CROSS_RULE_WINNERS` the instrument for — and not one of the seven was pinned.** Two pairs, and in both `fields` separates the rules not at all and file order is the whole decision: `fix(#325) a split credential followed by another suffix after a one-word family comma reads as suffixes` ahead of `fix(comma-family) lone post-comma piece routes to suffix/title, not first` over six names, and `fix(comma-family) a comma followed only by titles keeps the given/family split` ahead of `fix(comma-precomma-family) pre-comma run reads as family, not given` over `'Bob Jones, compositeur'`. Equal `fields` is the special case the 2026-09-02 entry reasons FROM — the class it leaves to the instrument is every pair where neither set contains the other, and the other seven names sit in that wider half, over eight of the fifteen rows. The instrument was named there and checked by nothing until this arc, and its clearest cases were the unowned ones. +- **Fourteen names, SEVEN (winner, loser) pairs, fifteen (name, pair) rows — and a first draft counted five, by grouping the rows by WINNER.** `fix(#271/#272/#298)` heads three distinct pairs, against the delimited-nickname rule, the comma-compound rule and the glued-peel rule, which collapse to one under a by-winner count; and `'田中さん, Dr.'` is unowned against TWO losers, so it carries two of the fifteen rows and appears under two of the verdict bullets above. Seven pairs over fifteen rows is what the sweep reports. The fourteen is a count of NAMES, which is the unit the pins are written in — one row per name in each roster, whatever the number of losers behind it — and the three figures answer three different questions rather than disagreeing. + +Declined: + +- The wider narrow-first predicate (2026-09-05, the scope decision above) — 9 more names at 1.4.0 and 23 in all, declined as a pin the declaration-free default already justifies. +- A `precedes_narrower`-shaped per-pair declaration for non-nested pairs (2026-09-05, the issue's option 1) — the roster is per-name and already carries the argument, and a second key would let a PAIR be declared without any name's shape being measured. That is the #501 adjudication's own finding one level down: an argument nothing can check, given a key of its own. +- A static sibling of the check (2026-09-05) — there is none to write, for the reason the first decision above gives: without a diff the population is the 71-pair reach — 60 overlapping pairs and 11 with equal `fields` — and the roster is unwritable. +- Reordering any of the seven pairs (2026-09-05) — a reorder moves which rule classifies a name, which is a behavior change to the ledger and breaks `_CROSS_RULE_WINNERS`; every current winner is argued where it sits, and the seven verdict bullets above are that argument, name by name. A wrong winner would have been closed by a narrowing or a declared reorder and not by a pin; the adjudication found none. + +The measurement, and how to redo it. `_RECORDED_DIFFS` reads 45 / 5 / 1 / 0 = 51 rows at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0 and `_WATCHED_DIFFS` 36 / 31 / 30 / 5 = 102 over the same 50 distinct names — five names moved rosters at 1.4.0 and stayed watched at both 2.x baselines, where each has one admitter and no contest to own, so only the 1.4.0 row count fell and the population did not move. 32 of the 45 rows at 1.4.0 sit on radar-tier names and 13 have a single admitter at their recorded shape; none of the fourteen joins those 13, every one having two or more admitters at its recorded shape by the definition of the sweep that found it. THE POPULATION FIGURES, recomputed: drive `main()` at each baseline with `classify()` wrapped, keep the calls whose issue is not None, and for each list the rules satisfying `compare._entry_matches` at that call's measured shape and order — fewer than two admitters is no contest; for each admitter after the first, the winner's `fields` a strict subset of the loser's is narrow-first and justified, the loser's a strict subset of the winner's WITH the loser's issue in `compare._declared_over(winner)` is declared and justified, and anything else is owed a pin unless the name is a key of `_RECORDED_DIFFS` for that ledger. That gives 47 / 3 / 1 / 0 contested, 18 / 3 / 1 / 0 pinned and 14 / 0 / 0 / 0 owed on the tree this arc opened on, and 0 owed at every baseline since. Do NOT replay the corpus load by hand: the `(name, order)` dedup, the baseline-minimum skip and the tier stamp all happen inside `main()`, so a row's tier is read as `main()` reads it — the default-order entry's where there is one, the first-loaded entry's otherwise — by wrapping `_run_worker` for the entries it was handed. THE VERDICTS' values come off that same run — the old side from the worker's rows for the entries `main()` was handed, the new side from `HumanName` and `_tree_parse` as `main()` computes them — and only the facade is compared at a pre-2.0 baseline, `_surfaces_for` returning `{"facade"}` below 2.0.0, so every old and new value in the bullets above is `HumanName`'s. THE "WHAT THE PIN BUYS" FIGURES are measured the same way and their recipe sits beside the rows in tests/v2/test_ledger_guards.py: move the loser above the winner in a COPY of the run's own sorted rule list and re-classify the run's own 352 classified calls, counting the labels that change; the `fields` readings are that recipe with one role dropped from the winner instead of a rule moved, and nothing on disk moves for either. MUTATION CONTROL, in memory around `main()` at 1.4.0 with the worktree untouched: `'Smith, Ph. D. Jr.'` deleted from `_RECORDED_DIFFS['expected_since_1.4.0.toml']` — the contract-tier row of the six-name equal-`fields` pair — makes the run exit 1 and print `UNPINNED CONTEST expected_since_1.4.0.toml: 1 contested pair(s), one per name and losing rule, that no rule outranks by nesting or declaration and no row pins`, naming the name under its winner as `'Smith, Ph. D. Jr.' [contract] ['suffix', 'title']` with the reason "equal `fields`, file order the whole decision". Unmutated, no `UNPINNED CONTEST` block prints at any of the four baselines; a corruption that changed nothing would have meant an inert check, and this one did not. The arc moved no classification: 352 / 247 / 155 / 14 intentional diffs, 0 unexplained and 0 radar unclassified, exit 0, summary lines byte-identical at every commit of the branch — a pin moves no classification, and neither does a check that fires on nothing. RECOMPUTE the row counts with `{k: len(v) for k, v in compare._RECORDED_DIFFS.items()}` and the same over `_WATCHED_DIFFS`; the radar and one-runner splits by taking each row's tier as above and classifying it at its recorded shape against `_sorted_rules`. + +## differential-ledger, a pin names one rule (2026-09-05, #508) + +The eleventh ledger arc, and a small one: it changes how a `_CROSS_RULE_WINNERS` pin is read, not what any pin says. The winner guard compared `classify()`'s answer to the pin with `startswith`, chosen without a recorded reason, so a pin was satisfied by any rule whose issue extended it. The #498 completeness check, earlier the same day, had borrowed that guard's meaning outright — a `_RECORDED_DIFFS` key counts as "a winner is pinned" because the two rosters' keys are equal — and its review found the case where the borrowed meaning was hollow. Mechanics are owned by `_pinned_rule`'s docstring in tests/v2/test_ledger_guards.py; these are the decisions. Every figure was measured 2026-09-05 on master at `449a7bc`. + +Decisions that landed: + +- 2026-09-05 #508 — **a pin names exactly one rule.** The guard resolves each pin to one rule's issue — exact where a rule's issue equals the pin, else the single rule whose issue starts with it — and refuses a pin that resolves to none or to several; the winner comparison is then EQUALITY against the resolved issue. Prefix pins stay: a family prefix that reaches one rule names it as well as the full string does, and 17 of the 51 rows are written that way. Measured over the 51 rows before the change: 31 full issue strings reaching only themselves; 17 short prefixes reaching one rule; 2 full strings that are also a strict prefix of a sibling's — `'Bob Jones, author'` and `'Bob Jones, compositeur'`, pinned to the all-titles rule whose issue `... keeps the given/family split, the C1 example` extends; 1 short prefix reaching three rules — `'Andrews, M.D.'` on the bare `fix(comma-family)`. All three ambiguous rows are in `expected_since_1.4.0.toml`, the 2.x rosters have none, and each was latent: `classify()` returned the intended rule. `'Andrews, M.D.'` is spelled out to the lone-post-comma rule that wins it; the Bob Jones rows are unchanged, exact comparison being what their comment already rested on. A second guard, `test_every_pinned_winner_names_exactly_one_rule`, refuses an ambiguous pin on its own at pytest speed, so the failure names the pin rather than a classification. RECOMPUTE: for each ledger and each pin, count the `[[change]]` issues that `startswith` the pin and whether one equals it (the loop is in the #508 issue body). + +Found rather than decided: + +- **The #498 arc's "a key in `_RECORDED_DIFFS` means a winner is pinned" was true of 48 rows and hollow for 3.** Nothing was wrong today, because `classify()` happened to return the intended rule for all three. What the prefix comparison could not do was FAIL when it stopped: the `'Bob Jones, compositeur'` roster comment rested its pin on the C1 rule's regex being unable to reach the string, an argument no test executed — measured 2026-09-05, forcing `classify()` to answer the C1 string for that name passed the old guard and fails the new one. + +Declined: + +- Exact match everywhere (2026-09-05) — rewriting the 18 short pins to full strings and ending the family-pin idiom. The idiom is harmless where the prefix reaches one rule, which is 17 of the 18, and the resolver makes the eighteenth loud; a reworded rule breaking its pins is the same outcome under either reading. +- Prefix plus an allowlist (2026-09-05, #508's option 2) — a second list to keep honest, for three rows the resolver handles without one. +- A word-boundary prefix (2026-09-05, #508's option 3) — answers the two Bob Jones rows and not `'Andrews, M.D.'`, and adds a tokenising rule to a string comparison. +- Refusing two rules in one ledger from sharing a prefix (2026-09-05) — a property of the ledger, not of a pin; the ledger's rule families are deliberate (`fix(comma-family)`, `fix(#296)`), and the resolver already makes the only case that matters loud. + +## differential-ledger, retired for a snapshot (2026-10-13, #664) + +The ledgers' last arc. They were built to show that 2.0 changed nothing against 1.4 except fixes the release log lists, and they did; as a standing check every later behavior change had to be classified once per ledger, and the guards policing the ledgers outgrew the ledgers. + +Decisions that landed: + +- 2026-10-13 #664 — **a committed snapshot replaces the classified gate.** `tools/differential/snapshot.jsonl` records every corpus name's reading (the seven role fields, ambiguity kinds, `initials()`, and the facade where it differs), one line per (name, order); `tests/v2/test_snapshot.py` fails on drift and prints the moves grouped by changed field. A change that moves readings re-records, and the PR diff of the snapshot is the list of names it moved. Nothing is classified: what a reviewer reads in the diff is what a ledger rule used to say in a regex. The six `expected_since_*.toml` ledgers and `tests/v2/test_ledger_guards.py` are deleted, and with them the contract/radar tiers, so the CJK corpus is one file again and the rules corpus harvests tolerated rules' examples too. +- 2026-10-13 #664 — **`compare.py` stays, unclassified.** It keeps the released-wheel runner and its checks against comparing one tree with itself (the version tell, the tree-side path check, the stripped `PYTHONPATH`; mechanisms.md#VERSION-TELL, #GENERATED-SCRIPT-OUTSIDE-THE-WORKTREE), prints the same grouped report as the snapshot and exits 0. It answers "how did release X read these names" for a release tagged before the snapshot existed; after that, `snapshot.py --since ` is the cheaper answer. +- 2026-10-13 #664 — **the CLDR agreement ratchet stays** (#654). It answers a different question, agreement with CLDR's stated fields rather than stability. + +Measured 2026-10-12, before the change, over the 40 first-parent commits ending at 04cc0814: 5,236 of 25,226 changed lines (21%) were ledger, `compare.py` or harness-test edits, and 21 of the 40 commits touched them. The share is uneven rather than steady: across the 29 of those commits that touch `nameparser/` it runs from 0% to 82%, median 11%, above 30% on 11 of them (#655: 236 of 448). Run against the code before and after #655, the snapshot reports five moved names, against that PR's 236 lines of ledger and guard edits; merging master afterwards, it reported #663's one new corpus name where #663 had added the same ten lines to each of six ledgers. RECOMPUTE (shares): `git log --first-parent -40 --format=%h 04cc0814`, and for each commit sum `git show --numstat` over every path except `*.jsonl` and `cldr_agreement.json`, the harness share over paths matching `expected_since|compare\.py|ledger_guards|test_differential|_differential_fixtures`; a commit "touches `nameparser/`" when any path starts with it. + +Accepted costs: + +- No forced explanation per moved name. A reviewer can approve a large snapshot diff without reading it; the grouping by changed fields is the mitigation. +- No bound on what a change may reach. `_CORPUS_CLAIMS`, the dormancy checks and the contest pins existed only to police ledgers; mechanisms.md#LEDGER-RULE-SEPARATION and #CROSS-RULE-OUTCOME-PINS are retired with them. +- The 1.4 compatibility promise is a question asked when needed (`compare.py --baseline 1.4.0`), not a gate at every release. The facade is deleted in 3.0. + +Declined: + +- Trimming the ledgers rather than retiring them (2026-10-13). The radar tier (#468, 2026-09-01) was that attempt: it stopped new corpora from failing the run, but every contract name still needed a rule in every ledger, and the cost grew by one ledger per release. + diff --git a/docs/design/decisions/generational-words.md b/docs/design/decisions/generational-words.md new file mode 100644 index 00000000..e56be1dc --- /dev/null +++ b/docs/design/decisions/generational-words.md @@ -0,0 +1,12 @@ +# generational-words — junior and senior in other languages (2026-10-14, #661) + +Closes #661. 31 spelled-out generational words join SUFFIX_WORDS, and one predicate moves with them (the run-start bullet below), so `Antonio Pavlović mlađi` reads family Pavlović, suffix mlađi, where 1.4.0 and 2.4.2 read middle Pavlović, family mlađi (checked against both wheels). The class is decided once — a word meaning junior/senior (or the Younger/the Elder) written spaced AFTER the name — and each word was then checked two ways. First, collisions: a suffix claim acts on the trailing word, so the risk is a borne name there, and `tools/name_usage.py` found none (US Census, SSA, Wikidata; nuorempi's one Wikidata bearer of each kind is noise, as the issue said). Second, usage, because the issue's list came partly from ilib's machine-translated test data: each word was found in the trailing spaced position in real titles on its own language's Wikipedia (2026-10-13, e.g. "Hans Holbein mlađi", "Плиний Младший", "Pieter Brueghel vanhempi", "बलबीर सिंह सीनियर"). The per-word list, grouped by language, is the #661 block in nameparser/config/suffixes.py. Feminine forms ship beside the masculine wherever the language has them (mlađa, младшая, молодша, jaunesnioji); lt vyresnioji is the grammatical feminine with no titled bearer found, and ships on grammar. ru/uk and cs/sk were not in the issue and were added on the same evidence (Derek, 2026-10-14). + +- **CLDR is corroboration, not a source.** Its person-name samples write mlađi/млађи (hr, sr — on a woman's name too), младши/младша (bg), ifj. (hu) and जूनियर (hi), but its `generation` field also holds translations of the field LABEL rather than of the word: isizukulwane (zu), जेनेरेसन (ne), نسل (ps) and jẹnereṣọn (pcm) all mean "generation". A word is not added because CLDR has it. The wider roster (fils, yngri, الابن and the rest) is open at [#679](https://github.com/derek73/python-nameparser/issues/679), to be vetted word by word the same way. +- **Excluded, each with its reason.** kıdemli (tr): a rank word (Kıdemli onbaşı); Turkish writes Jr. or a parenthesis, (oğul). sóisearach/sinsearach (ga): rank words too; Irish writes Óg and Mór, which are also inside given names (Seán Óg Ó hAilpín), so they are a separate decision and not taken here. ifj./id. (hu): written BEFORE the name (ifj. Kovács János, which H2's shape rule already reads as a title), and a bare `id` would make any trailing ID a suffix. млада (bg feminine of млади): Mlada is a borne given name, and under a family-first order the given name is the trailing word. ml./мл./st.: the abbreviations stay under the #269 deferral in suffixes.py, st. colliding with Saint besides. +- **Spaced only, and what that leaves.** bg, ru and uk write a modern Jr. hyphen-glued (Теодор Рузвелт-младши, Дональд Трамп-молодший), which no whole-token set reaches; for those three the gain is mainly the historical the Elder/the Younger names, written title case (Младший), which the lowercase fold matches. +- **The #602 run start reads the letters' script, not ASCII (Derek chose the Latin-script test, 2026-10-14). SUPERSEDED the same day by #679 (decisions/S2): no script is refused now, so the split this bullet accepts is gone.** rules/S2 lets "an unambiguous Latin suffix word of two or more letters" start the run, and `_pieces.starts_a_credential_run` had implemented Latin as `letters.isascii()`, exact while no accented Latin word was suffix vocabulary. This entry adds the first ones (mlađi, mlađa, mladší, starší), so without a change `Pieter Bruegel stariji Jones` would read suffix 'stariji Jones' while `Antonio Pavlović mlađi Jones` kept family Jones -- one language, one construction, split by the `đ`. The test now asks a non-ASCII word whether every letter's Unicode name is a LATIN one; the ASCII test stays first, so an ASCII word pays no frame for it. The same script test is what refuses the non-Latin honorifics (`Taro Tanaka さん Jones` keeps family Jones), and it refuses this entry's Cyrillic and Devanagari generational words with them, though the reason S2 gives for the refusal -- an honorific carries no credential mark -- is not true of them. So Serbian splits by script: `Ana Horvat stariji Jones` reads suffix 'stariji Jones' and `Ана Хорват старији Џонс` family Џонс, and after a comma C1 asks the same predicate, so `Ана Хорват, старији Џонс` reads given 'старији'. ACCEPTED for this entry (Derek, 2026-10-14) and stated at rules/S2's Accepted clause: the vocabulary does not mark which suffix words are honorifics, a script test is the only line available, and only a name word written after the generational word reaches it. Settling it is open at [#679](https://github.com/derek73/python-nameparser/issues/679), where the non-Latin generational words multiply. The first draft of this entry credited the ASCII test to S2's honorific exclusion instead and proposed deleting it, on the evidence that deleting it moved no test and no corpus name. The design-docs review found the attribution wrong and the evidence empty: nothing pinned the test, and deleting it reads `Taro Tanaka さん Jones` as suffix 'さん Jones' and `Гай Плиний Старший Иванов` as suffix 'Старший Иванов'. Both are rules/S2 boundary examples now. #678 was filed on the first draft and corrected. +- **Measurement (2026-10-14).** Six corpus names move, all in corpus_cldr.jsonl and all the CLDR hr/sr renderings carrying mlađi/млађи, each gaining suffix mlađi (or млађи) from a name field; cldr_agreement records 14 changes, every one toward CLDR's fields. The bg and hi renderings do not move because those locales' formats drop the generation field. The run-start change moves no name the corpus held before this entry; its one mover is S2's own `Antonio Pavlović mlađi Jones`. Of the 161 rows of tools/differential/cldr_fields.jsonl whose `generation` value `cldr_agreement.occurs()` in the text, none writing a non-ASCII generation word has a name word after it in the same part (two write given names after `II` across a comma, which ends the run's part). Recompute: `uv run python tools/differential/snapshot.py --since `. + +- **2026-10-14 #679 — the rest of CLDR's roster, vetted word by word (Derek).** 76 more words join SUFFIX_WORDS (counted against master's set, none removed): the #679 block in suffixes.py, 30-odd languages. Vetting as above, run by three research passes over each language's Wikipedia (langlinks of the Jr./Sr./the Younger/the Elder articles, then the language's own search) and `tools/name_usage.py`. CLDR's `generation` field proved unreliable in about a third of languages, beyond the field-label translations: an adjective written BEFORE the name (ta இளைய, kn ಕಿರಿ, si බාල, az Kiçik, uz Kichik, tk kiçi, ba кесе, ky Кенже, kk кіші), an abbreviation nobody writes after a name (mr ज्यु., pa ਜੂ., kok धा.), a word that means "second" (chr ᏔᎵᏁ, hi द्वितीय, zh 二世 — regnal, and glued), or a word no Wikipedia uses where another is standard. Where the language's Wikipedia writes a different word after names, that word ships instead of CLDR's: lo ຈູເນຍ for ຮຸ່ນລູກ, ti ጁንየር (and am ጁኒየር) for ማንጁስ, or ଜୁନିଅର/ଜୁନିୟର୍ for ଜେଆର୍, ta இளையவர் and ஜூனியர், si ජූනියර්, mk помладиот for јуниор. EXCLUDED, each with its reason: he סניור, which Israelis bear as a surname (אלי סניור and בוריס סניור, found on he Wikipedia), and ja シニア, whose only evidence is weaker: `tools/name_usage.py` gives both spellings the same 96 Wikidata bearers as Latin Senior, with the same citizenships, because Wikidata's Senior surname item carries he and ja labels, so the count measures the English surname and not Japanese writing; left out anyway (Derek, 2026-10-14), beside its Hebrew twin — English `senior` is no suffix either, Senior being a surname (3,864 US Census); hy Ավագ (an Armenian given name, in lists that write the surname first); km ជូនៀ (a given name; ជូនៀរ ships); Ajala (a Yoruba surname); kk Джуниор and mk јуниор (a given name or a club name there); mn Жр (the initials ЖР); be мал. (also "figure"); lt jaun.; the single-letter abbreviations mr क. and as ক., which are the initial shape; ne/kok जेआर (rare; also JR); fa پدر/پسر ("father"/"son" without the article that marks ar الابن and he הבן as never a name); and the CLDR forms that are label translations or unattested (cv, chr, ak, blo, kok-Latin, kk-Arabic). PACK-SHAPED, NOT SHIPPED: fr fils/père and the Latin abbreviations ml./st. (cs, sk, sl, hr) are real and standard, but their collisions are with OTHER languages written in the same script — `Marie Fils` (767 US Census bearers of Fils) and `Jiří Dienstbier ml.` beside ML initials and St. as Saint — which is the case a locale pack exists for (the #604 `nic` precedent). No fr or cs pack exists; proposing them is open at [#684](https://github.com/derek73/python-nameparser/issues/684). The Cyrillic мл. does ship: ru, bg and sr all write it after names and nothing collides with it in Cyrillic, so a pack would protect nobody; ст. is barely attested after a name and stays out. Neither existing pack takes anything: tr and az put the junior word BEFORE the name or in a parenthesis, and Turkish baba collides inside Turkish. Measured 2026-10-14: 16 corpus names move, all CLDR renderings in is, ar, fa, gu, kn, ml, ru and cv (cv's rendering through мл.) gaining their generation word as the suffix out of a name field; cldr_agreement's default view records 4 suffix fields newly agreeing (cv, fa, is, ru) and no regression, and 40 still-wrong fields changing across the 16 renderings: 12 suffixes now the generation word alone where CLDR also holds the comma-led credential (`yngri` for `yngri þingmaður`), and 28 family, family_base and given_names fields that lost the word but still disagree. No rendering newly agrees on every role. The Burmese sample glues the Burmese comma ၊ (U+104A) to ဂျူနီယာ and is not reached, the tokenizer not splitting on it. + diff --git a/docs/design/decisions/given-name-titles.md b/docs/design/decisions/given-name-titles.md new file mode 100644 index 00000000..fb0b8381 --- /dev/null +++ b/docs/design/decisions/given-name-titles.md @@ -0,0 +1,6 @@ +# given-name-titles — deliberately unvalidated + +Declined (rc1 arc; the full argument is AGENTS.md's gotcha): + +- Validating given_name_titles against titles, twice: the whole-entry check rejected legitimate multi-word entries; the per-word check rejected "sir and dame" (a conjunction inside a title run is itself a TITLE token, so the key is matchable while its middle word lives in conjunctions). No static relation over the vocabulary sets decides reachability, and an unreachable entry is inert — each guard cost a working configuration to forbid a condition that costs nothing. + diff --git a/docs/design/decisions/indic-honorifics.md b/docs/design/decisions/indic-honorifics.md new file mode 100644 index 00000000..32815a61 --- /dev/null +++ b/docs/design/decisions/indic-honorifics.md @@ -0,0 +1,17 @@ +# indic-honorifics — the renunciate class and the Indic honorific vocabulary (2026-09-06, #346/#344/#343) + +Closes #346, #344 and #343 as one bundle: each CLASS is decided once, and each script's membership in it is then argued word by word, so the three lists overlap without matching. No parser code moves in it. Four existing forks carry the whole thing — the renunciate fold (rules/H1), the spaced trailing suffix path, title vocabulary beating the one-letter-plus-period initial shape, and the unvalidated given_name_titles — and all four were confirmed on the pre-bundle tree with a throwaway override before any wordlist was edited — the recipe is `Parser(lexicon=Lexicon.default().add(given_name_titles={...}, titles={...}, suffix_words={...})).parse(text)`, and the fourth fork is the one a reader will not expect: `Lexicon` deliberately does not validate `given_name_titles` against `titles` (see #given-name-titles), so an entry in the given-name set alone is never recognized as a title at all. In the shipped data that is free, TITLES being GIVEN_NAME_TITLES | {...}. + +- **The renunciate class (#346).** What is decided once is the CLASS — renunciate given-name titles — against rules.md's H Background, which already carried the two criteria: membership in the given-name-title list follows HOW THE TITLE ADDRESSES, and whether an empty family is correct output follows SURNAME RETENTION. Renunciation answers both at once, so "Swami Vivekananda" reads given Vivekananda, family "". Each script's MEMBERSHIP is its own list, and the three overlap rather than coincide: Latin swami/guru/baba/lama (moved out of the TITLES-only block), Devanagari स्वामी/गुरु/बाबा/संत, Bengali স্বামী/শ্রীল/গুরু/বাবা. lama is a Tibetan title whose native script is Tibetan, and the Hindi and Bengali borrowings लामा/লামা are rare enough that no entry was argued for either — "Lama Zopa" folds, "लामा जोपा" and "লামা জোপা" do not. संत (Sant) and শ্রীল (Srila) run the other way: renunciate titles native to those scripts with NO Latin entry, — Latin sant collides with a real given name (the Excluded (TITLES) block in decisions/vocabulary-exclusions) and srila has no transliteration in use — "संत कबीर" folds, "Sant Kabir" does not. A sweep "harmonizing" the three lists must therefore argue each word on the two criteria, never on symmetry. rabbi and imam address by TITLE rather than by given name, so they fail the first criterion and stay out; the second agrees, those traditions keeping surnames. "Rabbi Cohen" keeps family Cohen and is pinned as the control beside the Swami row. The four Latin entries were already TITLES vocabulary, so the promotion changes only which field the one name word lands in, and C-i's position test was not re-run for them; the known leading-position collision is Guru Dutt, whose given name is Guru — family went from Dutt to "" — accepted as the pre-existing cost of guru being a title at all, and pinned by the row renunciate_title_swallows_a_real_given_name. +- **venerable is left in TITLES only, and is deliberately NOT an Excluded entry.** The traditions split on surname retention — Buddhist monastics drop the family name, Anglican archdeacons and Catholic candidates keep it — so neither answer is right for the single-name case. TITLES has no ambiguous subset and no AmbiguityKind, so, as for king in #v1-xfail-triage and roz in the maiden markers, the only two expressions available are ship and do not ship; this is the third thing, a membership that is right and an OUTPUT that is undecidable, and it stays where it is. +- **Rinpoche is postpositional** ("Sogyal Rinpoche", "Lama Zopa Rinpoche"), so it joins SUFFIX_WORDS and no title set. The Tibetan -la particle is vocative and glued and needs no entry. The Tibetan PREFIX titles (Tulku, Khenpo, Geshe, Kyabje, Jetsun) are out of this bundle — each re-asks the surname-retention question for a different tradition — and are deferred to a follow-up issue (#514). +- **The trailing set ships spaced only.** rinpoche, जी/साहब/साहिब/साहेब/महाराज and সাহেব/বাবু/মহারাজ are SUFFIX_WORDS entries and none is a honorific_tails entry. The standing prohibition below is what generalizes: it was recorded for जी alone and now covers the set, on the same evidence and with the same accepted cost. +- **Bengali বাবু is trailing where Devanagari बाबू is leading** ("অমল বাবু" against a leading बाबू). Different codepoints, so the two cannot interact; the divergence is stated in both wordlist comments so that a sweep "harmonizing" them has to argue with the data module rather than discover the split afterwards. +- **The Latin md history, corrected.** md entered TITLES in 8cd3123 (2015-09-05, #32) as the medical degree written before the name, alongside do, which #296's audit later took out (#vocabulary-collisions). The Muhammad reading is a 2026 reinterpretation, recorded in the case row audit_md_leading_stays_a_title; both readings agree on the field, which is why the entry survived #296's audit as the one deviation from that audit's table. Recorded here so the record stops implying md was added for Muhammad. +- **মোঃ/মো and মোসাঃ/মোছাঃ go into TITLES**, mirroring Latin md. Semantically they are religious name prefixes rather than titles. TITLES is the FUNCTIONAL home because `given` must stay the name the person is addressed by — মোঃ আবদুল করিম is called আবদুল করিম — and the parser has no name-prefix field. The bound-given-name set is the wrong home for the same kind of reason it is the right one for Arabic abdul: abdul is an inseparable half of one name, মোঃ is detachable. Latin Mst waits for the opt-in pack (#345); the most already in TITLES is the English "Most Rev." word and is unrelated. +- **Abbreviation marks and periods.** An entry carrying an abbreviation mark is spelled with it: the Bengali visarga ঃ (U+0983) is a spacing combining mark (Unicode category Mc, not a letter and not punctuation), and `_lexicon._normalize` lowercases, composes NFC for a non-ASCII word and strips edge whitespace and the four full stops of `FULL_STOPS` — no NFKC, no casefold anywhere in that module — so the mark is simply never touched and মোঃ reaches the lexicon and matches intact (corrected 2026-09-10, #cjk-full-stops: the fold was ASCII periods only and no NFC when this was written; the visarga argument is unchanged, ঃ having no canonical decomposition, so NFC leaves it exactly where it was). Recompute with `Lexicon.default().add(titles={"মোঃ"}).titles & {"মোঃ"}`, which is non-empty iff the mark survived, and `unicodedata.category("ঃ")`, which is `Mc`. Period spellings (মো., মোসা., ডা., प्रो., पं.) match the bare stem through the same edge-period strip, so every BENGALI abbreviation carrying the mark ships as two entries, the visarga form and the bare stem (মোঃ and মো, মোসাঃ and মোসা, মোছাঃ and মোছা, মিঃ and মি), while the Devanagari abbreviations carry no such mark and ship as the bare stem alone (प्रो, पं, प्रा, डा); the dotted form is never listed in either script. +- **Vocabulary beats the initial reading.** A one-letter-plus-period Bengali abbreviation is is_initial-shaped and the title entry wins: "ড. মুহাম্মদ ইউনূস" reads title ড., given মুহাম্মদ, family ইউনূস, while "র. কে. নারায়ণ" — real initials with no entry behind them — is unchanged. #343 verified the same fork at 2.1.0; it is re-verified here on the shipped set. This is MEASURED behavior pinned by rows, not a normative rule — no rule in rules.md states that title vocabulary outranks the one-letter-plus-period initial shape, rules/H2 speaking only to the bare-initial and the shape-versus-suffix cases — so a future shape veto must re-check "ড. মুহাম্মদ ইউনূস" before it lands. +- **rules/H2's Accepted clause keeps its claim and loses its example.** The clause says an unlisted abugida honorific stays a name word, because the shape rule cannot see an unbroken run of letters through combining vowel signs; that is still true. Its example used প্রফেসর, which this bundle lists, so the example is now "প্রকৌশলী. Sen" → given="প্রকৌশলী." — Engineer, a loanword deliberately not shipped. The example is a contract-tier name in corpus_rules.jsonl, so the swap landed in the Bengali commit rather than in the records commit; leaving it would have exited the gate 1 there. +- **Not recorded as excluded:** the Tier 4 candidates — transliterated military ranks, Advocate, Engineer, Marhum, and the kinship forms — are skipped pending a corpus check, not declined, and मां/माता/মা and the hafiz/syed/shahid forms are left alone the same way. An Excluded entry is a standing prohibition, and none of these has been argued to that standard. One of them carries a duty anyway: প্রকৌশলী (Engineer) is rules/H2's executable example of an unlisted abugida honorific, so whoever ships it must move the example to another unlisted word first, or the doc test and the contract-tier gate fail together. +- **Measurement (2026-09-06).** Four corpus names move, all under the renunciate fold, all in one direction (the family empties and the given fills): Swami Vivekananda, Guru Nanak, Baba Ramdev and Lama Zopa, every one from the radar-tier corpus_issues.jsonl. Recompute by parsing every name in the tools/differential/corpus*.jsonl glob twice — once with the shipped lexicon, once with `Lexicon.default().remove(given_name_titles={"swami","guru","baba","lama"})` — and diffing the seven name fields. The multi-name spellings are the boundary and do not move: Swami Vivekananda Saraswati and Guru Gobind Singh keep families Saraswati and Singh, rules/H1's fold requiring exactly one name word. The Devanagari and Bengali additions move no corpus name at all, the corpora holding no name written in either script apart from rules.md's own H2 example, which this bundle rewrote. The gate's intentional counts rose by four at every baseline; read today's from the `corpus:` line of `uv run python tools/differential/compare.py --baseline X`. + diff --git a/docs/design/decisions/initials-repertoire.md b/docs/design/decisions/initials-repertoire.md new file mode 100644 index 00000000..fc76606c --- /dev/null +++ b/docs/design/decisions/initials-repertoire.md @@ -0,0 +1,8 @@ +# initials-repertoire — is_initial across scripts (#320) + +- The principle: decided per script from CLDR's LOCALE-AUTHORED name patterns rather than from the script's typology — ja and ko author 29 and 32 patterns and use an initial in none of them, while ru uses one in 7 of 39 (`_policy._NO_INITIALS`' comment carries the counting recipe and the trap: `initialPattern` alone reports a period convention for locales that have none). Corrected 2026-09-10, #cjk-full-stops: this read "alphabetic-vs-CJK, not Latin-vs-non-Latin — Han, kana and hangul characters are morphemes or syllables, so a single one cannot stand in for a name word", which is the phonological restatement that comment forbids in as many words — Devanagari is an abugida and Arabic an abjad, neither has letters in that sense and both abbreviate, so a rule keyed on typology would put them here and their initials are real. _NO_INITIALS is ENUMERATED per script rather than derived from the script ranges, so a new Script member forces a decision instead of silently deciding Thai has no initials. + +Declined: + +- Narrowing is_initial to [A-Za-z] — measured: "Й." regresses to the #267 Ukrainian-conjunction misreading (the exact regression 2.1.0's release log claims to prevent), and the stable "initial" tag silently strips from every Cyrillic, Greek, Arabic and Hebrew initial, invisibly to the fields. + diff --git a/docs/design/decisions/legacy-rule-numbers.md b/docs/design/decisions/legacy-rule-numbers.md new file mode 100644 index 00000000..a65c217f --- /dev/null +++ b/docs/design/decisions/legacy-rule-numbers.md @@ -0,0 +1,4 @@ +# legacy-rule-numbers — the old docstring numbering + +Before rules.md, the post_rules stage docstring numbered its rules locally, and historical issue comments (#359, #364, #365, #367) use those numbers. The mapping: rule 1 → H1, rule 1b → P1, rule 2 → O1, rule 3 → O2, rule 4 → O3. Older still: bare "#11"-style citations from v1 comments live in the GOOGLE CODE namespace, not GitHub — GC issue 11 is decisions/P3's provenance — so a v1-era number that doesn't fit its GitHub issue is probably a GC number. + diff --git a/docs/design/decisions/ma-do.md b/docs/design/decisions/ma-do.md new file mode 100644 index 00000000..1ddaedc5 --- /dev/null +++ b/docs/design/decisions/ma-do.md @@ -0,0 +1,7 @@ +# ma-do — ambiguous acronyms by decision + +- 2026-07-17 (M12, Derek-approved) — ma and do joined the ambiguous acronym set because both are common surnames; the two-word "Jack Ma" is kept intact by S2's words-to-spare guard, while the periods gate governs the dotted spellings ("M.A." counts unambiguously). Documented side effect: parenthesized bare + "(MA)"/"(DO)" no longer escape to suffix as in 1.x. +- 2026-09-15 #289 — AMENDS the sentence above for one spelling and leaves it standing for the other. "The two-word 'Jack Ma' is kept intact by S2's words-to-spare guard" is still true of `Jack Ma`, and false of `Jack MA`: an all-caps member of the ambiguous set inside a mixed-case name is now read as a credential with nothing to spare, so `Jack MA` gives given 'Jack', suffix 'MA' — v1's reading was last 'MA', so this is a deliberate parity break and not a restoration. The periods gate is untouched and still governs the dotted spellings. The two ledger comments that quote this sentence carry the same amendment. See #S2. +- 2026-09-18 #531 — AND `do` IS NOW SPELLING-DEPENDENT AFTER A FAMILY COMMA, which neither sentence above anticipated: it is the one member of this set that is also particle vocabulary, so `Doe, John DO` reads suffix `DO` (capitals, mixed-case name) while every other spelling attaches to the family under rules/P6. The pairing and its accepted cost are in #S2's 2026-09-18 `do` bullet. + diff --git a/docs/design/decisions/nfd-shapes.md b/docs/design/decisions/nfd-shapes.md new file mode 100644 index 00000000..74831ebf --- /dev/null +++ b/docs/design/decisions/nfd-shapes.md @@ -0,0 +1,6 @@ +# nfd-shapes — shape tests read letters, and a decomposed letter is one letter + +- 2026-10-12 (#652) — a shape regex spelled a letter `[^\W\d_]`, and `\w` does not match a combining mark, so a name typed in NFD (`inž.` as i, n, z, U+030C) ended its word at the z: `Dr. inž. Janez Novak` made `inž.` the given name, and `Janez Novak, ŽŠČ` reversed because `str.isalpha()` refuses the caps shape. Vocabulary was never affected (`_normalize` NFC-folds); only the shape path was. The bug is as old as 1.4.0, whose `period_abbreviation` had the same class, so this is a fix and not a restoration. DECIDED: a letter in the four letter-shape regexes is `_vocab._LETTER` (letter plus the general combining blocks), `config`'s `period_abbreviation` carries the same class, and the caps predicates NFC-compose a word that is not already NFC before asking `isalpha()`, and the two-letter pair test counts letters rather than code points (`composed_len`; the first version of the fix split `John Smith, MÁ` the other way, found in review). The tests as written (`str.isalpha()`, `unicodedata.is_normalized`) are C-level, so a word already composed pays no frame: `Smith, José` 193 and `Janez Novak, Žiga` 237 frames before and after, default and EVERYWHERE (measured 2026-10-12, py3.11). Output keeps the encoding written; only the reading is made encoding-independent. MEASURED MOVES the release log cannot see, against the released 2.4.0: NFD `Janez Novak, ŽŠČ` went from given `ŽŠČ` (reversed) to suffix `ŽŠČ`, and NFD `Janez Novak, Ž.Š.` went from suffix `Ž.Š.` to given `Ž.Š.`, family `Janez Novak` — the composed twin's reading, which is C1's (paired initials are a name), so the second moves AWAY from the suffix reading and toward parity. Neither is in a corpus. +- DECLINED to widen the class to every script's marks: Indic vowel signs and Arabic and Hebrew points are marks too, but rules/H2's Accepted clause already records that an abugida abbreviation never reads as a title by shape, and a general `\p{M}` would change that without a measurement. KNOWN LIMITS, left open (#665): `_INITIAL` / `_render._INITIAL` / `config` `initial` (`\w\.`) are v1-verbatim and public, and an NFD single initial (`É.`) is not initial-shaped there. No role or `initials()` result differs for a lone initial (`Janez Ž. Novak`, 17 shapes × 3 orders), but one does beside a connective: NFC `É. y É. Smith` reads given `É.`, middle `y É.`, NFD reads given `É. y É.`. The same width problem makes the decomposed Cyrillic conjunction `й` (two code points) miss the single-letter tests at `_classify.py` (`len(token.text) == 1`) and `_pieces.py` (`len(text) == 1`): NFD `John й Smith` reads family `John й Smith`. Both predate #652 and read identically on 2.4.0 and master; neither is in a corpus. +- 2026-10-13 #665 — DONE, closing both KNOWN LIMITS above. The initial shape is asked of the composed spelling: `_vocab.is_initial_shaped` NFC-composes a non-ASCII word before `_INITIAL`, as `_render` already did for its two copies (#542). Composing was chosen over widening the class because the pattern is the public v1 `initial` regex, pinned verbatim by `test_regex_sync`, and because it gives parity BY CONSTRUCTION — a decomposed word reads as the word it composes to, and a letter-plus-mark with no precomposed form has no composed twin to disagree with, so it is left as the pattern reads it — and likewise the width tests count composed code points, so a caller's one-letter connective with no precomposed form (`q́`, added to `conjunctions`) is two letters to them and joins as a longer connective does (`John q́ Smith` reads given `John q́ Smith`), as rules/P3's "composes to one character" states. The three single-letter width tests (`_classify`'s connective fork and its one-case report, `_pieces`' three-word carve-out) count letters: the classify pair asks `composed_len` only for non-ASCII text that the vocabulary membership has already admitted and the C-level `len(...) == 1` has declined, and `_pieces` composes in C before `isalpha()`. ASCII text pays no frame and the width tests cost composed text nothing (`Dr. Juan Q. Xavier de la Vega III` 366, `Smith, José` 194, `Juan y Eva Garcia` 254, `John й Smith` 255 composed, all equal before and after); the veto below costs one frame per non-ASCII word that reaches it, its matcher being called through a wrapper (`É. y É. Smith` 305 → 307, `Janez Ž. Novak` 237 → 238, `田中. 太郎` 207 → 208; measured 2026-10-13, py3.11, all composed). Kept rather than composing inline at its four call sites: a rare shape no guard counts, against four copies of the one step whose omission this entry is about (AGENTS.md's frame rule, question 4). Decomposed `John й Smith` reads given John, middle й, family Smith, where 2.3.0 through 2.4.1 read the whole name as the given name and reported given-or-family (the KNOWN LIMITS bullet above says family; measured on 2.4.0, 2.4.1 and f9b2b81c it was given, and this bullet corrects it) — a regression of 2.3.0, whose NFC lookup fold (#cjk-full-stops) let the decomposed й match the conjunction vocabulary while the width tests still counted two code points; 1.4.0 through 2.2.0 never matched it and read it as middle й, so this restores their reading. Decomposed `É. y É. Smith` reads given É., middle `y É.`, where every release from 1.4.0 read given `É. y É.`, so that half is a fix. Both checked on the 1.4.0, 2.0.0, 2.2.0, 2.3.0, 2.4.0 and 2.4.1 wheels (2026-10-13). The #320 veto beside the shape is asked of the composed spelling too (`_vocab.in_initialless_script`, ASCII answering before the matcher): decomposed hangul is conjoining jamo, which no `_NO_INITIALS` range holds, so composing the shape alone made a decomposed `씨.` an initial — `김민준, 씨.` read title `씨.` — which the first version of this fix did, caught by `test_suffix_lookup_reads_every_full_stop`. The same veto read on jamo had long let a decomposed hangul word with a period take the leading-abbreviation shape (#H2): `김. 민준` read title `김.` decomposed in every release from 1.4.0 (checked on the 1.4.0, 2.0.0, 2.2.0, 2.3.0 and 2.4.0 wheels), and reads family `김.`, given `민준` as composed now; rules/H2's veto clause states it and carries the decomposed example. An UNSPACED decomposed hangul name is still not split (docs/usage.rst's documented exception), so `김민준. 이서연` keeps differing by encoding. The guard is `tests/v2/test_parser.py`'s twin test, which now compares ambiguity kinds and `initials()` as well as roles; its docstring records which rows each of the four sites holds. + diff --git a/docs/design/decisions/normalization-fold.md b/docs/design/decisions/normalization-fold.md new file mode 100644 index 00000000..ab3e395d --- /dev/null +++ b/docs/design/decisions/normalization-fold.md @@ -0,0 +1,6 @@ +# normalization-fold — lower(), not casefold(); comparison folds harder + +- 2026-07-17/19 — vocabulary storage normalizes with lower(), NOT casefold(): casefold mutated stored spellings (κος→κοσ, großfürst→grossfürst) while lower() keeps authored forms; the accepted cost, pinned deliberately, is that ASCII-SS GROSSFÜRST no longer matches. The paired half: comparison_key()/matches() use casefold() ON PURPOSE — comparison is the one surface that wants aggressive folding; this is a release-log-classified 2.0 CHANGE from 1.4's lower() (ß and final-sigma forms compare equal only since the v2 core; v1.4 has no casefold anywhere). The 1.x line was symmetric BY DESIGN — lower() everywhere, chosen for symmetry with the parser's lc() convention — so the storage-vs-comparison split is a v2 decision moving away from a real, deliberate alternative, not a fix of an oversight. Do not + "fix" it symmetric: symmetric is what 1.x was. +- 2026-07 (rc1 arc) — the fold must reach a FIXED POINT, and anything built on it must converge too: _title_key joined per-word folds and kept empty slots, so given_name_titles with a foldable word stored a key match-time could never rebuild — silently inert, and pickle round-trips then rejected the state. Shipped in the rc, caught in review, fixed by dropping words that fold away. + diff --git a/docs/design/decisions/parse-cost.md b/docs/design/decisions/parse-cost.md new file mode 100644 index 00000000..e11d648b --- /dev/null +++ b/docs/design/decisions/parse-cost.md @@ -0,0 +1,23 @@ +# parse-cost — what a parse is allowed to cost + +Every number below is a py3.11 measurement of 2026-08-31, recomputable with `uv run python tools/perf/call_count.py` (add `--modules`, `--stages`, or `--against `). That harness is in the tree BECAUSE of this entry: its first draft took figures across several sessions with throwaway scripts on whichever interpreter was to hand, and published a module table that spliced one interpreter's before-column onto another's after-column. Four of its eight module numbers were wrong, its four stage numbers were 2x (2000-parse totals labelled per 1000, which its own arithmetic contradicted — four stages summing past the whole parse), and it blamed the wrong PR. Quote nothing here without the interpreter beside it. + +- 2026-08-31 #475 — the benchmark bounds FUNCTION CALLS per parse, not seconds, as a per-interpreter BAND of ±2%. A loose 5s wall-clock backstop stays over BOTH entry points for what frame counts cannot see: a compiled regex that starts backtracking emits no `call` event, and neither does C-level work. (A quadratic inside a comprehension or generator IS visible — `call` fires once per generator resume.) + WHY THE CLOCK HAD TO GO. The 1.0s bound failed four times across #466 and #474, always on CI's 3.12 job, always between 1.01s and 1.08s, with master re-running green each time. Three local methods disagreed with CI and with each other: uninstrumented ~1% branch-over-parent (inside the parent's own spread), coverage-instrumented indistinguishable, and the whole benchmark file under `--cov` made the branch FASTER. The same harness measured `origin/master` at 94-96ms and again at 89-92ms twenty minutes later. A check that cannot separate a 1% change from a busy runner does not fail safe; it fails expensively. + WHY A BAND AND NOT A CEILING, which is the correction the review forced. A ceiling with headroom is ANOTHER threshold that happens to break — the failure this replaced. The first draft set 470 against a 408 baseline, and measured, that catches nothing smaller than +61 calls on 3.11 and +84 on 3.12: it would have missed the very regression it was built for on four of five interpreters. The cycle's +67 arrived across a dozen PRs at roughly five calls each, and no ceiling loose enough to be safe can see five. A ±2% band trips at +5, verified by injection. A DROP trips it too, which is intended: an unexplained fall is as much a signal as a rise. + PER INTERPRETER, because the count is not machine-independent — 410 on 3.11, 388 on 3.12, 406 on 3.13-3.15 for `parse`. PEP 709 inlined the comprehension frames 3.11 counts and 3.13 added others back. The first draft claimed "the same number on any machine" and took its baseline on 3.11 by luck: had it been taken on 3.12, the 3.11 job would have gone red with nothing in the file explaining why. A version with no row fails loudly with its own number rather than passing unguarded. + WHAT THE OLD BOUND HAD CAUGHT, which is the substantive half. Measured `--against v2.1.0`: `parse` 343 → 410 and `HumanName` 380 → 447, **+67 calls per parse**; by stage, per 1000 parses, `group` 13.5 → 22.2ms, `classify` 11.6 → 13.5, `assign` 11.1 → 13.0, `post_rules` 5.2 → 8.0. The bound was not too tight. The parser had grown into it, and the clock could not say so with enough confidence to act on. + WHERE THE CALLS WENT, per parse, v2.1.0 → here: `_group.py` 94 → 84, `_pieces.py` 0 → 51, `_post_rules.py` 16 → 33, `_vocab.py` 26 → 32, `_assign.py` 21 → 18, `_classify.py` 14 → 16. + AND WHICH PR SPENT THEM, which the first draft got backwards. It named #439's predicate extraction as "the largest contributor" and built a trade-off on it. Measured across its own merge boundary, #439 cost **zero**: 404 → 404, `_group` shedding 51 exactly as `_pieces` gained them. A pure relocation, and `mechanisms.md#ONE-PREDICATE-PER-QUESTION` says as much — the sharing between `group` and `assign` predated the module. The real largest single contributor is **#424 at +23** (381 → 404), the PR that made group's chain and walk mirror assign's; then #404 at +12, with #361, #367 and #434 at +8 each. So the entry's old conclusion — that recovering the cost means reintroducing the drift #439 removed — deterred work on a PR that spent nothing while +23 went unexamined. Recovering it, if ever wanted, starts at #424's peel walks. + ONE REPAIR WAS TRIED AND REVERTED: collapsing `post_rules`' three separate role-index scans into one pass. It saves **6 calls per parse** on 3.11 (the first draft said one, which is not even reachable — `_idx` is a one-line list comprehension, so each removed call frees two frames). Reverted because it was measured against a noisy timing harness that showed nothing; at 6 calls it is 9% of the cycle's growth and worth reconsidering against the band, which can now see it. + RAISING OR LOWERING A ROW IS A DECISION, not a maintenance chore. Append here with the interpreter and the harness invocation, as above. + +- 2026-09-26 #546 -- the stages copy their state with `_state.copy_with` instead of `dataclasses.replace`, and every row moves DOWN. `dataclasses.replace` walks `fields()` and calls `__init__` on each copy, three frames on 3.11 and 3.12 and four from 3.13 where `copy_with` is one. One parse of the reference name makes 18 copies: six of the state, one each from tokenize, segment, classify, group, assign and post_rules, and twelve of single tokens, six each from classify and assign (`extract_delimited` returns the state unchanged when there is no delimiter, and `script_segment` returns early on ASCII input). That is the whole of the drop, 18 x 2 = 36 frames on 3.11 and 3.12 and 18 x 3 = 54 from 3.13 (recompute: wrap `copy_with` with a counter in the eight stage modules and parse the reference name). A field copy builds what `replace` builds for a dataclass that is decorated itself rather than inheriting the decoration, keeps the generated `__init__`, and has no `__post_init__` and no `init=False` field. `_copyable_fields` checks exactly those four, and `_COPY_FIELDS` runs it over `WorkToken`, `PendingAmbiguity` and `ParseState` at import, so a class that stops qualifying fails there and `copy_with` copies nothing else. `test_the_guard_refuses_a_class_a_field_copy_would_get_wrong` records, for each refused shape, what `replace` builds and what an unguarded copy would build instead. To mypy, `copy_with` is `from dataclasses import replace as copy_with`, which keeps the dataclass plugin's keyword and type checks at every call site; an assignment (`copy_with = dataclasses.replace`) would not, since the plugin keys on the callee's full name (measured: a misspelled field and a wrong-typed value both pass through the assignment and both fail through the import). Measured 2026-09-26 with `uv run python tools/perf/call_count.py --against e0f1a2f`, each row on its own interpreter, parse/facade: 3.11 406/443 → 370/407, 3.12 384/421 → 348/385, 3.13, 3.14 and 3.15 402/439 → 348/385. The rows drop by 40 and 58 rather than 36 and 54 because e0f1a2f already read 4 under every row, inside the band, and the new rows are set to what the harness reads now. `_LINK_BASELINE`'s 64-link clause reads 2587 → 2301 on 3.11. By stage (`--stages`, py3.11, ms per 1000 parses of the reference name): group 19.6 → 17.3, classify 13.4 → 9.8, assign 11.1 → 8.0, tokenize 8.1 → 6.6, post_rules 7.9 → 6.5, segment 2.6 → 1.7. BEHAVIOR IDENTICAL: the differential gate's report at all five baselines matches e0f1a2f's line for line apart from the path header. +- 2026-09-29 #561 -- SUPERSEDED IN PART: the 2026-09-26 bullet above, whose four conditions ("decorated itself", "keeps the generated `__init__`", no `__post_init__`, no `init=False` field) were not sufficient. `replace` builds through `obj.__class__(...)`, and a field copy skips every hook that call runs; four shapes met all four conditions and still parted: a validating `__new__`, a validating `__setattr__` on a non-frozen dataclass and a validating metaclass `__call__`, each giving `ValueError` from `replace` where an unguarded field copy built `value=-1`, and an `InitVar` with no default, which `replace` demands (a `ValueError` through 3.12, a `TypeError` from 3.13) and a field copy never sees. #561's first tightening (80554266) found only the first three, and by swapping the class's OWN `__dataclass_params__` for an inherited lookup it also let through an `__init__` borrowed from another dataclass under that dataclass's name, which the 2026-09-26 guard had refused; review caught both. So the guard is now a list of conditions a class must MEET (`_state._copy_refusals`, one reason string per unmet condition): decorated as a dataclass itself, frozen, an `__init__` generated for it (compiled from `` and named `.__init__`) that takes exactly its `init` fields, the default metaclass, no `__new__` above `object`, no `__post_init__`, no `init=False` field. It is a TRIPWIRE for a realistic edit to the three pipeline classes and says so, not a proof against any class: a forged `__qualname__` on a borrowed `__init__` in a class that is also decorated itself, or a `__class__` property, still passes, and `copy_with` copies nothing but `WorkToken`, `PendingAmbiguity` and `ParseState`, which meet every condition on 3.11 to 3.15. `_UNGUARDED_EFFECT` records for each row what `replace` builds, what an unguarded copy builds, and which conditions refuse it; `test_every_guard_condition_alone_refuses_a_recorded_shape` asserts that each condition is the only refusal of some row, so dropping one fails a test rather than contradicting this paragraph (checked by removing the `__new__` condition in a scratch copy: two failures). Call counts unchanged, the guard running at import only. +- 2026-10-10 #624 -- every row moves DOWN by 4: `parse` 308 → 304 on 3.11 and 289 → 285 on 3.12-3.15, `facade` 345 → 341 and 326 → 322, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 8e6e524a (#631's merge) and of the change, the import asserted (8e6e524a read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 51 → 49 and `_group.py` 25 → 23 -- group's leading-position scan, a generator asking `is_leading_title` per piece, gave way to `leading_titles`, which it already called, and `chain_lead`, which asks `is_prefix_piece` only of the titles. +- 2026-10-10 #625 -- every row moves UP by 3: `parse` 304 → 307 on 3.11 and 285 → 288 on 3.12-3.15, `facade` 341 → 344 and 322 → 325, each measured on its own interpreter with `tools/perf/call_count.py` in isolated copies of 505d6dec (#632's merge) and of the change, the import asserted (505d6dec read exactly the recorded rows). `--modules` on 3.11: `_pieces.py` 49 → 52, `_group.py` 23 → 23. The reference name holds a particle run (`de la`), so it pays the merge's call and one more `merge_pieces` and `joined_tags`, the chain then merging `de la` with `Vega` where it had merged the three words at once; the first cut asked the title run and the lead in the merge and again in the read, +11, and group now asks the title run once and hands both on. Every name of three or more words read at the trailing site pays the call, +1 (`John Q. Smith` 202 → 203 on 3.11, warm); a name with a particle run pays the 3 (`Juan de la Vega` 230 → 233). +- 2026-10-06 #617 -- every row moves DOWN by 10: `parse` 374 → 364 on 3.11 and 353 → 343 on 3.12-3.15, `facade` 411 → 401 and 390 → 380, each measured on its own interpreter with `PYTHONPATH= pythonX.Y tools/perf/call_count.py` against 83f4e914 (every row had sat inside its band before, the 3.12-3.15 ones at +5 over a 348 baseline). `--modules` puts all ten in group and the piece layer (`_group.py` + `_pieces.py` 142 → 132, every other module unchanged): P3's connective joins moved out of `_group_segment` into `_pieces.join_connectives`, which asks `is_conj_piece`, `is_title_piece` and `is_prefix_piece` directly where the loops had asked through one-line closures over them, two frames per test becoming one. The link pin moves with it, 2,678 → 2,479 for the 64-link name on 3.11 (`_LINK_BASELINE`), and there for two reasons, counted per function in #617's review: the closures (`conj` 133 frames, `title` and `prefix` one each) and the single-letter test, which had joined each connective's text through a generator (65 frames, one per link, more than the pin's whole band), less the one `join_connectives` frame. A deliberate move, not drift: the change was a refactor (decisions/P3, 2026-10-06). The reference rows hold no comma; a comma part holding a connective now pays the shared loop (`John Smith, Mr. and Mrs.` 262 → 269 on 3.11, warm), and one holding none skips it and costs what it did (`Smith, John` 161), the review having found the first cut charging every family-comma parse two frames per word of the part. THEN 39 MORE on every row, in the branch's /simplify pass: group ran the rootname count (about five frames a piece, its only reader the carve-out) and the shared loop over every segment of three or more pieces, connective or not, and over the corpora and the case table under three orders 0 of the 3,153 group calls with no connective left free changed a piece. Group now skips both where its frozen-set walk, which already reads every piece's `conjunction` tag, finds no connective free, as `_comma._reading_pieces` does: `parse` 364 → 325 on 3.11 and 343 → 304 on 3.12-3.15, `facade` 401 → 362 and 380 → 341, `_group.py` + `_pieces.py` 132 → 93, measured on each interpreter as above; the link pin does not move, every link in it joining. Per name on 3.11, warm: the Vega name 436 → 386, `Smith, John Quincy Adams Bob` 326 → 297, `Smith, John` and `John Smith, Mr. and Mrs.` unchanged at 161 and 269. No output moved over the #617 grids (decisions/P3). +- 2026-10-07 -- every row moves DOWN by 8 more: `_group_segment`'s last three one-line predicate closures, `prefix` over `is_prefix_piece`, `suffix` over `is_suffix_piece` and `marker` over `_is_maiden_marker_piece`, are inlined as direct calls, finishing what #617 did for `title`, `conj` and `merge`. Each closure was a frame of its own ahead of the predicate's, and the chain's two inner scans ask `prefix` once per piece they pass, so the cost grows with particle pieces (frame-budget question 1 in AGENTS.md). Measured with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y tools/perf/call_count.py`, each row on its own interpreter, parse/facade against 0c54662b: 3.11 325/362 → 317/354, 3.12, 3.13, 3.14 and 3.15 304/341 → 296/333. `--modules` on 3.11 puts all eight in `_group.py` (34 → 26), every other module unchanged. The whole of the drop is the closures, counted per function on 0c54662b (a profile hook counting `call` events whose code is named `prefix`, `suffix` or `marker` in `_group.py`): the reference name asked `prefix` 7 times and `suffix` once, `Juan` + `de la Vega ` × 8 + `Smith` asked 40 and 9 and reads 937 → 888 on 3.11 (919 → 870 on 3.12-3.15), and the 64-link name asked `prefix` once, so `_LINK_BASELINE` moves 2,479 → 2,478 on 3.11 and gains rows for 3.12-3.15 at 2,459 (2,460 on 0c54662b), each measured on its own interpreter and the first the pin has had beyond 3.11; `marker` is reached only by P5's bound-join decline and the reference name never asks it. BEHAVIOR IDENTICAL: the full suite passes, and the differential gate's report at all five baselines matches 0c54662b's line for line apart from the path header. +- 2026-10-08 #620 -- no row moves: `parse` 308 on 3.11 and 289 on 3.12-3.15, `facade` 345 and 326, measured as the #614 bullet below on 4278693d (#621's merge) and on the change alike. The 3.12-3.15 pin read 290/327 until this date: #614's /simplify took one frame off those interpreters (289/326 on 4278693d, measured 2026-10-08) and the pin stayed, inside its band, so the #614 bullet's 290 is the pin's number, not that tree's. `--modules` on 3.11 unchanged. The merged copy and its per-word exclusion test are gone, and `_chain_units` asks `chain_run_end` per particle unit and `_weighed` per word behind one; the chain itself calls `chain_run_end` with the particle test inline where it had called `is_prefix_piece` per piece, which pays for the rest. decisions/S2, this date. +- 2026-10-07 #614 -- every row moves DOWN again, on top of the closure inlining above: `parse` 317 → 308 on 3.11 and 296 → 290 on 3.12-3.15, `facade` 354 → 345 and 333 → 327, each measured on its own interpreter with `PYTHONSAFEPATH=1 PYTHONPATH= pythonX.Y /tools/perf/call_count.py` against adf6da88 (#619's merge), the import asserted. `--modules` on 3.11: `_pieces.py` 59 → 51, `_group.py` 26 → 25, `_assign.py` 9 → 10, every other module unchanged (`_vocab.py` read 37 until #614's /simplify inlined the one roman test the counting view asks per word, 2026-10-07). S2's trailing run is read once between P3's joins and the particle chain, so the chain's stop and its re-ask, the reserve's two views and assign's own reading collapse into one, and the counting view it reads through is one call on every segment that reaches the read, and its unit walk (`_counted_as_one`) runs only on a segment holding a particle past its first piece; the link pin reads 2,478 → 2,482, inside its band, and stays. A segment of fewer than three pieces never reaches the read and costs what it did (`John Smith` 152, `Smith, John` 161, `John Smith, PhD` 206 on 3.11, warm). decisions/S2, this date. + diff --git a/docs/design/decisions/phd-merge.md b/docs/design/decisions/phd-merge.md new file mode 100644 index 00000000..4b4d6d07 --- /dev/null +++ b/docs/design/decisions/phd-merge.md @@ -0,0 +1,12 @@ +# phd-merge — the "Ph. D." split + +- 2026-07 (v2 core, PR #288; recorded plan deviation #1 of the core plan) — "Ph. D." tokenizes as two words and is merged back by vocabulary (v1 fix_phd), so the spaced and unspaced spellings read alike. (Scoped by #371 below: alike everywhere EXCEPT at the head of the string, where the merge declines and `Ph. D. Van Johnson` reads title 'Ph.' where `Ph.D. Van Johnson` reads given 'Ph.D.'.) + +- 2026-08-31 #371 (the merge declines at the head) — a suffix never BEGINS a name: position outranks the vocabulary match, and `rules/S2` now states it. The merge is what made a leading credential possible at all — every other suffix-shaped word standing first already falls out as a title (H2's abbreviation clause, or TITLES membership: `Jr.`, `MD`, `Esq.`, `Sr.`) or as a name word (`PhD`, `III`), so the `Ph.`/`D.` pair was the only shape that reached the defect. It reached it by emptying the family: `Ph. D. Van Johnson` read given 'Van Johnson' with no surname at all, which is #371's report. + THIS RESTORES 1.4.0 RATHER THAN DEVIATING FROM IT — on three of the four names, and the fourth is named below. v1's fix_phd healed the pair by regex on the raw string, `\s(ph\.?\s+d\.?)`, whose leading `\s` made a head match impossible while it fired everywhere else, mid-name as readily as trailing. (A first draft of this entry said "only where it TRAILED", which is false in both directions and would license narrowing the merge to the trailing position — `John Ph. D. Smith` reads suffix 'Ph. D.' at 1.4.0 and here.) At the head v1 split it — measured on the released wheel, `Ph. D. Van Johnson` gives title 'Ph.', first 'D.', last 'Van Johnson', which is exactly what this change gives. So `expected_since_1.4.0.toml` LOSES its fix(leading-credential) rule — and READ THE DIAGNOSIS, not just the error: the gate reported it SHADOWED, not reverted. `compare.py` separates the two ("shadowed -- an earlier rule claimed every diff it would have" versus "reverted -- matches no diffing name") because the fixes differ. Three of its four names became parity and the fourth, `Ph. D., Jr.`, still diverges under `fix(comma-precomma-family)`, which sits above it. #426 is the precedent for dropping a shadowed rule rather than keeping it. A first draft of this entry generalized EXPLAINED NOTHING into "how a parity restoration announces itself", which would teach the next maintainer to delete a shadowed rule whose behavior is still diverging and the 2.0.0 and 2.1.0 ledgers GAIN one, claiming the same four names, the same five roles and the same digest. What shipped in 2.0 was a regression from applying the merge unconditionally, not a reading anyone chose. + OPENING THE NAME IS NOT OPENING A PIECE LIST, and the first draft got that wrong: a credential run legitimately opens segment 1 after a family comma (`Smith, Ph. D. Jr.`), which C1 reads as a listing rather than a name beginning. Testing `k == 0` alone broke fifteen tests and took the gate red at all three baselines. The caller passes the distinction in, since `_group_segment` cannot see which segment it holds. + ACCEPTED, and found by review rather than by design: a TITLE before the pair keeps the credential reading, because `k` is a piece index and a title is a piece. `Sir Ph. D. Van Johnson` still reads given 'Van Johnson' with an empty family — #371's own symptom, surviving one word to the left. It is 1.4.0 parity, and the rule states it as a boundary rather than deviating: "the first piece of the name" cannot be computed before this merge, since H2's abbreviation test is true of `Ph.` itself, so a scan that stepped over titles would step over the very piece being judged. rules/S2 therefore says OPENS THE STRING, which is what the code holds and what v1 held. + THE FIRST DRAFT GOT "THE HEAD" WRONG IN THE OTHER DIRECTION, and the corpus could not see it. Testing the PIECE index alone declined the merge for `"Bob" Ph. D. John Smith`: `extract_delimited` removes a quoted or bracketed clause before grouping, so the pair reaches the merge at `k == 0` with a word standing before it in the INPUT. v1 merges there (its regex needed only a preceding space), and 112 measured names broke parity — every one opening with a quote or a bracket, and not one of them in any corpus. The test is now `a[0] == 0`, the first TOKEN, which is v1's boundary exactly. + ACCEPTED: `Parser.revise(suffix="Ph. D.")` now renders 'Ph., D.'. revise() runs a full sub-parse of the string it is given, and a field value has no head for a head-position rule to consult; a second draft carved that out by requiring a name to displace, and the carve-out made the head reading depend on what FOLLOWED it — appending a maiden clause changed whether 'Ph.' was a title, which `test_a_maiden_clause_changes_nothing_else` caught. Dropping the carve-out removed the inconsistency with it. The merge exists for a credential someone TYPED after a name; a caller who writes the spaced form into the suffix field is taken at their word (Derek's call, 2026-08-31). SUPERSEDED 2026-09-06 by #511, and half of the reasoning here still holds: the MERGE is a head-position rule and still does not fire in a field value. The other half — a caller who writes the spaced form is taken at their word — is what #511 reverses (Derek's call, 2026-09-06, choosing the entry rule over the accepted split when the round-trip limit was measured). What changed is that rules/R1's entry pass now runs inside `revise()` over the forced roles, and `Ph.` and `D.` share a comma bucket with nothing between them, so the ENTRY pass joins the pair for the reason it joins `MD` and `PhD` — a different rule reaching the same two words. `Parser.revise(suffix="Ph. D.")` renders 'Ph. D.' (decisions/C1, 2026-09-06 #511). + What the leading `Ph.` becomes is H2's business, not this rule's: an abbreviation before a name is almost always a title, which is the same clause that reads `Esq. van Gogh` as title 'Esq.' — so the pair reads title 'Ph.', given 'D.'. The issue proposed given 'Ph.', middle 'D.'; H2 claims the first word before the positional read sees it, and the two answers differ only in which field holds `Ph.`. + diff --git a/docs/design/decisions/removed-v1-surface.md b/docs/design/decisions/removed-v1-surface.md new file mode 100644 index 00000000..45af61c4 --- /dev/null +++ b/docs/design/decisions/removed-v1-surface.md @@ -0,0 +1,8 @@ +# removed-v1-surface + +- empty_attribute_default: removed in 2.0 (#255; deprecated in 1.4 per the bridge discipline). Origin #44 (2016): a DB-NULL convenience whose first answer — `name.title or None` — became the migration path. The in-band-signaling bug that sealed it (#254): the 2016 `.replace('None','')` scrub could not tell interpolated None from name text, so "Nonez Smith" rendered + "z Smith" — the fix shape, worth keeping as a one-liner, is + SUBSTITUTE BEFORE FORMAT, never scrub after. Two removal cautions shaped the implementation: tombstone-not-deletion (a plain deleted attribute would make assignment silently accepted-and-ignored, the #241 failure family) and readers-not-just-writers (#44's own thread advised asserting against the attribute, so reads existed downstream). The tombstone and pickle-tolerant load die with the shim. (An earlier 3-0-reevaluations bullet said "left untyped, typeable in 3.0" — promoted from a memory that was already three days stale when written; nothing is left to type.) +- bytes input: removed in 2.0 (#245; decode-first DeprecationWarnings shipped 1.3.0 per the bridge discipline); the shipped TypeError carries the decode hint. +- no_vowels: removed in 2.0 (#268, filed 2026-07-07, closed 2026-07-28) — never consulted by any parser version, ASCII-only; the facade carries no replacement because there was nothing to replace. + diff --git a/docs/design/decisions/render-default.md b/docs/design/decisions/render-default.md new file mode 100644 index 00000000..1cbe04f8 --- /dev/null +++ b/docs/design/decisions/render-default.md @@ -0,0 +1,4 @@ +# render-default — the default view's format + +- 2026-07-18 (bf1141c) — the default spec is '{title} {given} "{nickname}" {middle} {family} ({maiden}) {suffix}'. Declined the same day: a née-template maiden ('née {maiden}') that round-tripped LOSSLESSLY (née is a maiden marker), rejected on presentation grounds. Accepted cost of the shipped form: a parenthesized maiden re-parses as a nickname. + diff --git a/docs/design/decisions/review-agent-single-source.md b/docs/design/decisions/review-agent-single-source.md new file mode 100644 index 00000000..ae2c7518 --- /dev/null +++ b/docs/design/decisions/review-agent-single-source.md @@ -0,0 +1,6 @@ +# review-agent-single-source — the review axes live in one file + +- 2026-08-31 (#473) — `.claude/agents/design-docs-reviewer.md` restated AGENTS.md's review axes and, being untracked, drifted three ways against the file it implements: axis 10 never arrived, axis 4's outward half never arrived, and and the corpus-population caution kept a wording AGENTS.md itself records as corrected. That third one carried a second defect neither copy could show: BOTH quoted a corpus measurement that decisions/P6 had already superseded, so the endorsed wording was stale too. AGENTS.md's caution is rewritten in this change to carry the lesson without a standing count. The drift was one-directional — every correction landed in the tracked, reviewed, diffable file and none reached the copy. `.claude/agents/` and `.claude/commands/` are now tracked, and the agent points at AGENTS.md rather than restating it. The general form: when a duplicate carries no independent content, delete it; a duplicate is only worth keeping when it says something its source does not. + +Declined: a test pinning the agent's axis headings against AGENTS.md's list, which is what #473 proposed. Heading comparison would have caught the missing axis 10 and NEITHER of the other two drifts — both changed body prose and left headings intact — so the test detects a third of the defect while licensing the arrangement that produced all of it. Same shape as the `STABLE_TAGS` twin flagged in `nameparser/_types.py`, and the same remedy applies there if it is ever revisited. + diff --git a/docs/design/decisions/salutation-titles.md b/docs/design/decisions/salutation-titles.md new file mode 100644 index 00000000..6c08798c --- /dev/null +++ b/docs/design/decisions/salutation-titles.md @@ -0,0 +1,10 @@ +# salutation-titles — salutations, offices and ranks for the languages TITLES had none for (2026-10-04, #606) + +Closes #606 (#662 later added owura and bapak, last bullet). No parser code moves: 41 words join TITLES and one, knt (Knight, beside kt), joins SUFFIX_ACRONYMS with a `Knt` case mask. The class is decided once — a word that addresses or ranks a person and is written in FRONT of the name — and each word is then argued on #vocabulary-collisions C-i, the position test, because TITLES has no ambiguous subset and the only two answers available are ship and do not ship. The per-word lists, grouped by language, are the #606 block in nameparser/config/titles.py. + +- **The graf class, decided (Derek, 2026-10-04).** A word borne as a name, but RARELY as the first word of one, ships anyway, on the precedent of graf, which shipped though Steffi Graf exists. This is a judgment on each word's incidence in the leading position — the position C-i asks about — and not a rule, and it is an EXCEPTION to C-i rather than an application of it: C-i is an iff with no threshold, and herra, batoni, hrabia and kreivi were measured as borne in that position, so C-i says do not ship them (TITLES having no ambiguous subset to mark them in). What the exception adds is a weighing C-i does not make, a small measured incidence against the benefit of the title reading, and it is scoped to TITLES and to the words this entry names. It licenses nothing in the particle or suffix sets, and a later TITLES word that wants it is argued on its own counts. The title claim acts only in front, so most bearers are untouched in the forms they write: `Gladys Knight` and `Knight, Gladys` keep family Knight. The cost falls on a name that BEGINS with the word, the cost `Graf Steffi` already paid: `Greve Anna` reads title Greve, family Anna, and a given name is lost the same way, `Herra Wijaya` reading title Herra, family Wijaya. Declaring `FAMILY_FIRST` does not help, since the title claim is read before the order (`Hrabia Anna` reads title Hrabia under it); the family comma does. Two further limits, both pre-existing: a trailing period makes the word the title by rules/H5 (`Gladys Knight.` reads title `Knight.`, as `Mary Jane King.` does), and every word this entry adds widens that rule's reach, H5 claiming any listed title behind a period. Accepted rather than prevented, and stated at rules/H1's Accepted clause. The issue proposed the class for three words — greve (Scandinavian Count), graaf (Dutch Count), knight. The spot-check the issue asked for (forebears.io, 2026-10-04, worldwide counts rounded) found ten more of its "no known collision" words borne, and Derek extended the decision to all of them rather than excluding them: herra (~3,000 surnames, Costa Rica and the Dominican Republic; ~670 given-name uses, Indonesia and the Philippines), batoni (~1,100, the painter Pompeo Batoni; ~300 given-name uses, Tanzania), ponas (~4,000, Iran), counsel (~730, Australia, the US, England), ponia (~560), the Filipino ginoo, ginang and binibini (~500 each, Philippine lists writing Surname, Given with a comma), and two whose registers often write the surname first, hrabia (~750, Poland) and kreivi (~270, Finland). Those last two, and the given-name uses of herra and batoni, are the leading-position bearers this class accepts, and were decided on with their counts in hand. The line is NOT reproducible from this record, and that is said rather than hidden: the accepted side was counted, while the excluded side — thiru, puan, pan, rodina, ông/bà — was judged from general knowledge of each word, as #606 drafted it, with no count taken (the count source refused further lookups the same day). Each was judged a COMMON leading-position name, against counts in the hundreds on the accepted side; gróf was excluded for want of any evidence at all. Whoever decides κόμης or the next salutation should count the excluded words first, and is free to find the line in a different place. The comment `# borne` in titles.py marks each member, so a sweep meets the decision where it meets the word. +- **Accents are not folded, and two words depend on it.** `_lexicon._normalize` lowercases and composes NFC and does nothing else to letters, so paní and härra cannot match Pani (~76,000 surnames and ~65,000 given names, India and Indonesia) or Harra (~1,100). Accent-folding the lexicon would turn both into collisions; whoever proposes it re-argues this entry. frú depends on nothing: the unaccented fru (Swedish/Danish Mrs) already ships, so the Cameroonian surname Fru (~10,800, where the surname is often written first) already reads as a title in `Fru Anna` and did before this entry; that pre-existing collision is recorded here rather than argued, being no part of #606. +- **gravos was dropped, not excluded.** The issue listed it as the Latin-script Greek for Count; the spot-check could not confirm it is the Greek word at all, which is κόμης (kómis). Nothing was shown, so there is nothing to keep out — it is recorded here so that a later sweep starts from that rather than from the issue's table. κόμης itself is NOT added and NOT excluded: it is a Greek surname (~420 bearers in Greece, official lists writing the surname first), which by its counts sits inside the class above beside hrabia, and it was left out of this change for having no use in current Greek address rather than for its collision. Adding it is a decision still to make. +- **Measurement (2026-10-04).** One corpus name moves, `Greve Anna`, given Greve to title Greve: the rules/H1 example this entry added, and the only one of its three examples that moves. Recompute: take the 41 entries of the #606 block in titles.py — the lines that are a quoted string, not the words quoted inside its comments, which name existing entries and excluded ones — parse every distinct name in the tools/differential/corpus*.jsonl glob (1,518 on the day) with the shipped parser and with `Lexicon.default().remove(titles=, suffix_acronyms={"knt"})`, and diff `as_dict()`. Before the rules.md examples were added, the same diff moved nothing over the 1,809 corpus lines of the tree before #604 merged, gravos still in the word list. The old readings the release-log bullet quotes were checked against the 1.4.0 and 2.3.0 wheels. +- **2026-10-17 #662 — owura and bapak join TITLES; malam and bay are excluded.** Akan and Indonesian Mr., found in the CLDR corpus (`Owura Bertram…`, `Bapak Dwi Putro bin Adinata MP` read the salutation as the given name). Counts from `tools/name_usage.py` on 2026-10-17: owura and bapak each under 100 US Census bearers and none found on Wikidata (a zero proves nothing, per the tool). The two exclusions are in the Excluded (TITLES) block in decisions/vocabulary-exclusions. Measurement: two corpus names move, both in corpus_cldr.jsonl and both toward CLDR's fields (`Owura Bertram Wilberforce Henry Robert Wooster MP` and `Bapak Dwi Putro bin Adinata MP`, salutation from given to title); recompute with `uv run python tools/differential/snapshot.py --since `. The release-log reading was checked against the 1.4.0, 2.0.0 and 2.4.2 wheels. + diff --git a/docs/design/decisions/script-table-placement.md b/docs/design/decisions/script-table-placement.md new file mode 100644 index 00000000..87cfd5ec --- /dev/null +++ b/docs/design/decisions/script-table-placement.md @@ -0,0 +1,4 @@ +# script-table-placement — a recorded reversal + +- 2026-07-29 (script-ranges relocation) — REVERSES #271's documented choice ("tables deliberately live in _vocab"): the tables moved to _policy because packs cannot import the pipeline, evidenced by two hand copies with sync tests and a blocked 16–30× compiled-regex optimization (a module-level re.Pattern in a pack would flip its classification, hence the closure-held-pattern convention). Rider: U+3006 〆 joined the HAN span on a justification deliberately beyond UAX #24 — it is Script=Common but appears solely in Japanese surnames (〆木, 〆谷, 〆野). + diff --git a/docs/design/decisions/suffix-acronym-collisions.md b/docs/design/decisions/suffix-acronym-collisions.md new file mode 100644 index 00000000..fbaf490c --- /dev/null +++ b/docs/design/decisions/suffix-acronym-collisions.md @@ -0,0 +1,23 @@ +# suffix-acronym-collisions — the trailing-position collision class, decided (2026-09-07, #342/#454) + +Closes #342 (a wordlist question) and #454 (a rules.md question) together, because they are the same question asked of two different words and answering one without the other would leave the criterion half-stated. No parser code moves in either. Two existing forks carry the whole thing and both were confirmed on the pre-bundle tree with a throwaway override before any wordlist was edited: `Parser(lexicon=Lexicon.default().remove(suffix_acronyms={"rai","cha"}).add(suffix_acronyms_ambiguous={"ba"})).parse(text)`. + +- **The criterion, and it is a comparison of RELATIVE FREQUENCY.** #vocabulary-collisions C-i decides whether a word is borne as an ordinary name in the position the vocabulary claim acts on, and answers ship or mark. This entry adds the third answer and says when to reach for it. Being borne at all is only C-i's entry ticket; what chooses among the three answers is how common the word is as a borne name in the TRAILING position weighed against how common it is as a credential. Three outcomes. The NAME reading dominating removes the entry from the vocabulary altogether — a caller who needs it adds it back, which is the answer this file already gives for every locale-specific vocabulary — and that is rai and cha, far more common as surnames than RAI and CHA are as credentials. Rough balance earns the ambiguous marking, both readings being genuinely live so that a flag is the honest output, and that is ba, about as common a surname as BA is a credential. The CREDENTIAL dominating leaves the entry unambiguous, which is where the great majority of the set sits. LENGTH is a CORRELATE and not the test: a short acronym is more often a common credential and more often a name, which is why the four entries carrying the marking before this one — ma, do, ed, jd — are all two letters, and reading that as the rule is what this bullet exists to stop. +- **rai and cha left SUFFIX_ACRONYMS.** RAI is "RETA Authorized Instructor"; CHA is Certified Hotel Administrator or Certified Healthcare Auditor. Rai is a common surname across Hindi- and Bengali-speaking regions and Cha is the Korean 차. Both entries arrived in the 2019-12-11 bulk wikipedia post-nominal import (af5bdab) and neither was reviewed against a surname. Measured 2026-09-07: "Aishwarya Rai" reads family Rai where 2.0.0 through 2.2.0 read suffix Rai with no family name at all; "Lala Lajpat Rai" reads middle Lajpat, family Rai; "Kim Cha" reads family Cha. This SUPERSEDES the #vocabulary-collisions sentence "rai — Rai IS a borne surname, so it earns the marking rather than moving", which has a dated supersession note there rather than a rewrite; the sentence was right that Rai is borne and wrong about which of the three answers that earns. +- **The accepted cost, stated rather than discovered.** "John Smith RAI" reads family RAI with Smith a middle name, and "Ahmad Jayadi, CHA" reads given CHA, family "Ahmad Jayadi" — with no credential after the comma, C1 reads an ordinary family comma. Both are case rows, so the reversal is one measurement away for anyone who wants it back. The judgment behind accepting it: a wrong credential reading destroys a real person's surname on every record they appear in, and a missed credential reading leaves the letters in a name field where a human can still see them. +- **The roman-numeral accident, recorded because it looks like a survival and is not.** "John Smith R.A.I." still reads suffix R.A.I. after the removal — but by ACCIDENT, not by vocabulary: rules/S3 splits a token with interior periods on those periods and reads the chunks (`_pipeline._vocab.period_joined_vocab`), and the last chunk `i` is a Roman numeral in SUFFIX_WORDS, so "John Smith X.Y.I." and "John Smith R.A.V." read suffix while "John Smith R.A.X." does not — measured 2026-09-07, `period_joined_vocab('R.A.I.')` returning "suffix" and the chunks being r, a, i, never `ra`. "John Smith C.H.A." shows the other half: it read suffix BEFORE the removal, the lookup fold stripping the periods to reach the cha entry, and reads family C.H.A. after. Nothing is pinned on the accident; it is evidence for the parking lot below. +- **ba joined SUFFIX_ACRONYMS_AMBIGUOUS.** BA is a common credential and Ba a real surname (Vietnamese; Senegalese Fula) — the ma/Ma shape exactly. Measured: "Anna Ba" reads family Ba with a suffix-or-name flag (1.4.0's reading plus the flag), "John Smith BA" keeps suffix BA and is newly flagged, "John Smith B.A." is an unflagged suffix through the period gate, and "Smith, BA" reads given BA — the "Smith, Ed" cost, already Accepted under S2 and not reopened here. No corpus name carries a trailing ba, so nothing moves in the differential. +- **mai left SUFFIX_ACRONYMS (2026-10-16, #687).** The relative-frequency criterion above, asked of a third word and answered as rai and cha were. MAI is "Member, Appraisal Institute", a specialized US real-estate designation; Mai is a Vietnamese given and family name, a Chinese surname (麥) and a Japanese given name. `tools/name_usage.py Mai`: US Census 2010 17,445 family-name bearers (rank 2,075), SSA 5,697+ given-name bearers, Wikidata 822+ family and 458+ given. The entry arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. Measured: `Tran Thi Mai` under `FAMILY_FIRST_GIVEN_LAST` read suffix `Mai` with no given name (it now reads given `Mai`, middle `Thi`, family `Tran`), and `Nguyen Mai` read suffix `Mai` under every order (family `Mai` by default, 1.4.0's reading). The accepted cost is `John Smith MAI`, which reads middle `Smith`, family `MAI`, as `John Smith RAI` does; `Smith, John, MAI` still reads suffix (reported `comma-structure`) and `John Smith M.A.I.` still reads suffix by the dotted shape (rules/S3). A caller who needs it back adds it: `Lexicon.default().add(suffix_acronyms={"mai"})`. Marking it ambiguous, the `ba` outcome, was the alternative and was not taken: a trailing `Mai` is a name far more often than a credential. One corpus name moves, `Tran Thi Mai` under `FAMILY_FIRST` (snapshot, 2026-10-16). +- **#454 is BY DESIGN, and the words are mc and vd.** mc has no vowel and is not a borne name — which reverses #342's comment, where mc is listed among the entries "borne as surnames" and is the one member of that list its model-recall hedge does not cover; #360 measured it: mc and ste are contractions of Mac and Sainte, borne in neither position, which is why both left the ambiguous particle half. The Scottish prefix never detaches from the name it belongs to, so "Donald Mc" is not a name anyone writes and its suffix reading costs nothing real. vd needs a surname after it, so a bare trailing vd is the decoration. Neither joins the ambiguous subset and the `fix(suffix-routing)` ledger rule — a two-token name ending in a credential acronym keeps it in `suffix` — stays. What #454's example actually turns on is a different membership: "Mc Donald" reads family "Mc Donald" and "John van Mc" reads family "van Mc" because mc is a never-given PARTICLE (#360), which is why the trailing bare shape is the only one at issue. +- **What #454 got right, and it is a rules.md defect that is now fixed.** rules/P6's Accepted clause said that under the default order a comma-less name keeps its positional reading, with "Jong Anke de" → family "de" as its only example. Measured, that holds for de and for do and fails for exactly the two words that are in BOTH the particle vocabulary and the UNAMBIGUOUS suffix vocabulary: "Donald mc" and "Smith vd" lose the family entirely to `suffix`. The clause is repaired to say so, and the two places that already conceded it in prose — the `'Donald mc'` bullet of #differential-ledger's fields-only arc and the corresponding comment in expected_since_1.4.0.toml — are amended from "recorded here rather than fixed" to point at the repair. No example line moves: the clause's own example is de-shaped and stays true, which is why no test caught the defect and why regenerating corpus_rules.jsonl is a no-op here. Recompute the membership with `L = Parser().lexicon; sorted(L.particles & (L.suffix_acronyms - L.suffix_acronyms_ambiguous))`, which gives ['mc', 'vd'] on 2026-09-07, and `sorted(L.particles & L.suffix_acronyms_ambiguous)`, which gives ['do']. +- **Parking lot: the shape-plus-position credential heuristic.** #490's idea without the "collides with vocabulary" part, and what would recover "John Smith RAI" without any wordlist at all. Two shapes: an ALL-CAPS acronym standing in the suffix position of a MIXED-CASE name ("John Smith XYZ", "John Smith, XYZ"), where the case contrast is the signal; and a DOTTED acronym in that position regardless of case ("John Smith X.Y.Z.", "john smith x.y.z."), where the periods are. Measured 2026-09-07, none of the four is read as a suffix today — the bare form gives family XYZ, the comma form given XYZ with family "John Smith", and the second two family X.Y.Z. — while the roman-numeral accident above reads three dotted forms as suffixes on a fork that is about generations, not credentials. That contrast is the evidence the dotted shape is unhandled: the parser already produces the wanted answer for X.Y.I. and R.A.V. by coincidence and the unwanted one for R.A.X. A follow-up issue carries it; nothing in this entry depends on it. +- **esq left SUFFIX_ACRONYMS (2026-09-08, the #316/#489 bundle).** The same criterion asked of a word the frequency argument cannot reach: does the entry describe the WORD or the machinery. Esquire is a contraction, not an initialism, so the initialism set was the wrong home for it whatever its frequency — this is the criterion's third kind of answer, and it is the one rai and cha did not need. The entry arrived with the same 2019-12-11 bulk import (af5bdab, #93) that brought rai and cha and was never reviewed either; its only unique coverage was the multi-dot spelling `E.S.Q.`, the SUFFIX_WORDS membership carrying every single-token spelling. Measured over the corpus glob: one name moves, `John Smith E.S.Q.`, from suffix to family, at every baseline. What the removal buys is the invariant this entry's neighbours could not have: `SUFFIX_ACRONYMS ∩ SUFFIX_WORDS == ∅` is asserted at import, esq having been the only member of that intersection and the reason the assert could not exist. Recompute the intersection with `L = Parser().lexicon; sorted(L.suffix_acronyms & L.suffix_words)`, which gives `[]` on 2026-09-08. The standing keep-out is the Excluded (SUFFIX_ACRONYMS — esq) block in decisions/vocabulary-exclusions. +- **Not decided here:** sa stays where #296's audit put it (title and suffix dual, position decides); se and om stay unambiguous, there being no surname evidence worth standing behind and OM being the Order of Merit; the S2 comma-form boundary ("Smith, Ed" → given Ed) is not reopened. #342's own comment lists sa, se and om beside ba and cha as model recall rather than corpus-attested, and per the #360 lesson they need a human before they move. +- **Measurement (2026-09-07).** Five corpus names move, every one from the radar-tier corpus_issues.jsonl, in three diff shapes with one cause: Aishwarya Rai moves {family, suffix}, Lala Lajpat Rai and John Smith RAI move {middle, family, suffix}, and John Smith, RAI and Ahmad Jayadi, CHA move {given, family, suffix}. Recompute by parsing every name in the tools/differential/corpus*.jsonl glob twice — once with the shipped lexicon, once with `Lexicon.default().add(suffix_acronyms={"rai","cha"})`, which RESTORES the two entries this bundle removed so the diff read forwards is the removal's — and diffing the seven name fields plus `ambiguities`; the ba step of the same sweep moves nothing. Aishwarya Rai is the one name whose diff DISAPPEARS at 1.4.0, that release having read family Rai too, which is why the `fix(#342) NOT WANTED` ledger rule was deleted rather than rewritten and why the 1.4.0 gate lists four names under the replacement where the three 2.x ledgers list five. Read today's intentional counts off the `corpus:` line of `uv run python tools/differential/compare.py --baseline X`; they rose by three at 1.4.0 and by five at each 2.x baseline. +- **2026-09-15 (#289/#516) — the parking lot is CLOSED, and this entry's "measured 2026-09-07, none of the four is read as a suffix today" sentence is superseded rather than rewritten.** The shape-plus-position heuristic this entry parked is what shipped: the DOTTED shape as `Policy.unlisted_dotted_suffixes`, default on, and the ALL-CAPS shape as `Policy.unlisted_caps_suffixes`, default off. Of the four names the bullet measured, `John Smith X.Y.Z.` and `john smith x.y.z.` now read suffix at the default, and `John Smith XYZ` / `John Smith, XYZ` read suffix only with the opt-in switch. See #S2 for the whole decision and for why the two halves have different defaults. +- **2026-09-15 (#289/#516) — the roman-numeral accident is RETIRED, and here is what narrowly means.** This entry's bullet recorded the accident as evidence rather than as behavior anyone relied on. It now fires no longer where every chunk the vocabulary matches is a single ASCII character — measured, `{i, v, 2}` — so `John Smith R.A.I.` and `John Smith X.Y.I.` reach the shape class instead and read by position, which gives the same fields for the first and 1.4.0's family reading for `Jack X.Y.I.`. `John Smith C.H.A.`, the bullet's other half, is unaffected: nothing claimed it before or after. The WIDE retirement was measured and rejected; #S2 records the three names that pay for it. +- **2026-09-25 (#540) — meng and lac joined SUFFIX_ACRONYMS_AMBIGUOUS (Derek: the marking over removal, and no masks).** #vocabulary-collisions C-i asked at the position the suffix claim acts on, the last word of a name: Meng (孟) is a common Chinese surname and given name, and Lac (Lạc) is a Vietnamese given name — the trailing word in native order, `Nguyễn Văn Lạc` — and a French surname; both are borne in that slot (`wang meng`, `tran lac`, `nguyen van lac`). MEng (Master of Engineering) and LAc (Licensed Acupuncturist) are live credentials, and neither reading is rare enough beside the other to give it the word — the rough balance this entry's criterion marks (where it is uncertain, C-i's default is the marking too). They are the set's first entries longer than two letters, which is the LENGTH clause of the criterion bullet applied, not an exception to it. Neither was checked against a surname when it arrived: `lac` with the af5bdab import (2019-12-11, #93), `meng` with 3e14ea20 (2026-07-01, the German/Dutch title and degree batch, first shipped in 1.3.0). No rule changes: rules/S2 reads the two words exactly as it reads `ma` and `ba`. Measured 2026-09-25 on this tree, the parent 120033b5 and the five released wheels (each wheel run as the Gotchas in AGENTS.md prescribe, from a directory outside the checkout under `PYTHONSAFEPATH=1`, asserting `nameparser.__file__`). THE FIX: `wang meng`, `li meng` and `tran lac` read given + family with a `suffix-or-name` report, where every release from 2.0.0 read suffix and no family name — reporting nothing at 2.0.0 through 2.2.0, and `given-or-family` at 2.3.0 and the parent, the one-word-name fork, which was the wrong fork — and 1.4.0 read the family, unflagged. The family-comma spelling returns to 1.4.0 with them, `Wang, Meng` reading given `Meng`, family `Wang` where 2.0.0 through 2.3.0 read suffix `Meng`; and under `FAMILY_FIRST` the bare `Wang Meng` reads given `Meng` where 2.0.0 through 2.3.0 and the parent read family `Wang`, suffix `Meng`, so the comma spelling under the default order and the bare one under `FAMILY_FIRST` still agree with each other. WITH WORDS TO SPARE the credential stays: `john smith meng`, `JOHN SMITH MENG` and `nguyen van lac` keep their suffix, now reported, and so does `john smith m.eng.`, which S2's period gate does not settle, that gate counting one period after each letter. DECIDED (Derek, 2026-09-25): the bare `Wang M.Eng.` reading — family `M.Eng.`, nothing to spare — is accepted and recorded rather than widening the period gate to cover a chunked dotted spelling; S2's period gate counts a member as unambiguous only when it is written one period per letter (`M.A.`), and MEng and LAc are the first members whose conventional dotted spelling is chunked instead. `John Smith M.Eng.` keeps the suffix by the words-to-spare count, unaffected. THE ACCEPTED COST is the one `ma` and `ba` already carry, and it falls on the conventional spellings: in a name written in more than one case S2 reads a member written neither in capitals nor all lower as the name even with words to spare, and MEng and LAc are written exactly that way. `john smith MEng` reads middle `smith`, family `MEng`, where every release from 1.4.0 read suffix `MEng`; `John Smith Meng` and `John Smith LAc` move the same way; and the Title-case `Nguyen Van Lac` reads family `Van Lac` where every release read family `Van`, suffix `Lac` (for this Vietnamese name the gain, the mechanism being the cost's) — only the all-lower `nguyen van lac` keeps its suffix, one case saying nothing. A CREDENTIAL RUN ending in one of them pays the same walk-stopping cost: `Mary Jones PhD MEng` reads middle `Jones PhD`, family `MEng`, and `Jane Doe MS LAc` reads middle `Doe MS`, family `LAc`, where every release read the whole run as a suffix (`PhD, MEng` and `MS, LAc` through 2.2, `PhD MEng` and `MS LAc` in 2.3); `John Smith MEng PhD` reads middle `Smith`, family `MEng`, suffix `PhD`, the unambiguous credential behind the declined pick still peeled; the mechanism is S2's own — the case signal costs a genuine suffix standing behind a name-leaning acronym, the walk stopping at the declined pick rather than continuing past it — pinned by `tests/v2/cases.py`'s `a_declined_ambiguous_pick_stops_the_walk` and, for the family-comma shape, `a_declined_member_ends_the_trailing_run` (`Doe, John MA Ma`). A QUIETER COST sits behind a suffix comma: once the run's last word is a bare ambiguous member, the run is no longer wholly suffix-shaped, so C1 reads the comma as a family comma instead and the run's own words fall into the usual post-comma name slots — `John Smith, PhD MEng` reads given `PhD`, middle `MEng`, family `John Smith`, and `john smith, phd meng` reads given `phd`, family `john smith`, suffix `meng` (the count then peels `meng` with a word to spare); neither case form saves it, `JOHN SMITH, PHD MENG` reading the same way. A title-listed word standing in front of the run reads as a TITLE instead, with no report: `Jane Doe, MS LAc` reads title `MS`, given `LAc`, family `Jane Doe`, unflagged, where every release read suffix `MS LAc`. The path is not new: `john smith, phd ma` and `Jane Doe, MS Ma` already read this way, `ma` having been ambiguous all along. Pinned by `tests/v2/cases.py`'s `comma_credential_run_ending_in_meng_re_reads_the_comma` and `comma_lower_credential_run_ending_in_meng_re_reads_the_comma`. The comma re-read (with its silent title-dual case) and whether a listed member's chunked dotted spelling should pass the period gate are [#544](https://github.com/derek73/python-nameparser/issues/544)'s questions. After a comma, `Smith, MEng` reads given `MEng` (1.4.0's reading; 2.0.0 through 2.3.0 read suffix) — and so does the all-lower `Smith, meng`, a restored reading rather than a cost — and `Smith, John MEng` middle `MEng`, pinned as case rows (`Smith, meng`, `Smith, MEng`, `Smith, John MEng`) and classified by the `fix(#540)` rules, while `John Smith, MEng` keeps the suffix on C1's name-word count; bracketed, `John Smith (MEng)` reads nickname `MEng` with a `suffix-or-nickname` report, S1's escape declining an ambiguous member, where every release read suffix `MEng`. `John Smith MENG` keeps the credential on the capitals lean. The judgment is the one behind rai and cha: a missed credential leaves the letters in a name field where a human can still see them, and a wrong credential reading destroys a surname on every record it appears in. edd and ded are NOT marked: both are borne as GIVEN names (`Edd Smith`, `Ded Gjo Luli`), which is not the position the suffix claim acts on, and no measurement shows either losing a family name — a trailing bearer turning up reopens them. Recomputed with #vocabulary-collisions' recipe on 2026-09-25: suffix_acronyms 608 (unmoved by this change) with 7 ambiguous, 5 before this change (particles 70 with 37 and titles 758, unmoved). +- **Declined 2026-09-25 (#540), with measurement: removing the two words, and masking them.** REMOVAL, the rai/cha answer, measured with `Parser(lexicon=Lexicon.default().remove(suffix_acronyms={"meng", "lac"}, suffix_acronyms_ambiguous={"meng", "lac"}))`: `john smith meng` reads middle `smith`, family `meng`, and `JOHN SMITH MENG` family `MENG`, while `john smith m.eng.` stays a suffix by its dotted shape (S3), reported. Removal loses the bare credential outright where the marking keeps it with words to spare, and C-i's default under uncertainty is the marking. MASKS `meng → MEng` and `lac → LAc`, measured with both added to a private lexicon on top of the marking: the exceptions map is role-free (#R4, 2026-09-23), so `wang meng` renders `Wang MEng`, `tran lac` `Tran LAc` and `meng li` `MEng Li` — the mask re-spells the very surnames the marking restores. A suffix-gated mask would reverse the 2026-09-23 role-free decision for two words and is not taken. Without one, the default path renders `wang meng` as `Wang Meng` (the parent gave `Wang MENG`, the credential clause writing a suffix in capitals), `tran lac` as `Tran Lac`, `john smith meng` as `John Smith MENG` and `nguyen van lac` as `Nguyen Van LAC` by the acronym clause, and keeps a writer's `MEng` as written (#R5, 2026-09-24) — `john smith MEng` gives `john smith MEng`, and `John Smith Meng` under `force=True`, the word being the family name there. Pinned by `tests/v2/test_render.py::test_a_listed_acronym_that_is_a_name_word_gets_no_mask`, whose recorded negative control carries the two masks. +- **Measurement (2026-09-25, #540).** No corpus name written before this change carries a bare trailing `meng` or `lac` (the `tools/differential/corpus*.jsonl` glob at 120033b5), so the population that could move is this change's own shape-tagged case rows, and every mover is one of them. Classified by `fix(#540)` rules: at each 2.x baseline `wang meng` and `tran lac` move {family, suffix, _ambiguities}, `john smith MEng` and `John Smith MEng PhD` move {middle, family, suffix, _ambiguities}, and `john smith meng` and `nguyen van lac` `_ambiguities` alone; `Smith, John MEng` moves {middle, suffix, _ambiguities} at 2.x and {middle, suffix} at 1.4.0; `Smith, meng` and `Smith, MEng` move {given, suffix, _ambiguities} at 2.x and nothing at 1.4.0 (1.4.0's own reading); `John Smith, PhD MEng` moves {given, middle, family, suffix, _ambiguities} and `john smith, phd meng` {given, family, suffix, _ambiguities} at 2.x (without `_ambiguities` at 1.4.0); `Nguyen Van Lac` moves {family, suffix, _ambiguities} at 2.x and {family, suffix} at 1.4.0; and `Wang M.Eng.` moves {family, suffix, _ambiguities} at 2.x only, 1.4.0 reading the same family for an unrelated reason (its two-piece rule -- a lone word after the given name is the family -- not S2's gate). `nguyen van lac` also diffs at 1.4.0 on `_initials` (`n.` → `n. v.`) for fix(#385/#402)'s reason — its family `van` is a one-particle part — and joins that rule's literal list; the same case-insensitive regex also reaches `Nguyen Van Lac` at every baseline, without explaining anything there (its own family move, at 1.4.0 too, is explained by the Title-case Lac rule instead). Read today's intentional counts off the `corpus:` line of `uv run python tools/differential/compare.py --baseline X`; this change raised them by seven at 1.4.0 and by thirteen at each 2.x baseline. +- **2026-09-27 (#544) — #540's two pending questions are answered, and the #540 bullet stands as it landed.** The chunked dotted spelling passes S2's period gate — `Wang M.Eng.` reads suffix 'M.Eng.' as `Wang M.A.` does, and as 2.0.0 through 2.3.0 read it — and the comma is kept: a credential run after a full name that ends in a member is the credential run by C1's name-word count (`John Smith, PhD MEng` suffix 'PhD MEng'; `Jane Doe, MS LAc` suffix 'MS LAc' where #540 left title 'MS', given 'LAc', at the cost, accepted under S2, of `John Smith, Ms Ma` reading suffix 'Ms Ma'). The walk-stopping cost that bullet lists for a run ENDING in a member goes with them, by S2's company clause: `Mary Jones PhD MEng` and `Jane Doe MS LAc` read the whole run as a suffix again. Where the member LEADS the run nothing speaks for it, so `John Smith MEng PhD` keeps its cost. The `fix(#540)` ledger rules for the comma run and the dotted spelling are retired. `fix(#544)` rules classify the comma-run names; `Wang M.Eng.`, whose roles every release from 2.0.0 read, is classified by `feat(#449)`'s given-or-family report rule at 2.0.0 through 2.2.0 and by the dotted `fix(suffix-routing)` rule beside `Jack M.A.` at 1.4.0, and differs from 2.3.0 not at all. The case rows the #540 bullet cites were renamed with the readings they pin: `a_declined_ambiguous_pick_stops_the_walk` is `a_credential_in_front_anchors_a_declined_pick` (`abdul Smith Jr Ma`, suffix 'Jr Ma'), and `comma_credential_run_ending_in_meng_re_reads_the_comma` and `comma_lower_credential_run_ending_in_meng_re_reads_the_comma` are `comma_credential_run_ending_in_meng_keeps_the_suffix_comma` and `comma_lower_credential_run_ending_in_meng_keeps_the_suffix_comma`. The decision and its measurements are the #544 entry under S2. + diff --git a/docs/design/decisions/suffix-field-composition.md b/docs/design/decisions/suffix-field-composition.md new file mode 100644 index 00000000..3274346f --- /dev/null +++ b/docs/design/decisions/suffix-field-composition.md @@ -0,0 +1,17 @@ +# suffix-field-composition — three kinds of thing in one field + +- 2026-08-16 (suffix keystone; #326) — measured composition of suffix_words (40 entries): 11 generational (i, ii, iii, iv, v, jr, jnr, sr, snr, junior, 2), 5 neither (dr, esq, esquire, ret, vet), and 24 POSTNOMINAL HONORIFICS — 20 CJK (さん, さま, くん, ちゃん, 様, 殿, 氏, 先生, 博士, 教授, 女士, 小姐, 씨, 양, 군, 님, 박사, 박사님, 교수님, 선생님) and 4 Hebrew (ז"ל, ז״ל, שליט"א, שליט״א). The honorifics are the LARGEST group. +- Decision: do NOT split the field. Record the composition; #296, + #291, #325 and #289 proceed on their own terms rather than + waiting on it. +- Why #326 cannot be taken at face value: it argues the split is tractable because "the vocabulary is already split — suffix_acronyms is credentials, entirely; suffix_words is generational plus a handful". That was true when written (2026-08-02); the 2.1.0 East Asian work landed 2026-08-07 and put the 24 honorifics in the same set. CLDR's two buckets do not cover what the set now holds, so "adopt CLDR's model" is not available as the cheap answer. The issue's table is corrected on the issue. +- The unexamined question this leaves: whether the honorific bucket belongs in `suffix` at all. rules/W3 already argues an honorific is "no part of the name on either side" — the same language the H section uses for prenominal titles — so the conflation with PhD may be worse than the generation/credentials one #326 was filed about. Deliberately not decided here. +- Recorded as a documentation failure mode, not just a fact: `suffix` started generational, absorbed credentials, then absorbed postnominal honorifics, each step locally reasonable and none recorded as a widening of the field's MEANING. rules/S2's Background still calls it "two different things", which was accurate when written. The rules doc pinned the behavior faithfully; what slipped is the field's definition, which no rule states because no rule owns it. Field definitions need the same discipline rule statements get. +- The 11/5/24 split above is this entry's own version of the hazard it describes: a quoted vocabulary composition, dated, exactly as + #326 quoted one. A date does not stop rot — #326 carried one too + — so before relying on the split, check the set still looks like it: + + uv run python -c "from nameparser import Parser; print(sorted(Parser().lexicon.suffix_words))" + + What is durable here is the SHAPE of the finding — three kinds of thing, the honorifics the largest — not the three integers. A test asserting the counts is deliberately not the answer: that is the constant-content pattern, and it would fail on every legitimate vocabulary addition. + diff --git a/docs/design/decisions/two-input-invariants.md b/docs/design/decisions/two-input-invariants.md new file mode 100644 index 00000000..da1dfc33 --- /dev/null +++ b/docs/design/decisions/two-input-invariants.md @@ -0,0 +1,12 @@ +# two-input-invariants — checking a rule against something that is not the code + +- 2026-08-31 (#473; earned by #461 and #466) — AGENTS.md axis 11. Axes 1–10 compare a document to another document, to its own examples, or to the implementation, so all ten can only find code that disagrees with a rule; none can find a rule that is wrong. #461 was backed out and #466 closed unmerged for exactly that reason, a month apart, each after a review round that found real defects and missed the premise beneath them. An invariant relating two DIFFERENT inputs consults no rule statement and is not blind that way. + +- The worked invariant: for a name written `Family Given Particle`, the family-comma writing under the default order and the comma-less writing under FAMILY_FIRST must parse alike. Measured 2026-08-31 on 9f1d46e, comparing the non-empty role fields over `nameparser.config.particles.PARTICLES` × six families × six givens: it holds on all 36 pairs for 67 of the 70 particles, and fails on all 36 for `do`, `mc` and `vd` — precisely the words that are also suffix vocabulary, which rules/P6's Accepted consequence already names as the contested memberships. The invariant found that set from behavior alone, having read no rule. + +- Recompute: for each P in PARTICLES compare the non-empty role fields of `Parser().parse(f"{F}, {G} {P}")` against `Parser(policy=Policy(name_order=FAMILY_FIRST)).parse(f"{F} {G} {P}")` across any spread of families and givens that are not THEMSELVES particle vocabulary. That restriction is load-bearing, not hygiene: with F or G drawn from PARTICLES the correspondence fails for every particle, because the spread word joins the particle run and the invariant is then measuring a different shape (`Doe, Van de` gives given "Van" middle "de"; `Doe Van de` under FAMILY_FIRST gives given "Van de"). Measured, that is 0 of 70 rather than 67 of 70, so a reviewer who picks a particle word as a family — a natural choice when probing particle behavior — would report P6 broken. The argument survives the digits moving: what carries it is that the failures partition onto the particle-and-suffix words, not how many pairs there are. + +Declined: #473's "0 of 216 → 216 of 216" framing — but not re-argued here, because decisions/P6 already records that number's provenance (a script slicing `sorted(particles)[:14]`, which excluded exactly do, mc and vd), already names it axis 2 in its purest form, and already closes "Recompute over the WHOLE vocabulary, never a sample." Cite that entry for why the figure cannot be trusted; a second account of it in this file is the duplication decisions/review-agent-single-source was written to stop. #473's companion "72 of 216 — the ambiguous-particle half" measures a DIFFERENT partition of the vocabulary from the one above and must not be read as agreeing with it: the ambiguous particles are the 37 that are not never-given, and decisions/P6 records that #466 failed because a never-given test at the new site would have excluded all 37, `van` among them — not because of the three particle-and-suffix words. Two partitions, two different failures. + +Resolved: #469's invariant landed as `tests/v2/test_order_correspondence.py` — a generated parametrized sweep over a small deliberately-forked vocabulary (never-given/ambiguous/multi-token particles, an ASCII and a non-ASCII family, doubled middles), plus a second, non-parametrized test sweeping the entire shipped particle vocabulary (`nameparser.config.particles.PARTICLES`) in the minimal pair and asserting the disagreeing set is exactly `{do, mc, vd}` — the negative control this entry's Recompute bullet describes, now executable rather than a recipe to rerun by hand. + diff --git a/docs/design/decisions/v1-xfail-triage.md b/docs/design/decisions/v1-xfail-triage.md new file mode 100644 index 00000000..a3a147f4 --- /dev/null +++ b/docs/design/decisions/v1-xfail-triage.md @@ -0,0 +1,19 @@ +# v1-xfail-triage — the eight inherited aspirations, four retired and four kept (three of those eight since fixed, 2026-09-08 and 2026-09-23) + +The reconciled v1-style banks (`tests/test_*.py`) carried eight `@pytest.mark.xfail` tests inherited from v1, each an aspiration nobody had judged since. `xfail_strict = true`, so all eight were live claims that 2.0 still does not do the thing — but a bare marker says only that, never whether the thing is WANTED, and that is the gap this triage closes. After it, every surviving marker cites an issue and every retired one is a deliberate pin of current behavior. The parses quoted below are this entry's evidence and its own recompute: + + uv run python -c "from nameparser import HumanName; [print(repr(s), repr(HumanName(s))) for s in ('Maier, Amy I, Jr.', 'Dr King Jr', 'King, Dr Jr', 'Ahmad ben Husain', \"The Right Hon. the President of the Queen's Bench Division\")]" + +- 2026-09-01 (Derek's triage) — NOT FIXED, four. The aspiration is rejected, the marker is gone, and the test now pins what the parser does with the grounds recorded beside it so nobody re-derives the rejected proposal: + - `Maier, Amy I, Jr.` — v1 wanted suffix "I, Jr."; it reads given Amy, middle I, family Maier, suffix "Jr.". A middle initial `I` is far more common than an ordinal I borne without a Sr./Jr.-style companion, so where an explicit suffix comma has already named the suffix, the trailing I stays a middle initial. That is #vocabulary-collisions' commonality reasoning applied to a SHAPE rather than to a word — the criterion is written per-word, and the extension is named here rather than smuggled in. **This amends rules/C1 in the same diff.** C1's one-character clause said a bare single letter behind a name word is the generation, full stop, which predicts exactly the suffix "I, Jr." being rejected here. The discriminator the parser has always applied and C1 omitted is the FURTHER COMMA: `Maier, Amy I` reads suffix 'I' while `Maier, Amy I, Jr.` reads middle 'I', and `Smith, John V` against `Smith, John V, Jr.` is the same pair on a name C1 already exemplifies — all four measured. So the rule was incomplete, not the behavior: a rules.md defect rather than a parser one, which is the call this file already made for `'Donald mc'` in the fields-only arc entry (2026-08-27, #451). The qualifier is normative prose carrying no example line of its own; C1's Accepted block records why and names the bank pair as its executable witness. + - `Dr King Jr` — **FIXED 2026-09-08 (the #316/#489 bundle), and this is the one bullet of the four that moved out of NOT FIXED.** v1 wanted title 'Dr', family 'King', suffix 'Jr', and that is what the parse gives; the leading title peel now leaves a name word a suffix cannot be (#H3). The bullet is amended in place rather than moved, because the half it DECIDED is untouched and is still the reason the shape exists at all. What follows is the 2026-09-01 record, with the reading it describes corrected where the fix falsified it. It read title 'Dr King', family 'Jr' then. `king` stays in the titles vocabulary: it is there for the addressing forms ("King Charles"), and taking it out to serve the surname reading trades a common use for a rarer one, which is the direction #vocabulary-collisions cuts. TITLES has no ambiguous subset and no AmbiguityKind, so — as with MAIDEN_MARKERS and `roz` — the only two expressions of the criterion available here are ship and do not ship; #348 is the open work that would give this set a third answer. The comma format is the road to the surname reading, and is now pinned alongside rather than left as prose — `King, Dr Jr` reads title 'Dr', family 'King', suffix 'Jr'. The test cites [#27](https://github.com/derek73/python-nameparser/issues/27), which is closed; this is the decision it never got. + Two halves, and only one was decided. DECIDED: `king` stays in TITLES, which is unchanged and is what still puts `King` inside the title run in the first place. RECORDED, not endorsed: what becomes of the leftover `Jr`. rules/S2 predicts suffix 'Jr' with an empty family — its Accepted clause consumes an unambiguous suffix even when nothing is left to be the family (`Smith Jr.` → family "") — but once the title chain had taken `Dr King`, H1 claimed the one remaining word and it read family 'Jr', suffix ''. `Dr Smith Jr` isolated the cause: family 'Smith', suffix 'Jr', exactly as S2 states. S2 carried a descriptive note saying so. This entry then said that a future change moving `Dr King Jr` toward S2's prediction is an IMPROVEMENT and updates the pin — that change is #H3's peel floor and it landed 2026-09-08. The reading is now title 'Dr', family 'King', suffix 'Jr': the run gives back its last word where everything behind it is post-nominal and that word is not itself suffix vocabulary, so `Dr King Jr` reads like `Dr Smith Jr` rather than against it, and S2's note is replaced by the argument it asked for. The pin in `tests/test_suffixes.py::test_king` moved with it, by decision rather than by drift, and `king` being title vocabulary now buys the row a `title-or-name` report (#H4) rather than a different reading. + - `Ahmad ben Husain` — v1 wanted family "ben Husain"; it reads given Ahmad, middle ben, family Husain. Already decided in v0.2.5, when `ben` came out of the prefixes, and for the reason that still holds: `ben` collides with the given name Ben, in the position the particle claim would act on — `Ahmad Ben Husain` reads middle 'Ben' today, which is exactly the token a forward-joining particle claim would take. That is C-i's position test, and it keeps `ben` out. Recorded a second time as a standing keep-out in the particle-line Excluded block in decisions/vocabulary-exclusions, because that is where a wordlist sweep meets it: a keep-out that lives only in a triage entry is one the next Arabic/Hebrew patronymic-particle sweep never reads. Worth naming as a failure mode of its own — the marker was an aspiration that outlived its own resolution, and nothing in a bare xfail says which of the eight were like that. + - `The Right Hon. the President of the Queen's Bench Division` — v1 wanted the whole string as one title; it reads title "The Right Hon. the President of the Queen's Bench", family 'Division'. This is a name parser, not a title parser: handed an input that is all titles it assumes the last title-word is the name. Accepted as convention rather than defended as correct — and since 2026-09-07 the guess is no longer silent: rules/H4 states the convention and the parse reports `title-or-name`, which is what [#491](https://github.com/derek73/python-nameparser/issues/491) asked for and is a report rather than a change of reading. See decisions/H4. +- 2026-09-01 — FIX CANDIDATES, four. Each carries its issue, so `pytest -rx` names the work instead of listing anonymous aspirations. Two markers remain, both #490's (re-counted 2026-09-23, #492); #489 was fixed 2026-09-08 and #492 on 2026-09-23 and their markers are gone, which is why this line no longer says the marker stays: + - [#489](https://github.com/derek73/python-nameparser/issues/489) — **FIXED 2026-09-08 (the #316/#489 bundle).** `Her Majesty Queen Elizabeth` should address by given name (`tests/test_conjunctions.py::test_conjunction_in_an_address_with_a_first_name_title`), and it does: a title RUN addresses as its LAST title does, `queen` being a given-name title (#H1). The xfail marker is removed and the test is an ordinary passing pin. + - [#490](https://github.com/derek73/python-nameparser/issues/490) — `E.T. Smith` (`tests/test_conjunctions.py::test_two_initials_conflict_with_conjunction`) and `U.S. District Judge Marc Thomas Treadwell` (`tests/test_titles.py::test_chained_title_first_name_title_is_initials`). One issue for two tests deliberately: each test's own comment names the other's shape as what blocks a fix — dotted initials against dotted title and credential vocabulary — so they are one question, and fixing either alone is what has failed before. + - [#492](https://github.com/derek73/python-nameparser/issues/492) — `capitalize()` leaves `juan garcia III` lowercase (`tests/test_capitalization.py::test_capitalization_exception_for_already_capitalized_III_KNOWN_FAILURE`). The `_KNOWN_FAILURE` suffix is kept: it is still true, and beside the annotated reason it reads as redundant rather than misleading. +- 2026-09-23 [#492](https://github.com/derek73/python-nameparser/issues/492) — FIXED (the #459/#492/#478 bundle): `juan garcia III` repairs to `Juan Garcia III`, R5's gate leaving the suffixes out (decisions/R5, 2026-09-23). The marker is gone and the test is renamed `tests/test_capitalization.py::test_capitalization_exception_for_already_capitalized_III`, the `_KNOWN_FAILURE` suffix the bullet above kept being no longer true. Two markers remain, both #490's, and the marker counts above — this entry's heading, the FIX CANDIDATES line and the #485 bullet — were amended in place on this date, following the #489 fix's precedent. `tools/differential/corpus.jsonl` still carries the old test name as a label: that file is v1's own suite harvested at a pinned ref (tools/differential/README.md), so the label names where the row came from and is left as it is. +- [#485](https://github.com/derek73/python-nameparser/issues/485) is superseded. It proposed prefixing corpus labels with `xfail:` so a radar diff on a name like `Dr King Jr` reads as a known-bad parse improving rather than a regression. The premise was that a v1 xfail marker is a usable triage signal; after this triage it is not, because the marker no longer tracks the disposition — `Dr King Jr` is now a PIN, so a radar diff on it means a decided reading moved, which is precisely the signal the prefix would have suppressed. The four that remain carry issue numbers, which is the same information in a place that cannot go stale against the pinned historical ref. The 2026-09-08 fix STRENGTHENS this rather than unsettling it, and the argument is worth stating because the fix looks at first like the counterexample: `Dr King Jr` moved, so a diff on it is exactly what #485's prefix predicted — but it moved by DECISION, with a ledger rule and a rewritten pin, which is a classified change and not a known-bad parse quietly improving. Had the prefix shipped, that diff would have read as the aspiration finally coming true and nobody would have been asked to classify it. Two markers remain, both #490's (re-counted 2026-09-23, #492), and they still carry issue numbers. + diff --git a/docs/design/decisions/vocabulary-collisions.md b/docs/design/decisions/vocabulary-collisions.md new file mode 100644 index 00000000..a6a4a4bb --- /dev/null +++ b/docs/design/decisions/vocabulary-collisions.md @@ -0,0 +1,45 @@ +# vocabulary-collisions — when a word earns the ambiguous marking + +The mechanism shipped twice before anyone wrote down its criterion. Sizes as of 2.2.0dev: particles 67 with 39 ambiguous (58%, PARTICLE_OR_GIVEN); suffix_acronyms 613 with 4 ambiguous (0.65%, SUFFIX_OR_NAME); titles 711 with no ambiguous subset and no AmbiguityKind at all. Those counts are evidence from the decision date, not live facts — they drift with every vocabulary addition. The argument does not depend on the digits (it depends on the two shares differing by orders of magnitude), but recompute before quoting them: + + uv run python -c "from nameparser import Parser; L=Parser().lexicon; print({s: len(getattr(L,s)) for s in ('particles','particles_ambiguous','suffix_words','suffix_acronyms','suffix_acronyms_ambiguous','titles')})" + +Recomputed 2026-09-07 with that recipe, after this section's own #342 decision landed: particles 70 with 37 ambiguous, suffix_acronyms 610 with 5 ambiguous, titles 758. All THREE of the snapshot's figures were audited against the released 2.2.0 wheel, not the acronym one alone — run the recipe under a PEP 723 script pinning nameparser==2.2.0, from a directory outside the checkout so the tree cannot shadow the wheel, and it reports particles 70 with 37 ambiguous, suffix_acronyms 612 and titles 706. So the snapshot's particles and its titles were off at their own date exactly as its 613 was, and the one that matters to a reader subtracting the two rows is the particles: they have NOT moved since 2.2.0, where "67 with 39" against today's "70 with 37" reads as three additions and two demotions that never happened. What did move: rai and cha left the acronyms and ba joined the ambiguous subset (#suffix-acronym-collisions), and the titles gained 52 over the wheel. The snapshot above stays as written, being evidence from its own date; the argument survives either set of digits, since the two shares still differ by orders of magnitude. + +- 2026-08-16 (collision keystone; #348, #360, #342, #385) — the 58%-vs-0.65% gap is BASE RATE, not disagreement. Both sets apply the same test; most particles are short words that double as names (van, bin, le, do, bar, mac) while most credential acronyms are not (abpp, acp). Recorded because the gap reads as an inconsistency and is not one — a reviewer who "harmonizes" the two shares will break one of them. +- **C-i, vocabulary vs. name.** A word belongs in its set's ambiguous subset iff it is borne as an ordinary name IN THE POSITION THE VOCABULARY CLAIM ACTS ON. Existence of a bearer anywhere is not the test, and the first draft of this criterion (recorded 2026-08-16, corrected 2026-08-17) got that wrong. Under uncertainty, default to AMBIGUOUS. This generalizes the evidence standard already stated in NON_GIVEN_NAME_PARTICLES' docstring — a wrong unambiguous claim misparses a real person, a wrong ambiguous marking only adds a flag — from "which set" to "which subset". Applies uniformly to particles, suffix_acronyms and titles. +- 2026-10-04 #606 — C-i has ONE recorded exception, for TITLES only: #salutation-titles ships words measured as borne in the leading position at small incidence (herra, batoni, hrabia, kreivi), a weighing C-i itself does not make. It extends to no other set and no other word without its own counts. +- **C-ii, vocabulary vs. vocabulary.** Where two sets claim a word and NEITHER reading is a name, precedence is a frequency judgment recorded per word. vd is the live case: never-given particle AND credential acronym (the British Volunteer Decoration), neither of them a name. Decision: the Dutch van der reading, as the more common. That is what unblocks #380, whose trailing-orphan half is a separate decision recorded under its own rule. +- 2026-08-17 — C-i CORRECTED: the position qualifier. Writing the + per-word records for the never-given particles falsified the first + draft on the set's most load-bearing member. "De" is a borne Bengali + and Odia surname (parse("Bimal De").family == "De"), so C-i as first + written said mark `de` AMBIGUOUS -- which would break "de la Vega" + and every leading-particle reading. Nothing had tested the criterion + against `de`, and the error pointed at the safe side, so it would + never have announced itself as a misparse. + The qualifier the existing set was already obeying: P1 acts on the + LEADING position or a lone particle in the GIVEN role, so what + matters is whether the naming use occupies THAT position. + van Vietnamese Văn in given position collides ambiguous + bar Bar Refaeli, given position collides ambiguous + do Đỗ leads a surname collides ambiguous + de "De" is a TRAILING surname no clash never-given + Consequence, and the reason this is not merely tidier: `das` was + called ambiguous on #360 under the old reading because Das is a + borne Bengali surname. Measured, "Anjali Das" and "Bimal Das" are + unchanged by never-given `das` (the surname is trailing, the rule + acts leading) while "Maria das Neves" GAINS its particle -- + family="Neves" today, family="das Neves" with it. The old reading + would have declined a fix. `lo` and `el` need re-judging on the same + axis; `Lo` does lead in romanized Chinese ("Lo Wei"), so it may + genuinely collide where `das` does not. + Corrected on #360, which carried the wrong table publicly. +- The concrete demonstration is "do", which three vocabularies claimed — titles, particles/ambiguous, suffix_acronyms/ambiguous. Two mark it ambiguous; the third, TITLES, was the one that actually decided "Do Quang Minh" (title="Do", given="Quang") and reported nothing — until #296's audit (2026-08-23) took it out of TITLES, since when the name reads given "Do" and reports the fork. Same word behind #385's "Anh Do". +- Applications, each still its own work: #360 (mc, ste — neither is a borne given name, so both leave the ambiguous half); #342 (rai — Rai IS a borne surname, so it earns the marking rather than moving); #385 (do — resolved at decisions/R2). (#342's clause — "rai — Rai IS a borne surname, so it earns the marking rather than moving" — is SUPERSEDED 2026-09-07: Rai is borne, and what that earned was removal from the vocabulary rather than the marking. See #suffix-acronym-collisions for the relative-frequency criterion that decides between the two.) +- C-ii's per-word framing versus a rule stated for a SHAPE: measured 2026-08-16, the words that are both particle and suffix vocabulary are vd, do and mc — three, not the one this criterion adjudicated. rules/P6 states its precedence for the shape, so do and mc inherit vd's answer without being weighed. Recorded rather than papered over: stating a per-word judgement as a general clause is how an unexamined word acquires a decision, and the two are named here so the next reader knows which one was actually argued. +- 2026-10-04 #604 — five particles, judged by C-i one at a time. NEVER-GIVEN: `ó` (Irish "descendant of") and `ní` ("daughter of") — grammatical patronymic particles, no given-name use in the position P1 acts on, and the measured gain is the leading and mid-name surname both (`Liam Ó Murchú` and `Ó Murchú` read family `Ó Murchú`, where today Ó was a middle initial or the given name). AMBIGUOUS: `ua`, the older spelling of `ó` — no given-name use was found, but the search was thinner than for `ó`/`ní`, and C-i says default to ambiguous under uncertainty. Measured 2026-10-04, the two markings agree mid-name (`Seán Ua Buachalla` reads family `Ua Buachalla` either way) and differ in three places. Where `Ua` opens the name: the leading `Ua Buachalla` keeps given `Ua` and reports particle-or-given rather than becoming surname-only, and `Ua Buachalla Seán` under FAMILY_FIRST keeps family `Ua`, given `Buachalla` and reports. Where it opens the part after a family comma (P1's after-comma clause): `Seán, Ua Buachalla` reads given `Ua`, middle `Buachalla`, family `Seán`, where never-given would give family `Seán Ua Buachalla`. And where it ends the name (P6): `Seán, Seán Ua`, and `Seán Seán Ua` under FAMILY_FIRST, report particle-or-given only while `ua` is ambiguous, the fields being the same. Recompute: parse each with `Lexicon.default()` against `Lexicon.default().remove(particles_ambiguous={'ua'})`. The cost is small because `Ua` is the rarer spelling, but it is not leading-only, which matters to anyone re-pricing the marking under #360. `binti` and its Singapore spelling `binte` (Malay "daughter of") — ambiguous like `bin`, Binti being a Swahili given name ("daughter"); `Ina binti Navalamar` reads family `binti Navalamar`, `Binti Jua` keeps given `Binti` and reports. The one-letter `ó` needed a rule of its own against the initial it spells: rules/P7, decisions/P7. +- Caution when applying C-i to the particle set: TITLES ∩ ambiguous — {do, freiherr, st} until #296's audit took 'do' out of TITLES (2026-08-23; `Do Quang Minh` reads given 'Do' with the fork now), {freiherr, st} since — is load-bearing, per the particle-line Excluded note in decisions/vocabulary-exclusions. Emptying it makes the particle-or-given emitter dead code. + +Open: [#348](https://github.com/derek73/python-nameparser/issues/348) applying C-i to the 711 title entries, then titles_ambiguous plus a TITLE_OR_GIVEN kind. Blocked on data, not on judgement — the census needs a given-name frequency corpus this repo does not have, which is why the criterion is recorded here and the census is not attempted. + diff --git a/docs/design/decisions/vocabulary-exclusions.md b/docs/design/decisions/vocabulary-exclusions.md new file mode 100644 index 00000000..ec9626a7 --- /dev/null +++ b/docs/design/decisions/vocabulary-exclusions.md @@ -0,0 +1,111 @@ +# vocabulary-exclusions — words kept out of a vocabulary set, and contested memberships + +The `Excluded:` and `Open:` blocks keyed by a vocabulary set rather than a rule (the entry conventions in README.md). Until #713 they sat in decisions.md after whichever entry happened to precede them, so the split filed them into the P3, W2, P5 and R4 entries; they moved here together, in their old order, so a wordlist editor finds every set's prohibitions in one place. + +Excluded (Lexicon.conjunctions_ambiguous, the marked half of nameparser/config/conjunctions.py — an entry here reads as an INITIAL in a name written wholly in one case): + +- `y` — the commonest Hispanic double-surname link and this library's oldest fixture ("Velasquez y Garcia"); marking it would read the surname link of every lowercase Hispanic record as a middle initial. A caller who wants the Dutch reading adds it (2026-09-13, #383/#479). +- Cyrillic `и`, `і`, `й` — #267 blessed their joining and this change narrows nothing about it; marking one would break the reading #267 adjudicated, in the one-case spellings where nothing else decides (2026-09-13, #383/#479). +- `و` and any other caseless letter — inert rather than harmful, and excluded so nobody adds one believing it does something: the fork requires a token whose `upper()` and `lower()` differ, so a caseless member is never consulted (2026-09-13). +- Multi-letter entries (`and`, `та`, `και`) and `&` — the fork tests a single CHARACTER, so an entry longer than one letter is silently inert. `nameparser/config/conjunctions.py` asserts against it at import (2026-09-13). + +Excluded (Lexicon.conjunctions — nameparser/config/conjunctions.py): + +- `a` — the Māori link in `Te Awanui-a-Rangi` is the one hyphen-interior connective the #478 rule (rules/R4) leaves capitalized under force, `Te Awanui-A-Rangi Black`, and it must not be fixed by adding `a` here: a spaced `a` is an initial (`John A Smith`) or a name word elsewhere, and a connective entry would join or lowercase it in every one of those names. The non-reach is recorded under R4 with that name as its witness (2026-09-23, #478). + +Open (Lexicon.conjunctions — contested membership; the issue is canonical): none. The block held one entry, [#397](https://github.com/derek73/python-nameparser/issues/397) — whether Catalan `i` belongs in the conjunctions at all — answered 2026-09-20 by decisions/P3's bullet of that date: it does, and it joins `conjunctions_ambiguous` in the same change, which was never the contested half. The heading stays with nothing under it on purpose, the same reason an empty ledger roster section is a statement and a missing one is nobody having looked. + +Excluded (the never-given / ambiguous particle line, nameparser/config/particles.py — #360 owns the vocabulary question): + +- Only 9 of the ambiguous members were individually justified at #360's audit (2026-08); the rest sit there by the conservative default (ambiguous unless argued never-given). #604 (2026-10-04) argued three more on entry — ua, binti, binte — under vocabulary-collisions. +- mc, ste — measured misparses ("Mc Donald" → given "Mc"), tracked in #360; st is inert at the head because TITLES claims it first; mac must stay ambiguous because Mac is a real given name. +- ben — OUT of the set entirely, removed in v0.2.5 and re-affirmed 2026-09-01 (#v1-xfail-triage). Ben is a common given name standing exactly where a forward-joining particle claim would act: "Ahmad Ben Husain" reads middle 'Ben' today, and admitting `ben` would take that token into the family. This is the C-i position test, and the entry is here because the pressure to re-add is real and recurring — "Ahmad ben Husain" wants family "ben Husain", the v1 suite carried that aspiration as an xfail for years, and a sweep of Arabic and Hebrew patronymic particles is exactly how it comes back. A caller who needs the patronymic reading adds it to their own Lexicon. +- nic, unaccented o, bt, bte — left OUT by #604 (2026-10-04). `nic` is the Irish "daughter of" beside `ní`, but also an abbreviation of Nicolas and Nicole and a given name of its own (Nic Cage): a leading `Nic` would be marked ambiguous at best, and its natural home is an opt-in Irish pack. Bare `O` is a common periodless initial, so `o` as a particle would turn `Liam O Murchú`'s middle initial — and every `John O Smith` — into a surname; the anglicized form is glued (`O'Neil`) and needs no particle. `bt`/`bte`, the abbreviations of binti/binte, were not reviewed, and `bt` is not an unknown: it is already shipped as a suffix acronym (Baronet) with a `Bt` case mask, so admitting it as a particle would open a new particle ∩ suffix overlap, the contested class rules/P6 decides for vd, do and mc, and under R4's mask precedence its mask would capitalize the particle too. +- Encoding rationale (#293, predating #360's membership questions): the data layer stores the NEVER-GIVEN set and derives the ambiguous one, because that is safe-by-default for new particles — a one-place addition — and the v1 shim translates by one-directional complement. And the constants are FROZEN specifically to kill the cached-Lexicon.default()-vs-fresh- Constants desync that runtime module-constant mutation caused. +- Load-bearing dependency: TITLES ∩ ambiguous — {do, freiherr, st} until #296's audit, {freiherr, st} since — is what keeps the particle-or-given ambiguity emitter reachable at all; moving them all would leave the shipped vocabulary unable to reach it, which is why test_the_chained_emitter_is_reachable_by_construction (tests/v2/test_parser.py) pins reachability by construction rather than by shipped vocabulary — an empty intersection fails no test and does not mean the emitter is unreachable. + +Open (contested vocabulary memberships — the rule is right, the word's set is questioned; the issue is canonical): none. The block held three entries and all three were answered inside a fortnight — #346, #343 and #344 together on 2026-09-06 (see #indic-honorifics) and #342 on 2026-09-07 (see #suffix-acronym-collisions). The heading stays with nothing under it on purpose, the same reason an empty ledger roster section is a statement and a missing one is nobody having looked. sa, se and om are named as the next candidates by #342's own comment and carry no issue, so they are not entries here. + +Excluded (SUFFIX_ACRONYMS — esq, removed 2026-09-08, #316/#489 bundle): + +- esq is OUT of SUFFIX_ACRONYMS and must not be put back by a sweep that finds "Esq." parsing and assumes the acronym set is what carries it. The SUFFIX_WORDS membership carries every single-token spelling — Esq, Esq., ESQ, esq — and stops being inert with the acronym entry gone. What the acronym entry uniquely covered is the multi-dot spelling, so "John Smith E.S.Q." reads family E.S.Q. where every release since 1.4.0 read suffix; that is the whole of the cost and it is a spelling nobody writes. The criterion is #suffix-acronym-collisions', asked of the WORD rather than of the frequency: does the entry describe the word or the machinery. Esquire is a contraction, not an initialism, so the initialism set was never its home; it arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. This SUPERSEDES the dual-membership entry that stood here from the 2.2 cycle, which called the acronym membership load-bearing and the word membership inert and concluded that the singleton "is why the two sets cannot carry a disjointness assert". The assert now exists — `not (SUFFIX_ACRONYMS & SUFFIX_WORDS)` in suffixes.py's guard block — and it is what the removal buys: the two sets normalize differently (the word test strips edge periods, the acronym test strips all of them), so a word in both is matched by two rules and which one fired is unreadable from outside. AGENTS.md's esq gotcha, the two defending comment blocks in suffixes.py, the `# NOT asserted:` note and the `suffix_acronym_multidot_spelling` case row all retire with it; a row pinning `John Smith E.S.Q.` → family replaces the last of those. Measured over the corpus glob as it stood at the change (1123 distinct names, 1263 rows; the docs commit that follows adds this bundle's rules.md examples and makes it 1136 and 1284): exactly one moves, `John Smith E.S.Q.` itself, and it diffs at every baseline. Classified a behavior change, a 2.x parity break, on all four ledgers. + +Excluded (SUFFIX_ACRONYMS — ph, removed 2026-09-13, #459/#521): + +- ph is OUT of SUFFIX_ACRONYMS and must not be put back by a sweep that finds "Ph. D." parsing and assumes the acronym set is what carries it. The split spelling is merged by group's own rule (the `PH`/`D` regexes of `_pipeline/_vocab.py`, v1's fix_phd; see #phd-merge, whose "merged back by vocabulary" means that regex stage and not a lexicon lookup), which never consults acronym membership, and the single-token spellings `phd`/`Ph.D.` are the `phd` entry's. The fragment arrived in the 2019-12-11 bulk Wikipedia post-nominal import (af5bdab, #93) and was never reviewed. It did two things. One was to let the merged `Ph. D.` token pass the acronym test on its first piece, which nothing read until #459's all-caps repair did — with `ph` in the set that repair read `john smith ph. d.` as `John Smith PH. D.` on the default path, not only under force, and that is what forced the removal. The other, found by the docs review of the fix-up (2026-09-22) and not at the removal, was to read a bare trailing `Ph.` — no `D.` behind it — as a credential: `John Smith Ph.` gave family Smith, suffix Ph. on every release from 1.4.0 through 2.3.0 (wheels measured), and out of the set it falls to the positional read, middle Smith, family Ph., suffix empty; `Smith, John Ph.` moves the same way, to middle Ph. The cost is that CLASS and not one spelling, measured 2026-09-22 on HEAD and on the five wheels with `Lexicon.default().add(suffix_acronyms={"ph"})` as the comparator: the acronym test strips periods, so the undotted `John Smith Ph`/`PH`/`ph` move identically; inside a credential run `Ph.` no longer joins it, so `John Smith Ph. MD` reads family Ph., suffix MD, and `John Smith MD Ph.` reads middle `Smith MD`, family Ph., the real credential leaving the suffix field with the fragment; `John Smith Ph. Jr.` keeps only the Jr.; and the comma form `Smith, Ph.` reads title Ph. where 2.2.0 and 2.3.0 read suffix. `John Smith P.H.` does not move, the by-shape class reading it either way. Every shape in the class is a bare `ph` with no `D.` behind it, which nobody writes — the same criterion as esq's, asked of the machinery rather than a surname: the entry described the merge, not a word. Classified a behavior change, a 2.x parity break, on all five ledgers (`change(suffix-acronym-collisions) ph leaves the acronym set`), with the cases.py row `removed_ph_fragment_leaves_a_bare_trailing_ph_a_name_word` written to carry the plain shape; the ledger rule is literal to that one name, so a future row on any other shape of the class arrives unexplained at every baseline and is classified by widening that rule, not filed as a regression. No corpus name had any shape of the class, which is why the removal measured as 0 role movers at review time (2026-09-11, 1143 names), the corpus-blindness AGENTS.md warns of. Under `capitalized()` the 13 `PH. D.` shapes the repair had moved under `force=True` go back to `Ph. D.`; `john smith phd` and `ph.d.` still give `Ph.D.`. + +Excluded (BOUND_GIVEN_NAMES): + +- mohamad — a standalone given name in its own right; binding it would eat the middle name. +- abd — ADDED 2026-08-18, see decisions/P5's entry of that date; the exclusion stood on two reasons and only one of them survived. The collision with the post-nominal "ABD" turned out to cost nothing (position decides at both ends). The other reason stands as a known limit: the three-token spelling "abd al rahman" is still not joined, `al` being a particle that chains forward, so only the two-token "abd X" and the hyphenated "abd al-Rahman X" join. + +Excluded (DEFAULT_NICKNAME_DELIMITERS): + +- Curly single quotes ('‘','’') are deliberately absent from the default pairs: U+2019 is the typographic apostrophe (O'Connor in curly type), so shipping the pair would eat real names. #273's own proposal excluded them; pinned by the curly_apostrophe_stays_literal case. A sweeper "completing the typographic set" ships the regression. + +Excluded (given-name titles and post-nominals — the 2026-07-19 transliteration deferrals, each living in a data-module comment until now): + +- Arabic bare سيد/شيخ/أمير/سلطان — given-name collisions (Sayyid, Shaikha, Amir, Sultan); the honorific forms with the article (الدكتور, الشيخ) ship instead. +- The abbreviation د. — edge-period normalization leaves bare د, the single-letter trap; Greek bare κ deferred on the same grounds. +- Ottoman post-nominals باشا/بك/أفندي — surname collisions. +- Hebrew bare רב (an ordinary word, "many") and בר (Bar is a common modern Israeli given name) — deferred, #269's territory. +- Latin sri/shri deliberately absent while Devanagari श्री ships: the transliteration collides (Sri Mulyani), the native script cannot. +- Latin sant and pandit deliberately absent while Devanagari संत and पंडित ship: Sant is a Punjabi given name (Sant Singh) and Pandit a common surname (Vijaya Lakshmi Pandit), so the Latin forms fail #vocabulary-collisions C-i's position test at the position the claim acts on, and TITLES has no ambiguous subset to express a partial answer in. The native spellings carry neither collision. The Latin forms belong, if anywhere, to the opt-in packs of #345 (2026-09-06, #344). + +Excluded (multi-word vocabulary entries — a set outside #M2's two exceptions matches one written word, so a multi-word entry there is silently inert and warns at configuration; given_name_titles and, since #434, maiden_markers are the exceptions, and they match phrases by different mechanisms): + +- Eight entries shipped unmatchable from 2013 to 2.0. chargé d'affaires was SPLIT into the chainable chargé + d'affaires, plus unaccented charge (the attaché/attache precedent, Derek's call); leed ap, nicet i–iv and psm i/ii were REMOVED rather than split, on the name-swallowing measurements recorded in #291 (see the comma-suffix-arc Declined entry). The multi-word UserWarning is this story's enforcement. + +Excluded (Lexicon.honorific_tails — a glued tail peels only if it could never end a name; per-entry reasons live in nameparser/config/suffixes.py's vetting block): + +- 양, 군 — 김지양 and 김지군 are given names ending in these syllables, and 양 is a top-tier surname (Yang) besides. (The surname-LEADS argument is a different job: it is why both are safe in the SPACED set — a leading surname never meets the trailing-only suffix gate, so 양 미선 keeps family 양.) +- 氏 — 王氏 is a historical name form ("the Wang woman"). +- 博士 — 田中博士 is Tanaka Hiroshi as readily as Doctor Tanaka. +- 殿 — Japanese surnames end in it (鵜殿, 真殿, four-figure populations); peeling it would cut a real family name in two. +- 君 — 王君 is a complete Chinese name; its kana spelling くん does peel. +- 部長, 部长 — 長 alone is a borne given name, so 阿部長 can be 阿部 + 長 (#707). The measurement found no such bearer, as it found none for the words that shipped; the exclusion rests on the argument, which a lower-bound zero does not refute, so a re-measure that again comes back zero is no reason to add them. +- जी, and with it the whole spaced trailing set the 2026-09-06 Indic bundle shipped — जी, साहब, साहिब, साहेब, महाराज, সাহেব, বাবু, মহারাজ and Latin rinpoche. None is a honorific_tails entry and none may become one. The evidence is जी's and it carries the set: Banerjee, Mukherjee and Chatterjee end in the substring (बनर्जी/मुखर्जी/चटर्जी), and glued peeling strands a fragment on a bare virama (बनर् + जी). The 殿 criterion, in a non-CJK script — which also shows the criterion is not CJK-specific. The accepted cost is that गांधीजी stays unpeeled, and it is the right trade: an unrecognized honorific costs one field, a strand cuts a real family name in two. Both shapes are pinned in tests/v2/test_locales.py. See #indic-honorifics. + +Excluded (TITLES): + +- ঠাকুর — a genuine Bengali honorific (lord/master) that is also Tagore, the surname (#343 records it so a wordlist sweep does not ship it). +- ठाकुर — the Devanagari spelling of the entry above, excluded on the same ground and recorded with it so a sweep of one script does not ship the other (2026-09-06, #344). +- कुमारी / কুমারী (Kumari) — a given name and a family name in the very script the honorific would be written in, so #vocabulary-collisions C-i's position test fails at the position the claim acts on. TITLES has no ambiguous subset to express a partial answer in, so under C-i's "under uncertainty, default to ambiguous" the only available answer is do not ship (2026-09-06, #344/#343). +- बेगम / বেগম (Begum) — a trailing name element far more often than a leading title, so the leading claim would be wrong more often than right (2026-09-06, #343/#344). +- शेख / শেখ (Sheikh) — a clan name and a family name in both scripts ("শেখ হাসিনা"). The divergence from Latin is deliberate and is the sri/shri case run backwards: the Latin sheikh/sheik/shaykh/shaikh cluster SHIPS as given-name titles for the Arabic addressing form, while the Indic spellings name the family (2026-09-06, #344/#343). +- आचार्य (Acharya) — a Brahmin surname (2026-09-06, #344). +- राजा / रानी (Raja/Rani) — common given names (2026-09-06, #344). +- ông / bà (Vietnamese Mr / Mrs) — Ông is a Vietnamese surname and Vietnamese names lead with the surname, the position the title claim acts on; pinned by the case row salutation_that_leads_a_surname_stays_a_name (2026-10-04, #606, #salutation-titles). +- thiru (Tamil Mr) — a Tamil given name (2026-10-04, #606). +- puan (Malay Mrs) — an Indonesian given name, Puan Maharani (2026-10-04, #606). +- pan (Polish/Czech Mr) — the Chinese surname Pan, which leads (2026-10-04, #606). The feminine paní ships, its accent keeping it off the name Pani; the unaccented pani does not. +- rodina (Czech "family") — a Russian surname, often written surname-first (2026-10-04, #606). familie ships. +- gróf (Hungarian Count) — possibly a surname in a surname-first culture; unverified, and under uncertainty the answer is do not ship (2026-10-04, #606). +- cik, paron, tikin, janab (Latin spellings) — unverified, so not shipped. Latin janab follows the sri/shri pattern: Devanagari जनाब ships, and the Latin spelling needs its own check (2026-10-04, #606). +- marshal — a given name (Marshal Yanda). The accepted cost is that `Field Marshal Bernard Montgomery` keeps reading title Field, given Marshal (2026-10-04, #606). +- justice — a given name (Justice Smith). Likewise `Chief Justice John Roberts` keeps title Chief, given Justice (2026-10-04, #606). +- malam (Hausa teacher/Mr.) — a US Census surname (119), 5+ Wikidata family bearers and 1 Wikidata given-name bearer (Nigeria), a lower bound (2026-10-17, #662, `tools/name_usage.py malam`). CLDR's Hausa data files Malam as the title (`grep '"title": "Malam"' tools/differential/cldr_fields.jsonl`), the opposite of the evidence that decided shehu, so this is a decline on the counts, not a shehu-style removal: Derek dropped it for the lower-bound given-name use, and whoever re-argues it should count Hausa given-name use first. The accepted cost is `Malam Shehu Bello` reading given Malam. +- bay (Turkish Mr.) — a US surname with 6,350 bearers and 120 on Wikidata, 11 of them Hungarian; Hungarian writes the surname first, where the title claim acts, so it fails C-i (2026-10-17, #662). +- shehu (Hausa honorific) — a common Hausa given name and an Albanian and Nigerian surname, so the leading title claim took the given name of `Shehu Shagari`, a former president of Nigeria, who read title Shehu, family Shagari in every release from 1.4.0 through 2.4.1. It arrived with an early bulk import (1a51555f) and was never put to C-i; removed 2026-10-13 (#660). Counts from `tools/name_usage.py shehu` on 2026-10-13: 561 US Census surname bearers; on Wikidata 180 family-name bearers (Nigeria 20, Albania 11) and 1 given-name bearer (Nigeria), a lower bound, since Shagari's own entry has no given-name statement. Not weighed under the graf class (#salutation-titles), and the counts above would not settle that weighing: the one leading-position figure, the single Wikidata given-name bearer, is a lower bound and sits below herra's ~670, while the 561 and 180 are family-name bearers, a position C-i does not ask about. The evidence that Shehu leads a name is not a count. It is Shagari himself, and CLDR's Hausa sample data in this tree, whose six renderings carrying the word all file it as the given name (`grep '"given": "Shehu"' tools/differential/cldr_fields.jsonl`). Whoever re-argues the word should count Hausa given-name use first. The accepted cost is the honorific use, `Shehu Abubakar` reading given Shehu as any untitled name does; it belongs in an opt-in pack if anyone asks for it. Moved three CLDR corpus names (`Shehu Muhammed`, `Muhammed, Shehu`, `Modibbo, Shehu Bello`), all three now agreeing with CLDR on every role, so they left `tools/differential/cldr_agreement.json` (moves: `uv run python tools/differential/snapshot.py --since v2.4.1`). +- The trailing-position rule that must NOT be adopted, and 2026-09-08 says what the line is: TITLES holds hundreds of words in no suffix set, at least nineteen of them ordinary English surnames (king, judge, bishop, baron, sheriff, ...), so a blanket "vocabulary outranks position in the trailing slot" reading would turn "Mary Jane King" into title="King" with the family name gone. The leading half of this argument is AGENTS.md's "Dean is deliberately absent" gotcha; this is the trailing half, and it shadows the family name rather than the given (#316). What #316 settled is that the prohibition is on BARE words: "Mary Jane King" still reads family King, while "John Smith Prof." reads title Prof. **The second half of that argument is RETRACTED 2026-09-09**, in review of the docs commit: "a period-marked trailing word being one nobody writes as a surname" claimed the period keeps the collision out of the trailing slot, and it does not. Measured on the branch tree — "Mary Jane King." reads title "King.", given Mary, family Jane, the family name shadowed exactly as the bare reading would have shadowed it, and "John Smith Judge." reads title "Judge." The reach is the vocabulary rather than a handful: of the 746 titles in no suffix set, 627 read as a trailing title once a period is written behind them, and every one of the 119 that do not is kept out by the abbreviation SHAPE — a digit, a hyphen, an apostrophe, or a script whose letters carry combining marks (rules/H2 records that half) — no plain ASCII-letter title missing at all. So king, judge, bishop, baron and sheriff ARE reachable in the trailing slot; what the prohibition above buys is the bare spelling and nothing more. The period is a WRITING convention the rule can read, not evidence about the word, and reading it is an ACCEPTED COST under the premise that the input is a name (rules.md H Background): someone who ends a name with a period-marked word has written an abbreviation. rules/H5 carries it as an Accepted clause with "Mary Jane King." as the executable example. Recompute the reach with `L = Parser().lexicon; ws = [w for w in L.titles - L.suffix_acronyms - L.suffix_words if " " not in w]; len([w for w in ws if parse("Mary Jane %s." % w).title])` against `len(ws)` — 627 of 746 on 2026-09-09 — the digits move as the vocabulary grows and neither argument does. See #H5. + +Excluded (Policy.script_orders defaults): Script.KATAKANA is deliberately absent — a pure-katakana token is predominantly a transcribed foreign name kept in its source order, so nothing defaults on it (rule W4's boundary). Noted 2026-08-15: of the three Script-keyed axes, this is the one with no force-a-decision guard (mechanisms.md#FORCE-A-DECISION-TABLE), so a new Script member silently gets no order. + +Excluded (MAIDEN_MARKERS, per nameparser/config/maiden_markers.py): + +- Polish "z domu" — the exclusion is LIFTED and the phrase SHIPS; see the INCLUDED entry below for the argument. It was provisional from the day it was written, 2026-07-27, pending the multi-token matching decision — first tracked in #291, repointed to #434 on 2026-09-07 per Derek's amendment on #342, markers having no run predicate, so the multi-token question does not dissolve there the way the suffix one did — and #434 shipped whole-phrase marker matching on 2026-08-26, which is what lifted it. Kept rather than deleted because a provisional exclusion that was answered reads, once deleted, as an exclusion nobody ever considered. +- Contrast entry — unaccented "nee" SHIPS as a marker despite + #274 flagging "is nee safe as a default (it's also a rare + surname)" as open; the question resolved silently with the shipped set. Recorded here because the included risky member deserves its analysis as much as the excluded ones; M1's (Nee) boundary covers only the enclosure path, not this marker path. +- "born" — never shipped: a release-log drafting invention, caught by the 2.0 milestone audit and corrected (5ccf9f3). Recorded so nobody "restores" it; if ever proposed for real, Max Born is the counterexample to analyze. +- Scandinavian "f." — collides with the initial F.; only the full participles (født/fødd/född) are safe. Czech masculine "rozený" awaits the same vetting, on its own merits rather than following the feminine form in. +- Polish "z domu" — INCLUDED 2026-08-26 as the set's first phrase entry, and the contrast to draw against the roz removal below rather than against any included word. Its words fail C-i separately and it passes as a pair; the argument is in the #434 entry under #M2. The point to carry away is that the entry below could not have been rescued the same way: roz is a whole marker as written, so there is no longer unit to move the claim onto. +- Czech/Slovak "roz." — SHIPPED through 2.1 and removed 2026-08-26 (Derek's call). Roz is an ordinary English diminutive of Rosalind, and matching is whole-token, case-folded and period-insensitive, so "Roz", "roz" and "roz." are one string to this set. The collision is not theoretical and it is not new: on the bare path, which predates the 2.2 work entirely, "Rosalind Roz Smith" read maiden "Smith" and NO family name at all, and "Rosalind Roz Jones Smith" read maiden "Jones Smith", because M2 hands the marker every word after it. Grounded in #vocabulary-collisions C-i — a word belongs in its set's ambiguous subset iff it is borne as an ordinary name IN THE POSITION THE VOCABULARY CLAIM ACTS ON, and under uncertainty default to ambiguous. Roz is borne exactly there: the word after a given name, which is the only position M2's claim acts on. What C-i cannot do for this set is the marking: MAIDEN_MARKERS has no ambiguous subset, and no code path consults one, so the only two expressions of C-i available here are ship and do not ship. That is why the remedy is removal rather than an ambiguous marking, and it is worth knowing before someone proposes marking the next collision instead. The full participle rozená stays and is unambiguous; a Czech caller who needs the abbreviation adds it to their own Lexicon, which is the same answer this section gives for every locale-specific vocabulary. The cost is accepted and stated: "Anna Nováková roz. Svobodová" now reads middle "Nováková roz.", family "Svobodová" — which is how 1.4.0 read it — and "Rosalind Roz Smith" is back to 1.4.0's given/middle/family. Pinned by tests/v2/cases.py::diminutive_that_was_a_marker_keeps_the_family and ::full_participle_marker_still_consumes. + +Excluded (CAPITALIZATION_EXCEPTIONS — meng, edd, lac, ded, left out of the 2026-09-24 masks, #459): + +- `meng` (MEng) and `edd` (EdD) are conventionally mixed-case credentials and listed acronyms, and must not get a mask while the map stays role-free: a mask applies in every role, and both are borne as given names or surnames. Measured 2026-09-24 with each mask added to a private lexicon: `MENG LI` gives `MEng Li`, `meng wang` gives `MEng Wang`, `edd smith` gives `EdD Smith` and `EDD KIMBER` gives `EdD Kimber`, where the shipped map and every release give `Meng Li`, `Meng Wang`, `Edd Smith` and `Edd Kimber`. The same holds for two sweep candidates, `lac` (LAc, Licensed Acupuncturist; `Lac` is a Vietnamese given name — the trailing word in native order, `Nguyễn Văn Lạc` — and a French surname; corrected 2026-09-25, #540, from "a Vietnamese and French surname") and `ded` (DEd; `Ded` is an Albanian given name, `ded gjo luli` giving `DEd Gjo Luli` under the mask). A writer who spells one mixed-case keeps it (decisions/R5, 2026-09-24); a caller who wants the mask adds it to a private lexicon. A NOTE on `SUFFIX_ACRONYMS` itself, which this map does not decide. The two name-word collisions there come from two bulk additions, not one: `lac` (with `edd` and `ded`) from the Wikipedia post-nominals import, `af5bdab` (2019-12-11, #93), and `meng` from `3e14ea20` (2026-07-01, the German/Dutch title and degree batch, first shipped in 1.3.0). Both read as a suffix after a given name, and lose a family name to it (superseded 2026-09-25 by #540: both words are marked ambiguous, `wang meng` reads family `meng` and renders `Wang Meng` — see #suffix-acronym-collisions). Measured 2026-09-24 on the released wheels, the parent 4d0680e6 and this tree: `wang meng`, `li meng` and `zhang meng` read family `meng` on 1.4.0 and suffix `meng`, with no family, on every release from 2.0.0; and since #521 on master the suffix renders `Wang MENG` (parent and tree alike), where 2.3.0 gave `Wang Meng`. `nguyen van lac` reads suffix `lac` with family `van` on every release from 1.4.0, and since #521 renders `Nguyen Van LAC` where 2.3.0 gave `Nguyen Van Lac`. This is a #vocabulary-collisions C-i question about the acronym set, not about this map. + +Excluded (CAPITALIZATION_EXCEPTIONS — md, ii, iii, iv, removed 2026-09-23, #459): + +- `md`, `ii`, `iii` and `iv` are OUT of the exceptions map and must not be put back by a sweep that finds one of them title-cased somewhere. As a SUFFIX each is written in capitals with no entry — `md` by the acronym clause, the numerals by the numeral clause — and in any other role each repairs as the parse read it by decision (decisions/R4's item-4 bullet), so an entry would only bring back the role-free capitals that bullet retired: `iv smith` → `IV Smith`, `Md Abdul Karim` → `MD Abdul Karim`. A caller who wants them back adds them to a private lexicon, as masks. diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index b14185f4..374222da 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -29,7 +29,7 @@ Problem shape. A rule needs to know how words were JOINED (chained titles, parti ## UNIT-PARTITION — count the units the joining rules built -Problem shape. A rule counts "one name word", but the input holds words that another rule has already joined into one name — and the joining structure it would read has been merged away. Contract statement. Three rules build multi-word units: a particle chain (P2), a conjunction join (P3), and a bound given-name pair (P5). A rule that counts name words counts those units, and takes each whole or not at all — with one stated exception, rules.md#C1's count of the words before a comma, which counts particle surnames alone (decisions.md#C1, #575). How it works. group builds each join as a piece, but its own prefix chain then merges the joined piece into a longer one, so PIECES no longer carries the boundary — the units are rebuilt from the tags the vocabulary layer left (`particle`, `conjunction`, `vocab:bound-given`). The rebuild is RECURSIVE: what a conjunction or a bound word joins is the next UNIT, not the next word, and absorbing a single token instead strands a particle severed from the words it chains. Note the two joins arrive here for opposite reasons — the conjunction join was built and then swallowed, while the bound-given join was never built at all (P5 joins only at the first non-title piece), so restoring piece boundaries in group would fix the first and silently split the second. Lives in. nameparser/_pipeline/_vocab.py (`unit_ends`, the walk, shared since #575) and its two readers: _post_rules.py (`_units`, over classify's tags, for P1's fold) and rules.md#C1's counts before a comma: `_comma._whole_name`, over classify's tags (`surname_unit_facts`) since #613 moved the decision after classify, and `_vocab.name_word_count`, which builds its facts from the vocabulary (`surname_unit_tags`, held to classify's tags by an agreement test) -- a view script_segment needed while it asked a hand copy of C1 before classify ran, until #613's /simplify had it ask `_comma.decide` itself on a classified copy. A fourth walk is S2's read (#614): `_pieces._chain_units` marks each piece by the unit P2's chain WILL put it in -- opening a unit, or a name word the chain joins -- asking the chain's own `chain_run_end`, which `_group`'s chain calls too; a word the reading weighs (`_weighed`) keeps a unit of its own, as a position always counted it. The particles straight behind each other are merged ahead of the read, into the piece the chain would open its run with (`_pieces.merge_particle_runs`, #625), so a particle run is one piece the read takes whole or not at all, and no mark stands for a particle inside it. The trailing peel's words to spare and #602's run start count in those marks, the peel stops at a word the chain joins, and everything else -- the numeral fork's look at the word in front, the H5 chain, the run's title test -- reads the pieces as written. It runs before the chain joins name words to a particle run, which is why it cannot read the merged tags `unit_ends` walks; #614 had rebuilt the chain as a merged COPY instead, whose three defects in one review -- words the reading weighs hidden inside a unit, indices drifting from the pieces' -- were each patched in that copy, and #620 replaced it with the marks (decisions.md#S2). C1 counts particle chains only, reaching one word as the fold does: a bound pair builds a given name, and whether P3 joins a connective depends on the whole name the count is helping to decide, so neither is one surname before a comma (`_vocab.SURNAME_UNIT_TAGS`, decisions.md#C1). rules.md P1 and C1 are the counting rules, P2/P3/P5 the joining ones. Reach for it when. A rule says "one name word" and the input can contain a join — enumerate the joining rules out of rules.md rather than the ones you remember. +Problem shape. A rule counts "one name word", but the input holds words that another rule has already joined into one name — and the joining structure it would read has been merged away. Contract statement. Three rules build multi-word units: a particle chain (P2), a conjunction join (P3), and a bound given-name pair (P5). A rule that counts name words counts those units, and takes each whole or not at all — with one stated exception, rules/C1's count of the words before a comma, which counts particle surnames alone (decisions/C1, #575). How it works. group builds each join as a piece, but its own prefix chain then merges the joined piece into a longer one, so PIECES no longer carries the boundary — the units are rebuilt from the tags the vocabulary layer left (`particle`, `conjunction`, `vocab:bound-given`). The rebuild is RECURSIVE: what a conjunction or a bound word joins is the next UNIT, not the next word, and absorbing a single token instead strands a particle severed from the words it chains. Note the two joins arrive here for opposite reasons — the conjunction join was built and then swallowed, while the bound-given join was never built at all (P5 joins only at the first non-title piece), so restoring piece boundaries in group would fix the first and silently split the second. Lives in. nameparser/_pipeline/_vocab.py (`unit_ends`, the walk, shared since #575) and its two readers: _post_rules.py (`_units`, over classify's tags, for P1's fold) and rules/C1's counts before a comma: `_comma._whole_name`, over classify's tags (`surname_unit_facts`) since #613 moved the decision after classify, and `_vocab.name_word_count`, which builds its facts from the vocabulary (`surname_unit_tags`, held to classify's tags by an agreement test) -- a view script_segment needed while it asked a hand copy of C1 before classify ran, until #613's /simplify had it ask `_comma.decide` itself on a classified copy. A fourth walk is S2's read (#614): `_pieces._chain_units` marks each piece by the unit P2's chain WILL put it in -- opening a unit, or a name word the chain joins -- asking the chain's own `chain_run_end`, which `_group`'s chain calls too; a word the reading weighs (`_weighed`) keeps a unit of its own, as a position always counted it. The particles straight behind each other are merged ahead of the read, into the piece the chain would open its run with (`_pieces.merge_particle_runs`, #625), so a particle run is one piece the read takes whole or not at all, and no mark stands for a particle inside it. The trailing peel's words to spare and #602's run start count in those marks, the peel stops at a word the chain joins, and everything else -- the numeral fork's look at the word in front, the H5 chain, the run's title test -- reads the pieces as written. It runs before the chain joins name words to a particle run, which is why it cannot read the merged tags `unit_ends` walks; #614 had rebuilt the chain as a merged COPY instead, whose three defects in one review -- words the reading weighs hidden inside a unit, indices drifting from the pieces' -- were each patched in that copy, and #620 replaced it with the marks (decisions/S2). C1 counts particle chains only, reaching one word as the fold does: a bound pair builds a given name, and whether P3 joins a connective depends on the whole name the count is helping to decide, so neither is one surname before a comma (`_vocab.SURNAME_UNIT_TAGS`, decisions/C1). rules/P1 and rules/C1 are the counting rules, P2/P3/P5 the joining ones. Reach for it when. A rule says "one name word" and the input can contain a join — enumerate the joining rules out of the rules rather than the ones you remember. ## MARK-DONT-STRIP — record the decision, keep the fact @@ -47,7 +47,7 @@ How it works. The two layers compose without ordering bugs because the positiona ## READ-WITHOUT-THEN-BIND — read the name without the construct, then fix that reading -Problem shape. An optional construct sits inside a name — a maiden clause, a trailing title — and the requirement is that writing it changes nothing else about the name. Checked after the fact, that invariant has to be defended at every place the construct can end, and each defense has to model what later stages would do to the words it gives up. Contract statement. Read the name as if the construct were not written, using the rule that reads that position, not the whole parse. The construct takes only what that reading leaves. Where later stages could still move what the reading decided, give those words their roles at the take so no later join can reach them. The invariant then holds by construction and is not a property each stop must defend. How it works. M2 before #601 scanned forward, stopped at suffix-looking words, and kept a release check asking at each stop whether the name left standing would read what was given up as post-nominals. That check modelled the P2 chain, the P5 bound join and P6's attachment, and #548 was it failing at the one stop that never asked. #601 replaced the stops and the check with the clause-free reading plus the bind: a net 689 lines out of `_group.py` (+210/−899 in the #601 commit), and #535's grids went from 15,586 and 8,713 invariant violations to 0 (prototype measurement, 2026-10-03, #601's comments). The first half already appears twice. H5's trailing title is TRANSPARENT: "what stands once the chain is taken reads exactly as it would read written without the title, plus the title". P3 asks its word count and its one-case question of the name's OWN words, so a clause beside the name changes neither. M2 is the first to bind as well, because its construct sits in front of the run it reads and the joins come later. C1 is the second (#613): the part after the comma is read once, after classify and before any join, by the reading a part holding no name word gets, and a postnominal part's roles are set there. segment had decided the structure before the tags existed, so it modelled group's particle chain (#562's pair) and assign's reading of a run the capitals settle, and assign re-decided #296's positional read with a second count; all three went. S2 is the third (#614): the trailing run of a main segment is read once, between P3's connective joins and the particle chain, and split off, so the joins that had re-asked the peel before and after themselves -- the chain's stop and its rollback, the bound-given reserve's two views -- answer from that one reading, and assign places the run off it. Where the read sits is the lesson. Joins the run does not depend on but whose units it counts come FIRST: read before P3's connectives, the H5 chain took the second title of a bare `Mr. and Mrs.` and left 'and' as the family name, and a joined pair counted as two words (#614's review). Joins the run does depend on come after, and the read counts what they will build instead -- the particle chain's run and the words it joins as one unit, marked on the pieces rather than built, with every word the reading weighs still a word of its own: #614 counted through a merged copy, whose review found it hiding a shape-only roman numeral and a title inside a credential run, each patched there, until #620 counted the chain's units over the pieces as written (decisions.md#S2). Two limits, both measured on #601. Read with the TRAILING rule, not the full parse: a head the full parse reads differently (H1's lone title, a P5 join) would otherwise move where the construct ends. And binding means the run can read differently from the same words with no construct (`JANE Q. DOE GEB. LE DO DO` gives suffix `DO DO`, while the full parse of `JANE Q. DOE DO DO` chains them into the family), which rules.md#M2 accepts in so many words. Lives in. nameparser/_pipeline/_group.py (`_maiden_take`: the view, `tail_reading` over it, and the role-tagged pieces `group()` applies); nameparser/_pipeline/_pieces.py (`tail_reading`, H5's chain); the P3 own-word test in `_group.py`; nameparser/_pipeline/_comma.py (C1's read-and-bind); `_pieces.read_trailing_run` and `_chain_units`, and the read after P3's joins in `_group._group_segment` (S2's). Reach for it when. A rule is growing stops, exceptions or a release check whose job is to keep one construct from disturbing the rest of the name, and above all when that check has started to model another stage. Ask what the name reads as without the construct, and make that the rule. +Problem shape. An optional construct sits inside a name — a maiden clause, a trailing title — and the requirement is that writing it changes nothing else about the name. Checked after the fact, that invariant has to be defended at every place the construct can end, and each defense has to model what later stages would do to the words it gives up. Contract statement. Read the name as if the construct were not written, using the rule that reads that position, not the whole parse. The construct takes only what that reading leaves. Where later stages could still move what the reading decided, give those words their roles at the take so no later join can reach them. The invariant then holds by construction and is not a property each stop must defend. How it works. M2 before #601 scanned forward, stopped at suffix-looking words, and kept a release check asking at each stop whether the name left standing would read what was given up as post-nominals. That check modelled the P2 chain, the P5 bound join and P6's attachment, and #548 was it failing at the one stop that never asked. #601 replaced the stops and the check with the clause-free reading plus the bind: a net 689 lines out of `_group.py` (+210/−899 in the #601 commit), and #535's grids went from 15,586 and 8,713 invariant violations to 0 (prototype measurement, 2026-10-03, #601's comments). The first half already appears twice. H5's trailing title is TRANSPARENT: "what stands once the chain is taken reads exactly as it would read written without the title, plus the title". P3 asks its word count and its one-case question of the name's OWN words, so a clause beside the name changes neither. M2 is the first to bind as well, because its construct sits in front of the run it reads and the joins come later. C1 is the second (#613): the part after the comma is read once, after classify and before any join, by the reading a part holding no name word gets, and a postnominal part's roles are set there. segment had decided the structure before the tags existed, so it modelled group's particle chain (#562's pair) and assign's reading of a run the capitals settle, and assign re-decided #296's positional read with a second count; all three went. S2 is the third (#614): the trailing run of a main segment is read once, between P3's connective joins and the particle chain, and split off, so the joins that had re-asked the peel before and after themselves -- the chain's stop and its rollback, the bound-given reserve's two views -- answer from that one reading, and assign places the run off it. Where the read sits is the lesson. Joins the run does not depend on but whose units it counts come FIRST: read before P3's connectives, the H5 chain took the second title of a bare `Mr. and Mrs.` and left 'and' as the family name, and a joined pair counted as two words (#614's review). Joins the run does depend on come after, and the read counts what they will build instead -- the particle chain's run and the words it joins as one unit, marked on the pieces rather than built, with every word the reading weighs still a word of its own: #614 counted through a merged copy, whose review found it hiding a shape-only roman numeral and a title inside a credential run, each patched there, until #620 counted the chain's units over the pieces as written (decisions/S2). Two limits, both measured on #601. Read with the TRAILING rule, not the full parse: a head the full parse reads differently (H1's lone title, a P5 join) would otherwise move where the construct ends. And binding means the run can read differently from the same words with no construct (`JANE Q. DOE GEB. LE DO DO` gives suffix `DO DO`, while the full parse of `JANE Q. DOE DO DO` chains them into the family), which rules/M2 accepts in so many words. Lives in. nameparser/_pipeline/_group.py (`_maiden_take`: the view, `tail_reading` over it, and the role-tagged pieces `group()` applies); nameparser/_pipeline/_pieces.py (`tail_reading`, H5's chain); the P3 own-word test in `_group.py`; nameparser/_pipeline/_comma.py (C1's read-and-bind); `_pieces.read_trailing_run` and `_chain_units`, and the read after P3's joins in `_group._group_segment` (S2's). Reach for it when. A rule is growing stops, exceptions or a release check whose job is to keep one construct from disturbing the rest of the name, and above all when that check has started to model another stage. Ask what the name reads as without the construct, and make that the rule. ## LICENSED-BY-NEIGHBOR — a structural word is structural only beside the right neighbor @@ -55,7 +55,7 @@ Problem shape. A word in a structural vocabulary — a marker announcing a maide ## STATE-OFFSET-CHANNELS — early facts ride the state -Problem shape. A fact known during tokenization matters to a much later stage. Contract statement. A pre-token fact is recorded as offsets on the ParseState (comma_offsets, interpunct_offsets) and consulted later by position — or by presence alone where the fact is name-level, as both interpunct consumers do, or as a single boolean for the whole name, as `ParseState.one_case` is (#289/#516: the written-case fact three later sites read — the trailing suffix slot, the post-comma given slot, and the tail-segment reading — none of which may disagree about it) — rather than re-derived from text. How it works. The offsets survive every intermediate stage untouched; #298's transcription marker rides this channel from tokenize to order resolution (rules T3/W4). `one_case` differs from the offset channels in WHO writes it: it is computed once by whichever of segment and classify needs it first (classify always, segment only where a comma form could turn it on), recorded as `None` for "not asked yet", and never recomputed once set (decisions.md#S2). Lives in. nameparser/_pipeline/_state.py, produced in _tokenize (the offsets) or in segment/classify (`one_case`). Reach for it when. You are about to re-scan the original string in a late stage to rediscover something tokenize already knew. +Problem shape. A fact known during tokenization matters to a much later stage. Contract statement. A pre-token fact is recorded as offsets on the ParseState (comma_offsets, interpunct_offsets) and consulted later by position — or by presence alone where the fact is name-level, as both interpunct consumers do, or as a single boolean for the whole name, as `ParseState.one_case` is (#289/#516: the written-case fact three later sites read — the trailing suffix slot, the post-comma given slot, and the tail-segment reading — none of which may disagree about it) — rather than re-derived from text. How it works. The offsets survive every intermediate stage untouched; #298's transcription marker rides this channel from tokenize to order resolution (rules T3/W4). `one_case` differs from the offset channels in WHO writes it: it is computed once by whichever of segment and classify needs it first (classify always, segment only where a comma form could turn it on), recorded as `None` for "not asked yet", and never recomputed once set (decisions/S2). Lives in. nameparser/_pipeline/_state.py, produced in _tokenize (the offsets) or in segment/classify (`one_case`). Reach for it when. You are about to re-scan the original string in a late stage to rediscover something tokenize already knew. ## PIPELINE-STAGE-CONTRACTS — the ownership map @@ -63,11 +63,11 @@ Problem shape. "Which stage does X?" — asked before attributing behavior in pr ## ONE-PREDICATE-PER-QUESTION — one predicate answers it, and every other site calls that -Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix; is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage; and maiden_marker_run, the #434 instance and the clearest two-stage case, called by classify over token texts and by extract over a clause's whitespace words, with group reading the tags classify recorded because it runs later; and delimiter_cores, the #436/#437 instance, read by group where a tail segment DROPS a configured delimiter core and by post_rules where the suffix view's entry boundary asks whether a dropped token was one, with a third reader inside this same module, is_wholly_suffix, where a configured core counts as suffix-shaped; and in_initialless_script, the #322/#323 instance and the only one here that is a REPERTOIRE test rather than a vocabulary one — the script half of the #320 initial veto, read by is_initial one function away and by _pieces.is_leading_title, so "a script with no initials has no period abbreviations either" is one predicate over _policy._NO_INITIALS rather than a second reading of that table; it lost its leading underscore when the second caller arrived; and caps_shape_candidate, the #516 instance and the newest, called from the sites that each needed the identical question answered — classify's own tag emission and the comma decision's caps test (_segment.py's two comma tests until #613 moved the decision into _comma.py and classify's comma-run branch went with it), with its unit tests — where the usual reason for keeping such copies apart (a shared call costing every default-policy parse a frame it cannot use) does not hold: every caller asks first, in C, what the predicate would decline anyway — the setting and position, then alphabetic capitals and no listed suffix word — so only an unlisted all-caps word reaches the call, at the default as under any setting (decisions.md#S2)) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, leading_titles and peel_walk are called by both stages, as is is_title_piece since #602 and #603 gave assign two calls of its own (re-measured 2026-10-11), while is_leading_title and trailing_start are called by group alone (measured 2026-09-06 by call site: `is_leading_title` has no caller in `_assign.py`, which reads `leading_titles` instead — a first draft of this clause listed it among the shared ones) — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain stops at, and M2's walk wherever no trailing rule reads the clause (elsewhere, since #535, the walk stops where `tail_reading` says) — and segment_suffix_reading, by assign alone from #436/#437 and by `_comma.decide` alone since #613 moved the decision and its binding there, that last one being #430's instance, where THREE readers shared one answer until the render join, group's third, was replaced by a rule over the commas the writer typed (decisions.md#C1, 2026-09-06); it stays where it is, one call site being no reason to move a predicate that two sites will contest again. `trailing_titles` was that last shape for one day (2026-09-08, the #316/#489 bundle, rules.md#H5), and since the /simplify round of 2026-09-09 the SHARED predicate is `tail_reading` instead — the peel-and-chain fixed point that answers where the name pieces end (decisions.md#H5). Assign calls it at its main walk and group's bound-given reserve calls it twice, once per view the join compares, because that reserve reads the name words assign will leave and this walk is half of what leaves them (rules.md#P5; counting a trailing title word among them joined 'Prof. abdul rahman Prof.' where 'Prof. abdul rahman' does not). Since #535 group's maiden walk calls it as well, over the clause and over the view its take would leave, wherever a trailing rule reads the clause (rules.md#M2), for the same reason: the walk's stops must end the clause where assign's reading of the name will begin. `peel_trailing` and `trailing_titles` are what that fixed point is BUILT from, and neither is a two-stage question any longer: `peel_trailing` has no caller outside `_pieces.py`: `trailing_start` and `tail_reading`, its two callers in the leaf, fold the numeral's half and the acronym's half into one index. It had one until #601 — the maiden walk in `_group.py`, which asked the bare peel itself one half at a time (#424, #533) and from #535 only where no trailing rule read the clause; since #601 the maiden take reads `tail_reading` and `trailing_candidates` instead (re-measured 2026-10-11 by AST call-site census). `trailing_titles` has exactly one caller outside the leaf, assign's family-comma segment-1 walk, which reads the chain without the re-peel, and `_group.py` does not import it. The tail reading is in the leaf rather than inline because each assign site had been given a cheap frame-free gate written to match the walk's own first condition, which is a second implementation of the question and was removed in review; what the leaf costs is one frame per entry point, measured, and the walk's own first test is a compiled regex rather than a call, so an ordinary name pays a match and stops. The reserve's two calls cost the reference name nothing — it never enters that branch, having no bound given word — and the parse and facade frame counts did not move (measured 2026-09-09). Re-measured 2026-09-09 by an AST call-site census over `_pipeline/*.py` — every call node whose callee is one of these names, keyed by module and enclosing function, which is what caught the census claiming a share for `peel_trailing` that the round had just taken away — the rest of it holds unchanged: is_suffix_piece, leading_titles, peel_walk and now tail_reading shared, is_leading_title, is_title_piece and trailing_start group-only (is_title_piece has since joined the shared list, above) — assign still reads `leading_titles` and never `is_leading_title`, which is what keeps H2's shape inference out of the trailing slot. And nameparser/_pipeline/_post_rules.py over a state: suffix_entries, the #511 instance, the R1 entry pass as a function, the one instance living in a stage rather than in a leaf — it is a pass over a whole ParseState and no leaf takes one, and AGENTS.md names it as the exception — run by post_rules last in the stage (through its in-place worker) and by Parser.revise over a sub-parse whose roles it has forced, so a suffix value handed to revise() derives its entries by the rule a whole name uses rather than by a second reading of the value's commas (decisions.md#C1, 2026-09-06 #511). Two more since #610 (2026-10-04), both in `_pieces.py`: `particle_tail`, P6's walk, asked by post_rules' attachment and by assign's given-part credential run, which had been two walks agreeing only through the run's roles -- and since #613 asked once per parse, by assign's walk -- which both leaves the tail out of the run and attaches it -- or, behind a comma with nothing before it, by post_rules after H1 and M4; and `trailing_candidates`, the maiden take's choice of clause words, written beside the peel whose admissions it must cover, with a sweep test against `tail_reading` that found two shapes the hand copy had missed (decisions.md#M2). And `join_connectives` since #617, P3's connective joins, which `_comma.decide` had copied as `_title_chains` and which had already parted from group's three ways (no run merge, an unconditional single-letter skip, the title kind alone): one loop now, taking from each caller what genuinely differs between them -- the carve-out's answer, a titles-only mode, the frozen set. What the fold taught is the last of those: a parameter one caller had been filling with a guard is a question the other caller must answer too. The comma part had never been asked it -- its copy joined a generational connective of more letters as it joined any other (`og`, under caller vocabulary), and never met the default one-letter `i` at all -- and the shared run merge brought `i` into reach, where the property grid caught it (decisions.md#P3, 2026-10-06 #617). tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead. +Problem shape. Two stages need the same answer about the same input, and the one that does not own the decision is about to test for it. Contract statement. Where two sites ask the same question, exactly one predicate answers it and every other site calls that one — never a condition written to match it. The predicate belongs to the QUESTION, not to whichever stage decides: it may sit in a leaf both stages import, and for the leading-title test it must, since the deciding stage is assign and group cannot import assign. How it works. A hand-written mirror agrees with its original only until one of them moves, and the drift is invisible in both directions: each site keeps passing its own tests while they disagree about an input neither covers. Five instances, every one found as a defect before it was found as a pattern — #319 lifted the wholly-suffix predicate into the vocabulary layer "so the comma decision and the honorific peel's segment test cannot drift apart"; #401/#421 lifted the trailing-numeral fork out of assign so the bound-given reserve stopped carrying a copy, its hand-written mirror having been falsified in review more than once — the lesson recorded there being that what must be mirrored is assign's WALK, not merely its condition; #425 replaced that reserve's hand re-derivation of the trailing peel with one function over the view the join would leave; #424 moved assign's leading-title test down because group's own `title()` does not see H2's unlisted abbreviations, so `Xyz. van Johnson` chained where `Dr. van Johnson` did not; #429 moved the no-name-segment test down because group asked by segment INDEX where assign asks by CONTENT. The destination follows the LAYER, not the topic: a predicate over token text goes to `_vocab`, one over pieces and tags to `_pieces`. Both are leaves the stages sit on. The piece layer got its own module only in #439 — until then those predicates collected in `_group`, not because grouping owned them but because `_assign` imports `_group` and cannot be imported back, so group was the one place both stages could reach; five had accumulated across four PRs before the module existed. Stage order is this mechanism's limit, and it forecloses the alternative: where the reader comes AFTER the decider, record the answer on the state instead — `ParseState.order` is that shape, "Recorded rather than recomputed downstream, because the two can differ" — which is unavailable whenever the EARLIER stage is the one asking. (The concrete assign→group import that forced the `_group` collection is gone since #439; what remains is the ordering it was a symptom of, and tests/v2/test_layering.py is where the leaf's contract is now written down.) The cost is a second evaluation of the same predicate, measured for #429 at 1.2–2.2% of a family-comma parse and 0% of every other; recording that number was the right answer there over plumbing a state field the two sites would not otherwise share. Lives in. nameparser/_pipeline/_vocab.py over text (is_wholly_suffix; is_trailing_numeral_suffix — the #401/#421 instance, whose only caller since #439 is the shared peel rather than a stage; and maiden_marker_run, the #434 instance and the clearest two-stage case, called by classify over token texts and by extract over a clause's whitespace words, with group reading the tags classify recorded because it runs later; and delimiter_cores, the #436/#437 instance, read by group where a tail segment DROPS a configured delimiter core and by post_rules where the suffix view's entry boundary asks whether a dropped token was one, with a third reader inside this same module, is_wholly_suffix, where a configured core counts as suffix-shaped; and in_initialless_script, the #322/#323 instance and the only one here that is a REPERTOIRE test rather than a vocabulary one — the script half of the #320 initial veto, read by is_initial one function away and by _pieces.is_leading_title, so "a script with no initials has no period abbreviations either" is one predicate over _policy._NO_INITIALS rather than a second reading of that table; it lost its leading underscore when the second caller arrived; and caps_shape_candidate, the #516 instance and the newest, called from the sites that each needed the identical question answered — classify's own tag emission and the comma decision's caps test (_segment.py's two comma tests until #613 moved the decision into _comma.py and classify's comma-run branch went with it), with its unit tests — where the usual reason for keeping such copies apart (a shared call costing every default-policy parse a frame it cannot use) does not hold: every caller asks first, in C, what the predicate would decline anyway — the setting and position, then alphabetic capitals and no listed suffix word — so only an unlisted all-caps word reaches the call, at the default as under any setting (decisions/S2)) and nameparser/_pipeline/_pieces.py over pieces: is_suffix_piece, leading_titles and peel_walk are called by both stages, as is is_title_piece since #602 and #603 gave assign two calls of its own (re-measured 2026-10-11), while is_leading_title and trailing_start are called by group alone (measured 2026-09-06 by call site: `is_leading_title` has no caller in `_assign.py`, which reads `leading_titles` instead — a first draft of this clause listed it among the shared ones) — `trailing_start` being the one to know, since it answers where the trailing run begins and is what P2's chain stops at, and M2's walk wherever no trailing rule reads the clause (elsewhere, since #535, the walk stops where `tail_reading` says) — and segment_suffix_reading, by assign alone from #436/#437 and by `_comma.decide` alone since #613 moved the decision and its binding there, that last one being #430's instance, where THREE readers shared one answer until the render join, group's third, was replaced by a rule over the commas the writer typed (decisions/C1, 2026-09-06); it stays where it is, one call site being no reason to move a predicate that two sites will contest again. `trailing_titles` was that last shape for one day (2026-09-08, the #316/#489 bundle, rules/H5), and since the /simplify round of 2026-09-09 the SHARED predicate is `tail_reading` instead — the peel-and-chain fixed point that answers where the name pieces end (decisions/H5). Assign calls it at its main walk and group's bound-given reserve calls it twice, once per view the join compares, because that reserve reads the name words assign will leave and this walk is half of what leaves them (rules/P5; counting a trailing title word among them joined 'Prof. abdul rahman Prof.' where 'Prof. abdul rahman' does not). Since #535 group's maiden walk calls it as well, over the clause and over the view its take would leave, wherever a trailing rule reads the clause (rules/M2), for the same reason: the walk's stops must end the clause where assign's reading of the name will begin. `peel_trailing` and `trailing_titles` are what that fixed point is BUILT from, and neither is a two-stage question any longer: `peel_trailing` has no caller outside `_pieces.py`: `trailing_start` and `tail_reading`, its two callers in the leaf, fold the numeral's half and the acronym's half into one index. It had one until #601 — the maiden walk in `_group.py`, which asked the bare peel itself one half at a time (#424, #533) and from #535 only where no trailing rule read the clause; since #601 the maiden take reads `tail_reading` and `trailing_candidates` instead (re-measured 2026-10-11 by AST call-site census). `trailing_titles` has exactly one caller outside the leaf, assign's family-comma segment-1 walk, which reads the chain without the re-peel, and `_group.py` does not import it. The tail reading is in the leaf rather than inline because each assign site had been given a cheap frame-free gate written to match the walk's own first condition, which is a second implementation of the question and was removed in review; what the leaf costs is one frame per entry point, measured, and the walk's own first test is a compiled regex rather than a call, so an ordinary name pays a match and stops. The reserve's two calls cost the reference name nothing — it never enters that branch, having no bound given word — and the parse and facade frame counts did not move (measured 2026-09-09). Re-measured 2026-09-09 by an AST call-site census over `_pipeline/*.py` — every call node whose callee is one of these names, keyed by module and enclosing function, which is what caught the census claiming a share for `peel_trailing` that the round had just taken away — the rest of it holds unchanged: is_suffix_piece, leading_titles, peel_walk and now tail_reading shared, is_leading_title, is_title_piece and trailing_start group-only (is_title_piece has since joined the shared list, above) — assign still reads `leading_titles` and never `is_leading_title`, which is what keeps H2's shape inference out of the trailing slot. And nameparser/_pipeline/_post_rules.py over a state: suffix_entries, the #511 instance, the R1 entry pass as a function, the one instance living in a stage rather than in a leaf — it is a pass over a whole ParseState and no leaf takes one, and AGENTS.md names it as the exception — run by post_rules last in the stage (through its in-place worker) and by Parser.revise over a sub-parse whose roles it has forced, so a suffix value handed to revise() derives its entries by the rule a whole name uses rather than by a second reading of the value's commas (decisions/C1, 2026-09-06 #511). Two more since #610 (2026-10-04), both in `_pieces.py`: `particle_tail`, P6's walk, asked by post_rules' attachment and by assign's given-part credential run, which had been two walks agreeing only through the run's roles -- and since #613 asked once per parse, by assign's walk -- which both leaves the tail out of the run and attaches it -- or, behind a comma with nothing before it, by post_rules after H1 and M4; and `trailing_candidates`, the maiden take's choice of clause words, written beside the peel whose admissions it must cover, with a sweep test against `tail_reading` that found two shapes the hand copy had missed (decisions/M2). And `join_connectives` since #617, P3's connective joins, which `_comma.decide` had copied as `_title_chains` and which had already parted from group's three ways (no run merge, an unconditional single-letter skip, the title kind alone): one loop now, taking from each caller what genuinely differs between them -- the carve-out's answer, a titles-only mode, the frozen set. What the fold taught is the last of those: a parameter one caller had been filling with a guard is a question the other caller must answer too. The comma part had never been asked it -- its copy joined a generational connective of more letters as it joined any other (`og`, under caller vocabulary), and never met the default one-letter `i` at all -- and the shared run merge brought `i` into reach, where the property grid caught it (decisions/P3, 2026-10-06 #617). tests/v2/test_layering.py holds each module's contract, and a piece predicate growing a dependency on a STAGE shows up there as a widened entry. Reach for it when. You are about to write a condition that mirrors, matches or "does what X does" — or you find a comment saying one does. Grep for the other site's predicate and call it instead. ## RENDER-HONORS-THE-PARSE — the parse decides it, the views honor it -Problem shape. A render view needs a fact the parse already settled — whether a word is the conjunction or an initial, whether a particle is acting as one, which word renders first. Contract statement. The parse decides it; the render views honor those decisions and never re-evaluate them. Two directions break that, and each has been found here as a defect: a view RE-DERIVES the answer from the text, keeping its own copy of a pipeline predicate — that one shipped through 2.0 and 2.1 — or a view honors the record and then OVERRIDES it, readmitting what the decision excluded, which was filed and open here and is CLOSED (2026-09-20, #461). How it works. Re-deriving fails because the two copies stop being the same question long before anyone notices they are two: `_cap_word` re-ran the conjunction-versus-initial decision from the word's spelling against a hand-maintained copy of the pipeline's `_INITIAL` pattern while classify had already answered it and recorded it on the token, and `_classify.py` asks `is_initial()`, the shape test ANDed with a script-repertoire test since #320, where `_render.py` asked the bare pattern, and asked it per WORD of a token's text rather than per token, so `juan e-f smith` repaired to `Juan e-F Smith` (#458). Overriding fails more directly, and this entry's instance of it is now CLOSED (see the paragraph that closes it below): `initials()` honors the `conjunction` tag through `_SKIP_TAGS` and then readmits the token whenever it also carries UNJOINED_TAG, the mark of an all-particle part, where rules.md#R3 excludes a conjunction "even then" — so under a caller's vocabulary that puts a word in both sets the view readmits what the rule excluded, and under `Lexicon.default().add(particles={'y'})` — the lexicon those two readings need, and they hold under no other — `parse("Anh y Van")` initialed `A. y. V.` and `parse("Juan de y")` `J. d. y.` on 2026-08-29. Read those two values scoped to that lexicon or they are simply false: under the DEFAULT vocabulary the same strings gave `A. V.` and `J.` on that date, the second being a rules.md#R3 example line the doc runner asserts — and BOTH of those default readings have since moved, which the closing paragraph below carries. #461 is where that stands, and it is worth reading before re-fixing it: the narrowing was written, measured and BACKED OUT in the same PR, because honoring R3 there cost `initials()` its agreement with `family_base`, which reads that same `de y` as the base — this entry's other shape, arrived at from the render side, and the reason the question moved from the code to R3's clause (decisions.md carries the argument). What to carry away is the DIRECTION and not its verdict here: a view that honors a record and then readmits what the record excluded is overriding a decision it never took, whichever way this one settles. Case repair reads that same token and does NOT readmit it — `capitalized(force=True)` on `Anh y Van` gives `Anh y Van` under that same lexicon, R4 carrying the carve-out in its own words and, since #461, on its own reason rather than on R3's authority: a connective that initials because it joins nothing is still not written the way a name is written. Until 2026-09-20 R4's text read "being no name word in any part — the carve-out R3 states for initials", so a change to R3's clause reached R4's TEXT; #461 cut that cross-reference, and nothing now ties the two rules' wording. They were never tied in BEHAVIOR either, though an earlier wording of this sentence asserted it: over the corpus names carrying a conjunction in the GIVEN group, `parse("john and jane smith").capitalized()` kept `and` lowercase while `.initials()` gave `j. a. j. s.` before #461 (decisions.md#R2 carries that population). Since #461 a connective with a name word beside it contributes no initial, so that name initials `j. j. s.` and the two views agree on it, while a connective joining nothing still initials and still repairs lowercase — `parse("Juan de y")` initials `J. y.` and repairs to `Juan de y` under force — which is where they part today, by decision (rules.md#R3 and #R4 each say so in their own words); only the FORCED call witnesses the repair half, R5's gate refusing a mixed-case name before any of this is consulted. This is the CONSUMER-side rule over the producer-side entries — VOCAB-TAGS records what the vocabulary knew, MARK-DONT-STRIP what a stage decided about it, FOLDED_TAG what order to render in — and a view reads what they recorded, whichever kind it is. It sits where ONE-PREDICATE-PER-QUESTION's stated limit leaves off: where two live sites need one answer they share a predicate, but a render view always comes AFTER the decider, so the answer is recorded rather than shared — on the TOKEN, which is the views' equivalent of that entry's `ParseState.order`, no view being able to see a ParseState at all. Known limit, and the half most easily got wrong next: a token the parse never saw carries no decision to honor, so a view falls back to the vocabulary — the tell is `UNCLASSIFIED_TAG`, which `ParsedName.replace()` stamps when it splices raw text into a field and the facade's v1 pickle load stamps when it rebuilds a name from `*_list` strings. It is NOT untaggedness, since an ordinary parsed name word carries no tags either; and it is NOT `span is None`, which was tried and is wrong in the other direction — span-less means SYNTHETIC, and `Parser.revise()` builds span-less tokens from a full sub-parse whose tags it keeps on purpose, so the span reading overrode exactly the tags `revise()` exists to preserve (`revise(middle='e-f')` repaired to `e-F` where the parse gave `E-F`). A hand-built span-less token is unmarked and therefore classified, which is the same tag-driven default every other view applies. A view can only fall back if it is HANDED a vocabulary, and TWO are: `capitalized(lexicon=...)`, and — since #528 — the v1 facade's `HumanName.initials()`, which holds the bound `Lexicon` its `Parser` was built from and so never has to guess one. `family_base` and `family_particles` are properties on ParsedName, whose fields are original/tokens/ambiguities and nothing else, so a spliced field empties the particles view and leaves the base the whole field, with `Parser.revise()` the crossing there too (docs/usage.rst says so where it documents the degradation). The CORE `ParsedName.initials()` is the near miss and the instructive one, and it is still the near miss: it is a METHOD, so it looks like it could ask, but its signature is `(spec, delimiter, separator)` and carries no lexicon — a fallback there was written and dropped because it had to GUESS `Lexicon.default()`, and the guess erased a whole field under a caller's own vocabulary (decisions.md#R4). Read the two initials views apart wherever this entry says "initials", because #528 made them differ exactly here: the core cannot fall back and the facade does. `capitalized()` guesses nothing either: it reads the lexicon it was handed, and only defaults to `Lexicon.default()` when the caller passes none, which is the documented meaning of omitting the argument rather than a fallback. Within a view that can fall back, the fallback is drawn per QUESTION (rules.md#R4's Accepted clause). Whether a word is the conjunction or an initial is a property of the word, which a vocabulary answers alone, so case repair asks it. Whether a part is wholly particles is a property of the whole PART, which the pipeline answers once and records as UNJOINED_TAG. What `_cap_word` cannot do is RE-DERIVE that answer where no word of the part carries a tag — it is handed the whole token's tags and gates the particle conjunct on UNJOINED_TAG, so it is not blind to the part, it simply has no evidence to reconstruct one from — so repair leaves that half to plain particle treatment and rules.md#R4's Accepted boundary records the consequence — a spliced field is not repaired as a parsed one is, with `Parser.revise()` as the supported crossing. A fallback is right only while it answers as the pipeline would, and THAT is held by hand rather than mechanically: test_regex_sync pins the two `_INITIAL` copies to each other and to config, while the repertoire half of the pipeline's predicate (#320) is deliberately not carried across, layering forbidding the import. What that divergence can reach was argued to be nothing observable, and #528 RETRACTED that for the facade view while leaving it standing for the core. The shape it needs is a caller-added conjunction written initial-SHAPED in a script that has no initials (`太.`, `씨.`), and the old argument was that case repair is the fallback's only reader, so the two paths differ by `lower()` versus `capitalize()` over a caseless script — the same string either way. The facade's `HumanName.initials()` is a second reader now, and it CAN show the difference, because a word the fallback calls a connective contributes no initial while one it does not contributes a letter. Witness, measured 2026-09-13 under `Constants()` with `conjunctions.add("太")`: `HumanName("Wang Chen 太. Li").initials()` is "W. C. L." — classify tags the `太.` `conjunction` in the family and the view honors it — while splicing the identical family text in, `h.last = "Chen 太. Li"`, gives "W. C. 太. L.", the fallback reading `太.` as initial-shaped and admitting it. Same string in the field, two answers, and `capitalized(force=True)` shows nothing at all on either. ACCEPTED rather than repaired, on this entry's own terms: the fallback path is only ever reached for text no parse read, a spliced field is the caller's own text, and it is answered by the SAME helper R4 hands case repair — so the remedy is the documented crossing, WHICH DIFFERS BY VIEW and must not be copied across: for the parsed name's views it is `Parser.revise()` (rules.md#R3's Accepted clause, decisions.md#R3), while the facade has no `revise` at all — its setters splice through `replace()` by decision — so the v1 remedy is the v1 one, parsing the whole string again by assigning `full_name`, which restores "W. C. L." on the name above (measured 2026-09-13 with the rest of this witness). What stays true for the core is the sentence that used to close this: `ParsedName.initials()` was once the reader that could witness it and no longer falls back at all, being handed no vocabulary. A second limit, recorded rather than closed: `_cap_word`'s PARTICLE conjunct still keys on the lexicon handed to the view rather than on the `particle` tag, so a repair run with a lexicon other than the parse's re-decides a word the parse already read — a name parsed under the default vocabulary, where `parse('juan smith vega')` reads `vega` as the family, repairs to `Juan Smith vega` when `capitalized()` is handed `Lexicon.default().add(particles={'vega'})` instead — the divergence needs the two lexicons to differ, and repairing under the parse's own lexicon gives `Juan Smith Vega`. Making it read the tag moves a boundary rules.md#R4 states in prose, so it is a separate decision and not a cleanup (decisions.md#R4, "NOT DONE"). CLOSED 2026-09-20 (#461), and what closed it was the RULE rather than the code: rules.md#R3 no longer excludes a connective unconditionally, so the readmission this entry called an override is what the rule now says, decided once over the whole part and recorded on the token as UNJOINED_CONJUNCTION_TAG for both initials views to read. The scoped values above STAY TRUE and are re-measured on this date: under `Lexicon.default().add(particles={'y'})`, `parse("Anh y Van")` initials `A. y. V.` and `parse("Juan de y")` `J. d. y.`, exactly as on 2026-08-29. What changed is the pair of DEFAULT-vocabulary readings quoted beside them, and both of them: `parse("Juan de y")` gives `J. y.` where it gave `J.` — the rules.md#R3 example line moved with it — and `parse("Anh y Van")` gives `A. y. V.` where it gave `A. V.`, its middle `y` holding that part alone. On THAT string the two lexicons now agree, which is the shape of the fix: the readmission was never about a caller's overlapping vocabulary, it was about a part with nothing left to join. `Juan de y` still differs between them and legitimately so — under the default vocabulary `de` is a working particle and `y` alone is the base, while under the overlap the whole part is particles and R2's own mark readmits both words. The disagreement with case repair is not closed and is not meant to be: it is narrowed to a connective the parse placed among the NAME words with nothing to join, and it is R4's own rule now rather than an oversight or a borrowing from R3 (decisions.md#R4, 2026-09-20). Narrowed twice, the second time by a defect the record itself turned up (2026-09-20): where the parse read such a word as the GENERATION it also spells, repair was reading classify's `conjunction` tag without asking the role and lower-casing a suffix, so `parse("John Quincy Smith i")` forced gave 'John Quincy Smith i'. A view honors what the parse DECIDED, and the role is that decision where the tag is only what the word could have been -- the same distinction this entry draws between re-deriving an answer and reading the record, one level up. The direction this entry teaches is unchanged and is why the paragraph is closed rather than deleted: a view that honors a record and then readmits what the record excluded is overriding a decision it never took — the remedy was to settle what the record should say, not to make the view louder. Lives in. nameparser/_render.py (`capitalized`/`_cap_word`, `_reads_as_conjunction`, and `initials`, the CORE view, which honors tags and never falls back) and nameparser/_types.py (`_text_for` and `UNCLASSIFIED_TAG`, with the `ParsedName.replace()` producer beside it) and nameparser/_facade.py, which is on this list TWICE and for opposite reasons: the v1 pickle load is the SECOND producer of that mark — named here because a change that follows the list into `_types.py` alone leaves it behind, which is the site test_a_restored_pickle_keeps_v1_conjunction_repair exists to protect — and since #528 `_token_is_conjunction`/`_process_initial` are a CONSUMER, the facade's initials view reading the tag and calling `_render._reads_as_conjunction` for the mark's own tokens. All of them read what nameparser/_pipeline/ recorded — the mark those views read is recomputed producer-side in `_remarked`, which is deliberately silent about text nobody classified and is right to be. Reach for it when. A view is about to consult a Lexicon, a regex or an exception list about a word the parse already saw — or a view and a field disagree about the same parse. #408 was that second shape, and is CLOSED (2026-08-30): `initials()` walked tokens in written order where the family field applies FOLDED_TAG's ordering, so `parse("der, y van")` gave family `van der` and initials `y. d. v.` on 2026-08-29 and gives `y. v. d.` now, the view reading the tag as `_text_for` does. Worth keeping as the worked instance of the shape rather than deleting with the fix, and worth two notes on how it read once measured. The disagreement was not a judgment call anyone had taken: the FACADE already ordered folded-first through its own `*_list` views, so the core view was out of step with the field, with v1, and with the facade at once, and nothing in 6125 tests touched it. And where the change has a v1 reference at all it RESTORES rather than deviates, which is not what this entry's other instances have been -- but read that SCOPED to the population it was measured over, because an unscoped version of this sentence stood here until 2026-08-30 and overstated in both directions. Only the two DEFAULT-ORDER policies have a v1 reference: v1 had `middle_name_as_last` and no general `name_order`, so 588 of the 660 moving parses -- the two family-first orders -- restore nothing and break nothing, there being no v1 answer to come into or leave (decisions.md#R3 says the same and carries the rest of the measurement). Where the reference does exist the claim is exact and worth keeping: over the 1094-name corpus at the default order, 71 names move under `middle_as_family`, of which 54 return to 1.4.0's answer and none leaves it. And it is a claim about THOSE 71 rather than about every name the fix touches -- this entry's own lead example is the counterexample, `parse("der, y van")` giving `y. d. v.` before and `y. v. d.` after where 1.4.0 gives `y.`, v1 contributing nothing at all for a family that is all particles (rules.md#R2's territory, and a divergence decisions.md#R2 has already decided in favor of). A view that stopped honoring a record had been quietly reproducing a v1 bug that v1 did not have. +Problem shape. A render view needs a fact the parse already settled — whether a word is the conjunction or an initial, whether a particle is acting as one, which word renders first. Contract statement. The parse decides it; the render views honor those decisions and never re-evaluate them. Two directions break that, and each has been found here as a defect: a view RE-DERIVES the answer from the text, keeping its own copy of a pipeline predicate — that one shipped through 2.0 and 2.1 — or a view honors the record and then OVERRIDES it, readmitting what the decision excluded, which was filed and open here and is CLOSED (2026-09-20, #461). How it works. Re-deriving fails because the two copies stop being the same question long before anyone notices they are two: `_cap_word` re-ran the conjunction-versus-initial decision from the word's spelling against a hand-maintained copy of the pipeline's `_INITIAL` pattern while classify had already answered it and recorded it on the token, and `_classify.py` asks `is_initial()`, the shape test ANDed with a script-repertoire test since #320, where `_render.py` asked the bare pattern, and asked it per WORD of a token's text rather than per token, so `juan e-f smith` repaired to `Juan e-F Smith` (#458). Overriding fails more directly, and this entry's instance of it is now CLOSED (see the paragraph that closes it below): `initials()` honors the `conjunction` tag through `_SKIP_TAGS` and then readmits the token whenever it also carries UNJOINED_TAG, the mark of an all-particle part, where rules/R3 excludes a conjunction "even then" — so under a caller's vocabulary that puts a word in both sets the view readmits what the rule excluded, and under `Lexicon.default().add(particles={'y'})` — the lexicon those two readings need, and they hold under no other — `parse("Anh y Van")` initialed `A. y. V.` and `parse("Juan de y")` `J. d. y.` on 2026-08-29. Read those two values scoped to that lexicon or they are simply false: under the DEFAULT vocabulary the same strings gave `A. V.` and `J.` on that date, the second being a rules/R3 example line the doc runner asserts — and BOTH of those default readings have since moved, which the closing paragraph below carries. #461 is where that stands, and it is worth reading before re-fixing it: the narrowing was written, measured and BACKED OUT in the same PR, because honoring R3 there cost `initials()` its agreement with `family_base`, which reads that same `de y` as the base — this entry's other shape, arrived at from the render side, and the reason the question moved from the code to R3's clause (its decisions entry carries the argument). What to carry away is the DIRECTION and not its verdict here: a view that honors a record and then readmits what the record excluded is overriding a decision it never took, whichever way this one settles. Case repair reads that same token and does NOT readmit it — `capitalized(force=True)` on `Anh y Van` gives `Anh y Van` under that same lexicon, R4 carrying the carve-out in its own words and, since #461, on its own reason rather than on R3's authority: a connective that initials because it joins nothing is still not written the way a name is written. Until 2026-09-20 R4's text read "being no name word in any part — the carve-out R3 states for initials", so a change to R3's clause reached R4's TEXT; #461 cut that cross-reference, and nothing now ties the two rules' wording. They were never tied in BEHAVIOR either, though an earlier wording of this sentence asserted it: over the corpus names carrying a conjunction in the GIVEN group, `parse("john and jane smith").capitalized()` kept `and` lowercase while `.initials()` gave `j. a. j. s.` before #461 (decisions/R2 carries that population). Since #461 a connective with a name word beside it contributes no initial, so that name initials `j. j. s.` and the two views agree on it, while a connective joining nothing still initials and still repairs lowercase — `parse("Juan de y")` initials `J. y.` and repairs to `Juan de y` under force — which is where they part today, by decision (rules/R3 and #R4 each say so in their own words); only the FORCED call witnesses the repair half, R5's gate refusing a mixed-case name before any of this is consulted. This is the CONSUMER-side rule over the producer-side entries — VOCAB-TAGS records what the vocabulary knew, MARK-DONT-STRIP what a stage decided about it, FOLDED_TAG what order to render in — and a view reads what they recorded, whichever kind it is. It sits where ONE-PREDICATE-PER-QUESTION's stated limit leaves off: where two live sites need one answer they share a predicate, but a render view always comes AFTER the decider, so the answer is recorded rather than shared — on the TOKEN, which is the views' equivalent of that entry's `ParseState.order`, no view being able to see a ParseState at all. Known limit, and the half most easily got wrong next: a token the parse never saw carries no decision to honor, so a view falls back to the vocabulary — the tell is `UNCLASSIFIED_TAG`, which `ParsedName.replace()` stamps when it splices raw text into a field and the facade's v1 pickle load stamps when it rebuilds a name from `*_list` strings. It is NOT untaggedness, since an ordinary parsed name word carries no tags either; and it is NOT `span is None`, which was tried and is wrong in the other direction — span-less means SYNTHETIC, and `Parser.revise()` builds span-less tokens from a full sub-parse whose tags it keeps on purpose, so the span reading overrode exactly the tags `revise()` exists to preserve (`revise(middle='e-f')` repaired to `e-F` where the parse gave `E-F`). A hand-built span-less token is unmarked and therefore classified, which is the same tag-driven default every other view applies. A view can only fall back if it is HANDED a vocabulary, and TWO are: `capitalized(lexicon=...)`, and — since #528 — the v1 facade's `HumanName.initials()`, which holds the bound `Lexicon` its `Parser` was built from and so never has to guess one. `family_base` and `family_particles` are properties on ParsedName, whose fields are original/tokens/ambiguities and nothing else, so a spliced field empties the particles view and leaves the base the whole field, with `Parser.revise()` the crossing there too (docs/usage.rst says so where it documents the degradation). The CORE `ParsedName.initials()` is the near miss and the instructive one, and it is still the near miss: it is a METHOD, so it looks like it could ask, but its signature is `(spec, delimiter, separator)` and carries no lexicon — a fallback there was written and dropped because it had to GUESS `Lexicon.default()`, and the guess erased a whole field under a caller's own vocabulary (decisions/R4). Read the two initials views apart wherever this entry says "initials", because #528 made them differ exactly here: the core cannot fall back and the facade does. `capitalized()` guesses nothing either: it reads the lexicon it was handed, and only defaults to `Lexicon.default()` when the caller passes none, which is the documented meaning of omitting the argument rather than a fallback. Within a view that can fall back, the fallback is drawn per QUESTION (rules/R4's Accepted clause). Whether a word is the conjunction or an initial is a property of the word, which a vocabulary answers alone, so case repair asks it. Whether a part is wholly particles is a property of the whole PART, which the pipeline answers once and records as UNJOINED_TAG. What `_cap_word` cannot do is RE-DERIVE that answer where no word of the part carries a tag — it is handed the whole token's tags and gates the particle conjunct on UNJOINED_TAG, so it is not blind to the part, it simply has no evidence to reconstruct one from — so repair leaves that half to plain particle treatment and rules/R4's Accepted boundary records the consequence — a spliced field is not repaired as a parsed one is, with `Parser.revise()` as the supported crossing. A fallback is right only while it answers as the pipeline would, and THAT is held by hand rather than mechanically: test_regex_sync pins the two `_INITIAL` copies to each other and to config, while the repertoire half of the pipeline's predicate (#320) is deliberately not carried across, layering forbidding the import. What that divergence can reach was argued to be nothing observable, and #528 RETRACTED that for the facade view while leaving it standing for the core. The shape it needs is a caller-added conjunction written initial-SHAPED in a script that has no initials (`太.`, `씨.`), and the old argument was that case repair is the fallback's only reader, so the two paths differ by `lower()` versus `capitalize()` over a caseless script — the same string either way. The facade's `HumanName.initials()` is a second reader now, and it CAN show the difference, because a word the fallback calls a connective contributes no initial while one it does not contributes a letter. Witness, measured 2026-09-13 under `Constants()` with `conjunctions.add("太")`: `HumanName("Wang Chen 太. Li").initials()` is "W. C. L." — classify tags the `太.` `conjunction` in the family and the view honors it — while splicing the identical family text in, `h.last = "Chen 太. Li"`, gives "W. C. 太. L.", the fallback reading `太.` as initial-shaped and admitting it. Same string in the field, two answers, and `capitalized(force=True)` shows nothing at all on either. ACCEPTED rather than repaired, on this entry's own terms: the fallback path is only ever reached for text no parse read, a spliced field is the caller's own text, and it is answered by the SAME helper R4 hands case repair — so the remedy is the documented crossing, WHICH DIFFERS BY VIEW and must not be copied across: for the parsed name's views it is `Parser.revise()` (rules/R3's Accepted clause, decisions/R3), while the facade has no `revise` at all — its setters splice through `replace()` by decision — so the v1 remedy is the v1 one, parsing the whole string again by assigning `full_name`, which restores "W. C. L." on the name above (measured 2026-09-13 with the rest of this witness). What stays true for the core is the sentence that used to close this: `ParsedName.initials()` was once the reader that could witness it and no longer falls back at all, being handed no vocabulary. A second limit, recorded rather than closed: `_cap_word`'s PARTICLE conjunct still keys on the lexicon handed to the view rather than on the `particle` tag, so a repair run with a lexicon other than the parse's re-decides a word the parse already read — a name parsed under the default vocabulary, where `parse('juan smith vega')` reads `vega` as the family, repairs to `Juan Smith vega` when `capitalized()` is handed `Lexicon.default().add(particles={'vega'})` instead — the divergence needs the two lexicons to differ, and repairing under the parse's own lexicon gives `Juan Smith Vega`. Making it read the tag moves a boundary rules/R4 states in prose, so it is a separate decision and not a cleanup (decisions/R4, "NOT DONE"). CLOSED 2026-09-20 (#461), and what closed it was the RULE rather than the code: rules/R3 no longer excludes a connective unconditionally, so the readmission this entry called an override is what the rule now says, decided once over the whole part and recorded on the token as UNJOINED_CONJUNCTION_TAG for both initials views to read. The scoped values above STAY TRUE and are re-measured on this date: under `Lexicon.default().add(particles={'y'})`, `parse("Anh y Van")` initials `A. y. V.` and `parse("Juan de y")` `J. d. y.`, exactly as on 2026-08-29. What changed is the pair of DEFAULT-vocabulary readings quoted beside them, and both of them: `parse("Juan de y")` gives `J. y.` where it gave `J.` — the rules/R3 example line moved with it — and `parse("Anh y Van")` gives `A. y. V.` where it gave `A. V.`, its middle `y` holding that part alone. On THAT string the two lexicons now agree, which is the shape of the fix: the readmission was never about a caller's overlapping vocabulary, it was about a part with nothing left to join. `Juan de y` still differs between them and legitimately so — under the default vocabulary `de` is a working particle and `y` alone is the base, while under the overlap the whole part is particles and R2's own mark readmits both words. The disagreement with case repair is not closed and is not meant to be: it is narrowed to a connective the parse placed among the NAME words with nothing to join, and it is R4's own rule now rather than an oversight or a borrowing from R3 (decisions/R4, 2026-09-20). Narrowed twice, the second time by a defect the record itself turned up (2026-09-20): where the parse read such a word as the GENERATION it also spells, repair was reading classify's `conjunction` tag without asking the role and lower-casing a suffix, so `parse("John Quincy Smith i")` forced gave 'John Quincy Smith i'. A view honors what the parse DECIDED, and the role is that decision where the tag is only what the word could have been -- the same distinction this entry draws between re-deriving an answer and reading the record, one level up. The direction this entry teaches is unchanged and is why the paragraph is closed rather than deleted: a view that honors a record and then readmits what the record excluded is overriding a decision it never took — the remedy was to settle what the record should say, not to make the view louder. Lives in. nameparser/_render.py (`capitalized`/`_cap_word`, `_reads_as_conjunction`, and `initials`, the CORE view, which honors tags and never falls back) and nameparser/_types.py (`_text_for` and `UNCLASSIFIED_TAG`, with the `ParsedName.replace()` producer beside it) and nameparser/_facade.py, which is on this list TWICE and for opposite reasons: the v1 pickle load is the SECOND producer of that mark — named here because a change that follows the list into `_types.py` alone leaves it behind, which is the site test_a_restored_pickle_keeps_v1_conjunction_repair exists to protect — and since #528 `_token_is_conjunction`/`_process_initial` are a CONSUMER, the facade's initials view reading the tag and calling `_render._reads_as_conjunction` for the mark's own tokens. All of them read what nameparser/_pipeline/ recorded — the mark those views read is recomputed producer-side in `_remarked`, which is deliberately silent about text nobody classified and is right to be. Reach for it when. A view is about to consult a Lexicon, a regex or an exception list about a word the parse already saw — or a view and a field disagree about the same parse. #408 was that second shape, and is CLOSED (2026-08-30): `initials()` walked tokens in written order where the family field applies FOLDED_TAG's ordering, so `parse("der, y van")` gave family `van der` and initials `y. d. v.` on 2026-08-29 and gives `y. v. d.` now, the view reading the tag as `_text_for` does. Worth keeping as the worked instance of the shape rather than deleting with the fix, and worth two notes on how it read once measured. The disagreement was not a judgment call anyone had taken: the FACADE already ordered folded-first through its own `*_list` views, so the core view was out of step with the field, with v1, and with the facade at once, and nothing in 6125 tests touched it. And where the change has a v1 reference at all it RESTORES rather than deviates, which is not what this entry's other instances have been -- but read that SCOPED to the population it was measured over, because an unscoped version of this sentence stood here until 2026-08-30 and overstated in both directions. Only the two DEFAULT-ORDER policies have a v1 reference: v1 had `middle_name_as_last` and no general `name_order`, so 588 of the 660 moving parses -- the two family-first orders -- restore nothing and break nothing, there being no v1 answer to come into or leave (decisions/R3 says the same and carries the rest of the measurement). Where the reference does exist the claim is exact and worth keeping: over the 1094-name corpus at the default order, 71 names move under `middle_as_family`, of which 54 return to 1.4.0's answer and none leaves it. And it is a claim about THOSE 71 rather than about every name the fix touches -- this entry's own lead example is the counterexample, `parse("der, y van")` giving `y. d. v.` before and `y. v. d.` after where 1.4.0 gives `y.`, v1 contributing nothing at all for a family that is all particles (rules/R2's territory, and a divergence decisions/R2 has already decided in favor of). A view that stopped honoring a record had been quietly reproducing a v1 bug that v1 did not have. ## CLAUSE-CONTENT-OVERRULES-DELIMITER — content wins @@ -79,11 +79,11 @@ Problem shape. Two string shapes collide — the same written form means two dif ## RECORDED-ROSTERS — record the answer, don't re-derive it -Problem shape. A guard needs to know what the answer WAS, so it can detect the answer changing. Contract statement. Store the measured answer as literal data (a roster) and compare against it; never re-derive the expectation from the same inputs the check reads, because a derivation from the same data always agrees with itself. Known limit, and it is what the contract statement does NOT cover: a roster row can hold a recorded INPUT as well as a recorded expectation, and only the expectation half is asserted. The test to apply to a recorded input is not "is it checked" but "would the consumer answer DIFFERENTLY if it were wrong". The differential ledgers' rosters were this entry's worked cases (`_CROSS_RULE_WINNERS` carried wrong recorded shapes that agreed with themselves until #497 moved them where a run measured them), and were retired with the ledgers in #664; decisions.md keeps their measurements. How it works. The purest case is a recorded expectation whose only consumer is the check that compares it, re-derived by nothing: tools/differential/snapshot.jsonl records every corpus name's reading, and tests/v2/test_snapshot.py compares the tree against it. Lives in. tools/differential/snapshot.jsonl, tests/v2/test_facade_cases.py (_CORE_ONLY_IDS), tests/v2/test_benchmark.py (_CALL_BASELINE). Reach for it when. Writing a check whose expected value is computed by the code under test, or a comment that enumerates ids/counts — make it data the suite asserts. And when a roster row carries anything the check CONSUMES rather than compares, ask what measures that half. +Problem shape. A guard needs to know what the answer WAS, so it can detect the answer changing. Contract statement. Store the measured answer as literal data (a roster) and compare against it; never re-derive the expectation from the same inputs the check reads, because a derivation from the same data always agrees with itself. Known limit, and it is what the contract statement does NOT cover: a roster row can hold a recorded INPUT as well as a recorded expectation, and only the expectation half is asserted. The test to apply to a recorded input is not "is it checked" but "would the consumer answer DIFFERENTLY if it were wrong". The differential ledgers' rosters were this entry's worked cases (`_CROSS_RULE_WINNERS` carried wrong recorded shapes that agreed with themselves until #497 moved them where a run measured them), and were retired with the ledgers in #664; decisions/differential-ledger keeps their measurements. How it works. The purest case is a recorded expectation whose only consumer is the check that compares it, re-derived by nothing: tools/differential/snapshot.jsonl records every corpus name's reading, and tests/v2/test_snapshot.py compares the tree against it. Lives in. tools/differential/snapshot.jsonl, tests/v2/test_facade_cases.py (_CORE_ONLY_IDS), tests/v2/test_benchmark.py (_CALL_BASELINE). Reach for it when. Writing a check whose expected value is computed by the code under test, or a comment that enumerates ids/counts — make it data the suite asserts. And when a roster row carries anything the check CONSUMES rather than compares, ask what measures that half. ## LEDGER-RULE-SEPARATION — file order decides, fields narrow by subset -Problem shape. Two differential-ledger rules claimed overlapping names. Contract statement. Retired with the ledgers (#664): the snapshot that replaced them classifies nothing, so no rule can claim a name and no order can decide a contest. How it works. Through 2.4, each `tools/differential/expected_since_*.toml` rule carried a `name_regex` and `fields`, file order decided every contest, and wide-first pairs had to declare themselves; decisions.md#differential-ledger records the arc. Lives in. Nothing; git history before #664. Reach for it when. Reading a decisions.md entry that cites it. +Problem shape. Two differential-ledger rules claimed overlapping names. Contract statement. Retired with the ledgers (#664): the snapshot that replaced them classifies nothing, so no rule can claim a name and no order can decide a contest. How it works. Through 2.4, each `tools/differential/expected_since_*.toml` rule carried a `name_regex` and `fields`, file order decided every contest, and wide-first pairs had to declare themselves; decisions/differential-ledger records the arc. Lives in. Nothing; git history before #664. Reach for it when. Reading a decisions entry that cites it. ## CANONICAL-VOCABULARY-AT-THE-BOUNDARY — one vocabulary at the comparison @@ -95,7 +95,7 @@ Problem shape. A test's input depends on two config sets intersecting (a word th ## CROSS-RULE-OUTCOME-PINS — pin who wins the contest -Problem shape. Which differential-ledger rule won a contested name was measured by no total, and was exactly what a reorder or a narrowing changed. Contract statement. Retired with the ledgers (#664): with nothing classifying a diff there is no contest to pin. How it works. Through 2.4, `_CROSS_RULE_WINNERS` in tests/v2/test_ledger_guards.py pinned which rule classified each contested name, and `compare.unowned_contests` refused an unpinned one (#498, #508). Lives in. Nothing; git history before #664. Reach for it when. Reading a decisions.md entry that cites it. +Problem shape. Which differential-ledger rule won a contested name was measured by no total, and was exactly what a reorder or a narrowing changed. Contract statement. Retired with the ledgers (#664): with nothing classifying a diff there is no contest to pin. How it works. Through 2.4, `_CROSS_RULE_WINNERS` in tests/v2/test_ledger_guards.py pinned which rule classified each contested name, and `compare.unowned_contests` refused an unpinned one (#498, #508). Lives in. Nothing; git history before #664. Reach for it when. Reading a decisions entry that cites it. ## VOCABULARY-FEEDS-STRUCTURE — a wordlist edit can move the comma decision @@ -103,7 +103,7 @@ Problem shape. Editing suffix vocabulary looks field-local, but the comma-struct ## WARN-AT-THE-CALLER — walk out of the library, don't count frames -Problem shape. A warning should point at the caller's code, but the entry depth varies (constructor, add(), unpickle, dataclasses.replace, the shim snapshot), so any fixed stacklevel lands on library internals for most paths. Contract statement. The warner walks the stack outward until the first frame outside the library's own modules and warns there, instead of counting frames. Lives in. nameparser/_lexicon.py (_warn_dead_entry), and the related but distinct per-read-location choice is in the #293/#354 bridge (decisions.md#3-0-reevaluations). First measured instance of the hazard: the delegated bytes-deprecation warnings shipped with fixed stacklevel=2 and needed the follow-up fix (0044073). Reach for it when. Adding any warning reachable through more than one public entry point. +Problem shape. A warning should point at the caller's code, but the entry depth varies (constructor, add(), unpickle, dataclasses.replace, the shim snapshot), so any fixed stacklevel lands on library internals for most paths. Contract statement. The warner walks the stack outward until the first frame outside the library's own modules and warns there, instead of counting frames. Lives in. nameparser/_lexicon.py (_warn_dead_entry), and the related but distinct per-read-location choice is in the #293/#354 bridge (decisions/3-0-reevaluations). First measured instance of the hazard: the delegated bytes-deprecation warnings shipped with fixed stacklevel=2 and needed the follow-up fix (0044073). Reach for it when. Adding any warning reachable through more than one public entry point. ## LEGACY-STATE-SIGNATURE — subtract legacy defaults only as a complete set @@ -117,7 +117,7 @@ a fork for all ambiguous particles on "Dr. Van Jr.". Lives in. The AmbiguityKind ## MAKE-WRONG-STATES-UNREPRESENTABLE — the house meta-pattern -Problem shape. A convention keeps being violated no matter how clearly it is written down. Contract statement. Convert the consistency problem into a referential-integrity problem: make the wrong state impossible to express, or mechanically checked, rather than merely documented. How it works. Three instances built this documentation system: _CORE_ONLY_IDS replaced an enumerating comment; verbatim-excerpt citations replaced paraphrase; no-boundary: markers replaced silent omission. The executable examples in rules.md are the same move at document scale. Lives in. tests/v2/test_doc_citations.py, tests/v2/test_rules_doc.py, and every recorded roster. Reach for it when. Tempted to write "remember to keep X in sync with Y." +Problem shape. A convention keeps being violated no matter how clearly it is written down. Contract statement. Convert the consistency problem into a referential-integrity problem: make the wrong state impossible to express, or mechanically checked, rather than merely documented. How it works. Three instances built this documentation system: _CORE_ONLY_IDS replaced an enumerating comment; verbatim-excerpt citations replaced paraphrase; no-boundary: markers replaced silent omission. The executable examples in the rules are the same move at document scale. Lives in. tests/v2/test_doc_citations.py, tests/v2/test_rules_doc.py, and every recorded roster. Reach for it when. Tempted to write "remember to keep X in sync with Y." ## LOCALE-PACKS-PURE-DATA — packs configure, they never compute @@ -125,7 +125,7 @@ Problem shape. Language-specific behavior needs a home that cannot drift from th ## FACADE-CONTRACT — HumanName wraps the core; 1.4-warning-free code keeps working -Problem shape. Where does v1-compatibility behavior live, and what may it do? Contract statement. HumanName is a mutable facade over the immutable core: code that runs warning-free on 1.4 keeps working through 2.x with identical results EXCEPT release-log-classified fixes, verified for 2.0 by the differential ledger and since #664 visible against 1.4.0 through `tools/differential/compare.py --baseline 1.4.0` (the RFC's compatibility promise carries the same carve-out) — via validating setters, dirty-tracked re-parses, and pickle round-trips — and the facade never calls the v1 parsing hooks it still carries. "Warning-free" describes the code's 1.4 behavior, not a promise it STAYS warning-free: 2.x adds deliberate warnings (the multi-word-vocabulary and field-assignment ones), each release-log-classified, always with identical results. Lives in. nameparser/_facade.py; v1 import paths preserved by nameparser/parser.py and nameparser/config/. Reach for it when. A core change needs a v1-visible behavior — the facade, not the core, is where parity lives (and cite decisions.md#3-0-reevaluations when parity is the only reason). +Problem shape. Where does v1-compatibility behavior live, and what may it do? Contract statement. HumanName is a mutable facade over the immutable core: code that runs warning-free on 1.4 keeps working through 2.x with identical results EXCEPT release-log-classified fixes, verified for 2.0 by the differential ledger and since #664 visible against 1.4.0 through `tools/differential/compare.py --baseline 1.4.0` (the RFC's compatibility promise carries the same carve-out) — via validating setters, dirty-tracked re-parses, and pickle round-trips — and the facade never calls the v1 parsing hooks it still carries. "Warning-free" describes the code's 1.4 behavior, not a promise it STAYS warning-free: 2.x adds deliberate warnings (the multi-word-vocabulary and field-assignment ones), each release-log-classified, always with identical results. Lives in. nameparser/_facade.py; v1 import paths preserved by nameparser/parser.py and nameparser/config/. Reach for it when. A core change needs a v1-visible behavior — the facade, not the core, is where parity lives (and cite decisions/3-0-reevaluations when parity is the only reason). ## CONFIG-SHIM-SNAPSHOT — v1 config mutations reach the core by snapshot @@ -149,7 +149,7 @@ Contract statement. A user-supplied pattern is rejected as over-matching by prob ### VOCABULARY-EXERCISES-FORKS — pin the branch, not the wordlist -Contract statement. A vocabulary set a caller configures is data; the code that reads it is what has behavior, so a case row earns its place by exercising a FORK — a branch the parser takes for one INPUT and not another — never by covering one more member. The unit is the input, not the member, because most forks are keyed on the shape of the surrounding name rather than on which word appeared. Find them by reading the sites that consult the set and enumerating the branches they take, and expect that enumeration to come back empty: `maiden_markers` is read at exactly two sites (`_classify.py`'s tagging and `_extract.py`'s `_maiden_marked`), both plain whole-token membership after `_normalize`, and every shipped member parses to one structural class in bare, bracketed and lone-bracketed shapes with the marker text normalised out — zero member forks. What earns the maiden rows is clause SHAPE: marker first against marker later, a word after the marker against none, a marker a separator divides from the name against one glued to it. Two adjacent things are NOT forks of this set and do not want rows: the substring-versus-token split is a question tools/differential/README.md's coverage recipe asks wider than the parser does, and edge-period normalization lives in `_lexicon._normalize`, applying to every token of every vocabulary, so it discriminates spellings rather than members. Only "a member borne as an ordinary name" is member-keyed (decisions.md#vocabulary-collisions), and it has its own rows. The reduction does NOT carry across to delimiter pairs, and the difference is the line to hold: a pair the library SHIPS is caller-visible behavior, and vocabulary a caller configures is data — deleting `("«", "»")` from the defaults reddens three tests, so cases.py's one-row-per-shipped-pair convention is load-bearing and this criterion is not permission to delete those rows. Within the pairs the code does fork, on whether open and close are the same character (N2) and again inside that class, where `WORD_INTERNAL_DELIMITERS` carves the apostrophe out: `Jane Smith"` reports UNBALANCED_DELIMITER and `Jane Smith'` reports nothing. ASCII against fullwidth is not one of them — swapping `( )` for `（ ）` across every corpus name holding a paren changes no field and no ambiguity kind. A row may also earn its place a second way, with no fork behind it: pinning a name a release note advertises BY NAME, which is a promise to a reader rather than a branch in the parser. It follows that "N of M entries have no test" is not a finding; it reads as one because it carries a number, and it inflates with every entry added, which is the tell that it measures the data. The finding is "this fork has no test". State the argument without present-tense counts over a drifting wordlist — see docs/design/AGENTS.md — and where a count is genuinely wanted, ship the one-liner instead: +Contract statement. A vocabulary set a caller configures is data; the code that reads it is what has behavior, so a case row earns its place by exercising a FORK — a branch the parser takes for one INPUT and not another — never by covering one more member. The unit is the input, not the member, because most forks are keyed on the shape of the surrounding name rather than on which word appeared. Find them by reading the sites that consult the set and enumerating the branches they take, and expect that enumeration to come back empty: `maiden_markers` is read at exactly two sites (`_classify.py`'s tagging and `_extract.py`'s `_maiden_marked`), both plain whole-token membership after `_normalize`, and every shipped member parses to one structural class in bare, bracketed and lone-bracketed shapes with the marker text normalised out — zero member forks. What earns the maiden rows is clause SHAPE: marker first against marker later, a word after the marker against none, a marker a separator divides from the name against one glued to it. Two adjacent things are NOT forks of this set and do not want rows: the substring-versus-token split is a question tools/differential/README.md's coverage recipe asks wider than the parser does, and edge-period normalization lives in `_lexicon._normalize`, applying to every token of every vocabulary, so it discriminates spellings rather than members. Only "a member borne as an ordinary name" is member-keyed (decisions/vocabulary-collisions), and it has its own rows. The reduction does NOT carry across to delimiter pairs, and the difference is the line to hold: a pair the library SHIPS is caller-visible behavior, and vocabulary a caller configures is data — deleting `("«", "»")` from the defaults reddens three tests, so cases.py's one-row-per-shipped-pair convention is load-bearing and this criterion is not permission to delete those rows. Within the pairs the code does fork, on whether open and close are the same character (N2) and again inside that class, where `WORD_INTERNAL_DELIMITERS` carves the apostrophe out: `Jane Smith"` reports UNBALANCED_DELIMITER and `Jane Smith'` reports nothing. ASCII against fullwidth is not one of them — swapping `( )` for `（ ）` across every corpus name holding a paren changes no field and no ambiguity kind. A row may also earn its place a second way, with no fork behind it: pinning a name a release note advertises BY NAME, which is a promise to a reader rather than a branch in the parser. It follows that "N of M entries have no test" is not a finding; it reads as one because it carries a number, and it inflates with every entry added, which is the tell that it measures the data. The finding is "this fork has no test". State the argument without present-tense counts over a drifting wordlist — see docs/design/AGENTS.md — and where a count is genuinely wanted, ship the one-liner instead: ``` uv run python -c " @@ -167,7 +167,7 @@ print(len(MAIDEN_MARKERS), 'members ->', len(cls), 'structural classes')" ### FORCE-A-DECISION-TABLE — no silent defaults on growth -Contract statement. Where adding an enum member or a file must not silently inherit a default, a local table's key set is asserted equal to the population, so growth fails the suite until someone decides — against a local table, not the constant under test. Exemplar: tests/v2/pipeline/test_vocab.py's per-script initials check; reused for the pack registry (test_registry_is_the_pack_contract: every registered pack must ship DEVIATES and its rotator list or the suite fails structurally — decisions.md#deviates-registry). Known gap it exposes: DEFAULT_SCRIPT_ORDERS has no such guard. +Contract statement. Where adding an enum member or a file must not silently inherit a default, a local table's key set is asserted equal to the population, so growth fails the suite until someone decides — against a local table, not the constant under test. Exemplar: tests/v2/pipeline/test_vocab.py's per-script initials check; reused for the pack registry (test_registry_is_the_pack_contract: every registered pack must ship DEVIATES and its rotator list or the suite fails structurally — decisions/deviates-registry). Known gap it exposes: DEFAULT_SCRIPT_ORDERS has no such guard. ### SELF-EXPIRING-GUARD — a decline keyed to a measured defect @@ -184,7 +184,7 @@ Problem shape. A test pins an ordering, a sort, a dedup or a partition, and its ### Field notes — the traps themselves - Test a function INSIDE a stage on the inputs its caller hands it, not on the stage's finished output. #610's first sweep test built pieces by running `group` whole and handed them to the maiden take's candidate function, but the take runs before any join, so the test read `de VI Prof.` as one piece where production hands it three: a guard checking a structure production never passes in, green for the wrong reason. Review moved the oracle to the take itself by wrapping it (tests/v2/pipeline/test_pieces.py), and recorded the old loop's misses there as the negative control (432), which is what shows the moved harness can still see a difference. -- Enumerate the rules that BUILD a structure; do not recall them. #395's unit walk was written three times in one PR — P2's chain missing, then the conjunction and bound-given branches absorbing one token where the particle branch absorbed a unit, then the suffix stop — and each miss came from listing the joining rules from memory instead of reading them out of rules.md. Both later misses reproduced the very defect the first fix had just removed, mirrored. +- Enumerate the rules that BUILD a structure; do not recall them. #395's unit walk was written three times in one PR — P2's chain missing, then the conjunction and bound-given branches absorbing one token where the particle branch absorbed a unit, then the suffix stop — and each miss came from listing the joining rules from memory instead of reading them out of the rules. Both later misses reproduced the very defect the first fix had just removed, mirrored. - Prefer monkeypatching a module-level helper to re-exec'ing a module: a caller looks a module global up at CALL time, so replacing `_units` or `_name_positions` reaches it and none of the traps below apply. Re-exec is only needed for logic INLINE in a function, and "the thing I want to mutate is not a module-level helper" is a hint that it could be one. - Re-exec'ing a module redefines the classes that module DEFINES, not the ones it imports — the exec re-runs its own import statements, which resolve through sys.modules. So `is` comparisons break selectively: re-exec'ing `_group.py` yields a second `BoundJoin` (defined there) while `Structure` (imported) stays identical, and `bound_join is not BoundJoin.DISABLED` is then always true, joining a family comma's own segment and reporting 3,150 phantom movers. Rebind what the module defines (`mod.__dict__["BoundJoin"] = real.BoundJoin`) and assert the identity rather than assuming it either way. - `_pipeline.STAGES` holds function OBJECTS captured at import, so re-exec'ing a stage module leaves the pipeline running the old function and every result is a false negative. Rewire `STAGES` and assert the new function is in it. @@ -200,7 +200,7 @@ Problem shape. A test pins an ordering, a sort, a dedup or a partition, and its - A check that is quiet about a name is not evidence that the baseline read it the same. #603's first draft wrote that 1.4.0 read a title past the second comma as a title because no 1.4.0 name moved unexplained, when the 1.4.0 ledger's broad `fix(comma-family)` rule was claiming each one; the released wheel read suffix 'Jr., Attorney General'. Ask the wheel (`PYTHONSAFEPATH=1 uv run --isolated --no-project --with nameparser==X`, from outside the worktree, or `tools/differential/compare.py --baseline X`) before writing what a release did. - A skip is indistinguishable from "correctly declined": pytest turns an empty parametrize into a skip, and a filter that widens its own skip set cannot fail. After changing any selection shape, verify the guard still REACHES the code it watches — assert the selected set is non-empty, or force-a-decision on its size. - A differential corpus cannot evidence behavior keyed to OUT-of-vocabulary shapes: it holds only names someone wrote down, and an unrecognized word is by definition outside the vocabulary — a green run over the corpus proves nothing about such a rule. -- The corpus can be near-blind to a WRITING CONVENTION even where the vocabulary is well covered, and a small honest count then reads as a small blast radius. Measured for rules.md#P6: of 782 corpus names, 245 carry a comma, TWO of those end in a particle, and ONE of the two clears the words-to-spare guard and actually moves — so the Dutch trailing-tussenvoegsel listing the rule exists for is essentially unsampled. Before reporting "N names move", report the size of the population that COULD move; when that is ~1, the number is evidence about the corpus. This note's first wording said "exactly ONE ends in a particle", conflating the population with the movers — the very error it exists to prevent, committed inside the correction. Caught by re-running the count against AGENTS.md's axis 1 rather than by re-reading it. +- The corpus can be near-blind to a WRITING CONVENTION even where the vocabulary is well covered, and a small honest count then reads as a small blast radius. Measured for rules/P6: of 782 corpus names, 245 carry a comma, TWO of those end in a particle, and ONE of the two clears the words-to-spare guard and actually moves — so the Dutch trailing-tussenvoegsel listing the rule exists for is essentially unsampled. Before reporting "N names move", report the size of the population that COULD move; when that is ~1, the number is evidence about the corpus. This note's first wording said "exactly ONE ends in a particle", conflating the population with the movers — the very error it exists to prevent, committed inside the correction. Caught by re-running the count against AGENTS.md's axis 1 rather than by re-reading it. - An invariant judged on the part a JOIN PRODUCED is satisfied by whatever the join absorbed. #397's both-sides condition was stated as "a name word stands on each side of the link", and the invariant written from that sentence walks the joined part and asks whether a non-connective word stands on each side of the letter — which a swallowed credential answers yes to, being that word. `Josep Lluis Carod i III` read family 'Carod i III' and PASSED it. The merge has already destroyed the evidence, so the question cannot be re-asked downstream at all. Judge on the PRE-join classes, or against a second parse with the rule turned off — the off-switch parse puts the credential in `suffix`, and "no word the off-switch reading calls a suffix lands inside a joined name part" caught it on 639 parses of the shipped grid where the first invariant caught none (960 over the full cross product, a dated snapshot before the grid was trimmed by shape). The same pairing is worth reaching for whenever a rule MERGES tokens. - A detector that re-implements a rule's grouping will get the grouping wrong. Derive the boundary from the same vocabulary the rule reads, not from the half you happen to be thinking about: walking a particle run over the NEVER-GIVEN set alone (the rule chains through ANY particle) split "de la Vega" after "de la" and reported 50 false movers for #364, where the true count is one. Both wrong answers were plausible and printed cleanly. - Guard the whole family, parametrize over it: a defect on one of N parallel entry points hides behind a per-example test — three times in one session (a guard on one class of two, a decode hint on 3 of 5 entry points, a sync roster missing 4 copies) — and a {class}×{field}×{bad-value} parametrization is what caught each. diff --git a/docs/design/rules.md b/docs/design/rules.md deleted file mode 100644 index 90dfb841..00000000 --- a/docs/design/rules.md +++ /dev/null @@ -1,3053 +0,0 @@ -# Parsing rules - -This document is NORMATIVE, not descriptive: the rules state how names are written and what should happen when they are parsed, grounded in how people understand names — not in what the parser currently does. The parser implements these rules. Where it does not yet, the gap is a tracked deviation (`deviates:` marker below), not a counterexample. Statements are implementation-free: no stage names, no function names, no regexes. - -Authority, scoped: `tests/v2/cases.py` pins CURRENT behavior; this document states INTENDED behavior. A mismatch between them must be classified, never defaulted: either the rule is wrong (fix it here) or the parser is wrong (the example takes a `deviates:` marker and an issue). Where this document is silent, the behavior is pinned-but-undocumented — an extraction gap to close, not a specification, and not license to change the behavior. - -Rule IDs are stable forever: never renumbered, never reused. A retired rule keeps its ID with a one-line tombstone pointing at decisions.md. Cross-references use the anchor form `decisions.md#P2` / `mechanisms.md#SPANS`; a bare ID is never a citation. The `interacts:` field on a pointer line is advisory — the citation-integrity test checks the ID exists, not that the interaction is real. `implemented:` and `tracked:` are not advisory: every rule carries exactly one. `implemented:` names the modules honoring the rule, checked against the modules that cite it; `tracked:` names the issues that would ship a rule nothing implements yet, so a wholly-aspirational rule cannot sit here untracked, and a shipped rule cannot keep a stale tracking pointer. - -Every example line is EXECUTABLE. The grammar (its executable definition is `tests/v2/rules_doc.py`; `tests/v2/test_rules_doc.py` runs every line): - - "INPUT" [annotation] → field="value" [· boundary] - [deviates: #N (today: field="value")] - -An `annotation` names a policy, locale (`[ru]`), or extras gate (`[ja+segmenter]`) in the registry beside the test. `· boundary` marks the non-firing example every rule must carry: an input shaped like the rule's subject where its effect does NOT occur. That is usually the rule's OWN stated exception — H1's given-name title, P3's single-letter carve-out — not an input the rule never reaches, so the exception is executable rather than merely asserted. Or the rule declares `no-boundary: ` instead, so skipping the boundary is a recorded decision. `deviates:` states the INTENDED output on the example line while the marker records TODAY's output and the tracking issue; the runner asserts today's output strictly, so a parser change that closes the gap fails the suite until the marker is removed in the same PR. `grep deviates:` on this file is the deviation backlog (deviations from statable rules — coverage gaps are a separate, larger category no grep can see, and contested vocabulary memberships a third, tracked as Open blocks keyed to the vocabulary set in decisions.md). - -One class of rule stands outside the normative reading and declares it on a `tolerated: ` line beside its pointer line. Such a rule is ABOUT an input the writing systems it speaks about do not produce — a shape no convention writes, read best-effort because parsing is total over strings — and it describes what the parser does with that input instead of promising it. Its statement is still precedence-bearing and its examples are still executable and still run, but they illustrate rather than promise, and the behavior is changeable without notice. W3 is the only such rule today (the 2026-09-01 comma demotion). - -The marker's unit is the whole rule, so a clause that reaches past the tolerated shape does not belong in a marked rule — say it in a normative one. The marker changes what the rule promises, not what is watched: a marked rule's examples are in `tools/differential/corpus_rules.jsonl` like any other, so the snapshot shows when their reading moves (#664). - -## Not in scope - -- **Language detection.** The parser never infers a language from Latin-script text: transliteration destroys the signal ("Ali", - "Van", "Bin" each belong to several languages with conflicting - readings). Language-specific behavior is opt-in configuration. Script-conditional behavior exists only where the script itself settles the convention (see the W section). -- **Grammatical inflection.** Names inflect in many languages (vocative, genitive); this library neither produces nor consumes inflected forms. CLDR personNames draws the same line. -- **Validation.** Deciding whether a string IS a person's name is not parsing; `parse()` is total over strings and never rejects input. -- **Vocabulary completeness.** No shipped wordlist parses every name, and none is meant to. The promise is that every TYPE of vocabulary has a mechanism — markers, particles, titles, suffixes, bound given names — and that callers configure the members their data needs. A missing entry is a configuration answer, not a defect, and what the tests owe is one exercise per behavioral fork rather than one per entry (mechanisms.md#VOCABULARY-EXERCISES-FORKS). -- **Comparison.** matches()/comparison_key() are a value-API surface, not parsing; their design record is decisions.md#comparison-surface. - -## Titles & honorifics (H) - -Background: an honorific title precedes a name and is not itself part of it; it addresses or ranks the person. Most titles address by surname ("Mr. Johnson"), but a few — knighthoods, some clerical and courtesy titles — address by given name ("Sir John"). The library keeps a vocabulary of titles and, separately, of these given-name titles. A period-marked word is claimed by SHAPE at the front and by VOCABULARY at the back. At the front an unlisted abbreviation is a title (H2), the shape outranking the vocabulary; at the back there is no shape rule, so only a listed title word chains into the title (H5) and an unlisted abbreviation stays a name word. A BARE title word at the back is a name word too, TITLES holding ordinary surnames — king, judge, bishop — which is what bars the blanket vocabulary-wins answer (#316, decided 2026-09-08). An input the title peel eats down to one last title-vocabulary word is H4's, and what it does with that word is a convention rather than a reading of the vocabulary. Two criteria govern two different questions here. Membership in the given-name-title list follows HOW THE TITLE ADDRESSES: a title that precedes and addresses by the given name belongs (Sir, Sheikh, the Arabic honorifics الدكتور/الشيخ — which qualify even though those traditions fully retain family names). Whether an EMPTY FAMILY is correct output is the separate question, governed by surname retention: renunciation abolishes the surname, so for Swami, Guru, Baba or Lama family="" is right (#346), while rabbi and imam traditions keep surnames — "Rabbi Cohen" addresses by title and keeps family "Cohen". Conflating the two criteria either ejects the Arabic entries or sweeps in titles that break -"Rabbi Cohen". - -H1. Rationale: a title normally addresses by surname, so a title - followed by a single name word usually names the family; but a - given-name title addresses by given name. Several titles written - together are one form of address, and the one that does the - addressing is the last — the words in front of it rank the - person rather than name them. What stands beside the name word — - a suffix, a nickname, a maiden name — does not make the name any - longer, so it does not decide this reading. And a run written - BEHIND the one name word is the same form of address written on - the other side of it, so where no title run stands in front of - that word it decides the same reading. Where one does stand in - front, that run addresses: a title written behind the name is - the name plus that title (H5), and adding it cannot change how - the words in front of the name are read. Which side a title is - on is decided by the NAME WORD and by nothing else standing - beside it, so a run written around a nickname is one run: the - name Dr. 'Smitty' Sir John reads as Dr. Sir John does. - A title followed by exactly one name word makes that word the - family name, whatever suffix, nickname or maiden name stands - beside it, unless the title is a given-name title, which keeps - it the given name; a run of several titles addresses as its last - title does, and where a run stands BEFORE the one name word it - is the run that addresses, a run standing behind it deciding - that word's field only when none stands before. - "Mr. Johnson" → family="Johnson" - "Mrs. Garcia" → family="Garcia" - "Dr. Smith née Jones" → family="Smith" - "Her Majesty Queen Elizabeth" → given="Elizabeth" - "Dr. Sir John" → given="John" - "Smith Sir." → given="Smith" - "Sir John Prof." → given="John" - "Dr. Smith Sir." → family="Smith" - "His Excellency Lord Duncan" → family="Duncan" - "Her Royal Highness Princess Anne" → given="Anne" - "Sir John" → given="John" · boundary - Accepted: a given-name title plus one name word leaves the - family empty — the input names no family, and inventing one - would be worse. A run whose last title is a given-name title - reads the same way, `queen` doing the addressing where `her - majesty` only ranks. - "Sir John" → family="" - "Her Majesty Queen Elizabeth" → family="" - "Smith Sir." → family="" - Accepted: a title word in the run that no vocabulary knows does - not change what the run addresses by, the last word being the - one asked — `His Excellency Lord Duncan` reads family `Duncan` - because `lord` is not a given-name title, not because the run is - long. `lord` and `lady` stay out of that list by decision - (#519): each addresses by given name only as a courtesy style for - children of the senior ranks (`Lord Peter`, `Lady Diana`) and by - title or surname for every peer and every wife (`Lord Byron`, - `Lady Thatcher`), and the text does not say which the bearer is, - which the list cannot express; `prince` and `princess` are in it. - Accepted: a title word that is also borne as a name, though - rarely as the first word of one (`graf`, `greve`, `herra`), is - the title in front of a name, so a name that BEGINS with it -- a - surname written first without a comma, or a given name -- gives - that word to the title, under a declared family-first order too. - Written last without a period, or alone before a family comma, - it is the name; a period behind it makes it the title by H5, as - for `Mary Jane King.` (#606), and a name word behind it before - a family comma makes it the title by H6. - "Greve Smith, Anna" → title="Greve" - "Greve Anna" → family="Anna" - "Anna Greve" → family="Greve" - "Greve, Anna" → family="Greve" - history: decisions.md#H1 · interacts: H3, H5, H6, P2, P3, P5, M2, S1, S2, N1, N3 · implemented: nameparser/_pipeline/_post_rules.py - -H2. Rationale: before a name, an abbreviation is almost always a - title — "Rev.", "Ing.", "Mag." — and no vocabulary can list - every profession's abbreviations in every language. - An abbreviation opening the part of the name that carries the - given name — the whole name, or the part after a family comma — - reads as a title even when unlisted, provided it is an unbroken - run of two or more letters ending in its one period; a bare - initial does not, and neither does anything with interior - periods, hyphens or digits. Where it fires, the shape outranks - vocabulary: a period-marked opening word is a title even when - the word is suffix vocabulary — except after a family comma, - where a part that is nothing but suffix words is the credential - run (C1) and the abbreviation opens nothing. An accented Latin, - Greek or Cyrillic letter written as a base letter plus a - combining mark (NFD) is the same letter as the composed one, - for this shape and for the dotted and all-caps credential - shapes C1, S2 and S3 read (#652), so such a name parses the - same in either encoding and the output keeps the one written. - "Rev. John Smith" → title="Rev." - "Xyz. John Smith" → title="Xyz." - "Smith, Major. John" → title="Major." - "Esq. Smith" → title="Esq." - "J. Smith" → given="J." · boundary - "J.R. Smith" → given="J.R." · boundary - "Dr. inž. Janez Novak" → title="Dr. inž." - Accepted: the shape is only recognizable as an unbroken run of - letters, so an abbreviation in a script whose letters carry - combining vowel signs (Bengali, Devanagari) never reads as a - title by shape — an unlisted abugida honorific stays a name - word, and only vocabulary can recognize it — which is what #343 - and #344 supply for Bengali and Devanagari. - "প্রকৌশলী. Sen" → given="প্রকৌশলী." - Accepted: the shape reads a Latin convention, and a script with - no initials has no period abbreviations either, so a period- - marked opening word carrying a Han, kana or hangul character — - as the script table classifies them, halfwidth katakana - included since #594 — is a name word and never a - title by shape (#323; decisions.md#cjk-full-stops) — the same - veto that keeps 씨. from reading as an initial. The Latin - reading is unchanged, and W3's example block carries the CJK - reading this clause vetoes into. The veto reads the composed - spelling, so hangul typed decomposed, as conjoining jamo the - script table does not hold, is hangul to it too (#665); the - second line below is typed that way. - "Smith. John" → title="Smith." - "김. 민준" → title="" - Accepted: before a family comma the shape does not fire: the - titles there are the vocabulary's alone (H6), so an unlisted - abbreviation opening that part stays in the family. - "Xyz. Smith, John" → family="Xyz. Smith" - Accepted: after a family comma a part that is nothing but suffix - words is the credential run (C1), which the abbreviation does - not open: the vocabulary decides, and "Esq." is the postnominal - it is. - "Smith, Esq." → suffix="Esq." - history: decisions.md#H2 · interacts: C1, S2, S3, P4, H5, H6, W3, W4 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_vocab.py - -H3. Rationale: compound titles are written as a run of title words, - connectives included; a title word standing inside the name is - just a name word. And a title addresses somebody, so the run - leaves somebody there to address: a post-nominal is written - about a name rather than being one, so it cannot be the name the - run left. - Successive title words at the start of the part carrying the - given name chain into one title; a title word elsewhere in the - name does not. The run leaves one NAME word standing, and a word - of the suffix vocabulary is not that word — where everything - behind the run is post-nominal, the run gives its last word back - to the name, provided that word stands alone and is not itself - suffix vocabulary. - "Asst. Vice Chancellor John Smith" → title="Asst. Vice Chancellor" - "Marquess of Bath" → title="Marquess of Bath" - "Morse, Det. Insp. Jane" → title="Det. Insp." - "Dr King Jr" → family="King" - "Dr King Jr" → suffix="Jr" - "MD DDS" → family="DDS" - "John Doctor Smith" → middle="Doctor" · boundary - Accepted: before a family comma the run is H6's, which stops - where nothing but post-nominals follows for the reason this one - gives a word back, and always leaves the family one word, - whatever that word is. - "Dr. Jr., John" → family="Dr." - Accepted: the word given back must stand alone, so a title - written as one joined unit stays whole and the post-nominal - behind it is the name — turning the unit into a name and leaving - no title at all is the worse of the two readings. - "Prince of Wales Jr" → title="Prince of Wales" - Accepted: where the run's whole content is the word given back, - no title is left to make the reading H1's, so the word stands as - the name and the parse reports the doubt (H4). - "Dr Jr" → given="Dr" - "Dr Jr" → ambiguities=("title-or-name",) - history: decisions.md#H3 · interacts: C1, H1, H4, H5, H6, S2 · implemented: nameparser/_pipeline/_pieces.py - -H4. Rationale: this is a name parser, not a title parser. Handed a - string the title peel eats down to one last word which is itself - title vocabulary, it still has to name somebody, and that word is - the only candidate there is — a guess fixed in advance, so the - same input reads the same way every time, not a claim that the - word is a surname. The same reasoning covers a string that is - nothing but post-nominal vocabulary: something has to be the name. - An input whose only remaining name word after the title peel is - itself title vocabulary reads that word as the name by convention - and reports `title-or-name`; an input whose every word is - post-nominal vocabulary reads its first word as a name and - reports `suffix-or-name`. A single title word reads as a title - with no name beside it and reports nothing: the peel took the - whole string, so no word was left standing to be read as a name - and no reading was chosen. - "Lord Chancellor" → family="Chancellor" - "Lord Chancellor" → ambiguities=("title-or-name",) - "The Right Hon. the President of the Queen's Bench Division" → family="Division" - "The Right Hon. the President of the Queen's Bench Division" → ambiguities=("title-or-name",) - "Dr. King" → ambiguities=("title-or-name",) - "Dr. King MD" → ambiguities=("title-or-name",) - "Dr." → title="Dr." · boundary - "Dr." → ambiguities=() - "Dr. Smith" → ambiguities=() - Accepted: ONE site takes both halves — the assignment that places - the lone name word — so neither report depends on the declared - order. Under the default order H1 retags the title-vocabulary - word from given to family afterwards; under a declared - family-first order the assignment places it in the family - directly and H1 never runs. Same reading either way, so the same - kind is reported either way. - Accepted: the suffix half is reached only where no title was - peeled first, so a title in front of the run takes the input out - of that half. What it does NOT do is take the input out of the - rule: the title run leaves a name word standing (H3), and where - that word is title vocabulary the TITLE half claims it — - `Dr. King MD` reports `title-or-name` on `King`, while `MD DDS` - reports nothing, `DDS` being no title. - "MD DDS" → ambiguities=() - "Rinpoche" → ambiguities=("suffix-or-name",) - "QC MP" → given="QC" - "Jr." → title="Jr." - "Jr." → ambiguities=() - Accepted: `Jr.` is post-nominal vocabulary and still reads as a - title, H2's opening-abbreviation shape outranking the vocabulary - where it fires — so the suffix half never sees a dotted lone - credential. - Accepted: the suffix half is scoped to names no script order - placed, so a lone CJK honorific written by itself — さん, 씨, - 선생님 — reports nothing. The same shape, read through the - glued-honorific rules (W2) and the script's own order, and left - to the arc that revisits those readings. That silence is a SCOPE - on the script rule having placed the name, not a claim about the - honorific: a caller who declares no script orders at all gets the - report on the same input, since nothing then resolves the order - and the word reaches the carve-out as any other lone credential - does. The pair is pinned in the case table rather than here, - a policy that empties the script table having no example spelling - in this document. - Accepted: one name UNIT of more than one word carrying title - vocabulary reports `title-or-name` as well, the fork there being - whether the title word inside the unit is a title at all. Usually - a join (P3), and a particle chain (P4) is the same shape and - reports too — `St van Bishop` reads family "van Bishop" with - `bishop` title vocabulary inside it. The - clause reaches EVERY join whose non-leading member is TITLES - vocabulary, not the one word that prompted it: `Smith and King`, - `John and King`, `Smith and Bishop` and `John of Judge` all - report it, as `John of Prince` and `Smith and Prince` do. That - reach is the king/judge/bishop collision set met inside a join, - and the report is the honest answer there rather than a - misfire — a second surname that is also title vocabulary is - exactly the doubt the kind names, and those words stay in the - vocabulary for the addressing forms, which is - decisions.md#vocabulary-collisions' call. - A join OPENING the name with a title word is a title run (H3) - instead. Behind a peeled title the join is not: a title needs a - following piece and the join is the last one, so `Attorney General - of Minnesota` and `Deputy Secretary of State` keep `Attorney` and - `Deputy` as the title and report the rest — the two corpus names - the clause reaches, and `General` or `Secretary` being a title is - exactly the fork. Nothing in front of the join silences the - clause, a title, a maiden name or a claimed word each deciding - which FIELD the unit takes and none of them whether the word - inside it is a title. The other measured inputs are pinned in the - case table rather than here. - history: decisions.md#H4 · interacts: H1, H2, H3, H5, S2, O5 · implemented: nameparser/_pipeline/_assign.py - -H5. Rationale: a word abbreviated with a period at the END of a name - is standing where a post-nominal stands, and the parts of a name - that get abbreviated are the ones outside it. There is no shape - rule there — H2's belongs to the front slot — so only a word the - vocabulary knows as a title is one, and a bare title word is a - name word, TITLES holding ordinary surnames. - After the trailing suffix run has been taken, successive single - words that wear the abbreviation shape and are title vocabulary - chain into the title from the end, leaving one name word - standing. A bare title word there is a name word, and an - unlisted abbreviation there is a name word. The title is - TRANSPARENT to the suffix reading: where two or more name words - stand, what stands once the chain is taken reads exactly as it - would read written without the title, plus the title. Where the - chain leaves ONE name word and no title run stands in front of - it, there is no second reading for it to be transparent to, and - the title behind that word decides its field as a title in front - of it would (H1). Where a run DOES stand in front, that run - addresses and the chained title only joins the title field, so - the transparency holds for the one-word name too. - "John Smith Prof." → title="Prof." - "John Smith Prof." → family="Smith" - "John Smith Prof. Dr." → title="Prof. Dr." - "Dr. John Smith Prof." → title="Dr. Prof." - "Smith, John Prof." → title="Prof." - "Smith Prof." → family="Smith" - "John Smith Sir" → family="Sir" - "Mary Jane King" → family="King" - "John Smith Esq." → suffix="Esq." - "John Smith Xyz." → family="Xyz." · boundary - Accepted: the trailing title words join the title field in input - order after any leading run, because the title view reads the - tokens in the order they were written. - "Dr. John Smith Prof." → family="Smith" - Accepted: transparency is what makes the suffix reading proof - against a title standing among the post-nominals — a title AHEAD - of a suffix word is taken all the same, and the suffix reading - is then taken over what stands, not over what stood. - "John Smith Prof. Jr." → suffix="Jr." - "John Smith Jr. Prof." → suffix="Jr." - "John Prof. MA" → suffix="MA" - "JOHN PROF. MA" → family="MA" - Accepted: the reach is the whole title vocabulary, the ordinary - surnames in it included. TITLES holds king, judge and bishop, - and a period written behind one of them is enough to make it the - title and take it out of the name: `Mary Jane King.` reads title - `King.` with family `Jane`, where the bare spelling keeps family - `King`. That is a cost accepted under the premise that the input - is a name (H Background), not a case this rule prevents — the - period is a writing convention rather than evidence about the - word, so every title whose spelling wears the abbreviation shape - is reachable this way, and the BARE spelling is what the - trailing slot is protected from. - "Mary Jane King." → title="King." - "Mary Jane King." → family="Jane" - Accepted: no fork is reported. Under the premise that the input - is a name, a period-marked word the title vocabulary knows is - not a reading a reader would hesitate over — where the doubt is - real it is the word left STANDING that carries it, which is H4's - report and not this rule's. - Accepted: the chain is read together with the trailing suffix - run (S2), once the connectives have joined (P3) and before the - particle chain is made, so a particle chain (P2) stops in front - of a trailing title rather than taking it into the family name. - A title word a connective joined into a longer unit is no single - word for the chain to take. After a family comma the given part's - chain stops in front of the trailing run of suffixes and titles, - read as the comma-less name reads it, so there too a trailing - title is the title and P6's attachment reads the particle in - front of it as though the title were absent. A maiden clause ends where the run the end of the name - reads without it begins (M2), so where a trailing rule reads the - part a trailing title ends the clause and is a title. - "John van der Berg Prof." → title="Prof." - "John van der Berg Prof." → family="van der Berg" - "Mary Smith née Jones Prof." → title="Prof." - "Smith, John van der Berg Prof." → title="Prof." - "Smith, John van der Berg Prof." → middle="van der Berg" - "Smith, Jan van Prof." → family="van Smith" - Accepted: what the chain leaves is also what counts as a name - word to spare (P5). A trailing title word is not one, so a bound - given-name word behind one joins exactly as it joins with the - title absent — and lands in the same field, the run in FRONT of - the joined pair being the run H1 asks about. Said of the - comma-less writing, which is where the chain runs before the - reserve is read. After a family comma the reserve reads no peel, - but the join declines a trailing title word the chain takes (P5), - so the title is read there too, save the shapes P5's Accepted - clause records, as the segment written with an ordinary given - name reads it. - "Berg, abdul Prof." → title="Prof." - history: decisions.md#H5 · interacts: H1, H2, H3, H4, M2, P2, P5, P6, S2, C1 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py - -H6. Rationale: the listing form moves the surname to the front and - leaves everything else where it was written, so a title written - ahead of the name ends up ahead of the surname — `Dr. Smith, - John` is `Dr. John Smith` indexed. The comma names the family, - and a title is no part of it. But the slot in front of a family - comma is a surname's slot as well, so only the vocabulary claims - a word there, never H2's shape, and a word that is a particle as - well as a title opens a surname rather than ranking a person. - A run of title words opening the part before a family comma is - the title, joining the title field in input order with any title - after the comma. The run admits a title word only as vocabulary - knows it, stops at a word that is also a particle, or where - nothing but post-nominals stands behind it (H3), and always - leaves the family at least one word. The comma decides the - family whatever - the order declared, so the reading is the same under every - name_order. - "Dr. Smith, John" → title="Dr." - "Dr. Smith, John" → family="Smith" - "Prof. Dr. Brühl, Ada" → title="Prof. Dr." - "Dr. de la Vega, Juan" → family="de la Vega" - "Sir Smith, John" → given="John" - "Prof. Smith, Dr. John" → title="Prof. Dr." - "Rev. Sr. Smith, Mary" → title="Rev. Sr." - "Dr. Mc Donald, John" → family="Mc Donald" - "Prof. Smith, John, PhD" → family="Smith" - "Dr., John" → family="Dr." · boundary - "Dr. King, John" → family="King" · boundary - "MD PhD Jr., John" → family="MD" · boundary - "St Pierre, Ed" → family="St Pierre" · boundary - "Dr. St. Pierre, Ed" → family="St. Pierre" · boundary - Accepted: a title word that is also an ordinary surname (king, - judge, bishop, greve) opening a surname of two words is read as - the title, as it is in front of a name with no comma (H1). A - double-barrelled surname that begins with such a word loses it. - "King Smith, John" → title="King" - Accepted: what keeps a word in the family is the particle stop - and nothing else. 'Freiherr' and 'St' are particles as well as - titles, so they stay in either spelling, keeping 'St Pierre' - whole and 'Freiherr von Berg' as C1 reads it. The other German - ranks are titles only, so they peel, though since 1919 a rank - is part of the legal surname for all of them alike; that is the - vocabulary's distinction, not this rule's. A title in front of - a particle still peels. - "Graf Lambsdorff, Otto" → title="Graf" - "Freiherr von Berg, Ed" → family="Freiherr von Berg" - "Dr. Freiherr von Berg, Ed" → title="Dr." - history: decisions.md#H6 · interacts: C1, H1, H2, H3, M2, P1, P2, P4, S2, W3 · implemented: nameparser/_pipeline/_assign.py - -## Particles & surname prefixes (P) - -Background: particles ("de", "la", "van", "von", "bin") link forward to a surname and are written as part of it. Some are never anyone's given name; others ("Van", "Bin") are ordinary given names in some cultures, so the vocabulary distinguishes never-given particles from ambiguous ones, and only the never-given ones license special treatment. A separate small vocabulary binds forward to a GIVEN name instead: words like "abdul" that are not complete given names alone (P5). Which particles fall on which side of the never-given line is its own open question (#360). - -P1. Rationale: a particle OPENING a name has the whole rest of the - name to link forward to, so it is doing a particle's work and the - piece it heads is a surname written out in full. That is evidence - about the writing, not about the word, which is why no declared - order contradicts it. - A never-given particle opening the name marks the name as - surname-only: the particle run and the name words it attaches to - are the family. It needs another name word to attach to. The run - is every particle in sequence, never-given and ambiguous alike - ("de la Vega" is one group, not "de" plus a separate "la Vega"). - An ambiguous particle keeps whatever reading its position gives - it. That the particle claims the FAMILY rather than a given name - holds under every order: this is evidence about how the name is - written, and a declared order governs only what no vocabulary has - claimed (O4) — the same precedence the script license takes in W4. - The OPENING position is the whole of this rule's subject. A - particle standing where the given name would go was a second site - until #467, and dropping it is a correction rather than a - narrowing: that slot holds what the caller DECLARED to be the - given name, and the never-given vocabulary supplies a reading - where position leaves the question open rather than vetoing one - position has already given. A particle ending the name is P6's, - and only where it landed in a MIDDLE. - - How MANY name words it attaches to depends on the order. Under a - family-first order it takes exactly ONE — declaring that order - asserts that what follows the family is not more surname. Under - the default order it takes the rest of the name, and that is a - reading rather than a gap in one: a never-given particle marks - where a surname BEGINS, surnames are routinely several words, - and no declaration has said this one ends before the string - does. So `de Mesnil Jean` reports the whole string as the - family, which is 1.4.0's reach as well (#471, declined by - design). One name word means one UNIT — a particle chain (P2), - a conjunction join (P3) or a bound given-name pair (P5) is - taken whole or not at all. A title does not move the opening - position (P4), but a family comma does end the question: the - comma has already fixed the surname, so there is no positional - read left for an order to narrow, and a particle opening the - part AFTER it takes the rest of that part whatever order is - declared. What is left over is not read by O4's rule for a - whole name, which would make the first leftover a second family - name; it is laid out as the positions AFTER the family in the - declared order, the family slot being already filled. - "de la Vega" → family="de la Vega" - "Sir de Mesnil" → family="de Mesnil" - "Mesnil de" family-first → given="de" - "de Mesnil Jean" → family="de Mesnil Jean" - "de Mesnil Jean" family-first → family="de Mesnil" - "de Mesnil Jean" family-first → given="Jean" - "Smith, de Mesnil Jean" family-first → family="Smith de Mesnil Jean" - "de la Vega y Santos Juan" family-first → family="de la Vega y Santos" - "ibn Awf abdul Rahman" family-first → given="abdul Rahman" - "de la Cruz Juan Carlos" family-first-given-last → given="Carlos" - "Mc Donald" → family="Mc Donald" - "de los Santos" → family="de los Santos" - "van Gogh" → given="van" · boundary - Accepted: the default-order reach has two faces, and the second - is a real cost. A surname-only string is read right, which is - what the reach is for; a family-first listing read under the - default order is read wrong, all of it landing in the family. - The remedy is the declaration or the comma — declare a - family-first order, which gives family `de la Cruz`, given - `Juan`, middle `Carlos`, or write the family comma. What - NEITHER order settles is the comma's alone: `de la Family - Family Given` and `de la Family Given Given` are the same - string shape, and no reading of the words tells them apart. - "de la Torre Vega" → family="de la Torre Vega" - "de la Cruz Juan Carlos" → family="de la Cruz Juan Carlos" - Accepted: a bare "de" stays the given name — there is nothing to - fold into, and inventing a surname would be worse. - "de" → given="de" - Accepted: stopping the run leaves a MIDDLE where the fold never - left one before, so O3 has something to claim that it could not - reach until now. The family it then reports is discontiguous in - the input — words 1-3 plus word 5 — and renders the folded word - first, which is R1's order, not this rule's doing. - "de la Cruz Juan Carlos" family-first+middle_as_family → family="Carlos de la Cruz" - Accepted: only the OPENING position is this rule's subject. A - particle chain standing inside the name is grouped normally (P2) - and positioned by the declared order, so a family-first reading - may report it as the given name; what the vocabulary forbids is - the bare particle reading as a given name, not any name part - that begins with one. - "Juan de la Vega" family-first → family="Juan" - history: decisions.md#P1 · interacts: O3, O4, P2, P3, P4, P5, P6 · implemented: nameparser/_pipeline/_post_rules.py - -P2. Rationale: a particle is written as part of the surname it - precedes, and a title stands outside the name entirely. - A particle joins the words after it into one name part, the - join running until the next particle starts a group of its own, - a trailing suffix begins — the trailing run S2 reads, once, - before the chain is made and counting the chain's own run as the - one name word it will be, so the chain cannot change what it - counts: a trailing roman numeral, or a - bare acronym with words to spare, ends the chain as a suffix word - does, and so does a bare ambiguous acronym written in capitals in - a mixed-case name, which needs no words to spare; the same - acronym written Title-case in a mixed-case name ends nothing and - joins the chain as any name word does, words to spare or not, - and the fork the chain called is reported there exactly as it is - where no particle stands (S2) — - a maiden - marker takes the - words after it (M2), or the name ends. The final group reads as - the family name; - earlier groups read by position. The chain begins wherever the - name begins, and a preceding title does not move that point. - Where P1's fold has claimed the opening, the fold decides the - family instead — and may take only PART of the final group, - since it counts name words and the group is one part. - In the part before a family comma, which the comma named the - family, a particle that is also suffix vocabulary heads the name - word behind it, whatever case the name is written in, and is not - read as a suffix standing between two family words; with a suffix - word or nothing behind it, it reads as the suffix it is there. A - lone letter behind it reads as it does in the same part without - the particle: a capital is a name word ('SMITH VD V, JOHN' as - 'SMITH V, JOHN'), a lowercase letter the numeral ('smith vd v, - john' as 'smith v, john'). - "John van der Berg" → family="van der Berg" - "John van der Berg Smith" → family="van der Berg Smith" - "Vincent van Gogh van Beethoven" → middle="van Gogh" - "Dr. John van der Berg" → family="van der Berg" - "Juan de" → family="de" · boundary - "de la Cruz Juan Carlos" family-first → family="de la Cruz" - "John van der Berg PhD" → family="van der Berg" - "John van der Berg V" → family="van der Berg" - "John van der Berg V" → suffix="V" - "John van der Berg Ma" → family="van der Berg Ma" - "John van der Berg Ma" → ambiguities=("suffix-or-name",) - "john van der berg ma" → suffix="ma" - "John van der J. V" → family="van der J. V" · boundary - "Freiherr von Berg MA" → family="von Berg" - "Freiherr von Berg MA" → suffix="MA" - "Freiherr von Richthofen V" → suffix="V" · boundary - "SMITH VD MA, JOHN" → family="SMITH VD MA" - "SMITH VD, JOHN" → suffix="VD" · boundary - "John van der Berg née Jones" → family="van der Berg" - Accepted: a particle of the unambiguous suffix vocabulary too - (vd, mc) is a suffix piece to the peel, so where it opens the - trailing run the chain stops before it as before any suffix - word, and the peel takes it; where it continues a prefix run, - the run takes it as a particle, as P6 reads it after a comma — - and there the chain reports nothing, the particle reading being - P6's fork rather than S2's. - The same carve-out covers `do`, the one word that is both a - particle and an AMBIGUOUS credential acronym: where a particle - run takes it, the word is the run's and the chain reports - nothing, whatever case it is written in — so `Anh van Do` is - silent where `Anh Do`, with no particle standing, reports. - A word that is both belongs to the particle run and to P6's - fork, not to S2's. - "John Smith Mc V" → suffix="Mc V" - "John van Mc" → family="van Mc" - "anh van do" → family="van do" · boundary - "anh van do" → ambiguities=() · boundary - "Anh van Do" → family="van Do" · boundary - "Anh van Do" → ambiguities=() · boundary - Accepted: a caller wanting the combined double-surname reading - (#132's ask) has it as the surnames view rather than the - family field. - "Vincent van Gogh van Beethoven" → surnames="van Gogh van Beethoven" - history: decisions.md#P2 · interacts: P1, P4, H5, M2, S2, C2 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py - -P3. Rationale: connective words ("y", "of the") bind name words into - one name part; but a single letter in a short name is more - likely an initial than a connective, and where a name is written - in more than one case, a bare Latin capital standing alone is how - an initial is marked and a bare lowercase letter is how it is - not. - A recognized connective joins its neighbors into one name part, - connective runs included — except a single-letter connective in - a three-word name, which stays a name word, and a single-letter - connective that reads as an initial instead, which never joins. - A single-letter connective reads as an initial where the writing - says so: written as a bare Latin capital in a name that is not written - wholly in one case, or — in a name written wholly in one case, - where nothing says so — where the letter is one the vocabulary - marks as reading both ways. - A letter the vocabulary marks as reading both ways, read as an - initial in a name written wholly in one case, is a call that could - have gone the other way, and is reported. - The joined part is ONE name word wherever another rule counts - them, so a rule taking "one name word" takes the whole join and - never half of it — except C1's count of the words before a comma, - which counts particle surnames alone and states why. - A connective counts as a name word wherever this rule counts them, - whatever else the vocabulary says the word is, where it is placed - to join. A word can be a connective and a generation at once — the - Catalan link is also the roman numeral one — and counting it as - the generation would let a connective raise the bar for its own - join, so the shortest linked name, which is the commonest one, - would be the only one that failed to link; while counting it as a - connective where it joins nothing lowers the bar for a different - connective's join instead. - A connective that is also generational vocabulary joins only where - a name word stands on each side of it — a word the rest of the - parse reads as a name word rather than as a generation, a - credential or an honorific, looked for past any run of - connectives standing between. A connective with nothing to its right is connecting - nothing, and a word of that vocabulary ending a name, or standing - before the credential a name ends with, is the generation it also - spells. - A separator the caller declared, standing in a trailing suffix - part, ends the search there as a comma would, so a connective - never joins across it (C1); anywhere else it is a word. - Both questions this rule asks of a name — how many words it has, - and whether it is written in one case — are asked of the name's - OWN words: a maiden marker taken as one, and the words it takes - (M2), are not among them, and neither is a delimited clause (N1, - M1). So a clause beside the name changes neither whether the - connective joins nor how a letter in the name reads, and a letter - inside such a clause is the clause's word, read as it always was - and not by this rule. The two questions part at one point: a - marker the pass declines and leaves as a word (M2) is a word, and - counts toward the three — but its case is still not asked. - That span is not this rule's alone: wherever another rule asks - the case question — the suffix slot (S2) and the post-comma slot - (C1) — it is asked of these same words, so the answer is taken - once and read where each of them stands. - "Juan y Eva Garcia" → given="Juan y Eva" - "Jose E Maria Santos" → middle="E Maria" - "jose e maria santos" → middle="e maria" - "Jose e Maria Santos" → given="Jose e Maria" - "JUAN GARCIA Y LOPEZ" → family="GARCIA Y LOPEZ" - "juan garcia y lopez" → family="garcia y lopez" - "john e smith" → middle="e" - "Juan y Garcia" → middle="y" · boundary - "Juan y Garcia née Jones" → middle="y" - "Juan and Garcia" → given="Juan and Garcia" - "Juan & Garcia" → given="Juan & Garcia" - "Mr. Jack and Jill" → family="Jack and Jill" - "Mr. Jack Jill" → given="Jack" - "Josep Carod i Rovira" → family="Carod i Rovira" - "Carod i Rovira, Josep" → family="Carod i Rovira" - "Josep i Rovira" → middle="i" - "John Quincy Smith i" → suffix="i" - "Carod i" → suffix="i" - "Josep Lluis Carod i III" → suffix="i III" - "Carod y de Rovira i" → middle="y" - "John й Smith" → middle="й" - "É. y É. Smith" → given="É." - Both exceptions are about the written FORM, not the word: the - three-word carve-out counts letters, so a symbol connective joins - at any length, and it reaches every single-letter connective the - vocabulary holds — Catalan i and Cyrillic и/і/й and Arabic و as - well as y and e. The initial reading counts letters too, and asks - one more question of them: a letter with no case at all (و) can be - written against nothing, so it never reads as an initial. A Cyrillic - capital has case but is not the shape an initial is written in — - Cyrillic abbreviates with a dotted letter — so in a name of more - than one case it joins, the reading #267 blessed, and in a name - of one case it takes the vocabulary's answer like any other cased - letter. A letter written as a base letter plus a combining mark - (NFD) that composes to one character is one letter to every - count here, and a period-marked initial written that way is one - initial (#665), so the last two examples above, both typed - decomposed, read as their composed twins. - Which single letters a tradition actually wants joined differs by - language, and the marked set is where that answer lives: "y" is - the commonest Hispanic compound and stays out of it, while "e" and - Catalan "i" are both common bare initials and are the two entries - shipped. A caller with Portuguese data removes "e"; a caller with - Catalan or Polish data removes "i"; a caller with Dutch data adds - the letter "y" to it. - The initial reading is visible beyond the fields, on the two - derived views: parse("john e smith").initials() gives "j. e. s." - and .capitalized() gives "John E Smith", where the connective - reading gave "j. s." and "John e Smith" — a connective - contributing no initial (R3) and keeping its lowercase (R4), - where an initial does neither. The v1 facade's initials() reads - the same parse, so HumanName("john e smith").initials() is - also "j. e. s."; #528 closed the split and decisions.md#P3 - records it. - Accepted: two 1.4.0 parity breaks, one in each direction. A bare - capital in a name written wholly in upper case no longer reads as - an initial, so "JUAN GARCIA Y LOPEZ" joins where 1.4.0 and - 2.0–2.3 read middle "GARCIA Y"; and a marked lowercase letter in - a name written wholly in lower case no longer joins, so "jose e - maria santos" reads middle "e maria" where they read given "jose - e maria". That is #383 answered with "bless" for the mixed-case - half — a bare Latin capital among mixed case still reads as an - initial, the shape the veto always tested, which is why #267's - Cyrillic reading is untouched — and answered with evidence for - the one-case half, where the vocabulary decides. - A maiden marker's run and every word after it are outside the - name's own words from the moment classify tags the marker, and - they stay outside whether or not the marker pass later declines - the marker and leaves it a word (M2). A declined marker therefore - does two things at once. It still does not count toward the case - class, which shows when it is the only differently-cased token, - so "JUAN Y GARCIA née" joins where "JUAN Y GARCIA née Jones" - keeps "Y" a name word. And it leaves any connective standing - after it to the mixed-case rule rather than to the one-case fork, - so a bare Latin capital there reads as an initial while its - lowercase spelling joins — the one shape in which the name's OWN - words read differently in its two one-case spellings. So "DR. - NÉE JUAN Y LOPEZ" reads middle "JUAN Y" with its "Y" an initial, - while "dr. née juan y lopez" reads family "juan y lopez" with its - own "y" joined. A clause's words are read as they always were, - and that is not an exception to this: such a letter still takes - a different tag in the two spellings, so its case repair can - differ between them, which is a repair difference and not a - reading of the name's own words. - Accepted rather than repaired: classify cannot know what group - will decline, and a clause's words are read by the clause's own - rules. - "Хосе И Мария Сантос" → given="Хосе И Мария" - H1 is the counting rule that shows the one-word clause today: a - title plus the join reads the whole join as the family, where the - same two words unjoined are two name words and H1 does not fire. - P1's leading run is the second (#395, landed): its run takes - the "Vega y Santos" join whole or stops before it. - history: decisions.md#P3 · interacts: H1, P1, M2, R1, R3, R4, S2, C1 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_comma.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py - -P4. Rationale: a particle links forward from inside a name; at the - very front there is no name yet to be inside. - A particle in the name's leading position chains nothing: the - words stay separate, and any surname reading the name gets - comes from the fold (P1) or from position (O4), never from a - join. This is why a title before a leading particle changes - nothing (the title is not a name word), and why "Van Johnson" - is a given-name reading at all. An unlisted abbreviation before - the particle is as transparent as a listed title, since assign - reads it as one (H2). A word in the leading titles that is both a - title and a particle puts the name's leading position on itself, - so the particle behind it is inside a name ('Freiherr von Berg'); - of several such words the last does, so no particle chains inside - the titles (#624). The titles are those written, before H3 gives - the last one back to the name, so each such word reads as a title - unless H3 gives it back. - "Van Johnson" → given="Van" - "Sir de Mesnil" → pieces=[["Sir"], ["de"], ["Mesnil"]] - "Xyz. van Johnson" → given="van" - "John van der Berg" → pieces=[["John"], ["van", "der", "Berg"]] · boundary - "Freiherr St John Smith" → title="Freiherr St" - "Freiherr St John Smith" → given="John" - history: decisions.md#P2 · interacts: P1, P5, H2, H3 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py - -P5. Rationale: some given-name words are incomplete alone — "abdul" - is a bound form that the next word completes. - A recognized bound given-name word joins the word after it into - one given name. It needs a name word to spare, so two name words - alone do not join — the second is the family name — except after - a family comma, where the family is already fixed, or after a - given-name title, which asserts that a given name follows (H1): - there the two words join and the name has no family — two name - WORDS, so neither a particle chain (P2), which is the family - name, nor a suffix word is joined. A run of several titles is - read as H1 reads it, by the title that does the addressing, - which is its last. Where the join fires, a bound word that is - also a particle is read as the bound word: the join outranks the - leading-position reading (P4) and no fork is reported; where the - reserve blocks the join, P4's reading and its fork stand. Where - the word - is BOTH bound-given and suffix vocabulary, position decides and - both readings survive: leading, it is the bound word; trailing, - it is the suffix (S2). In the given slot after a family comma the - suffix reading wins. The marker and the words it will take are - not among the words to spare: they leave the name, so counting - them asks the question about a name that will not exist. The join - never absorbs a marker standing as a word of its own — a marker - is not a name word (M2) — nor a word of the unambiguous suffix - vocabulary (S2), wherever position will then place it, nor a - title word a trailing rule reads as a title: one the trailing - title run (H5) takes from the given part after a family comma, - read as the comma-less name reads it, or any title word in a part - past the second comma (C2); a bare - ambiguous acronym is a name word wherever the peel does not take - it; a marker left as a word that a particle join (P2) has already - taken travels with that join. A title word standing in the name - is a name word (H3) and joins like one, and so is a particle the - chain has not taken (P2) — unless it is of the unambiguous - suffix vocabulary too (vd, mc), which the join declines as the - suffix piece it is: - the pair is a given name whatever tag the word carried, and - after a family comma the join runs before the trailing - particle's attachment (P6) sees the name. What - there is to spare is what the name's trailing reading leaves: - the trailing suffix run (S2) and the trailing title run (H5), - each read over what the other leaves until neither takes - anything more, are read once, before the join is tried, and the - name words they leave are the words to spare. A trailing - roman numeral, or a bare acronym the run takes, or a trailing - title word the run takes, is no - word to spare. The join joins two name words into one and, read - after the trailing run, changes no suffix reading: a word the - run left as a name word stays one, whatever the join makes of - the word in front of it. After a - family comma the family is fixed and the joined pair is the given - whatever follows, so the reserve there reads no peel: the join - stands whenever the word after the bound word is a name word. - "abdul salam ahmed salem" → given="abdul salam" - "abd Allah Smith" → given="abd Allah" - "Salam, abd Allah" → given="abd Allah" - "abd Allah" → given="abd" - "Smith, Abd" → suffix="Abd" - "mohamad ali smith" → given="mohamad" · boundary - "Sheik abdul salam" → given="abdul salam" - "Sheik abdul salam Jr" → given="abdul salam" - "Sheik Abu Bakar" → given="Abu Bakar" - "Abu Bakar Salim" → given="Abu Bakar" - "Dr. abdul salam" → family="salam" · boundary - "Sir abdul van der Berg" → family="van der Berg" · boundary - "Sheik abdul Jr Smith" → given="abdul" · boundary - "Abu Bakar" → given="Abu" · boundary - "abdul Smith V" → family="Smith" - "abdul Smith V" → suffix="V" - "abdul Smith Jr V" → family="Smith" - "abdul Smith Jr Ma" → given="abdul" - "abdul Smith Jr Ma" → suffix="Jr Ma" - "abdul Smith Ma" → given="abdul Smith" - "abdul Smith Berg Ma" → middle="Berg" · boundary - "abdul Sir Smith Berg" → given="abdul Sir" - "Berg, abdul van" → given="abdul van" - "Berg, abdul vd" → family="vd Berg" - "Berg, abdul Prof." → given="abdul" - "Berg, abdul Prof. Ali" → given="abdul Prof." · boundary - "John Smith, Jr., abdul Dr. Smith" → title="Dr." - "abdul Jr Smith Berg" → given="abdul" - "abdul Jr Smith Berg" → middle="Jr Smith" - "abdul Ph. D. Smith Berg" → suffix="Ph. D." - "abdul V Smith" → given="abdul V" · boundary - "abd Berg née Jones" → family="Berg" - "abd Allah Smith née Jones" → given="abd Allah" - "abd née Jones" → given="abd" - "Berg, abd née Jones" → given="abd" · boundary - Accepted: after a family comma the join stands though, unjoined, - assign would read the word it takes as the suffix — the family - is fixed there and the pair is the given; and a bare ambiguous - acronym the peel does not take joins as any name word does. - "Berg, abdul V" → given="abdul V" - "abdul Ma Smith" → given="abdul Ma" - Accepted: a given-name title plus a bound pair leaves the family - empty, as H1 accepts for "Sir John" — the input names no family. - Under a family-first order the joined pair is the family and the - given name is empty instead, as "Sir John" reads there: the join - is order-blind, and H1's exemption never runs under that order. - "Sheik abdul salam" → family="" - "Sheik abdul salam" family-first → family="abdul salam" - "Sheik abdul salam" family-first → given="" - "Sheik abdul salam" family-first-given-last → family="abdul salam" - Accepted: the given part after a family comma reads the words - behind a title by its own walk, and where that walk reads them - otherwise than the comma-less name does, the join is declined and - no title is read — a shape-only numeral (VI to X), or V before a - third comma part, stays a name word there, and a title word that - is also a particle is attached by P6 (`Berg, abdul Prof. VI` - reads middle 'Prof. VI'; `Berg, abdul St.` reads family 'St. - Berg'), each as the same name with an ordinary given word reads. - history: decisions.md#P5 · interacts: C2, S2, M2, H1, H5, P2, P4, P6 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_post_rules.py - -P6. Rationale: a particle ending the name has nothing to link - forward to, so it is not doing a particle's work there. What it - is doing instead is decided by what the WRITING says, not by the - word: where something has already named the family AND left the - particle in a position that means nothing — after a family comma, - or in the middle of a FAMILY_FIRST reading — it belongs to that - family, which is how Dutch and Flemish names are listed - ("Beethoven, Ludwig van", the tussenvoegsel trailing the given - name but belonging to the surname). Where nothing has, it is - read where it stands, and a particle CAN be a given name there: - the never-given vocabulary supplies the reading position leaves - open, and forbids nothing (parse("de") reports given "de", and - so does every other word in that set). - Where a family comma has already named the family, a particle - ending the name attaches to that family name and is written - before it — provided at least one given word remains, so that a - name whose only given word is the particle keeps it (the - words-to-spare test S2 applies to ambiguous suffixes). A - post-nominal is written BEHIND the particle in this listing, so - it does not end the name for this purpose: the run is found by - looking past trailing words that hold no name — unless such a - word is itself particle vocabulary, which makes it part of the - run rather than something to look past. Where - the word is BOTH a particle and suffix vocabulary, this - attachment outranks the suffix reading (S2): a trailing - abbreviation after a family comma is the tussenvoegsel far more - often than the decoration it collides with. Three exceptions. The - first is where the capitals speak: a word of the AMBIGUOUS - credential class, written in capitals in a name written in more - than one case, reads as the credential and this attachment stands - down — unless a particle stands immediately in front of it, the - two being one particle run by then, which this rule takes whole. - The second is S2's company: such a word standing behind an - unambiguous credential in one run of suffix words reads as the - credential in any spelling and any case, reported as S2's - credential fork. The third is a lone word of both the particle and - the UNAMBIGUOUS suffix vocabulary standing INSIDE a run read as - post-nominals — one in front of it and another behind, a - credential or a generation, whatever made each one, and a title - between it and the one in front transparent (H5): the - tussenvoegsel stands right behind the given name, so a - post-nominal between them says the word is not one. It does not - end the name, and keeps the - post-nominal reading in any case, reported as S2's credential - fork. With nothing behind it, it ends the name and attaches, and - two such words side by side are one particle run, which this rule - takes whole. Every other spelling of such a word attaches as - it did before, and the kind rule below gives it this rule's - particle fork rather than S2's credential one. In a name written - wholly in one case, with nothing in front of the word to speak - for it, the two readings cannot be told apart and the particle - keeps it, which is right about a Portuguese record and wrong - about a credential; the report is how a caller finds the second. - "Jong, Anke de" → family="de Jong" - "Beethoven, Ludwig van" → family="van Beethoven" - "Berg, Jan vd" → family="vd Berg" - "Berg, Jan van der" → family="van der Berg" - "Vega, Juan de la" → family_particles="de la" - "Beethoven, Ludwig van" → family_base="Beethoven" - "Beethoven, Ludwig van" → family_particles="van" - "Nguyen, Van" → given="Van" · boundary - "Doe, John DO" → suffix="DO" · boundary - "Doe, John Do" → family="Do Doe" - "SMITH, JOHN DO" → family="DO SMITH" - "Doe, John van DO" → family="van DO Doe" - "DOE, JANE PHD VD MA" → suffix="PHD VD MA" - "doe, jane phd vd ma" → suffix="phd vd ma" - "DOE, JANE PHD VD" → family="VD DOE" · boundary - "DOE, JANE MA VD PHD" → suffix="MA VD PHD" - A particle an index form writes apart with its elision or its - hyphen still on it is a particle like any other here, where the - vocabulary lists that spelling — the elided "d'" and the Arabic - article in its plain and assimilated spellings, "al-" among them - — and of - the never-given kind, since written apart the word is the article - and nothing else. It joins with a space like every particle; - written glued to the family ("d'Ivoi") it is one word and this - rule never sees it. - "Ivoi, Paul d'" → family="d' Ivoi" - "Rashid, Harun al-" → family="al- Rashid" - "Zaʿīm, Ḥusnī az-" → family="az- Zaʿīm" - Without a comma, a declared family-first order has named the - family in the same way and the attachment fires there too — but - only where the run ENDS the name and stands in a MIDDLE — the one - position that means nothing for it, middles being further given - names, which a particle is not. Those two together name the - order without asking it: the default order ends with the FAMILY - and FAMILY_FIRST_GIVEN_LAST with the GIVEN name, so only - FAMILY_FIRST can put a name's last word in a middle at all. - Ending the name is not implied by the slot and has to be said. A - draft that tested the slot alone fired on 52 default-order names, - where a conjunction stops a particle's forward chain and leaves it - standing in a middle, and on 366 FAMILY_FIRST_GIVEN_LAST middles - with the given name still behind them. - Nothing is asked about the WORD, and that is what lets one rule - read both traditions: the same vocabulary that spells the Dutch - tussenvoegsel spells the Vietnamese given name, and the declared - order separates them where a comma cannot — one order, not both, - as the Accepted clause below records. - Nothing is re-laid-out either — under the order this can reach, - the roles run family, given, middle, middle…, so dropping a - trailing middle leaves every other piece where it was. That is a - property of FAMILY_FIRST alone, and it is the second thing the - two tests above secure. The family must hold a - base of its own, since a family that is all particles is not a - family written beside anything (R2). - "Jong Anke de" family-first → family="de Jong" - "Jong Anke de" family-first → given="Anke" - "Ménil Christophe de" family-first → family="de Ménil" - "Ménil Christophe de" family-first → given="Christophe" - "Beethoven Ludwig van" family-first → family="van Beethoven" - "Ménil Christophe de" family-first-given-last → given="de" - "Berg Jan de Jr." family-first → family="de Berg" - "Berg Jan de Jr." family-first → suffix="Jr." - "van Berg Jan de" family-first → family="van" · boundary - "Maria Luisa y de la Cruz" → family="la Cruz" · boundary - "de Anke van y" family-first-given-last → middle="van" · boundary - "Ménil de" family-first → given="de" · boundary - "Beethoven Ludwig van" family-first → ambiguities=("particle-or-given",) - "Jong Anke de" family-first → ambiguities=() · boundary - Accepted: an ambiguous particle attaches on the same terms as a - never-given one, so a Vietnamese name written in this listing - loses its given name — but only in the UNACCENTED - transliteration. Vân carries a diacritic and is not particle - vocabulary, so the correctly spelled name never reaches this - rule. It is the ASCII spelling that collides, and there the two - traditions write the same string. - "Nguyen, Thi Van" → family="Van Nguyen" - "Nguyễn, Thị Vân" → family="Nguyễn" - The fork is reported, in the kind naming the reading this - attachment OVERRODE — what the run was read as before it fired, - not what its words are. Where the run was read as a post-nominal, - which is the S2 reading this rule outranks, the report is - suffix-or-name. Otherwise, where the run holds an ambiguous - particle, the reading overridden is that word as a name word, and - the report is particle-or-given. Where the attachment overrode - neither — a never-given particle read as a name word, which it - could never have been — nothing was decided and nothing is - reported. - "Nguyen, Thi Van" → ambiguities=("particle-or-given",) - "Berg, Jan vd" → ambiguities=("suffix-or-name",) - "Jong, Piet de" → ambiguities=() · boundary - Accepted: the colliding spelling has a format that reads - correctly, and it is ONE order, not both — FAMILY_FIRST_GIVEN_LAST, - which is the order this name is written in. A line asserting what - it does under FAMILY_FIRST stood here until #467 and was removed - rather than updated: the name is not written in that format, so - its reading there is wrong by construction and pins nothing worth - keeping (#470). - "Nguyen Thi Van" family-first-given-last → family="Nguyen" - "Nguyen Thi Van" family-first-given-last → given="Van" - Accepted: under the DEFAULT order a comma-less name's written - shape is not settled — "Jong Anke de" may be a misformatted - listing, and a bare "Jong de" may be a given name beside a - particle — so nothing there names the family, the positional - reading stands, and the same input reports family "de" here. - That is the reading for a particle the UNAMBIGUOUS suffix - vocabulary does not also claim, which is all but two of them. - Where the word is both a particle and UNAMBIGUOUS suffix - vocabulary — vd and mc — nothing has named the family, so the - attachment above never fires and S2 takes the trailing word as - a post-nominal, even though that leaves no family name at all: - a bare "Donald mc" reports suffix "mc" with an empty family, - and so does "Smith vd". The comma is the contrast, and it is - where the precedence stated above acts, "Berg, Jan vd" reading - family "vd Berg". `do` follows `de` rather than `mc`, sitting - in the AMBIGUOUS acronym half, where S2's words-to-spare test - leaves a two-word name its family. - "Jong Anke de" → family="de" - Accepted: the precedence over S2 is stated for the shape, so it - sweeps in every word that is both particle and suffix - vocabulary — today vd, do and mc. Only vd's reading was - weighed; mc inherits it, which is the shape's cost and was - decided rather than tracked — #454 closed by design 2026-09-07 - (decisions.md#suffix-acronym-collisions). `do` sits in the - AMBIGUOUS acronym half and was already read as a name word - there, so the precedence decides nothing for it — and because - the report names the reading OVERRIDDEN, `do` reports - particle-or-given while vd and mc report suffix-or-name. The - same distinction decides a run of several words: a run read as - name words reports on its ambiguous member if it has one and - otherwise reports nothing, whatever its words could have been - read as standing alone. Both are pinned in tests/v2/cases.py. - Accepted: a bound given word ahead of the trailing particle takes - it as its pair first (P5), so the attachment never sees it — - unless the particle is of the unambiguous suffix vocabulary too - (vd, mc), which the join declines and the attachment then takes. - "Berg, abdul van" → given="abdul van" - "Berg, abdul vd" → family="vd Berg" - The comma site above and the no-comma family-first site earlier - in this rule are checked against each other directly, as a pair, - rather than only against their own examples: tests/v2/test_order_correspondence.py - generates both writings and asserts they parse alike, plus a - negative-control sweep pinning the disagreeing set the precedence - bullet above names. A change that breaks one side of that pair - should expect that test, not this file, to say so first. - history: decisions.md#P6 · interacts: A1, C1, H5, P1, S2, P5, M2 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py - -P7. Rationale: a one-letter particle is spelled with the same letter - as an initial, and the period is what tells them apart. An - initial is an abbreviation and is written with one; a particle is - a whole word and is not. The Irish Ó is never written with a - period, while Ó. is the initial of Óscar, Ólafur or Ólöf. - A one-letter word written with its period is an initial and not a - particle, whatever particle the same letter spells; written bare, - it is the particle. It is an initial in every position, so it - neither opens a surname, nor joins a chain, nor attaches to the - family behind a comma. - "Juan Ó. Pérez" → middle="Ó." - "J. Ó. Pérez" → middle="Ó." - "Ó. Pérez" → given="Ó." - "Pérez, Juan Ó." → middle="Ó." - "Juan Ó Pérez" → family="Ó Pérez" · boundary - Accepted: an accented initial written WITHOUT its period reads as - the particle. Such initials normally carry the period, and the - bare letter is exactly how the particle is written. - "Juan Ó Pérez" → middle="" - history: decisions.md#P7 · interacts: P1, P2, P3, P6, R3, R4 · implemented: nameparser/_lexicon.py, nameparser/_pipeline/_classify.py - -## Suffixes: generational & credentials (S) - -Background: what follows a name is one of two different things — generational suffixes (Jr., III), which attach to the name itself, and credentials (PhD, MD, MBA), which are earned attachments. The suffix sets match one written word at a time: a multi-word entry there can never match anything and is warned about at configuration. Only two sets are exempt from that rule -- given_name_titles and, since #434, maiden_markers -- and neither is a suffix set, so within the suffix vocabulary the one-word rule is absolute. That limit is on STORAGE, not on the shape a name may have: adjacent suffix tokens are reassembled after matching (`_vocab.is_wholly_suffix`), so a multi-word credential is reachable as its component words -- `John Smith, MD PhD` has read suffix `MD PhD` since 1.4.0 -- and a caller reaches an unshipped one by adding the words it is made of rather than the phrase (#433). The eight multi-word entries that shipped dead for years span the suffix sets and the titles alike and are the Excluded story in decisions.md. CLDR personNames keeps them as separate fields (`generation`, `credentials`) and formats them differently; this library currently reports both in one `suffix` field, a merge #326 examines. The vocabulary is largely split already: a generational word list and a credential acronym list, plus a short list of acronyms that are also ordinary names (MA, BA) and so are AMBIGUOUS as bare words. - -S1. Rationale: brackets set off more than nicknames — credentials - are routinely written parenthesized after a name, and a - credential is recognizable by its form. - A bracketed clause whose content is suffix-shaped is not a - nickname: the brackets are dropped and the content reads exactly - as if written bare. - "Andrew Perkins (MBA)" → suffix="MBA" - "Andrew Perkins (Andy)" → nickname="Andy" · boundary - interacts: M2 · implemented: nameparser/_pipeline/_extract.py - -S2. Rationale: generational suffixes and credentials are recognized - by vocabulary; an acronym that is also an ordinary name is only - unmistakably a credential when its periods are written. - A suffix never OPENS THE STRING: position outranks the - vocabulary match, so a suffix-shaped word written before anything - else reads as whatever its position and shape make it — a title - where it wears the abbreviation shape (H2), an ordinary name word - otherwise — and never as a credential belonging to a name that - has not been written yet. The string, not the name: a title, a - nickname or a bracketed clause standing first leaves the - credential reading intact, which is the ONE place this document - says "opening" and does not mean what P4 and the H Background - mean by it. A family comma is the other boundary: the comma has - already named the family, so a credential run may open the part - after it (C1), and the part before it is that family, less a - title run opening it (H6). - A trailing word of the suffix vocabulary reads as a suffix — - generational forms and credential acronyms alike, and an - ambiguous acronym written with its periods, one after each - letter or one after each of two or more letter chunks, counts - unambiguously; a single trailing period is the abbreviation - shape any word can wear and does not. The run at a name's - trailing slot is read ONCE, once the connectives have joined (P3) - and before the particle chain (P2) and the bound given-name join - (P5) are made, counting a particle run and the name words the - chain will take with it as the one name word it will make of them - — the words both particles and suffix vocabulary standing straight - behind a particle among them, while a word the reading weighs - counts as a word of its own wherever it stands; inside a - credential run a word stands on its own, a title word there - still a title — and neither - join reaches a word the run took. A - BARE ambiguous acronym is consumed only when the name has words - to spare — as the second of two words it stays the family - name — and at the slots that report, either reading carries the - ambiguity flag. Those slots are the trailing slot of a name, the - first slot after a family comma, the trailing slot of the GIVEN - part after that comma, and the trailing slot of a maiden - marker's clause (M2). A part beyond the second comma is not one - of them; what such a part does report is C2's to say. A word - this document - says is READ at one of those slots is not always a word one of - them reports: where a reading moves a word out of the slot that - asked about it, what reports is the slot it lands in, and that - may be none — M2 states the case where a clause does it. - After a family comma, a word of this class ending the GIVEN part - is read as the comma-less spelling reads a word ending the name, - and the count is not what decides it there. The comma has already - named the family and the first name word after it is the given - name, so the words to spare are there by construction and the - count says nothing — unless that first word is a never-given - particle, which is no given name and needs the next name word to - attach to (P1): that word is the surname the particle heads, not - a word to spare, so it reads as a name unless its capitals in a - mixed-case name say credential, as they do with no words to spare - anywhere: `SMITH, VD MA` reads as `Smith, vd Ma` does, `Smith, de - MA` keeps suffix `MA`, and `SMITH, VAN MA`, whose particle can be - a given name, keeps it and reads suffix `MA`. Otherwise a word - that ENDS that part reads as the - credential unless its WRITING says otherwise, and a word that - does not end it is never asked. "Ending the given part" reaches - past the credentials behind it and past a trailing title, which - is transparent to this reading as it is to the rest of S2's (H5); - a name word behind the word ends the reach, and so does a - PARTICLE the suffix vocabulary does not also claim: it belongs to - the family the comma already named and is taken there by a rule - that runs after this reading is made (P6), so `Doe, John MA do` - keeps its middle name though the capitals would otherwise have - taken the word. A particle the suffix vocabulary DOES claim is - looked past like any other credential, so `Doe, John MA vd` - reads suffix `MA`, with `vd` attaching behind it. Where the - reach ends, the word is an ordinary middle name, read in silence. - A particle in FRONT of the word takes it out of this slot as - well, and takes the word behind it too: where what follows the - particle reads as a name rather than a credential the two are - one name, so neither word is asked and neither reports — - `Doe, John van Ma` reads middle `van Ma` and `Doe, John DO Ed` - middle `DO Ed`, where `Doe, John van MA` reads suffix `MA` and - `Doe, John DO` alone reads the credential, both reporting. - Every word this slot does read reports the fork whichever way it - went, so a run of members all read as credentials reports once - for each, as the same words do without the comma — while a - member the writing keeps as a name - stops the reading there, and whatever stands in front of it is - name text, asked nothing and reporting nothing. One member of - this class is particle vocabulary as well, and where it stands - alone at this slot P6 decides it: the capitals take it as the - credential, as does an unambiguous credential in front of it (the - company below), and every other spelling attaches to the family, - reported there as P6's fork rather than as this one. Behind - another particle it does not stand alone — the two are one - particle run by then — and the run attaches whatever the capitals - say (P6). - Written case is the other evidence, and it speaks only in a name - written in more than one case: there a member of the ambiguous - set written in capitals reads as the credential even with no - words to spare, and one written in any other cased form that is - not wholly lower reads as the name even with words to spare. A - name written wholly in one case says nothing about any word in - it, and the count decides alone; so does a script with no case - to write in. After a family comma this evidence is SECOND at the - FIRST slot after it: the count of name words before the comma - decides there first (C1), and the case is read only where that - count leaves the word a name. C1 states the one exception: a part - of two or more suffix words and words of this class, every word of - the class listed and written in capitals in a mixed-case name, is - read on its capitals - before the count is asked, as the credential run behind one name - word as behind two ('Smith, MA MA' reads suffix 'MA MA'), - and only a word of the class opening the part reports, as the - first word after the comma ('John Smith, MA MA' reports on the - first 'MA' alone, where a counted flip reports over the whole - part). At the trailing slot of the given - part the comma has already settled the count, so the writing is - the only evidence there is. - Company is evidence that outranks both. A member of the ambiguous - set standing BEHIND an unambiguous credential in one run of - suffix words — members of the class between them passing the run - on — reads as the credential whatever its writing and whatever - the count, at every trailing slot this rule names: the degree in - front says what the run is. Only a credential in FRONT speaks; one - behind the member says nothing about it. A connective (P3), a - particle (P2) — a word of both the particle and the suffix - vocabulary heads the family name behind it — and a single-letter - roman numeral, in any case — one letter, the shape a bare middle - initial is written in, where a multi-letter one such as 'III' - speaks — speak for nothing and end the run; so does any name - word. A particle of this class standing behind a credential is - still spoken for: the exclusion is of the word in front. A word - of both the title and the suffix vocabulary standing in the given - part's leading title run is a title there and speaks for - nothing, and no credential behind it speaks for a word of this - class in that part: the title reading makes the next word the - given name, and from there the part reads as any given part does, - the given part's own company included. A part holding no such - word, or only words whose capitals decide them, reads as it did. - Anywhere else such a word speaks like any suffix word. - At the trailing slot of the given part this company outranks P6's - attachment, as the capitals do. It does not reach across a maiden - marker's clause (M2), whose name words stand between. A member - the company decides reports the fork as a counted pick does, - wherever it stands: at the trailing slots above, and in a part - after a family comma that the company leaves holding no name - word, which reads wholly as the credential run. A member its own - capitals already made the credential reports nothing new. - A credential after the name core starts a run to the end of its - part (#602). An unambiguous suffix word of two or more letters, - in any script, standing after two name words where no comma divides - the name or before a suffix comma, or after the given word in - the part after a family comma, makes every later word of that - part a suffix, except a title word, which reads as a title, and - in that given part the particles ending it, behind any - post-nominals, which P6 attaches to the family — other than a - lone member of the ambiguous class, which reads as the - credential. A lone particle is - not a name word for the count, 'de Mesnil' being one surname, - unless the credential stands right behind it, leaving it nothing - to join: then it counts as a name word of its own ('John von PhD - Jones'). A - particle run with the words the chain joins to it is one name - word, 'der la' and 'la Smith Secretary' each counting once (P2). - A title word - never starts the run, and neither does a member of the ambiguous - class, a single letter, a connective, a word that is also - particle or bound given-name vocabulary: a credential is the - writer's mark that the post-nominals have begun, and those carry - no such mark. An honorific or a generational word carries it as - a credential does, whatever its script. A name word the run - absorbs is reported `suffix-or-name`. - An unlisted word joins this same ambiguous class by SHAPE where - the caller asks for it. Two or more period-separated chunks is - one such shape, admitted by default (S3); an unlisted all-caps - alphabetic word of two or more letters, standing in a suffix - position of a mixed-case name and belonging to no wordlist, is - the other. By default that second shape is admitted in one - position only, the part after a comma with two or more name - words before it (C1): French and Korean records write the - SURNAME in capitals, and that convention puts the capitals at - the end of a name or before a comma, never after a comma behind - a full name. Everywhere else it is admitted only under the - caller setting the example lines below name, being a surname - there as often as a credential, which only the caller can know; - the cost of that setting is that a three-word name gives up its - family name to the acronym, while a two-word name keeps it — - there are no words to spare there, so the class is considered - and declined and only the fork is reported. The dotted shape does not take every promise of the - class with it: at the first slot after a comma, C1 reads paired - initials as the given name unless something speaks for them, and - makes a flip to the credential run in which no listed member takes - part in silence, unless paired initials speak only for each - other. - "John Smith Jr." → suffix="Jr." - "Antonio Pavlović mlađi" → family="Pavlović" - "Antonio Pavlović mlađi" → suffix="mlađi" - "Агриппина Младшая" → suffix="Младшая" - "جورج بوش الابن" → suffix="الابن" - "ロバート・ダウニー・ジュニア" → suffix="ジュニア" - "佐藤恵子 国会議員" → suffix="国会議員" - "陳子媚 二世" → suffix="二世" - "Питер Брейгель мл." → suffix="мл." - "Marie Fils" → family="Fils" · boundary - "Jiří Dienstbier ml." → family="ml." · boundary - "Pavlović, Antonio mlađi" → suffix="mlađi" - "Stariji Bruegel" → given="Stariji" · boundary - "John Smith M.A." → suffix="M.A." - "John Smith PhD" → suffix="PhD" - "John Ma" → family="Ma" · boundary - "Jack MA" → suffix="MA" - "Jack Ma" → family="Ma" - "JACK MA" → family="MA" - "JOHN SMITH MA" → suffix="MA" - "John Smith Ma" → family="Ma" - "wang meng" → family="meng" · boundary - "tran lac" → family="lac" - "john smith meng" → suffix="meng" - "john smith MEng" → family="MEng" - "Nguyen Van Lac" → family="Van Lac" - "Wang M.Eng." → suffix="M.Eng." - "John Smith PhD MEng" → suffix="PhD MEng" - "John Smith PhD Ed Ma" → suffix="PhD Ed Ma" - "Wang Ma PhD" → family="Ma" · boundary - "Smith, PhD MEng" → suffix="PhD MEng" - "Smith, Ms Ma" → given="Ma" · boundary - "Smith, PhD Ma" → suffix="PhD Ma" - "Smith, PhD Ma" → ambiguities=("suffix-or-name",) - "Smith, MD PhD Ma" → given="PhD" · boundary - "Doe, Jane PhD MEng" → suffix="PhD MEng" - "doe, jane v phd do" → suffix="v phd do" - "Smith, MA" → suffix="MA" - "Smith, Ma" → given="Ma" - "Doe, John MA" → suffix="MA" - "Doe, John Ma" → middle="Ma" · boundary - "Doe, John MA Smith" → middle="MA Smith" · boundary - "Doe, John DO" → suffix="DO" - "SMITH, JOHN DO" → family="DO SMITH" · boundary - "SMITH, VD MA" → family="SMITH VD MA" - "smith, de ma" → family="smith de ma" - "SMITH, VD DE LA MA" → family="SMITH VD DE LA MA" - "Doe, John MA JD" → ambiguities=("suffix-or-name", "suffix-or-name") - "Doe, John MA Ma" → middle="MA Ma" · boundary - "Doe, John MA Ma" → ambiguities=("suffix-or-name",) · boundary - "Doe, John van Ma" → middle="van Ma" · boundary - "Doe, John van Ma" → ambiguities=() · boundary - "Doe, John DO Ed" → middle="DO Ed" · boundary - "Doe, John DO Ed" → ambiguities=() · boundary - "John Smith XYZ" → family="XYZ" - "John Smith XYZ" unlisted_caps_suffixes-everywhere → suffix="XYZ" - "Jean DUPONT" unlisted_caps_suffixes-everywhere → family="DUPONT" - "Jean Pierre DUPONT" unlisted_caps_suffixes-everywhere → suffix="DUPONT" - "Jean Pierre DUPONT" unlisted_caps_suffixes-everywhere → family="Pierre" - "Jack Ma." → family="Ma." · boundary - "Ph. D. Van Johnson" → family="Van Johnson" - "Ph. D. Van Johnson" → title="Ph." - "Smith, Ph. D. Jr." → suffix="Ph. D. Jr." - "II Van Johnson" → given="II" · boundary - "Sir Ph. D. Van Johnson" → suffix="Ph. D." · boundary - "Ph. D., John" → family="Ph. D." · boundary - "John Smith PhD Jones" → suffix="PhD Jones" - "John Smith PhD Jones" → ambiguities=("suffix-or-name",) - "Eric H. Holder Jr. Attorney General" → title="Attorney General" - "John Smith PhD Prof. Ma" → suffix="PhD Ma" - "Smith, John PhD Jones" → suffix="PhD Jones" - "John von PhD Jones" → family="von" - "John von PhD Jones" → suffix="PhD Jones" - "Smith, John PhD de Jr." → family="de Smith" · boundary - "Mary Jane King Smith" → family="Smith" - "Josep Carod i Rovira" → family="Carod i Rovira" - "Mohamed Ali Abd Allah" → family="Allah" - "Antonio Pavlović mlađi Jones" → suffix="mlađi Jones" - "Pieter Bruegel stariji Jones" → suffix="stariji Jones" - "Ана Хорват старији Џонс" → suffix="старији Џонс" - "Гай Плиний Старший Иванов" → suffix="Старший Иванов" - "राम शर्मा साहब मोहन" → suffix="साहब मोहन" - "राम शर्मा साहब मोहन" → ambiguities=("suffix-or-name",) - "John PhD Smith" → family="Smith" · boundary - Accepted: a title before the credential keeps it a credential, - and the name loses its surname exactly as it did before this - clause — `Sir Ph. D. Van Johnson` reads given `Van Johnson` with - an empty family. That is 1.4.0's reading, and the clause cannot - reach it: the first piece of the NAME is not computable before - this decision, since the abbreviation test H2 uses is true of - `Ph.` itself, so a scan that stepped over titles would step over - the very piece being judged. - Accepted: an unambiguous suffix is consumed even when that - leaves no family name at all. - "Smith Jr." → family="" - Accepted: a name-leaning member with nothing in front of it to - speak for it ends the name, words to spare or not, and whatever - stands in front of it is name text. Where a credential does - stand in front, the company above decides instead, except in - one shape that keeps it out of reach and reads the member as the - name: a credential written split across two words with no comma - after the name. That reading is older than the company clause, - so it is witnessed by tests/v2/cases.py's - a_split_degree_in_front_is_out_of_the_walk rather than by a line - here, which would promise a reading this clause did not make. - A title standing between the credential and the member, and a - member a particle chain (P2) had taken, were two more such shapes - until the credential's run (#602) took both in: the rows - a_title_between_degree_and_member_keeps_it_a_name and - a_member_the_particle_chain_took_is_out_of_reach keep their ids - and now pin the run. - "Jack Wei Ma" → family="Ma" - "Jack Wei Ma" → ambiguities=("suffix-or-name",) - "abdul Smith Jr Ma" → suffix="Jr Ma" - Accepted: the title chain no longer takes the word this rule - needs, and the argument a descriptive note here asked for is - made. A title run leaves one NAME word standing and a - post-nominal cannot be it (H3, decided 2026-09-08), so `Dr King - Jr` reads family `King`, suffix `Jr` exactly as this rule - states, where it read title `Dr King`, family `Jr` before — - `Dr Smith Jr`, which always read as stated, was the contrast - that isolated the cause and now reads like its neighbour rather - than against it. What the floor cannot reach stays descriptive - and is small: where the run's whole content is the word given - back, no title is left to name anybody, so the word stands as - the name rather than as the family — `Dr Jr` reads given `Dr`, - suffix `Jr`, reporting `title-or-name` (H4), and `Sir Jr`, the - spelling this note first named, reads by the same branches — - given `Sir`, suffix `Jr`, the same kind reported — the run - being empty by then and no branch reading `vocab:given-title`. - The vocabulary half is decided - and unchanged (decisions.md#v1-xfail-triage: `king` stays a - title, for the addressing forms). - "Dr Jr" → suffix="Jr" - history: decisions.md#S2 · interacts: H1, H2, H3, H5, C1, C2, S3, P1, P2, P3, P5, P6, M2 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_classify.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_vocab.py - -S3. Rationale: credentials are often written run together with - periods; the chunks between the periods are what carry the - vocabulary, and a word of several chunks that no vocabulary - knows is still written the way a credential is written. - A word with interior periods reads as a suffix when any of its - period-separated chunks is suffix vocabulary — except where - every chunk the vocabulary matches is a single ASCII character, - the roman numerals and the lone digit the vocabulary lists, - which are about generations rather than credentials. - A word of two or more period-separated chunks that no - vocabulary claims is read by POSITION instead, as a bare - ambiguous acronym is (S2): a credential where the name has words - to spare, a name word where it does not, either reading - reported at the slots S2 reports at, and the same at a comma — - which means the FIRST piece after a family comma, the word - trailing the given part after one, the word ending a maiden - marker's clause (M2), and the part before a SUFFIX - comma. The part before a FAMILY comma never reports, the comma - having already named it the family. C1 states two exceptions at - the first slot after a comma: paired initials ('M.J.') read as - the given name unless something in the part speaks for them, and - a flip to the credential run in which no listed member of the - class takes part is not reported, unless paired initials speak - only for each other. Case says nothing here — the - periods are the evidence — and three shapes are outside it: a - single trailing period is not this shape at all, a chunk that is - not wholly alphabetic is no acronym letter, and a word carrying - a script that writes no abbreviations is not wearing an - abbreviation's periods (H2 refuses the same word for the same - reason). This second half is a caller switch, ON by default, and - the example lines name it; turned off, such a word is name - material and the chunk rule above still decides the rest. - "John Smith Msc.Ed." → suffix="Msc.Ed." - "John Smith Msc.Ed." unlisted_dotted_suffixes-off → suffix="Msc.Ed." - "Doe, John Msc.Ed." → suffix="Msc.Ed." - "John Smith J.u.n.i.o.r." → suffix="J.u.n.i.o.r." - "John Smith Q.W.E.R.T." → suffix="Q.W.E.R.T." - "John Smith X.Y.Z." → suffix="X.Y.Z." - "john smith x.y.z." → suffix="x.y.z." - "John Smith X.Y.Z." unlisted_dotted_suffixes-off → family="X.Y.Z." - "Jack X.Y.I." → family="X.Y.I." · boundary - "John Smith Xyz." → family="Xyz." · boundary - "John Smith 1.4" → family="1.4" · boundary - "Doe, John X.Y.Z." → suffix="X.Y.Z." - "Jane Doe nee Smith X.Y.Z." → suffix="X.Y.Z." - Accepted: the initialless-script clause carries no example line - of its own. Every input that exercises it composes a script that - writes no abbreviations with a period that only a Latin - convention writes — `John Smith 田.中.` is the shape — and a - composed form no writing system produces is tolerated input - rather than contract, which an example of a normative rule would - promise. So the witnesses are the tolerated row - tests/v2/cases.py's - an_initialless_script_glued_into_periods_is_not_this_shape and - the CJK assertions in tests/v2/pipeline/test_vocab.py's - test_period_joined_vocab_retires_the_single_character_chunk. - W3 states the same precedence for the same reason. - history: decisions.md#S2 · interacts: S2, C1, H2, W3 · implemented: nameparser/_pipeline/_vocab.py - -## Nicknames & quoted names (N) - -Background: a nickname is written beside the formal name, set off by quotes or brackets. Quotation conventions vary by language („…“, -«…», “…”), several share characters — one convention's closer is -another's opener — and the straight apostrophe doubles as a quotation mark and as a letter-like mark inside names (O'Connor). Which pairs delimit nicknames is caller configuration. - -N1. Rationale: a quoted or bracketed clause beside a name is an - informal alias, not part of the name. - A clause enclosed by a configured nickname delimiter pair reads - as the nickname and is lifted out of the name, a comma inside it - kept as written — the comma divides nothing there, though the - dots T2 and T3 name still divide a clause as they divide a name; - an empty enclosure is simply dropped. - "Andrew (Andy) Perkins" → nickname="Andy" - "Jane (Janey, Jo) Smith" → nickname="Janey, Jo" - "Jean 'JD' Smith" → nickname="JD" - "Anna () Smith" → nickname="" · boundary - "John ｢Jack｣ Smith" → nickname="Jack" - history: decisions.md#N1 · interacts: T2, T3 · implemented: nameparser/_pipeline/_extract.py, nameparser/_pipeline/_tokenize.py - -N2. Rationale: only a mark standing at word boundaries is quoting; - anywhere else it is part of the word. - A quote whose open and close are the same character opens only - at a word start and closes only at a word end, so an apostrophe - inside or at the end of a word is literal. Between conventions - that share a character, position in the text decides: the - leftmost valid opener wins — and a dangling-open report is - suppressed where its character sits inside another pair's - successful match, being literal content there rather than an - imbalance (A1 depends on that filter). - "Sean O'Connor" → family="O'Connor" - "Hans „Erster“ und “Zweiter” Müller" → nickname="Erster Zweiter" - "Mari' Aube'" → family="Aube'" · boundary - history: decisions.md#N2 · implemented: nameparser/_pipeline/_extract.py - -N3. Rationale: a person set down as a nickname plus one name word is - being identified by surname. - A name that is only a nickname and one name word reads that word - as the family name; with two or more name words the ordinary - positional reading applies. - "'Smitty' Jones" → family="Jones" - "'Smitty' John Jones" → given="John" · boundary - Accepted: the count does not set suffixes aside, so a nickname - plus one name word plus a suffix reads the name word as given - and leaves the family empty. A title counts against the count - too, but H1 then reads the title-plus-one-word name that is - left, so the family is named after all — unless the title is a - given-name title, which keeps the word in `given` and leaves no - family, exactly as it does anywhere else. - "'Smitty' Jones Jr." → family="" - "'Smitty' Dr. Jones" → family="Jones" - "'Smitty' Sir John" → given="John" - Accepted: a marker-led clause is a maiden clause and not a - nickname one (M3), so this rule does not reach a name written - that way, and the one name word keeps the reading the bare - spelling gives it. Since #445 that reading is the family name - all the same, M4 reaching both spellings from the other side. - The two rules agree on THIS name and not in general: M4 counts - name words alone, so where a suffix stands beside them it fires - and this rule declines. The Accepted line above reads a bare - nickname-plus-word-plus-suffix name as given with no family, and - the same name carrying a maiden clause reads family instead. - "Smith (née Jones)" → family="Smith" - history: decisions.md#N3 · interacts: H1, M3, M4 · implemented: nameparser/_pipeline/_assign.py - -## Maiden names (M) - -Background: a maiden name is written beside the current name, set off by a marker or by enclosure. Markers are attested across French née/né and the unaccented nee English writing uses for both, German geb./geborene, Dutch geboren, Czech/Slovak rozená (the abbreviation roz. shipped through 2.1 and was removed in 2.2 -- it collides with the English diminutive Roz, and a caller who needs it adds it to their own Lexicon), Scandinavian født/fødd/född, Russian урожд. and its full participles урождённая/урождённый (the participles in both the ё and е spellings, which case normalization does not fold), Japanese 旧姓, and Polish z domu — both grammatical genders where attested. Every one of these is a marker wherever it stands, the unaccented nee included: an enclosure holding a word past it reads as the maiden name and the marker is dropped, so a configured pair reading "(Nee Jones)" gives maiden Jones and not Nee Jones. A marker need not be one word. z domu is two, and a phrase marker is recognized only whole and only where its words stand together: neither of its words is a marker standing alone, and neither is the pair once a bracketed clause or a comma divides them. That is what makes shipping it safe where shipping its words would not be — z is an ordinary Polish preposition, and a name that merely contains one keeps its family name. Japanese more often writes the marker with a fullwidth colon (旧姓：佐藤), which is no separator, so marker and name arrive as a single word. Which enclosures mean "maiden" rather than "nickname" is a caller convention, so the maiden reading of a delimiter pair is opt-in — except where the clause announces itself. A clause of two words or more led by a marker word has said which convention it means, and reads as the maiden name inside a nickname pair as well (M3) — unless its content is suffix-shaped, which S1 takes ahead of both. A lone marker has said nothing, and neither has one the colon spelling above glues to the name. A marker also says something about the name standing beside it: it announces a surname the bearer no longer uses, and that is only worth writing where there is a current one to tell it apart from (M4). - -M1. Rationale: an enclosure the caller has declared to mean maiden - holds the former family name; a recognized marker word inside it - marks the clause and is not itself part of the name. - With a delimiter pair configured for maiden names, its enclosed - clause reads as the maiden name — unless the content is - suffix-shaped, which S1 takes first — a leading recognized - marker being dropped where the clause holds a word past it; a - clause of nothing but its marker keeps its words, which may - themselves be a surname (Nee). - Clauses are independent: two enclosures read as one maiden name, - each dropping or keeping its own marker. A pair configured for - both maiden and nickname reads maiden. Configuring the pair is - what this rule needs for a clause that does not announce itself - — markerless content, and a lone marker alike, whether that - marker is one word or several; a clause holding a word past its - marker reads as the maiden name inside a nickname pair as well - (M3), the suffix-shaped content S1 takes excepted there as it is - here. - "Jane Smith (née Jones)" maiden-parens → maiden="Jones" - "Jane Smith (Nee)" maiden-parens → maiden="Nee" · boundary - "Jane Smith (Nee) (Jones)" maiden-parens → maiden="Nee Jones" - "Andrew Perkins (MBA)" maiden-parens → suffix="MBA" · boundary - "Maria Kowalska (z domu)" maiden-parens → maiden="z domu" · boundary - "Maria Kowalska (z, domu Nowak)" maiden-parens → maiden="z, domu Nowak" · boundary - history: decisions.md#M1 · interacts: S1, M2, M3, M4 · implemented: nameparser/_pipeline/_extract.py, nameparser/_pipeline/_group.py - -M2. Rationale: a maiden marker announces that what follows it is the - former family name; the marker is an announcement, not a name. - A recognized maiden marker counts only in the part of the name - that holds the family name — the whole name where no comma - divides it or before a suffix comma, the part before a family - comma — and only with a - name word straight before it. A word of the leading title run is - no name word there, a lone title included; neither is a word of - the unambiguous suffix vocabulary, unless it opens the name (S2), - nor a connective. A particle - is one: with nothing after it to attach to, it is no particle - there. In the given part after a family comma, in a part after a - suffix comma, and behind a title, a suffix word or a connective, - the marker is an ordinary word; a name standing somewhere before - it is not enough. - A marker that counts takes the words after it as the maiden name, - and is itself dropped. It takes them up to the trailing run of - post-nominals and titles that the end of the name reads as if the - clause were not written — the trailing rules (S2, H5) asked over - the words before the marker and the post-nominal or title words - that end it, before any join — and always takes the first word - after it, the marker having announced a name. A suffix word - straight after the marker announces nothing, and the marker is - then a word. The run so found ends the name: its words are the - suffixes and titles that reading makes them, and no join reaches - into it. Before a family comma no trailing rule reads the part, so - the clause ends at the first suffix word, and the words from it - on read as that part reads them. - Where a trailing rule reads the part, a member of the ambiguous - class (S2) that ends the maiden name is reported where the clause - keeps it, and one in the run is reported where the run is read, - so no word is reported twice and none goes unreported. Before a - family comma no rule reads it, and the member the clause keeps - there is reported nowhere. - Delimiters outrank every reading inside them. Where a recognized - marker stands inside a delimited clause, the whole span is the - maiden name whatever its last word is, and whether or not the - pair is a configured maiden delimiter (M3): the writer drew the - boundary, so no fork is called and nothing is reported. A word - the writer left OUTSIDE the span is outside the clause and reads - as it would anywhere else. Bracketed content ending in a period - is the exception, because it never reaches this rule as a - delimited clause: a trailing period makes the content - suffix-shaped (M3 states this), so the brackets are dropped (S1) - and the clause is read as if written bare. - A marker - with nothing after it, or nothing before it, is just a word. - A marker may be more than one word, and is then recognized only - whole and only where its words stand together: its own first word - standing without the rest is not that marker, and neither is one - the writer divided from the rest by a bracketed clause or by a - comma. Whether the first word takes anything on its own is a - separate question with a separate answer — it does exactly when it - is a recognized marker in its own right, and the longer entry wins - wherever both could match. - A marker taken this way also bounds a particle join arriving from - its left (P2), so the family name's particles stop at the marker - instead of absorbing it; a marker left as a word bounds nothing. - Where the bound leaves a family of nothing but particles, they - are not in particle position and read as ordinary words (R2). - "Jane Smith née Jones" → maiden="Jones" - "Maria López née García Pérez" → maiden="García Pérez" - "Jane Smith née Jones PhD" → suffix="PhD" - "John née Jones Smith V" → maiden="Jones Smith" - "John née Jones Smith V" → suffix="V" - "Doe nee Smith, Jane" → maiden="Smith" - "Doe, Jane nee Smith" → maiden="" - "Doe, Jane nee Smith" → middle="nee Smith" - "Jane Smith, née Jones" → maiden="" - "Smith, John, PhD née Jones" → maiden="" - "Jane Doe PhD nee Smith" → maiden="" - "Jane and née Jones" → maiden="" - "Dr. nee Smith PhD Prof." → maiden="" · boundary - "Dr. nee Smith PhD Prof." → family="Smith" · boundary - "Mai Le née Nguyen" → maiden="Nguyen" - "abd née Jones" → maiden="Jones" - "Jane née Jr y Jones" → maiden="" - "Jane Doe nee Smith MA" → maiden="Smith" - "Jane Doe nee Smith MA" → suffix="MA" - "Jane Doe nee Smith Ma" → maiden="Smith Ma" · boundary - "Jane Doe nee MA" → maiden="MA" · boundary - "Jane Doe nee MA Smith" → maiden="MA Smith" · boundary - "John née Jones Smith Ma" → maiden="Jones Smith Ma" - "John nee Prof. ba MA" → maiden="Prof. ba" · boundary - "Jane Doe nee Puig i Soler" → maiden="Puig i Soler" - "Jane Doe nee Puig i" → maiden="Puig" · boundary - "Jane Doe nee Puig i" → suffix="i" · boundary - "Jane Doe nee Smith Prof." → maiden="Smith" - "Jane Doe nee Smith Prof." → title="Prof." - "Jane Doe nee Smith MA Prof." → suffix="MA" - "Jane Doe nee Smith Prof. MA" → maiden="Smith" - "Jane Doe nee Smith V Prof." → suffix="V" - "Jane Doe nee Smith VI Prof." → suffix="VI" - "Jane Doe nee King." → maiden="King." · boundary - "Jane Doe nee Prof. Dr." → maiden="Prof." · boundary - "Jane van der Berg nee Smith Prof." → title="Prof." - "Jane van der Berg nee Prof. King. MA" → maiden="Prof." · boundary - "Jane Doe nee Smith PhD Smith" → maiden="Smith" - "Jane Doe nee Smith PhD Smith" → suffix="PhD Smith" - "Doe nee Smith Jr. Prof., Jane" → family="Doe Prof." · boundary - "Jane Doe (nee Smith MA)" → maiden="Smith MA" - "Jane Doe (nee Smith Ma)" → maiden="Smith Ma" - "Jane Doe (nee Smith) MA" → suffix="MA" - "Jones née" → family="née" · boundary - "née Jones" → family="Jones" · boundary - "Jane van der Berg née Jones" → maiden="Jones" - "Jane de la née Jones" → maiden="Jones" - "Jane de la née Jones" → family="de la" - "Jane van der Berg née" → family="van der Berg née" - "Jane van der Berg née PhD" → family="van der Berg née" - "Jane van der Berg née y Jones" → maiden="y Jones" - "Maria Kowalska z domu Nowak" → maiden="Nowak" - "Anna z Nowak" → family="Nowak" · boundary - "Anna z (domu) Nowak" → family="Nowak" · boundary - Accepted: the fullwidth-colon spelling arrives as one word, so - the marker inside it goes unrecognized; #317 tracks whether it - should peel. - "山田 花子 旧姓：佐藤" → maiden="" - Accepted: a given name with no current surname beside a maiden - name reads by M4's lone-word rule, the lone name word before the - marker being the family. A writer giving a given name with its - birth name writes the clause delimited. - "Jane née Jones Smith" → maiden="Jones Smith" · boundary - Accepted: the run is what the trailing rules read, before any - join — not the whole reading of the clause-free name. Where a - particle chain would take the words of the run in that name, the - run keeps them suffixes and titles. - "Jane Doe nee Smith DO DO" → suffix="DO DO" · boundary - Accepted: the first word after the marker is the maiden name - whatever it is unless it is a suffix word, and a lone roman - numeral is not one — it is shaped like an initial — so it is the - maiden name where the clause-free name would read it as the - suffix. Decided with #601 (decisions.md#M2, 2026-10-04). - "Jane Smith née V" → maiden="V" · boundary - Accepted: bracket content ending in a period is suffix-shaped - (M3), so the brackets are dropped (S1) and the clause is read as - if written bare — the delimiter precedence above does not reach - it. - "Jane Doe (nee Smith Prof.)" → title="Prof." - Accepted: a marker behind a credential is an ordinary word, and - the credential's run (S2) takes it in with everything after it. - "Jane Doe Jr. nee Smith" → suffix="Jr. nee Smith" · boundary - history: decisions.md#M2 · interacts: P2, P3, P5, P6, R1, R2, M1, S1, S2, H1, H5, C1 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py - -M3. Rationale: an enclosure says nothing about whether it means - maiden, but a recognized marker word inside it does — the clause - announces itself, so the caller does not have to declare the - pair. - A bracketed clause whose content opens with a recognized marker - and carries a word after it reads as the maiden name, - whichever bucket the enclosing pair sits in — unless the content - is suffix-shaped, which S1 takes first — the marker itself - dropped, as M1 drops it. A marker with no word after it is just - a word in brackets, and so is a marker no separator divides from - the name, the fullwidth-colon spelling M2 records. And a marker - followed by a comma is no marker, whether the comma stands alone - or is glued to either word — here and in M1's drop alike, so a - clause the caller declared maiden keeps that marker. Where the - pair is already configured for maiden names M1 governs and this - adds nothing. Being keyed on the content rather than on the - pair, this reaches a nickname pair's clause too, and M1's - independence then governs what it produces: where a marker-led - clause stands beside another marker-led clause, both read as - maiden and join into one maiden name, leaving no nickname. - "Jane Smith (née Jones)" → maiden="Jones" - "Jane (née Jones) Smith" → family="Smith" - "Jane Smith (née Jr.)" → suffix="Jr." - "Jane Smith (née)" → nickname="née" · boundary - "Maria Kowalska (z domu Nowak)" → maiden="Nowak" - "Maria Kowalska (z domu)" → nickname="z domu" · boundary - "Jane (née , Jones) Smith" → nickname="née , Jones" · boundary - "Jane (née ,Jones) Smith" maiden-parens → maiden="née ,Jones" · boundary - Accepted: the word taken after the marker is not tested for - being a name word, so unlike M2's bare take this one does not - stop at a suffix word — the same two words read one way - bracketed and another way bare. This does not contradict the - S1 example above: S1 asks whether the WHOLE clause is - suffix-shaped, which the trailing period makes true of the one - and false of the other, so only the V clause reaches this rule. - "Jane Smith (née V)" → maiden="V" - history: decisions.md#M3 · interacts: M1, M2, S1, N1 · implemented: nameparser/_pipeline/_extract.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_vocab.py - -M4. Rationale: a maiden name is a FORMER family name, and a former - one only means something beside a current one — nobody announces - the surname they used to carry where there is no surname beside - it to tell it apart from. So where a maiden name has been read - out of a name and a single name word is left standing, that word - is the surname the bearer uses now; writing the clause at all is - what rules out the given-name reading O5's convention would - otherwise leave it with. How the maiden name was written does - not enter into it — a marker taking the words after it (M2), a - marker-led clause (M3), and a clause in a pair the caller - declared to mean maiden (M1) announce the same thing, and the - last of those carries no marker at all. - A maiden name standing beside exactly one name word makes that - word the family name, whatever suffix or nickname stands beside - it — an annotation beside the name is no part of the name, which - is how H1 reads the same shape. Two kinds of word are beyond its - reach, for one reason: the rule changes what POSITION would have - decided, so it cannot overrule what a word already IS - (mechanisms.md#TWO-LAYER-ASSIGN). A word the vocabulary has - claimed as a given name keeps that reading, and so does a word - read as an initial, which is nobody's family name. A name - carrying a TITLE is H1's rather than this rule's, H1's - given-name-title carve-out included, which keeps the word a - given name. A nickname holds nothing off: where N3 has already - named the family this rule finds nothing left to move, and where - N3 declined — its count does not set a suffix aside — this rule - names it. Where the maiden name stood does not matter, only what - is left beside it: a marker inside the name takes the rest of it - (M2), and one name word left that way is one name word. - "Smith née Jones" → family="Smith" - "Smith (née Jones)" → family="Smith" - "Smith (Jones)" maiden-parens → family="Smith" - "Jane née Jones Smith" → family="Jane" - "Smith née Jones PhD" → family="Smith" - "abd née Jones" → given="abd" · boundary - "J. née Jones Smith V" → given="J." · boundary - history: decisions.md#M4 · interacts: M1, M2, M3, H1, N3, O5 · implemented: nameparser/_pipeline/_post_rules.py - -## Commas & structure (C) - -Background: a comma in a name signals one of two conventions — the listing form "Family, Given" or trailing credentials "Name, PhD" — and which is meant can only be judged from what stands after the first comma. Recognizing a credential run is by nature a vocabulary judgment, so this is the one structural decision that consults the suffix word lists. Which characters COUNT as the comma is part of the rule: the Arabic comma (U+060C) and the fullwidth comma (U+FF0C) both signal the listing form, while the ideographic comma (U+3001) is not a name-structure comma at all (#265). - -C1. Rationale: a credential run after the comma means the name is in - natural order with suffixes appended; anything else after the - comma means the listing form. - With a comma present, the name reads as trailing suffixes when - the part after the first comma is entirely suffix words and more - than one word precedes the comma; otherwise it reads as the - listing form, the part before the comma being the family name, - less a title run opening it (H6), unless the part after it fixes - no family boundary (below). Only - the part after the first comma decides. - Wherever this rule counts the words before the comma, a particle - run and the one name word it attaches to are one word, the reach - P1's fold counts rather than the whole of P2's chain: the listing - form puts a surname before the comma, and a particle surname is - one surname ('De La Cruz, Ed' reads as 'Royce, Ed' does, 'van der - Berg, PhD' as 'Berg, PhD'). Where that surname is the whole part, - the comma settles P1's fork for a leading particle that could be - a given name, being the evidence that the part is a surname; a - word after the one the particle attaches to is a second word, and - a suffix word ends the particle's reach. A word that is also - title vocabulary is not a particle to this count where it opens - the part, reading there as the title it also is (H1); inside a - surname it chains ('de St Pierre, Ed' reads given 'Ed'). A - connective join and a bound given-name pair are not one word - here: whether P3 joins a connective depends on the words of the - whole name, which this count is part of deciding, and a bound - given name builds a given name rather than a surname (P5) — a - word that is a particle as well counts as the particle ('Abu - Bakar, Ed' reads given 'Ed'). A title in front is a word to the - count an unambiguous credential takes, so 'Dr. van der Berg, PhD' - reads as its part does alone. - For the ambiguous credential class — a bare acronym the - vocabulary marks as also an ordinary name, and a word admitted - to the class by shape, which S3 defines and bounds — the - count before the comma is of NAME words rather than of words: - two or more of them read the part after the comma as the - credential run, whatever case the name is written in. The count - goes FIRST and written case is asked only after it: where two - name words stand before the comma the part after it is the - credential run however that part is written, and only where the - count leaves the word a name — one name word before the comma — - is the case read, capitals in a mixed-case name making it the - credential there too (S2). Paired initials are the exception to - the count: two single letters joined by a period, with or without - a period after the second ('M.J.', 'M.J'), the one dotted shape a - person's own initials are written in, read as the given name - after the comma, because two words before the comma may be one - surname written in two ('García Márquez') and the count cannot - tell them from a given name and a family name. Only an - unambiguous suffix word in front of them that is not also title - vocabulary, or another word the class admits by its dotted shape - standing in the same part, makes them the credential run. A word - of this class in front of them speaks for nothing, as S2's - company has it, and where nothing but words of both the suffix - and the title vocabulary stands in front of them, those are the - titles of the given part the initials open, as S2 reads them - there. Behind two or more name words, paired initials that are - not the only shape word in the part are a credential however - little else speaks for them, since no one writes a person's - initials as two dotted groups. An unlisted all-caps word joins - the class in such a part only where the name carries the - contrast: one of its own words before the comma written the way a - name is written in mixed case, holding a capital with its last - cased letter lowercase — read after composing the word, passing - over every non-letter and caseless letter and any lowercase letter - whose capital is not a single letter, as ß's is not — with no - period and not claimed by the vocabulary - as a title, particle, connective, credential or generation. A - surname written in capitals ends in a capital whatever is glued in - front of it, and a word written wholly in lowercase holds none, so - such a record keeps its given name beside its lowercase particles, - titles and clauses ("GISCARD d'ESTAING, VALÉRY", 'LLOYD - FitzGERALD, RONALD'), and a name written wholly in lowercase reads - the listing form. Where the name carries it, an - unlisted word of two capitals after the comma is the paired - initials' shape undotted and is decided at this comma exactly as - they are, by the sentences above ('García Márquez, MJ' and 'García - Márquez, MJ PhD' keep given 'MJ'). The same count reads a part of two or - more words as the credential run when every word of it is a - suffix word or a word of this class, at least one of them of this - class, and none of them a single-letter roman numeral, in any - case: one letter is the shape a middle initial is written in, - and S3 retires single-character matches for the same reason, - while a multi-letter numeral or generational word stands in a - run like any suffix word ('John Smith, III Ma'). A word's - particle reading does not take it out of the run: a word of both - the particle and the unambiguous suffix vocabulary is a suffix - word there too ('John Smith, vd Ma'). Run membership asks what - the word is, and S2's company asks whether it speaks for the word - behind it, which a particle never does; the run S2 says such a - word ends is the company's, not this count's. A word of both the - title and the suffix vocabulary opening a part the count reads as - a run counts as a suffix word there, the name before the comma - being complete, except as the titles of paired initials, above. - Where every word - of this class in the part is a LISTED word written in capitals in - a mixed-case name, the writing has already made each of them the - credential (S2), and the part reads as the credential run on that - evidence rather than on the count; a word of the class by shape - alone carries no such lean, so a part holding one is read by the - count. A part read as postnominal — the credential run, titles - beside suffixes, or titles alone — is read as its words stand, - before any join, and - that reading is final: the one join made inside it is a - connective binding title words into one title (P3: 'Mr. and - Mrs.', a connective run included: 'Minister of the Interior'), - and no particle run, connective join or bound given name - reaches into it. The listing form's given part keeps every join - (P2, P3, P5, P6). - A decision either way at this comma - is reported; for a run of words the decision is the flip to the - credential run, reported once over the whole part. A flip in which - no listed word of this class takes part is the exception and is - made in silence, since only paired initials among such words are - ever taken for a name, and the flip reaches them only where - something else in the part has said otherwise — unless what said - so is nothing but other paired initials: each of those is a word - a reader takes for a name, and that flip is reported. Paired - initials the comma keeps as the given name report where they open - the part; behind a title they do not, the comma's report reaching - only the first word after it ('Smith, Ms G.J.'). A run the - count leaves in the listing form reports as S2 reads the words - in it: a word of this class read as the credential because a - credential in front speaks for it reports, and one its own - capitals made the credential does not unless it opens the part, - where it is the first word after the comma and reports as that - word always does ('Doe, MA PhD', 'John Smith, MA MA'). So a run - whose every such word is written in capitals reads whole in - silence only behind another credential ('John Smith, PhD MA', - 'Smith, PhD MA'), as does a part read as titles before a lone - given name ('Smith, Ms MD Ma'). It is one - of TWO places the comma's own decision is reported, the other being the word trailing the given part after - it (S2), which is a second decision about a second word and never - the same fork twice; an attachment decided after a family comma - (P6) reports on its own. C2's comma-structure flag reports what the - parse could not recognize, not a fork it called. - By default a recognized suffix word counts - even written like an initial ("V."), while strict mode vetoes - initial-shaped words. In the listing form the part after the - comma is still read for what it is: a part that is nothing but - suffix words is the credential run and reads as suffixes, whole - — the slot after a family comma is postnominal position, so the - vocabulary's verdict comes before any title reading of the same - word — and a part that holds no name word at all, titles and suffixes - only, fixes no family boundary, so a part before the comma with - more than one name word keeps its positional read, order and - all. A name word in the part after the comma makes it the - given name, with titles before it and suffixes after, unless a - credential opens the part. A credential opening the part makes it - the postnominal part however it goes on: where the first suffix - word of the part, with only titles in front of it, is one that - starts S2's run, which S2 bounds (the split 'Ph. D.' among them), - every word after it reads as that run reads, a title word as a - title and any other word as a suffix, under strict mode too, - whose veto refuses a word the suffix reading and not the run - ('Smith, PSM I.'). The comma has then fixed nothing, so a part - before it holding two or more name words keeps its positional - read, and a part holding one is the family ('John Smith, PhD - Jones', 'Doe, PhD Jones'). A word of both the title and the - suffix vocabulary opens nothing, and nor does a credential behind - one: in front of a given name it is the listing form's title - ('Smith, Ms Jane'), and no count of the words before the comma - tells a surname of two words from a given name and a family name. - A one-character suffix word — the only kind a reader could take - for an initial — is read by what stands before it. Behind - another suffix it is describing that suffix and continues the - run, written with a period or without, an initial being no shape - anyone writes there; behind a name word it is the generation - only when written bare, a period marking it the abbreviation of - a name and so a middle initial. Both branches then ask that the - generation slot still be open: where a further comma has already - named the suffix, a single letter ending the GIVEN part has no - generation left to be, and stays a middle initial — with a - period or without, and whether a name word or another suffix - word stands before it. Only the given part is touched; a part - after the comma that is nothing but suffix words is the - credential run, and a letter in it continues that run up to the - further comma. Longer suffix words are not in question either - way, and the strict knob above still vetoes the initial-shaped - ones, so the run ends at them there unless a credential opened - the part, above. - A delimiter the policy declares parts a trailing suffix part as a - comma would: the words on each side of it read exactly as the - parts of the same text written with a comma in its place, so no - join, maiden clause or connective reaches across it, and the - delimiter itself is dropped, unless it stands alone as the whole - of its part. Only a trailing part is parted: in the part before - the first comma, or in the given part of the listing form, the - delimiter is a word (the Accepted entry below). - "Smith, John" → family="Smith" - "سلمان، محمد" → family="سلمان" - "田中、太郎" → family="" - "John Smith, PhD" → suffix="PhD" - "John Smith, V." → suffix="V." - "John Smith, V." strict-comma-suffixes → family="John Smith" - "Smith, PhD" → family="Smith" · boundary - "Smith, PhD" → suffix="PhD" - "Smith, Jr." → suffix="Jr." - "Smith, Sr." → suffix="Sr." - "Smith, PSM I" → suffix="PSM I" - "Smith, PSM I." → suffix="PSM I." - "Smith, PSM I." strict-comma-suffixes → suffix="PSM I." - "Smith, John V." → middle="V." - "Smith, John PhD I." → suffix="PhD I." - "Smith, John V" → suffix="V" · boundary - "Smith, Ph. D. Jr." → suffix="Ph. D. Jr." - "Smith, MD PhD" → suffix="MD PhD" - "Doe, MA PhD" → ambiguities=("suffix-or-name",) - "Smith, Dr." → title="Dr." - "Smith, Dr. Jr." → suffix="Jr." - "John Smith, PhD Jones" → given="John" - "John Smith, PhD Jones" → suffix="PhD Jones" - "John Smith, PhD Jones" → ambiguities=("suffix-or-name",) - "Doe, PhD Jones" → suffix="PhD Jones" - "John Smith, Ph. D. Jones" → suffix="Ph. D. Jones" - "John Smith, MD Jones" → given="Jones" · boundary - "Smith, Ms Jane" → title="Ms" · boundary - "John Smith, Mr." → given="John" - "John Smith, Mr." → family="Smith" - "John Smith, Mr. Jr." → given="John" - "Smith Jr., Mr." → family="Smith" · boundary - "John Smith, Jones" → family="John Smith" - "John Smith, MA" → suffix="MA" - "John Smith, Ma" → suffix="Ma" - "Smith, MA" → suffix="MA" - "Smith, Ma" → given="Ma" - "John Smith, Ed" → suffix="Ed" - "Davis Royce, Ed" → suffix="Ed" - "Royce, Ed" → given="Ed" · boundary - "Smith Jr., MA" → suffix="Jr., MA" - "Smith Jr., Ma" → given="Ma" · boundary - "John Smith, PhD MEng" → suffix="PhD MEng" - "John Smith, Ed Ma" → suffix="Ed Ma" - "Jane Doe, MS LAc" → suffix="MS LAc" - "Smith, PhD MEng" → family="Smith" · boundary - "John Smith, PhD DO DO" → suffix="PhD DO DO" - "John Smith, PhD DO DO" → ambiguities=() - "John Smith, Mr. and Mrs." → given="John" - "John Smith, Mr. and Mrs." → title="Mr. and Mrs." - "Smith, Chancellor of the Exchequer MP" → suffix="MP" - "Ortega y Gasset, Dr." → given="Ortega" - "John Smith, PhD vd DO" → suffix="PhD vd DO" - "John Smith, vd Ma" → suffix="vd Ma" - "JOHN SMITH, VD MA" → suffix="VD MA" - "JOHN SMITH, VD MA" → ambiguities=("suffix-or-name",) - "John Smith, Jr vd" → ambiguities=() - "John Smith, Jr do" → ambiguities=("suffix-or-name",) - "John Smith, X.Y.Z." → suffix="X.Y.Z." - "John Smith, X.Y.Z." → ambiguities=() - "John Smith, X.Y.Z." unlisted_dotted_suffixes-off → given="X.Y.Z." - "John Smith, XYZ" → suffix="XYZ" - "John Smith, XYZ" → ambiguities=("suffix-or-name",) - "John Smith, XYZ" unlisted_caps_suffixes-off → given="XYZ" - "John Smith, LEED AP" → suffix="LEED AP" - "Smith, XYZ" → given="XYZ" · boundary - "García Márquez, MJ" → given="MJ" · boundary - "MÜLLER WEIß, HANS" → given="HANS" · boundary - "García Márquez, MJ PhD" → given="MJ" · boundary - "García Márquez, MJ JK" → suffix="MJ JK" - "John Smith, PhD XYZ" → suffix="PhD XYZ" - "Smith, A.B." → given="A.B." · boundary - "García Márquez, G.J." → given="G.J." - "García Márquez, G.J." → family="García Márquez" - "John Smith, A.B." → given="A.B." - "John Smith, A.B." → ambiguities=("suffix-or-name",) - "John Smith, X.Y. P.Q." → suffix="X.Y. P.Q." - "De La Cruz, M.J. K.L." → given="M.J." - "De La Cruz, M.J. K.L." → ambiguities=("suffix-or-name", "suffix-or-name") - "De La Cruz, Ed" → given="Ed" - "van der Berg, MA" → family="van der Berg" - "van der Berg, MA" → suffix="MA" - "Van Buren, Ed" → family="Van Buren" - "van der Berg, PhD" → family="van der Berg" - "John van Buren, Ed" → suffix="Ed" · boundary - "Ortega y Gasset, Ed" → suffix="Ed" · boundary - "abdul Salam, Ed" → suffix="Ed" · boundary - "John Smith, PhD X.Y." → suffix="PhD X.Y." - "García Márquez, G.J" → given="G.J" - "De La Cruz, M.J. PhD" → given="M.J." · boundary - "García Márquez, Ms G.J." → given="G.J." · boundary - "García Márquez, Ed G.J." → given="Ed" · boundary - "John Smith, A.B. Ph.D." → given="A.B." · boundary - "Smith, John, Puig - i Soler" extra_suffix_delimiters-dash → suffix="Puig, i Soler" - "Smith, John, PhD - and MD" extra_suffix_delimiters-dash → suffix="PhD, and MD" - Accepted: a title in front of a particle surname, before the - credential class's word, reads the listing form. The surname - counts once and the title is no name word, so the count reads the - listing form, and the listing form peels a title before the comma - (H6) unless the word is a particle too, as 'Freiherr' is, which - stays in the family. For a German rank that reading is the right one: - since 1919 a former noble title is part of the legal surname, - written between the given name and the particle, so 'Freiherr von - Berg' is the family name in 'Freiherr von Berg, Ed', as 2.0 - through 2.3 read it. Before an unambiguous credential the same - words keep the title apart ('Freiherr von Berg, PhD'), that count - taking the title as a word; both are readings of one name. - "Freiherr von Berg, Ed" → family="Freiherr von Berg" - Accepted: three or more initials run together with periods behind - a surname of two words read as a credential. Initials are - conventionally written apart ('J. R. R.'), and a token of three or - more single letters run together is far likelier a credential than - a given name and a middle name written as one word. - "García Márquez, G.J.R." → suffix="G.J.R." - Accepted: a word of both the title and the unambiguous suffix - vocabulary reads as the postnominal after a family comma in - every spelling, the honorific's too — position decides for the - duals, and the slot is postnominal. - "Smith, Ms." → suffix="Ms." - "Smith, Ms. Jane" → title="Ms." - Accepted: the vocabulary question above — is the part after the - comma nothing but suffix words — is asked a second time, without - the word-count condition, to decide whether a glued East Asian - honorific standing before a family comma comes off. That - composition is tolerated input rather than contract — a comma - between a family name and a given name is no part of CJK writing - — so the precedence is stated at W3, with its examples, and not - here. It is also the one consequence of this question Latin - script cannot witness, nothing there being glued to the end of a - name — which is why this clause carries no example of its own. - Accepted: a maiden marker in a part a declared delimiter - separates takes no clause, as no marker in a part after a suffix - comma does, the marker being an ordinary word there (M2). - Releases 2.0 through 2.3 read past the delimiter and took the - clause; the comma reading is accepted as the cost of one rule - for a declared separator. - "Smith, John, MD - née Jones Smith" extra_suffix_delimiters-dash → suffix="MD, née Jones Smith" - Accepted: a delimiter core the policy names (T1) is a word here, - not structure — v1 applied the delimiter to the suffix-comma - form alone, and that limitation is kept as parity: the core in - 'Smith, RN - CRNA' is a word under the policy as without it. - Since #603 it is a word of the run the credential opens, so the - part reads suffix "RN - CRNA" where 2.0 through 2.3, like v1, - read given "RN". - "Smith, RN - CRNA" → suffix="RN - CRNA" - "Steven Hardman, RN - CRNA" → given="Steven" - Accepted: the further-comma qualifier carries no example line of - its own. It discriminates PAIRS and spans both branches, so - exemplifying it means a with-comma partner for each — every one - of them a promise about behavior that has not moved since v1. - Its executable witness is instead the pair - already standing in the v1-style bank, - tests/test_suffixes.py's - test_roman_numeral_i_after_single_initial_lastname_comma_format - and test_roman_numeral_i_with_explicit_suffix_comma_stays_a_middle_initial. - Read the examples above with the qualifier in hand: `Smith, John - V` reads the suffix and `Smith, John PhD I.` continues the run, - while adding a suffix comma after either turns that same letter - into the middle initial. - history: decisions.md#C1 · interacts: H1, H2, H6, P1, P2, P3, P5, P6, W3, S2, S3, M2 · implemented: nameparser/_pipeline/_comma.py, nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py - -C2. Rationale: text beyond the recognized comma parts should be - taken in without silent guessing. - Parts beyond the second are consumed as suffixes either way, - except that a word of the title vocabulary there that is not also - suffix vocabulary is a title: a part past the second comma holds - no name word, so the surnames H5 keeps a bare trailing title word - from misreading are not there to protect, and a piece holding a - suffix word, a title/suffix dual among them, stays the postnominal - the slot makes it (#603). A non-empty extra part that - is not entirely suffix words is flagged as a structural ambiguity - rather than rejected — parsing never fails on content — unless - its words are title words and suffix words, one title word at - least, a word whose period-joined chunks read as a title - ('Lt.Gov.') counting as one. That is asked word by word, so a part - a connective joins into one title ('Secretary of State') keeps - the flag. An empty - part between doubled commas is consumed silently. - A part the parse reads as a credential run by some route other - than the suffix vocabulary is recognized and is not flagged: a - run of ambiguous acronyms whose written case leans credential - (S2), and a run every word of which is an unlisted DOTTED word - the position reads as a credential (S3). This is the one place - the ambiguous class QUIETS a report rather than adding one, and - it is narrow in two ways that the examples below pin. A member - whose written case does not lean CREDENTIAL keeps the flag, - whether the name is written in one case so that nothing leans at - all, or the member is written the way a name is written. And - S2's other by-shape half, the unlisted all-caps word, does not - reach here in any setting: the shape a tail segment is - recognized by is the dotted one alone. - A part the parse consumes wholly as suffixes raises no report - that a word of it was read as a name, whether it is the part - after a suffix comma (C1) or a part beyond the second: nothing - in it is read as one, a maiden marker there being an ordinary - word (M2), and no particle chain (P2) is made in it, so a title - word behind a particle run there is a title like any other, and - the part reports neither a particle chained onto a name word nor - an acronym taken into the name. What such a part reports is its - own — C1's report of the decision at its comma (the flip, or the - first word after the comma), S2's report on each word a credential - in front of it reads as a suffix although it may be a name word (a - name word the run absorbs, a member the company decides), and this - rule's flag. - "John Smith, MD, Bart" → suffix="MD, Bart" - "Eric H. Holder, Jr., Attorney General" → title="Attorney General" - "Eric H. Holder, Jr., Attorney General" → ambiguities=() - "Eric H. Holder, Jr., Secretary of State" → ambiguities=("comma-structure",) · boundary - "Smith, John, Prof." → title="Prof." - "John Smith, Jr., van der Berg Prof." → title="Prof." - "John Smith, Jr., van der Berg Prof." → suffix="Jr., van der Berg" - "John Smith, Jr., Lt.Gov." → ambiguities=() - "John Smith, MD, Ms" → suffix="MD, Ms" · boundary - "John Smith, MD,, Jr." → suffix="MD, Jr." · boundary - "John Smith, MD, R.A.I." → suffix="MD, R.A.I." - "John Smith, MD, R.A.I." → ambiguities=() - "John Smith, MD, R.A.I." unlisted_dotted_suffixes-off → ambiguities=("comma-structure",) - "John Smith, MD, Ma" → ambiguities=("comma-structure",) · boundary - "Steven Hardman, MD, DO, DDS" → ambiguities=() - "STEVEN HARDMAN, MD, DO, DDS" → ambiguities=("comma-structure",) · boundary - "John Smith, MD, XYZ" unlisted_caps_suffixes-everywhere → ambiguities=("comma-structure",) - Accepted: the no-name-reading clause's REPORT half carries no - example line of its own. What it moves is a report with no field - beside it, and an example line would promise that report alone; - its title half is witnessed by the two `John - Smith, Jr., van der Berg Prof.` lines above, and its report half's - executable witnesses are the case rows - a_credential_run_after_the_comma_reports_no_chain_fork and - a_part_past_the_second_reports_no_particle_fork in - tests/v2/cases.py, read beside the two rows the chain still - reports on outside a tail: - the_chain_reports_the_acronym_it_takes and - titled_particle_chain_survives_a_title_that_is_also_a_particle. - history: decisions.md#C1, decisions.md#S2 · interacts: C1, H5, P2, P5, S2, S3 · implemented: nameparser/_pipeline/_segment.py, nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py - -## Name order (O) - -Background: written name order varies by convention: given-first (the library's default reading), family-first, and family-first with the given name last (Vietnamese, where the person is called by the last element, given names are frequently two syllables — the given_names view stays correct wherever the internal boundary falls — and quốc ngữ is Latin script, so no native-script signal exists at all). The order is declared by the caller or a locale pack, never detected — but a few conventions leave a recognizable trace in the name itself. Patronymics are one: East Slavic names carry a father's-name derivative with distinctive endings between given and family, and Turkic names use a standalone marker word ("oglu" son-of, "qizi" daughter-of) after the father's name. Where such a trace is present and unambiguous, an opted-in parser can restore the intended reading from a family-first listing. -A declared order is a property of the DATA SOURCE rather than of any one string: the caller sets it to match how their records are written, and it governs what no vocabulary and no script license has already claimed (O4). A declared FAMILY-FIRST order outranks what the parser could infer from the shape of a particular name; the given-first reading is the parser's DEFAULT rather than a declaration it can tell apart from one, and the traces above are what refine it (O1, O2). And the declaration yields where a name's own script settles the order instead (W4, where a name written wholly in an East Asian script reads family-first whatever order the caller declared). Two consequences run through this document. Under the default given-first order a string opening with a never-given particle is a surname whose given name is simply absent, so the fold takes the rest of it (P1). Under a declared family-first order the caller has already said that what follows the family is not more surname — so the fold stops there, and the rotations below, whose whole job is to RESTORE the default reading from a family-first listing, have nothing left to restore. A shape neither order settles is the family comma's job, and the parser does not guess at it. - -O1. Rationale: an East Slavic name written family-first still shows - its patronymic — the distinctive ending identifies which word is - the patronymic, and the patronymic sits next to the given name. - With East Slavic patronymic handling active and no comma in the - name, a name of exactly three name words — titles, suffixes and - nicknames aside — whose last name word carries a patronymic - ending and whose middle name word does not reads as family-first: - the words are restored to given, patronymic, family. A middle - word that also carries the ending blocks the reading, because - the surname itself may be patronymic-derived. - The restoration is the default order's work: it recovers the - given-first reading a family-first listing hides, so where the - caller has DECLARED a family-first order there is nothing left - to restore and position decides. - "Сидоров Иван Петрович" [ru] → family="Сидоров" - "Sidorov Ivan Petrovich Jr." [ru] → family="Sidorov" - "Иван Петрович Абрамович" [ru] → family="Абрамович" · boundary - Accepted: under a declared family-first order the readings part - on natural-order input, and the declaration wins. A family-first - listing reads the same either way under FAMILY_FIRST — the first - example above is that same parse with FAMILY_FIRST declared, - and under FAMILY_FIRST_GIVEN_LAST the listing's given and middle - swap, so only the family is invariant — while with East Slavic - handling active and FAMILY_FIRST declared the natural-order - Иван Петрович Сидоров reads family Иван, given Петрович, middle - Сидоров. That is the caller's declaration being honored on input - they said was written family-first, not a defect (#384). No - registered example annotation combines a pack with an order, so - the parse is pinned in tests/v2/test_locales.py instead, and - decisions.md#O1 records why options 2 and 3 were declined. - history: decisions.md#O1 · implemented: nameparser/_pipeline/_post_rules.py - -O2. Rationale: a Turkic patronymic marker is a separate word that - follows the father's name; a four-word name ending in one is a - family-first listing. - With Turkic patronymic handling active and no comma in the name, - a name of exactly four name words — titles, suffixes and - nicknames aside — ending in a standalone patronymic marker reads - family-first: the first name word is the family name, and the - marker stays beside the father's name in the middle. - The scope is O1's: the restoration recovers the default reading, - so a declared family-first order stands in its place. - "Ali Ahmad Vali oglu" [tr_az] → family="Ali" - "Ali Ahmad Vali oglu Jr." [tr_az] → family="Ali" - Accepted: any other count of name words keeps its positional - reading, even when that leaves the marker itself in a name - field. - "Ali Ahmad oglu" [tr_az] → family="oglu" · boundary - Accepted: the order scope above has this rule's own consequence - reached by a second route — with Turkic handling active and - FAMILY_FIRST_GIVEN_LAST declared, Ali Ahmad Vali oglu reads - given oglu, the marker standing in a name field because the - declaration put the name's last word there (#384). - history: decisions.md#O2 · implemented: nameparser/_pipeline/_post_rules.py - -O3. Rationale: several traditions write compound family names - unmarked, so that every word after the given name belongs to the - family name. - With compound-family handling active, every middle word joins - the family name and is rendered before it; no word is a middle - name. - "Hassan Mohamad Ali" middle_as_family → family="Mohamad Ali" - "Hassan Mohamad Ali" → family="Ali" · boundary - implemented: nameparser/_pipeline/_post_rules.py - -O4. Rationale: what no vocabulary claims can only be read by where - it stands, under the order the caller declared. - Words no vocabulary has claimed read by position. In the default - given-first order the first name word is the given name, the - last is the family name, and everything between is middle names. - In a family-first order the first name word is the family; in - family-first-given-last the given name comes from the end, the - middles from between. - "Mary Beth Smith" → middle="Beth" - "Garcia Juan Carlos" family-first → family="Garcia" - "Nguyễn Thị Minh Khai" family-first-given-last → given="Khai" - no-boundary: this is the default reading every other rule carves - exceptions from; its boundaries are the other rules. - implemented: nameparser/_pipeline/_assign.py - -O5. Rationale: O4 reads a name by comparing where its words stand, - and a name of exactly one name word gives it nothing to compare - — the first name word is also the last, so the rule that decides - every longer name is silent here. Nothing in such an input says - whether the word is a given name or a family name: both readings - fit it equally well, and the library still has to report one. - A name of one name word that nothing else has decided reads that - word as the given name under the default given-first order, and - as the family name under a declared family-first one. That is a - convention rather than a determination — the same input has to - read the same way every time, so one of two equally consistent - readings is fixed in advance — and it is not evidence about the - word. Every rule that DOES decide such a name outranks it, and - each of them carves out a case it declines to decide — where the - convention is what is left. A title makes the word the family - name (H1), except a given-name title, which addresses by given - name; a nickname beside it does the same (N3), except where a - suffix stands beside them too, which its count does not set - aside; and a maiden name beside it does the same (M4), except - where the vocabulary or the word's own shape has claimed the - word already. The convention is reported: a name whose one name - word nothing else decided carries a `given-or-family` ambiguity - naming the field the convention chose. The report is exactly as - narrow as the convention, so every rule named above silences it - where it fires, a family comma — a comma with a name segment - after it — silences it, a script whose own convention settles - the order (W4) silences it, and so does the word's own claim — a - particle, a bound given name, an initial's shape. A comma with - nothing after it names no family and silences nothing, so - `سلمان،` and `Smith,` report where `Smith, Andrew` does not. A - comma with only a TITLE after it names no family either, but the - title still decides the field wherever it stands, so `John V, Dr.` - is silent for H1's reason and not the comma's. A word with no - letter or digit in it is no name word and - reports nothing (A2). Where the one name unit carries title - vocabulary in a word beside the one it places, or is itself title - vocabulary left standing by the title peel, the doubt reported is `title-or-name` - instead, which H4 states — so a title silences THIS kind, not - every report at the site. - "Smith" → given="Smith" - "Garcia" family-first → family="Garcia" - "Sir John" → given="John" - "'Smitty' Jones Jr." → given="Jones" - "abd née Jones" → given="abd" - "Mr. Johnson" → family="Johnson" · boundary - "'Smitty' Jones" → family="Jones" · boundary - "Smith née Jones" → family="Smith" · boundary - "Andrew" → ambiguities=("given-or-family",) - "Garcia" family-first → ambiguities=("given-or-family",) - "Juan & Garcia" → ambiguities=("given-or-family",) - "'Smitty' Jones Jr." → ambiguities=("given-or-family",) - "Smith Jr." → ambiguities=("given-or-family",) - "Dr. Smith" → ambiguities=() · boundary - "Sir John" → ambiguities=() - "Smith née Jones" → ambiguities=() - "abd née Jones" → ambiguities=() - history: decisions.md#O5 · interacts: O4, H1, N3, M4, H4, W4, P3, A2 · implemented: nameparser/_pipeline/_assign.py - -## Scripts & writing systems (W) - -Background: script-conditional behavior is permitted exactly where the writing system itself — not statistics about it — settles the convention; a language can never be inferred from Latin-script text, because transliteration destroys the signal. The facts this section builds on: Chinese and Japanese both write the family name first in native script, so the script settles the order without knowing the language. Hangul is written by exactly one language and Korean family names are a small closed census set. Han text does not identify its language — a Chinese surname list would divide Japanese 高橋一郎 as 高 + 橋一郎 — which is why Han division is opt-in and there is no Korean pack to opt into. Hiragana never transcribes a foreign name (transcriptions are katakana alone), so kanji-plus-kana is a Japanese name in Japanese order, while wholly-katakana may be a transcribed foreign name already in given-first order or a Japanese name's reading written family-first, and nothing in the script says which. Katakana has a halfwidth form (ﾀﾛｳ for タロウ), the only kana systems without kanji could store, and legacy data still carries it: it is katakana, read the same way. A voicing mark typed as its own character after a kana — the dakuten or handakuten after a kana with no precomposed voiced form, as in ア゙イ, or the spacing ゛ and ゜ — voices that kana and belongs to neither syllabary, so it never changes the script of the word it stands in. A mark standing anywhere else, first in a word or after a character that is not kana, voices nothing, and what the parser makes of such a word is not promised. Real Chinese text is unspaced (毛泽东); the spaced 毛 泽东 is an artifact. A fuller narrative lives in docs/usage.rst's East Asian section. One fact carries its own consequence: none of the three writing systems marks the family name with a comma — position in the written form is what identifies it, so a comma standing between the family name and the given name is a listing convention carried in from elsewhere rather than a form the script produces. That is why the rule reading one (W3) is tolerated rather than normative. CLDR's own locale data says the same where a contrary convention would have had to appear: across its ko, zh and ja personName patterns not one of the 126 pattern strings carries a comma of any width, the surname-first referring patterns separating surname from given by a single space, and the only comma in reach belongs to the locale-neutral root's sorting format — a list-ordering format, which ko and zh override comma-free and ja all but one inherited slot (decisions.md#cjk-comma-demotion carries the pull verbatim, with its URLs, its commit and its date). A full stop of any width — the ASCII period, the fullwidth ．, the ideographic 。 and its halfwidth ｡ — glued after a script-written word is punctuation and not part of the word: it is invisible to the script reading and to the vocabulary, and it stays in the text on the word it arrived with, because no East Asian script writes an initial or an abbreviation with a period (#322, #323; decisions.md#cjk-full-stops). A stop glued BEFORE the word is punctuation to the vocabulary lookup, which folds both edges away, so .씨 is still the honorific. The classification fold that feeds the two division sites reads the trailing edge only, so a word wearing a leading stop is given no script at all and never becomes a surname site: .김민준 stays one whole word, given, no script rule reaching it. The honorific peel (W2) is not gated by that fold — the tail alone is its license — so it reads such a token regardless of a leading stop, and its own trailing-edge fold is what decides there: .김민준씨 peels to .김민준 and 씨, and .김민준씨. peels to .김민준 and 씨. (tests/v2/pipeline/test_script_segment.py's test_the_peel_reads_the_trailing_stop_only). - -W1. Rationale: hangul is monoglot Korean and its surnames are a - closed census set, so an unspaced hangul name divides at a - certain point; Han carries no such certainty by default. - An undivided word in the family position of a name written in - an activated script divides after a recognized surname, the - longest recognized surname first; where the vocabulary - recognizes nothing, an optional segmenter may divide instead, - and with neither the word stays whole rather than divide in a - wrong place. Korean division is active by default; Han division - is opt-in. A segmenter — unlike the vocabulary, which ignores - spacing but not the interpunct (the Accepted below) — is - consulted only for a name whose written form is wholly - undivided, a spaced honorific counting as a written division - (decisions.md#W3 records what that trade keeps and costs). - "김민준" → family="김" - "남궁민수" → family="남궁" - "남궁민수 지훈" → family="남궁" - "毛泽东" → family="毛泽东" · boundary - "毛泽东" [zh] → family="毛" - "高橋一郎" [zh] → family="高" - Accepted: a name the interpunct divides is already divided in - the sense that matters — division stands down entirely there, - vocabulary and segmenter alike (#298; decisions.md#T3). - "安东尼·陈志明" [zh] → family="陈志明" - history: decisions.md#W1 · interacts: W3 · implemented: nameparser/_pipeline/_script_segment.py - -W2. Rationale: some East Asian honorifics glue directly onto the end - of the name (田中さん); a glued word peels off only if it could - never itself end a name, so the listed vocabulary carries its - own license and needs no other gate. - A listed honorific glued to the end of the name's last name - word splits off once and reads as a suffix. A part that is not - name text — a post-nominal word standing on its own — is never - the name's end: the split-off steps past it to the name word - behind, and never dissects it. - "田中さん" → suffix="さん" - "김민준씨" → suffix="씨" - "김민준 박사님" → suffix="박사님" - "马丁·路德·金씨" → suffix="씨" - "김지양" → suffix="" · boundary - "선생님" → family="선생님" · boundary - "王君" → family="王君" · boundary - "田中 社長" → suffix="社長" - "田中社長" → suffix="社長" - "陳怡君二世" → suffix="二世" - "田中部長" → family="田中部長" · boundary - Accepted: 部長 and its simplified spelling 部长 are vocabulary in - the spaced and after-comma positions only, and no peel splits - them off a glued name: 田中部長 keeps the word in the name, - because 長 alone is a borne given name and 阿部長 can be 阿部 + - 長. The other credential, office and generational words of #671 - (議員, 国会議員, 太平紳士, 医学博士, 二世, 社長, 課長 and their - variant spellings) met the glued bar when measured against real - names (#707) and peel. The Han segmenter's own division is not a - peel and exposes 部長 regardless: under the Chinese pack a - listed surname followed by it divides there and the word reads - as a suffix, so 李部長 reads family 李, suffix 部長, as 王博士 - already does. - Accepted: a longer office or degree word that ends in a listed - tail is split at it, whether glued or spaced, because the tail - is matched at the end of the token and the longer word is not - listed: 衆議院議員, 副社長, 代表取締役社長 and 獣医学博士 leave - their head (衆議院, 副, 代表取締役, 獣) in the name. Before the - tail was listed the whole word stayed in the name, which was no - more right (#707). - "田中 副社長" → given="副" - "岸田文雄衆議院議員" → family="岸田文雄衆議院" - history: decisions.md#W2 · interacts: W3 · implemented: nameparser/_pipeline/_script_segment.py - -W3. Rationale: a family name declared by a comma is the writer's - own division, and re-dividing it would invent a boundary nobody - drew — but none of the East Asian writing systems declares a - family name that way (the Background above), so every input this - rule reads is a listing convention wrapped around a name whose - own script has already arranged it — the comma that declares the - family name, and the punctuation such a listing carries in with - it. What follows describes what the parser does with such input; - it does not promise it. - Under a family comma the pre-comma text is the family by - declaration and never divides, and the post-comma side is given - text with no family to find. A title run opening the pre-comma - text is H6's title, not family text. Only the honorific - split-off (W2) - crosses the comma, an honorific being no part of the name on - either side — the crossing is stated here rather than in W2 - because it is a claim about the comma, and the comma is the part - nobody's writing system produces. Behind a part C1 reads as - postnominal there is no family comma, and the name before it - divides as it would with no comma ('마틴 킹, MA' as '마틴 킹'); - the division, and with it the split-off below, follows C1's own - reading of the comma, taken on the name as written before the - division ('Dr 김민준, MA' divides as 'Dr 김민준' does). Which side - the split-off takes - it from is decided by the SHAPE of the part after the comma, not - by which of C1's two readings that part selects — a one-word - family reads the listing form either way. A part that is nothing - but suffix words is not the name's end: it is declined as the - site, and the split-off falls back to the part before the comma - and takes the honorific there, provided that part offers a site - of its own. Any other part — a title, a name word — IS the - name's end, and a glued honorific before the comma stays glued. - The vocabulary question is C1's own, asked without C1's - word-count condition: the two differ in what else they require, - not in what they ask of the words — the written-case evidence - S2 reads included, so a post-comma acronym the case leans - credential declines the side it stands on here exactly as it - reads as the credential there, and the honorific comes off the - part before the comma as it does behind a settled credential. - A period the listing leaves - behind is not what licenses the step past a post-nominal word, - and it is not ignored either. The step is W2's, taken on the - vocabulary alone and taken with no punctuation anywhere in the - input; a period on a SEPARATE post-nominal word rides along into - the suffix and moves no division (田中さん 様. divides where - 田中さん 様 does). A period glued to the honorific's OWN word - rides with the honorific: the split-off matches the listed tail - through it and cuts before it, so 田中さん. and 김민준씨. divide - where 田中さん and 김민준씨 do (#323; until 2026-09-10 each read - as a title, measured 2026-09-05 and pinned by nothing, which is - why the move was free), and a period-marked Han opener divides - as its stop-less spelling does, 田中. 太郎 reading family-first - — the reading H2's Accepted clause vetoes a title into, carried - here because an edge full stop makes it tolerated input like the - rest of this block. All three are case rows now, tolerated, and - the sentence reports them rather than promising them, even by - this rule's standard. - decisions.md#cjk-comma-demotion carries the parses. - "남궁민수" → family="남궁" - "지훈, 남궁민수" → given="남궁민수" - "남궁민수, 지훈" → family="남궁민수" · boundary - "田中さん, Dr." → family="田中さん" - "田中さん, PhD" → suffix="さん, PhD" - "Kim김민준씨, MA" → suffix="씨, MA" - "田中さん 様." → suffix="さん 様." - "김민준씨." → suffix="씨." - "田中. 太郎" → family="田中." - tolerated: native CJK writing has neither a family-comma convention nor an edge full stop of any width on a name word, so the five comma lines above and the three period lines under them illustrate current behavior — changeable without notice — rather than promise it; the line carrying neither, beside them, is W1's claim, which is normative. All eight stay watched by the snapshot (tools/differential/snapshot.jsonl, over corpus_rules.jsonl and corpus_cjk.jsonl), and the `tolerated` rows of tests/v2/cases.py pin them at HEAD. - history: decisions.md#W3 · interacts: W1, W2, C1, H2, H6 · implemented: nameparser/_pipeline/_script_segment.py - -W4. Rationale: Chinese, Japanese and Korean all write the family - name first in native script — the script settles the order - without knowing the language — while a wholly-katakana name may - be a transcribed foreign name already in its source order or a - Japanese reading written family-first, so its script settles - nothing. - A name written wholly in one East Asian script, or in the - kana-licensed Japanese repertoire, reads family-first whatever - order the caller declared; a wholly-katakana name keeps the - declared order. - "김 민준" → family="김" - "山田 太郎" → family="山田" - "高橋 みなみ" → family="高橋" - "山田 ﾀﾛｳ" → family="山田" - "マイケル ジャクソン" → given="マイケル" · boundary - "ﾔﾏﾀﾞ ﾀﾛｳ" → given="ﾔﾏﾀﾞ" · boundary - "ア゙イ タロウ" → given="ア\u3099イ" · boundary - Accepted: a name the interpunct divides keeps the declared order - — the divider itself marks a transcription (T3) — so the override - stands down there, and the name reads as a wholly-katakana one - does: in its written order by default, reversed by a declared - family-first order like any other name; the katakana middle dot - (T2) carries no such signal, so a name it divides still reads by - the script license. - "毛·泽东" → given="毛" - "威廉·莎士比亚" family-first → family="威廉" - "威廉・莎士比亚" → family="威廉" - history: decisions.md#W4 · interacts: T3 · implemented: nameparser/_pipeline/_assign.py - -## Tokens, initials & punctuation (T) - -Background: every parsed name part is an exact piece of the input, located by its position — nothing rewrites the text before parsing. Some punctuation is a name divider only by convention of a particular writing system: Japanese writes name parts with a middle dot between them (マイケル・ジャクソン, 姓・名), while U+00B7 is both the Chinese divider for transcribed foreign names and the Catalan punt volat interior to legitimate words (Gal·la). - -T1. Rationale: a character that carries no name content (emoji, an - invisible directionality control) stands between words, not - inside them. - A name splits at whitespace and — unless the caller opts to - keep them — at ignorable characters: an ignorable character - separates its neighbors and never joins them. - "John😀Smith" → family="Smith" - "John😀Smith" keep-emoji → given="John😀Smith" · boundary - history: decisions.md#T1 · implemented: nameparser/_pipeline/_tokenize.py - -T2. Rationale: the katakana middle dot exists to divide name parts - and appears in no native name. - The katakana middle dot and its halfwidth twin divide a name - like whitespace, always. - "マイケル・ジャクソン" → given="マイケル" - "高橋・一郎" → family="高橋" - no-boundary: the separation is unconditional; the - context-sensitive interpunct is T3's subject. - implemented: nameparser/_pipeline/_tokenize.py - -T3. Rationale: U+00B7 is two marks in one codepoint — the Chinese - 间隔号 dividing a transcribed foreign name, and the Catalan punt - volat interior to words. - The interpunct divides a name only between two characters of a - classified East Asian script; anywhere else it is part of the - word. - "威廉·莎士比亚" → family="莎士比亚" - "Gal·la Serra" → given="Gal·la" · boundary - history: decisions.md#T3 · implemented: nameparser/_pipeline/_tokenize.py - -## Ambiguity & tie-breaking (A) - -Background: some name strings are genuinely ambiguous — the same written shape carries two readings ("Van Johnson": given name or particle?), or the text's structure is malformed. Parsing never fails and never silently discards; it completes on the best reading and says what it was unsure of. - -A1. Rationale: a caller can only act on doubt that is reported. - Parsing never fails on any input: where the text's structure or - a word's reading is genuinely uncertain, the parse completes on - the best reading and carries an ambiguity report naming the - doubt. A report names the reading the parse took, so a report - whose fork the rest of the parse then resolved to NEITHER branch - is withdrawn rather than carried beside a reading it - contradicts: a letter both a connective and an initial, read as - the generation it also spells, is neither of the two the - connective-or-initial fork offered, and only that fork's report - goes — the generation's own stands. And a report that names the - field its word was read into names the field the word holds in - the result, whatever rule moved it after the report was made. - "Van Johnson" → ambiguities=("particle-or-given",) - "JOHN QUINCY SMITH I" → ambiguities=("suffix-or-name",) - "Jane „JD Smith" → ambiguities=("unbalanced-delimiter",) - "John Smith, MD, Bart" → ambiguities=("comma-structure",) - "John Smith" → ambiguities=() · boundary - Accepted: the one exception to totality is a user-supplied - segmenter's own error, which propagates — a user-code error is - not a content error. (Needs the optional extra to demonstrate, - so no example line.) - history: decisions.md#A1 · interacts: H1, O1, O2, O3, P1, P3, P6, S2 · implemented: nameparser/_pipeline/_assemble.py, nameparser/_pipeline/_state.py - -A2. Rationale: an input with no name content names nobody, and - saying so beats inventing fields from punctuation. - A name with no name content parses to the empty name — every - field empty, false as a boolean — while ambiguities born from - its punctuation survive on the empty result. - ".," → family="" - "(" → ambiguities=("unbalanced-delimiter",) - "John . Smith" → family="Smith" · boundary - history: decisions.md#A2 · implemented: nameparser/_pipeline/_assemble.py - -## Rendering & views (R) - -Background: parsing produces words with roles; every string a caller reads is assembled from those words on request. Nothing about rendering changes the parse, and nothing about reading a field mutates anything. The default view renders '{title} {given} "{nickname}" {middle} {family} ({maiden}) {suffix}' — the choice and its declined née-template alternative are decisions.md#render-default. - -R1. Rationale: a field is a way of reading the parse, not a stored - string. - Every field is a view computed from the parsed words at read - time, joining its words in written order — except folded family - words — O3's fold and, since #379, P6's attached tussenvoegsel — - which render before the rest of the family wherever they stood in - the string. - Words are separated as the writer separated them. The suffix view - is the one place this is visible, because it is the only field - that can hold parts the writer comma-separated: a run of - post-nominals written with spaces renders with spaces, and one - written with commas keeps them. The separator is the comma the - writer typed, read off the words' positions once their roles are - known; a delimiter the configuration names is a separator too, - and so is any name word standing between two post-nominals. - "Dr. Juan Q. Xavier de la Vega III" → family="de la Vega" - "Hassan, Mohamad Ahmad Ali" middle_as_family → family="Ahmad Ali Hassan" - "Hassan, Mohamad Ahmad Ali" → family="Hassan" · boundary - "Smith, MD PhD" → suffix="MD PhD" - "John Smith MD PhD" → suffix="MD PhD" - "John Smith, MD, Bart" → suffix="MD, Bart" - "Smith, John, PhD - i Soler" extra_suffix_delimiters-dash → suffix="PhD, i Soler" - Accepted: a suffix value handed to revise() derives its entries the - same way, from the value's own commas, so a name's rendered suffix - revises back to itself wherever the value's words read as the - whole name read them — a glued CJK honorific peels off an initial - in a bare value where the whole name kept it glued, one corpus - name of the 372 with a suffix (decisions.md#C1, 2026-09-06, which - counted 368 of 1117 then; re-measured 2026-09-10 with this - bundle's rows in the corpora, 372 of 1156 distinct names and the - same single failure). A delimiter the configuration names parts a - value only where the value's own words, read as a name, give it a - tail segment for the core to be dropped on; a run of post-nominals - has none, with or without a comma of its own, so there the - delimiter stays a word of the run — write a comma at the boundary - instead. Stated without an example line because every line here - names an input string, and this shape needs a field revised after - the parse. - history: decisions.md#C1 · interacts: O3, P3, P6, R3, M2 · implemented: nameparser/_parser.py, nameparser/_pipeline/_post_rules.py, nameparser/_types.py - -R2. Rationale: callers need the surname with and without its - particles — sorting wants "Vega", display wants "de la Vega". - The family name splits into further views: the base (the family - without its leading particles) and the particles themselves. A - name part whose every word is particle vocabulary is a part where - none of them is doing a particle's work — nothing joins them to a - name — so they read as ordinary name words: they anchor the base - and leave the particles view. "Every word", not "standing alone": - a two-particle run has neither word alone and both are name words - there. Position decides that, not vocabulary; whether the word is - borne as a surname somewhere does not enter into it. - "Dr. Juan Q. Xavier de la Vega III" → family_base="Vega" - "Dr. Juan Q. Xavier de la Vega III" → family_particles="de la" - "Anh Do" → family_base="Do" - "Juan van der" → family_base="van der" - "Juan van der" → family_particles="" - "Juan de la Vega" → family_base="Vega" · boundary - "Juan de la Vega" → family_particles="de la" - "Sean O'Connor" → family_base="O'Connor" - Accepted, and the invariant it exists to hold: a non-empty - family always has a non-empty base. A particle needs a base to - attach to, so a family that is all particles is a family whose - words are not acting as particles. - "Del Toro" family-first → family_base="Del" - Accepted: the test runs after every rule that moves a token - between parts, so O3's fold decides it too — a middle folded into - the family can leave the family all particles, or can give a - trailing particle the name word it was missing. - "Anh Van Do" middle_as_family → family_base="Van Do" - "Nguyen, Van Le" middle_as_family → family_particles="Le" - history: decisions.md#R2 · interacts: R3 · implemented: nameparser/_types.py, nameparser/_pipeline/_post_rules.py, nameparser/_facade.py - -R3. Rationale: initials abbreviate the person's name words; titles, - suffixes, particles and nicknames are not name words. - Initials take the first letter of each given, middle, and base - family word; titles, suffixes, particles and nicknames - contribute nothing — except the particles of a part whose every - word is one, which are not acting as particles there (R2) and - initial like any other name word. A connective contributes - nothing where it is joining: a part holding another name word for - it to join — the part's working particles set aside — is a part - where the connective is doing a connective's work and is no name - word of its own. A part holding nothing else is a part where it - is joining nothing, and there it initials like any other name - word, agreeing with the base. - The first letter is the one the word's composed spelling begins - with, written in the form the writer used: a letter written as a - base letter followed by its combining accent initials whole, and - so does a syllable written as its separate parts. - The question is asked of the WHOLE PART, never of a word count. A - part of two words where one is the connective is still a part - where it joins — the base of "Jon Dough and" is "Dough and" and - its initials are "J. D." — and a part of three is no different. - One rule for every group. A connective written among given names - is a connective there too and contributes nothing where it joins, - so the group that was pinned-but-undocumented is now specified by - the same sentence as the other two. - Which words a group contributes is one question; the ORDER they - contribute in is a second, and its answer is the field's. Each - group initials in the order its field reads — written order, - except folded family words, which initial before the rest of the - family exactly as they render before it (R1). Stated here rather - than left to R1, whose subject is every FIELD and which this view - is not: what the view takes from the field is the ORDER it reads, - and about order the two never disagree. Membership is the first - question above and they can differ there — the family field of - the first example line below is de la Vega, where the initials - take its base — so this clause settles order and nothing else. - "Dr. Juan Q. Xavier de la Vega III" → initials="J. Q. X. V." - "Anh Do" → initials="A. D." - "Nguyen, Van Le" → initials="V. L. N." - "Hassan, Mohamad Ahmad Ali" middle_as_family → initials="M. A. A. H." - "Sean O'Connor" → initials="S. O." · boundary - "émile zola" → initials="e\u0301. z." - "マイケル ジャクソン" → initials="マ. シ\u3099." - A family that is ALL particles therefore contributes its words - rather than nothing: they are the base (R2), so they initial. - "Juan van der" → initials="J. v. d." - "Juan de y" → initials="J. y." - "John and Jane Smith" → initials="J. J. S." - "Duke of Edinburgh" → initials="D. E." - "Juan Velasquez y Garcia" → initials="J. V. G." - "Jon Dough and" → initials="J. D." - Accepted: this rule reads a part the parser read. A field set as - raw text after the parse carries no reading, and this view is - handed no vocabulary to supply one — it takes a format spec and - two separators and nothing else — so every word of such a field - initials, particles and conjunctions alike: a family set that way - to "de la vega" gives "j. d. l. v." where parsing the same - name gives "j. v.". Case repair IS handed a vocabulary, so it falls - back for the one question a word can answer on its own, and R4 - says which. Revising the field through the parser classifies it - and matches the parse in both of the parsed name's views. Stated - without an example line because every line here names an input - string, and this shape needs a field edited after the parse. - The v1 facade's HumanName.initials() is a second view of this - question and IS handed a vocabulary: it reads the parse's reading - wherever a word is backed by a parsed token, and falls back - wherever a word is not — a field set as raw text, or a name - restored from a v1 pickle or copied through the same state hooks. - The connective question falls back to the same helper case repair - uses; the particle question was never asked of the parse in this - view at all — `_is_particle` is a live - vocabulary lookup for every word, backed or spliced alike — so a - family spliced to "de la vega" initials "j. v." on this view - against "j. d. l. v." on the parsed name's own. So the - two views agree on WHICH WORDS initial in a parsed name; what - still differs there is GROUPING, the facade initialing a joined - run as one element, which is where the name "Ph. D., John" gives - a run-together "J. P D." on this view against "J. P. D." on the - other. Stated in prose and not as example lines because both - shapes need a field edited after the parse, or a rendering the - other view does not have. decisions.md#R3 carries what all of it - costs and where it is pinned. - Accepted: the given group's answer is a 1.4.0 parity break, and - a second one in the other direction sits beside it. A connective - among given names contributed an initial from 1.4.0 until this - rule reached it, so "John and Jane Smith" gives "J. J. S." where - every release through 2.3 gave "J. a. J. S." and 1.4.0 gave the - run-together "J a J. S."; and a connective holding a part alone - now contributes one, so "محمد و علي" gives "م. و. ع." on both - views, which is 1.4.0's own answer restored. The rule is one - sentence for all three groups, and taking it is what made the - four candidate answers this group once had into one. - history: decisions.md#R3 · interacts: O3, P3, P6, R1, R2, R4 · implemented: nameparser/_render.py, nameparser/_facade.py, nameparser/_pipeline/_post_rules.py - -R4. Rationale: case repair is a display concern, applied only on - request and never destructively. - Case repair returns a repaired copy and never mutates the parse. - Where it acts at all — R5 decides where — the copy honors the - casing a vocabulary entry records (PhD, BSc) and the Mac/Mc - convention (McDonald), not only ordinary word-by-word casing; a - particle is written in lowercase, but a particle the vocabulary - records a casing for takes that casing, the entry being how the - word is written wherever it stands; and - a part whose every word is particle vocabulary is repaired as - ordinary name words, since none of them is doing a particle's - work there (R2). A CONNECTIVE the parse placed among the name - words keeps its lowercase wherever it stands there, including - inside a part whose other words the unjoined mark has turned into - ordinary name words; one the parse read as the generation it also - spells is not a connective of this name at all, and is repaired - as the generation it was read as. That is this rule's own reading - and not a borrowing from R3: a connective that initials because - it joins nothing is still not written the way a name is written, - while a generation is written the way a generation is written. - The GENERATION is what that clause turns on and not the field: - a connective the suffix field merely holds, which the suffix - vocabulary does not know, was read as no generation and keeps its - lowercase there like any other connective — the third part of a - comma form is the shape that puts one there. - Repair changes case and nothing else: the repaired word is the - word as written, recased. A word is read whole however its letters - are encoded: a letter written as a base letter followed by its - combining accent is repaired as the same letter written whole, - and the repaired word keeps the encoding the writer used. A - vocabulary entry records casing as a - mask — its word's letters, each in the case it takes — and repair - lays it over the word as the writer punctuated it, so the one - entry for phd repairs phd to PhD and ph.d. to Ph.D.; the mask - applies wherever the word stands, whatever the parse read it as. - A single letter split off alone beside a full stop is written as - an initial and takes capitals whatever the mask says there — but - only where the mask itself writes that letter inside a run of two - or more letters: the writer split a chunk the mask keeps - together, so the split-off letter is an initial. A run is of - LETTERS only, so a digit ends one exactly as a full stop does. A - run of two or more letters beside a full stop takes the mask's - case letter by letter and is never an initial. Where the mask - writes the letter as a run of one too, there is no split to - repair, and the mask's own case stands. So ph.d. is Ph.D. under - the phd mask, and p.h.d. — three initials, not the mask's one run - 'PhD' — is P.H.D.; but h.c. under a caller's own h.c mask, which - spells each letter alone, stays h.c. - A name already written the way repair would - write it comes back unchanged, measured by repair's own - conventions rather than by the bearer's. A spelling written in a - single case is repaired even where its bearer meant it, because - nothing in the text marks it as a choice; where the text does - mark one, R5 defers to it. A credential acronym the exceptions map - does not carry is an initialism, so a single-case word the parse - put in the suffix role from the acronym vocabulary, or read as a - credential by its dotted shape alone (S3) or by its capitals - (S2), repairs to its all-caps spelling rather than a title-cased - one, and that repair - outranks the Mac/Mc convention where a word fits both (MCSE, not - McSe). A roman numeral the parse put in the suffix role is - written in capitals the way a generation is written, whether or - not the vocabulary lists it (VI as well as III). Those two - classes read the role and the mask does not, so a word of either - class that parsed as an ordinary name word repairs as that name - word unless the exceptions map carries it (qc mp gives Qc MP, - phd smith gives PhD Smith), and a suffix word that is neither an - acronym, a numeral nor an exceptions-map entry -- the - generational `jr`, `sr` -- keeps its title case. - Inside a hyphenated word, a part that is connective vocabulary - with a worded part on each side of it keeps its lowercase, the - hyphens being the writer joining the name around it - (Ortega-y-Gasset); at either end of the hyphenated word it is - repaired as ordinary name text. A single letter marked with a - period is read as an initial there, as the parse reads it, never - as the connective. A bare letter is not: the hyphen is the - writer's join, so the interior connective is read as one even in - a name written wholly in one case, where the same letter spaced - reads as an initial if the vocabulary marks it as reading both - ways (P3). - "juan mcdonald" → capitalized="Juan McDonald" - "Juan McDonald" → capitalized_forced="Juan McDonald" - "ANH DO" → capitalized="Anh Do" - "anh van do" → capitalized="Anh Van Do" - "SEÁN Ó MURCHÚ" → capitalized="Seán Ó Murchú" - "IVOI, PAUL D'" → capitalized="Paul d' Ivoi" - "zaʿīm, ḥusnī az-" → capitalized="Ḥusnī az- Zaʿīm" - "jacobus henricus van 't hoff" → capitalized="Jacobus Henricus van 't Hoff" - "ina binti navalamar" → capitalized="Ina binti Navalamar" · boundary - "john smith phd" → capitalized="John Smith PhD" - "john smith ph.d." → capitalized="John Smith Ph.D." - "JOHN SMITH PH.D." → capitalized="John Smith Ph.D." - "john smith p.h.d." → capitalized="John Smith P.H.D." - "john smith ph. d." → capitalized="John Smith Ph. D." - "john smith bsc" → capitalized="John Smith BSc" - "john smith psyd" → capitalized="John Smith PsyD" - "Dr. med. univ. Margit Popp, MSc" → capitalized_forced="Dr. Med. Univ. Margit Popp MSc" - "MSc Dr. med. univ." → capitalized_forced="MSc Dr. Med. Univ." - "Md Abdul Karim" → capitalized_forced="Md Abdul Karim" - "john smith md" → capitalized="John Smith MD" - "john smith m.d." → capitalized="John Smith M.D." - "john smith iii." → capitalized="John Smith III." - "john smith x.y.z." → capitalized="John Smith X.Y.Z." - "John Smith R.A.I." → capitalized_forced="John Smith R.A.I." - "john doe q.c." → capitalized="John Doe Q.C." - "Jack X.Y.Z." → capitalized_forced="Jack X.y.z." · boundary - "qc mp" → capitalized="Qc MP" · boundary - "john smith ed" → capitalized="John Smith ED" - "john smith vi" → capitalized="John Smith VI" - "john smith ix" → capitalized="John Smith IX" - "john smith iii" → capitalized="John Smith III" - "john smith, v" → capitalized="John Smith V" - "john smith xi" → capitalized="John Smith Xi" · boundary - "john smith, vi" → capitalized="Vi John Smith" · boundary - "john smith mba" → capitalized="John Smith MBA" - "john smith mcse" → capitalized="John Smith MCSE" - "john smith jr" → capitalized="John Smith Jr" · boundary - "John Quincy Smith i" → capitalized_forced="John Quincy Smith I" - "Carod i" → capitalized_forced="Carod I" - "Smith, John, and" → capitalized_forced="John Smith and" - "Doe, Jane, and Jr." → capitalized_forced="Jane Doe and Jr." - "juan de la vega" → capitalized="Juan de la Vega" · boundary - "josé garcía" → capitalized="Jose\u0301 Garci\u0301a" - "JOSÉ GARCÍA" → capitalized="Jose\u0301 Garci\u0301a" - "jose ortega-y-gasset" → capitalized="Jose Ortega-y-Gasset" - "JOSE ORTEGA-Y-GASSET" → capitalized="Jose Ortega-y-Gasset" - "maria silva-e-sousa" → capitalized="Maria Silva-e-Sousa" - "Jose Ortega-Y-Gasset" → capitalized_forced="Jose Ortega-y-Gasset" - "juan e-f smith" → capitalized="Juan E-F Smith" · boundary - "juan y-garcia" → capitalized="Juan Y-Garcia" · boundary - "donovan mcnabb-smith" → capitalized="Donovan McNabb-Smith" - "maria da-silva" → capitalized="Maria da-Silva" - "juan y-garcia-lopez" → capitalized="Juan Y-Garcia-Lopez" · boundary - "j.-e.-p. dupont" → capitalized_forced="J.-E.-P. Dupont" · boundary - "J-E-P DUPONT" → capitalized="J-e-P Dupont" · boundary - Accepted: a name written in one case whose hyphenated bare - initials spell a connective pays for the hyphen clause — its - interior letter is lowered as the connective though the writer - meant an initial, where the same letters spaced keep their - capitals. The hyphen is the writer's join, and reading the one - shape both ways would need the case evidence a one-case name does - not carry. - Accepted: the all-particle clause reaches a part the parser read. - A field spliced in as raw text after the parse carries no reading - of its own, so a family set that way to "de la" stays lowercase - where those same two words parsed from a name are repaired to "De - La". The acronym repair is the contrast: it asks the role and the - vocabulary and not a reading, so a suffix spliced in as "mba" is - repaired to "MBA" exactly as a parsed one is. - That is the boundary between splicing text into a field and - revising a field through the parser — revise() classifies the - value, so the repair follows it — rather than a gap between them. - It is also a limit and not a regression: a spliced-in family was - repaired exactly this way before the clause existed. Stated - without an example line because every line here names an input - string, and this shape needs a field edited after the parse. - Accepted, and the reason the boundary is drawn per question - rather than per field: the conjunction carve-out reaches a word - the parse read as a conjunction, and where the parse read nothing - at all it reaches what the vocabulary says. A word of that - vocabulary standing at either end of a longer written word is - not a conjunction there, because the hyphen clause above reaches - only a worded part with a worded part on each side, and an edge - part is outside that reach — but a field spliced in as raw - text was read by nobody, so repair asks the vocabulary and a - family set to "de y" - keeps its "y" lowercase. Whether a word is the conjunction or an - initial is a property of the word, which a vocabulary can answer; - whether a particle is acting as a particle is a property of the - whole part, which the parse settles and records; re-deriving it - needs a reading on every word of the part, and a spliced field - has none on any, so that half falls through to particle treatment - and the "de la" boundary above stands. Initials are the contrast - worth knowing, and R3 states it — but ask which initials view, - because the two answer oppositely. The parsed name's own view is - handed no vocabulary at all, so it falls back on neither question - and a spliced field's every word initials. The v1 facade's view IS - handed one: the connective question falls back to the helper this - rule uses, while the particle question is never asked of the - parse in this view at all — a live vocabulary lookup for every - word, backed or spliced alike — so a family spliced to "de la - vega" initials "j. v." there against the parsed view's "j. d. l. - v.". revise() classifies the value and crosses both questions, in - both of the parsed name's views: a middle revised to "e-f" - repairs to "E-F" as the parsed name does, where splicing the same - text in gives "e-F". - history: decisions.md#R4 · interacts: P3, R2, R3, R5, S3 · implemented: nameparser/_render.py, nameparser/_lexicon.py - -R5. Rationale: mixed case is evidence that the writer cased the name - deliberately, and a repair cannot tell a deliberate spelling from - a mistaken one. A name written wholly in one case leaves the - repair no such evidence to read either way — the writer who - always writes lowercase is indistinguishable from the writer who - could not be bothered — so repair proceeds there, which is a - choice about how to act absent evidence rather than a claim that - nothing can be lost by it. - Case repair acts only on a name written entirely in one case. - The suffixes are left out of that test: a suffix says nothing - about how the writer cased the name — III, PhD, Jr. are written - that way whatever the name's case — so the test reads every word - the parse did not put in the suffix role. A title is not left - out, and a cased title still holds repair back. A name whose - words outside the suffix are written in more than one case is - kept as it was written, and whether that casing is right does - not enter into it, unless repair was asked for anyway. The same - test is then asked of each suffix on its own: a suffix written - in more than one case is the writer's spelling and is kept as - written where repair was not forced; one written in a single - case is repaired like any other. - "juan mcdonald" → capitalized="Juan McDonald" - "SHIRLEY MACLAINE" → capitalized="Shirley MacLaine" - "Shirley Maclaine" → capitalized="Shirley Maclaine" - "Shirley Maclaine" → capitalized_forced="Shirley MacLaine" · boundary - "juan garcia III" → capitalized="Juan Garcia III" - "juan garcia PhD" → capitalized="Juan Garcia PhD" - "JUAN GARCIA Jr." → capitalized="Juan Garcia Jr." - "dr. juan garcia III" → capitalized="Dr. Juan Garcia III" - "Dr. juan garcia" → capitalized="Dr. juan garcia" · boundary - "Juan garcia III" → capitalized="Juan garcia III" · boundary - "john smith EdD" → capitalized="John Smith EdD" - "juan garcia Iii" → capitalized="Juan Garcia Iii" · boundary - "john smith EdD" → capitalized_forced="John Smith EDD" - history: decisions.md#R5 · interacts: R4, P3, S2 · implemented: nameparser/_render.py - -## Construction & configuration diagnostics (D) - -Background: configuration mistakes are reported when they are made — at construction — not when a name happens to hit them; and a diagnostic that hands the reader code must hand code that works and type-checks. - -D1. Rationale: a parser whose activated scripts nothing can divide - behaves like a working parser minus a feature, silently — the - one misconfiguration a caller cannot see in output. - Constructing a parser that activates division for scripts with - no covering surnames and no segmenter warns at construction, - naming the dead scripts and each way out. - [segmenterless-ja] → warns="deactivate with Policy(segment_scripts=frozenset())" - no-boundary: any covering surname vocabulary, configured - segmenter, or deactivation silences it — the default parser and - the zh pack never warn, which every other example in this - document exercises. - history: decisions.md#D1 · implemented: nameparser/_parser.py - -D2. Rationale: whatever a name contains, parsing answers; only a - broken configuration may raise, and it must name the field. - Configuration validation raises at construction with the - offending field and value named; applying a locale pack wraps - any such error with the locale's code, so a stacked - configuration names which layer broke. - [bad-name-order] → raises="name_order elements must be Role members" - [bad-order-none] → raises="name_order must be an iterable" - no-boundary: the non-raising side is every other rule's - examples; parse() itself is total (A1). - history: decisions.md#D2 · implemented: nameparser/_policy.py, nameparser/_parser.py diff --git a/docs/design/rules/A-ambiguity-and-tie-breaking.md b/docs/design/rules/A-ambiguity-and-tie-breaking.md new file mode 100644 index 00000000..2422f24c --- /dev/null +++ b/docs/design/rules/A-ambiguity-and-tie-breaking.md @@ -0,0 +1,4 @@ +## Ambiguity & tie-breaking (A) + +Background: some name strings are genuinely ambiguous — the same written shape carries two readings ("Van Johnson": given name or particle?), or the text's structure is malformed. Parsing never fails and never silently discards; it completes on the best reading and says what it was unsure of. + diff --git a/docs/design/rules/A1-ambiguity-report.md b/docs/design/rules/A1-ambiguity-report.md new file mode 100644 index 00000000..b4dd6822 --- /dev/null +++ b/docs/design/rules/A1-ambiguity-report.md @@ -0,0 +1,26 @@ +# A1 — the ambiguity report + +A1. Rationale: a caller can only act on doubt that is reported. + Parsing never fails on any input: where the text's structure or + a word's reading is genuinely uncertain, the parse completes on + the best reading and carries an ambiguity report naming the + doubt. A report names the reading the parse took, so a report + whose fork the rest of the parse then resolved to NEITHER branch + is withdrawn rather than carried beside a reading it + contradicts: a letter both a connective and an initial, read as + the generation it also spells, is neither of the two the + connective-or-initial fork offered, and only that fork's report + goes — the generation's own stands. And a report that names the + field its word was read into names the field the word holds in + the result, whatever rule moved it after the report was made. + "Van Johnson" → ambiguities=("particle-or-given",) + "JOHN QUINCY SMITH I" → ambiguities=("suffix-or-name",) + "Jane „JD Smith" → ambiguities=("unbalanced-delimiter",) + "John Smith, MD, Bart" → ambiguities=("comma-structure",) + "John Smith" → ambiguities=() · boundary + Accepted: the one exception to totality is a user-supplied + segmenter's own error, which propagates — a user-code error is + not a content error. (Needs the optional extra to demonstrate, + so no example line.) + history: decisions/A1 · interacts: H1, O1, O2, O3, P1, P3, P6, S2 · implemented: nameparser/_pipeline/_assemble.py, nameparser/_pipeline/_state.py + diff --git a/docs/design/rules/A2-empty-name.md b/docs/design/rules/A2-empty-name.md new file mode 100644 index 00000000..04cf360a --- /dev/null +++ b/docs/design/rules/A2-empty-name.md @@ -0,0 +1,12 @@ +# A2 — the empty name + +A2. Rationale: an input with no name content names nobody, and + saying so beats inventing fields from punctuation. + A name with no name content parses to the empty name — every + field empty, false as a boolean — while ambiguities born from + its punctuation survive on the empty result. + ".," → family="" + "(" → ambiguities=("unbalanced-delimiter",) + "John . Smith" → family="Smith" · boundary + history: decisions/A2 · implemented: nameparser/_pipeline/_assemble.py + diff --git a/docs/design/rules/C-commas-and-structure.md b/docs/design/rules/C-commas-and-structure.md new file mode 100644 index 00000000..51173524 --- /dev/null +++ b/docs/design/rules/C-commas-and-structure.md @@ -0,0 +1,4 @@ +## Commas & structure (C) + +Background: a comma in a name signals one of two conventions — the listing form "Family, Given" or trailing credentials "Name, PhD" — and which is meant can only be judged from what stands after the first comma. Recognizing a credential run is by nature a vocabulary judgment, so this is the one structural decision that consults the suffix word lists. Which characters COUNT as the comma is part of the rule: the Arabic comma (U+060C) and the fullwidth comma (U+FF0C) both signal the listing form, while the ideographic comma (U+3001) is not a name-structure comma at all (#265). + diff --git a/docs/design/rules/C1-suffix-comma-decision.md b/docs/design/rules/C1-suffix-comma-decision.md new file mode 100644 index 00000000..b905c8d5 --- /dev/null +++ b/docs/design/rules/C1-suffix-comma-decision.md @@ -0,0 +1,352 @@ +# C1 — the suffix-comma decision + +C1. Rationale: a credential run after the comma means the name is in + natural order with suffixes appended; anything else after the + comma means the listing form. + With a comma present, the name reads as trailing suffixes when + the part after the first comma is entirely suffix words and more + than one word precedes the comma; otherwise it reads as the + listing form, the part before the comma being the family name, + less a title run opening it (H6), unless the part after it fixes + no family boundary (below). Only + the part after the first comma decides. + Wherever this rule counts the words before the comma, a particle + run and the one name word it attaches to are one word, the reach + P1's fold counts rather than the whole of P2's chain: the listing + form puts a surname before the comma, and a particle surname is + one surname ('De La Cruz, Ed' reads as 'Royce, Ed' does, 'van der + Berg, PhD' as 'Berg, PhD'). Where that surname is the whole part, + the comma settles P1's fork for a leading particle that could be + a given name, being the evidence that the part is a surname; a + word after the one the particle attaches to is a second word, and + a suffix word ends the particle's reach. A word that is also + title vocabulary is not a particle to this count where it opens + the part, reading there as the title it also is (H1); inside a + surname it chains ('de St Pierre, Ed' reads given 'Ed'). A + connective join and a bound given-name pair are not one word + here: whether P3 joins a connective depends on the words of the + whole name, which this count is part of deciding, and a bound + given name builds a given name rather than a surname (P5) — a + word that is a particle as well counts as the particle ('Abu + Bakar, Ed' reads given 'Ed'). A title in front is a word to the + count an unambiguous credential takes, so 'Dr. van der Berg, PhD' + reads as its part does alone. + For the ambiguous credential class — a bare acronym the + vocabulary marks as also an ordinary name, and a word admitted + to the class by shape, which S3 defines and bounds — the + count before the comma is of NAME words rather than of words: + two or more of them read the part after the comma as the + credential run, whatever case the name is written in. The count + goes FIRST and written case is asked only after it: where two + name words stand before the comma the part after it is the + credential run however that part is written, and only where the + count leaves the word a name — one name word before the comma — + is the case read, capitals in a mixed-case name making it the + credential there too (S2). Paired initials are the exception to + the count: two single letters joined by a period, with or without + a period after the second ('M.J.', 'M.J'), the one dotted shape a + person's own initials are written in, read as the given name + after the comma, because two words before the comma may be one + surname written in two ('García Márquez') and the count cannot + tell them from a given name and a family name. Only an + unambiguous suffix word in front of them that is not also title + vocabulary, or another word the class admits by its dotted shape + standing in the same part, makes them the credential run. A word + of this class in front of them speaks for nothing, as S2's + company has it, and where nothing but words of both the suffix + and the title vocabulary stands in front of them, those are the + titles of the given part the initials open, as S2 reads them + there. Behind two or more name words, paired initials that are + not the only shape word in the part are a credential however + little else speaks for them, since no one writes a person's + initials as two dotted groups. An unlisted all-caps word joins + the class in such a part only where the name carries the + contrast: one of its own words before the comma written the way a + name is written in mixed case, holding a capital with its last + cased letter lowercase — read after composing the word, passing + over every non-letter and caseless letter and any lowercase letter + whose capital is not a single letter, as ß's is not — with no + period and not claimed by the vocabulary + as a title, particle, connective, credential or generation. A + surname written in capitals ends in a capital whatever is glued in + front of it, and a word written wholly in lowercase holds none, so + such a record keeps its given name beside its lowercase particles, + titles and clauses ("GISCARD d'ESTAING, VALÉRY", 'LLOYD + FitzGERALD, RONALD'), and a name written wholly in lowercase reads + the listing form. Where the name carries it, an + unlisted word of two capitals after the comma is the paired + initials' shape undotted and is decided at this comma exactly as + they are, by the sentences above ('García Márquez, MJ' and 'García + Márquez, MJ PhD' keep given 'MJ'). The same count reads a part of two or + more words as the credential run when every word of it is a + suffix word or a word of this class, at least one of them of this + class, and none of them a single-letter roman numeral, in any + case: one letter is the shape a middle initial is written in, + and S3 retires single-character matches for the same reason, + while a multi-letter numeral or generational word stands in a + run like any suffix word ('John Smith, III Ma'). A word's + particle reading does not take it out of the run: a word of both + the particle and the unambiguous suffix vocabulary is a suffix + word there too ('John Smith, vd Ma'). Run membership asks what + the word is, and S2's company asks whether it speaks for the word + behind it, which a particle never does; the run S2 says such a + word ends is the company's, not this count's. A word of both the + title and the suffix vocabulary opening a part the count reads as + a run counts as a suffix word there, the name before the comma + being complete, except as the titles of paired initials, above. + Where every word + of this class in the part is a LISTED word written in capitals in + a mixed-case name, the writing has already made each of them the + credential (S2), and the part reads as the credential run on that + evidence rather than on the count; a word of the class by shape + alone carries no such lean, so a part holding one is read by the + count. A part read as postnominal — the credential run, titles + beside suffixes, or titles alone — is read as its words stand, + before any join, and + that reading is final: the one join made inside it is a + connective binding title words into one title (P3: 'Mr. and + Mrs.', a connective run included: 'Minister of the Interior'), + and no particle run, connective join or bound given name + reaches into it. The listing form's given part keeps every join + (P2, P3, P5, P6). + A decision either way at this comma + is reported; for a run of words the decision is the flip to the + credential run, reported once over the whole part. A flip in which + no listed word of this class takes part is the exception and is + made in silence, since only paired initials among such words are + ever taken for a name, and the flip reaches them only where + something else in the part has said otherwise — unless what said + so is nothing but other paired initials: each of those is a word + a reader takes for a name, and that flip is reported. Paired + initials the comma keeps as the given name report where they open + the part; behind a title they do not, the comma's report reaching + only the first word after it ('Smith, Ms G.J.'). A run the + count leaves in the listing form reports as S2 reads the words + in it: a word of this class read as the credential because a + credential in front speaks for it reports, and one its own + capitals made the credential does not unless it opens the part, + where it is the first word after the comma and reports as that + word always does ('Doe, MA PhD', 'John Smith, MA MA'). So a run + whose every such word is written in capitals reads whole in + silence only behind another credential ('John Smith, PhD MA', + 'Smith, PhD MA'), as does a part read as titles before a lone + given name ('Smith, Ms MD Ma'). It is one + of TWO places the comma's own decision is reported, the other being the word trailing the given part after + it (S2), which is a second decision about a second word and never + the same fork twice; an attachment decided after a family comma + (P6) reports on its own. C2's comma-structure flag reports what the + parse could not recognize, not a fork it called. + By default a recognized suffix word counts + even written like an initial ("V."), while strict mode vetoes + initial-shaped words. In the listing form the part after the + comma is still read for what it is: a part that is nothing but + suffix words is the credential run and reads as suffixes, whole + — the slot after a family comma is postnominal position, so the + vocabulary's verdict comes before any title reading of the same + word — and a part that holds no name word at all, titles and suffixes + only, fixes no family boundary, so a part before the comma with + more than one name word keeps its positional read, order and + all. A name word in the part after the comma makes it the + given name, with titles before it and suffixes after, unless a + credential opens the part. A credential opening the part makes it + the postnominal part however it goes on: where the first suffix + word of the part, with only titles in front of it, is one that + starts S2's run, which S2 bounds (the split 'Ph. D.' among them), + every word after it reads as that run reads, a title word as a + title and any other word as a suffix, under strict mode too, + whose veto refuses a word the suffix reading and not the run + ('Smith, PSM I.'). The comma has then fixed nothing, so a part + before it holding two or more name words keeps its positional + read, and a part holding one is the family ('John Smith, PhD + Jones', 'Doe, PhD Jones'). A word of both the title and the + suffix vocabulary opens nothing, and nor does a credential behind + one: in front of a given name it is the listing form's title + ('Smith, Ms Jane'), and no count of the words before the comma + tells a surname of two words from a given name and a family name. + A one-character suffix word — the only kind a reader could take + for an initial — is read by what stands before it. Behind + another suffix it is describing that suffix and continues the + run, written with a period or without, an initial being no shape + anyone writes there; behind a name word it is the generation + only when written bare, a period marking it the abbreviation of + a name and so a middle initial. Both branches then ask that the + generation slot still be open: where a further comma has already + named the suffix, a single letter ending the GIVEN part has no + generation left to be, and stays a middle initial — with a + period or without, and whether a name word or another suffix + word stands before it. Only the given part is touched; a part + after the comma that is nothing but suffix words is the + credential run, and a letter in it continues that run up to the + further comma. Longer suffix words are not in question either + way, and the strict knob above still vetoes the initial-shaped + ones, so the run ends at them there unless a credential opened + the part, above. + A delimiter the policy declares parts a trailing suffix part as a + comma would: the words on each side of it read exactly as the + parts of the same text written with a comma in its place, so no + join, maiden clause or connective reaches across it, and the + delimiter itself is dropped, unless it stands alone as the whole + of its part. Only a trailing part is parted: in the part before + the first comma, or in the given part of the listing form, the + delimiter is a word (the Accepted entry below). + "Smith, John" → family="Smith" + "سلمان، محمد" → family="سلمان" + "田中、太郎" → family="" + "John Smith, PhD" → suffix="PhD" + "John Smith, V." → suffix="V." + "John Smith, V." strict-comma-suffixes → family="John Smith" + "Smith, PhD" → family="Smith" · boundary + "Smith, PhD" → suffix="PhD" + "Smith, Jr." → suffix="Jr." + "Smith, Sr." → suffix="Sr." + "Smith, PSM I" → suffix="PSM I" + "Smith, PSM I." → suffix="PSM I." + "Smith, PSM I." strict-comma-suffixes → suffix="PSM I." + "Smith, John V." → middle="V." + "Smith, John PhD I." → suffix="PhD I." + "Smith, John V" → suffix="V" · boundary + "Smith, Ph. D. Jr." → suffix="Ph. D. Jr." + "Smith, MD PhD" → suffix="MD PhD" + "Doe, MA PhD" → ambiguities=("suffix-or-name",) + "Smith, Dr." → title="Dr." + "Smith, Dr. Jr." → suffix="Jr." + "John Smith, PhD Jones" → given="John" + "John Smith, PhD Jones" → suffix="PhD Jones" + "John Smith, PhD Jones" → ambiguities=("suffix-or-name",) + "Doe, PhD Jones" → suffix="PhD Jones" + "John Smith, Ph. D. Jones" → suffix="Ph. D. Jones" + "John Smith, MD Jones" → given="Jones" · boundary + "Smith, Ms Jane" → title="Ms" · boundary + "John Smith, Mr." → given="John" + "John Smith, Mr." → family="Smith" + "John Smith, Mr. Jr." → given="John" + "Smith Jr., Mr." → family="Smith" · boundary + "John Smith, Jones" → family="John Smith" + "John Smith, MA" → suffix="MA" + "John Smith, Ma" → suffix="Ma" + "Smith, MA" → suffix="MA" + "Smith, Ma" → given="Ma" + "John Smith, Ed" → suffix="Ed" + "Davis Royce, Ed" → suffix="Ed" + "Royce, Ed" → given="Ed" · boundary + "Smith Jr., MA" → suffix="Jr., MA" + "Smith Jr., Ma" → given="Ma" · boundary + "John Smith, PhD MEng" → suffix="PhD MEng" + "John Smith, Ed Ma" → suffix="Ed Ma" + "Jane Doe, MS LAc" → suffix="MS LAc" + "Smith, PhD MEng" → family="Smith" · boundary + "John Smith, PhD DO DO" → suffix="PhD DO DO" + "John Smith, PhD DO DO" → ambiguities=() + "John Smith, Mr. and Mrs." → given="John" + "John Smith, Mr. and Mrs." → title="Mr. and Mrs." + "Smith, Chancellor of the Exchequer MP" → suffix="MP" + "Ortega y Gasset, Dr." → given="Ortega" + "John Smith, PhD vd DO" → suffix="PhD vd DO" + "John Smith, vd Ma" → suffix="vd Ma" + "JOHN SMITH, VD MA" → suffix="VD MA" + "JOHN SMITH, VD MA" → ambiguities=("suffix-or-name",) + "John Smith, Jr vd" → ambiguities=() + "John Smith, Jr do" → ambiguities=("suffix-or-name",) + "John Smith, X.Y.Z." → suffix="X.Y.Z." + "John Smith, X.Y.Z." → ambiguities=() + "John Smith, X.Y.Z." unlisted_dotted_suffixes-off → given="X.Y.Z." + "John Smith, XYZ" → suffix="XYZ" + "John Smith, XYZ" → ambiguities=("suffix-or-name",) + "John Smith, XYZ" unlisted_caps_suffixes-off → given="XYZ" + "John Smith, LEED AP" → suffix="LEED AP" + "Smith, XYZ" → given="XYZ" · boundary + "García Márquez, MJ" → given="MJ" · boundary + "MÜLLER WEIß, HANS" → given="HANS" · boundary + "García Márquez, MJ PhD" → given="MJ" · boundary + "García Márquez, MJ JK" → suffix="MJ JK" + "John Smith, PhD XYZ" → suffix="PhD XYZ" + "Smith, A.B." → given="A.B." · boundary + "García Márquez, G.J." → given="G.J." + "García Márquez, G.J." → family="García Márquez" + "John Smith, A.B." → given="A.B." + "John Smith, A.B." → ambiguities=("suffix-or-name",) + "John Smith, X.Y. P.Q." → suffix="X.Y. P.Q." + "De La Cruz, M.J. K.L." → given="M.J." + "De La Cruz, M.J. K.L." → ambiguities=("suffix-or-name", "suffix-or-name") + "De La Cruz, Ed" → given="Ed" + "van der Berg, MA" → family="van der Berg" + "van der Berg, MA" → suffix="MA" + "Van Buren, Ed" → family="Van Buren" + "van der Berg, PhD" → family="van der Berg" + "John van Buren, Ed" → suffix="Ed" · boundary + "Ortega y Gasset, Ed" → suffix="Ed" · boundary + "abdul Salam, Ed" → suffix="Ed" · boundary + "John Smith, PhD X.Y." → suffix="PhD X.Y." + "García Márquez, G.J" → given="G.J" + "De La Cruz, M.J. PhD" → given="M.J." · boundary + "García Márquez, Ms G.J." → given="G.J." · boundary + "García Márquez, Ed G.J." → given="Ed" · boundary + "John Smith, A.B. Ph.D." → given="A.B." · boundary + "Smith, John, Puig - i Soler" extra_suffix_delimiters-dash → suffix="Puig, i Soler" + "Smith, John, PhD - and MD" extra_suffix_delimiters-dash → suffix="PhD, and MD" + Accepted: a title in front of a particle surname, before the + credential class's word, reads the listing form. The surname + counts once and the title is no name word, so the count reads the + listing form, and the listing form peels a title before the comma + (H6) unless the word is a particle too, as 'Freiherr' is, which + stays in the family. For a German rank that reading is the right one: + since 1919 a former noble title is part of the legal surname, + written between the given name and the particle, so 'Freiherr von + Berg' is the family name in 'Freiherr von Berg, Ed', as 2.0 + through 2.3 read it. Before an unambiguous credential the same + words keep the title apart ('Freiherr von Berg, PhD'), that count + taking the title as a word; both are readings of one name. + "Freiherr von Berg, Ed" → family="Freiherr von Berg" + Accepted: three or more initials run together with periods behind + a surname of two words read as a credential. Initials are + conventionally written apart ('J. R. R.'), and a token of three or + more single letters run together is far likelier a credential than + a given name and a middle name written as one word. + "García Márquez, G.J.R." → suffix="G.J.R." + Accepted: a word of both the title and the unambiguous suffix + vocabulary reads as the postnominal after a family comma in + every spelling, the honorific's too — position decides for the + duals, and the slot is postnominal. + "Smith, Ms." → suffix="Ms." + "Smith, Ms. Jane" → title="Ms." + Accepted: the vocabulary question above — is the part after the + comma nothing but suffix words — is asked a second time, without + the word-count condition, to decide whether a glued East Asian + honorific standing before a family comma comes off. That + composition is tolerated input rather than contract — a comma + between a family name and a given name is no part of CJK writing + — so the precedence is stated at W3, with its examples, and not + here. It is also the one consequence of this question Latin + script cannot witness, nothing there being glued to the end of a + name — which is why this clause carries no example of its own. + Accepted: a maiden marker in a part a declared delimiter + separates takes no clause, as no marker in a part after a suffix + comma does, the marker being an ordinary word there (M2). + Releases 2.0 through 2.3 read past the delimiter and took the + clause; the comma reading is accepted as the cost of one rule + for a declared separator. + "Smith, John, MD - née Jones Smith" extra_suffix_delimiters-dash → suffix="MD, née Jones Smith" + Accepted: a delimiter core the policy names (T1) is a word here, + not structure — v1 applied the delimiter to the suffix-comma + form alone, and that limitation is kept as parity: the core in + 'Smith, RN - CRNA' is a word under the policy as without it. + Since #603 it is a word of the run the credential opens, so the + part reads suffix "RN - CRNA" where 2.0 through 2.3, like v1, + read given "RN". + "Smith, RN - CRNA" → suffix="RN - CRNA" + "Steven Hardman, RN - CRNA" → given="Steven" + Accepted: the further-comma qualifier carries no example line of + its own. It discriminates PAIRS and spans both branches, so + exemplifying it means a with-comma partner for each — every one + of them a promise about behavior that has not moved since v1. + Its executable witness is instead the pair + already standing in the v1-style bank, + tests/test_suffixes.py's + test_roman_numeral_i_after_single_initial_lastname_comma_format + and test_roman_numeral_i_with_explicit_suffix_comma_stays_a_middle_initial. + Read the examples above with the qualifier in hand: `Smith, John + V` reads the suffix and `Smith, John PhD I.` continues the run, + while adding a suffix comma after either turns that same letter + into the middle initial. + history: decisions/C1 · interacts: H1, H2, H6, P1, P2, P3, P5, P6, W3, S2, S3, M2 · implemented: nameparser/_pipeline/_comma.py, nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py + diff --git a/docs/design/rules/C2-parts-past-the-second-comma.md b/docs/design/rules/C2-parts-past-the-second-comma.md new file mode 100644 index 00000000..4b4395fc --- /dev/null +++ b/docs/design/rules/C2-parts-past-the-second-comma.md @@ -0,0 +1,76 @@ +# C2 — parts past the second comma + +C2. Rationale: text beyond the recognized comma parts should be + taken in without silent guessing. + Parts beyond the second are consumed as suffixes either way, + except that a word of the title vocabulary there that is not also + suffix vocabulary is a title: a part past the second comma holds + no name word, so the surnames H5 keeps a bare trailing title word + from misreading are not there to protect, and a piece holding a + suffix word, a title/suffix dual among them, stays the postnominal + the slot makes it (#603). A non-empty extra part that + is not entirely suffix words is flagged as a structural ambiguity + rather than rejected — parsing never fails on content — unless + its words are title words and suffix words, one title word at + least, a word whose period-joined chunks read as a title + ('Lt.Gov.') counting as one. That is asked word by word, so a part + a connective joins into one title ('Secretary of State') keeps + the flag. An empty + part between doubled commas is consumed silently. + A part the parse reads as a credential run by some route other + than the suffix vocabulary is recognized and is not flagged: a + run of ambiguous acronyms whose written case leans credential + (S2), and a run every word of which is an unlisted DOTTED word + the position reads as a credential (S3). This is the one place + the ambiguous class QUIETS a report rather than adding one, and + it is narrow in two ways that the examples below pin. A member + whose written case does not lean CREDENTIAL keeps the flag, + whether the name is written in one case so that nothing leans at + all, or the member is written the way a name is written. And + S2's other by-shape half, the unlisted all-caps word, does not + reach here in any setting: the shape a tail segment is + recognized by is the dotted one alone. + A part the parse consumes wholly as suffixes raises no report + that a word of it was read as a name, whether it is the part + after a suffix comma (C1) or a part beyond the second: nothing + in it is read as one, a maiden marker there being an ordinary + word (M2), and no particle chain (P2) is made in it, so a title + word behind a particle run there is a title like any other, and + the part reports neither a particle chained onto a name word nor + an acronym taken into the name. What such a part reports is its + own — C1's report of the decision at its comma (the flip, or the + first word after the comma), S2's report on each word a credential + in front of it reads as a suffix although it may be a name word (a + name word the run absorbs, a member the company decides), and this + rule's flag. + "John Smith, MD, Bart" → suffix="MD, Bart" + "Eric H. Holder, Jr., Attorney General" → title="Attorney General" + "Eric H. Holder, Jr., Attorney General" → ambiguities=() + "Eric H. Holder, Jr., Secretary of State" → ambiguities=("comma-structure",) · boundary + "Smith, John, Prof." → title="Prof." + "John Smith, Jr., van der Berg Prof." → title="Prof." + "John Smith, Jr., van der Berg Prof." → suffix="Jr., van der Berg" + "John Smith, Jr., Lt.Gov." → ambiguities=() + "John Smith, MD, Ms" → suffix="MD, Ms" · boundary + "John Smith, MD,, Jr." → suffix="MD, Jr." · boundary + "John Smith, MD, R.A.I." → suffix="MD, R.A.I." + "John Smith, MD, R.A.I." → ambiguities=() + "John Smith, MD, R.A.I." unlisted_dotted_suffixes-off → ambiguities=("comma-structure",) + "John Smith, MD, Ma" → ambiguities=("comma-structure",) · boundary + "Steven Hardman, MD, DO, DDS" → ambiguities=() + "STEVEN HARDMAN, MD, DO, DDS" → ambiguities=("comma-structure",) · boundary + "John Smith, MD, XYZ" unlisted_caps_suffixes-everywhere → ambiguities=("comma-structure",) + Accepted: the no-name-reading clause's REPORT half carries no + example line of its own. What it moves is a report with no field + beside it, and an example line would promise that report alone; + its title half is witnessed by the two `John + Smith, Jr., van der Berg Prof.` lines above, and its report half's + executable witnesses are the case rows + a_credential_run_after_the_comma_reports_no_chain_fork and + a_part_past_the_second_reports_no_particle_fork in + tests/v2/cases.py, read beside the two rows the chain still + reports on outside a tail: + the_chain_reports_the_acronym_it_takes and + titled_particle_chain_survives_a_title_that_is_also_a_particle. + history: decisions/C1, decisions/S2 · interacts: C1, H5, P2, P5, S2, S3 · implemented: nameparser/_pipeline/_segment.py, nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py + diff --git a/docs/design/rules/D-construction-and-configuration-diagnostics.md b/docs/design/rules/D-construction-and-configuration-diagnostics.md new file mode 100644 index 00000000..b3c8a8a0 --- /dev/null +++ b/docs/design/rules/D-construction-and-configuration-diagnostics.md @@ -0,0 +1,4 @@ +## Construction & configuration diagnostics (D) + +Background: configuration mistakes are reported when they are made — at construction — not when a name happens to hit them; and a diagnostic that hands the reader code must hand code that works and type-checks. + diff --git a/docs/design/rules/D1-segmenterless-activation-warning.md b/docs/design/rules/D1-segmenterless-activation-warning.md new file mode 100644 index 00000000..081edccf --- /dev/null +++ b/docs/design/rules/D1-segmenterless-activation-warning.md @@ -0,0 +1,15 @@ +# D1 — the segmenterless-activation warning + +D1. Rationale: a parser whose activated scripts nothing can divide + behaves like a working parser minus a feature, silently — the + one misconfiguration a caller cannot see in output. + Constructing a parser that activates division for scripts with + no covering surnames and no segmenter warns at construction, + naming the dead scripts and each way out. + [segmenterless-ja] → warns="deactivate with Policy(segment_scripts=frozenset())" + no-boundary: any covering surname vocabulary, configured + segmenter, or deactivation silences it — the default parser and + the zh pack never warn, which every other example in this + document exercises. + history: decisions/D1 · implemented: nameparser/_parser.py + diff --git a/docs/design/rules/D2-construction-raises-parse-never-does.md b/docs/design/rules/D2-construction-raises-parse-never-does.md new file mode 100644 index 00000000..22ce0018 --- /dev/null +++ b/docs/design/rules/D2-construction-raises-parse-never-does.md @@ -0,0 +1,13 @@ +# D2 — construction raises, parse never does + +D2. Rationale: whatever a name contains, parsing answers; only a + broken configuration may raise, and it must name the field. + Configuration validation raises at construction with the + offending field and value named; applying a locale pack wraps + any such error with the locale's code, so a stacked + configuration names which layer broke. + [bad-name-order] → raises="name_order elements must be Role members" + [bad-order-none] → raises="name_order must be an iterable" + no-boundary: the non-raising side is every other rule's + examples; parse() itself is total (A1). + history: decisions/D2 · implemented: nameparser/_policy.py, nameparser/_parser.py diff --git a/docs/design/rules/H-titles-and-honorifics.md b/docs/design/rules/H-titles-and-honorifics.md new file mode 100644 index 00000000..32aff09c --- /dev/null +++ b/docs/design/rules/H-titles-and-honorifics.md @@ -0,0 +1,5 @@ +## Titles & honorifics (H) + +Background: an honorific title precedes a name and is not itself part of it; it addresses or ranks the person. Most titles address by surname ("Mr. Johnson"), but a few — knighthoods, some clerical and courtesy titles — address by given name ("Sir John"). The library keeps a vocabulary of titles and, separately, of these given-name titles. A period-marked word is claimed by SHAPE at the front and by VOCABULARY at the back. At the front an unlisted abbreviation is a title (H2), the shape outranking the vocabulary; at the back there is no shape rule, so only a listed title word chains into the title (H5) and an unlisted abbreviation stays a name word. A BARE title word at the back is a name word too, TITLES holding ordinary surnames — king, judge, bishop — which is what bars the blanket vocabulary-wins answer (#316, decided 2026-09-08). An input the title peel eats down to one last title-vocabulary word is H4's, and what it does with that word is a convention rather than a reading of the vocabulary. Two criteria govern two different questions here. Membership in the given-name-title list follows HOW THE TITLE ADDRESSES: a title that precedes and addresses by the given name belongs (Sir, Sheikh, the Arabic honorifics الدكتور/الشيخ — which qualify even though those traditions fully retain family names). Whether an EMPTY FAMILY is correct output is the separate question, governed by surname retention: renunciation abolishes the surname, so for Swami, Guru, Baba or Lama family="" is right (#346), while rabbi and imam traditions keep surnames — "Rabbi Cohen" addresses by title and keeps family "Cohen". Conflating the two criteria either ejects the Arabic entries or sweeps in titles that break +"Rabbi Cohen". + diff --git a/docs/design/rules/H1-title-and-one-name-word.md b/docs/design/rules/H1-title-and-one-name-word.md new file mode 100644 index 00000000..9fe046ed --- /dev/null +++ b/docs/design/rules/H1-title-and-one-name-word.md @@ -0,0 +1,70 @@ +# H1 — a title and one name word + +H1. Rationale: a title normally addresses by surname, so a title + followed by a single name word usually names the family; but a + given-name title addresses by given name. Several titles written + together are one form of address, and the one that does the + addressing is the last — the words in front of it rank the + person rather than name them. What stands beside the name word — + a suffix, a nickname, a maiden name — does not make the name any + longer, so it does not decide this reading. And a run written + BEHIND the one name word is the same form of address written on + the other side of it, so where no title run stands in front of + that word it decides the same reading. Where one does stand in + front, that run addresses: a title written behind the name is + the name plus that title (H5), and adding it cannot change how + the words in front of the name are read. Which side a title is + on is decided by the NAME WORD and by nothing else standing + beside it, so a run written around a nickname is one run: the + name Dr. 'Smitty' Sir John reads as Dr. Sir John does. + A title followed by exactly one name word makes that word the + family name, whatever suffix, nickname or maiden name stands + beside it, unless the title is a given-name title, which keeps + it the given name; a run of several titles addresses as its last + title does, and where a run stands BEFORE the one name word it + is the run that addresses, a run standing behind it deciding + that word's field only when none stands before. + "Mr. Johnson" → family="Johnson" + "Mrs. Garcia" → family="Garcia" + "Dr. Smith née Jones" → family="Smith" + "Her Majesty Queen Elizabeth" → given="Elizabeth" + "Dr. Sir John" → given="John" + "Smith Sir." → given="Smith" + "Sir John Prof." → given="John" + "Dr. Smith Sir." → family="Smith" + "His Excellency Lord Duncan" → family="Duncan" + "Her Royal Highness Princess Anne" → given="Anne" + "Sir John" → given="John" · boundary + Accepted: a given-name title plus one name word leaves the + family empty — the input names no family, and inventing one + would be worse. A run whose last title is a given-name title + reads the same way, `queen` doing the addressing where `her + majesty` only ranks. + "Sir John" → family="" + "Her Majesty Queen Elizabeth" → family="" + "Smith Sir." → family="" + Accepted: a title word in the run that no vocabulary knows does + not change what the run addresses by, the last word being the + one asked — `His Excellency Lord Duncan` reads family `Duncan` + because `lord` is not a given-name title, not because the run is + long. `lord` and `lady` stay out of that list by decision + (#519): each addresses by given name only as a courtesy style for + children of the senior ranks (`Lord Peter`, `Lady Diana`) and by + title or surname for every peer and every wife (`Lord Byron`, + `Lady Thatcher`), and the text does not say which the bearer is, + which the list cannot express; `prince` and `princess` are in it. + Accepted: a title word that is also borne as a name, though + rarely as the first word of one (`graf`, `greve`, `herra`), is + the title in front of a name, so a name that BEGINS with it -- a + surname written first without a comma, or a given name -- gives + that word to the title, under a declared family-first order too. + Written last without a period, or alone before a family comma, + it is the name; a period behind it makes it the title by H5, as + for `Mary Jane King.` (#606), and a name word behind it before + a family comma makes it the title by H6. + "Greve Smith, Anna" → title="Greve" + "Greve Anna" → family="Anna" + "Anna Greve" → family="Greve" + "Greve, Anna" → family="Greve" + history: decisions/H1 · interacts: H3, H5, H6, P2, P3, P5, M2, S1, S2, N1, N3 · implemented: nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/H2-leading-abbreviation-title.md b/docs/design/rules/H2-leading-abbreviation-title.md new file mode 100644 index 00000000..9d448b30 --- /dev/null +++ b/docs/design/rules/H2-leading-abbreviation-title.md @@ -0,0 +1,59 @@ +# H2 — the leading-abbreviation title + +H2. Rationale: before a name, an abbreviation is almost always a + title — "Rev.", "Ing.", "Mag." — and no vocabulary can list + every profession's abbreviations in every language. + An abbreviation opening the part of the name that carries the + given name — the whole name, or the part after a family comma — + reads as a title even when unlisted, provided it is an unbroken + run of two or more letters ending in its one period; a bare + initial does not, and neither does anything with interior + periods, hyphens or digits. Where it fires, the shape outranks + vocabulary: a period-marked opening word is a title even when + the word is suffix vocabulary — except after a family comma, + where a part that is nothing but suffix words is the credential + run (C1) and the abbreviation opens nothing. An accented Latin, + Greek or Cyrillic letter written as a base letter plus a + combining mark (NFD) is the same letter as the composed one, + for this shape and for the dotted and all-caps credential + shapes C1, S2 and S3 read (#652), so such a name parses the + same in either encoding and the output keeps the one written. + "Rev. John Smith" → title="Rev." + "Xyz. John Smith" → title="Xyz." + "Smith, Major. John" → title="Major." + "Esq. Smith" → title="Esq." + "J. Smith" → given="J." · boundary + "J.R. Smith" → given="J.R." · boundary + "Dr. inž. Janez Novak" → title="Dr. inž." + Accepted: the shape is only recognizable as an unbroken run of + letters, so an abbreviation in a script whose letters carry + combining vowel signs (Bengali, Devanagari) never reads as a + title by shape — an unlisted abugida honorific stays a name + word, and only vocabulary can recognize it — which is what #343 + and #344 supply for Bengali and Devanagari. + "প্রকৌশলী. Sen" → given="প্রকৌশলী." + Accepted: the shape reads a Latin convention, and a script with + no initials has no period abbreviations either, so a period- + marked opening word carrying a Han, kana or hangul character — + as the script table classifies them, halfwidth katakana + included since #594 — is a name word and never a + title by shape (#323; decisions/cjk-full-stops) — the same + veto that keeps 씨. from reading as an initial. The Latin + reading is unchanged, and W3's example block carries the CJK + reading this clause vetoes into. The veto reads the composed + spelling, so hangul typed decomposed, as conjoining jamo the + script table does not hold, is hangul to it too (#665); the + second line below is typed that way. + "Smith. John" → title="Smith." + "김. 민준" → title="" + Accepted: before a family comma the shape does not fire: the + titles there are the vocabulary's alone (H6), so an unlisted + abbreviation opening that part stays in the family. + "Xyz. Smith, John" → family="Xyz. Smith" + Accepted: after a family comma a part that is nothing but suffix + words is the credential run (C1), which the abbreviation does + not open: the vocabulary decides, and "Esq." is the postnominal + it is. + "Smith, Esq." → suffix="Esq." + history: decisions/H2 · interacts: C1, S2, S3, P4, H5, H6, W3, W4 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_vocab.py + diff --git a/docs/design/rules/H3-title-runs-floor.md b/docs/design/rules/H3-title-runs-floor.md new file mode 100644 index 00000000..1bce276f --- /dev/null +++ b/docs/design/rules/H3-title-runs-floor.md @@ -0,0 +1,39 @@ +# H3 — the title run's floor + +H3. Rationale: compound titles are written as a run of title words, + connectives included; a title word standing inside the name is + just a name word. And a title addresses somebody, so the run + leaves somebody there to address: a post-nominal is written + about a name rather than being one, so it cannot be the name the + run left. + Successive title words at the start of the part carrying the + given name chain into one title; a title word elsewhere in the + name does not. The run leaves one NAME word standing, and a word + of the suffix vocabulary is not that word — where everything + behind the run is post-nominal, the run gives its last word back + to the name, provided that word stands alone and is not itself + suffix vocabulary. + "Asst. Vice Chancellor John Smith" → title="Asst. Vice Chancellor" + "Marquess of Bath" → title="Marquess of Bath" + "Morse, Det. Insp. Jane" → title="Det. Insp." + "Dr King Jr" → family="King" + "Dr King Jr" → suffix="Jr" + "MD DDS" → family="DDS" + "John Doctor Smith" → middle="Doctor" · boundary + Accepted: before a family comma the run is H6's, which stops + where nothing but post-nominals follows for the reason this one + gives a word back, and always leaves the family one word, + whatever that word is. + "Dr. Jr., John" → family="Dr." + Accepted: the word given back must stand alone, so a title + written as one joined unit stays whole and the post-nominal + behind it is the name — turning the unit into a name and leaving + no title at all is the worse of the two readings. + "Prince of Wales Jr" → title="Prince of Wales" + Accepted: where the run's whole content is the word given back, + no title is left to make the reading H1's, so the word stands as + the name and the parse reports the doubt (H4). + "Dr Jr" → given="Dr" + "Dr Jr" → ambiguities=("title-or-name",) + history: decisions/H3 · interacts: C1, H1, H4, H5, H6, S2 · implemented: nameparser/_pipeline/_pieces.py + diff --git a/docs/design/rules/H4-input-that-is-nothing-but-vocabulary-still-has-to-name-somebody.md b/docs/design/rules/H4-input-that-is-nothing-but-vocabulary-still-has-to-name-somebody.md new file mode 100644 index 00000000..16ab4bb3 --- /dev/null +++ b/docs/design/rules/H4-input-that-is-nothing-but-vocabulary-still-has-to-name-somebody.md @@ -0,0 +1,90 @@ +# H4 — an input that is nothing but vocabulary still has to name somebody + +H4. Rationale: this is a name parser, not a title parser. Handed a + string the title peel eats down to one last word which is itself + title vocabulary, it still has to name somebody, and that word is + the only candidate there is — a guess fixed in advance, so the + same input reads the same way every time, not a claim that the + word is a surname. The same reasoning covers a string that is + nothing but post-nominal vocabulary: something has to be the name. + An input whose only remaining name word after the title peel is + itself title vocabulary reads that word as the name by convention + and reports `title-or-name`; an input whose every word is + post-nominal vocabulary reads its first word as a name and + reports `suffix-or-name`. A single title word reads as a title + with no name beside it and reports nothing: the peel took the + whole string, so no word was left standing to be read as a name + and no reading was chosen. + "Lord Chancellor" → family="Chancellor" + "Lord Chancellor" → ambiguities=("title-or-name",) + "The Right Hon. the President of the Queen's Bench Division" → family="Division" + "The Right Hon. the President of the Queen's Bench Division" → ambiguities=("title-or-name",) + "Dr. King" → ambiguities=("title-or-name",) + "Dr. King MD" → ambiguities=("title-or-name",) + "Dr." → title="Dr." · boundary + "Dr." → ambiguities=() + "Dr. Smith" → ambiguities=() + Accepted: ONE site takes both halves — the assignment that places + the lone name word — so neither report depends on the declared + order. Under the default order H1 retags the title-vocabulary + word from given to family afterwards; under a declared + family-first order the assignment places it in the family + directly and H1 never runs. Same reading either way, so the same + kind is reported either way. + Accepted: the suffix half is reached only where no title was + peeled first, so a title in front of the run takes the input out + of that half. What it does NOT do is take the input out of the + rule: the title run leaves a name word standing (H3), and where + that word is title vocabulary the TITLE half claims it — + `Dr. King MD` reports `title-or-name` on `King`, while `MD DDS` + reports nothing, `DDS` being no title. + "MD DDS" → ambiguities=() + "Rinpoche" → ambiguities=("suffix-or-name",) + "QC MP" → given="QC" + "Jr." → title="Jr." + "Jr." → ambiguities=() + Accepted: `Jr.` is post-nominal vocabulary and still reads as a + title, H2's opening-abbreviation shape outranking the vocabulary + where it fires — so the suffix half never sees a dotted lone + credential. + Accepted: the suffix half is scoped to names no script order + placed, so a lone CJK honorific written by itself — さん, 씨, + 선생님 — reports nothing. The same shape, read through the + glued-honorific rules (W2) and the script's own order, and left + to the arc that revisits those readings. That silence is a SCOPE + on the script rule having placed the name, not a claim about the + honorific: a caller who declares no script orders at all gets the + report on the same input, since nothing then resolves the order + and the word reaches the carve-out as any other lone credential + does. The pair is pinned in the case table rather than here, + a policy that empties the script table having no example spelling + in this document. + Accepted: one name UNIT of more than one word carrying title + vocabulary reports `title-or-name` as well, the fork there being + whether the title word inside the unit is a title at all. Usually + a join (P3), and a particle chain (P4) is the same shape and + reports too — `St van Bishop` reads family "van Bishop" with + `bishop` title vocabulary inside it. The + clause reaches EVERY join whose non-leading member is TITLES + vocabulary, not the one word that prompted it: `Smith and King`, + `John and King`, `Smith and Bishop` and `John of Judge` all + report it, as `John of Prince` and `Smith and Prince` do. That + reach is the king/judge/bishop collision set met inside a join, + and the report is the honest answer there rather than a + misfire — a second surname that is also title vocabulary is + exactly the doubt the kind names, and those words stay in the + vocabulary for the addressing forms, which is + decisions/vocabulary-collisions' call. + A join OPENING the name with a title word is a title run (H3) + instead. Behind a peeled title the join is not: a title needs a + following piece and the join is the last one, so `Attorney General + of Minnesota` and `Deputy Secretary of State` keep `Attorney` and + `Deputy` as the title and report the rest — the two corpus names + the clause reaches, and `General` or `Secretary` being a title is + exactly the fork. Nothing in front of the join silences the + clause, a title, a maiden name or a claimed word each deciding + which FIELD the unit takes and none of them whether the word + inside it is a title. The other measured inputs are pinned in the + case table rather than here. + history: decisions/H4 · interacts: H1, H2, H3, H5, S2, O5 · implemented: nameparser/_pipeline/_assign.py + diff --git a/docs/design/rules/H5-trailing-title-run.md b/docs/design/rules/H5-trailing-title-run.md new file mode 100644 index 00000000..8fc75d27 --- /dev/null +++ b/docs/design/rules/H5-trailing-title-run.md @@ -0,0 +1,94 @@ +# H5 — the trailing title run + +H5. Rationale: a word abbreviated with a period at the END of a name + is standing where a post-nominal stands, and the parts of a name + that get abbreviated are the ones outside it. There is no shape + rule there — H2's belongs to the front slot — so only a word the + vocabulary knows as a title is one, and a bare title word is a + name word, TITLES holding ordinary surnames. + After the trailing suffix run has been taken, successive single + words that wear the abbreviation shape and are title vocabulary + chain into the title from the end, leaving one name word + standing. A bare title word there is a name word, and an + unlisted abbreviation there is a name word. The title is + TRANSPARENT to the suffix reading: where two or more name words + stand, what stands once the chain is taken reads exactly as it + would read written without the title, plus the title. Where the + chain leaves ONE name word and no title run stands in front of + it, there is no second reading for it to be transparent to, and + the title behind that word decides its field as a title in front + of it would (H1). Where a run DOES stand in front, that run + addresses and the chained title only joins the title field, so + the transparency holds for the one-word name too. + "John Smith Prof." → title="Prof." + "John Smith Prof." → family="Smith" + "John Smith Prof. Dr." → title="Prof. Dr." + "Dr. John Smith Prof." → title="Dr. Prof." + "Smith, John Prof." → title="Prof." + "Smith Prof." → family="Smith" + "John Smith Sir" → family="Sir" + "Mary Jane King" → family="King" + "John Smith Esq." → suffix="Esq." + "John Smith Xyz." → family="Xyz." · boundary + Accepted: the trailing title words join the title field in input + order after any leading run, because the title view reads the + tokens in the order they were written. + "Dr. John Smith Prof." → family="Smith" + Accepted: transparency is what makes the suffix reading proof + against a title standing among the post-nominals — a title AHEAD + of a suffix word is taken all the same, and the suffix reading + is then taken over what stands, not over what stood. + "John Smith Prof. Jr." → suffix="Jr." + "John Smith Jr. Prof." → suffix="Jr." + "John Prof. MA" → suffix="MA" + "JOHN PROF. MA" → family="MA" + Accepted: the reach is the whole title vocabulary, the ordinary + surnames in it included. TITLES holds king, judge and bishop, + and a period written behind one of them is enough to make it the + title and take it out of the name: `Mary Jane King.` reads title + `King.` with family `Jane`, where the bare spelling keeps family + `King`. That is a cost accepted under the premise that the input + is a name (H Background), not a case this rule prevents — the + period is a writing convention rather than evidence about the + word, so every title whose spelling wears the abbreviation shape + is reachable this way, and the BARE spelling is what the + trailing slot is protected from. + "Mary Jane King." → title="King." + "Mary Jane King." → family="Jane" + Accepted: no fork is reported. Under the premise that the input + is a name, a period-marked word the title vocabulary knows is + not a reading a reader would hesitate over — where the doubt is + real it is the word left STANDING that carries it, which is H4's + report and not this rule's. + Accepted: the chain is read together with the trailing suffix + run (S2), once the connectives have joined (P3) and before the + particle chain is made, so a particle chain (P2) stops in front + of a trailing title rather than taking it into the family name. + A title word a connective joined into a longer unit is no single + word for the chain to take. After a family comma the given part's + chain stops in front of the trailing run of suffixes and titles, + read as the comma-less name reads it, so there too a trailing + title is the title and P6's attachment reads the particle in + front of it as though the title were absent. A maiden clause ends where the run the end of the name + reads without it begins (M2), so where a trailing rule reads the + part a trailing title ends the clause and is a title. + "John van der Berg Prof." → title="Prof." + "John van der Berg Prof." → family="van der Berg" + "Mary Smith née Jones Prof." → title="Prof." + "Smith, John van der Berg Prof." → title="Prof." + "Smith, John van der Berg Prof." → middle="van der Berg" + "Smith, Jan van Prof." → family="van Smith" + Accepted: what the chain leaves is also what counts as a name + word to spare (P5). A trailing title word is not one, so a bound + given-name word behind one joins exactly as it joins with the + title absent — and lands in the same field, the run in FRONT of + the joined pair being the run H1 asks about. Said of the + comma-less writing, which is where the chain runs before the + reserve is read. After a family comma the reserve reads no peel, + but the join declines a trailing title word the chain takes (P5), + so the title is read there too, save the shapes P5's Accepted + clause records, as the segment written with an ordinary given + name reads it. + "Berg, abdul Prof." → title="Prof." + history: decisions/H5 · interacts: H1, H2, H3, H4, M2, P2, P5, P6, S2, C1 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py + diff --git a/docs/design/rules/H6-title-before-a-family-comma.md b/docs/design/rules/H6-title-before-a-family-comma.md new file mode 100644 index 00000000..3fe34e9a --- /dev/null +++ b/docs/design/rules/H6-title-before-a-family-comma.md @@ -0,0 +1,51 @@ +# H6 — a title before a family comma + +H6. Rationale: the listing form moves the surname to the front and + leaves everything else where it was written, so a title written + ahead of the name ends up ahead of the surname — `Dr. Smith, + John` is `Dr. John Smith` indexed. The comma names the family, + and a title is no part of it. But the slot in front of a family + comma is a surname's slot as well, so only the vocabulary claims + a word there, never H2's shape, and a word that is a particle as + well as a title opens a surname rather than ranking a person. + A run of title words opening the part before a family comma is + the title, joining the title field in input order with any title + after the comma. The run admits a title word only as vocabulary + knows it, stops at a word that is also a particle, or where + nothing but post-nominals stands behind it (H3), and always + leaves the family at least one word. The comma decides the + family whatever + the order declared, so the reading is the same under every + name_order. + "Dr. Smith, John" → title="Dr." + "Dr. Smith, John" → family="Smith" + "Prof. Dr. Brühl, Ada" → title="Prof. Dr." + "Dr. de la Vega, Juan" → family="de la Vega" + "Sir Smith, John" → given="John" + "Prof. Smith, Dr. John" → title="Prof. Dr." + "Rev. Sr. Smith, Mary" → title="Rev. Sr." + "Dr. Mc Donald, John" → family="Mc Donald" + "Prof. Smith, John, PhD" → family="Smith" + "Dr., John" → family="Dr." · boundary + "Dr. King, John" → family="King" · boundary + "MD PhD Jr., John" → family="MD" · boundary + "St Pierre, Ed" → family="St Pierre" · boundary + "Dr. St. Pierre, Ed" → family="St. Pierre" · boundary + Accepted: a title word that is also an ordinary surname (king, + judge, bishop, greve) opening a surname of two words is read as + the title, as it is in front of a name with no comma (H1). A + double-barrelled surname that begins with such a word loses it. + "King Smith, John" → title="King" + Accepted: what keeps a word in the family is the particle stop + and nothing else. 'Freiherr' and 'St' are particles as well as + titles, so they stay in either spelling, keeping 'St Pierre' + whole and 'Freiherr von Berg' as C1 reads it. The other German + ranks are titles only, so they peel, though since 1919 a rank + is part of the legal surname for all of them alike; that is the + vocabulary's distinction, not this rule's. A title in front of + a particle still peels. + "Graf Lambsdorff, Otto" → title="Graf" + "Freiherr von Berg, Ed" → family="Freiherr von Berg" + "Dr. Freiherr von Berg, Ed" → title="Dr." + history: decisions/H6 · interacts: C1, H1, H2, H3, M2, P1, P2, P4, S2, W3 · implemented: nameparser/_pipeline/_assign.py + diff --git a/docs/design/rules/M-maiden-names.md b/docs/design/rules/M-maiden-names.md new file mode 100644 index 00000000..7aaba010 --- /dev/null +++ b/docs/design/rules/M-maiden-names.md @@ -0,0 +1,4 @@ +## Maiden names (M) + +Background: a maiden name is written beside the current name, set off by a marker or by enclosure. Markers are attested across French née/né and the unaccented nee English writing uses for both, German geb./geborene, Dutch geboren, Czech/Slovak rozená (the abbreviation roz. shipped through 2.1 and was removed in 2.2 -- it collides with the English diminutive Roz, and a caller who needs it adds it to their own Lexicon), Scandinavian født/fødd/född, Russian урожд. and its full participles урождённая/урождённый (the participles in both the ё and е spellings, which case normalization does not fold), Japanese 旧姓, and Polish z domu — both grammatical genders where attested. Every one of these is a marker wherever it stands, the unaccented nee included: an enclosure holding a word past it reads as the maiden name and the marker is dropped, so a configured pair reading "(Nee Jones)" gives maiden Jones and not Nee Jones. A marker need not be one word. z domu is two, and a phrase marker is recognized only whole and only where its words stand together: neither of its words is a marker standing alone, and neither is the pair once a bracketed clause or a comma divides them. That is what makes shipping it safe where shipping its words would not be — z is an ordinary Polish preposition, and a name that merely contains one keeps its family name. Japanese more often writes the marker with a fullwidth colon (旧姓：佐藤), which is no separator, so marker and name arrive as a single word. Which enclosures mean "maiden" rather than "nickname" is a caller convention, so the maiden reading of a delimiter pair is opt-in — except where the clause announces itself. A clause of two words or more led by a marker word has said which convention it means, and reads as the maiden name inside a nickname pair as well (M3) — unless its content is suffix-shaped, which S1 takes ahead of both. A lone marker has said nothing, and neither has one the colon spelling above glues to the name. A marker also says something about the name standing beside it: it announces a surname the bearer no longer uses, and that is only worth writing where there is a current one to tell it apart from (M4). + diff --git a/docs/design/rules/M1-delimited-maiden-names.md b/docs/design/rules/M1-delimited-maiden-names.md new file mode 100644 index 00000000..19893f14 --- /dev/null +++ b/docs/design/rules/M1-delimited-maiden-names.md @@ -0,0 +1,28 @@ +# M1 — delimited maiden names + +M1. Rationale: an enclosure the caller has declared to mean maiden + holds the former family name; a recognized marker word inside it + marks the clause and is not itself part of the name. + With a delimiter pair configured for maiden names, its enclosed + clause reads as the maiden name — unless the content is + suffix-shaped, which S1 takes first — a leading recognized + marker being dropped where the clause holds a word past it; a + clause of nothing but its marker keeps its words, which may + themselves be a surname (Nee). + Clauses are independent: two enclosures read as one maiden name, + each dropping or keeping its own marker. A pair configured for + both maiden and nickname reads maiden. Configuring the pair is + what this rule needs for a clause that does not announce itself + — markerless content, and a lone marker alike, whether that + marker is one word or several; a clause holding a word past its + marker reads as the maiden name inside a nickname pair as well + (M3), the suffix-shaped content S1 takes excepted there as it is + here. + "Jane Smith (née Jones)" maiden-parens → maiden="Jones" + "Jane Smith (Nee)" maiden-parens → maiden="Nee" · boundary + "Jane Smith (Nee) (Jones)" maiden-parens → maiden="Nee Jones" + "Andrew Perkins (MBA)" maiden-parens → suffix="MBA" · boundary + "Maria Kowalska (z domu)" maiden-parens → maiden="z domu" · boundary + "Maria Kowalska (z, domu Nowak)" maiden-parens → maiden="z, domu Nowak" · boundary + history: decisions/M1 · interacts: S1, M2, M3, M4 · implemented: nameparser/_pipeline/_extract.py, nameparser/_pipeline/_group.py + diff --git a/docs/design/rules/M2-maiden-marker-rule.md b/docs/design/rules/M2-maiden-marker-rule.md new file mode 100644 index 00000000..cfac29ee --- /dev/null +++ b/docs/design/rules/M2-maiden-marker-rule.md @@ -0,0 +1,146 @@ +# M2 — the maiden-marker rule + +M2. Rationale: a maiden marker announces that what follows it is the + former family name; the marker is an announcement, not a name. + A recognized maiden marker counts only in the part of the name + that holds the family name — the whole name where no comma + divides it or before a suffix comma, the part before a family + comma — and only with a + name word straight before it. A word of the leading title run is + no name word there, a lone title included; neither is a word of + the unambiguous suffix vocabulary, unless it opens the name (S2), + nor a connective. A particle + is one: with nothing after it to attach to, it is no particle + there. In the given part after a family comma, in a part after a + suffix comma, and behind a title, a suffix word or a connective, + the marker is an ordinary word; a name standing somewhere before + it is not enough. + A marker that counts takes the words after it as the maiden name, + and is itself dropped. It takes them up to the trailing run of + post-nominals and titles that the end of the name reads as if the + clause were not written — the trailing rules (S2, H5) asked over + the words before the marker and the post-nominal or title words + that end it, before any join — and always takes the first word + after it, the marker having announced a name. A suffix word + straight after the marker announces nothing, and the marker is + then a word. The run so found ends the name: its words are the + suffixes and titles that reading makes them, and no join reaches + into it. Before a family comma no trailing rule reads the part, so + the clause ends at the first suffix word, and the words from it + on read as that part reads them. + Where a trailing rule reads the part, a member of the ambiguous + class (S2) that ends the maiden name is reported where the clause + keeps it, and one in the run is reported where the run is read, + so no word is reported twice and none goes unreported. Before a + family comma no rule reads it, and the member the clause keeps + there is reported nowhere. + Delimiters outrank every reading inside them. Where a recognized + marker stands inside a delimited clause, the whole span is the + maiden name whatever its last word is, and whether or not the + pair is a configured maiden delimiter (M3): the writer drew the + boundary, so no fork is called and nothing is reported. A word + the writer left OUTSIDE the span is outside the clause and reads + as it would anywhere else. Bracketed content ending in a period + is the exception, because it never reaches this rule as a + delimited clause: a trailing period makes the content + suffix-shaped (M3 states this), so the brackets are dropped (S1) + and the clause is read as if written bare. + A marker + with nothing after it, or nothing before it, is just a word. + A marker may be more than one word, and is then recognized only + whole and only where its words stand together: its own first word + standing without the rest is not that marker, and neither is one + the writer divided from the rest by a bracketed clause or by a + comma. Whether the first word takes anything on its own is a + separate question with a separate answer — it does exactly when it + is a recognized marker in its own right, and the longer entry wins + wherever both could match. + A marker taken this way also bounds a particle join arriving from + its left (P2), so the family name's particles stop at the marker + instead of absorbing it; a marker left as a word bounds nothing. + Where the bound leaves a family of nothing but particles, they + are not in particle position and read as ordinary words (R2). + "Jane Smith née Jones" → maiden="Jones" + "Maria López née García Pérez" → maiden="García Pérez" + "Jane Smith née Jones PhD" → suffix="PhD" + "John née Jones Smith V" → maiden="Jones Smith" + "John née Jones Smith V" → suffix="V" + "Doe nee Smith, Jane" → maiden="Smith" + "Doe, Jane nee Smith" → maiden="" + "Doe, Jane nee Smith" → middle="nee Smith" + "Jane Smith, née Jones" → maiden="" + "Smith, John, PhD née Jones" → maiden="" + "Jane Doe PhD nee Smith" → maiden="" + "Jane and née Jones" → maiden="" + "Dr. nee Smith PhD Prof." → maiden="" · boundary + "Dr. nee Smith PhD Prof." → family="Smith" · boundary + "Mai Le née Nguyen" → maiden="Nguyen" + "abd née Jones" → maiden="Jones" + "Jane née Jr y Jones" → maiden="" + "Jane Doe nee Smith MA" → maiden="Smith" + "Jane Doe nee Smith MA" → suffix="MA" + "Jane Doe nee Smith Ma" → maiden="Smith Ma" · boundary + "Jane Doe nee MA" → maiden="MA" · boundary + "Jane Doe nee MA Smith" → maiden="MA Smith" · boundary + "John née Jones Smith Ma" → maiden="Jones Smith Ma" + "John nee Prof. ba MA" → maiden="Prof. ba" · boundary + "Jane Doe nee Puig i Soler" → maiden="Puig i Soler" + "Jane Doe nee Puig i" → maiden="Puig" · boundary + "Jane Doe nee Puig i" → suffix="i" · boundary + "Jane Doe nee Smith Prof." → maiden="Smith" + "Jane Doe nee Smith Prof." → title="Prof." + "Jane Doe nee Smith MA Prof." → suffix="MA" + "Jane Doe nee Smith Prof. MA" → maiden="Smith" + "Jane Doe nee Smith V Prof." → suffix="V" + "Jane Doe nee Smith VI Prof." → suffix="VI" + "Jane Doe nee King." → maiden="King." · boundary + "Jane Doe nee Prof. Dr." → maiden="Prof." · boundary + "Jane van der Berg nee Smith Prof." → title="Prof." + "Jane van der Berg nee Prof. King. MA" → maiden="Prof." · boundary + "Jane Doe nee Smith PhD Smith" → maiden="Smith" + "Jane Doe nee Smith PhD Smith" → suffix="PhD Smith" + "Doe nee Smith Jr. Prof., Jane" → family="Doe Prof." · boundary + "Jane Doe (nee Smith MA)" → maiden="Smith MA" + "Jane Doe (nee Smith Ma)" → maiden="Smith Ma" + "Jane Doe (nee Smith) MA" → suffix="MA" + "Jones née" → family="née" · boundary + "née Jones" → family="Jones" · boundary + "Jane van der Berg née Jones" → maiden="Jones" + "Jane de la née Jones" → maiden="Jones" + "Jane de la née Jones" → family="de la" + "Jane van der Berg née" → family="van der Berg née" + "Jane van der Berg née PhD" → family="van der Berg née" + "Jane van der Berg née y Jones" → maiden="y Jones" + "Maria Kowalska z domu Nowak" → maiden="Nowak" + "Anna z Nowak" → family="Nowak" · boundary + "Anna z (domu) Nowak" → family="Nowak" · boundary + Accepted: the fullwidth-colon spelling arrives as one word, so + the marker inside it goes unrecognized; #317 tracks whether it + should peel. + "山田 花子 旧姓：佐藤" → maiden="" + Accepted: a given name with no current surname beside a maiden + name reads by M4's lone-word rule, the lone name word before the + marker being the family. A writer giving a given name with its + birth name writes the clause delimited. + "Jane née Jones Smith" → maiden="Jones Smith" · boundary + Accepted: the run is what the trailing rules read, before any + join — not the whole reading of the clause-free name. Where a + particle chain would take the words of the run in that name, the + run keeps them suffixes and titles. + "Jane Doe nee Smith DO DO" → suffix="DO DO" · boundary + Accepted: the first word after the marker is the maiden name + whatever it is unless it is a suffix word, and a lone roman + numeral is not one — it is shaped like an initial — so it is the + maiden name where the clause-free name would read it as the + suffix. Decided with #601 (decisions/M2, 2026-10-04). + "Jane Smith née V" → maiden="V" · boundary + Accepted: bracket content ending in a period is suffix-shaped + (M3), so the brackets are dropped (S1) and the clause is read as + if written bare — the delimiter precedence above does not reach + it. + "Jane Doe (nee Smith Prof.)" → title="Prof." + Accepted: a marker behind a credential is an ordinary word, and + the credential's run (S2) takes it in with everything after it. + "Jane Doe Jr. nee Smith" → suffix="Jr. nee Smith" · boundary + history: decisions/M2 · interacts: P2, P3, P5, P6, R1, R2, M1, S1, S2, H1, H5, C1 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py + diff --git a/docs/design/rules/M3-marker-led-clause.md b/docs/design/rules/M3-marker-led-clause.md new file mode 100644 index 00000000..fa4f696c --- /dev/null +++ b/docs/design/rules/M3-marker-led-clause.md @@ -0,0 +1,40 @@ +# M3 — the marker-led clause + +M3. Rationale: an enclosure says nothing about whether it means + maiden, but a recognized marker word inside it does — the clause + announces itself, so the caller does not have to declare the + pair. + A bracketed clause whose content opens with a recognized marker + and carries a word after it reads as the maiden name, + whichever bucket the enclosing pair sits in — unless the content + is suffix-shaped, which S1 takes first — the marker itself + dropped, as M1 drops it. A marker with no word after it is just + a word in brackets, and so is a marker no separator divides from + the name, the fullwidth-colon spelling M2 records. And a marker + followed by a comma is no marker, whether the comma stands alone + or is glued to either word — here and in M1's drop alike, so a + clause the caller declared maiden keeps that marker. Where the + pair is already configured for maiden names M1 governs and this + adds nothing. Being keyed on the content rather than on the + pair, this reaches a nickname pair's clause too, and M1's + independence then governs what it produces: where a marker-led + clause stands beside another marker-led clause, both read as + maiden and join into one maiden name, leaving no nickname. + "Jane Smith (née Jones)" → maiden="Jones" + "Jane (née Jones) Smith" → family="Smith" + "Jane Smith (née Jr.)" → suffix="Jr." + "Jane Smith (née)" → nickname="née" · boundary + "Maria Kowalska (z domu Nowak)" → maiden="Nowak" + "Maria Kowalska (z domu)" → nickname="z domu" · boundary + "Jane (née , Jones) Smith" → nickname="née , Jones" · boundary + "Jane (née ,Jones) Smith" maiden-parens → maiden="née ,Jones" · boundary + Accepted: the word taken after the marker is not tested for + being a name word, so unlike M2's bare take this one does not + stop at a suffix word — the same two words read one way + bracketed and another way bare. This does not contradict the + S1 example above: S1 asks whether the WHOLE clause is + suffix-shaped, which the trailing period makes true of the one + and false of the other, so only the V clause reaches this rule. + "Jane Smith (née V)" → maiden="V" + history: decisions/M3 · interacts: M1, M2, S1, N1 · implemented: nameparser/_pipeline/_extract.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_vocab.py + diff --git a/docs/design/rules/M4-maiden-marker-makes-the-lone-name-word-the-family.md b/docs/design/rules/M4-maiden-marker-makes-the-lone-name-word-the-family.md new file mode 100644 index 00000000..22ea8695 --- /dev/null +++ b/docs/design/rules/M4-maiden-marker-makes-the-lone-name-word-the-family.md @@ -0,0 +1,40 @@ +# M4 — a maiden marker makes the lone name word the family + +M4. Rationale: a maiden name is a FORMER family name, and a former + one only means something beside a current one — nobody announces + the surname they used to carry where there is no surname beside + it to tell it apart from. So where a maiden name has been read + out of a name and a single name word is left standing, that word + is the surname the bearer uses now; writing the clause at all is + what rules out the given-name reading O5's convention would + otherwise leave it with. How the maiden name was written does + not enter into it — a marker taking the words after it (M2), a + marker-led clause (M3), and a clause in a pair the caller + declared to mean maiden (M1) announce the same thing, and the + last of those carries no marker at all. + A maiden name standing beside exactly one name word makes that + word the family name, whatever suffix or nickname stands beside + it — an annotation beside the name is no part of the name, which + is how H1 reads the same shape. Two kinds of word are beyond its + reach, for one reason: the rule changes what POSITION would have + decided, so it cannot overrule what a word already IS + (mechanisms.md#TWO-LAYER-ASSIGN). A word the vocabulary has + claimed as a given name keeps that reading, and so does a word + read as an initial, which is nobody's family name. A name + carrying a TITLE is H1's rather than this rule's, H1's + given-name-title carve-out included, which keeps the word a + given name. A nickname holds nothing off: where N3 has already + named the family this rule finds nothing left to move, and where + N3 declined — its count does not set a suffix aside — this rule + names it. Where the maiden name stood does not matter, only what + is left beside it: a marker inside the name takes the rest of it + (M2), and one name word left that way is one name word. + "Smith née Jones" → family="Smith" + "Smith (née Jones)" → family="Smith" + "Smith (Jones)" maiden-parens → family="Smith" + "Jane née Jones Smith" → family="Jane" + "Smith née Jones PhD" → family="Smith" + "abd née Jones" → given="abd" · boundary + "J. née Jones Smith V" → given="J." · boundary + history: decisions/M4 · interacts: M1, M2, M3, H1, N3, O5 · implemented: nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/N-nicknames-and-quoted-names.md b/docs/design/rules/N-nicknames-and-quoted-names.md new file mode 100644 index 00000000..ae6025fb --- /dev/null +++ b/docs/design/rules/N-nicknames-and-quoted-names.md @@ -0,0 +1,6 @@ +## Nicknames & quoted names (N) + +Background: a nickname is written beside the formal name, set off by quotes or brackets. Quotation conventions vary by language („…“, +«…», “…”), several share characters — one convention's closer is +another's opener — and the straight apostrophe doubles as a quotation mark and as a letter-like mark inside names (O'Connor). Which pairs delimit nicknames is caller configuration. + diff --git a/docs/design/rules/N1-default-delimiter-pairs.md b/docs/design/rules/N1-default-delimiter-pairs.md new file mode 100644 index 00000000..d2655315 --- /dev/null +++ b/docs/design/rules/N1-default-delimiter-pairs.md @@ -0,0 +1,16 @@ +# N1 — the default delimiter pairs + +N1. Rationale: a quoted or bracketed clause beside a name is an + informal alias, not part of the name. + A clause enclosed by a configured nickname delimiter pair reads + as the nickname and is lifted out of the name, a comma inside it + kept as written — the comma divides nothing there, though the + dots T2 and T3 name still divide a clause as they divide a name; + an empty enclosure is simply dropped. + "Andrew (Andy) Perkins" → nickname="Andy" + "Jane (Janey, Jo) Smith" → nickname="Janey, Jo" + "Jean 'JD' Smith" → nickname="JD" + "Anna () Smith" → nickname="" · boundary + "John ｢Jack｣ Smith" → nickname="Jack" + history: decisions/N1 · interacts: T2, T3 · implemented: nameparser/_pipeline/_extract.py, nameparser/_pipeline/_tokenize.py + diff --git a/docs/design/rules/N2-same-character-quotes-and-shared-character-conventions.md b/docs/design/rules/N2-same-character-quotes-and-shared-character-conventions.md new file mode 100644 index 00000000..59941874 --- /dev/null +++ b/docs/design/rules/N2-same-character-quotes-and-shared-character-conventions.md @@ -0,0 +1,17 @@ +# N2 — same-character quotes and shared-character conventions + +N2. Rationale: only a mark standing at word boundaries is quoting; + anywhere else it is part of the word. + A quote whose open and close are the same character opens only + at a word start and closes only at a word end, so an apostrophe + inside or at the end of a word is literal. Between conventions + that share a character, position in the text decides: the + leftmost valid opener wins — and a dangling-open report is + suppressed where its character sits inside another pair's + successful match, being literal content there rather than an + imbalance (A1 depends on that filter). + "Sean O'Connor" → family="O'Connor" + "Hans „Erster“ und “Zweiter” Müller" → nickname="Erster Zweiter" + "Mari' Aube'" → family="Aube'" · boundary + history: decisions/N2 · implemented: nameparser/_pipeline/_extract.py + diff --git a/docs/design/rules/N3-lone-word-nickname-rule.md b/docs/design/rules/N3-lone-word-nickname-rule.md new file mode 100644 index 00000000..0ce892bc --- /dev/null +++ b/docs/design/rules/N3-lone-word-nickname-rule.md @@ -0,0 +1,32 @@ +# N3 — the lone-word nickname rule + +N3. Rationale: a person set down as a nickname plus one name word is + being identified by surname. + A name that is only a nickname and one name word reads that word + as the family name; with two or more name words the ordinary + positional reading applies. + "'Smitty' Jones" → family="Jones" + "'Smitty' John Jones" → given="John" · boundary + Accepted: the count does not set suffixes aside, so a nickname + plus one name word plus a suffix reads the name word as given + and leaves the family empty. A title counts against the count + too, but H1 then reads the title-plus-one-word name that is + left, so the family is named after all — unless the title is a + given-name title, which keeps the word in `given` and leaves no + family, exactly as it does anywhere else. + "'Smitty' Jones Jr." → family="" + "'Smitty' Dr. Jones" → family="Jones" + "'Smitty' Sir John" → given="John" + Accepted: a marker-led clause is a maiden clause and not a + nickname one (M3), so this rule does not reach a name written + that way, and the one name word keeps the reading the bare + spelling gives it. Since #445 that reading is the family name + all the same, M4 reaching both spellings from the other side. + The two rules agree on THIS name and not in general: M4 counts + name words alone, so where a suffix stands beside them it fires + and this rule declines. The Accepted line above reads a bare + nickname-plus-word-plus-suffix name as given with no family, and + the same name carrying a maiden clause reads family instead. + "Smith (née Jones)" → family="Smith" + history: decisions/N3 · interacts: H1, M3, M4 · implemented: nameparser/_pipeline/_assign.py + diff --git a/docs/design/rules/O-name-order.md b/docs/design/rules/O-name-order.md new file mode 100644 index 00000000..a1d8024f --- /dev/null +++ b/docs/design/rules/O-name-order.md @@ -0,0 +1,5 @@ +## Name order (O) + +Background: written name order varies by convention: given-first (the library's default reading), family-first, and family-first with the given name last (Vietnamese, where the person is called by the last element, given names are frequently two syllables — the given_names view stays correct wherever the internal boundary falls — and quốc ngữ is Latin script, so no native-script signal exists at all). The order is declared by the caller or a locale pack, never detected — but a few conventions leave a recognizable trace in the name itself. Patronymics are one: East Slavic names carry a father's-name derivative with distinctive endings between given and family, and Turkic names use a standalone marker word ("oglu" son-of, "qizi" daughter-of) after the father's name. Where such a trace is present and unambiguous, an opted-in parser can restore the intended reading from a family-first listing. +A declared order is a property of the DATA SOURCE rather than of any one string: the caller sets it to match how their records are written, and it governs what no vocabulary and no script license has already claimed (O4). A declared FAMILY-FIRST order outranks what the parser could infer from the shape of a particular name; the given-first reading is the parser's DEFAULT rather than a declaration it can tell apart from one, and the traces above are what refine it (O1, O2). And the declaration yields where a name's own script settles the order instead (W4, where a name written wholly in an East Asian script reads family-first whatever order the caller declared). Two consequences run through this document. Under the default given-first order a string opening with a never-given particle is a surname whose given name is simply absent, so the fold takes the rest of it (P1). Under a declared family-first order the caller has already said that what follows the family is not more surname — so the fold stops there, and the rotations below, whose whole job is to RESTORE the default reading from a family-first listing, have nothing left to restore. A shape neither order settles is the family comma's job, and the parser does not guess at it. + diff --git a/docs/design/rules/O1-east-slavic-rotation.md b/docs/design/rules/O1-east-slavic-rotation.md new file mode 100644 index 00000000..a178a0f7 --- /dev/null +++ b/docs/design/rules/O1-east-slavic-rotation.md @@ -0,0 +1,34 @@ +# O1 — East Slavic rotation + +O1. Rationale: an East Slavic name written family-first still shows + its patronymic — the distinctive ending identifies which word is + the patronymic, and the patronymic sits next to the given name. + With East Slavic patronymic handling active and no comma in the + name, a name of exactly three name words — titles, suffixes and + nicknames aside — whose last name word carries a patronymic + ending and whose middle name word does not reads as family-first: + the words are restored to given, patronymic, family. A middle + word that also carries the ending blocks the reading, because + the surname itself may be patronymic-derived. + The restoration is the default order's work: it recovers the + given-first reading a family-first listing hides, so where the + caller has DECLARED a family-first order there is nothing left + to restore and position decides. + "Сидоров Иван Петрович" [ru] → family="Сидоров" + "Sidorov Ivan Petrovich Jr." [ru] → family="Sidorov" + "Иван Петрович Абрамович" [ru] → family="Абрамович" · boundary + Accepted: under a declared family-first order the readings part + on natural-order input, and the declaration wins. A family-first + listing reads the same either way under FAMILY_FIRST — the first + example above is that same parse with FAMILY_FIRST declared, + and under FAMILY_FIRST_GIVEN_LAST the listing's given and middle + swap, so only the family is invariant — while with East Slavic + handling active and FAMILY_FIRST declared the natural-order + Иван Петрович Сидоров reads family Иван, given Петрович, middle + Сидоров. That is the caller's declaration being honored on input + they said was written family-first, not a defect (#384). No + registered example annotation combines a pack with an order, so + the parse is pinned in tests/v2/test_locales.py instead, and + decisions/O1 records why options 2 and 3 were declined. + history: decisions/O1 · implemented: nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/O2-turkic-rotation.md b/docs/design/rules/O2-turkic-rotation.md new file mode 100644 index 00000000..1b577478 --- /dev/null +++ b/docs/design/rules/O2-turkic-rotation.md @@ -0,0 +1,25 @@ +# O2 — Turkic rotation + +O2. Rationale: a Turkic patronymic marker is a separate word that + follows the father's name; a four-word name ending in one is a + family-first listing. + With Turkic patronymic handling active and no comma in the name, + a name of exactly four name words — titles, suffixes and + nicknames aside — ending in a standalone patronymic marker reads + family-first: the first name word is the family name, and the + marker stays beside the father's name in the middle. + The scope is O1's: the restoration recovers the default reading, + so a declared family-first order stands in its place. + "Ali Ahmad Vali oglu" [tr_az] → family="Ali" + "Ali Ahmad Vali oglu Jr." [tr_az] → family="Ali" + Accepted: any other count of name words keeps its positional + reading, even when that leaves the marker itself in a name + field. + "Ali Ahmad oglu" [tr_az] → family="oglu" · boundary + Accepted: the order scope above has this rule's own consequence + reached by a second route — with Turkic handling active and + FAMILY_FIRST_GIVEN_LAST declared, Ali Ahmad Vali oglu reads + given oglu, the marker standing in a name field because the + declaration put the name's last word there (#384). + history: decisions/O2 · implemented: nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/O3-compound-family-names-written-unmarked.md b/docs/design/rules/O3-compound-family-names-written-unmarked.md new file mode 100644 index 00000000..dfd6dd7b --- /dev/null +++ b/docs/design/rules/O3-compound-family-names-written-unmarked.md @@ -0,0 +1,12 @@ +# O3 — compound family names written unmarked + +O3. Rationale: several traditions write compound family names + unmarked, so that every word after the given name belongs to the + family name. + With compound-family handling active, every middle word joins + the family name and is rendered before it; no word is a middle + name. + "Hassan Mohamad Ali" middle_as_family → family="Mohamad Ali" + "Hassan Mohamad Ali" → family="Ali" · boundary + implemented: nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/O4-positional-assignment-and-declared-order.md b/docs/design/rules/O4-positional-assignment-and-declared-order.md new file mode 100644 index 00000000..8c37b7a7 --- /dev/null +++ b/docs/design/rules/O4-positional-assignment-and-declared-order.md @@ -0,0 +1,17 @@ +# O4 — positional assignment and declared order + +O4. Rationale: what no vocabulary claims can only be read by where + it stands, under the order the caller declared. + Words no vocabulary has claimed read by position. In the default + given-first order the first name word is the given name, the + last is the family name, and everything between is middle names. + In a family-first order the first name word is the family; in + family-first-given-last the given name comes from the end, the + middles from between. + "Mary Beth Smith" → middle="Beth" + "Garcia Juan Carlos" family-first → family="Garcia" + "Nguyễn Thị Minh Khai" family-first-given-last → given="Khai" + no-boundary: this is the default reading every other rule carves + exceptions from; its boundaries are the other rules. + implemented: nameparser/_pipeline/_assign.py + diff --git a/docs/design/rules/O5-lone-name-words-convention-and-reporting-it.md b/docs/design/rules/O5-lone-name-words-convention-and-reporting-it.md new file mode 100644 index 00000000..582f7484 --- /dev/null +++ b/docs/design/rules/O5-lone-name-words-convention-and-reporting-it.md @@ -0,0 +1,60 @@ +# O5 — the lone name word's convention, and reporting it + +O5. Rationale: O4 reads a name by comparing where its words stand, + and a name of exactly one name word gives it nothing to compare + — the first name word is also the last, so the rule that decides + every longer name is silent here. Nothing in such an input says + whether the word is a given name or a family name: both readings + fit it equally well, and the library still has to report one. + A name of one name word that nothing else has decided reads that + word as the given name under the default given-first order, and + as the family name under a declared family-first one. That is a + convention rather than a determination — the same input has to + read the same way every time, so one of two equally consistent + readings is fixed in advance — and it is not evidence about the + word. Every rule that DOES decide such a name outranks it, and + each of them carves out a case it declines to decide — where the + convention is what is left. A title makes the word the family + name (H1), except a given-name title, which addresses by given + name; a nickname beside it does the same (N3), except where a + suffix stands beside them too, which its count does not set + aside; and a maiden name beside it does the same (M4), except + where the vocabulary or the word's own shape has claimed the + word already. The convention is reported: a name whose one name + word nothing else decided carries a `given-or-family` ambiguity + naming the field the convention chose. The report is exactly as + narrow as the convention, so every rule named above silences it + where it fires, a family comma — a comma with a name segment + after it — silences it, a script whose own convention settles + the order (W4) silences it, and so does the word's own claim — a + particle, a bound given name, an initial's shape. A comma with + nothing after it names no family and silences nothing, so + `سلمان،` and `Smith,` report where `Smith, Andrew` does not. A + comma with only a TITLE after it names no family either, but the + title still decides the field wherever it stands, so `John V, Dr.` + is silent for H1's reason and not the comma's. A word with no + letter or digit in it is no name word and + reports nothing (A2). Where the one name unit carries title + vocabulary in a word beside the one it places, or is itself title + vocabulary left standing by the title peel, the doubt reported is `title-or-name` + instead, which H4 states — so a title silences THIS kind, not + every report at the site. + "Smith" → given="Smith" + "Garcia" family-first → family="Garcia" + "Sir John" → given="John" + "'Smitty' Jones Jr." → given="Jones" + "abd née Jones" → given="abd" + "Mr. Johnson" → family="Johnson" · boundary + "'Smitty' Jones" → family="Jones" · boundary + "Smith née Jones" → family="Smith" · boundary + "Andrew" → ambiguities=("given-or-family",) + "Garcia" family-first → ambiguities=("given-or-family",) + "Juan & Garcia" → ambiguities=("given-or-family",) + "'Smitty' Jones Jr." → ambiguities=("given-or-family",) + "Smith Jr." → ambiguities=("given-or-family",) + "Dr. Smith" → ambiguities=() · boundary + "Sir John" → ambiguities=() + "Smith née Jones" → ambiguities=() + "abd née Jones" → ambiguities=() + history: decisions/O5 · interacts: O4, H1, N3, M4, H4, W4, P3, A2 · implemented: nameparser/_pipeline/_assign.py + diff --git a/docs/design/rules/P-particles-and-surname-prefixes.md b/docs/design/rules/P-particles-and-surname-prefixes.md new file mode 100644 index 00000000..3c95598a --- /dev/null +++ b/docs/design/rules/P-particles-and-surname-prefixes.md @@ -0,0 +1,4 @@ +## Particles & surname prefixes (P) + +Background: particles ("de", "la", "van", "von", "bin") link forward to a surname and are written as part of it. Some are never anyone's given name; others ("Van", "Bin") are ordinary given names in some cultures, so the vocabulary distinguishes never-given particles from ambiguous ones, and only the never-given ones license special treatment. A separate small vocabulary binds forward to a GIVEN name instead: words like "abdul" that are not complete given names alone (P5). Which particles fall on which side of the never-given line is its own open question (#360). + diff --git a/docs/design/rules/P1-lone-particle-fold.md b/docs/design/rules/P1-lone-particle-fold.md new file mode 100644 index 00000000..1debba2e --- /dev/null +++ b/docs/design/rules/P1-lone-particle-fold.md @@ -0,0 +1,89 @@ +# P1 — lone particle fold + +P1. Rationale: a particle OPENING a name has the whole rest of the + name to link forward to, so it is doing a particle's work and the + piece it heads is a surname written out in full. That is evidence + about the writing, not about the word, which is why no declared + order contradicts it. + A never-given particle opening the name marks the name as + surname-only: the particle run and the name words it attaches to + are the family. It needs another name word to attach to. The run + is every particle in sequence, never-given and ambiguous alike + ("de la Vega" is one group, not "de" plus a separate "la Vega"). + An ambiguous particle keeps whatever reading its position gives + it. That the particle claims the FAMILY rather than a given name + holds under every order: this is evidence about how the name is + written, and a declared order governs only what no vocabulary has + claimed (O4) — the same precedence the script license takes in W4. + The OPENING position is the whole of this rule's subject. A + particle standing where the given name would go was a second site + until #467, and dropping it is a correction rather than a + narrowing: that slot holds what the caller DECLARED to be the + given name, and the never-given vocabulary supplies a reading + where position leaves the question open rather than vetoing one + position has already given. A particle ending the name is P6's, + and only where it landed in a MIDDLE. + + How MANY name words it attaches to depends on the order. Under a + family-first order it takes exactly ONE — declaring that order + asserts that what follows the family is not more surname. Under + the default order it takes the rest of the name, and that is a + reading rather than a gap in one: a never-given particle marks + where a surname BEGINS, surnames are routinely several words, + and no declaration has said this one ends before the string + does. So `de Mesnil Jean` reports the whole string as the + family, which is 1.4.0's reach as well (#471, declined by + design). One name word means one UNIT — a particle chain (P2), + a conjunction join (P3) or a bound given-name pair (P5) is + taken whole or not at all. A title does not move the opening + position (P4), but a family comma does end the question: the + comma has already fixed the surname, so there is no positional + read left for an order to narrow, and a particle opening the + part AFTER it takes the rest of that part whatever order is + declared. What is left over is not read by O4's rule for a + whole name, which would make the first leftover a second family + name; it is laid out as the positions AFTER the family in the + declared order, the family slot being already filled. + "de la Vega" → family="de la Vega" + "Sir de Mesnil" → family="de Mesnil" + "Mesnil de" family-first → given="de" + "de Mesnil Jean" → family="de Mesnil Jean" + "de Mesnil Jean" family-first → family="de Mesnil" + "de Mesnil Jean" family-first → given="Jean" + "Smith, de Mesnil Jean" family-first → family="Smith de Mesnil Jean" + "de la Vega y Santos Juan" family-first → family="de la Vega y Santos" + "ibn Awf abdul Rahman" family-first → given="abdul Rahman" + "de la Cruz Juan Carlos" family-first-given-last → given="Carlos" + "Mc Donald" → family="Mc Donald" + "de los Santos" → family="de los Santos" + "van Gogh" → given="van" · boundary + Accepted: the default-order reach has two faces, and the second + is a real cost. A surname-only string is read right, which is + what the reach is for; a family-first listing read under the + default order is read wrong, all of it landing in the family. + The remedy is the declaration or the comma — declare a + family-first order, which gives family `de la Cruz`, given + `Juan`, middle `Carlos`, or write the family comma. What + NEITHER order settles is the comma's alone: `de la Family + Family Given` and `de la Family Given Given` are the same + string shape, and no reading of the words tells them apart. + "de la Torre Vega" → family="de la Torre Vega" + "de la Cruz Juan Carlos" → family="de la Cruz Juan Carlos" + Accepted: a bare "de" stays the given name — there is nothing to + fold into, and inventing a surname would be worse. + "de" → given="de" + Accepted: stopping the run leaves a MIDDLE where the fold never + left one before, so O3 has something to claim that it could not + reach until now. The family it then reports is discontiguous in + the input — words 1-3 plus word 5 — and renders the folded word + first, which is R1's order, not this rule's doing. + "de la Cruz Juan Carlos" family-first+middle_as_family → family="Carlos de la Cruz" + Accepted: only the OPENING position is this rule's subject. A + particle chain standing inside the name is grouped normally (P2) + and positioned by the declared order, so a family-first reading + may report it as the given name; what the vocabulary forbids is + the bare particle reading as a given name, not any name part + that begins with one. + "Juan de la Vega" family-first → family="Juan" + history: decisions/P1 · interacts: O3, O4, P2, P3, P4, P5, P6 · implemented: nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/P2-particles-join-forward.md b/docs/design/rules/P2-particles-join-forward.md new file mode 100644 index 00000000..2649a706 --- /dev/null +++ b/docs/design/rules/P2-particles-join-forward.md @@ -0,0 +1,80 @@ +# P2 — particles join forward + +P2. Rationale: a particle is written as part of the surname it + precedes, and a title stands outside the name entirely. + A particle joins the words after it into one name part, the + join running until the next particle starts a group of its own, + a trailing suffix begins — the trailing run S2 reads, once, + before the chain is made and counting the chain's own run as the + one name word it will be, so the chain cannot change what it + counts: a trailing roman numeral, or a + bare acronym with words to spare, ends the chain as a suffix word + does, and so does a bare ambiguous acronym written in capitals in + a mixed-case name, which needs no words to spare; the same + acronym written Title-case in a mixed-case name ends nothing and + joins the chain as any name word does, words to spare or not, + and the fork the chain called is reported there exactly as it is + where no particle stands (S2) — + a maiden + marker takes the + words after it (M2), or the name ends. The final group reads as + the family name; + earlier groups read by position. The chain begins wherever the + name begins, and a preceding title does not move that point. + Where P1's fold has claimed the opening, the fold decides the + family instead — and may take only PART of the final group, + since it counts name words and the group is one part. + In the part before a family comma, which the comma named the + family, a particle that is also suffix vocabulary heads the name + word behind it, whatever case the name is written in, and is not + read as a suffix standing between two family words; with a suffix + word or nothing behind it, it reads as the suffix it is there. A + lone letter behind it reads as it does in the same part without + the particle: a capital is a name word ('SMITH VD V, JOHN' as + 'SMITH V, JOHN'), a lowercase letter the numeral ('smith vd v, + john' as 'smith v, john'). + "John van der Berg" → family="van der Berg" + "John van der Berg Smith" → family="van der Berg Smith" + "Vincent van Gogh van Beethoven" → middle="van Gogh" + "Dr. John van der Berg" → family="van der Berg" + "Juan de" → family="de" · boundary + "de la Cruz Juan Carlos" family-first → family="de la Cruz" + "John van der Berg PhD" → family="van der Berg" + "John van der Berg V" → family="van der Berg" + "John van der Berg V" → suffix="V" + "John van der Berg Ma" → family="van der Berg Ma" + "John van der Berg Ma" → ambiguities=("suffix-or-name",) + "john van der berg ma" → suffix="ma" + "John van der J. V" → family="van der J. V" · boundary + "Freiherr von Berg MA" → family="von Berg" + "Freiherr von Berg MA" → suffix="MA" + "Freiherr von Richthofen V" → suffix="V" · boundary + "SMITH VD MA, JOHN" → family="SMITH VD MA" + "SMITH VD, JOHN" → suffix="VD" · boundary + "John van der Berg née Jones" → family="van der Berg" + Accepted: a particle of the unambiguous suffix vocabulary too + (vd, mc) is a suffix piece to the peel, so where it opens the + trailing run the chain stops before it as before any suffix + word, and the peel takes it; where it continues a prefix run, + the run takes it as a particle, as P6 reads it after a comma — + and there the chain reports nothing, the particle reading being + P6's fork rather than S2's. + The same carve-out covers `do`, the one word that is both a + particle and an AMBIGUOUS credential acronym: where a particle + run takes it, the word is the run's and the chain reports + nothing, whatever case it is written in — so `Anh van Do` is + silent where `Anh Do`, with no particle standing, reports. + A word that is both belongs to the particle run and to P6's + fork, not to S2's. + "John Smith Mc V" → suffix="Mc V" + "John van Mc" → family="van Mc" + "anh van do" → family="van do" · boundary + "anh van do" → ambiguities=() · boundary + "Anh van Do" → family="van Do" · boundary + "Anh van Do" → ambiguities=() · boundary + Accepted: a caller wanting the combined double-surname reading + (#132's ask) has it as the surnames view rather than the + family field. + "Vincent van Gogh van Beethoven" → surnames="van Gogh van Beethoven" + history: decisions/P2 · interacts: P1, P4, H5, M2, S2, C2 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/P3-connectives.md b/docs/design/rules/P3-connectives.md new file mode 100644 index 00000000..c40ef1b5 --- /dev/null +++ b/docs/design/rules/P3-connectives.md @@ -0,0 +1,153 @@ +# P3 — connectives + +P3. Rationale: connective words ("y", "of the") bind name words into + one name part; but a single letter in a short name is more + likely an initial than a connective, and where a name is written + in more than one case, a bare Latin capital standing alone is how + an initial is marked and a bare lowercase letter is how it is + not. + A recognized connective joins its neighbors into one name part, + connective runs included — except a single-letter connective in + a three-word name, which stays a name word, and a single-letter + connective that reads as an initial instead, which never joins. + A single-letter connective reads as an initial where the writing + says so: written as a bare Latin capital in a name that is not written + wholly in one case, or — in a name written wholly in one case, + where nothing says so — where the letter is one the vocabulary + marks as reading both ways. + A letter the vocabulary marks as reading both ways, read as an + initial in a name written wholly in one case, is a call that could + have gone the other way, and is reported. + The joined part is ONE name word wherever another rule counts + them, so a rule taking "one name word" takes the whole join and + never half of it — except C1's count of the words before a comma, + which counts particle surnames alone and states why. + A connective counts as a name word wherever this rule counts them, + whatever else the vocabulary says the word is, where it is placed + to join. A word can be a connective and a generation at once — the + Catalan link is also the roman numeral one — and counting it as + the generation would let a connective raise the bar for its own + join, so the shortest linked name, which is the commonest one, + would be the only one that failed to link; while counting it as a + connective where it joins nothing lowers the bar for a different + connective's join instead. + A connective that is also generational vocabulary joins only where + a name word stands on each side of it — a word the rest of the + parse reads as a name word rather than as a generation, a + credential or an honorific, looked for past any run of + connectives standing between. A connective with nothing to its right is connecting + nothing, and a word of that vocabulary ending a name, or standing + before the credential a name ends with, is the generation it also + spells. + A separator the caller declared, standing in a trailing suffix + part, ends the search there as a comma would, so a connective + never joins across it (C1); anywhere else it is a word. + Both questions this rule asks of a name — how many words it has, + and whether it is written in one case — are asked of the name's + OWN words: a maiden marker taken as one, and the words it takes + (M2), are not among them, and neither is a delimited clause (N1, + M1). So a clause beside the name changes neither whether the + connective joins nor how a letter in the name reads, and a letter + inside such a clause is the clause's word, read as it always was + and not by this rule. The two questions part at one point: a + marker the pass declines and leaves as a word (M2) is a word, and + counts toward the three — but its case is still not asked. + That span is not this rule's alone: wherever another rule asks + the case question — the suffix slot (S2) and the post-comma slot + (C1) — it is asked of these same words, so the answer is taken + once and read where each of them stands. + "Juan y Eva Garcia" → given="Juan y Eva" + "Jose E Maria Santos" → middle="E Maria" + "jose e maria santos" → middle="e maria" + "Jose e Maria Santos" → given="Jose e Maria" + "JUAN GARCIA Y LOPEZ" → family="GARCIA Y LOPEZ" + "juan garcia y lopez" → family="garcia y lopez" + "john e smith" → middle="e" + "Juan y Garcia" → middle="y" · boundary + "Juan y Garcia née Jones" → middle="y" + "Juan and Garcia" → given="Juan and Garcia" + "Juan & Garcia" → given="Juan & Garcia" + "Mr. Jack and Jill" → family="Jack and Jill" + "Mr. Jack Jill" → given="Jack" + "Josep Carod i Rovira" → family="Carod i Rovira" + "Carod i Rovira, Josep" → family="Carod i Rovira" + "Josep i Rovira" → middle="i" + "John Quincy Smith i" → suffix="i" + "Carod i" → suffix="i" + "Josep Lluis Carod i III" → suffix="i III" + "Carod y de Rovira i" → middle="y" + "John й Smith" → middle="й" + "É. y É. Smith" → given="É." + Both exceptions are about the written FORM, not the word: the + three-word carve-out counts letters, so a symbol connective joins + at any length, and it reaches every single-letter connective the + vocabulary holds — Catalan i and Cyrillic и/і/й and Arabic و as + well as y and e. The initial reading counts letters too, and asks + one more question of them: a letter with no case at all (و) can be + written against nothing, so it never reads as an initial. A Cyrillic + capital has case but is not the shape an initial is written in — + Cyrillic abbreviates with a dotted letter — so in a name of more + than one case it joins, the reading #267 blessed, and in a name + of one case it takes the vocabulary's answer like any other cased + letter. A letter written as a base letter plus a combining mark + (NFD) that composes to one character is one letter to every + count here, and a period-marked initial written that way is one + initial (#665), so the last two examples above, both typed + decomposed, read as their composed twins. + Which single letters a tradition actually wants joined differs by + language, and the marked set is where that answer lives: "y" is + the commonest Hispanic compound and stays out of it, while "e" and + Catalan "i" are both common bare initials and are the two entries + shipped. A caller with Portuguese data removes "e"; a caller with + Catalan or Polish data removes "i"; a caller with Dutch data adds + the letter "y" to it. + The initial reading is visible beyond the fields, on the two + derived views: parse("john e smith").initials() gives "j. e. s." + and .capitalized() gives "John E Smith", where the connective + reading gave "j. s." and "John e Smith" — a connective + contributing no initial (R3) and keeping its lowercase (R4), + where an initial does neither. The v1 facade's initials() reads + the same parse, so HumanName("john e smith").initials() is + also "j. e. s."; #528 closed the split and decisions/P3 + records it. + Accepted: two 1.4.0 parity breaks, one in each direction. A bare + capital in a name written wholly in upper case no longer reads as + an initial, so "JUAN GARCIA Y LOPEZ" joins where 1.4.0 and + 2.0–2.3 read middle "GARCIA Y"; and a marked lowercase letter in + a name written wholly in lower case no longer joins, so "jose e + maria santos" reads middle "e maria" where they read given "jose + e maria". That is #383 answered with "bless" for the mixed-case + half — a bare Latin capital among mixed case still reads as an + initial, the shape the veto always tested, which is why #267's + Cyrillic reading is untouched — and answered with evidence for + the one-case half, where the vocabulary decides. + A maiden marker's run and every word after it are outside the + name's own words from the moment classify tags the marker, and + they stay outside whether or not the marker pass later declines + the marker and leaves it a word (M2). A declined marker therefore + does two things at once. It still does not count toward the case + class, which shows when it is the only differently-cased token, + so "JUAN Y GARCIA née" joins where "JUAN Y GARCIA née Jones" + keeps "Y" a name word. And it leaves any connective standing + after it to the mixed-case rule rather than to the one-case fork, + so a bare Latin capital there reads as an initial while its + lowercase spelling joins — the one shape in which the name's OWN + words read differently in its two one-case spellings. So "DR. + NÉE JUAN Y LOPEZ" reads middle "JUAN Y" with its "Y" an initial, + while "dr. née juan y lopez" reads family "juan y lopez" with its + own "y" joined. A clause's words are read as they always were, + and that is not an exception to this: such a letter still takes + a different tag in the two spellings, so its case repair can + differ between them, which is a repair difference and not a + reading of the name's own words. + Accepted rather than repaired: classify cannot know what group + will decline, and a clause's words are read by the clause's own + rules. + "Хосе И Мария Сантос" → given="Хосе И Мария" + H1 is the counting rule that shows the one-word clause today: a + title plus the join reads the whole join as the family, where the + same two words unjoined are two name words and H1 does not fire. + P1's leading run is the second (#395, landed): its run takes + the "Vega y Santos" join whole or stops before it. + history: decisions/P3 · interacts: H1, P1, M2, R1, R3, R4, S2, C1 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_comma.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/P4-particle-at-the-front-of-the-name.md b/docs/design/rules/P4-particle-at-the-front-of-the-name.md new file mode 100644 index 00000000..e2d17b98 --- /dev/null +++ b/docs/design/rules/P4-particle-at-the-front-of-the-name.md @@ -0,0 +1,26 @@ +# P4 — a particle at the front of the name + +P4. Rationale: a particle links forward from inside a name; at the + very front there is no name yet to be inside. + A particle in the name's leading position chains nothing: the + words stay separate, and any surname reading the name gets + comes from the fold (P1) or from position (O4), never from a + join. This is why a title before a leading particle changes + nothing (the title is not a name word), and why "Van Johnson" + is a given-name reading at all. An unlisted abbreviation before + the particle is as transparent as a listed title, since assign + reads it as one (H2). A word in the leading titles that is both a + title and a particle puts the name's leading position on itself, + so the particle behind it is inside a name ('Freiherr von Berg'); + of several such words the last does, so no particle chains inside + the titles (#624). The titles are those written, before H3 gives + the last one back to the name, so each such word reads as a title + unless H3 gives it back. + "Van Johnson" → given="Van" + "Sir de Mesnil" → pieces=[["Sir"], ["de"], ["Mesnil"]] + "Xyz. van Johnson" → given="van" + "John van der Berg" → pieces=[["John"], ["van", "der", "Berg"]] · boundary + "Freiherr St John Smith" → title="Freiherr St" + "Freiherr St John Smith" → given="John" + history: decisions/P2 · interacts: P1, P5, H2, H3 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py + diff --git a/docs/design/rules/P5-bound-given-names.md b/docs/design/rules/P5-bound-given-names.md new file mode 100644 index 00000000..4e482e04 --- /dev/null +++ b/docs/design/rules/P5-bound-given-names.md @@ -0,0 +1,115 @@ +# P5 — bound given names + +P5. Rationale: some given-name words are incomplete alone — "abdul" + is a bound form that the next word completes. + A recognized bound given-name word joins the word after it into + one given name. It needs a name word to spare, so two name words + alone do not join — the second is the family name — except after + a family comma, where the family is already fixed, or after a + given-name title, which asserts that a given name follows (H1): + there the two words join and the name has no family — two name + WORDS, so neither a particle chain (P2), which is the family + name, nor a suffix word is joined. A run of several titles is + read as H1 reads it, by the title that does the addressing, + which is its last. Where the join fires, a bound word that is + also a particle is read as the bound word: the join outranks the + leading-position reading (P4) and no fork is reported; where the + reserve blocks the join, P4's reading and its fork stand. Where + the word + is BOTH bound-given and suffix vocabulary, position decides and + both readings survive: leading, it is the bound word; trailing, + it is the suffix (S2). In the given slot after a family comma the + suffix reading wins. The marker and the words it will take are + not among the words to spare: they leave the name, so counting + them asks the question about a name that will not exist. The join + never absorbs a marker standing as a word of its own — a marker + is not a name word (M2) — nor a word of the unambiguous suffix + vocabulary (S2), wherever position will then place it, nor a + title word a trailing rule reads as a title: one the trailing + title run (H5) takes from the given part after a family comma, + read as the comma-less name reads it, or any title word in a part + past the second comma (C2); a bare + ambiguous acronym is a name word wherever the peel does not take + it; a marker left as a word that a particle join (P2) has already + taken travels with that join. A title word standing in the name + is a name word (H3) and joins like one, and so is a particle the + chain has not taken (P2) — unless it is of the unambiguous + suffix vocabulary too (vd, mc), which the join declines as the + suffix piece it is: + the pair is a given name whatever tag the word carried, and + after a family comma the join runs before the trailing + particle's attachment (P6) sees the name. What + there is to spare is what the name's trailing reading leaves: + the trailing suffix run (S2) and the trailing title run (H5), + each read over what the other leaves until neither takes + anything more, are read once, before the join is tried, and the + name words they leave are the words to spare. A trailing + roman numeral, or a bare acronym the run takes, or a trailing + title word the run takes, is no + word to spare. The join joins two name words into one and, read + after the trailing run, changes no suffix reading: a word the + run left as a name word stays one, whatever the join makes of + the word in front of it. After a + family comma the family is fixed and the joined pair is the given + whatever follows, so the reserve there reads no peel: the join + stands whenever the word after the bound word is a name word. + "abdul salam ahmed salem" → given="abdul salam" + "abd Allah Smith" → given="abd Allah" + "Salam, abd Allah" → given="abd Allah" + "abd Allah" → given="abd" + "Smith, Abd" → suffix="Abd" + "mohamad ali smith" → given="mohamad" · boundary + "Sheik abdul salam" → given="abdul salam" + "Sheik abdul salam Jr" → given="abdul salam" + "Sheik Abu Bakar" → given="Abu Bakar" + "Abu Bakar Salim" → given="Abu Bakar" + "Dr. abdul salam" → family="salam" · boundary + "Sir abdul van der Berg" → family="van der Berg" · boundary + "Sheik abdul Jr Smith" → given="abdul" · boundary + "Abu Bakar" → given="Abu" · boundary + "abdul Smith V" → family="Smith" + "abdul Smith V" → suffix="V" + "abdul Smith Jr V" → family="Smith" + "abdul Smith Jr Ma" → given="abdul" + "abdul Smith Jr Ma" → suffix="Jr Ma" + "abdul Smith Ma" → given="abdul Smith" + "abdul Smith Berg Ma" → middle="Berg" · boundary + "abdul Sir Smith Berg" → given="abdul Sir" + "Berg, abdul van" → given="abdul van" + "Berg, abdul vd" → family="vd Berg" + "Berg, abdul Prof." → given="abdul" + "Berg, abdul Prof. Ali" → given="abdul Prof." · boundary + "John Smith, Jr., abdul Dr. Smith" → title="Dr." + "abdul Jr Smith Berg" → given="abdul" + "abdul Jr Smith Berg" → middle="Jr Smith" + "abdul Ph. D. Smith Berg" → suffix="Ph. D." + "abdul V Smith" → given="abdul V" · boundary + "abd Berg née Jones" → family="Berg" + "abd Allah Smith née Jones" → given="abd Allah" + "abd née Jones" → given="abd" + "Berg, abd née Jones" → given="abd" · boundary + Accepted: after a family comma the join stands though, unjoined, + assign would read the word it takes as the suffix — the family + is fixed there and the pair is the given; and a bare ambiguous + acronym the peel does not take joins as any name word does. + "Berg, abdul V" → given="abdul V" + "abdul Ma Smith" → given="abdul Ma" + Accepted: a given-name title plus a bound pair leaves the family + empty, as H1 accepts for "Sir John" — the input names no family. + Under a family-first order the joined pair is the family and the + given name is empty instead, as "Sir John" reads there: the join + is order-blind, and H1's exemption never runs under that order. + "Sheik abdul salam" → family="" + "Sheik abdul salam" family-first → family="abdul salam" + "Sheik abdul salam" family-first → given="" + "Sheik abdul salam" family-first-given-last → family="abdul salam" + Accepted: the given part after a family comma reads the words + behind a title by its own walk, and where that walk reads them + otherwise than the comma-less name does, the join is declined and + no title is read — a shape-only numeral (VI to X), or V before a + third comma part, stays a name word there, and a title word that + is also a particle is attached by P6 (`Berg, abdul Prof. VI` + reads middle 'Prof. VI'; `Berg, abdul St.` reads family 'St. + Berg'), each as the same name with an ordinary given word reads. + history: decisions/P5 · interacts: C2, S2, M2, H1, H5, P2, P4, P6 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/P6-trailing-orphan-particle.md b/docs/design/rules/P6-trailing-orphan-particle.md new file mode 100644 index 00000000..401025d9 --- /dev/null +++ b/docs/design/rules/P6-trailing-orphan-particle.md @@ -0,0 +1,200 @@ +# P6 — the trailing orphan particle + +P6. Rationale: a particle ending the name has nothing to link + forward to, so it is not doing a particle's work there. What it + is doing instead is decided by what the WRITING says, not by the + word: where something has already named the family AND left the + particle in a position that means nothing — after a family comma, + or in the middle of a FAMILY_FIRST reading — it belongs to that + family, which is how Dutch and Flemish names are listed + ("Beethoven, Ludwig van", the tussenvoegsel trailing the given + name but belonging to the surname). Where nothing has, it is + read where it stands, and a particle CAN be a given name there: + the never-given vocabulary supplies the reading position leaves + open, and forbids nothing (parse("de") reports given "de", and + so does every other word in that set). + Where a family comma has already named the family, a particle + ending the name attaches to that family name and is written + before it — provided at least one given word remains, so that a + name whose only given word is the particle keeps it (the + words-to-spare test S2 applies to ambiguous suffixes). A + post-nominal is written BEHIND the particle in this listing, so + it does not end the name for this purpose: the run is found by + looking past trailing words that hold no name — unless such a + word is itself particle vocabulary, which makes it part of the + run rather than something to look past. Where + the word is BOTH a particle and suffix vocabulary, this + attachment outranks the suffix reading (S2): a trailing + abbreviation after a family comma is the tussenvoegsel far more + often than the decoration it collides with. Three exceptions. The + first is where the capitals speak: a word of the AMBIGUOUS + credential class, written in capitals in a name written in more + than one case, reads as the credential and this attachment stands + down — unless a particle stands immediately in front of it, the + two being one particle run by then, which this rule takes whole. + The second is S2's company: such a word standing behind an + unambiguous credential in one run of suffix words reads as the + credential in any spelling and any case, reported as S2's + credential fork. The third is a lone word of both the particle and + the UNAMBIGUOUS suffix vocabulary standing INSIDE a run read as + post-nominals — one in front of it and another behind, a + credential or a generation, whatever made each one, and a title + between it and the one in front transparent (H5): the + tussenvoegsel stands right behind the given name, so a + post-nominal between them says the word is not one. It does not + end the name, and keeps the + post-nominal reading in any case, reported as S2's credential + fork. With nothing behind it, it ends the name and attaches, and + two such words side by side are one particle run, which this rule + takes whole. Every other spelling of such a word attaches as + it did before, and the kind rule below gives it this rule's + particle fork rather than S2's credential one. In a name written + wholly in one case, with nothing in front of the word to speak + for it, the two readings cannot be told apart and the particle + keeps it, which is right about a Portuguese record and wrong + about a credential; the report is how a caller finds the second. + "Jong, Anke de" → family="de Jong" + "Beethoven, Ludwig van" → family="van Beethoven" + "Berg, Jan vd" → family="vd Berg" + "Berg, Jan van der" → family="van der Berg" + "Vega, Juan de la" → family_particles="de la" + "Beethoven, Ludwig van" → family_base="Beethoven" + "Beethoven, Ludwig van" → family_particles="van" + "Nguyen, Van" → given="Van" · boundary + "Doe, John DO" → suffix="DO" · boundary + "Doe, John Do" → family="Do Doe" + "SMITH, JOHN DO" → family="DO SMITH" + "Doe, John van DO" → family="van DO Doe" + "DOE, JANE PHD VD MA" → suffix="PHD VD MA" + "doe, jane phd vd ma" → suffix="phd vd ma" + "DOE, JANE PHD VD" → family="VD DOE" · boundary + "DOE, JANE MA VD PHD" → suffix="MA VD PHD" + A particle an index form writes apart with its elision or its + hyphen still on it is a particle like any other here, where the + vocabulary lists that spelling — the elided "d'" and the Arabic + article in its plain and assimilated spellings, "al-" among them + — and of + the never-given kind, since written apart the word is the article + and nothing else. It joins with a space like every particle; + written glued to the family ("d'Ivoi") it is one word and this + rule never sees it. + "Ivoi, Paul d'" → family="d' Ivoi" + "Rashid, Harun al-" → family="al- Rashid" + "Zaʿīm, Ḥusnī az-" → family="az- Zaʿīm" + Without a comma, a declared family-first order has named the + family in the same way and the attachment fires there too — but + only where the run ENDS the name and stands in a MIDDLE — the one + position that means nothing for it, middles being further given + names, which a particle is not. Those two together name the + order without asking it: the default order ends with the FAMILY + and FAMILY_FIRST_GIVEN_LAST with the GIVEN name, so only + FAMILY_FIRST can put a name's last word in a middle at all. + Ending the name is not implied by the slot and has to be said. A + draft that tested the slot alone fired on 52 default-order names, + where a conjunction stops a particle's forward chain and leaves it + standing in a middle, and on 366 FAMILY_FIRST_GIVEN_LAST middles + with the given name still behind them. + Nothing is asked about the WORD, and that is what lets one rule + read both traditions: the same vocabulary that spells the Dutch + tussenvoegsel spells the Vietnamese given name, and the declared + order separates them where a comma cannot — one order, not both, + as the Accepted clause below records. + Nothing is re-laid-out either — under the order this can reach, + the roles run family, given, middle, middle…, so dropping a + trailing middle leaves every other piece where it was. That is a + property of FAMILY_FIRST alone, and it is the second thing the + two tests above secure. The family must hold a + base of its own, since a family that is all particles is not a + family written beside anything (R2). + "Jong Anke de" family-first → family="de Jong" + "Jong Anke de" family-first → given="Anke" + "Ménil Christophe de" family-first → family="de Ménil" + "Ménil Christophe de" family-first → given="Christophe" + "Beethoven Ludwig van" family-first → family="van Beethoven" + "Ménil Christophe de" family-first-given-last → given="de" + "Berg Jan de Jr." family-first → family="de Berg" + "Berg Jan de Jr." family-first → suffix="Jr." + "van Berg Jan de" family-first → family="van" · boundary + "Maria Luisa y de la Cruz" → family="la Cruz" · boundary + "de Anke van y" family-first-given-last → middle="van" · boundary + "Ménil de" family-first → given="de" · boundary + "Beethoven Ludwig van" family-first → ambiguities=("particle-or-given",) + "Jong Anke de" family-first → ambiguities=() · boundary + Accepted: an ambiguous particle attaches on the same terms as a + never-given one, so a Vietnamese name written in this listing + loses its given name — but only in the UNACCENTED + transliteration. Vân carries a diacritic and is not particle + vocabulary, so the correctly spelled name never reaches this + rule. It is the ASCII spelling that collides, and there the two + traditions write the same string. + "Nguyen, Thi Van" → family="Van Nguyen" + "Nguyễn, Thị Vân" → family="Nguyễn" + The fork is reported, in the kind naming the reading this + attachment OVERRODE — what the run was read as before it fired, + not what its words are. Where the run was read as a post-nominal, + which is the S2 reading this rule outranks, the report is + suffix-or-name. Otherwise, where the run holds an ambiguous + particle, the reading overridden is that word as a name word, and + the report is particle-or-given. Where the attachment overrode + neither — a never-given particle read as a name word, which it + could never have been — nothing was decided and nothing is + reported. + "Nguyen, Thi Van" → ambiguities=("particle-or-given",) + "Berg, Jan vd" → ambiguities=("suffix-or-name",) + "Jong, Piet de" → ambiguities=() · boundary + Accepted: the colliding spelling has a format that reads + correctly, and it is ONE order, not both — FAMILY_FIRST_GIVEN_LAST, + which is the order this name is written in. A line asserting what + it does under FAMILY_FIRST stood here until #467 and was removed + rather than updated: the name is not written in that format, so + its reading there is wrong by construction and pins nothing worth + keeping (#470). + "Nguyen Thi Van" family-first-given-last → family="Nguyen" + "Nguyen Thi Van" family-first-given-last → given="Van" + Accepted: under the DEFAULT order a comma-less name's written + shape is not settled — "Jong Anke de" may be a misformatted + listing, and a bare "Jong de" may be a given name beside a + particle — so nothing there names the family, the positional + reading stands, and the same input reports family "de" here. + That is the reading for a particle the UNAMBIGUOUS suffix + vocabulary does not also claim, which is all but two of them. + Where the word is both a particle and UNAMBIGUOUS suffix + vocabulary — vd and mc — nothing has named the family, so the + attachment above never fires and S2 takes the trailing word as + a post-nominal, even though that leaves no family name at all: + a bare "Donald mc" reports suffix "mc" with an empty family, + and so does "Smith vd". The comma is the contrast, and it is + where the precedence stated above acts, "Berg, Jan vd" reading + family "vd Berg". `do` follows `de` rather than `mc`, sitting + in the AMBIGUOUS acronym half, where S2's words-to-spare test + leaves a two-word name its family. + "Jong Anke de" → family="de" + Accepted: the precedence over S2 is stated for the shape, so it + sweeps in every word that is both particle and suffix + vocabulary — today vd, do and mc. Only vd's reading was + weighed; mc inherits it, which is the shape's cost and was + decided rather than tracked — #454 closed by design 2026-09-07 + (decisions/suffix-acronym-collisions). `do` sits in the + AMBIGUOUS acronym half and was already read as a name word + there, so the precedence decides nothing for it — and because + the report names the reading OVERRIDDEN, `do` reports + particle-or-given while vd and mc report suffix-or-name. The + same distinction decides a run of several words: a run read as + name words reports on its ambiguous member if it has one and + otherwise reports nothing, whatever its words could have been + read as standing alone. Both are pinned in tests/v2/cases.py. + Accepted: a bound given word ahead of the trailing particle takes + it as its pair first (P5), so the attachment never sees it — + unless the particle is of the unambiguous suffix vocabulary too + (vd, mc), which the join declines and the attachment then takes. + "Berg, abdul van" → given="abdul van" + "Berg, abdul vd" → family="vd Berg" + The comma site above and the no-comma family-first site earlier + in this rule are checked against each other directly, as a pair, + rather than only against their own examples: tests/v2/test_order_correspondence.py + generates both writings and asserts they parse alike, plus a + negative-control sweep pinning the disagreeing set the precedence + bullet above names. A change that breaks one side of that pair + should expect that test, not this file, to say so first. + history: decisions/P6 · interacts: A1, C1, H5, P1, S2, P5, M2 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/P7-one-letter-particle-against-the-initial-it-spells.md b/docs/design/rules/P7-one-letter-particle-against-the-initial-it-spells.md new file mode 100644 index 00000000..4081c0d4 --- /dev/null +++ b/docs/design/rules/P7-one-letter-particle-against-the-initial-it-spells.md @@ -0,0 +1,23 @@ +# P7 — a one-letter particle against the initial it spells + +P7. Rationale: a one-letter particle is spelled with the same letter + as an initial, and the period is what tells them apart. An + initial is an abbreviation and is written with one; a particle is + a whole word and is not. The Irish Ó is never written with a + period, while Ó. is the initial of Óscar, Ólafur or Ólöf. + A one-letter word written with its period is an initial and not a + particle, whatever particle the same letter spells; written bare, + it is the particle. It is an initial in every position, so it + neither opens a surname, nor joins a chain, nor attaches to the + family behind a comma. + "Juan Ó. Pérez" → middle="Ó." + "J. Ó. Pérez" → middle="Ó." + "Ó. Pérez" → given="Ó." + "Pérez, Juan Ó." → middle="Ó." + "Juan Ó Pérez" → family="Ó Pérez" · boundary + Accepted: an accented initial written WITHOUT its period reads as + the particle. Such initials normally carry the period, and the + bare letter is exactly how the particle is written. + "Juan Ó Pérez" → middle="" + history: decisions/P7 · interacts: P1, P2, P3, P6, R3, R4 · implemented: nameparser/_lexicon.py, nameparser/_pipeline/_classify.py + diff --git a/docs/design/rules/R-rendering-and-views.md b/docs/design/rules/R-rendering-and-views.md new file mode 100644 index 00000000..782601df --- /dev/null +++ b/docs/design/rules/R-rendering-and-views.md @@ -0,0 +1,4 @@ +## Rendering & views (R) + +Background: parsing produces words with roles; every string a caller reads is assembled from those words on request. Nothing about rendering changes the parse, and nothing about reading a field mutates anything. The default view renders '{title} {given} "{nickname}" {middle} {family} ({maiden}) {suffix}' — the choice and its declined née-template alternative are decisions/render-default. + diff --git a/docs/design/rules/R1-fields-are-views-of-the-parsed-words.md b/docs/design/rules/R1-fields-are-views-of-the-parsed-words.md new file mode 100644 index 00000000..8cac089b --- /dev/null +++ b/docs/design/rules/R1-fields-are-views-of-the-parsed-words.md @@ -0,0 +1,42 @@ +# R1 — fields are views of the parsed words + +R1. Rationale: a field is a way of reading the parse, not a stored + string. + Every field is a view computed from the parsed words at read + time, joining its words in written order — except folded family + words — O3's fold and, since #379, P6's attached tussenvoegsel — + which render before the rest of the family wherever they stood in + the string. + Words are separated as the writer separated them. The suffix view + is the one place this is visible, because it is the only field + that can hold parts the writer comma-separated: a run of + post-nominals written with spaces renders with spaces, and one + written with commas keeps them. The separator is the comma the + writer typed, read off the words' positions once their roles are + known; a delimiter the configuration names is a separator too, + and so is any name word standing between two post-nominals. + "Dr. Juan Q. Xavier de la Vega III" → family="de la Vega" + "Hassan, Mohamad Ahmad Ali" middle_as_family → family="Ahmad Ali Hassan" + "Hassan, Mohamad Ahmad Ali" → family="Hassan" · boundary + "Smith, MD PhD" → suffix="MD PhD" + "John Smith MD PhD" → suffix="MD PhD" + "John Smith, MD, Bart" → suffix="MD, Bart" + "Smith, John, PhD - i Soler" extra_suffix_delimiters-dash → suffix="PhD, i Soler" + Accepted: a suffix value handed to revise() derives its entries the + same way, from the value's own commas, so a name's rendered suffix + revises back to itself wherever the value's words read as the + whole name read them — a glued CJK honorific peels off an initial + in a bare value where the whole name kept it glued, one corpus + name of the 372 with a suffix (decisions/C1, 2026-09-06, which + counted 368 of 1117 then; re-measured 2026-09-10 with this + bundle's rows in the corpora, 372 of 1156 distinct names and the + same single failure). A delimiter the configuration names parts a + value only where the value's own words, read as a name, give it a + tail segment for the core to be dropped on; a run of post-nominals + has none, with or without a comma of its own, so there the + delimiter stays a word of the run — write a comma at the boundary + instead. Stated without an example line because every line here + names an input string, and this shape needs a field revised after + the parse. + history: decisions/C1 · interacts: O3, P3, P6, R3, M2 · implemented: nameparser/_parser.py, nameparser/_pipeline/_post_rules.py, nameparser/_types.py + diff --git a/docs/design/rules/R2-all-particles-family-base-divergence.md b/docs/design/rules/R2-all-particles-family-base-divergence.md new file mode 100644 index 00000000..ae4dc3d5 --- /dev/null +++ b/docs/design/rules/R2-all-particles-family-base-divergence.md @@ -0,0 +1,34 @@ +# R2 — the all-particles family_base divergence + +R2. Rationale: callers need the surname with and without its + particles — sorting wants "Vega", display wants "de la Vega". + The family name splits into further views: the base (the family + without its leading particles) and the particles themselves. A + name part whose every word is particle vocabulary is a part where + none of them is doing a particle's work — nothing joins them to a + name — so they read as ordinary name words: they anchor the base + and leave the particles view. "Every word", not "standing alone": + a two-particle run has neither word alone and both are name words + there. Position decides that, not vocabulary; whether the word is + borne as a surname somewhere does not enter into it. + "Dr. Juan Q. Xavier de la Vega III" → family_base="Vega" + "Dr. Juan Q. Xavier de la Vega III" → family_particles="de la" + "Anh Do" → family_base="Do" + "Juan van der" → family_base="van der" + "Juan van der" → family_particles="" + "Juan de la Vega" → family_base="Vega" · boundary + "Juan de la Vega" → family_particles="de la" + "Sean O'Connor" → family_base="O'Connor" + Accepted, and the invariant it exists to hold: a non-empty + family always has a non-empty base. A particle needs a base to + attach to, so a family that is all particles is a family whose + words are not acting as particles. + "Del Toro" family-first → family_base="Del" + Accepted: the test runs after every rule that moves a token + between parts, so O3's fold decides it too — a middle folded into + the family can leave the family all particles, or can give a + trailing particle the name word it was missing. + "Anh Van Do" middle_as_family → family_base="Van Do" + "Nguyen, Van Le" middle_as_family → family_particles="Le" + history: decisions/R2 · interacts: R3 · implemented: nameparser/_types.py, nameparser/_pipeline/_post_rules.py, nameparser/_facade.py + diff --git a/docs/design/rules/R3-initials-honor-the-fold.md b/docs/design/rules/R3-initials-honor-the-fold.md new file mode 100644 index 00000000..2b1069f0 --- /dev/null +++ b/docs/design/rules/R3-initials-honor-the-fold.md @@ -0,0 +1,96 @@ +# R3 — initials honor the fold + +R3. Rationale: initials abbreviate the person's name words; titles, + suffixes, particles and nicknames are not name words. + Initials take the first letter of each given, middle, and base + family word; titles, suffixes, particles and nicknames + contribute nothing — except the particles of a part whose every + word is one, which are not acting as particles there (R2) and + initial like any other name word. A connective contributes + nothing where it is joining: a part holding another name word for + it to join — the part's working particles set aside — is a part + where the connective is doing a connective's work and is no name + word of its own. A part holding nothing else is a part where it + is joining nothing, and there it initials like any other name + word, agreeing with the base. + The first letter is the one the word's composed spelling begins + with, written in the form the writer used: a letter written as a + base letter followed by its combining accent initials whole, and + so does a syllable written as its separate parts. + The question is asked of the WHOLE PART, never of a word count. A + part of two words where one is the connective is still a part + where it joins — the base of "Jon Dough and" is "Dough and" and + its initials are "J. D." — and a part of three is no different. + One rule for every group. A connective written among given names + is a connective there too and contributes nothing where it joins, + so the group that was pinned-but-undocumented is now specified by + the same sentence as the other two. + Which words a group contributes is one question; the ORDER they + contribute in is a second, and its answer is the field's. Each + group initials in the order its field reads — written order, + except folded family words, which initial before the rest of the + family exactly as they render before it (R1). Stated here rather + than left to R1, whose subject is every FIELD and which this view + is not: what the view takes from the field is the ORDER it reads, + and about order the two never disagree. Membership is the first + question above and they can differ there — the family field of + the first example line below is de la Vega, where the initials + take its base — so this clause settles order and nothing else. + "Dr. Juan Q. Xavier de la Vega III" → initials="J. Q. X. V." + "Anh Do" → initials="A. D." + "Nguyen, Van Le" → initials="V. L. N." + "Hassan, Mohamad Ahmad Ali" middle_as_family → initials="M. A. A. H." + "Sean O'Connor" → initials="S. O." · boundary + "émile zola" → initials="e\u0301. z." + "マイケル ジャクソン" → initials="マ. シ\u3099." + A family that is ALL particles therefore contributes its words + rather than nothing: they are the base (R2), so they initial. + "Juan van der" → initials="J. v. d." + "Juan de y" → initials="J. y." + "John and Jane Smith" → initials="J. J. S." + "Duke of Edinburgh" → initials="D. E." + "Juan Velasquez y Garcia" → initials="J. V. G." + "Jon Dough and" → initials="J. D." + Accepted: this rule reads a part the parser read. A field set as + raw text after the parse carries no reading, and this view is + handed no vocabulary to supply one — it takes a format spec and + two separators and nothing else — so every word of such a field + initials, particles and conjunctions alike: a family set that way + to "de la vega" gives "j. d. l. v." where parsing the same + name gives "j. v.". Case repair IS handed a vocabulary, so it falls + back for the one question a word can answer on its own, and R4 + says which. Revising the field through the parser classifies it + and matches the parse in both of the parsed name's views. Stated + without an example line because every line here names an input + string, and this shape needs a field edited after the parse. + The v1 facade's HumanName.initials() is a second view of this + question and IS handed a vocabulary: it reads the parse's reading + wherever a word is backed by a parsed token, and falls back + wherever a word is not — a field set as raw text, or a name + restored from a v1 pickle or copied through the same state hooks. + The connective question falls back to the same helper case repair + uses; the particle question was never asked of the parse in this + view at all — `_is_particle` is a live + vocabulary lookup for every word, backed or spliced alike — so a + family spliced to "de la vega" initials "j. v." on this view + against "j. d. l. v." on the parsed name's own. So the + two views agree on WHICH WORDS initial in a parsed name; what + still differs there is GROUPING, the facade initialing a joined + run as one element, which is where the name "Ph. D., John" gives + a run-together "J. P D." on this view against "J. P. D." on the + other. Stated in prose and not as example lines because both + shapes need a field edited after the parse, or a rendering the + other view does not have. decisions/R3 carries what all of it + costs and where it is pinned. + Accepted: the given group's answer is a 1.4.0 parity break, and + a second one in the other direction sits beside it. A connective + among given names contributed an initial from 1.4.0 until this + rule reached it, so "John and Jane Smith" gives "J. J. S." where + every release through 2.3 gave "J. a. J. S." and 1.4.0 gave the + run-together "J a J. S."; and a connective holding a part alone + now contributes one, so "محمد و علي" gives "م. و. ع." on both + views, which is 1.4.0's own answer restored. The rule is one + sentence for all three groups, and taking it is what made the + four candidate answers this group once had into one. + history: decisions/R3 · interacts: O3, P3, P6, R1, R2, R4 · implemented: nameparser/_render.py, nameparser/_facade.py, nameparser/_pipeline/_post_rules.py + diff --git a/docs/design/rules/R4-case-repair-reads-the-unjoined-mark.md b/docs/design/rules/R4-case-repair-reads-the-unjoined-mark.md new file mode 100644 index 00000000..2c741561 --- /dev/null +++ b/docs/design/rules/R4-case-repair-reads-the-unjoined-mark.md @@ -0,0 +1,190 @@ +# R4 — case repair reads the unjoined mark + +R4. Rationale: case repair is a display concern, applied only on + request and never destructively. + Case repair returns a repaired copy and never mutates the parse. + Where it acts at all — R5 decides where — the copy honors the + casing a vocabulary entry records (PhD, BSc) and the Mac/Mc + convention (McDonald), not only ordinary word-by-word casing; a + particle is written in lowercase, but a particle the vocabulary + records a casing for takes that casing, the entry being how the + word is written wherever it stands; and + a part whose every word is particle vocabulary is repaired as + ordinary name words, since none of them is doing a particle's + work there (R2). A CONNECTIVE the parse placed among the name + words keeps its lowercase wherever it stands there, including + inside a part whose other words the unjoined mark has turned into + ordinary name words; one the parse read as the generation it also + spells is not a connective of this name at all, and is repaired + as the generation it was read as. That is this rule's own reading + and not a borrowing from R3: a connective that initials because + it joins nothing is still not written the way a name is written, + while a generation is written the way a generation is written. + The GENERATION is what that clause turns on and not the field: + a connective the suffix field merely holds, which the suffix + vocabulary does not know, was read as no generation and keeps its + lowercase there like any other connective — the third part of a + comma form is the shape that puts one there. + Repair changes case and nothing else: the repaired word is the + word as written, recased. A word is read whole however its letters + are encoded: a letter written as a base letter followed by its + combining accent is repaired as the same letter written whole, + and the repaired word keeps the encoding the writer used. A + vocabulary entry records casing as a + mask — its word's letters, each in the case it takes — and repair + lays it over the word as the writer punctuated it, so the one + entry for phd repairs phd to PhD and ph.d. to Ph.D.; the mask + applies wherever the word stands, whatever the parse read it as. + A single letter split off alone beside a full stop is written as + an initial and takes capitals whatever the mask says there — but + only where the mask itself writes that letter inside a run of two + or more letters: the writer split a chunk the mask keeps + together, so the split-off letter is an initial. A run is of + LETTERS only, so a digit ends one exactly as a full stop does. A + run of two or more letters beside a full stop takes the mask's + case letter by letter and is never an initial. Where the mask + writes the letter as a run of one too, there is no split to + repair, and the mask's own case stands. So ph.d. is Ph.D. under + the phd mask, and p.h.d. — three initials, not the mask's one run + 'PhD' — is P.H.D.; but h.c. under a caller's own h.c mask, which + spells each letter alone, stays h.c. + A name already written the way repair would + write it comes back unchanged, measured by repair's own + conventions rather than by the bearer's. A spelling written in a + single case is repaired even where its bearer meant it, because + nothing in the text marks it as a choice; where the text does + mark one, R5 defers to it. A credential acronym the exceptions map + does not carry is an initialism, so a single-case word the parse + put in the suffix role from the acronym vocabulary, or read as a + credential by its dotted shape alone (S3) or by its capitals + (S2), repairs to its all-caps spelling rather than a title-cased + one, and that repair + outranks the Mac/Mc convention where a word fits both (MCSE, not + McSe). A roman numeral the parse put in the suffix role is + written in capitals the way a generation is written, whether or + not the vocabulary lists it (VI as well as III). Those two + classes read the role and the mask does not, so a word of either + class that parsed as an ordinary name word repairs as that name + word unless the exceptions map carries it (qc mp gives Qc MP, + phd smith gives PhD Smith), and a suffix word that is neither an + acronym, a numeral nor an exceptions-map entry -- the + generational `jr`, `sr` -- keeps its title case. + Inside a hyphenated word, a part that is connective vocabulary + with a worded part on each side of it keeps its lowercase, the + hyphens being the writer joining the name around it + (Ortega-y-Gasset); at either end of the hyphenated word it is + repaired as ordinary name text. A single letter marked with a + period is read as an initial there, as the parse reads it, never + as the connective. A bare letter is not: the hyphen is the + writer's join, so the interior connective is read as one even in + a name written wholly in one case, where the same letter spaced + reads as an initial if the vocabulary marks it as reading both + ways (P3). + "juan mcdonald" → capitalized="Juan McDonald" + "Juan McDonald" → capitalized_forced="Juan McDonald" + "ANH DO" → capitalized="Anh Do" + "anh van do" → capitalized="Anh Van Do" + "SEÁN Ó MURCHÚ" → capitalized="Seán Ó Murchú" + "IVOI, PAUL D'" → capitalized="Paul d' Ivoi" + "zaʿīm, ḥusnī az-" → capitalized="Ḥusnī az- Zaʿīm" + "jacobus henricus van 't hoff" → capitalized="Jacobus Henricus van 't Hoff" + "ina binti navalamar" → capitalized="Ina binti Navalamar" · boundary + "john smith phd" → capitalized="John Smith PhD" + "john smith ph.d." → capitalized="John Smith Ph.D." + "JOHN SMITH PH.D." → capitalized="John Smith Ph.D." + "john smith p.h.d." → capitalized="John Smith P.H.D." + "john smith ph. d." → capitalized="John Smith Ph. D." + "john smith bsc" → capitalized="John Smith BSc" + "john smith psyd" → capitalized="John Smith PsyD" + "Dr. med. univ. Margit Popp, MSc" → capitalized_forced="Dr. Med. Univ. Margit Popp MSc" + "MSc Dr. med. univ." → capitalized_forced="MSc Dr. Med. Univ." + "Md Abdul Karim" → capitalized_forced="Md Abdul Karim" + "john smith md" → capitalized="John Smith MD" + "john smith m.d." → capitalized="John Smith M.D." + "john smith iii." → capitalized="John Smith III." + "john smith x.y.z." → capitalized="John Smith X.Y.Z." + "John Smith R.A.I." → capitalized_forced="John Smith R.A.I." + "john doe q.c." → capitalized="John Doe Q.C." + "Jack X.Y.Z." → capitalized_forced="Jack X.y.z." · boundary + "qc mp" → capitalized="Qc MP" · boundary + "john smith ed" → capitalized="John Smith ED" + "john smith vi" → capitalized="John Smith VI" + "john smith ix" → capitalized="John Smith IX" + "john smith iii" → capitalized="John Smith III" + "john smith, v" → capitalized="John Smith V" + "john smith xi" → capitalized="John Smith Xi" · boundary + "john smith, vi" → capitalized="Vi John Smith" · boundary + "john smith mba" → capitalized="John Smith MBA" + "john smith mcse" → capitalized="John Smith MCSE" + "john smith jr" → capitalized="John Smith Jr" · boundary + "John Quincy Smith i" → capitalized_forced="John Quincy Smith I" + "Carod i" → capitalized_forced="Carod I" + "Smith, John, and" → capitalized_forced="John Smith and" + "Doe, Jane, and Jr." → capitalized_forced="Jane Doe and Jr." + "juan de la vega" → capitalized="Juan de la Vega" · boundary + "josé garcía" → capitalized="Jose\u0301 Garci\u0301a" + "JOSÉ GARCÍA" → capitalized="Jose\u0301 Garci\u0301a" + "jose ortega-y-gasset" → capitalized="Jose Ortega-y-Gasset" + "JOSE ORTEGA-Y-GASSET" → capitalized="Jose Ortega-y-Gasset" + "maria silva-e-sousa" → capitalized="Maria Silva-e-Sousa" + "Jose Ortega-Y-Gasset" → capitalized_forced="Jose Ortega-y-Gasset" + "juan e-f smith" → capitalized="Juan E-F Smith" · boundary + "juan y-garcia" → capitalized="Juan Y-Garcia" · boundary + "donovan mcnabb-smith" → capitalized="Donovan McNabb-Smith" + "maria da-silva" → capitalized="Maria da-Silva" + "juan y-garcia-lopez" → capitalized="Juan Y-Garcia-Lopez" · boundary + "j.-e.-p. dupont" → capitalized_forced="J.-E.-P. Dupont" · boundary + "J-E-P DUPONT" → capitalized="J-e-P Dupont" · boundary + Accepted: a name written in one case whose hyphenated bare + initials spell a connective pays for the hyphen clause — its + interior letter is lowered as the connective though the writer + meant an initial, where the same letters spaced keep their + capitals. The hyphen is the writer's join, and reading the one + shape both ways would need the case evidence a one-case name does + not carry. + Accepted: the all-particle clause reaches a part the parser read. + A field spliced in as raw text after the parse carries no reading + of its own, so a family set that way to "de la" stays lowercase + where those same two words parsed from a name are repaired to "De + La". The acronym repair is the contrast: it asks the role and the + vocabulary and not a reading, so a suffix spliced in as "mba" is + repaired to "MBA" exactly as a parsed one is. + That is the boundary between splicing text into a field and + revising a field through the parser — revise() classifies the + value, so the repair follows it — rather than a gap between them. + It is also a limit and not a regression: a spliced-in family was + repaired exactly this way before the clause existed. Stated + without an example line because every line here names an input + string, and this shape needs a field edited after the parse. + Accepted, and the reason the boundary is drawn per question + rather than per field: the conjunction carve-out reaches a word + the parse read as a conjunction, and where the parse read nothing + at all it reaches what the vocabulary says. A word of that + vocabulary standing at either end of a longer written word is + not a conjunction there, because the hyphen clause above reaches + only a worded part with a worded part on each side, and an edge + part is outside that reach — but a field spliced in as raw + text was read by nobody, so repair asks the vocabulary and a + family set to "de y" + keeps its "y" lowercase. Whether a word is the conjunction or an + initial is a property of the word, which a vocabulary can answer; + whether a particle is acting as a particle is a property of the + whole part, which the parse settles and records; re-deriving it + needs a reading on every word of the part, and a spliced field + has none on any, so that half falls through to particle treatment + and the "de la" boundary above stands. Initials are the contrast + worth knowing, and R3 states it — but ask which initials view, + because the two answer oppositely. The parsed name's own view is + handed no vocabulary at all, so it falls back on neither question + and a spliced field's every word initials. The v1 facade's view IS + handed one: the connective question falls back to the helper this + rule uses, while the particle question is never asked of the + parse in this view at all — a live vocabulary lookup for every + word, backed or spliced alike — so a family spliced to "de la + vega" initials "j. v." there against the parsed view's "j. d. l. + v.". revise() classifies the value and crosses both questions, in + both of the parsed name's views: a middle revised to "e-f" + repairs to "E-F" as the parsed name does, where splicing the same + text in gives "e-F". + history: decisions/R4 · interacts: P3, R2, R3, R5, S3 · implemented: nameparser/_render.py, nameparser/_lexicon.py + diff --git a/docs/design/rules/R5-case-repair-gate.md b/docs/design/rules/R5-case-repair-gate.md new file mode 100644 index 00000000..7f6d5c9c --- /dev/null +++ b/docs/design/rules/R5-case-repair-gate.md @@ -0,0 +1,38 @@ +# R5 — the case-repair gate + +R5. Rationale: mixed case is evidence that the writer cased the name + deliberately, and a repair cannot tell a deliberate spelling from + a mistaken one. A name written wholly in one case leaves the + repair no such evidence to read either way — the writer who + always writes lowercase is indistinguishable from the writer who + could not be bothered — so repair proceeds there, which is a + choice about how to act absent evidence rather than a claim that + nothing can be lost by it. + Case repair acts only on a name written entirely in one case. + The suffixes are left out of that test: a suffix says nothing + about how the writer cased the name — III, PhD, Jr. are written + that way whatever the name's case — so the test reads every word + the parse did not put in the suffix role. A title is not left + out, and a cased title still holds repair back. A name whose + words outside the suffix are written in more than one case is + kept as it was written, and whether that casing is right does + not enter into it, unless repair was asked for anyway. The same + test is then asked of each suffix on its own: a suffix written + in more than one case is the writer's spelling and is kept as + written where repair was not forced; one written in a single + case is repaired like any other. + "juan mcdonald" → capitalized="Juan McDonald" + "SHIRLEY MACLAINE" → capitalized="Shirley MacLaine" + "Shirley Maclaine" → capitalized="Shirley Maclaine" + "Shirley Maclaine" → capitalized_forced="Shirley MacLaine" · boundary + "juan garcia III" → capitalized="Juan Garcia III" + "juan garcia PhD" → capitalized="Juan Garcia PhD" + "JUAN GARCIA Jr." → capitalized="Juan Garcia Jr." + "dr. juan garcia III" → capitalized="Dr. Juan Garcia III" + "Dr. juan garcia" → capitalized="Dr. juan garcia" · boundary + "Juan garcia III" → capitalized="Juan garcia III" · boundary + "john smith EdD" → capitalized="John Smith EdD" + "juan garcia Iii" → capitalized="Juan Garcia Iii" · boundary + "john smith EdD" → capitalized_forced="John Smith EDD" + history: decisions/R5 · interacts: R4, P3, S2 · implemented: nameparser/_render.py + diff --git a/docs/design/rules/README.md b/docs/design/rules/README.md new file mode 100644 index 00000000..ee145058 --- /dev/null +++ b/docs/design/rules/README.md @@ -0,0 +1,112 @@ +# Parsing rules + +This document is NORMATIVE, not descriptive: the rules state how names are written and what should happen when they are parsed, grounded in how people understand names — not in what the parser currently does. The parser implements these rules. Where it does not yet, the gap is a tracked deviation (`deviates:` marker below), not a counterexample. Statements are implementation-free: no stage names, no function names, no regexes. + +Authority, scoped: `tests/v2/cases.py` pins CURRENT behavior; this document states INTENDED behavior. A mismatch between them must be classified, never defaulted: either the rule is wrong (fix it here) or the parser is wrong (the example takes a `deviates:` marker and an issue). Where this document is silent, the behavior is pinned-but-undocumented — an extraction gap to close, not a specification, and not license to change the behavior. + +Rule IDs are stable forever: never renumbered, never reused. A retired rule keeps its ID with a one-line tombstone pointing at its decisions entry. Cross-references use the form `decisions/P2` / `mechanisms.md#SPANS`; a bare ID is never a citation. The `interacts:` field on a pointer line is advisory — the citation-integrity test checks the ID exists, not that the interaction is real. `implemented:` and `tracked:` are not advisory: every rule carries exactly one. `implemented:` names the modules honoring the rule, checked against the modules that cite it; `tracked:` names the issues that would ship a rule nothing implements yet, so a wholly-aspirational rule cannot sit here untracked, and a shipped rule cannot keep a stale tracking pointer. + +Every example line is EXECUTABLE. The grammar (its executable definition is `tests/v2/rules_doc.py`; `tests/v2/test_rules_doc.py` runs every line): + + "INPUT" [annotation] → field="value" [· boundary] + [deviates: #N (today: field="value")] + +An `annotation` names a policy, locale (`[ru]`), or extras gate (`[ja+segmenter]`) in the registry beside the test. `· boundary` marks the non-firing example every rule must carry: an input shaped like the rule's subject where its effect does NOT occur. That is usually the rule's OWN stated exception — H1's given-name title, P3's single-letter carve-out — not an input the rule never reaches, so the exception is executable rather than merely asserted. Or the rule declares `no-boundary: ` instead, so skipping the boundary is a recorded decision. `deviates:` states the INTENDED output on the example line while the marker records TODAY's output and the tracking issue; the runner asserts today's output strictly, so a parser change that closes the gap fails the suite until the marker is removed in the same PR. `grep -r deviates: docs/design/rules/` is the deviation backlog (deviations from statable rules — coverage gaps are a separate, larger category no grep can see, and contested vocabulary memberships a third, tracked as Open blocks keyed to the vocabulary set in decisions/). + +One class of rule stands outside the normative reading and declares it on a `tolerated: ` line beside its pointer line. Such a rule is ABOUT an input the writing systems it speaks about do not produce — a shape no convention writes, read best-effort because parsing is total over strings — and it describes what the parser does with that input instead of promising it. Its statement is still precedence-bearing and its examples are still executable and still run, but they illustrate rather than promise, and the behavior is changeable without notice. W3 is the only such rule today (the 2026-09-01 comma demotion). + +The marker's unit is the whole rule, so a clause that reaches past the tolerated shape does not belong in a marked rule — say it in a normative one. The marker changes what the rule promises, not what is watched: a marked rule's examples are in `tools/differential/corpus_rules.jsonl` like any other, so the snapshot shows when their reading moves (#664). + +## Not in scope + +- **Language detection.** The parser never infers a language from Latin-script text: transliteration destroys the signal ("Ali", + "Van", "Bin" each belong to several languages with conflicting + readings). Language-specific behavior is opt-in configuration. Script-conditional behavior exists only where the script itself settles the convention (see the W section). +- **Grammatical inflection.** Names inflect in many languages (vocative, genitive); this library neither produces nor consumes inflected forms. CLDR personNames draws the same line. +- **Validation.** Deciding whether a string IS a person's name is not parsing; `parse()` is total over strings and never rejects input. +- **Vocabulary completeness.** No shipped wordlist parses every name, and none is meant to. The promise is that every TYPE of vocabulary has a mechanism — markers, particles, titles, suffixes, bound given names — and that callers configure the members their data needs. A missing entry is a configuration answer, not a defect, and what the tests owe is one exercise per behavioral fork rather than one per entry (mechanisms.md#VOCABULARY-EXERCISES-FORKS). +- **Comparison.** matches()/comparison_key() are a value-API surface, not parsing; their design record is decisions/comparison-surface. + +## Index + +## [Titles & honorifics (H)](H-titles-and-honorifics.md) + +- [H1 — a title and one name word](H1-title-and-one-name-word.md) +- [H2 — the leading-abbreviation title](H2-leading-abbreviation-title.md) +- [H3 — the title run's floor](H3-title-runs-floor.md) +- [H4 — an input that is nothing but vocabulary still has to name somebody](H4-input-that-is-nothing-but-vocabulary-still-has-to-name-somebody.md) +- [H5 — the trailing title run](H5-trailing-title-run.md) +- [H6 — a title before a family comma](H6-title-before-a-family-comma.md) + +## [Particles & surname prefixes (P)](P-particles-and-surname-prefixes.md) + +- [P1 — lone particle fold](P1-lone-particle-fold.md) +- [P2 — particles join forward](P2-particles-join-forward.md) +- [P3 — connectives](P3-connectives.md) +- [P4 — a particle at the front of the name](P4-particle-at-the-front-of-the-name.md) +- [P5 — bound given names](P5-bound-given-names.md) +- [P6 — the trailing orphan particle](P6-trailing-orphan-particle.md) +- [P7 — a one-letter particle against the initial it spells](P7-one-letter-particle-against-the-initial-it-spells.md) + +## [Suffixes: generational & credentials (S)](S-suffixes-generational-and-credentials.md) + +- [S1 — a bracketed credential is not a nickname](S1-bracketed-credential-is-not-a-nickname.md) +- [S2 — the case signal at the suffix slot](S2-case-signal-at-the-suffix-slot.md) +- [S3 — credentials written with interior periods](S3-credentials-written-with-interior-periods.md) + +## [Nicknames & quoted names (N)](N-nicknames-and-quoted-names.md) + +- [N1 — the default delimiter pairs](N1-default-delimiter-pairs.md) +- [N2 — same-character quotes and shared-character conventions](N2-same-character-quotes-and-shared-character-conventions.md) +- [N3 — the lone-word nickname rule](N3-lone-word-nickname-rule.md) + +## [Maiden names (M)](M-maiden-names.md) + +- [M1 — delimited maiden names](M1-delimited-maiden-names.md) +- [M2 — the maiden-marker rule](M2-maiden-marker-rule.md) +- [M3 — the marker-led clause](M3-marker-led-clause.md) +- [M4 — a maiden marker makes the lone name word the family](M4-maiden-marker-makes-the-lone-name-word-the-family.md) + +## [Commas & structure (C)](C-commas-and-structure.md) + +- [C1 — the suffix-comma decision](C1-suffix-comma-decision.md) +- [C2 — parts past the second comma](C2-parts-past-the-second-comma.md) + +## [Name order (O)](O-name-order.md) + +- [O1 — East Slavic rotation](O1-east-slavic-rotation.md) +- [O2 — Turkic rotation](O2-turkic-rotation.md) +- [O3 — compound family names written unmarked](O3-compound-family-names-written-unmarked.md) +- [O4 — positional assignment and declared order](O4-positional-assignment-and-declared-order.md) +- [O5 — the lone name word's convention, and reporting it](O5-lone-name-words-convention-and-reporting-it.md) + +## [Scripts & writing systems (W)](W-scripts-and-writing-systems.md) + +- [W1 — unspaced CJK division](W1-unspaced-cjk-division.md) +- [W2 — the glued-honorific peel](W2-glued-honorific-peel.md) +- [W3 — the writer's divisions are respected](W3-writers-divisions-are-respected.md) +- [W4 — script-scoped order](W4-script-scoped-order.md) + +## [Tokens, initials & punctuation (T)](T-tokens-initials-and-punctuation.md) + +- [T1 — separators, not joiners](T1-separators-not-joiners.md) +- [T2 — the katakana middle dot divides like a space](T2-katakana-middle-dot-divides-like-a-space.md) +- [T3 — the interpunct's flank guard](T3-interpuncts-flank-guard.md) + +## [Ambiguity & tie-breaking (A)](A-ambiguity-and-tie-breaking.md) + +- [A1 — the ambiguity report](A1-ambiguity-report.md) +- [A2 — the empty name](A2-empty-name.md) + +## [Rendering & views (R)](R-rendering-and-views.md) + +- [R1 — fields are views of the parsed words](R1-fields-are-views-of-the-parsed-words.md) +- [R2 — the all-particles family_base divergence](R2-all-particles-family-base-divergence.md) +- [R3 — initials honor the fold](R3-initials-honor-the-fold.md) +- [R4 — case repair reads the unjoined mark](R4-case-repair-reads-the-unjoined-mark.md) +- [R5 — the case-repair gate](R5-case-repair-gate.md) + +## [Construction & configuration diagnostics (D)](D-construction-and-configuration-diagnostics.md) + +- [D1 — the segmenterless-activation warning](D1-segmenterless-activation-warning.md) +- [D2 — construction raises, parse never does](D2-construction-raises-parse-never-does.md) diff --git a/docs/design/rules/S-suffixes-generational-and-credentials.md b/docs/design/rules/S-suffixes-generational-and-credentials.md new file mode 100644 index 00000000..7a5eb21a --- /dev/null +++ b/docs/design/rules/S-suffixes-generational-and-credentials.md @@ -0,0 +1,4 @@ +## Suffixes: generational & credentials (S) + +Background: what follows a name is one of two different things — generational suffixes (Jr., III), which attach to the name itself, and credentials (PhD, MD, MBA), which are earned attachments. The suffix sets match one written word at a time: a multi-word entry there can never match anything and is warned about at configuration. Only two sets are exempt from that rule -- given_name_titles and, since #434, maiden_markers -- and neither is a suffix set, so within the suffix vocabulary the one-word rule is absolute. That limit is on STORAGE, not on the shape a name may have: adjacent suffix tokens are reassembled after matching (`_vocab.is_wholly_suffix`), so a multi-word credential is reachable as its component words -- `John Smith, MD PhD` has read suffix `MD PhD` since 1.4.0 -- and a caller reaches an unshipped one by adding the words it is made of rather than the phrase (#433). The eight multi-word entries that shipped dead for years span the suffix sets and the titles alike and are the Excluded story in decisions/vocabulary-exclusions. CLDR personNames keeps them as separate fields (`generation`, `credentials`) and formats them differently; this library currently reports both in one `suffix` field, a merge #326 examines. The vocabulary is largely split already: a generational word list and a credential acronym list, plus a short list of acronyms that are also ordinary names (MA, BA) and so are AMBIGUOUS as bare words. + diff --git a/docs/design/rules/S1-bracketed-credential-is-not-a-nickname.md b/docs/design/rules/S1-bracketed-credential-is-not-a-nickname.md new file mode 100644 index 00000000..cf6ca43a --- /dev/null +++ b/docs/design/rules/S1-bracketed-credential-is-not-a-nickname.md @@ -0,0 +1,12 @@ +# S1 — a bracketed credential is not a nickname + +S1. Rationale: brackets set off more than nicknames — credentials + are routinely written parenthesized after a name, and a + credential is recognizable by its form. + A bracketed clause whose content is suffix-shaped is not a + nickname: the brackets are dropped and the content reads exactly + as if written bare. + "Andrew Perkins (MBA)" → suffix="MBA" + "Andrew Perkins (Andy)" → nickname="Andy" · boundary + interacts: M2 · implemented: nameparser/_pipeline/_extract.py + diff --git a/docs/design/rules/S2-case-signal-at-the-suffix-slot.md b/docs/design/rules/S2-case-signal-at-the-suffix-slot.md new file mode 100644 index 00000000..c8cc3696 --- /dev/null +++ b/docs/design/rules/S2-case-signal-at-the-suffix-slot.md @@ -0,0 +1,329 @@ +# S2 — the case signal at the suffix slot + +S2. Rationale: generational suffixes and credentials are recognized + by vocabulary; an acronym that is also an ordinary name is only + unmistakably a credential when its periods are written. + A suffix never OPENS THE STRING: position outranks the + vocabulary match, so a suffix-shaped word written before anything + else reads as whatever its position and shape make it — a title + where it wears the abbreviation shape (H2), an ordinary name word + otherwise — and never as a credential belonging to a name that + has not been written yet. The string, not the name: a title, a + nickname or a bracketed clause standing first leaves the + credential reading intact, which is the ONE place this document + says "opening" and does not mean what P4 and the H Background + mean by it. A family comma is the other boundary: the comma has + already named the family, so a credential run may open the part + after it (C1), and the part before it is that family, less a + title run opening it (H6). + A trailing word of the suffix vocabulary reads as a suffix — + generational forms and credential acronyms alike, and an + ambiguous acronym written with its periods, one after each + letter or one after each of two or more letter chunks, counts + unambiguously; a single trailing period is the abbreviation + shape any word can wear and does not. The run at a name's + trailing slot is read ONCE, once the connectives have joined (P3) + and before the particle chain (P2) and the bound given-name join + (P5) are made, counting a particle run and the name words the + chain will take with it as the one name word it will make of them + — the words both particles and suffix vocabulary standing straight + behind a particle among them, while a word the reading weighs + counts as a word of its own wherever it stands; inside a + credential run a word stands on its own, a title word there + still a title — and neither + join reaches a word the run took. A + BARE ambiguous acronym is consumed only when the name has words + to spare — as the second of two words it stays the family + name — and at the slots that report, either reading carries the + ambiguity flag. Those slots are the trailing slot of a name, the + first slot after a family comma, the trailing slot of the GIVEN + part after that comma, and the trailing slot of a maiden + marker's clause (M2). A part beyond the second comma is not one + of them; what such a part does report is C2's to say. A word + this document + says is READ at one of those slots is not always a word one of + them reports: where a reading moves a word out of the slot that + asked about it, what reports is the slot it lands in, and that + may be none — M2 states the case where a clause does it. + After a family comma, a word of this class ending the GIVEN part + is read as the comma-less spelling reads a word ending the name, + and the count is not what decides it there. The comma has already + named the family and the first name word after it is the given + name, so the words to spare are there by construction and the + count says nothing — unless that first word is a never-given + particle, which is no given name and needs the next name word to + attach to (P1): that word is the surname the particle heads, not + a word to spare, so it reads as a name unless its capitals in a + mixed-case name say credential, as they do with no words to spare + anywhere: `SMITH, VD MA` reads as `Smith, vd Ma` does, `Smith, de + MA` keeps suffix `MA`, and `SMITH, VAN MA`, whose particle can be + a given name, keeps it and reads suffix `MA`. Otherwise a word + that ENDS that part reads as the + credential unless its WRITING says otherwise, and a word that + does not end it is never asked. "Ending the given part" reaches + past the credentials behind it and past a trailing title, which + is transparent to this reading as it is to the rest of S2's (H5); + a name word behind the word ends the reach, and so does a + PARTICLE the suffix vocabulary does not also claim: it belongs to + the family the comma already named and is taken there by a rule + that runs after this reading is made (P6), so `Doe, John MA do` + keeps its middle name though the capitals would otherwise have + taken the word. A particle the suffix vocabulary DOES claim is + looked past like any other credential, so `Doe, John MA vd` + reads suffix `MA`, with `vd` attaching behind it. Where the + reach ends, the word is an ordinary middle name, read in silence. + A particle in FRONT of the word takes it out of this slot as + well, and takes the word behind it too: where what follows the + particle reads as a name rather than a credential the two are + one name, so neither word is asked and neither reports — + `Doe, John van Ma` reads middle `van Ma` and `Doe, John DO Ed` + middle `DO Ed`, where `Doe, John van MA` reads suffix `MA` and + `Doe, John DO` alone reads the credential, both reporting. + Every word this slot does read reports the fork whichever way it + went, so a run of members all read as credentials reports once + for each, as the same words do without the comma — while a + member the writing keeps as a name + stops the reading there, and whatever stands in front of it is + name text, asked nothing and reporting nothing. One member of + this class is particle vocabulary as well, and where it stands + alone at this slot P6 decides it: the capitals take it as the + credential, as does an unambiguous credential in front of it (the + company below), and every other spelling attaches to the family, + reported there as P6's fork rather than as this one. Behind + another particle it does not stand alone — the two are one + particle run by then — and the run attaches whatever the capitals + say (P6). + Written case is the other evidence, and it speaks only in a name + written in more than one case: there a member of the ambiguous + set written in capitals reads as the credential even with no + words to spare, and one written in any other cased form that is + not wholly lower reads as the name even with words to spare. A + name written wholly in one case says nothing about any word in + it, and the count decides alone; so does a script with no case + to write in. After a family comma this evidence is SECOND at the + FIRST slot after it: the count of name words before the comma + decides there first (C1), and the case is read only where that + count leaves the word a name. C1 states the one exception: a part + of two or more suffix words and words of this class, every word of + the class listed and written in capitals in a mixed-case name, is + read on its capitals + before the count is asked, as the credential run behind one name + word as behind two ('Smith, MA MA' reads suffix 'MA MA'), + and only a word of the class opening the part reports, as the + first word after the comma ('John Smith, MA MA' reports on the + first 'MA' alone, where a counted flip reports over the whole + part). At the trailing slot of the given + part the comma has already settled the count, so the writing is + the only evidence there is. + Company is evidence that outranks both. A member of the ambiguous + set standing BEHIND an unambiguous credential in one run of + suffix words — members of the class between them passing the run + on — reads as the credential whatever its writing and whatever + the count, at every trailing slot this rule names: the degree in + front says what the run is. Only a credential in FRONT speaks; one + behind the member says nothing about it. A connective (P3), a + particle (P2) — a word of both the particle and the suffix + vocabulary heads the family name behind it — and a single-letter + roman numeral, in any case — one letter, the shape a bare middle + initial is written in, where a multi-letter one such as 'III' + speaks — speak for nothing and end the run; so does any name + word. A particle of this class standing behind a credential is + still spoken for: the exclusion is of the word in front. A word + of both the title and the suffix vocabulary standing in the given + part's leading title run is a title there and speaks for + nothing, and no credential behind it speaks for a word of this + class in that part: the title reading makes the next word the + given name, and from there the part reads as any given part does, + the given part's own company included. A part holding no such + word, or only words whose capitals decide them, reads as it did. + Anywhere else such a word speaks like any suffix word. + At the trailing slot of the given part this company outranks P6's + attachment, as the capitals do. It does not reach across a maiden + marker's clause (M2), whose name words stand between. A member + the company decides reports the fork as a counted pick does, + wherever it stands: at the trailing slots above, and in a part + after a family comma that the company leaves holding no name + word, which reads wholly as the credential run. A member its own + capitals already made the credential reports nothing new. + A credential after the name core starts a run to the end of its + part (#602). An unambiguous suffix word of two or more letters, + in any script, standing after two name words where no comma divides + the name or before a suffix comma, or after the given word in + the part after a family comma, makes every later word of that + part a suffix, except a title word, which reads as a title, and + in that given part the particles ending it, behind any + post-nominals, which P6 attaches to the family — other than a + lone member of the ambiguous class, which reads as the + credential. A lone particle is + not a name word for the count, 'de Mesnil' being one surname, + unless the credential stands right behind it, leaving it nothing + to join: then it counts as a name word of its own ('John von PhD + Jones'). A + particle run with the words the chain joins to it is one name + word, 'der la' and 'la Smith Secretary' each counting once (P2). + A title word + never starts the run, and neither does a member of the ambiguous + class, a single letter, a connective, a word that is also + particle or bound given-name vocabulary: a credential is the + writer's mark that the post-nominals have begun, and those carry + no such mark. An honorific or a generational word carries it as + a credential does, whatever its script. A name word the run + absorbs is reported `suffix-or-name`. + An unlisted word joins this same ambiguous class by SHAPE where + the caller asks for it. Two or more period-separated chunks is + one such shape, admitted by default (S3); an unlisted all-caps + alphabetic word of two or more letters, standing in a suffix + position of a mixed-case name and belonging to no wordlist, is + the other. By default that second shape is admitted in one + position only, the part after a comma with two or more name + words before it (C1): French and Korean records write the + SURNAME in capitals, and that convention puts the capitals at + the end of a name or before a comma, never after a comma behind + a full name. Everywhere else it is admitted only under the + caller setting the example lines below name, being a surname + there as often as a credential, which only the caller can know; + the cost of that setting is that a three-word name gives up its + family name to the acronym, while a two-word name keeps it — + there are no words to spare there, so the class is considered + and declined and only the fork is reported. The dotted shape does not take every promise of the + class with it: at the first slot after a comma, C1 reads paired + initials as the given name unless something speaks for them, and + makes a flip to the credential run in which no listed member takes + part in silence, unless paired initials speak only for each + other. + "John Smith Jr." → suffix="Jr." + "Antonio Pavlović mlađi" → family="Pavlović" + "Antonio Pavlović mlađi" → suffix="mlađi" + "Агриппина Младшая" → suffix="Младшая" + "جورج بوش الابن" → suffix="الابن" + "ロバート・ダウニー・ジュニア" → suffix="ジュニア" + "佐藤恵子 国会議員" → suffix="国会議員" + "陳子媚 二世" → suffix="二世" + "Питер Брейгель мл." → suffix="мл." + "Marie Fils" → family="Fils" · boundary + "Jiří Dienstbier ml." → family="ml." · boundary + "Pavlović, Antonio mlađi" → suffix="mlađi" + "Stariji Bruegel" → given="Stariji" · boundary + "John Smith M.A." → suffix="M.A." + "John Smith PhD" → suffix="PhD" + "John Ma" → family="Ma" · boundary + "Jack MA" → suffix="MA" + "Jack Ma" → family="Ma" + "JACK MA" → family="MA" + "JOHN SMITH MA" → suffix="MA" + "John Smith Ma" → family="Ma" + "wang meng" → family="meng" · boundary + "tran lac" → family="lac" + "john smith meng" → suffix="meng" + "john smith MEng" → family="MEng" + "Nguyen Van Lac" → family="Van Lac" + "Wang M.Eng." → suffix="M.Eng." + "John Smith PhD MEng" → suffix="PhD MEng" + "John Smith PhD Ed Ma" → suffix="PhD Ed Ma" + "Wang Ma PhD" → family="Ma" · boundary + "Smith, PhD MEng" → suffix="PhD MEng" + "Smith, Ms Ma" → given="Ma" · boundary + "Smith, PhD Ma" → suffix="PhD Ma" + "Smith, PhD Ma" → ambiguities=("suffix-or-name",) + "Smith, MD PhD Ma" → given="PhD" · boundary + "Doe, Jane PhD MEng" → suffix="PhD MEng" + "doe, jane v phd do" → suffix="v phd do" + "Smith, MA" → suffix="MA" + "Smith, Ma" → given="Ma" + "Doe, John MA" → suffix="MA" + "Doe, John Ma" → middle="Ma" · boundary + "Doe, John MA Smith" → middle="MA Smith" · boundary + "Doe, John DO" → suffix="DO" + "SMITH, JOHN DO" → family="DO SMITH" · boundary + "SMITH, VD MA" → family="SMITH VD MA" + "smith, de ma" → family="smith de ma" + "SMITH, VD DE LA MA" → family="SMITH VD DE LA MA" + "Doe, John MA JD" → ambiguities=("suffix-or-name", "suffix-or-name") + "Doe, John MA Ma" → middle="MA Ma" · boundary + "Doe, John MA Ma" → ambiguities=("suffix-or-name",) · boundary + "Doe, John van Ma" → middle="van Ma" · boundary + "Doe, John van Ma" → ambiguities=() · boundary + "Doe, John DO Ed" → middle="DO Ed" · boundary + "Doe, John DO Ed" → ambiguities=() · boundary + "John Smith XYZ" → family="XYZ" + "John Smith XYZ" unlisted_caps_suffixes-everywhere → suffix="XYZ" + "Jean DUPONT" unlisted_caps_suffixes-everywhere → family="DUPONT" + "Jean Pierre DUPONT" unlisted_caps_suffixes-everywhere → suffix="DUPONT" + "Jean Pierre DUPONT" unlisted_caps_suffixes-everywhere → family="Pierre" + "Jack Ma." → family="Ma." · boundary + "Ph. D. Van Johnson" → family="Van Johnson" + "Ph. D. Van Johnson" → title="Ph." + "Smith, Ph. D. Jr." → suffix="Ph. D. Jr." + "II Van Johnson" → given="II" · boundary + "Sir Ph. D. Van Johnson" → suffix="Ph. D." · boundary + "Ph. D., John" → family="Ph. D." · boundary + "John Smith PhD Jones" → suffix="PhD Jones" + "John Smith PhD Jones" → ambiguities=("suffix-or-name",) + "Eric H. Holder Jr. Attorney General" → title="Attorney General" + "John Smith PhD Prof. Ma" → suffix="PhD Ma" + "Smith, John PhD Jones" → suffix="PhD Jones" + "John von PhD Jones" → family="von" + "John von PhD Jones" → suffix="PhD Jones" + "Smith, John PhD de Jr." → family="de Smith" · boundary + "Mary Jane King Smith" → family="Smith" + "Josep Carod i Rovira" → family="Carod i Rovira" + "Mohamed Ali Abd Allah" → family="Allah" + "Antonio Pavlović mlađi Jones" → suffix="mlađi Jones" + "Pieter Bruegel stariji Jones" → suffix="stariji Jones" + "Ана Хорват старији Џонс" → suffix="старији Џонс" + "Гай Плиний Старший Иванов" → suffix="Старший Иванов" + "राम शर्मा साहब मोहन" → suffix="साहब मोहन" + "राम शर्मा साहब मोहन" → ambiguities=("suffix-or-name",) + "John PhD Smith" → family="Smith" · boundary + Accepted: a title before the credential keeps it a credential, + and the name loses its surname exactly as it did before this + clause — `Sir Ph. D. Van Johnson` reads given `Van Johnson` with + an empty family. That is 1.4.0's reading, and the clause cannot + reach it: the first piece of the NAME is not computable before + this decision, since the abbreviation test H2 uses is true of + `Ph.` itself, so a scan that stepped over titles would step over + the very piece being judged. + Accepted: an unambiguous suffix is consumed even when that + leaves no family name at all. + "Smith Jr." → family="" + Accepted: a name-leaning member with nothing in front of it to + speak for it ends the name, words to spare or not, and whatever + stands in front of it is name text. Where a credential does + stand in front, the company above decides instead, except in + one shape that keeps it out of reach and reads the member as the + name: a credential written split across two words with no comma + after the name. That reading is older than the company clause, + so it is witnessed by tests/v2/cases.py's + a_split_degree_in_front_is_out_of_the_walk rather than by a line + here, which would promise a reading this clause did not make. + A title standing between the credential and the member, and a + member a particle chain (P2) had taken, were two more such shapes + until the credential's run (#602) took both in: the rows + a_title_between_degree_and_member_keeps_it_a_name and + a_member_the_particle_chain_took_is_out_of_reach keep their ids + and now pin the run. + "Jack Wei Ma" → family="Ma" + "Jack Wei Ma" → ambiguities=("suffix-or-name",) + "abdul Smith Jr Ma" → suffix="Jr Ma" + Accepted: the title chain no longer takes the word this rule + needs, and the argument a descriptive note here asked for is + made. A title run leaves one NAME word standing and a + post-nominal cannot be it (H3, decided 2026-09-08), so `Dr King + Jr` reads family `King`, suffix `Jr` exactly as this rule + states, where it read title `Dr King`, family `Jr` before — + `Dr Smith Jr`, which always read as stated, was the contrast + that isolated the cause and now reads like its neighbour rather + than against it. What the floor cannot reach stays descriptive + and is small: where the run's whole content is the word given + back, no title is left to name anybody, so the word stands as + the name rather than as the family — `Dr Jr` reads given `Dr`, + suffix `Jr`, reporting `title-or-name` (H4), and `Sir Jr`, the + spelling this note first named, reads by the same branches — + given `Sir`, suffix `Jr`, the same kind reported — the run + being empty by then and no branch reading `vocab:given-title`. + The vocabulary half is decided + and unchanged (decisions/v1-xfail-triage: `king` stays a + title, for the addressing forms). + "Dr Jr" → suffix="Jr" + history: decisions/S2 · interacts: H1, H2, H3, H5, C1, C2, S3, P1, P2, P3, P5, P6, M2 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_classify.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_vocab.py + diff --git a/docs/design/rules/S3-credentials-written-with-interior-periods.md b/docs/design/rules/S3-credentials-written-with-interior-periods.md new file mode 100644 index 00000000..9fc069a2 --- /dev/null +++ b/docs/design/rules/S3-credentials-written-with-interior-periods.md @@ -0,0 +1,61 @@ +# S3 — credentials written with interior periods + +S3. Rationale: credentials are often written run together with + periods; the chunks between the periods are what carry the + vocabulary, and a word of several chunks that no vocabulary + knows is still written the way a credential is written. + A word with interior periods reads as a suffix when any of its + period-separated chunks is suffix vocabulary — except where + every chunk the vocabulary matches is a single ASCII character, + the roman numerals and the lone digit the vocabulary lists, + which are about generations rather than credentials. + A word of two or more period-separated chunks that no + vocabulary claims is read by POSITION instead, as a bare + ambiguous acronym is (S2): a credential where the name has words + to spare, a name word where it does not, either reading + reported at the slots S2 reports at, and the same at a comma — + which means the FIRST piece after a family comma, the word + trailing the given part after one, the word ending a maiden + marker's clause (M2), and the part before a SUFFIX + comma. The part before a FAMILY comma never reports, the comma + having already named it the family. C1 states two exceptions at + the first slot after a comma: paired initials ('M.J.') read as + the given name unless something in the part speaks for them, and + a flip to the credential run in which no listed member of the + class takes part is not reported, unless paired initials speak + only for each other. Case says nothing here — the + periods are the evidence — and three shapes are outside it: a + single trailing period is not this shape at all, a chunk that is + not wholly alphabetic is no acronym letter, and a word carrying + a script that writes no abbreviations is not wearing an + abbreviation's periods (H2 refuses the same word for the same + reason). This second half is a caller switch, ON by default, and + the example lines name it; turned off, such a word is name + material and the chunk rule above still decides the rest. + "John Smith Msc.Ed." → suffix="Msc.Ed." + "John Smith Msc.Ed." unlisted_dotted_suffixes-off → suffix="Msc.Ed." + "Doe, John Msc.Ed." → suffix="Msc.Ed." + "John Smith J.u.n.i.o.r." → suffix="J.u.n.i.o.r." + "John Smith Q.W.E.R.T." → suffix="Q.W.E.R.T." + "John Smith X.Y.Z." → suffix="X.Y.Z." + "john smith x.y.z." → suffix="x.y.z." + "John Smith X.Y.Z." unlisted_dotted_suffixes-off → family="X.Y.Z." + "Jack X.Y.I." → family="X.Y.I." · boundary + "John Smith Xyz." → family="Xyz." · boundary + "John Smith 1.4" → family="1.4" · boundary + "Doe, John X.Y.Z." → suffix="X.Y.Z." + "Jane Doe nee Smith X.Y.Z." → suffix="X.Y.Z." + Accepted: the initialless-script clause carries no example line + of its own. Every input that exercises it composes a script that + writes no abbreviations with a period that only a Latin + convention writes — `John Smith 田.中.` is the shape — and a + composed form no writing system produces is tolerated input + rather than contract, which an example of a normative rule would + promise. So the witnesses are the tolerated row + tests/v2/cases.py's + an_initialless_script_glued_into_periods_is_not_this_shape and + the CJK assertions in tests/v2/pipeline/test_vocab.py's + test_period_joined_vocab_retires_the_single_character_chunk. + W3 states the same precedence for the same reason. + history: decisions/S2 · interacts: S2, C1, H2, W3 · implemented: nameparser/_pipeline/_vocab.py + diff --git a/docs/design/rules/T-tokens-initials-and-punctuation.md b/docs/design/rules/T-tokens-initials-and-punctuation.md new file mode 100644 index 00000000..ea481d31 --- /dev/null +++ b/docs/design/rules/T-tokens-initials-and-punctuation.md @@ -0,0 +1,4 @@ +## Tokens, initials & punctuation (T) + +Background: every parsed name part is an exact piece of the input, located by its position — nothing rewrites the text before parsing. Some punctuation is a name divider only by convention of a particular writing system: Japanese writes name parts with a middle dot between them (マイケル・ジャクソン, 姓・名), while U+00B7 is both the Chinese divider for transcribed foreign names and the Catalan punt volat interior to legitimate words (Gal·la). + diff --git a/docs/design/rules/T1-separators-not-joiners.md b/docs/design/rules/T1-separators-not-joiners.md new file mode 100644 index 00000000..2eea4c24 --- /dev/null +++ b/docs/design/rules/T1-separators-not-joiners.md @@ -0,0 +1,12 @@ +# T1 — separators, not joiners + +T1. Rationale: a character that carries no name content (emoji, an + invisible directionality control) stands between words, not + inside them. + A name splits at whitespace and — unless the caller opts to + keep them — at ignorable characters: an ignorable character + separates its neighbors and never joins them. + "John😀Smith" → family="Smith" + "John😀Smith" keep-emoji → given="John😀Smith" · boundary + history: decisions/T1 · implemented: nameparser/_pipeline/_tokenize.py + diff --git a/docs/design/rules/T2-katakana-middle-dot-divides-like-a-space.md b/docs/design/rules/T2-katakana-middle-dot-divides-like-a-space.md new file mode 100644 index 00000000..4636558f --- /dev/null +++ b/docs/design/rules/T2-katakana-middle-dot-divides-like-a-space.md @@ -0,0 +1,12 @@ +# T2 — the katakana middle dot divides like a space + +T2. Rationale: the katakana middle dot exists to divide name parts + and appears in no native name. + The katakana middle dot and its halfwidth twin divide a name + like whitespace, always. + "マイケル・ジャクソン" → given="マイケル" + "高橋・一郎" → family="高橋" + no-boundary: the separation is unconditional; the + context-sensitive interpunct is T3's subject. + implemented: nameparser/_pipeline/_tokenize.py + diff --git a/docs/design/rules/T3-interpuncts-flank-guard.md b/docs/design/rules/T3-interpuncts-flank-guard.md new file mode 100644 index 00000000..9639a7dd --- /dev/null +++ b/docs/design/rules/T3-interpuncts-flank-guard.md @@ -0,0 +1,12 @@ +# T3 — the interpunct's flank guard + +T3. Rationale: U+00B7 is two marks in one codepoint — the Chinese + 间隔号 dividing a transcribed foreign name, and the Catalan punt + volat interior to words. + The interpunct divides a name only between two characters of a + classified East Asian script; anywhere else it is part of the + word. + "威廉·莎士比亚" → family="莎士比亚" + "Gal·la Serra" → given="Gal·la" · boundary + history: decisions/T3 · implemented: nameparser/_pipeline/_tokenize.py + diff --git a/docs/design/rules/W-scripts-and-writing-systems.md b/docs/design/rules/W-scripts-and-writing-systems.md new file mode 100644 index 00000000..20872811 --- /dev/null +++ b/docs/design/rules/W-scripts-and-writing-systems.md @@ -0,0 +1,4 @@ +## Scripts & writing systems (W) + +Background: script-conditional behavior is permitted exactly where the writing system itself — not statistics about it — settles the convention; a language can never be inferred from Latin-script text, because transliteration destroys the signal. The facts this section builds on: Chinese and Japanese both write the family name first in native script, so the script settles the order without knowing the language. Hangul is written by exactly one language and Korean family names are a small closed census set. Han text does not identify its language — a Chinese surname list would divide Japanese 高橋一郎 as 高 + 橋一郎 — which is why Han division is opt-in and there is no Korean pack to opt into. Hiragana never transcribes a foreign name (transcriptions are katakana alone), so kanji-plus-kana is a Japanese name in Japanese order, while wholly-katakana may be a transcribed foreign name already in given-first order or a Japanese name's reading written family-first, and nothing in the script says which. Katakana has a halfwidth form (ﾀﾛｳ for タロウ), the only kana systems without kanji could store, and legacy data still carries it: it is katakana, read the same way. A voicing mark typed as its own character after a kana — the dakuten or handakuten after a kana with no precomposed voiced form, as in ア゙イ, or the spacing ゛ and ゜ — voices that kana and belongs to neither syllabary, so it never changes the script of the word it stands in. A mark standing anywhere else, first in a word or after a character that is not kana, voices nothing, and what the parser makes of such a word is not promised. Real Chinese text is unspaced (毛泽东); the spaced 毛 泽东 is an artifact. A fuller narrative lives in docs/usage.rst's East Asian section. One fact carries its own consequence: none of the three writing systems marks the family name with a comma — position in the written form is what identifies it, so a comma standing between the family name and the given name is a listing convention carried in from elsewhere rather than a form the script produces. That is why the rule reading one (W3) is tolerated rather than normative. CLDR's own locale data says the same where a contrary convention would have had to appear: across its ko, zh and ja personName patterns not one of the 126 pattern strings carries a comma of any width, the surname-first referring patterns separating surname from given by a single space, and the only comma in reach belongs to the locale-neutral root's sorting format — a list-ordering format, which ko and zh override comma-free and ja all but one inherited slot (decisions/cjk-comma-demotion carries the pull verbatim, with its URLs, its commit and its date). A full stop of any width — the ASCII period, the fullwidth ．, the ideographic 。 and its halfwidth ｡ — glued after a script-written word is punctuation and not part of the word: it is invisible to the script reading and to the vocabulary, and it stays in the text on the word it arrived with, because no East Asian script writes an initial or an abbreviation with a period (#322, #323; decisions/cjk-full-stops). A stop glued BEFORE the word is punctuation to the vocabulary lookup, which folds both edges away, so .씨 is still the honorific. The classification fold that feeds the two division sites reads the trailing edge only, so a word wearing a leading stop is given no script at all and never becomes a surname site: .김민준 stays one whole word, given, no script rule reaching it. The honorific peel (W2) is not gated by that fold — the tail alone is its license — so it reads such a token regardless of a leading stop, and its own trailing-edge fold is what decides there: .김민준씨 peels to .김민준 and 씨, and .김민준씨. peels to .김민준 and 씨. (tests/v2/pipeline/test_script_segment.py's test_the_peel_reads_the_trailing_stop_only). + diff --git a/docs/design/rules/W1-unspaced-cjk-division.md b/docs/design/rules/W1-unspaced-cjk-division.md new file mode 100644 index 00000000..45a03cfc --- /dev/null +++ b/docs/design/rules/W1-unspaced-cjk-division.md @@ -0,0 +1,28 @@ +# W1 — unspaced CJK division + +W1. Rationale: hangul is monoglot Korean and its surnames are a + closed census set, so an unspaced hangul name divides at a + certain point; Han carries no such certainty by default. + An undivided word in the family position of a name written in + an activated script divides after a recognized surname, the + longest recognized surname first; where the vocabulary + recognizes nothing, an optional segmenter may divide instead, + and with neither the word stays whole rather than divide in a + wrong place. Korean division is active by default; Han division + is opt-in. A segmenter — unlike the vocabulary, which ignores + spacing but not the interpunct (the Accepted below) — is + consulted only for a name whose written form is wholly + undivided, a spaced honorific counting as a written division + (decisions/W3 records what that trade keeps and costs). + "김민준" → family="김" + "남궁민수" → family="남궁" + "남궁민수 지훈" → family="남궁" + "毛泽东" → family="毛泽东" · boundary + "毛泽东" [zh] → family="毛" + "高橋一郎" [zh] → family="高" + Accepted: a name the interpunct divides is already divided in + the sense that matters — division stands down entirely there, + vocabulary and segmenter alike (#298; decisions/T3). + "安东尼·陈志明" [zh] → family="陈志明" + history: decisions/W1 · interacts: W3 · implemented: nameparser/_pipeline/_script_segment.py + diff --git a/docs/design/rules/W2-glued-honorific-peel.md b/docs/design/rules/W2-glued-honorific-peel.md new file mode 100644 index 00000000..5440d5f9 --- /dev/null +++ b/docs/design/rules/W2-glued-honorific-peel.md @@ -0,0 +1,45 @@ +# W2 — the glued-honorific peel + +W2. Rationale: some East Asian honorifics glue directly onto the end + of the name (田中さん); a glued word peels off only if it could + never itself end a name, so the listed vocabulary carries its + own license and needs no other gate. + A listed honorific glued to the end of the name's last name + word splits off once and reads as a suffix. A part that is not + name text — a post-nominal word standing on its own — is never + the name's end: the split-off steps past it to the name word + behind, and never dissects it. + "田中さん" → suffix="さん" + "김민준씨" → suffix="씨" + "김민준 박사님" → suffix="박사님" + "马丁·路德·金씨" → suffix="씨" + "김지양" → suffix="" · boundary + "선생님" → family="선생님" · boundary + "王君" → family="王君" · boundary + "田中 社長" → suffix="社長" + "田中社長" → suffix="社長" + "陳怡君二世" → suffix="二世" + "田中部長" → family="田中部長" · boundary + Accepted: 部長 and its simplified spelling 部长 are vocabulary in + the spaced and after-comma positions only, and no peel splits + them off a glued name: 田中部長 keeps the word in the name, + because 長 alone is a borne given name and 阿部長 can be 阿部 + + 長. The other credential, office and generational words of #671 + (議員, 国会議員, 太平紳士, 医学博士, 二世, 社長, 課長 and their + variant spellings) met the glued bar when measured against real + names (#707) and peel. The Han segmenter's own division is not a + peel and exposes 部長 regardless: under the Chinese pack a + listed surname followed by it divides there and the word reads + as a suffix, so 李部長 reads family 李, suffix 部長, as 王博士 + already does. + Accepted: a longer office or degree word that ends in a listed + tail is split at it, whether glued or spaced, because the tail + is matched at the end of the token and the longer word is not + listed: 衆議院議員, 副社長, 代表取締役社長 and 獣医学博士 leave + their head (衆議院, 副, 代表取締役, 獣) in the name. Before the + tail was listed the whole word stayed in the name, which was no + more right (#707). + "田中 副社長" → given="副" + "岸田文雄衆議院議員" → family="岸田文雄衆議院" + history: decisions/W2 · interacts: W3 · implemented: nameparser/_pipeline/_script_segment.py + diff --git a/docs/design/rules/W3-writers-divisions-are-respected.md b/docs/design/rules/W3-writers-divisions-are-respected.md new file mode 100644 index 00000000..79cdf26a --- /dev/null +++ b/docs/design/rules/W3-writers-divisions-are-respected.md @@ -0,0 +1,72 @@ +# W3 — the writer's divisions are respected + +W3. Rationale: a family name declared by a comma is the writer's + own division, and re-dividing it would invent a boundary nobody + drew — but none of the East Asian writing systems declares a + family name that way (the Background above), so every input this + rule reads is a listing convention wrapped around a name whose + own script has already arranged it — the comma that declares the + family name, and the punctuation such a listing carries in with + it. What follows describes what the parser does with such input; + it does not promise it. + Under a family comma the pre-comma text is the family by + declaration and never divides, and the post-comma side is given + text with no family to find. A title run opening the pre-comma + text is H6's title, not family text. Only the honorific + split-off (W2) + crosses the comma, an honorific being no part of the name on + either side — the crossing is stated here rather than in W2 + because it is a claim about the comma, and the comma is the part + nobody's writing system produces. Behind a part C1 reads as + postnominal there is no family comma, and the name before it + divides as it would with no comma ('마틴 킹, MA' as '마틴 킹'); + the division, and with it the split-off below, follows C1's own + reading of the comma, taken on the name as written before the + division ('Dr 김민준, MA' divides as 'Dr 김민준' does). Which side + the split-off takes + it from is decided by the SHAPE of the part after the comma, not + by which of C1's two readings that part selects — a one-word + family reads the listing form either way. A part that is nothing + but suffix words is not the name's end: it is declined as the + site, and the split-off falls back to the part before the comma + and takes the honorific there, provided that part offers a site + of its own. Any other part — a title, a name word — IS the + name's end, and a glued honorific before the comma stays glued. + The vocabulary question is C1's own, asked without C1's + word-count condition: the two differ in what else they require, + not in what they ask of the words — the written-case evidence + S2 reads included, so a post-comma acronym the case leans + credential declines the side it stands on here exactly as it + reads as the credential there, and the honorific comes off the + part before the comma as it does behind a settled credential. + A period the listing leaves + behind is not what licenses the step past a post-nominal word, + and it is not ignored either. The step is W2's, taken on the + vocabulary alone and taken with no punctuation anywhere in the + input; a period on a SEPARATE post-nominal word rides along into + the suffix and moves no division (田中さん 様. divides where + 田中さん 様 does). A period glued to the honorific's OWN word + rides with the honorific: the split-off matches the listed tail + through it and cuts before it, so 田中さん. and 김민준씨. divide + where 田中さん and 김민준씨 do (#323; until 2026-09-10 each read + as a title, measured 2026-09-05 and pinned by nothing, which is + why the move was free), and a period-marked Han opener divides + as its stop-less spelling does, 田中. 太郎 reading family-first + — the reading H2's Accepted clause vetoes a title into, carried + here because an edge full stop makes it tolerated input like the + rest of this block. All three are case rows now, tolerated, and + the sentence reports them rather than promising them, even by + this rule's standard. + decisions/cjk-comma-demotion carries the parses. + "남궁민수" → family="남궁" + "지훈, 남궁민수" → given="남궁민수" + "남궁민수, 지훈" → family="남궁민수" · boundary + "田中さん, Dr." → family="田中さん" + "田中さん, PhD" → suffix="さん, PhD" + "Kim김민준씨, MA" → suffix="씨, MA" + "田中さん 様." → suffix="さん 様." + "김민준씨." → suffix="씨." + "田中. 太郎" → family="田中." + tolerated: native CJK writing has neither a family-comma convention nor an edge full stop of any width on a name word, so the five comma lines above and the three period lines under them illustrate current behavior — changeable without notice — rather than promise it; the line carrying neither, beside them, is W1's claim, which is normative. All eight stay watched by the snapshot (tools/differential/snapshot.jsonl, over corpus_rules.jsonl and corpus_cjk.jsonl), and the `tolerated` rows of tests/v2/cases.py pin them at HEAD. + history: decisions/W3 · interacts: W1, W2, C1, H2, H6 · implemented: nameparser/_pipeline/_script_segment.py + diff --git a/docs/design/rules/W4-script-scoped-order.md b/docs/design/rules/W4-script-scoped-order.md new file mode 100644 index 00000000..2f0fec6d --- /dev/null +++ b/docs/design/rules/W4-script-scoped-order.md @@ -0,0 +1,31 @@ +# W4 — script-scoped order + +W4. Rationale: Chinese, Japanese and Korean all write the family + name first in native script — the script settles the order + without knowing the language — while a wholly-katakana name may + be a transcribed foreign name already in its source order or a + Japanese reading written family-first, so its script settles + nothing. + A name written wholly in one East Asian script, or in the + kana-licensed Japanese repertoire, reads family-first whatever + order the caller declared; a wholly-katakana name keeps the + declared order. + "김 민준" → family="김" + "山田 太郎" → family="山田" + "高橋 みなみ" → family="高橋" + "山田 ﾀﾛｳ" → family="山田" + "マイケル ジャクソン" → given="マイケル" · boundary + "ﾔﾏﾀﾞ ﾀﾛｳ" → given="ﾔﾏﾀﾞ" · boundary + "ア゙イ タロウ" → given="ア\u3099イ" · boundary + Accepted: a name the interpunct divides keeps the declared order + — the divider itself marks a transcription (T3) — so the override + stands down there, and the name reads as a wholly-katakana one + does: in its written order by default, reversed by a declared + family-first order like any other name; the katakana middle dot + (T2) carries no such signal, so a name it divides still reads by + the script license. + "毛·泽东" → given="毛" + "威廉·莎士比亚" family-first → family="威廉" + "威廉・莎士比亚" → family="威廉" + history: decisions/W4 · interacts: T3 · implemented: nameparser/_pipeline/_assign.py + diff --git a/docs/modules.rst b/docs/modules.rst index 9ed5e9e5..5bb12bbc 100644 --- a/docs/modules.rst +++ b/docs/modules.rst @@ -52,7 +52,7 @@ Results ``conjunction`` (a joining word, "and"/"y"), ``initial`` (a word READ as an initial — initial-shaped in a script that HAS initials, "J." or "А.", never "씨.", or a marked single-letter connective in - a name written in one case, rules.md#P3), and ``joined`` (a + a name written in one case, rules/P3), and ``joined`` (a continuation of the token before it — within one merged piece the tag is role-blind and every view joins the pair with a space, so it renders "Ph. D." as one diff --git a/docs/release_log.rst b/docs/release_log.rst index 5136cbc5..2a1fb260 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -7,7 +7,7 @@ Release Log **Behavior Changes** - **Add junior and senior in other languages as suffixes.** ``Antonio Pavlović mlađi`` gives last ``Pavlović``, suffix ``mlađi``, where 1.4.0 through 2.4.2 gave middle ``Pavlović``, last ``mlađi``. The words come from 45 languages and are read when written as a separate word after the name. :issue:`661`, :issue:`679` - - **Remove mai from the default post-nominal acronyms, so a trailing Mai keeps the name.** ``Tran Thi Mai`` read in Vietnamese order (``FAMILY_FIRST_GIVEN_LAST``) gives given ``Mai``, middle ``Thi``, family ``Tran``, where 2.0.0 through 2.4.2 gave suffix ``Mai`` and no given name; ``Nguyen Mai`` gives family ``Mai``, as 1.4.0 did. The cost is that ``John Smith MAI`` gives last ``MAI`` instead of suffix; ``Smith, John, MAI`` and ``John Smith M.A.I.`` still read as credentials. A caller who needs it back adds it: ``Lexicon.default().add(suffix_acronyms={"mai"})``. See the ``suffix-acronym-collisions`` entry of ``docs/design/decisions.md`` (:issue:`687`) + - **Remove mai from the default post-nominal acronyms, so a trailing Mai keeps the name.** ``Tran Thi Mai`` read in Vietnamese order (``FAMILY_FIRST_GIVEN_LAST``) gives given ``Mai``, middle ``Thi``, family ``Tran``, where 2.0.0 through 2.4.2 gave suffix ``Mai`` and no given name; ``Nguyen Mai`` gives family ``Mai``, as 1.4.0 did. The cost is that ``John Smith MAI`` gives last ``MAI`` instead of suffix; ``Smith, John, MAI`` and ``John Smith M.A.I.`` still read as credentials. A caller who needs it back adds it: ``Lexicon.default().add(suffix_acronyms={"mai"})``. See the ``suffix-acronym-collisions`` entry of ``docs/design/decisions/`` (:issue:`687`) - **Add Chinese and Japanese credential, office and generational words as suffixes.** ``彭德威 议员`` under ``locales.ZH`` gives family ``彭``, given ``德威``, suffix ``议员``, where 2.1 through 2.4.2 gave middle ``议员``; ``田中社長`` gives family ``田中``, suffix ``社長``, where 2.1 through 2.4.2 gave family ``田中社長``. Read when written as a separate word, after a comma, or glued to the end of the name, except that 部長 and 部长 are not split off a glued name (``田中部長`` is unchanged) unless ``locales.ZH`` divides it at a listed surname (``李部長``). :issue:`671`, :issue:`707` - **Add the Akan and Indonesian salutations owura and bapak as titles.** ``Bapak Dwi Putro`` gives title ``Bapak``, first ``Dwi``, last ``Putro``, where 1.4.0 through 2.4.2 gave first ``Bapak``, middle ``Dwi``. :issue:`662` - **Fix a title written before a family comma reading as part of the family name.** ``Dr. Smith, John`` gives title ``Dr.``, last ``Smith``, where 1.4.0 through 2.4.2 gave last ``Dr. Smith``. Only listed titles are read there, so ``Xyz. Smith, John`` keeps its family, and a title that is also a surname particle, such as ``St`` or ``Freiherr``, stays in the surname. :issue:`701` @@ -133,35 +133,35 @@ Release Log **Behavior Changes** - - **Fix HumanName.initials() dropping a middle- or family-group initial that is also a one-letter conjunction.** ``HumanName("Scott E. Werner").initials()`` gives ``S. E. W.`` again where 2.0.0 through 2.2.0 gave ``S. W.``; ``Juan Y. Garcia`` and a bare ASCII capital ``John E Smith`` likewise. v1 excluded initial-shaped words from its conjunction test and the 2.0 facade had not; ``parse(...).initials()`` was already right and is unchanged. A bare lowercase ``john e smith`` still reads the ``e`` as the connective. See the ``R3`` entry of ``docs/design/decisions.md`` (closes :issue:`462`) + - **Fix HumanName.initials() dropping a middle- or family-group initial that is also a one-letter conjunction.** ``HumanName("Scott E. Werner").initials()`` gives ``S. E. W.`` again where 2.0.0 through 2.2.0 gave ``S. W.``; ``Juan Y. Garcia`` and a bare ASCII capital ``John E Smith`` likewise. v1 excluded initial-shaped words from its conjunction test and the 2.0 facade had not; ``parse(...).initials()`` was already right and is unchanged. A bare lowercase ``john e smith`` still reads the ``e`` as the connective. See the ``R3`` entry of ``docs/design/decisions/`` (closes :issue:`462`) - - **Record a 2.0.0 change to HumanName.initials() that no release note had classified:** since 2.0.0 the facade initials each WORD of a name part, where 1.4.0 initialed a joined run as one group -- ``HumanName("Juan Velasquez y Garcia").initials()`` is ``J. V. G.`` and was ``J. V G.``; ``Abdul Salam Hassan`` is ``A. S. H.`` and was ``A S. H.``. Nothing changes in 2.3.0; the differential gate now compares ``initials()`` (:issue:`484`) and this is what it found. See the ``differential-ledger, the initials view`` entry of ``docs/design/decisions.md`` + - **Record a 2.0.0 change to HumanName.initials() that no release note had classified:** since 2.0.0 the facade initials each WORD of a name part, where 1.4.0 initialed a joined run as one group -- ``HumanName("Juan Velasquez y Garcia").initials()`` is ``J. V. G.`` and was ``J. V G.``; ``Abdul Salam Hassan`` is ``A. S. H.`` and was ``A S. H.``. Nothing changes in 2.3.0; the differential gate now compares ``initials()`` (:issue:`484`) and this is what it found. See the ``differential-ledger, the initials view`` entry of ``docs/design/decisions/`` - - **Fix a space-separated run of post-nominals rendering with a comma the writer never typed.** ``HumanName("John Smith MD PhD").suffix`` is ``MD PhD`` and was ``MD, PhD`` at every release since 1.4.0; ``Kenneth Clarke QC MP`` gives ``QC MP``, and the CJK honorific runs (``김민준 박사 씨``) follow the same rule. This is a deliberate deviation from 1.4.0, and the v1-parity suite is re-pinned to match. The comma forms are unchanged -- ``HumanName("Smith, MD, PhD").suffix`` is still ``MD, PhD`` -- because the separator is now the comma the writer typed; a configured suffix delimiter still parts a run, as does a name word standing between two post-nominals. Round-tripping is fixed for these runs, which the 2.2.0 note below recorded as broken: ``str(HumanName("Smith, MD PhD"))`` is ``Smith MD PhD`` and re-parses to suffix ``MD PhD``. ``str()`` is still a rendering rather than a canonical form. See the ``C1`` entry of ``docs/design/decisions.md`` (closes :issue:`436`, closes :issue:`437`) + - **Fix a space-separated run of post-nominals rendering with a comma the writer never typed.** ``HumanName("John Smith MD PhD").suffix`` is ``MD PhD`` and was ``MD, PhD`` at every release since 1.4.0; ``Kenneth Clarke QC MP`` gives ``QC MP``, and the CJK honorific runs (``김민준 박사 씨``) follow the same rule. This is a deliberate deviation from 1.4.0, and the v1-parity suite is re-pinned to match. The comma forms are unchanged -- ``HumanName("Smith, MD, PhD").suffix`` is still ``MD, PhD`` -- because the separator is now the comma the writer typed; a configured suffix delimiter still parts a run, as does a name word standing between two post-nominals. Round-tripping is fixed for these runs, which the 2.2.0 note below recorded as broken: ``str(HumanName("Smith, MD PhD"))`` is ``Smith MD PhD`` and re-parses to suffix ``MD PhD``. ``str()`` is still a rendering rather than a canonical form. See the ``C1`` entry of ``docs/design/decisions/`` (closes :issue:`436`, closes :issue:`437`) - - **Fix Parser.revise() splitting a space-separated suffix value into comma-separated entries.** ``Parser().revise(n, suffix="MD PhD").suffix`` is ``MD PhD`` and was ``MD, PhD``. A suffix value's entries are now derived from the value's own commas by the rule a whole name uses -- a comma parts two credentials and a space joins them -- so ``revise(n, suffix="MD, PhD")`` is still two entries, and ``revise(n, suffix="Ph. D.")`` renders ``Ph. D.`` where the 2.2.0 note below accepted ``Ph., D.``. One limit: a delimiter configured through ``extra_suffix_delimiters`` parts a value only where the value's own words read as a name with a tail segment, so in a run of post-nominals it stays a word; write a comma at the boundary instead. ``ParsedName.replace()`` is unchanged. See the ``C1`` entry of ``docs/design/decisions.md`` (closes :issue:`511`) + - **Fix Parser.revise() splitting a space-separated suffix value into comma-separated entries.** ``Parser().revise(n, suffix="MD PhD").suffix`` is ``MD PhD`` and was ``MD, PhD``. A suffix value's entries are now derived from the value's own commas by the rule a whole name uses -- a comma parts two credentials and a space joins them -- so ``revise(n, suffix="MD, PhD")`` is still two entries, and ``revise(n, suffix="Ph. D.")`` renders ``Ph. D.`` where the 2.2.0 note below accepted ``Ph., D.``. One limit: a delimiter configured through ``extra_suffix_delimiters`` parts a value only where the value's own words read as a name with a tail segment, so in a run of post-nominals it stays a word; write a comma at the boundary instead. ``ParsedName.replace()`` is unchanged. See the ``C1`` entry of ``docs/design/decisions/`` (closes :issue:`511`) - - **Remove rai and cha from the default post-nominal acronyms, so a trailing Rai or CHA keeps the family name.** ``HumanName("Aishwarya Rai")`` gives last ``Rai``, where 2.0.0 through 2.2.0 gave suffix ``Rai`` and no last name at all -- 1.4.0's reading, restored. The cost is that a genuine credential written after a full name is no longer recognized: ``John Smith RAI`` gives last ``RAI``, and ``John Smith, RAI`` gives first ``RAI``, last ``John Smith``. A caller who needs either back adds it: ``Lexicon.default().add(suffix_acronyms={"cha"})``. See the ``suffix-acronym-collisions`` entry of ``docs/design/decisions.md`` (closes :issue:`342`) + - **Remove rai and cha from the default post-nominal acronyms, so a trailing Rai or CHA keeps the family name.** ``HumanName("Aishwarya Rai")`` gives last ``Rai``, where 2.0.0 through 2.2.0 gave suffix ``Rai`` and no last name at all -- 1.4.0's reading, restored. The cost is that a genuine credential written after a full name is no longer recognized: ``John Smith RAI`` gives last ``RAI``, and ``John Smith, RAI`` gives first ``RAI``, last ``John Smith``. A caller who needs either back adds it: ``Lexicon.default().add(suffix_acronyms={"cha"})``. See the ``suffix-acronym-collisions`` entry of ``docs/design/decisions/`` (closes :issue:`342`) - **Mark ba as an acronym that is also an ordinary name, so a bare trailing Ba keeps the family name.** ``HumanName("Anna Ba")`` gives last ``Ba`` and reports a suffix-or-name ambiguity, where 2.0.0 through 2.2.0 gave suffix ``Ba`` and no last name. The spaced full-name form keeps the credential reading -- ``John Smith BA`` still gives suffix ``BA``, now flagged -- and the dotted ``John Smith B.A.`` is an unflagged suffix. The comma forms move, which is the marking's cost: ``John Smith, BA`` gives first ``BA``, last ``John Smith``, as ``Smith, Ed`` already did for the other ambiguous acronyms. Write ``B.A.`` to keep the credential reading. Ba is a real surname in Vietnamese and Senegalese Fula, the ``ma``/``Ma`` shape exactly (:issue:`342`) - - **Fix a title run addressing by its first title rather than its last.** ``HumanName("Her Majesty Queen Elizabeth")`` gives first ``Elizabeth`` with an empty last name, where every release since 1.4.0 gave last ``Elizabeth``. Several titles written together are one form of address and the one that does the addressing is the last, so the run is now matched whole or by its last word: ``Reverend Mother Teresa``, ``Dr. Sir John`` and ``Sir Sheikh abdul rahman`` move the same way. A run whose last word addresses by surname does not move: ``His Excellency Lord Duncan`` still gives last ``Duncan``, ``lord`` not being a given-name title. A caller's multi-word entry still matches as a phrase. See the ``H1`` entry of ``docs/design/decisions.md`` (closes :issue:`489`) + - **Fix a title run addressing by its first title rather than its last.** ``HumanName("Her Majesty Queen Elizabeth")`` gives first ``Elizabeth`` with an empty last name, where every release since 1.4.0 gave last ``Elizabeth``. Several titles written together are one form of address and the one that does the addressing is the last, so the run is now matched whole or by its last word: ``Reverend Mother Teresa``, ``Dr. Sir John`` and ``Sir Sheikh abdul rahman`` move the same way. A run whose last word addresses by surname does not move: ``His Excellency Lord Duncan`` still gives last ``Duncan``, ``lord`` not being a given-name title. A caller's multi-word entry still matches as a phrase. See the ``H1`` entry of ``docs/design/decisions/`` (closes :issue:`489`) - - **Fix the leading title peel taking a name word and leaving a post-nominal to be the name.** ``HumanName("Dr King Jr")`` gives title ``Dr``, last ``King``, suffix ``Jr``, where every release since 1.4.0 gave title ``Dr King``, last ``Jr`` and no suffix at all; ``Dr. King MD`` moves the same way, and both now read as the comma spelling ``King, Dr Jr`` always has. A name that is nothing but titles or nothing but post-nominals is untouched (``Marquess of Bath``, ``MD DDS``), and so is a title written as one joined unit: ``Prince of Wales Jr`` keeps title ``Prince of Wales``. Where the only word left is the title itself, ``Dr Jr`` gives first ``Dr``, suffix ``Jr`` and reports a title-or-name ambiguity. See the ``H3`` entry of ``docs/design/decisions.md`` + - **Fix the leading title peel taking a name word and leaving a post-nominal to be the name.** ``HumanName("Dr King Jr")`` gives title ``Dr``, last ``King``, suffix ``Jr``, where every release since 1.4.0 gave title ``Dr King``, last ``Jr`` and no suffix at all; ``Dr. King MD`` moves the same way, and both now read as the comma spelling ``King, Dr Jr`` always has. A name that is nothing but titles or nothing but post-nominals is untouched (``Marquess of Bath``, ``MD DDS``), and so is a title written as one joined unit: ``Prince of Wales Jr`` keeps title ``Prince of Wales``. Where the only word left is the title itself, ``Dr Jr`` gives first ``Dr``, suffix ``Jr`` and reports a title-or-name ambiguity. See the ``H3`` entry of ``docs/design/decisions/`` - - **Fix a trailing abbreviated title reading as a name word.** ``HumanName("John Smith Prof.")`` gives title ``Prof.``, first ``John``, last ``Smith``, where every release since 1.4.0 gave last ``Prof.`` and lost the surname; ``John Smith Dr.``, ``John Smith Rev.`` and ``Andrew Perkins (Mgr.)`` move the same way, and the comma form ``Smith, John Prof.`` now agrees with the bare one where it gave middle ``Prof.``. A run chains from the end (``John Smith Prof. Dr.`` gives title ``Prof. Dr.``), a leading title keeps its place (``Dr. John Smith Prof.`` gives title ``Dr. Prof.``), and a post-nominal is read through it (``John Smith Jr. Prof.`` gives suffix ``Jr.``). Only a listed title word wearing the abbreviation period is claimed: an unlisted abbreviation (``John Smith Xyz.``), a bare title word (``John Smith Sir``) and a post-nominal (``John Smith Esq.``) do not move. The reach is the whole title vocabulary, ordinary surnames in it included, so ``Mary Jane King.`` gives title ``King.`` where the bare ``Mary Jane King`` keeps last ``King`` -- accepted rather than prevented, the period being evidence the bare spelling never gives. The same argument holds in a native script: ``毛 泽东 Dr.`` gives title ``Dr.``, first ``泽东``, last ``毛``, where 2.2.0 gave last ``Dr.``. See the ``H5`` entry of ``docs/design/decisions.md`` (closes :issue:`316`) + - **Fix a trailing abbreviated title reading as a name word.** ``HumanName("John Smith Prof.")`` gives title ``Prof.``, first ``John``, last ``Smith``, where every release since 1.4.0 gave last ``Prof.`` and lost the surname; ``John Smith Dr.``, ``John Smith Rev.`` and ``Andrew Perkins (Mgr.)`` move the same way, and the comma form ``Smith, John Prof.`` now agrees with the bare one where it gave middle ``Prof.``. A run chains from the end (``John Smith Prof. Dr.`` gives title ``Prof. Dr.``), a leading title keeps its place (``Dr. John Smith Prof.`` gives title ``Dr. Prof.``), and a post-nominal is read through it (``John Smith Jr. Prof.`` gives suffix ``Jr.``). Only a listed title word wearing the abbreviation period is claimed: an unlisted abbreviation (``John Smith Xyz.``), a bare title word (``John Smith Sir``) and a post-nominal (``John Smith Esq.``) do not move. The reach is the whole title vocabulary, ordinary surnames in it included, so ``Mary Jane King.`` gives title ``King.`` where the bare ``Mary Jane King`` keeps last ``King`` -- accepted rather than prevented, the period being evidence the bare spelling never gives. The same argument holds in a native script: ``毛 泽东 Dr.`` gives title ``Dr.``, first ``泽东``, last ``毛``, where 2.2.0 gave last ``Dr.``. See the ``H5`` entry of ``docs/design/decisions/`` (closes :issue:`316`) - - **Remove esq from the default post-nominal acronyms, and assert the two post-nominal sets disjoint.** ``HumanName("John Smith E.S.Q.")`` gives middle ``Smith``, last ``E.S.Q.``, where every release since 1.4.0 gave suffix ``E.S.Q.``; ``Esq``, ``Esq.``, ``ESQ`` and ``esq`` are unchanged, the post-nominal word list carrying every single-token spelling. Esquire is a contraction rather than an initialism, and it was the one word in both post-nominal sets, which can now assert they do not overlap. A caller who needs the dotted spelling back adds it: ``Lexicon.default().add(suffix_acronyms={"esq"})``. See the ``suffix-acronym-collisions`` entry of ``docs/design/decisions.md`` + - **Remove esq from the default post-nominal acronyms, and assert the two post-nominal sets disjoint.** ``HumanName("John Smith E.S.Q.")`` gives middle ``Smith``, last ``E.S.Q.``, where every release since 1.4.0 gave suffix ``E.S.Q.``; ``Esq``, ``Esq.``, ``ESQ`` and ``esq`` are unchanged, the post-nominal word list carrying every single-token spelling. Esquire is a contraction rather than an initialism, and it was the one word in both post-nominal sets, which can now assert they do not overlap. A caller who needs the dotted spelling back adds it: ``Lexicon.default().add(suffix_acronyms={"esq"})``. See the ``suffix-acronym-collisions`` entry of ``docs/design/decisions/`` - - **Fix the East Slavic and Turkic patronymic rotations overriding a declared family-first name order.** With ``patronymic_rules`` opted in and ``Policy(name_order=FAMILY_FIRST)``, ``Мицкевич Адам Юзеф`` gave last ``Адам`` through 2.2.0 and now gives last ``Мицкевич`` -- the reading the declaration asks for -- and ``oglu Ahmad Vali Ali`` with Turkic handling gave last ``Ahmad`` and now ``oglu``. The rotations exist to restore the given-first reading a family-first listing hides, so under a declared family-first order the declaration decides. See the ``O1`` entry of ``docs/design/decisions.md`` (closes :issue:`384`) + - **Fix the East Slavic and Turkic patronymic rotations overriding a declared family-first name order.** With ``patronymic_rules`` opted in and ``Policy(name_order=FAMILY_FIRST)``, ``Мицкевич Адам Юзеф`` gave last ``Адам`` through 2.2.0 and now gives last ``Мицкевич`` -- the reading the declaration asks for -- and ``oglu Ahmad Vali Ali`` with Turkic handling gave last ``Ahmad`` and now ``oglu``. The rotations exist to restore the given-first reading a family-first listing hides, so under a declared family-first order the declaration decides. See the ``O1`` entry of ``docs/design/decisions/`` (closes :issue:`384`) - - **Fix CJK honorifics and names written with a full stop of any width.** ``HumanName("김민준 씨．").suffix`` is ``씨．`` (the fullwidth stop a Japanese or Chinese IME produces by default), with last ``김`` and first ``민준``, where every release since 2.1.0 gave first ``김``, middle ``민준``, last ``씨．`` and no suffix; the ideographic ``씨。`` and halfwidth ``씨｡`` spellings move the same way. The vocabulary lookup now composes NFC, so a decomposed ``Señor`` or ``née`` from macOS-origin data is recognized too. A period glued to a name word no longer breaks the name: ``양. 지훈`` gives last ``양.``, first ``지훈`` where 2.1.0 through 2.2.0 gave first ``양.``, middle ``지``, last ``훈``, and ``김민준씨.`` peels its honorific as the stop-less spelling does. The period stays on the word it was written with; nothing is rewritten. A Latin name written with ASCII periods is untouched (``Smith. John`` still reads title ``Smith.``), but a Latin or Cyrillic word wearing one of the three wider stops now reaches the vocabulary: ``Dr。 John Smith`` reads title ``Dr。`` where it read first ``Dr。``. This retires the 2.2.0 note below that read *period* strictly. One incompatibility, by decision: a ``Lexicon`` pickled by 2.1.x or 2.2.x that carries a caller-added entry the widened fold now changes -- a non-ASCII entry written with a fullwidth or ideographic stop, or in NFD -- no longer loads (``ValueError: incompatible Lexicon pickle: entries are not normalized``); the shipped vocabulary is unaffected, and the remedy is to rebuild the ``Lexicon`` from its source rather than unpickle it. See the ``cjk-full-stops`` entry of ``docs/design/decisions.md`` (closes :issue:`322`, closes :issue:`323`) + - **Fix CJK honorifics and names written with a full stop of any width.** ``HumanName("김민준 씨．").suffix`` is ``씨．`` (the fullwidth stop a Japanese or Chinese IME produces by default), with last ``김`` and first ``민준``, where every release since 2.1.0 gave first ``김``, middle ``민준``, last ``씨．`` and no suffix; the ideographic ``씨。`` and halfwidth ``씨｡`` spellings move the same way. The vocabulary lookup now composes NFC, so a decomposed ``Señor`` or ``née`` from macOS-origin data is recognized too. A period glued to a name word no longer breaks the name: ``양. 지훈`` gives last ``양.``, first ``지훈`` where 2.1.0 through 2.2.0 gave first ``양.``, middle ``지``, last ``훈``, and ``김민준씨.`` peels its honorific as the stop-less spelling does. The period stays on the word it was written with; nothing is rewritten. A Latin name written with ASCII periods is untouched (``Smith. John`` still reads title ``Smith.``), but a Latin or Cyrillic word wearing one of the three wider stops now reaches the vocabulary: ``Dr。 John Smith`` reads title ``Dr。`` where it read first ``Dr。``. This retires the 2.2.0 note below that read *period* strictly. One incompatibility, by decision: a ``Lexicon`` pickled by 2.1.x or 2.2.x that carries a caller-added entry the widened fold now changes -- a non-ASCII entry written with a fullwidth or ideographic stop, or in NFD -- no longer loads (``ValueError: incompatible Lexicon pickle: entries are not normalized``); the shipped vocabulary is unaffected, and the remedy is to rebuild the ``Lexicon`` from its source rather than unpickle it. See the ``cjk-full-stops`` entry of ``docs/design/decisions/`` (closes :issue:`322`, closes :issue:`323`) **Additions** - - **Add the renunciate titles to the given-name title list, so a renunciate's one name is a given name.** ``HumanName("Swami Vivekananda")`` gives first ``Vivekananda`` with an empty last name, where every release since 1.4.0 gave last ``Vivekananda``; ``Guru Nanak``, ``Baba Ramdev`` and ``Lama Zopa`` move the same way, and so do the Devanagari and Bengali spellings added below. Two name words behind the title are unchanged -- ``Swami Vivekananda Saraswati`` keeps last ``Saraswati`` -- and a surname-retaining title is untouched: ``Rabbi Cohen`` still gives last ``Cohen``. ``venerable`` is deliberately not in the list, the traditions using it splitting on whether the family name survives. See the ``indic-honorifics`` entry of ``docs/design/decisions.md`` (closes :issue:`346`) + - **Add the renunciate titles to the given-name title list, so a renunciate's one name is a given name.** ``HumanName("Swami Vivekananda")`` gives first ``Vivekananda`` with an empty last name, where every release since 1.4.0 gave last ``Vivekananda``; ``Guru Nanak``, ``Baba Ramdev`` and ``Lama Zopa`` move the same way, and so do the Devanagari and Bengali spellings added below. Two name words behind the title are unchanged -- ``Swami Vivekananda Saraswati`` keeps last ``Saraswati`` -- and a surname-retaining title is untouched: ``Rabbi Cohen`` still gives last ``Cohen``. ``venerable`` is deliberately not in the list, the traditions using it splitting on whether the family name survives. See the ``indic-honorifics`` entry of ``docs/design/decisions/`` (closes :issue:`346`) - - **Add prince and princess to the given-name title list, so a royal's one name is a given name.** ``HumanName("Prince Harry")`` gives first ``Harry`` with an empty last name, where every release since 1.4.0 gave last ``Harry``; ``Princess Anne`` and ``Her Royal Highness Princess Anne`` move the same way. Two name words behind the title are unchanged -- ``Prince Harry Windsor`` keeps last ``Windsor`` -- and so is ``Prince of Wales Jr``, the joined title addressing by its last word. ``lord`` and ``lady`` are deliberately NOT in the list: they address by given name only as a courtesy style (``Lord Peter``, ``Lady Diana``) and by surname for every peer and every wife (``Lord Byron``, ``Lady Thatcher``), which the list cannot express. The given-name collision is untouched: ``Prince Fielder`` now gives first ``Fielder`` (:issue:`348`). See the ``H1`` entry of ``docs/design/decisions.md`` (closes :issue:`519`) + - **Add prince and princess to the given-name title list, so a royal's one name is a given name.** ``HumanName("Prince Harry")`` gives first ``Harry`` with an empty last name, where every release since 1.4.0 gave last ``Harry``; ``Princess Anne`` and ``Her Royal Highness Princess Anne`` move the same way. Two name words behind the title are unchanged -- ``Prince Harry Windsor`` keeps last ``Windsor`` -- and so is ``Prince of Wales Jr``, the joined title addressing by its last word. ``lord`` and ``lady`` are deliberately NOT in the list: they address by given name only as a courtesy style (``Lord Peter``, ``Lady Diana``) and by surname for every peer and every wife (``Lord Byron``, ``Lady Thatcher``), which the list cannot express. The given-name collision is untouched: ``Prince Fielder`` now gives first ``Fielder`` (:issue:`348`). See the ``H1`` entry of ``docs/design/decisions/`` (closes :issue:`519`) - **Add trailing honorifics as post-nominal vocabulary:** Latin ``rinpoche``, Devanagari ``जी``, ``साहब``, ``साहिब``, ``साहेब``, ``महाराज``, and Bengali ``সাহেব``, ``বাবু``, ``মহারাজ``. ``HumanName("Lama Zopa Rinpoche")`` reads title ``Lama``, first ``Zopa``, suffix ``Rinpoche``; ``नरेन्द्र मोदी जी`` reads last ``मोदी``, suffix ``जी``. They are recognized SPACED only and are deliberately absent from ``Lexicon.honorific_tails``: Banerjee, Mukherjee and Chatterjee end in the ``जी`` substring (``बनर्जी``, ``मुखर्जी``, ``चटर्जी``), so a glued peel would cut a real family name in two, and ``गांधीजी`` staying unpeeled is the accepted cost. Bengali ``বাবু`` is trailing where Devanagari ``बाबू`` is a leading title (:issue:`344`, :issue:`343`) @@ -169,9 +169,9 @@ Release Log - **Add Bengali honorifics -- the first Bengali vocabulary in the default lexicon (#343):** ``ড``, ``ডঃ``, ``ডক্টর``, ``ডাঃ``, ``ডা``, ``ডাক্তার``, ``শ্রী``, ``শ্রীমতী``, ``জনাব``, ``অধ্যাপক``, ``প্রফেসর``, ``বিচারপতি``, ``মাওলানা``, ``মুফতি``, ``আলহাজ্ব``, ``আলহাজ``, ``মিঃ``, ``মি``, ``মিসেস``, ``মোঃ``, ``মো``, ``মোসাঃ``, ``মোসা``, ``মোছাঃ`` and ``মোছা`` as titles, and ``স্বামী``, ``শ্রীল``, ``গুরু``, ``বাবা`` as given-name titles. ``ড. মুহাম্মদ ইউনূস`` reads title ``ড.``, first ``মুহাম্মদ``, last ``ইউনূস`` -- the vocabulary beats the initial reading -- while real initials are untouched: ``র. কে. নারায়ণ`` is unchanged. ``মোঃ আবদুল করিম`` reads title ``মোঃ``, first ``আবদুল``, last ``করিম``, the mirror of Latin ``Md``; the visarga spelling and the ``মো.`` period spelling both match, and the women's ``মোসাঃ``/``মোসা.`` rides the same pair of entries. ``ঠাকুর`` stays out, being Tagore. Latin transliterations (``Sri``, ``Pandit``, ``Mst``) are not added -- they collide with real given names where the native scripts cannot -- and belong to the opt-in packs of :issue:`345` (closes :issue:`343`) - - **Add AmbiguityKind.GIVEN_OR_FAMILY, reported when a name of one name word had nothing to decide which field it is:** ``parse("Andrew")`` still gives given ``Andrew`` and now says that field was a convention rather than a reading -- the library picks the given name under the default order and the family name under a declared family-first one, and ``detail`` names the field it picked. ``parse("Smith Jr.")`` reports it too, the suffix being peeled first. A name something DID decide stays silent -- ``"Dr. Smith"``, ``"Smith, Andrew"``, ``"abdul"`` (bound given-name vocabulary), ``"J."`` (an initial's shape) -- and so does ``"毛泽东"``, where the writing system settles the order. No field moves anywhere. See the ``O5`` entry of ``docs/design/decisions.md`` (closes :issue:`449`) + - **Add AmbiguityKind.GIVEN_OR_FAMILY, reported when a name of one name word had nothing to decide which field it is:** ``parse("Andrew")`` still gives given ``Andrew`` and now says that field was a convention rather than a reading -- the library picks the given name under the default order and the family name under a declared family-first one, and ``detail`` names the field it picked. ``parse("Smith Jr.")`` reports it too, the suffix being peeled first. A name something DID decide stays silent -- ``"Dr. Smith"``, ``"Smith, Andrew"``, ``"abdul"`` (bound given-name vocabulary), ``"J."`` (an initial's shape) -- and so does ``"毛泽东"``, where the writing system settles the order. No field moves anywhere. See the ``O5`` entry of ``docs/design/decisions/`` (closes :issue:`449`) - - **Add AmbiguityKind.TITLE_OR_NAME, reported when an input that is nothing but honorifics had its last word read as the name:** ``parse("Lord Chancellor")`` still gives title ``Lord``, family ``Chancellor``, and now says so -- this is a name parser, not a title parser, so handed a string with no name in it, it reads the last title word as one. ``"His Holiness"`` and ``"Dr. King"`` move the same way, ``king`` being title vocabulary. A title with an ordinary word behind it is silent (``"Dr. Smith"``, ``"King Charles"``), and so is a lone title word: ``parse("Dr.")`` is a title with no name beside it. The same convention on the post-nominal vocabulary reports the existing suffix-or-name: ``parse("Rinpoche")`` gives given ``Rinpoche`` and flags it, as does ``"QC MP"``. No field moves for this change; ``Dr King Jr``, ``Dr. King MD`` and ``Dr Jr`` gain the report from the title-peel fix above. See the ``H4`` entry of ``docs/design/decisions.md`` (closes :issue:`491`) + - **Add AmbiguityKind.TITLE_OR_NAME, reported when an input that is nothing but honorifics had its last word read as the name:** ``parse("Lord Chancellor")`` still gives title ``Lord``, family ``Chancellor``, and now says so -- this is a name parser, not a title parser, so handed a string with no name in it, it reads the last title word as one. ``"His Holiness"`` and ``"Dr. King"`` move the same way, ``king`` being title vocabulary. A title with an ordinary word behind it is silent (``"Dr. Smith"``, ``"King Charles"``), and so is a lone title word: ``parse("Dr.")`` is a title with no name beside it. The same convention on the post-nominal vocabulary reports the existing suffix-or-name: ``parse("Rinpoche")`` gives given ``Rinpoche`` and flags it, as does ``"QC MP"``. No field moves for this change; ``Dr King Jr``, ``Dr. King MD`` and ``Dr Jr`` gain the report from the title-peel fix above. See the ``H4`` entry of ``docs/design/decisions/`` (closes :issue:`491`) **Documentation** @@ -206,7 +206,7 @@ Release Log **Behavior Changes** - - **Fix a title changing how the name behind it is read.** ``"Dr. Van Johnson"`` gave family ``Van Johnson`` with no given name, and ``"Sir Van Johnson"`` gave given ``Van Johnson`` with no family at all; both now read given ``Van``, family ``Johnson`` -- the reading the untitled ``"Van Johnson"`` has always had. A leading word that is *both* a title and a particle is unchanged: ``"St John Smith"``, ``"Do John Smith"`` and ``"Freiherr von Richthofen"`` keep their readings. See the ``P2`` entry of ``docs/design/decisions.md`` (closes :issue:`367`) + - **Fix a title changing how the name behind it is read.** ``"Dr. Van Johnson"`` gave family ``Van Johnson`` with no given name, and ``"Sir Van Johnson"`` gave given ``Van Johnson`` with no family at all; both now read given ``Van``, family ``Johnson`` -- the reading the untitled ``"Van Johnson"`` has always had. A leading word that is *both* a title and a particle is unchanged: ``"St John Smith"``, ``"Do John Smith"`` and ``"Freiherr von Richthofen"`` keep their readings. See the ``P2`` entry of ``docs/design/decisions/`` (closes :issue:`367`) - **Fix a given-name title keeping a bound given name from joining the word after it.** ``"Sheik abdul salam"`` read given ``abdul``, family ``salam``, and now reads given ``abdul salam`` with an empty family, as ``"Sir John"`` does; ``"الشيخ عبد الله"`` reads given ``عبد الله``. A title that addresses by family is unchanged (``"Dr. abdul salam"``). This also restores ``"Sheik Abu Bakar"`` to given ``Abu Bakar``, which the fix above had regressed, and drops the ``PARTICLE_OR_GIVEN`` ambiguity that name reported through 2.1 (closes :issue:`369`) @@ -236,19 +236,19 @@ Release Log - **Fix a name opening with a particle that is never a given name being split at the particle under a family-first name order.** ``"de Mesnil"`` read as family ``de``, given ``Mesnil`` and ``"de la Vega"`` as family ``de``, given ``la Vega``; each is now the whole surname, as it has always been in the default order, under ``FAMILY_FIRST`` and ``FAMILY_FIRST_GIVEN_LAST`` alike. A word that can never be a given name leaves ``name_order`` nothing to decide. Standing alone is the whole of it: ``"Juan de la Vega"`` under ``FAMILY_FIRST`` still reports given ``de la Vega``. A leading particle that *may* be a given name is genuinely order-dependent and is untouched, so ``"van Gogh"`` still reads family ``van``, given ``Gogh`` under both family-first orders (closes :issue:`359`) - - **Fix a family name made only of particle words reporting no base, so the surname vanished from family_base and from the initials.** ``parse("Anh Do")`` gave family ``Do`` with ``family_base`` ``''`` and initials ``A.``, and under ``Policy(name_order=FAMILY_FIRST)`` ``"Del Toro"`` gave family ``Del`` the same way. A particle standing alone in a name part is not doing a particle's work there and now reads as an ordinary name word: ``"Anh Do"`` is base ``Do``, initials ``A. D.``; ``"Juan van der"`` is base ``van der``, initials ``J. v. d.``; ``"Nguyen, Van Le"`` initials ``V. L. N.`` where the middle name used to be dropped. The parse fields themselves do not move -- only the derived views and the initials. See the ``R2`` entry of ``docs/design/decisions.md`` (closes :issue:`385`, closes :issue:`402`) + - **Fix a family name made only of particle words reporting no base, so the surname vanished from family_base and from the initials.** ``parse("Anh Do")`` gave family ``Do`` with ``family_base`` ``''`` and initials ``A.``, and under ``Policy(name_order=FAMILY_FIRST)`` ``"Del Toro"`` gave family ``Del`` the same way. A particle standing alone in a name part is not doing a particle's work there and now reads as an ordinary name word: ``"Anh Do"`` is base ``Do``, initials ``A. D.``; ``"Juan van der"`` is base ``van der``, initials ``J. v. d.``; ``"Nguyen, Van Le"`` initials ``V. L. N.`` where the middle name used to be dropped. The parse fields themselves do not move -- only the derived views and the initials. See the ``R2`` entry of ``docs/design/decisions/`` (closes :issue:`385`, closes :issue:`402`) - **Fix case repair lowercasing the words of a family name made only of particle words, where every other view already reads them as ordinary name words.** ``HumanName("ANH DO").capitalize()`` gives ``Anh Do`` where it gave ``Anh do``, and ``"anh van do"`` gives ``Anh Van Do``. This DIFFERS FROM 1.4.0 deliberately and does not restore it: 1.4.0 returned ``Anh do``. The accepted cost is that a family which is nothing but particles capitalizes too, so ``"juan van der"`` gives ``Juan Van Der``. A conjunction is untouched (``"der, y van"`` gives ``y Van Der``), and where the particles DO join a name word nothing changes (``"juan de la vega"`` still gives ``Juan de la Vega``) (closes :issue:`407`) - - **Change case repair to read the parser's own conjunction tag instead of re-deciding, from the word's spelling, whether a word is a conjunction or an initial.** Two spellings of one name disagreed because of it: ``"jose ortega-y-gasset"`` capitalized to ``Jose Ortega-y-Gasset`` while ``"JOSE ORTEGA-Y-GASSET"`` gave ``Jose Ortega-Y-Gasset``; both give ``Jose Ortega-Y-Gasset`` now, a hyphenated token being one word to the parse whatever it contains. The spaced spelling is untouched and still repairs to ``Jose Ortega y Gasset``. A field assigned after the parse was never classified, so repair asks the vocabulary there -- today's vocabulary, which is narrower than 1.4.0 parity: ``h.last = "хосе и мария сантос"`` gives ``Хосе И Мария Сантос`` on 1.4.0 and ``Хосе и Мария Сантос`` here. One reading changes for hand-built ``Token``\ s in the 2.0 API: an untagged token whose text is conjunction vocabulary now capitalizes as an ordinary name word. See the ``R4`` entry of ``docs/design/decisions.md`` (closes :issue:`458`) + - **Change case repair to read the parser's own conjunction tag instead of re-deciding, from the word's spelling, whether a word is a conjunction or an initial.** Two spellings of one name disagreed because of it: ``"jose ortega-y-gasset"`` capitalized to ``Jose Ortega-y-Gasset`` while ``"JOSE ORTEGA-Y-GASSET"`` gave ``Jose Ortega-Y-Gasset``; both give ``Jose Ortega-Y-Gasset`` now, a hyphenated token being one word to the parse whatever it contains. The spaced spelling is untouched and still repairs to ``Jose Ortega y Gasset``. A field assigned after the parse was never classified, so repair asks the vocabulary there -- today's vocabulary, which is narrower than 1.4.0 parity: ``h.last = "хосе и мария сантос"`` gives ``Хосе И Мария Сантос`` on 1.4.0 and ``Хосе и Мария Сантос`` here. One reading changes for hand-built ``Token``\ s in the 2.0 API: an untagged token whose text is conjunction vocabulary now capitalizes as an ordinary name word. See the ``R4`` entry of ``docs/design/decisions/`` (closes :issue:`458`) - - **Change the parse-cost benchmark to bound function calls per parse rather than wall-clock seconds.** The old one-second bound failed four times on CI while the same code re-ran green on master; frame counts do not move under load. The bound is a per-interpreter band of ±2%, with a loose five-second backstop for what frame counts cannot see. Recomputable with ``uv run python tools/perf/call_count.py --against v2.1.0``; the counts and the per-PR attribution are in the ``parse-cost`` entry of ``docs/design/decisions.md``. No user-visible behavior changes (closes :issue:`475`) + - **Change the parse-cost benchmark to bound function calls per parse rather than wall-clock seconds.** The old one-second bound failed four times on CI while the same code re-ran green on master; frame counts do not move under load. The bound is a per-interpreter band of ±2%, with a loose five-second backstop for what frame counts cannot see. Recomputable with ``uv run python tools/perf/call_count.py --against v2.1.0``; the counts and the per-PR attribution are in the ``parse-cost`` entry of ``docs/design/decisions/``. No user-visible behavior changes (closes :issue:`475`) - **Fix a name that opens with a spaced Ph. D. losing its surname.** ``parse("Ph. D. Van Johnson")`` read given ``Van Johnson`` with an empty ``family`` and suffix ``Ph. D.``; it now reads title ``Ph.``, given ``D.``, family ``Van Johnson``. A suffix never begins a name, and the split credential was the only shape that reached the defect. A family comma still opens a listing rather than a name (``"John Smith Ph. D."`` and ``"Smith, Ph. D. Jr."`` keep their suffixes), and "the head" means the head of the string rather than of the name, so ``"Sir Ph. D. Van Johnson"`` is unchanged. This RESTORES 1.4.0. One accepted consequence: ``Parser.revise(suffix="Ph. D.")`` renders ``Ph., D.`` (closes :issue:`371`) - - **Fix a trailing surname particle being stranded as a standalone middle name under a family-first name order.** Under ``Policy(name_order=FAMILY_FIRST)`` the same listing written with a comma reads it as part of the surname: ``"Jong Anke de"`` gave family ``Jong`` with ``de`` left as a middle name and now gives family ``de Jong``, given ``Anke`` -- the answer ``parse("Jong, Anke de")`` has always given. ``FAMILY_FIRST`` is the only order that puts a trailing piece in a middle; ``FAMILY_FIRST_GIVEN_LAST`` puts it in the given slot, so ``"Nguyen Thi Van"`` under that order still reads given ``Van``. ``"Beethoven Ludwig van"`` under ``FAMILY_FIRST`` now gives family ``van Beethoven``. A particle standing alone in the given slot is no longer folded into the family either, so ``"Ménil de"`` reports given ``de``. Nothing moves under the DEFAULT name order. See :doc:`customize` for what declaring an order settles, and the ``P6`` entry of ``docs/design/decisions.md`` for the reasoning (closes :issue:`467`) + - **Fix a trailing surname particle being stranded as a standalone middle name under a family-first name order.** Under ``Policy(name_order=FAMILY_FIRST)`` the same listing written with a comma reads it as part of the surname: ``"Jong Anke de"`` gave family ``Jong`` with ``de`` left as a middle name and now gives family ``de Jong``, given ``Anke`` -- the answer ``parse("Jong, Anke de")`` has always given. ``FAMILY_FIRST`` is the only order that puts a trailing piece in a middle; ``FAMILY_FIRST_GIVEN_LAST`` puts it in the given slot, so ``"Nguyen Thi Van"`` under that order still reads given ``Van``. ``"Beethoven Ludwig van"`` under ``FAMILY_FIRST`` now gives family ``van Beethoven``. A particle standing alone in the given slot is no longer folded into the family either, so ``"Ménil de"`` reports given ``de``. Nothing moves under the DEFAULT name order. See :doc:`customize` for what declaring an order settles, and the ``P6`` entry of ``docs/design/decisions/`` for the reasoning (closes :issue:`467`) - - **Fix initials() ordering a name differently from the fields of the same parse.** Two rules fold words into the family and render them ahead of it -- ``Policy(middle_as_family=True)`` and the tussenvoegsel attachment after a family comma -- and the ``family`` field honored the fold where ``initials()`` did not: ``parse("der, y van")`` gave family ``van der`` but initials ``y. d. v.``, and now gives ``y. v. d.``. Under ``middle_as_family`` this RESTORES v1, that option being ``middle_name_as_last``'s successor: ``"Doe, Dr. John A."`` gives ``J. A. D.`` again where 2.0 through 2.2 gave ``J. D. A.``. ``HumanName.initials()`` was already right and is unchanged. See the ``R3`` entry of ``docs/design/decisions.md`` (closes :issue:`408`) + - **Fix initials() ordering a name differently from the fields of the same parse.** Two rules fold words into the family and render them ahead of it -- ``Policy(middle_as_family=True)`` and the tussenvoegsel attachment after a family comma -- and the ``family`` field honored the fold where ``initials()`` did not: ``parse("der, y van")`` gave family ``van der`` but initials ``y. d. v.``, and now gives ``y. v. d.``. Under ``middle_as_family`` this RESTORES v1, that option being ``middle_name_as_last``'s successor: ``"Doe, Dr. John A."`` gives ``J. A. D.`` again where 2.0 through 2.2 gave ``J. D. A.``. ``HumanName.initials()`` was already right and is unchanged. See the ``R3`` entry of ``docs/design/decisions/`` (closes :issue:`408`) - **Fix a tussenvoegsel attached to the family name after a comma deciding a genuinely uncertain reading and reporting nothing.** ``"Van Johnson"`` reports a ``PARTICLE_OR_GIVEN`` ambiguity -- ``Van`` is a Dutch particle and a Vietnamese given name -- while ``"Nguyen, Thi Van"`` picked the same word the same way, silently. The attachment now reports the fork it decides: ``"Nguyen, Thi Van"``, ``"Berg, Jan van der"`` and ``"Vega, Juan de la"`` each gain a ``PARTICLE_OR_GIVEN``, while a particle already read as a post-nominal reports ``SUFFIX_OR_NAME`` instead (``"Berg, Jan vd"``). Worth knowing before you filter on this: ``"Beethoven, Ludwig van"`` -- read exactly right -- now carries a report too, nothing in the input separating it from ``"Nguyen, Thi Van"``. ``ambiguities`` is the only value that grows (closes :issue:`405`) @@ -256,13 +256,13 @@ Release Log - **Add abd to BOUND_GIVEN_NAMES, so the spellings that write the article as its own word join like the others do:** ``"abd Allah Smith"`` was given ``abd``, middle ``Allah`` and is now given ``abd Allah``. ``abdul``, ``abdel`` and ``abdal`` were already there, and the Arabic-script ``عبد`` has covered the same word since 2.0, so only the Latin spelling was short. The word is also the postnominal ABD ("All But Dissertation") and stays in ``SUFFIX_ACRONYMS``: position tells the two readings apart, so ``"Jane Smith ABD"``, ``"Jane Smith, ABD"`` and ``"Jane Smith A.B.D."`` all still read the credential as a suffix (:issue:`400`) - - **Change how far a leading never-given particle takes the surname when a family-first name_order is declared.** ``Policy(name_order=FAMILY_FIRST)`` read ``"de Mesnil Jean"`` as family ``de Mesnil Jean`` -- the whole name -- and now reads family ``de Mesnil``, given ``Jean``. The default order is unchanged, deliberately: with no order declared nothing marks where the surname ends, and a particle followed by several words really can be all surname (``von Bergen Wessels``); a caller who means family ``de la Vega`` plus given ``Juan`` there writes the comma. The stop cannot land inside a conjunction-joined run or a bound given-name pair: ``"de la Vega y Santos Juan"`` reads family ``de la Vega y Santos``, ``"ibn Awf abdul Rahman"`` given ``abdul Rahman``. Where two or more words are left over, the two family-first orders differ from each other for the first time: ``"de la Cruz Juan Carlos"`` reads given ``Juan``, middle ``Carlos`` under ``FAMILY_FIRST`` and the reverse under ``FAMILY_FIRST_GIVEN_LAST``. See :doc:`customize`, and the ``P1`` entry of ``docs/design/decisions.md`` for the reasoning (closes :issue:`395`) + - **Change how far a leading never-given particle takes the surname when a family-first name_order is declared.** ``Policy(name_order=FAMILY_FIRST)`` read ``"de Mesnil Jean"`` as family ``de Mesnil Jean`` -- the whole name -- and now reads family ``de Mesnil``, given ``Jean``. The default order is unchanged, deliberately: with no order declared nothing marks where the surname ends, and a particle followed by several words really can be all surname (``von Bergen Wessels``); a caller who means family ``de la Vega`` plus given ``Juan`` there writes the comma. The stop cannot land inside a conjunction-joined run or a bound given-name pair: ``"de la Vega y Santos Juan"`` reads family ``de la Vega y Santos``, ``"ibn Awf abdul Rahman"`` given ``abdul Rahman``. Where two or more words are left over, the two family-first orders differ from each other for the first time: ``"de la Cruz Juan Carlos"`` reads given ``Juan``, middle ``Carlos`` under ``FAMILY_FIRST`` and the reverse under ``FAMILY_FIRST_GIVEN_LAST``. See :doc:`customize`, and the ``P1`` entry of ``docs/design/decisions/`` for the reasoning (closes :issue:`395`) - **Change the detail text of a PARTICLE_OR_GIVEN ambiguity to name the role the leading particle was actually given.** It said "read as a given name" under every ``name_order``, which is false under ``Policy(name_order=FAMILY_FIRST)`` -- there ``"Van Johnson"`` reads family ``Van``, given ``Johnson``, and the report described the reading not taken. It now ends "read as a family name" in that case. The ``kind`` is unchanged and stays ``PARTICLE_OR_GIVEN``; only the human-readable text moved, and default-order output is identical (:issue:`355`) - - **Fix a maiden name being lost when a particle stood in front of the marker.** ``"Ursula Leyen geb. Albrecht"`` reported maiden ``Albrecht`` correctly, but ``"Ursula von der Leyen geb. Albrecht"`` -- the same words one particle chain apart -- gave family ``von der Leyen geb. Albrecht`` and no maiden name at all, as did ``"Jane van der Berg née Jones"``. A suffix already stopped the particle chain; a marker now does too, so those read family ``von der Leyen`` maiden ``Albrecht`` and family ``van der Berg`` maiden ``Jones``. Under a family-first order ``"de la Cruz née Vega"`` now reads family ``de la Cruz``, maiden ``Vega``. Two limits remain, both recorded in ``rules.md#M2``: a conjunction join and a bound given-name join each still absorb a marker first (closes :issue:`399`) + - **Fix a maiden name being lost when a particle stood in front of the marker.** ``"Ursula Leyen geb. Albrecht"`` reported maiden ``Albrecht`` correctly, but ``"Ursula von der Leyen geb. Albrecht"`` -- the same words one particle chain apart -- gave family ``von der Leyen geb. Albrecht`` and no maiden name at all, as did ``"Jane van der Berg née Jones"``. A suffix already stopped the particle chain; a marker now does too, so those read family ``von der Leyen`` maiden ``Albrecht`` and family ``van der Berg`` maiden ``Jones``. Under a family-first order ``"de la Cruz née Vega"`` now reads family ``de la Cruz``, maiden ``Vega``. Two limits remain, both recorded in ``rules/M2``: a conjunction join and a bound given-name join each still absorb a marker first (closes :issue:`399`) - - **Move mc and ste into the never-given half of the particle vocabulary, and add los, las and das.** The Spanish and Portuguese articles were absent from it entirely. A never-given particle opening a name folds into the family (``rules.md#P1``) instead of being read as a given name, so ``"Mc Donald"`` was first ``Mc``, last ``Donald`` and is now last ``Mc Donald``; ``"Ste Marie"``, ``"Los Santos"``, ``"Las Casas"`` and ``"Das Silva"`` move the same way, and ``"Mc Donald Smith"`` becomes last ``Mc Donald Smith``. The ``PARTICLE_OR_GIVEN`` ambiguity goes with it for ``mc`` and ``ste``. ``los``, ``las`` and ``das`` were not particles at all, so for those three the ordinary particle join fires from a non-leading position too: ``"Maria das Neves"`` is now last ``das Neves``. (closes :issue:`360`) + - **Move mc and ste into the never-given half of the particle vocabulary, and add los, las and das.** The Spanish and Portuguese articles were absent from it entirely. A never-given particle opening a name folds into the family (``rules/P1``) instead of being read as a given name, so ``"Mc Donald"`` was first ``Mc``, last ``Donald`` and is now last ``Mc Donald``; ``"Ste Marie"``, ``"Los Santos"``, ``"Las Casas"`` and ``"Das Silva"`` move the same way, and ``"Mc Donald Smith"`` becomes last ``Mc Donald Smith``. The ``PARTICLE_OR_GIVEN`` ambiguity goes with it for ``mc`` and ``ste``. ``los``, ``las`` and ``das`` were not particles at all, so for those three the ordinary particle join fires from a non-leading position too: ``"Maria das Neves"`` is now last ``das Neves``. (closes :issue:`360`) - **Fix a bound given-name join leaving no family name when the name also carries a maiden clause, and stop the join absorbing the marker itself.** ``"abdul Berg née Jones"`` read given ``abdul Berg`` with an EMPTY family, where ``"abdul Berg"`` alone reads given ``abdul``, family ``Berg``; it now reads given ``abdul``, family ``Berg``, maiden ``Jones``. The join also declines when the piece it would absorb is a marker, so ``"van der Berg, abdul née Jones"`` reads given ``abdul``, family ``van der Berg``, maiden ``Jones`` where it read given ``abdul née``. Where the bound word is ALSO suffix vocabulary, a declining join after a family comma leaves the post-nominal reading: ``"Berg, abd née Jones"`` reads family ``Berg``, suffix ``abd``, maiden ``Jones``, as ``"Berg, abd"`` alone always has (closes :issue:`411`) diff --git a/nameparser/_config_shim.py b/nameparser/_config_shim.py index 461cc9fd..5a87491b 100644 --- a/nameparser/_config_shim.py +++ b/nameparser/_config_shim.py @@ -1109,7 +1109,7 @@ def matchable(field: str) -> frozenset[str]: # tests/v2/test_config_shim.py::test_snapshot_field_translation) # A capitalization_exceptions value that does not spell its key # is the one DECIDED exception to this method's never-raise - # rule (#459, decisions.md#R4 and #3-0-reevaluations). + # rule (#459, decisions/R4 and #3-0-reevaluations). # _normpairs raises _MaskValueError for exactly that shape, and # the except clause re-spells its offered fix for a v1 caller. # Caught by TYPE, so every other Lexicon error passes through diff --git a/nameparser/_facade.py b/nameparser/_facade.py index 1725b616..97a270b9 100644 --- a/nameparser/_facade.py +++ b/nameparser/_facade.py @@ -271,7 +271,7 @@ def capitalize(self, force: bool | None = None) -> None: """Re-capitalize the current parse against the bound lexicon. force=None reads the bound Constants' render default (force_mixed_case_capitalization); the core's capitalized() - implements the one-case gate (rules.md#R5, suffixes left out + implements the one-case gate (rules/R5, suffixes left out since #492) -- not re-implemented here.""" self._resolve() @@ -511,7 +511,7 @@ def _token_is_conjunction(self, tok: Token) -> bool: # answers -- the same fallback _render._cap_word takes for the # same tokens and through the same helper, which is why the # helper is imported rather than the predicate rewritten - # (decisions.md#R4 for why one question and not two: whether a + # (decisions/R4 for why one question and not two: whether a # word is a connective is a fact the word can answer alone, # whether a particle is acting as one is a fact about the part). # @@ -533,7 +533,7 @@ def _token_is_conjunction(self, tok: Token) -> bool: and UNJOINED_CONJUNCTION_TAG not in tok.tags) def _split_last(self) -> tuple[list[str], list[str]]: - # rules.md#R2: "a name part whose every word is particle + # rules/R2: "a name part whose every word is particle # vocabulary is a part where none of them is doing a # particle's work" -- the all-particle guard # below is this rule, and predates its statement: v1 assumed a @@ -617,7 +617,7 @@ def _process_initial(self, name_part: str, # # Particles are NOT decided per token: _is_particle stays a # live vocabulary lookup, as _render._cap_word keeps it -- - # rules.md#R4 draws this boundary per question, not per field. + # rules/R4 draws this boundary per question, not per field. self._resolve() if tokens is None: # STRING PATH. split() rather than split(" ") because @@ -700,7 +700,7 @@ def all_particle_guard(got: list[str], words: list[str]) -> list[str]: if got or not words or not all(self._is_particle(w) for w in words): return got - # rules.md#R3: "except the particles of a part whose every + # rules/R3: "except the particles of a part whose every # word is one, which are not acting as particles there" # -- nothing survived # the filter, so the whole group is particles. The diff --git a/nameparser/_lexicon.py b/nameparser/_lexicon.py index a757a1b3..c3bdcb45 100644 --- a/nameparser/_lexicon.py +++ b/nameparser/_lexicon.py @@ -129,7 +129,7 @@ #: it is roman five -- suffix -- where 'V.' stays a middle #: initial through that same carve-out; 'Jr。' simply reaches its #: entry, and 'Dr。' its title. -#: decisions.md#cjk-full-stops carries the measurements. +#: decisions/cjk-full-stops carries the measurements. FULL_STOPS = ".．。｡" @@ -152,7 +152,7 @@ def _normalize(word: str) -> str: NFC, since #322, and only for a non-ASCII word (ASCII is already NFC, and this runs per token on the parse path -- - decisions.md#parse-cost). NFD hangul decomposes to jamo and missed + decisions/parse-cost). NFD hangul decomposes to jamo and missed every vocabulary entry; composing at lookup AND at storage keeps the two symmetric, and no shipped entry changes under it (measured 2026-09-09: fold every string member of every Lexicon.default() @@ -175,7 +175,7 @@ def _normalize(word: str) -> str: word = stripped -# rules.md#P7: "A one-letter word written with its period is an +# rules/P7: "A one-letter word written with its period is an # initial and not a particle" def _spells_an_initial(n: str, text: str) -> bool: """Whether `text` is ONE letter written with its period ('Ó.', @@ -195,7 +195,7 @@ def _spells_an_initial(n: str, text: str) -> bool: -- its script table is _policy's, which this module may not import -- so a caller's one-character particle in such a script, written with a period, is vetoed where is_initial would call it no initial - (decisions.md#P7). No shipped particle is one character of those + (decisions/P7). No shipped particle is one character of those scripts.""" return len(n) == 1 and text[-1] == "." @@ -224,7 +224,7 @@ def _fold_words(words: Iterable[str]) -> list[str]: map/filter rather than a comprehension: both are C calls where a comprehension is a Python frame on 3.11, and this runs on the parse - path for every name carrying a title run (decisions.md#parse-cost). + path for every name carrying a title run (decisions/parse-cost). """ return list(filter(None, map(_normalize, words))) @@ -247,7 +247,7 @@ def _run_addresses_by_given(words: Iterable[str], Two sites ask it and they must agree -- post_rules for H1, group for the P5 licence -- because a run read two ways is a rule - contradicting itself (decisions.md#P5, the 2026-08-22 #369 entry). + contradicting itself (decisions/P5, the 2026-08-22 #369 entry). One predicate, so they cannot drift. The WHOLE run's key, or that key's LAST word (#489): a run is @@ -542,7 +542,7 @@ def _normpairs( # punctuation marks which letters are joined into one run # (_render._apply_mask reads that) and is never written into # the word. A raise, not a warning: a mismatched value used to - # be SUBSTITUTED for the word, and rules.md#R4: "Repair + # be SUBSTITUTED for the word, and rules/R4: "Repair # changes case and nothing else", so there is no reading of # one that repair can honor. if _alnum(_normalize(v)) != _alnum(normalized_key): @@ -599,7 +599,7 @@ class Lexicon: #: outright; written bare, capitals decide in a mixed-case name #: ("Jack MA" is the credential, "Jack Ma" keeps his family name) #: and otherwise the word counts as a suffix only where a name - #: word is left beside it ("John Smith ma"), rules.md#S2. Full + #: word is left beside it ("John Smith ma"), rules/S2. Full #: default list: #: :data:`~nameparser.config.suffixes.SUFFIX_ACRONYMS_AMBIGUOUS`. suffix_acronyms_ambiguous: frozenset[str] = frozenset() diff --git a/nameparser/_parser.py b/nameparser/_parser.py index faf8e85d..fadb2b6f 100644 --- a/nameparser/_parser.py +++ b/nameparser/_parser.py @@ -41,7 +41,7 @@ class Parser: vocabulary DECLINES, so a locale pack's surnames always win where they match; returning None declines in turn and the token stays whole. Two promises narrow when one is supplied (the first is - rules.md#A1's Accepted clause): + rules/A1's Accepted clause): parse-totality gains its one exception -- an exception raised by the segmenter propagates, because a user-supplied callable's own error is a user-code error, not a content error -- and this Parser @@ -220,7 +220,7 @@ def revise(self, name: ParsedName, **fields: str) -> ParsedName: # tags are KEPT, "joined" included: a within-piece mark # (the Ph. D. merge) is role-blind and right for every # role, and a clear was measured and backed out -- - # decisions.md#C1 (2026-09-06 #511) carries the rest: the + # decisions/C1 (2026-09-06 #511) carries the rest: the # between-piece mark that rides onto a non-suffix role, and # which views read it. Dropped tokens keep their role: the # pass filters them by index and assemble omits them. @@ -228,7 +228,7 @@ def revise(self, name: ParsedName, **fields: str) -> ParsedName: tok if i in dropped else dataclasses.replace(tok, role=role) for i, tok in enumerate(state.tokens)) - # rules.md#R1: "a run of post-nominals written with spaces + # rules/R1: "a run of post-nominals written with spaces # renders with spaces, and one written with commas keeps # them" -- the same pass post_rules runs last, over the # value's own comma offsets, so the harvest carries exactly @@ -346,13 +346,13 @@ def parser_for(*locales: Locale, base: Parser | None = None, # a subclass with extra mandatory args would break this rewrap raise type(exc)( f"while applying locale {loc.code!r}: {exc}") from exc - # rules.md#D1: "constructing a parser that activates division for + # rules/D1: "constructing a parser that activates division for # scripts with no covering surnames and no segmenter warns at # construction, naming the dead scripts and each way out" - # (history: decisions.md#D1) - # rules.md#D2: "applying a locale pack wraps any such error with + # (history: decisions/D1) + # rules/D2: "applying a locale pack wraps any such error with # the locale's code, so a stacked configuration names which layer - # broke" (history: decisions.md#D2) + # broke" (history: decisions/D2) # Construction warnings (the segmenterless-activation check in # Parser.__post_init__) re-emit from THIS frame: its stacklevel is # sized for direct Parser(...) construction, and through this diff --git a/nameparser/_pipeline/_assemble.py b/nameparser/_pipeline/_assemble.py index a3921d4f..363509b6 100644 --- a/nameparser/_pipeline/_assemble.py +++ b/nameparser/_pipeline/_assemble.py @@ -26,10 +26,10 @@ ) -# rules.md#A2: "a name with no name content parses to the empty +# rules/A2: "a name with no name content parses to the empty # name — every field empty, false as a boolean — while ambiguities # born from its punctuation survive on the empty result" -# (history: decisions.md#A2) +# (history: decisions/A2) def assemble(state: ParseState) -> ParsedName: dropped = set(state.dropped) final: dict[int, Token] = {} @@ -85,7 +85,7 @@ def assemble(state: ParseState) -> ParsedName: # referents did not lose a contest, they were discarded # wholesale, and the report still describes the input. continue - # rules.md#A1: "a report names the reading the parse took, so + # rules/A1: "a report names the reading the parse took, so # a report whose fork the rest of the parse then resolved to # NEITHER branch is withdrawn rather than carried beside a # reading it contradicts". classify's connective-or-initial diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 613a4fab..b3f30696 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -1,7 +1,7 @@ """Stage: assign. Consumes: pieces + piece_tags (grouped), segments, structure, tokens, -one_case, and tail_reads -- rules.md#S2's trailing run as group read it +one_case, and tail_reads -- rules/S2's trailing run as group read it once, after P3's joins and before the particle chain (#614), which the main segment's placement takes rather than peeling again, save where group's joins left no name word in front of it. @@ -16,7 +16,7 @@ applied by classify (plus the leading-title period rule). Implements rules H2, H4, H5, N3, O4, O5, W4, and P6 after a family comma, -of docs/design/rules.md, +of docs/design/rules/, each cited at its code below. Ports v1's assignment loops. NO_COMMA (per name_order): leading title pieces chain while no given-position name has been seen @@ -32,11 +32,11 @@ reading however many titles are written (_pieces.tail_reading). Where the run takes nothing -- almost every name -- the first peel is the only one and its answer stands. -The v1 single-name+nickname rule lives here (decisions.md#N3): a +The v1 single-name+nickname rule lives here (decisions/N3): a nonempty nickname beside exactly one piece in total puts that piece in FAMILY. FAMILY_COMMA: segment 0 FAMILY but for a leading run of listed titles -(rules.md#H6, #701; v1 kept them in the family) and the post-nominals +(rules/H6, #701; v1 kept them in the family) and the post-nominals behind its first family piece (SUFFIX, 'Smith Jr., John'). A part after the comma that holds no name word, or that a credential opens (#603), is not read here: group's head binds its SUFFIX and TITLE roles before @@ -112,7 +112,7 @@ def _set_roles(tokens: list[WorkToken], piece: tuple[int, ...], def _absorbed(piece: Sequence[int], tokens: Sequence[WorkToken]) -> PendingAmbiguity: - """rules.md#S2's report for a name word #602's run absorbs, one + """rules/S2's report for a name word #602's run absorbs, one spelling for the no-comma and the given-part run.""" text = " ".join(tokens[i].text for i in piece) return PendingAmbiguity( @@ -133,7 +133,7 @@ def _absorbed(piece: Sequence[int], _WORD_ALREADY_CLAIMED = _NEVER_FLIPPED | frozenset({"particle"}) -# rules.md#H2: "an abbreviation opening the part of the name that +# rules/H2: "an abbreviation opening the part of the name that # carries the given name — the whole name, or the part after a # family comma — reads as a title even when unlisted" -- the count is # _pieces.leading_titles since #424 (the chain's leading position is @@ -164,10 +164,10 @@ class EffectiveOrder(NamedTuple): by_script: bool -# rules.md#W4: "a name written wholly in one East Asian script, or in +# rules/W4: "a name written wholly in one East Asian script, or in # the kana-licensed Japanese repertoire, reads family-first whatever # order the caller declared; a wholly-katakana name keeps the declared -# order" (history: decisions.md#W4) +# order" (history: decisions/W4) def _effective_order(policy: Policy, pieces: Sequence[tuple[int, ...]], name_pieces: Sequence[int], @@ -202,7 +202,7 @@ def _effective_order(policy: Policy, shape every piece-layer predicate takes: a caller holding those indices had to build a second list of the same pieces to hand over otherwise, which on 3.11 is a comprehension frame on every - parse (decisions.md#parse-cost). + parse (decisions/parse-cost). Returns an EffectiveOrder: the order triple, and `by_script` set only on the one path where an entry answered. Every fallback below is a @@ -210,7 +210,7 @@ def _effective_order(policy: Policy, """ declared = EffectiveOrder(policy.name_order, by_script=False) # #298 transcription marker -- see the docstring; codepoint-scoped - # (only U+00B7 records; decisions.md#T3) + # (only U+00B7 records; decisions/T3) if dot_divided: return declared if not policy.script_orders: @@ -240,7 +240,7 @@ def _effective_order(policy: Policy, return declared -# rules.md#O4: "words no vocabulary has claimed read by position. In +# rules/O4: "words no vocabulary has claimed read by position. In # the default given-first order the first name word is the given name, # the last is the family name, and everything between is middle names" def _name_positions(order: tuple[Role, Role, Role], @@ -251,7 +251,7 @@ def _name_positions(order: tuple[Role, Role, Role], family, middles..., given. One piece takes order[0]'s role; two pieces take order[0] and the other primary.""" first, second = order[0], order[1] - # rules.md#O5: "a name of one name word that nothing else has + # rules/O5: "a name of one name word that nothing else has # decided reads that word as the given name under the default # given-first order, and as the family name under a declared # family-first one" -- a convention, not a determination: O4 has @@ -326,15 +326,15 @@ def _assign_main(seg_idx: int, state: ParseState, # from 3.12, where this tree's reference parse costs 395 written # either way (measured 2026-09-09, against 416 and 417 on 3.11). # 3.11 is the interpreter the band is quoted for, so 3.11 is what - # the shape is chosen on (decisions.md#parse-cost). + # the shape is chosen on (decisions/parse-cost). for k in range(n, len(pieces)): if "suffix" in ptags[k]: _set_roles(tokens, pieces[k], Role.SUFFIX) rest = peel_walk(n, ptags) if not rest: return None - # rules.md#N3: "a name that is only a nickname and one name word - # reads that word as the family name" (history: decisions.md#N3) + # rules/N3: "a name that is only a nickname and one name word + # reads that word as the family name" (history: decisions/N3) # -- v1's p_len == 1 counted # the WHOLE segment before any title peeling -- 'Xyz. (Bud) Smith' # has two pieces, so the title peel wins and Smith stays the given @@ -343,17 +343,17 @@ def _assign_main(seg_idx: int, state: ParseState, # every interpreter resumes once per token (seven call events on # the reference name), and only a one-piece segment ever reads its # answer. Hoisting it to the top of the read, where it once stood, - # is what put 3.12-3.15 one call over the band (decisions.md#parse-cost). + # is what put 3.12-3.15 one call over the band (decisions/parse-cost). if (len(pieces) == 1 and len(rest) == 1 and any(t.role is Role.NICKNAME for t in tokens)): _set_roles(tokens, pieces[rest[0]], Role.FAMILY) return None - # rules.md#S2's trailing peel and rules.md#H5's title chain, read + # rules/S2's trailing peel and rules/H5's title chain, read # together to their fixed point by _pieces.tail_reading -- one # function since the /simplify round, shared with the bound-given # reserve (P5), which must count the name words this leaves. # - # rules.md#H5: "successive single words that wear the abbreviation + # rules/H5: "successive single words that wear the abbreviation # shape and are title vocabulary chain into the title from the end, # leaving one name word standing" -- the titles are set BEFORE # _name_positions, so the shortened list is what the positional @@ -397,7 +397,7 @@ def _assign_main(seg_idx: int, state: ParseState, state.one_case) for piece_idx in titled_tail: _set_roles(tokens, pieces[piece_idx], Role.TITLE) - # rules.md#S2: "except a title word, which reads as a title" -- the + # rules/S2: "except a title word, which reads as a title" -- the # credential run (#602) is placed here, so a name word it absorbed # is reported here too for piece_idx in peeled.run_titles: @@ -436,7 +436,7 @@ def _assign_main(seg_idx: int, state: ParseState, # Both emitters below turn on the PEEL's count, so neither can # reach a name that kept two name pieces, and the two scans are # nested under the count rather than run beside it: they cost the - # call budget on every parse otherwise (decisions.md#parse-cost). + # call budget on every parse otherwise (decisions/parse-cost). if peeled.names <= 1: head = pieces[name_pieces[0]] # Both conventions here turn on a lone name word, so both @@ -455,9 +455,9 @@ def _assign_main(seg_idx: int, state: ParseState, field_undecided = ( not resolved.by_script and not titled and not any(t.role is Role.MAIDEN for t in tokens)) - # rules.md#H4: "an input whose every word is post-nominal + # rules/H4: "an input whose every word is post-nominal # vocabulary reads its first word as a name and reports - # `suffix-or-name`" (history: decisions.md#H4) -- only the + # `suffix-or-name`" (history: decisions/H4) -- only the # word made into a name reports, and no name piece survived # the peel, so the carve-out above made the first post-nominal # the name. Its field is worded by assemble from the final role @@ -479,11 +479,11 @@ def _assign_main(seg_idx: int, state: ParseState, # WORD a title), anything else O5's (which FIELD it took), so # only the second reads `field_undecided`. if peeled.names == 1 and not resolved.by_script: - # rules.md#H4: "an input whose only remaining name word + # rules/H4: "an input whose only remaining name word # after the title peel is itself title vocabulary reads # that word as the name by convention and reports - # `title-or-name`" (history: decisions.md#H4), and H4's - # join clause, stated at rules.md#O5 as its exception -- + # `title-or-name`" (history: decisions/H4), and H4's + # join clause, stated at rules/O5 as its exception -- # one branch, its detail naming a field only in the join # shape ('John of Prince', 'Attorney General of # Minnesota'), where the unit is more than the title word. @@ -502,10 +502,10 @@ def _assign_main(seg_idx: int, state: ParseState, f"vocabulary to a name word; read as ", tuple(head), field_tail=None if lone else " by convention")) - # rules.md#O5: "a name of one name word that nothing else has + # rules/O5: "a name of one name word that nothing else has # decided reads that word as the given name under the default # given-first order, and as the family name under a declared - # family-first one" (history: decisions.md#O5). Past + # family-first one" (history: decisions/O5). Past # `field_undecided`, two clauses of O5's own: the word's # own reading may have claimed it, and A2's content test # -- a piece with no alphanumeric character is no name @@ -531,7 +531,7 @@ def _assign_main(seg_idx: int, state: ParseState, assert token.role is not None declined = ("a name part" if token.role is Role.SUFFIX else "a post-nominal") - # A word in the class by SHAPE (rules.md#S2, #S3) is in no + # A word in the class by SHAPE (rules/S2, #S3) is in no # wordlist and may be written with its periods, so the listed # member's wording would misdescribe it on both counts (#563) what = ("is shaped like a post-nominal but listed in no " @@ -585,7 +585,7 @@ def _reads_as_a_trailing_suffix(piece: Sequence[int], a no-comma rule and the opposite of this path's v1 parity -- 'Chang, Andy C I' is first Andy, middle C, suffix I, and asking that predicate here made the numeral a middle. The #401/#421 entry - under decisions.md#P5 records the same fork declining to transfer + under decisions/P5 records the same fork declining to transfer to this walk under LENIENT. """ text = tokens[piece[0]].text @@ -599,7 +599,7 @@ def _inside_a_credential_run(seg: Sequence[Sequence[int]], tokens: list[WorkToken], given_at: int, k: int, end: int, ambiguities: list[PendingAmbiguity]) -> bool: - """rules.md#P6's third exception for the run `seg[k:end]`: a + """rules/P6's third exception for the run `seg[k:end]`: a run assign read as post-nominals, with a credential read in front of it and another behind, stands INSIDE the credential run assign read whole rather than ending the name, so it keeps that @@ -658,7 +658,7 @@ def _attach_particle_tail(pieces: Sequence[Sequence[int]], tokens: list[WorkToken], given_at: int, k: int, end: int, ambiguities: list[PendingAmbiguity]) -> None: - """rules.md#P6's attachment after a family comma: the particle run + """rules/P6's attachment after a family comma: the particle run `pieces[k:end]` ending the given part joins the family the comma named and is written before it -- the Dutch alphabetized listing, "Beethoven, Ludwig van" being how "Ludwig van Beethoven" is filed. @@ -681,7 +681,7 @@ def _attach_particle_tail(pieces: Sequence[Sequence[int]], leave a GIVEN word ahead of it, so "Nguyen, Van" keeps its only given word. Only the DEGENERATE Vietnamese listing is protected by that -- "Nguyen, Thi Van" has a given word to spare, so `Van` - attaches and the given name is lost. rules.md#P6 records why that + attaches and the given name is lost. rules/P6 records why that is accepted. mechanisms.md#FOLDED_TAG does the rest: tokens never move, so the @@ -712,7 +712,7 @@ def _attach_particle_tail(pieces: Sequence[Sequence[int]], # only words in both the particle and the unambiguous # suffix vocabularies, and neither is ambiguous particle # vocabulary -- but a caller's Lexicon may put one word in - # both, and rules.md#A1 says parsing never fails on any + # both, and rules/A1 says parsing never fails on any # input, so this decides instead of raising. (Measured: an # assert here DID fire on `Berg, Jan zz` under a Lexicon # adding `zz` to particles_ambiguous and suffix_acronyms.) @@ -796,11 +796,11 @@ def assign(state: ParseState) -> ParseState: fam_tags = state.piece_tags[0] # The part after the comma reaches here as the given part, or # bound and with no pieces left: group's `_comma.decide` binds a - # postnominal one before any join (#613, rules.md#C1), so this + # postnominal one before any join (#613, rules/C1), so this # path is the listing form's, and the positional read #296 gave # a comma followed by no name word behind two name words is # decide's suffix comma now, read by the NO_COMMA branch above. - # rules.md#C1's exception, scoped to the ambiguous credential + # rules/C1's exception, scoped to the ambiguous credential # class: this is the given-name half of the comma's OWN # decision (listing or credential run), where the writing left # the fork open. `_comma.decide` reports the credential half, @@ -848,7 +848,7 @@ def assign(state: ParseState) -> ParseState: # keeps as a name stops that run, which is why # 'Doe, John MA Ma' reports once and for 'Ma' alone -- 'MA' # then has a name word behind it and is never asked - # (rules.md#S2). + # (rules/S2). if state.pieces[1] and len(state.pieces[1][0]) == 1: i = state.pieces[1][0][0] if not tokens[i].tags.isdisjoint(_AMBIGUOUS_CREDENTIAL_TAGS): @@ -957,7 +957,7 @@ def trailing_floor(m: int, titled: frozenset[int]) -> int: down as `m` needs it: `m >= trailing_floor(m, titled)` is exactly "every kept piece behind `m` reads as a suffix", which is what ENDING the given part means - (rules.md#S2, #531). + (rules/S2, #531). ONE walk per `titled` value, shared by the predicate below and by that rule's report at the foot of this @@ -1030,7 +1030,7 @@ def attaches_the_lead(m: int, titled: frozenset[int]) -> bool: member's writing says nothing: capitals in a mixed-case name still make it the credential ('Smith, de MA' keeps suffix 'MA'), as they do with no words to spare - anywhere (rules.md#S2). Asked only of a class member + anywhere (rules/S2). Asked only of a class member the slot would otherwise take.""" if not is_lone_never_given_particle(pieces[n], tokens): return False @@ -1077,7 +1077,7 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: """ if is_suffix_piece(pieces[m], ptags[m], tokens): return True - # rules.md#S2, the given part's trailing slot (#531). + # rules/S2, the given part's trailing slot (#531). # INLINE, and that is the frame budget talking rather # than taste: the candidates pass asks this closure once # per piece of every family-comma segment, so a helper @@ -1135,7 +1135,7 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: # leans nothing, so the positional reading # would take it -- the wrong answer there, # not merely a stray report - # (decisions.md#S2, 2026-09-18). + # (decisions/S2, 2026-09-18). # # A FUNCTION since #533 rather than a # condition written here (mechanisms.md @@ -1171,15 +1171,15 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: pieces[m], pieces[prev], ptags[prev], tokens, state.lexicon)) - # rules.md#C1: "anything else after the comma means the + # rules/C1: "anything else after the comma means the # listing form" -- and this walk reads it. A segment of no # name word never reaches here: it is `_comma.decide`'s, read piece by piece and bound at - # group's head (#613, rules.md#C1), the suffix verdict + # group's head (#613, rules/C1), the suffix verdict # before the title reading of the same word (#296, #325). # A name word in the segment makes it v1's walk ('Smith, # John Jr.'). n = _peel_leading_titles(pieces, ptags, tokens) - # rules.md#H5: "the title is TRANSPARENT to the suffix + # rules/H5: "the title is TRANSPARENT to the suffix # reading: where two or more name words stand, what # stands once the chain is taken reads exactly as it # would read written without the title, plus the title" @@ -1248,7 +1248,7 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: # memo, not a second spelling of the question, and it # cannot have gone stale because nothing the chain does # moved the end of the name. - # rules.md#S2: "or after the given word in the part after a + # rules/S2: "or after the given word in the part after a # family comma" -- the given part's own credential run # (#602), the comma having fixed the family. The tag tests are inline ahead of the # predicate, as in `_pieces.credential_run`, so a name word @@ -1264,7 +1264,7 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: tokens)): sticky_from = m break - # rules.md#P6: "a particle ending the name attaches to that + # rules/P6: "a particle ending the name attaches to that # family name" -- decided here, in the pass that reads the # given part, rather than predicted for post_rules to take # a stage later (#613). The tail is `pieces[k6:end6]`, found @@ -1340,7 +1340,7 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: # only if a reader would hesitate too" is why that # must stay silent rather than why it happens to (the # sentence is the 2.0-conventions section's, not - # rules.md#A1's, which this cited until 2026-09-19). + # rules/A1's, which this cited until 2026-09-19). # # The `do` carve-out is a REPORT carve-out on top of # the reading one above: a particle-tagged member this @@ -1414,7 +1414,7 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: tokens)): _attach_particle_tail(pieces, tokens, n, k6, end6, ambiguities) - # rules.md#H6: "A run of title words opening the part before a + # rules/H6: "A run of title words opening the part before a # family comma is the title" -- by vocabulary only, never H2's # shape, leaving one piece for the family, and stopping at a # word that is a particle too ('St Pierre', 'Freiherr von @@ -1449,7 +1449,7 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: and "particle" not in tokens[fam_pieces[lead][0]].tags): _set_roles(tokens, fam_pieces[lead], Role.TITLE) lead += 1 - # rules.md#P2: a particle "joins the words after it into one + # rules/P2: a particle "joins the words after it into one # name part" -- so a particle that is suffix vocabulary too # (vd, mc) with a name piece behind it in this part heads # that name, and is not peeled from between two family words @@ -1478,7 +1478,7 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: else: _set_roles(tokens, piece, Role.FAMILY) tail = 2 - # rules.md#C2: segments past the structure's name segments are + # rules/C2: segments past the structure's name segments are # consumed as suffixes, except "a word of the title vocabulary there # that is not also suffix vocabulary is a title" (#603) -- a part # past the second comma holds no name word, so the surnames H5 diff --git a/nameparser/_pipeline/_classify.py b/nameparser/_pipeline/_classify.py index b104497e..6e117797 100644 --- a/nameparser/_pipeline/_classify.py +++ b/nameparser/_pipeline/_classify.py @@ -70,7 +70,7 @@ -# rules.md#S2: "a trailing word of the suffix vocabulary reads as a +# rules/S2: "a trailing word of the suffix vocabulary reads as a # suffix — generational forms and credential acronyms alike, and an # ambiguous acronym written with its periods, one after each # letter or one after each of two or more letter chunks, counts @@ -90,7 +90,7 @@ def _tags_for(token: WorkToken, n: str, state: ParseState, `one_case_own` is true when the name's OWN words are written in one case AND this token is one of the name's own words -- a maiden clause and any delimited (nickname) content are not, so the fork never - reads them either (rules.md#P3): a clause's words are not the + reads them either (rules/P3): a clause's words are not the name's own words, and appending one must not change how THIS token reads. @@ -121,19 +121,19 @@ def _tags_for(token: WorkToken, n: str, state: ParseState, tags.add("vocab:suffix-word") if n in lex.suffix_acronyms_ambiguous: tags.add(AMBIGUOUS_ACRONYM_TAG) - # rules.md#P7: "A one-letter word written with its period is an + # rules/P7: "A one-letter word written with its period is an # initial and not a particle" -- both particle tags, so 'Ó.' is # neither half of the vocabulary if n in lex.particles and not _spells_an_initial(n, token.text): tags.add("particle") if n in lex.particles_ambiguous: tags.add("vocab:particle-ambiguous") - # rules.md#P3: "a single-letter connective reads as an initial + # rules/P3: "a single-letter connective reads as an initial # where the writing says so: written as a bare Latin capital in a # name that is not written wholly in one case, or — in a name # written wholly in one case, where nothing says so — where the # letter is one the vocabulary marks as reading both ways" - # (#383/#479; history: decisions.md#P3) + # (#383/#479; history: decisions/P3) # one letter as the writer sees it: a decomposed 'й' is two code # points (#665), so non-ASCII text is counted composed, after the # C-level tests have declined everything else @@ -246,7 +246,7 @@ def _tags_for(token: WorkToken, n: str, state: ParseState, # tags there when it binds the part (#564, #613), so the # reading carries the same marks under either setting -- # case repair keeps 'XYZ' in 'John Smith, XYZ' - # (rules.md#R4) rather than title-casing a credential the + # (rules/R4) rather than title-casing a credential the # default admitted. # The setting comes FIRST, then C-level checks of what the # predicate would decline, so the default never calls it @@ -268,7 +268,7 @@ def classify(state: ParseState) -> ParseState: folded = [_normalize(t.text) for t in state.tokens] marker_tags = tag_marker_runs(state.tokens, state.comma_offsets, state.lexicon.maiden_markers, folded) - # rules.md#P3 says a maiden marker, taken as one, and the words it + # rules/P3 says a maiden marker, taken as one, and the words it # takes, are not among the name's own words -- so the span and its # clause cut are _pieces.own_words', shared with the site that # needs the same answer two stages earlier (#289/#516). The marker @@ -281,7 +281,7 @@ def classify(state: ParseState) -> ParseState: # ONE fact per parse, and it is recorded now (ParseState.one_case): # segment writes it first where C2's flag on a tail part asks, and # a fact two stages decide apart is what recording it prevents - # (decisions.md#S2). + # (decisions/S2). one_case = state.one_case if one_case is None: one_case = is_one_case(own) @@ -331,9 +331,9 @@ def classify(state: ParseState) -> ParseState: # hands classify tokens it did not build; no such path exists # today. `conjunctions` is the clause doing real work: it keeps # an orphan marker (a conjunctions_ambiguous entry no longer in - # conjunctions) inert rather than reported (decisions.md#P3). + # conjunctions) inert rather than reported (decisions/P3). # `i < clause_at and token.role is None` mirrors the fork's own - # "own words" test above, for the same reason (rules.md#P3): a + # "own words" test above, for the same reason (rules/P3): a # clause's words were never eligible for the fork, so they must # never be eligible to report either. Emitted at the decision # site (mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE), per diff --git a/nameparser/_pipeline/_comma.py b/nameparser/_pipeline/_comma.py index 9010436d..8771e49c 100644 --- a/nameparser/_pipeline/_comma.py +++ b/nameparser/_pipeline/_comma.py @@ -1,4 +1,4 @@ -"""Not a stage: rules.md#C1's decision, made by group at its head. +"""Not a stage: rules/C1's decision, made by group at its head. The comma's structure is decided ONCE, after classify has tagged every word and before group joins any of them (#613): the part after the @@ -76,7 +76,7 @@ def _reading_pieces(part: Sequence[int], tokens: Sequence[WorkToken], can hold.""" pieces, ptags = _pieces(part, tokens) # The one join a postnominal part can hold, made here so the reading - # sees it: rules.md#P3's connective joining its neighbours where the + # sees it: rules/P3's connective joining its neighbours where the # neighbour it takes its kind from is a title, which makes the # joined part a title ('Mr. and Mrs.', 'Secretary of State', 'Dr. # and'), a run of connectives merging first as it does in the name @@ -90,7 +90,7 @@ def _reading_pieces(part: Sequence[int], tokens: Sequence[WorkToken], # Dr.'). Group's own joins, the one loop since #617: its copy here # merged no run and joined 'Minister of the' alone. # - # rules.md#P3: "A connective that is also generational vocabulary + # rules/P3: "A connective that is also generational vocabulary # joins only where a name word stands on each side of it" -- and in # a part read for credentials none does, so every such connective # is frozen here, where group freezes only the ones it finds without @@ -147,7 +147,7 @@ def _name_contrast(state: ParseState, seg0: Sequence[int]) -> bool: def _whole_name(seg0: Sequence[int], tokens: Sequence[WorkToken]) -> bool: - """rules.md#C1's count before the comma for an unambiguous + """rules/C1's count before the comma for an unambiguous credential, read off classify's tags (the ambiguous class's count of NAME words, which leaves titles out, is `_vocab.name_word_count`'s, asked in `decide`): two or more units (a particle run and @@ -191,7 +191,7 @@ def decide(state: ParseState) -> ParseState: return state tokens = list(state.tokens) lex, pol = state.lexicon, state.policy - # rules.md#C1: "A delimiter the policy declares parts a trailing + # rules/C1: "A delimiter the policy declares parts a trailing # suffix part as a comma would" -- behind a whole name its cores are # no word of the reading, and leave as group drops them in a tail; # behind one name word the core is a word, v1 having applied the @@ -246,7 +246,7 @@ def decide(state: ParseState) -> ParseState: if maybe: names = name_word_count([tokens[i].text for i in seg0], lex, pol) if names >= 2: - # rules.md#C1: "An unlisted all-caps word joins the class in such + # rules/C1: "An unlisted all-caps word joins the class in such # a part only where the name carries the contrast" contrast: bool | None = None for i in maybe: @@ -267,7 +267,7 @@ def decide(state: ParseState) -> ParseState: # for, a lone one included ('John Smith, XY') if contrast: caps.add(i) - # rules.md#C1: "The same count reads a part of two or more words + # rules/C1: "The same count reads a part of two or more words # as the credential run when every word of it is a suffix word # or a word of this class", none of them a single-letter numeral. # the words not of the class are asked as C1's suffix test asks @@ -290,7 +290,7 @@ def decide(state: ParseState) -> ParseState: if not run: licensed.clear() else: - # rules.md#C1: "Paired initials are the exception to the + # rules/C1: "Paired initials are the exception to the # count" -- "Only an unambiguous suffix word in front of # them that is not also title vocabulary, or another word # the class admits by its dotted shape standing in the same @@ -333,7 +333,7 @@ def decide(state: ParseState) -> ParseState: anchored, absorbed, licensed) if whole is None: whole = _whole_name(seg0, tokens) - # rules.md#C1: "By default a recognized suffix word counts even + # rules/C1: "By default a recognized suffix word counts even # written like an initial" -- behind a whole name ('John Smith, V.') if reading is None and whole and is_wholly_suffix( [tokens[i].text for i in part], lex, pol, one_case=None): @@ -350,10 +350,10 @@ def decide(state: ParseState) -> ParseState: tags = tokens[i].tags if i in caps: # #564: the caps shape's marks, as classify writes them - # EVERYWHERE, so case repair keeps 'XYZ' (rules.md#R4) + # EVERYWHERE, so case repair keeps 'XYZ' (rules/R4) tags = tags | {SHAPE_ACRONYM_TAG, AMBIGUOUS_ACRONYM_TAG} tokens[i] = copy_with(tokens[i], role=role, tags=tags) - # rules.md#C1: "A decision either way at this comma is reported; + # rules/C1: "A decision either way at this comma is reported; # for a run of words the decision is the flip to the credential # run, reported once over the whole part. A flip in which no listed # word of this class takes part is the exception and is made in @@ -406,7 +406,7 @@ def decide(state: ParseState) -> ParseState: AmbiguityKind.SUFFIX_OR_NAME, f"{tokens[i].text!r} after the comma is also an ordinary name " f"word; read as a credential", (i,))) - # rules.md#S2's company, reported where it decided: a member the + # rules/S2's company, reported where it decided: a member the # anchor read as a credential after its own writing declined (#544) for k in anchored: i = p1[k][0] @@ -414,7 +414,7 @@ def decide(state: ParseState) -> ParseState: AmbiguityKind.SUFFIX_OR_NAME, f"{tokens[i].text!r} behind a credential after the comma is " f"also an ordinary name word; read as a credential", (i,))) - # rules.md#C1: "A credential opening the part makes it the + # rules/C1: "A credential opening the part makes it the # postnominal part however it goes on" -- each word it takes that # would have made it the given part, as #602's run reports them for k in absorbed: diff --git a/nameparser/_pipeline/_extract.py b/nameparser/_pipeline/_extract.py index 56fd1eb5..a9066bd1 100644 --- a/nameparser/_pipeline/_extract.py +++ b/nameparser/_pipeline/_extract.py @@ -17,7 +17,7 @@ Lexicon.suffix_words / suffix_acronyms / suffix_acronyms_ambiguous through _suffix_shaped. -Implements rules N1, N2, S1, M1 and M3 of docs/design/rules.md (the #273 +Implements rules N1, N2, S1, M1 and M3 of docs/design/rules/ (the #273 matching mechanism); each is cited at its code below. One scan mechanic worth stating up front: matching is one left-to-right pass, no nesting, and delimiter characters inside a matched region are @@ -44,7 +44,7 @@ from nameparser._types import AmbiguityKind, Role, Span -# rules.md#S1: "a bracketed clause whose content is suffix-shaped is +# rules/S1: "a bracketed clause whose content is suffix-shaped is # not a nickname: the brackets are dropped and the content reads # exactly as if written bare" def _suffix_shaped(content: str, lexicon: Lexicon) -> bool: @@ -60,7 +60,7 @@ def _suffix_shaped(content: str, lexicon: Lexicon) -> bool: or content.endswith(".")) -# rules.md#M3: "a bracketed clause whose content opens with a +# rules/M3: "a bracketed clause whose content opens with a # recognized marker and carries a word after it reads as the # maiden name, whichever bucket the enclosing pair sits in" def _maiden_marked(content: str, lexicon: Lexicon) -> bool: @@ -80,7 +80,7 @@ def _maiden_marked(content: str, lexicon: Lexicon) -> bool: WHITESPACE, so a marker the writer glued to punctuation is not one here ('née,') -- nor to classify, since #653 keeps a clause's comma on its word. A nakaguro still divides the two: the tokenizer splits - '旧姓・田中' at it (rules.md#T2) and classify tags '旧姓', which is + '旧姓・田中' at it (rules/T2) and classify tags '旧姓', which is what keeps _group's Role.MAIDEN filter reachable (cases.py's marker_divided_by_a_nakaguro_keeps_the_clause_a_nickname). Sharing the PREDICATE does not merge the two @@ -97,7 +97,7 @@ def _maiden_marked(content: str, lexicon: Lexicon) -> bool: and not comma_follows_marker(words[run])) -# rules.md#N2: "a quote whose open and close are the same character +# rules/N2: "a quote whose open and close are the same character # opens only at a word start and closes only at a word end, so an # apostrophe inside or at the end of a word is literal" def _open_ok(text: str, i: int) -> bool: @@ -169,11 +169,11 @@ def _unmatched(open_: str, offset: int) -> tuple[int, PendingAmbiguity]: )) -# rules.md#N1: "a clause enclosed by a configured nickname delimiter +# rules/N1: "a clause enclosed by a configured nickname delimiter # pair reads as the nickname and is lifted out of the name" (its # punctuation is tokenize's half of the rule) -# rules.md#M1: "with a delimiter pair configured for maiden names, its -# enclosed clause reads as the maiden name" (history: decisions.md#M1) +# rules/M1: "with a delimiter pair configured for maiden names, its +# enclosed clause reads as the maiden name" (history: decisions/M1) def extract_delimited(state: ParseState) -> ParseState: text = state.original policy = state.policy diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index e0ce9ae8..8a6178a7 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -6,7 +6,7 @@ `_comma.decide`, comma_offsets (the own-words span of the name's case contrast) and dropped, which decide extends. Produces: structure -- its head decides a comma form once the words -are tagged (rules.md#C1, #613, `_comma.decide`), binding the SUFFIX and +are tagged (rules/C1, #613, `_comma.decide`), binding the SUFFIX and TITLE roles of a postnominal part after the comma, emptying its segment and dropping its delimiter cores before any join, plus the SUFFIX_OR_NAME ambiguities that decision reports and the caps shape's acronym tags on @@ -15,7 +15,7 @@ anti-#100 invariant); maiden tail tokens get role=MAIDEN, and the trailing run a maiden take gives up gets its SUFFIX and TITLE roles here (#601), so no join can reach it; marker tokens land in dropped; -and tail_reads, rules.md#S2's trailing run of each main segment read +and tail_reads, rules/S2's trailing run of each main segment read ONCE, after P3's connective joins and before the particle chain and the bound-given join, and split off so neither reaches it (#614), which assign places and reports rather than peeling again. @@ -37,7 +37,7 @@ content, which M2's pieces walk cannot reach because extract's content never enters pieces); each is cited at its code below. Also implements rule P5 (cited below at the bound-given join) and ports -the "Ph. D."-split merge (v1 fix_phd; decisions.md#phd-merge). +the "Ph. D."-split merge (v1 fix_phd; decisions/phd-merge). The piece-level predicates moved to _pieces in #439 -- the S2 trailing peel, the leading-title and title-piece tests, the @@ -50,7 +50,7 @@ mechanisms.md#ONE-PREDICATE-PER-QUESTION describes; group imports them back like any other caller, and still does the work H3 and S2 describe with them. is_prefix_piece and is_conj_piece followed in #617, with -the merge and rules.md#P3's connective joins (join_connectives), when +the merge and rules/P3's connective joins (join_connectives), when the comma decision's own copy of those joins was folded into them. What remains defined here is group's own: _is_rootname and _is_maiden_marker_piece. @@ -106,7 +106,7 @@ class ClauseSite(IntEnum): """Where a maiden clause stands, which decides whether a marker counts there at all and which rule reads the end of the part - (rules.md#M2, #601). + (rules/M2, #601). A marker counts only in the part holding the family name, so GIVEN_SLOT and TAIL are statements, not defaults: a marker there is @@ -144,9 +144,9 @@ class BoundJoin(IntEnum): STRICT = 2 # main segments (reserve_last=True: keep a family piece) -# rules.md#M2: "A marker that counts takes the words after it as the +# rules/M2: "A marker that counts takes the words after it as the # maiden name, and is itself dropped" -- where a marker counts, and up -# to what, are `_maiden_take`'s (history: decisions.md#M2) +# to what, are `_maiden_take`'s (history: decisions/M2) # # A marker piece is a LONE marker -- M2's own "standing as a word of # its own". The consumer runs before every join but the Ph. D. merge @@ -243,7 +243,7 @@ def _maiden_take(pieces: Sequence[Sequence[int]], an ordinary word. Piece indices into `pieces`, not the TOKEN indices `_group_segment` builds out of them. - Three questions, in order (rules.md#M2, #601): + Three questions, in order (rules/M2, #601): * WHERE A MARKER COUNTS. Only in the part holding the family name -- never in the given part after a family comma, never in a tail @@ -354,7 +354,7 @@ def _maiden_take(pieces: Sequence[Sequence[int]], #: its left and on its right that is NOT a connective -- `-1` and #: `len(pieces)` where the run reaches the end. Built by #: `_run_neighbours`, and an ALIAS rather than a NamedTuple because a -#: NamedTuple's __new__ is a frame of its own (decisions.md#parse-cost). +#: NamedTuple's __new__ is a frame of its own (decisions/parse-cost). _Beside = tuple[list[int], list[int]] @@ -436,7 +436,7 @@ def _is_rootname(piece: Sequence[int], ptags: Set[str], tokens: Sequence[WorkToken]) -> bool: if len(piece) == 1 and "initial" in tokens[piece[0]].tags: return False - # rules.md#P3: "A connective counts as a name word wherever this + # rules/P3: "A connective counts as a name word wherever this # rule counts them, whatever else the vocabulary says the word is" # (#397). The order against the `initial` refusal above decides # nothing, and what makes that safe lives in classify rather than @@ -455,7 +455,7 @@ def _is_rootname(piece: Sequence[int], ptags: Set[str], or is_suffix_piece(piece, ptags, tokens)) -# rules.md#P3: "a word the rest of the parse reads as a name word +# rules/P3: "a word the rest of the parse reads as a name word # rather than as a generation, a credential or an honorific, looked # for past any run of connectives standing between. A connective with # nothing to its right is connecting nothing, and a word of that @@ -568,7 +568,7 @@ def _group_segment(seg: tuple[int, ...], additional: int, # the TRAILING site, which group() never suppresses. # ph-d merge first: "Ph." "D." adjacent -> one suffix piece - # (decisions.md#phd-merge; v1 fix_phd did this by regex on the + # (decisions/phd-merge; v1 fix_phd did this by regex on the # raw string) # A suffix never BEGINS a name: position outranks the vocabulary # match (#371). This merge is what makes a leading "Ph." "D." a @@ -604,7 +604,7 @@ def _group_segment(seg: tuple[int, ...], additional: int, # where the NAME begins, this asks where the STRING does. `Sir # Ph. D. Van Johnson` keeps suffix 'Ph. D.' with an empty family, # which is #371's symptom surviving one word to the left -- 1.4.0 - # parity, stated as a boundary in rules.md#S2 rather than fixed + # parity, stated as a boundary in rules/S2 rather than fixed # here, because "the first piece of the name" cannot be computed # before this merge: `is_leading_title` is true of `Ph.` itself, # so the scan would step over the very piece being judged. @@ -619,9 +619,9 @@ def _group_segment(seg: tuple[int, ...], additional: int, else: k += 1 - # rules.md#M2: "A marker that counts takes the words after it as + # rules/M2: "A marker that counts takes the words after it as # the maiden name, and is itself dropped" (history: - # decisions.md#M2) -- the marker pass (#274), and it runs + # decisions/M2) -- the marker pass (#274), and it runs # BEFORE every join below. Each join rule asks a question about the # name -- how many words it has (P3's carve-out, P5's reserve), # what the word after a particle or a bound word is -- and the @@ -654,7 +654,7 @@ def _group_segment(seg: tuple[int, ...], additional: int, tail_pieces: list[Piece] = [] tail_ptags: list[set[str]] = [] if len(pieces) + additional >= 3: - # rules.md#P3: "A connective that is also generational + # rules/P3: "A connective that is also generational # vocabulary joins only where a name word stands on each side # of it" (#397). `frozen` holds the TOKEN index of every such # connective missing that name word on one side or the other. @@ -772,7 +772,7 @@ def _group_segment(seg: tuple[int, ...], additional: int, # the `y` join). join_connectives(pieces, ptags, tokens, letter_stays=total < 4, titles_only=False, frozen=frozen) - # rules.md#S2's trailing run is read ONCE, here -- after P3's + # rules/S2's trailing run is read ONCE, here -- after P3's # connective joins, which do not depend on it and make the # units it counts, and before the particle chain and the # bound-given join, which do -- and bound (#614): the run is @@ -805,7 +805,7 @@ def _group_segment(seg: tuple[int, ...], additional: int, tail_pieces, tail_ptags = pieces[start:], ptags[start:] del pieces[start:] del ptags[start:] - # rules.md#S2: "a trailing word of the suffix vocabulary reads as a + # rules/S2: "a trailing word of the suffix vocabulary reads as a # suffix" -- on a main segment the run read above decides that # and is split off already; elsewhere group does not decide # it, and stops before whatever @@ -815,13 +815,13 @@ def _group_segment(seg: tuple[int, ...], additional: int, # (#424). The maiden take reads the title-aware # `tail_reading` over the clause-free view instead (#601), so the # clause ends where assign's peel and title chain together begin. - # rules.md#P2: "a particle joins the words after it into one name + # rules/P2: "a particle joins the words after it into one name # part, the join running until the next particle starts a group of # its own, a trailing suffix begins" -- and on to the maiden marker # (M2) or the name's end; the final group reads as the family name, - # earlier groups by position. (history: decisions.md#P2) - # rules.md#P4: "a particle in the name's leading position chains - # nothing: the words stay separate" (history: decisions.md#P2) + # earlier groups by position. (history: decisions/P2) + # rules/P4: "a particle in the name's leading position chains + # nothing: the words stay separate" (history: decisions/P2) # # prefix chains: a non-leading prefix run absorbs everything to # the next prefix or suffix (v1's leading_first_name rule keeps @@ -893,7 +893,7 @@ def _group_segment(seg: tuple[int, ...], additional: int, # assign read both words as titles. name_start = leading_titles(pieces, ptags, tokens) leading = chain_lead(pieces, ptags, tokens, name_start) - # rules.md#P2: "a trailing suffix begins" -- where it begins + # rules/P2: "a trailing suffix begins" -- where it begins # is read by assign's peel over the pieces as they stand # (#424), once per segment and kept as a length from the end, # which the chain's merges ahead of it do not move -- except @@ -910,7 +910,7 @@ def _group_segment(seg: tuple[int, ...], additional: int, # the pieces the fork counted (below). # the run read above is already split off (#614), so the chain # stops at the end of what is left. - # rules.md#H5: "the title is TRANSPARENT to the suffix reading" + # rules/H5: "the title is TRANSPARENT to the suffix reading" # -- in the given part after a family comma too, where assign's # walk reads the trailing title run, so the chain stops before # the run of suffixes and titles `tail_reading` takes there @@ -920,7 +920,7 @@ def _group_segment(seg: tuple[int, ...], additional: int, # #623). The bound matters only where a particle stands past the # chain's lead, so it is not asked otherwise: the chain then # merges nothing (the test is `is_prefix_piece`'s, inline). - # rules.md#C2: "a part past the second comma holds no name word" + # rules/C2: "a part past the second comma holds no name word" # -- assign reads such a part wholly as suffixes and its title # words as titles, so the chain builds no name unit there and is # not run, and a title word behind a particle run stays a word @@ -1029,7 +1029,7 @@ def chain(tail: int) -> None: f"following name piece; it is also a given " f"name in other names", (i,))) - # rules.md#S2: "A BARE ambiguous acronym is consumed + # rules/S2: "A BARE ambiguous acronym is consumed # only when the name has words to spare — as the second # of two words it stays the family name — and at the # slots that report, either reading carries the @@ -1162,8 +1162,8 @@ def chain(tail: int) -> None: peel.names, peel.numeral, tuple([p for p in peel.picks if p not in took]), peel.anchors, peel.run_titles, peel.absorbed)) - # rules.md#P5: "a recognized bound given-name word joins the - # word after it into one given name" (history: decisions.md#P5) + # rules/P5: "a recognized bound given-name word joins the + # word after it into one given name" (history: decisions/P5) # -- bound given names: the first non-title piece joins the next # ONCE (pairwise, v1 parity: 'Salem, Abdul Rahman Ahmed' keeps # Ahmed a middle name). BoundJoin encodes v1's reserve_last. @@ -1185,18 +1185,18 @@ def chain(tail: int) -> None: # nee' clears the LENIENT reserve the same way. Measured # rather than assumed -- dropping this undid #411 on # exactly that row. Nor is a suffix piece (#421) -- - # rules.md#P5: "nor a word of the unambiguous suffix + # rules/P5: "nor a word of the unambiguous suffix # vocabulary (S2), wherever position will then place it" # -- and declining it is also what keeps merge_pieces' tag # union from making the joined piece a suffix piece. - # rules.md#P5: "nor a title word a trailing rule reads as a + # rules/P5: "nor a title word a trailing rule reads as a # title" -- after a family comma one H5's trailing run takes # from the given part, read as the comma-less name reads it, # which the LENIENT join reads no peel for and so asks here, # only where a title word follows ('Berg, abdul Prof.', # #637; where the given part's own walk then reads the words # behind it otherwise, the title stays a name word, which - # rules.md#P5 accepts); in a part past the second + # rules/P5 accepts); in a part past the second # comma any title word, which C2 reads as a title ('John # Smith, Jr., abdul Dr. Smith', #640). A title word standing # in the name is a name word and joins ('abdul Sir Smith @@ -1215,19 +1215,19 @@ def chain(tail: int) -> None: elif bound_join is BoundJoin.LENIENT: # post-comma the family is fixed and the pair is the # given whatever follows, so no peel is read - # (decisions.md#P5, #423), save the title test above + # (decisions/P5, #423), save the title test above merge_pieces(pieces, ptags, fk, fk + 2, drop={"title"}) else: - # rules.md#P5: "the trailing suffix run (S2) and the + # rules/P5: "the trailing suffix run (S2) and the # trailing title run (H5), each read over what the other # leaves until neither takes anything more, are read # once, before the join is tried, and the name words they - # leave are the words to spare" (history: decisions.md#P5). + # leave are the words to spare" (history: decisions/P5). # That reading is the one above, after P3's joins and # before the chain (#614): the run is # split off, every piece left is a name piece, and the # join leaves one fewer. Nor can the join move a suffix - # -- rules.md#P5: "a word the run left as a name word + # -- rules/P5: "a word the run left as a name word # stays one, whatever the join makes of the word in # front of it" -- since nothing in group reads the run # again after it. @@ -1261,7 +1261,7 @@ def chain(tail: int) -> None: == [tuple(pieces[j]) for j in rest[before.names:]]) chained = fk + 1 in chain_took names_after = after.names - # rules.md#P5: "a trailing roman numeral, or a bare + # rules/P5: "a trailing roman numeral, or a bare # acronym the run takes, or a trailing title word the # run takes, is no word to spare" -- and the join joins # two NAME words, so a piece the chain takes is no more @@ -1310,7 +1310,7 @@ def chain(tail: int) -> None: if (not chained and same_suffixes and names_after >= reserve): # the pair is a given name whatever tag the word - # carried (rules.md#P5); joined_tags says why the + # carried (rules/P5); joined_tags says why the # title tag is dropped. Pinned in test_group.py. merge_pieces(pieces, ptags, fk, fk + 2, drop={"title"}) if reads is not None: @@ -1321,7 +1321,7 @@ def chain(tail: int) -> None: def group(state: ParseState) -> ParseState: - # rules.md#C1's decision, read once now that classify has tagged + # rules/C1's decision, read once now that classify has tagged # the words and before any join below can reach the part after the # comma (#613); a postnominal part comes back bound, its segment # emptied (a name with no comma asks nothing) @@ -1353,8 +1353,8 @@ def group(state: ParseState) -> ParseState: # suppresses it there: the family name is already fixed, so # there is no fork left to report. A tail segment needs no # suppression: group makes no particle chain there (#636) and - # takes no maiden clause (rules.md#M2, #601), so nothing in group - # reports in one. rules.md#C2: "a part the parse consumes wholly + # takes no maiden clause (rules/M2, #601), so nothing in group + # reports in one. rules/C2: "a part the parse consumes wholly # as suffixes raises no report that a word of it was read as a # name" tail = tail_start is not None and seg_idx >= tail_start @@ -1368,7 +1368,7 @@ def group(state: ParseState) -> ParseState: else ClauseSite.FAMILY_PART) else: site = ClauseSite.TRAILING - # rules.md#C1: "a delimiter the policy declares parts a + # rules/C1: "a delimiter the policy declares parts a # trailing suffix part as a comma would" (#549): the segment is # grouped as the parts between its cores, each read exactly as # the comma twin's own segment would be, and the cores are @@ -1433,7 +1433,7 @@ def group(state: ParseState) -> ParseState: # index, ORed since #429 with segment_suffix_reading's # per-piece content verdict, gated per piece and sticky across # an interleaved title. post_rules derives that from the - # commas the writer typed instead (#436/#437, rules.md#R1), + # commas the writer typed instead (#436/#437, rules/R1), # which is what the input answers directly. The stickiness # survives by construction rather than by code: # 'Smith, MD Dr. PhD' has no comma between MD and PhD and a @@ -1451,7 +1451,7 @@ def group(state: ParseState) -> ParseState: all_pieces.append(tuple(tuple(p) for p in pieces)) all_ptags.append(tuple(frozenset(t) for t in ptags)) all_reads.append(seg_reads[0] if len(seg_reads) == 1 else None) - # rules.md#M1: "a leading recognized marker being dropped where + # rules/M1: "a leading recognized marker being dropped where # the clause holds a word past it; a clause of nothing but its # marker keeps its words" — a marker inside EXTRACTED maiden # content (#329). @@ -1523,7 +1523,7 @@ def group(state: ParseState) -> ParseState: # test, since the tokens inside a clause are contiguous in # index order -- a clause holding nothing but its marker # keeps its words, exactly as a one-token "(Nee)" does. - # rules.md#M3: "a marker followed by a comma is no marker" + # rules/M3: "a marker followed by a comma is no marker" # -- asked of the same word extract's M3 test asks it of if (first + run < len(tokens) and tokens[first + run].span.end <= clause.end diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index 8fd953f4..ddaca7b8 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -29,7 +29,7 @@ it, and tests/v2/test_layering.py is where it is written down -- a piece predicate may not depend on a stage, in either direction. -The S2 trailing peel travels as the unit decisions.md describes -- +The S2 trailing peel travels as the unit the decisions describe -- peel_walk, peel_trailing and trailing_start together -- though only the first two cross a stage boundary. trailing_titles joins them because it reads what that peel left: the two answer one question @@ -67,11 +67,11 @@ ) -# rules.md#P3: "both questions this rule asks of a name — how many +# rules/P3: "both questions this rule asks of a name — how many # words it has, and whether it is written in one case — are asked of # the name's OWN words: a maiden marker taken as one, and the words it # takes (M2), are not among them, and neither is a delimited clause -# (N1, M1)" (history: decisions.md#P3) +# (N1, M1)" (history: decisions/P3) def own_words(tokens: Sequence[WorkToken], comma_offsets: Sequence[int], markers: frozenset[str], marker_tags: Mapping[int, str] | None = None, @@ -95,7 +95,7 @@ def own_words(tokens: Sequence[WorkToken], comma_offsets: Sequence[int], could disagree with on a name where a marker's head opens an entry but no run completes ('z' of 'z domu'): measured, 'ANNA z Nowak, MD' flipped the recorded one-case verdict under that approximation - (decisions.md#P3). Sharing the exact function instead makes the + (decisions/P3). Sharing the exact function instead makes the two paths agree by construction, not by corpus luck. Takes tokens/comma_offsets/markers rather than a whole ParseState: @@ -131,7 +131,7 @@ def own_words(tokens: Sequence[WorkToken], comma_offsets: Sequence[int], clause_at) -# rules.md#H3: "successive title words at the start of the part +# rules/H3: "successive title words at the start of the part # carrying the given name chain into one title; a title word # elsewhere in the name does not" def is_title_piece(piece: Sequence[int], ptags: Set[str], @@ -141,7 +141,7 @@ def is_title_piece(piece: Sequence[int], ptags: Set[str], return len(piece) == 1 and "vocab:title" in tokens[piece[0]].tags -# rules.md#P4: "a particle in the name's leading position chains +# rules/P4: "a particle in the name's leading position chains # nothing" -- WHERE that position is, asked once, by group's chain and # the trailing read's unit count (#624) def chain_lead(pieces: Sequence[Sequence[int]], ptags: Sequence[Set[str]], @@ -190,11 +190,11 @@ def is_prefix_piece(piece: Sequence[int], ptags: Set[str], return len(piece) == 1 and "particle" in tokens[piece[0]].tags -# rules.md#P3: "a recognized connective joins its neighbors into one +# rules/P3: "a recognized connective joins its neighbors into one # name part, connective runs included — except a single-letter # connective in a three-word name, which stays a name word, and a # single-letter connective that reads as an initial instead, which -# never joins" (history: decisions.md#P3) +# never joins" (history: decisions/P3) def is_conj_piece(piece: Sequence[int], ptags: Set[str], tokens: Sequence[WorkToken]) -> bool: if "conjunction" in ptags: @@ -254,7 +254,7 @@ def join_connectives(pieces: list[list[int]], ptags: list[set[str]], tokens: Sequence[WorkToken], *, letter_stays: bool, titles_only: bool, frozen: Set[int] = frozenset()) -> None: - """rules.md#P3's connective joins, for both parts that make them: + """rules/P3's connective joins, for both parts that make them: group's name segments, and the part after a comma, which `_comma.decide` reads before group joins anything (#617, where the two had been copies that differed). Contiguous connectives merge @@ -300,7 +300,7 @@ def join_connectives(pieces: list[list[int]], ptags: list[set[str]], merge_pieces(pieces, ptags, k, k + 2, add={"conjunction"}) else: k += 1 - # each conjunction joins its neighbors, rules.md#P3: "except a + # each conjunction joins its neighbors, rules/P3: "except a # single-letter connective in a three-word name, which stays a # name word" (v1's Google Code issue 11 carve-out, the # "john e smith" bug). @@ -351,10 +351,10 @@ def join_connectives(pieces: list[list[int]], ptags: list[set[str]], # trailing-run walk must start where assign starts. -# rules.md#H2: "an abbreviation opening the part of the name that +# rules/H2: "an abbreviation opening the part of the name that # carries the given name — the whole name, or the part after a # family comma — reads as a title even when unlisted" -# (history: decisions.md#H2) +# (history: decisions/H2) def is_leading_title(piece: Sequence[int], ptags: Set[str], tokens: Sequence[WorkToken]) -> bool: if is_title_piece(piece, ptags, tokens): @@ -393,7 +393,7 @@ def leading_titles(pieces: Sequence[Sequence[int]], piece when that piece is a name candidate: where everything behind the run is suffix pieces, the run gives back its last piece, when that piece is one word and is not itself suffix vocabulary - (rules.md#H3, decisions.md#H3 -- the block at the floor below + (rules/H3, decisions/H3 -- the block at the floor below carries the examples of each half, and the ordering its two inline tag reads were measured on). One definition, read by assign (which sets the roles), by the @@ -407,7 +407,7 @@ def leading_titles(pieces: Sequence[Sequence[int]], n += 1 continue break - # rules.md#H3: "where everything behind the run is post-nominal, + # rules/H3: "where everything behind the run is post-nominal, # the run gives its last word back to the name, provided that word # stands alone and is not itself suffix vocabulary" # @@ -423,7 +423,7 @@ def leading_titles(pieces: Sequence[Sequence[int]], # as the title and reads the numeral as the name -- given 'V', # 'king' being a given-name title and the run's last word, where # 'Dr Smith V' reads suffix 'V'. That numeral fork is outside this - # floor (decisions.md#H3). + # floor (decisions/H3). # # The two inline tag reads are the cheapest NECESSARY condition for # the piece behind the run to be a suffix piece at all -- @@ -433,7 +433,7 @@ def leading_titles(pieces: Sequence[Sequence[int]], # plan's ordering rather than the shape below: asking the # authoritative predicate first cost 8 calls per parse of the # reference name (leading_titles runs four times), against a band - # with room for two (decisions.md#parse-cost). is_suffix_piece + # with room for two (decisions/parse-cost). is_suffix_piece # stays the predicate that ANSWERS, here and in the walk. if (n and n < len(pieces) and ("suffix" in ptags[n] @@ -460,7 +460,7 @@ def is_suffix_piece(piece: Sequence[int], ptags: Set[str], return "vocab:suffix" in tags and "initial" not in tags -# rules.md#S2: "A title word never starts the run, and neither does a +# rules/S2: "A title word never starts the run, and neither does a # member of the ambiguous class, a single letter, a connective" -- what # starts #602's credential run, one predicate for the trailing peel and # the given-part walk (mechanisms.md#ONE-PREDICATE-PER-QUESTION) @@ -477,7 +477,7 @@ def starts_a_credential_run(piece: Sequence[int], ptags: Set[str], than a numeral, and carries no `initial` tag), a connective, a word that is also bound given-name or particle vocabulary (`abd` heads `Abd Allah`; `vd`, `mc`). No script is refused: an honorific - starts the run as a credential does (#679, decisions.md#S2), so + starts the run as a credential does (#679, decisions/S2), so `Ана Хорват старији Џонс` reads as `Ana Horvat stariji Jones` does. The split credential group merges ('Ph. D.', the one piece carrying @@ -514,7 +514,7 @@ def starts_a_credential_run(piece: Sequence[int], ptags: Set[str], def is_lone_never_given_particle(piece: Sequence[int], tokens: Sequence[WorkToken]) -> bool: - """A piece that is one never-given particle (rules.md#P1's fold + """A piece that is one never-given particle (rules/P1's fold site). One predicate, asked by the fold in post_rules and by two sites in assign that predict the fold will take the particle forward: the given slot, which then leaves no given name in front @@ -559,10 +559,10 @@ def _anchors(piece: Sequence[int], tokens: Sequence[WorkToken]) -> bool: roman numeral. A connective anchors nothing because between two name words it is - a link -- the generational 'i' is also Catalan's 'i' (rules.md#P3) + a link -- the generational 'i' is also Catalan's 'i' (rules/P3) -- and 'Jane Doe nee Puig i Ma' must keep its clause. A particle anchors nothing for the same reason from the other side: it joins - the name word behind it (rules.md#P2), so a word that is both + the name word behind it (rules/P2), so a word that is both particle and suffix vocabulary ('vd', 'mc') standing in front of a member is the head of a family name, not a credential list -- 'Smith vd Ma, John' keeps family 'Smith vd Ma' and 'Jan vd Ma' @@ -759,12 +759,12 @@ def segment_suffix_reading(pieces: Sequence[Sequence[int]], (mechanisms.md#ONE-PREDICATE-PER-QUESTION); it answered for a third until #436, group's one-entry join. - `licensed` holds the pieces rules.md#C1's count has licensed as + `licensed` holds the pieces rules/C1's count has licensed as credentials -- ambiguous-class words in a credential run behind two name words -- and each reads as a suffix whatever its writing or company. - rules.md#S2's initial veto keeps a roman numeral out of a suffix + rules/S2's initial veto keeps a roman numeral out of a suffix reading, which is right after a NAME word: 'Smith, John V.' is a middle initial (#432). After a SUFFIX word the numeral is describing that suffix -- 'PSM I' is Professional Scrum Master @@ -773,7 +773,7 @@ def segment_suffix_reading(pieces: Sequence[Sequence[int]], that: what follows a bare title is not continuing a credential. A part OPENED by a credential is the postnominal part however it - goes on (#603, rules.md#C1): where an unambiguous suffix word that + goes on (#603, rules/C1): where an unambiguous suffix word that starts #602's run (`starts_a_credential_run`) is the first suffix word of the part, with only titles in front of it (a title/suffix dual among them ends the chance: a credential behind one opens @@ -863,7 +863,7 @@ def segment_suffix_reading(pieces: Sequence[Sequence[int]], leading = False out.append(True) continue - # A word rules.md#C1's count licenses as a credential: the + # A word rules/C1's count licenses as a credential: the # caller (`_comma.decide`) counted two name words before the # comma and found the part a credential run, which settles the # ambiguous word here before S2's anchors are asked (#613) @@ -934,7 +934,7 @@ class Peel(NamedTuple): absorbed: tuple[tuple[int, ...], ...] = () -# rules.md#S2: "a trailing word of the suffix vocabulary reads as a +# rules/S2: "a trailing word of the suffix vocabulary reads as a # suffix — generational forms and credential acronyms alike, and an # ambiguous acronym written with its periods, one after each letter or # one after each of two or more letter chunks, counts unambiguously; a @@ -1098,7 +1098,7 @@ def peel_trailing(rest: Sequence[int], pieces: Sequence[Sequence[int]], `one_case` is ParseState.one_case, the recorded fact: None means nobody asked, which is every caller that has no state to ask with, - and reads as "no lean" -- rules.md#S2's count alone, the behavior + and reads as "no lean" -- rules/S2's count alone, the behavior of every release before this one. `start` and `anchors` resume a walk rather than begin one: the @@ -1119,7 +1119,7 @@ def peel_trailing(rest: Sequence[int], pieces: Sequence[Sequence[int]], it, and a word the chain joins to a run without the reading weighing it is a name word the walk stops at. The numeral fork still reads the piece in front of the numeral as written: an initial there keeps it a name word whatever - unit the initial is in ('John van der J. V', rules.md#P2). + unit the initial is in ('John van der J. V', rules/P2). """ picks: list[tuple[int, ...]] = [] numeral: tuple[int, ...] | None = None @@ -1177,7 +1177,7 @@ def peel_trailing(rest: Sequence[int], pieces: Sequence[Sequence[int]], # ("Jack MA"), a Title-case one is declined with plenty # ("John Smith Ma"). The lean is the LISTED set's alone: a # token admitted to this class by SHAPE carries no writing - # convention to read, so it takes the count (decisions.md#S2). + # convention to read, so it takes the count (decisions/S2). # # k < 2 means it is the only piece left, which is not the fork # this reports -- and not a floor the lean moves: the walk @@ -1223,7 +1223,7 @@ def peel_trailing(rest: Sequence[int], pieces: Sequence[Sequence[int]], return Peel(k, numeral, tuple(picks), anchors) -# rules.md#S2: "A credential after the name core starts a run to the +# rules/S2: "A credential after the name core starts a run to the # end of its part" -- applied to a finished peel (#602), so every # reader of the trailing run (assign, P5's reserve, P2's chain stop, # the maiden take) sees one answer. @@ -1245,7 +1245,7 @@ def run_start(rest: Sequence[int], names: int, every piece of an ordinary name -- and a connective or particle never pay the predicate's frame: this runs on every parse, once per piece, and a clause holding a run of links ('Puig i i i ... - Soler') otherwise paid two frames per link (decisions.md#parse-cost). + Soler') otherwise paid two frames per link (decisions/parse-cost). The predicate repeats the refusal; the inline copy is only the cheap half of the same question. Under three names no run can start, two core pieces having to stand in front of it. @@ -1280,7 +1280,7 @@ def run_start(rest: Sequence[int], names: int, return names -# rules.md#P6: "a particle ending the name attaches to that family +# rules/P6: "a particle ending the name attaches to that family # name" -- WHICH particles, asked once, by assign's given-part walk, # which both leaves them out of #602's run and attaches them (#613; # #610 had first made the run and post_rules' attachment one walk) @@ -1346,7 +1346,7 @@ def particle_tail(seg: Sequence[Sequence[int]], def has_name_content(piece: Sequence[int], tokens: Sequence[WorkToken]) -> bool: """Whether a piece holds a letter or digit: a piece with none is - no name word (rules.md#A2), so a run that takes it in absorbs no + no name word (rules/A2), so a run that takes it in absorbs no name and reports nothing. C-level, no frame per character.""" return any(map(str.isalnum, "".join([tokens[i].text for i in piece]))) @@ -1381,13 +1381,13 @@ def credential_run(rest: Sequence[int], peel: Peel, p: int, tuple(run_titles), tuple(absorbed)) -# rules.md#H5: "only a word the vocabulary knows as a title is one, +# rules/H5: "only a word the vocabulary knows as a title is one, # and a bare title word is a name word" # -- the trailing run's own predicate. NOT is_leading_title: # that predicate carries H2's unlisted-abbreviation inference, which is # the LEADING slot's shape rule and has no trailing counterpart, so # with it 'John Smith Xyz.' would lose its family name to a title -# (decisions.md#H5). The vocabulary read is is_title_piece's, shared +# (decisions/H5). The vocabulary read is is_title_piece's, shared # with the leading run so the two cannot disagree about what a title # WORD is while disagreeing, deliberately, about what a title SHAPE is. def trailing_titles(rest: Sequence[int], pieces: Sequence[Sequence[int]], @@ -1426,7 +1426,7 @@ def trailing_titles(rest: Sequence[int], pieces: Sequence[Sequence[int]], is a compiled regex (a C call, no Python frame) where is_title_piece is a call, and almost no name ends in a period-marked word, so the ordinary parse pays the one match and - stops (decisions.md#parse-cost). + stops (decisions/parse-cost). """ k = len(rest) if end is None else end while k > 1: @@ -1445,7 +1445,7 @@ def trailing_titles(rest: Sequence[int], pieces: Sequence[Sequence[int]], return k -# rules.md#H5: "successive single words that wear the abbreviation +# rules/H5: "successive single words that wear the abbreviation # shape and are title vocabulary chain into the title from the end" # -- one word's half of that, for a caller asking it of a piece the # chain did not walk to: the maiden take, choosing the words the @@ -1470,7 +1470,7 @@ def is_trailing_title_word(piece: Sequence[int], ptags: Set[str], "vocab:suffix"}) -# rules.md#M2: "It takes them up to the trailing run of post-nominals +# rules/M2: "It takes them up to the trailing run of post-nominals # and titles that the end of the name reads as if the clause were not # written" -- which words of a clause might be in that run, written # beside the peel and the chain whose admissions it has to cover (#610) @@ -1527,7 +1527,7 @@ def trailing_candidates(lo: int, pieces: Sequence[Sequence[int]], c) -# rules.md#H5: "the title is TRANSPARENT to the suffix reading: where +# rules/H5: "the title is TRANSPARENT to the suffix reading: where # two or more name words stand, what stands once the chain is taken # reads exactly as it would read written without the title, plus the # title" @@ -1552,7 +1552,7 @@ def tail_reading(rest: list[int], pieces: Sequence[Sequence[int]], `rest[:peel.names]` the name pieces, `rest[peel.names:]` the suffixes. A bare tuple rather than a named one because a NamedTuple's __new__ is a frame of its own on every parse - (decisions.md#parse-cost), and `_group_segment` returns its three + (decisions/parse-cost), and `_group_segment` returns its three the same way. Transparency is what the fixed point buys: 'X Prof. Y' reads @@ -1567,7 +1567,7 @@ def tail_reading(rest: list[int], pieces: Sequence[Sequence[int]], One function for assign's placement, group's bound-given reserve (P5), which must count the name words assign will leave, and the - maiden take (rules.md#M2), which must end the clause where the + maiden take (rules/M2), which must end the clause where the reading of the name will begin -- and since #614, for a main segment, the one read group takes between P3's joins and the particle chain (`read_trailing_run`), which assign and the reserve @@ -1700,7 +1700,7 @@ class TailRead(NamedTuple): peel: Peel -# rules.md#P2: "A particle joins the words after it into one name part, +# rules/P2: "A particle joins the words after it into one name part, # the join running until the next particle starts a group of its own" # -- how far, asked by the chain and by S2's read alike (#620), so the # units the read counts are the ones the chain then builds @@ -1729,7 +1729,7 @@ def chain_run_end(k: int, pieces: Sequence[Sequence[int]], return j -# rules.md#P2: "A particle joins the words after it into one name part" +# rules/P2: "A particle joins the words after it into one name part" # -- the particles straight behind it first, merged ahead of S2's read # so the read counts and absorbs the run as the one piece the chain # makes of it (#625) @@ -1858,7 +1858,7 @@ def read_trailing_run(pieces: Sequence[Sequence[int]], lead: int | None = None, ) -> tuple[TailRead, int] | None: """`tail_reading` read once over a segment's pieces as group holds - them after rules.md#P3's connective joins and before the particle + them after rules/P3's connective joins and before the particle chain (#614), counting name words by the units the chain will make (`_chain_units`, #620): a `TailRead`, and the index in `pieces` of the first piece of the run, which runs to the end. None where there @@ -1927,7 +1927,7 @@ def trailing_run_start(start: int, pieces: Sequence[Sequence[int]], return first -# rules.md#H5: "the title is TRANSPARENT to the suffix reading: where +# rules/H5: "the title is TRANSPARENT to the suffix reading: where # two or more name words stand, what stands once the chain is taken # reads exactly as it would read written without the title, plus the # title" diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index 01d921b8..d74ef151 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -16,9 +16,9 @@ cores group drops); Lexicon.given_name_titles. Implements rules H1, M4, P1, P6 (its no-comma site, and after a comma -naming no family), O1, O2, O3 and R1 of docs/design/rules.md; +naming no family), O1, O2, O3 and R1 of docs/design/rules/; each is cited at its code below, and H1/P1/O1/O2's history lives in -docs/design/decisions.md. `suffix_entries` is the R1 entry pass as a +docs/design/decisions/. `suffix_entries` is the R1 entry pass as a state-in/state-out function, for Parser.revise to run over a forced-role sub-parse (#511); `post_rules` runs the same worker last. """ @@ -70,10 +70,10 @@ def _rotations_apply(state: ParseState) -> bool: # Both patronymic rotations RESTORE the given-first reading a - # family-first listing hides, so rules.md#O1's scope clause holds + # family-first listing hides, so rules/O1's scope clause holds # them to the default order: a caller who declared family-first has # already said what the rotation would infer, and position decides - # (decisions.md#O1, the 2026-09-07 entry on #384). `state.order`, + # (decisions/O1, the 2026-09-07 entry on #384). `state.order`, # not policy.name_order, for the reason the P1 fold gives -- a # script_orders entry can override the policy, and the roles the # rotations read are the ones assign actually made. None means @@ -98,7 +98,7 @@ def _mark_suffix_entries(tokens: list[WorkToken], state: ParseState) -> None: # Reads dropped, comma_offsets and the policy off `state`, which # post_rules does not change, so the un-rebuilt state is current. # - # rules.md#R1: "a run of post-nominals written with spaces renders + # rules/R1: "a run of post-nominals written with spaces renders # with spaces, and one written with commas keeps them" # # The entry boundary, read off the text the writer typed rather @@ -141,7 +141,7 @@ def _mark_suffix_entries(tokens: list[WorkToken], state: ParseState) -> None: # policy's own declaration deciding it: the writer's configuration # named that text a separator, so the run parts there. # - # Four shapes were declined, recorded in decisions.md#C1 by the + # Four shapes were declined, recorded in decisions/C1 by the # bundle that landed this pass: marking the boundary at the # core-drop site (`dropped` already holds the fact with its # span, so a second recording of it is the duplication @@ -205,7 +205,7 @@ def _mark_suffix_entries(tokens: list[WorkToken], state: ParseState) -> None: def suffix_entries(state: ParseState) -> ParseState: - """rules.md#R1's entry pass over a whole state, wrapping + """rules/R1's entry pass over a whole state, wrapping _mark_suffix_entries: tag a SUFFIX token "joined" when it continues the entry of the SUFFIX token before it. post_rules runs the worker last, over the roles assign settled; this wrapper exists @@ -214,7 +214,7 @@ def suffix_entries(state: ParseState) -> ParseState: role, so a suffix value derives its entries from its own commas by the rule a whole name uses. The pass ADDS the tag and never removes one; a caller forcing roles keeps what the sub-parse marked, a - within-piece mark being role-blind (decisions.md#C1, the #436 + within-piece mark being role-blind (decisions/C1, the #436 DECLINED list) and every between-piece mark on a suffix value being one this pass sets again. Reads comma_offsets, dropped and Policy.extra_suffix_delimiters; writes the "joined" tag and @@ -257,12 +257,12 @@ def _retag(tokens: list[WorkToken], i: int, role: Role) -> None: tokens[i] = copy_with(tokens[i], role=role) -# rules.md#P2: "a particle joins the words after it into one name +# rules/P2: "a particle joins the words after it into one name # part, the join running until the next particle starts a group of # its own, a trailing suffix begins" -# rules.md#P3: "the joined part is ONE name word wherever another +# rules/P3: "the joined part is ONE name word wherever another # rule counts them" -# rules.md#P5: "a recognized bound given-name word joins the word +# rules/P5: "a recognized bound given-name word joins the word # after it into one given name" def _units(tokens: list[WorkToken], idx: list[int]) -> list[list[int]]: """`idx` split into the units other rules COUNT: one name word @@ -311,7 +311,7 @@ def _fold_reach(tokens: list[WorkToken], name_idx: list[int]) -> int: def _addressing_run(titles: list[int], name_word: int) -> list[int]: """The title run H1 asks about: the LEADING one where one stands, - else the whole (trailing) run -- rules.md#H1 -- the run standing + else the whole (trailing) run -- rules/H1 -- the run standing BEFORE the one name word being the run that addresses, and a run standing behind it deciding that word's field only when none stands before. @@ -319,7 +319,7 @@ def _addressing_run(titles: list[int], name_word: int) -> list[int]: Every title token is in the TITLE role by the time this runs, both ends of `Sir John Prof.` among them, so `titles` is not a run -- keeping the two ends apart is what makes a trailing title - TRANSPARENT (rules.md#H5): `Sir John Prof.` is `Sir John` plus a + TRANSPARENT (rules/H5): `Sir John Prof.` is `Sir John` plus a title, and reading both ends as one run keyed 'sir prof' made adding the title flip the name word's field (#489, #316). @@ -349,16 +349,16 @@ def post_rules(state: ParseState) -> ParseState: middles = _idx(tokens, Role.MIDDLE) families = _idx(tokens, Role.FAMILY) - # rules.md#H1: "a title followed by exactly one name word makes + # rules/H1: "a title followed by exactly one name word makes # that word the family name, whatever suffix, nickname or maiden # name stands beside it, unless the title is a given-name title, # which keeps it the given name; a run of several titles addresses # as its last title does" -- counting suffix, nickname and maiden # as further name words is what emptied the family (#410) # (known gap: the guard tests which roles are unoccupied, it does - # not count units -- decisions.md#H1) (v1 handle_firstnames) + # not count units -- decisions/H1) (v1 handle_firstnames) # - # rules.md#H1: "a run of several titles addresses as its last + # rules/H1: "a run of several titles addresses as its last # title does, and where a run stands BEFORE the one name word it # is the run that addresses, a run standing behind it deciding # that word's field only when none stands before" -- #489. WHICH @@ -378,7 +378,7 @@ def post_rules(state: ParseState) -> ParseState: middles = _idx(tokens, Role.MIDDLE) families = _idx(tokens, Role.FAMILY) - # rules.md#M4: "a maiden name standing beside exactly one name + # rules/M4: "a maiden name standing beside exactly one name # word makes that word the family name, whatever suffix or # nickname stands beside it" (#445) -- the MAIDEN role, not a # marker: M1's configured pair produces one with no marker @@ -398,11 +398,11 @@ def post_rules(state: ParseState) -> ParseState: # name word another rule joined counts as several here and the # rule declines -- `Dr. Dean of Chemistry` reads family, while # `Dean of Chemistry née Jones` keeps given 'Dean of Chemistry'. - # rules.md#P3 says a joined part is one name word wherever + # rules/P3 says a joined part is one name word wherever # another rule counts them, so the two disagree; widening moves # no corpus name and is not this change's to make - # (decisions.md#M4). - # rules.md#M4: "a word the vocabulary has claimed as a given name + # (decisions/M4). + # rules/M4: "a word the vocabulary has claimed as a given name # keeps that reading, and so does a word read as an initial" # -- read off the tags classify already recorded rather than a # predicate of this rule's own, because this rule changes what @@ -418,12 +418,12 @@ def post_rules(state: ParseState) -> ParseState: middles = _idx(tokens, Role.MIDDLE) families = _idx(tokens, Role.FAMILY) - # rules.md#P1: "A never-given particle opening the name marks the + # rules/P1: "A never-given particle opening the name marks the # name as surname-only: the particle run and the name words it # attaches to are the family." (v1 handle_non_first_name_prefix; - # history: decisions.md#P1) + # history: decisions/P1) # How far the fold reaches depends on the order the name was READ - # under (#395; decisions.md#P1, 2026-08-17): declaring a + # under (#395; decisions/P1, 2026-08-17): declaring a # family-first order asserts that what follows the family is not # more surname, which is the very question of where the run stops. # Under that declaration the run takes its own particles and ONE @@ -483,10 +483,10 @@ def post_rules(state: ParseState) -> ParseState: # patronymics first, then handle_middle_name_as_last) rules = state.policy.patronymic_rules rotations_apply = _rotations_apply(state) - # rules.md#O1: "a name of exactly three name words — titles, + # rules/O1: "a name of exactly three name words — titles, # suffixes and nicknames aside — whose last name word carries a # patronymic ending and whose middle name word does not reads as - # family-first" (history: decisions.md#O1) + # family-first" (history: decisions/O1) if rotations_apply and PatronymicRule.EAST_SLAVIC in rules and \ len(givens) == 1 and len(middles) == 1 and len(families) == 1: tail = tokens[families[0]].text @@ -498,10 +498,10 @@ def post_rules(state: ParseState) -> ParseState: _retag(tokens, m, Role.GIVEN) _retag(tokens, f, Role.MIDDLE) _retag(tokens, g, Role.FAMILY) - # rules.md#O2: "a name of exactly four name words — titles, + # rules/O2: "a name of exactly four name words — titles, # suffixes and nicknames aside — ending in a standalone # patronymic marker reads family-first: the first name word is - # the family name" (history: decisions.md#O2) + # the family name" (history: decisions/O2) if rotations_apply and PatronymicRule.TURKIC in rules and \ len(givens) == 1 and len(middles) == 2 and len(families) == 1: tail = tokens[families[0]].text @@ -511,7 +511,7 @@ def post_rules(state: ParseState) -> ParseState: _retag(tokens, m2, Role.MIDDLE) _retag(tokens, f, Role.MIDDLE) _retag(tokens, g, Role.FAMILY) - # rules.md#P6: "a particle ending the name attaches to that family + # rules/P6: "a particle ending the name attaches to that family # name and is written before it" -- the no-comma site (#467). # A comma is not the only thing that names the family; a declared # family-first order does too, and the same listing is written @@ -535,7 +535,7 @@ def post_rules(state: ParseState) -> ParseState: # FAMILY_FIRST_GIVEN_LAST keeps given "Van" though the same word # is in the same set. (Under the WRONG order each loses: the # Vietnamese name read as FAMILY_FIRST gives family "Van Nguyen". - # rules.md#P6 records that as accepted -- it is one order, not + # rules/P6 records that as accepted -- it is one order, not # both.) A never-given test here would have excluded every # ambiguous particle -- von, di, da, del, le and `van` itself, # the flagship word of the listing this rule is named for. @@ -595,7 +595,7 @@ def post_rules(state: ParseState) -> ParseState: for j in families): # mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE: "Emit at # the site that takes the branch, not where an ambiguous - # tag sits". decisions.md#P6 (#405) settled that this + # tag sits". decisions/P6 (#405) settled that this # attachment reports the fork it decides, and the state it # fixed is the one this site would otherwise recreate -- # `Beethoven, Ludwig van` reporting while `Beethoven Ludwig @@ -626,7 +626,7 @@ def post_rules(state: ParseState) -> ParseState: middles = _idx(tokens, Role.MIDDLE) families = _idx(tokens, Role.FAMILY) - # rules.md#P6: "a particle ending the name attaches to that family + # rules/P6: "a particle ending the name attaches to that family # name" -- after a family comma with NOTHING before it. assign takes # the attachment wherever the comma names a family (#613); where it # names none, H1 and M4 above may make the given word the family @@ -646,7 +646,7 @@ def post_rules(state: ParseState) -> ParseState: _attach_particle_tail(seg, tokens, given_at, k, end, ambiguities) - # rules.md#O3: "every middle word joins the family name and is + # rules/O3: "every middle word joins the family name and is # rendered before it" (v1 handle_middle_name_as_last). v1 # PREPENDED middle_list to last_list; mechanisms.md#FOLDED_TAG: # "tokens never move: a rule that needs different rendering order @@ -656,7 +656,7 @@ def post_rules(state: ParseState) -> ParseState: tokens[i] = copy_with( tokens[i], role=Role.FAMILY, tags=tokens[i].tags | {FOLDED_TAG}) - # rules.md#R2: "a name part whose every word is particle + # rules/R2: "a name part whose every word is particle # vocabulary is a part where none of them is doing a particle's # work — nothing joins them to a name — so they read as ordinary # name words" @@ -675,7 +675,7 @@ def post_rules(state: ParseState) -> ParseState: # and FAMILY moves 0 of 4,506 parses. The GIVEN arm is marked so a # future view reading the mark gets a consistent answer. # - # rules.md#R3: "A part holding nothing else is a part where it is + # rules/R3: "A part holding nothing else is a part where it is # joining nothing, and there it initials like any other name word" # -- #461's mark, decided HERE for the same reason R2's is: the # question is about the PART, and a view that re-derives a diff --git a/nameparser/_pipeline/_script_segment.py b/nameparser/_pipeline/_script_segment.py index d772c878..e5f756d1 100644 --- a/nameparser/_pipeline/_script_segment.py +++ b/nameparser/_pipeline/_script_segment.py @@ -45,9 +45,9 @@ Implements rules W1 (the vocabulary/segmenter division), W2 (the glued-honorific peel) and W3 (the writer's divisions are respected) -of docs/design/rules.md, cited at their code below; the decision +of docs/design/rules/, cited at their code below; the decision chain (#308, #312, #319, the vetting bars, the measured -spaced-honorific trade) is decisions.md#W1, #W2 and #W3. Both splits +spaced-honorific trade) is decisions/W1, #W2 and #W3. Both splits make sub-slices of one token, rewriting nothing -- spans still index the original exactly, so the anti-#100 invariant holds by construction. Both also match on the token's CORE, its @@ -245,10 +245,10 @@ def _is_post_nominal(state: ParseState, i: int) -> bool: return is_suffix_strict(state.tokens[i].text, state.lexicon) -# rules.md#W2: "A part that is not name text — a post-nominal word +# rules/W2: "A part that is not name text — a post-nominal word # standing on its own — is never the name's end: the split-off steps # past it to the name word behind, and never dissects it." (history: -# decisions.md#W2) +# decisions/W2) # The scan-back below is that clause, and it needs no punctuation to # fire. '김민준 박사님' steps past 박사님 to 김민준, finds no listed # tail there and returns None -- so 박사님 is left whole for suffix @@ -256,7 +256,7 @@ def _is_post_nominal(state: ParseState, i: int) -> bool: # same input gives when the step is removed. '선생님' is post-nominal # entire with no name word behind it, so the scan yields no site at # all and the token stays whole. Both are corpus names and -# rules.md#W2 example lines; decisions.md#cjk-comma-demotion +# rules/W2 example lines; decisions/cjk-comma-demotion # carries the forced-predicate measurement behind them. def _peel_site(state: ParseState, flat: Sequence[int], tails: frozenset[str]) -> tuple[int, int] | None: @@ -315,15 +315,15 @@ def _peel_site(state: ParseState, flat: Sequence[int], return None -# rules.md#W2: "a listed honorific glued to the end of the name's +# rules/W2: "a listed honorific glued to the end of the name's # last name word splits off once and reads as a suffix." (history: -# decisions.md#W2) +# decisions/W2) # That the peel also reaches ACROSS a family comma is stated at -# rules.md#W3 instead, which is a tolerated rule since the +# rules/W3 instead, which is a tolerated rule since the # 2026-09-01 comma demotion -- the crossing is what the parser does # today, not something W2 promises. W3 also carries the reading of a -# period a listing leaves behind (decisions.md#cjk-comma-demotion, -# amended by decisions.md#cjk-full-stops): a period on a SEPARATE +# period a listing leaves behind (decisions/cjk-comma-demotion, +# amended by decisions/cjk-full-stops): a period on a SEPARATE # post-nominal word rides into the suffix and moves nothing ('様.' # is post-nominal-strict and the scan steps past it as it steps past # '様'), and since #323 a period glued to the honorific's OWN token @@ -348,7 +348,7 @@ def _one_case(state: ParseState) -> bool: def _postnominal_behind_a_whole_name(state: ParseState) -> bool: """Whether the part after the comma is the postnominal part of a whole name, so the part before it is a name and not a surname -- - rules.md#C1's question, which group decides once the words are + rules/C1's question, which group decides once the words are tagged (#613). This stage runs first, so it asks THAT decision, on the name as written: `_comma.decide` over a classified copy of the state, which is the one implementation of the question. A hand copy @@ -646,7 +646,7 @@ def _split_surname_site(state: ParseState) -> ParseState: # entry's stored form is its NFC composition with no case or # edge-stop change (#322) -- so an entry AUTHORED in NFD is # composed on the way in, raw NFD input matches nothing here, and - # the name goes unsplit, which is the no-split rules.md#W1 already + # the name goes unsplit, which is the no-split rules/W1 already # accepts for NFD text. An empty vocabulary skips the match rather # than bailing the stage (_longest_entry's max() has nothing to # take): a surname-less lexicon declines every token, which is @@ -714,7 +714,7 @@ def _split_surname_site(state: ParseState) -> ParseState: and _PEELED_TAG not in state.tokens[j].tags for j in state.segments[0]): return state - # No try/except around the call: rules.md#A1's Accepted clause + # No try/except around the call: rules/A1's Accepted clause # ("a user-supplied segmenter's own error, which propagates"). The # two checks below are that same doctrine, curated, and they are # where the line this module draws is easiest to state: @@ -770,13 +770,13 @@ def _split_surname_site(state: ParseState) -> ParseState: return _split(state, i, answer.splits, detail) -# rules.md#W1: "an undivided word in the family position of a name +# rules/W1: "an undivided word in the family position of a name # written in an activated script divides after a recognized surname, # the longest recognized surname first; where the vocabulary # recognizes nothing, an optional segmenter may divide instead" -# rules.md#W3: "under a family comma the pre-comma text is the +# rules/W3: "under a family comma the pre-comma text is the # family by declaration and never divides, and the post-comma side -# is given text with no family to find" (history: decisions.md#W3) +# is given text with no family to find" (history: decisions/W3) def script_segment(state: ParseState) -> ParseState: if state.original.isascii(): # spans index the original exactly (the anti-#100 invariant), diff --git a/nameparser/_pipeline/_segment.py b/nameparser/_pipeline/_segment.py index ac95b615..7c44f7c6 100644 --- a/nameparser/_pipeline/_segment.py +++ b/nameparser/_pipeline/_segment.py @@ -22,8 +22,8 @@ _vocab.period_joined_vocab for a period-joined title, for a part of titles and suffixes (#603). -Implements rule C2 of docs/design/rules.md, and C1's segmentation; -C1's decision is `_comma.decide`'s. History in decisions.md#C1. +Implements rule C2 of docs/design/rules/, and C1's segmentation; +C1's decision is `_comma.decide`'s. History in decisions/C1. """ from __future__ import annotations @@ -74,7 +74,7 @@ def segment(state: ParseState) -> ParseState: # The case fact, asked LAZILY: here only C2's flag on a part past # the second comma turns on it. A name that never asks pays nothing; # classify asks for itself later where this did not - # (decisions.md#S2, and #429's precedent for paying a predicate + # (decisions/S2, and #429's precedent for paying a predicate # twice rather than plumbing a field two sites would not otherwise # share). one_case = state.one_case @@ -113,7 +113,7 @@ def class_run(seg: tuple[int, ...]) -> bool: # whether the parse says it did not RECOGNIZE the segment -- and # a run the parse itself reads as a credential run by shape is # recognized. Without it, the narrow roman retirement - # (rules.md#S3) moved 'R.A.I.', 'X.Y.I.' and 'J.u.n.i.o.r.' out + # (rules/S3) moved 'R.A.I.', 'X.Y.I.' and 'J.u.n.i.o.r.' out # of the vocabulary verdict and into the shape class, and every # one of them gained a COMMA_STRUCTURE flag in a third segment # that 2.3 did not raise -- a report about the parser's own new @@ -138,7 +138,7 @@ def class_run(seg: tuple[int, ...]) -> bool: for i in seg) def titles_and_suffixes(seg: tuple[int, ...]) -> bool: - # rules.md#C2: "a word of the title vocabulary there that is not + # rules/C2: "a word of the title vocabulary there that is not # also suffix vocabulary is a title" -- at least one title # word, and every other word a suffix word # period_joined_vocab for 'Lt.Gov.', one token that classify @@ -152,7 +152,7 @@ def titles_and_suffixes(seg: tuple[int, ...]) -> bool: != "title")]) return len(rest) < len(seg) and (not rest or suffixy(rest)) - # rules.md#C1's decision is not this stage's since #613: every comma + # rules/C1's decision is not this stage's since #613: every comma # form leaves here as the FAMILY comma -- the listing form, what a # comma means until the part after it has been read -- and group # decides at its head, once classify has tagged every word, whether @@ -162,7 +162,7 @@ def titles_and_suffixes(seg: tuple[int, ...]) -> bool: # second count in assign besides. What stays here is the # segmentation and C2's flag on a part past the second comma. ambiguities = list(state.ambiguities) - # rules.md#C2: "a non-empty extra part that is not entirely suffix + # rules/C2: "a non-empty extra part that is not entirely suffix # words is flagged as a structural ambiguity rather than rejected" # -- parts[2:] are consumed as suffixes either way, their title # words as titles (#603), so a tail segment that is neither all diff --git a/nameparser/_pipeline/_state.py b/nameparser/_pipeline/_state.py index dd37336d..530b883d 100644 --- a/nameparser/_pipeline/_state.py +++ b/nameparser/_pipeline/_state.py @@ -99,7 +99,7 @@ class WorkToken: _AMBIGUOUS_CREDENTIAL_TAGS = frozenset( {AMBIGUOUS_ACRONYM_TAG, SHAPE_ACRONYM_TAG}) -#: The roles that make a piece a name word (rules.md#P6's walk), and +#: The roles that make a piece a name word (rules/P6's walk), and #: the roles of a word not yet read as one: a suffix, or nothing yet #: -- what `_pieces.particle_tail` reads, asked by assign's given-part #: walk when it reaches the credential run, whose words are unread. @@ -118,7 +118,7 @@ class Structure(Enum): @dataclass(frozen=True, slots=True) -# rules.md#A1: "parsing never fails on any input: where the text's +# rules/A1: "parsing never fails on any input: where the text's # structure or a word's reading is genuinely uncertain, the parse # completes on the best reading and carries an ambiguity report # naming the doubt" @@ -215,7 +215,7 @@ class ParseState: pieces: tuple[tuple[tuple[int, ...], ...], ...] = () piece_tags: tuple[tuple[frozenset[str], ...], ...] = () dropped: tuple[int, ...] = () # structural tokens (maiden markers) - #: rules.md#S2's trailing run per segment, as group read it ONCE + #: rules/S2's trailing run per segment, as group read it ONCE #: after P3's joins and before the particle chain (#614), for #: assign to place and report #: rather than peel again; None where group read none -- a family @@ -239,7 +239,7 @@ class ParseState: order: tuple[Role, Role, Role] | None = None #: Whether the name's OWN words are written wholly in one case -- #: all upper or all lower alike -- and so carry no case EVIDENCE - #: about any word in them (rules.md#P3's own-words span, + #: about any word in them (rules/P3's own-words span, #: _pieces.own_words). None means NOT ASKED YET: the fact is #: computed by whichever of segment and classify needs it first, #: segment only when C2's flag on a part past the second comma asks @@ -250,7 +250,7 @@ class ParseState: #: reading, the prefix chain's own tail measure (_group) and the #: glued-honorific peel's decline of a post-comma run #: (_script_segment) all consult it and must not disagree - #: (#289/#516, decisions.md#S2). Five stages read it in all -- + #: (#289/#516, decisions/S2). Five stages read it in all -- #: segment, script_segment, classify, group and assign -- plus the #: two predicate layers they read it through (_pieces, _vocab). #: A fact segment records SURVIVES script_segment, the one stage @@ -342,7 +342,7 @@ def copy_with(obj: _T, /, **changes: object) -> _T: """`dataclasses.replace` for the pipeline's own dataclasses, without its per-call cost: a direct field copy, which for the classes in `_COPY_FIELDS` builds the same object - (decisions.md#parse-cost has the measurement).""" + (decisions/parse-cost has the measurement).""" cls = type(obj) names = _COPY_FIELDS.get(cls) if names is None: diff --git a/nameparser/_pipeline/_tokenize.py b/nameparser/_pipeline/_tokenize.py index e988c835..d960b281 100644 --- a/nameparser/_pipeline/_tokenize.py +++ b/nameparser/_pipeline/_tokenize.py @@ -8,10 +8,10 @@ interpunct_offsets (间隔号 transcription markers, #298; never tokens). Reads: Policy.strip_emoji, Policy.strip_bidi. -Implements rules T1, T2 and T3 of docs/design/rules.md, cited at +Implements rules T1, T2 and T3 of docs/design/rules/, cited at their code below. There is NO text-rewriting normalize stage: all three are character-classification rules, so spans always index the -original exactly as given (v1 contrast: decisions.md#T1). +original exactly as given (v1 contrast: decisions/T1). """ from __future__ import annotations @@ -35,7 +35,7 @@ (0x2600, 0x26FF), (0x2700, 0x27BF)) _BIDI = re.compile('[\u061C\u200E\u200F\u202A-\u202E\u2066-\u2069]+') -# rules.md#T2: "the katakana middle dot and its halfwidth twin divide +# rules/T2: "the katakana middle dot and its halfwidth twin divide # a name like whitespace, always." They record no offset: the # nakaguro also divides kanji roster pairs (高橋・一郎, read # family-first by the script license, #272), so it is not a @@ -43,9 +43,9 @@ _NAME_DOT_SEPARATORS = frozenset({"\u30FB", "\uFF65"}) _INTERPUNCT = "\u00B7" -# rules.md#T3: "the interpunct divides a name only between two +# rules/T3: "the interpunct divides a name only between two # characters of a classified East Asian script; anywhere else it is -# part of the word" (history: decisions.md#T3). Per-CHAR classifier +# part of the word" (history: decisions/T3). Per-CHAR classifier # for the flank guard: a one-char string is wholly-classified iff the # character is. Context-sensitivity is why this lives in # _tokenize_region (where the index exists) and not in _ignorable. @@ -57,8 +57,8 @@ def _is_emoji(ch: str) -> bool: return any(lo <= cp <= hi for lo, hi in _EMOJI_RANGES) -# rules.md#T1: "an ignorable character separates its neighbors and -# never joins them" (v1 contrast: decisions.md#T1) +# rules/T1: "an ignorable character separates its neighbors and +# never joins them" (v1 contrast: decisions/T1) def _stripped(ch: str, policy: Policy) -> bool: """True when the strip policy removes `ch` from the token stream. The ONE definition of that set, shared by _ignorable and _flank: a @@ -111,7 +111,7 @@ def _tokenize_region(state: ParseState, start: int, end: int, tok_start: int | None = None for i in range(start, end): ch = text[i] - # rules.md#N1: "a comma inside it kept as written — the comma + # rules/N1: "a comma inside it kept as written — the comma # divides nothing there" -- a comma separates only where it # records a segmentation offset, the main stream; inside an # extracted region it is the clause's text, like ';' (#653) diff --git a/nameparser/_pipeline/_vocab.py b/nameparser/_pipeline/_vocab.py index 3a477b82..a04f016b 100644 --- a/nameparser/_pipeline/_vocab.py +++ b/nameparser/_pipeline/_vocab.py @@ -164,7 +164,7 @@ def in_initialless_script(text: str) -> bool: # divergence before the move: 'Dr. Smith, Ed' (LISTED title 'Dr.') and # 'Xyz. Smith, Ed' (UNLISTED, H2-shaped) both read family 'Dr. # Smith'/'Xyz. Smith', given 'Ed' at the peel (since #701 the listed -# 'Dr.' then peels into the title, rules.md#H6), because is_leading_title +# 'Dr.' then peels into the title, rules/H6), because is_leading_title # reads H2's shape and a bare `folded in lexicon.titles` lookup does # not -- but name_word_count's OWN count disagreed: 'Xyz.' counted as # a name word where 'Dr.' did not, so 'Xyz. Smith, Ed' alone flipped @@ -186,16 +186,16 @@ def in_initialless_script(text: str) -> bool: # it at the z. `_MARKS` is the general combining blocks -- what NFD of # a Latin, Greek or Cyrillic letter produces -- and not the marks that # only some scripts write (Indic vowel signs, Arabic and Hebrew -# points); rules.md#H2's Accepted clause owns that limit. +# points); rules/H2's Accepted clause owns that limit. _MARKS = '[\u0300-\u036f\u1ab0-\u1aff\u1dc0-\u1dff\u20d0-\u20ff\ufe20-\ufe2f]' _LETTER = r'[^\W\d_]' + _MARKS + '*' _PERIOD_ABBREV = re.compile(r'^(?:' + _LETTER + r'){2,}\.$') -# rules.md#H2: "an abbreviation opening the part of the name that +# rules/H2: "an abbreviation opening the part of the name that # carries the given name — the whole name, or the part after a # family comma — reads as a title even when unlisted" -# (history: decisions.md#H2) +# (history: decisions/H2) def is_title_shaped(text: str) -> bool: """Whether TEXT wears H2's shape alone -- vocabulary-free, the LISTED half being a separate lookup at each caller's own site @@ -252,7 +252,7 @@ def is_initial_shaped(text: str) -> bool: _ROMAN = re.compile(r'^(X|IX|IV|V?I{0,3})$', re.I) -# rules.md#S2: "a trailing word of the suffix vocabulary reads as a +# rules/S2: "a trailing word of the suffix vocabulary reads as a # suffix" -- the roman-numeral half of that rule, which the initial # veto would otherwise take: V, I and X are suffix vocabulary AND bare # capitals, and a bare capital inside a name is a middle initial. @@ -275,7 +275,7 @@ def is_trailing_numeral_suffix(text: str, preceding: str) -> bool: # #544: a single-letter roman numeral ('V', 'v', 'I.'), in any case, # is excluded for its SHAPE -- one letter is how a middle initial is -# written, and rules.md#S3 retires single-character matches for the +# written, and rules/S3 retires single-character matches for the # same reason -- not for being a generation: a multi-letter suffix # word ('III', 'Jr') anchors and runs like any other. So it neither # anchors an ambiguous member behind it nor counts toward C1's @@ -303,7 +303,7 @@ def is_initial(text: str) -> bool: def is_one_case(texts: Sequence[str]) -> bool: """Whether a name is written wholly in ONE case -- all upper or all lower alike -- and so carries no case EVIDENCE about any letter in - it (rules.md#P3, #383/#479). The caller passes the name's OWN + it (rules/P3, #383/#479). The caller passes the name's OWN words: a maiden marker's clause and any delimited (nickname) content are not among them, and appending one must not flip the reading of words that did not change. @@ -311,14 +311,14 @@ def is_one_case(texts: Sequence[str]) -> bool: Mirrors the SHAPE of the comparison the R5 gate in `_render.capitalized` makes, not its SPAN: R5 joins every token but the suffixes, nickname and maiden content included, while - classify's caller hands in only the name's own words (rules.md#P3's + classify's caller hands in only the name's own words (rules/P3's own-words doctrine, see above) -- so a clause-bearing name can be one-case to this function and mixed to R5 (measured: `'JUAN GARCIA Y LOPEZ née Jones'` is one case here, mixed there), and a name with a cased suffix mixed here and one-case there (`'john e jones III'`). Not shared by import today -- render is a layer this module does not reach into -- and #492 left the two - spans apart by decision (decisions.md#R5, "THE R5 / PARSER SPLIT"). + spans apart by decision (decisions/R5, "THE R5 / PARSER SPLIT"). `Sequence`, not `Iterable`: the caller passes a list it already built rather than a fresh generator, so `is_one_case` costs one @@ -357,14 +357,14 @@ def is_one_case(texts: Sequence[str]) -> bool: Lean = Literal["credential", "name"] -# #289's credential lean: rules.md#S2 records the words-to-spare count +# #289's credential lean: rules/S2 records the words-to-spare count # today; this predicate is the WRITING evidence that decides ahead of -# it, in a mixed-case name only (history: decisions.md#S2). +# it, in a mixed-case name only (history: decisions/S2). def ambiguous_lean(text: str, one_case: bool) -> Lean | None: """Which way the WRITING leans for a member of the ambiguous credential set: "credential", "name", or None for no lean at all. - None is the fall-through to rules.md#S2's count, and it is the + None is the fall-through to rules/S2's count, and it is the answer for three inputs. A name written wholly in one case says nothing about any word in it. A member written wholly in lower case says nothing either -- lower is how most of a mixed-case @@ -412,7 +412,7 @@ def _dotted(text: str) -> bool: spelling a member with a lower-case tail is written in. A single trailing period ('Ma.', 'Ed.', 'Ms.') is the abbreviation shape any word can wear -- the honorific's, a name's -- and is not the gate's - "written with periods" (rules.md#S2). Until #296's review the gate + "written with periods" (rules/S2). Until #296's review the gate was "any period", and 'Smith, Ms.' passed it as the degree.""" return (_DOTTED.fullmatch(text) is not None or _CHUNKED.fullmatch(text) is not None) @@ -429,7 +429,7 @@ def is_paired_initials(text: str) -> bool: """Two single letters run together with periods ('M.J.'): the one unlisted dotted shape a person's own initials take, so after a comma it is a name until something in front of it, or another - dotted word beside it, says it is a credential (rules.md#C1). + dotted word beside it, says it is a credential (rules/C1). Three or more letters ('X.Y.Z.') and any longer chunk ('B.Tech.') are outside it.""" return _PAIRED_INITIALS.fullmatch(text) is not None @@ -506,7 +506,7 @@ def splits_into_suffixes(text: str, cores: frozenset[str], return False -# rules.md#S3: "a word with interior periods reads as a suffix when +# rules/S3: "a word with interior periods reads as a suffix when # any of its period-separated chunks is suffix vocabulary — except # where every chunk the vocabulary matches is a single ASCII # character, the roman numerals and the lone digit the vocabulary @@ -537,7 +537,7 @@ def period_joined_vocab( any chunk match yields to the shape, was measured and rejected: it costs 'Doe, John Msc.Ed.' a real credential and re-routes the honorific peel of '김민준씨, J.씨' for nothing this design wants - (decisions.md#S2). + (decisions/S2). The ALPHABETIC requirement is a SEPARATE, narrower gate on the shape verdict alone (#516 review round, decided by the @@ -781,7 +781,7 @@ def claimed_as_non_name(n: str, lexicon: Lexicon) -> bool: # The candidate test for a comma part read before classify -# (rules.md#C1, decisions.md#S2): segment's C2 flag on a tail part. +# (rules/C1, decisions/S2): segment's C2 flag on a tail part. def ambiguous_class_candidate(text: str, lexicon: Lexicon, policy: Policy) -> bool: """Whether TEXT is a CANDIDATE for the ambiguous credential class @@ -876,7 +876,7 @@ def unit_ends(tags: Sequence[Set[str]], chain: bool = True) -> list[int]: particle run and every word after it up to the next stop ("van der Berg Smith" is one unit). False is P1's fold reach: the particle run and the ONE unit it attaches to, so "de Mesnil Jean" is two -- - the count rules.md#C1 needs before a comma, where under a + the count rules/C1 needs before a comma, where under a family-first order that part is family 'de Mesnil', given 'Jean' (#575). @@ -927,11 +927,11 @@ def end_of(i: int) -> int: return ends -#: The facts rules.md#C1's count before a comma reads (#575): a +#: The facts rules/C1's count before a comma reads (#575): a #: particle starts a surname unit, and a suffix word stops it. A word #: that is ALSO title vocabulary ('Freiherr', 'St') LEADING the part is #: not a particle here: in front of a name it reads as the title it -#: also is (rules.md#H1), so 'Freiherr von Berg, PhD' keeps its title, +#: also is (rules/H1), so 'Freiherr von Berg, PhD' keeps its title, #: while inside a surname it chains as P1 and P2 chain it ('de St #: Pierre' is one surname). A bound given-name head that is also a #: particle ('Abu') stays a particle: 'Abu Bakar, Ed' is a surname @@ -994,11 +994,11 @@ def name_word_count(texts: Sequence[str], lexicon: Lexicon, """How many of these texts are NAME words -- not suffix vocabulary, not title vocabulary. - rules.md#C1's count for the ambiguous class, and it is of names + rules/C1's count for the ambiguous class, and it is of names rather than of words because 'Smith Jr., MA' is two tokens and one name: counting tokens there flips the structure and hands the family to `given`, which no reading of that string wants - (decisions.md#S2). The suffix half asks the POLICY-selected + (decisions/S2). The suffix half asks the POLICY-selected predicate, the same one is_wholly_suffix asks, so the two agree about what a suffix word is; the title half asks BOTH the listed lookup and H2's shape test (`is_title_shaped`) -- a bare @@ -1223,7 +1223,7 @@ def maiden_marker_run(words: Sequence[str], markers: frozenset[str]) -> int: -# rules.md#M3: "a marker followed by a comma is no marker" +# rules/M3: "a marker followed by a comma is no marker" def comma_follows_marker(word: str) -> bool: """Whether `word`, the one standing after a marker run inside a delimited clause, opens with a comma -- standing alone (' , ') or @@ -1426,7 +1426,7 @@ def _normalized_for_script(text: str) -> str | None: same way), so an NFD suffix word reaches its NFC entry. What stays raw is SEGMENTATION -- the surname site's direct membership test and the peel's tail slice index the token's own text -- where NFD - degrades to no-split, never to a wrong split (decisions.md#W1, + degrades to no-split, never to a wrong split (decisions/W1, the 2026-07-29 ja amendment). """ text = text.rstrip(FULL_STOPS) diff --git a/nameparser/_policy.py b/nameparser/_policy.py index 8c0e230b..cc3d2868 100644 --- a/nameparser/_policy.py +++ b/nameparser/_policy.py @@ -132,14 +132,14 @@ class Script(StrEnum): # personal name uses them. Halfwidth kana (U+FF65-U+FF9F, #594) IS # katakana here: legacy bank, payroll and CSV data written for JIS X # 0201 systems spells names in it, and 山田 ﾀﾛｳ must read as 山田 タロウ -# does. Classifying it is a range, not a fold. rules.md's T Background +# does. Classifying it is a range, not a fold. rules/T's Background # says "nothing rewrites the text before parsing", which rules out a # fold at tokenize, and an NFKC fold at classification would reach far # past the kana (fullwidth Latin, ㈱), and when #594 decided it a # voicing mark with no precomposed form to join (ｱﾞ) folded to the # combining U+3099, which is in the HIRAGANA block -- so the folded # token took the kana license and a wholly-katakana name turned -# family-first (decisions.md#W4; #596 has since given every voicing +# family-first (decisions/W4; #596 has since given every voicing # mark after a kana its base's script, see below). The span takes # in the halfwidth nakaguro U+FF65, which tokenize turns into a separator, # on the same direct-call grounds as U+30FB below, and the voicing @@ -732,7 +732,7 @@ class Policy: #: and a single trailing period is not this shape ("John Smith #: Xyz." keeps family ``Xyz.``). ``False`` reads such a token as #: name material everywhere, as 2.3 did for a token no chunk - #: claimed; the roman-chunk retirement (rules.md#S3) is not + #: claimed; the roman-chunk retirement (rules/S3) is not #: behind this switch, and still reports the fork. unlisted_dotted_suffixes: bool = True #: Where an UNLISTED all-caps word of two or more letters, with no @@ -774,7 +774,7 @@ class Policy: __getstate__ = _guarded_getstate __setstate__ = _guarded_setstate - # rules.md#D2: "configuration validation raises at construction + # rules/D2: "configuration validation raises at construction # with the offending field and value named" def __post_init__(self) -> None: object.__setattr__( diff --git a/nameparser/_render.py b/nameparser/_render.py index 036d8798..0bbc383b 100644 --- a/nameparser/_render.py +++ b/nameparser/_render.py @@ -41,13 +41,13 @@ #: Tags whose tokens contribute no initial outside the given group -- #: unless the token also carries UNJOINED_TAG, i.e. the whole part is #: particles, in which case they are the part's only words and do -#: contribute (rules.md#R3, #404). The mark readmits a token carrying +#: contribute (rules/R3, #404). The mark readmits a token carrying #: EITHER tag: a conjunction with nothing to join is not acting as a #: conjunction any more than a particle with nothing to join is acting #: as a particle, so it is a name word of the part like the rest. #: Not STABLE_TAGS -- that also contains "initial", which must contribute. _SKIP_TAGS = frozenset({"particle", "conjunction"}) -#: The given group's own skip set (rules.md#R3, #461): a connective +#: The given group's own skip set (rules/R3, #461): a connective #: contributes no initial in ANY group where it is joining words, so #: the given group no longer exempts it -- "one rule for every group". #: The particle exemption stays: a given-group particle is a name word @@ -127,7 +127,7 @@ def _reads_as_conjunction(word: str, lex: Lexicon) -> bool: from the word alone, the initial carve-out included ('E.' assigned to middle is an initial, not the Italian conjunction). What it cannot give is an answer the parse reached by looking at the whole - NAME -- rules.md#P3's one-case fork is the live example -- which + NAME -- rules/P3's one-case fork is the live example -- which is why it is the fallback and the tags are the rule. """ # the shape on the composed spelling, as _normalize composes the @@ -175,7 +175,7 @@ def render(name: ParsedName, spec: str) -> str: return _format_spec(spec, values, "render", _RENDER_KEYS) -# rules.md#R3: "initials take the first letter of each given, middle, +# rules/R3: "initials take the first letter of each given, middle, # and base family word; titles, suffixes, particles and nicknames # contribute nothing" def _first_letter(text: str) -> str: @@ -228,8 +228,8 @@ def initials(name: ParsedName, spec: str, delimiter: str, separator: str) -> str This view takes NO lexicon, so it has none to fall back to for that text: `replace(family='de la vega')` initials every word of that field where the same name parsed gives 'j. v.' - (rules.md#R3's Accepted - clause, and decisions.md#R4 for why the fallback was tried + (rules/R3's Accepted + clause, and decisions/R4 for why the fallback was tried and dropped -- #464 is the crossing that would make it answerable). Valid spec keys: given, middle, family.""" if not isinstance(delimiter, str): @@ -289,10 +289,10 @@ def letter(c: str | None) -> bool: for i, c in enumerate(text) if c.isalnum()] -# rules.md#R4: "the writer split a chunk the mask keeps together, so +# rules/R4: "the writer split a chunk the mask keeps together, so # the split-off letter is an initial" -- the one override below. def _apply_mask(word: str, mask: str) -> str | None: - """rules.md#R4's mask: `word` with each letter or digit recased to + """rules/R4's mask: `word` with each letter or digit recased to the case of the mask's alphanumeric in the same position, every other character kept where the writer put it ('ph.d.' under 'PhD' is 'Ph.D.'). None where the two alphanumeric counts differ -- @@ -348,7 +348,7 @@ def _lowers_as_particle(normalized: str, word: str, role: Role, and not _spells_an_initial(normalized, word) and role in (Role.MIDDLE, Role.FAMILY) and UNJOINED_TAG not in tags - # rules.md#R4: "a particle the vocabulary records a + # rules/R4: "a particle the vocabulary records a # casing for takes that casing" -- the mask below # outranks the particle's lowercase (#604) and normalized not in lex.capitalization_exceptions_map) @@ -363,11 +363,11 @@ def _cap_word(word: str, role: Role, tags: frozenset[str], # the mask ahead of the acronym clause is what gives bsc 'BSc' # though bsc is a listed acronym too. normalized = _normalize(word) - # rules.md#R4: "a part whose every word is particle vocabulary is + # rules/R4: "a part whose every word is particle vocabulary is # repaired as ordinary name words, since none of them is doing a # particle's work there" -- UNJOINED_TAG is that mark (#407). # Only the PARTICLE conjunct is gated on it, and that is the rule - # rather than an omission -- rules.md#R4: "A CONNECTIVE the parse + # rather than an omission -- rules/R4: "A CONNECTIVE the parse # placed among the name words keeps its lowercase wherever it # stands there, including inside a part whose other words the # unjoined mark has turned into ordinary name words" -- so a @@ -469,7 +469,7 @@ def _cap_word(word: str, role: Role, tags: frozenset[str], # word as the writer punctuated it, so 'Ph.D.' under 'PhD' stays # 'Ph.D.' and 'phd' becomes 'PhD'. v1 substituted the value, which # is how 'md' became 'M.D.' and 'iii.' lost its period; - # rules.md#R4: "Repair changes case and nothing else". Role-free, + # rules/R4: "Repair changes case and nothing else". Role-free, # as the map always was: an entry is the caller saying how a word # is written wherever it stands. undotted = normalized.replace(".", "") @@ -484,7 +484,7 @@ def _cap_word(word: str, role: Role, tags: frozenset[str], # not a word to title-case: a one-case name repairs to the # acronym's caps instead of 'Mba' (#459). So is a word classify # admitted to the credential class by its dotted SHAPE alone - # (SHAPE_ACRONYM_TAG; rules.md#S3's unlisted 'x.y.z.'), which has + # (SHAPE_ACRONYM_TAG; rules/S3's unlisted 'x.y.z.'), which has # no vocabulary entry to be listed in. The mask above is asked # first, which is what keeps a conventionally mixed-case acronym as # it is written (bsc -> BSc) though it is listed here too. Gated @@ -496,7 +496,7 @@ def _cap_word(word: str, role: Role, tags: frozenset[str], undotted in lex.suffix_acronyms or SHAPE_ACRONYM_TAG in tags): return word.upper() - # rules.md#R4: "A roman numeral the parse put in the suffix role is + # rules/R4: "A roman numeral the parse put in the suffix role is # written in capitals" whether or not the vocabulary lists it: 'vi' # through 'x' carry no vocabulary tag and title-cased to 'Vi'/'Ix' # until #459, while 'ii'/'iii'/'iv' rode the exceptions map, which is @@ -567,7 +567,7 @@ def cap(word: str) -> str: # A particle written with a mark ("d'", "az-", "'t", #700) is one # vocabulary word that the per-word walk below would split at the # mark, asking the particle test of its stem ('d', 'az', 't') - # instead -- so rules.md#R4's "a particle is written in lowercase" + # instead -- so rules/R4's "a particle is written in lowercase" # is asked of the whole token first. Only the lowercase is taken # here; wherever the arm declines (an all-particle part, a role # other than middle or family, an initial's spelling, a recorded @@ -590,7 +590,7 @@ def cap(word: str) -> str: if any(c.isalnum() for c in part)] if len(named) < 3: return _sub_words(cap, text) - # rules.md#R4: "Inside a hyphenated word, a part that is + # rules/R4: "Inside a hyphenated word, a part that is # connective vocabulary with a worded part on each side of it # keeps its lowercase" (#478). Decided here, not in _cap_word, # whose word has lost its neighbours. Only a part holding an @@ -619,7 +619,7 @@ def _in_one_case(text: str) -> bool: return text in (text.upper(), text.lower()) -# rules.md#R4: "case repair returns a repaired copy and never mutates +# rules/R4: "case repair returns a repaired copy and never mutates # the parse" def capitalized(name: ParsedName, lexicon: Lexicon | None, *, force: bool) -> ParsedName: @@ -633,7 +633,7 @@ def capitalized(name: ParsedName, lexicon: Lexicon | None, *, than one case is kept as written -- 'EdD', 'B.Tech.', and the garbled 'Iii' alike -- while one written in a single case is repaired like any other token. - Repair changes case and nothing else (see the rules.md#R4 citation + Repair changes case and nothing else (see the rules/R4 citation in _cap_word): an exceptions-map value is a mask recasing the word as written (#459), never a replacement. The repair reads token TAGS as well as texts: a part whose every @@ -649,14 +649,14 @@ def capitalized(name: ParsedName, lexicon: Lexicon | None, *, 'de la' where the same words parsed give 'De La'; one set to 'de y' keeps the 'y' lowercase, as the parse does and as 1.4.0 did. Parser.revise() is the edit that classifies the value, and - gives 'De La' (rules.md#R4's Accepted boundary). + gives 'De La' (rules/R4's Accepted boundary). Idempotent: every _cap_word rule, and the hyphen rule in _cap_text, is a fixpoint on its own output, so a repaired name comes back unchanged whether or not the gate admits it again (a name whose non-suffix words are caseless, 'Kim Minjun' in hangul with a 'phd', is admitted every time) -- except where a LETTER'S OWN CASE MAPPING changes its length - (decisions.md#R4's Unicode boundary; 'ß' recasing to 'SS' through + (decisions/R4's Unicode boundary; 'ß' recasing to 'SS' through a mask is one example, not the only one). Lengthening: a mask's own per-character casing fallback can turn one letter into a different LETTER SEQUENCE ('ß' upper is 'SS', not one recased @@ -681,18 +681,18 @@ def capitalized(name: ParsedName, lexicon: Lexicon | None, *, # silent no-op on mixed-case input or a deep AttributeError raise TypeError(f"lexicon must be a Lexicon or None, got {lexicon!r}") lex = Lexicon.default() if lexicon is None else lexicon - # rules.md#R5: "case repair acts only on a name written entirely + # rules/R5: "case repair acts only on a name written entirely # in one case" -- and "the suffixes are left out of that test": a # credential or a generation written the way one is written # ('III', 'PhD', 'Jr.') says nothing about how the writer cased # the NAME (#492). Titles stay in because title repair is not yet - # trusted to act on a cased title (decisions.md#R5 names the three + # trusted to act on a cased title (decisions/R5 names the three # titles that showed why). gate = " ".join(t.text for t in name.tokens if t.role is not Role.SUFFIX) if not force and not _in_one_case(gate): return name - # rules.md#R5: "a suffix written in more than one case is the + # rules/R5: "a suffix written in more than one case is the # writer's spelling and is kept as written where repair was not # forced" -- the gate's own test, asked of each suffix token. The # gate leaves the suffixes out because a cased one says nothing diff --git a/nameparser/_types.py b/nameparser/_types.py index 159bddb1..cb046cf3 100644 --- a/nameparser/_types.py +++ b/nameparser/_types.py @@ -88,7 +88,7 @@ def __add__(self, other: object) -> NoReturn: # type: ignore[override] #: word READ as an initial -- initial-shaped in a script that HAS #: initials ("J." or "А.", never "씨.", #320), or a marked #: single-letter connective in a name written in one case -#: (rules.md#P3, see CONJUNCTION_OR_INITIAL); +#: (rules/P3, see CONJUNCTION_OR_INITIAL); #: "joined" a continuation of the token before it -- within one #: merged piece the tag is role-blind and every view joins the pair #: with a space ("Ph." + "D."; 'Ph. D., Jr.' gives last_list @@ -107,7 +107,7 @@ def __add__(self, other: object) -> NoReturn: # type: ignore[override] #: where none of them is doing a particle's work -- nothing joins them #: to a name -- so FOUR views read them as ordinary name words: they #: anchor `family_base`, drop out of `family_particles`, contribute -#: initials (rules.md#R2), and capitalize like any other name word +#: initials (rules/R2), and capitalize like any other name word #: rather than being lowercased as particles (#407). #: MARKED rather than untagged: `particle` is stable API and says the #: word IS particle vocabulary wherever it lands, which stays true, and @@ -117,7 +117,7 @@ def __add__(self, other: object) -> NoReturn: # type: ignore[override] #: A name part holding no word a connective could join -- once the #: part's WORKING particles are set aside -- is a part where the #: connective is not doing a connective's work, so it initials like an -#: ordinary name word, agreeing with `family_base` (rules.md#R3, +#: ordinary name word, agreeing with `family_base` (rules/R3, #: #461). The twin of UNJOINED_TAG above, computed in the same pass #: and for the same reason: the fact is about a PART, so it is decided #: once where the parts are settled rather than re-derived by each @@ -436,7 +436,7 @@ class AmbiguityKind(StrEnum): #: family name remains; "JACK MA", written wholly in one case, keeps #: it as the name because none would, while in a mixed-case name #: the capitals decide outright -- "Jack MA" is the credential and - #: "Jack Ma" the name, rules.md#S2) and a trailing roman numeral, which is a suffix where any + #: "Jack Ma" the name, rules/S2) and a trailing roman numeral, which is a suffix where any #: other single letter would be a name ("John Smith V" vs "John #: Smith B"). It also covers a family-comma listing, where a #: trailing abbreviation that is both a post-nominal and a surname @@ -455,7 +455,7 @@ class AmbiguityKind(StrEnum): #: trailing slot of that listing's GIVEN part, and the trailing #: slot of a maiden marker's clause. A part beyond the second #: comma is not among them, so "Smith, John, MA" is silent; what - #: such a part does report is rules.md#C2's to say. + #: such a part does report is rules/C2's to say. #: Since 2.4 the given part's trailing slot reports whichever way #: it read the word: "Doe, John MA" reads suffix ``MA`` and says #: so, "Doe, John Ma" keeps middle ``Ma`` and says so too. Not in @@ -467,7 +467,7 @@ class AmbiguityKind(StrEnum): #: Since #544 C1's name-word count reports its flip once over a #: whole RUN after the comma ("John Smith, Ed Ma" reads suffix #: ``Ed Ma``), and a member an unambiguous credential in front of - #: it speaks for (rules.md#S2) is a pick like a counted one, + #: it speaks for (rules/S2) is a pick like a counted one, #: reported where it was decided: by the peel, by the given part's #: trailing slot, or by the reading of a part after a family comma #: that the company leaves holding no name word ("John Smith PhD @@ -485,7 +485,7 @@ class AmbiguityKind(StrEnum): #: reports: a clause in the FAMILY segment of a comma listing #: ("Smith nee Jones MA, Jane" keeps maiden ``Jones MA``) is #: silent. After any other comma a marker is an ordinary word and - #: makes no clause at all (rules.md#M2). + #: makes no clause at all (rules/M2). #: FOUR positions stay silent, and all four are boundaries rather #: than omissions. The NO-READER clause just named is the first of #: them, and the three that follow are about the member's own @@ -513,7 +513,7 @@ class AmbiguityKind(StrEnum): #: "Doe, MA Smith" reports its ``MA``, and "Doe, Dr. MA Smith" -- #: the same member, one title in front of it -- reads given ``MA`` #: in silence. A maiden marker cannot reach this position: after - #: a family comma it is an ordinary word (rules.md#M2), so "Doe, + #: a family comma it is an ordinary word (rules/M2), so "Doe, #: Dr. nee Smith MA" reads given ``nee``, middle ``Smith`` and #: suffix ``MA`` -- and reports, the ``MA`` ending a given part #: like any other. @@ -621,7 +621,7 @@ class AmbiguityKind(StrEnum): #: MULTI-letter connective ("and", "та", "και") or a symbol ("&") is #: no initial's shape at any casing. And a letter inside a maiden or #: delimited clause never reaches the fork either (the own-words - #: doctrine, rules.md#P3): appending " née Jones" or a nickname + #: doctrine, rules/P3): appending " née Jones" or a nickname #: changes no report, because the clause's words were never #: eligible for it. #: ``detail`` names the token, the kind naming neither the field nor @@ -948,7 +948,7 @@ def middle(self) -> str: return self._text_for(Role.MIDDLE) @property - # rules.md#R1: "every field is a view computed from the parsed + # rules/R1: "every field is a view computed from the parsed # words at read time, joining its words in written order — except # folded family words" (O3's fold and P6's tussenvoegsel, which # render before the rest of the family) @@ -970,7 +970,7 @@ def maiden(self) -> str: # -- derived views (filters over roles + STABLE tags only) ---------- @property - # rules.md#R2: "the family name splits into further views: the + # rules/R2: "the family name splits into further views: the # base (the family without its leading particles) and the # particles themselves" def family_particles(self) -> str: @@ -1135,6 +1135,6 @@ def capitalized(self, lexicon: Lexicon | None = None, *, outside the suffix are written in more than one case, and in a name it does repair keeps a suffix written in more than one case as written ('EdD'), repairing only one written in a - single case (rules.md#R5). Idempotent.""" + single case (rules/R5). Idempotent.""" import nameparser._render as _render return _render.capitalized(self, lexicon, force=force) diff --git a/nameparser/config/__init__.py b/nameparser/config/__init__.py index 52967adc..6cf7183f 100644 --- a/nameparser/config/__init__.py +++ b/nameparser/config/__init__.py @@ -23,7 +23,7 @@ # above no longer says "this package is deleted in 3.0". The 3.0 # removal schedule names "nameparser.config in its entirety" while # actually enumerating only the five shim exports re-exported below; -# that gap is recorded in docs/design/decisions.md#3-0-reevaluations +# that gap is recorded in docs/design/decisions/3-0-reevaluations # rather than left to a section number of an uncommitted document. # Whether the DATA modules keep this package as their # home in 3.0 or move under the core is an open decision, not something diff --git a/nameparser/config/bound_given_names.py b/nameparser/config/bound_given_names.py index 56d263df..1b6e5845 100644 --- a/nameparser/config/bound_given_names.py +++ b/nameparser/config/bound_given_names.py @@ -32,7 +32,7 @@ # credential -- but not everywhere: in the given slot of a # family-comma name the credential still wins, so "Smith, Abd" # reports suffix 'Abd' and no given name, where "Smith, Abdul" - # reports the given name. Recorded at decisions.md#P5. + # reports the given name. Recorded at decisions/P5. 'abd', 'abu', 'abou', diff --git a/nameparser/config/capitalization.py b/nameparser/config/capitalization.py index 9c41b23e..4cffd7cc 100644 --- a/nameparser/config/capitalization.py +++ b/nameparser/config/capitalization.py @@ -8,8 +8,8 @@ # (#459): without a mask the acronym repair writes each in # capitals. A mask applies wherever its word stands, so a word # borne as a name must not get one -- meng and edd are listed - # acronyms that are, and stay out on purpose (decisions.md#R4's - # Excluded block). + # acronyms that are, and stay out on purpose (the meng/edd/lac/ded + # CAPITALIZATION_EXCEPTIONS block in decisions/vocabulary-exclusions). 'bt': 'Bt', 'chfc': 'ChFC', 'cpht': 'CPhT', @@ -29,7 +29,7 @@ # Irish patronymic particles (#604). Case repair writes a particle # in lowercase ('de la Vega'), but Irish writes these capitalized # ('Liam Ó Murchú', 'Sinéad Ní Mhurchú'), and a mask outranks the - # particle lowercase (rules.md#R4). Each mask is the word's own + # particle lowercase (rules/R4). Each mask is the word's own # title case, so a bare word in any other role repairs as it would # anyway; the dotted 'u.a.' and 'n.í.' now repair to 'U.A.' and # 'N.Í.' (were 'U.a.' and 'N.í.'). @@ -55,8 +55,8 @@ ``PharmD``), so ``psyd`` repairs to ``PsyD`` rather than ``PSYD``. The exception is an acronym that is also a name word (``meng``, ``edd``): a mask applies in every role, so it gets none, and -``john smith edd`` repairs to ``EDD`` (decisions.md#R4's Excluded -block for CAPITALIZATION_EXCEPTIONS -- meng, edd, lac, ded). A +``john smith edd`` repairs to ``EDD`` (decisions/vocabulary-exclusions, +CAPITALIZATION_EXCEPTIONS -- meng, edd, lac, ded). A suffix the writer spelled in more than one case (``EdD``) needs no entry either: repair keeps it as written unless forced. A caller's OWN acronym -- one absent from ``suffix_acronyms`` -- depends on how diff --git a/nameparser/config/conjunctions.py b/nameparser/config/conjunctions.py index e69d3c28..25229b32 100644 --- a/nameparser/config/conjunctions.py +++ b/nameparser/config/conjunctions.py @@ -14,7 +14,7 @@ # entry that is ALSO generational vocabulary -- 'i' is the roman # numeral I, a bare entry of SUFFIX_WORDS -- so the carve-out # counts it as a name word for its own sake and it joins only - # with a name word on each side (rules.md#P3). + # with a name word on each side (rules/P3). 'i', # #269: Cyrillic (ru/uk/bg) "and": и, і, та. Ukrainian writes і and # й for the same conjunction, alternating on the surrounding @@ -81,7 +81,7 @@ # deliberately absent from Lexicon's subset checks, so a CALLER'S # orphan is never rejected -- it is inert instead, the classify fork # and its emitter both requiring the base entry before they read the -# marker (see decisions.md#P3, the 2026-09-13 entries). +# marker (see decisions/P3, the 2026-09-13 entries). assert CONJUNCTIONS_AMBIGUOUS <= CONJUNCTIONS, \ "CONJUNCTIONS_AMBIGUOUS must stay a subset of CONJUNCTIONS" assert all(len(w) == 1 and w.upper() != w.lower() diff --git a/nameparser/config/maiden_markers.py b/nameparser/config/maiden_markers.py index afd0543f..637ee9ed 100644 --- a/nameparser/config/maiden_markers.py +++ b/nameparser/config/maiden_markers.py @@ -67,7 +67,7 @@ needs no space in front of 旧姓 at all. The bare "山田花子 旧姓 佐藤" and the bracketed "山田 花子（旧姓 佐藤）" alike give maiden 佐藤, and since #335 the bracketed form needs no configuration to do it: the -fullwidth pair is a NICKNAME delimiter by default, and rules.md#M3 +fullwidth pair is a NICKNAME delimiter by default, and rules/M3 reads a clause that opens with a marker word AND carries a word after it as the maiden name, whichever bucket its pair sits in. The second word is part of the condition, not a detail of it -- a lone "（旧姓）" @@ -85,7 +85,7 @@ it splits the clause on WHITESPACE, and "旧姓：佐藤" is one whitespace word, so there is no marker word for the clause to open with and by default it stays a nickname. The two tests agree here and are -deliberately not the same test -- see decisions.md#M3. The wholly +deliberately not the same test -- see decisions/M3. The wholly unspaced "山田花子（旧姓佐藤）" reads as one token for the same reason. Peeling a marker off the head of a token is #317's job. @@ -100,9 +100,9 @@ period-insensitive, so "Roz", "roz" and "roz." are one string to this set: with it shipped, "Rosalind Roz Smith" read maiden "Smith" and NO family name at all. The collision is in the position the claim acts -on, which is the test decisions.md#vocabulary-collisions states, and +on, which is the test decisions/vocabulary-collisions states, and this set has no ambiguous subset to express a partial answer in — see -decisions.md's Excluded (MAIDEN_MARKERS) block. The full participle +decisions/vocabulary-exclusions, Excluded (MAIDEN_MARKERS). The full participle rozená stays, being unambiguous; a caller who needs the abbreviation adds it to their own Lexicon. """ diff --git a/nameparser/config/particles.py b/nameparser/config/particles.py index 583d0129..f97ff5dc 100644 --- a/nameparser/config/particles.py +++ b/nameparser/config/particles.py @@ -16,13 +16,13 @@ #: opening *piece*, not the first word of the string: a particle that has #: already chained onto the word behind it is part of that piece rather #: than standing alone. -#: OPENING the name is the whole of that rule's subject (rules.md#P1; +#: OPENING the name is the whole of that rule's subject (rules/P1; #: the pre-2.2 docstrings called it rule 1b). A member standing alone #: where a declared order puts the GIVEN name was a second site until #: #467, and is not one now: that slot holds what the caller declared, #: so ``Policy(name_order=FAMILY_FIRST)`` reads "Mesnil de" as family #: "Mesnil", given "de". A member ENDING the name under a family-first -#: order is rules.md#P6's, which joins it to the family. A bare "de" stays as it is. Where a chain is +#: order is rules/P6's, which joins it to the family. A bare "de" stays as it is. Where a chain is #: reported as the given name anyway it is because the member is no #: longer standing alone: under ``Policy(name_order=FAMILY_FIRST)`` the #: given position of "Juan de la Vega" holds the whole three-token @@ -46,7 +46,7 @@ # the AMBIGUOUS side; no amount of searching settles absence, so what # is recorded is what the word IS rather than a failed search. # - # The test is POSITIONAL (decisions.md#vocabulary-collisions C-i, + # The test is POSITIONAL (decisions/vocabulary-collisions C-i, # corrected 2026-08-17): membership asks whether the word is borne as # a name in the position this rule ACTS on -- leading, or alone in the # given role -- not whether a bearer exists anywhere. 'de' is why the @@ -125,7 +125,7 @@ # name (#604). The one ONE-LETTER particle, so the letter # written with its period is still the initial it # spells -- 'Juan Ó. Pérez' is Óscar, not a surname - # (rules.md#P7). Unaccented 'o' is EXCLUDED: bare 'O' is + # (rules/P7). Unaccented 'o' is EXCLUDED: bare 'O' is # a common periodless initial, and the anglicized form # is glued ("O'Neil") anyway. 'ste', # Contraction of 'Sainte'; has a vowel, which is why the @@ -137,7 +137,7 @@ 'vd', # Dutch abbreviation of "van der". Also the British # Volunteer Decoration, a suffix acronym: two non-name # readings, so C-ii decides it on frequency and the Dutch - # one wins (decisions.md#vocabulary-collisions). + # one wins (decisions/vocabulary-collisions). 'vom', # German "from the" 'zu', # German "at/to", nobiliary ("zu Guttenberg") @@ -214,7 +214,7 @@ #: with no drift -- mirroring ``TITLES = GIVEN_NAME_TITLES | {...}`` in #: :py:mod:`nameparser.config.titles`. # The AMBIGUOUS half: words that are also borne as names in the position -# the leading-particle rule acts on (decisions.md#vocabulary-collisions +# the leading-particle rule acts on (decisions/vocabulary-collisions # C-i). A commented entry has been examined; a BARE one is sitting on the # conservative default and nobody has looked at it. #360 examined the 12 # that appear leading in tools/differential's corpora plus a few more, @@ -253,7 +253,7 @@ # particle (DiCaprio) 'dí', 'do', # Vietnamese Đỗ leads a great many surnames. Also in - # TITLES -- decisions.md records TITLES n ambiguous == + # TITLES -- the decisions record TITLES n ambiguous == # {do, freiherr, st} as load-bearing for the emitter 'du', # Du is a Chinese surname (Du Fu), leading under a # family-first reading diff --git a/nameparser/config/suffixes.py b/nameparser/config/suffixes.py index 12b49f85..598731c6 100644 --- a/nameparser/config/suffixes.py +++ b/nameparser/config/suffixes.py @@ -37,7 +37,7 @@ # (US Census, SSA, Wikidata via tools/name_usage.py; nuorempi has one # Wikidata bearer each way). Feminine forms ship beside # the masculine where the language has them. Left out, with reasons, - # in decisions.md#generational-words: kıdemli, sóisearach and + # in decisions/generational-words: kıdemli, sóisearach and # sinsearach (rank words, not generations), Hungarian ifj./id. # (written BEFORE the name), Bulgarian млада (a given name), and the # abbreviations ml./st. (#679 ships мл.; see the deferral above). @@ -58,9 +58,9 @@ # word the language's Wikipedia writes after names ships instead # (lo, ti, or, ta, si, mk). Transliterations of "senior" ship, but # not Hebrew סניור, which Israelis bear as a surname (אלי סניור), nor - # Japanese シニア (decisions.md#generational-words); English + # Japanese シニア (decisions/generational-words); English # `senior` is not a suffix at all, Senior being a surname. - # Left out with reasons in decisions.md#generational-words, + # Left out with reasons in decisions/generational-words, # including fils/père and the Latin ml./st., which collide outside # their languages and wait on a pack. 'yngri', 'eldri', # is @@ -169,7 +169,7 @@ # spaced position is a token boundary the writer drew, so the # vetting bar there is just "is a word standing alone ever a # name?". The glued bar ("can it END a name?") was measured - # separately in #707 (decisions.md#W2), and every entry but + # separately in #707 (decisions/W2), and every entry but # 部長/部长 passed it and is in GLUED_HONORIFICS below. The # Chinese pack's SEGMENTER is not a peel and exposes them after a # listed surname regardless: 王二世 and 李部長 divide at the @@ -200,7 +200,7 @@ # written बनर्जी, मुखर्जी, चटर्जी, so a glued peel would strand a # fragment on a bare virama (बनर् + जी) -- the 殿 criterion in a # non-CJK script. गांधीजी therefore stays unpeeled, which is the - # accepted trade (decisions.md#indic-honorifics). + # accepted trade (decisions/indic-honorifics). # NOTE the leading/trailing split across the two scripts, which is # real and must not be "harmonized": Devanagari बाबू is a LEADING # honorific -- a TITLES entry (#344), not a suffix -- while Bengali @@ -226,7 +226,7 @@ set claims goes to ``period_joined_vocab``, which splits it on its periods and, no chunk being a title, calls the whole thing a suffix if ANY chunk is suffix vocabulary -- except where every chunk it matches is a SINGLE ASCII -CHARACTER, which since 2.4 retires rather than claims (decisions.md#S2): the +CHARACTER, which since 2.4 retires rather than claims (decisions/S2): the one-character suffix vocabulary is the Roman numerals and the digit "2" -- ASCII is load-bearing there, since the seven single-character CJK honorifics in THIS set (様, 殿, 氏, 군, 님, 씨, 양 -- three of them also @@ -236,7 +236,7 @@ no longer reaches this set's "i" entry as a generational claim; it is read by POSITION instead, as an unlisted multi-chunk word (``Policy.unlisted_dotted_suffixes``, default on), and with words to -spare it is still a suffix (rules.md#S3). A multi-character chunk match +spare it is still a suffix (rules/S3). A multi-character chunk match is untouched -- "Msc.Ed." still derives from ``ed`` and reads as a suffix outright, and "Lt.Gov." still derives a TITLE from its own multi-character chunk match. @@ -289,7 +289,7 @@ # argument (above), which a lower-bound zero does not refute. # Accepted cost: a longer office word ending in one of these is # split at it (田中 副社長 → given 副, suffix 社長; 獣医学博士 leaves - # 獣), since nothing longer is listed (decisions.md#W2, #707). + # 獣), since nothing longer is listed (decisions/W2, #707). '二世', '議員', '议员', '国会議員', '國會議員', '太平紳士', '太平绅士', @@ -315,7 +315,7 @@ মহারাজ) and Latin rinpoche are spaced-only for the same reason 殿 is: जी ends Banerjee/Mukherjee/Chatterjee (बनर्जी, मुखर्जी, चटर्जी) and a glued peel would strand बनर् on a bare virama. The criterion is not - CJK-specific, which is the point (decisions.md#indic-honorifics). + CJK-specific, which is the point (decisions/indic-honorifics). Three more are in NEITHER set, so neither spelling is recognized. 君: 王君 is a complete Chinese name (君 is a common given-name final), so the honorific @@ -327,7 +327,7 @@ SUFFIX_ACRONYMS_AMBIGUOUS = frozenset({ # Suffix acronyms that also commonly work as given-name nicknames on # their own (e.g. "Ed", "JD"). Read wherever the pipeline decides an - # acronym's bare reading (rules.md#S2) and by the delimited-content + # acronym's bare reading (rules/S2) and by the delimited-content # escape (content matching one of these stays a nickname, the more # common reading in ambiguous, delimiter-only context). _classify # also tags membership as "vocab:suffix-ambiguous". @@ -342,7 +342,7 @@ # written in capitals inside a mixed-case name; inside a # mixed-case name any other cased form that is not wholly lower # ('Ma', 'MEng') reads as the name even with words to spare -- - # rules.md#S2. The parse reports the fork where it decides one, + # rules/S2. The parse reports the fork where it decides one, # so 'Jack Ma' keeps its family name. # # The other half of the criterion, added 2026-09-07 with #342. @@ -374,7 +374,7 @@ # # Removal takes the DOTTED spelling with it too, except by # accident: "John Smith R.A.I." still reads suffix 'R.A.I.' only - # because rules.md#S3 splits an interior-period token on its + # because rules/S3 splits an interior-period token on its # periods and the chunk 'i' happens to be a Roman numeral in # SUFFIX_WORDS. "John Smith R.A.X." reads family, and so does # "John Smith C.H.A." after the removal. Do not count on a dotted @@ -383,7 +383,7 @@ # which is the answer this library gives for every # locale-specific vocabulary. The cost is stated and accepted: # with the entry gone, "John Smith RAI" reads family 'RAI'. See - # decisions.md#suffix-acronym-collisions. + # decisions/suffix-acronym-collisions. # # NOT 'ms' or 'sa', though #296's audit table put them here for the # leading-title collision (bare "Ms" the honorific, "M.S." the @@ -392,13 +392,13 @@ # title 'MS', and 'Smith, Ms.' passed the gate on its one period # and read as a credential anyway. Both words are genuine duals -- # title and unambiguous suffix -- and position decides, as for - # 'sr' and 'lt' (decisions.md#C1). + # 'sr' and 'lt' (decisions/C1). # # NOT 'se' or 'om' either, weighed 2026-09-07 with #342 and left # alone: no surname evidence worth standing behind, and OM is the # Order of Merit. And NOT 'mc' or 'vd', which are also PARTICLES: # #454 closed by design -- neither is a borne name, so a bare - # trailing one is the decoration, and rules.md#P6's Accepted + # trailing one is the decoration, and rules/P6's Accepted # clause names them as the two words whose positional reading # does not hold. 'ba', @@ -418,7 +418,7 @@ Acronym suffixes from SUFFIX_ACRONYMS that also plausibly collide with a word borne as a name -- a given name, nickname or surname. Not a partition of SUFFIX_ACRONYMS -- a small, standalone exception list, read -wherever the pipeline decides an acronym's bare reading (rules.md#S2). +wherever the pipeline decides an acronym's bare reading (rules/S2). """ SUFFIX_ACRONYMS = frozenset({ @@ -431,7 +431,7 @@ # Arabic-script counterpart عبد is in that set only) -- the one # word in both sets, kept in both deliberately. Position decides # at the two ends but not in a family comma's given slot; see - # decisions.md#P5. + # decisions/P5. 'abd', 'abpp', 'abr', @@ -1057,7 +1057,7 @@ # periods, the acronym test strips all of them -- so a word in both is # matched twice by two rules, and which one fired is unreadable from the # outside. The single overlap was 'esq', dropped 2026-09-08 -# (decisions.md#suffix-acronym-collisions), and the assert is what keeps +# (decisions/suffix-acronym-collisions), and the assert is what keeps # a bulk import from quietly re-creating one. It guards the SHIPPED sets # only: Lexicon has no matching invariant, so a caller who wants the # overlap in their own vocabulary may still have it. diff --git a/nameparser/config/titles.py b/nameparser/config/titles.py index 13e0be7b..dc4e8ce5 100644 --- a/nameparser/config/titles.py +++ b/nameparser/config/titles.py @@ -6,7 +6,7 @@ # #346: the renunciate class -- 'baba' here, and 'guru', 'lama', # 'swami' in their own alphabetical places below. All four were in # the TITLES-only block until 2026-09-06. Two separate criteria - # both say yes (rules.md#H Background): the title addresses by the + # both say yes (rules/H Background): the title addresses by the # GIVEN name, which is what membership here means, and the # traditions renounce the surname, which is what makes the empty # family the right output for "Swami Vivekananda". 'rabbi' and @@ -30,7 +30,7 @@ 'pope', # #519: prince and princess address by the GIVEN name ("Prince # Harry", "Princess Anne"), which is what membership here means - # (rules.md#H Background); both were in the TITLES-only block + # (rules/H Background); both were in the TITLES-only block # until 2026-09-10. 'lord' and 'lady' stay there, decided rather # than deferred: both split by the bearer's rank, which the text # does not carry -- the given-name form is a courtesy style for @@ -97,7 +97,7 @@ 'संत', # Sant # #343: Bengali renunciate titles. Same two criteria as the Latin - # and Devanagari sets (rules.md#H Background), and no NEW Latin + # and Devanagari sets (rules/H Background), and no NEW Latin # transliterations -- srila has none; baba/guru/swami already ship # above. শ্রীল is the Vaishnava honorific (শ্রীল প্রভুপাদ); # unprefixed শ্রী is civil and sits in TITLES below. @@ -770,17 +770,17 @@ # had none for, so "Herra Väinö Johansson" stops reading given # 'Herra'. TITLES has no ambiguous subset, so a word ships only if # it is borne as no name in the LEADING position - # (decisions.md#vocabulary-collisions C-i) -- with one recorded + # (decisions/vocabulary-collisions C-i) -- with one recorded # exception. Those marked "borne" are names somewhere, but rarely # the FIRST word of one, mostly surnames written last, and ship # anyway on the 'graf' precedent above, paying the cost "Graf # Steffi" already pays: a name that begins with one gives it to - # the title (rules.md#H1). That is a judgment on each word's + # the title (rules/H1). That is a judgment on each word's # counts, scoped to these words, not a rule to sweep with. Counts # and sources are in - # decisions.md#salutation-titles, and so are the words that failed: + # decisions/salutation-titles, and so are the words that failed: # among them Vietnamese ông/bà, Tamil thiru and pl/cs pan, all in - # its Excluded (TITLES) block. + # decisions/vocabulary-exclusions, Excluded (TITLES). # English offices, completing runs like "Police Commissioner". # 'marshal' and 'justice' stay out: both are given names. 'administrator', @@ -843,7 +843,7 @@ # #662: Akan and Indonesian Mr.; under 100 census bearers each and # none found on Wikidata. Hausa malam and Turkish bay are excluded - # (decisions.md, Excluded (TITLES)). + # (decisions/vocabulary-exclusions, Excluded (TITLES)). 'owura', 'bapak', @@ -912,7 +912,7 @@ # Excluded under the collision rule and recorded here so a sweep # does not ship them -- ठाकुर (Tagore, the surname), कुमारी, बेगम, # शेख, आचार्य, राजा/रानी are all borne as ordinary names. Full - # argument: decisions.md Excluded (TITLES). + # argument: decisions/vocabulary-exclusions, Excluded (TITLES). 'डॉक्टर', # Doctor, full form 'डा', # Dr. -- the डा. abbreviation (Nepali and older Hindi), # edge-period normalized @@ -941,7 +941,7 @@ # collide with real given names where the native script cannot. # No NEW Latin transliterations here (sri, mst); Latin md already # ships, and for the medical degree - # (decisions.md#indic-honorifics). Doctor abbreviations + # (decisions/indic-honorifics). Doctor abbreviations # split by profession: ড./ডঃ is the PhD's, ডাঃ/ডা. the Bangladeshi # physician's. Abbreviation marks are spelled as written -- the # visarga ঃ (U+0983) is a spacing combining mark, and the lookup @@ -953,7 +953,7 @@ # Excluded under the collision rule and recorded here so a sweep # does not ship them -- ঠাকুর (Tagore, the surname), কুমারী, বেগম # and শেখ are all borne as ordinary names. Full argument: - # decisions.md Excluded (TITLES). + # decisions/vocabulary-exclusions, Excluded (TITLES). 'ড', # Dr. abbreviation (PhD) 'ডঃ', # Dr. abbreviation, visarga spelling 'ডক্টর', # Doctor, full form @@ -976,7 +976,7 @@ 'মোঃ', # Md. (Mohammad) -- a name PREFIX, not a title # semantically; TITLES is the functional home # because `given` must stay the name the person is - # addressed by (decisions.md#indic-honorifics) + # addressed by (decisions/indic-honorifics) 'মো', # the same, period spelling মো. 'মোসাঃ', # Mst. (Mosammat), the women's counterpart 'মোসা', # the same, period spelling মোসা. diff --git a/nameparser/locales/ja.py b/nameparser/locales/ja.py index ad9cce84..1478c115 100644 --- a/nameparser/locales/ja.py +++ b/nameparser/locales/ja.py @@ -15,7 +15,7 @@ why this pack alone is inert: it activates a stage that then has nothing to segment with. * No order, because a kana-bearing Japanese name already reads - family-first by default (decisions.md#W4, the kana license: + family-first by default (decisions/W4, the kana license: hiragana identifies native Japanese as certainly as hangul identifies Korean), and wholly Han names have read family-first since the 2026-07-27 amendment. @@ -46,7 +46,7 @@ ``ja_segmenter(gbdt=True)`` path, which is what loads that file; the default BasicNameDivider reads namedivider's own bundled kanji.csv and never touches it. Its BERT model for katakana division is CC-BY-SA and -is NOT used (katakana division is out of scope; decisions.md#W1's +is NOT used (katakana division is out of scope; decisions/W1's kana-gating bullet). Declared deviations (mechanisms.md#LOCALE-PACKS-PURE-DATA): the diff --git a/tests/test_capitalization.py b/tests/test_capitalization.py index 8ead3862..707d1edf 100644 --- a/tests/test_capitalization.py +++ b/tests/test_capitalization.py @@ -19,7 +19,7 @@ def test_capitalization_exception_for_III(self) -> None: # A known failure since 2012 (the Google Code tracker's issue 22) # until #492: the one-case gate read the SUFFIX as evidence that # the writer cased the whole name, so 'III' held the lowercase - # name back. The gate now leaves the suffixes out (rules.md#R5): + # name back. The gate now leaves the suffixes out (rules/R5): # a generation written the way one is written says nothing about # how the name was cased. The name words still count, and so # does a title -- the two boundary rows. @@ -168,7 +168,7 @@ def test_capitalize_exceptions_still_win_over_acronyms(self) -> None: # key's own letters recased -- so a v1 Constants carrying any other # value raises at the first parse, where the snapshot builds the # Lexicon. A DECIDED exception to the shim's never-raise rule - # (decisions.md#R4, and #3-0-reevaluations): 1.4.0 substituted + # (decisions/R4, and #3-0-reevaluations): 1.4.0 substituted # such a value for the word, repair now only recases, and no such # value was found in the tracker, the docs or any test. The raise # carries a v1-spelled hint: the 2.0 message's own offer @@ -203,7 +203,7 @@ def test_a_mismatched_exception_value_raises_at_the_first_parse( hn.capitalize() self.m(str(hn), 'John Smith JR', hn) - # rules.md#R5 (#492): a suffix ASSIGNED to the facade after + # rules/R5 (#492): a suffix ASSIGNED to the facade after # construction carries UNCLASSIFIED_TAG (no `vocab:suffix` tag, no # roman-numeral shape read from a parse) and is still excluded from # the one-case gate -- the gate reads the ROLE, not the tag. @@ -265,13 +265,13 @@ def test_capitalize_prefix_clash_on_first_name(self) -> None: hn.capitalize() self.m(str(hn), 'Van Nguyen', hn) - # #407, rules.md#R4. The family is one word and that word is + # #407, rules/R4. The family is one word and that word is # particle vocabulary, so nothing joins it to a name and it is not # doing a particle's work: family_base and initials have read it as # an ordinary name word since #404, and case repair now agrees. # Deliberately the v1 facade, and deliberately single-case input -- # this MOVES v1-visible behavior (1.4.0 returns 'Anh do'), and the - # mixed-case gate would return the input untouched (rules.md#R5). + # mixed-case gate would return the input untouched (rules/R5). def test_capitalize_all_particle_family_is_a_name_word(self) -> None: hn = HumanName('ANH DO') hn.capitalize() @@ -316,7 +316,7 @@ def test_capitalize_working_particle_stays_lowercase(self) -> None: # an all-particle family whose `y` carries the mark, giving # 'Anh y Van' where gating the conjunct too would give # 'Anh Y Van'. So the ungated conjunct decides something: it is - # rules.md#R4's stated carve-out, matching R3's for initials, + # rules/R4's stated carve-out, matching R3's for initials, # rather than a no-op. The `y` here is a GIVEN-part word besides, # not a word of the all-particle family. def test_capitalize_all_particle_family_beside_a_conjunction(self) -> None: @@ -324,7 +324,7 @@ def test_capitalize_all_particle_family_beside_a_conjunction(self) -> None: hn.capitalize() self.m(str(hn), 'y Van Der', hn) - # rules.md#R5's override, stated as a property rather than a + # rules/R5's override, stated as a property rather than a # single row: mixed case is the writer making an explicit choice # and repair defers to it, so asking for repair REGARDLESS should # ignore the input's case entirely -- one name, one repaired @@ -341,7 +341,7 @@ def test_capitalize_all_particle_family_beside_a_conjunction(self) -> None: # only 25 move a role, and the other 37 parse byte-identically and # differ inside the repair itself. Through the v2 core -- # `parse(n).capitalized(force=True)`, rendering all seven roles - # -- the counts are 63 and 38, which is what decisions.md#R5 + # -- the counts are 63 and 38, which is what decisions/R5 # states. The one name the facade cannot see is # `Jane van der Berg nee y Jones`, whose conjunction sits in the # MAIDEN name: `str(HumanName)` renders the default spec, and @@ -353,7 +353,7 @@ def test_capitalize_all_particle_family_beside_a_conjunction(self) -> None: # count collapses and the direction flips, uppercase now differing # on fewer names than lowercase rather than more. The dated # re-measurement and its recipe (with the actual counts) live - # under decisions.md#R5. + # under decisions/R5. # # THROUGH 2.3.0 the mechanism was v1's initial carve-out, taken in # the PARSE since #458 and read off the tag by the repair: a word @@ -363,9 +363,9 @@ def test_capitalize_all_particle_family_beside_a_conjunction(self) -> None: # conjunction became an initial; lowercase one and a middle # initial `E` became the Italian conjunction. `Velasquez y Garcia, # Dr. Juan Q.` forced kept `y`; uppercased then repaired gave `Y` - # (decisions.md#R5). + # (decisions/R5). # - # #383/#479 (rules.md#P3, decisions.md#P3) removed "one CAPITAL + # #383/#479 (rules/P3, decisions/P3) removed "one CAPITAL # letter" as the whole test: a name written wholly in one case # carries no case evidence, so the vocabulary decides instead of # the shape. Only the letters `Lexicon.conjunctions_ambiguous` @@ -381,7 +381,7 @@ def test_capitalize_all_particle_family_beside_a_conjunction(self) -> None: # giving `Dr. Juan Q. Velasquez y Garcia`; the lowercase half is # shown instead by `john e smith` # (tests/v2/test_render.py::test_capitalized_one_case_connective_that_reads_as_an_initial). - # decisions.md#R5 carries the dated amendment for this. So the + # decisions/R5 carries the dated amendment for this. So the # property is pinned over names # carrying no single-letter word whose class case decides, and the # `conjunctions_ambiguous` exception is pinned beside it as data @@ -401,13 +401,13 @@ def test_forcing_repair_ignores_the_case_it_was_given(self) -> None: # This WAS the recorded exception to the property above through # 2.3, and the reason it was scoped rather than universal. The - # #383/#479 fork (rules.md#P3, decisions.md#P3) reads a bare + # #383/#479 fork (rules/P3, decisions/P3) reads a bare # capital single-letter conjunction in a one-case name as the # connective -- no case evidence says otherwise -- so # 'JUAN Y GARCIA' and 'juan y garcia' now repair to the same # string and the property above holds for them too. 1.4.0's # 'Juan Y Garcia' (measured on the released wheel) is the parity - # break decisions.md#R5's 2026-09-13 amendment records; no ledger + # break decisions/R5's 2026-09-13 amendment records; no ledger # compares case repair, which is why this test is the pin. # # The name is kept for the blame trail even though it now @@ -426,7 +426,7 @@ def test_a_one_letter_conjunction_is_case_sensitive_to_repair(self) -> None: # 'Y' carries no case evidence -- 'y' is not in # conjunctions_ambiguous, so it reads as the connective rather # than as an initial, and R4's carve-out lowercases it exactly - # as it lowercases the all-lower spelling above (decisions.md#P3) + # as it lowercases the all-lower spelling above (decisions/P3) self.m(str(uppered), 'Juan y Garcia', uppered) # mixed-case control: here the capital IS evidence, so 'Y' # reads as an initial exactly as it always has and repair @@ -482,7 +482,7 @@ def test_a_conjunction_inside_a_longer_token_is_a_name_word(self) -> None: # #478: Ortega y Gasset is routinely hyphenated in catalogues, and # inside one hyphenated word a connective with a part on each side # keeps its lowercase -- the hyphens are the writer joining the name - # around it, as the spaced connective does (rules.md#R4). This + # around it, as the spaced connective does (rules/R4). This # reverses #458's answer for the INTERIOR position ('Jose # Ortega-Y-Gasset' at 2.2.0 and 2.3.0) and keeps it at the edges, # where a part has a neighbour on one side only and is ordinary diff --git a/tests/test_initials.py b/tests/test_initials.py index b8e324e4..592a9987 100644 --- a/tests/test_initials.py +++ b/tests/test_initials.py @@ -44,7 +44,7 @@ def test_initials_middle_name_all_prefixes(self) -> None: # "Nguyen, Van Le" parses with middle name "Le", every word of # which is particle vocabulary. A particle standing alone in a # name part is not doing a particle's work there, so it - # initials as an ordinary name word (rules.md#R2/#R3) rather + # initials as an ordinary name word (rules/R2/#R3) rather # than being dropped -- this read "V. N." until #404, losing # the middle name entirely. hn = HumanName("Nguyen, Van Le") @@ -279,15 +279,15 @@ def test_initials_follow_the_one_case_fork_on_both_letters(self) -> None: # #528 made this view read the parse's tags instead of # re-deriving from vocabulary and shape. The core's # parse(...).initials() has given 'j. e. s.' since #383/#479 - # and the two agree now; decisions.md#P3 and decisions.md#R3 + # and the two agree now; decisions/P3 and decisions/R3 # carry the split and its closing. hn = HumanName("john e smith") self.m(hn.initials(), "j. e. s.", hn) # 'maria y lopez' moved with #461, not with the one-case fork: # its 'y' holds the middle part alone, so there is nothing # there for it to join and it initials like any other name - # word, agreeing with the parse's own view. rules.md#R3 states - # the rule for all three groups and decisions.md#R3 records + # word, agreeing with the parse's own view. rules/R3 states + # the rule for all three groups and decisions/R3 records # what it costs. The letter's TAG is unchanged -- a lowercase # 'y' outside the marked set is still the connective. hn = HumanName("maria y lopez") diff --git a/tests/test_suffixes.py b/tests/test_suffixes.py index 91eb240f..d21343c7 100644 --- a/tests/test_suffixes.py +++ b/tests/test_suffixes.py @@ -22,7 +22,7 @@ def test_two_suffixes(self) -> None: self.m(hn.first, "Kenneth", hn) self.m(hn.last, "Clarke", hn) # The writer spaced "QC MP", so #436 renders it as written - # (rules.md#R1). A deliberate deviation from 1.4.0, which added a + # (rules/R1). A deliberate deviation from 1.4.0, which added a # comma the original format did not have. self.m(hn.suffix, "QC MP", hn) @@ -31,7 +31,7 @@ def test_two_suffixes_lastname_comma_format(self) -> None: self.m(hn.first, "Franklin", hn) self.m(hn.last, "Washington", hn) # The writer spaced "Jr. MD", so #436 renders it as written - # (rules.md#R1). This name HAS a writer's comma, but it stands + # (rules/R1). This name HAS a writer's comma, but it stands # between MD and Franklin -- it parts the family from the run, not # the run from itself. A deliberate deviation from 1.4.0, which # added a comma the original format did not have. @@ -148,16 +148,16 @@ def test_king(self) -> None: # v1 aspired to read "Dr King Jr" as title 'Dr', family 'King', # suffix 'Jr', and shipped this as an xfail. # Two halves, and this pin means different things about each. - # DECIDED 2026-09-01 (v1-xfail triage, decisions.md#v1-xfail-triage): + # DECIDED 2026-09-01 (v1-xfail triage, decisions/v1-xfail-triage): # 'king' stays in the titles vocabulary, because removing it breaks # the addressing forms it is there for ("King Charles"). - # decisions.md#vocabulary-collisions cuts toward keeping it, so the + # decisions/vocabulary-collisions cuts toward keeping it, so the # title chain reaches 'King' at all. The comma format reads the # surname without any of that, and is pinned below as the contrast. # WHAT BECOMES OF THE LEFTOVER 'Jr': the improvement this pin - # anticipated, taken 2026-09-08 (#489, decisions.md#H3). The title + # anticipated, taken 2026-09-08 (#489, decisions/H3). The title # chain used to eat two words and leave 'Jr' to be the name, against - # rules.md#S2, which reads a trailing suffix-vocabulary word as a + # rules/S2, which reads a trailing suffix-vocabulary word as a # suffix -- and against 'Dr Smith Jr', which reads family 'Smith', # suffix 'Jr' exactly as S2 states. The leading peel now leaves a # name word a suffix cannot be: everything behind the run is a @@ -238,16 +238,16 @@ def test_roman_numeral_i_with_explicit_suffix_comma_stays_a_middle_initial(self) # v1 aspired to read the trailing 'I' as an ordinal joining the explicit # suffix ("I, Jr."), and shipped this as an xfail. # NOT FIXED, decided 2026-09-01 (v1-xfail triage, - # decisions.md#v1-xfail-triage): a middle initial 'I' is far more common + # decisions/v1-xfail-triage): a middle initial 'I' is far more common # than an ordinal I borne without a Sr./Jr.-style companion, so when an # explicit suffix comma has already named the suffix, the trailing 'I' # stays a middle initial. The commonality reasoning is the - # vocabulary-collision criterion (decisions.md#vocabulary-collisions) + # vocabulary-collision criterion (decisions/vocabulary-collisions) # applied to a shape rather than to a word. # Contrast test_roman_numeral_i_after_single_initial_lastname_comma_format # above, where no explicit suffix comma is present and the 'I' does go to # suffix. - # rules.md#C1 gained a qualifier with this triage: where a further + # rules/C1 gained a qualifier with this triage: where a further # comma has already named the suffix, the trailing single letter has # no generation left to be. C1 carries no example line for it (see the # Accepted note there), so this pair of tests is its witness. @@ -305,7 +305,7 @@ def test_suffix_delimiter_none_by_default_known_limitation(self) -> None: # Through 2.3 (and in v1) that broke parsing -- first 'RN', last # 'Steven Hardman', suffix 'CRNA' -- and this test pinned the # limitation. Since #603 a credential opening the part after the - # comma makes it the suffix part (rules.md#C1), so the undeclared + # comma makes it the suffix part (rules/C1), so the undeclared # delimiter is one more word of that part and the name parses; # the name of the test keeps the history. hn = HumanName("Steven Hardman, RN - CRNA") diff --git a/tests/test_titles.py b/tests/test_titles.py index 11b32123..28dc59a7 100644 --- a/tests/test_titles.py +++ b/tests/test_titles.py @@ -101,7 +101,7 @@ def test_title_multiple_titles_with_apostrophe_s(self) -> None: # v1 aspired to read the whole string as one title, and shipped this # as an xfail. # NOT FIXED, decided 2026-09-01 (v1-xfail triage, - # decisions.md#v1-xfail-triage): this is a name parser, not a title + # decisions/v1-xfail-triage): this is a name parser, not a title # parser. Handed an input that is all titles, it assumes the last # title-word is the name, so 'Division' becomes the family. Accepted # convention rather than a defect; #491 tracks reporting the guess. @@ -201,10 +201,10 @@ def test_ben_is_not_a_particle(self) -> None: # v1 aspired to read 'ben' as a particle joining the family # ("ben Husain"), and shipped this as an xfail. # NOT FIXED, decided 2026-09-01 (v1-xfail triage, - # decisions.md#v1-xfail-triage): the question was already settled in + # decisions/v1-xfail-triage): the question was already settled in # v0.2.5, when 'ben' was removed from the prefixes. 'ben' collides with # the given name Ben, which is the same criterion - # (decisions.md#vocabulary-collisions) that keeps it out today. + # (decisions/vocabulary-collisions) that keeps it out today. # test_ben_as_first_name below is the collision it protects. hn = HumanName("Ahmad ben Husain") self.m(hn.title, "", hn) diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 51a508fc..eb8b2db0 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -1,4 +1,4 @@ -"""THE shared behavior case table (rules.md cites it as the pin of +"""THE shared behavior case table (the rules cite it as the pin of CURRENT behavior). Format is fixed here, in the first pipeline PR, and never per-PR: @@ -69,7 +69,7 @@ def _stray_ascii(text: str) -> str: messages of their own because each names a composed form with its own doctrine, and this takes everything else -- a period, a digit, a parenthesis. A space is the one ASCII character a pure CJK - arrangement writes (rules.md#W4's '山田 太郎'), so it is the one + arrangement writes (rules/W4's '山田 太郎'), so it is the one exemption.""" return "".join(sorted({c for c in text if c.isascii() and c != " "})) @@ -116,7 +116,7 @@ class Case: #: policy fork rather than an input shape stays untagged. shape: int | None = None #: Marks a row's text as TOLERATED input (2026-09-01 CJK demotion): - #: read best-effort, rules.md#W3. This table still asserts its + #: read best-effort, rules/W3. This table still asserts its #: expectation, and a behavior change on a tolerated row still edits #: its classification. Mutually exclusive with `shape`. Every #: composed/wrapped CJK form (a comma listing, a Latin title or @@ -279,7 +279,7 @@ def _check_cjk_shape_purity(self) -> None: # U+00B7 (间隔号) marks a name transcription in SOURCE order # (W1 Accepted) -- shape 7's ground, not shape 6's family- # first one. The fullwidth nakaguro U+30FB is NOT a source- - # order marker on its own: decisions.md#T3 scopes that reading + # order marker on its own: decisions/T3 scopes that reading # to the codepoint, so U+30FB on non-katakana text is an # ordinary Han/Hangul separator and the text reads family- # first (cases.py's own ja_nakaguro_han_takes_the_han_order, @@ -359,7 +359,7 @@ def _check_cjk_shape_purity(self) -> None: "(af5bdab, #93), and the multi-dot spelling was its " "only unique coverage -- so for one bundle this read " "family 'E.S.Q.' (a deliberate 2.x parity break, " - "decisions.md#suffix-acronym-collisions; same criterion " + "decisions/suffix-acronym-collisions; same criterion " "as the rai/cha rows above, asked of the machinery " "instead of a surname: does the entry describe the " "WORD or the set's normalization; the bundle that " @@ -383,7 +383,7 @@ def _check_cjk_shape_purity(self) -> None: "#516's by-shape class now reads the SAME single " "pre-comma word as a candidate for the ambiguous class " "-- one word is never enough to flip the structure " - "(rules.md#C1), so the FIELDS stay at the 1.4.0 " + "(rules/C1), so the FIELDS stay at the 1.4.0 " "reading, but the class is now considered, and the " "consideration reports: the parity note above is about " "the reading, not about whether a fork was called", @@ -405,7 +405,8 @@ def _check_cjk_shape_purity(self) -> None: "positional read, family 'Ph.', where every release since " "1.4.0 read suffix. Same criterion as the esq rows above, " "asked of the machinery: the entry described the merge, " - "not a word. decisions.md, Excluded (SUFFIX_ACRONYMS -- ph)", + "not a word. decisions/vocabulary-exclusions, Excluded " + "(SUFFIX_ACRONYMS -- ph)", shape=1), Case("suffix_word_esq_still_reads_as_a_suffix", "John Smith Esq", {"given": "John", "family": "Smith", "suffix": "Esq"}, @@ -432,7 +433,7 @@ def _check_cjk_shape_purity(self) -> None: "carry vocab:folded-middle and every view of them orders " "them first (spans cannot reorder). This table has no " "initials column, so the half #408 moved is pinned in " - "rules.md#R3 and tests/v2/test_render.py: initials were " + "rules/R3 and tests/v2/test_render.py: initials were " "'M. H. A. A.' here and are 'M. A. A. H.' now, which is " "also 1.4.0's answer. Since #484 the differential compares " "initials() too, for names whose roles are identical -- " @@ -441,7 +442,7 @@ def _check_cjk_shape_purity(self) -> None: # MOVED by #289, not deleted: 'MA' is written in capitals inside a # mixed-case name, so it now leans CREDENTIAL and is taken with no # words to spare -- the reserve/count this row used to pin no - # longer decides here, the written case does (decisions.md#S2). + # longer decides here, the written case does (decisions/S2). Case("ambiguous_surname_acronyms", "Jack MA", {"given": "Jack", "suffix": "MA"}, classification="fix(#289)", @@ -455,7 +456,7 @@ def _check_cjk_shape_purity(self) -> None: "what turns on GIVEN_OR_FAMILY. 1.4.0 read LAST 'MA' " "here (family 'MA'), so this row BREAKS 1.4.0 parity " "deliberately, in the direction the East Asian surname " - "wants (decisions.md#S2) -- corrected 2026-09-17, F3 " + "wants (decisions/S2) -- corrected 2026-09-17, F3 " "review finding: an earlier version of this note " "wrongly claimed the opposite"), Case("ambiguous_surname_acronym_with_suffix", "Jack MA Jr", @@ -465,7 +466,7 @@ def _check_cjk_shape_purity(self) -> None: notes="'Jr' peels first as unambiguous vocabulary; 'MA' is " "then the last piece, and the caps lean takes it with " "no words to spare rather than leaving it the family " - "the old reserve kept (decisions.md#S2) -- both suffix " + "the old reserve kept (decisions/S2) -- both suffix " "words render as one run"), Case("ambiguous_acronym_is_a_suffix_when_a_family_name_remains", "John Smith MA", @@ -475,7 +476,7 @@ def _check_cjk_shape_purity(self) -> None: "peeling 'MA' still leaves given+family, so the " "credential reading wins (v1 parity)"), # -- #342: rai and cha left SUFFIX_ACRONYMS. The criterion is in - # decisions.md#suffix-acronym-collisions and it compares + # decisions/suffix-acronym-collisions and it compares # FREQUENCIES: how common the word is as a borne name in the # trailing position against how common it is as a credential. # Rough balance earns the ambiguous marking (the members of @@ -505,7 +506,7 @@ def _check_cjk_shape_purity(self) -> None: "no signal the parser reads, so the word families and " "'Smith' becomes a middle name. The shape-plus-position " "heuristic that would recover it is a parking-lot " - "bullet of decisions.md#suffix-acronym-collisions"), + "bullet of decisions/suffix-acronym-collisions"), Case("removed_credential_after_a_comma_reads_by_its_capitals", "Ahmad Jayadi, CHA", {"given": "Ahmad", "family": "Jayadi", "suffix": "CHA"}, @@ -514,7 +515,7 @@ def _check_cjk_shape_purity(self) -> None: notes="with 'cha' gone from the vocabulary (#342) the word is " "unlisted, and since #564 an unlisted all-caps word " "after a comma behind two name words is a credential by " - "default (rules.md#C1, S2), so the comma is a suffix " + "default (rules/C1, S2), so the comma is a suffix " "comma again and reports the call. 2.3.0 read given " "'CHA', family 'Ahmad Jayadi'; 'John Smith, RAI' moves " "with it"), @@ -529,7 +530,7 @@ def _check_cjk_shape_purity(self) -> None: "(the contrast was 'John Smith R.A.I.', which kept " "reading suffix as an accident of 'i' being a " "SUFFIX_WORDS numeral rather than a survival of the " - "vocabulary, decisions.md#suffix-acronym-collisions). " + "vocabulary, decisions/suffix-acronym-collisions). " "#516's by-shape class now reads BOTH the same way, by " "POSITION rather than by any chunk claim -- three " "single-letter chunks, none of them vocabulary, still " @@ -582,7 +583,7 @@ def _check_cjk_shape_purity(self) -> None: # mixed-case name, so it now leans CREDENTIAL and the post-comma # slot takes it with one word to spare where the count alone would # not -- back to what 2.0.0 through 2.2.0 read, though for a - # different reason (decisions.md#S2, Derek's own #289 comment: + # different reason (decisions/S2, Derek's own #289 comment: # positive evidence outranks position here). Case("comma_ambiguous_acronym_ba", "Smith, BA", {"family": "Smith", "suffix": "BA"}, @@ -604,12 +605,12 @@ def _check_cjk_shape_purity(self) -> None: "ambiguous_acronym_is_a_suffix_when_a_family_name_remains " "above"), # -- #540: 'meng' and 'lac' take the marking 'ba' took, on the same - # criterion (decisions.md#suffix-acronym-collisions). MEng and LAc + # criterion (decisions/suffix-acronym-collisions). MEng and LAc # are live credentials; Meng is a Chinese surname and given name, # Lac a Vietnamese given name (the trailing word in native order, # Nguyen Van Lac) and a French surname, and both stand in the one # slot the suffix vocabulary claims, the last word of a name. No rule - # moves: these rows are rules.md#S2's forks read through two more + # moves: these rows are rules/S2's forks read through two more # members, written all-lower so the count decides unless the # writing says otherwise, and shape-tagged so the snapshot sees # them -- no corpus name had either word. @@ -855,7 +856,7 @@ def _check_cjk_shape_purity(self) -> None: "reported the part. Since #613 the part is read and bound " "before any join, so no chain reaches it: each 'DO' leans " "credential by its capitals and the run reads whole in " - "silence behind another credential (rules.md#C1). 1.4.0 " + "silence behind another credential (rules/C1). 1.4.0 " "read the same fields; 2.0.0 read title 'PhD', given 'DO " "DO', and 2.3.0 given 'PhD', family 'DO DO John Smith'", shape=3), @@ -959,7 +960,7 @@ def _check_cjk_shape_purity(self) -> None: "suffix": "PhD Ma"}, classification="fix(#602)", ambiguities=("suffix-or-name",), - notes="rules.md#S2 (#602): the credential after two name words " + notes="rules/S2 (#602): the credential after two name words " "starts a run to the end of the part, the title inside it " "reads as a title, and the member is in the run. The id " "names the reading #544 pinned (middle 'Smith PhD Prof.', " @@ -972,7 +973,7 @@ def _check_cjk_shape_purity(self) -> None: "suffix": "Ph. D."}, classification="fix(#540)", ambiguities=("suffix-or-name",), - notes="rules.md#S2's Accepted limit: the merged 'Ph. D.' is " + notes="rules/S2's Accepted limit: the merged 'Ph. D.' is " "read as a suffix at any position and so is not in the " "no-comma walk the company is read over, and the " "member's writing decides. The comma and given-slot " @@ -984,13 +985,13 @@ def _check_cjk_shape_purity(self) -> None: {"given": "John", "family": "Smith", "suffix": "PhD Do Do"}, classification="fix(#602)", ambiguities=("suffix-or-name",), - notes="rules.md#S2 (#602): the credential starts the run and " + notes="rules/S2 (#602): the credential starts the run and " "P2's chain stops where it starts, so the 'Do Do' the " "chain joined is absorbed and reported. The id names " "#544's Accepted limit, which #602 retires; 1.4.0 and " "2.3.0 read middle 'Smith PhD', family 'Do Do'"), # #602: a credential after the name core starts a run to the end of - # its part (rules.md#S2). The movers, then the exclusions that keep + # its part (rules/S2). The movers, then the exclusions that keep # a run from starting -- each a negative control for the movers. Case("a_credential_starts_a_run_to_the_end_of_the_part", "John Smith PhD Jones", @@ -1034,7 +1035,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "다우니", "family": "로버트", "suffix": "주니어"}, classification="fix(#679)", notes="the suffix peels before the wholly-hangul name reads " - "family-first by script (rules.md#W4), so the transcribed " + "family-first by script (rules/W4), so the transcribed " "Robert is the family name as it was before; 2.4.2 read " "middle '주니어', and 1.4.0 family '주니어'"), Case("a_generational_word_after_a_nakaguro", "ロバート・ダウニー・ジュニア", @@ -1262,7 +1263,7 @@ def _check_cjk_shape_purity(self) -> None: "slot's given is there and the member behind it reads " "as the credential, as for 'SMITH, JOHN MA'. 1.4.0 " "reads the same"), - # #575: rules.md#C1's count before the comma counts a particle + # #575: rules/C1's count before the comma counts a particle # chain (P2) as ONE name word -- it builds one family name -- so a # particle surname reads as a one-word surname does ('Royce, Ed', # 'Berg, MA', 'Berg, PhD'). @@ -1414,8 +1415,8 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name", "suffix-or-name", "suffix-or-name"), notes="#601: a marker behind a title or a suffix word is an ordinary " - "word (rules.md#M2). Before #601: given 'Jane', family 'Doe', " - "suffix 'Jr.', maiden 'Smith Ma'. As pinned then: rules.md#M2's " + "word (rules/M2). Before #601: given 'Jane', family 'Doe', " + "suffix 'Jr.', maiden 'Smith Ma'. As pinned then: rules/M2's " "boundary: the clause's name words stand between 'Jr.' and " "'Ma', so the credential in front speaks for nothing past the " "marker and the clause keeps its member, where 'Jane Doe Jr. " @@ -1491,7 +1492,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'PhD MEng', maiden 'Smith'. As pinned then: the " "maiden walk's given-slot reader asks the same company over the " "name the take would leave. 2.3.0 read the same; #540 had read " @@ -1502,7 +1503,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name", "suffix-or-name"), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'MA Jr Ed', maiden 'Smith'. As pinned then: the " "release check the maiden walk asks at the member it stops on " "reads the span behind it the way assign's given slot does, " @@ -1516,7 +1517,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: title 'Dr.', given " + "ordinary word (rules/M2). Before #601: title 'Dr.', given " "'Jane', family 'Doe', suffix 'Jr Ma', maiden 'Smith'. As " "pinned then: the same release check at a title the walk stops " "on: the given part's title chain runs from the end through " @@ -1597,7 +1598,7 @@ def _check_cjk_shape_purity(self) -> None: "borne name: it has no vowel, and the Scottish prefix " "never detaches from the name it belongs to, so " "'Donald Mc' is not a name anyone writes and its " - "suffix reading costs nothing real. rules.md#P6's " + "suffix reading costs nothing real. rules/P6's " "attachment is scoped to shapes where something has " "already named the family, which this has not, so S2 " "takes the trailing word -- 'an unambiguous suffix is " @@ -1607,7 +1608,7 @@ def _check_cjk_shape_purity(self) -> None: "Mc', both because 'mc' is a never-given particle " "(#360), which is the membership that actually bears " "on #454's example. See " - "decisions.md#suffix-acronym-collisions. The peel " + "decisions/suffix-acronym-collisions. The peel " "leaves 'Donald' the only name word, so O5's " "convention places it and says so (#449): roles " "parity, the flag is #449's"), @@ -1616,7 +1617,7 @@ def _check_cjk_shape_purity(self) -> None: "suffix": "MA"}, ambiguities=("suffix-or-name",)), # #604: the one-letter particle against the initial it spells - # (rules.md#P7). The period decides, in every position a particle + # (rules/P7). The period decides, in every position a particle # reading could otherwise claim the letter. Case("one_letter_particle_written_bare", "Juan Ó Pérez", {"given": "Juan", "family": "Ó Pérez"}, @@ -1640,7 +1641,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Ua", "family": "Buachalla"}, ambiguities=("particle-or-given",), notes="ua joined the AMBIGUOUS half, not the never-given one " - "(#604, decisions.md#vocabulary-collisions): leading, " + "(#604, decisions/vocabulary-collisions): leading, " "it keeps the given reading and reports the fork, " "where never-given would have made the string " "surname-only"), @@ -1705,7 +1706,7 @@ def _check_cjk_shape_purity(self) -> None: notes="Vietnamese ông (Mr) is excluded from TITLES: Ông is a " "surname and Vietnamese names lead with the surname, " "the position the title claim acts on " - "(decisions.md#salutation-titles)"), + "(decisions/salutation-titles)"), Case("office_completes_a_title_run", "Police Commissioner James Gordon", {"title": "Police Commissioner", "given": "James", @@ -1719,7 +1720,7 @@ def _check_cjk_shape_purity(self) -> None: notes="greve is a Scandinavian count AND a surname; shipped on " "the graf precedent, so the uncommaed family-first form " "loses its surname as 'Graf Steffi' does " - "(rules.md#H1's Accepted clause). The two rows below are " + "(rules/H1's Accepted clause). The two rows below are " "the readings it keeps"), Case("surname_borne_title_trailing_is_the_surname", "Anna Greve", {"given": "Anna", "family": "Greve"}, @@ -1732,7 +1733,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Bapak", "given": "Dwi", "family": "Putro"}, classification="fix(#662)", notes="bapak (Indonesian Mr.) was read as the given name " - "(decisions.md#salutation-titles)"), + "(decisions/salutation-titles)"), Case("salutation_in_akan", "Owura Kwame Mensah", {"title": "Owura", "given": "Kwame", "family": "Mensah"}, classification="fix(#662)", @@ -1842,7 +1843,7 @@ def _check_cjk_shape_purity(self) -> None: "last 'PhD', suffix 'Jr.'. The comma between them was " "the shape machinery's artifact and never a reading " "anyone chose: the writer typed no comma, so #436 " - "renders none (rules.md#R1)"), + "renders none (rules/R1)"), Case("title_plus_one_word_nickname_and_suffix", "Dr. (Bud) Smith Jr.", {"title": "Dr.", "family": "Smith", "suffix": "Jr.", @@ -1878,13 +1879,13 @@ def _check_cjk_shape_purity(self) -> None: "'pennie von bergen wessels' as the same SHAPE, which " "it is not: pennie is the given name there and von is " "ambiguous, so this fold cannot fire on it. See " - "decisions.md#P1, 2026-08-17.) 1.4.0 gives last 'de la " + "decisions/P1, 2026-08-17.) 1.4.0 gives last 'de la " "Cruz Juan Carlos' too, so this row must not move when " "#395 lands. #471 DECLINED 2026-09-07: the reach is " "the reading and not a gap in one -- a declared order " "is a property of the data source, so under the " "default order nothing has said this surname ends " - "before the string does. See decisions.md#P1"), + "before the string does. See decisions/P1"), Case("leading_never_given_particle_two_leftovers_family_first", "de la Cruz Juan Carlos", {"family": "de la Cruz", "given": "Juan", "middle": "Carlos"}, @@ -2120,7 +2121,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the words-to-spare test asks whether ANY word ahead " "of the run holds a given role, not whether all of " "them do: the title does not, and the rule must still " - "fire. decisions.md#P6 calls that guard load-bearing"), + "fire. decisions/P6 calls that guard load-bearing"), Case("tussenvoegsel_takes_the_vietnamese_reading", "Nguyen, Thi Van", {"given": "Thi", "family": "Van Nguyen"}, classification="fix(#379)", @@ -2135,7 +2136,7 @@ def _check_cjk_shape_purity(self) -> None: "made load-bearing by giving comma-less FAMILY_FIRST the " "same attachment (there it reads family Van Nguyen too). " "That surviving format is what makes the " - "loss acceptable -- see rules.md#P6. Since #405 the " + "loss acceptable -- see rules/P6. Since #405 the " "loss is at least REPORTED: 'Van' is ambiguous " "vocabulary, so the attachment declines a live reading " "as a name word and says so"), @@ -2185,7 +2186,7 @@ def _check_cjk_shape_purity(self) -> None: "the case that guard protects"), # A family made only of particle vocabulary. Position decides: # nothing joins these words to a name, so they are not acting as - # particles and they anchor the base (rules.md#R2, #404). + # particles and they anchor the base (rules/R2, #404). Case("all_particle_family_anchors_its_own_base", "Anh Do", {"given": "Anh", "family": "Do"}, classification="parity", @@ -2193,7 +2194,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the ROLES are parity -- 1.4.0 gives first 'Anh', last " "'Do' too -- and the views are what #404 moved, which " "this table cannot assert (no initials or base column; " - "rules.md#R2/#R3 carry those). Worth recording which " + "rules/R2/#R3 carry those). Worth recording which " "way: 1.4.0's own _split_last guard kept last_base " "'Do', so the empty family_base was a 2.0 regression " "and this restores it. The initials half was broken in " @@ -2316,7 +2317,7 @@ def _check_cjk_shape_purity(self) -> None: # control lives beside the capitalize() pin instead, in # tests/test_capitalization.py::test_a_one_letter_conjunction_is_case_sensitive_to_repair; # 'Juan y Garcia' (mixed-lower, three words) is absent from this - # table because rules.md#P3 carries it as its boundary example and + # table because rules/P3 carries it as its boundary example and # test_render.py pins its initials. # # The other five rows are three DIFFERENT reasons a row does not @@ -2426,7 +2427,7 @@ def _check_cjk_shape_purity(self) -> None: "ROLE is unchanged. What moves is initials() -- " "ParsedName.initials() gives 'J. G.' where 2.3 gave " "'J. Y. G.', a conjunction contributing none " - "(rules.md#R3) -- and the v1 facade's " + "(rules/R3) -- and the v1 facade's " "HumanName.initials() gave 'J. Y. G.' until #528 made " "it read the same tags on 2026-09-13. That is the " "split the 'john e smith' row records, closed on the " @@ -2513,7 +2514,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name", "given-or-family"), notes="the lean reads CASE, not periods: a single trailing " "period is the abbreviation shape any word can wear " - "(rules.md#S2) and changes nothing here, where the " + "(rules/S2) and changes nothing here, where the " "capitals have already spoken", shape=1), Case("title_case_ambiguous_leans_surname_two_words", "Jack Ma", @@ -2528,7 +2529,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), notes="the one-case control for 'Jack MA': all-caps against " "all-caps is no contrast at all, so nothing leans and " - "rules.md#S2's words-to-spare count decides alone -- " + "rules/S2's words-to-spare count decides alone -- " "two pieces, so the acronym stays the family name", shape=1), Case("one_case_lower_ambiguous_takes_the_count", "jack ma", @@ -2549,7 +2550,7 @@ def _check_cjk_shape_purity(self) -> None: "word beside 'John Smith' is how a surname is written. " "1.4.0 read suffix 'Ma', so this is a deliberate " "parity break in the direction the East Asian surname " - "wants (decisions.md#S2)", + "wants (decisions/S2)", shape=1), Case("one_case_upper_ambiguous_with_words_to_spare", "JOHN SMITH MA", @@ -2587,7 +2588,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the floor the lean does not move, measured " "2026-09-15: the trailing peel's walk starts after the " "leading title run, so one piece is all it sees and " - "rules.md#S2's two-piece floor refuses it. A lean that " + "rules/S2's two-piece floor refuses it. A lean that " "reached below the floor would read this as a title " "with a credential and no name at all -- and nothing " "is reported, because no fork was taken", @@ -2603,7 +2604,7 @@ def _check_cjk_shape_purity(self) -> None: "decision rides with it -- C2's structural flag already " "reports on the comma path, but reports what the parse " "could not recognize, not a fork it called " - "(rules.md#C1's exception, scoped to this class; P6's " + "(rules/C1's exception, scoped to this class; P6's " "attachment fork has reported on a family-comma path " "since 2.3, e.g. 'Berg, Jan vd')", shape=2), @@ -2619,13 +2620,13 @@ def _check_cjk_shape_purity(self) -> None: {"family": "Smith", "suffix": "Jr., MA"}, classification="fix(#289)", ambiguities=("suffix-or-name",), - notes="rules.md#C1's count for this class is of NAME words, " + notes="rules/C1's count for this class is of NAME words, " "not of words: two tokens and one name here, so the " "structure does not flip and the family survives. " "Counting tokens read it as given 'Smith' with suffix " "'Jr., MA' -- no reading of this string puts the " "family anywhere but where this row does " - "(decisions.md#S2)", + "(decisions/S2)", shape=2), Case("the_by_shape_class_never_reaches_is_wholly_suffix", "Smith Jr., A.B.", @@ -2668,7 +2669,7 @@ def _check_cjk_shape_purity(self) -> None: "count of name words before the comma decides, and two " "of them make it a credential. 1.4.0 read suffix 'Ed' " "too, so this restores parity rather than opening " - "distance from it (decisions.md#S2, 2026-09-15)", + "distance from it (decisions/S2, 2026-09-15)", shape=3), Case("the_comma_count_reaches_the_listed_set_lower", "john smith, ma", @@ -2696,7 +2697,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#289)", ambiguities=("suffix-or-name",), notes="the two-word twin of 'Royce, Ed' below, and the case " - "row item 5 of decisions.md#S2 turns on -- no corpus " + "row item 5 of decisions/S2 turns on -- no corpus " "name has this shape, so this row carries the whole " "weight of that decision", shape=3), @@ -2731,7 +2732,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the one place this design QUIETS a report rather than " "adding one: 'DO' is a listed ambiguous acronym " "written in capitals inside a mixed-case name, so the " - "third comma segment is wholly suffix and rules.md#C2 " + "third comma segment is wholly suffix and rules/C2 " "stops flagging it. The FIELDS do not move -- a " "quieted flag is invisible to a reader sweeping for " "new reports, which is why it has a row of its own", @@ -2839,7 +2840,7 @@ def _check_cjk_shape_purity(self) -> None: "three pieces, so the unlisted acronym is the " "credential. 1.4.0 read last 'X.Y.Z.', so this breaks " "parity deliberately -- the parking-lot entry of " - "decisions.md#suffix-acronym-collisions is what it " + "decisions/suffix-acronym-collisions is what it " "closes", shape=1), Case("unlisted_dotted_is_case_blind", "john smith x.y.z.", @@ -2872,7 +2873,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "John", "family": "Smith", "suffix": "Q.W.E.R.T."}, classification="fix(#516)", ambiguities=("suffix-or-name",), - notes="rules.md#S3's own boundary example, which said this " + notes="rules/S3's own boundary example, which said this " "shape stays a name. It moves, and the rule moves with " "it", shape=1), @@ -2917,7 +2918,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#516)", notes="two NAME words before the comma read the part after " "it as the credential run, for the by-shape half " - "exactly as for the listed one (rules.md#C1). Silent " + "exactly as for the listed one (rules/C1). Silent " "since #563: three letters run together have no name " "reading a reader would weigh", shape=3), @@ -2927,7 +2928,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), notes="#563: two words before the comma may be ONE surname, " "and two dotted letters are how a person's own initials " - "are written, so the count gives way (rules.md#C1). " + "are written, so the count gives way (rules/C1). " "2.4's first reading was given 'García', family " "'Márquez', suffix 'G.J.'; 1.4.0 and 2.3.0 read it as " "here, and only the report is new", @@ -2956,7 +2957,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#563)", notes="the contrast to the row above: the degree in front " "makes the pair a credential, and a flip on a dotted " - "word admitted by shape is silent (rules.md#C1). 2.3.0 " + "word admitted by shape is silent (rules/C1). 2.3.0 " "read given 'PhD', middle 'X.Y.'", shape=3), Case("a_title_in_front_of_paired_initials_does_not_speak", @@ -2964,7 +2965,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Ms", "given": "G.J.", "family": "García Márquez"}, notes="#563 review round: 'Ms' is both title and suffix " "vocabulary, and in front of paired initials it is the " - "title of the given part they open (rules.md#C1, as S2 " + "title of the given part they open (rules/C1, as S2 " "reads a dual in that part's title run). The first fix " "let it speak and split the surname in silence. Every " "release reads it this way", @@ -2984,7 +2985,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), notes="nobody writes initials as two dotted groups, so a " "second by-shape word makes the part the credential " - "run (rules.md#C1) -- reported, since here the pairs " + "run (rules/C1) -- reported, since here the pairs " "speak only for each other and each is a word a reader " "takes for a name. 1.4.0 read given 'X.Y.', middle " "'P.Q.'", @@ -3017,7 +3018,7 @@ def _check_cjk_shape_purity(self) -> None: "García Márquez, G.J.R.", {"given": "García", "family": "Márquez", "suffix": "G.J.R."}, classification="fix(#516)", - notes="the ACCEPTED cost of #563's line (rules.md#C1): " + notes="the ACCEPTED cost of #563's line (rules/C1): " "initials are conventionally written apart ('J. R. " "R.'), so three letters run together read as a " "credential even where the name before the comma is " @@ -3041,7 +3042,7 @@ def _check_cjk_shape_purity(self) -> None: shape=2), Case("a_leading_dotted_run_is_untouched", "X.Y.Z. Smith", {"given": "X.Y.Z.", "family": "Smith"}, - notes="rules.md#S2 already says a suffix never opens the " + notes="rules/S2 already says a suffix never opens the " "string, and the shape class is read only by the " "trailing peel and the post-comma slot -- so every " "leading dotted run, 'J.R.R. Tolkien' included, is " @@ -3061,7 +3062,7 @@ def _check_cjk_shape_purity(self) -> None: "and this real credential is not handed to the shape " "class. The WIDE retirement -- any chunk match yielding " "to the shape -- would have read 'Msc.Ed.' as a middle " - "name, which is what rejected it (decisions.md#S2)", + "name, which is what rejected it (decisions/S2)", shape=2), Case("a_glued_honorific_chunk_is_not_ascii", "J.씨", {"given": "J.", "suffix": "씨"}, @@ -3148,7 +3149,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "John", "family": "van der Berg Ma"}, classification="fix(#289)", ambiguities=("suffix-or-name",), - notes="rules.md#S2 says either reading carries the flag, and " + notes="rules/S2 says either reading carries the flag, and " "this reading carried none. assign reports from the " "peel's picks and a pick reaches it only as a LONE " "piece, so once the prefix chain merged 'Ma' into the " @@ -3178,7 +3179,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("comma-structure",), classification="fix(#636)", notes="a part past the second comma holds no name word " - "(rules.md#C2), so group makes no particle chain in it " + "(rules/C2), so group makes no particle chain in it " "and the title word behind 'van der' is a title as in " "'John Smith, Jr., Prof.'; the chain from 'der' had " "taken it into the suffix (1.4.0 and 2.3.0 read suffix " @@ -3190,7 +3191,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("comma-structure",), classification="fix(#636)", notes="the row above after a FAMILY comma: the part past the " - "second comma holds no name word there either (rules.md#C2) " + "second comma holds no name word there either (rules/C2) " "and makes no chain (#636's review: a chain kept after a " "family comma alone passed every other test); 1.4.0 and " "2.3.0 read suffix 'van der Berg Prof.'"), @@ -3201,7 +3202,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#636)", notes="the empty head (AGENTS.md's comma sweep): the part past " "the second comma makes no chain, so the title word is a " - "title (rules.md#C2, #636); 1.4.0 and 2.3.0 read suffix " + "title (rules/C2, #636); 1.4.0 and 2.3.0 read suffix " "'van der Berg Prof.'"), Case("a_credential_run_after_the_comma_reports_no_chain_fork", "John Smith, PhD Do Ma", @@ -3211,7 +3212,7 @@ def _check_cjk_shape_purity(self) -> None: "is the credential run C1 flips to, and assign reads a " "tail segment wholly as suffixes, so 'Ma' ends a suffix. " "The chain may not report taking it into the name, a " - "reading the parse never makes (rules.md#C2), and since " + "reading the parse never makes (rules/C2), and since " "#613 it is not run in the part at all, decide binding the " "part before any join; the one report is " "C1's flip over the whole part. Carried a second, " @@ -3296,7 +3297,7 @@ def _check_cjk_shape_purity(self) -> None: "John Smith, MD, R.A.I.", {"given": "John", "family": "Smith", "suffix": "MD, R.A.I."}, classification="fix(#516)", - notes="rules.md#S3's narrow roman retirement moved 'R.A.I.' " + notes="rules/S3's narrow roman retirement moved 'R.A.I.' " "out of the vocabulary verdict and into the shape " "class, and segment's tail-segment test reads " "`is_wholly_suffix`, which is blind to that class by " @@ -3377,7 +3378,7 @@ def _check_cjk_shape_purity(self) -> None: "without meeting it: three name words, so the count " "reads the capitalised surname as a credential and the " "family becomes 'Pierre'. This row is the argument for " - "the default being off (decisions.md#S2)"), + "the default being off (decisions/S2)"), # #516 review round (quality-review finding): the TWO-word shape # ('Jean DUPONT') is the row `_policy.py`'s own docstring needed # and did not have -- turning the switch on reports a genuine @@ -3434,7 +3435,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), notes="the all-caps SURNAME convention never writes the " "capitals after a comma behind a full name, so the " - "default reads the comma position (rules.md#C1, S2). " + "default reads the comma position (rules/C1, S2). " "2.3.0 read given 'XYZ', family 'John Smith'"), Case("the_comma_caps_reading_can_be_switched_off", "John Smith, XYZ", @@ -3628,7 +3629,7 @@ def _check_cjk_shape_purity(self) -> None: "all-caps words, and every token in the run must be a " "candidate for the run itself to be one. Since #564 the " "DEFAULT reads it the same way, the comma position being " - "on by default, and rules.md#C1's `deviates: #291` " + "on by default, and rules/C1's `deviates: #291` " "marker came off with that change"), Case("the_caps_comma_multi_word_run_declines_at_one_word", "Smith, LEED AP", @@ -3653,7 +3654,7 @@ def _check_cjk_shape_purity(self) -> None: # the switch on -- the caps branch must never ride SHAPE_ACRONYM_ # TAG beside a listed member's own membership tag, which is what # silenced `listed_lean` for these before the fix (F1, decided by - # the reviewer; decisions.md#S2). Both rows are byte-identical to + # the reviewer; decisions/S2). Both rows are byte-identical to # their DEFAULT-policy siblings above (`caps_ambiguous_leans_ # credential`, `the_lean_reaches_the_post_comma_slot`) and carry # no `shape=` tag for the same reason every other policy-on row @@ -3676,7 +3677,7 @@ def _check_cjk_shape_purity(self) -> None: # #516 review round: UNLISTED means in no wordlist at all, not # merely "no whole-token suffix vocabulary" -- a capitalized # particle or particle phrase must not join the class either - # (F1b, decided by the reviewer; decisions.md#S2's amendment). + # (F1b, decided by the reviewer; decisions/S2's amendment). Case("caps_switch_does_not_claim_a_capitalized_particle", "John Smith DE", {"given": "John", "middle": "Smith", "family": "DE"}, @@ -3839,7 +3840,7 @@ def _check_cjk_shape_purity(self) -> None: "name word and the count declines the flip. Without " "that arm two tokens would read as two names and hand " "the family to `given`"), - # rules.md#H6 (#701): a title opening the part before a family + # rules/H6 (#701): a title opening the part before a family # comma. The rows sweep the spellings and comma neighbours the rule # reads; its own examples carry the plain shapes. Case("a_title_before_a_family_comma_peels_beside_a_nickname", @@ -3867,7 +3868,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#701)", notes="'Freiherr' is a particle as well as a title in either " "spelling, so the run stops at it; the bare spelling is " - "rules.md#H6's example"), + "rules/H6's example"), Case("a_period_marked_dual_opening_a_family_comma_stays_family", "Freiherr. von Berg, Ed", {"given": "Ed", "family": "Freiherr. von Berg"}, @@ -3879,7 +3880,7 @@ def _check_cjk_shape_purity(self) -> None: notes="'Ms' is suffix vocabulary too, and a name word follows " "it, so it is a title as in 'Hon. Ms Jane Smith'. The " "second #701 draft stopped in front of any suffix word " - "and read family 'Hon. Smith', suffix 'Ms' (rules.md#H6's " + "and read family 'Hon. Smith', suffix 'Ms' (rules/H6's " "'Rev. Sr.' example; the design-docs review)"), Case("a_title_before_a_family_comma_stops_at_a_bare_saint", "Dr. St Pierre, Ed", @@ -3934,15 +3935,15 @@ def _check_cjk_shape_purity(self) -> None: notes="'gov' joined TITLES in the #701 review, as the listed " "'governor' abbreviated; 1.4.0 through 2.4.2 read given " "'Gov'. 236 US Census surname bearers, so a surname " - "written first loses it (decisions.md#H6)"), + "written first loses it (decisions/H6)"), Case("the_comma_count_counts_names_not_words_behind_a_suffix", "Smith Jr, Ma", {"given": "Ma", "family": "Smith", "suffix": "Jr"}, ambiguities=("suffix-or-name",), notes="the SUFFIX arm of the same count, and the shape " - "decisions.md#S2 names: 'Smith Jr., MA' is two tokens " + "decisions/S2 names: 'Smith Jr., MA' is two tokens " "and one name. The Title-case spelling declines where " - "the all-caps one flips, which is the pair rules.md#C1 " + "the all-caps one flips, which is the pair rules/C1 " "states"), Case("strict_comma_reads_the_dotted_numeral_as_a_name_word", "Smith V., Ma", @@ -3950,7 +3951,7 @@ def _check_cjk_shape_purity(self) -> None: policy=Policy(lenient_comma_suffixes=False), ambiguities=("suffix-or-name",), notes="MEASURED, and a known rough edge recorded rather than " - "repaired (decisions.md#S2, 2026-09-18). Under strict " + "repaired (decisions/S2, 2026-09-18). Under strict " "the initial-shaped 'V.' fails the suffix test, so " "name_word_count sees TWO name words before the comma, " "flips the structure, and the positional read of " @@ -4029,8 +4030,8 @@ def _check_cjk_shape_purity(self) -> None: "John Smith 1.4.2", {"given": "John", "middle": "Smith", "family": "1.4.2"}, classification="fix(#516)", - notes="ACCEPTED and SILENT, recorded at decisions.md#S2. " - "rules.md#S3's narrow retirement drops a dotted token " + notes="ACCEPTED and SILENT, recorded at decisions/S2. " + "rules/S3's narrow retirement drops a dotted token " "whose every matched chunk is a single ASCII " "character, and the vocabulary's lone digit '2' is one " "of them -- so the chunk claim goes, and the shape " @@ -4264,7 +4265,7 @@ def _check_cjk_shape_purity(self) -> None: notes="why the condition asks assign's TRAILING-RUN walk " "rather than the suffix-piece test: a bare 'V' is " "suffix vocabulary written like an initial, so " - "is_suffix_piece refuses it (rules.md#S2's initial " + "is_suffix_piece refuses it (rules/S2's initial " "veto) and a class test spelled with that predicate " "alone would let the link swallow it. The peel takes " "it, so trailing_start puts it outside the name's own " @@ -4439,7 +4440,7 @@ def _check_cjk_shape_purity(self) -> None: shape=2, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', maiden 'Puig i Soler'. As pinned then: the comma form, " "and a different reader rather than a second member of one " "shape: segment 1 is read at the GIVEN slot, so the leak landed " @@ -4480,7 +4481,7 @@ def _check_cjk_shape_purity(self) -> None: "'Puig' for want of maiden support", shape=1), # ---- #397 second review: the one-case role movers -------------- - # decisions.md#P3's 2026-09-20 bullet describes these four names + # decisions/P3's 2026-09-20 bullet describes these four names # in prose and pinned none of them. They are where the marked # subset's riskiest movement is -- an ALL-LOWER name whose letter # the parent read as the GENERATION and this reads as an initial @@ -4502,7 +4503,7 @@ def _check_cjk_shape_purity(self) -> None: "ALL-CAPS twin 'ROVIRA, I' gave first 'I' at BOTH of " "them and gives given 'I' here -- the two spellings " "disagreed at 2.3.0 and agree now. NO SHAPE TAG on " - "any of these four: decisions.md#P3's one-case " + "any of these four: decisions/P3's one-case " "paragraph says of them that none is a corpus name, " "and admitting them to a corpus is a change of its " "own rather than this row's business. The cross-version readings " @@ -4564,7 +4565,7 @@ def _check_cjk_shape_purity(self) -> None: # part holds nothing else. Asked of the whole PART, never of a # word count. These rows carry the ROLES, which this table's # expectations are; the initials each one is about are named in - # its notes and asserted by rules.md#R3's example lines and by + # its notes and asserted by rules/R3's example lines and by # tests/v2/test_render.py. Case("a_connective_among_given_names_stops_initialing", "John and Jane Smith", @@ -4647,7 +4648,7 @@ def _check_cjk_shape_purity(self) -> None: "connective has given names on both sides of it, so it " "joins. 'Х. И. М. С.' becomes 'Х. М. С.'. A Cyrillic " "capital is not the shape an initial is written in " - "(rules.md#P3), so the letter is the connective in " + "(rules/P3), so the letter is the connective in " "every spelling of this name", shape=1), # ---- #531: the given part's trailing slot ---------------------- @@ -4668,7 +4669,7 @@ def _check_cjk_shape_purity(self) -> None: "construction and the positional reading at this slot " "IS the credential. Same answer as the comma-less " "'John Doe MA', which is the whole point " - "(decisions.md#S2)", + "(decisions/S2)", shape=2), Case("the_given_parts_trailing_dotted_slot_reads_the_credential", "Doe, John X.Y.Z.", @@ -4683,7 +4684,7 @@ def _check_cjk_shape_purity(self) -> None: "at this slot is the credential -- matching the " "comma-less 'John Doe X.Y.Z.' and the 2.4 shape rule " "rather than v1. One name, one new divergence, accepted " - "(decisions.md#S2)", + "(decisions/S2)", shape=2), Case("the_given_parts_trailing_slot_in_all_caps", "DOE, JOHN MA", @@ -4709,7 +4710,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#531)", ambiguities=("suffix-or-name",), notes="'ba' is another listed member (one of those " - "decisions.md#suffix-acronym-collisions marked " + "decisions/suffix-acronym-collisions marked " "ambiguous rather than removing), so the slot is not a " "rule about 'ma' -- it is the whole class", shape=2), @@ -4839,7 +4840,7 @@ def _check_cjk_shape_purity(self) -> None: "the walk stops at the declined pick rather than " "continuing past it -- so the all-caps 'MA' in front " "is never asked and never reports. ONE report, for " - "'Ma'. Identical in shape to rules.md#S2's accepted " + "'Ma'. Identical in shape to rules/S2's accepted " "'Jack Wei Ma' clause", shape=2), Case("a_taken_member_behind_a_declined_one_still_reports", @@ -4938,7 +4939,7 @@ def _check_cjk_shape_purity(self) -> None: "only if a reader would hesitate too' is why this must " "stay silent rather than why it happens to -- that " "sentence is the 2.0-conventions section's, and this " - "note cited rules.md#A1 for it until 2026-09-19. NOT " + "note cited rules/A1 for it until 2026-09-19. NOT " "1.4.0 " "parity, and not #531's doing either: v1 read middle " "'Smith', suffix 'MA', and 2.0.0 already read what " @@ -4977,7 +4978,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'MA', maiden 'Smith'. As pinned then: the row " "that named the silence, now naming the reading. The maiden " "marker no longer claims a trailing credential: the words it " @@ -5072,7 +5073,7 @@ def _check_cjk_shape_purity(self) -> None: "word the walk was asked about is the LAST piece of " "the maiden name, and a member with a name word behind " "it is never that piece. Nothing was consulted, so " - "nothing reports -- which is what rules.md#A1's " + "nothing reports -- which is what rules/A1's " "hesitating reader asks for rather than an accident of " "where the emitter sits. Unchanged from 2.0.0; 1.4.0 " "read middle 'Doe nee MA', family 'Smith'", @@ -5216,7 +5217,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Prof.", "given": "Jane", "family": "Doe", "maiden": "Smith"}, classification="fix(#535)", - notes="rules.md#M2's title stop: a period-marked word the " + notes="rules/M2's title stop: a period-marked word the " "trailing title chain takes (H5) ends the clause, as " "'Jane Doe Prof.' reads the title. 2.3.0 read maiden " "'Smith Prof.'", @@ -5250,7 +5251,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: title 'Prof.', given " + "ordinary word (rules/M2). Before #601: title 'Prof.', given " "'Jane', family 'Doe', suffix 'MA', maiden 'Smith'. As pinned " "then: the given part's own slot reads the chain too ('Doe, " "Jane MA Prof.' gives title and suffix). 2.3.0 read maiden " @@ -5261,7 +5262,7 @@ def _check_cjk_shape_purity(self) -> None: "family": "Smith"}, classification="fix(#623)", notes="the given part's particle chain stops in front of the " - "trailing run its walk reads, titles included (rules.md#H5, " + "trailing run its walk reads, titles included (rules/H5, " "#623), so the title is the title as in 'Smith, John " "Prof.' and 'John van der Berg Prof.'; 1.4.0 and 2.3.0 " "read middle 'van der Berg Prof.'"), @@ -5272,7 +5273,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#623)", notes="P6 reads the given part as though the trailing title " "were absent, so 'van' attaches to the family as in " - "'Smith, Jan van' (rules.md#H5, #P6; #623); 1.4.0 and " + "'Smith, Jan van' (rules/H5, #P6; #623); 1.4.0 and " "2.3.0 read middle 'van Prof.'"), Case("a_title_particle_heading_the_given_part_keeps_the_bound", "Smith, Freiherr von Berg Prof.", @@ -5309,7 +5310,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Freiherr", "given": "van Berg X.Y.Z.", "family": "Smith"}, classification="parity", - notes="group's re-ask after the chain (rules.md#P2): behind a " + notes="group's re-ask after the chain (rules/P2): behind a " "word in both the title and particle vocabularies the " "chain's merges leave the dotted acronym no word to " "spare, so the chain runs again without its stop and " @@ -5323,12 +5324,12 @@ def _check_cjk_shape_purity(self) -> None: "family": "Smith"}, classification="parity", notes="the contrast to the row above: a bare title word ending " - "a name is a name word (rules.md#H5), so the chain takes " + "a name is a name word (rules/H5), so the chain takes " "it (#623 moves only the period-marked spelling)"), Case("the_first_word_after_the_marker_stays_even_as_a_title", "Jane Doe nee King.", {"given": "Jane", "family": "Doe", "maiden": "King."}, - notes="rules.md#M2's first-word floor covers the title stop: " + notes="rules/M2's first-word floor covers the title stop: " "the marker announced a name, and 'King.' is a borne " "surname the title vocabulary also lists. Unchanged " "from 2.3.0"), @@ -5347,7 +5348,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Dr. Prof.", "given": "nee", "middle": "Jones", "family": "Smith"}, classification="fix(#601)", notes="#601: a marker behind a title or a suffix word is an ordinary " - "word (rules.md#M2). Before #601: title 'Dr.', maiden 'Jones " + "word (rules/M2). Before #601: title 'Dr.', maiden 'Jones " "Smith Prof.'. As pinned then: the view check: the take would " "leave 'Dr. Prof.', where H5's chain has no name word to stand " "behind and 'Prof.' would be the family name, so the clause " @@ -5364,9 +5365,9 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Prof.", "given": "Jane", "family": "van der Berg", "maiden": "Smith"}, classification="fix(#601)", notes="#601: the clause ends at the clause-free name's trailing run, " - "which the take consumes (rules.md#M2). Before #601: given " + "which the take consumes (rules/M2). Before #601: given " "'Jane', family 'van der Berg', maiden 'Smith Prof.'. As pinned " - "then: P2's chain runs on over a trailing title (rules.md#H5's " + "then: P2's chain runs on over a trailing title (rules/H5's " "Accepted 'John van der Berg Prof.'), so releasing the title " "would put it in the family; the clause keeps it. Unchanged " "from 2.3.0"), @@ -5375,7 +5376,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "abdul", "middle": "nee Smith", "family": "Berg", "suffix": "V"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'abdul', " + "ordinary word (rules/M2). Before #601: given 'abdul', " "family 'Berg', maiden 'Smith V'. As pinned then: the numeral " "stop now asks M2's join question too: released, the V was " "taken by the bound-given join after the comma (P5) and read " @@ -5386,7 +5387,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Prof.", "given": "Jane", "family": "Doe", "maiden": "Smith"}, classification="fix(#535)", - notes="rules.md#M2 Accepted: bracket content ending in a " + notes="rules/M2 Accepted: bracket content ending in a " "period is not extracted, so the brackets drop and the " "clause is read bare -- and the bare clause now gives " "the title up. 2.3.0 read maiden 'Smith Prof.' for the " @@ -5406,9 +5407,9 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Jane", "middle": "nee Smith V", "family": "Doe", "suffix": "PhD"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'PhD', maiden 'Smith V'. As pinned then: " - "rules.md#M2's given-slot reader declines the numeral where a " + "rules/M2's given-slot reader declines the numeral where a " "third comma part follows (#144's own condition, asked of the " "maiden clause too): the given part is not the LAST comma part, " "so the release is withdrawn and the clause keeps 'V'. 2.3.0 " @@ -5419,7 +5420,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Jane", "middle": "nee V", "family": "Doe", "suffix": "PhD"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'PhD', maiden 'V'. As pinned then: the numeral " "stop reads FROM the marker and is not held to the first-word " "floor; here the given slot does not read a lone numeral as a " @@ -5434,7 +5435,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: title 'Prof.', given " + "ordinary word (rules/M2). Before #601: title 'Prof.', given " "'Jane', family 'Doe', suffix 'MA', maiden 'Smith Rev.'. As " "pinned then: after a family comma the given part's title chain " "runs from the end over what its first suffix pass leaves, and " @@ -5448,7 +5449,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Prof.", "given": "Jane", "middle": "nee Smith", "family": "Doe", "suffix": "V"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: title 'Prof.', given " + "ordinary word (rules/M2). Before #601: title 'Prof.', given " "'Jane', family 'Doe', suffix 'V', maiden 'Smith'. As pinned " "then: the lenient trailing numeral (#144) is a suffix of the " "given part's first pass once the numeral stop has asked it, so " @@ -5460,7 +5461,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Prof.", "given": "Jane", "middle": "nee Smith V", "family": "Doe", "suffix": "PhD"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: title 'Prof.', given " + "ordinary word (rules/M2). Before #601: title 'Prof.', given " "'Jane', family 'Doe', suffix 'PhD', maiden 'Smith V'. As " "pinned then: the same third-comma-part decline reaches through " "the title chain: the numeral stays maiden text and the title " @@ -5492,13 +5493,13 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Prof.", "given": "Jane", "family": "Doe", "maiden": "V"}, classification="fix(#601)", notes="#601: the first word after the marker is the maiden name (open " - "on #601) (rules.md#M2). Before #601: title 'Prof.', given " + "on #601) (rules/M2). Before #601: title 'Prof.', given " "'Jane', middle 'Doe', family 'nee', suffix 'V'. As pinned " "then: the numeral stop reads FROM the marker by design, unlike " "the credential and title stops, so a numeral standing straight " "after it is not held to the first-word floor, and here it " "declines the clause -- as 'Jane Smith née V' does bare " - "(rules.md#M2) -- and the title behind the numeral then reads " + "(rules/M2) -- and the title behind the numeral then reads " "the declined name as 'Jane Doe nee V' plus the title would. " "2.3.0 read maiden 'V Prof.'"), Case("a_title_behind_a_tail_bound_numeral_keeps_both", @@ -5506,7 +5507,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Jane", "middle": "nee Smith Prof. V", "family": "Doe", "suffix": "PhD"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'PhD', maiden 'Smith Prof. V'. As pinned then: " "NOT the transparent twin of 'Doe, Jane nee Smith V Prof., PhD' " "(title 'Prof.', maiden 'Smith V'): the third comma part's " @@ -5525,7 +5526,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: the clause ends at the clause-free name's trailing run, " - "which the take consumes (rules.md#M2). Before #601: title " + "which the take consumes (rules/M2). Before #601: title " "'Prof.', given 'Jane', family 'Doe', maiden 'Smith i DO'. As " "pinned then: with the title chained, the DO is the trailing " "peel's, so the link exception refuses 'i' and the walk would " @@ -5541,7 +5542,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Prof.", "given": "abdul", "middle": "nee Smith V", "family": "Berg", "suffix": "i, MD"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'abdul', " + "ordinary word (rules/M2). Before #601: given 'abdul', " "family 'Berg', suffix 'MD', maiden 'Smith i V Prof.'. As " "pinned then: the given-slot twin of the row above: giving up " "'i V Prof.' would leave the V a middle name behind the bound " @@ -5554,7 +5555,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("particle-or-given",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', maiden 'Smith St.'. As pinned then: 'St.' is a title " "and a particle. Released after a family comma, P6 would attach " "it to the family ('Doe, Jane St.' reads family 'St. Doe'), a " @@ -5566,7 +5567,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("particle-or-given", "suffix-or-name"), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', maiden 'Smith MA St.'. As pinned then: the same guard " "with a credential in front of the title: releasing 'MA St.' " "would hand 'St.' to the family, so the clause keeps both. " @@ -5587,7 +5588,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'MA Ph. D.', maiden 'Smith'. As pinned then: the " "given-slot twin of the row above. 2.3.0 read maiden 'Smith " "MA', suffix 'Ph. D.'"), @@ -5743,7 +5744,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', maiden 'Smith Ma'. As pinned then: after a family comma " "the comma has already settled the count, so the reader is " "#531's slot and the writing decides alone -- Title case in a " @@ -5756,7 +5757,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'ma', maiden 'Smith'. As pinned then: the row " "that makes the COMMA reader load-bearing rather than " "decorative: the count-based reader declines an all-lower " @@ -5772,7 +5773,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("particle-or-given",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', maiden 'Smith do'. As pinned then: 'do' is the one " "class member that is also particle vocabulary, so after a " "comma the clause's trailing slot, #531's slot and P6's " @@ -5787,11 +5788,11 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'DO', maiden 'Smith'. As pinned then: the same " "word where the capitals speak: a positive credential lean is " "the one spelling that outranks P6's attachment " - "(decisions.md#S2, 2026-09-18), so the clause gives the word " + "(decisions/S2, 2026-09-18), so the clause gives the word " "up. 1.4.0 read suffix 'DO'"), Case("the_no_comma_do_reads_by_the_count_instead", "Jane Doe nee Smith do", @@ -5822,7 +5823,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("particle-or-given",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', maiden 'Smith MA do'. As pinned then: where #531's " "FLOOR earns its place, and a check that asked only about 'MA' " "got this wrong: the take would leave 'Jane MA do', where 'do' " @@ -5830,7 +5831,7 @@ def _check_cjk_shape_purity(self) -> None: "'MA' as a MIDDLE name -- releasing it from the clause would " "move a word from one person's name into another's. With the " "floor the clause keeps 'MA do' whole, and reports the 'do' it " - "kept. rules.md#S2's own 'Doe, John MA do' clause is the " + "kept. rules/S2's own 'Doe, John MA do' clause is the " "reading this rests on"), Case("the_clamp_never_takes_the_first_word_after_the_marker", "Doe, J. nee MA ba", @@ -5839,7 +5840,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name", "suffix-or-name"), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'J.', family " + "ordinary word (rules/M2). Before #601: given 'J.', family " "'Doe', suffix 'ba', maiden 'MA'. As pinned then: the floor is " "a CLAMP, not a veto, and this row is why. The peel takes 'ba' " "and then 'MA', so the first piece the peel took IS the only " @@ -5857,7 +5858,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the recorded control for the clamp: this is what the " "released 'ba' must go on reading, and it is #531's " "slot doing the reading. 'ba' is the fifth listed " - "member, the one decisions.md#suffix-acronym-collisions " + "member, the one decisions/suffix-acronym-collisions " "marked ambiguous rather than removing. 1.4.0 parity", shape=2), Case("the_clause_leaves_a_middle_initial_alone", @@ -5867,7 +5868,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', middle " + "ordinary word (rules/M2). Before #601: given 'Jane', middle " "'Q.', family 'Doe', suffix 'MA', maiden 'Smith'. As pinned " "then: the member leaves the clause and 'Q.' stays the middle " "initial it always was -- the stop reaches the clause's " @@ -5879,8 +5880,8 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: title 'Dr.', family " - "'Doe', maiden 'Smith MA'. As pinned then: rules.md#M2's " + "ordinary word (rules/M2). Before #601: title 'Dr.', family " + "'Doe', maiden 'Smith MA'. As pinned then: rules/M2's " "invariant, and the row that used to record the opposite: an " "earlier round of #533 read suffix 'MA' here and said so in " "silence. Once 'Smith' leaves with the marker, segment 1 is " @@ -5900,7 +5901,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'abdul', " + "ordinary word (rules/M2). Before #601: given 'abdul', " "family 'Berg', maiden 'Jones MA'. As pinned then: the other " "half of M2's invariant: P5's LENIENT post-comma join runs " "BELOW the marker pass and would swallow the released 'MA' into " @@ -5927,9 +5928,9 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Degenerate input, pinned to " + "ordinary word (rules/M2). Degenerate input, pinned to " "detect change rather than for a reading anyone wants " - "(decisions.md#M2). Before #601: given 'Jane', family " + "(decisions/M2). Before #601: given 'Jane', family " "'van der Berg', maiden 'Smith DO'. As pinned then: M2's " "invariant against P2 rather than P5. 'DO' is particle " "vocabulary standing behind a particle piece, so the chain " @@ -5946,7 +5947,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name", "suffix-or-name"), notes="#601: the clause ends at the clause-free name's trailing run, " - "which the take consumes (rules.md#M2). Before #601: given " + "which the take consumes (rules/M2). Before #601: given " "'Jane', family 'Doe', maiden 'Smith DO DO'. As pinned then: " "the no-comma spelling of the same decline, where what would do " "the joining is the OTHER released member: the trailing peel " @@ -5990,7 +5991,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("particle-or-given",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', maiden 'Smith Do'. As pinned then: #531's reading at " "the given slot, reached through a clause: a member that is " "also particle vocabulary is the credential on a POSITIVE lean " @@ -6014,7 +6015,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Jane", "family": "Doe", "maiden": "Smith MA"}, classification="parity", ambiguities=(), - notes="rules.md#M2, the delimited clause: the writer drew " + notes="rules/M2, the delimited clause: the writer drew " "the boundary and it outranks every reading inside it, " "so no fork is called and none is reported. The marker " "inside the pair is what says the span is a maiden " @@ -6078,7 +6079,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("comma-structure",), notes="#601: a marker in a part after a suffix comma is an ordinary " - "word (rules.md#M2). Before #601: given 'John', family 'Smith', " + "word (rules/M2). Before #601: given 'John', family 'Smith', " "suffix 'Jr', maiden 'Jones MA'. As pinned then: the other NONE " "reader, and the row that killed the first prototype: a segment " "past the second comma is read as credentials whole, so no " @@ -6093,7 +6094,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'MA', maiden 'Smith'. As pinned then: the knob " "governs the LENIENT trailing predicate, which this slot does " "not inherit, so the reading is the default's"), @@ -6104,7 +6105,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'MA', maiden 'Smith'. As pinned then: name_order " "does not enter it: the peel is order-independent and the comma " "has already named the family, so all three orders move the " @@ -6118,7 +6119,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#601)", ambiguities=("suffix-or-name",), notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', family " + "ordinary word (rules/M2). Before #601: given 'Jane', family " "'Doe', suffix 'MA', maiden 'Smith'. As pinned then: the third " "order, for the same reason as the row above"), Case("a_leading_title_takes_the_slot_the_member_would_have_had", @@ -6462,7 +6463,7 @@ def _check_cjk_shape_purity(self) -> None: "applies -- and at this slot the positional reading IS " "the credential. 1.4.0 parity, measured. A composed " "CJK comma form, so tolerated rather than shape-tagged " - "(rules.md#W3's 2026-09-01 demotion)", + "(rules/W3's 2026-09-01 demotion)", tolerated=True), Case("a_hangul_family_takes_the_trailing_credential_too", "김, 민준 MA", @@ -6481,7 +6482,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Steven", "family": "Hardman", "suffix": "RN - CRNA"}, classification="fix(#603)", notes="a credential opening the part after the comma makes it " - "the postnominal part (rules.md#C1, #603), so the " + "the postnominal part (rules/C1, #603), so the " "pre-comma name keeps its positional read and the " "undeclared delimiter is one more word of the run. v1 and " "2.0 through 2.3 read v1's walk order here, the first " @@ -6493,7 +6494,7 @@ def _check_cjk_shape_purity(self) -> None: "John Smith, Ma Jones", {"given": "Ma", "middle": "Jones", "family": "John Smith"}, ambiguities=("suffix-or-name",), - notes="the boundary of #603's opened part (rules.md#C1): only " + notes="the boundary of #603's opened part (rules/C1): only " "a word that starts S2's run opens it, and a member of " "the ambiguous credential class does not, so the part " "keeps the listing form's given name and the first " @@ -6501,7 +6502,7 @@ def _check_cjk_shape_purity(self) -> None: Case("a_dual_opens_no_part_behind_a_two_word_surname", "García Márquez, Ms Gabriela", {"title": "Ms", "given": "Gabriela", "family": "García Márquez"}, - notes="#603's boundary for a title/suffix dual (rules.md#C1): it " + notes="#603's boundary for a title/suffix dual (rules/C1): it " "opens nothing, being the listing form's title in front " "of a given name, and no count before the comma tells a " "two-word surname from a given name and a family name. A " @@ -6526,7 +6527,7 @@ def _check_cjk_shape_purity(self) -> None: "suffix": "Ph. D. and Mary Jones"}, ambiguities=("suffix-or-name", "suffix-or-name", "suffix-or-name"), classification="fix(#603)", - notes="the two spellings agree (rules.md#C1): since #613 the " + notes="the two spellings agree (rules/C1): since #613 the " "part after the comma is read before group's connective " "join, so 'Ph. D.' opens it as 'PhD' does ('John Smith, " "PhD and Mary Jones' reads the same) and the words it takes " @@ -6576,7 +6577,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=(), classification="fix(#613)", notes="the one join made inside a part read as postnominal " - "(rules.md#C1, P3): a connective between titles binds " + "(rules/C1, P3): a connective between titles binds " "them into one title. #613's first draft bound each word " "on its own and lost it (`_comma._title_chains` then, " "group's `_pieces.join_connectives` since #617)"), @@ -6596,7 +6597,7 @@ def _check_cjk_shape_purity(self) -> None: "family": "Smith"}, ambiguities=(), classification="fix(#617)", - notes="rules.md#P3's connective runs ('of the') merge before " + notes="rules/P3's connective runs ('of the') merge before " "they join, in the part after the comma as in the name: " "the comma decision joins with group's own loop since " "#617. Its copy joined 'Minister of the' alone, left " @@ -6628,7 +6629,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Mr. y and Mrs.", "given": "John", "family": "Smith"}, ambiguities=(), classification="fix(#617)", - notes="rules.md#P3's carve-out keeps a LONE single-letter " + notes="rules/P3's carve-out keeps a LONE single-letter " "connective a word; one beside another connective is a " "member of their run, which joins whole. 2.3.0 read the " "same; 83f4e914 kept the 'y' out of the copy's chain and " @@ -6644,7 +6645,7 @@ def _check_cjk_shape_purity(self) -> None: "the 'y' a word in a part this short ('Smith, Sr. y Sra. " "Juan Maria' joins it); the comma-less 'Sr. y Sra. Juan " "Garcia Lopez' joins it too. Read so by 2.3.0 and " - "83f4e914 alike (decisions.md#P3, #617)"), + "83f4e914 alike (decisions/P3, #617)"), Case("a_comma_part_of_one_connective_is_the_given_name", "Smith, and", {"given": "and", "family": "Smith"}, @@ -6677,7 +6678,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Otto", "family": "von und zu Habsburg"}, ambiguities=(), classification="parity", - notes="rules.md#P3's join takes the particle's prefix kind, so " + notes="rules/P3's join takes the particle's prefix kind, so " "'von und zu' chains onto the name as one particle (v1 PR " "#191); the derivation had only v1 tests behind it " "(#617's test review)"), @@ -6785,7 +6786,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Mr.", "given": "y", "family": "John Smith"}, ambiguities=(), classification="fix(#613)", - notes="rules.md#P3's single-letter connective is left as it stands by the comma decision's title join (`_comma._title_chains` in #613, `_pieces.join_connectives` since #617), so it is a name word and the part the listing form's (mutation measured in #613's PR review)"), + notes="rules/P3's single-letter connective is left as it stands by the comma decision's title join (`_comma._title_chains` in #613, `_pieces.join_connectives` since #617), so it is a name word and the part the listing form's (mutation measured in #613's PR review)"), Case("a_periodless_name_word_carries_the_contrast_a_period_does_not", "Chas. SMITH JONES, XYZ", {"given": "XYZ", "family": "Chas. SMITH JONES"}, @@ -6805,7 +6806,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), classification="fix(#613)", tolerated=True, - notes="rules.md#W3 tolerated: peels as '김민준씨, PhD' does; b547c247 kept family '김민준씨'"), + notes="rules/W3 tolerated: peels as '김민준씨, PhD' does; b547c247 kept family '김민준씨'"), Case("an_unlisted_caps_credential_divides_a_cjk_name_as_decide_reads_it", "Smith 김민준씨, XYZ", {"given": "Smith", "middle": "김", "family": "민준", @@ -6813,7 +6814,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), classification="fix(#613)", tolerated=True, - notes="rules.md#W3 tolerated: script_segment asks the comma's " + notes="rules/W3 tolerated: script_segment asks the comma's " "decision itself (#613's /simplify), which reads 'XYZ' " "as a credential behind a whole name, so the name " "divides and the honorific peels, as b547c247 read it; " @@ -6824,7 +6825,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), classification="fix(#613)", tolerated=True, - notes="rules.md#W3 tolerated: the split-off follows the comma's " + notes="rules/W3 tolerated: the split-off follows the comma's " "own decision, a credential behind a whole name, as " "b547c247 read it; the hand copy #613's /simplify " "replaced had kept 'さん' glued"), @@ -6835,7 +6836,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=(), classification="fix(#613)", tolerated=True, - notes="rules.md#W3 tolerated: decide reads the dotted shape as " + notes="rules/W3 tolerated: decide reads the dotted shape as " "a credential behind two name words, and the division " "follows it, as b547c247 read it"), Case("a_title_and_one_cjk_name_word_before_a_credential", @@ -6844,7 +6845,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), classification="fix(#613)", tolerated=True, - notes="rules.md#W3 tolerated, decided with Derek (#613's " + notes="rules/W3 tolerated, decided with Derek (#613's " "/simplify): script_segment asks the comma's decision, " "which licenses 'MA' by the capitals lean behind a title " "and a name word and reads a suffix comma, so the name " @@ -6868,7 +6869,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=(), classification="fix(#613)", tolerated=True, - notes="rules.md#W3 tolerated: the comma's decision reads 'Dr. " + notes="rules/W3 tolerated: the comma's decision reads 'Dr. " "J.씨' as a title and a credential behind a whole name, " "so the peel stays in the part before the comma and the " "part reads as 'John Smith, J.씨' does; b547c247's hand " @@ -6889,7 +6890,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "MA", "family": "Smith", "suffix": "PhD Jones"}, ambiguities=("suffix-or-name", "suffix-or-name"), notes="#603's opener is the part's FIRST suffix word with only " - "titles in front of it (rules.md#C1): 'MA', a member of " + "titles in front of it (rules/C1): 'MA', a member of " "the ambiguous class, opens nothing and ends the leading " "run, so the 'PhD' behind it opens nothing either and the " "part is the walk's. Letting a later credential open it " @@ -6898,7 +6899,7 @@ def _check_cjk_shape_purity(self) -> None: {"family": "Smith", "suffix": "Jr. John"}, ambiguities=("suffix-or-name",), classification="fix(#603)", - notes="decisions.md#C1's accepted cost: a generational word " + notes="decisions/C1's accepted cost: a generational word " "starts S2's run, so it opens the part and 'John' is " "absorbed, one word before the comma reading the whole " "part as suffixes (Derek). 2.3.0 read title 'Jr.', given " @@ -6910,7 +6911,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("comma-structure",), classification="fix(#603)", notes="C2's tail title read off the piece group's connective join " - "built (rules.md#C2): the join tags the piece a title " + "built (rules/C2): the join tags the piece a title " "though its first token, 'and', is no title word, so the " "test is is_title_piece's whole and not a first-token " "check (the /simplify review found a first-token check " @@ -7211,7 +7212,7 @@ def _check_cjk_shape_purity(self) -> None: "given part the dash stays a word -- and since #602 a " "word after the credential is in the run: one entry " "'RN - CRNA', the dash unreported, holding no letter or " - "digit (rules.md#A2). Before #602 the stray token kept " + "digit (rules/A2). Before #602 the stray token kept " "its per-piece walk role, middle '-', suffix 'RN, CRNA' " "(v1 parity, pinned live 2026-07-16)"), Case("suffix_delimiter_suffix_comma", "John Smith, RN - CRNA", @@ -7252,7 +7253,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("comma-structure",), notes="a declared core parts a tail as a comma would, so a " "connective beside it joins nothing across it and the " - "core is dropped (rules.md C1, #549) -- 1.4.0's " + "core is dropped (rules/C1, #549) -- 1.4.0's " "expand_suffix_delimiter reading, where 2.0.0 through " "2.3.0 merged the core into the joined piece and " "rendered 'PhD - and MD'"), @@ -7276,7 +7277,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("comma-structure",), notes="the marker ends its part with nothing behind it, as it " "does before a typed comma, so it takes no clause " - "(rules.md C1's Accepted entry, #549). 2.0.0 through " + "(rules/C1's Accepted entry, #549). 2.0.0 through " "2.3.0 read maiden 'Jones', stepping past the core"), Case("suffix_delimiter_core_that_survives_is_a_boundary_too", "Smith, MD - PhD - FACS", @@ -7393,7 +7394,7 @@ def _check_cjk_shape_purity(self) -> None: "name word, so the convention picks the field and says " "so (#449). Roles parity; the flag is #449's"), Case("title_only", "Dr.", {"title": "Dr."}, - notes="rules.md#H4's boundary as well as a shape row: a LONE " + notes="rules/H4's boundary as well as a shape row: a LONE " "title word demotes nothing, so no reading was chosen " "and nothing is reported. The title-vs-given-name " "collision on a word like `Baron` is #348's"), @@ -7430,7 +7431,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "John", "middle": "Jack", "family": "Kennedy", "suffix": "PhD MD"}, classification="fix(#436/#437)", - notes="the NO-COMMA path, which is the one rules.md#R1 " + notes="the NO-COMMA path, which is the one rules/R1 " "described and the parser violated from 1.4.0 to " "2.2.0: no comma stands between PhD and MD, so none " "is rendered. The corpus twin is 'John Doe MD PhD'"), @@ -7483,7 +7484,7 @@ def _check_cjk_shape_purity(self) -> None: "the roz collision -- a marker entry that is also an " "ordinary word eats the family name, and a phrase entry " "is how that is avoided rather than accepted " - "(decisions.md#vocabulary-collisions). Parity"), + "(decisions/vocabulary-collisions). Parity"), Case("phrase_marker_split_by_a_clause_is_not_a_marker", "Anna z (domu) Nowak", {"given": "Anna", "middle": "z", "family": "Nowak", @@ -7716,7 +7717,7 @@ def _check_cjk_shape_purity(self) -> None: "maiden": "von der Berg"}, classification="fix(#399)", notes="a particle chain on each side of the marker. This is " - "the shape decisions.md#M2 first cited against moving " + "the shape decisions/M2 first cited against moving " "the maiden handler ahead of the chain and then " "retracted -- merged and unmerged pieces hand the " "consumer the same words -- and the handler now does " @@ -7789,7 +7790,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "abd", "maiden": "Jones Jr Smith Berg"}, classification="fix(#601)", notes="#601: a credential in the clause starts a run the take " - "consumes (with #602) (rules.md#M2). Before #601: given 'abd', " + "consumes (with #602) (rules/M2). Before #601: given 'abd', " "middle 'Jr Smith', family 'Berg', maiden 'Jones'. As pinned " "then: #411's shape, re-pinned twice. The maiden walk stops at " "the inner suffix and takes only 'Jones'; with the marker pass " @@ -7841,7 +7842,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "abd", "middle": "née Jones", "family": "Berg"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: family 'Berg', " + "ordinary word (rules/M2). Before #601: family 'Berg', " "suffix 'abd', maiden 'Jones'. As pinned then: 'abd' is the one " "word in both the bound-given and the suffix vocabulary, and P5 " "says the suffix reading wins in the given slot after a family " @@ -7881,7 +7882,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "abdul", "middle": "née Jones", "family": "van der Berg"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'abdul', " + "ordinary word (rules/M2). Before #601: given 'abdul', " "family 'van der Berg', maiden 'Jones'. As pinned then: was the " "second of #412's two join-swallows and is now fixed as a side " "effect of #411, which is why the row is renamed rather than " @@ -7918,7 +7919,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Jane", "middle": "van der Berg née Jones", "family": "Smith"}, classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " - "ordinary word (rules.md#M2). Before #601: given 'Jane', middle " + "ordinary word (rules/M2). Before #601: given 'Jane', middle " "'van der Berg', family 'Smith', maiden 'Jones'. As pinned " "then: the listing form, where the chain and the marker are " "both on the given side of the comma. Before #399 the marker " @@ -8047,7 +8048,7 @@ def _check_cjk_shape_purity(self) -> None: "Jane Smith (née Jones)", {"given": "Jane", "family": "Smith", "maiden": "Jones"}, classification="fix(#335)", - notes="rules.md#M3 -- the clause says 'maiden' out loud, so the " + notes="rules/M3 -- the clause says 'maiden' out loud, so the " "pair enclosing it does not have to be configured. " "maiden_marker_delimited above is the same input under " "Policy(maiden_delimiters=...) and reads identically -- " @@ -8284,7 +8285,7 @@ def _check_cjk_shape_purity(self) -> None: Case("lone_name_word_reports_the_convention", "Andrew", {"given": "Andrew"}, ambiguities=("given-or-family",), classification="feat(#449)", - notes="rules.md#O5 -- nothing decided this reading, so the " + notes="rules/O5 -- nothing decided this reading, so the " "convention picked the field and says so"), Case("lone_name_word_reports_the_convention_family_first", "Garcia", {"family": "Garcia"}, policy=Policy(name_order=FAMILY_FIRST), @@ -8294,7 +8295,7 @@ def _check_cjk_shape_purity(self) -> None: Case("lone_name_word_joined_by_a_connective_is_one_word", "Juan & Garcia", {"given": "Juan & Garcia"}, ambiguities=("given-or-family",), classification="feat(#449)", - notes="rules.md#P3 -- a joined part is one name word wherever " + notes="rules/P3 -- a joined part is one name word wherever " "another rule counts them, and O5 counts them"), Case("lone_name_word_bound_given_vocabulary_decides", "abdul", {"given": "abdul"}, classification="parity", @@ -8313,14 +8314,14 @@ def _check_cjk_shape_purity(self) -> None: Case("lone_name_word_beside_a_suffix_still_reports", "Smith Jr.", {"given": "Smith", "suffix": "Jr."}, ambiguities=("given-or-family",), classification="feat(#449)", - notes="rules.md#S2 peels the suffix and leaves ONE name word, " + notes="rules/S2 peels the suffix and leaves ONE name word, " "which O5's convention then places -- the peel decided " "the suffix, not the field"), Case("lone_name_word_beside_a_nickname_and_a_suffix_reports", "'Smitty' Jones Jr.", {"given": "Jones", "suffix": "Jr.", "nickname": "Smitty"}, ambiguities=("given-or-family",), classification="feat(#449)", - notes="rules.md#O5's own example: N3's count does not set " + notes="rules/O5's own example: N3's count does not set " "aside a suffix standing beside the nickname, so N3 " "declines and the convention is what places 'Jones'"), Case("lone_name_word_title_decides_it", "Dr. Smith", @@ -8341,14 +8342,14 @@ def _check_cjk_shape_purity(self) -> None: Case("all_titles_input_reports_the_demoted_word", "Lord Chancellor", {"title": "Lord", "family": "Chancellor"}, ambiguities=("title-or-name",), classification="feat(#491)", - notes="rules.md#H4 -- nothing but title vocabulary, so the " + notes="rules/H4 -- nothing but title vocabulary, so the " "last title word is the name by convention and says so"), Case("all_titles_input_the_queens_bench_string", "The Right Hon. the President of the Queen's Bench Division", {"title": "The Right Hon. the President of the Queen's Bench", "family": "Division"}, ambiguities=("title-or-name",), classification="feat(#491)", - notes="decisions.md#v1-xfail-triage's fourth NOT FIXED entry: " + notes="decisions/v1-xfail-triage's fourth NOT FIXED entry: " "the reading stands, and the guess stops being silent"), Case("all_titles_input_a_title_vocabulary_surname", "Dr. King", {"title": "Dr.", "family": "King"}, @@ -8370,7 +8371,7 @@ def _check_cjk_shape_purity(self) -> None: "Dr. King MD", {"title": "Dr.", "family": "King", "suffix": "MD"}, ambiguities=("title-or-name",), classification="fix(#489)", - notes="rules.md#H3's floor: everything behind the run is a " + notes="rules/H3's floor: everything behind the run is a " "suffix piece and `King` is not one, so the run gives " "it back and the credential is a credential. `king` " "being title vocabulary, the word left standing is " @@ -8380,7 +8381,7 @@ def _check_cjk_shape_purity(self) -> None: "Dr King Jr", {"title": "Dr", "family": "King", "suffix": "Jr"}, ambiguities=("title-or-name",), classification="fix(#489)", notes="v1 wanted exactly this reading and the 2026-09-01 " - "triage pinned the old one as NOT FIXED; rules.md#S2's " + "triage pinned the old one as NOT FIXED; rules/S2's " "descriptive note said a change toward S2's prediction " "would be an improvement, and this is it"), Case("title_run_floor_gives_back_a_given_name_title", @@ -8390,7 +8391,7 @@ def _check_cjk_shape_purity(self) -> None: "peeled as the suffix, no title is left to make the " "reading H1's, so the lone name word is H4's -- " "`dr` is title vocabulary and the word standing is it. " - "This shape is what rules.md#S2's descriptive note " + "This shape is what rules/S2's descriptive note " "named, in its `Sir Jr` spelling, as the reading it " "predicted and did not get; once the floor empties the " "run no branch reads `vocab:given-title` at all, so " @@ -8402,7 +8403,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Lord", "family": "Chancellor", "suffix": "Jr"}, ambiguities=("title-or-name",), classification="fix(#489)", notes="the floor takes back ONE word, the run's last, and " - "`Lord Chancellor` is the input decisions.md#H4 already " + "`Lord Chancellor` is the input decisions/H4 already " "uses for the all-titles convention"), Case("title_run_floor_keeps_a_joined_title_unit", "Prince of Wales Jr", @@ -8435,7 +8436,7 @@ def _check_cjk_shape_purity(self) -> None: "Her Majesty Queen Elizabeth", {"title": "Her Majesty Queen", "given": "Elizabeth"}, classification="fix(#489)", - notes="rules.md#H1 -- `queen` is a given-name title and it is " + notes="rules/H1 -- `queen` is a given-name title and it is " "the last word of the run, so the run addresses as it " "does; the empty family is H1's Accepted outcome"), Case("title_run_addresses_by_its_last_title_with_a_suffix", @@ -8471,7 +8472,7 @@ def _check_cjk_shape_purity(self) -> None: "Sir Sheikh abdul rahman", {"title": "Sir Sheikh", "given": "abdul rahman"}, classification="fix(#489)", - notes="rules.md#P5's licence keys the same way H1 does, the " + notes="rules/P5's licence keys the same way H1 does, the " "invariant the #369 entry set: behind a given-name " "title there is no family to spare"), Case("title_run_does_not_address_by_a_non_given_name_title", @@ -8481,7 +8482,7 @@ def _check_cjk_shape_purity(self) -> None: notes="negative control: `lord` is not a given-name title, so " "the run's last word does not address by given name"), # -- #519: prince and princess join the given-name titles. The - # criterion is rules.md#H Background's: a title that precedes and + # criterion is rules/H Background's: a title that precedes and # addresses by the GIVEN name belongs. "Prince Harry", "Princess # Anne" address by given name and no surname reading of the word # behind them exists. lord and lady stay out, decided: both split @@ -8507,7 +8508,7 @@ def _check_cjk_shape_purity(self) -> None: Case("prince_licences_the_bound_given_join", "Prince abdul Rahman", {"title": "Prince", "given": "abdul Rahman"}, classification="fix(#519)", - notes="the second site that reads the set, rules.md#P5's " + notes="the second site that reads the set, rules/P5's " "licence: behind a given-name title there is no family " "to spare, so the bound word joins forward as it does " "behind Sir. Read family 'Rahman' before #519"), @@ -8540,7 +8541,7 @@ def _check_cjk_shape_purity(self) -> None: Case("all_suffix_input_reports_suffix_or_name", "Rinpoche", {"given": "Rinpoche"}, ambiguities=("suffix-or-name",), classification="feat(#491)", - notes="rules.md#H4's suffix half: post-nominal vocabulary with " + notes="rules/H4's suffix half: post-nominal vocabulary with " "no name word beside it, read as the name"), Case("all_suffix_input_two_words", "QC MP", {"given": "QC", "suffix": "MP"}, ambiguities=("suffix-or-name",), @@ -8550,7 +8551,7 @@ def _check_cjk_shape_purity(self) -> None: Case("lone_joined_unit_carrying_title_vocabulary", "John of Prince", {"given": "John of Prince"}, ambiguities=("title-or-name",), classification="feat(#491)", - notes="rules.md#O5's exception: the one name unit is a join " + notes="rules/O5's exception: the one name unit is a join " "(P3) carrying title vocabulary, so the doubt is " "whether `Prince` is a title rather than which field " "the unit takes -- one of the inputs measured to reach " @@ -8622,7 +8623,7 @@ def _check_cjk_shape_purity(self) -> None: "is TITLES vocabulary, not just `prince`: `John and " "King`, `Smith and Bishop` and `John of Judge` report " "it too. That is the king/judge/bishop collision set " - "(decisions.md#vocabulary-collisions) met inside a " + "(decisions/vocabulary-collisions) met inside a " "join, and the report is deliberate -- a second surname " "that is also title vocabulary is the doubt the kind " "names"), @@ -8711,7 +8712,7 @@ def _check_cjk_shape_purity(self) -> None: "and lets the content read as if written bare, and " "bare 'Jane Smith née Jr.' has no name word after the " "marker for M2 to take, so 'née' stays an ordinary " - "word and lands in the family. rules.md#M3 carries the " + "word and lands in the family. rules/M3 carries the " "same input as an example line, but the runner checks " "one field per line; this row is the other three. " "Parity, and unchanged by #335 -- 1.4.0 and 2.1.0 read " @@ -8846,7 +8847,7 @@ def _check_cjk_shape_purity(self) -> None: "Jane (n\u00e9e \u200f, Jones) Smith", {"given": "Jane", "family": "Smith", "nickname": "n\u00e9e , Jones"}, classification="parity", - notes="rules.md#M3's comma clause where the clause's whitespace " + notes="rules/M3's comma clause where the clause's whitespace " "word is '\\u200f,' and its token is ',' -- tokenize drops " "the right-to-left mark under the default strip_bidi. " "Extract's M3 test and group's M1 drop ask " @@ -8869,7 +8870,7 @@ def _check_cjk_shape_purity(self) -> None: "the marker question of different things: M3 splits " "the clause on WHITESPACE, so its first word is " "'旧姓・田中' and it declines, while tokenize divides " - "at the nakaguro (rules.md#T2), so the clause's first " + "at the nakaguro (rules/T2), so the clause's first " "TOKEN is '旧姓' with vocab:maiden-marker -- exactly " "what the branch refuses to drop from a nickname. " "Measured 2026-10-12: with the branch removed this " @@ -9058,7 +9059,7 @@ def _check_cjk_shape_purity(self) -> None: # 'Ed' is Title-case in a mixed-case name, so it leans SURNAME and # the post-comma slot declines it exactly as before -- but the # decision is now reported, both directions of the fork being - # worth telling the caller about (decisions.md#S2). + # worth telling the caller about (decisions/S2). Case("comma_ambiguous_acronym", "Smith, Ed", {"given": "Ed", "family": "Smith"}, ambiguities=("suffix-or-name",)), @@ -9068,7 +9069,7 @@ def _check_cjk_shape_purity(self) -> None: # opener counted as a NAME word where a LISTED one did not, and # this pair split on that alone: 'Dr. Smith, Ed' (LISTED 'Dr.') # read family 'Dr. Smith' unflipped (the count is unchanged; since - # #701 the listed 'Dr.' then peels, rules.md#H6), but 'Xyz. Smith, Ed' + # #701 the listed 'Dr.' then peels, rules/H6), but 'Xyz. Smith, Ed' # (UNLISTED, H2-shaped) flipped the comma structure and read title # 'Xyz.', family 'Smith', suffix 'Ed' -- a different STRUCTURE for # the same shape, from a title-vocabulary difference the count had @@ -9104,7 +9105,7 @@ def _check_cjk_shape_purity(self) -> None: notes="#371's own subject: the leading pair read given " "'Van Johnson' with an EMPTY family until the merge " "learned to decline at the head of the input. A Case " - "row rather than a rules.md line because the empty " + "row rather than a rules line because the empty " "fields are the symptom -- test_case asserts the whole " "dict, so `suffix` and `middle` being empty is pinned " "here and nowhere else"), @@ -9194,7 +9195,7 @@ def _check_cjk_shape_purity(self) -> None: "'MD' after the name is the degree. Position decides, " "as for 'sr'. The shape-1 tag rides on that disposition " "-- 'md' is the ONE deviation from the approved " - "2026-07-30 audit table (decisions.md#comma-suffix-arc), " + "2026-07-30 audit table (decisions/comma-suffix-arc), " "so if the deviation is " "ever reversed this row stops being a Title Given " "Family arrangement and the tag has to move to another " @@ -9206,9 +9207,9 @@ def _check_cjk_shape_purity(self) -> None: notes="the third leg of 'position decides' for 'md'"), # -- #346: the renunciate class. A title of religious renunciation # addresses the GIVEN name AND belongs to a tradition that - # abolishes the surname, so rules.md#H1's given-name-title fold is + # abolishes the surname, so rules/H1's given-name-title fold is # the right reading and family="" is the right output. The two - # criteria are separate (rules.md#H Background) and the control + # criteria are separate (rules/H Background) and the control # row below is what keeps them separate. Case("renunciate_title_keeps_the_given_name", "Swami Vivekananda", {"title": "Swami", "given": "Vivekananda"}, @@ -9222,7 +9223,7 @@ def _check_cjk_shape_purity(self) -> None: Case("surname_retaining_title_keeps_the_family", "Rabbi Cohen", {"title": "Rabbi", "family": "Cohen"}, - notes="makes rules.md#H Background's 'Rabbi Cohen' sentence " + notes="makes rules/H Background's 'Rabbi Cohen' sentence " "executable -- no other row parses a Rabbi. 'rabbi' " "addresses by title and the tradition keeps surnames, " "so it stays out of GIVEN_NAME_TITLES and H1 families " @@ -9257,7 +9258,7 @@ def _check_cjk_shape_purity(self) -> None: "into GIVEN_NAME_TITLES is what moved 'Dutt' from " "family to given. Accepted as the pre-existing cost of " "'guru' being a title at all " - "(decisions.md#indic-honorifics); TITLES has no " + "(decisions/indic-honorifics); TITLES has no " "ambiguous subset to say otherwise"), Case("audit_do_leading_is_a_name", "Do Nguyen", {"given": "Do", "family": "Nguyen"}, @@ -9364,7 +9365,7 @@ def _check_cjk_shape_purity(self) -> None: # MOVED by #289, not deleted: 'DO' is written in capitals inside a # mixed-case name, so it now leans CREDENTIAL and the post-comma # slot takes it with one word to spare rather than falling to the - # given position (decisions.md#S2). + # given position (decisions/S2). Case("audit_bare_caps_do_after_comma_is_a_credential", "Smith, DO", {"family": "Smith", "suffix": "DO"}, classification="fix(#289)", @@ -9509,7 +9510,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "John", "family": "Smith", "suffix": "MD PhD"}, classification="fix(#436/#437)", notes="the NO-COMMA path #429 recorded as left alone, and " - "the rules.md#R1 example line. 'MD, PhD' at 1.4.0, " + "the rules/R1 example line. 'MD, PhD' at 1.4.0, " "2.0.0, 2.1.0 and 2.2.0 alike -- a comma the writer " "never typed -- because the entry boundary was derived " "from segment SHAPE and this string has no comma to " @@ -9522,7 +9523,7 @@ def _check_cjk_shape_purity(self) -> None: notes="segment 0 is the family segment even when it is wholly " "credential-shaped, so 'MD' is the FAMILY and not the " "head of a credential run, and that is what this row " - "still guards. Since #701 it guards rules.md#H6's stop " + "still guards. Since #701 it guards rules/H6's stop " "too: 'MD' is title vocabulary, and a title before the " "comma peels only in front of a family word, which 'PhD' " "is not. The SEPARATOR is no longer the row's " @@ -9597,7 +9598,7 @@ def _check_cjk_shape_purity(self) -> None: "suffix 'MD I'. Spelled with the dual 'MD' since #603: " "'Smith, PSM Dr. I', this row until then, is a part the " "credential 'PSM' OPENS, which reads as #602's run reads " - "whatever stands behind it (rules.md#C1) -- title 'Dr.', " + "whatever stands behind it (rules/C1) -- title 'Dr.', " "suffix 'PSM I' -- so the reset no longer decides it. A " "dual opens nothing, and the reset is what is left " "deciding this spelling. 1.4.0 read the " @@ -9752,7 +9753,7 @@ def _check_cjk_shape_purity(self) -> None: Case("title_word_trailing_is_a_title", "John Smith Prof.", {"title": "Prof.", "given": "John", "family": "Smith"}, classification="fix(#316)", - notes="rules.md#H5 -- after the trailing suffix run, a " + notes="rules/H5 -- after the trailing suffix run, a " "period-marked title word chains into the title from " "the end. The comma path already read it this way " "('Smith, Prof.'); the two paths agree now"), @@ -9826,7 +9827,7 @@ def _check_cjk_shape_purity(self) -> None: # MOVED by #289, not deleted: spliced, this is 'John MA' plus a # title, and 'MA' is written in capitals inside a mixed-case name # -- the caps lean now takes it with no words to spare where the - # old reserve kept it the family (decisions.md#S2). + # old reserve kept it the family (decisions/S2). Case("title_word_trailing_run_is_read_to_a_fixed_point", "John Prof. MA Prof.", {"title": "Prof. Prof.", "family": "John", "suffix": "MA"}, @@ -9845,7 +9846,7 @@ def _check_cjk_shape_purity(self) -> None: "break either way and the round decided only which " "reading it is. #289 moves it again: the caps lean " "takes 'MA' as a credential rather than the family " - "(decisions.md#S2)"), + "(decisions/S2)"), Case("title_word_trailing_reads_as_the_leading_title_does", "John Prof. MA", {"title": "Prof.", "family": "John", "suffix": "MA"}, @@ -9856,7 +9857,7 @@ def _check_cjk_shape_purity(self) -> None: "row: 'MA' is written in capitals inside a mixed-case " "name, so the caps lean takes it with no words to " "spare, so this spelling no longer shows the reserve at " - "work. The one-case row below does (decisions.md#S2)"), + "work. The one-case row below does (decisions/S2)"), Case("title_word_trailing_keeps_the_bare_acronym_reserve", "JOHN PROF. MA", {"title": "PROF.", "given": "JOHN", "family": "MA"}, @@ -9894,7 +9895,7 @@ def _check_cjk_shape_purity(self) -> None: "Mary Jane King", {"given": "Mary", "middle": "Jane", "family": "King"}, classification="parity", - notes="negative control: decisions.md's trailing-position " + notes="negative control: the decisions' trailing-position " "rule that must NOT be adopted, still not adopted"), Case("title_word_trailing_credential_is_a_suffix", "John Smith Esq.", @@ -10042,7 +10043,7 @@ def _check_cjk_shape_purity(self) -> None: "fired and read given 'abdul rahman', family 'MA'. One " "shared reading of the tail is what makes the two " "agree -- found by the /simplify round on the bundle, " - "2026-09-09 (decisions.md#H5). 1.4.0 read first 'abdul " + "2026-09-09 (decisions/H5). 1.4.0 read first 'abdul " "rahman' / middle 'MA' / last 'Prof.' (measured " "2026-09-09)"), Case("title_word_trailing_behind_a_licensed_bound_pair", @@ -10063,7 +10064,7 @@ def _check_cjk_shape_purity(self) -> None: "by neither word, so the pair became the family. H1 " "now asks the LEADING run, and 'Sir John Prof.' " "against 'Sir John' is the same movement in one word " - "(rules.md#H1, both are examples there). 1.4.0 read " + "(rules/H1, both are examples there). 1.4.0 read " "title 'Sir' / first 'abdul rahman' / last 'Prof.' " "(measured 2026-09-09)"), Case("title_word_trailing_behind_a_bound_pair_after_a_comma", @@ -10071,7 +10072,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Prof.", "given": "abdul", "family": "Berg"}, classification="fix(#637)", notes="after a family comma the reserve reads no peel " - "(rules.md#P5), but the join declines a trailing title " + "(rules/P5), but the join declines a trailing title " "word the H5 run takes, so the title is read as the " "segment with an ordinary given name reads it ('Smith, " "John Prof.'), and as the comma-less 'Sir abdul rahman " @@ -10085,8 +10086,8 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("comma-structure",), classification="fix(#640)", notes="a part past the second comma holds no name word, so " - "every title word in it is a title (rules.md#C2) and the " - "bound join declines one (rules.md#P5, #640), as a " + "every title word in it is a title (rules/C2) and the " + "bound join declines one (rules/P5, #640), as a " "family comma's tail reads since #603 ('Smith, John, " "abdul Dr. Smith'); 1.4.0 and 2.3.0 read suffix 'Jr., " "abdul Dr. Smith'"), @@ -10115,7 +10116,7 @@ def _check_cjk_shape_purity(self) -> None: "Sir John Prof.", {"title": "Sir Prof.", "given": "John"}, classification="fix(#316)", - notes="rules.md#H1's clause in one word: the run BEFORE the " + notes="rules/H1's clause in one word: the run BEFORE the " "one name word addresses, so this is 'Sir John' plus a " "title and the given name survives the title being " "added. Keying every TITLE token as one run gave 'sir " @@ -10133,7 +10134,7 @@ def _check_cjk_shape_purity(self) -> None: "'dr sir' matched on 'sir' and read given 'Smith'. " "Where NO run stands in front the trailing one does " "decide -- 'Smith Sir.' reads given 'Smith', a " - "rules.md#H1 example unchanged by this. 1.4.0 read " + "rules/H1 example unchanged by this. 1.4.0 read " "title 'Dr.' / first 'Smith' / last 'Sir.' (measured " "2026-09-09)"), Case("title_word_trailing_after_a_maiden_take", @@ -10141,7 +10142,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Prof.", "given": "Mary", "family": "Smith", "maiden": "Jones"}, classification="fix(#535)", - notes="rules.md#H5's M2 boundary, removed by #535: the maiden " + notes="rules/H5's M2 boundary, removed by #535: the maiden " "walk now reads the trailing title chain, so the title " "ends the clause and reads as 'Mary Smith Prof.' would " "read it. 2.3.0 read maiden 'Jones Prof.'; 1.4.0 had no " @@ -10201,7 +10202,7 @@ def _check_cjk_shape_purity(self) -> None: {"title": "Freiherr", "family": "von Berg X.Y.Z."}, ambiguities=("particle-or-given", "suffix-or-name"), classification="parity", - notes="rules.md#S2 counts the particle run and the word the " + notes="rules/S2 counts the particle run and the word the " "chain takes with it as one name word (#614), so the " "dotted acronym has no word to spare, as every release " "read it; counted word by word it was the credential, " @@ -10244,7 +10245,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("particle-or-given", "suffix-or-name"), classification="fix(#624)", notes="of two title-particles at the head the second leads the " - "chain (rules.md#P4), so both are titles and 'van Berg' " + "chain (rules/P4), so both are titles and 'van Berg' " "is the family, as 'Dr. St van Johnson' reads; the chain " "and the read's count take that one answer (1.4.0 given " "'St van Berg', family 'MA'; 2.0-2.3 family 'St van Berg " @@ -10267,7 +10268,7 @@ def _check_cjk_shape_purity(self) -> None: "suffix": "MD van Jones"}, ambiguities=("suffix-or-name", "suffix-or-name"), classification="fix(#602)", - notes="rules.md#S2 (#602): inside the run a title word reads as " + notes="rules/S2 (#602): inside the run a title word reads as " "a title. #614's read merged 'van Secretary' into one " "piece and the title went into the suffix (its second " "review); since #620 the read counts the chain's units " @@ -10281,7 +10282,7 @@ def _check_cjk_shape_purity(self) -> None: "family": "de la Berg"}, classification="parity", notes="each particle run joins the words after it, the final " - "group the family (rules.md#P2); group merges both runs " + "group the family (rules/P2); group merges both runs " "ahead of S2's read (#625), whose bookkeeping across the " "first merge no other row reached (#625's review: an " "off-by-one there read family 'van der Smith de la " @@ -10292,7 +10293,7 @@ def _check_cjk_shape_purity(self) -> None: "family": "van mc"}, classification="parity", notes="'van mc' is one particle run, the dual a particle the " - "run took and no suffix (rules.md#P2, #S2); merged " + "run took and no suffix (rules/P2, #S2); merged " "ahead of the read after a first run (#625's review: an " "off-by-one left 'mc' a suffix)"), Case("a_second_particle_run_is_a_unit_of_its_own", @@ -10597,7 +10598,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("particle-or-given", "title-or-name"), classification="fix(#614)", notes="garbage in: the trailing 'Prof.' is read before the " - "chain (rules.md#H5), the chain then joins a piece " + "chain (rules/H5), the chain then joins a piece " "carrying the joined 'Dr. and Ed.' title, which reads as " "a leading title, and assign reads the name again and " "keeps 'Prof.' as its word; 0c54662b chained the whole " @@ -10609,7 +10610,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the trailing slot is a segment away: a second comma " "makes the last part its own segment, which the tail " "reads before any trailing walk reads a piece. Since " - "#603 a title word there is a title (rules.md#C2), so " + "#603 a title word there is a title (rules/C2), so " "'Smith, John, Prof.' reads title 'Prof.' as 'Smith, " "John Prof.' does, and the part is recognized rather " "than flagged. Until then, and in 1.4.0, it read suffix " @@ -10652,7 +10653,7 @@ def _check_cjk_shape_purity(self) -> None: Case("ko_family_comma_given_side_stays_whole", "지훈, 남궁민수", {"family": "지훈", "given": "남궁민수"}, notes="the mirror of the row above, and one of " - "rules.md#W3's comma illustrations: with the " + "rules/W3's comma illustrations: with the " "comma naming 지훈 the family, the post-comma side is " "given text with no family to find, so 남궁민수 is " "not segmented THERE either -- the same inertness " @@ -10708,7 +10709,7 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), classification="feat(#491)", notes="core-only for the reason given on the Han row above. " "The suffix half's positive control, and what makes " - "rules.md#H4's CJK-honorific silence a `by_script` " + "rules/H4's CJK-honorific silence a `by_script` " "scope rather than a claim about the word: empty the " "script table and the same lone honorific reaches the " "bare-suffix carve-out and reports"), @@ -10865,7 +10866,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "ﾔﾏﾀﾞ", "family": "ﾀﾛｳ"}, notes="parity row guarding the license's boundary in " "halfwidth: wholly-katakana keeps the declared order " - "(rules.md#W4). Legacy halfwidth data writes Japanese " + "(rules/W4). Legacy halfwidth data writes Japanese " "names this way too, family-first, but the script " "cannot tell a reading from a transcription, so the " "caller opts in through script_orders (#594)"), @@ -10873,7 +10874,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "ﾀﾛｳ", "family": "ﾔﾏﾀﾞ"}, classification="fix(#594)", notes="the 间隔号 divides between two classified characters " - "(rules.md#T3), and halfwidth kana is classified now, " + "(rules/T3), and halfwidth kana is classified now, " "so the dot divides here as it does in タロウ·ヤマダ " "and the parts keep source order. 2.3.0 kept the whole " "text one token"), @@ -10885,7 +10886,7 @@ def _check_cjk_shape_purity(self) -> None: "combining dakuten U+3099 standing, and the mark sits in " "the HIRAGANA block. A voicing mark takes the script of " "the kana before it, so the name is wholly katakana and " - "keeps the declared order (rules.md#W4). 2.1.0 through " + "keeps the declared order (rules/W4). 2.1.0 through " "2.3.0 read the mark as hiragana and the token as " "kana-licensed Japanese: family ア゙イ"), Case("ja_combining_voicing_mark_beside_plain_katakana", @@ -10929,7 +10930,7 @@ def _check_cjk_shape_purity(self) -> None: "the token then classified as hangul, the surname site " "divided the RAW text after 김, and the stranded mark " "opened the family: given 김, family \u3099민준. " - "Tolerated: rules.md's W Background leaves a mark after " + "Tolerated: rules/W's Background leaves a mark after " "a non-kana unpromised, so this reading is watched by " "the snapshot rather than pinned as contract", tolerated=True), @@ -10987,7 +10988,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the SAME transcription typed with the Japanese " "nakaguro reads as the dot's own typography says -- a " "姓・名 roster pair, family-first (#272) -- because the " - "nakaguro records nothing (per decisions.md#T3, " + "nakaguro records nothing (per decisions/T3, " "codepoint-scoped; only the Chinese B7 marks a " "transcription). A limitation row: chosen, not " "accidental -- cross-convention input reads by the " @@ -11324,7 +11325,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the lookup fold composes NFC, so decomposed 씨. reaches " "the suffix entry (2.2.0 read title 씨.). The family " "stays WHOLE and stays NFD: segmentation matches raw " - "text on purpose (decisions.md#W1, 2026-07-29 ja " + "text on purpose (decisions/W1, 2026-07-29 ja " "amendment), so NFD degrades to no-split, never to a " "wrong split, and no " "token text is rewritten. Contrast 'NFD(田中さん, 様.)', " @@ -11356,7 +11357,7 @@ def _check_cjk_shape_purity(self) -> None: "which _is_post_nominal's docstring says the surname " "site reads as an ANSWER rather than a token to step " "past -- and declines, leaving 지훈 whole. The stop " - "stays on the token: nothing rewrites text (rules.md T " + "stays on the token: nothing rewrites text (rules/T's " "Background). 2.2.0 read given 양., middle 지, family " "훈.", tolerated=True), @@ -11382,7 +11383,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the ORDER reader of the same None: assign's script " "walk returns the declared order the moment one piece " "has no script, so a stop on the LAST token flipped " - "the name to given-first. rules.md#W4's family-first " + "the name to given-first. rules/W4's family-first " "reading survives the stop now. 2.2.0 read given 양, " "family 지훈..", tolerated=True), @@ -11436,7 +11437,7 @@ def _check_cjk_shape_purity(self) -> None: classification="fix(#323)", notes="the H2 veto's witness: Han has no default segmentation, " "so the period-marked word reaches assign whole, and " - "rules.md#H2's shape -- a Latin convention, an " + "rules/H2's shape -- a Latin convention, an " "abbreviation's period -- declines it because Han has " "no period abbreviations (the #320 veto, extended from " "is_initial to H2). A lone name word is the family " @@ -11449,7 +11450,7 @@ def _check_cjk_shape_purity(self) -> None: notes="the two-word Han shape, where the veto is the whole " "difference: without it H2 fires on 田中. and 太郎 " "becomes the entire name. With it the pair reads as " - "'田中 太郎' does, family-first by script (rules.md#W4), " + "'田中 太郎' does, family-first by script (rules/W4), " "with the stop kept. The stop stays on 田中, the word " "that carried it: family 田中., given 太郎. 2.2.0 read " "title 田中., family 太郎. Stays tolerated like its " @@ -11464,7 +11465,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "マイケル."}, ambiguities=("given-or-family",), classification="fix(#323)", notes="the katakana arm of the H2 veto, and the one where it " - "cuts across rules.md#W4 -- a wholly-katakana name keeps " + "cuts across rules/W4 -- a wholly-katakana name keeps " "the declared order, so the lone word is the GIVEN name " "-- the veto decides title-or-name, the order rule " "decides which name. 2.2.0 read title マイケル.", @@ -11477,13 +11478,13 @@ def _check_cjk_shape_purity(self) -> None: "initials (_NO_INITIALS), and halfwidth kana is katakana " "now, so H2's opening-abbreviation shape declines it. " "2.3.0 read title ﾏｲｹﾙ., the halfwidth limit " - "decisions.md#cjk-full-stops recorded", + "decisions/cjk-full-stops recorded", tolerated=True), Case("ja_halfwidth_katakana_opener_with_a_period_is_a_name", "ﾀﾅｶ. John", {"given": "ﾀﾅｶ.", "family": "John"}, classification="fix(#594)", - notes="decisions.md#cjk-full-stops' recorded misroute: a " + notes="decisions/cjk-full-stops' recorded misroute: a " "period-marked halfwidth opener read as a TITLE, where " "タナカ. John reads given. Now both read given", tolerated=True), @@ -11511,7 +11512,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "김", "middle": "민준. John", "family": "Smith"}, classification="fix(#323)", notes="a recorded DEGRADATION, pinned so it cannot move " - "silently. rules.md#S1's bracketed-credential escape in " + "silently. rules/S1's bracketed-credential escape in " "_extract calls a clause suffix-shaped when it ends in " "an ASCII period -- an unwidened test, deliberately -- " "so the brackets are dropped and the content reads as " @@ -11529,7 +11530,7 @@ def _check_cjk_shape_purity(self) -> None: "front rather than the position. Latin wrapped around a " "CJK name is the 2026-09-01 demotion's own ground, so " "the row is tolerated and the degradation is recorded " - "in decisions.md#cjk-full-stops rather than fixed here", + "in decisions/cjk-full-stops rather than fixed here", tolerated=True), Case("latin_suffix_with_an_ideographic_full_stop", "John Smith, Jr。", {"given": "John", "family": "Smith", "suffix": "Jr。"}, @@ -11561,7 +11562,7 @@ def _check_cjk_shape_purity(self) -> None: "unfoldable then. Neither reading was designed; the " "bundle widened the stop set and this is where the " "widening lands on Latin text, which " - "decisions.md#cjk-full-stops records as an unasked-for " + "decisions/cjk-full-stops records as an unasked-for " "reach rather than a promise"), Case("ko_honorific_glued_teacher", "김선생님", {"family": "김", "suffix": "선생님"}, diff --git a/tests/v2/design_docs.py b/tests/v2/design_docs.py new file mode 100644 index 00000000..92f47bc0 --- /dev/null +++ b/tests/v2/design_docs.py @@ -0,0 +1,107 @@ +"""The folder layout of docs/design/rules/ and docs/design/decisions/. + +The one place that knows it: every other reader of the design docs goes +through these functions, so the parsers keep reading the text they +always read, and a change of layout is a change here. + +Each folder holds a README.md whose Markdown links, in order, are the +index: one link per file, its text `` — ``. A rule, a +section and a rule-keyed decisions entry are filed as +``-.md``; a decisions entry keyed by a slug is +filed under the slug alone, which already is a name. A citation names +the key only (``rules/S2``, ``decisions/vocabulary-collisions``), so +renaming a file never breaks one. +""" +from __future__ import annotations + +import re +import unicodedata +from dataclasses import dataclass +from pathlib import Path + +DESIGN_DIR = Path(__file__).resolve().parents[2] / "docs" / "design" +RULES_DIR = DESIGN_DIR / "rules" +DECISIONS_DIR = DESIGN_DIR / "decisions" +MECHANISMS_DOC = DESIGN_DIR / "mechanisms.md" +#: rules order, the order the decisions index groups its entries in +SECTION_ORDER = "HPSNMCOWTARD" + +# A citation: the folder and the key, or mechanisms.md and its slug. A +# key is not followed by a word character, nor by a hyphen and another +# word, which is how a file path in the rules folder (a stem like +# S2-case-signal-at-the-suffix-slot) is told from a citation, and a `.md` +# after it marks a file name (the README); a sentence's full stop is +# allowed, and so is a hyphen ending the line, where a comment wraps a +# long key. A decisions key starting with a +# digit (`3-0-reevaluations`) is not matched, as it was not before the +# split. +MENTION_RE = re.compile( + r"(?:(?Prules|decisions)/|(?Pmechanisms)\.md#)" + r"(?P[A-Za-z][A-Za-z0-9_]*(?:-[A-Za-z0-9_]+)*)" + r"(?![A-Za-z0-9_]|-[A-Za-z0-9_])(?!\.md\b)") + +_LINK_RE = re.compile(r"\[(?P[^\]]+)\]\((?P<file>[^)/]+\.md)\)") +_KEY_RE = re.compile(r"[^\s—,]+") +_SECTION_RE = re.compile(r"\(([A-Z])\)$") +_RULE_KEY_RE = re.compile(r"[A-Z]\d*") +_ARTICLE_RE = re.compile(r"^(?:a|an|the)\s+", re.IGNORECASE) + + +@dataclass(frozen=True) +class IndexEntry: + key: str + title: str # the link text, "S2 — the case signal at the suffix slot" + path: Path + + +def slugify(name: str) -> str: + """A name as a file-name slug: `&` read as "and", one leading + article dropped, an apostrophe dropped ("run's" is "runs"), folded + to lowercase ASCII, every run of other characters one hyphen.""" + name = _ARTICLE_RE.sub("", name.replace("&", " and ").strip(' "\'')) + name = re.sub(r"(?<=\w)['’](?=\w)", "", name) + ascii_name = (unicodedata.normalize("NFKD", name) + .encode("ascii", "ignore").decode()) + return re.sub(r"[^a-z0-9]+", "-", ascii_name.lower()).strip("-") + + +def stem_for(key: str, name: str) -> str: + """The file stem for an entry: a rule or section key with its + name's slug, any other key alone.""" + if _RULE_KEY_RE.fullmatch(key): + return f"{key}-{slugify(name)}" + return key + + +def read_index(folder: Path) -> list[IndexEntry]: + text = (folder / "README.md").read_text(encoding="utf-8") + entries = [] + for m in _LINK_RE.finditer(text): + # a section is linked by its heading, "Suffixes: ... (S)" + km = (_SECTION_RE.search(m.group("title")) + or _KEY_RE.match(m.group("title"))) + assert km, f"{folder.name}/README.md: link without a key: {m[0]}" + entries.append(IndexEntry(km.group(km.lastindex or 0), + m.group("title"), + folder / m.group("file"))) + return entries + + +def read_rules_text(folder: Path = RULES_DIR) -> str: + """Every section and rule file, in index order, as one text.""" + return "\n".join(e.path.read_text(encoding="utf-8") + for e in read_index(folder)) + + +def read_decision_files(folder: Path = DECISIONS_DIR) -> dict[str, str]: + return {e.key: e.path.read_text(encoding="utf-8") + for e in read_index(folder)} + + +def resolve(doc: str, cid: str, folder: Path | None = None) -> Path | None: + """The one file a citation names, or None when none or several do.""" + if folder is None: + folder = RULES_DIR if doc == "rules" else DECISIONS_DIR + hits = [p for p in folder.glob("*.md") if p.name != "README.md" + and (p.stem == cid or p.stem.startswith(cid + "-"))] + return hits[0] if len(hits) == 1 else None diff --git a/tests/v2/pipeline/test_assemble.py b/tests/v2/pipeline/test_assemble.py index 79efbad2..ee2a7168 100644 --- a/tests/v2/pipeline/test_assemble.py +++ b/tests/v2/pipeline/test_assemble.py @@ -135,7 +135,7 @@ def test_ambiguity_with_all_indices_dropped_is_omitted() -> None: def test_a_connective_or_initial_report_is_withdrawn_where_the_role_is_a_suffix( ) -> None: - """rules.md#A1, the withdrawal (#397 second review). + """rules/A1, the withdrawal (#397 second review). classify's connective-or-initial fork offers a connective and an initial and its detail says which it took. A generation is diff --git a/tests/v2/pipeline/test_assign.py b/tests/v2/pipeline/test_assign.py index d2ed35ad..7f13abfd 100644 --- a/tests/v2/pipeline/test_assign.py +++ b/tests/v2/pipeline/test_assign.py @@ -400,7 +400,7 @@ def test_the_trailing_title_is_transparent_to_the_suffix_peel() -> None: # 'Jack MA' shape -- an ALL-CAPS ambiguous acronym in a mixed-case # name leans credential and is taken with no words to spare, so # the reserve that used to keep it the family now takes it as the - # suffix instead (decisions.md#S2). + # suffix instead (decisions/S2). out = _assigned("John Mr. MA", lexicon=lex) assert _by_role(out, Role.TITLE) == "Mr." assert _by_role(out, Role.GIVEN) == "John" @@ -840,7 +840,7 @@ def test_the_trailing_given_slot_ignores_the_two_segment_floor() -> None: credential reading likelier rather than less.""" out = _assigned("Doe, John MA, PhD", lexicon=Lexicon.default()) # _by_role space-joins by role; the written comma is preserved by - # post_rules' entry pass (#436/#437, rules.md#R1), a later stage + # post_rules' entry pass (#436/#437, rules/R1), a later stage # this assign-only helper does not run -- the full-pipeline case # row `the_trailing_slot_survives_a_third_comma_part` pins the # comma-rendered "MA, PhD". What this unit test checks is the role diff --git a/tests/v2/pipeline/test_classify.py b/tests/v2/pipeline/test_classify.py index 237b138a..fc10c6e9 100644 --- a/tests/v2/pipeline/test_classify.py +++ b/tests/v2/pipeline/test_classify.py @@ -153,7 +153,7 @@ def test_bare_ambiguous_acronym_in_acronyms_is_not_suffix() -> None: def test_one_case_subset_member_is_an_initial_and_reports() -> None: - # rules.md#P3 says a one-case name carries no case evidence, so + # rules/P3 says a one-case name carries no case evidence, so # the vocabulary decides, and 'e' is marked as reading both ways. # (No colon after the rule id on purpose: tests/ is swept by # test_doc_citations.py, and the colon form must quote the rule @@ -219,7 +219,7 @@ def test_the_catalan_link_reads_as_an_initial_in_a_one_case_name( # #397's letter through #527's fork, and the reason it ships # marked: a bare I initial is as common as a bare E, so a name # with no case evidence reads the letter as an initial and says - # so. rules.md#P3 says the marked set is where that answer lives. + # so. rules/P3 says the marked set is where that answer lives. for text in ("josep carod i rovira", "JOSEP CAROD I ROVIRA"): out = _classified(text) letter = "i" if text.islower() else "I" @@ -257,7 +257,7 @@ def test_removing_the_link_from_the_subset_restores_joining() -> None: def test_an_orphan_marker_is_inert_even_in_the_emitter() -> None: # 'e' left in conjunctions_ambiguous but removed from conjunctions - # -- legal since the pair is not in _SUBSET_FIELDS (decisions.md#P3) + # -- legal since the pair is not in _SUBSET_FIELDS (decisions/P3) # -- never takes the fork, so 'E' in an all-upper name falls to the # else branch and is tagged "initial" by is_initial() alone, same as # any bare capital. Without the emitter's base-vocabulary gate this @@ -303,7 +303,7 @@ def test_a_dotted_letter_is_an_initial_by_shape_in_any_case_class() -> None: def test_a_maiden_clause_does_not_count_toward_the_case_class() -> None: - # rules.md#P3 says a maiden marker, taken as one, and the words it + # rules/P3 says a maiden marker, taken as one, and the words it # takes, are not among the name's own words -- so appending # ' née Jones' must not flip 'Y' back to an initial: the name's own # words ("JUAN GARCIA Y LOPEZ") are still all-upper on their own, @@ -373,7 +373,7 @@ def test_a_clauses_own_marker_word_does_not_truncate_the_own_words() -> None: # must not move clause_at. Before the fix, "NEE"'s match truncated # "own" at its own index, excluding the trailing "GARCIA Y LOPEZ" # from the case class and misreading 'Y' as an initial instead of - # joining the family (rules.md#P4). + # joining the family (rules/P4). out = _classified("JUAN (NEE JONES) GARCIA Y LOPEZ") nee = next(t for t in out.tokens if t.text == "NEE") # the parenthesised clause opens as Role.NICKNAME (parens are a @@ -454,7 +454,7 @@ def test_classify_records_the_one_case_fact_on_the_state() -> None: # a caseless script has only one case, and answers True harmlessly assert _classified("毛泽东").one_case is True # the span is the name's OWN words: a maiden clause beside a - # one-case name does not make it mixed (rules.md#P3) + # one-case name does not make it mixed (rules/P3) assert _classified("JUAN GARCIA Y LOPEZ née Jones").one_case is True @@ -751,7 +751,7 @@ def _surname_unit_sweep() -> list[str]: for w in getattr(lex, field): if " " not in w: words |= {w, w.title(), w.upper()} - # rules.md#P7's spelling: a one-letter particle written with its + # rules/P7's spelling: a one-letter particle written with its # period is an initial, and both copies must say so (#604) words |= {w + "." for w in set(words) if len(w) == 1 and w.lower() in lex.particles} @@ -782,7 +782,7 @@ def _surname_unit_disagreements() -> list[str]: def test_surname_unit_tags_agree_with_classify() -> None: - # #575: rules.md#C1's count before the comma reads two facts per + # #575: rules/C1's count before the comma reads two facts per # token -- particle, suffix -- and `segment` must build them from # the vocabulary (`_vocab.surname_unit_tags`), since it runs before # classify has tagged anything, while assign derives them from @@ -809,7 +809,7 @@ def test_the_surname_unit_agreement_test_sees_the_initial_veto( # The same guard over the other half of the mirror: with # `surname_unit_tags`' one-letter-initial veto switched off, the # period-written particle reads as a particle there and not in - # classify (#604, rules.md#P7). + # classify (#604, rules/P7). monkeypatch.setattr(_vocab, "_spells_an_initial", lambda n, text: False) assert _surname_unit_disagreements() == ["Ó.", "ó."] diff --git a/tests/v2/pipeline/test_comma.py b/tests/v2/pipeline/test_comma.py index 58f3de02..1c140344 100644 --- a/tests/v2/pipeline/test_comma.py +++ b/tests/v2/pipeline/test_comma.py @@ -1,4 +1,4 @@ -"""rules.md#C1's decision, made at the head of group once classify has +"""rules/C1's decision, made at the head of group once classify has tagged the words (#613): `_comma.decide`. These tests were segment's until the decision moved; they run the stages up to classify and then the decision, as group does.""" @@ -79,11 +79,11 @@ def test_the_credential_pair_merges_anywhere_in_the_run() -> None: def test_structure_flips_for_the_ambiguous_class_on_a_name_word_count() -> None: - # rules.md#C1 for the ambiguous class: two or more NAME words + # rules/C1 for the ambiguous class: two or more NAME words # before the comma read the part after it as the credential run. # 'John Smith, MA' flips; 'Smith Jr., MA' does not, having two # TOKENS and one name word -- which is what keeps Smith in the - # family (#289, decisions.md#S2). + # family (#289, decisions/S2). assert _decided("John Smith, MA").structure is Structure.SUFFIX_COMMA assert _decided("Smith Jr., MA").structure is Structure.FAMILY_COMMA assert _decided("Smith, MA").structure is Structure.FAMILY_COMMA @@ -114,7 +114,7 @@ def test_structure_flips_for_a_by_shape_member_too() -> None: def test_paired_initials_need_a_word_to_speak_for_them() -> None: - # rules.md#C1 (#563): two dotted single letters are how a person's + # rules/C1 (#563): two dotted single letters are how a person's # own initials are written, and two words before the comma may be # one surname, so alone they keep the family comma. Each contrast # pair differs in the one thing that speaks: a third letter, a @@ -135,7 +135,7 @@ def test_paired_initials_need_a_word_to_speak_for_them() -> None: def test_a_flip_no_listed_member_takes_part_in_is_silent() -> None: - # rules.md#C1 (#563): a flip in which no listed member takes part + # rules/C1 (#563): a flip in which no listed member takes part # is silent, while a LISTED member in the same position still # reports the flip -- the control that keeps the silence from # being a lost emitter -- and so do paired initials spoken for @@ -184,7 +184,7 @@ def test_the_structure_flip_report_counts_the_real_pre_comma_words() -> None: def test_the_name_word_count_reads_a_run_as_it_reads_one_word() -> None: - # rules.md#C1 (#544): a part of two or more words, every one of + # rules/C1 (#544): a part of two or more words, every one of # them suffix vocabulary or a class member and at least one a # member, reads by the same NAME-word count as the single token, # and the flip reports once, over the whole part. @@ -237,7 +237,7 @@ def test_the_run_test_reads_a_delimiter_core_transparently() -> None: def test_the_run_test_reads_lenient_comma_suffixes() -> None: - # rules.md#C1's leftovers ask `is_wholly_suffix`'s own POLICY- + # rules/C1's leftovers ask `is_wholly_suffix`'s own POLICY- # selected predicate (#544): an # initial-shaped suffix WORD ('B.') that is not a single-letter # roman numeral is a member of neither the ambiguous class nor @@ -354,7 +354,7 @@ def test_an_empty_part_before_the_comma_still_reads_the_part_after_it() -> None: def test_a_bound_caps_word_keeps_its_capitals_in_case_repair() -> None: # decide writes the caps shape's marks on a word it binds (#564, - # rules.md#R4), so case repair keeps the capitals -- also where the + # rules/R4), so case repair keeps the capitals -- also where the # run test declined and #603's opener bound the part anyway; the # marks had been cleared there, and 'XYZ' repaired to 'Xyz' # (#613's PR review). A name in one case carries no contrast. diff --git a/tests/v2/pipeline/test_group.py b/tests/v2/pipeline/test_group.py index c387b5c7..490d7dfa 100644 --- a/tests/v2/pipeline/test_group.py +++ b/tests/v2/pipeline/test_group.py @@ -462,7 +462,7 @@ def test_where_the_marker_lands_when_the_consumer_declines() -> None: # any other to the bound-given join (P5 declines a marker "standing # as a word of its own", and this one is not) -- but since #602 the # 'Jr' behind it starts a credential run to the end of the part - # (rules.md#S2), so the reserve finds no name word to spare and the + # (rules/S2), so the reserve finds no name word to spare and the # join declines for that reason instead assert _piece_texts(_grouped("abdul van der Berg née Jr Jones")) == [ ["abdul", "van der Berg née", "Jr", "Jones"]] @@ -514,7 +514,7 @@ def test_a_delimiter_core_in_a_suffix_tail_is_not_maiden_text() -> None: def test_a_marker_in_a_suffix_part_is_a_word() -> None: # A marker counts only in the part holding the family name - # (rules.md#M2, #601), so in a tail part it takes nothing, core or + # (rules/M2, #601), so in a tail part it takes nothing, core or # no core. This was the #549 test that a trailing core is cut # before the walk, and the walk no longer runs in a tail part. out = _grouped("Smith, John, PhD née Jones V -", policy=_DASH) @@ -602,7 +602,7 @@ def test_the_bound_given_join_never_absorbs_a_marker() -> None: def test_a_given_name_title_licenses_the_bound_join_with_one_word_to_spare() -> None: - # rules.md#P5 (#369): a given-name title asserts that a given name + # rules/P5 (#369): a given-name title asserts that a given name # follows -- the same assertion H1 reads when it keeps "Sir John" a # given name -- so behind one the reserve is the post-comma one, # and two name words join where STRICT keeps the second back as @@ -646,7 +646,7 @@ def test_the_licence_joins_a_word_not_a_particle_chain() -> None: # it is the family name P2 built -- so behind a given-name title # the bound word must not absorb it: untitled, "abdul van der # Berg" keeps the chain as its family, and the title cannot make - # the surname vanish. Found in review; the rules.md example carries + # the surname vanish. Found in review; the rules example carries # the shape into the rules corpus, so the snapshot watches it too. out = _grouped("sir abdul van der Berg", lexicon=_GIVEN_NAME_TITLE_LEX) assert _piece_texts(out) == [["sir", "abdul", "van der Berg"]] @@ -686,7 +686,7 @@ def test_the_licence_still_declines_a_lone_marker() -> None: def test_the_reserve_counts_a_trailing_numeral_as_the_suffix_assign_reads() -> None: - # rules.md#P5 says the reserve "needs a name word to spare". A bare + # rules/P5 says the reserve "needs a name word to spare". A bare # 'V' carries both vocab:suffix and the initial tag, so the # suffix-piece test vetoes it -- right for assign's middle-initial # question, wrong here: assign reads a FINAL roman numeral after a @@ -725,7 +725,7 @@ def test_the_reserve_counts_the_names_the_read_left_before_a_numeral() -> None: # Since #614 the reserve reads no view: S2's run is read once # before the join, and here it takes nothing -- the initial-shaped # 'J.' suppresses the fork -- so three name words stand and the - # join leaves two, which the reserve allows (decisions.md#P5, + # join leaves two, which the reserve allows (decisions/P5, # 2026-10-07). The join no longer re-reads the fork it would move. out = _grouped("abdul J. V") assert _piece_texts(out) == [["abdul J.", "V"]] @@ -757,7 +757,7 @@ def test_the_numeral_fork_is_not_mirrored_after_a_family_comma() -> None: def test_the_join_never_absorbs_a_suffix_piece() -> None: - # rules.md#P5 says the join takes "the word after it"; a suffix is + # rules/P5 says the join takes "the word after it"; a suffix is # not a name word -- the same decline the join already makes for a # marker. The reserve counted only non-suffix pieces, so 'abdul jr # Jones' declined, but with a word to spare the join took the @@ -780,7 +780,7 @@ def test_the_join_never_absorbs_a_split_credential() -> None: def test_the_join_declines_a_suffix_after_a_family_comma_too() -> None: - # The post-comma LENIENT reserve is the path decisions.md#P5 first + # The post-comma LENIENT reserve is the path decisions/P5 first # recorded the absorb-a-suffix shape on ('Berg, abdul jr Smith' # read given 'abdul jr'); the decline is the join's, so it holds # under every reserve. @@ -823,7 +823,7 @@ def test_the_join_never_turns_a_suffix_into_a_name() -> None: # MOVED by #289, not deleted: 'Ma' now leans SURNAME (Title case, # mixed-case name) on BOTH sides of the join, so the two views' # suffix readings still agree and the join stands -- 'abdul Smith' - # given, 'Ma' family (decisions.md#S2's corpus row). + # given, 'Ma' family (decisions/S2's corpus row). out = _grouped("abdul Smith Ma", lexicon=_AMBIGUOUS_LEX) assert _piece_texts(out) == [["abdul Smith", "Ma"]] out = _grouped("abdul Smith Berg Ma", lexicon=_AMBIGUOUS_LEX) @@ -878,7 +878,7 @@ def test_the_licence_joins_where_the_read_took_nothing() -> None: # there is no joined view to compare: the run, read once before # the join, takes nothing, and the join stands, as for 'abdul J. # V' (test_the_reserve_counts_the_names_the_read_left_before_a_ - # numeral, decisions.md#P5 2026-10-07). + # numeral, decisions/P5 2026-10-07). out = _grouped("sir abdul J. V", lexicon=_GIVEN_NAME_TITLE_LEX) assert _piece_texts(out) == [["sir", "abdul J.", "V"]] @@ -921,7 +921,7 @@ def test_the_chain_keeps_an_acronym_assign_will_not_peel() -> None: # as a credential); now 'Ma' leans SURNAME (Title case, mixed-case # name) and the peel declines it regardless of the count, so the # second re-ask absorbs it into the particle run too - # (decisions.md#S2). + # (decisions/S2). out = _grouped("St John van Berg Ma", lexicon=lex) assert _piece_texts(out) == [["St", "John", "van Berg Ma"]] @@ -945,7 +945,7 @@ def test_the_chain_stops_before_a_bare_acronym_with_words_to_spare() -> None: # MOVED by #289, not deleted: 'Ma' is Title-case in a mixed-case # name, so it now leans SURNAME and the chain's re-ask no longer # stops before it -- 'Ma' joins the particle run instead - # (decisions.md#S2's corpus row). + # (decisions/S2's corpus row). out = _grouped("John van der Berg Ma", lexicon=_AMBIGUOUS_LEX) assert _piece_texts(out) == [["John", "van der Berg Ma"]] @@ -1394,7 +1394,7 @@ def test_the_marker_placements_reach_both_answers() -> None: def test_a_connective_piece_counts_toward_the_carve_outs_total() -> None: - # rules.md#P3's count (#397). Four words, one of them the link, + # rules/P3's count (#397). Four words, one of them the link, # and the link is suffix vocabulary -- so the total reaches four # only because a connective counts ITSELF. Without the count arm # the total is three, the carve-out declines, and the link stays @@ -1438,7 +1438,7 @@ def test_the_right_hand_test_reads_the_peel_not_the_suffix_piece( ) -> None: # the arm that forces trailing_start rather than is_suffix_piece: # a ONE-CHARACTER suffix word is initial-shaped, so the - # suffix-piece test refuses it (rules.md#S2's initial veto) while + # suffix-piece test refuses it (rules/S2's initial veto) while # assign's trailing peel takes it. Spelled with the piece test # alone, the link would swallow the generation here. lex = _LINK_LEX.add(suffix_words={"v"}) @@ -1468,7 +1468,7 @@ def test_a_leading_title_on_the_left_is_no_name_word() -> None: def test_an_unlisted_leading_abbreviation_is_no_name_word_either( ) -> None: # and THIS is the `lo` bound's own row, the listed spelling above - # being caught by the title-piece test as well. rules.md#H2 reads + # being caught by the title-piece test as well. rules/H2 reads # an unlisted abbreviation opening a name as a title by SHAPE, so # the vocabulary says nothing about 'Xyz.' and only the bound # assign's own title run draws keeps the link from joining it. @@ -1613,7 +1613,7 @@ def test_a_one_case_letter_is_an_initial_and_never_counts() -> None: def test_the_class_reaches_a_callers_own_connective() -> None: - # rules.md#P3 is keyed on the class throughout: a caller who adds + # rules/P3 is keyed on the class throughout: a caller who adds # 'v' to their connectives gets the Catalan link's behavior for # it, because 'v' is generational vocabulary the way 'i' is. p = Parser(lexicon=Lexicon.default().add(conjunctions={"v"})) @@ -1650,12 +1650,12 @@ def test_the_count_reaches_a_connective_that_is_particle_vocabulary( # the cost that comes with it, pinned rather than hidden: # family_base drops a particle wherever it stands and not only # leading, so the joined run loses the letter here. A standing - # rules.md#R2 limit this row surfaces, not one it creates. + # rules/R2 limit this row surfaces, not one it creates. assert out.family_base == "Velasquez Garcia" # --- #397 review: the link inside a MAIDEN CLAUSE ------------------- -# rules.md#M2's link clause. The walk ends the birth name at the first +# rules/M2's link clause. The walk ends the birth name at the first # suffix WORD after the marker, and the link is one -- so it ended the # clause there, and what #397 added was to JOIN the words it left # standing into the current surname. Each branch below has the same @@ -1755,7 +1755,7 @@ def test_the_marker_is_not_the_name_word_on_the_links_left() -> None: # carries: the marker announces the name and is no word of it, so # a link standing first inside the clause joins nothing there. The # walk then stops at its very first piece and the pass declines - # altogether, leaving the marker an ordinary word (rules.md#M2). + # altogether, leaving the marker an ordinary word (rules/M2). out = _grouped("Jane Doe née i Soler", lexicon=_LINK_LEX) assert _maiden_texts(out) == [] assert _piece_texts(out) == [["Jane", "Doe", "née i Soler"]] @@ -1804,7 +1804,7 @@ def test_a_marker_with_nothing_after_it_declines_before_the_bound( def test_a_trailing_title_does_not_hide_the_suffix_run_from_the_join( ) -> None: - # rules.md#H5 read where the JOIN asks its question (#397 second + # rules/H5 read where the JOIN asks its question (#397 second # review). `trailing_start` reads the peel over the pieces as # WRITTEN, so a title standing behind the suffix run makes the # peel take nothing and the answer is `len(pieces)` -- and a @@ -1846,7 +1846,7 @@ def test_a_trailing_title_does_not_hide_the_suffix_run_from_the_join( assert lower_title.suffix == "i MA" # the ONE-CASE spellings never reached the defect and are pinned # as the control: written wholly in one case the letter reads as - # an initial (rules.md#P3's marked subset), no join is attempted + # an initial (rules/P3's marked subset), no join is attempted # at all, and both spellings already agreed with each other and # with the untitled name at dc3bdf9c -- which is what says the # bound and not the marking is what this row is about. @@ -1859,7 +1859,7 @@ def test_a_trailing_title_does_not_hide_the_suffix_run_from_the_join( def test_a_multi_letter_link_of_the_suffix_vocabulary_joins_by_the_same_rule( ) -> None: - # rules.md#P3's both-sides clause names a CLASS -- "a connective + # rules/P3's both-sides clause names a CLASS -- "a connective # that is also generational vocabulary" -- and says nothing about # how the word is spelled (#397 second review). A `len(text) != 1` # filter stood in the stage and narrowed the clause to one-letter @@ -1918,7 +1918,7 @@ def test_the_first_word_floor_holds_a_title_out_of_the_chain() -> None: def test_the_floor_keeps_the_first_word_in_the_chains_count() -> None: - """rules.md#M2 (#535 review): the floor lives in the chain now, + """rules/M2 (#535 review): the floor lives in the chain now, not in a clamp read after the fact -- the chain may not take the first word after the marker, so the re-peel's count still holds it and a trailing suffix behind a chained title reads as diff --git a/tests/v2/pipeline/test_pieces.py b/tests/v2/pipeline/test_pieces.py index ae2c063f..fc7b8e45 100644 --- a/tests/v2/pipeline/test_pieces.py +++ b/tests/v2/pipeline/test_pieces.py @@ -348,7 +348,7 @@ def test_peel_trailing_takes_the_three_lean_outcomes() -> None: # #289 at the trailing slot, the three outcomes against the peel's # three: a CREDENTIAL lean consumes with no words to spare, a NAME # lean declines WITH words to spare, and no lean at all leaves - # rules.md#S2's count deciding exactly as it did. + # rules/S2's count deciding exactly as it did. # Every case still REPORTS: the pick is appended either way, which # is what the fork's report is built from. for text, one_case, names, picked in ( @@ -380,7 +380,7 @@ def test_peel_trailing_keeps_its_two_piece_floor_under_a_lean() -> None: def test_peel_trailing_takes_a_declined_pick_an_anchor_stands_before( ) -> None: - # #544 reverses the cost decisions.md#S2 had accepted: the surname + # #544 reverses the cost decisions/S2 had accepted: the surname # lean declines 'Ma', but the unambiguous 'Jr' IN FRONT of it # anchors it, so the walk takes it and reaches 'Jr' too -- and the # member is still a pick, reported as a counted one is. @@ -427,7 +427,7 @@ def test_the_walks_own_leading_piece_never_anchors_what_follows_it( def test_a_reserve_kept_leading_piece_beside_a_genuine_family_loss( ) -> None: - # rules.md#S2's Accepted clause ("an unambiguous suffix is + # rules/S2's Accepted clause ("an unambiguous suffix is # consumed even when that leaves no family name at all", 'Smith # Jr.' -> family='') applies just the same when the LEADING piece # is itself listed suffix vocabulary rather than an ordinary name: @@ -476,7 +476,7 @@ def test_credential_anchors_stops_at_a_numeral_or_a_name_word() -> None: def test_a_connective_or_a_numeral_suffix_word_anchors_nothing() -> None: # the generational 'i' is also Catalan's conjunction, and between - # two name words it is a link (rules.md#P3); a single-letter roman + # two name words it is a link (rules/P3); a single-letter roman # numeral, in any case, is ONE LETTER, the shape a middle initial # is written in -- not excluded for being "no credential", since a # multi-letter numeral ('III') anchors like any other suffix word @@ -633,7 +633,7 @@ def test_a_by_shape_token_takes_the_count_and_never_a_lean() -> None: @pytest.mark.parametrize("text, expected", [ - ("Xyz.", True), # H2 (rules.md): an unlisted Latin abbreviation + ("Xyz.", True), # rules/H2 -- an unlisted Latin abbreviation ("김민준.", False), # #323: hangul has no period abbreviations ("田中.", False), # nor Han ("たなか.", False), # nor kana @@ -672,7 +672,7 @@ def test_is_title_shaped_and_is_leading_title_agree(text: str) -> None: def test_own_words_is_the_name_s_own_span_and_its_clause_cut() -> None: - # rules.md#P3 says the case question is asked of the name's OWN + # rules/P3 says the case question is asked of the name's OWN # words. This helper is that span, taken off the token stream so # both the site that runs before classify and classify itself ask # one function (#289/#516). @@ -853,7 +853,7 @@ def test_a_title_the_fixed_point_splices_out_keeps_no_peel_pick() -> None: assert name.ambiguities == () -# rules.md#S2 (#602): a credential after the name core starts a run to +# rules/S2 (#602): a credential after the name core starts a run to # the end of its part. The exclusion tests pass before the change too: # they are the negative controls that make the first two meaningful. def test_a_credential_after_two_name_words_starts_the_run() -> None: @@ -1004,7 +1004,7 @@ def _joined(text: str, *, letter_stays: bool, titles_only: bool, def test_a_connective_run_merges_before_it_joins_in_both_modes() -> None: - """rules.md#P3's "connective runs included", for both callers: the + """rules/P3's "connective runs included", for both callers: the run joins as one link, so a title before it takes the word after it too. The comma decision's copy of the loop merged no run and joined 'Minister of the' alone (#617).""" diff --git a/tests/v2/pipeline/test_post_rules.py b/tests/v2/pipeline/test_post_rules.py index 9cc913da..714afc8c 100644 --- a/tests/v2/pipeline/test_post_rules.py +++ b/tests/v2/pipeline/test_post_rules.py @@ -29,7 +29,7 @@ particles=frozenset({"de", "der", "ibn", "la", "van", "vd"}), particles_ambiguous=frozenset({"la", "van"}), # `씨` is a ONE-LETTER suffix word, which starts no credential run - # (rules.md#S2's two-letter minimum -- any longer suffix word would, + # (rules/S2's two-letter minimum -- any longer suffix word would, # an honorific included since #679), the filler a test needs where # a credential would now take the rest of the part suffix_words=frozenset({"jr", "씨"}), @@ -199,7 +199,7 @@ def test_east_slavic_skips_when_middle_is_also_patronymic() -> None: @pytest.mark.parametrize("text,title", [ ("Мицкевич Адам Юзеф", ""), # a peeled title must not move the shape the gate is claimed of: - # rules.md#O1 counts name words "titles, suffixes and nicknames + # rules/O1 counts name words "titles, suffixes and nicknames # aside", so the rotation still reaches this input pre-gate # (_PRE_GATE_FAMILY records the same family as the bare form) and # the title has to survive the stand-down @@ -208,7 +208,7 @@ def test_east_slavic_skips_when_middle_is_also_patronymic() -> None: @pytest.mark.parametrize("order", _FAMILY_FIRST) def test_east_slavic_stands_down_under_a_declared_family_first_order( order: Policy, text: str, title: str) -> None: - # rules.md#O1's scope clause (decisions.md#O1, 2026-09-07, #384): + # rules/O1's scope clause (decisions/O1, 2026-09-07, #384): # the rotation RESTORES the default reading a family-first listing # hides, so a declared family-first order supplies that reading and # the rotation stands down. The gate is on the order assign READ, @@ -238,7 +238,7 @@ def test_turkic_rotation() -> None: @pytest.mark.parametrize("order", _FAMILY_FIRST) def test_turkic_stands_down_under_a_declared_family_first_order( order: Policy) -> None: - # The other pack, gated by the same read: rules.md#O2 is the same + # The other pack, gated by the same read: rules/O2 is the same # restoration of a hidden given-first reading, and the gate is one # condition covering both rotations -- so pinning only the East # Slavic half would leave half the gate free to come back out. @@ -342,7 +342,7 @@ def test_leading_piece_scan_skips_pieces_that_hold_no_name( # non-name piece, which no fixture in this module produces. # # The title is deliberately UNDOTTED: a period makes any opening - # abbreviation a title by shape (rules.md#H2), so a dotted one + # abbreviation a title by shape (rules/H2), so a dotted one # would pass this test with the title vocabulary empty. out = _parsed("Mr de MD Mesnil", policy) assert _by_role(out, Role.TITLE) == "Mr" @@ -547,7 +547,7 @@ def test_the_run_must_end_the_name() -> None: def test_the_site_reports_the_fork_it_decides() -> None: - # decisions.md#P6 (#405): the attachment reports the fork it + # decisions/P6 (#405): the attachment reports the fork it # decides. The state that decision fixed is the one this site # would otherwise recreate -- the comma writing reporting while # the comma-less writing decides the identical fork in silence. @@ -685,17 +685,17 @@ def test_family_first_run_stops_at_the_first_name_word( @pytest.mark.parametrize("policy", _FAMILY_FIRST) @pytest.mark.parametrize("text,family,rest", [ - # the conjunction join is ONE name word (rules.md#P3), so the run + # the conjunction join is ONE name word (rules/P3), so the run # takes 'Vega y Santos' whole rather than stopping inside it ("de la Vega y Santos Juan", "de la Vega y Santos", "Juan"), # ... and the same clause applies to what is LEFT: 'Juan y Eva' is # one unit, so it lands in one field rather than being split ("de la Cruz Juan y Eva", "de la Cruz", "Juan y Eva"), # a bound given-name word and the word it completes are one unit - # (rules.md#P5), so the fold cannot leave half of the pair behind + # (rules/P5), so the fold cannot leave half of the pair behind ("ibn Awf abdul Rahman", "ibn Awf", "abdul Rahman"), # a particle in what is LEFT chains forward the same way it would - # if the fold had never run (rules.md#P2). Handing its words out + # if the fold had never run (rules/P2). Handing its words out # separately would report given='van' -- a bare particle as the # given name, which is the reading P1 exists to forbid -- and # would disagree with the same tail parsed alone: 'Mesnil van @@ -832,7 +832,7 @@ def test_a_suffix_word_mid_run_ends_the_chain( # orders are pinned apart at a count other than two. The suffix word # is one letter because a longer one -- a credential, or since #679 # an honorific -- would start #602's run and take `Juan` with it - # (rules.md#S2); it was `MD` until #602. + # (rules/S2); it was `MD` until #602. out = _parsed("de Mesnil van Berg 씨 Juan", policy) assert _by_role(out, Role.FAMILY) == "de Mesnil" assert _by_role(out, Role.GIVEN) == given @@ -842,7 +842,7 @@ def test_a_suffix_word_mid_run_ends_the_chain( @pytest.mark.parametrize("policy", _FAMILY_FIRST) def test_a_bound_pair_survives_the_leading_particle_stop( policy: Policy) -> None: - # The stop leaves the bound pair whole (rules.md#P5 via _units). + # The stop leaves the bound pair whole (rules/P5 via _units). # Asserted under a family-first order deliberately: in the DEFAULT # order the fold takes the whole name into the family, so the join # is invisible there and the assertion would pass with the join @@ -862,7 +862,7 @@ def test_a_dual_membership_word_decides_rather_than_raising() -> None: # sets, and neither is ambiguous particle vocabulary, so no shipped # input reaches both arms. A caller's Lexicon can put one word in # both, and an `assert` written on that premise DID fire here - # before this test existed -- rules.md#A1 says parsing never fails + # before this test existed -- rules/A1 says parsing never fails # on any input, and an AssertionError out of a stage is exactly the # failure it forbids. # @@ -952,7 +952,7 @@ def _entry_tags(text: str, policy: Policy | None = None) -> list[tuple[str, bool def test_a_space_separated_run_is_one_entry() -> None: - """rules.md#R1's no-comma path, which is the whole of #436.""" + """rules/R1's no-comma path, which is the whole of #436.""" assert _entry_tags("John Smith MD PhD") == [("MD", False), ("PhD", True)] @@ -1067,7 +1067,7 @@ def test_a_surviving_name_word_parts_two_entries() -> None: test alone cannot reach: a name word standing between two post-nominals parts their entries. `abd` is suffix vocabulary that starts no credential run (it is bound given-name vocabulary too, - rules.md#S2), so `Jones` behind it survives as a name word before + rules/S2), so `Jones` behind it survives as a name word before the credential. Drop the conjunct and this renders 'abd PhD'. The fixture was 'Smith, MD - PhD - FACS' until #602, whose dashes @@ -1178,7 +1178,7 @@ def test_p6_still_claims_an_unambiguous_suffix_particle() -> None: def test_p6_keeps_every_non_capital_spelling_of_the_particle_member( ) -> None: """Capitals decide; the particle rule keeps the rest, and each - reports P6's own kind ONCE (decisions.md#S2, 2026-09-18).""" + reports P6's own kind ONCE (decisions/S2, 2026-09-18).""" for text, family in (("Doe, John do", "do Doe"), ("Doe, John Do", "Do Doe"), ("DOE, JOHN DO", "DO DOE"), @@ -1190,7 +1190,7 @@ def test_p6_keeps_every_non_capital_spelling_of_the_particle_member( def test_a_lone_connective_is_marked_as_joining_nothing() -> None: - # rules.md#R3's mark (#461), decided where the parts are settled. + # rules/R3's mark (#461), decided where the parts are settled. r = parse("Juan de y") y = [t for t in r.tokens if t.text == "y"][0] assert UNJOINED_CONJUNCTION_TAG in y.tags diff --git a/tests/v2/pipeline/test_segment.py b/tests/v2/pipeline/test_segment.py index 364e2088..8df9c5ef 100644 --- a/tests/v2/pipeline/test_segment.py +++ b/tests/v2/pipeline/test_segment.py @@ -51,7 +51,7 @@ def test_excess_non_suffix_segment_flags_comma_structure() -> None: def test_the_comma_structure_report_names_no_field() -> None: # #629: a title word past the second comma reads as a title - # (rules.md#C2, #603), so the part can land in two fields; the + # (rules/C2, #603), so the part can land in two fields; the # report says the part was not recognized and leaves where each # word went to the fields. 2.3.0 said 'consumed as suffix'. out = _segmented("John Smith, Jr., Dr. Bart") @@ -93,7 +93,7 @@ def test_leading_comma_yields_empty_first_segment() -> None: def test_a_tail_segment_of_leaning_credentials_is_not_flagged() -> None: # The third reading site, and the one place this design QUIETS a # report: 'DO' leans credential in a mixed-case name, so the third - # segment is a credential run and rules.md#C2's flag stops firing. + # segment is a credential run and rules/C2's flag stops firing. state = _segmented("Steven Hardman, MD, DO, DDS") assert not [a for a in state.ambiguities if a.kind is AmbiguityKind.COMMA_STRUCTURE] diff --git a/tests/v2/pipeline/test_vocab.py b/tests/v2/pipeline/test_vocab.py index 37a11ba5..6dc200be 100644 --- a/tests/v2/pipeline/test_vocab.py +++ b/tests/v2/pipeline/test_vocab.py @@ -463,7 +463,7 @@ def test_is_title_shaped_is_h2_s_shape_alone() -> None: def test_name_word_count_counts_names_not_tokens() -> None: - # rules.md#C1's count for the ambiguous class: 'Smith Jr.' is two + # rules/C1's count for the ambiguous class: 'Smith Jr.' is two # tokens and ONE name word, which is what keeps its family where a # token count would hand it to `given`. lex, pol = Lexicon.default(), Policy() @@ -663,7 +663,7 @@ def test_script_classification_ignores_edge_full_stops() -> None: # and a configured segmenter is then handed the raw token and can # answer offset 1 -- '.' as the given name, the name as the family. # No script means no site, which is the pre-#323 reading and the - # no-split rules.md#W1 already accepts. + # no-split rules/W1 already accepts. assert effective_script(".양") is None assert effective_script("．양") is None assert effective_script("。양") is None @@ -700,7 +700,7 @@ def test_a_voicing_mark_takes_the_script_of_the_kana_before_it( assert effective_script("김" + mark + "민준") is None assert effective_script("山" + mark) is Script.HIRAGANA # a mark with nothing before it has no base either. Degenerate - # input, recorded rather than promised (rules.md's W Background) + # input, recorded rather than promised (rules/W's Background) assert effective_script(mark + "アイ") is Script.HIRAGANA assert effective_script(mark) is Script.HIRAGANA @@ -786,7 +786,7 @@ def test_is_one_case() -> None: assert not is_one_case(["john", "e", "jones", "III"]) # a caseless script is "one case" harmlessly: the fork that reads # this ALSO requires a cased token, so a caseless letter never - # enters it (rules.md#P3, decisions.md#P3) + # enters it (rules/P3, decisions/P3) assert is_one_case(["محمد", "و", "علي"]) assert is_one_case(["山田", "太郎"]) # the empty and single-token edges diff --git a/tests/v2/rules_doc.py b/tests/v2/rules_doc.py index 44477afa..5812cd9d 100644 --- a/tests/v2/rules_doc.py +++ b/tests/v2/rules_doc.py @@ -1,6 +1,6 @@ -"""Parser and annotation registry for docs/design/rules.md example lines. +"""Parser and annotation registry for docs/design/rules/ example lines. -The grammar (described for humans in rules.md's preamble; this module +The grammar (described for humans in the rules README's preamble; this module is its executable definition): "INPUT" [annotation] → field=VALUE [· boundary] @@ -38,9 +38,7 @@ import ast import re from dataclasses import dataclass, field as dc_field -from pathlib import Path -RULES_DOC = Path(__file__).resolve().parents[2] / "docs" / "design" / "rules.md" _RULE_RE = re.compile(r"^([A-Z])(\d+)\.\s") _VALUE = r"\"[^\"]*\"|\([^)]*\)|\[(?:[^\[\]]|\[[^\]]*\])*\]" @@ -124,7 +122,7 @@ def has_boundary_or_waiver(self) -> bool: "strict-comma-suffixes": Policy(lenient_comma_suffixes=False), #: The two switches over the by-shape credential class (#516). #: Named after the Policy FIELD rather than after the behavior, - #: which is deliberate: rules.md's statements stay + #: which is deliberate: the rules' statements stay #: implementation-free, so the annotation slot is the one place #: the doc can put the caller-facing name of the switch its prose #: describes. The suffix says which way the field is set, since diff --git a/tests/v2/test_benchmark.py b/tests/v2/test_benchmark.py index 04c908cf..bc122208 100644 --- a/tests/v2/test_benchmark.py +++ b/tests/v2/test_benchmark.py @@ -57,7 +57,7 @@ #: reason beside it. A DROP is a signal too, and trips the same test. #: #: Raising or lowering a row is a decision to record in -#: decisions.md#parse-cost, not a maintenance chore. Recompute with +#: decisions/parse-cost, not a maintenance chore. Recompute with #: `uv run python tools/perf/call_count.py`, which is the harness every #: number in that entry comes from. #: @@ -67,7 +67,7 @@ #: the first green run -- a wrong seed fails with the real number. #: #: Lowered 2026-09-26 when the stages stopped copying state through -#: dataclasses.replace (decisions.md#parse-cost). All five rows were +#: dataclasses.replace (decisions/parse-cost). All five rows were #: re-measured with that harness, each on its own interpreter. #: #: Lowered 2026-10-06 (#617) when P3's connective joins left group's @@ -77,18 +77,18 @@ #: +4/+5 inside the old 370/407 and 348/385). Then 39 more on every #: row, in #617's /simplify, when group stopped running the rootname #: count and the shared loop over segments with no connective left -#: free to join (decisions.md#parse-cost). +#: free to join (decisions/parse-cost). #: #: Lowered 2026-10-07 by 8 on every row when `_group_segment`'s last #: three predicate closures (`prefix`, `suffix`, `marker`) were #: inlined: the reference name asked `prefix` 7 times and `suffix` -#: once, each a frame of its own (decisions.md#parse-cost). And 9 on +#: once, each a frame of its own (decisions/parse-cost). And 9 on #: 3.11, 6 on 3.12-3.15, in #614, when S2's trailing run came to be #: read once, between P3's joins and the particle chain, rather than #: re-asked by the chain and the reserve and read again by assign -#: (decisions.md#parse-cost). The 3.12-3.15 rows read 289/326 from +#: (decisions/parse-cost). The 3.12-3.15 rows read 289/326 from #: #614's /simplify on and were corrected in #620, which moved no row -#: (decisions.md#parse-cost). +#: (decisions/parse-cost). _CALL_BASELINE = { (3, 11): {"parse": 307, "facade": 344}, (3, 12): {"parse": 288, "facade": 325}, @@ -140,7 +140,7 @@ def _check_budget(kind: str, fn: Callable[[str], object]) -> None: Skips rather than clobbers when something else owns the profile slot: `sys.setprofile(None)` in the helper CLEARS the hook, and a maintainer profiling the parser -- the very workflow that produced - decisions.md#parse-cost -- would otherwise get silently truncated + decisions/parse-cost -- would otherwise get silently truncated data. Restoring is not an option: `sys.getprofile()` hands back a `Profile` object that `sys.setprofile()` then refuses. """ @@ -152,7 +152,7 @@ def _check_budget(kind: str, fn: Callable[[str], object]) -> None: pytest.fail( f"no call baseline for Python {version[0]}.{version[1]}; " f"{kind} measures {actual:.1f} here. Add the row to " - f"_CALL_BASELINE and record it in decisions.md#parse-cost") + f"_CALL_BASELINE and record it in decisions/parse-cost") baseline = _CALL_BASELINE[version][kind] actual = _calls_per_parse(fn) low, high = baseline * (1 - _BAND), baseline * (1 + _BAND) @@ -162,7 +162,7 @@ def _check_budget(kind: str, fn: Callable[[str], object]) -> None: f"baseline of {baseline}. Growth and shrinkage are both signals: " f"recompute with tools/perf/call_count.py, then either make it " f"cheaper or move the baseline deliberately -- see " - f"decisions.md#parse-cost") + f"decisions/parse-cost") def test_parse_cost_stays_within_its_band() -> None: @@ -741,7 +741,7 @@ def _clause_run(members: int) -> str: """A maiden clause whose birth name is a RUN of links. Mixed case on purpose: written wholly in one case the letter reads - as an initial (rules.md#P3's marked subset). Since #601 the clause + as an initial (rules/P3's marked subset). Since #601 the clause keeps the run because nothing in it ends the clause, so this measures the take's walk over a long clause; until then it measured the clause's link exception, which #601 retired. @@ -813,7 +813,7 @@ def test_the_paired_initials_title_scan_does_not_cost_quadratically() -> None: large_text = "John Smith, " + "MD " * _PAIR_SCAN_LARGE + "G.J. " * _PAIR_SCAN_LARGE # REACHABILITY: the flip REPORTS only where the scan ran and found # nothing but titles in front of the first pair, every shape word - # being a pair (rules.md#C1's "nothing but other paired initials"). + # being a pair (rules/C1's "nothing but other paired initials"). # A change that stopped the scan being asked -- 'MD' leaving the # titles, the pair test moving -- loses the report and fails here # rather than leaving this guard measuring nothing. @@ -1061,7 +1061,7 @@ def test_a_fixed_point_does_not_reread_what_it_has_read( # interpreter gets a row the same way: run it there, then add it. # # Lowered 2026-09-26 from 2587 with `_CALL_BASELINE` above, for the -# same reason (decisions.md#parse-cost). 2,678 from 2026-10-04, the +# same reason (decisions/parse-cost). 2,678 from 2026-10-04, the # re-pointed name-level shape (above), measured on py3.11. 2,479 # from 2026-10-06 (#617), 199 below 83f4e914's 2,678, for two # reasons, counted per function: P3's joins call the piece @@ -1124,7 +1124,7 @@ def test_a_link_costs_what_it_is_pinned_at() -> None: def test_a_member_opening_a_comma_part_asks_no_anchor_pass() -> None: - """rules.md#S2's company clause speaks only for a member with a + """rules/S2's company clause speaks only for a member with a credential in FRONT of it, so a member opening the part after a one-word family comma ('Smith, Ed') is never anchored, and `segment_suffix_reading` builds no `credential_anchors` pass for diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 76bfc60d..8def295d 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -71,7 +71,7 @@ def _swept(case_id: str, @pytest.mark.parametrize("case,order", _SWEPT_ROWS, ids=_SWEPT_IDS) def test_the_family_partitions_into_particles_and_base( case: Case, order: tuple[Role, Role, Role] | None) -> None: - """rules.md#R2's invariant: a particle needs a base to attach to, + """rules/R2's invariant: a particle needs a base to attach to, so a family made only of particles is a family whose words are not acting as particles -- and therefore a non-empty family always has a non-empty base. @@ -315,7 +315,7 @@ def test_case_construction_rejects_a_bad_shape_tag( #: The constructions the battery above proves nothing rejects: a pure #: shape-6 row, a Han shape-6 row divided by a nakaguro that does NOT #: mark it as source order (U+30FB is not a divider outside katakana -#: -- decisions.md#T3; this is family-first per +#: -- decisions/T3; this is family-first per #: ja_nakaguro_han_takes_the_han_order), an interpunct-divided shape-7 #: row, a SPACED wholly-katakana shape-7 row (the subtler admission -- #: a transcription with no U+00B7 at all is still a shape, not a diff --git a/tests/v2/test_corpora.py b/tests/v2/test_corpora.py index 195d6af5..ddf6033b 100644 --- a/tests/v2/test_corpora.py +++ b/tests/v2/test_corpora.py @@ -3,7 +3,7 @@ snapshot can read. A generated corpus that falls behind its source silently narrows the -names snapshot.jsonl watches: a case row or a rules.md example added +names snapshot.jsonl watches: a case row or a rules example added without regenerating is then read by nothing but its own test, which moves with the change it pins. These pins make a stale corpus fail instead. @@ -73,7 +73,7 @@ def test_case_shape_ids_exist_in_the_inventory() -> None: def test_rules_corpus_matches_the_rules_doc() -> None: - """corpus_rules.jsonl is generated from docs/design/rules.md's own + """corpus_rules.jsonl is generated from docs/design/rules/'s own examples (#414), through the doc's parser. test_rules_doc executes every example against an expectation stored beside it, so a deliberate behavior change edits both and stays green; the snapshot @@ -87,12 +87,11 @@ def test_rules_corpus_matches_the_rules_doc() -> None: # regenerating then agrees with the degraded selection. A rule # whose ID line stops matching, or that gains a heading mid-block, # loses its examples to the previous rule or to nothing. + from .design_docs import read_rules_text from .rules_doc import parse_rules_doc - rules = parse_rules_doc( - (_TOOLS.parents[1] / "docs" / "design" / "rules.md") - .read_text(encoding="utf-8")) + rules = parse_rules_doc(read_rules_text()) assert len(rules) >= 38, ( - f"rules.md parsed to {len(rules)} rules, fewer than the 38 this " + f"docs/design/rules/ parsed to {len(rules)} rules, fewer than the 38 this " f"corpus was built from. A rule block that stops parsing hands " f"its examples to its neighbour and shrinks the corpus " f"silently. Ratchet this floor up when rules are added") diff --git a/tests/v2/test_design_docs.py b/tests/v2/test_design_docs.py new file mode 100644 index 00000000..6f113d60 --- /dev/null +++ b/tests/v2/test_design_docs.py @@ -0,0 +1,150 @@ +"""The design docs' folder layout: names, the index, citations.""" +from __future__ import annotations + +import re +from pathlib import Path + +import pytest + +from tests.v2.design_docs import ( + DECISIONS_DIR, MENTION_RE, RULES_DIR, read_decision_files, read_index, + read_rules_text, resolve, slugify, stem_for) +from tests.v2.rules_doc import parse_rules_doc + +_FIXTURE_FILES = ("S-suffixes.md", "S1-x.md", + "S2-case-signal-at-the-suffix-slot.md", + "vocabulary-collisions.md") + + +@pytest.fixture +def folder(tmp_path: Path) -> Path: + (tmp_path / "README.md").write_text( + "# Index\n\n" + "## [Suffixes (S)](S-suffixes.md)\n\n" + "- [S1 — x](S1-x.md)\n" + "- [S2 — the case signal at the suffix slot]" + "(S2-case-signal-at-the-suffix-slot.md)\n\n" + "## Named topics\n\n" + "- [vocabulary-collisions — when a word earns the ambiguous " + "marking](vocabulary-collisions.md)\n", + encoding="utf-8") + for name in _FIXTURE_FILES: + (tmp_path / name).write_text(f"# {name}\n", encoding="utf-8") + return tmp_path + + +def test_slugify_follows_the_spec() -> None: + assert slugify("the case signal at the suffix slot") == ( + "case-signal-at-the-suffix-slot") + assert slugify("Titles & honorifics") == "titles-and-honorifics" + assert slugify("a title and one name word") == "title-and-one-name-word" + assert slugify("Suffixes: generational & credentials") == ( + "suffixes-generational-and-credentials") + assert slugify('the "Ph. D." split') == "ph-d-split" + assert slugify("the title run's floor") == "title-runs-floor" + assert slugify("the writer’s divisions") == "writers-divisions" + + +def test_stem_for_names_rules_and_keeps_slugs() -> None: + assert stem_for("S2", "the case signal at the suffix slot") == ( + "S2-case-signal-at-the-suffix-slot") + assert stem_for("S", "Suffixes") == "S-suffixes" + assert stem_for("vocabulary-collisions", "when a word …") == ( + "vocabulary-collisions") + + +def test_read_index_keeps_order_and_keys(folder: Path) -> None: + entries = read_index(folder) + assert [e.key for e in entries] == [ + "S", "S1", "S2", "vocabulary-collisions"] + assert entries[2].title == "S2 — the case signal at the suffix slot" + assert entries[2].path == folder / "S2-case-signal-at-the-suffix-slot.md" + (folder / "README.md").write_text( + "- [differential-ledger — the differential ledger's arcs]" + "(differential-ledger.md)\n", encoding="utf-8") + assert read_index(folder)[0].key == "differential-ledger" + + +@pytest.mark.parametrize(("text", "doc", "cid"), [ + ("see rules/S2.", "rules", "S2"), + ("rules/S2's Accepted clause", "rules", "S2"), + ("rules/H Background", "rules", "H"), + ("decisions/vocabulary-collisions, which", "decisions", + "vocabulary-collisions"), + ("mechanisms.md#VERSION-TELL", "mechanisms", "VERSION-TELL"), +]) +def test_mention_re_reads_each_shape(text: str, doc: str, cid: str) -> None: + m = MENTION_RE.search(text) + assert m is not None + assert (m.group("doc") or m.group("mdoc"), m.group("cid")) == (doc, cid) + + +def test_mention_re_reads_a_key_wrapped_at_its_hyphen() -> None: + # a comment wraps a long key at a hyphen; the checker joins the + # next line's half on, so the first half must still match + for text in ("mechanisms.md#ONE-PREDICATE-PER-", + "decisions/two-input-'"): + m = MENTION_RE.search(text) + assert m is not None, text + assert m.group("cid") in ("ONE-PREDICATE-PER", "two-input") + + +@pytest.mark.parametrize("text", [ + "docs/design/rules/S2-case-signal-at-the-suffix-slot.md", + "decisions/README.md", + "rules/README.md", + "docs/design/rules/", +]) +def test_mention_re_skips_file_paths(text: str) -> None: + assert MENTION_RE.search(text) is None + + +def test_resolve_section_letter_is_not_a_rule(folder: Path) -> None: + def name(cid: str) -> str | None: + p = resolve("rules", cid, folder) + return p.name if p else None + assert name("S") == "S-suffixes.md" + assert name("S2") == "S2-case-signal-at-the-suffix-slot.md" + assert name("S9") is None + assert name("README") is None + assert resolve("decisions", "vocabulary-collisions", folder) == ( + folder / "vocabulary-collisions.md") + + +# -- the real tree --------------------------------------------------------- + +@pytest.mark.parametrize("folder", [RULES_DIR, DECISIONS_DIR], + ids=["rules", "decisions"]) +def test_every_entry_file_is_indexed_once(folder: Path) -> None: + indexed = [e.path for e in read_index(folder)] + assert len(indexed) == len(set(indexed)), "an index lists a file twice" + on_disk = {p for p in folder.glob("*.md") if p.name != "README.md"} + assert set(indexed) == on_disk + + +@pytest.mark.parametrize("folder", [RULES_DIR, DECISIONS_DIR], + ids=["rules", "decisions"]) +def test_index_title_matches_heading_and_file_name(folder: Path) -> None: + for e in read_index(folder): + first = e.path.read_text(encoding="utf-8").split("\n", 1)[0] + if len(e.key) == 1: # a section keeps its old heading + assert first == "## " + e.title, e.path.name + name = e.title.rsplit(" (", 1)[0] + else: + assert first == "# " + e.title, e.path.name + name = e.title.split(" — ", 1)[1] if " — " in e.title else "" + assert e.path.stem == stem_for(e.key, name), e.path.name + + +def test_each_citation_key_resolves_to_one_file() -> None: + for doc, folder in (("rules", RULES_DIR), ("decisions", DECISIONS_DIR)): + for e in read_index(folder): + assert resolve(doc, e.key) == e.path, (doc, e.key) + + +def test_the_folders_are_not_empty() -> None: + rule_files = [p for p in RULES_DIR.glob("*.md") + if re.match(r"[A-Z]\d+-", p.name)] + rules = parse_rules_doc(read_rules_text()) + assert len(rules) == len(rule_files) >= 46 + assert len(read_decision_files()) >= 60 diff --git a/tests/v2/test_doc_citations.py b/tests/v2/test_doc_citations.py index 39da99cb..27941d26 100644 --- a/tests/v2/test_doc_citations.py +++ b/tests/v2/test_doc_citations.py @@ -20,32 +20,40 @@ from __future__ import annotations import re +import subprocess from pathlib import Path from typing import NamedTuple -from tests.v2.rules_doc import ( - _POINTER_RE, _RULE_RE, RULES_DOC, parse_rules_doc) +from tests.v2.design_docs import ( + DECISIONS_DIR, MECHANISMS_DOC, MENTION_RE, RULES_DIR, + read_decision_files, read_rules_text) +from tests.v2.rules_doc import _POINTER_RE, _RULE_RE, parse_rules_doc REPO = Path(__file__).resolve().parents[2] -MECH_DOC = REPO / "docs" / "design" / "mechanisms.md" +MECH_DOC = MECHANISMS_DOC SWEEP_DIRS = ("nameparser", "tests", "tools") ENFORCE_NO_LEGACY = True # armed 2026-08-15, the rewrite complete _LEGACY_RES = (re.compile(r"§"), re.compile(r"superpowers"), re.compile(r"plan[\s#]+deviation"), re.compile(r"spec\s+[S§]?\d")) -# A reference to a design doc: the doc and the ID or entry key. A -# decisions.md key may be a lowercase slug ('suffix-acronym-collisions'). -_MENTION_RE = re.compile( - r"(?P<doc>rules|mechanisms|decisions)\.md#" - r"(?P<cid>[A-Za-z][A-Za-z0-9_]*(?:-[A-Za-z0-9_]+)*)") +# A reference to a design doc: the doc and the ID or entry key +# (design_docs.MENTION_RE). A decisions key may be a lowercase slug +# ('suffix-acronym-collisions'). +_MENTION_RE = MENTION_RE +# The same reference in the docs themselves, where a key starting with a +# digit (decisions/3-0-reevaluations) is checked too. +_DOC_ANCHOR_RE = re.compile(MENTION_RE.pattern.replace( + "[A-Za-z][A-Za-z0-9_]*", "[A-Za-z0-9][A-Za-z0-9_]*", 1)) +assert _DOC_ANCHOR_RE.pattern != MENTION_RE.pattern, ( + "MENTION_RE was reworded; widen its key's first character here again") # A reference is a CITATION when a quote follows it in one of the -# shapes the tree writes: the colon form (`rules.md#S2: "..."`, where +# shapes the tree writes: the colon form (`rules/S2: "..."`, where # prose may stand before the quote), or the ID, an optional "'s", up # to four words and optionally a dash, colon, comma or parenthesis, # then the -# quote (`rules.md#P5 -- "..."`, `rules.md#M2's Accepted clause -# ("...")`, `rules.md#S2 consumes "..."`, `rules.md#M2 -- the +# quote (`rules/P5 -- "..."`, `rules/M2's Accepted clause +# ("...")`, `rules/S2 consumes "..."`, `rules/M2 -- the # numeral is taken "..."`). Any other reference is a # pointer, which must still name something that exists. _COLON_RE = re.compile(r"\s*:") @@ -59,7 +67,6 @@ # An excerpt may elide with "[...]": each fragment must then be # verbatim, non-empty, and in order. _ELISION = "[...]" -DEC_DOC = REPO / "docs" / "design" / "decisions.md" class Citation(NamedTuple): @@ -72,6 +79,10 @@ class Citation(NamedTuple): lead: str # the text between the ID and its quote +def _cite(doc: str, cid: str) -> str: + return f"{doc}.md#{cid}" if doc == "mechanisms" else f"{doc}/{cid}" + + def _norm(s: str) -> str: # Case folds because a quote routinely lowercases the first letter # of the sentence it lifts; "--" is how an ASCII comment spells the @@ -85,14 +96,14 @@ def _statements() -> dict[str, str]: """What a citation may quote: for a rule, its prose -- the statement and its Accepted: clauses, everything in the block but example, pointer and marker lines; for a section letter - (`rules.md#H Background`), the section's Background paragraph; for + (`rules/H Background`), the section's Background paragraph; for a mechanism, its Contract statement.""" out: dict[str, list[str]] = {} current: list[str] | None = None background: list[str] | None = None section = None marker = False # inside a wrapped no-boundary: marker - for line in RULES_DOC.read_text(encoding="utf-8").splitlines(): + for line in read_rules_text().splitlines(): m = _RULE_RE.match(line) hm = re.match(r"## .*\(([A-Z])\)\s*$", line) if background is not None: @@ -137,18 +148,12 @@ def _statements() -> dict[str, str]: def _decision_entries() -> dict[str, str]: - """decisions.md entry bodies keyed by their ### key. An entry is a - dated record rather than a statement, so a citation of one need - not quote it; but what it does quote must be verbatim. An arc - spread over several headings ('### differential-ledger, the ... - arc') answers to its key with all of them.""" - out: dict[str, str] = {} - text = DEC_DOC.read_text(encoding="utf-8") - for sec in re.split(r"^(?=#{2,3} )", text, flags=re.M): - hm = re.match(r"### ([^\s—,]+)", sec) - if hm: - out[hm.group(1)] = out.get(hm.group(1), "") + " " + _norm(sec) - return out + """decisions entry bodies keyed by their key, one file each. An + entry is a dated record rather than a statement, so a citation of + one need not quote it; but what it does quote must be verbatim. An + arc spread over several headings (the differential-ledger arcs) + answers to its key with all of them.""" + return {key: _norm(text) for key, text in read_decision_files().items()} def _is_excerpt(excerpt: str, body: str) -> bool: @@ -165,9 +170,12 @@ def _is_excerpt(excerpt: str, body: str) -> bool: def _swept_files() -> list[Path]: - # this module's own comments show citation shapes, citing nothing + # this module's own comments show citation shapes, citing nothing, + # and so do test_design_docs.py's fixtures for the citation pattern + shapes = {Path(__file__).resolve(), + REPO / "tests" / "v2" / "test_design_docs.py"} files = [p for d in SWEEP_DIRS for p in sorted((REPO / d).rglob("*.py")) - if p != Path(__file__).resolve()] + if p not in shapes] return files @@ -212,7 +220,7 @@ def _citations() -> list[Citation]: if nxt: text = text[:nxt.start()] colon = bool(_COLON_RE.match(text)) - doc = m.group("doc") + doc = m.group("doc") or m.group("mdoc") if colon and doc != "decisions": # the colon form's quote may follow a paraphrase q = text.find('"') @@ -233,7 +241,7 @@ def test_citations_are_verbatim_excerpts() -> None: where = f"{c.path.relative_to(REPO)}:{c.line}" known = entries if c.doc == "decisions" else statements if c.cid not in known: - problems.append(f"{where}: {c.doc}.md#{c.cid} names nothing") + problems.append(f"{where}: {_cite(c.doc, c.cid)} names nothing") continue if c.colon and c.doc != "decisions" and not c.excerpts: problems.append( @@ -242,7 +250,7 @@ def test_citations_are_verbatim_excerpts() -> None: if not _is_excerpt(excerpt, known[c.cid]): problems.append( f"{where}: not a verbatim excerpt of " - f"{c.doc}.md#{c.cid}: {excerpt[:60]!r}") + f"{_cite(c.doc, c.cid)}: {excerpt[:60]!r}") assert not problems, "\n".join(problems) @@ -252,8 +260,8 @@ def test_the_sweep_reaches_every_citation_shape() -> None: # nothing would let the excerpt test pass on what is left. cites = _citations() leads = [c.lead for c in cites if c.excerpts and not c.colon] - # The possessive lead (`rules.md#X's Accepted clause ("...")`), the - # dash lead (`rules.md#X -- "..."`) and the "[...]" elision had + # The possessive lead (`rules/X's Accepted clause ("...")`), the + # dash lead (`rules/X -- "..."`) and the "[...]" elision had # their only witnesses in the differential ledgers' comments, # deleted with them (#664). The grammar still reads both leads, and # test_an_elided_excerpt_must_keep_its_order holds the elision. @@ -266,6 +274,18 @@ def test_the_sweep_reaches_every_citation_shape() -> None: assert any(len(c.excerpts) > 1 for c in cites) +def test_the_sweep_finds_the_tree_s_citations() -> None: + # A change to the citation pattern can narrow what it reads without + # failing anything: #713's first pattern dropped three keys wrapped + # at a hyphen, found only by diffing citation sets (that shape is + # pinned by test_mention_re_reads_a_key_wrapped_at_its_hyphen). + # 1,010 references, pointers included, on 2026-10-17 (#713): the + # floor catches a pattern that stops reading a large shape -- the + # mechanisms.md form, slug keys, hyphenated keys -- and not a small + # one, nor a lead change that turns citations into pointers. + assert len(_citations()) >= 1000 + + def test_an_elided_excerpt_must_keep_its_order() -> None: assert _is_excerpt("a b [...] d", "a b c d") assert not _is_excerpt("d [...] a b", "a b c d") @@ -318,7 +338,7 @@ def test_implemented_matches_citing_modules() -> None: continue citing.setdefault(c.cid, set()).add(str(c.path.relative_to(REPO))) problems = [] - for rule in parse_rules_doc(RULES_DOC.read_text(encoding="utf-8")): + for rule in parse_rules_doc(read_rules_text()): actual = citing.get(rule.rule_id, set()) if rule.implemented: declared = set(rule.implemented) @@ -340,7 +360,7 @@ def test_implemented_matches_citing_modules() -> None: def test_interacts_ids_exist() -> None: - rules = parse_rules_doc(RULES_DOC.read_text(encoding="utf-8")) + rules = parse_rules_doc(read_rules_text()) ids = {r.rule_id for r in rules} missing = [f"{r.rule_id} -> {t}" for r in rules for t in r.interacts if t not in ids] @@ -348,28 +368,28 @@ def test_interacts_ids_exist() -> None: def test_doc_internal_anchors_resolve() -> None: - """Every rules.md#X / decisions.md#Y / mechanisms.md#Z reference + """Every rules/X / decisions/Y / mechanisms.md#Z reference INSIDE the three docs points at a real rule ID, ### key, or entry heading (## or ###). The code-side citations are covered above; this closes the doc-to-doc half.""" - docs = {name: (REPO / "docs" / "design" / f"{name}.md") - .read_text(encoding="utf-8") - for name in ("rules", "decisions", "mechanisms")} - rule_ids = set(re.findall(r"^([A-Z]\d+)\.\s", docs["rules"], re.M)) - dec_keys = set(re.findall(r"^### ([^\s—]+)", docs["decisions"], re.M)) + paths = [*sorted(RULES_DIR.glob("*.md")), + *sorted(DECISIONS_DIR.glob("*.md")), MECH_DOC] + docs = {str(p.relative_to(REPO)): p.read_text(encoding="utf-8") + for p in paths} + rule_ids = set(re.findall(r"^([A-Z]\d+)\.\s", read_rules_text(), re.M)) + dec_keys = set(read_decision_files()) mech_slugs = set(re.findall(r"^#{2,3} ([A-Z][A-Z0-9_-]+)", - docs["mechanisms"], re.M)) + docs[str(MECH_DOC.relative_to(REPO))], re.M)) problems = [] for name, text in docs.items(): - for m in re.finditer( - r"(rules|decisions|mechanisms)\.md#([A-Za-z0-9_-]+)", text): - doc, anchor = m.group(1), m.group(2) + for m in _DOC_ANCHOR_RE.finditer(text): + doc, anchor = m.group("doc") or m.group("mdoc"), m.group("cid") ok = (anchor in rule_ids if doc == "rules" else anchor in dec_keys if doc == "decisions" else anchor in mech_slugs) if not ok: line = text[:m.start()].count("\n") + 1 - problems.append(f"{name}.md:{line} -> {doc}.md#{anchor}") + problems.append(f"{name}:{line} -> {_cite(doc, anchor)}") assert not problems, "\n".join(problems) @@ -391,3 +411,29 @@ def test_no_legacy_citations() -> None: problems.append( f"{path}: matches {pat.pattern!r}") assert not problems, "\n".join(problems) + + +_OLD_ANCHOR_RE = re.compile(r"\b(rules|decisions)\.md#") + + +def test_no_old_design_doc_anchors() -> None: + """rules.md and decisions.md are folders since #713, cited as + rules/S2 and decisions/key. An old-form citation written from habit + would not be read by the citation check at all, so its quote would + go unchecked; this keeps the form out of every tracked file.""" + tracked = subprocess.run(["git", "ls-files"], cwd=REPO, check=True, + capture_output=True, text=True).stdout.split() + problems = [] + for rel in tracked: + if not rel.endswith((".py", ".md", ".rst")): + continue + path = REPO / rel + if not path.exists(): + continue + for i, line in enumerate( + path.read_text(encoding="utf-8").splitlines(), 1): + if _OLD_ANCHOR_RE.search(line): + problems.append(f"{rel}:{i}: {line.strip()[:80]}") + assert not problems, ( + f"{len(problems)} old-form citations; write rules/ID or " + "decisions/key:\n" + "\n".join(problems[:40])) diff --git a/tests/v2/test_doc_spellings.py b/tests/v2/test_doc_spellings.py index aca2297c..e54abced 100644 --- a/tests/v2/test_doc_spellings.py +++ b/tests/v2/test_doc_spellings.py @@ -1,6 +1,6 @@ """Recorded roster of spellings that have actually shipped wrong in docs, docstrings, or messages. Add an entry when one is caught; never -remove one without a decisions.md note.""" +remove one without a decisions note.""" from __future__ import annotations from pathlib import Path diff --git a/tests/v2/test_facade.py b/tests/v2/test_facade.py index 3c5b7c72..bfae80cd 100644 --- a/tests/v2/test_facade.py +++ b/tests/v2/test_facade.py @@ -698,7 +698,7 @@ def test_token_is_conjunction_reads_the_tag_then_the_vocabulary() -> None: # #528, the two-way decision. A token the parser classified answers # from its tags -- the source the core's initials() reads # (mechanisms.md#RENDER-HONORS-THE-PARSE) -- so a one-case 'e' that - # rules.md#P3 tagged `initial` is a name word and a bare capital 'Y' + # rules/P3 tagged `initial` is a name word and a bare capital 'Y' # that P3 tagged `conjunction` is not, which is the reverse of what # the vocabulary-and-shape test said about either. parsed = HumanName("JUAN Y GARCIA") @@ -720,7 +720,7 @@ def test_token_is_conjunction_reads_the_tag_then_the_vocabulary() -> None: # A field spliced in as raw text was read by no parse, so there is # no tag to honor and the vocabulary answers -- the same fallback - # rules.md#R4's case repair takes, through the same helper. + # rules/R4's case repair takes, through the same helper. spliced = HumanName("john smith") spliced.middle = "e" lower = spliced._parsed.tokens_for(Role.MIDDLE)[0] @@ -773,7 +773,7 @@ def test_process_initial_with_tokens_reads_the_parse() -> None: # which is what the bare-string path above already said about the # same text. The flip the first half pinned is re-pinned below on # a 'Y' that IS joining, where the two paths still disagree; - # rules.md#R3 states the rule and decisions.md#R3 records it. + # rules/R3 states the rule and decisions/R3 records it. hn = HumanName("JUAN Y GARCIA") middle = hn._list_tokens_for("middle")[0] assert hn._process_initial("", firstname=False, tokens=middle) == "Y" @@ -821,14 +821,14 @@ def test_an_override_that_forwards_tokens_keeps_working() -> None: # PRE-#528 answer, computed from the vocabulary fallback rather # than the parse. "john e smith" is where the two views disagree # -- the parse TAGS the middle "e" an initial - # (test_process_initial_with_tokens_reads_the_parse); rules.md#P3's + # (test_process_initial_with_tokens_reads_the_parse); rules/P3's # one-case fork reads it from the vocabulary, not from its shape, # which is exactly why the bare-word vocabulary fallback below # reads it the other way -- "e" is not initial-SHAPED by any # pattern over the bare word, so a fallback with no parse behind it # reads lowercase "e" as the # without #528's fix. Accepting AND FORWARDING `tokens` is still - # the only way to receive the fix. decisions.md#R3's 2026-09-13 + # the only way to receive the fix. decisions/R3's 2026-09-13 # entry (amended 2026-09-14) and the 2.4.0 release note both # state this. class Forwards(HumanName): @@ -946,7 +946,7 @@ def test_initials_of_a_spliced_field_ask_the_vocabulary() -> None: # nothing rather than read and found plain. There is no tag to # honor, so this view falls back to the vocabulary for the one # question a word can answer alone -- the same fallback and the - # same helper as rules.md#R4's case repair. The lowercase spelling + # same helper as rules/R4's case repair. The lowercase spelling # reads as the connective and drops; the capital is initial-shaped # and stays, keeping the letter's case as every initial does. lower = HumanName("john smith") @@ -969,7 +969,7 @@ def test_initials_freeze_the_connective_answer_at_parse_time() -> None: # ability to WITNESS the freeze: its 'y' holds the middle part # alone, so it initials as a marked connective and would initial # again as a plain name word, and all three readings below are now - # the same string (rules.md#R3, decisions.md#R3). The name is kept + # the same string (rules/R3, decisions/R3). The name is kept # for its moved value and for the capitalize() precedent; the # freeze itself is re-pinned under it on a name where the # connective is JOINING, which is where a vocabulary edit still @@ -1000,10 +1000,10 @@ def test_initials_of_an_unpickled_or_copied_name_ask_the_vocabulary_too() -> Non # v1 pickle carries the *_list STRINGS and no tags, so a restored # name is spliced text throughout and takes the fallback above. # That makes it disagree with a live parse of the same string on - # the two names rules.md#P3's one-case fork moved -- which + # the two names rules/P3's one-case fork moved -- which # capitalize() has done since the tag was introduced, for the same # reason and through the same helper. Pinned rather than left to - # prose; decisions.md#R3 records it. + # prose; decisions/R3 records it. # # copy.copy and copy.deepcopy go through the same __getstate__/ # __setstate__ hooks as pickle -- HumanName's own pair, defined @@ -1019,8 +1019,8 @@ def test_initials_of_an_unpickled_or_copied_name_ask_the_vocabulary_too() -> Non # only the missing parse explains the difference. # 'JUAN Y GARCIA' no longer contrasts on INITIALS: #461 gave the # live parse the restored answer, both views now saying 'J. Y. G.' - # because the 'Y' holds its part alone (rules.md#R3, - # decisions.md#R3). Kept for its CAPITALIZE half, which still + # because the 'Y' holds its part alone (rules/R3, + # decisions/R3). Kept for its CAPITALIZE half, which still # differs, and replaced on the initials side by # 'JUAN GARCIA Y LOPEZ', where the letter IS joining and the live # parse still drops it. 'john e smith' is untouched throughout. @@ -1051,13 +1051,13 @@ def test_initials_of_an_unpickled_or_copied_name_ask_the_vocabulary_too() -> Non def test_the_v1_off_switch_restores_the_pre_397_fields() -> None: - """decisions.md#P3's off switch on the V1 SURFACE (#397 second + """decisions/P3's off switch on the V1 SURFACE (#397 second review). The bullet says deleting `i` from `C.conjunctions` restores the parent's readings; the core-side switch is pinned by the off-switch grid in tests/v2/test_properties.py and this is the facade half, which has no `Lexicon` of its own to remove from. - The three role movers of decisions.md#P3's one-case paragraph are + The three role movers of decisions/P3's one-case paragraph are the rows, because they are the ones whose FIELDS move: every value on the right below is what the released 2.3.0 wheel gives, measured 2026-09-20. diff --git a/tests/v2/test_lexicon.py b/tests/v2/test_lexicon.py index 04992577..2eb42645 100644 --- a/tests/v2/test_lexicon.py +++ b/tests/v2/test_lexicon.py @@ -31,7 +31,7 @@ def test_default_sources_v1_vocabulary() -> None: # flipped model: 'dos' is never-given in v1, so NOT ambiguous here assert "dos" in lex.particles and "dos" not in lex.particles_ambiguous assert "van" in lex.particles_ambiguous - # the policy is 'e' and not 'y' (decisions.md#P3): a bare E initial is + # the policy is 'e' and not 'y' (decisions/P3): a bare E initial is # common, where 'y' between two surnames is the commonest Hispanic compound assert "e" in lex.conjunctions_ambiguous and "y" not in lex.conjunctions_ambiguous # CAPITALIZATION_EXCEPTIONS maps 'phd' -> 'PhD', a case mask since @@ -425,7 +425,7 @@ def test_the_shipped_suffix_sets_are_disjoint() -> None: # periods, the acronym test strips all of them -- so a word in both # is matched by two rules and which one fired is unreadable from # outside. The last overlap, 'esq', left the acronyms 2026-09-08 - # (decisions.md#suffix-acronym-collisions). + # (decisions/suffix-acronym-collisions). # # The SHIPPED sets only: Lexicon has no such invariant, and a # caller who wants the overlap in their own vocabulary keeps it. @@ -604,7 +604,7 @@ def test_removing_a_conjunction_leaves_its_ambiguous_marker_alone() -> None: # conjunctions_ambiguous is the same shape as given_name_titles: not # in _SUBSET_FIELDS, so an orphan is inert rather than rejected -- # the marker is never consulted once its base entry is gone - # (decisions.md#P3). + # (decisions/P3). lean = Lexicon.default().remove(conjunctions={"e"}) assert "e" not in lean.conjunctions assert "e" in lean.conjunctions_ambiguous @@ -675,7 +675,7 @@ def test_an_nfd_authored_entry_is_stored_composed() -> None: # the storage fold composes NFC (#322), so a caller who authors a # surname in NFD gets the NFC entry -- and raw NFD input, which # segmentation matches without folding, then finds no entry and - # goes unsplit (rules.md#W1's accepted degradation). Pinned + # goes unsplit (rules/W1's accepted degradation). Pinned # because the surname site's comment relies on it. nfd = unicodedata.normalize("NFD", "김") lex = Lexicon(surnames=frozenset({nfd})) diff --git a/tests/v2/test_locales.py b/tests/v2/test_locales.py index 15d2665c..6d6be1d3 100644 --- a/tests/v2/test_locales.py +++ b/tests/v2/test_locales.py @@ -220,7 +220,7 @@ def test_ja_pack_contents() -> None: assert locales.JA.code == "ja" # segmentation activation ONLY: no vocabulary (no list settles a # kanji name) and no order (the kana license already reads - # Japanese family-first by default, decisions.md#W4) + # Japanese family-first by default, decisions/W4) assert locales.JA.policy.segment_scripts == frozenset( {Script.HAN, Script.HIRAGANA}) assert locales.JA.policy.name_order is UNSET @@ -755,8 +755,8 @@ def test_the_rotation_stands_down_under_a_declared_family_first_order( The family-first listing reads the same either way, and the natural-order name reads as the caller declared it -- family 'Иван' -- which is what #384 measured, Accepted in - docs/design/decisions.md#O1 rather than treated as a defect. - rules.md#O1 states it in prose because an example line there + docs/design/decisions/O1 rather than treated as a defect. + rules/O1 states it in prose because an example line there takes ONE annotation, a pack or an order, never both. """ ff = parser_for(locales.RU, @@ -779,7 +779,7 @@ def test_the_rotation_stands_down_under_a_declared_family_first_order( def test_the_turkic_marker_lands_in_a_name_field_under_both_orders( ) -> None: - """Pins the prose decisions.md#O1's 2026-09-07 entry states for O2. + """Pins the prose decisions/O1's 2026-09-07 entry states for O2. Gate-independent: `Ali Ahmad Vali oglu` is natural-order input, so the rotation would not fire on it under the default order @@ -789,7 +789,7 @@ def test_the_turkic_marker_lands_in_a_name_field_under_both_orders( Under FAMILY_FIRST that costs nothing visible, the family being the word the rotation would have chosen anyway; under FAMILY_FIRST_GIVEN_LAST the marker becomes the GIVEN name, which - is the reading rules.md#O2 carries as Accepted. + is the reading rules/O2 carries as Accepted. """ ff = parser_for(locales.TR_AZ, base=Parser( policy=Policy(name_order=FAMILY_FIRST))) @@ -923,7 +923,7 @@ def _default_corpus() -> list[str]: # ja has no marker regexes and no vocabulary either: its rotators cover # the shapes only the pack PLUS its segmenter changes. Every row is # UNSPACED on purpose -- a spaced kana-licensed name (高橋 みなみ) now -# reads family-first by DEFAULT (decisions.md#W4), so it would +# reads family-first by DEFAULT (decisions/W4), so it would # not deviate and would fail the must-deviate assertion below. _ROTATORS["ja"] = [ "山田太郎", # 2-kanji family + 2-kanji given, the common shape @@ -1089,7 +1089,7 @@ def test_non_interference_all_packs_combined() -> None: # The Devanagari block grew and a Bengali one opened on 2026-09-06 # with the Indic honorific bundle (#346/#344/#343); those rows were confirmed the # same way, live against a runtime-augmented Lexicon.default() before -# the data landed. See decisions.md#indic-honorifics. +# the data landed. See decisions/indic-honorifics. @pytest.mark.parametrize("name, field, expected", [ # Cyrillic (ru/uk) titles. ("г-н Иван Петров", "title", "г-н"), @@ -1184,7 +1184,7 @@ def test_non_interference_all_packs_combined() -> None: ("الحاج عبد الرحمن السيد", "given", "عبد الرحمن"), ("الحاج عبد الرحمن السيد", "family", "السيد"), # a given-name title plus the bound pair ALONE: the licence - # (rules.md#P5, #369) lifts the family reserve, so the pair is the + # (rules/P5, #369) lifts the family reserve, so the pair is the # given name and, as for "Sir John", there is no family. Thirteen # of the given-name titles are this script's, so this is where # the licence mostly fires. @@ -1264,7 +1264,7 @@ def test_non_interference_all_packs_combined() -> None: # বাবু is TRAILING in Bengali where Devanagari बाबू is LEADING. # Different codepoints, so the two cannot interact -- and a sweep # "harmonizing" them changes four parses and, without these rows, - # reddens nothing (decisions.md#indic-honorifics). + # reddens nothing (decisions/indic-honorifics). ("অমল বাবু", "suffix", "বাবু"), ("অমল বাবু", "given", "অমল"), ("बाबू अमल", "title", "बाबू"), @@ -1328,7 +1328,8 @@ def test_non_interference_all_packs_combined() -> None: ("মি. রহমান", "title", "মি."), ("মি. রহমান", "family", "রহমান"), # The exclusion holds: ঠাকুর is Tagore, a surname, and is - # deliberately not a title (decisions.md's Excluded (TITLES)). + # deliberately not a title (decisions/vocabulary-exclusions, + # Excluded (TITLES)). # LEADING position is where the exclusion is load-bearing -- # TITLES is leading-only, so the trailing spelling reads the same # whether or not the word ships and cannot guard anything. diff --git a/tests/v2/test_order_correspondence.py b/tests/v2/test_order_correspondence.py index 6fa9a406..a42c2f85 100644 --- a/tests/v2/test_order_correspondence.py +++ b/tests/v2/test_order_correspondence.py @@ -9,18 +9,18 @@ equal, which consults no vocabulary and no rule -- so it catches a rule that is WRONG, which review cannot. -Provenance (decisions.md#P6, the 2026-08-30 #467 entry): over ALL 70 +Provenance (decisions/P6, the 2026-08-30 #467 entry): over ALL 70 particles x 3 families x 3 givens = 630 generated names, 0 of 630 agreed before the #467 change and 603 of 630 after. The 27 failures are the particle-and-suffix trio -- vd, mc, do -- a real limit and -not noise (rules.md#P6, restated as a third bullet below); this +not noise (rules/P6, restated as a third bullet below); this module's own parametrized sweep below excludes that trio on purpose and covers 2 x 1 x 3 x 4 x 2 x 2 pairs of its own -- families, givens, middles, particles, titles, suffixes -- a SEPARATE, smaller count that must never be quoted as the 630 measurement or vice versa. Written as the product rather than as a figure because the figure was -the thing that went stale: the tuples below are what decides it. decisions.md -#two-input-invariants recomputes the same correspondence a second +the thing that went stale: the tuples below are what decides it. +decisions/two-input-invariants recomputes the same correspondence a second way -- a 6x6 spread of families and givens per particle rather than 3x3 -- and gets 67 of 70 particles agreeing on all 36 of their pairs, the same trio failing all 36; its Recompute recipe is what this @@ -28,12 +28,12 @@ predecessor, has no comparable number on record; its defects are qualitative -- it lost a given name outright on "van Berg Jan de" and promoted a post-nominal into the given slot on "Berg Jan Jr. -de" (decisions.md#P6). An earlier draft of this module cited -"0/216 -> 216/216", which decisions.md#P6 explicitly retracts: that +de" (decisions/P6). An earlier draft of this module cited +"0/216 -> 216/216", which decisions/P6 explicitly retracts: that number came from a script slicing the vocabulary (`sorted(particles)[:14]`), which happened to exclude exactly vd, mc and do -- the project's own example of a detector inheriting the -design's blind spot (decisions.md#P6 says: "A FIRST DRAFT OF THIS +design's blind spot (decisions/P6 says: "A FIRST DRAFT OF THIS ENTRY CLAIMED 216 of 216 ... Recompute over the WHOLE vocabulary, never a sample"). @@ -52,21 +52,21 @@ writings read it differently and the correspondence genuinely fails. It is the ASYMMETRY that breaks it, not a precedence applying to both sides: the attachment outranks the suffix reading - on the COMMA side, which is the scope rules.md#P6 states it in, so + on the COMMA side, which is the scope rules/P6 states it in, so `Ménil, Christophe vd` reads family `vd Ménil` while the family-first `Ménil Christophe vd` reads family `Ménil` and suffix - `vd`. Not a bug, but rules.md#P6 stating it plainly: + `vd`. Not a bug, but rules/P6 stating it plainly: "the word is BOTH a particle and suffix vocabulary, this attachment outranks the suffix reading (S2): a trailing abbreviation after a family comma is the tussenvoegsel far more - often than the decoration it collides with" -- and decisions.md - #P6 calling it "a real limit, not noise". The parametrized + often than the decoration it collides with" -- and decisions/P6 + calling it "a real limit, not noise". The parametrized vocabulary below deliberately excludes vd/mc/do; the negative control test owns them instead. A mass failure here -- most or all pairs turning red, rather than a -handful -- means the correspondence itself has narrowed and rules.md -#P6 needs amending alongside this file, not that the vocabulary +handful -- means the correspondence itself has narrowed and rules/P6 +needs amending alongside this file, not that the vocabulary needs shrinking to get back to green. """ import itertools @@ -86,7 +86,7 @@ def _reading(p: ParsedName) -> dict[str, object]: """The comparator shape tools/differential/compare.py uses: the seven role fields (never None -- ParsedName's role properties are - typed `-> str`) plus the ambiguity-kind SET. decisions.md#P6 + typed `-> str`) plus the ambiguity-kind SET. decisions/P6 (#405) records that this design's own repaired failure was exactly an ambiguity ASYMMETRY between two writings of one name, so a role-only comparison would miss the most fragile part of the @@ -151,8 +151,8 @@ def test_form_4_parses_as_its_comma_twin(comma: str, inline: str) -> None: f"shape written two ways and must agree") -# The recorded negative control (decisions.md#P6, decisions.md#two- -# input-invariants, rules.md#P6): the ONLY members of the shipped +# The recorded negative control (decisions/P6, decisions/two- +# input-invariants, rules/P6): the ONLY members of the shipped # particle vocabulary where the correspondence fails are the words # that are both particle and suffix vocabulary. Swept over the entire # shipped vocabulary -- nameparser.config.particles.PARTICLES, the @@ -168,12 +168,12 @@ def test_form_4_parses_as_its_comma_twin(comma: str, inline: str) -> None: def test_only_the_particle_suffix_trio_breaks_the_correspondence() -> None: - # rules.md#P6 states the exception this sweep measures: "the word + # rules/P6 states the exception this sweep measures: "the word # is BOTH a particle and suffix vocabulary, this attachment # outranks the suffix reading (S2)" # # The shape is the minimal "Family Given Particle" -- one family, - # one given -- rather than decisions.md#two-input-invariants' 6x6 + # one given -- rather than decisions/two-input-invariants' 6x6 # spread, because that entry's own Recompute bullet says a spread # only matters to rule out families/givens that are THEMSELVES # particle vocabulary (the precondition asserted below); once @@ -183,7 +183,7 @@ def test_only_the_particle_suffix_trio_breaks_the_correspondence() -> None: # mechanisms.md#VOCABULARY-OVERLAP-AS-PRECONDITION: "assert the # intersection as a precondition" -- inverted from the mechanism's # usual shape: what has to hold here is the intersection being - # EMPTY, not populated. decisions.md#two-input-invariants records + # EMPTY, not populated. decisions/two-input-invariants records # that a particle-vocabulary family or given measures a different # shape and reports 0 of 70 rather than 67 of 70. assert not ({fam.lower(), giv.lower()} & _SHIPPED_PARTICLES), ( diff --git a/tests/v2/test_parser.py b/tests/v2/test_parser.py index 83d9eee6..a4fa465a 100644 --- a/tests/v2/test_parser.py +++ b/tests/v2/test_parser.py @@ -206,7 +206,7 @@ def test_ambiguous_acronym_reports_the_reading_it_took() -> None: # MOVED by #289, not deleted: 'MA' is written in capitals inside a # mixed-case name, so it now leans CREDENTIAL and is taken with no # words to spare -- 'Jack' is the only name word left, which is - # what also turns on the GIVEN_OR_FAMILY fork (decisions.md#S2). + # what also turns on the GIVEN_OR_FAMILY fork (decisions/S2). took_family = parse("Jack MA") assert took_family.suffix == "MA" assert not took_family.family @@ -514,7 +514,7 @@ def test_bound_given_name_that_is_also_a_particle() -> None: # claimed 'Bakar' -- a side effect of the #367 bug, not a rule. Now # the bound given-name join reads it: "Sheik" is a given-name # title, which licenses the join with one word to spare - # (rules.md#P5), so the particle never enters into it -- and a + # (rules/P5), so the particle never enters into it -- and a # bound word read as the bound word is not a fork, so the # PARTICLE_OR_GIVEN report the chain used to emit is gone. The # same precedence, untitled, is what "Abu Bakar Salim" has always @@ -527,7 +527,7 @@ def test_bound_given_name_that_is_also_a_particle() -> None: def test_a_given_name_title_licenses_the_bound_given_join() -> None: - # rules.md#P5 (#369): the contrast #367's release note draws -- + # rules/P5 (#369): the contrast #367's release note draws -- # "Sheik abdul salam", whose lead is a bound given name and NOT a # particle -- now joins the same way, and for the same reason. A # plain title does not license it: "Dr." addresses by family, so @@ -543,7 +543,7 @@ def test_a_given_name_title_licenses_the_bound_given_join() -> None: # 'Sheik John Ma' reports its pick (the #369 precedent). S2's run is # read once now, before the join, and the read weighs 'Ma' as a # credential and declines it before the join makes it part of the - # given pair -- so the pick reports in both (decisions.md#P5, + # given pair -- so the pick reports in both (decisions/P5, # 2026-10-07) for text in ("Sheik abdul Ma", "Sheik John Ma"): assert [a.kind for a in parse(text).ambiguities] == \ @@ -577,7 +577,7 @@ def test_the_bound_given_join_leaves_a_suffix_where_it_stands() -> None: ("abdul", "Smith", "Berg", "Ph. D.") # after a family comma the decline holds under the LENIENT reserve, # and since #602 the 'Jr' starts the given part's credential run, - # taking 'Smith' with it (rules.md#S2) + # taking 'Smith' with it (rules/S2) n = parse("Berg, abdul Jr Smith") assert (n.given, n.middle, n.family, n.suffix) == \ ("abdul", "", "Berg", "Jr Smith") @@ -590,7 +590,7 @@ def test_the_reserve_counts_what_the_trailing_read_left() -> None: # takes nothing (the initial-shaped 'J.' suppresses the fork), three # name words stand, and the join leaves two, so it stands -- given # 'abdul J.', family 'V'. The join no longer re-reads the fork it - # would move (decisions.md#P5, 2026-10-07). + # would move (decisions/P5, 2026-10-07). n = parse("abdul J. V") assert (n.given, n.middle, n.family, n.suffix) == ("abdul J.", "", "V", "") n = parse("John J. V") @@ -631,7 +631,7 @@ def test_the_reserve_spares_the_family_the_acronym_fork_would_take() -> None: # # MOVED by #289, not deleted: 'Ma' is Title-case in a mixed-case # name, so it leans SURNAME on both sides of the join and stays a - # name word instead of a credential (decisions.md#S2). + # name word instead of a credential (decisions/S2). n, m = parse("abdul Smith Ma"), parse("John Smith Ma") assert (n.family, n.suffix) == (m.family, m.suffix) == ("Ma", "") @@ -675,7 +675,7 @@ def test_the_licence_joins_where_the_read_took_nothing() -> None: # suffix after the join: the run, read once before it, takes # nothing, and the pair joins with the V still the family -- the # suffix reading the equality guarded against no longer arises - # (decisions.md#P5, 2026-10-07). + # (decisions/P5, 2026-10-07). n = parse("Sir abdul J. V") assert (n.given, n.family, n.suffix) == ("abdul J.", "V", "") n = parse("Sir John J. V") @@ -694,7 +694,7 @@ def test_the_chain_and_the_walk_stop_where_the_peel_begins() -> None: # mixed-case name, so it now leans SURNAME, the # run's read leaves it a name word (S2's read ahead of the # chain since #614, its re-ask before that), so the chain - # takes it into the particle run (decisions.md#S2). + # takes it into the particle run (decisions/S2). ("John van der Berg Ma", "van der Berg Ma", "")): n = parse(text) assert (n.family, n.suffix) == (family, suffix), text @@ -721,7 +721,7 @@ def test_the_chain_and_the_walk_stop_where_the_peel_begins() -> None: # word (#367), and the acronym it leaves has no words to spare for # assign: the chain keeps it rather than leave it as the family # - # MOVED by #289, ACCEPTED COST (decisions.md#S2): 'MA' is written + # MOVED by #289, ACCEPTED COST (decisions/S2): 'MA' is written # in capitals inside a mixed-case name, so it now leans CREDENTIAL # and the caps lean makes it reachable where the count alone was # not -- the lean ends the chain's re-ask with no words to spare, @@ -782,7 +782,7 @@ def test_the_p5_licence_and_h1_read_a_title_run_the_same_way( assert (parse(f"{title} John").family == "") == \ (parse(f"{title} abdul rahman").family == "") # The same invariant with a trailing title behind the pair, and - # stated the way rules.md#H5's transparency clause states it + # stated the way rules/H5's transparency clause states it # rather than as agreement between the two halves: `X Prof.` is # `X` plus a title, so adding the title moves NEITHER field. That # is what discriminates. Comparing the two spellings' "no family" @@ -977,7 +977,7 @@ def test_revise_preserves_particle_tags() -> None: def test_revise_derives_suffix_entries_from_the_values_commas() -> None: # A suffix value's entry structure is the value's own commas, read - # by rules.md#R1's pass after the named role is forced (#511). The + # by rules/R1's pass after the named role is forced (#511). The # sub-parse of a bare "MD PhD" reads a title and a family name and # so joins nothing on its own; revise() runs the pass again over # the forced roles, where the two share a comma bucket. @@ -989,7 +989,7 @@ def test_revise_derives_suffix_entries_from_the_values_commas() -> None: # consult in a field value, and it still does not fire here; the # ENTRY pass joins the pair for the reason it joins MD and PhD. # Supersedes the 2026-08-31 acceptance of 'Ph., D.' on this path - # (decisions.md#phd-merge). + # (decisions/phd-merge). assert p.revise(n, suffix="Ph. D.").suffix == "Ph. D." # a dash is a token under the default policy, and a suffix word # once the role is forced @@ -1056,7 +1056,7 @@ def test_revise_clears_a_stale_unjoined_mark() -> None: def test_revise_sets_a_missing_unjoined_mark() -> None: - # The other direction, and the one that made rules.md#R2's + # The other direction, and the one that made rules/R2's # invariant false through this path: "St" alone parses as a TITLE # (a word in both the title and particle vocabularies), so the # sub-parse marks nothing, and the harvest then re-roles a bare @@ -1078,7 +1078,7 @@ def test_revise_recomputes_the_connective_mark_in_both_directions( # The mark says a connective stands in a part with nothing to # join, which is a fact about the PART -- so the harvest that # splices a sub-parse's tokens into one field invalidates it both - # ways, exactly as it does the particle mark above (rules.md#R3, + # ways, exactly as it does the particle mark above (rules/R3, # #461). p = Parser() lone = p.parse("Juan de y") @@ -1312,7 +1312,7 @@ def test_a_rendered_suffix_revises_back_to_itself(name: str) -> None: def test_revise_keeps_the_sub_parses_within_piece_mark() -> None: # The forcing keeps the sub-parse's tags: a "joined" made WITHIN a # piece (the Ph. D. merge, mid-value where the head rule lets it - # fire) is role-blind and right for every role (decisions.md#C1's + # fire) is role-blind and right for every role (decisions/C1's # #436 DECLINED list), and a draft that cleared it with the role # lost it on every non-suffix value. For a suffix value the pass # writes the marks itself ('Ph. D. MD' sub-parses with none), so @@ -1423,7 +1423,7 @@ def test_nakaguro_split_han_tokens_take_the_han_order() -> None: def test_halfwidth_nakaguro_splits_at_parse_level_too() -> None: # decision, not accident: halfwidth kana is katakana (#594), and a # name written wholly in katakana keeps the declared order - # (rules.md#W4) as マイケル・ジャクソン does -- KATAKANA has no + # (rules/W4) as マイケル・ジャクソン does -- KATAKANA has no # script_orders entry -- while the dot divides the tokens regardless text = "ﾏｲｹﾙ･ｼﾞｬｸｿﾝ" n = parse(text) @@ -1697,7 +1697,7 @@ def test_stacked_activation_warns_only_for_uncovered_scripts() -> None: #: The corpus names a maiden clause can be appended to without the #: clause itself being the variable: Latin script, no comma (a clause -#: behind a comma is post-comma text, rules.md#M2's Accepted row), and +#: behind a comma is post-comma text, rules/M2's Accepted row), and #: no marker already present. _LATIN = re.compile(r"^[\x00-\u024f]*$") @@ -1721,7 +1721,7 @@ def _clause_free_latin_corpus_names() -> list[str]: # a delimiter character is still a marker, and neither the # predicate nor the vocabulary strips a bracket, so '(geb.' must # lose it before being asked. Stripping only the period admitted - # every corpus name that brackets its marker, and once rules.md#M3 + # every corpus name that brackets its marker, and once rules/M3 # read such a clause as the maiden name, six of them had a maiden # clause of their own. The strip turns away TEN names in all: those # six, plus four that M3 declines and that would have been safe to @@ -1790,7 +1790,7 @@ def test_the_clause_free_corpus_is_not_empty() -> None: @pytest.mark.parametrize("name", _clause_free_latin_corpus_names()) def test_a_maiden_clause_changes_nothing_else(name: str) -> None: """The grouping rules count and join only the words that remain - once the marker and the maiden name leave (rules.md#M2, #418), so + once the marker and the maiden name leave (rules/M2, #418), so appending a clause adds a maiden name and moves no other field -- wherever the appended marker counts at all. @@ -1813,7 +1813,7 @@ def test_a_maiden_clause_changes_nothing_else(name: str) -> None: skipped. One class of name moves ON PURPOSE, and is asserted MOVING rather - than skipped: rules.md#M4 makes the clause decide a name that + than skipped: rules/M4 makes the clause decide a name that holds exactly one name word, because a marker announces a former surname and only means anything beside a current one. The flip is exactly given -> family with every other field standing still -- @@ -1852,7 +1852,7 @@ def test_a_maiden_clause_changes_nothing_else(name: str) -> None: or "conjunction" in last.tags), ( f"{name!r}: the appended marker took nothing, but the word " f"before it, {last.text if last else None!r}, is none of " - f"rules.md#M2's refusals") + f"rules/M2's refusals") return # M4's guard, read off the base parse: one GIVEN token, no other # name word, no title (a titled name is H1's), and neither @@ -1869,7 +1869,7 @@ def test_a_maiden_clause_changes_nothing_else(name: str) -> None: if flips: assert (with_clause.family, with_clause.given) == (base.given, ""), ( f"{name!r}: one name word beside a maiden clause is the " - f"family name (rules.md#M4), but it reads family " + f"family name (rules/M4), but it reads family " f"{with_clause.family!r} / given {with_clause.given!r}") for field in moved: assert getattr(with_clause, field) == getattr(base, field), ( @@ -1897,7 +1897,7 @@ def test_a_phrase_marker_outranks_the_word_it_starts_with() -> None: def test_the_catalan_recipe_unmarks_the_link_for_one_case_names() -> None: - """rules.md#P3's per-caller answer for Catalan and Polish data + """rules/P3's per-caller answer for Catalan and Polish data (#397 second review), pinned rather than described. `i` ships in the MARKED subset, so a name written wholly in one @@ -1929,7 +1929,7 @@ def test_the_catalan_recipe_unmarks_the_link_for_one_case_names() -> None: _NFD_ROWS = [ - # the period-abbreviation shape (rules.md#H2, #H5), in each slot + # the period-abbreviation shape (rules/H2, #H5), in each slot "Dr. inž. Janez Novak", "inž. Janez Novak", "prof. dr. inž. Janez Novak", @@ -1939,7 +1939,7 @@ def test_the_catalan_recipe_unmarks_the_link_for_one_case_names() -> None: # the comma shapes AGENTS.md asks of every comma grid ", inž.", "(Bob), inž.", - # paired initials (rules.md#C1) + # paired initials (rules/C1) "Janez Novak, Ž.Š.", "Janez Novak, Ž.Š., Jr.", # the all-caps credential shape: default policy reads it in the @@ -2059,7 +2059,7 @@ def test_a_decomposed_abbreviation_reads_as_its_composed_twin( def test_a_decomposed_conjunction_reports_as_its_composed_twin() -> None: """#665: the one-case report on a letter that is both a connective - and an initial (rules.md#P3) counted code points, so it is reached + and an initial (rules/P3) counted code points, so it is reached by `й` only once a caller marks it ambiguous -- no default entry is decomposable.""" parser = Parser(lexicon=Lexicon.default().add( diff --git a/tests/v2/test_policy.py b/tests/v2/test_policy.py index 1b985eb4..ad0e0a8d 100644 --- a/tests/v2/test_policy.py +++ b/tests/v2/test_policy.py @@ -800,7 +800,7 @@ def test_unlisted_dotted_suffixes_is_a_validated_bool_defaulting_on() -> None: def test_unlisted_dotted_suffixes_off_reads_name_material() -> None: # The switch's whole behavior, both directions, on one name -- # "name material" for a token no chunk claims; the roman-chunk - # retirement (rules.md#S3) and real chunk-level vocabulary + # retirement (rules/S3) and real chunk-level vocabulary # ('Msc.Ed.', 'JD.CPA') are not behind this switch either way. from nameparser import Parser @@ -871,7 +871,7 @@ def test_unlisted_caps_suffixes_on_reads_an_all_caps_word() -> None: # assertions are about the TRAILING slot EVERYWHERE adds. Default # costs, same harness against master: 'Smith, John' 183 -> 183, # 'Smith, XYZ' 182 -> 182 (the comma test needs two words before - # the comma), 'John Smith, XYZ' 251 -> 266 (decisions.md#S2). + # the comma), 'John Smith, XYZ' 251 -> 266 (decisions/S2). from nameparser import Parser on = Parser(policy=Policy(unlisted_caps_suffixes=CapsSuffixes.EVERYWHERE)) @@ -886,7 +886,7 @@ def test_unlisted_caps_suffixes_on_reads_an_all_caps_word() -> None: assert default.parse("John Smith XYZ").ambiguities == () # the boundaries: one case, one letter, a digit, and vocabulary. # 'John Smith X' is NOT the single-letter control -- 'X' is a bare - # roman numeral (rules.md#S2's numeral fork) and reads as suffix + # roman numeral (rules/S2's numeral fork) and reads as suffix # 'X' with the switch either way, unrelated to this one; 'Z' is # not vocabulary at all and stays unaffected by the switch, which # is the actual boundary (a single capital never satisfies the diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index 48dd8712..38028ae1 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -185,9 +185,9 @@ def test_a_leading_ambiguous_particle_is_reported_once_and_only_once( #: with the rule that owns each one: a fourth exception fails, which is #: the whole point of carrying them by name rather than by count. _COMMA_AGREEMENT_EXCEPTIONS = { - "John Doe do": "rules.md#P6 attaches the lower-case particle member", - "John Q. Doe do": "rules.md#P6 attaches the lower-case particle member", - "Mary Jo Doe do": "rules.md#P6 attaches the lower-case particle member", + "John Doe do": "rules/P6 attaches the lower-case particle member", + "John Q. Doe do": "rules/P6 attaches the lower-case particle member", + "Mary Jo Doe do": "rules/P6 attaches the lower-case particle member", } @@ -258,13 +258,13 @@ def test_the_comma_agreement_exceptions_are_all_still_exceptions( #: Structural rather than a name list: a name whose OWN words are #: written in one case, where the member's own writing is the only #: contrast in the string, so the clause hides it. `own_words` stops -#: at the maiden marker (rules.md#P3), so a member inside the clause +#: at the maiden marker (rules/P3), so a member inside the clause #: cannot contribute the case contrast `one_case` is computed from -- -#: which is decisions.md#S2's own "THE PREDICATE KEEPS THE JUDGED +#: which is decisions/S2's own "THE PREDICATE KEEPS THE JUDGED #: TOKEN IN THE SPAN" (the 2026-09-14 #289/#516 entry) failing #: structurally, the judged token never being in the span at this #: slot. Accepted by Derek 2026-09-19 and recorded in -#: decisions.md#S2 as the M2 instance of #492's deferred question. +#: decisions/S2 as the M2 instance of #492's deferred question. #: The class is asserted beside the count because a structural #: allowlist cannot notice a new member of it, and a COUNT cannot #: notice a swap -- one pair leaving and another arriving keeps the @@ -279,7 +279,7 @@ def test_the_comma_agreement_exceptions_are_all_still_exceptions( #: #: The ANCHORED-HEAD class beside it (#544), pinned the same way: a #: head ending in an unambiguous credential anchors a member written -#: straight after it and not one written after a clause (rules.md#M2's +#: straight after it and not one written after a clause (rules/M2's #: boundary). Recorded 2026-09-27 on the #544 tree: 810 of the 20,412 #: pairs, 270 for each head ending in a credential ('Jane Doe Jr.', #: 'Jane Doe PhD', 'Doe, Jane PhD') and every one a Title-case member @@ -294,7 +294,7 @@ def test_the_comma_agreement_exceptions_are_all_still_exceptions( #: credential heads, 108 each, which the head's credential run (#602) #: took in. Then EMPTIED the same day by #601, and kept at 0 rather #: than deleted, so a member arriving fails here: a marker behind a -#: credential is an ordinary word now (rules.md#M2), the credential's +#: credential is an ordinary word now (rules/M2), the credential's #: run takes the whole clause in, and the clause form reads the member #: as a credential exactly as the plain form does. _MAIDEN_ANCHORED_HEAD_EXCEPTIONS = 0 @@ -315,7 +315,7 @@ def _one_case(text: str) -> bool | None: def _qualifying_front(toks: list[tuple[Token, tuple[int, int]]], i: int, original: str) -> tuple[Token, int] | None: - """rules.md#S2's company clause: walk back from `toks[i]` (a + """rules/S2's company clause: walk back from `toks[i]` (a member of the ambiguous class) through lone members to the piece that would speak for their company, and return it -- with its OWN index, which a caller's role question needs -- if it structurally @@ -323,7 +323,7 @@ def _qualifying_front(toks: list[tuple[Token, tuple[int, int]]], i: int, not a connective, not a particle, not a single-letter roman numeral, and neither a comma nor a maiden marker stands between it and `toks[i]` (the company is read within one comma part, and does - not reach across a clause, rules.md#M2). None if no such piece exists or it fails one + not reach across a clause, rules/M2). None if no such piece exists or it fails one of those tests. Says NOTHING about either token's ROLE -- not `toks[i]`'s, and not @@ -355,7 +355,7 @@ def _qualifying_front(toks: list[tuple[Token, tuple[int, int]]], i: int, def _reserve_kept(toks: list[tuple[Token, tuple[int, int]]], j: int, original: str) -> bool: - """Whether `toks[j]` is the reserve rules.md#H4's no-comma + """Whether `toks[j]` is the reserve rules/H4's no-comma carve-out keeps regardless of its own vocabulary: the name's first non-TITLE piece, in a name with NO COMMA anywhere before it, and no GIVEN/MIDDLE/FAMILY token before it either. 'PhD Ma' @@ -385,7 +385,7 @@ def _reserve_kept(toks: list[tuple[Token, tuple[int, int]]], j: int, def _outside_its_company(name: ParsedName) -> list[str]: - """rules.md#S2's company clause, read off a finished parse (#544): + """rules/S2's company clause, read off a finished parse (#544): a LISTED member of the ambiguous class written behind an unambiguous credential -- with nothing but listed members between them, in the same comma part and on the same side of a maiden @@ -421,7 +421,7 @@ def _outside_its_company(name: ParsedName) -> list[str]: ('Smith, PhD Ma') -- every head either has no comma at all or already carries a given name across it. That comma-side shape is pinned separately: `tests/v2/cases.py`'s `family_comma_lone_degree` - and `family_comma_two_credentials` rows, and rules.md#S2's + and `family_comma_two_credentials` rows, and rules/S2's 'Smith, PhD' boundary example. DELIBERATELY A SECOND IMPLEMENTATION of `_pieces.credential_anchors`, @@ -467,7 +467,7 @@ def _credential_without_suffix_role(name: ParsedName) -> list[str]: outside both TITLE and SUFFIX. Every other member is decided by role tests alone. - Skips a member whose OWN case-based lean (rules.md#S2, #289 -- an + Skips a member whose OWN case-based lean (rules/S2, #289 -- an ALL-CAPS member of a mixed-case name) already reads it as a credential with no company at all: that class predates #544 and reaches the very same H4 carve-out on its own ('PhD MA' -> given @@ -521,7 +521,7 @@ def test_a_maiden_clause_does_not_change_how_a_trailing_word_reads( #544 joins this walk rather than opening a grid: every parse it takes is also asked `_outside_its_company` and - `_credential_without_suffix_role` (rules.md#S2's company clause, + `_credential_without_suffix_role` (rules/S2's company clause, both directions), and the two heads ending in 'PhD' were added so that the given slot holds members behind a credential beside the comma-less trailing slot 'Jane Doe Jr.' already held. A third @@ -589,7 +589,7 @@ def side(name: ParsedName, word: str) -> str: # after it ('Jane Doe Jr. Ma' reads suffix 'Jr. Ma'), while the # clause puts its name words between the two ('Jane Doe Jr. nee # Smith Ma'), and the anchor never reaches across a clause - # (rules.md#M2, decided, a boundary). So the pair differs by + # (rules/M2, decided, a boundary). So the pair differs by # construction, and in ONE direction only: the clause reads a name, # the plain form a credential. Admitted only where the parser # itself gives the head's last token the suffix role and the pair @@ -683,7 +683,7 @@ def audit(label: str, text: str, name: ParsedName) -> None: #: The tags that make a token a member of the ambiguous credential #: class as the reader sees it -- the listed one and the by-shape one #: a 2.4 switch writes. Spelled here rather than imported so the -#: property is stated in the terms rules.md#M2 states it in, and so a +#: property is stated in the terms rules/M2 states it in, and so a #: rename in the pipeline cannot quietly narrow what this checks. _CLASS_TAGS = frozenset({"vocab:suffix-ambiguous", "shape:acronym"}) @@ -697,7 +697,7 @@ def audit(label: str, text: str, name: ParsedName) -> None: def _maiden_clause_grid() -> list[tuple[str, Parser, str]]: """(name, parser, policy label) for the M2 release grid. - Heads where a marker counts (rules.md#M2, #601): particle heads, a + Heads where a marker counts (rules/M2, #601): particle heads, a bound-given head and pair, a title-led head, a lone name word and a Vietnamese head whose surname is particle vocabulary, with by-shape and caps-on members, two adjacent particle members, mixed/ALL-CAPS/ @@ -737,7 +737,7 @@ def _maiden_clause_grid() -> list[tuple[str, Parser, str]]: def _released_but_not_suffix(text: str, parser: Parser) -> list[str]: - """Words rules.md#M2 says must be SUFFIX-roled and are not. + """Words rules/M2 says must be SUFFIX-roled and are not. The clause's parent-style reach is everything from the marker to the first CERTAIN suffix word -- suffix vocabulary that is not @@ -770,7 +770,7 @@ def _released_but_not_suffix(text: str, parser: Parser) -> list[str]: def test_a_word_the_clause_gives_up_lands_in_suffix() -> None: - """rules.md#M2's invariant, over the whole release grid. + """rules/M2's invariant, over the whole release grid. The clause may hand a word to the trailing rule that reads it as a credential, and it may keep the word. There is no third answer: @@ -815,7 +815,7 @@ def test_a_word_the_clause_gives_up_lands_in_suffix() -> None: def test_a_trailing_title_is_transparent_to_the_maiden_clause() -> None: - """rules.md#H5's transparency, asked where #535 found it broken: + """rules/H5's transparency, asked where #535 found it broken: a maiden clause ending '<run> Prof.' and one ending 'Prof. <run>' must read alike in every field but the order of the title words, wherever a trailing rule reads the clause -- no comma, the part @@ -830,7 +830,7 @@ def test_a_trailing_title_is_transparent_to_the_maiden_clause() -> None: in or ahead of the clause would take the title they differ too ('Jane van der Berg nee Smith PhD Prof.' against '... Prof. PhD', 'Jane Doe nee Smith do MA Prof.' against '... do Prof. MA') -- - rules.md#M2's Accepted pairs, which are examples rather than a + rules/M2's Accepted pairs, which are examples rather than a complete list. No head or run below has a particle, a bound given name or an empty name in or ahead of the clause. @@ -866,7 +866,7 @@ def test_a_trailing_title_is_transparent_to_the_maiden_clause() -> None: def test_a_title_the_clause_gives_up_lands_in_title() -> None: - """rules.md#M2's invariant for a title in the run (#535, #601): a + """rules/M2's invariant for a title in the run (#535, #601): a period-marked title word written after the marker ends the parse in the maiden name or in the title field, never in a name part. @@ -905,7 +905,7 @@ def test_a_title_the_clause_gives_up_lands_in_title() -> None: def test_a_title_first_word_counts_as_a_word() -> None: - """rules.md#M2's first-word floor, over two INPUTS: the floor is + """rules/M2's first-word floor, over two INPUTS: the floor is the title chain's own (`trailing_titles` takes it), so whether the first word after the marker IS title vocabulary ('King.') or not ('Smith') must not change what a trailing credential behind it @@ -1424,7 +1424,7 @@ def test_any_valid_config_still_parses_totally( assert isinstance(str(parsed), str) assert isinstance(parsed.capitalized().given, str) assert isinstance(parsed.initials(), str) - # rules.md#R4 under configuration: what this buys is COVERAGE of + # rules/R4 under configuration: what this buys is COVERAGE of # the non-mask clauses (shape-acronym, numeral, plain word) across # drawn vocabularies and policies -- it falsifies on e.g. a clause # that strips a period -- not DETECTION of a mask substitution. @@ -1639,7 +1639,7 @@ def _dropped_rows(texts: list[str], lexicons: _Variants) -> list[ @functools.cache def _class_letters(lexicon: Lexicon) -> frozenset[str]: - """The class rules.md#P3's both-sides condition is about: a + """The class rules/P3's both-sides condition is about: a connective that is ALSO generational vocabulary. NO LENGTH TEST, matching the rule's own scope and the stage's @@ -1708,7 +1708,7 @@ def _readmitted(tok: Token) -> bool: def _has_something_to_join(tok: Token, part: tuple[Token, ...]) -> bool: - """rules.md#R3's question, asked of the WHOLE PART. + """rules/R3's question, asked of the WHOLE PART. Working particles are set aside -- a particle the unjoined mark has NOT readmitted is doing a particle's work and is no name word @@ -1745,7 +1745,7 @@ def _record_contradicting_reports(name: ParsedName, label: str, text: str, """INV10 (#397 second review). Two statements of one rule, the second wider than the first and both about the same defect. - rules.md#A1's own sentence, that a report names the reading the + rules/A1's own sentence, that a report names the reading the parse took. The connective-or-initial fork offers a connective and an initial and says which it took, so a token the parse roles SUFFIX or TITLE @@ -2244,7 +2244,7 @@ def _initial_spans(name: ParsedName, def _initial_reading_moved(on: ParsedName, off: ParsedName, letters: frozenset[str]) -> bool: """Whether the two parses disagree about a class letter being an - INITIAL -- rules.md#P3's marked-subset clause rather than its + INITIAL -- rules/P3's marked-subset clause rather than its both-sides one, and the one thing an off-switch comparison cannot hold fixed: taking the word out of the connectives decides that question too, in either direction. A marked letter in a one-case @@ -2303,7 +2303,7 @@ def _off_switch_findings() -> dict[str, list[str]]: continue on = parser.parse(text) # INV6b's rows, and its scope: the MARKED subset. Its claim - # is decisions.md#P3's own -- a one-case name reads the + # is decisions/P3's own -- a one-case name reads the # marked letter as an initial and so reads as its ALL-CAPS # twin already did -- and a class letter the caller's lexicon # leaves UNMARKED is the shape P3's Accepted block records @@ -2434,7 +2434,7 @@ def _a_case_reading_the_docs_already_own( e i' against 'CAROD E I' are the same pair. TWO: a member of the AMBIGUOUS credential class reading - differently in the two spellings. That is decisions.md#S2's own + differently in the two spellings. That is decisions/S2's own lean -- an ALL-CAPS bare member of the class reads as the credential and a lower-case one as a name word -- and it decides the COMMA STRUCTURE, so its effect reaches every token of the @@ -2546,7 +2546,7 @@ def test_a_trailing_credential_never_joins_into_a_name_part() -> None: def _placed_as_a_connective(on: ParsedName, letters: frozenset[str]) -> bool: """Whether the parse put a class letter among the NAME words as a - connective -- rules.md#R4's own reading, and the one thing an + connective -- rules/R4's own reading, and the one thing an off-switch comparison of case repair cannot hold fixed: a connective keeps its lowercase there and the off-switch parse, where the letter is no connective at all, capitalizes it. Asked @@ -2586,7 +2586,7 @@ def test_a_letter_that_did_not_join_repairs_as_the_off_switch_does() -> None: Three exemptions. The first two are INV6's -- a name whose FIELDS moved is a name the join changed, and a letter the two parses disagree about being an INITIAL is the marked-subset clause. The - third is rules.md#R4's own sentence: a connective the parse + third is rules/R4's own sentence: a connective the parse placed among the name words keeps its lowercase there, which the off-switch parse cannot agree with, since for it the letter is not a connective at all. What is left is the generation, and the @@ -2610,7 +2610,7 @@ def test_a_letter_that_did_not_join_repairs_as_the_off_switch_does() -> None: def test_a_one_case_name_reads_the_same_in_either_case() -> None: - """INV6b (#397 second review). decisions.md#P3 states in prose + """INV6b (#397 second review). decisions/P3 states in prose that a name written wholly in lower case now reads as its ALL-CAPS twin already did, and INV6 cannot hold it: the `_initial_reading_moved` exemption there swallows the one-case @@ -2663,7 +2663,7 @@ def test_a_one_case_name_reads_the_same_in_either_case() -> None: # any of these names, so removing `i` from the connectives changes # nothing and the comparison would be vacuous -- the silence a # reachability probe exists to catch. The oracle is the RULE instead, -# the RULE instead, rules.md#R4 read in its own words: a suffix-roled +# the RULE instead, rules/R4 read in its own words: a suffix-roled # connective that the suffix vocabulary does NOT hold was read as no # generation and keeps its lowercase. # The literal strings the released wheels give are pinned separately, @@ -2827,14 +2827,14 @@ def _link_texts() -> dict[str, str]: def _link_joins_inside_the_clause(maiden: str) -> bool: """Whether the ORACLE parse put a link inside the birth name with - a birth-name word on each side of it -- rules.md#P3's both-sides + a birth-name word on each side of it -- rules/P3's both-sides condition, read off the `y` spelling's own maiden field. The guard, and it is narrower than "the twin keeps the link" for a measured reason: 'Jane Doe nee Puig y' keeps its `y` in maiden 'Puig y', and 'Jane Doe nee Puig i' reads maiden 'Puig' with suffix 'i' -- deliberately, the link there joining nothing and - being the generation it also spells (rules.md#M2). So a guard + being the generation it also spells (rules/M2). So a guard asking only whether the twin kept the letter would demand the two agree where the rules say they must not. `Puig y III` and `y Soler` are the same shape from the other two sides. @@ -2905,7 +2905,7 @@ def test_a_clause_link_reads_as_its_y_twin_does() -> None: the one the maiden walk was never able to get wrong; `i` is that same connective AND the roman numeral, and the walk used to end the birth name at it. The pair is the whole statement of - rules.md#M2's link clause, and it needs no expected values of its + rules/M2's link clause, and it needs no expected values of its own. Mutation-checked, 2026-09-20: it fails on all 10,540 guarded rows @@ -3004,7 +3004,7 @@ def _case_only_violations() -> list[str]: def test_case_repair_changes_case_and_nothing_else() -> None: - """rules.md#R4 read as an invariant: a repaired token equals its + """rules/R4 read as an invariant: a repaired token equals its input under str.casefold(), on both surfaces, plain and forced, under the shipped lexicon and under one whose exceptions map is empty. casefold(), not a character-for-character check, is the @@ -3016,7 +3016,7 @@ def test_case_repair_changes_case_and_nothing_else() -> None: shape under re.I admits the dotless 'ı', which upper()s to 'I', and 'I' does NOT fold back to 'ı', so a name spelling a numeral with it would fail this walk; none in the walk does. Both cases - are decisions.md#R4's Unicode boundary. + are decisions/R4's Unicode boundary. Recorded negative control, measured 2026-09-23 over this walk: at the parent 4d0680e6, where the exceptions map SUBSTITUTED its @@ -3113,7 +3113,7 @@ def nfc(text: str) -> str: def test_a_decomposed_name_renders_as_its_composed_twin() -> None: - """rules.md#R3 and #R4 read as an invariant over encodings: a + """rules/R3 and #R4 read as an invariant over encodings: a letter written as a base letter and its combining accent initials (#585) and repairs (#542) as the letter written whole does. Core and facade; repair plain and forced. The output is compared @@ -3202,7 +3202,7 @@ def _comma_twin_findings(parser: Parser) -> tuple[list[str], int]: def test_a_delimiter_core_in_a_tail_reads_as_its_comma_twin() -> None: - """rules.md's C1: "a delimiter the policy declares parts a trailing + """rules/C1's statement: "a delimiter the policy declares parts a trailing suffix part as a comma would" (#549). Every role field of a name with a declared core standing in a part after the suffix comma, and every kind it reports but `comma-structure` (`_twin_record`), @@ -3226,7 +3226,7 @@ def test_a_delimiter_core_in_a_tail_reads_as_its_comma_twin() -> None: # A one-letter word written with its period is an initial and not a -# particle (rules.md#P7), so a name holding 'Ó.' must read exactly as +# particle (rules/P7), so a name holding 'Ó.' must read exactly as # the same name holding any other initial does. Two INPUTS, # not two views of one parse (docs/design/AGENTS.md axis 11): the rule # is not consulted, so the test can find the rule wrong as well as the @@ -3303,7 +3303,7 @@ def test_a_one_letter_particle_with_its_period_reads_as_any_initial() -> None: #: some, so none is decoration. _render's is measured without the #: particle case masks, as for a caller's one-letter particle that has #: none: with the shipped 'ó' mask the repair takes the mask before -#: the particle arm is asked (rules.md#R4), so the default vocabulary +#: the particle arm is asked (rules/R4), so the default vocabulary #: alone cannot see that veto. _P7_SITE_EFFECT = { "nameparser._pipeline._classify": 404, diff --git a/tests/v2/test_render.py b/tests/v2/test_render.py index e3885eae..448143bc 100644 --- a/tests/v2/test_render.py +++ b/tests/v2/test_render.py @@ -154,7 +154,7 @@ def test_initials_skips_tagged_particles_outside_given() -> None: def test_repair_keeps_a_conjunction_lowercase_in_a_particle_part() -> None: - """rules.md#R4's conjunction carve-out, which no shipped name reaches. + """rules/R4's conjunction carve-out, which no shipped name reaches. The unjoined mark makes the words of an all-particle part ordinary name words, since none of them is acting as a particle @@ -163,7 +163,7 @@ def test_repair_keeps_a_conjunction_lowercase_in_a_particle_part() -> None: a part". Witnessing it needs a word in `particles` AND `conjunctions`; the shipped sets are disjoint, in the default vocabulary and in every locale pack, so no input string can - witness it and rules.md can carry no example line for it. Gating + witness it and the rules can carry no example line for it. Gating the conjunct on the mark passed the entire suite until this test existed. @@ -187,7 +187,7 @@ def test_repair_keeps_a_conjunction_lowercase_in_a_particle_part() -> None: def test_initials_readmits_a_conjunction_in_a_particle_part() -> None: - """rules.md#R3, settled (#461). + """rules/R3, settled (#461). This pinned TODAY's output on an open question so that settling it would fail the suite until the pin moved with it. It fired, and @@ -197,14 +197,14 @@ def test_initials_readmits_a_conjunction_in_a_particle_part() -> None: base. The all-particle rows below are unchanged, because R2's mark already readmitted the word there; what moved is everything under the default vocabulary, where a lone connective now initials too. - decisions.md#R3 records the rule and what it costs. + decisions/R3 records the rule and what it costs. - Still not a `deviates:` marker and still not a rules.md example + Still not a `deviates:` marker and still not a rules example line: markers hang on example lines and every line there names an input parsed with the DEFAULT vocabulary, over which `particles` and `conjunctions` are disjoint, so no string reaches the all-particle-with-a-connective shape. It is also what keeps the - values quoted in prose by decisions.md, mechanisms.md and + values quoted in prose by decisions/, mechanisms.md and `_render.py` from going stale unnoticed. """ assert "y" in Lexicon.default().conjunctions, ( @@ -244,7 +244,7 @@ def test_initials_readmits_a_conjunction_in_a_particle_part() -> None: def test_repair_keeps_a_lone_connective_lowercase_where_it_initials( ) -> None: - """rules.md#R4's own reading, and the view split it accepts. + """rules/R4's own reading, and the view split it accepts. A connective that initials BECAUSE it joins nothing is still not written the way a name is written, so case repair leaves it @@ -252,7 +252,7 @@ def test_repair_keeps_a_lone_connective_lowercase_where_it_initials( rather than a borrowing from R3 -- the two rules answer different questions about the same token and this is the row where their answers part. mechanisms.md#RENDER-HONORS-THE-PARSE records the - shape; decisions.md#R4 records the split. + shape; decisions/R4 records the split. """ assert parse("Juan de y").initials() == "J. y." assert parse("Juan de y").capitalized().family == "de y" @@ -262,7 +262,7 @@ def test_repair_keeps_a_lone_connective_lowercase_where_it_initials( def test_repair_capitalizes_a_generation_the_connective_also_spells( ) -> None: - """rules.md#R4's other half (#397 review), and the row that made + """rules/R4's other half (#397 review), and the row that made the sentence need two clauses. A word can be the connective and the generation at once -- 'i' is @@ -300,7 +300,7 @@ def test_repair_capitalizes_a_generation_the_connective_also_spells( def test_repair_keeps_a_plain_connective_the_suffix_field_holds_lower( ) -> None: - """rules.md#R4, the GENERATION half read narrowly (#397 second + """rules/R4, the GENERATION half read narrowly (#397 second review). The clause turns on the generation and not on the field: the test is the suffix ROLE **and** the suffix VOCABULARY, and a role test alone capitalized every plain connective that merely @@ -557,7 +557,7 @@ def test_capitalized_preserves_mixed_case_unless_forced() -> None: def test_the_gate_leaves_the_suffixes_out() -> None: - """rules.md#R5 (#492): a credential or a generation written the + """rules/R5 (#492): a credential or a generation written the way one is written ('III', 'PhD', 'Jr.') is no evidence about how the NAME was cased, so the one-case test reads every token but the SUFFIX-roled ones. Titles stay in -- a cased title still holds @@ -604,7 +604,7 @@ def test_the_gate_leaves_the_suffixes_out() -> None: assert str(parse("김민준 Phd").capitalized()) == "민준 김 Phd" assert str(parse("김민준 PHD").capitalized()) == "민준 김 PhD" # no non-suffix token at all -- a synthetic name only, since every - # parse names somebody (rules.md#H4): the empty text is one-case + # parse names somebody (rules/H4): the empty text is one-case only_suffixes = _pn("phd md", [ Token("phd", Span(0, 3), Role.SUFFIX), Token("md", Span(4, 6), Role.SUFFIX), @@ -615,7 +615,7 @@ def test_the_gate_leaves_the_suffixes_out() -> None: def test_a_suffix_written_in_more_than_one_case_is_kept_as_written( ) -> None: - """rules.md#R5 (decided 2026-09-24): the gate leaves the suffixes + """rules/R5 (decided 2026-09-24): the gate leaves the suffixes out because a cased suffix says nothing about the NAME, and read to its end that also means repair has no business re-spelling one. So where repair was not forced, a SUFFIX token written in @@ -654,8 +654,8 @@ def test_a_suffix_written_in_more_than_one_case_is_kept_as_written( def test_the_gate_and_the_parser_read_a_cased_suffix_differently() -> None: - """decisions.md#R5's split: the parser's own one-case readings - (rules.md#P3, #S2) still count a cased suffix, while R5's gate + """decisions/R5's split: the parser's own one-case readings + (rules/P3, #S2) still count a cased suffix, while R5's gate leaves it out. So 'john e jones III' is MIXED to the parser, which reads its 'e' as a connective, and one-case to the gate, which repairs it -- keeping the connective lowercase -- where the one-case @@ -697,7 +697,7 @@ def _repaired_under(pairs: tuple[tuple[str, str], ...], text: str, *, def test_a_mask_recases_the_word_as_the_writer_punctuated_it() -> None: - """rules.md#R4's mask (#459): an exceptions-map value is its key's + """rules/R4's mask (#459): an exceptions-map value is its key's letters in the case each takes, laid over the word as written -- one entry covers every punctuation of the word, and repair adds and removes nothing.""" @@ -740,7 +740,7 @@ def test_a_word_that_left_the_map_repairs_by_the_role_it_took() -> None: and numeral clauses write them in capitals -- keeping whatever punctuation the writer used, which the map's substitution did not. A word the parse put in a NAME role is repaired as a name - word whatever vocabulary holds it (decisions.md#R4's given-role + word whatever vocabulary holds it (decisions/R4's given-role half), which is also how the abbreviated Mohammed is written.""" assert str(parse("john smith md").capitalized()) == "John Smith MD" assert str(parse("john smith m.d.").capitalized()) == "John Smith M.D." @@ -758,7 +758,7 @@ def test_a_word_that_left_the_map_repairs_by_the_role_it_took() -> None: def test_a_credential_read_by_its_shape_repairs_to_capitals() -> None: - """rules.md#R4, #516's by-shape half (#459): a suffix classify + """rules/R4, #516's by-shape half (#459): a suffix classify admitted to the credential class by its dotted shape alone carries SHAPE_ACRONYM_TAG and no vocabulary entry, and repairs as a listed acronym does.""" @@ -784,7 +784,7 @@ def test_a_credential_read_by_its_shape_repairs_to_capitals() -> None: def test_a_suffix_numeral_repairs_to_capitals_by_its_shape() -> None: - """rules.md#R4 (#459): vi through x carry no vocabulary tag and + """rules/R4 (#459): vi through x carry no vocabulary tag and title-cased to 'Vi'/'Ix'; the numeral clause reads the roman shape of a SUFFIX-roled word. xi and up are no suffix to the parse (its roman shape stops at x), and a numeral after a family @@ -837,7 +837,7 @@ def test_the_mask_keeps_every_non_letter_and_declines_a_miscount() -> None: def test_a_split_initial_is_capitalized_only_where_the_mask_keeps_it_joined( ) -> None: - """rules.md#R4 (#459 review, narrowed): a letter written alone + """rules/R4 (#459 review, narrowed): a letter written alone beside a full stop is an initial ONLY where the MASK writes that same letter inside a run of two or more letters. Where the mask spells it alone too, the mask's own case stands ('h.c' on 'h.c.' @@ -1028,7 +1028,7 @@ def test_a_mark_with_no_letter_before_it_heads_no_word() -> None: def test_a_masks_upper_fallback_can_lengthen_a_word_through_ss() -> None: - """decisions.md#R4 (2026-09-24 review): where the whole-word + """decisions/R4 (2026-09-24 review): where the whole-word upper-casing does not keep the word's length, _apply_mask falls back to a per-character `c.upper()`, which lengthens 'ß' to 'SS' as the acronym/numeral clauses' `word.upper()` does -- whether @@ -1044,7 +1044,7 @@ def test_a_masks_upper_fallback_can_lengthen_a_word_through_ss() -> None: def test_a_lengthening_mask_is_not_a_fixpoint_under_a_second_forced_pass( ) -> None: - """decisions.md#R4 (2026-09-24 review, sub-clause (a)): capitalized()'s + """decisions/R4 (2026-09-24 review, sub-clause (a)): capitalized()'s own docstring claims every clause is a fixpoint, so a repaired name comes back unchanged if repaired again -- true everywhere except this one boundary. The FIRST forced pass over 'a.ß' under the @@ -1199,7 +1199,7 @@ def test_capitalized_lowers_the_words_the_parse_tagged_conjunction() -> None: def test_a_link_inside_a_hyphenated_word_keeps_its_lowercase() -> None: - """rules.md#R4 (#478): inside one hyphenated word, a part that is + """rules/R4 (#478): inside one hyphenated word, a part that is connective vocabulary with a worded part on each side of it keeps its lowercase, in every role. At either END it is ordinary name text -- #458's answer, kept -- and a word outside the connective @@ -1249,7 +1249,7 @@ def test_a_link_inside_a_hyphenated_word_keeps_its_lowercase() -> None: mixed = parse("Jose Ortega-Y-Gasset") assert mixed.capitalized() == mixed assert str(mixed.capitalized(force=True)) == "Jose Ortega-y-Gasset" - # not connective vocabulary, so not reached (decisions.md#R4) + # not connective vocabulary, so not reached (decisions/R4) assert "a" not in Lexicon.default().conjunctions assert str(parse("Te Awanui-a-Rangi Black").capitalized( force=True)) == "Te Awanui-A-Rangi Black" @@ -1272,8 +1272,8 @@ def test_a_link_inside_a_hyphenated_word_keeps_its_lowercase() -> None: def test_the_hyphen_is_the_writers_join_even_in_a_one_case_name() -> None: - """rules.md#R4's hyphen clause against rules.md#P3's one-case fork, - a split DECIDED 2026-09-24 (decisions.md#R4): spaced, a marked + """rules/R4's hyphen clause against rules/P3's one-case fork, + a split DECIDED 2026-09-24 (decisions/R4): spaced, a marked letter in a name written in one case reads as an initial and repairs to a capital; hyphenated, the writer joined the surname on purpose, so the interior word is the connective whatever case the @@ -1303,7 +1303,7 @@ def test_the_hyphen_is_the_writers_join_even_in_a_one_case_name() -> None: def test_a_shipped_mask_spells_its_suffix_in_either_single_case() -> None: - """#459 (decisions.md#R4, 2026-09-24): the PROPERTY every shipped + """#459 (decisions/R4, 2026-09-24): the PROPERTY every shipped mask pair serves, derived from the pairs rather than restating them -- `john smith <key>` puts the key in the suffix role, and repair spells it as the mask whether it was written all lower or @@ -1327,7 +1327,7 @@ def test_a_shipped_mask_spells_its_suffix_in_either_single_case() -> None: def test_a_shipped_particle_mask_outranks_the_particle_lowercase() -> None: - """rules.md#R4, #604: case repair writes a particle in lowercase + """rules/R4, #604: case repair writes a particle in lowercase ('de la Vega') unless the vocabulary records a casing for it, as it does for the Irish particles. The property every shipped particle mask serves, derived from the pairs: `john <key> smith` puts the @@ -1349,7 +1349,7 @@ def test_a_shipped_particle_mask_outranks_the_particle_lowercase() -> None: def test_a_caller_particle_mask_outranks_the_particle_lowercase() -> None: - """rules.md#R4, #604, decided for CALLER maps too (decisions.md#R4, + """rules/R4, #604, decided for CALLER maps too (decisions/R4, 2026-10-04): 1.4.0 through 2.3.0 kept a particle lowercase whatever the map held. The control is the same name with no entry. Both surfaces.""" @@ -1367,7 +1367,7 @@ def test_a_caller_particle_mask_outranks_the_particle_lowercase() -> None: def test_a_listed_acronym_that_is_a_name_word_gets_no_mask() -> None: - """decisions.md#R4's Excluded block for CAPITALIZATION_EXCEPTIONS + """decisions/vocabulary-exclusions, the CAPITALIZATION_EXCEPTIONS block (meng, edd, lac, ded): a mask applies in every role, so an acronym that is also a name word must not carry one. The fork it protects: in a NAME role the word repairs as a title-cased name word, and in @@ -1377,7 +1377,7 @@ def test_a_listed_acronym_that_is_a_name_word_gets_no_mask() -> None: Since #540 `meng` and `lac` are ambiguous acronyms, so a bare trailing one after a lone given name is the FAMILY name - (decisions.md#suffix-acronym-collisions): `wang meng` repairs to + (decisions/suffix-acronym-collisions): `wang meng` repairs to `Wang Meng`, where the credential reading gave `Wang MENG`. With words to spare the credential reading, and its capitals, stay. That puts the trailing slot on both sides of the fork, and the @@ -1584,10 +1584,10 @@ def test_render_malformed_specs_surface_raw_format_errors() -> None: def test_capitalized_one_case_connective_that_reads_as_an_initial() -> None: - """#383/#479, the half no differential gate can see (decisions.md#R4). + """#383/#479, the half no differential gate can see (decisions/R4). Repair lowercases a CONJUNCTION even inside a part it otherwise - capitalizes (rules.md#R4's carve-out). Once a marked single letter + capitalizes (rules/R4's carve-out). Once a marked single letter in a one-case name is read as an INITIAL instead, that carve-out no longer reaches it and the letter capitalizes like any name word. The 'y' line is the control: it stays the connective, so it stays @@ -1608,11 +1608,11 @@ def test_facade_initials_follow_the_one_case_fork() -> None: The core's initials() follows the parse's tags: in a name written wholly in one case an 'e' is an INITIAL and a 'y' is the connective - (rules.md#P3), so R3's "each given, middle, and base family word" + (rules/P3), so R3's "each given, middle, and base family word" reaches the first and not the second. Until #528 HumanName.initials() re-derived that from the lexicon and the part's raw shape instead, and kept 1.4.0's answer on both letters; it now reads the same tags, - so the two views of one parse agree. decisions.md#R3 records it. + so the two views of one parse agree. decisions/R3 records it. """ assert parse("john e smith").initials() == "j. e. s." assert HumanName("john e smith").initials() == "j. e. s." @@ -1629,12 +1629,12 @@ def test_facade_initials_follow_the_one_case_fork() -> None: assert HumanName("John E Smith").initials() == "J. E. S." assert HumanName("Juan Y. Garcia").initials() == "J. Y. G." # 'maria y lopez' is a ONE-CASE control, not a mixed-case one: written - # wholly in lowercase, its 'y' is outside the marked set (rules.md#P3), + # wholly in lowercase, its 'y' is outside the marked set (rules/P3), # so it stays the CONNECTIVE on both surfaces rather than reading as # an initial the way 'e' does. What it no longer # witnesses is the DROP: #461 gave the letter its initial back here, # because it holds the middle part alone and so joins nothing - # (decisions.md#R3). The joining one-case control below is where + # (decisions/R3). The joining one-case control below is where # a lowercase 'y' still drops, which is what keeps this row's point # -- the tag, not the value -- observable. assert HumanName("maria y lopez").initials() == "m. y. l." @@ -1656,7 +1656,7 @@ def test_facade_initials_follow_the_one_case_fork() -> None: # The one corpus name where the two views still differ, and it is # not this rule's: the facade merges 'Ph.' + 'D.' into one list # element and renders it with no inner delimiter - # (fix(initials-per-word) the Ph. D. merge, decisions.md#phd-merge) + # (fix(initials-per-word) the Ph. D. merge, decisions/phd-merge) assert HumanName("Ph. D., John").initials() == "J. P D." assert parse("Ph. D., John").initials() == "J. P. D." @@ -1681,7 +1681,7 @@ def test_initials_separator_is_honored_on_the_live_token_path() -> None: def test_a_caps_credential_after_a_comma_keeps_its_capitals_in_every_setting( ) -> None: # A credential read by its capitals (S2) repairs to its all-caps - # spelling (R4 in rules.md). #564 made the comma reading a default + # spelling (rules/R4). #564 made the comma reading a default # decided in `segment` from the text, and classify's shape tag -- # what repair reads -- has to follow it there too, or the default # title-cases a credential EVERYWHERE keeps: the first draft gave @@ -1698,7 +1698,7 @@ def test_a_caps_credential_after_a_comma_keeps_its_capitals_in_every_setting( def test_a_marked_particle_lowercases_only_where_the_particle_arm_would( ) -> None: - """rules.md#R4, #700: a particle written with its mark ("d'", + """rules/R4, #700: a particle written with its mark ("d'", "al-", "'t") is asked the particle arm's question as a whole token, since the per-word walk splits it at the mark and asks the stem. Only the arm's lowercase is taken whole; wherever the arm declines, diff --git a/tests/v2/test_rules_doc.py b/tests/v2/test_rules_doc.py index bec234b9..015ddde0 100644 --- a/tests/v2/test_rules_doc.py +++ b/tests/v2/test_rules_doc.py @@ -1,4 +1,4 @@ -"""Executes every example line in docs/design/rules.md. +"""Executes every example line in docs/design/rules/. Wrong-on-arrival defense: each example runs against the live parser. ``deviates:`` examples assert TODAY's output (strict, like an xfail) @@ -15,14 +15,15 @@ from nameparser import Parser, parse, parser_for from nameparser import locales from nameparser._policy import Policy +from tests.v2.design_docs import read_rules_text from tests.v2.rules_doc import ( - RULES_DOC, Example, Rule, parse_rules_doc, resolve_annotation) + Example, Rule, parse_rules_doc, resolve_annotation) -RULES = parse_rules_doc(RULES_DOC.read_text(encoding="utf-8")) +RULES = parse_rules_doc(read_rules_text()) def test_doc_has_rules() -> None: - assert RULES, "rules.md holds no rules yet -- extraction not started" + assert RULES, "docs/design/rules/ holds no rules" @pytest.mark.parametrize("rule", RULES, ids=lambda r: r.rule_id) @@ -152,15 +153,13 @@ def test_example(rule: Rule, example: Example) -> None: "김. 민준", # H2's script veto on hangul typed as jamo (#665) ]) def test_the_nfd_examples_are_still_decomposed(decomposed: str) -> None: - """The decomposed example lines in rules.md are the only executable + """The decomposed example lines in the rules are the only executable witnesses of its encoding clauses (H2, P3), and an editor or formatter that NFC-normalizes the file turns each into the composed case, which always worked and still passes. Each entry here is typed decomposed, which the first assert checks so that normalizing this file cannot hollow the test out either.""" import unicodedata - from pathlib import Path assert not unicodedata.is_normalized("NFC", decomposed) - text = (Path(__file__).parents[2] / "docs" / "design" / "rules.md" - ).read_text(encoding="utf-8") + text = read_rules_text() assert decomposed in text, f"the NFD example {decomposed!r} was recomposed" diff --git a/tests/v2/test_rules_doc_grammar.py b/tests/v2/test_rules_doc_grammar.py index 768e5947..77c68bd7 100644 --- a/tests/v2/test_rules_doc_grammar.py +++ b/tests/v2/test_rules_doc_grammar.py @@ -1,9 +1,9 @@ -"""Unit tests for the rules.md example-line grammar parser. +"""Unit tests for the rules example-line grammar parser. DOC below is a SYNTHETIC fixture: it exists only to exercise the grammar and is never executed against the parser. Its rule IDs and content are illustrative (they echo the spec's worked examples for -readability) and deliberately do NOT track docs/design/rules.md — +readability) and deliberately do NOT track docs/design/rules/ — do not "fix" this fixture when the real doc changes. """ from __future__ import annotations @@ -13,7 +13,7 @@ from tests.v2.rules_doc import Example, parse_rules_doc DOC = '''\ -# Parsing rules (synthetic grammar fixture -- not the real rules.md) +# Parsing rules (synthetic grammar fixture -- not the real rules) ## Particles @@ -26,7 +26,7 @@ "Mesnil de" family-first → family="Mesnil de" "de" → given="de" · boundary Accepted: a bare "de" keeps given="de". - history: decisions.md#P2 · interacts: H1 · implemented: nameparser/_pipeline/_post_rules.py + history: decisions/P2 · interacts: H1 · implemented: nameparser/_pipeline/_post_rules.py P3. Statement with a tracked gap. "Swami Vivekananda" → family="" deviates: #346 (today: family="Vivekananda") diff --git a/tests/v2/test_snapshot.py b/tests/v2/test_snapshot.py index 7b5306ba..5e8988fc 100644 --- a/tests/v2/test_snapshot.py +++ b/tests/v2/test_snapshot.py @@ -36,7 +36,7 @@ def test_a_name_only_cldr_holds_is_counted_out() -> None: # A release figure counts CLDR's fictional renderings out (AGENTS.md, # Release-log claims); a name another corpus also holds stays in. # One of #653's movers, a CLDR rendering; 'Jane (Janey, Jo) Smith' - # moved with it and is a rules.md example only. 'マイケル' is in CLDR + # moved with it and is a rules example only. 'マイケル' is in CLDR # AND the CJK corpus, which is the half the subtraction decides. cldr_name = "Macáková, Alexandra Zuzana (pani, PhD)" shared = "マイケル" diff --git a/tools/differential/README.md b/tools/differential/README.md index 4d55ff81..d36c4afb 100644 --- a/tools/differential/README.md +++ b/tools/differential/README.md @@ -191,7 +191,7 @@ invocation shows it. | `corpus_wikidata.jsonl` | Wikidata, real public figures (deceased, or with at least 10 sitelinks) whose item states a second surname (P1950: Spain, Mexico, Colombia) or a patronymic (P5056: Russia, the Soviet Union, the Russian Empire, Iceland, Malaysia): each P1559 native-language name and own-language label whose every word aligns with exactly one stated part, via `build_wikidata_corpus.py` (2026-10-16); CC0, credit in `wikidata_LICENSE` | everything the strict alignment drops: short display forms without the second surname, nicknames, and names with an unrecorded part, so tidy names are over-represented; only those eight citizenships; no titles, credentials or suffixes | | `corpus_wikidata_native.jsonl` | Wikidata, real public figures (deceased, or with at least 10 sitelinks) with a name written in their own script: Hangul, Han and kana, Arabic (Syria, Egypt, Saudi Arabia, Iran), Hebrew, Georgian, Armenian, Greek, Cyrillic beyond Russian (Serbia, Bulgaria, Ukraine, Kazakhstan), Bengali, Devanagari and Thai, 20 strata sampled by country and language. Each person's first P1559 or own-language label whose every word, or every character in unspaced text, aligns with exactly one stated part, via `build_wikidata_native_corpus.py` (2026-10-16). CC0, credit in `wikidata_native_LICENSE` | everything the strict alignment drops, which in these scripts is most names: Arabic nasab chains (`بن` is no connective in Arabic script), `آل` family names and kunyas (most Arabic items state no family name), Chinese names whose given name has no item (most), Japanese names whose given-name item holds only a kana reading, and names whose parts are labelled only in Latin script (most Devanagari, Bengali and Thai items, so those strata are 10–25 names); transcriptions with the 间隔号; Urdu and Marathi entirely; no titles, credentials or suffixes | | `corpus_conventions.jsonl` | invented names patterned on naming conventions the other corpora barely cover -- no family name, religious names, positional patronymics, honorifics after the name -- from the hand-written table in `conventions.py`, via `build_conventions_corpus.py` (#686) | real names, and every convention nobody has written a row for: about one row per convention, so it shows that a convention moved, not how often | -| `corpus_rules.jsonl` | every example in `docs/design/rules.md`, via `build_rules_corpus.py` (#414) | anything the rules doc has no example for | +| `corpus_rules.jsonl` | every example in `docs/design/rules/`, via `build_rules_corpus.py` (#414) | anything the rules doc has no example for | | `corpus_shapes.jsonl` | shape-tagged rows of `tests/v2/cases.py`, via `build_shapes_corpus.py` (#468) | anything no one has tagged a row for | Each covers a blind spot of the others. `corpus.jsonl` is reproducible @@ -306,7 +306,7 @@ Every distinct CJK-bearing `text` in `tests/v2/cases.py`, `tolerated` rows included. The generated corpora (`corpus_cjk`, `corpus_conventions`, `corpus_rules`, `corpus_shapes`) are each pinned by `tests/v2/test_corpora.py` against their generator's selection, so a -case row, a rules.md example or a `conventions.py` row added without +case row, a rules example or a `conventions.py` row added without regenerating fails the suite instead of silently narrowing what the snapshot watches. @@ -398,7 +398,7 @@ So what is opt-in is neither the marker words nor the delimited path as a whole: it is the delimited path for content that does not announce itself. Since #335 a clause holding a word past its marker reads as the maiden name whichever bucket its pair sits in -(`rules.md#M3`), so `Jane Smith (née Jones)` needs no configuration +(`rules/M3`), so `Jane Smith (née Jones)` needs no configuration either. A word past the MARKER, not a second word in the clause: since #434 a marker may be a phrase, and `Maria Kowalska (z domu)` is a two-word clause that stays a nickname because both its words are the diff --git a/tools/differential/build_cjk_corpus.py b/tools/differential/build_cjk_corpus.py index 4dc6cb54..e71abdaf 100644 --- a/tools/differential/build_cjk_corpus.py +++ b/tools/differential/build_cjk_corpus.py @@ -18,7 +18,7 @@ row added for new CJK work enters the corpus by being written down in the table everyone already reviews (#298's 间隔号 forms arrived exactly this way). Rows marked `tolerated` (the 2026-09-01 CJK -demotion, rules.md#W3) are harvested like any other: the flag says +demotion, rules/W3) are harvested like any other: the flag says the reading is best-effort, not that nobody should see it move. Regenerate after editing CJK case rows: diff --git a/tools/differential/build_rules_corpus.py b/tools/differential/build_rules_corpus.py index cd4c9a1a..e63b5574 100644 --- a/tools/differential/build_rules_corpus.py +++ b/tools/differential/build_rules_corpus.py @@ -1,4 +1,4 @@ -"""Regenerate corpus_rules.jsonl from the examples in docs/design/rules.md. +"""Regenerate corpus_rules.jsonl from the examples in docs/design/rules/. A corpus with its own provenance (#414). Each corpus covers a blind spot the others have -- the roster, with each file's source and the @@ -44,7 +44,7 @@ The one example form that carries no name is skipped. A `[subject]` example names a policy or locale rather than a name string, so its -`text` is empty -- rules.md has three, and without the filter all +`text` is empty -- the rules have three, and without the filter all three collapse into a single "" that the doc never wrote. Skipping them is also why the promise above is honest: an example enters the corpus by being written down, and a form with nothing to enter is @@ -57,7 +57,7 @@ argues its edges explicitly, which is a reason to keep them rather than to drop them. -Regenerate after editing rules.md examples: +Regenerate after editing rules examples: uv run python tools/differential/build_rules_corpus.py @@ -75,16 +75,16 @@ ROOT = HERE.parents[1] sys.path.insert(0, str(ROOT)) +from tests.v2.design_docs import read_rules_text # noqa: E402 from tests.v2.rules_doc import parse_rules_doc # noqa: E402 OUT = HERE / "corpus_rules.jsonl" -RULES_DOC = ROOT / "docs" / "design" / "rules.md" def selected_names() -> list[str]: """Every distinct example text in the rules doc, sorted for a deterministic file.""" - rules = parse_rules_doc(RULES_DOC.read_text(encoding="utf-8")) + rules = parse_rules_doc(read_rules_text()) return sorted({example.text for rule in rules for example in rule.examples diff --git a/tools/differential/shapes.py b/tools/differential/shapes.py index 865c22b3..cd984d79 100644 --- a/tools/differential/shapes.py +++ b/tools/differential/shapes.py @@ -23,7 +23,7 @@ because they are order-less, but because the family-first (resp. source-order) reading is SCRIPT-carried rather than declared: a pure shape 6/7 string parses correctly under the DEFAULT policy already -(rules.md#W4, T2/T3), the same way DEFAULT_SCRIPT_ORDERS routes Han/ +(rules/W4, T2/T3), the same way DEFAULT_SCRIPT_ORDERS routes Han/ Hangul/Hiragana to family-first without a Policy saying so. A shape whose reading depends on the string's own script has nothing for `order` to name. diff --git a/tools/name_usage.py b/tools/name_usage.py index 87be7b17..7b8031e3 100644 --- a/tools/name_usage.py +++ b/tools/name_usage.py @@ -1,10 +1,10 @@ """How often is a word borne as a given name, and as a family name? WHY THIS IS IN THE TREE. Vocabulary claims are weighed by -decisions.md#vocabulary-collisions: a word the parser claims by +decisions/vocabulary-collisions: a word the parser claims by position (titles, never-given particles) is safe to claim outright only if it is not borne as a name IN THE POSITION THE CLAIM ACTS ON (C-i; -decisions.md#salutation-titles records the one TITLES-only exception, +decisions/salutation-titles records the one TITLES-only exception, decided on counts). That check was often run from memory, which misses what the data shows at once: `shehu`, a title until #660, is also an Albanian and Nigerian surname, with 561 bearers in the 2010 US Census. diff --git a/tools/perf/call_count.py b/tools/perf/call_count.py index b7270bea..0c0779f6 100644 --- a/tools/perf/call_count.py +++ b/tools/perf/call_count.py @@ -12,7 +12,7 @@ docs/design/AGENTS.md already asks for this: "Where an entry quotes something that drifts, give the one-liner that recomputes it." This is -that one-liner. Every number in decisions.md#parse-cost comes from a +that one-liner. Every number in decisions/parse-cost comes from a mode of this script, named beside it. uv run python tools/perf/call_count.py # both entry points