Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md

Large diffs are not rendered by default.

23 changes: 22 additions & 1 deletion docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1138,6 +1138,22 @@ Through 2.3 the undeclared reading was given ``RN``, family ``Jane
Smith`` and suffix ``CRNA``, so the delimiter was the only way to get
the name right.

A hyphen or a slash written between two credentials with no space needs
no declaring when every part is a listed credential that is not also a
name, because the word then reads as one credential (since 2.7). A part
that may be a name, such as ``MA``, keeps the word a name word, since a
hyphen also joins surnames, unless the word is written in capitals,
which :ref:`all-caps-credentials` reads on its own terms:

.. doctest::

>>> joined = parse("Jane Smith, RN/CRNA")
>>> joined.given, joined.family, joined.suffix
('Jane', 'Smith', 'RN/CRNA')
>>> joined = parse("Jane Smith, MA/PhD")
>>> joined.given, joined.family
('MA/PhD', 'Jane Smith')

.. _unlisted-credentials:

Credentials the vocabulary doesn't list
Expand Down Expand Up @@ -1253,7 +1269,9 @@ All-caps words
^^^^^^^^^^^^^^

``unlisted_caps_suffixes`` reads an unlisted word of two or more
capital letters, with no period in it. Its value is a
capital letters, with no period in it, and since 2.7 one of capitals
joined by a hyphen or a slash (``PU-PH``, ``MS-OMPT``) with a part of
two or more letters. Its value is a
:class:`~nameparser.CapsSuffixes`, because where capitals mean a
credential depends on a second convention: many records write the
SURNAME in capitals (``"Jean DUPONT"``, ``"DUPONT, Jean"``).
Expand Down Expand Up @@ -1282,6 +1300,9 @@ all-caps surname convention never writes capitals there:
'XYZ'
>>> parse("JOHN SMITH, XYZ").given
'XYZ'
>>> cred = parse("John Smith, PU-PH")
>>> cred.family, cred.suffix
('Smith', 'PU-PH')

``EVERYWHERE``
""""""""""""""
Expand Down
2 changes: 1 addition & 1 deletion docs/design/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Entry conventions:

- [S1 — a bracketed credential is not a nickname](S1-bracketed-credential-is-not-a-nickname.md)
- [S2 — a suffix word ends a name and never opens the string](S2-suffix-word-ends-a-name-and-never-opens-the-string.md)
- [S3 — credentials written with interior periods](S3-credentials-written-with-interior-periods.md)
- [S3 — credentials written in joined chunks](S3-credentials-written-in-joined-chunks.md)
- [S4 — the ambiguous acronym needs words to spare](S4-ambiguous-acronym-needs-words-to-spare.md)
- [S5 — the given part's trailing slot after a family comma](S5-given-parts-trailing-slot-after-a-family-comma.md)
- [S6 — the case signal at the suffix slot](S6-case-signal-at-the-suffix-slot.md)
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Examples name each field as the default order reads it; under the family-first o
- The comma's case contrast is carried by a word of the name before the comma that holds a capital, ends in a lowercase cased letter and is claimed by no wordlist as a non-name, so an all-caps record keeps its given name (`MÜLLER WEIß, HANS`, `LLOYD WEBBER, ANDREW PhD`) and a name written in lowercase reads the listing form (`john smith, XYZ` → given 'XYZ'); a lone two-letter word is never admitted at the comma (`García Márquez, MJ` → given 'MJ') — 2026-10-01 #564.
- Accepted: a given name written in capitals behind a two-word surname reads as the credential (`García Márquez, GABRIEL` → suffix 'GABRIEL', reported), OFF being the escape; and case repair follows the reading, a caps word in the part a suffix comma opened being tagged under AFTER_COMMA, one in a third or later part only under EVERYWHERE — 2026-10-01 #564.
- Under EVERYWHERE the cost is the three-word name (`Jean Pierre DUPONT` → family 'Pierre', suffix 'DUPONT'), while a two-word name keeps its family and reports (`Jean DUPONT`); unlisted is checked by wordlist membership rather than tags (`John Smith NEE` keeps family 'NEE'), and a roman numeral is the numeral fork's before the shape is asked (`John Smith VI` → suffix 'VI') — 2026-09-14 (Derek), corrected 2026-09-18.
- The caps shape has a joined spelling: an unlisted word of capitals whose letter chunks are joined by a hyphen or a slash, one of them of two or more letters, joins the class wherever the caps shape is admitted, gated as it is (`John Smith, PU-PH` → suffix 'PU-PH', reported; `JOHN SMITH, PU-PH` and `Smith, PU-PH` → given; `John Smith, A-B` and `John Smith, Pu-Ph` → given), and under EVERYWHERE at every suffix position as the unjoined word is (`Jean Pierre DUPONT-MORETTI` → suffix) — 2026-10-25 #752.
- Neither switch reaches the v1 shim, which translates neither: `HumanName` follows both defaults and a v1 caller cannot turn them off (`HumanName("John Smith X.Y.Z.").suffix` and `HumanName("John Smith, XYZ").suffix` are 'X.Y.Z.' and 'XYZ'), the caps half since the AFTER_COMMA default — 2026-09-14 (Derek), corrected 2026-09-18; 2026-10-01 #564.
- A rule's statement names no `Policy` field; the example annotations carry the field names — 2026-09-18, RULES.MD STATEMENTS.
- An example's text is a default-policy corpus name whatever its annotation, so an annotated-only example is checked for a default-policy diff like any other — 2026-09-18, corrected.
Expand Down Expand Up @@ -50,3 +51,4 @@ Open: none in this file.
COST, `sys.setprofile` call counts against `git archive origin/master`: ordinary and listed-credential comma names unchanged, `John Smith, XYZ` 251 → 266, and an all-caps record with a capitalized given name and a credential (`LLOYD WEBBER, ANDREW PhD`) a constant +26 (the second draft's per-character contrast test had made `'LLOYD'*64 + ' WEBBER, ANDREW PhD'` +350), the contrast linear in the words before the comma and constant in a word's letters, all paid only once a caps word is in hand (PR #577). The comma test is on by default, so three C-level checks go before any call: two or more words before the comma (the first draft lacked it, and `Smith, JOHN`, a common record format, paid +25 for a flip it can never make — the code review), the first word in capitals, and the lone-two-letter length. Every caller of the caps predicate asks first, in C, what it would decline anyway — alphabetic capitals, not a listed suffix word — so a listed credential never pays for the call (the review of the first fix found `John Smith, CPA` +4 and `John Smith, Ph. D.` +5 before these).
- 2026-10-17 #682 — SPLIT FROM decisions/S2 with the rule. The entries above were written under decisions/S2 while this rule was part of rules/S2, and moved here unchanged but for pointers the move broke; where one says rules/S2 it means the rule as it then stood, which held this one.
- 2026-10-24 #864 — PAIRED INITIALS LEAVE THE CLASS AT THE TRAILING SLOTS. The 2026-09-30 #563 entry's choice (iii) left those slots with #516's reading; for paired initials of a script with capitals that question is answered under decisions/S3, the rule whose position reading took them, and this bullet is S9's pointer to it.
- 2026-10-25 #752 — THE CAPS SHAPE'S JOINED SPELLING (phase 2d, the unlisted half of item 5 of decisions/vocabulary-model's 2026-10-23 #752 entry, agreed by Derek on 2026-10-23; #797's half A). An unlisted word of capitals in letter chunks joined by `-` or `/`, one chunk of two or more letters, is the caps shape: `John Smith, PU-PH` reads suffix 'PU-PH' with the class's report where 1.4.0 through 2.6.0 read given 'PU-PH', family 'John Smith'. It joins the class exactly where and as the unjoined word does, so the C4 count and C6's case contrast gate it after a comma by default (`JOHN SMITH, PU-PH`, `Smith, PU-PH` → given) and EVERYWHERE takes it at every suffix position, its cost the unjoined word's (`Jean Pierre DUPONT-MORETTI` reads as `Jean Pierre DUPONT` does); keeping it to the comma under EVERYWHERE too was not built, since it would make one shape two by position. "Unlisted" is the whole word in no wordlist, as for the unjoined word; a word whose every chunk counts as a suffix as written is rules/S3's joined credential, read unambiguously before this shape is asked (`MD-PHD`), so the two overlap in no reading. Single-letter chunks alone are how initials are joined (`A-B`) and stay out, and a chunk holding a digit or a period is no letter chunk. REACHABILITY: the comma's candidate list, C12's case-repair marks and classify under EVERYWHERE each ask `isalpha()` and `isupper()` in C before calling the predicate, and the joined spelling fails `isalpha()`, so each pre-filter widened with it to a capitals word holding a joiner; the 2026-10-23 measurement's first run moved nothing for the same reason. MEASURED (2026-10-25, against the phase 2c commit 15d5426f, every local corpus recorded from it with `--tree`): `tools/differential/snapshot.py` moves nothing but the rules' own new examples; `cldr_agreement.py` matches every record; `local_agreement.py psc-individuals` matches its record; tse-donors and fec-individuals, none (FEC is written in one case, so the contrast never admits a word there); ctgov-officials, 529 of 352,104 entries. Of the 529, 519 are a joined capitals word after a comma, now the suffix with a suffix-or-name report (5 of them in a third part, where the comma-structure report stays): 516 had read it as the given name with the name reversed around it, and 3 had read a listed credential opening the part as the given name and the joined word as the middle (`John Smith, MSN ACNP-BC`), the whole part now the suffix. 10 move only in `capitalized(force=True)`, a joined capitals word already in a suffix run now keeping its capitals (rules/R4). The words are 104 distinct credentials and job titles, `PU-PH` (258) and `MCU-PH` (30), the French hospital-university ranks, and `MS-OMPT` (32) the commonest; none is a name. The 2026-10-23 figure, 759, was measured without phase 2c, which now reads about 230 of them as listed credentials. One of the 516 also takes a word before the comma into the suffix, as the unjoined twin reads: a capitals word between a listed credential and the comma (`Jean Claude MD DUPONT, PU-PH` → family 'Claude', suffix 'MD DUPONT, PU-PH', reported), where the part before the comma reads as the name without it (rules/C1) and `Jean Claude MD DUPONT` alone already gives suffix 'MD DUPONT' by rules/S8's run; ACCEPTED here, since it is S8's reading of a capitals surname behind a credential, the same with `XYZ` or `PhD` after the comma. COST: a capitals word holding a joiner pays the caps shape's path, as an unjoined capitals word already does (`SMITH-JONES, MARY-JANE` 175 → 189 profiler frames, py3.11), and a word with a lowercase letter or no joiner pays nothing new (`Mary Smith-Jones` 159, `John Smith, XYZ` 264, before and after). Pinned by the #752 rows of tests/v2/cases.py and `test_caps_shape_candidate_reads_joined_capitals` (tests/v2/pipeline/test_vocab.py).
2 changes: 2 additions & 0 deletions docs/design/decisions/vocabulary-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,3 +195,5 @@ Open: #752 (phase 2, derived ambiguity; its sub-phases 2a to 2m, each its own PR
- **One reading kept rather than widened: a one-letter numeral before a credential stays a name word** (decisions/S10's #721 entry, `John Smith V PhD` → family 'V', reported). Reading it with the count instead moved 6 tse-donors names, every one wrong: a lone `V` or `I` behind a surname or a particle before `JR` or `JUNIOR` (shape `Aa A Aa de V JUNIOR` read family 'de', suffix 'V JUNIOR'), where the letter abbreviates a surname; and 3 committed names (`John Smith V PhD`, `Jane Doe nee Smith V PhD`, `Jane Doe nee Smith V MA`). A split credential behind such a numeral (`John Smith V Ph. D.`) is left to the split-credential sub-phase (item 6), which must keep its suffix 'V Ph. D.' by its own rule.
- **Measured** (2026-10-25, against master 2af8da8e, each local baseline re-recorded from that checkout with `--tree` immediately before its check, since the shared cache's `snapshot.jsonl` is overwritten by any session measuring in parallel): the committed snapshot moves 6 names, every one intended: the three GND regnal rows (`Georg VI.`, `Konstantin VII.`, `Bonifatius VIII.`, family → suffix), `Smith VI, AD` (family 'Smith VI' → 'Smith', suffix 'VI'), and #774's `John Smith V, M.J.` and `John Smith VI, M.J.`. GND agreement goes from 618 to 621 of 801 on family and 779 to 782 on suffix, three progress rows each, and every other agreement record matches. ctgov-officials moves 1 reading (a job title ending in a level before a comma, `Aa Aa I, A` → suffix 'I', reported); fec-individuals 22, each a numeral ending the given part or the family part (`A, A VI` → suffix 'VI'; `A VI, A` → suffix 'VI'; `A A V, A` → suffix 'V'), about 14 read as generations and about 8 a middle name or a fragment read as one, all reported; tse-donors none. psc-individuals (1,039,171 records scored): 3 fixed, all Vietnamese names ending in the surname `Vi` that the Title-case lean keeps a name; 1 broken, a Title-case `Viii` written as the bearer's generation, which the lean reads as a name (the lean's cost, rules/S10); 1 changed, a record whose title box the scorer sets aside. Spike figures above (listing `vi` to `x`: 152 FEC moves) are the 2026-10-23 entry's and were not re-taken.
- **Recompute:** `uv run python tools/differential/snapshot.py` on the parent of this change's merge with its code, and for each local corpus `snapshot.py --local <source> --record --tree <base checkout>` then `snapshot.py --local <source>` from this tree, back to back; `uv run python tools/differential/local_agreement.py psc-individuals` the same way, `--list N` printing the moved records locally.
- **2026-10-25 #752 — phase 2c: a credential joined by a hyphen or a slash reads as one where every chunk counts as a suffix as written**, carrying out the listed half of item 5 of the 2026-10-23 #752 entry. `John Smith, MD/PhD` reads suffix 'MD/PhD' where it read given 'MD/PhD', family 'John Smith'; a bare member of the ambiguous class is no chunk the test counts (`John Smith, MA/PhD` keeps its reading). The record, with what was measured and what was left unread, is decisions/S3's entry of this date (rules/S3).
- **2026-10-25 #752 — phase 2d: an unlisted word of capitals joined by a hyphen or a slash is the caps shape**, carrying out the unlisted half of item 5 of the 2026-10-23 #752 entry (#797's half A). `John Smith, PU-PH` reads suffix 'PU-PH', reported, where it read given 'PU-PH', family 'John Smith'; C6's case contrast gates it after a comma as it gates the unjoined word. The record is decisions/S9's entry of this date (rules/S9, rules/C6).
1 change: 1 addition & 0 deletions docs/design/mechanisms.md
Original file line number Diff line number Diff line change
Expand Up @@ -432,6 +432,7 @@ A second axis cuts across the jobs: whether the names are real or invented, labe

- Test a function INSIDE a stage on the inputs its caller hands it, not on the stage's finished output. #610's first sweep test built pieces by running `group` whole and handed them to the maiden take's candidate function, but the take runs before any join, so the test read `de VI Prof.` as one piece where production hands it three: a guard checking a structure production never passes in, green for the wrong reason. Review moved the oracle to the take itself by wrapping it (tests/v2/pipeline/test_pieces.py), and recorded the old loop's misses there as the negative control (432), which is what shows the moved harness can still see a difference.
- Enumerate the rules that BUILD a structure; do not recall them. #395's unit walk was written three times in one PR — P2's chain missing, then the conjunction and bound-given branches absorbing one token where the particle branch absorbed a unit, then the suffix stop — and each miss came from listing the joining rules from memory instead of reading them out of the rules. Both later misses reproduced the very defect the first fix had just removed, mirrored.
- Widening a predicate widens nothing its callers filter out first. A predicate on a hot path often sits behind a C-level pre-check at each call site, a necessary half of it inlined so a word that cannot pass pays no frame (`isalpha()` and `isupper()` ahead of `caps_shape_candidate`), and a widened predicate behind an unwidened pre-check is unreachable: #752's joined capitals (`PU-PH`) failed `isalpha()` at all three call sites, and the 2026-10-23 measurement of the change first moved nothing. Before measuring a widened predicate, grep its call sites for the pre-check and widen each with it, and hold one input the change must move.
- Prefer monkeypatching a module-level helper to re-exec'ing a module: a caller looks a module global up at CALL time, so replacing `_units` or `_name_positions` reaches it and none of the traps below apply. Re-exec is only needed for logic INLINE in a function, and "the thing I want to mutate is not a module-level helper" is a hint that it could be one.
- Re-exec'ing a module redefines the classes that module DEFINES, not the ones it imports — the exec re-runs its own import statements, which resolve through sys.modules. So `is` comparisons break selectively: re-exec'ing `_group.py` yields a second `BoundJoin` (defined there) while `Structure` (imported) stays identical, and `bound_join is not BoundJoin.DISABLED` is then always true, joining a family comma's own segment and reporting 3,150 phantom movers. Rebind what the module defines (`mod.__dict__["BoundJoin"] = real.BoundJoin`) and assert the identity rather than assuming it either way.
- `_pipeline.STAGES` holds function OBJECTS captured at import, so re-exec'ing a stage module leaves the pipeline running the old function and every result is a false negative. Rewire `STAGES` and assert the new function is in it.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@
"John Smith, LEED AP" → suffix="LEED AP"
"Smith, XYZ" → given="XYZ" · boundary
"John Smith, PhD XYZ" → suffix="PhD XYZ"
"John Smith, PU-PH" → suffix="PU-PH"
"JOHN SMITH, PU-PH" → given="PU-PH" · boundary
"Smith, PU-PH" → given="PU-PH" · boundary
```

---
Expand Down
2 changes: 1 addition & 1 deletion docs/design/rules/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ Terms the rules use without defining them each time, each with the rule that def

- [S1 — a bracketed credential is not a nickname](S1-bracketed-credential-is-not-a-nickname.md)
- [S2 — a suffix word ends a name and never opens the string](S2-suffix-word-ends-a-name-and-never-opens-the-string.md)
- [S3 — credentials written with interior periods](S3-credentials-written-with-interior-periods.md)
- [S3 — credentials written in joined chunks](S3-credentials-written-in-joined-chunks.md)
- [S4 — the ambiguous acronym needs words to spare](S4-ambiguous-acronym-needs-words-to-spare.md)
- [S5 — the given part's trailing slot after a family comma](S5-given-parts-trailing-slot-after-a-family-comma.md)
- [S6 — the case signal at the suffix slot](S6-case-signal-at-the-suffix-slot.md)
Expand Down
Loading
Loading