diff --git a/AGENTS.md b/AGENTS.md index 25ce3de2..3298f5eb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -173,7 +173,7 @@ The library has two layers: `nameparser/config/` (data) and `nameparser/parser.p **Sweep the forms a change can reach: comma shapes, then `name_order`.** No comma, a FULL name before the comma, a ONE-WORD name before the comma and NOTHING before the comma are four paths, and the last two are the misses — #429, #430 and #432 are all the one-word path disagreeing with the full-name path on inputs the full-name path reads correctly, and #613's PR review found `, PhD` and `, Jr.` (an empty surname field, as CSV data writes it) read as given and title by a decision that returned early on an empty head, after two review rounds whose every grid had a name before the comma. Put an empty head and a nickname-only head (`(Bob), PhD`) in every comma grid. Count the one-word path the way C1 counts, which is neither tokens nor pieces: a particle run and the one word it attaches to are one word (#575), a connective join and a bound given-name pair count as their words (#613, so `Ortega y Gasset, Dr.` is a whole name), and a suffix beside the surname counts as no name word at all. #603's review found the suffix half (`Vega y Lopez Jr., PhD Jones` lost its family where `Vega y Lopez, PhD Jones` kept it); #613 then reversed the connective half, and both now read given 'Vega', middle 'y', family 'Lopez' — so test a particle surname, a connective surname and a surname with a suffix, each before a comma. For orders the sweep already exists (`tests/v2/test_cases.py` runs every row under all three) but carries one assertion, R2's family partition, so it checks nothing a new change moves; coverage there has been vacuous before (PR #394's review found the suite passed with `name_order` discarded from grouping). Parse your change's names in each comma shape and each order, and read the ones you did not predict. And read a generated grid's moves by SHAPE, not by corpus membership: #614's first design passed the suite, the five gates and a fingerprint of every corpus name, because its grid moves were sorted into 'corpus name moved' and 'garbage', and the review found the moves that mattered among the 'garbage' -- a bare salutation (`Mr. and Mrs.` lost its connective to the family name) and a name with one trailing word (`Freiherr von Berg X.Y.Z.`). Pull out the head-plus-one-word slice of the grid and a salutation with nothing after it, and read every move there. Two more were the second review's (#614): no grid put TWO words that are both a title and a particle at the head (`Freiherr St van Berg MA`, where an index into the pieces was applied to a merged view of them -- `TITLES ∩ particles_ambiguous` is `{freiherr, st}`, so put both in every head list, each in its bare and its period-marked spelling -- #637's grid had `St` and `Freiherr` alone and missed that `St.` and `Freiherr.` are declined by the bound join and then attached by P6), and the grids compared ambiguity KINDS, so a report telling the caller the wrong field (`Freiherr von Berg Ma`, 'read as a given name' of a word in the family) passed every pin. #701's review added one: put a title/suffix dual (`Ms`, `Sr.`, `Cpl`) SECOND in a title run, behind an ordinary title. A stop that asks only "is this word suffix vocabulary" passes every row that opens with the dual and fails `Rev. Sr. Smith, Mary`; it was found by comparing `T Smith, John` against `T John Smith` over every title word, and a comma-versus-comma-less comparison is the cheap check for any rule that moves a word between the two slots. Compare each report's `detail` against the field its tokens land in -- since #626 `test_a_report_names_the_field_its_word_lands_in` (tests/v2/test_cases.py) does it over every non-locale case row under all three orders (a row carrying its own policy only as declared), for the phrasings its `_FIELD_CLAIMS` lists -- so a row with the shape is the test only where the report's wording is listed: a new emitter's wording goes in that list, and segment's report on a part past the second comma names no field (#629). #626's own first grid then missed an emitter for want of one comma shape (`, Ma Dr.`: an empty head, an ambiguous word, a trailing title, which H1 moves to the family): a trailing title belongs in every empty-head comma grid too. And #620's: an exclusion list in code being replaced is a set of rules, not noise -- #614's merged copy excluded every word the trailing read weighs from its units, the first #620 prototype dropped the list as a symptom of the copy, and its review found the list had been keeping positional counting's rule that a member in front counts as a word (`Freiherr von Berg MA X.Y.Z.` lost its suffix). Before deleting one, mutate each entry out and read what moves. -**Then sweep the spellings: a class reached by vocabulary is also reached by SHAPE.** A test that uses only the listed spelling of a word class walks only the vocabulary path, and the shape path is a separate branch that can be missing while every test passes. Roman numerals are the sharpest case: `ii`, `iii` and `iv` are suffix vocabulary; `i` and `v` are vocabulary too but initial-shaped, so `is_suffix_piece` vetoes them and only the numeral fork reads them; and `vi` through `x` are in no list at all, read by the fork's shape test alone (checked 2026-10-04 against `Lexicon.default()`) -- which stops at `x`: `_vocab._ROMAN` matches I to X, so `John Smith XI` reads `XI` as a name word (re-checked 2026-10-07, #614's /simplify, which found a docstring claiming "any length"). And the fork counts a ONE-letter numeral besides, since #718 (rules/S10): it is the suffix only where the name has a word to spare, so `John V` and `John Smith V` take different branches of one fork, while a one-case lowercase `v`, a listed suffix word, never reaches it and is not counted (`john v` keeps suffix 'v' where `john x` and `john i` read family -- `i` arrives as P3's connective, read as an initial) -- put a head-plus-numeral row beside every words-to-spare row. So a change that handles `V` and `III` can still miss `VI` — #610 found exactly that, the maiden take's candidate loop admitting a shape-only numeral in one position where the peel reads it in two (`Jane Doe nee Smith VI Prof.` kept maiden `Smith VI` while `V` and `III` worked). The same split runs through the credentials (`MA` listed, `X.Y.Z.` by the dotted shape, `XYZ` by the caps shape, and `Ph. D.` merged by group into one flagged piece the peel's walk never holds -- #603 found #602's run-start predicate refusing it too, so `Smith, John Ph. D. Jones` kept middle 'Jones' where `Smith, John PhD Jones` read suffix 'PhD Jones') and the titles (`Prof.` period-marked and read at the end of a name, `Prof` bare and a name word there). When a change touches a word class, test one spelling from each path. The same holds for a separator's SPACING: a comma (or any punctuation) can stand alone, glue to the word before it or glue to the word after, and each is a different token stream. #653's first fix swept `(née, Jones)` and missed `(née , Jones)`, where the kept comma became a word after the marker and M3 read the clause as maiden ', Jones'; its review found it. Put all three spacings in any grid around a separator. +**Then sweep the spellings: a class reached by vocabulary is also reached by SHAPE.** A test that uses only the listed spelling of a word class walks only the vocabulary path, and the shape path is a separate branch that can be missing while every test passes. Roman numerals are the sharpest case: `ii`, `iii` and `iv` are suffix vocabulary; `i` and `v` are vocabulary too but initial-shaped, so `is_suffix_piece` vetoes them and only the numeral fork reads them; and `vi` through `x` are in no list at all, read by the fork's shape test alone (checked 2026-10-04 against `Lexicon.default()`) -- which stops at `x`: `_vocab._ROMAN` matches I to X, so `John Smith XI` reads `XI` as a name word (re-checked 2026-10-07, #614's /simplify, which found a docstring claiming "any length"). And the fork counts a ONE-letter numeral besides, since #718 (rules/S10): it is the suffix only where the name has a word to spare, so `John V` and `John Smith V` take different branches of one fork, a one-case name being counted like capitals whatever the letter -- since #721 a one-case one-letter numeral carries the `initial` tag its capital does, and a bare letter in front of the numeral is initial-shaped as its capital is, so `john v` reads as `JOHN V` does over `test_a_one_case_name_reads_as_its_capitals`'s grid (comma shapes, orders, a maiden clause, a particle chain, a bare letter in front, a trailing title, a non-ASCII letter in front: 315 disagreeing pairs at 2bfa9533, before #721, and none after). Its first version claimed "every shape" with no bare letter in front and no trailing title in the grid, and #721's reviews found both broken; a grid's claim is the grid -- put a head-plus-numeral row beside every words-to-spare row. So a change that handles `V` and `III` can still miss `VI` — #610 found exactly that, the maiden take's candidate loop admitting a shape-only numeral in one position where the peel reads it in two (`Jane Doe nee Smith VI Prof.` kept maiden `Smith VI` while `V` and `III` worked). The same split runs through the credentials (`MA` listed, `X.Y.Z.` by the dotted shape, `XYZ` by the caps shape, and `Ph. D.` merged by group into one flagged piece the peel's walk never holds -- #603 found #602's run-start predicate refusing it too, so `Smith, John Ph. D. Jones` kept middle 'Jones' where `Smith, John PhD Jones` read suffix 'PhD Jones') and the titles (`Prof.` period-marked and read at the end of a name, `Prof` bare and a name word there). When a change touches a word class, test one spelling from each path. The same holds for a separator's SPACING: a comma (or any punctuation) can stand alone, glue to the word before it or glue to the word after, and each is a different token stream. #653's first fix swept `(née, Jones)` and missed `(née , Jones)`, where the kept comma became a word after the marker and M3 read the clause as maiden ', Jones'; its review found it. Put all three spacings in any grid around a separator. **And the encoding: a letter class written `[^\W\d_]` or tested with `isalpha()` is blind to a combining mark.** Python's `\w` and `isalpha()` do not match Mn, so a letter written as base plus accent (NFD) ends a word early; the vocabulary path is immune because `_normalize` NFC-folds, the shape path is not. #652: `Dr. inž. Janez Novak` read `inž.` as the given name when the ž was decomposed, and `Janez Novak, ŽŠČ` reversed, in two families that share the blind spot — the letter-shape regexes (`period_abbreviation`, `_DOTTED`, `_CHUNKED`, `_PAIRED_INITIALS`), now spelled with `_vocab._LETTER` (a letter plus the general combining blocks; `test_regex_sync` compares the two `_MARKS` strings and pins `period_abbreviation` against config's copy), and the caps predicates, which NFC-compose a non-ASCII word before `isalpha()` (`is_caps_word`, behind C-level `isalpha()`/`is_normalized` tests, so already-composed text pays no frame — measured 2026-10-12, `Smith, José` was 193 frames before and after; the first version called it for every non-ASCII word and cost one frame each, and no guard saw it because the reference name is ASCII). A width test must count letters, not code points (`composed_len`): admitting a decomposed word to the caps set put `John Smith, MÁ` on the pair exception's wrong side, and the review's grid found it where the rows (all three letters long) did not — put a two-letter word in every caps grid. The initial shape (`_INITIAL`, `\w\.`, v1-verbatim and public) is not widened but asked of the COMPOSED word, in `is_initial_shaped` and in `_render`'s two copies; and the single-letter connective tests count composed letters, since a decomposed `й` is two code points (#665, where `É. y É. Smith` and `John й Smith` read differently decomposed). Compose the WHOLE predicate, not one half: `is_initial` is a shape ANDed with the #320 script veto, and composing the shape alone made a decomposed `씨.` an initial, since decomposed hangul is conjoining jamo outside every script range (#665's first version; `in_initialless_script` composes too now). Not covered, deliberately: the marks only some scripts write (Indic vowel signs, Arabic and Hebrew points; rules/H2's Accepted clause). The guard is `test_a_decomposed_name_renders_as_its_composed_twin` over the CLDR corpus plus `test_a_decomposed_abbreviation_reads_as_its_composed_twin` (rows × three orders × EVERYWHERE); the corpus guard reaches a shape only if some corpus name does, which is how the caps shape was missed until review, so when adding a shape rule add a row there. Four of #652's rows and two of #665's pass with every fix reverted (measured) and are parity pins, and the test compares ambiguity kinds and `initials()` too, since #665's `John й Smith` differed in its report; `_DOTTED` has no observable input and is pinned by a direct call. diff --git a/docs/design/decisions/P3-connectives.md b/docs/design/decisions/P3-connectives.md index 6f030eff..80489e23 100644 --- a/docs/design/decisions/P3-connectives.md +++ b/docs/design/decisions/P3-connectives.md @@ -62,3 +62,4 @@ - 2026-10-07 #614 — CROSS-REFERENCE: P3's joins are made before S2's trailing run is read, so the run counts a joined part as one name word, as this rule says every count does (`Juan y Garcia X.Y.Z.` keeps family 'X.Y.Z.'), and a salutation keeps its connective (`Mr. and Mrs.` reads title 'Mr. and Mrs.'). #614's first design read before these joins too and broke both; #614's review reversed it (decisions/S2, this date). - Provenance: the single-letter-connective guard is v1's fix for Google Code issue 11 ("john e smith", 2013, commit 33676c9) — the "#11" citations that circulated pointed at a GitHub accident, not the real source. Recorded so the archaeology stays done. +- 2026-10-18, #721 — THE OFF SWITCH NO LONGER RESTORES THE ONE-CASE ROLE MOVERS. With `i` removed from the conjunctions, a one-case `i` is a one-letter roman numeral, which since #721 is the initial its capital is (rules/S10), so `rovira, i`, `john smith i jr` and `maier, amy i, jr.` read with the switch off exactly as with it on, and as their all-caps twins read either way — where 2.3.0 read them as suffixes, the disagreement with the capitals that decisions/P3's 2026-09-20 bullet "ACCEPTED, AND THE HONEST STATEMENT OF THE ONE-CASE HALF" measured. The off-switch grid's property is unchanged, and these rows no longer lean on its exemption (2): the two readings now agree on them (on: `initial`; off: `initial`, where 2.3.0's off reading was not), so they pass on equal fields. The exemption still guards the rows it was written for (#721's design-docs review measured the rows only it saves falling from thousands on master to a few hundred, all of them a connective `i` before a strict-comma numeral such as `I ROVIRA, V` — a dated snapshot). And a connective letter ending its part is not reported (rules/P3, "unless nothing of its part stands to its right"): the fork had no other way to go. diff --git a/docs/design/decisions/S10-trailing-roman-numeral.md b/docs/design/decisions/S10-trailing-roman-numeral.md index c9a3d796..28691d24 100644 --- a/docs/design/decisions/S10-trailing-roman-numeral.md +++ b/docs/design/decisions/S10-trailing-roman-numeral.md @@ -10,3 +10,13 @@ ONE STAND-DOWN, FOUND BY INV10 rather than designed. In a one-case name the connective `i` reaches the fork as a letter classify has already reported as `conjunction-or-initial`, read as an initial, and the new report put a second report on the same token: 138 rows of the connective grid (measured on this branch with the `reported_as_initial` check out of assign's numeral report, a dated snapshot) in tests/v2/test_properties.py (`AND ROVIRA I`, `HENRY I`) tripped INV10's kind pair. The two reports did not contradict each other — both read the letter as a name word — but they named one reading twice, so the numeral's report stands down where the initial reading is already reported, the shape of rules/S7's "a member its own capitals already made the credential reports nothing new". The guard was left untouched: its kind pair still means "two readings" everywhere else, and narrowing it to suit this change would have retaken its recorded control. `HENRY I` therefore reports `conjunction-or-initial` alone, and moved in test_assemble's contrast list from the withdrawn side to the side where the fork resolved to a name word. - 2026-10-18, #718 review — THE COUNT IS NOT ASKED IN THE GIVEN PART AFTER A FAMILY COMMA. The code review's grid (family-comma and empty-head shapes behind a title-particle, about 558,000 parses) found 2,880 parses where the first form moved a numeral from the suffix into the given or family name with no report: `Smith, Freiherr von Berg V` read given 'von Berg V', `, Freiherr von V` family 'von V', `Smith, St van V` given 'van V', where `Smith, von V` and `Smith, Freiherr John V` kept suffix V. The cause was not a choice: assign's walk over that part never counts the numeral (rules/S4: the given part's trailing slot "is read without it (S5)"), but the particle chain's bound there asks `tail_reading`, which the count had reached, so the chain took a word assign would have read as the suffix. The bound now reads with the count off (`trailing_run_start`, `count_numeral=False`), and all four read as they did on master (2.2.0 onward, since #424; 1.4.0 read the numeral into the name). The class is the one the AGENTS.md gotcha on group-side predictors of the given part names, met from the other side: the predictor did not change, the peel it asks did. The S4 analogue, `Smith, Freiherr von Berg Ma` → given 'von Berg Ma', is master's reading and is left alone here. - 2026-10-18, #718's PR review — TWO OF THE THREE REPORT SITES ARE ONE, AND THE CLAUSE ASKS THE CASE. The test-coverage review mutated each new emitter out in a copy of the tree. Removing the particle chain's numeral report moved no field and no kind — assign's pick loop reported the same `suffix-or-name`, worded from the final role ('read as a family name') — so the chain's branch was only choosing a sentence, and it is gone: a declined numeral stays one of the read's picks, and assign alone reports it, the decision having been the read's and not the chain's. The maiden clause's stand-down on a letter already reported as an initial could not fire at all (classify reads no connective inside a clause), so that call is gone too. And the clause's report on a numeral its take gave up did not ask the case: `Jane Doe nee Smith vi` and `x` reported where every other path had gone silent, the drift mechanisms.md#ONE-PREDICATE-PER-QUESTION names; the take now asks `numeral_settled_by_case` as assign does, `VI` still reporting beside them. Five rows the first draft labelled `fix(#718)` read field for field as 1.4.0 does and are `parity`, as are the two Freiherr rows added here; a restored 1.4.0 reading is parity by the table's own definition (rules/S4's `bare_surname_is_not_a_credential` is the precedent). +- 2026-10-18 (Derek), #721 — A ONE-CASE ONE-LETTER NUMERAL IS THE INITIAL ITS CAPITAL IS, AND A LETTER ENDING ITS PART IS NO CONNECTIVE. Two Accepted gaps of the entries above close. First, `john v` read suffix 'v' with no family while `JOHN V` and `john x` were counted: `V` carries the `initial` tag (a bare capital) and every reader of the trailing run honors it — the suffix veto, the particle chain, the credential run — while a lowercase `v` was a plain listed suffix word. A first prototype (not shipped) routed a one-case `v` to the numeral fork at the walk's last position only, and a lower/upper invariant — every one-case name parses as its capitals, fields and report kinds — showed it too shallow (`john v phd`, `john v, phd` and `freiherr von smith v` still disagreed). The fix that holds tags the letter where its capital is tagged: in a name written in one case, a one-letter roman numeral is an `initial` (classify, reading the NAME-level one-case fact so a maiden clause's `v` is one too, as #516's caps branch reads it). And #721's reviews found the numeral fork's own test of the word in front case-dependent: a bare lowercase letter is never initial-shaped, so `john j v` read suffix 'v' where `JOHN J V` reads family 'V' (on master too). In a one-case name a bare cased letter in front now counts as written as an initial. The invariant is `test_a_one_case_name_reads_as_its_capitals` in tests/v2/test_properties.py, its grid and its two recorded controls stated there: master disagreed on 288 of its 1,575 pairs, the branch on none. + Second, Derek on the issue: `john i` reporting `conjunction-or-initial` is wrong "because there's nothing for a conjunction to join to there". P3 already said a connective with nothing to its right connects nothing; classify's report now skips a letter that is the last own word of its part — before a comma, a marker's clause or the end. Such a letter is read as an initial and not reported, and where it is a roman numeral the numeral fork reports `suffix-or-name` instead (`john i`, `HENRY I`, `carod i`); a marked letter that is no numeral is simply silent (`john e`, `jones, john e`). Not all such letters are the walk's last word: a trailing title (rules/H5) is an own word to classify that the walk peels first, so in `john i dr.` the letter keeps P3's report and is still the numeral fork's declined pick. The stand-down the entry above added (a letter already reported as an initial reports nothing new) is what keeps that to one report, and it stays: a first draft of this change deleted it as dead on a mutation run over an intermediate tree, and both reviews then found `john i dr.`, `JOHN I DR.` and `henry i prof.` carrying both reports on one token, where master carries `conjunction-or-initial` alone. Two comma rows go silent rather than move — `rovira, i` and `maier, amy i, jr.` — since the comma path has never reported the numeral (decisions/C1, "Not done"). + WHAT MOVES, the snapshot against master at 2bfa9533: six corpus names, `john v` (family 'v'), `john smith v` and `larry james edward johnson v` (gain the report), `doe, jane v phd do` (middle 'v', suffix 'phd do', as `DOE, JANE V PHD DO` and `Doe, Jane V PhD Do` read; rules/S7's example keeps its input, whose point — `do` anchored by the degree in front — is unchanged), `john i` (report kind), `jones, john e` (report dropped). And a seventh reading the snapshot shows as a removal rather than a move, since the rules example that pinned it was deleted with S10's Accepted clause: `john nee smith i` read suffix 'i' and now reads maiden 'smith i', reported — the clause-free name is `john i`, whose letter has no word to spare, so M2's clause keeps it, as it keeps `john nee smith v`. And decisions/P3's off switch no longer restores 2.3.0's three one-case readings (that file's #721 entry). +- 2026-10-18 (Derek), #721's PR review — A NUMERAL BEFORE A CREDENTIAL IS REPORTED, AND THE TWO-WORD COUNT STANDS. Derek: `john i phd` should report `suffix-or-name` as `john i, phd` does, and `John Smith V PhD` keeps family 'V' with the report — so the Accepted clause's reading stands and only the report is added; the fields do not move (1.4.0 read the same). The part's last name word with nothing but the trailing run behind it is the site: assign, once the trailing read has said what that run is, withdraws classify's connective report on a marked letter there (rules/P3 now says "unless no name word of its part stands to its right") and reports a roman numeral there, worded for what follows it. So `john i dr.` reports the numeral once, `john e phd` and `jose e jr` report nothing, and `john smith i jr` reports `suffix-or-name` where it reported `conjunction-or-initial`. #718's stand-down is kept as a defensive check, and says so: measured on this commit's tree, the full suite and a 1,680-parse trailing-title sweep fail and pass alike without it. + Declined: two stricter readings Derek weighed on the PR, both measured over the snapshot corpus. (A) a one-letter numeral is the generation only when given, middle and family are all filled — 11 readings move, among them `Hardie Leannon V`; (B) a comma-less one-letter numeral is never the generation — 18 move, among them `John W. Ingram V`. Both are real people who write the numeral without a comma: #36 (2015) reported `Hardie Leannon V` reading last 'V' as a bug, #136 (2022) came from John W. Ingram V writing his name with and without the comma, and Derek found the same sheriff written `John W Ingram V` and, once, `John Ingram V` (Brunswick Beacon, his candidate profile). The two-word count reads all four as the generation; the rows `a_real_generation_with_a_middle_initial` and `a_real_generation_with_no_middle_name` pin the last two. + And the review's other defect: the one-case test of the word in front accepted any one-character cased letter, so `IVAN Ж V` and `JOSÉ É V` moved off their mixed-case reading and a decomposed `é` read otherwise than a composed one. It asks now whether the letter's capital is initial-shaped, which is the rule the docstring states (`john j v` as `JOHN J V`; `é`, `ж` and an uncased word as before). The grid gains a non-ASCII letter in front and a shape-only three-letter numeral; master disagrees on 315 of its 1,872 pairs, the branch on none. +- 2026-10-18 (Derek), #722's last review — ONE WITHDRAWAL, AT THE END OF ASSIGN, AND CORRECTIONS TO THE ENTRY ABOVE. Two reviews of the round above found P3's broadened exception ("unless no name word of its part stands to its right") implemented only where the part's last name word is one token on the no-comma path: `john van e phd` (the letter ending a particle chain), `smith, john e phd` (the given part after a family comma) and `, john i phd` kept `conjunction-or-initial`, and `freiherr von i dr.` reached #718's stand-down and lost the numeral's report to the connective's. The withdrawal is one role-based pass at the end of the stage now, over every segment: a marked letter that is its segment's last name-role word with only suffix- and title-roled words after it loses classify's report. The numeral's own report is emitted before it, so one token never carries both, and the stand-down is gone — not "kept as a defensive check", which the entry above said on a measurement that reached no particle chain. The maiden clause gains the numeral report's twin (`Jane Doe nee Smith V PhD`, maiden 'Smith V', reported), the same fork taken a stage earlier. Corrections to the entry above, each measured on this commit's tree: option B moves 19 readings, not 18, and A 11, over the snapshot's 5,711 names with the stated patches to `_pieces.peel_trailing`'s count (A: `k - 3 >= second`; B: the count dropped) — the first figures were over an earlier snapshot plus fifteen named inputs. #36 and #136 do not say their names are real people's: #36 reports 'Hardie Leannon V' (possibly generated test data), #136 a reporter writing 'John W. Ingram, V' and 'John W. Ingram V'. The real person is the sheriff Derek found, written 'John W Ingram V' and 'John Ingram V'. And "1.4.0 read the same" holds for the capital spellings only: 1.4.0 read `john smith v phd` and `john smith i jr` with the letter in the suffix. + THE PREMISE BEHIND #718, REVISED. #718's entry quotes Derek that a generational V is "mostly archaic". On this PR he found otherwise: searching shows many people named John Ingram with every variation of generational marker and middle initial, written without a comma unless a credential follows ("MBA", "MD"). Nothing decided here moves — the two-word count reads each of those as the generation — but #718's lone-numeral reading (`John V`, family 'V') now rests on the count alone, a name of two words having no word to spare for a generation, not on the generation being rare. +- 2026-10-18, #722's /simplify pass — P3'S "CONNECTS NOTHING" IS ASKED IN ONE PLACE, OVER FINAL ROLES. The last two rounds answered rules/P3's "unless no name word of its part stands to its right" twice: classify skipped its report on a letter that was its part's last own word by position (`ends`), and assign withdrew it at the end of the stage for a letter followed only by suffix and title roles. Both go; assemble, which already withdrew this report where the letter was roled SUFFIX or TITLE (rules/A1), withdraws it too where no given, middle or family word follows the letter in its segment — the roles being final there and nothing downstream moving them. Measured, the one site reads everything the two did, and restores one reading the position cut had wrongly silenced: where no clause forms and the marker is read as a name word, a name word does follow the letter, so `jose e nee` (family 'nee') and `smith, e nee smith` report `conjunction-or-initial` again, as master did. The two reports on a numeral a post-nominal follows (assign's, and the maiden take's twin) share one predicate, `_vocab.reads_as_a_reported_numeral`. + Open: "a name written wholly in one case says nothing about any letter in it" is applied to the roman-numeral letters only. In a one-case name a bare capital is tagged an initial and a bare lowercase letter is not, unless it is `v`, `i` or `x` (this PR) or a marked connective (P3), so the lowercase and all-caps spellings of a one-case name read differently wherever a reader honors the tag — before #721 too: M4 makes a lone name word before a marker the family but not an initial (`j nee smith` family 'j', `J NEE SMITH` given 'J'), O5 is silenced by one (a lone `j` reports given-or-family, `J` does not), and the connective join defers to one (`jose j y lopez` family 'j y lopez', `JOSE J Y LOPEZ` middle 'J Y', family 'LOPEZ'). Tagging every one-case single letter as its capital is tagged is a behavior change beyond this PR: #723. diff --git a/docs/design/rules/P3-connectives.md b/docs/design/rules/P3-connectives.md index 505fcb69..01dd0e06 100644 --- a/docs/design/rules/P3-connectives.md +++ b/docs/design/rules/P3-connectives.md @@ -17,7 +17,11 @@ P3. Rationale: connective words ("y", "of the") bind name words into marks as reading both ways. A letter the vocabulary marks as reading both ways, read as an initial in a name written wholly in one case, is a call that could - have gone the other way, and is reported. + have gone the other way, and is reported — unless no name word + of its part stands to its right, nothing at all or only + post-nominals and titles, where it connects nothing and there is + no other way for the call to go: it is not reported, and where + S10 reads it as a numeral S10's report is the one. The joined part is ONE name word wherever another rule counts them, so a rule taking "one name word" takes the whole join and never half of it — except C1's count of the words before a comma, @@ -74,6 +78,10 @@ P3. Rationale: connective words ("y", "of the") bind name words into "Josep i Rovira" → middle="i" "John Quincy Smith i" → suffix="i" "Carod i" → suffix="i" + "carod i" → ambiguities=("suffix-or-name",) · boundary + "jose e phd" → ambiguities=() · boundary + "smith, john e phd" → ambiguities=() · boundary + "john van e phd" → ambiguities=() · boundary "Josep Lluis Carod i III" → suffix="i III" "Carod y de Rovira i" → middle="y" "John й Smith" → middle="й" @@ -149,5 +157,5 @@ P3. Rationale: connective words ("y", "of the") bind name words into same two words unjoined are two name words and H1 does not fire. P1's leading run is the second (#395, landed): its run takes the "Vega y Santos" join whole or stops before it. - history: decisions/P3 · interacts: H1, P1, M2, R1, R3, R4, C1, S6 · implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_comma.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py + history: decisions/P3 · interacts: H1, P1, M2, R1, R3, R4, C1, S6, S10 · implemented: nameparser/_pipeline/_assemble.py, nameparser/_pipeline/_classify.py, nameparser/_pipeline/_comma.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py diff --git a/docs/design/rules/S10-trailing-roman-numeral.md b/docs/design/rules/S10-trailing-roman-numeral.md index a6ba308f..fcbe7459 100644 --- a/docs/design/rules/S10-trailing-roman-numeral.md +++ b/docs/design/rules/S10-trailing-roman-numeral.md @@ -8,16 +8,17 @@ S10. Rationale: a roman numeral ending a name is its generation, the generation — the numerals the vocabulary lists, and those up to X it does not, by their shape alone — unless the word in front of it is written as an initial, which makes it a name word - too. A single letter written with a period is an initial, and a + too; in a name written wholly in one case a bare letter in front + is written as one. A single letter written with a period is an initial, and a numeral past X is a name word. A numeral of one letter needs words to spare, as a bare ambiguous acronym does (S4): as the second of two name words it is the family name, a title in front being no word to spare, and either reading is reported, as either reading of an ambiguous acronym is. A numeral read by its shape alone reports too; a numeral the vocabulary lists in two - or more letters is not in question and reports nothing, and a - letter already reported as read as an initial reports nothing - new. Written case + or more letters is not in question and reports nothing; a marked + letter (P3) that is such a numeral reports this way and not as a + connective. Written case settles the fork in one way only: a numeral written in lower case in a name of mixed case is the generation and reports nothing, since nobody writes an initial that way. A name written @@ -28,6 +29,8 @@ S10. Rationale: a roman numeral ending a name is its generation, "John Smith V" → suffix="V" "John Smith V" → ambiguities=("suffix-or-name",) "John W. Ingram V" → suffix="V" + "John W Ingram V" → suffix="V" + "John Ingram V" → suffix="V" "John Smith VI" → suffix="VI" "John Smith VI" → ambiguities=("suffix-or-name",) "John Smith III" → ambiguities=() @@ -43,21 +46,22 @@ S10. Rationale: a roman numeral ending a name is its generation, "John Smith V." → family="V." · boundary "John J. V" → family="V" · boundary "John Smith XI" → family="XI" · boundary - Accepted: a lower-case `v` in a name written wholly in lower - case is read as the suffix word the vocabulary lists, words to - spare or not, and reports nothing, where the same letter in - capitals is counted and reported; `i`, which is also a - connective, is read as an initial there (P3) and counted. - "john v" → family="" - "john smith v" → ambiguities=() + "john v" → family="v" + "john smith v" → ambiguities=("suffix-or-name",) "john i" → family="i" - "john i" → ambiguities=("conjunction-or-initial",) - Accepted: inside a maiden marker's clause the connective is not - read (M2), so a one-case `i` there is the suffix word the - vocabulary lists, as `v` is. - "john nee smith i" → suffix="i" + "john i" → ambiguities=("suffix-or-name",) + "john i dr." → ambiguities=("suffix-or-name",) + "john j v" → family="v" Accepted: a numeral with another suffix word behind it is not the last word of the name and is read as any word there is, so a - one-letter numeral before a credential is the family name. + one-letter numeral before a credential is the family name, and + the reading is reported, as either reading of the fork is. "John Smith V PhD" → family="V" - history: decisions/S10 · interacts: S2, S4, S7, C1, P2, P3, M2, H1, H3 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_vocab.py + "John Smith V PhD" → ambiguities=("suffix-or-name",) + "john i phd" → ambiguities=("suffix-or-name",) + "Jane Doe nee Smith V PhD" → ambiguities=("suffix-or-name",) + Accepted: a credential written split across two words is read + as one with the numeral in front of it, the run taking both, as + 1.4.0 read it. + "John Smith V Ph. D." → suffix="V Ph. D." + history: decisions/S10 · interacts: S2, S4, S7, C1, P2, P3, M2, H1, H3 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_classify.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_vocab.py diff --git a/docs/design/rules/S7-company-of-an-unambiguous-credential.md b/docs/design/rules/S7-company-of-an-unambiguous-credential.md index 94ac75e9..ffad9fa8 100644 --- a/docs/design/rules/S7-company-of-an-unambiguous-credential.md +++ b/docs/design/rules/S7-company-of-an-unambiguous-credential.md @@ -44,7 +44,7 @@ S7. Rationale: post-nominals are written as a run, and an "Smith, PhD Ma" → ambiguities=("suffix-or-name",) "Smith, MD PhD Ma" → given="PhD" · boundary "Doe, Jane PhD MEng" → suffix="PhD MEng" - "doe, jane v phd do" → suffix="v phd do" + "doe, jane v phd do" → suffix="phd do" Accepted: a name-leaning member with nothing in front of it to speak for it ends the name, words to spare or not, and whatever stands in front of it is name text. Where a credential does diff --git a/docs/release_log.rst b/docs/release_log.rst index 96d99047..a02be1e6 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -14,7 +14,7 @@ Release Log - **Fix a surname particle such as d' or al- reading as a middle name when a listing puts it after the given name.** ``Ivoi, Paul d'`` gives last ``d' Ivoi``, where 1.4.0 through 2.4.2 gave middle ``d'``. The Arabic article reads the same in its plain and assimilated spellings (``al-``, ``az-``, ``aš-``). A particle written joined to the surname, as in ``Paul d'Ivoi``, is unchanged. :issue:`700` - **Fix case repair capitalizing the Dutch particle 't.** ``jacobus henricus van 't hoff`` repairs to ``Jacobus Henricus van 't Hoff``, where 1.4.0 through 2.4.2 gave ``van 'T Hoff``. :issue:`700` - **Add Gov as a title.** ``Gov Smith`` gives title ``Gov``, last ``Smith``, where 1.4.0 through 2.4.2 gave first ``Gov``, and ``Lt. Gov. Smith, John`` gives title ``Lt. Gov.`` with the change above. :issue:`701` - - **Fix a capital V, I or X after a lone first name being read as a generational suffix, leaving no last name.** ``John V`` gives first ``John``, last ``V``, and ``Malcolm X`` last ``X``, where 2.0.0 through 2.4.2 gave suffix ``V`` and ``X`` with no last name, restoring 1.4.0's reading. A title is no word to spare, so ``Dr. Smith V`` gives first ``Smith``, last ``V`` (1.4.0's reading too) where 2.0.0 through 2.4.2 gave last ``Smith``, suffix ``V``; a name with a word to spare keeps the suffix, and ``John Smith V`` still gives suffix ``V``. :issue:`718` + - **Fix a capital V, I or X after a lone first name being read as a generational suffix, leaving no last name.** ``John V`` gives first ``John``, last ``V``, and ``Malcolm X`` last ``X``, where 2.0.0 through 2.4.2 gave suffix ``V`` and ``X`` with no last name, restoring 1.4.0's reading. A title is no word to spare, so ``Dr. Smith V`` gives first ``Smith``, last ``V`` (1.4.0's reading too) where 2.0.0 through 2.4.2 gave last ``Smith``, suffix ``V``; a name with a word to spare keeps the suffix, and ``John Smith V`` still gives suffix ``V``. A name written all in lower case reads the same way (``john v`` gives last ``v``). :issue:`718`, :issue:`721` * 2.4.2 - October 13, 2026 diff --git a/nameparser/_lexicon.py b/nameparser/_lexicon.py index 06acc453..6331fc0c 100644 --- a/nameparser/_lexicon.py +++ b/nameparser/_lexicon.py @@ -644,8 +644,10 @@ class Lexicon: #: decided by the writing instead and never consults this set, and #: neither does a caseless letter (Arabic و), which has no case to #: read. A member additionally reports - #: :attr:`~nameparser.AmbiguityKind.CONJUNCTION_OR_INITIAL`; a - #: non-member reports nothing, its reading not being in doubt. + #: :attr:`~nameparser.AmbiguityKind.CONJUNCTION_OR_INITIAL` where a + #: name word of its part follows it (with none, it connects + #: nothing); a non-member reports nothing, its reading not being in + #: doubt. #: Full default list: #: :data:`~nameparser.config.conjunctions.CONJUNCTIONS_AMBIGUOUS`. #: Entries need not also be in ``conjunctions``; one that is not is diff --git a/nameparser/_pipeline/_assemble.py b/nameparser/_pipeline/_assemble.py index 363509b6..58d1e03f 100644 --- a/nameparser/_pipeline/_assemble.py +++ b/nameparser/_pipeline/_assemble.py @@ -2,7 +2,8 @@ Consumes: tokens (all roles set), dropped, ambiguities (by index; one carrying ``field_tail`` is worded from its word's final role, -#626). +#626), and segments, for the part a withdrawn connective report's +letter ends (rules/P3, #721). Produces: a validated ParsedName -- the constructor re-checks every invariant (span order/bounds, ambiguity subset), so a pipeline bug that would produce an invalid result dies HERE, not in a renderer @@ -20,7 +21,7 @@ """ from __future__ import annotations -from nameparser._pipeline._state import FIELD_READING, ParseState +from nameparser._pipeline._state import FIELD_READING, NAME_ROLES, ParseState from nameparser._types import ( Ambiguity, AmbiguityKind, ParsedName, Role, Token, ) @@ -74,6 +75,7 @@ def assemble(state: ParseState) -> ParsedName: if contentless: final = {} ambiguities = [] + segment_of: dict[int, tuple[int, ...]] | None = None for pending in state.ambiguities: materialized = tuple(final[i] for i in pending.indices if i in final) @@ -114,6 +116,23 @@ def assemble(state: ParseState) -> ParsedName: and any(t.role in (Role.SUFFIX, Role.TITLE) for t in materialized)): continue + # rules/P3: "unless no name word of its part stands to its + # right" -- nothing after the letter in its segment, or only + # suffixes, titles, a maiden clause or a nickname: it connects + # nothing, so the report goes (#721). Asked here for the reason + # above: the roles are final. Where the letter is also a + # numeral the fork read, its own report (rules/S10) is the one + # left, so one token never carries both. Built only for a + # connective report, so other names pay nothing. + if pending.kind is AmbiguityKind.CONJUNCTION_OR_INITIAL: + if segment_of is None: + segment_of = {j: seg for seg in state.segments + for j in seg} + i = pending.indices[0] + if not any(j > i and j in final + and final[j].role in NAME_ROLES + for j in segment_of.get(i, ())): + continue detail = pending.detail if pending.field_tail is not None: # A report naming a field, in a name emptied above (no diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index f4a885b6..da2b5841 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -89,7 +89,8 @@ from nameparser._lexicon import Lexicon from nameparser._pipeline._vocab import ( - effective_script, is_suffix_lenient, resolve_script_set, + _ROMAN, effective_script, is_suffix_lenient, + reads_as_a_reported_numeral, resolve_script_set, ) from nameparser._pipeline._pieces import ( anchor_in_reach, credential_at_the_given_slot, given_slot_anchors, @@ -306,21 +307,6 @@ def _placed(read: TailRead, rest: list[int], peel.absorbed) -# rules/S10: "a letter already reported as read as an initial reports -# nothing new" -- asked only by the numeral fork's declined pick -# (#718). The maiden clause's report on the numeral it keeps cannot -# meet the case: classify reads no connective inside a clause -def _reported_as_initial(ambiguities: Sequence[PendingAmbiguity], - index: int) -> bool: - """Whether classify's connective-or-initial fork already reported - the token at `index`, reading it as an initial. Declining the - numeral reads the letter as that same name word, so a second - report would name one reading twice. A rare shape, so the scan - costs ordinary names nothing.""" - return any(a.kind is AmbiguityKind.CONJUNCTION_OR_INITIAL - and index in a.indices for a in ambiguities) - - def _assign_main(seg_idx: int, state: ParseState, tokens: list[WorkToken], ambiguities: list[PendingAmbiguity], @@ -540,6 +526,25 @@ def _assign_main(seg_idx: int, state: ParseState, f"decides it; read as ", tuple(head), field_tail=" by convention, which follows the read " "order")) + # rules/S10: "a one-letter numeral before a credential is the + # family name, and the reading is reported" -- the part's last name + # word with nothing but the trailing run behind it (#721, Derek: + # 'John Smith V PhD' keeps family 'V' and reports, as 'john i, phd' + # does). Decided here, where the trailing read has said what that + # run is + # `_ROMAN` first, C-level, so an ordinary name word pays no frame + if (len(name_pieces) >= 2 and (suffix_pieces or titled_tail) + and len(last := pieces[name_pieces[-1]]) == 1 + and _ROMAN.match(word := tokens[last[0]].text) is not None + and tuple(last) not in peeled.picks + and reads_as_a_reported_numeral( + word, tokens[pieces[name_pieces[-2]][0]].text, + state.one_case)): + ambiguities.append(PendingAmbiguity( + AmbiguityKind.SUFFIX_OR_NAME, + f"{word!r} is a roman numeral, but a post-nominal or title " + f"follows it; read as ", tuple(last), + field_tail=" rather than a generational suffix")) for piece in peeled.picks: # every pick is in rest, so the loops above just gave it a # role; whether it was read as a suffix or a name settles what @@ -550,12 +555,11 @@ def _assign_main(seg_idx: int, state: ParseState, # declined pick (#718), which carries neither credential tag # (the acronym fork's picks carry one) and is never a suffix if token.tags.isdisjoint(_AMBIGUOUS_CREDENTIAL_TAGS): - if not _reported_as_initial(ambiguities, piece[0]): - ambiguities.append(PendingAmbiguity( - AmbiguityKind.SUFFIX_OR_NAME, - f"{token.text!r} is a roman numeral, but the name " - f"has no other word to spare for it; read as ", piece, - field_tail=" rather than a generational suffix")) + ambiguities.append(PendingAmbiguity( + AmbiguityKind.SUFFIX_OR_NAME, + f"{token.text!r} is a roman numeral, but the name has " + f"no other word to spare for it; read as ", piece, + field_tail=" rather than a generational suffix")) continue declined = ("a name part" if token.role is Role.SUFFIX else "a post-nominal") diff --git a/nameparser/_pipeline/_classify.py b/nameparser/_pipeline/_classify.py index 9017b519..5e9b8f78 100644 --- a/nameparser/_pipeline/_classify.py +++ b/nameparser/_pipeline/_classify.py @@ -62,7 +62,7 @@ ) from nameparser._types import AmbiguityKind, Role from nameparser._pipeline._vocab import ( - caps_shape_candidate, composed_len, is_initial, is_one_case, + _ROMAN, caps_shape_candidate, composed_len, is_initial, is_one_case, period_joined_vocab, suffix_as_written, tag_marker_runs, ) from nameparser._pipeline._pieces import own_words @@ -159,7 +159,21 @@ def _tags_for(token: WorkToken, n: str, state: ParseState, # the mixed-case rule, unchanged. v1's is_conjunction excludes # initials: 'e.' in 'john e. smith' is a middle initial, not # the Spanish conjunction 'e' - initial = is_initial(token.text) + # rules/S10: "A name written wholly in one case says nothing + # about any letter in it" -- so a one-letter roman numeral + # there is the initial its capital is, and every reader of the + # tag after classify vetoes 'v' as it vetoes 'V' (#721) -- not + # the honorific peel upstream, which asks the shape itself + # ('田中さん v' still reads otherwise than '田中さん V'). + # `one_case` is the fact about the name's own words, applied to + # a clause's words too, as #516's caps branch applies it, since + # a capital 'V' in a clause is an initial as well. One + # character already, so `_ROMAN` alone asks what + # `is_single_letter_numeral` would, at C level: no frame per + # one-letter word + initial = is_initial(token.text) or ( + one_case and len(token.text) == 1 + and _ROMAN.match(token.text) is not None) if n in lex.conjunctions and not initial: tags.add("conjunction") if initial: @@ -323,12 +337,15 @@ def classify(state: ParseState) -> ParseState: # Of the two clauses beside it, one is load-bearing and one is # not. `conjunctions_ambiguous` is IMPLIED by the others for a # fresh parse -- the contrapositive of what it looks like: - # `is_initial` matches an ASCII capital only, so ONLY the - # fork's branch can tag a bare LOWERCASE letter "initial" - # ("jose e maria santos" tags a lowercase e), and it does so - # only for a conjunctions_ambiguous member. So for a letter of - # EITHER case, "initial" plus len 1 implies membership. It is - # kept only because `_tags_for` starts from `set(token.tags)`, so + # `is_initial` matches an ASCII capital only, so a bare + # LOWERCASE letter is tagged "initial" only by the fork's + # branch ("jose e maria santos" tags a lowercase e), for a + # conjunctions_ambiguous member, or by the one-case numeral + # (#721), which never tags an own word that is a single-letter + # connective (the fork's branch takes those first). So for a + # letter of EITHER case in `conjunctions`, "initial" plus len 1 + # implies membership. It is kept only because `_tags_for` + # starts from `set(token.tags)`, so # this clause is what keeps the emitter honest if a runner ever # hands classify tokens it did not build; no such path exists # today. `conjunctions` is the clause doing real work: it keeps @@ -339,7 +356,8 @@ def classify(state: ParseState) -> ParseState: # clause's words were never eligible for the fork, so they must # never be eligible to report either. Emitted at the decision # site (mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE), per - # token -- 'e and e' reports twice. + # token -- 'e and e e' emits twice; assemble withdraws the + # report on a letter no name word follows (rules/P3, #721). if (one_case and "initial" in token.tags and folded[i] in state.lexicon.conjunctions_ambiguous and folded[i] in state.lexicon.conjunctions diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index b71b8aae..3383bf12 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -81,7 +81,7 @@ import nameparser._pipeline._comma as _comma from nameparser._pipeline._vocab import D, PH from nameparser._pipeline._vocab import ( - comma_follows_marker, delimiter_cores, + comma_follows_marker, delimiter_cores, reads_as_a_reported_numeral, ) from nameparser._types import AmbiguityKind, Role @@ -360,6 +360,19 @@ def _maiden_take(pieces: Sequence[Sequence[int]], f"but the name has no other word to spare for it; the " f"maiden marker's clause keeps it rather than reading it as " f"a generational suffix", tuple(last))) + elif (released and len(last) == 1 and end - 2 >= lo + and reads_as_a_reported_numeral( + word.text, tokens[pieces[end - 2][0]].text, one_case)): + # rules/S10: "a one-letter numeral before a credential is the + # family name, and the reading is reported" -- the clause's + # twin of assign's report (#721's review): the clause keeps the + # numeral its released post-nominal stands behind + ambiguities.append(PendingAmbiguity( + AmbiguityKind.SUFFIX_OR_NAME, + f"{word.text!r} ending the maiden name is a roman numeral, " + f"but a post-nominal follows it; the maiden marker's clause " + f"keeps it rather than reading it as a generational suffix", + tuple(last))) return (list(range(m, lo)), [(j, Role.MAIDEN) for j in range(lo, end)] + released) diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index e772fec0..e3f14252 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -1159,7 +1159,7 @@ def peel_trailing(rest: Sequence[int], pieces: Sequence[Sequence[int]], if (k == len(rest) and k >= 2 and len(piece) == 1 and is_trailing_numeral_suffix( text := tokens[piece[0]].text, - tokens[pieces[rest[k - 2]][0]].text)): + tokens[pieces[rest[k - 2]][0]].text, one_case)): # rules/S10: "a numeral written in lower case in a name of # mixed case is the generation and reports nothing" -- # decided here once: taken as any suffix word is, and not @@ -1541,7 +1541,7 @@ def trailing_candidates(lo: int, pieces: Sequence[Sequence[int]], not tokens[piece[0]].tags.isdisjoint(_TRAILING_WORD_TAGS) or (titles_behind and is_trailing_numeral_suffix( tokens[piece[0]].text, - tokens[pieces[k - 1][0]].text))))): + tokens[pieces[k - 1][0]].text, None))))): break # a group-flagged suffix piece is outside the peel's walk # altogether (`peel_walk` drops it), so it is behind the diff --git a/nameparser/_pipeline/_vocab.py b/nameparser/_pipeline/_vocab.py index 52514325..c5fc5c82 100644 --- a/nameparser/_pipeline/_vocab.py +++ b/nameparser/_pipeline/_vocab.py @@ -256,9 +256,10 @@ def is_initial_shaped(text: str) -> bool: # reads as the generation" -- which the initial veto would otherwise # take: V and I are suffix vocabulary, X a numeral by shape alone, and # all three are bare capitals, which inside a name are middle initials. -def is_trailing_numeral_suffix(text: str, preceding: str) -> bool: - """assign's roman-numeral fork, shared with group's bound-given - reserve (#401): a FINAL single-token piece that is a roman numeral +def is_trailing_numeral_suffix(text: str, preceding: str, + one_case: bool | None) -> bool: + """The trailing walk's roman-numeral fork: a FINAL single-token + piece that is a roman numeral reads as the suffix when the piece before it does not look like part of an initial run -- for a one-letter numeral a necessary condition only, the caller's words-to-spare count deciding @@ -270,16 +271,26 @@ def is_trailing_numeral_suffix(text: str, preceding: str) -> bool: piece looks like part of an initial run, which is a question about layout, and #320 narrowed the tag to initials that can really stand in for a name. Reading the tag here made '씨.' stop - suppressing the fork and cost 'John 씨. V' its family name.""" + suppressing the fork and cost 'John 씨. V' its family name. + + `one_case` is ParseState.one_case: in a name written in one case + the word in front is read as its capital would be, so a bare 'j' + is initial-shaped as 'J' is (rules/S10, #721's review: 'john j v' + reads as 'JOHN J V'), while 'é', 'ж' or an uncased word, whose + capitals are not, stay as they were. None (the maiden take's + superset view passes it) gets the shape test alone, which admits + more and so stays a superset.""" return (_ROMAN.match(text) is not None - and not is_initial_shaped(preceding)) + and not is_initial_shaped(preceding) + and not (one_case and is_initial_shaped(preceding.upper()))) # rules/S10: "a numeral written in lower case in a name of mixed case is # the generation and reports nothing" -- the one writing that settles # the fork, since nobody writes an initial in lower case beside -# capitals. Asked by the walk alone, which takes such a numeral as any -# suffix word and records no call for a reporter to report. +# capitals. Asked by the walk, which takes such a numeral as any suffix +# word and records no call, and by the report on a numeral before a +# credential (`reads_as_a_reported_numeral`). def numeral_settled_by_case(text: str, one_case: bool | None) -> bool: """Whether a numeral the trailing fork reads is lower case in a name written in more than one case. `one_case` is @@ -287,6 +298,18 @@ def numeral_settled_by_case(text: str, one_case: bool | None) -> bool: return one_case is False and text.islower() +# rules/S10: "either reading is reported" -- the one question the two +# reports on a numeral a post-nominal follows ask (assign's, and the +# maiden take's for a numeral its clause keeps): the fork would read +# this numeral, and its writing does not settle it (#721) +def reads_as_a_reported_numeral(text: str, preceding: str, + one_case: bool | None) -> bool: + """Whether a numeral the walk left a name word, with `preceding` + in front of it, is one the fork would weigh and so reports.""" + return (is_trailing_numeral_suffix(text, preceding, one_case) + and not numeral_settled_by_case(text, one_case)) + + # #544: a single-letter roman numeral ('V', 'v', 'I.'), in any case, # is excluded for its SHAPE -- one letter is how a middle initial is # written, and rules/S3 retires single-character matches for the diff --git a/nameparser/_types.py b/nameparser/_types.py index c31e5a5d..b6b1f6aa 100644 --- a/nameparser/_types.py +++ b/nameparser/_types.py @@ -88,7 +88,9 @@ def __add__(self, other: object) -> NoReturn: # type: ignore[override] #: word READ as an initial -- initial-shaped in a script that HAS #: initials ("J." or "А.", never "씨.", #320), or a marked #: single-letter connective in a name written in one case -#: (rules/P3, see CONJUNCTION_OR_INITIAL); +#: (rules/P3, see CONJUNCTION_OR_INITIAL), or a one-letter roman +#: numeral in a name whose own words are written in one case, as its +#: capital is (rules/S10); #: "joined" a continuation of the token before it -- within one #: merged piece the tag is role-blind and every view joins the pair #: with a space ("Ph." + "D."; 'Ph. D., Jr.' gives last_list @@ -625,6 +627,14 @@ class AmbiguityKind(StrEnum): #: doctrine, rules/P3): appending " née Jones" or a nickname #: changes no report, because the clause's words were never #: eligible for it. + #: A letter with no name word after it in its part -- nothing + #: before a comma, a clause or the end, or only post-nominals and + #: titles, as in "JOSE E", "rovira, i" or "jose e phd" -- connects + #: nothing, so it is read as an initial and not reported. Where + #: such a letter is a roman numeral at the end of the name the + #: numeral fork reports it instead (SUFFIX_OR_NAME: "HENRY I", + #: "john i phd"); one ending a comma part reports nothing + #: ("rovira, i"). #: ``detail`` names the token, the kind naming neither the field nor #: the letter: which field the reading lands in follows the name's #: shape and its ``name_order``, the PARTICLE_OR_GIVEN precedent. diff --git a/nameparser/config/conjunctions.py b/nameparser/config/conjunctions.py index 25229b32..ebe552b5 100644 --- a/nameparser/config/conjunctions.py +++ b/nameparser/config/conjunctions.py @@ -56,7 +56,9 @@ # 'i' (Catalan/Polish) ships here beside 'e' for the same reason # (#397): a bare I initial is as common as a bare E, so a name # written wholly in one case reads the letter as an initial and - # reports the fork rather than joining in silence. + # reports the fork rather than joining in silence -- where a name + # word follows it in its part, there being nothing to join + # otherwise (#721). # # A caller edits the vocabulary rather than a switch: remove 'e' to # restore joining for Portuguese data, remove 'i' for Catalan or diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 7a7328ef..eaed1afb 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -1574,16 +1574,19 @@ def _check_cjk_shape_purity(self) -> None: "read middle 'ed'"), Case("an_anchored_particle_member_outranks_p6", "doe, jane v phd do", - {"given": "jane", "family": "doe", "suffix": "v phd do"}, + {"given": "jane", "middle": "v", "family": "doe", + "suffix": "phd do"}, classification="fix(#436/#437)", ambiguities=("suffix-or-name",), notes="'do' is a member AND a particle, and written in one " "case it leans nothing, so P6's attachment would take " "it (as in 'NASCIMENTO, EDSON ARANTES DO'); the degree " "in front outranks the attachment as the capitals do " - "(Derek, 2026-09-27). The roles are 1.4.0's; 2.3.0 " - "read family 'do doe', and 1.4.0 wrote the suffix " - "'v, phd, do'", + "(Derek, 2026-09-27). 1.4.0 read suffix 'v, phd, do' " + "and 2.3.0 family 'do doe'. The 'v' is the one-case " + "initial its capital is since #721, as 'DOE, JANE V " + "PHD DO' reads middle 'V' (S10, the trailing roman " + "numeral), where 1.4.0 and #718 read it in the suffix", shape=2), Case("by_design_trailing_mc_reads_as_a_credential", "Donald Mc", {"given": "Donald", "suffix": "Mc"}, @@ -4499,7 +4502,6 @@ def _check_cjk_shape_purity(self) -> None: "rovira, i", {"given": "i", "family": "rovira"}, classification="fix(#397)", - ambiguities=("conjunction-or-initial",), notes="written wholly in lower case, so the letter reads as " "an INITIAL and takes the given slot the comma leaves " "for it. 2.3.0 read family 'rovira' with suffix 'i'; " @@ -4518,7 +4520,7 @@ def _check_cjk_shape_purity(self) -> None: {"given": "john", "middle": "smith", "family": "i", "suffix": "jr"}, classification="fix(#397)", - ambiguities=("conjunction-or-initial",), + ambiguities=("suffix-or-name",), notes="the same movement with a generation behind it: the " "letter reads as an initial, so the peel stops before " "it and 'jr' alone is the suffix. 2.3.0 read family " @@ -4526,13 +4528,15 @@ def _check_cjk_shape_purity(self) -> None: "The ALL-CAPS twin 'JOHN SMITH I JR' read middle " "'SMITH', last 'I', suffix 'JR' at both of them -- " "again the reading this row now gives. No shape " - "tag, for the reason the row above gives"), + "tag, for the reason the row above gives. Since #721 " + "the letter, with only 'jr' behind it, connects nothing " + "(P3, the connectives rule), so the report is the " + "numeral's"), Case("a_one_case_link_in_a_comma_suffix_run_is_a_middle_initial", "maier, amy i, jr.", {"given": "amy", "middle": "i", "family": "maier", "suffix": "jr."}, classification="fix(#397)", - ambiguities=("conjunction-or-initial",), notes="the third comma shape, and the one whose MIXED-CASE " "twin is already in this table: 'Maier, Amy I, Jr.' " "reads middle 'I' and does not move, the writing " @@ -6021,10 +6025,13 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Jane", "family": "Doe", "suffix": "MA", "maiden": "Smith V"}, classification="fix(#533)", - ambiguities=("suffix-or-name",), + ambiguities=("suffix-or-name", "suffix-or-name"), notes="both stops read the TRAILING word, so the numeral is " "not the word either fork asks about: the acronym fork " "stops at 'MA' and 'V' stays maiden text behind it. " + "Since #721 the kept 'V', a numeral with a post-nominal " + "behind it, reports as 'John Smith V MA' reports its " + "own (S10, the trailing roman numeral). " "The mirror image, 'Jane Smith née Jones Ma V', keeps " "maiden 'Jones Ma' and reads suffix 'V'", shape=2), @@ -7513,13 +7520,14 @@ def _check_cjk_shape_purity(self) -> None: "declined numeral is still the read's pick, reported " "from the role it ends in. 2.2.0 to 2.4.2 read suffix " "'V'; 1.4.0 read last 'von Richthofen V'"), - Case("a_lone_numeral_the_chain_takes_already_an_initial", + Case("a_lone_one_case_connective_numeral_the_chain_takes_reports", "FREIHERR VON RICHTHOFEN I", {"title": "FREIHERR", "family": "VON RICHTHOFEN I"}, - ambiguities=("conjunction-or-initial", "particle-or-given"), - notes="the row above in one case with the connective 'I', " - "which classify has already reported as an initial: the " - "declined numeral reports nothing new (S10)"), + ambiguities=("particle-or-given", "suffix-or-name"), + notes="the row above in one case with the connective 'I'. " + "Ending the name it connects nothing, so classify does " + "not report it (P3, #721), and the declined numeral is " + "what reports, as for 'V'"), Case("a_lowercase_shape_numeral_in_a_mixed_case_name_is_settled", "John x", {"given": "John", "suffix": "x"}, ambiguities=("given-or-family",), classification="fix(#718)", @@ -7555,6 +7563,127 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("suffix-or-name",), classification="fix(#274)", notes="the contrast that makes the silence above a decision " "about case"), + Case("a_real_generation_with_a_middle_initial", "John W Ingram V", + {"given": "John", "middle": "W", "family": "Ingram", + "suffix": "V"}, + ambiguities=("suffix-or-name",), + notes="a sheriff in North Carolina, consistently written this " + "way (Derek, #721); #136's reporter wrote the same name " + "'John W. Ingram V' and '..., V', expecting the suffix " + "both ways -- the evidence that kept #718's two-word " + "count against reading a comma-less letter as never " + "the generation"), + Case("a_real_generation_with_no_middle_name", "John Ingram V", + {"given": "John", "family": "Ingram", "suffix": "V"}, + ambiguities=("suffix-or-name",), + notes="the same sheriff as a local paper wrote him (Derek, " + "#721), two name words and the numeral: the shape #36's " + "'Hardie Leannon V' was reported as a bug for reading " + "otherwise in 2015"), + Case("a_lone_numeral_before_a_credential_reports", + "John Smith V PhD", + {"given": "John", "middle": "Smith", "family": "V", + "suffix": "PhD"}, + ambiguities=("suffix-or-name",), + notes="S10's Accepted clause: a numeral with a credential " + "behind it is the family name, and since #721 the " + "reading is reported (Derek). 1.4.0 read the same " + "fields"), + Case("a_one_case_link_before_a_credential_reports_the_numeral", + "john i phd", + {"given": "john", "family": "i", "suffix": "phd"}, + ambiguities=("suffix-or-name",), + notes="the letter has only a credential behind it, so it " + "connects nothing (P3) and the numeral is what reports, " + "as for 'john i, phd' (Derek, #721)"), + Case("a_one_case_link_before_a_title_reports_the_numeral", + "john i dr.", + {"title": "dr.", "given": "john", "family": "i"}, + ambiguities=("suffix-or-name",), classification="fix(#316)", + notes="the same with a trailing title (H5): one report on " + "the letter, the numeral's. 1.4.0 read suffix 'dr.'"), + Case("a_one_case_non_numeral_link_before_a_credential_is_silent", + "jose e phd", + {"given": "jose", "family": "e", "suffix": "phd"}, + notes="nothing to join and no numeral fork: no report"), + Case("a_one_case_link_before_a_maiden_marker_is_silent", + "jose e nee smith", + {"given": "jose", "family": "e", "maiden": "smith"}, + classification="fix(#274)", + notes="the clause is one of the ends of a part, so the letter " + "before the marker connects nothing (P3, #721); 1.4.0 " + "had no maiden markers"), + Case("a_one_case_letter_ending_a_comma_part_is_silent", + "jones, john e", + {"given": "john", "middle": "e", "family": "jones"}, + notes="a corpus name: 'e' ends its part, so it connects " + "nothing, and being no numeral nothing else reports"), + Case("a_one_case_bare_letter_in_front_keeps_the_numeral_a_name", + "john j v", {"given": "john", "middle": "j", "family": "v"}, + classification="fix(#721)", + notes="read as 'JOHN J V' is: in a one-case name the bare 'j' " + "in front is the initial its capital is (S10). 1.4.0 " + "read last 'j', suffix 'v'"), + Case("an_uncased_word_in_front_is_no_initial", + "김 v", {"given": "김", "family": "v"}, + ambiguities=("suffix-or-name",), + notes="hangul has no capital, so the one-case test of the word " + "in front does not make it an initial; the count reads " + "the numeral as it reads '김 V'"), + Case("a_non_ascii_capital_in_front_is_no_initial", "IVAN Ж V", + {"given": "IVAN", "family": "Ж", "suffix": "V"}, + ambiguities=("suffix-or-name",), + notes="the one-case test of the word in front asks whether its " + "CAPITAL is initial-shaped, and 'Ж' is not (an initial is " + "a Latin capital), so this reads as 'Ivan Ж V' does"), + Case("a_non_ascii_lowercase_letter_in_front_is_no_initial", + "josé é v", + {"given": "josé", "family": "é", "suffix": "v"}, + ambiguities=("suffix-or-name",), + notes="the lowercase twin of the row above; its decomposed " + "spelling reads the same (rules/H2's encoding note)"), + Case("a_one_case_link_ending_a_particle_chain_before_a_credential", + "john van e phd", + {"given": "john", "family": "van e", "suffix": "phd"}, + notes="the letter ends the chain the particle made, with only " + "a credential behind it: it connects nothing (P3), so " + "nothing reports"), + Case("a_one_case_link_before_a_credential_after_a_family_comma", + "smith, john e phd", + {"given": "john", "middle": "e", "family": "smith", + "suffix": "phd"}, + notes="the given part after a family comma: only a " + "credential follows the letter in its part (P3)"), + Case("a_chained_one_case_numeral_before_a_title_reports_once", + "freiherr von i dr.", + {"title": "freiherr dr.", "family": "von i"}, + ambiguities=("particle-or-given", "suffix-or-name"), + classification="fix(#316)", + notes="a letter ending a particle chain with only a title " + "behind it: the numeral's report, the connective's " + "withdrawn (P3). 1.4.0 read first 'freiherr von', last " + "'i', suffix 'dr.'"), + Case("a_numeral_kept_by_a_maiden_clause_before_a_credential_reports", + "Jane Doe nee Smith V PhD", + {"given": "Jane", "family": "Doe", "suffix": "PhD", + "maiden": "Smith V"}, + ambiguities=("suffix-or-name",), classification="fix(#274)", + notes="the clause's twin of 'John Smith V PhD': the clause " + "keeps the numeral a post-nominal stands behind, and " + "reports it (S10). 1.4.0 had no maiden markers"), + Case("a_one_case_link_before_a_marker_read_as_a_name_reports", + "jose e nee", + {"given": "jose", "middle": "e", "family": "nee"}, + ambiguities=("conjunction-or-initial",), + notes="no clause forms, so the marker is the family name and a " + "name word follows the letter: P3 reports it, as 1.4.0's " + "successors did -- the contrast with 'jose e nee smith'"), + Case("a_lowercase_shape_numeral_before_a_credential_is_settled", + "John x PhD", + {"given": "John", "family": "x", "suffix": "PhD"}, + notes="lower case beside capitals settles the numeral (S10), " + "so the report on a numeral a post-nominal follows does " + "not fire"), Case("a_lowercase_numeral_in_a_one_case_name_still_reports", "john smith i", {"given": "john", "family": "smith", "suffix": "i"}, diff --git a/tests/v2/pipeline/test_assemble.py b/tests/v2/pipeline/test_assemble.py index 425e67d9..1d2041fe 100644 --- a/tests/v2/pipeline/test_assemble.py +++ b/tests/v2/pipeline/test_assemble.py @@ -164,10 +164,11 @@ def test_a_connective_or_initial_report_is_withdrawn_where_the_role_is_a_suffix( # the contrast that keeps the withdrawal honest: where the parse # roles the same letter a NAME word the fork DID resolve to one # of its branches, and the report stands - # -- 'HENRY I' among them since #718: with no word to spare the - # letter is the family name (rules/S10), where it was the suffix + # -- each with a word to the letter's right: ending its part, as + # in 'HENRY I', the letter connects nothing and classify reports + # nothing (rules/P3, #721) for text in ("JOSEP CAROD I ROVIRA", "josep carod i rovira", - "JOHN I SMITH", "JOSEP CAROD I ROVIRA III", "HENRY I"): + "JOHN I SMITH", "JOSEP CAROD I ROVIRA III"): name = parse(text) assert "conjunction-or-initial" in [ a.kind.value for a in name.ambiguities], text diff --git a/tests/v2/pipeline/test_pieces.py b/tests/v2/pipeline/test_pieces.py index 08dcc9fa..79bb3198 100644 --- a/tests/v2/pipeline/test_pieces.py +++ b/tests/v2/pipeline/test_pieces.py @@ -517,7 +517,7 @@ def test_a_particle_in_suffix_vocabulary_anchors_nothing( # MEMBER is still anchored by a credential in front of it. n = parse(text) assert {k: v for k, v in n.as_dict().items() if v} == fields - assert parse("doe, jane v phd do").suffix == "v phd do" + assert parse("doe, jane v phd do").suffix == "phd do" def test_anchor_in_reach_is_false_only_where_the_pass_is() -> None: diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 1cc3e0ab..8e37449b 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -190,9 +190,10 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # #718's declined-numeral report, which names the field the letter # was read into, on those rows under each order; 1191 -> 1210 the # same day: the PR review's rows (the chain-taken, family-first, - # lowercase and maiden-clause numerals). - assert claims == 1210, ( - f"the field sweep checks {claims} claims, recorded as 1210 on " + # lowercase and maiden-clause numerals); 1210 -> 1243 the same day: + # #721's rows (#722). + assert claims == 1243, ( + f"the field sweep checks {claims} claims, recorded as 1243 on " f"2026-10-18") diff --git a/tests/v2/test_facade.py b/tests/v2/test_facade.py index bfae80cd..5a386edd 100644 --- a/tests/v2/test_facade.py +++ b/tests/v2/test_facade.py @@ -1050,43 +1050,31 @@ def test_initials_of_an_unpickled_or_copied_name_ask_the_vocabulary_too() -> Non assert str(built) == restored_cap -def test_the_v1_off_switch_restores_the_pre_397_fields() -> None: +def test_the_v1_off_switch_unjoins_the_link_only() -> None: """decisions/P3's off switch on the V1 SURFACE (#397 second - review). The bullet says deleting `i` from `C.conjunctions` - restores the parent's readings; the core-side switch is pinned by - the off-switch grid in tests/v2/test_properties.py and this is the - facade half, which has no `Lexicon` of its own to remove from. + review). The core-side switch is pinned by the off-switch grid in + tests/v2/test_properties.py and this is the facade half, which + has no `Lexicon` of its own to remove from. - The three role movers of decisions/P3's one-case paragraph are - the rows, because they are the ones whose FIELDS move: every - value on the right below is what the released 2.3.0 wheel gives, - measured 2026-09-20. + The switch moves the Catalan join and leaves the three one-case + role movers of decisions/P3's paragraph alone: with `i` out of the + conjunctions it is a one-letter roman numeral, the initial its + capital is (rules/S10), so each reads as with the switch on. A local `Constants`, never the shared `CONSTANTS`: a removal on the singleton would leak into every later test in the process. """ constants = Constants() constants.conjunctions.remove("i") - for text, on_fields, off_fields in ( - ("rovira, i", - {"first": "i", "last": "rovira", "suffix": ""}, - {"first": "", "last": "rovira", "suffix": "i"}), - ("john smith i jr", - {"first": "john", "middle": "smith", "last": "i", - "suffix": "jr"}, - {"first": "john", "middle": "", "last": "smith", - "suffix": "i jr"}), - ("maier, amy i, jr.", - {"first": "amy", "middle": "i", "last": "maier", - "suffix": "jr."}, - {"first": "amy", "middle": "", "last": "maier", - "suffix": "i, jr."})): - on = HumanName(text) - off = HumanName(text, constants) - for field, value in on_fields.items(): - assert getattr(on, field) == value, (text, field) - for field, value in off_fields.items(): - assert getattr(off, field) == value, (text, field) + for text, fields in ( + ("rovira, i", {"first": "i", "last": "rovira", "suffix": ""}), + ("john smith i jr", {"first": "john", "middle": "smith", + "last": "i", "suffix": "jr"}), + ("maier, amy i, jr.", {"first": "amy", "middle": "i", + "last": "maier", "suffix": "jr."})): + for name in (HumanName(text), HumanName(text, constants)): + for field, value in fields.items(): + assert getattr(name, field) == value, (text, field) # and the join itself, the reading the bullet leads with joined = HumanName("Josep Carod i Rovira") assert joined.last == "Carod i Rovira" diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index 251009b1..45ac067b 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -3332,3 +3332,76 @@ def test_every_one_letter_particle_veto_is_load_bearing( monkeypatch.setattr(importlib.import_module(module), "_spells_an_initial", lambda n, text: False) assert len(_p7_disagreements(lexicon)) == _P7_SITE_EFFECT[module] + + +# --- #721: a name written in one case reads as its capitals --------- +# S10 (the trailing roman numeral) says a name written wholly in one +# case says nothing about any letter in it -- so a wholly lower-case +# name parses exactly as the same name in capitals, fields (compared +# case-folded) and report kinds alike. The grid crosses the shapes a one-letter numeral meets +# -- every comma shape, a maiden clause, a title, a particle chain, a +# bound given name, an initial, a nickname and a credential before or +# after it -- with the numerals a listed and a shape-only spelling of +# each length, three heads (one Catalan, one holding an ambiguous +# acronym) and all three orders: +# 1,872 pairs, about 3,750 parses, 0.29s on py3.11 (2026-10-18). +_ONE_CASE_SHAPES = ( + "{a} {n}", "{a} {b} {n}", "{a} {n}, phd", "{a} {b} {n}, phd", + "{a} {n}, jr.", "{b}, {a} {n}", ", {a} {n}", "{a} nee {b} {n}", + "{a} {b} nee {c} {n}", "dr {a} {n}", "dr {a} {b} {n}", "sir {a} {n}", + "{a} van {b} {n}", "freiherr von {b} {n}", "abdul {a} {n}", + "{a} j. {n}", "({c}) {a} {n}", "{a} {b} jr {n}", "{a} {n} phd", + # a bare letter in front, a trailing title, a doubled numeral and + # a trailing period + "{a} b {n}", "{a} e {n}", "{a} {n} prof.", "{a} {b} {n} prof.", + "{a} {b} {n} {n}", "{a} {b} {n}.", + # a non-ASCII letter in front, whose capital is no initial + "{a} é {n}") +_ONE_CASE_NUMERALS = ("v", "i", "x", "vi", "iii", "ii", "iv", "vii") +_ONE_CASE_HEADS = (("john", "smith", "jones"), ("carod", "puig", "rovira"), + ("mary", "ma", "lee")) +#: the negative controls, measured 2026-10-18 on this grid, which +#: master (2bfa9533) disagreed on 315 of: with classify's one-case +#: numeral initial taken out, and with the numeral fork's one-case +#: letter in front taken out -- each guard seen on its own +_ONE_CASE_UNGUARDED = 225 +_ONE_CASE_LETTER_IN_FRONT_UNGUARDED = 117 + + +def _one_case_disagreements() -> list[str]: + def sig(name: ParsedName) -> tuple[object, ...]: + return ({k: v.lower() for k, v in name.as_dict().items() if v}, + tuple(sorted(a.kind.value for a in name.ambiguities))) + out = [] + for order in (GIVEN_FIRST, FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST): + parser = Parser(policy=Policy(name_order=order)) + for shape, numeral, (a, b, c) in itertools.product( + _ONE_CASE_SHAPES, _ONE_CASE_NUMERALS, _ONE_CASE_HEADS): + lower = shape.format(a=a, b=b, c=c, n=numeral) + if sig(parser.parse(lower)) != sig(parser.parse(lower.upper())): + out.append(f"[{order}] {lower!r}") + return out + + +def test_a_one_case_name_reads_as_its_capitals() -> None: + failures = _one_case_disagreements() + assert not failures, "\n".join(failures[:10]) + + +def test_the_one_case_grid_sees_the_numeral_initial( + monkeypatch: pytest.MonkeyPatch) -> None: + # classify's one use of `_ROMAN` is the one-case numeral initial + monkeypatch.setattr("nameparser._pipeline._classify._ROMAN", + re.compile(r"(?!)")) + assert len(_one_case_disagreements()) == _ONE_CASE_UNGUARDED + + +def test_the_one_case_grid_sees_the_letter_in_front( + monkeypatch: pytest.MonkeyPatch) -> None: + from nameparser._pipeline import _vocab + monkeypatch.setattr( + "nameparser._pipeline._pieces.is_trailing_numeral_suffix", + lambda text, preceding, one_case=None: + _vocab.is_trailing_numeral_suffix(text, preceding, None)) + assert (len(_one_case_disagreements()) + == _ONE_CASE_LETTER_IN_FRONT_UNGUARDED) diff --git a/tests/v2/test_regex_sync.py b/tests/v2/test_regex_sync.py index 0e55094c..03338891 100644 --- a/tests/v2/test_regex_sync.py +++ b/tests/v2/test_regex_sync.py @@ -142,6 +142,11 @@ def test_dotted_initial_is_the_period_alternative_of_initial() -> None: # SAME object, imported for the trailing read's weighed-word test # (#620), as _PERIOD_ABBREV is above ("_pieces", "_ROMAN"): "roman_numeral", + # SAME object, imported for the one-case numeral initial (#721) + ("_classify", "_ROMAN"): "roman_numeral", + # SAME object, imported for the report on a numeral before a + # credential (#721) + ("_assign", "_ROMAN"): "roman_numeral", ("_post_rules", "_EAST_SLAVIC"): "east_slavic_patronymic", ("_post_rules", "_EAST_SLAVIC_CYR"): "east_slavic_patronymic_cyrillic", ("_post_rules", "_TURKIC"): "turkic_patronymic_marker", diff --git a/tools/differential/corpus_cjk.jsonl b/tools/differential/corpus_cjk.jsonl index 5bf09ad6..d18a8191 100644 --- a/tools/differential/corpus_cjk.jsonl +++ b/tools/differential/corpus_cjk.jsonl @@ -109,6 +109,7 @@ "高橋・一郎" "高橋一郎" "鵜殿" +"김 v" "김 민준" "김, 민준 MA" "김, 민준씨" diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 904c4500..e19b471d 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -171,6 +171,7 @@ "Jane Doe nee Smith PhD Smith" "Jane Doe nee Smith Prof." "Jane Doe nee Smith Prof. MA" +"Jane Doe nee Smith V PhD" "Jane Doe nee Smith V Prof." "Jane Doe nee Smith VI Prof." "Jane Doe nee Smith X.Y.Z." @@ -204,6 +205,7 @@ "John . Smith" "John Doctor Smith" "John III" +"John Ingram V" "John J. V" "John Ma" "John PhD Smith" @@ -233,6 +235,7 @@ "John Smith R.A.I." "John Smith Sir" "John Smith V" +"John Smith V Ph. D." "John Smith V PhD" "John Smith V." "John Smith VI" @@ -279,6 +282,7 @@ "John Smith, vd Ma" "John V" "John V, Dr." +"John W Ingram V" "John W. Ingram V" "John and Jane Smith" "John nee Prof. ba MA" @@ -491,6 +495,7 @@ "abdul V Smith" "abdul salam ahmed salem" "anh van do" +"carod i" "de" "de Anke van y" "de Mesnil Jean" @@ -511,7 +516,9 @@ "john doe q.c." "john e smith" "john i" -"john nee smith i" +"john i dr." +"john i phd" +"john j v" "john smith EdD" "john smith MEng" "john smith bsc" @@ -539,7 +546,9 @@ "john smith, vi" "john v" "john van der berg ma" +"john van e phd" "jose e maria santos" +"jose e phd" "jose ortega-y-gasset" "josé garcía" "juan de la vega" @@ -557,6 +566,7 @@ "née Jones" "qc mp" "smith, de ma" +"smith, john e phd" "tran lac" "van Berg Jan de" "van Gogh" diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 8a5b5bab..8aded66a 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -1356,6 +1356,7 @@ {"name": "Jane Doe nee Smith Prof. MA", "title": "Prof.", "given": "Jane", "family": "Doe", "suffix": "MA", "maiden": "Smith", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. D."} {"name": "Jane Doe nee Smith V", "given": "Jane", "family": "Doe", "suffix": "V", "maiden": "Smith", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. D."} {"name": "Jane Doe nee Smith V MA", "given": "Jane", "family": "Doe", "suffix": "MA", "maiden": "Smith V", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. D."} +{"name": "Jane Doe nee Smith V PhD", "given": "Jane", "family": "Doe", "suffix": "PhD", "maiden": "Smith V", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. D."} {"name": "Jane Doe nee Smith V Prof.", "title": "Prof.", "given": "Jane", "family": "Doe", "suffix": "V", "maiden": "Smith", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. D."} {"name": "Jane Doe nee Smith VI Prof.", "title": "Prof.", "given": "Jane", "family": "Doe", "suffix": "VI", "maiden": "Smith", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. D."} {"name": "Jane Doe nee Smith X.Y.Z.", "given": "Jane", "family": "Doe", "suffix": "X.Y.Z.", "maiden": "Smith", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. D."} @@ -1490,6 +1491,7 @@ {"name": "John Hámish Watson", "given": "John", "middle": "Hámish", "family": "Watson", "initials": "J. H. W."} {"name": "John I Smith", "given": "John", "middle": "I", "family": "Smith", "initials": "J. I. S."} {"name": "John III", "given": "John", "suffix": "III", "ambiguities": ["GIVEN_OR_FAMILY"], "initials": "J."} +{"name": "John Ingram V", "given": "John", "family": "Ingram", "suffix": "V", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. I."} {"name": "John J. V", "given": "John", "middle": "J.", "family": "V", "initials": "J. J. V."} {"name": "John Jack Andrew Kennedy", "given": "John", "middle": "Jack Andrew", "family": "Kennedy", "initials": "J. J. A. K."} {"name": "John Jones (Google Docs)", "given": "John", "family": "Jones", "nickname": "Google Docs", "initials": "J. J."} @@ -1547,7 +1549,8 @@ {"name": "John Smith Rev.", "title": "Rev.", "given": "John", "family": "Smith", "initials": "J. S."} {"name": "John Smith Sir", "given": "John", "middle": "Smith", "family": "Sir", "initials": "J. S. S."} {"name": "John Smith V", "given": "John", "family": "Smith", "suffix": "V", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S."} -{"name": "John Smith V PhD", "given": "John", "middle": "Smith", "family": "V", "suffix": "PhD", "initials": "J. S. V."} +{"name": "John Smith V Ph. D.", "given": "John", "family": "Smith", "suffix": "V Ph. D.", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S."} +{"name": "John Smith V PhD", "given": "John", "middle": "Smith", "family": "V", "suffix": "PhD", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S. V."} {"name": "John Smith V.", "given": "John", "middle": "Smith", "family": "V.", "initials": "J. S. V."} {"name": "John Smith VI", "given": "John", "family": "Smith", "suffix": "VI", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S."} {"name": "John Smith X.Y.Z.", "given": "John", "family": "Smith", "suffix": "X.Y.Z.", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. S."} @@ -1612,6 +1615,7 @@ {"name": "John V", "given": "John", "family": "V", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. V."} {"name": "John V, Dr.", "title": "Dr.", "given": "John", "family": "V", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. V."} {"name": "John V. Smith", "given": "John", "middle": "V.", "family": "Smith", "initials": "J. V. S."} +{"name": "John W Ingram V", "given": "John", "middle": "W", "family": "Ingram", "suffix": "V", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. W. I."} {"name": "John W. Ingram", "given": "John", "middle": "W.", "family": "Ingram", "initials": "J. W. I."} {"name": "John W. Ingram V", "given": "John", "middle": "W.", "family": "Ingram", "suffix": "V", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "J. W. I."} {"name": "John W. Ingram, V", "given": "John", "middle": "W.", "family": "Ingram", "suffix": "V", "initials": "J. W. I."} @@ -3286,6 +3290,7 @@ {"name": "bin Stromae Elizabeth Windsor Maria Teresa MP", "order": "FAMILY_FIRST", "given": "Stromae", "middle": "Elizabeth Windsor Maria Teresa", "family": "bin", "suffix": "MP", "ambiguities": ["PARTICLE_OR_GIVEN"], "initials": "S. E. W. M. T. b."} {"name": "bin Stromae, Elizabeth Windsor Maria Teresa MP", "given": "Elizabeth", "middle": "Windsor Maria Teresa", "family": "bin Stromae", "suffix": "MP", "initials": "E. W. M. T. S."} {"name": "bob v. de la macdole-eisenhower phd", "given": "bob", "middle": "v.", "family": "de la macdole-eisenhower", "suffix": "phd", "initials": "b. v. m."} +{"name": "carod i", "given": "carod", "family": "i", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "c. i."} {"name": "da Silva Sr. José Pedro Eduardo Jr., MP", "order": "FAMILY_FIRST", "given": "Silva", "middle": "Sr. José Pedro Eduardo", "family": "da", "suffix": "Jr., MP", "ambiguities": ["PARTICLE_OR_GIVEN"], "initials": "S. S. J. P. E. d."} {"name": "de", "given": "de", "initials": "d."} {"name": "de Anke van y", "family": "de Anke van y", "initials": "A."} @@ -3336,7 +3341,7 @@ {"name": "der, y van", "given": "y", "family": "van der", "ambiguities": ["PARTICLE_OR_GIVEN"], "initials": "y. v. d."} {"name": "dhr. Bertus Wevers Harry Robert de Jong jr. mp", "title": "dhr.", "given": "Bertus", "middle": "Wevers Harry Robert", "family": "de Jong", "suffix": "jr. mp", "initials": "B. W. H. R. J."} {"name": "doe, jane phd vd ma", "given": "jane", "family": "doe", "suffix": "phd vd ma", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. d."} -{"name": "doe, jane v phd do", "given": "jane", "family": "doe", "suffix": "v phd do", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. d."} +{"name": "doe, jane v phd do", "given": "jane", "middle": "v", "family": "doe", "suffix": "phd do", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. v. d."} {"name": "doe, john ma", "given": "john", "family": "doe", "suffix": "ma", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. d."} {"name": "donovan mcnabb-smith", "given": "donovan", "family": "mcnabb-smith", "initials": "d. m."} {"name": "dos Santos", "family": "dos Santos", "initials": "S."} @@ -3379,8 +3384,10 @@ {"name": "john e jones, III", "given": "john", "middle": "e", "family": "jones", "suffix": "III", "initials": "j. e. j."} {"name": "john e smith", "given": "john", "middle": "e", "family": "smith", "ambiguities": ["CONJUNCTION_OR_INITIAL"], "initials": "j. e. s."} {"name": "john e. smith", "given": "john", "middle": "e.", "family": "smith", "initials": "j. e. s."} -{"name": "john i", "given": "john", "family": "i", "ambiguities": ["CONJUNCTION_OR_INITIAL"], "initials": "j. i."} -{"name": "john nee smith i", "family": "john", "suffix": "i", "maiden": "smith", "initials": "j."} +{"name": "john i", "given": "john", "family": "i", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. i."} +{"name": "john i dr.", "title": "dr.", "given": "john", "family": "i", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. i."} +{"name": "john i phd", "given": "john", "family": "i", "suffix": "phd", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. i."} +{"name": "john j v", "given": "john", "middle": "j", "family": "v", "initials": "j. j. v."} {"name": "john smith", "given": "john", "family": "smith", "initials": "j. s."} {"name": "john smith EdD", "given": "john", "family": "smith", "suffix": "EdD", "initials": "j. s."} {"name": "john smith MA", "given": "john", "family": "smith", "suffix": "MA", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. s."} @@ -3402,7 +3409,7 @@ {"name": "john smith ph.d.", "given": "john", "family": "smith", "suffix": "ph.d.", "initials": "j. s."} {"name": "john smith phd", "given": "john", "family": "smith", "suffix": "phd", "initials": "j. s."} {"name": "john smith psyd", "given": "john", "family": "smith", "suffix": "psyd", "initials": "j. s."} -{"name": "john smith v", "given": "john", "family": "smith", "suffix": "v", "initials": "j. s."} +{"name": "john smith v", "given": "john", "family": "smith", "suffix": "v", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. s."} {"name": "john smith vi", "given": "john", "family": "smith", "suffix": "vi", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. s."} {"name": "john smith x.y.z.", "given": "john", "family": "smith", "suffix": "x.y.z.", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. s."} {"name": "john smith xi", "given": "john", "middle": "smith", "family": "xi", "initials": "j. s. x."} @@ -3411,11 +3418,13 @@ {"name": "john smith, phd meng", "given": "john", "family": "smith", "suffix": "phd meng", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. s."} {"name": "john smith, v", "given": "john", "family": "smith", "suffix": "v", "initials": "j. s."} {"name": "john smith, vi", "given": "vi", "family": "john smith", "initials": "v. j. s."} -{"name": "john v", "given": "john", "suffix": "v", "ambiguities": ["GIVEN_OR_FAMILY"], "initials": "j."} +{"name": "john v", "given": "john", "family": "v", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. v."} {"name": "john van der berg ma", "given": "john", "family": "van der berg", "suffix": "ma", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "j. b."} +{"name": "john van e phd", "given": "john", "family": "van e", "suffix": "phd", "initials": "j. e."} {"name": "johnny y", "given": "johnny", "family": "y", "initials": "j. y."} -{"name": "jones, john e", "given": "john", "middle": "e", "family": "jones", "ambiguities": ["CONJUNCTION_OR_INITIAL"], "initials": "j. e. j."} +{"name": "jones, john e", "given": "john", "middle": "e", "family": "jones", "initials": "j. e. j."} {"name": "jose e maria santos", "given": "jose", "middle": "e maria", "family": "santos", "ambiguities": ["CONJUNCTION_OR_INITIAL"], "initials": "j. e. m. s."} +{"name": "jose e phd", "given": "jose", "family": "e", "suffix": "phd", "initials": "j. e."} {"name": "jose ortega-y-gasset", "given": "jose", "family": "ortega-y-gasset", "initials": "j. o."} {"name": "josep carod i rovira", "given": "josep", "middle": "carod i", "family": "rovira", "ambiguities": ["CONJUNCTION_OR_INITIAL"], "initials": "j. c. i. r."} {"name": "josé garcía", "given": "josé", "family": "garcía", "initials": "j. g."} @@ -3430,7 +3439,7 @@ {"name": "juan y garcia", "given": "juan", "middle": "y", "family": "garcia", "initials": "j. y. g."} {"name": "juan y-garcia", "given": "juan", "family": "y-garcia", "initials": "j. y."} {"name": "juan y-garcia-lopez", "given": "juan", "family": "y-garcia-lopez", "initials": "j. y."} -{"name": "larry james edward johnson v", "given": "larry", "middle": "james edward", "family": "johnson", "suffix": "v", "initials": "l. j. e. j."} +{"name": "larry james edward johnson v", "given": "larry", "middle": "james edward", "family": "johnson", "suffix": "v", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "l. j. e. j."} {"name": "lt. gen. john a. kenneth doe iv", "title": "lt. gen.", "given": "john", "middle": "a. kenneth", "family": "doe", "suffix": "iv", "initials": "j. a. k. d."} {"name": "m.d ph.d. Inger Marie Adele Synnøve von Wolf jr.", "given": "m.d", "middle": "ph.d. Inger Marie Adele Synnøve", "family": "von Wolf", "suffix": "jr.", "initials": "m. p. I. M. A. S. W."} {"name": "m.d ph.d. von Wolf Inger Marie Adele Synnøve", "order": "FAMILY_FIRST", "given": "ph.d.", "middle": "von Wolf Inger Marie Adele Synnøve", "family": "m.d", "initials": "p. W. I. M. A. S. m."} @@ -3483,6 +3492,7 @@ {"name": "scott l. werner", "given": "scott", "middle": "l.", "family": "werner", "initials": "s. l. w."} {"name": "señora María García", "title": "señora", "given": "María", "family": "García", "initials": "M. G."} {"name": "smith, de ma", "family": "smith de ma", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "s. m."} +{"name": "smith, john e phd", "given": "john", "middle": "e", "family": "smith", "suffix": "phd", "initials": "j. e. s."} {"name": "st.repr. Gregers Werle Marve Almar von Fleksnes jr.", "title": "st.repr.", "given": "Gregers", "middle": "Werle Marve Almar", "family": "von Fleksnes", "suffix": "jr.", "initials": "G. W. M. A. F."} {"name": "st.repr. von Fleksnes Gregers Werle Marve Almar", "order": "FAMILY_FIRST", "title": "st.repr.", "given": "Fleksnes", "middle": "Gregers Werle Marve Almar", "family": "von", "ambiguities": ["PARTICLE_OR_GIVEN"], "initials": "F. G. W. M. A. v."} {"name": "test", "given": "test", "ambiguities": ["GIVEN_OR_FAMILY"], "initials": "t."} @@ -5713,6 +5723,7 @@ {"name": "공옥진", "given": "옥진", "family": "공", "initials": "옥. 공."} {"name": "권재진", "given": "재진", "family": "권", "initials": "재. 권."} {"name": "권중동", "given": "중동", "family": "권", "initials": "중. 권."} +{"name": "김 v", "given": "김", "family": "v", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "김. v."} {"name": "김 민준", "given": "민준", "family": "김", "initials": "민. 김."} {"name": "김, 민준 MA", "given": "민준", "family": "김", "suffix": "MA", "ambiguities": ["SUFFIX_OR_NAME"], "initials": "민. 김."} {"name": "김, 민준씨", "given": "민준", "family": "김", "suffix": "씨", "initials": "민. 김."}