Skip to content

Split rules.md and decisions.md into one named file per entry - #715

Merged
derek73 merged 10 commits into
masterfrom
feat/split-design-docs
Oct 18, 2026
Merged

derek73 merged 10 commits into
masterfrom
feat/split-design-docs

Conversation

@derek73

@derek73 derek73 commented Oct 18, 2026

Copy link
Copy Markdown
Owner

Closes #713.

docs/design/rules.md (31k words) and docs/design/decisions.md (205k words, more than an agent's context window holds) are now folders of named files. Each folder has a README with the old preamble and an index:

  • docs/design/rules/: one file per rule (S2-case-signal-at-the-suffix-slot.md) and one per section Background (S-suffixes-generational-and-credentials.md).
  • docs/design/decisions/: one file per key (S2-case-signal-at-the-suffix-slot.md, vocabulary-collisions.md). The 11 differential-ledger arcs share one file. The vocabulary-keyed Excluded:/Open: blocks are gathered into the new vocabulary-exclusions.md, unedited and in their old order. They had no heading of their own, so the split had filed them by position into P3, W2, P5 and R4.
  • mechanisms.md is unchanged.

Citations are now rules/S2 and decisions/key (mechanisms.md#X is unchanged). A script rewrote 1,347 of them across 159 tracked files. test_no_old_design_doc_anchors keeps the old form out of every tracked .py/.md/.rst file, because an old-form citation would no longer be read, and its quote would go unchecked. Citations carry only the ID, so renaming a file breaks nothing.

Names. 39 rules take the name their decisions entry already had. Seven are new: C2 "parts past the second comma", O3 "compound family names written unmarked", P4 "a particle at the front of the name", R1 "fields are views of the parsed words", S1 "a bracketed credential is not a nickname", S3 "credentials written with interior periods", T2 "the katakana middle dot divides like a space". The ledger file is named "the differential ledger's arcs". AGENTS.md now asks that a rule cited in chat, a commit or a PR give its name with its ID.

Verification

  • Lossless split: the migration script (committed in a839084, deleted in 526630e) refused to write anything unless joining the pieces back reproduced both files byte for byte. It printed round trip: rules.md and decisions.md both byte-identical. The final review rebuilt both documents independently. The only differences it found are the intended layout-describing prose edits, and it also found the NFD example lines intact. corpus_rules.jsonl regenerates byte-identical.
  • No citation lost: before the rewrite the checker saw 990 references; afterwards it sees 1,011. Diffed by file, line and key: 0 lost. The gains are prose mentions the sweep turned into real, checked citations, plus three keys wrapped at a hyphen across lines that the old checker never read. The first version of the new pattern dropped those three wrapped keys, which a set diff caught. test_mention_re_reads_a_key_wrapped_at_its_hyphen now pins that shape, and a floor (>= 1000) guards against losing a whole shape.
  • Negative control: citing rules/S99, misquoting an M3 excerpt, and appending rules.md#S2 to docs/design/AGENTS.md each fails test_doc_citations.py (3 failures). Breaking the folder names in MENTION_RE drops the count to 102 and fails the floor.
  • New layout tests (tests/v2/test_design_docs.py): every file is indexed exactly once; each index title matches its file's heading and file name; every key resolves to exactly one file; rules/S resolves to the section file, never to S1-…; a sentence-final rules/S2. is a citation; a file path is not.
  • Full suite (20,867 passed), mypy, ruff, the Sphinx HTML and doctest builds (no output), and python -m doctest README.rst.

Reviews

A whole-branch code review and a design-docs review ran in parallel, and a third reviewer checked the fix commits. Every finding is fixed:

  • Prose still describing the old layout (grep deviates: on "this file", "this file is the record", the guard list in docs/design/AGENTS.md).
  • The misfiled vocabulary blocks.
  • Thirteen positional references ("the bullet above", "this file's Excluded block") left pointing at nothing by the move. The lesson is recorded in docs/design/AGENTS.md.

Known and left alone

  • A gap in the citation check. A decisions key starting with a digit (decisions/3-0-reevaluations) is still not read by the code-side check, same as before the split. The docs-internal check does read it.
  • History keeps the old names. Dated decisions entries and past release-log bullets still say rules.md/decisions.md, since they record what the files were then. Bare #key shorthands inside dated entries (about 90) now read like issue numbers.
  • Deferred minors: MENTION_RE has no leading word boundary; nothing checks that each rule file holds the rule its name claims; a rules/P5-style mention is silently not a citation.

🤖 Generated with Claude Code

derek73 and others added 10 commits October 17, 2026 20:51
tests/v2/design_docs.py is the one place that will know how
docs/design/rules/ and docs/design/decisions/ are laid out: the README
index, file-name slugs, the rules/ID citation pattern and how a
citation resolves to its file. Nothing reads it yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A one-shot migration, deleted once the split lands. `split` refuses to
write unless joining the pieces back reproduces rules.md and
decisions.md byte for byte; a dry run on 901ab66 gives 59 files for
rules/ and 68 for decisions/, both round-tripping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"the title run's floor" slugged to title-run-s-floor; it is
title-runs-floor now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/design/rules/ holds a README (the old preamble, "Not in scope"
and an index), one file per section Background and one per rule;
docs/design/decisions/ holds a README (the entry conventions and an
index grouped by rule section, then named topics) and one file per
entry, the eleven differential-ledger arcs in one file. Each file is
named <ID>-<name>.md, a slug-keyed entry <slug>.md.

Generated by `uv run python tools/split_design_docs.py split`, which
printed "round trip: rules.md and decisions.md both byte-identical":
joining the pieces back reproduces both files byte for byte, and the
script writes nothing unless both do. The only text added is each rule
or entry file's `# ID — name` heading and the two indexes.

The parsers read the folders through tests/v2/design_docs.py; the
citation form is unchanged in this commit. corpus_rules.jsonl
regenerates byte-identical. The checker sees 991 citations (153 with
quotes): 990/152 on 901ab66 plus one fixture string in the new
test_design_docs.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generated by `uv run python tools/split_design_docs.py rewrite`:
1347 citations in 159 tracked .py/.md/.rst files. The checker reads
the new form through design_docs.MENTION_RE, and
test_no_old_design_doc_anchors keeps rules.md#/decisions.md# out of
every tracked file, since an old-form citation would no longer be read
and its quote would go unchecked.

Every citation the checker saw before the rewrite it sees after
(diffed by file, line and key); the count moves 991 -> 992 only by two
citations in design_docs.py's docstring and one fixture string, now
exempt with test_design_docs.py's other citation-shape fixtures. A key
wrapped at its hyphen at the end of a comment line (mechanisms.md#ONE-
PREDICATE-PER-, three sites) is matched again, pinned by
test_mention_re_reads_a_key_wrapped_at_its_hyphen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every prose mention of rules.md or decisions.md that describes the
current docs now names the folders (rules/, decisions/) or the entry
it means; dated history entries keep the old names, since they record
what the files were then. AGENTS.md says each rule and decisions key
is its own named file indexed by its folder's README, and asks that a
rule or decision cited in chat, a commit or a PR carry its name with
its ID. decisions/README.md says how to add an entry under the new
layout.

Three wrapped references in test_order_correspondence.py
(decisions.md / #two-input-invariants, and two #P6 ones) had never
been read by the citation check; joined onto one line, they are now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tools/split_design_docs.py did the split and the citation rewrite;
history keeps it. The citation check no longer needs to exempt it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- docs/design/AGENTS.md lists test_design_docs.py among the guards a
  reviewer can skip, and root AGENTS.md names the loader and its test.
- rules/README.md: the deviation backlog is a grep over the folder,
  not over the README, which holds only the grammar's mentions.
- decisions/README.md: the folder is the record, and its cross-
  references use the decisions/key form.
- test_doc_citations.py: a floor on the citations the sweep finds
  (1,004 today), and an assert that the docs-side anchor pattern really
  differs from MENTION_RE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
decisions.md kept the blocks keyed by a vocabulary set (Excluded
(TITLES), (MAIDEN_MARKERS), (SUFFIX_ACRONYMS — esq), the contested-
memberships Open block, ...) after whichever entry preceded them, so
the split filed them by position into the P3, W2, P5 and R4 entries,
where nothing in the index says they are. Both reviews of the split
flagged it: a wordlist editor following a comment to the TITLES
exclusions faced 68 files. They now live, unedited and in their old
order, in decisions/vocabulary-exclusions.md, which the README names as
the home for any new set's block; every comment and doc that pointed at
one of them points there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…713)

The review of the block move found thirteen positional references left
pointing at nothing: three inside the moved blocks ("the bullet of
that date above", "see the entry above", "the item-4 bullet
above") and ten entries that located a block by position ("the
Excluded block below", "this file's Excluded block", "the W2
section"), plus titles.py's salutation comment. Each now names
decisions/vocabulary-exclusions or the entry it meant.

Also: decisions/README.md no longer claims one block per set, and says
an entry may still record keep-outs inline (generational-words); the
CAPITALIZATION_EXCEPTIONS pointers name which of its two blocks; the
citation floor's comment says what 1,000 can and cannot catch. The
lesson goes in docs/design/AGENTS.md: a move breaks positional
references, and the citation check cannot see them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added docs Documentation fixes and updates tests labels Oct 18, 2026
@derek73 derek73 self-assigned this Oct 18, 2026
@codecov

codecov Bot commented Oct 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.03%. Comparing base (901ab66) to head (3dac055).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #715   +/-   ##
=======================================
  Coverage   99.03%   99.03%           
=======================================
  Files          46       46           
  Lines        4557     4557           
=======================================
  Hits         4513     4513           
  Misses         44       44           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@derek73
derek73 merged commit e7e84c6 into master Oct 18, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation fixes and updates tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split rules.md and decisions.md into one file per entry, with named indexes

1 participant