Repository navigation
fix(code-index): bound cold builds and resume refused publication - #4510
Merged
ScriptedAlchemy merged 12 commits intoOct 25, 2026
Merged
ScriptedAlchemy merged 12 commits into
ScriptedAlchemy merged 12 commits into
Conversation
|
A cold build the resident-memory checkpoint stops gives back what it built. The latch clearing behind that release must not wake its own retry; memory another holder gives back must. Refs #4505 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A cold reconcile the resident-memory checkpoint stops drops what it built. RSS falls under the low watermark, the latch clears, and the admission headroom epoch advances. Both the worker loop and the headroom watcher read that epoch as memory given back, so the pass rebuilt and was refused again on every maintenance sample. Record where measured RSS stood when such a pass began. Admission headroom now retries it only once the process measures below that point, and the bound registers with the resident-memory authority so a later sample under it still advances the epoch. Ledger refusals and passes admitted while already over the watermark keep retrying on the latch as before; owner releases by others still wake the retry. Fixes #4505 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptedAlchemy
force-pushed
the
fix/cold-pass-memory-retry-loop
branch
from
October 24, 2026 21:46
f083b0f to
3434b35
Compare
JSON and TOML documents whose keys read as high-entropy tokens are withheld whole today. Pin the target behavior: the key becomes a placeholder unique within its object, the values under it are redacted, and passphrase-shaped secrets stay redacted in every position. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A JSON or TOML object key that trips a credential or entropy detector withheld the whole document, because the sanitizer had no way to rewrite a key. Long identifiers used as keys (schema $defs type names, benchmark target names, path keys) score above the entropy floor, so files such as dashboard-contracts.schema.json were never indexed. The span-reporting JSON and TOML parsers already give each key's byte range. Replace a flagged key with a placeholder unique within its object (TraceDecay-redacted-key-N), redact every scalar under it, and keep the rest of the document byte-exact. Detection is unchanged: no token is exempted. A document is still withheld when the key has no span (YAML) or the rewritten text no longer parses as its format. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 25, 2026
ScriptedAlchemy
changed the base branch from
master
to
fix/privacy-redact-keys-in-place
October 25, 2026 01:30
This was referenced Oct 25, 2026
This was referenced Oct 25, 2026
ScriptedAlchemy
changed the base branch from
fix/privacy-redact-keys-in-place
to
fix/lcm-retry-transient-summarizer-failures
October 25, 2026 01:32
This was referenced Oct 25, 2026
This was referenced Oct 25, 2026
ScriptedAlchemy
merged commit Oct 25, 2026
aef1060
into
fix/lcm-retry-transient-summarizer-failures
16 checks passed
ScriptedAlchemy
added a commit
that referenced
this pull request
Oct 25, 2026
This was referenced Oct 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge #4478 first, followed by #4526, #4510, #4524, #4489, #4485, #4527, #4528, #4530, #4531, #4529, #4534, and #4541.
This PR is number 3 of 13, based on #4526. Merge #4524 next.
#4516 has already merged into this branch.
Use merge commits. After merging a parent PR, change its child's base to
masterbefore merging the child. Auto-merge is disabled.A cold build that exceeds the resident-memory limit now waits until enough memory is available for the full reconstruction. Cold publication also avoids holding a decoded repository alongside its encoded output.
The existing memory authority tracks allocations owned by the operation. It retains the full memory requirement after a refused build unwinds. A prepared publication can resume without repeating completed extraction. Worker admission measures current pressure and reserves memory for the installed worker count. Before activation, publication verifies every referenced digest and exact extent, including inherited segments.
Cold builds now reserve memory before replacement allocations and compact copies. They limit temporary carry data to one file, reuse authenticated unchanged segments, release resolution facts before building the graph, and defer projection rows. Symbol-name lookup and resolution-index page construction request memory in batches. Digest serialization streams explicitly ordered objects and preserves the exact bytes.
Fixes #4505.
Validation
Before the final integration of ancestor commits, the following tests passed:
Additional checks passed 96 retention tests, 4 resident-accounting tests, 75 query integration tests, and 27 evaluator tests. Changed benchmark callers compiled. Scoped Clippy and formatting checks passed.
The ignored scale fixture with 6,600 files and 422,400 symbols published under the unchanged 1 GiB limit in 241.64 seconds. Peak admission occupancy was 805,367,808 bytes. Peak RSS was 850,829,312 bytes. The run did not decode the full generation eagerly.
At stacked head
3e10b5fbb0a5d49b59c7fbe7216e43c6f10b0a2d, withnightly-2026-10-25and inherited dependencies, fresh execution passed 542 runtime-core, 357 index unit, 223 index integration, and 664 scheduler runtime tests. The 2 existing ignored tests remained ignored. All three affected formatting targets passed. Bazel invocation27ebabd4-4bd2-4576-83c9-64be264208ccexited with code 0.The scale result covers cold publication. It does not establish a bound for retained-text serving or every allocator. Existing limits in decoder preallocation and measurement of native, parser, and Go internal workspaces remain. No budget or timeout increased.
Publication cancellation correction
A source epoch changing inside final durable commit surfaced as
CompareAndSwapand could park the project. The scheduler now restores cancellation only when the attempt control confirms it, preserving genuine pointer conflicts and memory-refusal precedence. Fix: dab12b4.Focused regression BEFORE: 0 passed / 1 failed (
fa27d3a7-0aa4-4164-a350-6f943e556f4c). AFTER publication-store module: 45 passed plus formatting (836e4ba5-46ab-4200-aa71-af891b3acd38). Full daemon suite: 79 passed, including all three CI failures (711bbf0f-9197-456f-be73-3746d23f04bb). Scoped Clippy passed (bfa5119d-e483-4697-9846-3d10e9ff3330). These ran on the #4510 correction before stack propagation. The separate refresh-memory journey passed with existing phase tracing in 158.23s (a7c28c80-c892-4279-ba35-5d4beed34574), then passed again after removing all investigative edits: 2 tests, 0 failures, 157.87s (fc944727-2cd0-4620-b7bc-86cea6ea5047). Both completed cold indexing and all four refreshes under the unchanged 6 GiB limit and 300-second target timeout. The slower CI execution was not reproduced; no performance fix or timeout increase is claimed.