Repository navigation
docs(code-index): design a smaller decoded generation - #4560
Open
ScriptedAlchemy wants to merge 1 commit into
Open
ScriptedAlchemy wants to merge 1 commit into
ScriptedAlchemy wants to merge 1 commit into
Conversation
The sealed codec already writes per-chunk revisions, the projection request, and clone token streams compactly. The decoded generation expands them, so the 3.24 GB that issue #4506 measured is a decoded representation cost, and no sealed-format revision is needed. The design orders four decoded-form changes, keeps format revision 21, explains why a later byte change rebuilds from source instead of migrating, and records how to measure each change. It also records the current cold-pass peaks: 1.58 GB on this repository and 2.45 GB on t3code, with the peak now in the uncharged interactive catalog warm. Refs #4506 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #4506
Problem
Issue #4506 reported a cold build that held a 3.24 GB decoded generation that admission did not charge. That build peaked at a 4.11 GB footprint on this repository.
What master already fixes
#4510 removed that hold. Cold construction now writes segments to the store preparation and charges what it retains as
code_index.construction.v1. A refused build parks until its demand fits.publish_atomicallyandCodeIndexSealedPublicationV1::encodeno longer exist. Those two functions are the frames the live beta.92 daemon is stuck in.I measured the release daemon at
bbfcad7c03in an isolated profile with a fresh clone. Each journey ran once.During construction, the reservations stayed above occupancy. The remaining peak comes after publication, from the interactive catalog warm, which takes no reservation. That gap is now #4559.
This PR
This PR adds a design doc and changes no code. The doc is
docs/plans/CODE-GENERATION-RESIDENT-SIZE-DESIGN.md, and it covers the sealed-codec candidates from the issue. The sealed codec already writes per-chunk revisions, projection rows, and clone token streams compactly. The 3.24 GB comes from expanding them during decode. The doc orders four decoded-form changes and keeps format revision 21. It explains why a later byte change rebuilds from source instead of migrating, and it gives the measurement method.🤖 Generated with Claude Code