Repository navigation
fix(code-graph): charge interactive catalog builds to resident memory - #4563
ScriptedAlchemy wants to merge 2 commits into
Conversation
The catalog warm that follows a publication builds the interactive catalog without a resident-memory reservation, so admission cannot see it. This test mounts a worktree on the persistent graph runtime and fails while the warm reserves nothing. Refs #4559 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Graph activation warmed the interactive catalog after publication without a reservation. On a fresh t3code clone occupancy rose by about 1.2 GB during the 17 s build while 268 MB was reserved, so a concurrent cold build was admitted against occupancy that excluded the warm. Graph activation now binds the daemon's resident-memory authority to each seated graph store. Every catalog build (the post-publication warm, a re-warm after a memory release, and a successor carry) grows a `code_graph.catalog.build.v1` reservation with what it holds, the way cold construction charges `code_index.construction.v1`. A build that does not fit stops with the typed resident-memory refusal and leaves the catalog warmable for the next read instead of failed. The charge drops once the catalog is published and measured as `graph_catalog`. The scan now records every entity before the first relation, so each `calls`/`uses` edge folds into file dependencies and each import link is validated as it streams past. It no longer holds every edge's endpoint ids or a clone of every import relation until the end. Closes #4559 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| // A carried catalog is a full successor of the parent's size. | ||
| charge.cover(usize::try_from(parent.retained_bytes()).unwrap_or(usize::MAX))?; | ||
| let carried = { |
There was a problem hiding this comment.
🔴 Successor catalog carry exceeds its reservation
When a layered successor carries its predecessor, charge.cover reserves only the predecessor's size. The carry clones its maps and allocates edits without another admission, so the build can exceed the resident-memory limit.
Learn more
A layered generation can derive its catalog from a ready predecessor instead of scanning all rows. carry_interactive_catalog gathers changed rows and builds a successor catalog by cloning the predecessor's sorted maps. Reserving the predecessor's retained size only covers roughly one catalog, but the predecessor remains resident while a successor and temporary edits are allocated. There is no later cover call on the successful carry path, so a large refresh can allocate beyond the admitted amount.
Example: A predecessor catalog occupies 600 MB. The successor initially reserves 600 MB, then clones much of the predecessor into another catalog while the first stays live. The operation can cross a 1 GB resident-memory limit without another refusal.
Recommended fix: Account for the peak memory of the carry, including the retained predecessor, changed-row buffers, sorted-map clones and temporary edits. Grow the reservation before each material increase, or reserve a validated upper bound before calling carry_interactive_catalog; refuse before allocating beyond it.
Was this helpful? React with 👍 or 👎 to provide feedback.
| charge.cover(scan.held_bytes().saturating_add(scan.frozen_bytes()))?; | ||
| { | ||
| let _span = tracing::trace_span!("code_graph.catalog.finish").entered(); | ||
| scan.finish(projection_node_count) |
There was a problem hiding this comment.
🔴 Final catalog allocations escape memory admission
When a scan nears the resident-memory limit, charge.cover admits its trees and frozen arrays but not final aggregates. finish builds file adjacency and rankings afterward, so the catalog can exceed the limit without refusal.
Learn more
The final cover call uses CatalogScan::frozen_bytes, which counts sorted arrays for existing maps. CatalogBuilder::finish also allocates a file-language map, file ranking, and fold_file_dependencies allocates a hash adjacency with cloned file paths. These allocations happen after the last admission check. On a graph with many files or dependency pairs, this is meaningful additional memory even though the scan's reservation is already close to the limit.
Example: A scan's trees and frozen arrays fit a 1 GB policy with 10 MB to spare. Finalizing thousands of distinct file dependencies builds an additional hash adjacency and cloned strings beyond that spare capacity, but the build publishes successfully.
Recommended fix: Include bounded transient and retained allocations of CatalogBuilder::finish in the final reservation, including adjacency buckets, path clones, language counts and file rankings; grow the reservation before allocating these aggregates.
Was this helpful? React with 👍 or 👎 to provide feedback.
| let slot = occurrence | ||
| .as_str() | ||
| .len() | ||
| .saturating_add(2 * size_of::<SymbolOccurrenceId>()); |
There was a problem hiding this comment.
🟡 Unique symbol lists undercharge catalog builds
When many names have one symbol, slot budgets two list entries while each new vector allocates four. The catalog scan undercharges its live lookup lists, so memory admission can accept a build beyond its limit.
Learn more
Each new lookup-key vector uses Rust's initial Vec capacity, which is four entries for SymbolOccurrenceId, not two. CatalogBuilder::insert can create both a qualified-name and a simple-name vector for every symbol, and CatalogBuilder::held_bytes relies on the accumulated heap for their allocation. When most names are unique, the two-entry estimate undercounts both vectors throughout the scan. This is separate from any memory allocated during finalization.
Example: For 100,000 symbols with distinct qualified and simple names, 200,000 one-element lists each allocate four slots while the reservation counts two per list.
Recommended fix: Track actual vector capacities, or conservatively bound Vec's initial allocation as well as subsequent growth when charging list slots. Verify the bound against the per-thread allocation measurement for predominantly unique names.
Was this helpful? React with 👍 or 👎 to provide feedback.
Closes #4559
Problem
After a publication, graph activation warms the interactive catalog (
code_graph.catalog.build) with no resident-memory reservation. On a fresh t3code clone, occupancy rose by about 1.2 GB during the 17 s build while 268 MB was reserved. A cold build admitted beside the warm is sized against occupancy that leaves out everything the warm still has to allocate, and nothing stops the warm itself at the limit.Root cause
PendingInteractiveCatalogWarmV1::runbuilds the catalog on a graph store that holds no memory authority, sobuild_interactive_catalognever reserves anything. The same gap applies to the re-warm a read starts after a memory release and to a successor's carried catalog.The scan also kept data the catalog never keeps. Entities and relations were read in interleaved pages, so it held every
calls/usesedge's two endpoint ids and a clone of every import relation until the end. On this repository that was 193k edge endpoint pairs and 100k import relations.Fix
CodeGraphProjectionStore::from_verified_snapshot_charged).code_graph.catalog.build.v1reservation, the way cold construction growscode_index.construction.v1. The scan charges an upper bound of what it holds after each page, plus the sorted arrays it freezes into beforefinish. A carry charges the parent catalog's size first. The charge drops when the catalog is published, and from then on thegraph_catalogowner measures it.Released, so status says it was released for memory and the next graph read retries it. A refusal does not leave the catalog failed.Verification
Release build in an isolated profile (
TRACEDECAY_DATA_DIRandHOMEin a temp dir, socket dir mode 700).footprintgives the physical footprint peak.tracedecay status --jsongives occupancy (memory.resident_bytes) andmemory.reserved_bytes.Restart warm. Each run restarts the daemon on a store the baseline published, so the post-mount catalog warm runs on its own. There were 3 runs per side, alternating baseline and fix.
The fix numbers are medians. While the warm runs, the catalog charge peaks at 849 to 992 MB on tracedecay and 1367 to 1432 MB on t3code. The build takes as long as before, 13.4 to 14.0 s on tracedecay versus 13.6 to 13.9 s.
Fresh clone, cold pass, one run each.
Both peaks still come during the catalog warm. With the fix, the
code_graph.catalog.build.v1charge accounts for most of that occupancy (849 and 1170 MB). The tracedecay fix run used a build from before the final commit, which differs only in the post-refusal catalog state. One earlier t3code fix run never reached the warm. Under machine load, its sealed graph publication spent its background budget, which happens before the catalog and in code this PR does not touch.Admission on the real daemon. I restarted the tracedecay profile with
TRACEDECAY_RESIDENT_MEMORY_LIMIT_BYTES=1100000000. The baseline still warms the catalog and reaches a 1452 MB footprint, 32% over the limit. The fix refuses the build:The footprint peaks at 992 MB. Status reports
{"state": "warming", "reason": "code graph interactive catalog was released for memory; the next graph read re-warms it"}, and exact and lexical search keep serving.Tests.
the_catalog_warm_after_publication_reserves_resident_memory(crates/tracedecay) mounts a worktree on the persistent graph runtime. It fails on master (the catalog warm must reserve what it builds) and passes with the fix. It lands first, in its own commit.catalog_builds_reserve_the_heap_they_hold(crates/tracedecay-store-runtime) warms a real sealed graph and measures the build's heap peak with the per-thread counting allocator. An authority at 90% of that peak refuses the build with the typed refusal, keeps no charge, and leaves the catalog warmable. An authority at twice the peak admits the build and returns every byte. On a 4000-function fixture, the charge at its peak (found by bisecting the refusal threshold) is 1.14x the measured heap peak.bazel testof//crates/tracedecay-code-index/...,//crates/tracedecay-code-index-runtime/...,//crates/tracedecay-store-runtime/..., and//crates/tracedecay:unit_testall pass with three exceptions. Two tests also fail on master on this macOS host:unbounded_snapshot_captures_a_listing_beyond_the_default_read_limitandreplay_and_fence_read_awaited_on_a_native_worker_never_park_it. One test is flaky and passed on rerun:aborted_socket_curator_preparation_keeps_its_writer_owned_through_shutdown.bazel build --config=clippyon all four crates and//:rustfmt_checkare clean.Not addressed
At its peak, the scan still holds more than the frozen catalog. The B-tree builder has node slack, and
finishallocates each sorted array while the tree it replaces is still live. Removing that needs a sorted-vector builder. The charge covers this transient, but the PR does not shrink it.Related #4558. The re-warm a read starts after a memory release is now charged too. Under pressure that re-warm can be refused, and the read then gets the typed warming answer. This PR does not change how long the build or the engine reseat takes.
🤖 Generated with Claude Code