diff --git a/README.md b/README.md index 69fee697e..1f2e7e3f1 100644 --- a/README.md +++ b/README.md @@ -198,7 +198,7 @@ Call resolution the fork adds: ### Measured results -Upstream `main` at `69af19ac` against the fork at `891e1966`, each run on Node.js 24.22.1 and on Bun 1.4.4. Index and sync figures were re-measured 2026-10-27 on the same 16-vCPU WSL2 host after the ambient-`declare module` scan fix and the resolve-pipeline changes below; MCP, command-line and explore figures are carried from the earlier 2026-10-27 run at `bc27fdb4` (unchanged by resolution-path changes). Seven corpora, arms in mirrored order; index figures are the median of two runs and sync figures the median of four. A comparison counts as a change only when the gap exceeds that comparison's own run-to-run spread — the larger of the two arms' (max − min) / median — and cells that cannot show that are marked inconclusive or no clear change rather than a fixed band. Upstream refuses to start on Bun, which reports itself as Node 26, because upstream blocks Node 25 and newer; it ran here with `CODEGRAPH_ALLOW_UNSAFE_NODE=1`. The host shared other work during the run, so spread-based cells carry more noise than a quiet host would; the numbers, the method, the graph sizes and the disclosures are in [`docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md`](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md). The scripts that produced them are in [`scripts/benchmarks/runtime/`](scripts/benchmarks/runtime/). +Upstream `main` at `69af19ac` against the fork at `6b2e7ae1`, each run on Node.js 24.22.1 and on Bun 1.4.4. Index and sync figures were re-measured 2026-10-27 on the same 16-vCPU WSL2 host after the ambient-`declare module` scan fix and the resolve-pipeline changes below; MCP, command-line and explore figures are carried from the earlier 2026-10-27 run at `bc27fdb4` (unchanged by resolution-path changes). Seven corpora, arms in mirrored order; index figures are the median of two runs and sync figures the median of four. A comparison counts as a change only when the gap exceeds that comparison's own run-to-run spread — the larger of the two arms' (max − min) / median — and cells that cannot show that are marked inconclusive or no clear change rather than a fixed band. Upstream refuses to start on Bun, which reports itself as Node 26, because upstream blocks Node 25 and newer; it ran here with `CODEGRAPH_ALLOW_UNSAFE_NODE=1`. The host shared other work during the run, so spread-based cells carry more noise than a quiet host would; the numbers, the method, the graph sizes and the disclosures are in [`docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md`](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md). The scripts that produced them are in [`scripts/benchmarks/runtime/`](scripts/benchmarks/runtime/). ![Fork against upstream CodeGraph on n8n and the command line: the fork indexes n8n 1.5 times faster, syncs an edited file 1.3 times faster, answers a warm question 1.8 times faster and uses 33% less memory while indexing. The fork is better in 57 of 76 comparisons, shows no clear change in 10 and is worse in 9.](assets/fork-vs-upstream-performance.svg?v=8) @@ -220,7 +220,7 @@ A [2026-10-15 mid-size comparison](docs/benchmarks/midsize-sweep-2026-10-15.md) | supabase (React + Next.js + TS, 10,716 files) | 18.2 s, 4.86 GiB | 15.5 s, 5.45 GiB | 14.6 s, 4.73 GiB | | n8n (Vue + TS, 24,434 files) | 74.9 s, 12.39 GiB | 49.5 s, 8.26 GiB | 45.2 s, 8.28 GiB | -The fork's index was slower than upstream's on five of the seven corpora in the first pass of this run — the supabase and n8n regressions traced to upstream's ambient-`declare module` lookup (merged in fork #548): the Rust port scanned every declaration file per out-of-repo import. A per-file memo ([#554](https://github.com/bompus/codegraph/pull/554)) ended the reparse-every-reference quadratic, and the follow-up above lands the rest: a port of upstream's text tokenizer plus a `specifier → files` index, so each import checks only candidate declaration files instead of every `.d.ts`. The re-measured numbers in this table are the result: supabase went from 41.5 s to 15.5 s on Node and n8n from 178 s to 49.5 s — n8n now beats upstream 1.5× and CPython 2.2×. The remaining index losses from that run — pretix +33% and Alamofire +18% on Node, and the Bun arm of supabase and n8n — traced not to the ambient scan but to the resolve loop's batch boundaries: every 5,000-ref page drained the worker pipeline before the next fanned out. This run stacks three fixes for that — 40,000-ref pages with a 12-worker pool cap (same batch-boundary semantics; pool workers resolve against a fixed snapshot), then bounded work-stealing dispatch with earlier batch fan-out and non-blocking worker recycling, then a memoized near-duplicate pass that stopped re-tokenizing bodies it had just signed — and those losses are gone: pretix-Node now wins (8.14 s against 9.04 s), the Bun arms of supabase and n8n win, and pretix-Bun is within ~2% of upstream (the last residual). The benchmark doc has the [full accounting](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md#the-supabase-resolution-regression). The trade is memory: indexing peaks higher on pretix and supabase now, lower on the other five, and its one-file sync is faster on six of seven (below). +The fork's index was slower than upstream's on five of the seven corpora in the first pass of this run — the supabase and n8n regressions traced to upstream's ambient-`declare module` lookup (merged in fork #548): the Rust port scanned every declaration file per out-of-repo import. A per-file memo ([#554](https://github.com/bompus/codegraph/pull/554)) ended the reparse-every-reference quadratic, and the follow-up above lands the rest: a port of upstream's text tokenizer plus a `specifier → files` index, so each import checks only candidate declaration files instead of every `.d.ts`. The re-measured numbers in this table are the result: supabase went from 41.5 s to 15.5 s on Node and n8n from 178 s to 49.5 s — n8n now beats upstream 1.5× and CPython 2.2×. The remaining index losses from that run — pretix +33% and Alamofire +18% on Node, and the Bun arm of supabase and n8n — traced not to the ambient scan but to the resolve loop's batch boundaries: every 5,000-ref page drained the worker pipeline before the next fanned out. This run stacks three fixes for that — 40,000-ref pages with a 12-worker pool cap (same batch-boundary semantics; pool workers resolve against a fixed snapshot), then bounded work-stealing dispatch with earlier batch fan-out and non-blocking worker recycling, then a memoized near-duplicate pass that stopped re-tokenizing bodies it had just signed — and those losses are gone: pretix-Node now wins (8.14 s against 9.04 s), the Bun arms of supabase and n8n win, and pretix-Bun is within ~2% of upstream (the last residual). Later in the same stack, the deferred-decode path stopped turning off on framework projects: a framework `extract()` hook no longer forces every file onto decoded-object extraction — its output rides the kernel-buffer transport and merges post-decode ([#563](https://github.com/bompus/codegraph/pull/563); pretix went from 0 to 1,416 kernel-path files, n8n-Node index RSS −10%). The benchmark doc has the [full accounting](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md#the-supabase-resolution-regression). The trade is memory: indexing peaks higher on pretix and supabase now, lower on the other five, and its one-file sync is faster on six of seven (below). Upstream at `6560052` took 2.0 to 2.7 times as long to index Python as at `290e03f` (pretix 9.2 s to 18.8 s, CPython 35.6 s to 95.3 s; [a same-host check](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md#since-2026-09-28) put the cause in upstream's code). At `69af19ac` upstream's CPython index is slower than at `986de95c` (55.0 s against 40.2 s on this host) while the fork's keeps improving, so the fork's CPython lead is 2.2× this run.