Skip to content

perf(sync): keep re-emitted synthesized rows on publish instead of rewriting the set - #577

Merged
bompus merged 1 commit into
fork/consolidatedfrom
perf/sync-synth-reconcile
Oct 28, 2026
Merged

perf(sync): keep re-emitted synthesized rows on publish instead of rewriting the set#577
bompus merged 1 commit into
fork/consolidatedfrom
perf/sync-synth-reconcile

Conversation

@bompus

@bompus bompus commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

  • Synchronized-edge refresh now reconciles instead of rebuilding: resynthesize captures the emitted edge set; stored synthesized edges a pass re-emitted keep their rows, only un-emitted ones are deleted, and drifted metadata updates in place (QueryBuilder.reconcileSynthesizedEdges).
  • saveSynthSkips no longer rewrites a row whose merged result is identical to what's stored.
  • Failure semantics unchanged: a failed refresh still drops all synthesized edges (readers see missing edges, never wrong ones, until the retry lands — same end-state as the old delete-up-front).

Why

  • Root-caused the discourse sync regression via page-level DB diffs: a one-file sync changed 1,386 DB pages vs upstream's 71. The churn was idx_edges_* leaves (~800 identical edges deleted+recreated by the unscoped delete-all + full re-synthesis) plus ~900 synth_skips leaf pages (20k rows rewritten every refresh). Upstream's input-scoped refresh never fires on config/boot.rb; ours recomputed over the whole graph — the precision guarantee stays — but now publishes only the delta.

Measured

  • Discourse one-file sync: filesystem outputs 35,832 → 2,784 sectors (upstream: 4,600); changed DB pages 1,386 → 80 (upstream: 71); synthesized-edge rowid churn 787 → 0 replaced, 0 added, 0 removed — byte-identical graph.

Test plan

  • sync-rebuild-convergence (25), sync-resynthesis (13), synth-skips — 42/42 pass on this base
  • Lab-verified on discourse lab copy: identical graph, same node/edge counts

@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 40 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 76529b1 and d8ab781.

Files selected for processing: 4
  • src/codegraph.ts
  • src/db/queries.ts
  • src/resolution/callback-synthesizer.ts
  • src/resolution/index.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@bompus

bompus commented Oct 28, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@bompus
bompus force-pushed the perf/sync-synth-reconcile branch from 7929861 to 31137ec Compare October 28, 2026 13:47
@bompus

bompus commented Oct 28, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@bompus
perf(sync): keep re-emitted synthesized rows on refresh instead of re…
d8ab781 
…writing the set

A one-file sync deleted every synthesized edge and reinserted them all, and
rewrote each recorded synth_skips row — ~800 identical edge rows (plus their
index pages) and ~900 skip-table leaf pages churned on a 20k-file repo where
upstream's input-scoped refresh writes ~70 pages. The refresh still recomputes
over the whole graph — the precision guarantee stands — but publish now
reconciles: re-emitted edges keep their rows, only stale ones are deleted, and
drifted metadata updates in place. synth_skips skips the write when the merged
row is identical. Failure paths keep the old drop-all semantics (missing
edges, never wrong ones). Measured on discourse: filesystem outputs 35.8k ->
2.8k sectors on a one-file sync, below upstream's 4.6k; changed DB pages
1,386 -> 80 vs upstream's 71.
@bompus
bompus force-pushed the perf/sync-synth-reconcile branch from 31137ec to d8ab781 Compare October 28, 2026 14:04
@bompus

bompus commented Oct 28, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@bompus
bompus merged commit 705a9ab into fork/consolidated Oct 28, 2026
4 checks passed
@bompus
bompus deleted the perf/sync-synth-reconcile branch October 28, 2026 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@bompus