[waleedlatif1]

Summary

• A completed listing's reconciliation read every window of the connector to count, then the ACL, soft-delete and hard-delete walks each read the same windows again: up to four passes over a connector whenever any removal was due, the count pass cold and the walks warm right after it
• tallyReconciliationWindows replaces countReconciliationWindows: the same one-statement-per-window keyset walk over doc_connector_reconciliation_v2_idx now also returns, per condition, the ids (and tombstone flag) of the matching rows, each condition evaluated once per row
• visitReconciliationMatches pages those retained rows to the existing removal callbacks, so a completed listing reads each window once; the walks stay only as the fallback for a condition matching more than RECONCILIATION_RETAINED_MATCHES (10,000), which keeps memory and each window's result bounded
• Every decision is unchanged: the same four counts feed the suspect-listing guard, the per-generation cap and its hold notice, ACL revocation still runs when removals are held, and every removal still re-checks its target in its own statement (soft and ACL writes re-check absence, hard deletes keep the connector/exclusion/archive guard). The rows acted on are now exactly the rows counted, rather than whatever a later walk found
• Prod (plain EXPLAIN and one replica EXPLAIN ANALYZE of a full window): the tally keeps the old plan, a 5,000-step recursive Limit 1 probe of doc_connector_reconciliation_v2_idx with no other document access, plus an in-memory sort of the window. A cold window is I/O-bound (~1.3s, nearly all of it reads), a warm one ~47ms
• Effect: a completed listing with removals due issues one statement per window instead of two to four. Over the last day of query insights, the soft- and hard-delete walk windows this removes accounted for ~60% of the rows reconciliation windows read and ~10% of their DB time. The cold count pass itself remains, since it has to read every owned row's seen stamp
• Not changed: window size, the index, source_seen_at stays out of every index (HOT updates), member-resurrection walks, statement timeouts

Type of Change

• Other: performance

Testing

• sync-content-pass.integration.ts (real Postgres) gains a suite for the removal decisions: under, at and over the cap for soft, hard and both together (exact hold notices, ACLs revoked either way), full sync past the cap, an empty connector, a connector of three windows (exact soft/hard/ACL sets, 3 tally statements and no walks), more than 10,000 removals (walk fallback, each purged exactly once in pages of 25), a user excluding a tallied document mid-pass (skipped) and one in a later window (never counted), and a deadline cut mid-removal resumed by the next pass (each removed once)
• The same suite run against the pre-change code: every decision test passes unchanged; only the statement-count assertion fails (3 tallies + 9 walk statements before, 3 + 0 now)
• Mutation checks: acting on truncated rows past the bound, keeping only the first window's rows, and an off-by-one bound each turn a test red
• listing-continuation.integration.ts reconciliation-window tests now assert no walks and keep the EXPLAIN ANALYZE bound (every tally statement reads at most one window) on the real schema
• Unit: reconciliation-window.test.ts (sums and retains across windows, bound edge), plus the updated sync-content-pass, sync-engine and member-sync-engine tests; vitest run lib/knowledge (1,793 tests)
• bun run lint, bun run type-check, bun run check:audits, check:test-types, block-registry check

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing (new tests pass the test-audit authoring gate)
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Oct 29, 2026 6:33pm UTC

[cubic-dev-ai]

No issues found across 9 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

_{Turn on auto-fix | Re-trigger cubic}

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

Completed listings now count documents and retain removal candidates during one windowed pass.

• Completed listings reuse each connector window for removal work.

No actionable issues were found. Tests were inspected but not run during this review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Completed listing] --> B[Tally each document window]
  B --> C[Apply listing safety checks and removal caps]
  C --> D{Matching rows retained?}
  D -->|Yes| E[Page retained rows]
  D -->|More than 10000 matches| F[Walk windows again]
  E --> G[Run guarded removal callbacks]
  F --> G

Loading

_{Reviews (1) · Last reviewed commit: "perf(knowledge): reconcile a completed l..." · Reviewed by Greptile}