Repository navigation
perf(audit-logs): read a listing narrowed only by time through per-workspace keyset scans - #9450
Conversation
…rkspace keyset scans An organization's audit listing is one OR predicate over the organization's workspaces and its organization-level rows. With no filter, or only a time range (the audit page's default view), PostgreSQL has no index that returns that OR in listing order, so it collects every row of the scope inside the range and sorts it for each page; for a busy organization that is a scan of a large part of the table. Such a listing now reads its page ids as a disjoint union: one keyset scan of audit_log_workspace_created_at_id_idx per organization workspace, the organization-level metadata rows through audit_log_org_level_created_at_id_idx, and the organization resource rows that do not name it in metadata. Each part stops after a page, and the first page of the union is exactly the first page of the scope. The time range is kept verbatim and also stated as implied bounds on the sort key, so each scan starts and stops at the range; the cursor is the same keyset position written as a row comparison so the scan seeks to it. A listing with any other filter (action, resource, actor, workspace, search) keeps the single predicate unchanged, byte for byte: per-workspace scans cannot stop early under such a filter, which is what made #9419 slow, while the single predicate lets the filter's own index drive. queryAuditLogs now takes the scope and the filters rather than prebuilt SQL so it can make that choice. list-scope.integration.ts pages every filter shape, with and without departed members, across page sizes and the byte budget, and requires exactly the single predicate's order, including sub-millisecond ties, bounds inside a millisecond and a workspace the organization no longer holds. list-plans.integration.ts pins the keyset plan for the unfiltered and time-bounded shapes (failing on the previous single-predicate listing) and the single predicate for every other one.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
[Medium impact] The PR appears safe to merge; no new actionable issue was found. SummaryThe PR uses per-workspace keyset scans for audit listings filtered only by time. Other filters keep the existing query.
The earlier unnumbered finding is addressed by the changed actor mix.
|
…stly by a departed member
|
@cubic-dev-ai review this PR |
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
Summary
An organization's audit listing scopes rows with one
ORpredicate (the org's workspaces, or workspace-less rows naming the org). With no filter, or only a time range (the audit page's default "Past 30 days" view), no index returns thatORin listing order, so each page collects every in-scope row inside the range and sorts it. For a busy organization that becomes a scan over a large part ofaudit_log, long enough to hit replica recovery conflicts.This PR changes only those listings, the ones narrowed by nothing but a time range:
audit_log_workspace_created_at_id_idx, throughunnest(...) cross join lateral (... limit page). The metadata-attributed org-level rows are read throughaudit_log_org_level_created_at_id_idx(0444). The organization-resource rows that don't name the org in metadata get a smallresource_idxread. Every part is ordered and limited to one page, so the first page of the union is exactly the first page of the scope, andunion allcan't produce duplicates because the parts are disjoint. Workspace ids are deduplicated.created_at >= / <=). It is also stated as implied bounds on the sort key (date_trunc('milliseconds', created_at) >= / <= date_trunc(bound)). Becausedate_truncnever decreases, these bounds drop no row the exact predicate keeps, and each scan starts and stops at the range.Any other filter (action, resource type/id, actor, actor email, workspace, search, alone or combined with time) keeps the single predicate unchanged, SQL and parameters byte for byte. Under such a filter a per-workspace scan cannot stop after a page; that is what made #9419's fan-out slow, and #9438 reverted it. The single predicate lets the filter's own index drive.
queryAuditLogsnow takes the scope and the filters instead of prebuilt SQL so it can make this choice. Callers updated: list and export use cases,/api/v1/audit-logs.buildOrgScopeConditionoutput is unchanged, so the detail read and the data drain are untouched.No schema change.
audit_log_workspace_created_at_id_idxstays: this path depends on it, and workspace-filtered listings already use it today.Type of Change
Testing
list-scope.integration.tspages through 13 filter shapes (none, start, end inside a millisecond, range, action, action+time, resource type(s), resource, actor, workspace, search). Each runs with current members only and with departed members included, at page sizes 1/3/7/100, with and without the byte budget. Every run must match the single predicate's order exactly. The fixture covers workspace rows, org-level rows of both kinds, another org's rows, departed and null actors, four rows per PostgreSQL millisecond (ties broken by id), a duplicated workspace id, and a workspace the organization no longer holds.list-plans.integration.tschecks the plans. Unfiltered, departed-included, after-cursor, start-date and date-range listings must read each workspace with anIndex Scan Backwardon the keyset index directly under the limit that stops it, with the range and cursor in itsIndex Cond. These checks fail on the previous single-predicate listing. Every narrowed shape must keep the single predicate (no fan-out).staging.lint,check:audits,type-check,test:scripts.Checklist