Skip to content

Native insert_many: decide with a timed A/B whether to pause the cyclic garbage collector for a list or tuple input #237

Closed as not planned

Description

@tae898

Native insert_many: should it pause Python's cyclic garbage collector for a list or tuple input, as the JVM build now does (#235, PR #236)?

On the arcadedb-jvm build, insert_many spends 19% to 28% of its wall inside Python's cyclic collector (about 1,270 young and 32 to 46 full collections per 1,000,000-row load, triggered by the call's own allocations and paid on the caller's live row lists), and a pause inside the call took 0.819 to 0.903 of the wall (95% CI, #235). On the arcadedb (native) build the same untimed probe found zero collections inside insert_many (the crossing allocates no object the collector tracks), so the expected gain there is nothing. That is an expectation from untimed runs on earlier native wheels, not a timed measurement of the current native path.

The question is therefore open for native and is decided by a timed A/B, not by analogy: arm A is the native CI wheel of main, arm B is the same wheel with only _native/database.py replaced by the draft PR's file, arm Z is A again as the A/A control. The pass bar is the one the JVM study used (insert wall CI upper end at most 0.95, whole-load wall at most 1.00, peak RSS at most 1.10, A/A clean), plus a control load where the caller has already disabled the collector, which the change must not move. The rule, the round plan and the policy for a GRAY result are written down before the first timed run.

The draft PR carries the same guard as the JVM change (a list or a tuple is paused, a generator or any other lazy input is not, the previous state is restored in finally), written as a decorator on NativeDatabase.insert_many so that it does not touch the lines that other open native insert_many PRs change. It ships only if the A/B passes. If it does not, this issue is closed with the measurements and the native default stays.

Activity

  1. tae898 commented on Nov 4, 2026

    @tae898
    CollaboratorAuthor

    Closing as not planned: the timed A/B on PR 240 found no effect on the native build (insert and load B/A 1.000 and 1.001, inside the equivalence band), and no cyclic collection runs inside the native insert_many call, so a pause has nothing to remove. Details and figures: #240. The JVM build keeps its pause (PR 236).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions