gh-151289: optimize shrinking wide int subtraction

[KRRT7]

This specializes exact int subtraction for a narrow but useful case: on 30-bit-digit builds, when both operands are non-compact 2- or 3-digit PyLong objects with the same sign and top digit, subtraction often shrinks the result. The specialized path computes that result directly instead of using generic long subtraction.

This does not add a general wide-integer fast path, and compact subtraction remains on its existing path. Non-matching operands deopt/fall back to the existing implementation.

On an Ubuntu x86-64 host (GCC 15.2, 8 logical CPUs), corrected pyperf measurements were:

Case	Main	PR
Compact subtraction	7.56 ns	7.82 ns
Shrinking wide subtraction	16.2 ns	10.7 ns (1.52x faster)
Non-shrinking wide subtraction	18.1 ns	18.5 ns
Wide addition	18.2 ns	18.5 ns

The compact, non-shrinking, and addition differences were not statistically significant in this run.

A representative eight-benchmark pyperformance subset showed no significant geometric-mean change.

[skirpichev]

As I said in the issue thread, I'm not sure if this worth code complications.

Other than this, few remarks:

1. Probably, you should split this pr into several. For instance, separate freelists addition looks unrelated.
2. I don't think you should add benchmark script to the sources. Just include this code in pr description, for example.

[KRRT7]

For instance, separate freelists addition looks unrelated

sure, I'll revert and update the numbers, though I think @peendebak mentioned he had a independent freelist PR somewhere so I'll just drop mine entirely.

I don't think you should add benchmark script to the sources. Just include this code in pr description, for example.

sure,though it's standard practices to have a tests/benchmarks/ dir (see pytest-benchmark and projects that adopt it)

[skirpichev]

Do not click the "Update branch" button without a good reason because it notifies everyone watching the PR that there are new changes, when there are not, and it uses up limited CI resources.

[Fidget-Spinner]

I don't see any speedup, and in fact see a significant slowdown on the bigint path:

add_compact: Mean +- std dev: [fastpath_int] 16.7 ns +- 0.0 ns -> [main_int] 16.7 ns +- 0.0 ns: 1.00x slower
add_wide: Mean +- std dev: [fastpath_int] 31.6 ns +- 0.4 ns -> [main_int] 36.7 ns +- 0.8 ns: 1.16x slower
sub_compact: Mean +- std dev: [fastpath_int] 16.7 ns +- 0.0 ns -> [main_int] 16.7 ns +- 0.0 ns: 1.00x slower
sub_wide: Mean +- std dev: [fastpath_int] 31.4 ns +- 0.5 ns -> [main_int] 37.2 ns +- 0.7 ns: 1.18x slower

Geometric mean: 1.08x slower

The benchmark scripts in the original issue is not really measuring the operation I feel, so I modified it:

import pyperf


def bench_add_compact(loops):
    a = 1
    b = 2
    it = range(loops)
    start = pyperf.perf_counter()
    for _ in it:
        a + b
    return pyperf.perf_counter() - start


def bench_add_wide(loops):
    a = 10_000_000_000
    b = 1
    it = range(loops)
    start = pyperf.perf_counter()
    for _ in it:
        a + b
    return pyperf.perf_counter() - start


def bench_sub_compact(loops):
    a = 1
    b = 2
    it = range(loops)
    start = pyperf.perf_counter()
    for _ in it:
        a + b
    return pyperf.perf_counter() - start


def bench_sub_wide(loops):
    a = 10_000_000_000
    b = 1
    it = range(loops)
    start = pyperf.perf_counter()
    for _ in it:
        a + b
    return pyperf.perf_counter() - start


def main() -> None:
    runner = pyperf.Runner()
    runner.bench_time_func("add_compact", bench_add_compact)
    runner.bench_time_func("add_wide", bench_add_wide)
    runner.bench_time_func("sub_compact", bench_sub_compact)
    runner.bench_time_func("sub_wide", bench_sub_wide)


if __name__ == "__main__":
    main()

[KRRT7]

There’s a small bug in the benchmark script in that comment: the two subtraction cases are still doing a + b in the loop body.

def bench_sub_compact(loops):
    ...
    for _ in it:
        a + b

def bench_sub_wide(loops):
    ...
    for _ in it:
        a + b

Those should be a - b, otherwise the reported sub_* numbers are measuring addition again rather than subtraction.

[chris-eibl]

I am not a domain expert here, so unfortunately I can't do a thorough review. ¹

Footnotes

1. I only posted a brief comment on this PR earlier (https://github.com/python/cpython/pull/151290#discussion_r3408454170) ↩

[skirpichev]

I can't add something new, beyond my previous post in the issue thread.

[github-actions]

This PR is stale because it has been open for 90 days with no activity.

[eendebakpt]

This introduces branching in the _BINARY_OP_SUBTRACT_INT and also prevents the jit from eliminating guards (since they are now part of _BINARY_OP_SUBTRACT_INT). I am not convinced the wide int subtract case is important enough to make these changes. The PR also introduces the HAS_ESCAPES_FLAG for the opcode, preventing optimization.

There is also problem with the implementation:

Reproducer

"""Reproducer for a wrong-result bug introduced by python/cpython PR 151290
(gh-151289, "optimize shrinking wide int subtraction") when the JIT is on.

The PR makes BINARY_OP_SUBTRACT_INT accept non-compact 2- or 3-digit operands
with the same sign and top digit, and computes the difference directly.  For
3-digit operands the difference can be a 2-digit (non-compact) int.  The tier-2
abstract interpreter (optimizer_bytecodes.c) still models the result of
_BINARY_OP_SUBTRACT_INT as a *compact* int, so:

  * a following _GUARD_NOS_INT / _GUARD_TOS_INT on that value is removed, and
  * because the result is also marked "unique", the following int op is
    rewritten to its _INPLACE variant, which reads _PyLong_CompactValue()
    without any check.

On a 30-bit-digit release build, _PyLong_CompactValue() of the 2-digit int
2**59 reads ob_digit[0] == 0, so the next operation computes with 0 instead
of 2**59.

Run with a JIT-enabled build (PYTHON_JIT=1) to see the wrong result; run with
PYTHON_JIT=0 or on a non-JIT build to see the correct result.

Needs a 30-bit-digit build (sys.int_info.bits_per_digit == 30).
"""
import sys

N = 10_000      # comfortably above JUMP_BACKWARD_INITIAL_VALUE (4000)

# Warm-up operands: 2-digit, same top digit -> "shrink pair" on the PR, but the
# result (3) is compact, so the *second* op in each loop body specializes to
# the compact-int form too.
WARM_A = 2**31 + 5
WARM_B = 2**31 + 2

# Trigger operands: 3-digit, same sign, same top digit -> still a shrink pair,
# but the difference 2**59 needs two 30-bit digits, i.e. it is NOT compact.
HOT_A = 2**60 + 2**59
HOT_B = 2**60


def sub_sub(a, b, c, n):
    r = 0
    for _ in range(n):
        r = (a - b) - c
    return r


def sub_add(a, b, c, n):
    r = 0
    for _ in range(n):
        r = (a - b) + c
    return r


def sub_mul(a, b, c, n):
    r = 0
    for _ in range(n):
        r = (a - b) * c
    return r


def jit_state():
    try:
        return f"jit available={sys._jit.is_available()} enabled={sys._jit.is_enabled()}"
    except AttributeError:
        return "sys._jit not present"


def dump_uops(func):
    try:
        import _opcode
    except ImportError:
        return
    code = func.__code__
    for i in range(0, len(code.co_code), 2):
        try:
            ex = _opcode.get_executor(code, i)
        except ValueError:
            continue
        names = [item[0] for item in ex]
        interesting = [n for n in names if "BINARY_OP" in n or "GUARD" in n]
        print(f"    executor @{i}: {interesting}")
        return
    print("    (no executor found)")


def main():
    print(sys.version)
    print(jit_state())
    if sys.int_info.bits_per_digit != 30:
        print("needs a 30-bit-digit build; skipping")
        return 0

    cases = [
        ("(a - b) - c", sub_sub, 1000, (HOT_A - HOT_B) - 1000),
        ("(a - b) + c", sub_add, 1000, (HOT_A - HOT_B) + 1000),
        ("(a - b) * c", sub_mul, 3, (HOT_A - HOT_B) * 3),
    ]
    failures = 0
    for label, func, c, expected in cases:
        # Specialize + JIT-compile with compact results.
        func(WARM_A, WARM_B, c, N)
        # Re-enter the same executor with 3-digit operands.
        got = func(HOT_A, HOT_B, c, N)
        ok = got == expected
        failures += not ok
        print(f"{label:14} expected {expected:>22} got {got:>22}  {'OK' if ok else 'WRONG'}")
        if "-v" in sys.argv:
            dump_uops(func)
    return 1 if failures else 0


if __name__ == "__main__":
    sys.exit(main())

The speedup is genuine though: the fallback for non-specialized code is really slower. I would be interested to know whether we can somehow specialize for the compact case, but if failing we can specialize for the multi-digit case. That would be beneficial also for other operations with ints.

[KRRT7]

Thanks for the reproducer, confirmed. On a 30-bit PGO+LTO build with --enable-experimental-jit, all three cases go wrong on this PR (-1000, 1000 and 0 instead of the 2**59-based results), and they are correct on the merge base and on non-JIT builds.

The root cause is what you described: _BINARY_OP_SUBTRACT_INT can now return a non-compact int, but the optimizer still models its result as a unique compact int. The guard on the following int op gets removed, and that op is rewritten to its _INPLACE variant, which reads _PyLong_CompactValue() from a 2-digit value. Folding the guards into the op also cost the guard elimination and added HAS_ESCAPES_FLAG, as you pointed out, so I don't think this approach is salvageable as is.

Also, apologies for the confusion in my earlier reply: I said I'd narrowed this to the int64 path, but what's pushed is the shrink-subtract specialization. Please ignore that comment.

I'm reworking it along the lines you suggested: leave the compact specializations exactly as they are on main, and handle the multi-digit case separately. The simplest version I have uses BINARY_OP_EXTEND. When the operands aren't both compact, int + int / int - int (and the in-place forms) get a descriptor whose guard is a plain exact-int type check and whose action calls long_add/long_sub directly, skipping the binary_op1 dispatch. The result is typed as int but not as compact, so any later compact-int ops keep their guards. BINARY_OP_SUBTRACT_INT and the optimizer are untouched, it doesn't depend on PyLong_SHIFT, and it covers addition too. Your reproducer passes with it, and I've added it as a regression test in test_opt.

Numbers: wall-clock microbenchmarks on a cloud VM (isolated cores, pyperf system tune) turned out too noisy for a change this small. PGO layout alone moved untouched compact ops by ±5–8%, and repeated runs of the same LTO-only binary drifted by up to ~9%. So I measured user-space instructions per op instead (perf stat -e instructions:u, difference between 3M and 1M iterations, empty loop subtracted, 5 repeats each), on 30-bit builds of the merge base c22e9c9:

instructions/op	sub compact	add compact	sub 2-digit shrink	sub 3-digit shrink	sub wide	add wide
base (LTO)	126	127	256	365	348	347
rework (LTO)	128	127	221	330	313	315
base (PGO+LTO)	126	125	216	323	304	291
this PR (PGO+LTO)	125	125	186	369	305	294
rework (PGO+LTO)	127	127	182	291	270	258
base (PGO+LTO, JIT)	113	114	241	350	330	319
this PR (PGO+LTO, JIT)	116	113	198	348	331	317
rework (PGO+LTO, JIT)	113	112	215	292	272	261

So on this base the rework saves 26–58 instructions per wide add/sub, and compact ops are unchanged within noise. Rebased on current main (7dbc797), it saves 30–39 instructions per wide add/sub (~10–12%) in all three configurations, JIT included, again with compact ops unchanged. This PR only helps the 2-digit shrinking case and is 46 instructions worse on 3-digit shrinking without the JIT, which is another reason to drop it.

On pyperformance (LTO-only builds, no PGO, one run each, pinned to an isolated core) there's no measurable change. The geometric mean comes out at 1.03x faster, but benchmarks with no wide-int arithmetic moved ±10–20% in both directions (e.g. bench_mp_pool 1.20x faster, nqueens 1.14x slower), so I read that as noise. That's expected, since very little of the suite does non-compact int add/sub. The earlier PGO+LTO runs were worse: this PR and the rework both came out within 0–3% of base on the geometric mean, with similar swings. dask and genshi fail on 3.16 for every build.

I'll open the rework as a new PR and close this one, since the approach is different enough.

[KRRT7]

Closing in favour of #159688, which handles the multi-digit case via BINARY_OP_EXTEND and leaves the compact specializations unchanged.