gh-158675: Fix Pool.imap(buffersize=...) dropping results after close()

[oyiakoumis]

With buffersize, calling close() before the imap() / imap_unordered() iterator was exhausted made the iterator stop early without an error. The items not yet submitted to the workers were dropped.

This came from the fix for gh-155477 (gh-155478): close() woke the throttled task generator, and the generator stopped as soon as the pool was no longer in the RUN state.

Changes

• close() no longer releases the buffersize semaphores. The task generator keeps submitting items as results are consumed, so nothing is dropped.
• join() now sets a flag and releases the semaphores. From then on, the generator submits the remaining items without throttling. This keeps multiprocessing.Pool.imap(buffersize=...) deadlocks on close()/join() after partial consumption #155477 fixed: close() + join() on a partially consumed iterator does not hang.
• The task generator stops early only when the pool is terminated.

Behaviour after this change

Call	Remaining items	buffersize limit
close()	submitted as results are consumed	still applies
join()	all submitted	no longer applies
terminate()	dropped	not relevant

One consequence: close() + join() with buffersize on an infinite iterable no longer returns, which matches imap() without buffersize.

Tests and docs

• New test: close() followed by iteration returns every result.
• The existing multiprocessing.Pool.imap(buffersize=...) deadlocks on close()/join() after partial consumption #155477 test now also checks that all results are available after close() + join().
• Added a sentence to the imap() documentation about join() and buffersize.

No NEWS entry, because buffersize is new in 3.16 and has not been released (same as gh-155478).

• Issue: Pool.imap(buffersize=...) silently drops results after close() #158675

I used Claude Code (Claude Opus 5.5, effort: Medium) to help investigate the bug and develop the fix, tests, and whatsnew entry. I reviewed, understood, modified, and tested the resulting changes myself.

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #35225409 | 📁 Comparing 956e103 against main (1b015e6)

  🔍 Preview build  

105 files changed · ± 105 modified

± Modified

• index.html
• _static/tachyon-example-heatmap.html
• builtins/functions.html
• builtins/stdtypes.html
• c-api/bytes.html
• c-api/capsule.html
• c-api/complex.html
• c-api/extension-modules.html
• c-api/function.html
• c-api/import.html
• and 95 more...

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[picnixz]

Please add a NEWS entry.

[oyiakoumis]

Please add a NEWS entry.

@picnixz Are you sure? buffersize hasn't been released yet, so I thought the skip news label would be appropriate here. Happy to add a NEWS entry if you prefer, though!

[picnixz]

We released 3.16.0alpha so if it has been in alpha 1, we need a NEWS entry

[oyiakoumis]

@picnixz my bad. added.