Repository navigation
gh-155126: Implement PEP 828 - #155127
gh-155126: Implement PEP 828#155127ZeroIntensity wants to merge 16 commits into
Conversation
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
Documentation build overview
417 files changed ·
|
|
cc @1st1 |
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
…/implementation
…/implementation
…/implementation
Treat a deleted `StopAsyncIteration.value` as `None` when handling delegated iterator exhaustion. Add a regression test covering immediate exhaustion and exhaustion after yielding an item.
| // this was called from athrow(). This broke some tests/expected behavior | ||
| // for yield from in asyncgens. Removing the parameter didn't seem to cause | ||
| // any new test failures, nor could I reproduce any different behavior | ||
| // when experimenting with it, but we need to be careful. |
There was a problem hiding this comment.
Neither configuration is correct. Removing the flag breaks asynchronous cleanup; restoring it fixes that case but breaks the required close semantics for GeneratorExit.
I rebuilt and tested both configurations, restoring the original flag plumbing from main: gen_throw() passes 1, the initial aclose()/athrow() calls pass 0, and recursive calls preserve the flag.
| Case | Current branch, flag removed | Flag restored |
|---|---|---|
| Delegate’s cleanup completes synchronously | Works | Works |
| Delegate’s cleanup suspends during aclose() | Fails with RuntimeError; cleanup incomplete | Works |
| Delegate’s cleanup suspends during athrow(GeneratorExit) | Same failure | Works |
| Delegate suppresses GeneratorExit during outer athrow(GeneratorExit) | Correctly reraises the outer exception | Incorrectly allows delegation to finish normally |
| Delegate yields an item while handling GeneratorExit | Correctly raises RuntimeError | Outer athrow() incorrectly returns that item |
| Delegate raises its own GeneratorExit during closing | Preserves expected outer exception identity | Identity test fails |
| Ordinary async generator’s own cleanup awaits | Works | Works |
| Custom async iterator implements aclose()/athrow() | Methods skipped | Methods skipped |
The missing case in the tests is cleanup that actually suspends. This reproduces the current branch’s problem:
import asyncio
async def inner():
try:
yield 1
finally:
await asyncio.sleep(0)
print("cleanup completed")
async def outer(sub):
yield from sub
async def main():
sub = inner()
gen = outer(sub)
await anext(gen)
await gen.aclose()
asyncio.run(main())Currently, aclose() raises:
RuntimeError: async generator ignored GeneratorExit
The cleanup never reaches the print. With the flag restored, cleanup completes and aclose() succeeds.
The cause is gen_close_iter() (Objects/genobject.c:427): the PR adds async generators to the branch that calls synchronous gen_close(). When cleanup suspends at an await, gen_close() treats the suspension as an illegal yield and raises RuntimeError.
Restoring the flag only exchanges one problem for another. It makes _gen_throw() forward GeneratorExit into the delegate. But async delegation must await the delegate’s aclose() and then reraise the original exception. Forwarding through throw lets the delegate suppress the exception or yield another item. That distinction is explicit in PEP 828’s reference expansion.
The custom iterator failures expose the same underlying mismatch: the current helpers look for synchronous .close() and .throw(), rather than calling and awaiting .aclose() and .athrow(). I confirmed this against a Python implementation of the PEP’s expansion; the reference calls those methods, while native delegation skips them in both configurations.
The fix needs to distinguish closing an async delegate from throwing into it, and provide a resumable path for awaiting its aclose(). A boolean change alone cannot provide both behaviors. I would investigate handling this in the async delegation bytecode expansion, where the operation can suspend, rather than extending synchronous gen_close_iter().
Validation:
- Current branch: all 285 tests pass across test_yield_from_async, test_asyncgen, test_yield_from, and test_coroutines.
- Flag restored: 4 errors and 3 failures, all in test_yield_from_async; the other suites pass.
- Additional probes confirm the missing cleanup and custom iterator failures above.
The source and interpreter are restored to your current implementation; the working tree is clean. The comparison probe (/private/tmp/pep828-protocol-probe.py) and delegation edge-case probe (/private/tmp/pep828-delegation-probe.py) remain available for review.
4 participants
Uh oh!
There was an error while loading. Please reload this page.