Repository navigation
Avoid releasing the GIL in nonblocking socket operations #89977
Description
| BPO | 45819 |
|---|---|
| Nosy | @rhettinger, @tiran, @asvetlov, @1st1, @jakirkham, @jcrist |
| PRs | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2021-11-16.18:51:40.323>
labels = ['3.11', 'expert-asyncio', 'expert-IO', 'performance']
title = 'Avoid releasing the GIL in nonblocking socket operations'
updated_at = <Date 2022-02-02.23:18:52.305>
user = 'https://github.com/jcrist'bugs.python.org fields:
activity = <Date 2022-02-02.23:18:52.305>
actor = 'jakirkham'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['IO', 'asyncio']
creation = <Date 2021-11-16.18:51:40.323>
creator = 'jcristharif'
dependencies = []
files = ['50443']
hgrepos = []
issue_num = 45819
keywords = ['patch']
message_count = 3.0
messages = ['406422', '406431', '406436']
nosy_count = 6.0
nosy_names = ['rhettinger', 'christian.heimes', 'asvetlov', 'yselivanov', 'jakirkham', 'jcristharif']
pr_nums = ['29579']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue45819'
versions = ['Python 3.11']
Activity
jcrist commentedon Nov 16, 2021
In https://bugs.python.org/issue7946 an issue with how the current GIL interacts with mixing IO and CPU bound work. Quoting this issue:
This issue can come up in any application where IO and CPU bound work are mixed (we've found it to be a cause of performance issues in https://dask.org for example). Fixing the general problem is tricky and likely requires changes to the GIL's internals, but in the specific case of mixing asyncio running in one thread and CPU work happening in background threads, there may be a simpler fix - don't release the GIL if we don't have to.
Asyncio relies on nonblocking socket operations, which by definition shouldn't block. As such, releasing the GIL shouldn't be needed for many operations (
send,recv, ...) onsocket.socketobjects provided they're in nonblocking mode (as suggested in https://bugs.python.org/issue7946#msg99477). Likewise, dropping the GIL can be avoided when callingselectonselectors.BaseSelectorobjects with a timeout of 0 (making it a non-blocking call).I've made a patch (https://github.com/jcrist/cpython/tree/keep-gil-for-fast-syscalls) with these two changes, and run a benchmark (attached) to evaluate the effect of background threads with/without the patch. The benchmark starts an asyncio server in one process, and a number of clients in a separate process. A number of background threads that just spin are started in the server process (configurable by the
-tflag, defaults to 0), then the server is loaded to measure the RPS.Here are the results:
A few comments on the results:
On the main branch, any GIL contention sharply decreases the number of RPS an asyncio server can handle, regardless of the number of clients. This makes sense - any socket operation will release the GIL, and the server thread will have to wait to reacquire it (up to the switch interval), rinse and repeat. So if every request requires 1 recv and 1 send, a server with background GIL contention is stuck at a max of
1 / (2 * switchinterval)or 200 RPS with default configuration. This effectively prioritizes the background thread over the IO thread, since the IO thread releases the GIL very frequently and the background thread never does.With the patch, we still see a performance degradation, but the degradation is less severe and improves with the number of clients. This is because with these changes the asyncio thread only releases the GIL when doing a blocking poll for new IO events (or when the switch interval is hit). With low load (1 client), the IO thread becomes idle more frequently and releases the GIL. Under higher load though the event loop frequently still has work to do at the end of a cycle and issues a
selector.selectcall with a 0 timeout (nonblocking), avoiding releasing the GIL at all during that loop (note the nonlinear effect of adding more clients). Since the IO thread still releases the GIL sometimes, the background thread still holds the GIL a larger percentage of the time than the IO thread, but the difference between them is less severe than without this patch.I have also tested this patch on a Dask cluster running some real-world problems and found that it did improve performance where IO was throttled due to GIL contention.
rhettinger commentedon Nov 16, 2021
+1 There is almost no upside for the current behavior.
tiran commentedon Nov 16, 2021
Do POSIX and Windows APIs guarantee that operations on a nonblocking socket can never ever block?
We cannot safely keep the GIL unless you can turn "nonblocking socket operations by definition shouldn't block" into "standard guarantees that nonblocking socket operations never block".
7 remaining items
pythongh-89977: Generate socket news with blurb and link APIs
pythongh-89977: Correct setblocking news reference
pythongh-89977: Keep the GIL for short socket and epoll control calls
methane commentedon Sep 29, 2026
We should keep releasing the GIL for
EPOLL_CTL_DEL. Linux acquires the sameep->mtxforADD,MOD, andDEL, andADD/MODmay hold it while waiting for a FUSE poll response.If another Python thread in the same process handles that FUSE request, holding the GIL during
DELcan create a deadlock:DELwaits for the mutex,ADD/MODwaits for the FUSE response, and the FUSE handler waits for the GIL. This can happen even when unregistering an ordinary socket, since the mutex belongs to the epoll instance, not the target fd.hosu3das20s05s commentedon Sep 29, 2026
zzffeggs commentedon Sep 29, 2026
dqqFA8995 commentedon Sep 29, 2026
pythongh-89977: Revert epoll GIL changes
pythongh-89977: Remove unrelated socket name test
gh-89977: Keep the GIL for short socket calls (#157981)