Skip to content

MDEV-38741 Abort the running statement when its client disconnects - #6035

Draft
vaintroub wants to merge 1 commit into
mainfrom
main-MDEV-38741
Draft

vaintroub wants to merge 1 commit into
mainfrom
main-MDEV-38741

Conversation

@vaintroub

@vaintroub vaintroub commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Abort a running statement when its client disconnects. The server notices a gone client only when it reads from or writes to the socket, so a statement that waits for something else, like an InnoDB row lock, keeps running, and its transaction keeps owning its locks, while other connections wait for them.

Design

  • The client socket is watched from the end of login until the connection ends or the client sends COM_QUIT (the watch is dropped when it is parsed, as the client closes its socket right after it, which is not a failure), using only events the OS reports on its own. The global disconnect_watcher variable (dynamic, ON by default, affects new connections) switches it off. It exists only where there is a watcher: Linux, Windows and the BSDs, not in the embedded server. The watcher finds the connection by thread id, like KILL does, and touches the THD only through awake_no_mutex().
  • A disconnect aborts the running statement with a new, statement-level state KILL_DISCONNECT, like KILL QUERY. An idle or quitting connection ignores it, so COM_QUIT, and a plain close, behave as before, with the usual "Aborted connection" line.
  • When the kill does abort a statement, the error of that statement is sent (ER_CONNECTION_KILLED), the reason "Client disconnected" is logged once, and the connection ends, so the commands that the client pipelined behind the statement do not run. The disconnect is kept in THD::client_disconnected until then, and a failed write of the error is not reported. If the kill aborts nothing, for example the statement had already finished, nothing happens.
  • Replication dump threads are watched like other connections.

Platforms

mechanism
Linux an epoll set, polled by the connection accepting loop. TCP also requests EPOLLRDHUP, as a close and a crashed client are only a FIN. A unix socket does not, it reports a close with EPOLLHUP. The family is taken from getsockname(), as vio_type() hides the transport under TLS
Windows sockets WSAEventSelect(FD_CLOSE) and a wait per socket, on a thread pool of our own with a single thread, so the callbacks run one by one like the epoll events. Polling with the AFD driver was tried and rejected: a pending poll makes select(), which the vio layer waits with, lose the data notification
Windows named pipes wait for the exit of the local client process (GetNamedPipeClientProcessId(), the client is impersonated if the process cannot be opened). Remote pipes are skipped
BSD, macOS kqueue, EV_EOF. It passed on the aarch64-macos builder at an earlier revision of this PR, not tested locally

Limitations

  • A FIN cannot be told from a half-close, on any platform but a Linux unix socket. A client that closes only its sending side while a statement runs can get that statement aborted (it receives ER_CONNECTION_KILLED), and its connection ended.
  • A named pipe client that only closes the pipe and keeps running is not noticed.
  • Each connection is watched once, after the first event the watch is spent.

Tests

  • main.disconnect_lock_wait (new): kills a waiting client over the default connection, over TCP closed with a FIN, and with a reset, and checks that the reason is logged once.
  • main.disconnect_lock_wait_pipe (new, Windows): a real client is started on the pipe, waits for a lock, and is killed.
  • sys_vars.disconnect_watcher_basic (new). The tests that depend on the watcher run with it ON and OFF (include/disconnect_watcher.combinations); with it OFF the lock wait is not aborted, so the test releases the lock instead (rdiff files).
  • mysqltest: dirty_close <connection> [linger|nolinger]. nolinger resets the TCP connection, linger, the default, is what it was.
  • innodb.xa_recovery no longer needs the shortened lock wait timeout. rpl_ssl1 and rpl_password_boundaries wait for the dump thread of the slave to end, max_binlog_total_size and rpl_fail_register kill it only if it is still there, as it may have ended already.

The full main suite was run on Windows and Linux (Debug). The only failures are in my environment: large pages on Windows, and on WSL a world-writable config file and a missing git revision.

Cost

Connect and disconnect churn with libmariadb, 16 threads, 2000 connections per run, against the commit this PR is based on: no difference that is distinguishable from the noise (4 runs per cell). Plain TCP and AF_UNIX on Linux, and plain TCP on Windows, are within about 5%. With TLS, the handshake dominates (about 2000 conn/s on both systems) and the difference is within 2%. A watcher event still fires for about 12% of the connections on Linux and about a third on Windows, when the FIN arrives before the server has parsed COM_QUIT. Waiting for the server's EOF before closing the socket in the client removes all of them, but it is not part of this PR.

If the VIO refactoring in #5584 lands first, vio_transport_type() and vio_handle() replace the getsockname() family check and the direct hPipe access.

🤖 Generated with Claude Code

@vaintroub
vaintroub force-pushed the main-MDEV-38741 branch 2 times, most recently from c31cfc0 to 70e9cd4 Compare October 30, 2026 02:49
The server notices that a client is gone only when it reads from or
writes to the socket. While a statement runs, for example waiting for
an InnoDB row lock, a disconnect of the client went unnoticed and the
statement kept running and holding its locks.

Watch the client socket from the end of login until the connection
ends, or the client sends COM_QUIT, using only the events the OS
reports on its own. When the client has gone, abort the running
statement with the new KILL_DISCONNECT state, found by thread id like
KILL does. An idle or quitting connection ignores it.

KILL_DISCONNECT is statement-level, like KILL QUERY. When it does abort
a statement, the connection ends after the error of that statement is
sent, so the commands that the client pipelined behind it do not run.
The reason, "Client disconnected", is logged once, and a failed write
of the error is not reported. An orderly COM_QUIT, or a close without
it, is handled as before.

The global variable disconnect_watcher, ON by default, switches the
watcher off. It affects the connections made after the change, and
exists only where there is a watcher: Linux, Windows and the BSD
systems, not the embedded server.

Linux: an epoll set, polled by the connection accepting loop. TCP
requests EPOLLRDHUP, as a close and a crashed client are only a FIN.
A unix socket does not, it reports a close with EPOLLHUP.
Windows sockets: WSAEventSelect(FD_CLOSE), with a wait per socket on a
thread pool of our own that has a single thread, so the callbacks run
one by one.
Windows named pipes: the wait is for the exit of the local client
process. A client that only closes the pipe is not noticed.
BSD and macOS: kqueue, EV_EOF.
Replication dump threads are watched too.

mysqltest: dirty_close takes an optional "linger" (the default, a FIN)
or "nolinger" (SO_LINGER 0, so the server gets a reset) argument.

main.disconnect_lock_wait, disconnect_lock_wait_pipe (Windows) and
sys_vars.disconnect_watcher_basic are new. The tests that depend on the
watcher run with it ON and OFF. innodb.xa_recovery no longer needs the
shortened lock wait timeout. rpl_ssl1 and rpl_password_boundaries wait
for the dump thread of the slave to end before DROP USER,
max_binlog_total_size and rpl_fail_register kill it only if it is
still there.

Assisted-By: Claude:claude-sonnet-5-5
@vaintroub vaintroub changed the title MDEV-38741 Client disconnect fails to abort a lock wait MDEV-38741 Abort the running statement when its client disconnects Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants