MDEV-41862: Syntax error with DELETE and IN subquery in FederatedX pushdown by bsrikanth-mariadb · Pull Request #6036 · MariaDB/server

bsrikanth-mariadb · 2026-10-29T16:24:55Z

A single-table DELETE with a subquery on the same table, e.g.

DELETE FROM t3 WHERE a IN (SELECT a FROM t3);

was pushed down to the remote server as

delete from federated.t3 using dual where ... in (select ...)

which the remote server rejects with ER_PARSE_ERROR. The same statement works with federated_pushdown=OFF.

The pushed-down text is produced by st_select_lex::print(). For a DELETE it prints the target list, " using ", and then the tables via print_join() over top_join_list when the statement has a JOIN. The grammar rule for a single-table DELETE adds its table with add_table_to_list(), so the table is never in top_join_list. print_join() has nothing to print and writes "dual". Without a subquery there is no JOIN, so the leaf_tables branch was used and the problem did not show up. EXPLAIN EXTENDED was affected the same way, and printed "using dual" for most single-table DELETEs.

Fix: in st_select_lex::print(), use the leaf_tables branch when top_join_list is empty. For a DELETE it prints the tables with the new helper print_delete_using_table() instead of TABLE_LIST::print(), because print() repeats the FOR SYSTEM_TIME clause that the FROM part of the statement already carries (versioning.delete_history). The helper prints the db-qualified name and alias, and expands a merged view to its underlying tables so that the USING part names the same tables as the target list. PARTITION (...) is not repeated in the USING part. UPDATE keeps using TABLE_LIST::print(), and explicit multi-table DELETE is unchanged.

The change is not limited to FederatedX. The EXPLAIN EXTENDED notes and the optimizer trace of single-table DELETE now print "using db.t" instead of "using dual", so the affected results are re-recorded: main.explain_non_select, main.myisam_explain_non_select_all, main.opt_hints, main.opt_hints_index, main.opt_trace and versioning.delete_history.

Test: federated.federatedx_pushdown_upd_del covers the statement above with pushdown on, using rows that must survive, ORDER BY ... LIMIT, a prepared statement executed twice, and an UPDATE with a subquery on the same table. It checks the text received by the remote server through the general log.

Give storage engines a way to take over an entire multi-table UPDATE/DELETE, the way they can already take over a SELECT. Without it the join, the row matching and every modification run in the SQL layer even when an engine could do the whole statement itself in one step; a single-table UPDATE/DELETE already avoids this via direct_update_rows()/direct_delete_rows(), but a multi-table statement has no primary handler object to drive that path. This adds a generic, engine-agnostic pushdown interface: the SQL layer offers the statement to the engine, and if the engine accepts it, it performs the whole thing and reports only the row counts. - Split select_handler into a pushdown_handler base with select_handler (result set) and a new multi_upddel_handler (runs a whole UPDATE/DELETE, reports row counts, reported as PUSHED UPDATE/PUSHED DELETE); add handlerton::create_multi_upddel, looked up in Sql_cmd_dml::execute_inner(). - multi_update/multi_delete gain direct_update_delete_done(), which records the engine's counts so send_eof() binlogs and replies without the SQL-layer loop; it forces statement-format binlogging so the change still replicates under binlog_format=ROW, and errors out instead of silently dropping counts for an unsupported result object. - FederatedX implements the interface as the reference engine used to test correctness: it prints the statement back and runs it remotely, passes the engine's error code/SQLSTATE through, reads the matched count from the remote info string, executes IGNORE locally, and only pushes down when all tables share one remote server (same as SELECT/derived/unit pushdown). Test: federated.federatedx_pushdown_upd_del.

A multi-table UPDATE or DELETE referring to a view (merged or materialized) was pushed down whole to the remote server. The pushed statement is printed with the view name, which exists only on the local server, so the remote server failed to find it. Reject views in get_fed_table_for_pushdown(): a table list entry for which is_view() is true makes the pushdown decline, and the statement is executed locally, row by row. get_fed_table_for_pushdown() is shared by the select, unit, derived and multi-table UPDATE/DELETE handlers, so SELECT and INSERT ... SELECT referring to a view are no longer pushed down with the local view name either. The view's own body can still be pushed as a derived table (shown as PUSHED DERIVED in EXPLAIN); only the outer statement is not pushed. Add tests to federatedx_pushdown_upd_del covering multi-table UPDATE and DELETE with a materialized (LIMIT) view and with a mergeable view, a merged view over several tables, a view of a view, a view inside a subquery, and SELECT and INSERT ... SELECT from views.

When a statement is pushed down to an engine, the once-per-statement part of the optimization (e.g. the first_cond_optimization part of JOIN::optimize(), or the whole of JOIN::optimize() for a pushed down UNION) is skipped. Re-executing the same prepared statement or stored routine without pushdown then allocated from a mem_root already marked ROOT_FLAG_READ_ONLY, hitting an assertion in alloc_root(). Add LEX::dont_freeze_mem_root, set by the pushdown_handler and derived_handler constructors so it covers every engine implementing them. Prepared_statement::execute_loop(), sp_head::execute() (through sp_head::dont_freeze_mem_root) and the SP instruction reparse path no longer freeze the mem_root of such statements. The flag is only declared and used in PROTECT_STATEMENT_MEMROOT builds. The flag is per execution for prepared statements: Prepared_statement::execute_loop() clears it after each execution, so the mem_root is frozen again after the first later execution that is not pushed down. For a stored routine the instruction's LEX keeps the flag, so once one instruction has been pushed down the routine's mem_root stays unfrozen, even if later executions are not pushed down, unless the routine is re-parsed into a new LEX. Add tests for prepared SELECT, UNION, UPDATE and DELETE, prepared EXPLAIN of a pushed UNION, derived tables, stored procedures, functions, triggers, cursors and metadata-invalidation reparse, with pushdown switched on and off. Most prepared statement and routine tests check the statements received by the remote server, or EXPLAIN, to verify that pushdown actually happened.

A multi-table UPDATE/DELETE pushed down to FederatedX failed on the remote server with a syntax error when the WHERE clause had a BETWEEN whose bounds are constants, e.g. UPDATE t1 AS a, t1 AS b SET a.x=NULL WHERE a.x BETWEEN NULL AND NULL; The bounds are wrapped in Item_cache objects. The cache was not filled yet when the query was printed, so Item_cache::print() emitted "<cache>(NULL)", which is not valid SQL. Fix: when printing with QT_PARSABLE, an Item_cache that holds no value now prints the item it caches. A cache that already holds a value still prints it as a literal. Output for callers that do not pass QT_PARSABLE is unchanged. Only FederatedX pushdown uses QT_PARSABLE. Add a test that pushes down multi-table SELECT, UPDATE and DELETE with NULL and DATE BETWEEN bounds, and checks both the statements the remote server received and the resulting data.

A pushed-down statement containing "x < SOME (subquery)" failed on the remote server with a syntax error, as the query text contained the internal "<nop>(...)" wrapper. "x < ALL (subquery)" is wrapped in "<not>(...)" in the same way. Item_func_not_all::print() prints the wrapper whenever `show` is set. Omit it when QT_PARSABLE is requested and print the wrapped predicate in parentheses, as precedence() reports the wrapper, so a parent expression would not add parentheses itself. While the ALL/ANY subquery item is intact, args[0] prints the original "x op ALL|ANY (subquery)". When the subquery is reduced to a constant ("x < ALL (SELECT 2)"), it is replaced by a comparison with the inverted comparator, and the negation is held by "<not>". Print it explicitly as "NOT (x >= 2)" for ALL, otherwise the remote server returns the complementary rows. ANY/SOME needs no negation. Item_allany_subselect::print() printed its left operand without parentheses, so "(a XOR b) < ALL (subquery)" was pushed as "a xor b < all (...)" and parsed by the remote server as "a XOR (b < ALL ...)". Use print_parenthesised(), as Item_in_subselect::print() does. Add tests to federatedx_pushdown_upd_del covering SOME and ALL in UPDATE, DELETE and SELECT, with results compared with pushdown OFF and the pushed-down text checked in the remote general log.

Problem: When reading the reply of the remote server times out, the client library compiled into the server returns without closing the connection and without setting a client error. FederatedX took the raw ER_NET_READ_INTERRUPTED for an error of the remote server and went on using a connection that still had the reply of the failed statement to arrive. As a result: * the next statement on that connection failed in various ways, either with ER_QUERY_ON_FOREIGN_DATA_SOURCE ("Server has gone away") or with an error packet that mysqltest took for a malformed packet; * inside a transaction, the following statements continued on a new remote session in autocommit mode, so they were committed on the remote server one by one even though the local transaction was later rolled back. The savepoints of the lost session were also still remembered by FederatedX. The same happened when the connection was lost while reading the reply. Fix: * After a timeout FederatedX closes the connection, so that the next statement opens a new one. The client still gets the error of the timeout, ER_NET_READ_INTERRUPTED. * Whenever the connection is gone after a failed statement, timeout or not, the state of the lost session (autocommit mode, savepoints) is forgotten and the local transaction is marked to be rolled back, so a new connection starts the transaction again. This includes the case where the connection is lost while SET AUTOCOMMIT=0 is sent for the first statement of the transaction. The closed connection is also freed, which was leaked before. * Errors of the client library are reported as ER_QUERY_ON_FOREIGN_DATA_SOURCE by ha_federatedx::info() as well, the same way as by the pushed down multi-table UPDATE/DELETE. * The error state of ha_federatedx, read by info(), is initialized. * A non-debug build checks the socket for EOF before it sends a command (net_clear()). If the remote server has already closed the connection, the write fails without the network layer raising an error, and the client would get ER_GET_ERRMSG ("Got error 10000 ...") intermittently. When the connection is lost and no error has been raised, raise ER_QUERY_ON_FOREIGN_DATA_SOURCE with the error of the client library. The COMMIT and ROLLBACK of the transaction still return their error to the caller. Test: federatedx_pushdown_upd_del covers the timeout of a pushed down multi-table UPDATE and DELETE and of reading a result, the statements after it, the same inside a transaction, a connection killed in the middle of a statement, a connection lost while SET AUTOCOMMIT=0 is sent (also followed by a disconnect and a transaction in another session), client library errors in info(), and that an error of the remote server which merely has the code of ER_NET_READ_INTERRUPTED is not taken for a timeout. The two statements after the idle connection was killed accept ER_NET_READ_ERROR (reported by the network layer in a debug build) or ER_QUERY_ON_FOREIGN_DATA_SOURCE (non-debug build).

…shdown A single-table DELETE with a subquery on the same table, e.g. DELETE FROM t3 WHERE a IN (SELECT a FROM t3); was pushed down to the remote server as delete from `federated`.`t3` using dual where ... in (select ...) which the remote server rejects with ER_PARSE_ERROR. The same statement works with federated_pushdown=OFF. The pushed-down text is produced by st_select_lex::print(). For a DELETE it prints the target list, " using ", and then the tables via print_join() over top_join_list when the statement has a JOIN. The grammar rule for a single-table DELETE adds its table with add_table_to_list(), so the table is never in top_join_list. print_join() has nothing to print and writes "dual". Without a subquery there is no JOIN, so the leaf_tables branch was used and the problem did not show up. EXPLAIN EXTENDED was affected the same way, and printed "using dual" for most single-table DELETEs. Fix: in st_select_lex::print(), use the leaf_tables branch when top_join_list is empty. For a DELETE it prints the tables with the new helper print_delete_using_table() instead of TABLE_LIST::print(), because print() repeats the FOR SYSTEM_TIME clause that the FROM part of the statement already carries (versioning.delete_history). The helper prints the db-qualified name and alias, and expands a merged view to its underlying tables so that the USING part names the same tables as the target list. PARTITION (...) is not repeated in the USING part. UPDATE keeps using TABLE_LIST::print(), and explicit multi-table DELETE is unchanged. The change is not limited to FederatedX. The EXPLAIN EXTENDED notes and the optimizer trace of single-table DELETE now print "using `db`.`t`" instead of "using dual", so the affected results are re-recorded: main.explain_non_select, main.myisam_explain_non_select_all, main.opt_hints, main.opt_hints_index, main.opt_trace and versioning.delete_history. Test: federated.federatedx_pushdown_upd_del covers the statement above with pushdown on, using rows that must survive, ORDER BY ... LIMIT, a prepared statement executed twice, and an UPDATE with a subquery on the same table. It checks the text received by the remote server through the general log.

spetrunia · 2026-10-30T12:44:38Z

Note that it prints

create table t10 (a int, b int) partition by hash(a) partitions 4;
explain extended delete  from t10 partition (p0) where a <3;
+------+-------------+-------+------+---------------+------+---------+------+------+----------+-------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+-------------+
|    1 | SIMPLE      | t10   | ALL  | NULL          | NULL | NULL    | NULL | 2    |   100.00 | Using where |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 1 warning (5.716 sec)

show warnings\G
*************************** 1. row ***************************
  Level: Note
   Code: 1003
Message: delete  from `test`.`t10` PARTITION (`p0`) using `test`.`t10` where `test`.`t10`.`a` < 3
1 row in set (0.001 sec)

Let's try running what it produces in the Message:

MariaDB [test]> delete  from `test`.`t10` PARTITION (`p0`) using `test`.`t10` where `test`.`t10`.`a` < 3;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'using `test`.`t10` where `test`.`t10`.`a` < 3' at line 1

Let's move the PARTITION p0 to be after the USING :

MariaDB [test]> delete  from `test`.`t10` using `test`.`t10` partition (p0) where `test`.`t10`.`a` < 3;
Query OK, 0 rows affected (0.003 sec)

spetrunia · 2026-10-30T13:12:10Z

One can observe similar effect with hints inside VIEW:

MariaDB [test]> create view v10b as select a, b from t10 ignore index(b) ;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> explain extended delete from v10b where a<3;
+------+-------------+-------+------+---------------+------+---------+------+------+----------+-------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+-------------+
|    1 | SIMPLE      | t10   | ALL  | NULL          | NULL | NULL    | NULL | 4    |   100.00 | Using where |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.002 sec)

MariaDB [test]> show warnings\G
*************************** 1. row ***************************
  Level: Note
   Code: 1003
Message: /* select#1 */ delete  from `test`.`t10` IGNORE INDEX (`b`) using `test`.`t10` where `v10b`.`a` < 3
1 row in set (0.000 sec)

Note that also the WHERE has a reference to "v10b.a" .

Can this work at all for VIEWs?

spetrunia · 2026-10-30T13:14:08Z

We did try to prevent pushdown for queries using VIEWs : 2db22df:

commit 2db22df80520f0196b0c518e6bb146b07739ff3f (origin/13.2-MDEV-41340-merged-view-update-query-not-pushed-down)
Author: bsrikanth-mariadb <srikanth.bondalapati@mariadb.com>
Date:   Thu Oct 1 19:32:11 2026 +0530

    MDEV-41340: Don't push down statements that use a view in FederatedX
    
    A multi-table UPDATE or DELETE referring to a view (merged or
    materialized) was pushed down whole to the remote server. The pushed
    statement is printed with the view name, which exists only on the local
    server, so the remote server failed to find it.
    
    Reject views in get_fed_table_for_pushdown(): a table list entry for
    which is_view() is true makes the pushdown decline, and the statement
    is executed locally, row by row.

spetrunia

See above. We did try to refuse pushdown for queries using VIEWs. Why this one was not refused?

spetrunia · 2026-10-31T11:30:17Z

Wait, I was asking the wrong question.
The testcase doesn't use VIEWs.
But in the patch, the print_delete_using_table does handle printing of DELETE FROM multi_table_view.
Even if print_delete_using_table is not invoked for DELETE FROM multi_table_view.

spetrunia · 2026-10-31T11:31:24Z

What about the alternative.
Instead of trying to produce

DELETE FROM t1 USING t1 WHERE ...

Can we write the original form

DELETE FROM t1 WHERE ...

?

Definitely, this is much better.

spetrunia · 2026-10-31T11:32:30Z

Pushed 111a6a9 to check this hypothesis...

bsrikanth-mariadb · 2026-11-02T07:23:54Z

Pushed 111a6a9 to check this hypothesis...

The main merge created a lot of merge conflicts. So, moved these changes to #6050
I am closing this PR.

spetrunia · 2026-11-02T15:47:46Z

Ok, actually closing as said above.

bsrikanth-mariadb added 7 commits September 4, 2026 17:32

bsrikanth-mariadb added the MariaDB Corporation label Oct 29, 2026

bsrikanth-mariadb requested a review from spetrunia October 30, 2026 03:14

bsrikanth-mariadb force-pushed the 13.2-MDEV-39226-direct-multi_table-update-delete branch from daf3ee7 to 6ea128f Compare October 30, 2026 10:53

spetrunia force-pushed the 13.2-MDEV-41862-syntax-err-with-delete-and-in-op-when-pushdown branch from a1c3fb9 to 212b1ee Compare October 30, 2026 12:32

spetrunia requested changes Oct 30, 2026

View reviewed changes

MDEV-41862: Followup: print single-table DELETE without USING.

111a6a9

spetrunia force-pushed the 13.2-MDEV-41862-syntax-err-with-delete-and-in-op-when-pushdown branch from 212b1ee to 111a6a9 Compare October 31, 2026 09:52

bsrikanth-mariadb mentioned this pull request Nov 2, 2026

MDEV-41862: Syntax error with DELETE and IN in FederatedX pushdown #6050

Closed

spetrunia closed this Nov 2, 2026

Navigation Menu

Repository navigation

MDEV-41862: Syntax error with DELETE and IN subquery in FederatedX pushdown - #6036

MDEV-41862: Syntax error with DELETE and IN subquery in FederatedX pushdown#6036
bsrikanth-mariadb wants to merge 8 commits into
13.2-MDEV-39226-direct-multi_table-update-deletefrom
13.2-MDEV-41862-syntax-err-with-delete-and-in-op-when-pushdown

bsrikanth-mariadb commented Oct 29, 2026

Uh oh!

spetrunia commented Oct 30, 2026

Uh oh!

spetrunia commented Oct 30, 2026

Uh oh!

spetrunia commented Oct 30, 2026

Uh oh!

spetrunia left a comment

Uh oh!

spetrunia commented Oct 31, 2026

Uh oh!

spetrunia commented Oct 31, 2026 •

edited by bsrikanth-mariadb

Loading

Uh oh!

spetrunia commented Oct 31, 2026

Uh oh!

bsrikanth-mariadb commented Nov 2, 2026 •

edited

Loading

Uh oh!

spetrunia commented Nov 2, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Conversation

bsrikanth-mariadb commented Oct 29, 2026

Uh oh!

spetrunia commented Oct 30, 2026

Uh oh!

spetrunia commented Oct 30, 2026

Uh oh!

spetrunia commented Oct 30, 2026

Uh oh!

spetrunia left a comment

Choose a reason for hiding this comment

Uh oh!

spetrunia commented Oct 31, 2026

Uh oh!

spetrunia commented Oct 31, 2026 • edited by bsrikanth-mariadb Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

spetrunia commented Oct 31, 2026

Uh oh!

bsrikanth-mariadb commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

spetrunia commented Nov 2, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

spetrunia commented Oct 31, 2026 •

edited by bsrikanth-mariadb

Loading

bsrikanth-mariadb commented Nov 2, 2026 •

edited

Loading