Skip to content

Fix ORCA EXISTS subquery planning with LIMIT/OFFSET - #2223

Open
Alena0704 wants to merge 2 commits into
apache:REL_2_STABLEfrom
Alena0704:fix/orca-exists-subqueries
Open

Alena0704 wants to merge 2 commits into
apache:REL_2_STABLEfrom
Alena0704:fix/orca-exists-subqueries

Conversation

@Alena0704

Copy link
Copy Markdown
Collaborator

Fixes #2221

What does this PR do?

Fix two planning errors for correlated EXISTS/NOT EXISTS queries containing LIMIT/OFFSET and a nested scalar subquery:

  • Missing MotionState after a projection rewrite drops a SubPlan reference.
  • A spurious scalar cardinality violation when an EXISTS input can legitimately return multiple rows.

Example 1: missing MotionState

This is the minimal reproduction from #2221:

CREATE TEMP TABLE motion_repro (id integer) DISTRIBUTED BY (id);
INSERT INTO motion_repro VALUES (1);
ANALYZE motion_repro;

SET optimizer = on;

SELECT y.id
FROM motion_repro x CROSS JOIN motion_repro y
WHERE EXISTS (
    SELECT (SELECT count(*) FROM motion_repro)
    FROM motion_repro z
    WHERE y.id IS NULL
    LIMIT 1
);

The only value of y.id is 1, so y.id IS NULL is false and the query must return zero rows.

Before the fix, with optimizer=on:

SQLSTATE: XX000
ERROR: could not find MotionState for slice 5 in executor tree

After the fix, with optimizer=on, and with optimizer=off on either build:

 id
----
(0 rows)
Before the fix — EXPLAIN (COSTS OFF), optimizer=on
Gather Motion 3:1  (slice1; segments: 3)
  ->  Nested Loop
        Join Filter: true
        ->  Seq Scan on motion_repro y
              Filter: ((id IS NULL) AND (SubPlan 2))
              SubPlan 2
                ->  Result
                      ->  Limit
                            ->  Result
                                  One-Time Filter: (y.id IS NULL)
                                  ->  Materialize
                                        ->  Broadcast Motion 3:3  (slice3; segments: 3)
                                              ->  Seq Scan on motion_repro z
        ->  Materialize
              ->  Broadcast Motion 3:3  (slice2; segments: 3)
                    ->  Seq Scan on motion_repro x
Optimizer: GPORCA
After the fix — EXPLAIN (COSTS OFF), optimizer=on
Gather Motion 3:1  (slice1; segments: 3)
  ->  Nested Loop
        Join Filter: true
        ->  Seq Scan on motion_repro y
              Filter: ((id IS NULL) AND (SubPlan 2))
              SubPlan 2
                ->  Result
                      ->  Result
                            ->  Limit
                                  ->  Result
                                        One-Time Filter: (y.id IS NULL)
                                        ->  Materialize
                                              ->  Broadcast Motion 3:3  (slice3; segments: 3)
                                                    ->  Seq Scan on motion_repro z
                            SubPlan 1
                              ->  Materialize
                                    ->  Broadcast Motion 1:3  (slice4)
                                          ->  Aggregate
                                                ->  Gather Motion 3:1  (slice5; segments: 3)
                                                      ->  Seq Scan on motion_repro
        ->  Materialize
              ->  Broadcast Motion 3:3  (slice2; segments: 3)
                    ->  Seq Scan on motion_repro x
Optimizer: GPORCA

When removing a Result node, the planner loses the reference to SubPlan 1, which computes count(*).
The executor still expects this subquery’s Motion slices, but cannot find them in the execution tree.
The fix keeps the Result node and SubPlan 1, so the query runs successfully and returns the expected zero rows.

Example 2: a scalar cardinality error inside EXISTS

Run this example in a fresh connection:

CREATE TEMP TABLE card_l(id int) DISTRIBUTED BY (id);
CREATE TEMP TABLE card_r(id int, lid int, k int) DISTRIBUTED BY (id);
CREATE TEMP TABLE card_s(id int, k int) DISTRIBUTED BY (id);
INSERT INTO card_l VALUES (1), (2), (3), (4);
INSERT INTO card_r VALUES (10,1,7), (11,1,NULL), (12,3,8), (13,5,NULL);
INSERT INTO card_s VALUES (100,7), (101,7), (102,8), (103,NULL), (104,9);
ANALYZE card_l;
ANALYZE card_r;
ANALYZE card_s;

SET optimizer = on;

SELECT l.id, r.id,
       EXISTS (SELECT (SELECT count(*) FROM card_s)
               FROM card_s s
               WHERE s.k = r.k
               LIMIT NULL OFFSET NULL)
FROM card_l l FULL JOIN card_r r ON l.id = r.lid;

For r.id = 10, r.k = 7 matches two rows in card_s.
This is valid: EXISTS tests whether any row exists. The nested SELECT count(*) FROM card_s is
a separate scalar subquery returning exactly one row; it does not make the surrounding EXISTS input scalar.
LIMIT NULL OFFSET NULL preserves the input rows.

Before the fix, with optimizer=on:

SQLSTATE: 21000
ERROR: more than one row returned by a subquery used as an expression

After the fix, with optimizer=on, and with optimizer=off on either build:

 l.id | r.id | exists
------+------+--------
    1 |   10 | t
    1 |   11 | f
    2 | NULL | f
    3 |   12 | t
    4 | NULL | f
 NULL |   13 | f
(6 rows)

Rows are sorted here for presentation, with NULL displayed explicitly. The example has no ORDER BY; results were compared as multisets.

Before the fix — EXPLAIN (COSTS OFF), optimizer=on
Gather Motion 3:1  (slice1; segments: 3)
  ->  HashAggregate
        Group Key: l.id, l.ctid, l.gp_segment_id, r.id, r.lid, r.k, r.ctid, r.gp_segment_id, ((SubPlan 2))
        ->  Redistribute Motion 3:3  (slice2; segments: 3)
              Hash Key: l.id, l.ctid, l.gp_segment_id, r.id, r.lid, r.k, r.ctid, r.gp_segment_id, ((SubPlan 2))
              ->  Hash Full Join
                    Hash Cond: (r.lid = l.id)
                    ->  Redistribute Motion 3:3  (slice3; segments: 3)
                          Hash Key: r.lid
                          ->  Seq Scan on card_r r
                    ->  Hash
                          ->  Seq Scan on card_l l
                    SubPlan 2
                      ->  Result
                            ->  Result
                                  ->  Limit
                                        ->  Result
                                              Filter: (s.k = r.k)
                                              ->  Materialize
                                                    ->  Broadcast Motion 3:3  (slice4; segments: 3)
                                                          ->  Seq Scan on card_s s
                                  SubPlan 1
                                    ->  Materialize
                                          ->  Broadcast Motion 1:3  (slice5)
                                                ->  Aggregate
                                                      ->  Gather Motion 3:1  (slice6; segments: 3)
                                                            ->  Seq Scan on card_s
Optimizer: GPORCA
After the fix — EXPLAIN (COSTS OFF), optimizer=on
Result
  ->  Gather Motion 3:1  (slice1; segments: 3)
        ->  Hash Full Join
              Hash Cond: (r.lid = l.id)
              ->  Redistribute Motion 3:3  (slice2; segments: 3)
                    Hash Key: r.lid
                    ->  Seq Scan on card_r r
              ->  Hash
                    ->  Seq Scan on card_l l
  SubPlan 2
    ->  Result
          ->  Result
                ->  Limit
                      ->  Result
                            Filter: (s.k = r.k)
                            ->  Materialize
                                  ->  Gather Motion 3:1  (slice3; segments: 3)
                                        ->  Seq Scan on card_s s
                SubPlan 1
                  ->  Aggregate
                        ->  Materialize
                              ->  Gather Motion 3:1  (slice4; segments: 3)
                                    ->  Seq Scan on card_s
Optimizer: GPORCA

The failing alternative introduces an outer Apply below a grouping operation. When decorrelation fails, ORCA can implement this Apply as a scalar SubPlan even though its inner input can produce multiple rows. The resulting scalar cardinality check fails before the grouping above it can run.

The fix allows this scalar plan only when ORCA can prove that the input returns at most one row, otherwise, ORCA can choose a plan that handles EXISTS correctly. In this example, the query succeeds and the outer HashAggregate is no longer needed.

Implementation

The changes are split into two commits:

  1. In orca.c, preserve every SubPlan referenced by a child's target list when folding Result projections. Build the proposed replacement on copies and keep the Result if folding would lose a reference.
  2. In CXformApply2Join, restrict scalar correlated outer-Apply generation to inner inputs whose maximum cardinality is at most one. Other correlated alternatives remain available, and genuine scalar cardinality violations remain errors.

Both fixes are required: the cardinality fix removes the incorrect 21000 error, but the new plan still fails with a missing MotionState unless the other fix is also applied.

The regression tests cover correlated EXISTS and NOT EXISTS queries with outer joins, NULL values, and LIMIT/OFFSET. They also check nested subqueries, ANY/ALL, and scalar aggregates, and verify that scalar subqueries still raise an error when they return more than one row.

Type of Change

  • Bug fix (non-breaking change)
  • New feature (non-breaking change)
  • Breaking change (fix or feature with breaking changes)
  • Documentation update

Test Plan

  • Added orca_motion_subplan and orca_exists_cardinality to greenplum_schedule, outside the parallel group containing bfv_temp.
  • Added paired optimizer=off / optimizer=on executions: all 12 SELECT pairs return identical expected rows. A genuine scalar cardinality error is also checked in both modes.
  • Validated the fixes against a matrix of 1,581 queries. All queries returned the expected results with both optimizer=off and optimizer=on.
  • Unit tests added/updated
  • Integration tests added/updated
  • Passed make installcheck
  • Passed make -C src/test installcheck-cbdb-parallel

Checklist


Keep a Result node when replacing its child's target list would drop a
SubPlan reference after Motion slices have been allocated.
Build the proposed projection on copies so rejecting the fold leaves
the child expressions intact.

Add regression coverage for EXISTS and NOT EXISTS with nullable outer joins,
LIMIT/OFFSET, and multiple unused SubPlans.
An outer Apply introduced below grouping for an existential or
quantified subquery can have a multirow inner input. If decorrelation fails,
implementing it as a scalar SubPlan raises a spurious cardinality violation
before the grouping can run.

Only generate that scalar correlated alternative when the inner input is proven
to return at most one row. Keep the correlated alternatives generated from
the original subquery. Add regression coverage for nullable FULL JOIN correlations,
LIMIT/OFFSET, quantified subqueries, and genuine scalar cardinality errors.
@Alena0704
Alena0704 marked this pull request as ready for review October 30, 2026 19:43
* plan tree. Keep the Result unless all child SubPlans survive.
*/
if (!preserves_subplans(child_plan->targetlist, tlist))
return node;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return node here bypasses the plan_tree_mutator() call at the bottom of remove_redundant_results_mutator(), so when the fold is rejected nothing ever walks this Result's own target list. Redundant Result nodes inside the plan trees of the SubPlans referenced from that target list are therefore no longer removed (plan_tree_mutator() is called with recurse_into_subplans = true).

Before this patch, every path that kept a Result fell through to plan_tree_mutator() — the if body always eliminated the Result — so this is a newly introduced behavioural difference rather than a pre-existing one. The child plan itself is fine: it is walked explicitly a few lines above.

It is only a missed optimization and not a correctness problem, so feel free to leave it; but it would be nice to either fall through to plan_tree_mutator() on this path (keeping the Result and avoiding the double recursion into child_plan), or to walk the SubPlans of result_plan->plan.targetlist explicitly before returning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants