[kumarUjjawal]

Which issue does this PR close?

Related to #25620. Split from #25648 following the review request. This PR can be reviewed and merged independently.

Rationale for this change

Joins repeat or drop input rows, and outer joins add NULL padding. Copying each input column's null count unchanged can therefore produce inconsistent output statistics. For example, a left join can repeat a NULL in a non-key left column many times, while an unmatched left row introduces NULLs in every right column.

What changes are included in this PR?

• Estimate inner and outer join null counts from the rows carrying each side's values and the rows added as NULL padding.
• Estimate unmatched input rows using the existing semi-join estimator. Inner-join row counts count matching pairs, so they cannot alone determine how many input rows are unmatched.
• Account for NULL keys under ordinary and null-safe equality, while retaining unknown contributions and conservative precision.
• Rescale null counts when a statistics provider replaces the row estimate. Cover projected joins and swapped inputs.

The semi/anti-join estimator itself is unchanged. The rules follow join semantics and do not contain benchmark-specific cases.

What is the testing strategy for this PR?

Added unit tests cover NULL padding, repeated matches, unknown counts, null-safe keys, projection, input swapping, and provider rescaling. Existing cardinality assertions remain in place.

Passed on this independent branch:

• cargo test --profile ci -p datafusion-physical-plan --lib (2,369 tests)
• cargo clippy --profile ci -p datafusion-physical-plan --all-targets --all-features -- -D warnings
• cargo fmt --all and git diff --check

Benchmarks were not rerun for the split. This extracts the join implementation already present in #25648; its earlier combined measurements are not isolated measurements of this PR.

Are there any user-facing changes?

Join row and null-count estimates can change, which can affect EXPLAIN output and plan choices. No public API changes.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 97.69504% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.64%. Comparing base (dbf62c1) to head (ebd3acd).
⚠️ Report is 65 commits behind head on main.

Files with missing lines	Patch %	Lines
...usion/physical-plan/src/operator_statistics/mod.rs	92.68%	4 Missing and 5 partials ⚠️
datafusion/physical-plan/src/joins/utils.rs	99.09%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #25861      +/-   ##
==========================================
+ Coverage   82.58%   82.64%   +0.06%     
==========================================
  Files        1142     1147       +5     
  Lines      440884   445806    +4922     
  Branches   440884   445806    +4922     
==========================================
+ Hits       364088   368433    +4345     
- Misses      54799    55025     +226     
- Partials    21997    22348     +351     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kumarUjjawal]

@gabotechs can you take a look at this when you get the chance, it should block the other pr after merged.

[gabotechs]

Thanks @kumarUjjawal!

This looks very good, just spotted a couple of edge cases that are probably worth covering.

[gabotechs]

For a multi-key NullEqualsNull inner join, null_matches is Absent, so this branch makes a nullable key's output null count Absent even when the corresponding key on the other input has Exact(0) nulls. For example, with (a, b) join keys and right a null-free, no matched row can have NULL in left a; its output null count is provably Exact(0). Could we derive that per-key zero from the opposite key's null count and add a regression test? This keeps a usable statistic for a downstream IS NULL filter.

[gabotechs]

The semi-join estimator uses non-NULL NDV overlap for NullEqualsNull and can report zero matched rows even when NULL keys match. For a left join with 100 left rows (90 NULL keys and 10 distinct non-NULL keys) and one right NULL row, this calculation gives inner_rows = 90, unmatched_left = 100, hence 190 output rows; the actual output has 100 rows because the 90 NULL-key rows match. I checked the case with a focused unit test: it passes at the PR base and fails at this head (190 vs 100). Could we include the null-safe NULL matches when estimating matched, or avoid treating the semi estimate as complete in this case, and add a regression test?

[kumarUjjawal]

@gabotechs can you take another look please

[gabotechs]

Looks good 👍, thanks @kumarUjjawal!