Update Vortex and DataFusion Rust crates by renovate[bot] · Pull Request #19 · vortex-data/vortex-datafusion-cli

renovate · 2026-10-08T03:46:25Z

This PR contains the following updates:

Package	Type	Update	Change
datafusion (source)	dependencies	minor	`55.1.0` → `55.2.0`
datafusion-cli (source)	dependencies	minor	`55.1.0` → `55.2.0`
vortex-datafusion	dependencies	minor	`0.87.0` → `0.90.0`

Release Notes

apache/datafusion (datafusion)

`v55.2.0`

Compare Source

vortex-data/vortex (vortex-datafusion)

`v0.90.0`

Compare Source

Changes

⚠️ Breaks

Remove legacy_session and the deprecated APIs that rely on it (#10595) @robert3005
Validate List and ListView arrays with an explicit execution context (#10594) @robert3005
Validate VarBin arrays with an explicit execution context (#10593) @robert3005
Thread execution contexts through Patches (#10592) @robert3005

✨ Features

Add select_unset to BitBuffer (#10666) @rapour
Poc index layout (#9706) @thorfour
Support List/Struct/Map scalars in DuckDB conversion (#10639) @robert3005
Prune in_list expressions (#10617) @robert3005
feat(aggregate): register is_sorted and min_max for temporal and UUID extension types (#10616) @joseph-isaacs

🚀 Performance

perf(sequence): take through the fallible lane kernels (#10142) @robert3005
Decode DateTimeParts values in single pass (#10140) @robert3005
Track kept patches in a bitmap when filtering patches (#10642) @robert3005
feat(scan): cache V2 scan state across scans of one file (#10631) @joseph-isaacs

🐛 Bug Fixes

Return an empty array from fill_null on an empty input (#10627) @ump45nose
Drop an empty ListView's elements when rebuilding (#10655) @mhk197
Preserve extension metadata on Arrow dictionary import (#10645) @asubiotto
Push down comparisons through REE-to-dictionary casts (#10637) @yiquanzhou
fix(ipc): fill the buffer on partial reads in the sync message reader (#10284) @jackylee-ch

🧰 Maintenance

15 changes

duckdb: replace hash to 2.0.0 (#10709) @myrrc
Refactor Interleave to use named child slots (#10688) @jiengup
Split TPC-H and TPC-DS benchmark tables into ~200MB files (#10519) @joseph-isaacs
Update dependency storybook to v10.6.2 [SECURITY] (#10682) @renovate[bot]
ci: Alpine 3.24 musl container and move small-runner jobs to amd64-ci-medium (#10667) @joseph-isaacs
Use same tpchgen-rs for sqllogic test and bench (#10656) @robert3005
random-access-bench: read only the selected rows from Parquet (#10647) @joseph-isaacs
random-access-bench: take Lance rows from its single fragment (#10649) @joseph-isaacs
Update webpki packages to resolve security warnings (#10662) @robert3005
Update dependency vite to v8.3.3 [SECURITY] (#10654) @renovate[bot]
Add Ashvin Agrawal and Matthew Katz as Committers (#10644) @mhk197
file_segment_source_panic_propagates_to_all_concurrent_readers test no longer depends on time (#10636) @robert3005
Fix random fail in runtime::tokio::tests::test_spawn_and_abort (#10632) @robert3005
Drop VarBinView's debug validation in new_unchecked (#10584) @robert3005
Validate FSST codes bounds only where an execution context exists (#10588) @robert3005

`v0.89.0`

Compare Source

Changes

⚠️ Breaks

C++ FFI field access method doesn't require a session (#10611) @robert3005
Pass an execution context to array value display (#10583) @robert3005
Take a session in vx_array_get_field and vx_array_invalid_count (#10589) @robert3005
duplicate ZstdBuffer's validity into a slot instead of using the legacy session (#10586) @robert3005
Take an execution context in CanonicalCudaExt::into_host (#10590) @robert3005
Blocked FoR: store a global reference scalar as metadata rather than child (#10222) @mhk197
Move Shared array out of vortex-array into vortex-layout (#10495) @robert3005

✨ Features

Convert FixedSizeList scalars to and from DuckDB ARRAY values (#10629) @robert3005
feat(aggregate): add ExtensionStoragePushdown to push an aggregate into an extension type's storage (#10566) @joseph-isaacs
Expose list membership with SQL null semantics in Java and Python (#10053) @robert3005
Duckdb 2.0 (#9141) @myrrc
feat(array): enable CaseWhen expression serialization (#10604) @blaginin
feat(sparse): push LIKE through Sparse arrays (#10538) @robert3005
feat(layout): zoned pruning, shares, and dictionary value reuse in physical plans (#10462) @joseph-isaacs
perf(python): release the GIL once per scan export (#10497) @robert3005
feat(python): open local files as memory maps (#10498) @robert3005
Optimize list_contains with prepared constant sets (#10052) @robert3005
feat: expose Spark scan schema and pushed predicates (#9956) @xiaoh1024
Blocked BitPacked: Add Compressor support via BitPackingScheme mode (#10245) @mhk197
Support Polars 2 (#10398) @robert3005
feat(layout): compile evals, row indices, and takes to pipelines (#10393) @joseph-isaacs
feat(layout): compile segment scans, filters, concatenations, and packs to pipelines (#10392) @joseph-isaacs
feat(layout): add push-based pipelines and the scan that runs them (#10391) @joseph-isaacs
Support filters with limit in ScanBuilder (#8766) @AdamGS
Add more scalar literal types to the Python and JNI bindings (#10079) @robert3005
feat(array): support fill null for UTF-8 and binary arrays (#10044) @1fanwang
feat(python): support row indices in VortexFile.to_arrow (#10373) @joseph-isaacs

🚀 Performance

Plan V2 random access from known row selections (#10625) @joseph-isaacs
perf(fastlanes): answer BitPacked compare and between from the bit width, between through the fused kernel (#10559) @mightsleep
perf(array): snapshot aggregate kernels in ExecutionCtx (#10561) @joseph-isaacs
Decode short RunEnd runs with fixed-size block writes (#10488) @joseph-isaacs
Size the list layout split grid by row count (#10531) @mhk197
Preserve contiguous index ranges in BitPacked take (#10520) @blaginin
perf(fsst): gather sparse takes from small arrays with large payloads (#10509) @blaginin
Deduplicate execute is_done check with done_canonical check (#10387) @robert3005
perf(datafusion): avoid Arrow session lookup for ordinary scalars (#10541) @joseph-isaacs
perf: make FieldSelection normalization linear (#10517) @joseph-isaacs
perf: read pass-through struct projections directly (#10515) @joseph-isaacs
perf: move decoded stats into arrays without stats (#10516) @joseph-isaacs
Optimise bitpacked filtering and take for valid runs (#10281) @robert3005
fix(datetime-parts): keep null rows null and order by parts (#10145) @robert3005
perf: exit compute_referenced_values_mask early once every value is referenced (#10423) @joseph-isaacs
Decode nullable RunEnd validity by clearing null streaks (#10487) @joseph-isaacs
Drop DuckDB file scan state outside the multi-file lock (#10486) @robert3005
perf(layout): skip chunks without selected rows (#10491) @robert3005
Bound DashMap shard counts for per-reader caches (#10485) @robert3005
Blocked BitPacked: fuse FoR decode over blocked children (#10251) @mhk197
perf(mask): read intersect_by_rank rank bits at an absolute offset (#10480) @mightsleep
avx2 branch for bool take kernel (#10385) @myrrc
perf: enable POPCNT in the BMI2, AVX2 and AVX-512 kernels (#10456) @mightsleep
feat(compressor): measure inputs and blocks by the bytes their rows reference (#10445) @joseph-isaacs
perf(mask): SVE2 BDEP and a branch-free portable deposit for intersect_by_rank (#10168) @mightsleep
feat(array): add precision option to uncompressed_size_in_bytes (#10439) @joseph-isaacs
perf(layout): lower zoned pruning predicates once (#10428) @robert3005
Optimize SearchSorted to reuse probes and avoid repeated context creation (#9904) @joseph-isaacs
Speed up scans with a roaring row selection (#10359) @joseph-isaacs
perf: batch fragmented CASE branch execution (#9820) @gatesn

🐛 Bug Fixes

37 changes

fix: consult segment cache before registering reads (#10535) @gatesn
Don't pass extension arrays through the compressor unchanged (#10424) @robert3005
Validate MaskedArray with an explicit execution context (#10591) @robert3005
fix: avoid fill_null crashes on lazy fill values (#10576) @mvanhorn
chore: fix intermittent failure in fused_decode_is_wrapping_add (#10544) @mightsleep
Fix constant decimal array overflow for decimals bigger than i128 (#10459) @robert3005
fix: account for NaN in float stats exported to DuckDB and DataFusion (#10479) @jackylee-ch
fix(datetime-parts): keep constant nullability in comparison pushdown (#10512) @robert3005
Use SQL null semantics for DuckDB IN and NOT IN pushdown (#10054) @robert3005
fix(datafusion): infer singleton distinct counts from exact bounds (#10448) @joseph-isaacs
fix(python): translate Polars struct field access (#10194) @danking
Struct reader partitioning respects struct validity (#10449) @myrrc
Update rustls to 0.23.46 (#10492) @palaska
fix: omit zone-map stats that a nullable struct's nulls would corrupt (#10413) @joseph-isaacs
Fix DataFusion expression conversion and pushdown semantics (#9785) @AdamGS
fix(array): scale decimal-to-float scalar casts across the whole scale domain (#9883) @jackylee-ch
Only gather sparse takes from OnPair and FSST (#10421) @joseph-isaacs
fix(layout): stop dict writer looping on an oversized value (#10400) @vortex-claude[bot]
fix(file): validate metadata segments against the file size too (#9881) @jackylee-ch
fix(python): support Polars datetime literals and timezone expressions (#10183) @danking
fix: report a canonical NaN for a float sum, not the platform's default (#10122) @gerchowl
fix(python): translate Polars fill_null with expression fills (#10192) @danking
feat(expr): add timestamp timezone replacement (#10182) @danking
fix(python): cast scan indices to u64 in read_url and to_array (#10277) @jackylee-ch
fix(cloud): parse HF ALLOW_HTTP the way object_store does (#10278) @jackylee-ch
Fix nullability of bool take (#10411) @robert3005
Expression optimisation rules don't introduce irreducible children (#9625) @robert3005
fix(python): translate Polars boolean NOT expressions (#10190) @danking
fix(python): translate Polars is_null expressions (#10189) @danking
fix(python): translate Polars binary scalar literals (#10188) @danking
fix(python): translate Polars date literals (#10185) @danking
fix(python): support Polars is_not_null predicates (#10184) @danking
fix(python): support unsigned Polars scalar literals (#10170) @danking
fix(python): translate Polars is_between expressions (#10191) @danking
fix(python): translate Polars decimal literals (#10187) @danking
fix(python): translate Polars conditional expressions (#10193) @danking
Canonical dict execute handles edge cases in Decimal/Bool logic (#10098) @robert3005

🧰 Maintenance

34 changes

GetItem reduction rule produces new_constant instead of new_node(Literal) (#10606) @robert3005
compress-bench: write and scan Parquet the way DataFusion does by default (#10571) @robert3005
rust toolchain 1.99.1 (#10608) @blaginin
Validate RunEnd arrays without the legacy session (#10587) @robert3005
Make Null take an execute kernel (#10585) @robert3005
Lock file maintenance (#10598) @renovate[bot]
Fix the manual_map lint in TrustedLenAdapter::next (#10597) @robert3005
Use mimalloc with purging disabled in each e2e benchmark binary (#10490) @joseph-isaacs
Audit tests for duplication and coverage (#10458) @robert3005
Reuse unpack helpers in BitPacked range take (#10565) @robert3005
ci(compress-bench): give Parquet and Vortex the same parallel CPU budget (#10510) @joseph-isaacs
Write the list layout when the enabled editions include it (#10563) @mhk197
Make the list layout opt-in and edition-gated (#10529) @mhk197
Rewrite range pairs into between once per scan, and push down DataFusion dynamic filters (#10505) @robert3005
bench: generate TPC-DS with tpcdsgen and share Parquet writer settings with TPC-H (#10511) @joseph-isaacs
ci(python): run Python benchmarks with CodSpeed (#10500) @robert3005
Use mimalloc as the global allocator in every benchmark binary (#10483) @joseph-isaacs
Strengthen test assertions and remove fuzz reference checks (#10465) @robert3005
Blocked BitPacked: add fastlanes.bitpacked.v2 serialization (#10244) @mhk197
[bench] sparse bitpacked filter and take (#9900) @robert3005
Update lance benchmark dependencies (major) (#10436) @renovate[bot]
Update gradle/actions digest to 3f5f9ad (#10434) @renovate[bot]
Update CodSpeedHQ/action digest to f842b9a (#10432) @renovate[bot]
Update dtolnay/rust-toolchain digest to 686976e (#10433) @renovate[bot]
Lock file maintenance (#10438) @renovate[bot]
Update taiki-e/install-action digest to 1168f68 (#10435) @renovate[bot]
Update Rust crate zip to v9 (#10437) @renovate[bot]
Update actions/upload-artifact digest to cf430e0 (#10431) @renovate[bot]
Update actions/setup-node digest to 949feb2 - autoclosed (#10430) @renovate[bot]
Update actions/download-artifact digest to 9000827 (#10429) @renovate[bot]
Return owned results from the optimizer and log every parent rewrite (#10407) @robert3005
Expose slots, dtype, len and is_empty on TypedArrayRef (#10406) @robert3005
Don't require boxing ChunkedArray chunk iterators (#9918) @robert3005
Cache aggregate results on the requested array, not its canonical form (#10380) @robert3005

`v0.88.0`

Compare Source

Important breaks

None

Changes

⚠️ Breaks

Speed up scalar construction, stats reads and aggregate partial merging (#10326) @robert3005
break(arrow): add export options and allow skipping view buffer compaction (#10220) @lorenzhs
Blocked BitPacked: store block offsets as a child (#10004) @mhk197
Delta validity is a top level child (#9970) @robert3005
Move users of map_each_with_validity and map_each_in_place to lane kernels and remove those methods (#10139) @robert3005

🚧 Deprecation

Remove usages of null_as_false (#10124) @myrrc

✨ Features

Add support for reading vortex files through native python readers (#10097) @robert3005
duckdb: don't execute validity in exporters (#10376) @myrrc
Blocked BitPacked: encode blocked bitwidths (#10203) @mhk197
AllFalse early exit (#10303) @myrrc
Scan protocol traits (#10047) @joseph-isaacs
Add combine_chunks option to Python Arrow export (#10279) @robert3005
feat(python): support integer column indices in read_url projection (#10287) @jackylee-ch

🚀 Performance

Bool take doesn't fill_null validity (#10383) @myrrc
perf(array): expand FixedSizeList filter masks directly into a bitmap (#10274) @robert3005
Make opening a Vortex file with a cached footer cheaper (#10358) @joseph-isaacs
perf: classify canonical arrays by concrete vtable (#10238) @joseph-isaacs
Add OnPair take kernel that shares the pair dictionary (#10363) @joseph-isaacs
perf(arrow): preserve source nullability during export (#10365) @joseph-isaacs
Read integer validity a word per chunk, and fill nulls for runs (#10342) @joseph-isaacs
Read VarBin UTF-8 RowFn inputs through their offsets (#10340) @connortsui20
Execute FSST iteratively (#10209) @joseph-isaacs
perf(array): avoid errors when comparison kernels decline arithmetic (#10301) @connortsui20
Faster ScalarFnArray creation, lazy ArrayStats (#10329) @myrrc
perf(buffer): use fearless_simd for popcount (#10327) @joseph-isaacs
Initialise ArrayParts directly with provided slots (#10072) @robert3005
Execute OnPair iteratively (#10210) @joseph-isaacs
perf(compressor): detect constant primitives before generating stats (#10043) @robert3005
perf: vectorize small u8 table take with AVX2 (#9572) @joseph-isaacs
perf: vectorize small u8 table take with NEON (#9571) @joseph-isaacs
Optimise min/max computation for VarBinView arrays (#9740) @robert3005
Execute DecimalByteParts iteratively (#10207) @joseph-isaacs
Execute RunEnd iteratively (#10206) @joseph-isaacs
perf(zstd): reuse one decompression context per thread (#10270) @joseph-isaacs
feat: add between kernel for DecimalByteParts (#10152) @joseph-isaacs
python: release the GIL only around expensive work (#10280) @robert3005
Convert trivial filters into slices during reduction, MaskValues::last uses BitBuffer::last_set_index (#9831) @robert3005
perf(array): forward probe_scalar through pass-through encodings (#9905) @joseph-isaacs
Perform validation and null view replacement in two passes (#10148) @robert3005

🐛 Bug Fixes

13 changes

fix(python): translate Polars time literals (#10186) @danking
fix(buffer): require both halves of a Zip to be TrustedLen (#10293) @jianhe25
fix(buffer): validate offsets in Buffer::into_arrow_offset_buffer (#10292) @jianhe25
fix(buffer): don't create &mut [u64] over uninitialized memory in collect_words_in (#10294) @jianhe25
Trim child data on Arrow slice imports (#10379) @palaska
duckdb: i16 Decimal on duckdb side may fit in Vortex's i8 decimal buffer (#10155) @myrrc
Keep the file sum absent when a chunk sum overflows (#10328) @connortsui20
Ignore empty constant chunks in aggregate states (#10254) @connortsui20
fix(parquet-variant): decline all_non_distinct when only one side is shredded (#10275) @jackylee-ch
fix(python): preserve nullability when converting a chunked array with nulls (#10283) @jackylee-ch
fix(buffer): enable and detect bmi1 for _tzcnt_u64 in select_in_word (#10295) @jianhe25
fix(array): preserve result nullability when simplifying and/or/zip (#10286) @jackylee-ch
fix(datafusion): preserve list element field metadata in physical schema (#10285) @jackylee-ch

🧰 Maintenance

14 changes

Skip validation when accessing storage scalar of extension scalar (#9875) @robert3005
Validate TPC-DS results against Vortex and Parquet (#9863) @joseph-isaacs
Test IO file sizes in Rust and array sizes in Python doctests (#9792) @robert3005
chore(array): use fearless_simd for AVX2 1- and 2-byte filter compress (#10324) @joseph-isaacs
Add sum oracle to the fuzzer and reject samples that have order dependent overflow behaviour (#9801) @robert3005
Do not build a Sequence array for slices that are not sequences (#10338) @robert3005
perf(buffer): portable fearless_simd kernels for bitmap rank-select (#10319) @joseph-isaacs
fix: reconcile asyncband references in Cargo.lock (#10306) @joseph-isaacs
Fix deprecations flagged by cargo build on new rust version (#10042) @robert3005
Add list_contains membership benchmarks (#10060) @robert3005
Clean up vortex_ensure_eq! (#10237) @connortsui20
Lock file maintenance (#10290) @renovate[bot]
Split SQL PR benchmarks into bench-sql and bench-sql-extended (#10213) @joseph-isaacs
Update bit-vec to 0.10.1 (#10288) @jianhe25

Configuration

📅 Schedule: (UTC)

Branch creation
- At any time (no schedule defined)
Automerge
- At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

renovate Bot force-pushed the renovate/vortex-and-datafusion-rust-crates branch from fbeb81c to f284a2c Compare October 10, 2026 04:03

renovate Bot changed the title ~~Update Vortex and DataFusion Rust crates to 55.2.0~~ Update Vortex and DataFusion Rust crates Oct 10, 2026

Update Vortex and DataFusion Rust crates

aab14b5

renovate Bot force-pushed the renovate/vortex-and-datafusion-rust-crates branch from f284a2c to aab14b5 Compare October 24, 2026 07:28

Navigation Menu

Repository navigation

Update Vortex and DataFusion Rust crates - #19

Update Vortex and DataFusion Rust crates#19
renovate[bot] wants to merge 1 commit into
mainfrom
renovate/vortex-and-datafusion-rust-crates

renovate Bot commented Oct 8, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

0 participants

Conversation

renovate Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release Notes

v55.2.0

v0.90.0

Changes

⚠️ Breaks

✨ Features

🚀 Performance

🐛 Bug Fixes

🧰 Maintenance

v0.89.0

Changes

⚠️ Breaks

✨ Features

🚀 Performance

🐛 Bug Fixes

🧰 Maintenance

v0.88.0

Important breaks

Changes

⚠️ Breaks

🚧 Deprecation

✨ Features

🚀 Performance

🐛 Bug Fixes

🧰 Maintenance

Configuration

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

0 participants

renovate Bot commented Oct 8, 2026 •

edited

Loading

`v55.2.0`

`v0.90.0`

`v0.89.0`

`v0.88.0`