LoginSign Up
← Back to all sparks
A

Apache Arrow Rust

DEVOPS
Velocity6.3

Rust implementation of the Apache Arrow columnar in-memory format

arrow.apache.org/rust ↗

Apache Arrow Rust 60.0.0 ships Parquet page index APIs and removes an ownership bottleneck in the Flight path

apache-arrowparquetrustcolumnar-datadata-engineeringflight
◆Current state
Apache Arrow Rust released version 60.0.0 with two notable changes: new PageIndexBuilder and PageIndexProvider APIs for Parquet page indexes, and a breaking ownership fix to batches_to_flight_data() that eliminates a redundant data copy. The project runs a stable major-release cadence with active maintenance backports for security and correctness fixes.
◆Where it's heading
The Parquet work is the consistent throughline — each major release expands codec support, compression capabilities, or predicate pushdown infrastructure. Page index APIs close a gap for Rust-based query engines that already handle bloom filters but lacked native page-level access. The Flight API ergonomics changes reflect ongoing work to remove unnecessary clones from Arrow-native data pipelines.
◆Prediction
Future releases will continue refining Parquet's ALP floating-point encoding (changes began in 60.0.0) and likely add more granular page-level statistics access. The IPC sans-IO encoder shipped in 59.2.0 is likely to get broader adoption hooks in the next few versions.

◆Recent moves

  1. 20h ago

    arrow 60.0.0

    ⚡ SPARK

    Arrow Rust 60.0.0 ships the first stable PageIndexBuilder and PageIndexProvider APIs for Parquet page-level indexes — a capability query engines need for fine-grained predicate pushdown beyond row-group filtering. The breaking change to batches_to_flight_data() also removes a long-standing unnecessary ownership requirement in the Flight path.

    View source ↗
  2. 5d ago

    arrow 60.0.0 release candidate

    Release candidate preparation for 60.0.0: version bump and changelog additions. No functional changes versus the prior development commits.

    View source ↗
  3. 15d ago

    arrow 59.3.0

    Arrow Rust 59.3.0 backports a security fix for a CVE in the h2 dependency and patches a correctness bug in concat_run_arrays with all-empty arrays. Maintenance release targeting the 59.x stable series.

    View source ↗
  4. 20d ago

    arrow 59.3.0 RC2 (maintenance backport)

    59.3.0 second release candidate. Adds a parquet-testing revision update for ALP encoder tests to the maintenance backport stream.

    View source ↗
  5. 20d ago

    arrow 59.3.0 RC1

    59.3.0 first release candidate. Version bump and changelog prep for the 59.x maintenance branch.

    View source ↗
  6. 1mo ago

    arrow 59.2.0

    Arrow Rust 59.2.0 adds unchecked array builder methods for performance-sensitive paths, a sans-IO IPC stream encoder for embedding Arrow IPC in non-async contexts, and deprecates the object_store integration in arrow-avro. These APIs expand what Rust-native tools can build without reaching for unsafe code or the C++ library.

    View source ↗