LoginSign Up
← Back to all sparks
A

Apache Iceberg

ANALYTICS
Velocity0.0

Open table format for huge analytic datasets with schema evolution

iceberg.apache.org ↗

Iceberg 1.12 drops Spark 3.4 and starts laying groundwork for the v4 table spec.

table-formatlakehouserest-catalogv4-specvariant-type
◆Current state
Apache Iceberg maintains several minor lines at once, and most visible work is backports and dependency updates. 1.12.0 on September 30 is broader: it removes Spark 3.4 support, adds V4 groundwork such as relative paths in the spec and Parquet or Avro manifests, adds a REST unregister-table endpoint and content stats in the spec, and writes shredded Variant from Flink. Kafka Connect now reports commit failures instead of swallowing them.
◆Where it's heading
The project is moving from maintenance-only releases back to spec work, with V4 pieces landing in stages. Older engine versions are being dropped as that happens.
◆Prediction
Expect more V4 spec and core pieces in 1.13 and patch releases on 1.11 and 1.12; a V4 release date is not visible in the entries.

◆Recent moves

  1. 1mo ago

    Iceberg 1.12.0: Spark 3.4 dropped, first V4 spec pieces

    1.12.0 removes Spark 3.4, starts V4 work with relative paths and new manifest formats, and adds a REST unregister-table endpoint and Flink shredded Variant writes. It is the first release in months that changes the format and catalog rather than only backporting fixes.

    View source ↗
  2. 5mo ago

    1.11.0 opens a new line on Spark 4.0.1

    The new minor line arrives dominated by dependency bumps and build maintenance, with the Spark version moved to 4.0.1 and a schema-evolution spec note reverted. Opening a third live line while 1.10.x still takes backports is the cadence pattern this project now runs on.

    View source ↗
  3. 5mo ago

    Backport release fixes delete ordering and a compression CVE

    A pure backport release: non-deterministic equality-delete schema ordering, snapshot loading that could trigger accidental file cleanup, non-deterministic Flink operator UIDs, and a CVE in the aircompressor dependency. The cleanup-on-failure fixes are the ones that matter — they protect data, not throughput.

    View source ↗
  4. 10mo ago

    Nullability and REST catalog validation fixes

    Fixes required fields nested in optional structs producing nulls, namespace drops that ignored contained views, timestamp-nanos default overflow, and PlanTableScanRequest validation. The concentration on nullability and REST validation shows where the format's remaining sharp edges are.

    View source ↗
  5. 1y ago

    Flink 2.0 support and Variant type reaches Parquet

    The substantive minor in this window: Flink 2.0 support lands, and Variant gains timestamp-nanos, time and UUID types plus a Parquet array reader. Engine breadth and semi-structured data are where Iceberg is spending, and the subsequent patch stream is largely cleaning up after it.

    View source ↗
  6. 1y ago

    Stop retrying object-store 502 and 504 responses

    A single-change patch that stops retrying on 502 and 504 from object storage. Narrow, but it belongs to the same theme as the newer cleanup fixes: getting failure handling right rather than adding surface.

    View source ↗