DuckDB v1.5.4 Released: JSON Handling, Parquet Optimization & ADBC 1.1.0 Support

On June 17, 2026, the DuckDB team released v1.5.4 (Variegata), a significant maintenance release that brings a wealth of improvements across JSON data processing, Parquet file optimization, ADBC protocol upgrades, and over 100 stability fixes. This article provides an in-depth look at the new features and their practical value.

Release Overview

DuckDB v1.5.4 is the latest maintenance release in the 1.5.x series, reinforcing DuckDB’s position as the leading embedded analytical database. The three primary improvement areas are:

Comprehensive JSON Processing Upgrades

Wildcard Path Fixes

v1.5.4 fixes several critical issues with the json_keys function when handling wildcard paths. These fixes make it possible to precisely extract key-value pairs from complex nested JSON structures.

-- Create test table with nested JSON data
CREATE TABLE json_data AS
SELECT * FROM read_json_auto('https://raw.githubusercontent.com/duckdb/duckdb/main/data/json/ticket_order.json');

-- Extract customer names from all order items using wildcard paths
SELECT 
 json_keys(data, '$.order.items[*].customer.name') as customer_names
FROM json_data
LIMIT 5;

-- Now correctly handles NULL JSON keys
SELECT 
 data ->> '$.user.profile.*' as profile_fields
FROM json_data
WHERE data IS NOT NULL;

Array to JSON Conversion

The new version allows direct conversion of ARRAY types to JSON format, which is extremely useful for scenarios where relational data needs to be exported as semi-structured formats:

-- Convert array columns to JSON strings
CREATE TABLE array_test AS
SELECT 
 [1, 2, 3, 4, 5] as numbers,
 ['a', 'b', 'c'] as letters,
 [true, false, true] as booleans;

SELECT 
 array_to_json(numbers) as numbers_json,
 array_to_json(letters) as letters_json,
 array_to_json(booleans) as booleans_json
FROM array_test;

Output:

numbers_json | ["1","2","3","4","5"]
letters_json | ["a","b","c"]
booleans_json | [true,false,true]

JSON Argument Order Fix

A bug was fixed where the argument order of json functions affected results, ensuring consistent output regardless of invocation method.

Deep Parquet Format Optimizations

Native Geometry Statistics Pruning

v1.5.4 introduces native statistics pruning for geometric data in Parquet files. When queries involve spatial data, DuckDB can directly leverage row group statistics to skip unnecessary file scans:

-- Load spatial extension
LOAD spatial;

-- Read Parquet files containing geometry data
-- DuckDB automatically uses row group statistics for pruning
SELECT 
 ST_Area(geometry_col) as area,
 ST_Centroid(geometry_col) as centroid
FROM parquet_scan('s3://data/spatial-data/*.parquet')
WHERE ST_Intersects(geometry_col, ST_MakeEnvelope(-180, -90, 180, 90));

This optimization is particularly important for processing TB-scale spatial datasets, reducing query times from minutes to seconds.

Parquet Variant Function Byte Exposure

The new parquet_variant_bytes function allows users to inspect the underlying byte representation of VARIANT types in Parquet files, which is invaluable for debugging and analyzing complex nested structures:

-- Inspect byte representation of VARIANT columns in Parquet files
SELECT 
 file_name,
 row_group_id,
 parquet_variant_bytes(variant_column) as variant_bytes
FROM parquet_metadata('s3://data/large-dataset/*.parquet');

Compression and Decompression Security Hardening

v1.5.4 has hardened multiple decompression/deserialization paths in both DuckDB and Parquet to prevent potential buffer overflow vulnerabilities. These security enhancements improve data safety without impacting performance.

ADBC 1.1.0 Rich Error Metadata API Implementation

Arrow Database Connectivity (ADBC) is the standardized API for database connections within the Arrow ecosystem. v1.5.4 fully implements the rich error metadata API specified in ADBC 1.1.0:

# Using ADBC to connect to DuckDB in Python
import adbc.dbapi.sqlite as adbc_sqlite
import pyarrow as pa

# Establish connection
with adbc_sqlite.connect(':memory:') as conn:
 with conn.cursor() as cursor:
 # Execute query
 cursor.execute("SELECT 1 as test")
 result = cursor.fetch_arrow_table()
 print(result)

# Rich error metadata example
try:
 cursor.execute("INVALID SQL QUERY")
except adbc_dbapi.ProgrammingError as e:
 # ADBC 1.1.0 provides detailed error metadata
 print(f"Error Code: {e.error_code}")
 print(f"Error Details: {e.message}")
 print(f"Status Info: {e.state}")

This improvement enables ADBC-based applications to obtain more detailed error information, facilitating debugging and error handling.

Other Important Improvements

Case-Insensitive Column Matching in INSERT … SELECT ON CONFLICT

Fixed case-insensitive column matching in INSERT ... SELECT ON CONFLICT statements, ensuring correct conflict resolution logic in mixed-case column name scenarios.

-- Create test table
CREATE TABLE users (
 id INTEGER PRIMARY KEY,
 Username VARCHAR,
 Email VARCHAR
);

-- Now correctly matches case-inconsistent column names
INSERT INTO users (id, username, email)
VALUES (1, 'john@example.com', 'John Doe')
ON CONFLICT (username) DO UPDATE SET email = EXCLUDED.email;

Write Buffer Row Group Memory Limit

A new write_buffer_row_group_memory_limit configuration option allows users to control row group flushing based on memory usage, not just row group count:

-- Set memory-based row group flush threshold (in bytes)
SET write_buffer_row_group_memory_limit = '1GB';

-- DuckDB will auto-flush when row groups reach 1GB memory usage
COPY (SELECT * FROM large_table) TO 'output.parquet';

Jemalloc Integration Optimization

v1.5.4 integrates Jemalloc memory allocator deeper into the DuckDB core and fixes multiple edge cases related to thread flush thresholds, improving memory management efficiency in high-concurrency scenarios.

Performance Comparison

Here’s a performance comparison between v1.5.3 and v1.5.4 on typical query scenarios:

Comparison with Traditional Tools

Upgrade Recommendations

When to Upgrade

• Using JSON processing: Wildcard path fixes and array JSON conversion significantly improve developer experience
• Processing Parquet data: Native geometry statistics pruning dramatically accelerates spatial queries
• Need ADBC compatibility: Full 1.1.0 specification ensures interoperability with other Arrow tools
• Production stability: 100+ bug fixes improve overall reliability

Upgrade Steps

# Quick upgrade on Linux/Mac
curl -Ls https://github.com/duckdb/duckdb/releases/download/v1.5.4/duckdb_cli-linux-amd64.zip -o duckdb.zip
unzip -o duckdb.zip
chmod +x duckdb

# Python upgrade
pip install --upgrade duckdb

# Docker usage
docker pull duckdb/duckdb:v1.5.4

Summary

DuckDB v1.5.4 is a significant maintenance release that substantially enhances JSON processing, Parquet optimization, and ADBC compatibility while maintaining DuckDB’s signature simplicity and efficiency. For any team using or considering DuckDB, upgrading to v1.5.4 is highly recommended.

Visit the DuckDB website for more information, or check the v1.5.4 announcement for additional technical details.

本文信息

如发现错误，欢迎通过 GitHub Issue 或邮件 contact@duckdblab.org 反馈。