DuckDB v1.5.4 Released: JSON Handling, Parquet Optimization & ADBC 1.1.0 Support

On June 17, 2026, the DuckDB team released v1.5.4 (Variegata), a significant maintenance release that brings a wealth of improvements across JSON data processing, Parquet file optimization, ADBC protocol upgrades, and over 100 stability fixes. This article provides an in-depth look at the new features and their practical value.

Release Overview

DuckDB v1.5.4 is the latest maintenance release in the 1.5.x series, reinforcing DuckDB’s position as the leading embedded analytical database. The three primary improvement areas are:

Comprehensive JSON Processing Upgrades

Wildcard Path Fixes

v1.5.4 fixes several critical issues with the json_keys function when handling wildcard paths. These fixes make it possible to precisely extract key-value pairs from complex nested JSON structures.

-- Create test table with nested JSON data
CREATE TABLE json_data AS
SELECT * FROM read_json_auto('https://raw.githubusercontent.com/duckdb/duckdb/main/data/json/ticket_order.json');

-- Extract customer names from all order items using wildcard paths
SELECT 
    json_keys(data, '$.order.items[*].customer.name') as customer_names
FROM json_data
LIMIT 5;

-- Now correctly handles NULL JSON keys
SELECT 
    data ->> '$.user.profile.*' as profile_fields
FROM json_data
WHERE data IS NOT NULL;

Array to JSON Conversion

The new version allows direct conversion of ARRAY types to JSON format, which is extremely useful for scenarios where relational data needs to be exported as semi-structured formats:

-- Convert array columns to JSON strings
CREATE TABLE array_test AS
SELECT 
    [1, 2, 3, 4, 5] as numbers,
    ['a', 'b', 'c'] as letters,
    [true, false, true] as booleans;

SELECT 
    array_to_json(numbers) as numbers_json,
    array_to_json(letters) as letters_json,
    array_to_json(booleans) as booleans_json
FROM array_test;

Output:

numbers_json    | ["1","2","3","4","5"]
letters_json    | ["a","b","c"]
booleans_json   | [true,false,true]

JSON Argument Order Fix

A bug was fixed where the argument order of json functions affected results, ensuring consistent output regardless of invocation method.

Deep Parquet Format Optimizations

Native Geometry Statistics Pruning

v1.5.4 introduces native statistics pruning for geometric data in Parquet files. When queries involve spatial data, DuckDB can directly leverage row group statistics to skip unnecessary file scans:

-- Load spatial extension
LOAD spatial;

-- Read Parquet files containing geometry data
-- DuckDB automatically uses row group statistics for pruning
SELECT 
    ST_Area(geometry_col) as area,
    ST_Centroid(geometry_col) as centroid
FROM parquet_scan('s3://data/spatial-data/*.parquet')
WHERE ST_Intersects(geometry_col, ST_MakeEnvelope(-180, -90, 180, 90));

This optimization is particularly important for processing TB-scale spatial datasets, reducing query times from minutes to seconds.

Parquet Variant Function Byte Exposure

The new parquet_variant_bytes function allows users to inspect the underlying byte representation of VARIANT types in Parquet files, which is invaluable for debugging and analyzing complex nested structures:

-- Inspect byte representation of VARIANT columns in Parquet files
SELECT 
    file_name,
    row_group_id,
    parquet_variant_bytes(variant_column) as variant_bytes
FROM parquet_metadata('s3://data/large-dataset/*.parquet');

Compression and Decompression Security Hardening

v1.5.4 has hardened multiple decompression/deserialization paths in both DuckDB and Parquet to prevent potential buffer overflow vulnerabilities. These security enhancements improve data safety without impacting performance.

ADBC 1.1.0 Rich Error Metadata API Implementation

Arrow Database Connectivity (ADBC) is the standardized API for database connections within the Arrow ecosystem. v1.5.4 fully implements the rich error metadata API specified in ADBC 1.1.0:

# Using ADBC to connect to DuckDB in Python
import adbc.dbapi.sqlite as adbc_sqlite
import pyarrow as pa

# Establish connection
with adbc_sqlite.connect(':memory:') as conn:
    with conn.cursor() as cursor:
        # Execute query
        cursor.execute("SELECT 1 as test")
        result = cursor.fetch_arrow_table()
        print(result)

# Rich error metadata example
try:
    cursor.execute("INVALID SQL QUERY")
except adbc_dbapi.ProgrammingError as e:
    # ADBC 1.1.0 provides detailed error metadata
    print(f"Error Code: {e.error_code}")
    print(f"Error Details: {e.message}")
    print(f"Status Info: {e.state}")

This improvement enables ADBC-based applications to obtain more detailed error information, facilitating debugging and error handling.

Other Important Improvements

Case-Insensitive Column Matching in INSERT … SELECT ON CONFLICT

Fixed case-insensitive column matching in INSERT ... SELECT ON CONFLICT statements, ensuring correct conflict resolution logic in mixed-case column name scenarios.

-- Create test table
CREATE TABLE users (
    id INTEGER PRIMARY KEY,
    Username VARCHAR,
    Email VARCHAR
);

-- Now correctly matches case-inconsistent column names
INSERT INTO users (id, username, email)
VALUES (1, '[email protected]', 'John Doe')
ON CONFLICT (username) DO UPDATE SET email = EXCLUDED.email;

Write Buffer Row Group Memory Limit

A new write_buffer_row_group_memory_limit configuration option allows users to control row group flushing based on memory usage, not just row group count:

-- Set memory-based row group flush threshold (in bytes)
SET write_buffer_row_group_memory_limit = '1GB';

-- DuckDB will auto-flush when row groups reach 1GB memory usage
COPY (SELECT * FROM large_table) TO 'output.parquet';

Jemalloc Integration Optimization

v1.5.4 integrates Jemalloc memory allocator deeper into the DuckDB core and fixes multiple edge cases related to thread flush thresholds, improving memory management efficiency in high-concurrency scenarios.

Performance Comparison

Here’s a performance comparison between v1.5.3 and v1.5.4 on typical query scenarios:

Comparison with Traditional Tools

Upgrade Recommendations

When to Upgrade

• Using JSON processing: Wildcard path fixes and array JSON conversion significantly improve developer experience
• Processing Parquet data: Native geometry statistics pruning dramatically accelerates spatial queries
• Need ADBC compatibility: Full 1.1.0 specification ensures interoperability with other Arrow tools
• Production stability: 100+ bug fixes improve overall reliability

Upgrade Steps

# Quick upgrade on Linux/Mac
curl -Ls https://github.com/duckdb/duckdb/releases/download/v1.5.4/duckdb_cli-linux-amd64.zip -o duckdb.zip
unzip -o duckdb.zip
chmod +x duckdb

# Python upgrade
pip install --upgrade duckdb

# Docker usage
docker pull duckdb/duckdb:v1.5.4

Monetization Suggestions

Data Product Opportunities

1. JSON Analytics SaaS: Leverage v1.5.4’s enhanced JSON processing to build SaaS products for e-commerce, log analysis, and more. For example, provide structured analysis services for cross-border e-commerce product reviews.

2. Spatial Data Visualization Platform: Combine native Parquet geometry statistics pruning to build high-performance spatial data analysis platforms serving logistics optimization, real estate analytics, and similar industries.

3. Real-time Data Pipelines: Utilize ADBC 1.1.0’s rich error metadata API to build more robust real-time data pipelines, providing reliable data integration for finance, IoT, and other sectors.

Cost Savings

• Replace traditional BI tools: DuckDB’s embedded architecture can reduce BI infrastructure costs by 60-80%
• Simplify ETL complexity: Native Parquet/JSON support reduces data transformation layers, lowering operational costs
• Improve query efficiency: Compared to in-memory solutions like Pandas, DuckDB can save 50%+ memory usage on GB-scale data

Commercialization Path

Summary

DuckDB v1.5.4 is a significant maintenance release that substantially enhances JSON processing, Parquet optimization, and ADBC compatibility while maintaining DuckDB’s signature simplicity and efficiency. For any team using or considering DuckDB, upgrading to v1.5.4 is highly recommended.

Visit the DuckDB website for more information, or check the v1.5.4 announcement for additional technical details.