Ecosystem Tools on DuckDB Lab

Build a SQL Dashboard in 10 Minutes with Shaper: DuckDB's Open-Source Viz Tool

Thu, 14 May 2026 00:00:00 +0000

The Problem: Data Visualization is a Pain

You’ve got your data neatly organized in DuckDB. Your boss wants a dashboard — yesterday.

Your options:

Tableau → $75/user/month. For a 3-person team that’s $2,700/year.
Metabase → Free but heavy. Need a separate server, Java runtime, lots of config.
Python + Plotly → 200+ lines of code for what should be a 5-line query.
Excel charts → 1990 called, they want their workflow back.

What if you could just write SQL and get a chart?

Shaper is exactly that — an open-source, SQL-driven dashboard tool powered by DuckDB under the hood. Write SQL queries with type annotations, and Shaper renders them as bar charts, line charts, pie charts, tables, and more.

GitHub: https://github.com/taleshape-com/shaper (1.1k ⭐, actively maintained)

What is Shaper?

Shaper’s positioning is crystal clear: the DuckDB visualization layer for SQL users.

“As Official says: All in SQL, Powered by DuckDB.”

The core idea: You don’t need to learn any new API or DSL. Just append ::BARCHART, ::XAXIS, etc. to your SQL columns and Shaper figures out how to visualize them.

-- Shaper SQL example
SELECT
 date_trunc('week', created_at)::XAXIS,
 category::CATEGORY,
 count()::BARCHART_STACKED
FROM dataset
GROUP BY ALL ORDER BY ALL;

This SQL produces a stacked bar chart — no JavaScript, no JSON config, no drag-and-drop.

Key Features

Capability	Description
Fully Open Source	MPL-2.0 license, self-hosted
SQL-First	Define charts via SQL type annotations
Git Workflow	Version-control your dashboards
Multi-Source	Query across DuckDB, CSV, Parquet, MySQL, Postgres
White-Label Embed	Embed in iframe or via JS/React SDK without branding
Export	PDF, PNG, CSV, Excel with one click
Scheduled Reports	Auto-generate and deliver reports
Row-Level Security	JWT tokens for data access control

10-Minute Quickstart

Minute 1: Start Shaper

The fastest way to try Shaper is via Docker:

docker run --rm -it -p5454:5454 taleshape/shaper

Open http://localhost:5454 — you’ll see a clean dashboard editor.

No Docker? Shaper also ships as npm and pip packages:

npm install @taleshape/shaper

pip install shaper

Minutes 2-5: Import Data + First Query

Click “New Query” and write your first SQL:

-- See what we're working with
SELECT * FROM read_csv_auto('sales_2024.csv') LIMIT 10;

Shaper uses DuckDB’s engine natively, so all DuckDB features work — read_csv_auto, read_parquet, ATTACH for MySQL/PostgreSQL…

Now for a real chart:

SELECT
 strftime(order_date, '%Y-%m')::XAXIS,
 product_category::CATEGORY,
 SUM(amount)::BARCHART_STACKED
FROM read_csv_auto('sales_2024.csv')
GROUP BY ALL ORDER BY ALL;

Minutes 6-10: Assemble Your Dashboard

Add more queries to build a complete dashboard:

KPI Card:

SELECT
 'Total Revenue'::LABEL,
 '$' || FORMAT('%,.0f', SUM(amount))::VALUE,
 'vs last month ' || CASE WHEN SUM(amount) - LAG(SUM(amount)) OVER () > 0 THEN '↑' ELSE '↓' END || FORMAT('%.1f%%', ABS((SUM(amount) - LAG(SUM(amount)) OVER ()) / LAG(SUM(amount)) OVER () * 100))::SUBTITLE
FROM read_csv_auto('sales_2024.csv');

Trend Line:

SELECT
 order_date::XAXIS,
 SUM(amount)::LINE
FROM read_csv_auto('sales_2024.csv')
GROUP BY ALL ORDER BY ALL;

Top 10 Products:

SELECT
 product_name::LABEL,
 SUM(amount)::BARCHART
FROM read_csv_auto('sales_2024.csv')
GROUP BY ALL ORDER BY SUM(amount) DESC LIMIT 10;

Ten minutes. A 4-component professional dashboard. All SQL. No frontend code.

Advanced Features

4.1 KPI Monitoring & Alerts

Shaper supports scheduled scans that trigger alerts when metrics cross thresholds:

-- Alert if today's sales < $10,000
SELECT
 SUM(amount)::VALUE,
 CASE WHEN SUM(amount) < 10000 THEN '🚨 Below Target'
 ELSE '✅ On Track' END::STATUS
FROM read_csv_auto('sales_2024.csv')
WHERE order_date = CURRENT_DATE;

4.2 Embedded Dashboards

Want to embed dashboards in your product? Shaper supports:

iframe embedding: One line of HTML
JS/React SDK: Full control over styling, no iframe needed
White-label mode: Hide Shaper branding

<!-- iframe embed -->
<iframe src="https://your-shaper.com/d/sales-dashboard"
 width="100%" height="600px"
 frameborder="0"></iframe>

// React SDK approach
import { ShaperDashboard } from '@taleshape/shaper-react';

function App() {
 return <ShaperDashboard dashboardId="sales-dashboard"
 token="your-jwt-token" />;
}

4.3 Scheduled PDF Reports

Configure a cron job to generate PDF reports on schedule:

# Export via CLI
curl -X POST https://your-shaper.com/api/dashboards/sales-weekly/export \
 -H "Authorization: Bearer your-token" \
 -d '{"format": "pdf"}'

When Should You Use Shaper?

Best for:

Teams already using DuckDB for data analysis
SQL analysts who need quick charts
Developers building client-facing dashboards
Backend engineers who want to avoid JavaScript

Not ideal for:

Complex interactive dashboards with drill-down animations
Non-technical users who need drag-and-drop (Shaper is SQL-driven)

Bottom line: If you already use DuckDB, Shaper is the shortest path from query to dashboard to share.

Monetization Ideas

Shaper is free and open-source, but it opens up several business opportunities:

DuckDB + Shaper Dashboard Service: Build data dashboards for SMBs, $300-800/project
Embedded Analytics Module: Integrate Shaper dashboards into your SaaS product
Shaper Training + Templates: Video courses or downloadable dashboard templates
Industry-Specific Dashboard Packs: Pre-built templates for e-commerce, logistics, finance

Summary

Shaper fills the missing visualization layer in the DuckDB ecosystem. It bridges the gap between “querying data with SQL” and “seeing charts that drive decisions” — no new tools, no frontend code, just the SQL you already know.

10 minutes. Zero to a professional dashboard. That’s Shaper’s promise.

Get started now:

docker run --rm -it -p5454:5454 taleshape/shaper

Open http://localhost:5454 and give it a try.

DuckLake v1.0: Lightweight Lakehouse Solution

Sun, 10 May 2026 00:00:00 +0000

Introduction

Over the past decade, the line between data warehouses and data lakes has blurred, giving rise to the Lakehouse architecture. Databricks’ Delta Lake, Apache Iceberg, and Apache Hudi have dominated the lakehouse format landscape — but they all share a common problem: they are too heavy.

Running any of these three formats typically requires Spark, Hive Metastore, HDFS or object storage, and a catalog service. For small-to-medium teams, this is not just a steep learning curve — it’s an operational nightmare.

DuckDB v1.5’s DuckLake format is designed to solve exactly this problem.

DuckLake is not a replacement for Parquet or Delta Lake. Rather, it provides a lightweight lakehouse format suited for embedded scenarios — no Spark, no Metastore, no catalog service. Just DuckDB handling everything from writes and queries to lifecycle management.

What is DuckLake?

DuckLake is a structured lakehouse storage format natively supported by DuckDB. Under the hood, it is a collection of Parquet files accompanied by metadata files, tracked through a Transaction Log that records every write operation. This provides ACID transactions, time-travel queries, and incremental read capabilities.

Core Features

Zero external dependencies: No Spark, Hive, HDFS, or catalog services required
ACID transactions: Concurrent write isolation through file-level optimistic locking
Schema evolution: Add/drop columns, modify types
Time travel: Query any historical version
Incremental queries: Read only newly written data shards
Open format compatibility: Underlying data stored as Parquet — any Parquet reader can consume it

The name “DuckLake” is straightforward: Duck + Lake. DuckDB is the duck, the Lake is the lakehouse — putting the lake into the duck perfectly captures the essence of lightweight lakehousing.

Installation & Setup

DuckLake ships as a built-in feature of DuckDB v1.5. No extension installation required:

-- Verify DuckDB version (requires v1.5+)
SELECT version();

-- Confirm DuckLake support
SELECT * FROM duckdb_extensions() WHERE extension_name = 'ducklake';

For Python users, it’s equally straightforward:

import duckdb

con = duckdb.connect()
# DuckLake is immediately available — no extra pip packages needed

COPY TO Syntax Deep Dive

The core write interface for DuckLake is the COPY TO statement. In v1.5, COPY TO has been significantly extended to support direct DuckLake format writes:

Basic Syntax

-- Write query results in DuckLake format
COPY (SELECT * FROM orders)
TO 'data/orders.ducklake'
(FORMAT DUCKLAKE, APPEND FALSE);

-- Append writes (create new version)
COPY (SELECT * FROM new_orders)
TO 'data/orders.ducklake'
(FORMAT DUCKLAKE, APPEND TRUE);

Key Parameters

Parameter	Type	Default	Description
`FORMAT`	Enum	None	Must be set to `DUCKLAKE`
`APPEND`	Boolean	`FALSE`	`TRUE` appends new data; `FALSE` overwrites the entire Lake
`COMPRESSION`	Enum	`ZSTD`	Parquet compression: `ZSTD`/`SNAPPY`/`LZ4`/`UNCOMPRESSED`
`ROW_GROUP_SIZE`	Integer	`122880`	Rows per Row Group
`OVERWRITE_SCHEMA`	Boolean	`FALSE`	Allows schema changes on append
`PARTITION_BY`	Column list	Empty	Partition storage by specified columns

Advanced Usage

-- Partitioned write with ZSTD compression
COPY (SELECT * FROM events WHERE year = 2026)
TO 'data/events.ducklake'
(FORMAT DUCKLAKE, PARTITION_BY (region, dt), COMPRESSION 'ZSTD');

-- Overwrite schema on append
COPY (SELECT id, name, email, signup_date FROM users_v2)
TO 'data/users.ducklake'
(FORMAT DUCKLAKE, APPEND TRUE, OVERWRITE_SCHEMA TRUE);

Reading DuckLake

-- Basic read (latest version)
SELECT * FROM 'data/orders.ducklake';

-- Time travel: read specific version
SELECT * FROM 'data/orders.ducklake' (VERSION 3);

-- Time travel: read at specific timestamp
SELECT * FROM 'data/orders.ducklake' (TIMESTAMP '2026-05-09 12:00:00');

-- View version history
SELECT * FROM ducklake_versions('data/orders.ducklake');

Management Operations

-- Compact (merge small files)
CALL ducklake_compact('data/orders.ducklake');

-- Clean up expired versions
CALL ducklake_vacuum('data/orders.ducklake', KEEP_VERSIONS 10);

-- Get statistics
SELECT * FROM ducklake_stats('data/orders.ducklake');

Multi-Client Support

A key strength of DuckLake is its broad ecosystem compatibility. Here’s how to access DuckLake from various clients:

Python (DuckDB + PyArrow)

import duckdb
import pandas as pd

con = duckdb.connect()

# Write DuckLake
con.execute("""
 COPY (SELECT * FROM range(1000000) t(id))
 TO 'test.ducklake' (FORMAT DUCKLAKE)
""")

# Read as Pandas DataFrame
df = con.execute(
 "SELECT * FROM 'test.ducklake'"
).df()

# Read as PyArrow Table
import pyarrow as pa
table = con.execute(
 "SELECT * FROM 'test.ducklake'"
).arrow()

R Language

library(duckdb)
library(dplyr)

con <- dbConnect(duckdb())

# Read DuckLake
df <- tbl(con, "test.ducklake") %>%
 filter(id > 500000) %>%
 collect()

print(df)

Java / JDBC

// pom.xml: add duckdb-jdbc dependency
Connection conn = DriverManager.getConnection("jdbc:duckdb:");
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery(
 "SELECT count(*) FROM 'data/orders.ducklake'"
);
while (rs.next()) {
 System.out.println(rs.getLong(1));
}

Node.js

const duckdb = require('duckdb');
const db = new duckdb.Database(':memory:');
db.all("SELECT * FROM 'data/orders.ducklake' LIMIT 10",
 (err, rows) => {
 if (err) throw err;
 console.log(rows);
 }
);

Command-Line CLI

# Query directly via DuckDB CLI
duckdb -c "SELECT region, count(*) FROM 'data/sales.ducklake' GROUP BY region"

# Export to CSV
duckdb -c "COPY (SELECT * FROM 'data/sales.ducklake') TO 'export.csv' (HEADER TRUE)"

Parquet vs Delta Lake vs Iceberg vs DuckLake

Here is a comprehensive comparison matrix to help you make informed technology decisions:

Dimension	Parquet	Delta Lake	Apache Iceberg	DuckLake v1.0
Type	Columnar file format	Lakehouse table format	Lakehouse table format	Lightweight lakehouse format
ACID transactions	❌ Not supported	✅ Optimistic concurrency	✅ Optimistic concurrency	✅ File-level optimistic lock
Schema evolution	❌ Not supported	✅ Supported	✅ Supported	✅ Supported
Time travel	❌ Not supported	✅ 30-day default	✅ By snapshot	✅ By version/timestamp
Incremental reads	❌ Full scan	✅ By version	✅ By snapshot	✅ By version
Partition pruning	✅ Statistics-based	✅ Partition trimming	✅ Hidden partitioning	✅ Partition pruning
File compaction	❌ External tools	✅ OPTIMIZE command	✅ Rewrite operations	✅ `ducklake_compact`
Metadata management	❌ None	Hive Metastore / AWS Glue	Hive / REST / Nessie	No Metastore needed
Execution engines	Any engine	Spark / Flink / Trino / DuckDB	Spark / Flink / Trino / DuckDB	DuckDB native
CPU architecture	x86 / ARM / RISC-V	x86 / ARM	x86 / ARM	x86 / ARM / RISC-V
Embedded scenarios	⚠️ Usable, no transactions	❌ Too heavy	❌ Too heavy	✅ Born for it
External dependencies	None	Spark + Hive + HDFS	Spark + Hive + HDFS	Zero dependencies
Query performance	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐⭐
Write performance	⭐⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐	⭐⭐⭐⭐
Ecosystem maturity	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐ (Rapidly evolving)
Open source license	Apache 2.0	Apache 2.0	Apache 2.0	MIT

Selection Guide

Already using Spark/Flink ecosystem → Stick with Delta Lake or Iceberg
Need only a file format → Use Parquet
Small-to-medium team wanting lakehouse capabilities without Spark → DuckLake is the best choice
Embedded or mobile data solutions → DuckLake (DuckDB’s embedded design is a natural fit)
Startup validating an idea quickly → DuckLake, with zero operational overhead

Complete Runnable SQL Example

Below is an end-to-end hands-on example simulating an e-commerce order analytics scenario:

-- ===========================================
-- DuckLake in Action: E-Commerce Order Analytics
-- ===========================================

-- 1. Prepare sample data
CREATE OR REPLACE TABLE raw_orders AS
SELECT * FROM (VALUES
 (1001, 'Alice', 'Electronics', 2999.00, '2026-05-01'::DATE),
 (1002, 'Bob', 'Clothing', 459.00, '2026-05-01'::DATE),
 (1003, 'Charlie', 'Food', 89.90, '2026-05-02'::DATE),
 (1004, 'Alice', 'Books', 79.00, '2026-05-03'::DATE),
 (1005, 'David', 'Electronics', 1599.00, '2026-05-03'::DATE),
 (1006, 'Bob', 'Food', 120.50, '2026-05-04'::DATE),
 (1007, 'Eve', 'Clothing', 899.00, '2026-05-04'::DATE),
 (1008, 'Charlie', 'Electronics', 4599.00, '2026-05-05'::DATE),
 (1009, 'Alice', 'Food', 210.00, '2026-05-06'::DATE),
 (1010, 'David', 'Books', 150.00, '2026-05-06'::DATE)
) t(order_id, customer, category, amount, order_date);

-- 2. Write to DuckLake (Version 1)
COPY raw_orders TO 'ecommerce.ducklake' (FORMAT DUCKLAKE);

-- 3. View version history
SELECT * FROM ducklake_versions('ecommerce.ducklake');

-- 4. Query: category sales summary
SELECT category,
 count(*) AS order_count,
 round(sum(amount), 2) AS total_sales,
 round(avg(amount), 2) AS avg_amount
FROM 'ecommerce.ducklake'
GROUP BY category
ORDER BY total_sales DESC;

-- 5. Append new orders (Version 2)
INSERT INTO raw_orders VALUES
 (1011, 'Eve', 'Electronics', 3200.00, '2026-05-07'),
 (1012, 'Bob', 'Books', 55.00, '2026-05-07');

COPY (SELECT * FROM raw_orders WHERE order_id > 1010)
TO 'ecommerce.ducklake' (FORMAT DUCKLAKE, APPEND TRUE);

-- 6. Time travel: view Version 1 data
SELECT sum(amount) AS version_1_total
FROM 'ecommerce.ducklake' (VERSION 1);

-- 7. Time travel: view latest data
SELECT sum(amount) AS latest_total
FROM 'ecommerce.ducklake' (VERSION 2);

-- 8. Schema evolution: add new column
ALTER TABLE raw_orders ADD COLUMN shipping_address VARCHAR;
UPDATE raw_orders SET shipping_address = CASE
 WHEN customer = 'Alice' THEN 'Haidian, Beijing'
 WHEN customer = 'Bob' THEN 'Pudong, Shanghai'
 WHEN customer = 'Charlie' THEN 'Tianhe, Guangzhou'
 WHEN customer = 'David' THEN 'Nanshan, Shenzhen'
 WHEN customer = 'Eve' THEN 'Xihu, Hangzhou'
END;

-- 9. Overwrite with new schema (Version 3)
COPY raw_orders TO 'ecommerce.ducklake'
(FORMAT DUCKLAKE, OVERWRITE_SCHEMA TRUE);

-- 10. Verify schema evolution succeeded
DESCRIBE SELECT * FROM 'ecommerce.ducklake' (VERSION 3);

-- 11. Advanced analysis: window functions
SELECT customer, category, amount,
 sum(amount) OVER (PARTITION BY customer) AS customer_total,
 rank() OVER (PARTITION BY category ORDER BY amount DESC) AS category_rank
FROM 'ecommerce.ducklake' (VERSION 3)
ORDER BY category, category_rank;

-- 12. Compaction and cleanup
CALL ducklake_compact('ecommerce.ducklake');
CALL ducklake_vacuum('ecommerce.ducklake', KEEP_VERSIONS 3);

-- 13. Final verification
SELECT category,
 sum(amount) AS total,
 count(*) AS orders
FROM 'ecommerce.ducklake'
GROUP BY category;

Expected Results

┌──────────────┬──────────────┬──────────┐
│ category │ order_count │ total │
│ varchar │ int64 │ decimal │
├──────────────┼──────────────┼──────────┤
│ Electronics │ 3 │ 9798.00 │
│ Clothing │ 2 │ 1358.00 │
│ Food │ 3 │ 420.40 │
│ Books │ 3 │ 284.00 │
└──────────────┴──────────────┴──────────┘

Monetization Strategies

DuckLake, as an emerging lightweight lakehouse format, presents multiple commercialization opportunities:

1. DuckLake Data Pipeline Service

Target audience: SMBs and independent developers

Build a DuckLake-based data pipeline orchestration service (lightweight Airbyte alternative)
SaaS platform: users configure data sources, data is automatically written in DuckLake format
Pricing model: storage volume + API call count
Estimated monthly fee: $29–$199/month depending on data volume

2. DuckLake-Native Visualization Tool

Target audience: Business analysts, non-technical users

Build a DuckLake-native BI visualization tool (lightweight Metabase alternative)
Leverage DuckDB’s embedded nature: browser-side + DuckDB WASM reads DuckLake directly
Key selling point: no backend service needed — drag and drop files to analyze
Business model: Open-source community edition + Enterprise (permissions, team collaboration)

3. DuckLake Conversion Service

Target audience: Enterprises with legacy data

One-click conversion service: JSON / CSV / Database → DuckLake format
Enterprise tier supports incremental sync and CDC (Change Data Capture)
TAM (Total Addressable Market): All SMBs using CSV and JSON for data analysis

4. DuckLake Data Marketplace

Target audience: Data providers and consumers

Build a data marketplace built on DuckLake format
Data providers upload DuckLake-formatted datasets
Consumers pay per-use or via subscription
Core advantage: DuckLake’s time-travel capability enables historical version tracking

5. Embedded / IoT Solutions

Target audience: Edge computing devices, IoT gateways

Run DuckDB + DuckLake on Raspberry Pi / Jetson Nano
Use cases: data collection, local aggregation, incremental upload
Compared to traditional approach: eliminates the two-step SQLite-to-Parquet pipeline
Pricing: per-node licensing ($5/node/month)

6. Training & Consulting

Target audience: DuckDB and Lakehouse beginners

Create “DuckLake from Zero to Hero” paid course (Udemy / indie platform)
Enterprise training: DuckDB + DuckLake best practices
Technical consulting: Legacy warehouse migration to DuckLake
Pricing reference: Beginner course $49.9, enterprise training $2000–$5000/day

Conclusion

DuckLake v1.0 is a strategically significant addition to the DuckDB ecosystem. It breaks the long-held assumption that “lakehouse = heavy infrastructure,” proving that a full lakehouse capability can be realized within a single embedded OLAP engine.

Its core value proposition is clear:

Zero-dependency deployment — No Spark, Hive Metastore, or HDFS required
Out-of-the-box ACID — Every DuckDB instance is a complete lakehouse engine
Drastically lower TCO — Hardware, operations, and personnel costs all significantly reduced
Seamless compatibility — Underlying Parquet files ensure data isn’t locked in

What makes DuckLake most exciting for data practitioners is that it brings lakehouse capabilities from the data center to your laptop, a Raspberry Pi, or even a browser. When you can run a full ACID lakehouse on a notebook, the boundaries of what’s possible in data engineering expand dramatically.

DuckLake isn’t here to replace Delta Lake or Iceberg — in large-scale data center scenarios, the mature ecosystem of the three giants still provides irreplaceable advantages. But for small teams, startups, individual developers, and edge computing use cases, DuckLake is arguably the most elegant option available today.

We’d love to hear your thoughts and experiences with DuckLake in the comments below!