Featured image of post Build a Lightweight Data Analytics API with DuckDB + FastAPI

Build a Lightweight Data Analytics API with DuckDB + FastAPI

Build a zero-maintenance data analytics API with DuckDB and FastAPI, package it as a service product for SMEs. Complete architecture, runnable code, pricing strategies, and customer acquisition playbook included. saving your first server cost back in 23 days

The Data Analyst’s Dilemma

If you’re a data analyst trying to DuckDB skills on the side, you’ve probably been stuck in this loop:

You help someone run an analysis → They think it’s great → You quote ¥500 → They say “let me think about it” → Three months later they’re still asking around for “cheap data analysis services”

The problem is clear: you’re selling one-time labor, not a repeatable product.

But there’s a massively underrated alternative: Use DuckDB + FastAPI to build a lightweight data analytics API, package it as a SaaS product, and charge monthly.

The insight is simple: you productize the capability to analyze. Customers upload their data, your system automatically provides insights and reports. Your operational cost? Near zero.

This article is a complete walkthrough of the architecture, code, deployment, and path.


Why DuckDB Is the Perfect Engine for This

When choosing your tech stack, consider these options:

ApproachDeploy DifficultyOps CostQuery SpeedPer-Customer Cost10-Customer Monthly Profit
PostgreSQL + PandasMediumHighMediumHigh (needs VPS + connection pool)~¥500
ClickHouseHighHighFastHigh (memory hungry)~¥0 (can’t cover costs)
Excel TemplateLowLowPoorNegligibleNegative (customers don’t value it)
DuckDB + FastAPIVery LowVery LowFastNegligible (10MB file)~¥2,500

Three numbers capture DuckDB’s advantage:

  1. 10MB — The DuckDB binary is a complete database. No installation, no configuration, no DBA needed
  2. 0ms startup — Embedded in-process execution. No waiting for connection pools, no warmup needed
  3. File-level isolation — Each customer gets their own .db file. Natural multi-tenancy. Backup = copy a file

This means you can run dozens of customers on a single 2C4G cloud server (about ¥50-80/month), with per-customer cost approaching zero.


Complete Architecture

Customer uploads data (CSV / Excel / DB export)
 ↓
 ┌──────────────────────────────────────┐
 │ FastAPI Router Layer (API Endpoints)│
 │ - /api/{customer}/upload │
 │ - /api/{customer}/query │
 │ - /api/{customer}/report │
 └──────────────┬───────────────────────┘
 ↓
 ┌──────────────────────────────────────┐
 │ DuckDB Engine Layer │
 │ - Independent .db per customer │
 │ - SQL template asset library │
 │ - Cross-database ATTACH queries │
 └──────────────┬───────────────────────┘
 ↓
 ┌──────────────────────────────────────┐
 │ cron Scheduled Task Layer │
 │ - Daily auto-generated reports │
 │ - Telegram / Email push │
 │ - Anomaly alerts │
 └──────────────────────────────────────┘

Three layers, all in a single Python process. Zero external dependencies — no Redis, no message queues, no separate database server.


Phase 1: Data Ingestion (20 Lines of Code)

Customers upload CSV, Parquet, or Excel files. DuckDB’s read_csv_auto automatically infers the schema. Zero configuration required:

import duckdb
from fastapi import FastAPI, UploadFile, File, HTTPException
from pydantic import BaseModel
import os

app = FastAPI(title="Lightweight Analytics API")
DB_DIR = "./customer_dbs"

def get_conn(customer_id: str):
 """Each customer gets their own DuckDB database file"""
 os.makedirs(DB_DIR, exist_ok=True)
 db_path = f"{DB_DIR}/{customer_id}.db"
 conn = duckdb.connect(db_path)
 conn.execute("INSTALL 'httpfs'; LOAD 'httpfs';")
 conn.execute("INSTALL 'json'; LOAD 'json';")
 conn.execute("INSTALL 'spatial'; LOAD 'spatial';")
 return conn

@app.post("/api/{customer_id}/upload")
async def upload_data(customer_id: str, table_name: str, file: UploadFile = File(...)):
 """
 Customer uploads CSV/Parquet file, table auto-created.
 DuckDB infers column names and types automatically.
 """
 conn = get_conn(customer_id)
 temp_path = f"/tmp/{customer_id}_{file.filename}"

 with open(temp_path, "wb") as f:
 content = await file.read()
 f.write(content)

 try:
 conn.execute(f"""
 CREATE OR REPLACE TABLE {table_name} AS
 SELECT * FROM read_csv_auto('{temp_path}', header=true, all_varchar=false)
 """)
 row_count = conn.execute(f"SELECT count(*) FROM {table_name}").fetchone()[0]
 os.remove(temp_path)
 return {"status": "ok", "table": table_name, "rows": row_count}
 except Exception as e:
 os.remove(temp_path)
 raise HTTPException(status_code=400, detail=str(e))

Why this makes money: Your customers don’t need to know SQL or configure a database. They upload data and instantly get results. While your competitors are still explaining “why we need a schema,” your customers are already looking at their dashboard.


Phase 2: Assetizing SQL Queries (Your Core IP)

Package high-frequency analytical queries as callable API endpoints. These SQL templates ARE your product:

# SQL Template Asset Library — each query is a competitive advantage
SQL_TEMPLATES = {
 "daily_sales": """
 SELECT
 strftime(order_date, '%Y-%m-%d') as date,
 sum(amount) as total_sales,
 count(distinct customer_id) as unique_customers,
 sum(amount) / nullif(count(*), 0) as avg_order_value
 FROM orders
 WHERE order_date >= current_date - interval '30 days'
 GROUP BY strftime(order_date, '%Y-%m-%d')
 ORDER BY date DESC
 """,
 "top_products": """
 SELECT
 product_name,
 sum(quantity) as units_sold,
 sum(amount) as revenue,
 count(distinct customer_id) as buyers
 FROM orders
 WHERE order_date >= current_date - interval '7 days'
 GROUP BY product_name
 ORDER BY revenue DESC
 LIMIT 10
 """,
 "customer_segments": """
 WITH customer_value AS (
 SELECT
 customer_id,
 sum(amount) as lifetime_value,
 count(*) as order_count,
 datediff('day', max(order_date), current_date) as days_since_last
 FROM orders
 GROUP BY customer_id
 )
 SELECT
 CASE
 WHEN lifetime_value > 10000 THEN 'High Value'
 WHEN lifetime_value > 3000 THEN 'Mid Value'
 ELSE 'Low Value'
 END as segment,
 count(*) as customer_count,
 avg(lifetime_value) as avg_lifetime_value,
 avg(days_since_last) as avg_recency
 FROM customer_value
 GROUP BY segment
 ORDER BY avg_lifetime_value DESC
 """,
 "inventory_warning": """
 SELECT
 product_name,
 stock_quantity,
 reorder_point,
 CASE WHEN stock_quantity <= reorder_point THEN '⚠️ Reorder Needed' ELSE 'OK' END as status
 FROM inventory
 WHERE stock_quantity <= reorder_point
 ORDER BY stock_quantity ASC
 """,
 "weekly_trend": """
 SELECT
 date_trunc('week', order_date) as week,
 sum(amount) as weekly_sales,
 count(*) as order_count,
 sum(amount) / nullif(count(*), 0) as avg_order_value
 FROM orders
 WHERE order_date >= current_date - interval '12 weeks'
 GROUP BY week
 ORDER BY week DESC
 """
}

class QueryRequest(BaseModel):
 query_name: str
 params: dict = {}

@app.post("/api/{customer_id}/query")
async def run_query(customer_id: str, req: QueryRequest):
 """Execute a predefined query and return JSON results"""
 if req.query_name not in SQL_TEMPLATES:
 raise HTTPException(status_code=404, detail="Query template not found")

 conn = get_conn(customer_id)
 sql = SQL_TEMPLATES[req.query_name]

 try:
 result = conn.execute(sql).fetchdf()
 return {
 "query": req.query_name,
 "columns": list(result.columns),
 "data": result.values.tolist(),
 "rows": len(result)
 }
 except Exception as e:
 raise HTTPException(status_code=400, detail=str(e))

Each SQL template is optimized for DuckDB’s strengths. The daily_sales query uses DuckDB’s native strftime — 10x faster than converting dates in Python. The customer_segments query uses a CTE for one-pass customer value computation.

You’re not selling “SQL queries” to customers. You’re selling business insights — that’s what they pay for.


Phase 3: Automated Scheduled Reports (The Most Valuable Feature)

Recurring, scheduled value delivery is the core of subscription revenue. DuckDB’s cross-database analysis makes this surprisingly simple:

import httpx
import json

# cron task: scan all customers at 09:00 daily, generate and push reports
async def generate_daily_reports():
 customers = ["Customer_A", "Customer_B", "Customer_C"]

 for cid in customers:
 conn = get_conn(cid)

 # Check if customer has data
 has_data = conn.execute(
 "SELECT count(*) FROM information_schema.tables WHERE table_name='orders'"
 ).fetchone()[0]
 if not has_data:
 continue

 # Single DuckDB query for multi-dimensional KPI aggregation
 report = conn.execute("""
 SELECT
 strftime(current_date, '%Y-%m-%d') as report_date,
 (SELECT sum(amount) FROM orders WHERE order_date = current_date) as today_sales,
 (SELECT sum(amount) FROM orders WHERE order_date = current_date - interval '1 day') as yesterday_sales,
 round(
 (SELECT sum(amount) FROM orders WHERE order_date = current_date)
 / nullif((SELECT sum(amount) FROM orders WHERE order_date = current_date - interval '1 day'), 0) * 100 - 100
 , 1) as mom_change,
 (SELECT count(*) FROM orders WHERE order_date = current_date) as today_orders,
 (SELECT count(DISTINCT customer_id) FROM orders WHERE order_date = current_date) as today_customers
 """).fetchdf()

 # Format push notification
 message = f"""
📊 Daily Report — {cid}
📅 Date: {report['report_date'][0]}
💰 Today's Sales: ¥{report['today_sales'][0]:,.2f}
📉 MoM Change: {report['mom_change'][0]}%
🛒 Orders: {report['today_orders'][0]}
👤 Customers: {report['today_customers'][0]}
 """

 # Push via Telegram Bot
 async with httpx.AsyncClient() as client:
 await client.post(
 f"https://api.telegram.org/bot{TOKEN}/sendMessage",
 data={"chat_id": CHAT_ID, "text": message}
 )

DuckDB’s SQL functions like current_date and interval follow SQL standards — everything you learned in MySQL or PostgreSQL transfers directly.


Comparison with Traditional Solutions

DimensionTraditional BI (Tableau/Power BI)DuckDB + FastAPI Solution
Monthly Costper user¥50-80 per server (unlimited users)
Deployment Time3-7 days2 hours
Customer Learning CurveHigh (needs training)Zero (API-based)
CustomizationLimited (drag-and-drop constraints)Unlimited (any SQL)
Data VolumeModerate100GB+ on a single machine
ScalabilityVendor-dependentCopy .db file = instant backup
Marginal Ops CostPer-user licensingNear zero

The fundamental difference is in cost structure. BI tools price by “per user.” Your solution prices by “per value delivered.” Adding a customer with DuckDB costs you exactly one more .db file on disk — less than 1MB of metadata overhead.


Deployment Guide: From Code to Production

Step 1: Server Setup

# Cloud server (recommended: 2C4G, Ubuntu 22.04)
apt update && apt install -y python3 python3-pip
pip install duckdb fastapi uvicorn httpx pydantic

# Configure systemd for auto-restart
cat > /etc/systemd/system/analytics-api.service << 'EOF'
[Unit]
Description=Lightweight Analytics API
After=network.target

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/home/ubuntu/analytics-api
ExecStart=/usr/bin/python3 -m uvicorn main:app --host 0.0.0.0 --port 8000
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

systemctl enable analytics-api
systemctl start analytics-api

Step 2: Configure Cron Jobs

# Daily report at 9 AM
crontab -e
# Add:
0 9 * * * cd /home/ubuntu/analytics-api && python3 -c "import asyncio; from main import generate_daily_reports; asyncio.run(generate_daily_reports())"

# Weekly report every Monday 9:30 AM
30 9 * * 1 cd /home/ubuntu/analytics-api && python3 generate_weekly.py

Step 3: Security Hardening

# In production, add authentication and rate limiting
from fastapi import Depends, HTTPException, status
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials

security = HTTPBearer()
CUSTOMER_TOKENS = {
 "Customer_A": "token_a_xxxx",
 "Customer_B": "token_b_xxxx",
}

def verify_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
 token = credentials.credentials
 for cid, t in CUSTOMER_TOKENS.items():
 if t == token:
 return cid
 raise HTTPException(status_code=403, detail="Invalid token")

@app.get("/api/my/report")
async def get_report(customer_id: str = Depends(verify_token)):
 # Authenticated customer
 ...

Growth: Every New Feature Is a New Revenue Stream

Once the base version is running, layer on premium features:

Advanced Analytics (+¥100/month)

  • Cohort analysis for retention tracking
  • RFM customer segmentation model
  • Market basket analysis (association rules)

AI Enhancement (+¥200/month)

  • DuckDB + LLM natural language query interface
  • Anomaly detection with automatic alerts
  • Sales forecasting (time series)

Data Integration (+¥150/month)

  • Auto-pull from WeChat Pay / Alipay APIs
  • E-commerce platform integration (Taobao, JD, Pinduoduo)
  • ERP system auto-sync

本文信息

项目内容
DuckDB 版本v1.5.x(部分功能基于 v2.0 Preview)
最后验证2026-09-12
测试环境Linux / x86_64 / 16GB RAM
官方文档DuckDB Documentation
GitHubpengzz9527/duckdb-blog

如发现错误,欢迎通过 GitHub Issue 或邮件 [email protected] 反馈。

📺 Watch video tutorials → Olap Studio YouTube

Subscribe for more DuckDB & AI automation tutorials