Featured image of post Build an Automated Sales Reporting System with DuckDB in 30 Minutes

Build an Automated Sales Reporting System with DuckDB in 30 Minutes

Learn how to build a fully automated sales reporting system using DuckDB + Python. Connect data sources, aggregate data, generate HTML reports, and set up scheduled delivery — all in one afternoon.

DuckDB Sales Reporting Automation Architecture

Why Sales Reports Are a Side-Hustle Goldmine

In the data analytics world, there’s an underrated income stream: automated reporting as a service.

Many small and medium businesses generate sales data daily, but their owners either stare at Excel spreadsheets for hours or hire assistants to manually export and consolidate. You can build them a fully automated system that produces and delivers reports to WeChat, email, or Telegram every morning — and charge 500-2000 RMB per month. Ten clients means 5000-20000 RMB in passive income.

DuckDB is the perfect tool for this project. No database server to install — just pip install duckdb and you’re running. It aggregates millions of rows in seconds and reads CSV, Excel, Parquet, and remote databases natively.

Complete Project Structure

sales-report-system/
├── config.yaml          # Data source and report configuration
├── generate_report.py   # Core report generation script
├── data/
│   └── sales.csv        # Sample sales data
├── output/
│   └── report_2026-08-22.html  # Generated report
└── requirements.txt

Step 1: Prepare Sample Data

Create data/sales.csv with simulated sales records:

date,product,category,region,revenue,quantity,cost
2026-08-01,iPhone 15,Electronics,North,8999,2,12000
2026-08-01,MacBook Pro,Electronics,South,14999,1,18000
2026-08-01,T-shirt,Clothing,East,199,5,3000
2026-08-02,AirPods Pro,Electronics,North,1899,3,4500
2026-08-02,Jeans,Clothing,West,399,2,2000
2026-08-02,Sneakers,Clothing,South,699,4,4000
2026-08-03,iPad Air,Electronics,East,4799,1,5000
2026-08-03,Jacket,Clothing,North,599,3,3500
2026-08-04,Mac mini,Electronics,West,4999,2,6000
2026-08-04,Dress,Clothing,South,349,6,2800

Step 2: Core Report Generation Script

Create generate_report.py — the heart of the system:

import duckdb
import os
from datetime import datetime
from jinja2 import Template

# ─── Configuration ───────────────────────────────────────
DATA_DIR = "data"
OUTPUT_DIR = "output"
os.makedirs(OUTPUT_DIR, exist_ok=True)

# ─── Connect and load data ──────────────────────────────
conn = duckdb.connect(":memory:")

# Auto-infer column types, read CSV
conn.execute(f"""
    CREATE TABLE sales AS
    SELECT * FROM read_csv_auto('{DATA_DIR}/sales.csv')
""")

# ─── Core Analysis Queries ──────────────────────────────
daily_revenue = conn.execute("""
    SELECT 
        date,
        SUM(revenue) AS daily_revenue,
        SUM(quantity) AS total_units,
        ROUND(SUM(revenue - cost), 2) AS profit
    FROM sales
    GROUP BY date
    ORDER BY date
""").fetchall()

# Category dimension analysis
category_analysis = conn.execute("""
    SELECT 
        category,
        COUNT(*) AS order_count,
        SUM(revenue) AS total_revenue,
        ROUND(SUM(revenue - cost), 2) AS profit,
        ROUND(AVG(revenue), 2) AS avg_order_value,
        ROUND(SUM(revenue - cost) * 100.0 / NULLIF(SUM(revenue), 0), 1) AS profit_margin
    FROM sales
    GROUP BY category
    ORDER BY total_revenue DESC
""").fetchall()

# Region dimension analysis
region_analysis = conn.execute("""
    SELECT 
        region,
        COUNT(*) AS orders,
        SUM(revenue) AS revenue,
        ROUND(SUM(revenue - cost), 2) AS profit,
        ROW_NUMBER() OVER (ORDER BY SUM(revenue) DESC) AS rank
    FROM sales
    GROUP BY region
""").fetchall()

# Top 5 best-selling products
top_products = conn.execute("""
    SELECT 
        product,
        SUM(quantity) AS units_sold,
        SUM(revenue) AS revenue,
        ROUND(AVG(revenue / quantity), 2) AS avg_price
    FROM sales
    GROUP BY product
    ORDER BY revenue DESC
    LIMIT 5
""").fetchall()

# Last 7 days trend with moving average
trend_data = conn.execute("""
    SELECT 
        date,
        SUM(revenue) AS daily_revenue,
        ROUND(AVG(SUM(revenue)) OVER (
            ORDER BY date 
            ROWS BETWEEN 6 PRECEDING AND CURRENT ROW
        ), 2) AS moving_avg_7d
    FROM sales
    GROUP BY date
    ORDER BY date
""").fetchall()

# ─── Generate HTML Report ───────────────────────────────
report_date = datetime.now().strftime("%Y-%m-%d")
report_file = f"{OUTPUT_DIR}/report_{report_date}.html"

# ... (HTML template with Jinja2 rendering — see GitHub for full code)

Step 3: Configuration File (config.yaml)

Production systems need flexible data source configuration. Create config.yaml:

data_sources:
  primary:
    type: csv
    path: data/sales.csv
  # Switch to database easily
  # primary:
  #   type: postgres
  #   database: sales_db
  #   host: localhost
  #   port: 5432
  #   user: analyst
  #   password: xxx

report_settings:
  output_format: html
  include_chart: true
  push_to:
    - email: [email protected]
    - telegram: "@sales_bot"

schedule:
  cron: "0 8 * * *"  # Every day at 8 AM

Why DuckDB Is the Core Engine

1. Zero Deployment Cost

Traditional approaches require installing PostgreSQL or MySQL. With DuckDB, you just need pip install duckdb and import duckdb in Python. For small businesses, this eliminates the cost of a database administrator entirely.

2. Unified Query Interface Across Formats

# Same query interface for multiple data sources:
conn.execute("SELECT * FROM read_csv_auto('sales.csv')")           # CSV
conn.execute("SELECT * FROM read_excel('sales.xlsx')")             # Excel
conn.execute("SELECT * FROM 'sales.parquet'")                      # Parquet
conn.execute("SELECT * FROM postgres('host=localhost db=sales')")  # Remote DB

No need to write different reading code for each format.

3. Native SQL Analytics Power

DuckDB’s SQL engine is designed for OLAP (analytical) workloads, not transactions. This means:

  • Columnar storage: Read only the columns you need — 3 columns from a 10GB file in under 1 second
  • Vectorized execution: 5-10x faster than Pandas
  • Built-in window functions: Rolling averages, ranking, cumulative sums — all in pure SQL

4. Zero-Copy Arrow Integration

# DuckDB query results directly convert to Pandas / Polars / PyArrow, zero-copy
df = conn.execute("SELECT * FROM big_table").df()        # → Pandas
df_arrow = conn.execute("SELECT * FROM big_table").arrow()  # → PyArrow
df_pl = pl.from_arrow(conn.execute("SELECT * FROM big_table").arrow())  # → Polars

This lets DuckDB integrate seamlessly into existing Python data stacks.

Monetization Paths: 4 Ways to Earn

Method 1: Custom Report Outsourcing (Fastest)

List your service on Taobao, Xianyu, or猪八戒: “Custom sales report system, delivered in 3 days” — charging 2000-5000 RMB per setup. Your only cost is time; DuckDB is completely free.

Method 2: SaaS Subscription (Recurring Revenue)

Build a web app with FastAPI + DuckDB backend and Streamlit frontend. Charge 299-999 RMB/month per client. Ten clients = 3000-10000 RMB monthly recurring.

Method 3: Data Analysis Training Courses

Turn your project experience into a course teaching “How to Build Automated Reports with DuckDB.” Sell on Bilibili, knowledge platforms, or Udemy at 99-299 RMB per enrollment.

Method 4: Corporate Training

SMEs don’t understand the tech but will pay for results. Build a complete data system for one enterprise and charge 10,000-50,000 RMB. DuckDB’s zero-deployment advantage makes proposals much easier to close.

Advanced: Scheduled Delivery

Use Python’s schedule library or system crontab:

# schedule.py -定时任务
import schedule
import time

def send_report():
    from generate_report import generate_and_save
    report_path = generate_and_save()
    send_email_report(report_path)

schedule.every().day.at("08:00").do(send_report)

while True:
    schedule.run_pending()
    time.sleep(60)
# Or use crontab (lighterweight)
crontab -e
# Add:
# 0 8 * * * cd /path/to/project && python3 schedule.py >> /var/log/sales_report.log 2>&1

Complete Architecture

┌─────────────────────────────────────────────────────┐
│                 Data Source Layer                   │
│  CSV / Excel / Parquet / PostgreSQL / MySQL / S3  │
└──────────────────────┬──────────────────────────────┘
                       │ DuckDB Unified Query Interface
                       ▼
┌─────────────────────────────────────────────────────┐
│             DuckDB Compute Engine                    │
│  • Auto Type Inference  • Columnar Storage           │
│  • Vectorized Execution  • Window Functions          │
│  • CTEs  • UNION BY NAME                            │
└──────────────────────┬──────────────────────────────┘
                       │ .df() / .arrow() / JSON
                       ▼
┌─────────────────────────────────────────────────────┐
│              Python Processing Layer                 │
│  • Jinja2 Template Rendering                         │
│  • Aggregation  • Trend Analysis  • Anomaly Detection│
└──────────────────────┬──────────────────────────────┘
                       │
          ┌────────────┼────────────┐
          ▼            ▼            ▼
    ┌──────────┐  ┌──────────┐  ┌──────────┐
    │ HTML Report│  │ JSON API │  │ Email    │
    └──────────┘  └──────────┘  └──────────┘

Pitfalls to Avoid

  1. File size limits: DuckDB in-memory mode handles up to ~10GB on a single machine. Beyond that, use duckdb.connect('file.duckdb') for persistent mode or upgrade to DuckDB Cloud.

  2. Timezone issues: DuckDB defaults to UTC. If your business needs local time, convert explicitly: SELECT date + INTERVAL '8' HOUR FROM sales.

  3. Type inference bias: read_csv_auto() sometimes infers numeric columns as integers instead of floats. Use SELECT CAST(col AS DOUBLE) FROM read_csv_auto('file.csv') to specify types explicitly.

  4. Concurrent writes: DuckDB doesn’t support multi-writer concurrency. If multiple processes need to write, use Parquet in batches, then have DuckDB read uniformly.


This system goes from zero to production in 30 minutes, but its business value extends far beyond that. Once you’ve delivered for one client, you can rapidly replicate to other industries — e-commerce, retail, logistics, SaaS — all use the same pattern.

📖 The complete deployment guide with full code is published at duckdblab.org, including PostgreSQL connection, Telegram message delivery, and production error handling patterns.

📺 Watch video tutorials → Olap Studio YouTube

Subscribe for more DuckDB & AI automation tutorials

Built with Hugo
Theme Stack designed by Jimmy

⚠️ This site is an independent community project, not affiliated with, endorsed by, or sponsored by the DuckDB Foundation or official DuckDB project.

"DuckDB" is a registered trademark of the DuckDB Foundation. This site uses the name solely for factual description purposes.

All content is for educational and community promotion purposes only and does not constitute any commercial service.