Featured image of post Build a Subscription Industry Data Briefing Product with DuckDB: From Zero to $2,800/Month

Build a Subscription Industry Data Briefing Product with DuckDB: From Zero to $2,800/Month

Step-by-step guide to building an automated industry data briefing product using DuckDB. Learn multi-source data integration, SQL-driven report generation, and monetization strategies.

Industry Data Briefing Product Architecture

Difficulty: ⭐⭐⭐ | Estimated Setup Time: 2 hours for prototype, then 2-3 hours/week to maintain


One person, one computer, collecting $3,000-5,000/month in subscriptions from 50-80 paying users. No team, no office, no fundraising needed.

Sounds too good to be true? It’s real. Today we’ll walk through the complete blueprint for building a subscription-based “Industry Data Briefing” product using DuckDB—from data sourcing to automated report generation to multi-channel monetization.

一、What Problem Does This Solve?

Many small business owners, investors, and industry professionals need to track the latest data trends in their field—but lack the time or resources to do it themselves.

Specific scenarios:

  • A restaurant owner wants to know “the growth rate and top brands in the bubble tea category over the past month”
  • A startup founder wants to know “funding data and standout companies in the AI Agent space over the past six months”
  • An investor wants to know “monthly capacity changes and pricing trends in the new energy battery sector”

They’re willing to pay because they don’t have time or ability to do the research themselves. A “weekly updated, data-reliable, insight-first” industry briefing commands $15-70/month in the market.

二、Why DuckDB? The Core Advantage

Before this approach, you might consider pandas for ETL + Excel for output. But DuckDB delivers a qualitative leap:

DimensionPandas + Excel ApproachDuckDB Approach
Data Source AccessDownload then process locallyread_csv_auto/read_json_auto reads remote files directly
Type InferenceManual schema specification requiredAutomatic column type detection
Data ProcessingMixed Python + SQL codePure SQL handles 90% of operations
Execution SpeedMemory-limited, slow on large dataVectorized execution, GB-scale data in seconds
Deployment CostRequires server or local environmentSingle .db file, zero infrastructure
PortabilityEnvironment-dependentRuns anywhere, any time

Key strategy: More data sources = stronger competitive moat. Your competitors might use 1-2 data sources; you use 5-6 with cross-validation for more reliable conclusions.

三、Step 1: Build the Data Processing Pipeline

Assuming your briefing theme is “China New Energy Vehicle Market Weekly Report”, you need to integrate:

  1. China Association of Automobile Manufacturers monthly sales data (CSV)
  2. Brand model pricing data (web scraping)
  3. Charging infrastructure data (government open data)
  4. Raw material lithium price data (financial data API)
import duckdb
from pathlib import Path

DB_PATH = "nev_industry.db"
RAW_DIR = Path("./data/raw")
PROCESS_DIR = Path("./data/processed")

con = duckdb.connect(DB_PATH)

# ── Create tables: truncate and reload on each update ──
con.execute("""
CREATE TABLE IF NOT EXISTS sales_data (
    month VARCHAR,
    brand VARCHAR,
    model VARCHAR,
    sales_volume INTEGER,
    yoy_growth_pct DECIMAL(10,2),
    data_source VARCHAR
)
""")

con.execute("""
CREATE TABLE IF NOT EXISTS charging_stations (
    province VARCHAR,
    city VARCHAR,
    total_stations INTEGER,
    fast_chargers INTEGER,
    quarter VARCHAR
)
""")

con.execute("""
CREATE TABLE IF NOT EXISTS battery_prices (
    date VARCHAR,
    price_per_kwh DECIMAL(10,2),
    change_pct DECIMAL(10,2),
    source VARCHAR
)
""")

print("✅ Data table structure created")

四、Step 2: Multi-Source Data Auto Refresh

When updating data monthly, simply call the corresponding refresh function:

def refresh_sales_data(csv_path):
    """Monthly sales data update"""
    con.execute("TRUNCATE TABLE sales_data")
    con.execute(f"""
        INSERT INTO sales_data
        SELECT * FROM read_csv_auto('{csv_path}', 
            headers=true, 
            autoparse=true,
            sep=','
        )
    """)
    print(f"✅ Sales data updated: {con.execute('SELECT COUNT(*) FROM sales_data').fetchone()[0]} records")

def refresh_charging_data(json_path):
    """Update charging station data"""
    con.execute("TRUNCATE TABLE charging_stations")
    con.execute(f"""
        INSERT INTO charging_stations
        SELECT * FROM read_json_auto('{json_path}')
    """)
    print(f"✅ Charging station data updated")

def refresh_battery_prices(csv_path):
    """Update battery price data"""
    con.execute("TRUNCATE TABLE battery_prices")
    con.execute(f"""
        INSERT INTO battery_prices
        SELECT * FROM read_csv_auto('{csv_path}', 
            headers=true, 
            timezone='Asia/Shanghai'
        )
    """)
    print(f"✅ Battery price data updated")

Key insight: Using TRUNCATE + INSERT achieves idempotent updates—each run reloads from raw data, avoiding the complexity of incremental updates. For daily or weekly briefings with limited data volume, this simple strategy works perfectly.

五、Step 3: SQL-Driven Automated Report Generation

This is the core of the entire system. You need a set of SQL queries that automatically extract key metrics from multi-source data:

def generate_weekly_report():
    """Generate core data for weekly report"""
    
    # ── Metric 1: Top 10 Brands by Sales ──
    top_brands = con.execute("""
        SELECT 
            brand,
            SUM(sales_volume) AS total_sales,
            ROUND(AVG(yoy_growth_pct), 1) AS avg_yoy_growth,
            COUNT(DISTINCT model) AS model_count
        FROM sales_data
        WHERE month = (SELECT MAX(month) FROM sales_data)
        GROUP BY brand
        ORDER BY total_sales DESC
        LIMIT 10
    """).fetchdf()
    
    # ── Metric 2: Price Segment Market Share ──
    price_segment = con.execute("""
        WITH order_data AS (
            SELECT 
                model,
                brand,
                CASE 
                    WHEN avg_price < 10 THEN '<100k'
                    WHEN avg_price < 20 THEN '100k-200k'
                    WHEN avg_price < 30 THEN '200k-300k'
                    ELSE '300k+'
                END AS price_segment,
                SUM(sales_volume) AS sales
            FROM (
                SELECT 
                    s.model,
                    s.brand,
                    s.sales_volume,
                    CASE 
                        WHEN s.brand IN ('BYD Dolphin', 'Wuling Bingo') THEN 8
                        WHEN s.brand IN ('BYD Qin PLUS', 'GAC Aion Y') THEN 12
                        WHEN s.brand IN ('BYD Han', 'XPeng P7') THEN 20
                        ELSE 35
                    END AS avg_price
                FROM sales_data s
                WHERE s.month = (SELECT MAX(month) FROM sales_data)
            ) sub
            GROUP BY model, brand, price_segment
        )
        SELECT 
            price_segment,
            SUM(sales) AS total_sales,
            ROUND(SUM(sales) * 100.0 / NULLIF(SUM(SUM(sales)) OVER (), 0), 1) AS market_share_pct
        FROM order_data
        GROUP BY price_segment
        ORDER BY total_sales DESC
    """).fetchdf()
    
    # ── Metric 3: Charging Infrastructure Coverage ──
    charging_coverage = con.execute("""
        SELECT 
            province,
            SUM(total_stations) AS total_stations,
            SUM(fast_chargers) AS total_fast_chargers,
            ROUND(SUM(fast_chargers) * 100.0 / NULLIF(SUM(total_stations), 0), 2) AS fast_ratio_pct
        FROM charging_stations
        WHERE quarter = (SELECT MAX(quarter) FROM charging_stations)
        GROUP BY province
        ORDER BY total_stations DESC
        LIMIT 10
    """).fetchdf()
    
    # ── Metric 4: Battery Cost Trend (Key Signal for Vehicle Profit) ──
    battery_trend = con.execute("""
        SELECT 
            date,
            price_per_kwh,
            change_pct,
            ROUND(AVG(price_per_kwh) OVER (
                ORDER BY date 
                ROWS BETWEEN 29 PRECEDING AND CURRENT ROW
            ), 2) AS ma_30d
        FROM battery_prices
        ORDER BY date DESC
        LIMIT 14
    """).fetchdf()
    
    return {
        'top_brands': top_brands,
        'price_segment': price_segment,
        'charging_coverage': charging_coverage,
        'battery_trend': battery_trend
    }

Key technical points:

  • Window functions: SUM(SUM(sales)) OVER () calculates total sales for share percentage
  • CTE nesting: Layered CTE processing for clear logic—segment by price first, then aggregate
  • Moving average: ROWS BETWEEN 29 PRECEDING AND CURRENT ROW computes 30-day moving average for trend detection

六、Step 4: Translate Data into “Human Language”

Data is ready. Now translate it into readable insights—that’s what makes your briefing valuable:

def format_report_text(data):
    """Convert data into natural language briefing"""
    import datetime
    lines = []
    lines.append("📊 New Energy Vehicle Industry Weekly | " + datetime.date.today().strftime("%Y-%m-%d"))
    lines.append("")
    
    # ── Headline: This Week's Biggest Story ──
    top_brand = data['top_brands'].iloc[0]
    lines.append(f"🏆 Spotlight: {top_brand['brand']} leads with {int(top_brand['total_sales']):,} units sold, up {top_brand['avg_yoy_growth']}% year-over-year")
    lines.append("")
    
    # ── Segment Insights ──
    lines.append("📈 Price Segment Competition:")
    for _, row in data['price_segment'].iterrows():
        emoji = "🔥" if row['market_share_pct'] > 30 else "📌"
        lines.append(f"  {emoji} {row['price_segment']}: {int(row['total_sales']):,} units ({row['market_share_pct']}% share)")
    lines.append("")
    
    # ── Infrastructure Progress ──
    lines.append("🔋 Charging Infrastructure (Top 3 Provinces):")
    for _, row in data['charging_coverage'].head(3).iterrows():
        lines.append(f"  {row['province']}: {int(row['total_stations']):,} stations, {row['fast_ratio_pct']}% fast chargers")
    lines.append("")
    
    # ── Cost Signals ──
    latest_battery = data['battery_trend'].iloc[0]
    prev_battery = data['battery_trend'].iloc[1] if len(data['battery_trend']) > 1 else latest_battery
    change_note = "📉 Down" if latest_battery['change_pct'] < 0 else "📈 Up"
    lines.append(f"⚡ Battery Cost Signal: Lithium carbonate at {latest_battery['price_per_kwh']} RMB/kWh ({change_note} {abs(latest_battery['change_pct'])}%), 30-day avg {latest_battery['ma_30d']} RMB/kWh")
    lines.append("")
    
    # ── Key Conclusions (Most Valuable Part) ──
    lines.append("💡 Key Takeaways:")
    
    # Auto-generated insight logic
    if top_brand['avg_yoy_growth'] > 20:
        lines.append("  1. Top brands still maintaining high growth, industry concentration increasing")
    if len(data['price_segment']) > 0 and data['price_segment'].iloc[0]['price_segment'] == '100k-200k':
        lines.append("  2. 100k-200k RMB segment remains the main competitive battlefield")
    if latest_battery['change_pct'] < -2:
        lines.append("  3. Rapid battery cost decline may ease price war pressure H2")
    
    lines.append("")
    lines.append("---")
    lines.append("Data Sources: CAAM, MIIT, Brand Announcements, Industry Public Data")
    lines.append("Report auto-generated by DuckDB, Data as of latest reporting period")
    
    return "\n".join(lines)

# Generate and save the report
report_data = generate_weekly_report()
report_text = format_report_text(report_data)

# Save as Markdown
output_path = f"./reports/report_{datetime.date.today().strftime('%Y%m%d')}.md"
with open(output_path, 'w', encoding='utf-8') as f:
    f.write(report_text)

print(f"✅ Report generated: {output_path}")

七、Step 5: Multi-Channel Distribution & Monetization

The report is ready. How do you reach paying users?

Option A: WeChat Official Account / Knowledge Planet Subscription

Auto-distribute weekly to your official account. Free tier: headline + conclusions. Paid tier: full data + deep analysis.

Option B: Paid Newsletter (Substack/小报童)

Launch a paid subscription on Substack or 小报童. Monthly fee $15, includes exclusive data interpretation + downloadable raw data.

Option C: Enterprise Custom Reports

Provide custom briefings for specific enterprises (e.g., a car manufacturer needs deep competitor analysis). Monthly fee $300-700, adjusted to client’s data dimension requirements.

Option D: Data API

Package core metrics as an API for other developers. Pay-per-call pricing, e.g., $0.01/call.

def publish_report(report_text, report_data):
    """Multi-channel distribution"""
    PROCESS_DIR.mkdir(parents=True, exist_ok=True)
    
    # Save raw data (for paying users)
    report_data['top_brands'].to_csv(
        PROCESS_DIR / f"top_brands_{datetime.date.today().strftime('%Y%m%d')}.csv",
        index=False, encoding='utf-8-sig'
    )
    report_data['price_segment'].to_csv(
        PROCESS_DIR / f"price_segment_{datetime.date.today().strftime('%Y%m%d')}.csv",
        index=False, encoding='utf-8-sig'
    )
    
    print("✅ Data files saved for paying users")
    print("📤 Report text ready for distribution channels")

八、Revenue Projection

Taking the “NEV Weekly Briefing” as an example:

TierMonthly FeeTarget UsersMonthly Revenue
Individual$15100$1,500
Enterprise$7010$700
Custom Reports$2803$840
Total$3,040

Cost Structure:

  • DuckDB runtime: Nearly zero (runs locally)
  • Data source costs: Mostly free, some paid sources ~$70/month
  • Time cost: 2-3 hours/week (data update + content review)
  • Server costs: $15/month (storage and distribution)

Net Profit: ~$2,600-3,000/month

九、Comparison with Traditional Approach

StageTraditional (Pandas + Excel)DuckDB Approach
Data AcquisitionManual CSV download → local processingread_csv_auto reads remote URLs directly
Data ProcessingPython ETL codePure SQL for aggregation and computation
Type HandlingManual dtype and schema specificationautoparse automatic inference
Output FormatExcel + manual formattingSQL results directly to Markdown
SchedulingManual run or complex cron scriptsDuckDB + Python one-click generation
DeploymentEnvironment and library dependenciesSingle .db file, zero dependencies

十、Action Items for Tonight

  1. Pick an industry you know well (being an insider in that industry is ideal)
  2. Find 3-5 public data sources (government websites, industry associations, open datasets)
  3. Build a minimal viable version with DuckDB: auto-read from data sources → generate simple briefing
  4. Send to 10 potential users for free trial, collect feedback
  5. Iterate and optimize, then start charging

Remember: The best data product isn’t the one with the most data—it’s the one that helps people make the best decisions. DuckDB lets you focus on “distilling insights” instead of “moving data around.”


📖 A complete industry briefing project template (including data source checklist, SQL templates, and automation scheduling scripts) is available at duckdblab.org. You can directly use the template, swap in your data sources, and launch your own data subscription product quickly.

💡 Want to systematically learn how to build commercializable data products with DuckDB? → duckdblab.org has a complete 0-to-1 tutorial series

📺 Watch video tutorials → Olap Studio YouTube

Subscribe for more DuckDB & AI automation tutorials

Built with Hugo
Theme Stack designed by Jimmy