Query Faster, Query Smarter: Our Move to DuckDB and What We Learned

Back to Devoured - July 02, 2026

DEVOURED

Query Faster, Query Smarter: Our Move to DuckDB and What We Learned

Data performance Medium

Arcesium cut query costs and runtimes by 50% by ditching cloud-native engines like Athena and Trino for DuckDB on specialized workloads.

What: After migrating thousands of SQL queries, the team found that DuckDB’s columnar, single-node performance outperformed distributed engines for small-to-medium datasets, requiring careful management of Parquet compaction and schema evolution.

Why it matters: The industry's move toward massive distributed SQL engines often creates unnecessary overhead for medium-scale data; DuckDB proves that simpler, single-node memory-efficient engines are often superior.

Decoder

Trino: A distributed SQL query engine designed to query large data sets across multiple heterogeneous data sources.

Original article

Arcesium migrated thousands of SQL queries from Athena to Trino to DuckDB over 18 months, cutting query costs by ~50% and reducing query runtime by ~50% for small-to-medium workloads. Athena hit account/service limits, while Trino solved scalability but increased resource cost. DuckDB delivered the needed speed with ~40% lower memory footprint. The migration required handling Glue-less schema evolution, Parquet compaction, JSON fallbacks for STRUCT mismatches, and thread parallelism tuning.

Explore all 51 articles in today's digest