DuckFlight (GitHub Repo)
DuckFlight extends DuckDB to support PostgreSQL and Arrow Flight SQL protocols directly, enabling live access to data without intermediate services.
Decoder
- Arrow Flight SQL: A framework for high-performance transport of large datasets using the Apache Arrow columnar format.
- PgWire: The wire protocol used by PostgreSQL, which allows external applications to communicate with the database backend.
Original article
DuckFlight DuckDB extension
DuckFlight turns any DuckDB database into a PostgreSQL and Arrow Flight SQL server. Query it with psql, PostgreSQL drivers and BI tools, ADBC Flight SQL clients, or another DuckDB instance through Airport. Every client reads and writes the same live database—there is no second service to run and no data to export or copy.
Install and load
Install the signed extension directly from DuckDB Community Extensions:
install duckflight from community;
load duckflight;
DuckDB downloads the build matching your DuckDB version and platform. DuckFlight currently supports Linux and macOS on amd64 and arm64.
Download the artifact for your platform from the latest release. Release artifacts are not signed by DuckDB, so start the CLI with unsigned extensions enabled:
duckdb -unsigned
Then load the downloaded file by its absolute path:
load '/absolute/path/to/duckflight-v1.5.5-osx_arm64.duckdb_extension';
select * from duckflight_core_status();
Choose the asset matching linux_amd64, linux_arm64, osx_amd64, or osx_arm64. The -unsigned flag weakens DuckDB's extension-signature protection for that process, so use it only with an artifact downloaded from this repository's releases and verify its checksum when moving it through another system.
SQL API
DuckFlight adds server-control table functions and, when the real core initializes, PostgreSQL compatibility macros and catalog views. This guide covers the basics useful to extension users. The public test mock only implements server lifecycle operations; it does not provide the core's SQL compatibility layer.
Start, inspect, and stop servers
Run these in the DuckDB instance that loads the extension. Both protocols use that same database, including an in-memory database. Keep the host process running while clients connect.
load duckflight;
select * from duckflight_core_status();
select * from duckflight_pg_serve('127.0.0.1:5433', '/path/to/duckflight.toml');
select * from duckflight_flight_serve('127.0.0.1:31337', '/path/to/duckflight.toml');
select * from duckflight_servers();
select * from duckflight_stop('pgwire', '127.0.0.1:5433');
select * from duckflight_stop('flight', '127.0.0.1:31337');
| Function | Arguments | Returned columns |
|---|---|---|
duckflight_core_status() |
None | loaded (boolean), abi_version (unsigned bigint), detail (varchar) |
duckflight_pg_serve(address, config_file) |
Two varchar values | protocol, address (varchar) |
duckflight_flight_serve(address, config_file) |
Two varchar values | protocol, address (varchar) |
duckflight_servers() |
None | protocol, address (varchar), one row per listener |
duckflight_stop(protocol, address) |
Two varchar values | status (varchar) |
Start functions return after starting a background listener. Use host:port for the address; port 0 asks the operating system to choose a free port. The returned address is authoritative: use it when connecting or stopping, including when a hostname resolves to an IP address.
Common PostgreSQL compatibility helpers
| Area | Useful signatures | Purpose |
|---|---|---|
| Strings | initcap(s), btrim(s, chars) |
Capitalize space-separated words; trim specified characters |
| SQL quoting | quote_literal(s), quote_ident(s), quote_nullable(s) |
Quote SQL values or identifiers; represent a null as SQL text |
| Numbers | div(a, b), width_bucket(x, minv, maxv, count) |
Truncated division; equal-width numeric buckets |
| Aggregates | every(x) |
Boolean AND across rows |
| Arrays | array_ndims(arr), array_remove(arr, val), array_replace(arr, old, new) |
Inspect dimensions or transform array elements |
| JSON | json_typeof(j), jsonb_array_length(j) |
Inspect a JSON value or array length |
| JSON aggregates | json_agg(x), json_object_agg(k, v) |
Build JSON arrays or objects from rows |
| Date/time | to_char(val, fmt), to_date(val, fmt), to_timestamp(val, fmt) |
Format or parse supported date/time patterns |
DuckFlight session and diagnostic utilities
| Function | Result | Purpose in a PgWire session |
|---|---|---|
duckflight_current_user() |
varchar | User associated with the client session |
duckflight_current_database() |
varchar | Client-facing database name, which can differ from the underlying DuckDB catalog |
duckflight_visible_database() |
varchar | Underlying DuckDB database selected for catalog visibility |
duckflight_session_pid() |
integer | PgWire backend/session identifier, not the host operating-system process ID |
duckflight_runtime_databases() |
Table with datname |
Distinct database names represented in tracked PgWire sessions, not all attached databases |
duckflight_pg_stat_activity() |
Table with session and query metadata | Inspect tracked PgWire sessions, including pid, datname, usename, application_name, state, and query |
duckflight_pg_stat_ssl() |
Table with TLS metadata | Inspect pid and ssl for sessions belonging to the current user |
duckflight_database_oid(name) |
unsigned integer | Generate the database identifier used by the PostgreSQL compatibility catalogs |
Authentication setup
Both extension listeners take the path to one duckflight.toml file:
select * from duckflight_pg_serve('127.0.0.1:5433', '/run/secrets/duckflight.toml');
select * from duckflight_flight_serve('127.0.0.1:31337', '/run/secrets/duckflight.toml');
Secure defaults
Every listener requires at least one configured user, bearer token, or mTLS identity. An empty configuration does not start an anonymous server.
| Client | Authentication on the wire | Required config |
|---|---|---|
| PostgreSQL | PostgreSQL SCRAM-SHA-256 | [users.*] |
| ADBC Flight SQL username/password | Basic authorization during the Flight handshake, then a server-issued bearer | [users.*]; TLS outside loopback |
| Airport or direct bearer ADBC | Bearer on every Flight RPC; no username/password handshake | [tokens.*]; TLS outside loopback |
| mTLS | Verified client certificate mapped by SHA-256 fingerprint | [tls] with client_ca and identities |
Build and test
The repository follows DuckDB's Rust Community Extension template and pins extension-ci-tools as a submodule. Clone with submodules, then run:
git submodule update --init --recursive
make configure
make debug
make test_debug
Security
The bundled core executes native code in the DuckDB process. Release inputs must be immutable and checksum-verified. SQL callers are trusted to manage DuckFlight listeners, just as they are trusted to operate the DuckDB instance. The core rejects non-loopback listeners without TLS and rejects listeners with no authentication method.
License
This extension is licensed under the MIT License. Sidequery also licenses the bundled core binaries under MIT; the core source code remains private and is not covered by that license.
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi