Back to Devoured - November 19, 2026
DEVOURED
How one SurrealQL query scans 10 billion rows a second

How one SurrealQL query scans 10 billion rows a second

Data SurrealDB
SurrealDB Earth claims over 10 billion rows per second scan throughput by parallelizing query execution across 1,000 Cloudflare Workers.
What: The architecture splits a single query into 'morsels' (data chunks) processed by stateless worker functions. Orchestration is managed by a tree of 'Durable Objects' that coordinate state, while raw data is read directly from immutable R2 storage files.
Why it matters: This reveals a design pattern for massive-scale distributed databases that leverage ephemeral serverless compute to bypass the throughput limitations of single-node database executors.
Deep dive
  • Uses a tree-structured fan-out architecture: Executors manage planners, which distribute morsels to Opener Durable Objects, then to Leaf Workers.
  • Leverages Cloudflare R2 for immutable data storage (Runs), allowing any worker to read files without needing to coordinate locks.
  • Implements 'hedging' by assigning slow tasks to additional workers and accepting the first successful response to reduce tail latency.
  • Uses a 'sequencer' to establish a strictly ordered commit history in SQLite-backed Durable Objects.
  • Reads data at a fixed snapshot position to ensure consistency across distributed leaves.
  • Scaling is achieved by adding depth to the tree (more levels of openers) to manage connections, rather than widening a single node.
Decoder
  • Durable Object: A Cloudflare-specific serverless entity that maintains state and allows for consistent coordination across worker functions.
  • Isolate: A lightweight, sandboxed environment for running code, much faster to spin up than a traditional virtual machine or container.
Original article

How one SurrealQL query scans 10 billion rows a second

SurrealDB Earth reaches more than 10 billion rows a second by splitting one SurrealQL query into many small pieces of work, running them on around 1,000 Cloudflare Workers at the same time, and merging the results into one answer that reads the whole database at a single moment. This post explains each step of that path. Earth launched on 3 November 2026, alongside SurrealDB 4.

What is Earth built from?

Earth runs SurrealDB's own code on four building blocks of Cloudflare's platform:

Cloudflare building block What it is What Earth uses it for
Worker A short-lived, stateless function that runs in its own isolate, a lightweight sandbox Leaves, which read data during a scan
Durable Object A single-threaded object with a fixed identity and its own small storage The executor, the sequencer, the gateways and the openers, which hold state that needs one strict order
R2 Cloudflare's object storage, similar to Amazon S3 Runs, the immutable files that hold the database's data
Colo cache The cache in each Cloudflare data centre ("colo" is short for colocation facility) Keeping recently read parts of runs close to the leaves

A Durable Object and an R2 object are unrelated despite the shared word. A Durable Object is a running program with state, and an R2 object is a stored file.

Everything else in this post is SurrealDB: the query engine, the roles below, the way a scan is cut into pieces, and the checks that make the result correct.

What happens when one query fans out?

A single executor receives the query, runs the SurrealDB engine, and decides whether the scan is too large for one isolate. If it is, the executor cuts the scan into morsels, which are key ranges of the table, and sends them down a tree of openers to stateless leaves.

Each role does one job:

Role Runs as Job
Executor Durable Object Runs the engine, holds the transaction, pins the snapshot, plans the scan and merges the results
Gateway Durable Object Gathers commits from many executors into batches and passes them up to the sequencer, so the sequencer never has to talk to every writer
Sequencer Durable Object Gives every commit its place in one order, and holds recent small commits until they are written to R2
Opener Durable Object Passes a group of morsels to leaves and streams their answers back
Leaf Worker Reads one morsel from R2 and streams back rows or partial results
Run R2 object Immutable, sorted data that leaves read directly

The sequencer holds recent commits in its Durable Object's own storage, which Cloudflare builds on SQLite. That storage is only a short-term holding area for commits before they are written to R2 as runs. The SurrealDB engine runs every query, and the data that a scan reads comes from the runs in R2.

How is the snapshot chosen?

The scan reads at the snapshot of the open transaction, and a reader lease keeps that snapshot available throughout the scan. Before it serves a fresh read, the executor asks the sequencer for its latest position and waits until it has applied every commit up to that position.

The executor then fixes a snapshot position and builds each morsel request from three things:

  1. The key range of the morsel.
  2. The run list at that position, which is every run whose first position is at or below it.
  3. The tail, which is the commits accepted after the last write to R2 and up to the snapshot, limited to the morsel's key range.

Because every leaf receives the same position, the same run list and its part of the same tail, each leaf sees the database exactly as it was at one moment. Commits that arrive after the snapshot are invisible to every morsel. The tail travels with the request, so a leaf never has to call back to the executor for it.

The sequencer keeps a history floor that only rises, and it never lets the floor pass the oldest snapshot that an unexpired lease holds. A scan therefore reads the same runs from start to finish, even while compaction and new commits continue.

How do writes scale?

Reads are not the only part of Earth that grows by adding more of the same thing. Writes take a path of their own, and it is built the same way.

A write starts at an executor, which sends its commit to a gateway. A gateway is a Durable Object that serves a group of executors. It collects the commits from them, checks that commits in the same batch do not conflict, and sends the batch up as one unit. Small commits travel with the batch itself. Large commits are written to R2 once, and only a short record of them moves up the tree.

Gateways sit in levels, in the same way that openers sit between the executor and the leaves. Each level merges the batches from the level below, so the sequencer receives a few large, ordered batches instead of a message from every writer. Adding writers adds gateways, and the sequencer's job stays the same size. Every commit still receives its place in one strict order, so the snapshot rules in the section above hold no matter how many writers there are.

Gateways hold no data that the sequencer or R2 does not also hold, and a gateway that has no work to do holds no connections. A quiet database costs almost nothing, and a busy one takes on more gateways as its writers grow.

What do the leaves do?

Each leaf is a Worker that runs in its own isolate. Earth deploys the same leaf code under many script names so that calls spread over many isolates.

A leaf receives a morsel request on POST /morsel, an internal endpoint of Earth's leaf code. It reads its key range from R2 through the colo cache, merges the tail in by position, and streams the results back with running counts of rows and bytes. An aggregate such as a count returns as a partial result, and the executor adds the partial results together.

How wide does the tree get?

An executor cannot manage a thousand connections at once, so openers sit between it and the leaves. A few limits keep each part of the tree to a size that one object can manage:

  • Each node passes work to at most 32 children.
  • By default, the executor sends work through 6 openers.
  • A read keeps at most 64 R2 requests in progress at once.

To reach more leaves, the tree adds levels instead of making any one node wider. Each leaf takes its next morsel when it finishes one, so a scan can have many more morsels than there are leaves.

How do results merge into one answer?

Morsels cover separate key ranges in ascending order, and together they cover the whole scan range. Verified rows can therefore be released in key order without a sort. Counts add up correctly because every key falls in exactly one morsel and is counted once, from a verified stream.

The executor verifies every morsel before it releases a row. The stream must have consecutive sequence numbers starting from zero, running row and byte counts that match what arrived, and a final frame whose totals equal the rows, bytes and frames received. A stream that is missing, incomplete or inconsistent, or an opener that stops answering, fails the check, and the morsel runs again through another opener.

A morsel gets up to 3 attempts, and each attempt may take up to 8,000 ms. A morsel that never verifies fails the whole scan, so Earth never returns a partial answer.

Slow leaves are hedged. Once every morsel has started, a morsel whose leaf is slower than a threshold is also given to an idle leaf, and the first verified answer wins.

Why does the scan scale with the number of leaves?

Durable Objects hold the state that has to stay in one order. The sequencer gives every commit its place in one order, and the executor holds sessions, the transaction and the dispatch logic. Openers are a pool of Durable Objects that limits how many connections the executor manages.

R2 holds all the data that has been written into runs. Runs never change, so neither the executor's cache nor the colo cache ever needs invalidating, and every leaf reads the same bytes without coordinating with any other. Leaves hold no data and no state, so any of them can stop in the middle of a scan, and the verifier turns that into a retry.

No node owns a part of the data, so adding more leaves needs no data to move. That is why scan throughput grows with the number of leaves while the data stays in object storage.

What do the measurements show?

Measure Result
Table size 5 billion rows
Query SELECT count() AS n FROM t WHERE b < 100 GROUP ALL
Peak rate over 100 ms 22.3 billion rows per second
Leaves reading at the peak 22,085
Best single second 4.55 billion rows
Morsels 24,501
Query wall time 14.5 s

The peak is the best 100 ms window of the scan, not an average over the query. The scan starts its morsels together at a set time, so the load arrives as a burst: the work reaches its peak in about a tenth of a second, and the best full second reads 4.55 billion rows. All of these figures are computed from the leaves' own logs.

Where is the limit, and what comes next?

The executor is the next limit. Each morsel costs about 4 ms of the executor's single thread, which caps a scan of small morsels at about 200 to 280 morsels a second. Planning also takes time, because the executor reads run indexes to decide where morsel boundaries go. The next step is to move planning and the carrying of results from the executor into the node levels of the tree, so that the executor does less work for each morsel.

We are also working on these improvements to Earth:

  • Faster cold starts. A cold start currently takes about 300ms, and we are working to bring that down.
  • Database branching. Branching is coming to Earth, where R2 does much of the work.
Explore all 60 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi