You can now send OpenTelemetry (OTel) traces from Claude Managed Agents, the Messages API, and Claude SDKs to any OTel tool, and follow each step your agents take. If you already send Claude Code or Cowork traces to your observability tool, the agents you build on the Claude Platform can also show up in the same place.

Trace agents with OpenTelemetry

Trusting an agent with more access starts with tracing its model calls, tool use, and MCP server calls. Review its traces, then widen its access as it proves reliable.

Add a destination in the Claude Console, and every agent turn, model call, and tool use arrives in your observability tool as a standard OpenTelemetry trace, with latency, tokens, and IDs.

Works with your observability tools

Claude traces follow the OpenTelemetry GenAI semantic conventions, so data from Claude and other providers works in the same tools and dashboards.

• Debug agents next to the rest of your stack. In Grafana Cloud, Honeycomb, Sentry, and Splunk, agent traces sit beside your logs and metrics, so you can see which service a slow or failed step came from.
• Improve agents with production data. With Arize AI, Braintrust, Datadog, Langfuse, LangSmith by LangChain, New Relic, and PostHog, you can turn real traces into evals, catch quality issues, and measure whether a change helped.
• Track usage across your organization. Send traces to Databricks Unity Gateway so platform and security teams can see token usage by team and flag unusual activity.

Claude traces also work with any other OpenTelemetry tool that accepts a standard OTLP endpoint.

Get started

OpenTelemetry traces on the Claude Platform is available in public beta. Managed Agents and the Messages API need a destination in the Claude Console, while Claude SDKs export traces automatically.

See the docs to get started.

“At Foxglove, we’re building agents that help robotics teams debug their robots. Native OpenTelemetry export from the Claude Platform lets us analyze agent interactions in PostHog alongside product usage data. That helps us evaluate how our agents perform in practice, identify where they fall short, and prioritize improvements for robotics teams.”

Roman Shtylman, Co-Founder and CTO

“Claude powers some of Ramp’s most capable internal and customer-facing agents. We built Reflect, an agentic system that scores their Langfuse traces, finds recurring failures, and proposes fixes. On some agents, we’ve already cut token usage by 10–20% and made sessions 30% faster. But Reflect can only improve what it observes. Native OpenTelemetry export from the Claude API can feed high-fidelity instrumentation directly into that improvement loop, with no extra steps.”

David Traina, Senior Software Engineer

“At Sterling Labs, we tune our systems like a race team tunes its engine. For agents analyzing live markets 24/7, Langfuse is our pit wall. Native OpenTelemetry export from the Claude Platform streams every trace straight in with no custom instrumentation, so my team can focus on the edge, not the wiring.”

Hyon Lee, CTO

“Agents that touch a company’s books have to show their work, the same way an accountant does. Our agents run on the Claude Platform and our telemetry lives in Grafana Cloud, and native OpenTelemetry export from Claude puts each model call in the same trace as the reconciliation or journal-entry workflow it ran inside. We see the prompt version, tool calls, latency, and token spend alongside the rest of our stack, so when an auditor asks what an agent did and why, we can answer from one timeline.”

Ankush Sharma, Head of Infrastructure

Anthropic told investors recently that the company's revenue for the third quarter jumped about 45% from the prior quarter to $16.8 billion. The AI lab is currently pursuing what may become the largest-ever initial public offering. Its IPO could value the company at $2 trillion. The San Francisco company is expected to begin trading as early as next month. A successful launch from Anthropic could reopen the door for other listings and provide a vote of confidence for the industry.

pg_vectorsearch

A PostgreSQL extension for high-performance approximate nearest neighbor (ANN) search.

Status: alpha. Meant for testing, not production. Upgrading between prereleases may not be possible; plan to drop and recreate the extension and its indexes.

• PRISM index (100M+ vectors) — IVF-style, with hierarchical routing
• RaBitQ quantization with theoretical error bounds for two-stage search
• SIMD-optimized distance computation (AVX2, AVX512, NEON)
• Dynamic updates — inserts, updates and deletes, with posting-list splits via prism_rebalance
• pgvector compatible — index existing vector and halfvec columns as-is

PRISM Index

PRISM (Partitioned Routing Index for Similarity Matching) is a PostgreSQL-native index access method that is conceptually similar to pgvector's IVFFlat, but it is built around state-of-the-art quantization, routing, and partitioning techniques. It is implemented in C for high performance and seamless integration with PostgreSQL.

The result: an index that is significantly faster than pgvector's IVFFlat and HNSW, for both querying and index building. In fact, compared to graph-based HNSW, PRISM is faster at equivalent recall on a 100-million-vector dataset, on an instance with a quarter of the memory.

To achieve this performance, PRISM uses a hierarchical centroid tree to partition vectors into clusters, and relies on the PostgreSQL buffer cache to keep that tree warm in memory. Vectors are RaBitQ-quantized and stored in posting lists with a page layout optimized for SIMD distance computation. The RaBitQ encoding carries error bounds that let a search rule out vectors that cannot possibly be nearest neighbors, so far fewer of them need reranking against the full-precision vectors stored in the table.

Query Flow:
1. Traverse centroid tree to find nearest clusters (buffer cache)
2. Scan posting lists for candidates (buffer cache, async I/O)
3. RaBitQ two-stage filtering: estimate → error bound → rerank
4. Full-precision reranking from heap
5. Return k nearest neighbors

Usage

Install the extension first.

There are two ways to use a PRISM index. Pick the one that matches where your vectors live:

• Native types. Starting fresh, or not using pgvector? Store vectors in pg_vectorsearch's own vec32 (float32) or vec16 (float16) columns. No other extension is needed.
• Existing pgvector tables. Already have vector or halfvec columns? Install pg_vectorsearch next to pgvector and build a PRISM index on those columns as they are. Your table and queries stay unchanged.

Native types

CREATE EXTENSION pg_vectorsearch;

CREATE TABLE items (
    id serial PRIMARY KEY,
    embedding vec32(3) STORAGE PLAIN
);

INSERT INTO items (embedding)
SELECT ARRAY[i::real, (i % 7)::real, (i % 5)::real]::vec32
FROM generate_series(1, 1000) i;

CREATE INDEX ON items USING prism (embedding vec32_l2_ops);

SET prism.nprobe = 40;

SELECT * FROM items
ORDER BY embedding <-> '[3,1,2]'
LIMIT 5;

Existing pgvector tables

If you already have a table with pgvector's vector or halfvec columns, you can build a PRISM index on it directly. Nothing about the table or your queries needs to change. vec32 and vec16 are binary-compatible with pgvector's vector and halfvec but have different names, so the two extensions can be installed side by side, even in the same schema.

CREATE EXTENSION vector;

CREATE TABLE documents (
    id        bigserial PRIMARY KEY,
    body      text,
    embedding vector(384)
);
CREATE INDEX documents_embedding_hnsw
    ON documents USING hnsw (embedding vector_l2_ops);

CREATE EXTENSION pg_vectorsearch;

CREATE INDEX documents_embedding_prism
    ON documents USING prism (embedding vec32_l2_ops);

Operator classes

Tuning

Defaults are tuned from large-scale benchmarks; most deployments only ever adjust prism.nprobe.

Maintenance

An insert appends to whichever posting list its vector routes to, so lists grow as rows arrive and never split on their own. Besides REINDEX, the prism_rebalance procedure splits oversized lists in place.

Building

Requirements:

• PostgreSQL 18+ (with development headers)
• GNU C11 compiler (GCC 11+ or Clang 14+)
• Meson build system
• Optional: CBLAS, for faster RaBitQ encoding.

# Release build
meson setup builddir --buildtype=release

# Build and install to PostgreSQL
meson install -C builddir

License

The PostgreSQL License

Full article content is not available for inline reading.

Read the original article →

At Anthropic, coding is no longer the bottleneck to building systems or doing research. Claude is able to autonomously iterate on research loops, micro-optimize complex programs, and build new services from the ground up at a speed and scale we haven’t seen before. The bottleneck, now, is all of the developer infrastructure built for human scale: source control, builds, and CI.

In May 2026, Anthropic averaged around 2,500 code changes merged per day; as of September, we merged over 30,000 per day. We don’t expect this rate of growth to slow down anytime soon; the latent demand for merging code is practically infinite, and every bottleneck we remove immediately leads to a further increase in the amount of code we produce.

In order to support this growth, we built Scion, an in-house version control system, and fully migrated our monorepo from git in five months. We also built and shipped a code review surface (called Antique), code search, a rich source control IAM system, novel ways of managing and landing large amounts of reviewed code, and more, all in that same time period. In this post, we’ll focus on how we built Scion specifically. It makes for an interesting case study: Claude’s ability to write code was both what put us in this situation in the first place and how we were able to quickly build a high-quality, scalable system to deal with it.

Growing our monorepo

Before we explore what we built, we should acknowledge how unusual our workload is. Unsurprisingly, Anthropic uses Claude to write software far more than almost any other company, so we’re often the first to see where tools built for human scale start to strain. Source control is just one example of this. While we don’t expect many organizations to have these problems in the very near term, we suspect it won’t be too long before others who use AI coding tools experience these same pressure points, which is why we wanted to share what we built and what we learned.

Git is a decentralized system: every clone is a complete, self-contained repository that can compute diffs, read history and blame, and check out any revision while fully offline. For the vast majority of repositories, this is the right design, and it’s a large part of why git became the standard.

As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB. We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.

Git offers tools for working with a slice of a repository, such as shallow clones, sparse checkouts, and promisor remotes. Beyond these, there are standard workarounds for large monorepos, like internal git mirrors and repository snapshots (seeds) kept in blob storage. We’ve used all of them—at our peak, we operated several hundred internal git mirrors to deal with the load. This bought us time, but they became harder to operate as our merge rate increased. Seeds fell further from the tip of main between refreshes, and it took more and more care to keep operations that depend on history, like diff and blame, working on partial clones.

Taking a step back, we noticed that, like most companies, we were using git as a centralized system with one authoritative copy, but nearly all of our clients were agents and jobs that are online by definition. The properties that make git excellent for distributed, offline work were actively undermining our ability to scale as the repo grew. So we asked ourselves: if we designed source control with the fact that agents are always online in mind from the start, what would it look like?

Principles for agent-scale source control

With agentic coding, many common assumptions about how software is developed are evolving. We embraced these changes when building Scion, which allowed us to take a fundamentally different approach than git on several core principles. Source control for huge monorepos has prior art, notably Google’s Piper and Meta’s Sapling, so we were able to build Scion with their hard-won lessons in mind. Our principles are:

• Clients are always online. Git assumes that clients work offline and need a complete copy of the repository. Since nearly all of our clients are agents and jobs, we designed Scion’s client and protocols around a server that is always reachable, and have clients fetch data when they need it.
• Client-server asymmetry. In git, every client is capable of being a full-fledged server. In Scion, the server answers questions about file history and blame, creates commits, performs merges and rebases, and so on. The client maintains a bounded local object cache, lays out the working copy files on disk, monitors for filesystem changes, and gives users an ergonomic way to work with the server. It can compute very little on its own but, in exchange, it requires much less data locally.
• A comprehensive source control service can power many novel applications. With git, a CI job or tool that needs to ask a question about the repository, such as who last changed a line or how two commits are related, has to maintain a local copy of the repo or hope that vendor-specific APIs provide the answer. With Scion, all of the source control queries the client uses are also exposed via API, so in many cases jobs don’t need a local repository at all. Many internal tools, such as our automated bisect and revert pipelines, wouldn’t be practical at scale without this cheap, programmatic access to source control.
• Everything must be horizontally scalable. At our peak, we ran several hundred git mirrors, which significantly strained capacity and no longer scaled once the repo outgrew the mirrors’ disks or memory limits. In Scion, the repository’s data is distributed, and every component is independently scalable; there is no fully materialized copy of the repo anywhere, even on the server. There is no equivalent of packfiles (git’s compressed bundles of objects), so no repository maintenance is needed.
• Commits must be cheap, incremental, and lock-free. In git, you commit locally and then push to share your work. In Scion, every commit, rebase, or merge happens on the server, so there’s nothing to push, but each of those operations needs to be fast. The cost of creating a commit and updating its downstream data structures has to scale with the size of the change, not the overall size of the repo, or the server can’t keep up. The development lifecycle in Scion discourages refs (pointers to commits, such as branches and tags), except for the main ref or release branches, because global state that every server has to agree on is a looming scaling problem.
• The working copy is optional, and sparse by design. Because our monorepo is so large, and because we check out code tens of millions of times a day for CI, we wanted to avoid checkouts whenever we could. With Scion, a CI job that needs the diff between a commit and its merge base, for example, can ask the server for it. The same goes for writes: many write workloads can also skip the checkout and function entirely via API. Scion’s protocol is centered around sparse checkouts, which only contain the directories the client asks for, and virtualized checkouts, which materialize files on demand, so when a job does need files on disk, the client only pays for what it actually reads.

How Scion works

Data structures

In Scion, the unit of work is a “changeset,” an entry in a write-ahead log. A changeset stores most of what you’d expect from a commit in git. But where a git commit stores a pointer to the tree, a changeset stores a list of the files that changed relative to its first parent, along with content-addressed pointers to the compressed file contents in blob storage. Changesets are created server-side and are available everywhere, instantly.

This structure lets us accept changesets with very high throughput because there are no packfiles to accept, no locking or cross-server coordination, and no refs to move around. Scion does technically support refs, but since neither creating nor fetching commits involves them, moving one is a compare-and-swap on a single Spanner row that happens separately from creating changesets.

A changeset records only what’s changed, so on its own, it can’t quickly answer questions like who last edited a line or which commits affected a file. To answer those, Scion builds additional data structures from the changesets, including dedicated structures for file blame and path history and skip lists for the commit graph. We call everything that isn’t directly part of the changeset “derived data,” and we compute it all through one generic mechanism. Each type of derived data is defined recursively, so its value for a changeset is computed from its values for that changeset’s parents.

This mechanism has three benefits. First, we can easily add a new type of derived data when a new data structure would make some operation faster. Second, for each type, we can choose whether to compute it eagerly or defer the work, so we avoid materializing data we don’t need. And third, because the client never sees the internals of these data structures, we can optimize them without changing the client. We’ve added new types of derived data many times as the system has grown, such as new indices for performing copy detection or path-based file search.

Scaling Scion

A small fleet of Scion servers provides source control APIs for any operation a client needs, such as getting changesets, computing diffs, finding merge bases, and fetching blobs. All source control data is content-addressed, so a piece of data never changes once it’s stored, and any server can cache it without checking with the others. As a result, the servers are horizontally scalable and require little coordination. They are essentially large caches that can also compute derived data on the fly and write the results back to Spanner.

Scion’s many consumers no longer need to maintain local repositories of their own. Fetching and maintaining the repository used to be a significant maintenance burden for CI jobs or tools that needed to ask source control questions, particularly about commit ancestry and blame. Early in the migration, before we had our own client, repo format, or protocols, Scion only answered these “metadata” queries. Fetching file contents and materializing working copies are far more data-intensive operations, and they needed a design of their own.

Scion’s scn client deviates from git’s fetch protocols in several important ways and, as a result, lends itself to a highly scalable architecture that requires significantly less CPU than git servers. Scion’s negotiation doesn’t tell the server what the client already has; instead, it operates exclusively on trees and blobs the client wants. Starting from a root tree, the client checks its local store for missing trees and requests them recursively, then repeats the process for blobs. Every request is for a specific object that never changes, so any cache can answer it, and the architecture closely resembles traditional tiered caches. The process is also efficient and resumable, because there is no single multi-gigabyte packfile response. Sparse and even virtualized checkouts don’t need special support, because the client simply doesn’t ask for trees it doesn’t want.

In the fleet, the client talks over a socket to a daemon, scion-daemon, that stores objects for the whole node. Most of our nodes need access to the monorepo, and jobs on the same node reuse most of the same objects. Each daemon starts from a periodically refreshed snapshot, then talks only to the tree and blob cache servers in its region, called scion-cache. These cache servers stream objects directly from memory, in the disk format that the daemon will consume, so they don’t need to spend time computing the client’s missing objects or performing delta compression. When a cache is missing an object, it asks the authoritative scion-servers, which read from Spanner or cloud storage as needed.

The scion-daemon, scion-cache, and scion-server all respond to identical RPCs for this protocol, so the client doesn’t need to know the cache topology. Any tier could theoretically talk to any other, but we strictly enforce the tiering to isolate failure domains. We want to avoid a situation where a failure in a regional scion-cache overwhelms the upstream scion-servers and brings down the entire system.

Each tier of this system maintains a cache hit rate of over 99% and does not require long-lived state, so it’s easy to scale up if needed. Our Scion infrastructure footprint is now 200x smaller than our previous git mirroring infrastructure in terms of CPU and memory requirements. CI and research jobs check out code two orders of magnitude faster (in seconds rather than minutes), which frees up a sizable amount of capacity across the fleet; CI alone realized a 25% CPU capacity win. Neither clients nor caches need to perform repository maintenance, which simplifies the lifecycle management of our jobs.

What we learned building Scion

We started building Scion around the same time as Claude Mythos Preview and Claude Tag became available internally. Both of these releases simultaneously amplified the need for Scion and meaningfully accelerated how quickly we could build it.

Here’s a particularly striking example. Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.

What we learned building and operating Scion offers some interesting lessons for what building agent-scale software could look like.

• Rapid development with a small team will become more common. A team of seven people built both Scion and Antique, not to mention half a dozen related systems, and fully migrated to them in five months. We did all this while the volume of code merged per day increased by 10x. In the past, developing and deploying a source control system like this would have taken a company years of iterative migration, with many more engineers. We think this sort of fast, nimble migration will become more common in the future.
• Engineers are no less valuable, but their work is changing. All of the engineers who worked on this system would now rate Claude as better at writing code than them. But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place. This was particularly common in our migration.
• We were able to substantially improve optimization and quality. By virtue of Claude’s ability to write immense amounts of code much faster than we could ourselves, Scion is among the most well-tested and well-defended codebases we have ever worked on. Even on our tight timeline, we were able to build hermetic simulations of every non-trivial infrastructure and source control interaction we’ve ever encountered, as well as fuzzing and sophisticated rate limiting.
• We laid the groundwork for new forms of monorepo security. Managing safety, security, and compliance in a massive monorepo is as important as managing the size and scale. We built Scion with these in mind from the start. It supports rich access control on repositories, as well as the ability to apply permissions to individual directories in order to protect the most sensitive code without losing the benefits of a monorepo.
• A new level of operational excellence is possible. Our team uses Claude Tag heavily, which has helped us monitor each component of the system. For example, we created Slack channels where we tasked Claude with being obsessively focused on the health of a single component. We call these Claudes “superfans.” We have a scion-daemon superfan, a scion-cache superfan, an Antique merge queue superfan, and so on.
• We’re completely rethinking our priors. This project prompted us to take a very hard look at other workflows designed for human scale. A proper merge queue, for example, doesn’t make much sense at agent scale. What does CI even mean when hundreds or even thousands of changes can merge while tests are running? We haven’t been able to effectively replicate the industry standard of a global repository lock on the merge queue, so instead, we’ve invested in post-merge and pre-deploy automated testing.

Contributing to the wider ecosystem

We see Scion as just one part of the answer to the scaling concerns that increasingly capable AI agents have raised. We’re sharing these details as an example of how teams can tackle similar challenges, though we don’t expect our solution, or our considerations, to suit every case.

Scion is by no means a suitable replacement for git for general use. The concepts we’ve outlined here really only make sense for truly enormous monorepos, with either tens of thousands of engineers or swarms of agents prolifically writing code. Plus, Anthropic is unique in that our ability to spend tokens for internal use far exceeds what any other business would realistically want to spend. We do this both to address our own software development needs, and to test Claude’s capabilities.

However, we anticipate that this class of problems will become more commonplace over the next six to 12 months, and it would be an inefficient use of engineering resources for every company using git to face similar challenges alone. Given that, we think the most impactful thing we can do is to share our thinking, so that other developers can apply some of the most successful parts of Scion to git without departing from the overall model. A git server doesn’t have to work the same way as a git client does; novel server architectures exist that provide significant scaling improvements.

Acknowledgements

Written by Nick G. and the source control team: S.G., R.K., Dan C., Garrett H., Ben L., Colin M., Joe P. We’re grateful to Vilas J. and Sergiu D. for supporting the source control team on this ambitious journey and Amir O. for early contributions to Antique.

Footnotes

1. Storing changed files relative to the first parent of a merge is a stylistic choice; in practice, we’ve found that people only care about history or blame for commits on the main line in trunk-based development in a monorepo.

AI agent developer Manus raises $500M+ at reported $4B valuation

Manus, the creator of the eponymous artificial intelligence agent, has raised more than $500 million in funding.

The Chinese startup announced the round today. It said the private equity firm Boyu Capital led the investment with contributions from Tencent Holdings Ltd., one of China’s largest tech firms, and several others.

Bloomberg reported last month that the deal was expected to value Manus at $4 billion. That’s about double what Meta Platforms Inc. had reportedly offered for the company last December. The acquisition was scrapped this past April after Chinese regulators blocked it.

Manus launched as Butterfly Effect Co. Ltd. in 2022. The company’s first product was a browser extension that provided access to multiple AI models through a single interface. In 2024, it raised $10 million and refocused on developing the product that would eventually become the Manus agent.

The product launched last March through an invite-only program. Videos of the agent performing multistep tasks on users’ computers quickly went viral, which drew the attention of venture capital firm Benchmark. Manus raised $75 million from the firm in April 2025 and relocated its headquarters to Singapore.

After Meta’s attempted acquisition fell through earlier this year, Manus announced that it would resume operating as a standalone company. The software maker has spent the time since improving its product. Manus 2.0, the first major update to the agent since the company’s relaunch, rolled out a few days before today’s funding announcement.

Manus 2.0 features a new harness, a collection of software modules designed to customize the underlying LLMs’ behavior. It promises to reduce the agent’s token use by 23.2% across some tasks while speeding up processing by more than 28%.

It also ships with a tool that consumers can use to generate clips and video games. A second new feature, Cloud Computer, hosts AI-generated games’ backend code in a managed environment. Cloud Computing also lends itself to running other workloads. It can host websites and perform long-running tasks even when the user’s local machine is offline.

The third addition to Manus, Cue, makes it possible to equip AI agents with resources such as an email account and a digital wallet. Furthermore, users can create a message board for their Cue-powered agents. The agents can use it to collaborate on complex tasks.

Manus’ funding milestone comes a day after Nous Research Inc., a startup with a competing AI agent for consumers, raised $90 million. Meta, for its part, stepped up its in-house AI agent development efforts after the Manus deal fell apart. The effort produced an app called Muse that recently surpassed ChatGPT on the U.S. edition of the App Store.

Introducing Grok Voice 3.0

The world’s most intelligent voice model.

Today, we’re announcing Grok Voice 3.0, the world’s most intelligent speech-to-speech model. It sets a new record for resolving customer support calls, has class-leading latency at under one second, and costs less than other frontier models. Grok Voice powers millions of conversations a day across Tesla and SpaceX, and is available today in the API at the same price as its predecessor.

The Most Accurate Voice Model

In Artificial Analysis’s independent testing, Grok Voice 3.0 ranks #1 on the Speech-to-Speech Index, with a score of 86.9. The Index combines speech reasoning, τ-Voice, and preference and task success from blind head-to-head calls. Grok Voice 3.0 also scores 97% on Full Duplex Bench, a measure of how naturally it handles pauses, turn-taking, and interruptions.

τ-Voice measures how well agents handle support calls with diverse accents, background noise, and interruptions. Grok Voice 3.0 scores 71.6%, comparable with GPT-Live-1 (Astra), but at 40% lower latency, setting a new frontier in agent intelligence versus responsiveness.

Agentic Performance vs. Speed

It's also cheaper to run than comparable models. Artificial Analysis measures what each model costs to complete a task in its Big Bench Audio run, and Grok Voice 3.0 comes in under both GPT-Live-1 (Astra, Medium) and GPT-Realtime-2.1.

On EVA-Bench, ServiceNow’s benchmark for enterprise voice agents, Grok Voice 3.0 achieves the highest accuracy score at 74% and is the best-performing model when jointly scored across accuracy and customer experience.

“Grok Voice 3.0 sets a new Pareto frontier on EVA-Bench, ServiceNow’s open-source benchmark scoring voice agents on accuracy and experience across 200+ realistic enterprise scenarios. It pairs top accuracy with a natural, responsive conversation, setting a new standard for what enterprises can expect from AI on the front line of customer service.”

Nitin Aggarwal, Senior Director of Product Management, Agentic Evals and RAI, ServiceNow

Used by Millions of Customers at Tesla & SpaceX

Grok Voice 3.0 handles millions of conversations a day in Tesla vehicles. Drivers can use Connectors for simple requests like checking their calendar or answering an email, and Grok Bot for bigger tasks like ordering a coffee or booking dinner—all completely hands-free. Grok Voice 3.0 also powers Grok Bot’s voice mode.

Starlink call centers use Grok Voice to answer 20,000 customer calls every day across multiple languages. Roughly 70% are resolved with no human in the loop, and one in four sales calls ends in a purchase before the caller hangs up. A single agent built on Grok Voice 3.0 verifies customers, troubleshoots hardware, issues credits, and ships replacement equipment.

Pricing and Availability

Grok Voice 3.0 is available today in the Voice Agent API at $0.08 / min ($4.80 / hr), the same price as Grok Voice 2.0. Agents using grok-voice-latest have already been switched over automatically, with no changes needed.

Full article content is not available for inline reading.

Read the original article →

Recursive Agent Harnessing for an Open World

A harness is the system around a model that enables it to act as an agent: prompts, tools, memory, skills, orchestration, and more. Harnessing is the process of optimizing that system.

Harness² is a plug-and-play framework for recursive agent harnessing in an open world, where agents encounter tasks unseen during harness design. It connects two levels of recursion while keeping model weights fixed:

• Task-level recursion refines the execution harness for the current task through a contrastive Propose–Probe–Compose step: propose edits, probe their effects on the same task, and compose a refined harness from behavioral contrasts between executions, without ground-truth feedback.
• Harnessing-level recursion updates a persistent improvement harness with adopted harnesses and evidence-supported lessons from each task to guide future refinement.

Across three professional benchmarks and four model–harness configurations, a single Harness² step improves GLM-5 and Gemini 3.5 Flash over their base harnesses, with gains of up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB's partial pass rate. Increasing the task-level recursion budget brings further gains.

The release supports parallel and sequential recursion, OpenCode and Codex executors, and JobBench, LAB, and WorkBuddy-Bench.

Harness components

Harness² exposes eight editable components around the model: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, and plugins. They control context construction, the tool interface, delegation, and behavior inside the tool loop.

Install

Use Linux, Python 3.12, and uv. From the repository root:

uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

The local demo runs both recursion modes with deterministic editors and real Python executions. It requires no model credentials.

For model-backed benchmarks, install Node.js 20+, the Google Cloud CLI, and the benchmark dependencies. LAB needs Pandoc and Poppler; WorkBuddy-Bench needs Docker. Then configure your models:

cp .env.example .env
# Edit .env with your Vertex AI project and model IDs.
gcloud auth application-default login

Run a benchmark

The following example installs LAB and runs one task, then evaluates and reports its result. Model-backed runs incur API costs; start with a small task selection.

harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel \
  --only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel

Sequential recursion runs the same commands with --mode sequential. --k is the task-level recursion budget. Judging and reporting must use the same benchmark, executor, subset, --k, --mode, and seed as the refinement run. Add --dry-run to harness2 to preview task selection without model calls.

Choose one benchmark per command. WorkBuddy also takes --subset office, --subset web, or --subset code on every command after setup.

Experiments

Scale task-level recursion in two ways: parallel recursion explores multiple candidate harnesses and composes refinements from their execution evidence; sequential recursion carries the refined harness and accumulated evidence through successive Propose–Probe–Compose steps.

Parallel recursion explores wider; sequential recursion refines deeper.

After each task, harnessing-level recursion retains the adopted harness and evidence-supported lessons to guide future refinement.

Web access

Executor network access is unrestricted, and Codex runs with its sandbox disabled. Public benchmark rubrics may be accessible online. Before evaluation, configure network restrictions to prevent access to benchmark answers and grading criteria.

Results and resume

Results are saved under runs/, organized by benchmark and subset where applicable. They include task summaries, deliverables, adopted harnesses, reflection notes, and evaluation results.

Repeat the same command to resume. Use a new HARNESS2_RUNS_DIR when changing models or task selection.

Citation

@article{xu2026harness2,
  title   = {Harness$^2$: Recursive Agent Harnessing for an Open World},
  author  = {Xu, Ruiyao and Chen, Yanfei and CuiZhu, Zhongying and Dalvi Mishra, Bhavana and Ming, Yifei and Yu, Han and Han, Rujun and Lee, Chen-Yu and Pfister, Tomas},
  journal = {arXiv preprint arXiv:2610.14224},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.14224}
}

Analyzing technical documents with Deep Review

Following successful deployments across 16,000+ submissions at STOC, ICML, NeurIPS, and ICLR, Deep Review is launching publicly. It’s expanding from deep mathematical and scientific peer review to provide rigorous, grounded verification across knowledge documents, including Google Docs and presentations.

Generative AI has accelerated many technical fields. It is now common for AI to play some role in writing scientific papers and technical documents, running code, designing experiments, and drafting proposals. However, accelerating content creation has created a review challenge: the human capacity for verification has remained largely fixed amid this rapid increase in AI-generated content. Reviewing complex documents like long scientific manuscripts or complex legal agreements requires specialized expertise and advanced subject knowledge. This imbalance creates a validation bottleneck, where information is being generated faster than it can be rigorously reviewed.

Today, we’re excited to announce the public release of Deep Review, an agentic AI system engineered to provide rigorous feedback and technical verification on complex documents. Initially designed for computer science academic paper analysis, the system’s capabilities have expanded to review documents across many domains, such as legal and finance. It can be used as both an iterative improvement tool during the drafting process and a tool for the scientific process verification, claims, and evaluation during the review process.

Deep Review represents the evolution of our Paper Assistant Tool (PAT) — an experimental initiative developed to help researchers stress-test manuscripts prior to conference submission. Over the past year, we partnered with four flagship computer science conferences (STOC 2026, ICML 2026, NeurIPS 2026, and ICLR 2027) to provide private, pre-submission feedback to authors of more than 16,000 manuscripts who opted in. Further details on PAT and these pilots can be found in our manuscript, Towards Automating Scientific Review with Google’s Paper Assistant Tool.

With this release, Deep Review is now accessible worldwide:

• Tiered access and scalable API: Every user receives two complimentary document reviews per week. Users with high-volume workflows can connect their Gemini API key for higher usage limits and programmatic integration via the Deep Review API.
• Format flexibility: Deep Review will initially support PDFs and plain text as input formats, with support for multi-file inputs coming soon.
• Privacy and data policy: Documents uploaded to Deep Review are never used to train, fine-tune, or improve Google models. Uploaded documents are stored for one year for the sole purpose of display to the user in the Deep Review UI.

Deep Review accelerates technical validation

One of the most significant examples of the validation bottleneck is in academic peer review. This can be seen in the uptick of submissions to academic publication venues, especially venues for AI research itself, which have seen a dramatic rise. This year, there were over 74,000 submissions to three major AI conferences (ICLR, NeurIPS, and ICML) up from approximately 45,000 in 2025, 32,000 in 2024, and 23,000 in 2023. Each of these submissions requires multiple high-quality reviews to ensure the integrity of the peer review process, which can place a larger burden on human experts.

Deep Review aims to help alleviate this expanding burden on human reviewers across technical fields. By providing detailed, objective evaluations of documents, Deep Review checks logical consistency and mathematical statements, suggests improvements, grounds facts, and compares with existing work. It addresses some of the most cognitively burdensome facets of review, while leaving higher-level judgements to humans.

A proven track record for scientific review

Deep Review has been extensively tested and deployed under its experimental name, the Paper Assistant Tool (PAT), in four pilot partnerships with the flagship computer science conferences STOC 2026, ICML 2026, NeurIPS 2026, and ICLR 2027. During these pilots, authors were able to opt-in to receive feedback and analysis on their paper via PAT in the weeks before the submission deadline. The goal was to provide authors with actionable feedback to help improve the quality of papers in submission. This feedback was completely decoupled from the official conference review process. PAT did not collect any of the data for training, and papers were deleted from Google systems promptly after each program concluded.

Author reactions to the tool have been extremely positive, and reflect continued improvements to both underlying models and the PAT harness. With the addition of Gemini 4 Argon support in Deep Review, the underlying capabilities of the tool have further increased beyond the older iterations of PAT described below.

How Deep Review works

Single-prompt LLM calls often fail when evaluating long, complex documents due to exhausting their token reasoning budget, overlooking long-context cross-section dependencies, or accepting nuanced claims without fully scrutinizing them. Deep Review solves these challenges through a four-stage agentic pipeline:

1. Document segmentation: The document is first ingested and autonomously decomposed by a segmenter agent into logical segments (e.g., Introduction, experiments, methodology, etc.). These segments can be overlapping, and must cover all the content of the input document.
2. Adaptive budgeting: Not all sections require equal cognitive depth. The pipeline inspects information density and dynamically allocates a compute budget, along with a specialized inference scaling pipeline for each section. For instance, sections with dense mathematical proofs may receive more compute than an introduction or a related work section in an academic paper.
3. Deep parallel verification: Specialized reasoning agents scrutinize each segment in parallel. Each agent has access to the full document context to track cross-references, but focuses its reasoning power specifically on the task of reviewing its specific target section. This ensures both that reviewing tasks are manageable for each review swarm, and that parallel swarms are properly coordinated.
4. Global synthesis and agentic grounding: A dedicated synthesis agent consolidates the segment-level reports, and does final consistency checks between sections, rerunning logical checks if required. The agent further performs external grounding checks and tool calls as a final preventative measure against hallucinations.

Universal verification and review: Beyond research papers

While Deep Review originated in academic paper reviews, long-context review is generalizable: reasoning mechanisms used to check whether a mathematical proof holds up in a research paper can also help surface logical gaps and inconsistencies in other types of documents, where claims must be backed by evidence. Therefore, Deep Review can be used to bring structured critique to a broader range of everyday workflows.

Users can now apply Deep Review to analyze complex documents across many domains. For instance, it can be run on:

• Design documents: Catch unstated architectural assumptions, spot edge cases, and suggest concrete improvements.
• Presentations: Ensure slide transitions flow logically and bulleted takeaways match visual charts.
• Legal and financial documents: Check internal consistency across clauses and data tables, flag ambiguous terms or downstream consequences, and ground claims against supporting evidence.

Deep Review meets Antigravity

We’re also introducing the Teamwork Deep Document Review Pattern, a document review implementation pattern inspired by Deep Review built into Antigravity using the Teamwork multi-agent orchestration framework. This pattern is designed systematically to evaluate research papers, manuscript drafts, and technical proposals using a multi-agent hierarchy with self-correction and segment-based rubrics. Antigravity users can invoke this pattern using the /teamwork-preview command.

Making thorough, high-quality technical reviews widely accessible

Comprehensive pre-submission critique in computer science has generally depended on access to extensive review networks — like senior faculty, well-resourced academic laboratories, or extensive peer advisory networks. Providing reliable, automated logical verification can help many academics stress-test their assertions, identify avoidable flaws, refine and strengthen their documents based on verified foundations, and present their work in a review process with more confidence.

Get started

We invite researchers, engineers, and creators everywhere to explore Deep Review:

• Try Deep Review: Visit the Google Research Portal to review your first documents.
• Integrate with the API: Access programmatic verification in your continuous integration and drafting workflows using your Gemini API key.
• Read the research: Read the preprint to explore the underlying methodology and pilot evaluations.

Conclusion and looking ahead

Given the utility it demonstrated during our pilot programs with academic conferences, we believe that tools like Deep Review will play a significant role across the peer review process in the coming years by empowering both authors and reviewers.

Moving forward, we are continuing to expand the platform’s capabilities. With the rollout of Gemini 4 Argon support, Deep Review’s underlying capabilities further improve, delivering even higher quality feedback and analysis to authors. Additionally, with multi-file support coming soon, users will also be able to run these same consistency checks across entire sets of related documents, spreadsheets, and exhibits simultaneously. We hope these advancements will make high-quality technical reviews more widely accessible for knowledge workers everywhere.

Acknowledgments

Deep Review is a collaborative effort across Google Research, Google DeepMind, and Google Cloud. We thank our core contributors and research collaborators — Santiago Balseiro, Taylan Cemgil, Tommaso D'Orsi, Kirk Draheim, Lindi Emoungu, Ayse Erduran, Palash Goyal, Caroline Hartnell, Shobana Kannan, Amy Keeling, Siyi Kou, Bridget Lee, Ian Li, Anoushka Mazumdar, Taylor Montgomery, Mouad Naciri, Uche Okonkwo, Mihir Parmar, Anthony Phalen, Rory Pilgrim, Sarah Presotto Post, Rob Tierney, Drew Tyler, Vishnu V., Vahab Mirrokni, Laura Vardoulakis, Johannes Welbl, David Woodruff, Yunhan Xu, Jinsung Yoon — as well as our executive sponsors Katherine Chou, Corinna Cortes, Burak Gokturk, Vahab Mirrokni, Tomas Pfister, Yossi Matias and the Program Chairs of STOC, ICML, NeurIPS, and ICLR for their partnership.

Partners

Every few months, frontier models unlock harder problems and new domains. No company can afford to sit that out, but few can afford frontier prices on every request. Which model should handle each piece of work, and when does the performance of frontier models justify their cost?

At Vals, we evaluate models from across the major labs on their ability to carry out complex work across domains such as finance, software engineering, law, and tax. The Vals Index brings these evaluations together to measure AI’s potential economic impact. From that vantage point, the frontier is jagged: five different models hold the top spots on the Index’s eight benchmarks, and no model ranks in the top three on all of them. Choosing only one model means compromising performance on some tasks.

On real work, the frontier is jagged

The five models that hold the top spots on the Index’s eight benchmarks, and their rank on each, out of 48 models.

Claude Sonnet 5.5 and Claude Opus 5.5 rank second and third overall but 35th and 33rd on Harvey’s Legal Agent Benchmark, which Muse Spark 1.2 tops from 33rd overall. Vals Index v2.1, published scores.

The range of models to choose from is continuously and rapidly expanding. This past month alone, ten developers released 16 language models tracked by Vals. Each release raises another set of decisions for companies: which workloads would benefit, whether the improvement is worth the cost, and whether it should replace a model already in production. Sticking with one model can mean missing better options; maintaining separate model choices across many workflows is an operational burden requiring continuous engineering and evaluation effort.

Routers have emerged as a new primitive of the AI stack, promising to manage that complexity and help companies benefit from new models without revisiting model selection across every application themselves. The value is both better use of today’s models and a way to keep benefiting from progress across labs.

Every router shares the same fundamental design. It sits in front of a pool of models and, for each request, decides which one should answer, right-sizing the intelligence to the task by weighing quality, cost, and latency. Where routers differ is in how they decide. Some judge the prompt itself, such as the type of task or how hard it looks. Others draw on external signals like benchmark scores or usage statistics. Increasingly, routers make this call at every step of an agent’s work rather than once per task.

An ensemble of models, used well, can beat any single model on the frontier. To measure by how much, we calculated the performance of an oracle router, one that picks the perfect model for every task in hindsight. On the Vals Index, that oracle would beat the best single model by 14 points while costing about 40% less.

Perfect hindsight is impossible, of course, but it shows the full potential of choosing the right model for each task. The real question is how much of that potential today’s routers capture in practice, and existing benchmarks weren’t built to answer it. Most academic router benchmarks score a single choice on a single public question, often from datasets frontier models have long since saturated. When every model in the pool gets the answer right, a router that always picks the cheapest one looks perfect. But routers today are asked to do something much harder, choosing a model at every step of agent work that can run for hours.

Router providers have begun publishing serious evaluations of their own, and we think the natural next step is an independent comparison, with every router run through the same harness on private, long-horizon tasks across real professional domains. Vals has always set out to chart the frontier of AI performance, which is why we’re now measuring how today’s routers compare with single models on quality and cost, with the task-level oracle as a reference point.

From isolated prompts to agent workflows

Academic benchmarks have established a foundation for evaluating model routers. RouterBench (Hu et al., 2024) helped establish an early framework using precomputed model outcomes. RouterArena (Lu et al., 2025) broadened comparisons across domains and difficulty levels, while LLMRouterBench (Li et al., 2026) brought routing methods into a shared framework that includes tool-use and software tasks. More recently, TwinRouterBench (Yang et al., 2026) introduced live coding-agent evaluation, selecting a model at every call and measuring task success and actual API spend.

Today’s agents complete complex work over dozens of steps, where each routing decision affects the context and choices that follow. A cheaper model may save money on one call but introduce an error that takes several more calls to recover from, while a weak routing decision can derail the rest of the task. Predicting how a task will decompose into subtasks is another challenge, and any decision to switch models must account for the cost of losing the existing prompt cache. For long-running sessions, switching to a cheaper model does not automatically bring cost savings.

Isolated prompts miss these consequences, and TwinRouterBench’s live evaluation remains focused on coding. Difficulty matters too: Stanford’s 2025 AI Index identifies benchmarks such as MMLU and GSM8K as saturated, making them less revealing of the accuracy sacrificed for cheaper models. Newer benchmarks address parts of this gap, but public tasks still carry a risk of prior exposure.

The next step is to bring these requirements together: challenging, realistic work across professional domains, live execution that exposes the consequences of each routing decision, and private test data to reduce contamination risk.

What router providers are already measuring

Router providers are also evaluating how their systems perform in practice. OpenRouter compares its own and third-party routers against single models on quality, time, and cost.

The way an agent session unfolds affects its cost. Fireworks evaluated FireRouter through randomized A/B tests on live sessions, accounting for the cost of lost prompt caching when switching models. Not Diamond’s coding-agent evaluation methodology uses simulated users to model long sessions and pauses between user interactions, capturing the costs of growing context and cache expiration.

Benchmarks also shape routing decisions themselves. Ramp’s Benchmark Routing strategy lets customers select and weight benchmarks, then routes requests to the model with the highest weighted score. The relevance of those benchmarks to the customer’s work therefore matters to the routing decision.

These efforts provide useful evidence about router performance. An independent comparison can build on them by running router products through the same harness on shared, private tasks across professional domains, with each product’s configuration documented.

Evaluating routers on the Vals Index

The Vals Index was built to measure the impact of AI on real-world agentic tasks—the very things routers are now aiming to optimize.

The Vals Index is agentic and long-horizon. All eight components evaluate tool-using agents spanning finance, coding, legal, and tax work: Finance Agent v2, the Excel Modeling Benchmark, Terminal-Bench 4.0, Vibe Code Bench, Code Migration, Legal Research Bench, Harvey’s Legal Agent Benchmark (HLAB), and Tax Agent Bench. Many of these tasks are long: the median Terminal-Bench 4.0 task is estimated at four hours of expert work, each Excel Modeling task takes an expert five hours or more, and on Legal Research Bench, some models work for upwards of an hour on certain tasks.

The Index is expert-written and mostly private. Six of its eight benchmarks are private, and their tasks, answers, and grading rubrics are written by people who do this work professionally: financial analysts for the finance benchmarks, practicing lawyers for Legal Research Bench, and tax professionals for Tax Agent Bench. Coding tasks are graded by running tests against what the agent actually built. Our private test set has never been released, and our evaluations are run with ZDR (Zero Data Retention) policies in place, meaning that model providers cannot retain or train on our prompts.

Importantly, no single model leads every component. The model that tops Harvey’s Legal Agent Benchmark ranks 40th on Terminal-Bench 4.0, different models lead other components, and of the top two models, the second costs about 36% more per task than the first. This kind of spread is the opportunity routing can exploit.

Finally, the Index is continuously updated. We regularly evaluate frontier models on it with consistent harnesses, and retire components as they saturate. This means every router can be compared against single-model baselines run in the same harness, along with an oracle that keeps pace with the models being routed between.

Choosing a model to answer a trivia question is a very different problem from choosing a model for every step of a corporate tax research task, and we think router evaluations should reflect that. Concretely, good router evaluations put each router through live, end-to-end agent runs. The router takes the place of a model in the same harnesses and is consulted on every model call, allowing it to retain the current model or select another as the task unfolds. Prompts, tools, and graders stay fixed. Each router runs as shipped, with its default model pool and settings, and that configuration is documented for every run. Task outcomes, cost per task, cache usage, and completion time are compared against baselines run in the same harness (the best and the cheapest single model in the router’s own pool), with the task-level oracle as a reference point.

Beyond picking a single model

If routers are judged primarily on isolated public questions, evaluations will reward a narrower set of capabilities than production agentic work requires. As routing increasingly determines what AI work costs, we believe it deserves to be measured on that work, with the same care as the models themselves.

That is what our router evaluation is designed to do, and we will publish results on a public leaderboard soon. If you build a router and are interested in having it evaluated, get in touch with us here.

HiRO-ACEv2: Simulating extreme weather at 3 km resolution for a fraction of the cost

We introduce HiRO-ACEv2, the newest update to our High Resolution Output Ai2 Climate Emulator (HiRO-ACE) system. HiROv2 can reproduce even extremely rare rain, wind, and pressure events with high accuracy, while the full system can generate global 3 km climate data at about 1/300th the cost of X-SHiELD, a high-resolution global atmospheric model that explicitly simulates storms and other small-scale weather.

HiRO-ACE, introduced earlier in 2026, has two parts: ACE, an AI model that operates at a coarser 100 km resolution, and HiRO, a model that sharpens those coarse outputs into fine 3 km detail. The biggest change in HiRO-ACEv2 is the new downscaling model, HiROv2, which can now 1) predict several high-resolution fields at once instead of just rainfall, so it can capture a much wider range of extreme weather, and 2) has improved skill in predicting extreme events.

Our main focus with HiRO-ACEv2 was to simulate extreme events well. Changes to how we train HiROv2 improved its ability to reproduce the most intense rain, wind, and pressure values. When given coarsened X-SHiELD data, it matches the target simulation's high-resolution statistics to within 3%, even for events so rare they occur only once in a million data points. It also accurately estimates the total energy of tropical cyclones, including storms it never saw during training, and generates hurricanes with realistic eyes and rain bands, though winds and pressure around the eye remain somewhat more lopsided than in X-SHiELD. Additionally, HiROv2 realistically generates strong winds flowing down mountain slopes.

We also trained an updated version of ACE for this system. Over a 10-year simulation, its hurricanes, average climate patterns, and overall distributions of rain, wind, and pressure all line up well with X-SHiELD. That tells us ACE's coarse simulations are a solid foundation for producing long-term high-resolution climate data.

Putting HiROv2 and ACE together, we generated a full year of global 3 km data. Its statistics closely match X-SHiELD's, even though HiRO was working from a simulation ACE ran on its own rather than from real X-SHiELD data. Across a 10-year run, the energy of HiRO-ACEv2's hurricanes is comparable to X-SHiELD's over the same period, and snapshots from that run show a variety of other extreme events that resemble those in X-SHiELD.

Overall, HiRO-ACEv2 offers researchers a flexible and inexpensive way to generate many versions of high-resolution rain, wind, and pressure data for extreme events over 10 years or more. Producing global high-resolution data with it costs roughly 300 times less than running X-SHiELD. If only one region is needed, the savings can be more than a hundred times greater, depending on the size of the area. We're also working on a streamlined version of HiROv2 that could make it more than ten times faster still.

We still have future work ahead of us: because HiRO learned from X-SHiELD, it only works with the version of ACE tuned to match X-SHiELD. That means it can't yet produce high-resolution data for other climates, such as a warmer future or the historical record in datasets like ERA5. Next, we plan to train on new X-SHiELD datasets, including one simulating a world 4°C warmer, so the system can handle different climates. We also plan to add outputs like near-surface temperature and snow and ice, which would allow for even more uses, such as snowpack prediction.

Visualizing and using HiRO outputs

To make it easy to see what HiRO produces without running any inference, we built a web demo that shows pre-generated precipitation, wind, and sea-level pressure output from HiROv1 and HiROv2 side by side with X-SHiELD, the simulation they were trained on. The demo complements the quantitative comparisons in our tech report: it offers a quick visual check of how the outputs look, and lets users zoom in on their own region of interest to judge performance there.

For more in-depth analysis, the models are available to download, and our tutorial notebooks walk through how to run them.

Join us

At Ai2 we’re building the future of transparent, open-source AI — built in the open to empower scientific progress and fundamental understanding of this world changing technology. We’re not here to make profits, we’re here to make sure benefits of AI are shared widely and for the benefit of humanity. If this appeals to you, please take a look at our open roles.

vLLM is an open-source inference and serving engine for large language models (LLMs). LLM inference has two distinct phases: prefill, which processes the input prompt and builds the key-value (KV) cache, and decode, which uses that cache to generate output one token at a time. These phases have different compute, memory, and scaling characteristics. Prefill-decode (P-D) disaggregation runs them on separate engine instances, allowing each phase to scale and be optimized independently. This can improve resource utilization, throughput, and latency under load. However, it also requires transferring the KV cache from the prefill instance to the decode instance before token generation can begin, making efficient KV transfer critical to time-to-first-token (TTFT).

One of the ways vLLM performs this transfer is through the NVIDIA Inference Xfer Library (NIXL) connector, which moves KV blocks over RDMA-capable interconnects such as AWS Elastic Fabric Adapter (AWS EFA, the low latency RDMA network used across servers in AWS). On supported instance types, AWS EFA supports GPUDirect RDMA and Neuron Direct RDMA, enabling NIXL to transfer KV blocks directly between GPU or Trainium device memory without staging through host memory. The default mode for NIXL connector is pull-based: After the prefill instance finishes computing the prompt KV cache, the router relays the source metadata, including the prefill block IDs, to the decode instance. The decode instance then allocates destination blocks, prepares the NIXL transfer, and initiates NIXL READ to pull the KV blocks from the prefill instance. Because these coordination and setup steps occur sequentially after prefill and before decoding can begin, they extend the TTFT critical path.

This post introduces a push-based NIXL connector that moves post-prefill transfer coordination and setup off the critical path. For customers running disaggregated prefill/decode inference with vLLM, this can improve application responsiveness by reducing TTFT and TPOT, particularly at high request rates.

In our benchmark on a 1P1D deployment using Amazon EC2 P5en instances connected with EFA, push mode delivered up to 3x better TTFT and up to 30% better time-per-output-token (TPOT) compared to pull mode for Qwen3-32B model; results depend on workload and configuration.

We built and validated the feature on AWS Trainium instance clusters connected with EFA, and then we contributed it to vLLM as an accelerator-agnostic feature. The results here are from an AWS p5en (GPU) cluster with EFA, showing the gains carry over to GPUs. This post explains the push-based architecture, transfer flow, threading model, performance results, and how to reproduce the benchmark.

Push-based KV transfer

Architecture overview

Push-based transfer improves TTFT and decode throughput for two reasons:

1. It removes the serialization of the pull model by overlapping producer and consumer work. The decode instance registers its target blocks in advance while the producer is still computing KV. The transfer then begins as soon as the producer's blocks are ready, rather than after a consumer-side identifier-discovery round trip, removing that round trip from the path to first token.
2. It moves the transfer-setup time off the decode instance. That setup — preparing the NIXL descriptors and submitting the transfer — runs on the prefill side and on a background thread, so the decode instance does not spend those cycles during its latency-critical token-generation phase.

The net effect is lower TTFT and higher output-token throughput on the decode instance.

The push connector coordinates three actors: the decode instance, which registers its target blocks up front; the prefill instance, which matches each request and issues the transfer; and a dedicated background thread that carries the KV write off the engine's forward-pass loop.

Implementation details

The protocol has three logical steps.

1. Registration (decode side). When the decode instance allocates local blocks for a request requiring remote prefilling, it prepares a registration describing where the prefill instance should write the KV: its own identity and the block locations it has just reserved. Within the same engine step, the worker transmits this registration to the prefill instance as a NIXL notification. Because registration happens at allocation time, the connector knows the destination blocks before the prefill instance completes prefill.

2. Rendezvous (prefill side). The registration from the decoder and the completion of prefill on the producer can arrive in either order, and the connector handles both. If the registration arrives first, the producer holds it until its own prefill finishes. If prefill finishes first, the producer holds the request's blocks until the registration arrives. Whichever event happens second triggers the transfer. The connector pairs requests by identifier, and the matching accounts for reissued requests, correctly associating preempted or retried requests.

3. Transfer (prefill side). Once a request is matched, the producer establishes a connection to the decode instance (if one does not already exist) and issues an NIXL WRITE that copies the KV blocks directly into the decoder's registered memory, addressing each destination tensor-parallel rank. The producer and consumer may use different internal block layouts. Each side maps block locations to its own physical layout at transfer time, allowing different block sizes on each side. Each WRITE carries a completion notification that NIXL delivers to the decode instance once the data arrives. The decoder counts these notifications, one per producer rank that writes into it, and treats the KV as received only after all expected notifications have arrived, at which point the request proceeds to decode.

Threading model

The push connector separates model execution from KV-transfer preparation by using a dedicated background transfer thread on each prefill worker. At the start of a request, the decode main loop allocates its destination KV blocks and registers them with the prefill instance while the prefill main loop computes the prompt KV cache. After both the destination registration and prompt KV are ready, the prefill main loop hands the KV blocks to the background thread and quickly returns to model execution.

The background thread establishes or reuses the NIXL connection, maps the request's KV block IDs to memory descriptors, merges contiguous blocks into larger transfer regions, and submits the NIXL WRITE to the decode instance. When NIXL completes the transfer, it sends a completion notification to the decode instance, which then begins token generation. Keeping descriptor preparation and transfer submission off the latency-critical main loops prevents this per-request work from delaying model execution.

Timing comparison: Pull vs push

Although both modes use the same NIXL and EFA/RDMA data path, pull performs the metadata relay and decode-side setup after prefill, while push overlaps decode-side setup with prefill. As a result, push removes these operations from the post-prefill critical path, allowing decoding to begin sooner.

The two modes also involve a routing tradeoff. Pull mode allows the router to select a decode instance after prefill completes, which provides greater flexibility for load-aware routing and quality-of-service management. Push mode selects the decode instance earlier so that destination setup can overlap with prefill. In return, under comparable load, the prefill instance can release its KV blocks sooner because the transfer starts as soon as both the prompt KV and destination registration are ready.

Performance data

For evaluation, we used Qwen3-32B model on a 1P1D deployment using Amazon EC2 P5en instances connected with EFA. To isolate P→D transfer performance, we configured the benchmark to disable prefix caching with --no-enable-prefix-caching. This configuration makes the decode node retrieve the full KV cache from the prefill node for every request.

The following chart compares TTFT for push and pull KV transfer. We sweep input lengths 1k–16k and request rates 1–8 QPS. At low load and short prompts, the two modes perform almost identically. As QPS and input length grow, pull mode degrades sharply, reaching about 1,744 ms at 8 QPS and 16k. Push mode stays far lower at roughly 540 ms, around 3x reduction. The gap opens because pull mode posts NIXL descriptors inline on the decode worker, synchronously within its engine loop, so that work stalls token generation. Push mode moves it to a background thread, off the critical path.

This chart shows TPOT under the same conditions. The pattern mirrors TTFT. At light load, push and pull are close, around 5.6–5.8 ms. As load and input length rise, pull climbs to about 10.4 ms at 4–8 QPS and 16k, while push holds near 7 ms. The reason is that push offloads descriptor preparation and transfer submission from the decode worker. Token generation is never starved, so per-token latency stays flat even at high concurrency and long contexts.

How to run benchmark

To reproduce our 1P1D benchmark configuration, one prefill node and one decode node, using Amazon EC2 P5en instances connected with EFA, refer to the detailed instructions in the Reproducing our results guide. The guide covers environment setup, launching the vLLM prefill and decode servers, running the benchmark, and collecting results. The feature can run on other supported configurations, but performance results will vary depending on the hardware, model, workload, and network configuration.

Conclusion

In this post, we showed how the push-based KV connector overlaps prefill computation with decode-side block allocation and registration, removing the post-prefill router round trip and decode-side transfer setup from the critical path. This reduces TTFT and TPOT, particularly at high request rates. In our Qwen3-32B benchmark on a 1P1D deployment using Amazon EC2 P5en instances connected with EFA, push mode provided up to a 3x improvement in TTFT and 30% improvement in TPOT compared with pull mode. Results vary by workload and configuration. To use push mode, configure both the prefill and decode servers to use NixlPushConnector instead of NixlConnector.

Choose the push-based connector when early decode node selection is acceptable and lower TTFT is the priority. The pull-based connector remains the better fit for workloads that require bidirectional KV transfer or defer decode-instance selection until after prefill for load-aware routing. vLLM supports both push- and pull-based KV connectors, allowing users to select the approach that best fits their deployment.

As a future improvement, we are exploring bringing the background NIXL threading model to the pull-mode connector, moving NIXL setup off the critical path so that pull-mode workloads could benefit from the same optimizations; we have opened a pull request (#45211) to explore this.

Related resources

• vLLM disaggregated serving guide
• AWS EFA
• Push-mode NIXL KV connector

Acknowledgements

We thank the vLLM community for their support in maintaining this feature.

Appendix: Reproducing our results

The topology is two p5en GPU nodes (prefill on :8100, decode on :8200) with EFA enabled, plus a third smaller node that runs the proxy and the benchmark client.

Test environment

Both GPU nodes were configured identically:

EFA device selection by NIXL. The LIBFABRIC backend opens all 16 EFA devices as rails with the efa provider. It then registers each GPU's KV cache only on the 2 rails that share that GPU's PCIe switch, so every transfer stays PCIe-local. To confirm it on your own nodes, launch the servers with NIXL_LOG_LEVEL=DEBUG and check the registration lines:

grep -E "Created rail [0-9]+ \(device=|mapped to rail" server.log

1. Provision the nodes

Install the EFA userspace stack on both GPU nodes and confirm the provider is present. We used installer 1.50.0:

curl -O https://efa-installer.amazonaws.com/aws-efa-installer-1.50.0.tar.gz
tar -xf aws-efa-installer-1.50.0.tar.gz && cd aws-efa-installer
sudo ./efa_installer.sh -y
fi_info --version         # libfabric 2.6.0amzn1.0
fi_info -p efa            # verify the EFA provider is present

The two GPU nodes must reach each other on 8100/8200 and on the NIXL side channel (5600).

2. Install vLLM and NIXL

On both GPU nodes:

python3.12 -m venv ~/vllm_env && source ~/vllm_env/bin/activate
pip install --upgrade pip
pip install vllm==0.29.0
pip install nixl==1.4.1

3. Download the model

export HF_HOME=/shared/huggingface
hf download Qwen/Qwen3-32B

4. Launch the P/D pair

Run serve_pd.sh once per GPU node. The only difference between the two modes is kv_connector: NixlConnector for pull, NixlPushConnector for push. Ports, engine IDs, roles, and ranks stay the same.

# on the prefill node
PREFILL_IP=<PREFILL_IP> DECODE_IP=<DECODE_IP> ROLE=prefill MODE=pull ./serve_pd.sh
# on the decode node
PREFILL_IP=<PREFILL_IP> DECODE_IP=<DECODE_IP> ROLE=decode  MODE=pull ./serve_pd.sh

Which expands to, on the prefill side:

vllm serve Qwen/Qwen3-32B --served-model-name Qwen --dtype bfloat16 \
  --tensor-parallel-size 8 --max-model-len 32768 --max-num-seqs 8 --port 8100 \
  --kv-transfer-config '{"kv_connector":"NixlConnector","kv_buffer_device":"cuda",
    "kv_role":"kv_producer","engine_id":"prefill-engine-001","kv_rank":0,
    "kv_parallel_size":2,"kv_connector_extra_config":{"backends":["LIBFABRIC"]}}'

5. Run the sweep

git clone https://github.com/vllm-project/vllm.git ~/vllm && cd ~/vllm
git fetch origin pull/57567/head:pd-bench && git checkout pd-bench
export VLLM_SRC=~/vllm
 
VLLM_SRC=~/vllm PREFILL_IP=<PREFILL_IP> DECODE_IP=<DECODE_IP> \
  MODE=pull ./run_sweep.sh

Then stop both servers, relaunch step 4 with MODE=push on each node, and rerun the sweep with MODE=push.

Cleanup

./cleanup.sh proxy      # on the proxy / benchmark driver node
./cleanup.sh server     # on each GPU node

Netflix described the infrastructure it uses to detect faulty GPUs and network hardware, automatically remediate unhealthy machines, and keep distributed training jobs alive through hardware failures.

When you write code by hand, most of your taste gets used without your noticing. You pick a name, decide a function is getting too long, notice that two cases are really one case. These decisions happen at typing speed, which is slow, so there's time to make them. If you have bad taste, you make bad decisions slowly, and the slowness limits the damage.

AI coding tools take away the slowness. You can now get a few hundred lines of plausible code in less time than it takes to read them. So what happens to taste when you're no longer the one typing?

The obvious answer is that it becomes a filter. The model proposes, you judge. That's true as far as it goes, and it already makes taste matter more than it did, because you're judging far more code than you'd ever have written, and judging is harder than writing. When you write a function yourself you know why each line is there. When you read one someone else wrote, you have to reconstruct that, and code that looks fine is the hardest kind to check, because nothing makes you stop.

But I think the filter view misses the bigger part. By the time you're reviewing output, most of the important choices have already been made: what to build, how to split it up, what to leave out. If you ask for the wrong thing, the model will build the wrong thing well, and good taste at review time can only tell you it's well built. So the taste that matters most comes earlier, in knowing what to ask for. Suppose you ask for a caching layer when the real problem is that you're fetching the same data in three places. You'll get a nice cache. What you needed was to delete two of the fetches.

Won't the models just develop taste themselves? I suspect they'll get very good at one kind of it. They already know the idioms of most languages better than most programmers, and local judgment about code (this loop should be a map, this name is misleading) is the sort of thing you can learn from lots of examples. What's harder to learn from examples is taste about your particular problem: which complexity is essential and which is accidental, what users will actually do, which part of the system is going to change next year. That depends on knowing what the program is for, and that knowledge mostly lives with the people who own it.

So the kind of taste worth developing is shifting. Being able to tell clean code from messy code is still useful, but it's becoming the part the tools can help with. Knowing what shape a program should have before anyone writes it is the part they can't do for you yet, and it's also the part with the biggest effect, since every line the model writes inherits it.

Which suggests something slightly odd. If AI writes most of the code, the best way to get better at AI coding might be to spend more time thinking about problems and less time thinking about prompts. I'm not sure how far that goes. But if I had to bet on which programmers will get the most out of these tools, I'd bet on the ones who already knew what they wanted.

Giving AI agents a way to independently verify their work heavily improves the quality of output. Write end-to-end tests that simulate real user flows and act as the ground truth. Use property-based testing to specify what should never happen and generate thousands of test cases that attempt to break those constraints. Require tests to run quickly and deterministically so that agents retry when the loop is slow or unreliable.

AI is mass decompiling games. That's both awesome and scary.

Introduction

I'm going to preface this post by saying I generally don't like AI. What an original take, I know. I hate that the internet is full of slop and bots, I hate that it's used as justification to fire hundreds of thousands of workers, I hate that the AI industry is making hardware inaccessible for consumers, I hate that it's being shoved into everything, and so forth. I like to do everything by my hand and take pride in my work.

That said, every now and then there's AI news that becomes impossible to ignore or act like it doesn't matter. This is especially one of those times. AI is now able to very quickly decompile video games and software, allowing them to easily get modified and enhanced and recompiled to all sorts of different systems.

And honestly? I have to admit - it's very cool. So many old, abandoned games are getting native ports to other platforms, enhanced with high FPS and running at dynamic native resolutions, with modding support. These type of enhancements usually aren't possible through an emulator.

Game Decompilations???

Decompiling games and software is the process of reverse-engineering it from a compiled version back to source code. Once decompiled, you can practically do anything you want to that code then recompile it back to an executable that runs on your computer. Or mobile phone. Or smart fridge, really.

Decompiling old games isn't a new trend. The process of picking apart an old game and re-writing the code was doable by hand, but it was so complicated and time-consuming that it took years to decompile a game. The only ones that ever make it to the finish lines are through massive community efforts. Notable decompilations tirelessly made by hand are OpenGOAL (Jak and Daxter games) and Ship of Harkinian (Zelda Ocarina of Time/Majora's Mask). These developers are legends.

Thanks to the games being decompiled, modding all these games is now possible, and there are some really fun ones. Here's Jak and Daxter but you play as Gordon from Half Life. The mod can be found here, and it was not made with AI.

Newer AI models and workflows are now able to decompile games in a matter of days, or maybe less. The code isn't nearly as clean as doing it by hand, but it works. Some ports are a mess made by people who have no idea what they're doing, but most ports are undeniably impressive and stable.

Some Examples

A great example of this being used for good is IsaacRecomp, which ports The Binding of Isaac in C++! No, not the remake. This is for the original Flash version made in 2011. Flash Isaac ran at a locked 30FPS and a low, blurry resolution. It was also a bit unstable because you're trying to run Adobe Flash in 2026. But now... Look at this.

The game now runs at a native 1440p, smoothly interpolated to 165FPS! If you've ever played Flash Isaac, you can immediately tell how beautiful that looks compared to the original. And it was all made automatically. In a few days. By an AI. That's insane.

There are hundreds of these projects now, with more coming every day. From a game preservation standpoint, it's incredible how we can take old janky experiences and get them to play on modern systems while also modernizing them. You can absolutely have your opinions regarding AI, but you have to admit this is cool.

Another one I like is Colin McRae Rally 2.0, a very old game that also barely ran on modern systems. Or Final Fantasy Tactics Advance, now running at any aspect ratio with dynamic zooming in and out. There's a Metroid Prime port. There's Ratchet and Clank which should be finished soon. There are too many more to mention and I'm struggling to keep track of them all.

The Scary Part

Most people already realized the scary part by now. If AI can decompile games and software, then the entiry tech industry is about to explode. Anybody can decompile other software and use it any way they'd like, and nobody can stop them. Companies can steal from competitors. Your paid game or app can be open-sourced against your will. This isn't exclusive to old video games, we can apply this to any software.

Decompiling software is completely legal, and even if it won't be - it doesn't matter. You can't prove it's stolen code. I can make 1:1 copies of other things and say it's totally original, then publish it online. Someone made a fantastic joke website about how you can copy code with AI and claim it's made in a clean room, so technically you're not under any licensing agreements.

That's scary. The implications of this haven't been fully realized by people yet, but it's only a matter of time before alarms start ringing. I expect that in the next few months, we'll see some major application get decompiled and shared that will cause a serious response from companies.

Maybe open source licenses are completely unenforceable now. Maybe critical software and hardware can be easily hacked into now, and we haven't realized just how bad it is yet. Maybe everything will have more invasive DRM on it soon. Who knows. We're about to find out in the next few months.

But Isn't This Also Awesome?

This is still a big win for consumers, though. Old abandoned games and software can be ported and polished for modern systems! I can't help but be giddy when seeing something like Skate 3 and Black Ops Zombies running on your browser. We live in weird times. I wasn't kidding, here's Black Ops 1 Zombies running on my browser (at a bad framerate, but still).

This is also a big win for preservation. Old games can be ported, modified, enhanced, and made much more easily accessible than before. Shouldn't that be celebrated more?

There's a project in development called AnyPS5 which will allow you to automatically port any PS5 game to your PC, making them fully playable. This isn't hypothetical vaporware, some PS5 games have already been ported to PC such as Animal Well! This also isn't an emulator, it's more of a compatibility layer similar to Proton (which allows you to play Windows games on Linux).

With console makers going all in on digital and games becoming entirely unplayable due to servers shutting down, this is fantastic news. We could own software we bought regardless of what the company thinks.

I expect modding to become way easier too since AI can find ways to hook into a game's code. You might've already seen some sloppy mods that combine two games together. Those are dumb, but they make a point that AI could potentially create modding frameworks for previously unmoddable games. That's neat.

Speaking of "dumb but neat", I also wanted to share this AI assisted port of Mario 64... to Scratch. Yes, Scratch - the 2D-only game engine for children where you code with blocks. I'm not messing with you. It only includes the first level and some things like signs don't work, but it's a great proof of concept that AI can help you run anything anywhere. I tried it, and it works. We live in very weird times. Note that I'm playing this on TurboWarp, which is just Scratch but runs faster.

"Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should"

I know. AI still sucks, it's still causing a lot of issues in the world, and it's still accelerating us to a future that is too uncertain. But this is one of the positives of having such a technology, and I hope instead of random internet users attacking those using AI for decompilation and blindly hating these projects, we can just accept the win and play those old games you love at 4K 240FPS.

• At Meta, we’ve built a system that enhances ads recommendations by leveraging idle inference capacity in our fleet.
• Two elasticity levers in the system allow ads recommendation work to expand across opportunistic capacity, without compromising baseline performance.
• This post describes how we designed and now operate the system at Meta’s scale and how it drives significant advertiser value and capacity efficiency.

Inference fleets face a capacity paradox: They can be globally constrained while meaningful pockets of capacity remain locally idle. At Meta, we built a system that aggregates these pockets into a shared opportunistic capacity pool and turns them into better ads recommendations. By putting otherwise idle capacity to work, the system helps people see more relevant ads and improves advertiser outcomes without requiring additional capacity orders.

Using this opportunistic capacity safely is difficult. It’s distributed across team-owned capacity allocations, regions, and accelerator types, and may be reclaimed with little notice. The ads inference system must therefore adapt workload as capacity supply changes without overloading the dedicated capacity that serves baseline workload.

We address this challenge with two complementary elasticity levers. Rank More Ads (RMA) adjusts the number of candidates evaluated by the ranking stages. Serve Better Models (SBM) adjusts the share of requests routed to a more powerful model.

Physical and logical isolation protects the capacity reserved for baseline serving. Separate feedback loops control RMA candidate volume and SBM model coverage, while value-aware load shedding at both servers and clients allows the system to degrade gracefully when capacity is suddenly reclaimed. Together, these mechanisms allow optional work to expand and contract with opportunistic capacity while protecting latency and reliability.

The Inference Capacity Paradox: Globally Constrained, but Locally Idle

Inference fleets must maintain enough capacity to meet demand under strict latency and reliability targets. However, demand and capacity rarely align perfectly across time, regions, resource combinations, and accelerator types. Capacity can therefore remain idle in parts of a fleet even when the fleet is constrained overall. Common causes include:

• Peak-provisioning and demand-variation mismatch: Inference fleets must be provisioned for peak demand, while traffic varies by time of day and season. Capacity needed during peak periods can therefore remain temporarily idle when demand is lower.
• Model-development and capacity-delivery timing mismatch: Inference capacity arrives on supply-chain timelines, whereas models are developed and launched on separate schedules. This creates a gap between when capacity becomes available and when models are ready to consume it.
• Regional resource-ratio mismatch: Inference workloads require accelerators as well as supporting resources such as CPUs. When these resources are not available in the required proportions, one resource can remain idle while another limits serving capacity.
• Accelerator-type mismatch: Inference fleets can span multiple accelerator vendors, generations, and architectures. Workloads are not always portable across these accelerator types, which can strand capacity on hardware that cannot immediately serve the available demand.

These mismatches create an industry-wide inference capacity paradox: A fleet can be globally constrained while meaningful pockets of capacity remain locally idle. At Meta’s scale, these temporary pockets can collectively represent substantial capacity and create better ads recommendations when used safely.

From Fragmented Reservations to a Shared Opportunistic Capacity Pool

Across a heterogeneous fleet at Meta’s scale, instantaneous idle inference capacity is inevitable — even with proactive planning designed to minimize structural mismatches. However, because this capacity is inherently tied to fluctuating baseline demands, it comes with distinct operational constraints — it’s preemptible (can be reclaimed at any time), highly volatile in its availability, and carries no long-term guarantees.

To successfully consume this volatile resource without compromising baseline performance, workloads must exhibit certain structural properties:

• A high degree of workload elasticity: The workload can quickly add or remove a large amount of optional work as the pool changes without affecting baseline expectations.
• Sustained quality at scale: The workload can continue to meaningfully improve output quality across the range the opportunistic pool can supply. This type of workload often already uses a large amount of dedicated capacity due to this property.
• Limited cross-service dependencies: A self-contained workload reduces the number of upstream and downstream systems that must respond to volatile capacity.

Two Complementary Elasticity Levers

Based on these structural properties, we identified two complementary levers that meet these criteria to consume this idle capacity and generate large value upside.

Rank More Ads (RMA)

For each user request, the ads recommendation system must select a small number of ads from millions of candidates in less than a second. The system uses a sequence of ranking stages in which increasingly powerful machine learning models evaluate progressively smaller candidate sets. The later stages only evaluate the top candidates from the previous stage.

RMA increases the number of candidates evaluated by the ranking stages when opportunistic capacity is available. Evaluating more candidates increases the chance of finding ads that are relevant to the user. Because the additional candidates pass through CPU-heavy upstream (e.g., retrieval, feature processing) as well as accelerator-heavy inference, RMA generally requires both CPU and accelerator surplus.

Serve Better Models (SBM)

The ads evaluation can use models with different levels of complexity. More powerful models can improve prediction quality, but it’s not efficient to use them for every request with dedicated capacity.

SBM changes the share of requests evaluated by the more powerful models as accelerator capacity changes. It keeps the number of ads entering the ranking stages constant, so upstream CPU demand remains comparatively stable. This allows SBM to use accelerator-only surplus that RMA cannot consume.

The SBM model-router component uses request-value signals to prioritize the requests expected to benefit most from a more powerful model.

Why Both Levers Are Needed

The two levers consume different combinations of resources. RMA requires both CPU and accelerator capacity, while SBM can use accelerator capacity without substantially increasing upstream CPU work. Together, they can use a broader range of opportunistic capacity than either lever alone.

The incremental impact of each lever also changes with traffic, current coverage, and the available resource mix. At a given moment, the next unit of opportunistic capacity may surface more relevant ads by ranking additional candidates to serve better ads or by routing more requests to a powerful model for improved prediction accuracy. We periodically adjust the allocation between RMA and SBM based on their measured performance and resource requirements.

Safely Using Opportunistic Capacity for Ads Inference

Opportunistic capacity can be reclaimed faster than the control loops that regulate work across a large serving fleet. We therefore design the system around one requirement – losing opportunistic capacity may reduce incremental ranking work, but it must not overload the capacity that serves the baseline workload. We support this requirement through isolated routing targets, two workload-specific feedback loops, and fast local load shedding.

Protecting Baseline Traffic Through Physical and Logical Isolation

Optional work must not consume the capacity reserved for baseline serving. SBM and RMA enforce this separation differently.

For SBM, the isolation is physical. The baseline and more powerful models use separate routing targets. Dedicated replicas are sized to serve requests with the baseline model, while replicas on opportunistic capacity serve requests selected for the more powerful model. When opportunistic capacity contracts, the system reduces the share of requests routed to the more powerful model. Those requests use the baseline model instead, whose capacity is already provisioned for them.

For RMA, the isolation is logical. The system calculates how much additional candidate-ranking work can be supported by the model replicas deployed on opportunistic capacity and uses total capacity to set a global budget for RMA. Each model replica may receive a mixture of baseline and additional work; individual requests do not need to be physically separated. Baseline traffic is protected by limiting the total incremental work to the amount that these replicas can support.

When opportunistic capacity contracts, RMA reduces its global candidate-volume target and SBM reduces its more-powerful-model coverage. In both cases, the system removes optional work rather than allowing it to spill onto the capacity reserved for baseline serving. This keeps baseline traffic within its latency and reliability targets.

Controlling Candidate Volume With Latency Feedback

RMA uses a closed feedback loop composed of a central proportional-integral-derivative (PID) controller and client-side enforcement. Looking through a simplified lens, the controller uses observed tail latency to calculate a target for the number of ads sent to downstream ranking, and clients apply that target to new requests. When latency is below the target, the controller increases the number of ads ranked. When latency rises, it reduces the number. This allows RMA to use the latency signal, which is influenced by the capacity situation, to converge on a sustainable candidate volume as conditions change.

Routing Requests With Value-Aware Model Selection

SBM uses a corresponding feedback loop with central control and client-side routing. The central controller sets the target coverage for the more powerful model, and the client-side model router decides which eligible requests use that model and which continue through the baseline model.

The router uses a request-value signal to group requests into segments based on predicted likelihood of user engagement with the ad. An explore-and-exploit process periodically measures the incremental outcome of using the more powerful model in each segment and updates the allocation between the two models. This is necessary because the incremental benefit of the more powerful model does not always align with the absolute value of a request. The process also preserves exploratory traffic across segments so the system continues to collect the observations needed to adapt, rather than reinforcing an outdated routing decision.

A calibration loop keeps the realized traffic share close to its target as the traffic mix changes across regions and over time. When capacity contracts, the central controller reduces the target coverage and the client routes fewer requests to the more powerful model. Requests removed from the opportunistic path continue through the baseline model.

Multi-Layer Value-Aware Load Shedding

The RMA and SBM feedback loops continuously align opportunistic work with available capacity, but their central components operate on the order of minutes. A sudden preemption can therefore remove capacity before clients receive new targets. The system adds value-aware protection for both workloads that can react immediately:

Server-side shedding: If the opportunistic pool loses capacity before the clients’ view has caught up, servers immediately remove lower-value opportunistic work. For RMA, they reduce the number of candidates processed within individual requests; SBM applies an equivalent value-aware shedding policy to requests routed to the more powerful model. This absorbs rapid changes close to the serving bottleneck.

Client-side shedding: The RMA and SBM clients use the latest controller targets as the guideline but also observe the server status directly. When it’s observed that the server is overloaded, clients shed lower-value opportunistic work directly, i.e., RMA requests fewer additional ads, while SBM routes fewer requests to the more powerful model.

Both forms of load shedding react within a few milliseconds, bridging the interval before the controller-client loops converge. Both server-side shedding and client-side shedding use value signals when selecting work to remove, which minimizes the effect during the transition.

Current Work and What Comes Next

Increasing Workload Elasticity

We aim to use more opportunistic capacity productively by extending the operating range of RMA and SBM and broadening the set of elasticity levers. For RMA, this means supporting higher candidate volumes across ranking stages; for SBM, using more powerful models that turn additional accelerator capacity into better predictions. We are also developing new levers such as precomputation elasticity, which performs work ahead of request time and is therefore less constrained by the location and timing of live traffic.

Expanding the Usable Capacity Pool

Opportunistic capacity is fragmented across regions and accelerator types, and it’s useful only when compatible models and supporting resources are available in the same place. We are expanding model portability and cross-region inference capabilities so that more of the available capacity can be used efficiently.

Automating Value-Based Capacity Allocation

The incremental impact of each lever changes with traffic, coverage, and the available resource mix. At a given moment, the next unit of opportunistic capacity may surface more relevant ads by ranking additional candidates or by routing more requests to a powerful model. We are building an optimization-based system to automatically adjust the allocation of AI inference capacity to its possible elasticity levers based on the benefits to users and advertisers. The goal is to maximize the return of the capacity subject to the constraints that each use case has in terms of regional constraints, hardware compatibility, and durability requirements for the idle capacity, among others.

More than 20% of the web sits behind Cloudflare’s network, and our team, the Data team, owns the data pipeline responsible for plumbing that data (and more) from our global network to applicable central data centers. The Cloudflare network is a globally distributed system supporting tens of thousands of servers. Data that power Cloudflare Logs, SQL API, observability, internal analytics, and more flow through this network and use our data pipeline to serve customers.

In May of this year, we were providing data delivery at roughly two billion events per second. This includes customer traffic as well as data sent from internal teams. Year after year, we tackle the same problem — exponential traffic growth against linear hardware growth. We have to react to this increase in order to keep resource costs under control. Over the past five years, we’ve made various improvements to compensate for the data volume increase in comparison to hardware allocation, equivalent to deferring two and a half years of hardware expansion. This post details those changes: breaking up our monolithic data schema in 2022, adjusting compression in 2024, rearchitecting pipeline replication in spring 2026, decentralizing sampling in summer 2026, and introducing aggregations into the pipeline in fall 2026.

Background

Our pipeline starts with an internal service called Logfwdr, where we collect from highly distributed data sources (think: Workers, FL). We then feed this data to a set of centralized receivers, which eventually push them onto our buffering system, Buftee. Consumers fetch data from Buftee and distribute data to their final destinations, like ClickHouse, which we rely on heavily.

Unpacking the monolith

In 2022, we dismantled our one and only schema. Previously, there was a single stream packed with every product's fields, forcing Cloudflare-internal consumers to ingest everything and demux or discard records via filters. At the time, most of Cloudflare’s data products were built on HTTP, Security and CDN metadata.

Some fields here belonged under the same product, but some were completely irrelevant to each other. The hypothesis was that none of these teams require 100% of fields in the monolithic schema. In other words, we can create small messages for each product’s required fields. Although we were going to duplicate fields, some of them in every message, replacing this firehose with product-specific schemas actually reduced the backbone bandwidth usage by 35-40%, knocking out roughly a year’s worth of compressed traffic growth.

Compression

By 2024, as traffic steadily grew, we were tackling network congestion issues on a daily basis. To react to this, we changed our compression algorithm from LZ4 to zstd. We had already experimented with zstd compression before, but we noticed a higher CPU usage at the time. Therefore, we were careful to track the metrics to ensure the trade-off was worth it.

Initially, we rolled this change out to a few servers. To our relief, we didn’t observe a noticeable difference in CPU usage.

With more confidence, we gradually rolled out to our entire fleet, which also showed little impact to CPUs. In the end, there was nearly a 40% reduction in compressed traffic throughput. The uncompressed bytes remained at the same growth, meaning we could support more than 1.5x the amount of traffic with the same size pipe. Or another way to put it, the estimated growth-equivalent reductions put us at about 10 months prior.

Now, in 2026, we attempted three strategies to reduce uncompressed bytes (and thus, compressed bytes).

Load balancing

To make analytics resilient against regional disasters, our systems require redundant data to be stored in multiple centralized regions. Historically, the simplest way to achieve disaster resilience was double-writing: Logfwdr transmitted two full copies of every non-regional analytics event — one to each centralized region. Each region then independently processed a complete copy through to ClickHouse. We also apply sampling in each region, an incredibly scalable technique that works for all volumes of data during any traffic conditions. We downsample high-volume traffic while rare events are preserved.

This infrastructure was manageable when only a handful of streams used secondary regions. However, as we expanded disaster resilience across almost all analytics streams, Logfwdr traffic doubled across the board. Sending every event, twice, over wide-area transit was unsustainable against our exponential traffic growth.

We had already tackled a similar challenge for Logpush. For Logpush, Logfwdr treats the centralized data centers as a unified, shared pool of receivers: each message is sent to only one region, failing over to another destination if a machine or facility is unreachable. That model had already proven its worth during large incidents, shifting traffic seamlessly from one location to another without human intervention. This automatic failover was possible because Logpush delivery is stateless, as customer destinations do not care which centralized data center pushes the logs.

Analytics was trickier. Unlike stateless logs, ClickHouse clusters in both regions still need a full copy of non-regional data to allow instant, independent query failover. The breakthrough was decoupling pipeline transit from database redundancy:

1. Shared Ingest Pool: We adopted Logpush’s model to the analytics streams. Logfwdr dispatches each event exactly once across wide-area transit to whichever centralized receiver is available.

2. Late Replication from centralized regions: Instead of replicating traffic at source, replication happens as late as possible in the pipeline. Data lands in our distributed buffering layer, Buftee, and remote inserters stream cross-data-center copies downstream.

3. Resilient Failure Handling: If a centralized region goes down, Logfwdrs immediately routes all non-regional traffic to the surviving region. If the inter-data center connection blips, each cluster continues ingesting its local share without backpressure; remote inserters catch up from Buftee's retention buffer once the backbone recovers.

For the datasets we migrated, the cutover eliminated approximately 43 GB/s of compressed transit traffic while adding only ~7 GB/s of cross-data-center replication on our pipeline.

Sampling from the get-go

Another strategy we employed was further expanding our sampling mechanism so that we pulled critical processing earlier into the pipeline. Sampling was already used in our centralized services, particularly the receivers, easing pressure on downstream systems.

The solution here was to simply sample in Logfwdr, in addition to maintaining the sampling in centralized receivers. When sampling, we first shuffle the records for each key to eliminate bias. Then, for each group of records, we select a sample size equal to the square root of the total count of the records. The result is a representative sample, whether sampled once or twice. To produce estimates, we scale calculations against the sample rate.

To ensure this approach was working well, we compared the number of rows written to ClickHouse against the estimated sum for total count using a variety of keys across a variety of traffic footprints.

The result is that for keys with more data, the sample rate decreases. However, keys with less data were sampled comparably. Thus, we still kept a representative sample while delivering and storing fewer events. We’re not compromising the quality of data delivered, even when we deliver less in quantity.

Aggregating

Cloudflare offers several analytics datasets built on rollups: aggregates of the raw events computed at ingest time. This makes the opposite tradeoff to sampling: rather than answering any question approximately, we answer a fixed set of questions exactly, provided we choose the dimensions and metrics up front.

Historically, this aggregation occurred towards the end of the pipeline, inside the inserters which read from Buftee and insert into ClickHouse. Decentralizing this aggregation was an opportunity to spread out load and reduce the amount of data we centralize.

Hence, we built an aggregation functionality that is utilized every step of the way, creating repeated opportunities to combine matching partial aggregates as data converges. Much like sampling, records are grouped by a key, which determines which records can be combined. We aggregate on each instance of Logfwdr. We aggregate again after the partially aggregated events reach the centralized receivers, which push these re-aggregated events onto dedicated partitioned buffers. Because these buffers bring together records from multiple receivers, we aggregate another time after they’ve been read in batches by the ClickHouse inserters. Finally, we aggregate in ClickHouse with aggregating table engines.

Impact

The starkest proof of this exponential growth is how rapidly traffic growth consumes headroom. Across the entire pipeline altogether, from spring to fall, load balancing (~38 GB/s saved), sampling (~25 GB/s saved across top datasets), and aggregating (~7+ GB/s so far), eliminated ~70 GB/s of compressed traffic from our backbone. These cliff drops provide immediate relief.

If we zoom out such that we look at five years from now, the changes made in this year alone have just as much impact as the short-term benefits. We’ve halved the amount of data we’d be processing in another half a decade’s time. The structural and algorithmic optimizations we have and will continue to make are essential to keep hardware demands sustainable.

Four PostgreSQL arguments that came out even

Some advice gets repeated in code review because it sounds like it should matter. I picked four I have seen argued about and measured them. I expected at least two clear winners. I got three ties and one result that went the opposite way from the advice.

Setup

PostgreSQL 18.6, default settings. 2020 13-inch MacBook Pro, Intel i5-8257U, 8 GB RAM. A 2,000,000-row table u with an id, an email, a status and a timestamp. Times are execution times from EXPLAIN (ANALYZE, TIMING OFF), warm cache, first run discarded, median of the rest.

IN, ANY or a VALUES join for 1,000 ids

All three read exactly the same pages. The medians differ, but one IN run took 54 ms and another 15 ms, so I would not read anything into the gap. Use whichever form your driver makes easy. ANY has one practical advantage when the array is passed as a single parameter: the statement text stays the same whatever the number of ids, which is kinder to prepared statements and to pg_stat_statements.

text, smallint or enum for a status column

Three values, two million rows, each stored three ways in its own one-column table.

Every row has a 24-byte header and is padded to 8 bytes, so a 2-byte smallint and a 7-byte 'active' end up the same size. The indexes are identical because B-tree deduplication stores each of the three values once with a list of row pointers. In a wider table the column order can change the padding, but the type of a short status column is not where the space goes. Choose it for what it lets you check.

Columns that are usually empty

One million rows with an id and ten more integer columns. In one table the ten columns are all NULL, in the other all filled.

A NULL takes no space in the row. It costs one bit in the row's null bitmap. Optional columns that are mostly empty are close to free, which is worth knowing before splitting a table to save space.

count(DISTINCT) against GROUP BY

The advice I have seen most often: count(DISTINCT x) is slow, rewrite it as a count over a GROUP BY subquery. On two million unique emails:

The rewrite was slower. I am not claiming it always is. Which one wins depends on the plan the GROUP BY gets and on work_mem, and both change between versions and settings. That is the point: this is advice to check with EXPLAIN on your own data, not to apply.

What I did not test

• IN lists with tens of thousands of values.
• Status columns inside a wider row, where alignment can differ.
• count(DISTINCT) with more work_mem, or on columns with few distinct values.
• Other hardware. Compare the ratios, not the seconds.

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

Python used to have limited parallelism, due to its global interpreter lock (GIL). Only one thread could run Python code at a time. Python 3.14 comes in a free-threaded version (python3.14t), without the global interpreter lock. Several threads can now run Python code at the same time.

Let us pick a realistic problem. GitHub publishes everything that happens on its public repositories: pushes, issues, pull requests, stars, and so forth. The GH Archive project collects these events as JSON, one event per line (NDJSON), one file per hour. I took eight hours of March 13, 2024: 7.6 GB of JSON once decompressed, 2.2 million events. I want to know which programming languages get the most new pull requests. So I count the events by type, and for each pull request that was opened, I look up the language of the repository. Each event is a large JSON object, about 3.4 kB on average: I need only a few of its fields.

The answer, for these eight hours: 65,269 pull requests were opened, mostly toward TypeScript (11,565), JavaScript (7,778), Python (7,057) and Java (4,668) repositories. About 5,857 targeted repositories without a language.

With fastsimdjson, our Python binding for simdjson, the code looks as follows. The function map_many calls my function on each event, using as many threads as I ask for.

import collections, threading
import fastsimdjson

local = threading.local()
parts = []

def one_event(e):
    acc = getattr(local, "acc", None)
    if acc is None:  # first event seen by this thread
        acc = local.acc = (collections.Counter(), collections.Counter())
        parts.append(acc)
    t = e["type"]
    acc[0][t] += 1
    if t == "PullRequestEvent":
        p = e["payload"]
        if p["action"] == "opened":
            acc[1][p["pull_request"]["base"]["repo"]["language"]] += 1

data = open("events.json", "rb").read()
fastsimdjson.map_many(one_event, data, format="lines", threads=64, lazy=True)

types, languages = collections.Counter(), collections.Counter()
for t, lang in parts:
    types.update(t)
    languages.update(lang)

Each thread keeps its own counters. The first event a thread sees creates the pair and appends it to parts. Later events on that thread update the same pair. Nothing in the counters is shared, so the threads do not wait for each other. map_many returns only once every event has been counted. The main thread then walks parts, one pair per thread, and Counter.update adds the counts. A few dozen event types and about a hundred languages: the merge is tiny next to the parsing.

With lazy=True, my function receives a lazy view of the event instead of a dictionary: the event is parsed, but Python objects are created only for the values I access. The lazy approach was first developed by Tyler Kennedy in another simdjson binding, pysimdjson.

How does it compare? I tried the usual suspects: the standard library (json), orjson and msgspec. msgspec deserves special mention: it can decode JSON into typed structures that declare only the fields you need, skipping the rest. It is a lot like our lazy views, except that you must declare a schema beforehand. I gave msgspec this advantage. In each case, I verify that we get exactly the same counts.

I use a server with two Intel Xeon Gold 6548N processors (64 cores), Python 3.14.6, and the current versions of the libraries (fastsimdjson 0.6.0, orjson 3.13.0, msgspec 0.22.0). The data is decompressed in memory before the timer starts. I report the best of seven runs, and one run for the slow methods: the single-thread parsers, and the standard library on 64 threads. Let us first use a single core.

The standard library needs 42 seconds to go through the eight hours. With fastsimdjson and lazy views, it takes 3.6 seconds. Notice how the methods that materialize everything (json, orjson and fastsimdjson without lazy views) are slow: most of the time goes into creating Python objects that we discard right away.

Now let us use all 64 cores, still in one process. On free-threaded Python, I use a thread pool for json and msgspec, and map_many(..., threads=64) for fastsimdjson. The threads share the input.

We go through 7.6 GB of JSON in about 0.12 seconds, at 66 GB/s, in one Python process. That is 31 times faster than with one core. It is faster than msgspec using 64 threads (25 GB/s).

So, can you parse JSON at tens of gigabytes per second in Python? Yes.

fastsimdjson 0.6.0 is available on PyPI. My benchmark is available.

Here's a question from our test set. A caller rings a multi-trade maintenance company.

"Hi, I had a plumber round last week and there's a problem with what he did, it's still leaking. Who do I contact about that?"

Our AI receptionist does what we built it to do. It searches the company's website for context and hands the closest matches to the language model. The closest match, according to our production embeddings, was a blog post called How to Find a Reliable Plumber.

Which, if you think about it, is about the worst page you could hand an LLM while it's on the phone with one of the company's unhappy customers.

It wasn't a one-off. In a test we ran at the start of October, our search handed the LLM a wrong page on a third of 168 caller questions. This post is about what we tried instead. A model that can't write a word, picking pages off a list. It handed our LLM the wrong page a tenth as often, 3% of the time against our search's 33%. It put the right page first 88% of the time, against 70%. And, the bit I didn't see coming, it knew when the website didn't have the answer.

How our receptionist reads a website today

Sayora answers the phone for small businesses in the UK: plumbers, estate agents, electricians, bars. Most calls are routine. Bookings, messages, "are you open on Saturday?". The interesting ones are questions about the business itself. Is the valuation free? Do you fit water softeners? Can I bring the dog?

The answers are usually on the business's own website, so today we fetch them the standard way:

• For a small site, we paste the homepage (or up to eight pages) straight into the prompt.
• For anything bigger, we cut every page into 1,200-character chunks with 200 characters of overlap, and embed them with OpenAI's text-embedding-3-large at 768 dimensions.
• On every caller turn, we embed what they said (plus a version with the turn before, for follow-ups) and inject the closest chunks scoring above a cosine similarity of 0.36.

It's the standard RAG recipe, and it works well enough that we shipped it. It also always bugged me.

Websites are already organised

A small business's website isn't a pile of documents. Someone who knows the business has already organised it into pages, and each page is about one thing: boiler servicing, fees, the team, the Christmas menu. Shredding those pages into confetti and trusting cosine similarity to put the meaning back together always felt backwards.

So we tried the obvious alternative:

1. Keep pages whole, and write each one a short description.
2. Give the receptionist a tool. When it hits something it doesn't know, it says "let me check that for you" and calls it.
3. The tool asks a model a multiple-choice question: which of these pages answers what the caller just said? "None of these" is one of the options.
4. The best page or two go to the LLM. If the answer is "none", the receptionist says it doesn't know and passes the question on.

One lookup from the test, on an estate agent's 74-page site.

The model doing the picking is Jev, a decision model TypeSafe launched on 15 September. It doesn't generate text. You send it some state and some typed questions, and it sends back a choice with a probability for every option. A choice question takes up to 255 options and a request up to 64k tokens. It costs $0.042 per million input tokens, and output is free. On a 100-page site, that's about a twentieth of a cent per lookup.

How we tested it

We took four of our customers' websites and removed the pages a page picker should never see. Individual property listings went, because a feed already covers those. So did near-identical "electrician in [town]" landing pages, because a coverage list answers "do you cover X?" better than a choice between 31 copies of the same page. That left:

• a bar and restaurant, 22 pages
• an estate agent, 74 pages
• an electrician, 75 pages
• a multi-trade maintenance company, 232 pages

Claude Haiku 4.5 read every page once and wrote a description: the title, the path, a 30-word factual summary and a few questions a caller might ask. About 100 tokens a page.

Separately, Claude Opus 5.5 read the page text (never the descriptions) and wrote questions the way callers actually talk. That gave us 168 questions: 128 the sites answer and 40 they don't. A quarter of the answerable ones also got a follow-up form ("and how much is that?") after two turns of conversation.

Then nine pickers had a go at every question:

• Jev: one choice question over every page on the site, plus "none"
• Jev with thinner descriptions: title and summary only, and title only
• Embeddings shortlist, then Jev: the top 40 pages by embedding, then Jev picks
• GPT-6 Luna: the same list, asked for up to three page ids, or none
• Embeddings on page descriptions: the same descriptions Jev read, embedded
• Embeddings on chunks: our production retrieval, exactly as it runs today
• BM25: keyword search, on the descriptions and on the page text

Judging was pooled. For each question, every page that any picker ranked in its top three went to a judge (Claude Opus 5.5), which read the actual page text and decided whether a receptionist could answer from it. That's 1,668 judgements. No picker was graded against pages another picker chose. When the judge found an answer to a question we'd written as unanswerable, or nothing for one we'd written as answerable, the question switched sides. That happened six times.

The result: 88% against 70%

Jev put a right page first on 88% of the answerable questions, and in its top three on 95%. Our production search managed 70%. That's 18 points, with a 95% interval of 11 to 26. Or, if you prefer headline maths, 26% better.

The fairer fight is embeddings over the exact descriptions Jev read. That narrows the gap to 14 points (7 to 21), still well clear of zero. Keyword search came last, at 53% to 57%.

The part that surprised us: knowing when to stop

Ranking is only half the job. On a phone call, the worse mistake is handing the LLM a page that doesn't answer the question, because a fluent model will often build an answer out of it anyway. That's how a receptionist ends up quoting a guide's average prices as if they were the business's own.

At our production cut-off of 0.36, chunk search handed over a wrong page on 33% of all questions. It said the website didn't have the answer for just 9 of the 40 questions the sites can't answer. Jev handed over a wrong page 3% of the time and said "not on the website" for 38 of the 40.

Why embeddings lose on websites

I read every question where chunk search put a wrong page first. Of the 39, 23 went to a generic guide or blog post. "Rewiring a house in the UK" for "will I get a certificate?". "How to find a reliable plumber" for the leak complaint. A long advice article shares more words with a caller's question than the business's own short page does, so it wins the cosine contest.

Jev reads the list more like a person would. When it was wrong, it was usually wrong between near twins ("Certified electrician" instead of "Part P certificate"), and only one of its 16 wrong first pages was a guide.

Couldn't an LLM just pick?

Fair question. It's the first one I got asked. We gave GPT-6 Luna the same list and asked for up to three page ids, or none. OpenAI's new Decisions API is reportedly built on a tuned GPT-6 Luna. We don't have access to it yet, so this stands in for its judgement, not its speed.

Luna tied Jev on picking: 86% right first against Jev's 88%, a difference of 2 points with an interval of −4 to +7. But it was slower, at 0.66 s median and 1.27 s at p95 (reasoning effort low), against Jev's 0.28 s and 0.42 s. It was also less willing to say "none". It caught 80% of the unanswerable questions and handed over a wrong page 8% of the time.

Embeddings still earn their keep

Some of the best numbers in the whole test came from using both. Embeddings shortlist 40 pages, then Jev picks among them: 88% right first, in 0.25 s. On the 232-page site, a shortlist of 100 got 95% right first and never handed over a wrong page.

Embeddings are good at "somewhere around here". Jev is good at "this one". You don't have to choose.

How many pages can it take?

A choice question tops out at 255 options, so we checked whether that bites. Of the 449 business websites in our customer records, 353 publish a sitemap. The median has 14 pages, and 95% have 255 or fewer. Small businesses, it turns out, have small websites. Who knew.

Then we pushed it. For each question we built option lists of 16 up to 232 pages, filling them with the most similar pages first (from the same section of the site), so every extra option was a plausible distractor.

Accuracy drifts down as the list fills with lookalikes. On the biggest site, Jev went from 100% at 16 options to 92% at 232, and the right page stayed in its top three every time. Embeddings over the same lists fell from 89% to 74%.

The descriptions do the heavy lifting

Jev only knows what the description tells it. Page titles alone cost 9 points on the right page first and 16 points end to end. Title plus a one-line summary did as well as the full description with example questions, so the example questions can go. Writing the summaries cost about $0.002 a page with Claude Haiku, once per page and again only when a page changes.

Follow-ups come free

On the phone, people don't repeat themselves. "And how much is that?" only makes sense with the two turns before it. We gave Jev those turns as state and the caller's last words as the question. It put the right page first 91% of the time, exactly the same as when we handed it a cleanly rewritten question.

The hard part was the crawler

Picking the page was the easy bit. Reading real small-business websites broke something different almost every time.

What's next

Real caller questions from transcripts, then a test on live calls. If it holds up, our receptionist reads a business's whole website instead of its homepage, the owner chooses which parts it can use, and every "not on the website" becomes a question for the owner to answer. We'd also like to put OpenAI's Decisions API through the same test once we can get in.

What gets TOASTed in Postgres

Insert 2,000 copies of the letter x into a table with an int ID and a text column, and Postgres stores all 2,000 bytes uncompressed. Insert 2,001, and Postgres compresses the value to 26 bytes with LZ4.

That one extra byte is enough to trigger TOAST, the part of Postgres that compresses values once their row gets too big, and moves them out of the row when compressing isn't enough.

Postgres 19, now in beta, changes how TOAST compresses, switching the default from pglz to LZ4 in builds with LZ4 support. PlanetScale already made this switch on Postgres 17 and 18.

Here, we look at which values get TOASTed, what TOAST does to them, and when LZ4 is the right choice. Let's dig in!

What a row costs in bytes

Compression is all about saving bytes, so we'll start with where the bytes in a row go.

Postgres stores table data in fixed 8 KB pages. A row can't be split across pages, so every row has to fit inside one.

Each row starts with a 24-byte header that Postgres uses to track which transactions can see it, followed by the values of its columns. The space those values take depends on their type, and every type falls into one of two groups:

• Fixed-width types like bigint, timestamptz, and boolean always take the same number of bytes.
• Variable-length types like text, jsonb, and bytea take as many bytes as the value needs, plus a length header: 1 byte for values up to 126 bytes, and 4 bytes for anything longer.

pg_column_size shows both groups in a single row:

SELECT pg_column_size(ROW(1::bigint, now(), true, 'Ada'::text));
 pg_column_size
----------------
             45
(1 row)

That's the 24-byte header, 17 bytes for the bigint, timestamptz, and boolean (8, 8, and 1), and 4 bytes for 'Ada': three characters plus a 1-byte length header. Change 'Ada' to 'Ada Lovelace', and the row grows to 54 bytes while the other columns stay at 17.

Variable-length values are what make a row grow, and the only ones Postgres can compress.

Small rows don't trigger compression

TOAST stands for The Oversized-Attribute Storage Technique. As the name suggests, TOAST only steps in when a row gets too big.

Postgres treats a row as oversized once it's larger than 2,032 bytes, about a quarter of an 8 KB page. This threshold was set so that at least four rows fit on every page, after room for the page header and a pointer to each row.

Here's what happens when we store the most compressible value possible, a single repeated letter, at lengths just below and just above the threshold. pg_column_compression returns the algorithm that compressed each value, or null if the value isn't compressed, which we display as none. The COMPRESSION lz4 clause picks the algorithm.

CREATE TABLE t (id int PRIMARY KEY, body text COMPRESSION lz4);

INSERT INTO t VALUES
  (1500, repeat('x', 1500)),
  (2000, repeat('x', 2000)),
  (2001, repeat('x', 2001));

SELECT id AS chars,
       pg_column_size(body) AS stored_bytes,
       coalesce(pg_column_compression(body), 'none') AS compression
FROM t ORDER BY id;
 chars | stored_bytes | compression
-------+--------------+-------------
  1500 |         1504 | none
  2000 |         2004 | none
  2001 |           26 | lz4
(3 rows)

The first two values aren't compressed. Each takes 4 bytes more than its length, for the length header. The third value, however, is just one character longer, and it's compressed to 26 bytes.

The 2,000-character row is exactly 2,032 bytes: 24 for the row header, 4 for the int, and 2,004 for the text. That's right on the threshold, so TOAST leaves it alone. The 2,001-character row is one byte over.

TOAST does two different jobs

The threshold applies to the whole row, but TOAST works on individual variable-length values. Once a row is over the threshold, TOAST has two ways to shrink it:

• Compress a value, and keep the smaller version in the row, like the 2,001-character row.
• Move a value out of the row, into a separate table.

That separate table is called a TOAST table. Tables with columns that can hold large values get one automatically in the pg_toast schema.

A value moved to the TOAST table isn't stored in one piece, though. TOAST splits it into chunks of up to 1,996 bytes and stores each chunk as its own row, tagged with the value's ID (chunk_id) and its position in the value (chunk_seq). The rest of the 2,032 bytes holds the chunk row's header and IDs, so four chunk rows fit on a page. The original row keeps an 18-byte pointer, so Postgres can find the chunks and reassemble them.

TOAST works through a row's values from largest to smallest. As soon as the row is back under 2,032 bytes, the table's toast_tuple_target, it stops.

To see every outcome, we store four very different values in one table: a short string, 10 KB of repeated text, 200 KB of random characters, and 1.1 MB of a repeated sentence. stored_bytes is the value's size wherever it ended up, including the TOAST table for a moved-out value. octet_length returns a value's uncompressed size, and pg_column_toast_chunk_id (Postgres 17+) returns its ID in the TOAST table, or null if the value is still in the row.

CREATE TABLE docs (id int PRIMARY KEY, body text COMPRESSION lz4);

INSERT INTO docs VALUES
  (1, repeat('x', 200)),
  (2, repeat('abc', 3400)),
  (3, (SELECT string_agg(md5(random()::text), '') FROM generate_series(1, 6250))),
  (4, repeat('The quick brown fox jumps over the lazy dog. ', 25000));

SELECT id,
       octet_length(body) AS raw_bytes,
       pg_column_size(body) AS stored_bytes,
       coalesce(pg_column_compression(body), 'none') AS compression,
       pg_column_toast_chunk_id(body) IS NOT NULL AS moved_out
FROM docs ORDER BY id;
 id | raw_bytes | stored_bytes | compression | moved_out
----+-----------+--------------+-------------+-----------
  1 |       200 |          204 | none        | f
  2 |     10200 |           60 | lz4         | f
  3 |    200000 |       200000 | none        | t
  4 |   1125000 |         4471 | lz4         | t
(4 rows)

Each value lands somewhere different:

• Row 1 is under the threshold, so TOAST never touches it.
• Row 2 compresses from 10 KB to 60 bytes, small enough to stay in the row.
• Row 3 is random, so compressing it doesn't save space. TOAST moves the value out uncompressed.
• Row 4 compresses to 4,471 bytes, which is still too big for the row, so TOAST compresses it and moves it out.

Each column's storage strategy decides which of those jobs TOAST can do:

• PLAIN: neither. Used by fixed-width types like int.
• EXTENDED: compress, then move out if needed. The default for text, jsonb, and bytea.
• EXTERNAL: move out, but never compress.
• MAIN: compress, but keep the value in the row even above the usual TOAST target if the row still fits on a page. Move it out only as a last resort. The default for numeric.

TOAST isn't a large object

Postgres has a second way to store big values, called large objects, built for whole files like videos or PDFs.

TOAST and large objects both store big values in chunks, but they differ in three ways:

• TOAST is automatic. It applies to any variable-length column without you asking. A large object only exists if you create one.
• TOAST values top out at 1 GB. Large objects can hold up to 4 TB.
• TOAST values are deleted with their row. A large object stays until you remove it with lo_unlink, even after the row that references it is gone.

How pglz and LZ4 work

Postgres can compress a TOAST value with one of two algorithms, pglz or LZ4. A column uses the one its COMPRESSION clause names, or the default_toast_compression setting if it doesn't name one. Changing either only affects values compressed from then on. Values already compressed keep their algorithm, even when you copy them with INSERT ... SELECT or rewrite the table with VACUUM FULL.

zstd isn't an option for column compression. Postgres uses it elsewhere, like write-ahead log (WAL) compression through wal_compression (Postgres 15+) and backups.

Both algorithms work the same basic way. They scan a value from start to finish, looking for bytes that already appeared earlier. When they find a repeat, they replace it with a short note saying where the earlier copy is and how long it is.

pglz wins on these JSON documents

In a JSON array, the same keys come back in every object, close enough for both algorithms to find them.

We loaded 10,000 order documents into three tables on PlanetScale Postgres 18: one using EXTERNAL so nothing gets compressed, one using pglz, and one using LZ4. Each document has 25 to 100 line items, and every line item repeats the same five keys: sku, name, qty, price_cents, and tags.

The uncompressed table is 103 MB. The pglz table is 19 MB and the LZ4 table is 24 MB.

Since both find the keys, the size comes down to pglz's extra searching, which finds longer matches. LZ4 makes up for it in speed: in two runs each, it loaded the same documents in 0.48 and 1.09 s, against 1.59 and 1.73 s for pglz.

pglz is faster for previews

pglz also wins when you only read the start of a large value, like a 200-character preview with left(body, 200). pglz never needs more than 9 bits to store a byte of the original, plus 2 bytes overall, so Postgres can put a ceiling on how many compressed bytes a prefix needs. A character takes 1 to 4 bytes, so 200 characters are at most 800 bytes, and 800 bytes of output need at most 902 compressed bytes. That's less than the 1,996 bytes in a chunk, so Postgres fetches only the first one. Postgres has no equivalent ceiling for LZ4, so it fetches the whole value first.

When to skip compression

Compressed values take fewer pages, which means less disk, more of a table in memory, and less WAL to send to replicas and back up. The cost is CPU, on every write that crosses the threshold and every read of a compressed value. Compression also doesn't reduce egress, since Postgres decompresses values before sending them.

Two kinds of data are better off uncompressed, with EXTERNAL:

• Already-compressed data, like JPEG frames. LZ4 spends CPU on every frame and saves little or nothing.
• Large bytea values read from the middle, like seeking in a file.

Wrapup

TOAST only steps in once a row passes 2,032 bytes, so plenty of values, like the 2,000 xs from the start, never get compressed at all. For the ones that do, LZ4 is the right default for most tables: its speed matters more than the size difference, which can go either way. For a few kinds of data, it's worth choosing pglz or EXTERNAL by hand.

I like sparse interfaces. My approach to design owes a lot to Danish functionalism, and I’m usually looking for something I can remove before I start adding another control. But a screen can look beautifully restrained whilst asking the person using it to do a surprising amount of work.

I’ve written about affordances before, and the argument gets a little abstract if we leave it at making controls easier to discover. I want to work through a small example instead: saving an article to read later.

The versions below are illustrative designs, not a usability study or a teardown of a particular product. They contain the same article and offer the same action. What changes is how much of that action they explain.

Saving an article

Imagine a reading list where each row shows a title, a source and a date. In the sparse version, a bookmark outline appears when you hover over a row. Press it and the outline becomes a filled icon.

That feels like a reasonable amount of interface if you already know what the icon does. You reveal it, press it, notice the change and move on. The article gets most of the space, which is what we wanted.

Now remove the assumptions. You haven’t used this product before, so you don’t know whether the bookmark means save, open saved items or add this article to a collection. Before you can decide, you need to find it. On a touchscreen, hovering isn’t available as a dependable way to reveal a control, so the design needs another route.

We could reveal it when the row receives keyboard focus too. That helps a keyboard user reach the action, but it doesn’t resolve the uncertainty about what the icon means. An accessible name helps someone using a screen reader; it doesn’t give a sighted reader the visible explanation we’ve removed.

For a more explicit version, put a small Save button beside the title. After activation it becomes Saved, and a nearby message offers Undo. The interface now gives the action a name, confirms its state and offers a way back.

The second version contains more visible information. I still think it asks less of someone encountering the task for the first time. They can read what will happen rather than reconstructing it from an icon and a change in fill.

That’s my judgement looking at these examples. I’d still want to watch people use them, because a label that seems helpful to me isn’t proof that the whole task has become easier.

Kevin Bonett made a useful point in response to this article (paraphrased here): explicit labels can take up more space whilst giving people less to work out. And if every article has a Save button, its accessible name needs to include the article title too, so someone using a screen reader or voice control can tell which article they’re saving.

What happens on the fiftieth visit?

The explicit version has costs too. A Save button on every row repeats the same word. Confirmation messages can interrupt reading, particularly if every press produces an announcement in a distant corner of the screen. A dense list may become harder to scan if every action competes with the article titles.

I’d keep Save visible and visually subordinate to the title. I’d show the saved state in the same place, so you can check it without moving your attention. Undo could appear beside that state for the immediate correction, with a persistent, discoverable Remove from saved action available afterwards. The short-lived message shouldn’t be the only route back.

If the product has a dedicated saved-articles view, it may need a different action label there. You’re already looking at saved items, so repeating Saved beside every one adds little. Remove would explain the available action more directly.

In that view, we already know the items are saved, so I can use the space to explain what somebody can do next. The context is doing some of the work the repeated label did before.

A control can be quick and familiar in one setting and baffling in another. We can’t settle that from an isolated component screenshot.

Remove the work as well

For this example, my revised version would retain the quiet layout, keep the action visible on every input method and use a short word to explain it. I’d remove the separate success toast if the local Saved state and Undo action communicate enough. I’d also avoid adding an explanatory tooltip that merely repeats Save.

I’d end up with a fairly quiet layout, a visible action and feedback close to where the person used it. There’s still some explanation on the screen, but fewer separate things asking to be read.

To check the result, I’d ask someone to save an article, find it again and remove it. I’d watch where they pause and what they expect the controls to do. Then I’d repeat the task with touch and keyboard input, and check that its state and actions make sense with a screen reader. Counting visible elements wouldn’t tell me any of that.

I’m still inclined to remove things. I just want to check what they were helping somebody understand before I take them away. It’s quite easy to forget how much I already know about an interface I’ve been designing.

AI has accelerated engineering at Anthropic to the point where deciding what deserves to be built is becoming a greater constraint than implementing it, pushing designers closer to production code and working prototypes. Designers routinely build their own tools, generate interactive variations using Anthropic's design system, and collaborate through smaller, more frequent critiques and working demos rather than lengthy specifications. Technical fluency is increasingly useful, but Anthropic also values designers who continually reinvent their workflows, have a point of view about where software is heading, and can move fluidly between strategic direction and hands-on making.

Fluid Functionalism

Refined UI components with satisfying hover.

Why is every other input box so stiff? Because nothing about them moves with you.

What is Fluid Functionalism?

A design philosophy where every animation serves a functional purpose — motion is information, not decoration.

How does fluid hover work?

The closest item to your cursor is highlighted before you click, reducing targeting errors.

Why spring physics?

Springs respond naturally to interruption — if a user reverses mid-transition, the animation adapts.

Is it compatible with shadcn/ui?

Yes. Your existing theme, radius tokens, and setup apply automatically.

How do I install a component?

One CLI command — dependencies and shared utilities resolve themselves.

Full article content is not available for inline reading.

Read the original article →

Just a few days ago, I wrote about how great Apple's advertising has been recently with its focus on human craftmanship. Now, I'm being forced to eat my own words after sitting through the reveal of the company's first new product line since the Vision Pro in early 2024.

After the whimsical delight of the embroidered Apple logo for the Welcome Home event invite, the launch advert for Apple HomeView is stressful viewing.

For those concerned that they don't have enough screens in their lives with only an iMac, MacBook, Apple TV, iPad, iPhone and Apple Watch, Apple HomeView is finally here. Coming in two formats: tabletop or wall-mountable, it offers another surface on which to check your messages, speak to your AI assistant, control smart devices and watch stuff.

The novelty is that it's designed with a shared experience in mind rather than being geared towards a single user. HomeView can recognise up to six faces and dynamically change its interface depending on who's standing in front of it and how far away they are.

Both models have a 7-inch Liquid Retina display, while the tabletop version combines an adjustable screen with a speaker base (and looks a bit like a G4 iMac, some think). The interface changes with viewing distance, showing simpler information when the user is further away and more detail when they approach. There's also a range of customisable faces.

HomeView will allow access to some general content for everyone, including conversational requests via Siri AI, but private Messages and Calendar content will appear only when an enrolled person is recognised and their paired iPhone or iPad is present and recently unlocked.

Apps supported include Apple Music, Apple Podcasts, Apple TV and various third-party media apps. There's a 12MP Center Stage camera for FaceTime, and Handoff allows some activities to be continued from an iPhone. Apple says expanded access to certain server-based AI features will require a separate subscription fee in the future.

The ad above shows a kind of Modern Family scenario with a sickly yellowish colour grade. The HomeView somehow makes a morning routine even more chaotic, and the ad seems to drag on and on, with every second making me feel more anxious than excited about the product.

Part of the problem is the characterisation of the family. Many people might be able to relate to them, but would anyone aspire to be them? They're so unlikeable, so stressed out over a morning routine, with the screen they drilled a hole in the kitchen counter to install becoming the centre of their world. If they're going to fight over a shared device, perhaps it would be better for each person to have their own screen to look at.

Maybe Apple should invent it: it would be like a portable digital tablet that could run apps, messaging and provide information about the weather. Oh... wait.

Both the tabletop and wall-mounted Apple HomeView models start at $299/£299. A mount for the later costs extra. Preorders are open on the Apple website in initial launch markets with delivery starting from 13 November, 2026.

It could be the ultimate smart home device for the family that hates each other.

Preorders are also open for the new Apple TV 4K and HomePod mini 2. Meanwhile, iPhone Duo pre-orders begin on Friday 16 October.

Jev: For the builders AND dreamers

TL;DR: We're releasing an improved Jev 1.14.0 along with experimental features for our dreamers: images, a new type, and longer context. Enter the danger zone at your own risk. 🔥

Our philosophy

A huge debate we’ve been having is on the balance between reliability and creativity.

To really care about the prod builders, we should be shipping models that meet our quality bar for intelligence and have been battle-tested to the limits of what we can do internally.

On the other hand, there’s no way that our team has a full understanding of what is needed to spark delight in the world and build sick sci-fi. We don’t want to impose our values of extreme reliability on those dreamers.

Our solution: share those dreamer-ready features under “danger zone” flags in our API headers. These are features that will be here to stay for future models, are (as far as we can tell) better than what exists out there, and will continue to get better by leaps and bounds. But as a warning, we think there is still way too much jaggedness for us to vouch for these for production use cases!

See these new capabilities as a continuation of what we’ve built over 2 years of stealth. They’re less ready for automation, but very fun to play with. They’re also an early glimpse into what we’re focused on making reliable enough to meet TypeSafe standards.

Composable Images

Not only are we enabling multimodality for future Jev models, we’re enabling it for the current Jev that you all know and love. Surprise: the Jev you all know and love (jev-1.13.0) was multimodal all along!

Image support is fully composable via a find-and-replace in assets. You can put them anywhere that our JSON interface supports: in the state, instructions, or even criteria! Play around with it, push the boundaries of technology, and let us know where Jev can improve.

As a warning, all models (Jev included) perform worse on multimodality over pure text. Breaking down questions into the simplest possible decompositions helps a lot.

TypeSafe-Danger-Zone-Images: true

Check out docs: docs.typesafe.ai/go/images

Types Beyond Decisions

We’re introducing another new type as well: box. It returns a single bounding box - no type errors allowed. It’s our first type that’s not purely meant for smart-if statements, and is a preview for our explorations beyond simple decisions into what else System One Models can support.

We made huge leaps and bounds on this one recently, so we’re enabling this only for jev-1.14.0 (spoilers).

Check out docs: docs.typesafe.ai/go/box

Size does matter, apparently 💅

We’ve heard you loud and clear on context lengths. Jev actually supports a much longer context length than we’ve enabled in production. We chose to limit it intentionally because Jev, like all models, is less reliable on longer contexts, and this constraint encourages more effective usage patterns in prod.

We aren’t your parents though, so while we strongly recommend limiting context length, you no longer have to:

TypeSafe-Danger-Zone-Long-Context: true

Check out docs: docs.typesafe.ai/models#context-length

Opt-in to help us

We love the community’s desire to help make models better! We’ve added an opt-in to allow you to help find model jaggedness with your queries. We will still not train on your queries, but this does make it easier for us to find issues and help debug your use cases.

jev-1.14.0

We’re also releasing a new version of Jev under jev-preview! Not meant to be flashy but instead be small, subtle improvements to everything. Most things should be a tiny bit better, but we expect the largest gain in multi-lingual support.

Model releases should generally also be strict improvements, but we’re well aware of how much developers hate updating their dependencies. jev-latest will be updated to jev-1.14.0 in a week. jev-1.13.0 will be available for a while.

We intend to give 60 days warning before jev-1.13.0 is formally deprecated, and 30 days for jev-1.14.0. We’ll release a deprecation policy prior to this.

Measuring Intelligence

As you all know, we dislike public benchmarks. However, we have our own internal best guesses on how to measure the form factor of intelligence that matters most for developers.

We’re trying something new out here: we’re going to treat you guys like adults and share this with you all.

Some nuance

• This measurement is our best guess and will continue to evolve. If ever we show you our internal numbers again, don’t expect them to be consistent.
• Don’t believe it entirely. Always favor your own experimentation and evals with your use cases.
• We’re including only open weight models because we guard our validation set very closely and are spooked out about the data policies of our model providers.
• This is likely biased towards our models. Feel free to ignore our model, and instead focus on the relative ranking of open source models.

What’s next

Lots of people love to ask what kind of sci-fi we’re releasing next. While we are working on some mindblowing stuff, expect most releases to be beautifully boring (like we think AI should be): small improvements across the board, better stability, and features for real deployment.

1) Trying again. TLDR: Everyone can win. Open-source, frontier labs, infrastructure, applications and most importantly the American people.

I understand all of the jitters about Anthropic losing share, real rates, spreads and second derivatives. Still think the most likely future outcome is a hybrid, multi-model future where almost everyone wins, open-weight tokens are likely the overwhelming majority of tokens, the dominant frontier model companies still do well albeit perhaps not the outcome that seemed possible in May and AI infrastructure/applications benefit immensely from the increased fragmentation, competition and lower margins at the model layer.

Lower margins at the model layer mechanically drive higher compute demand as long as someone is willing to pay for the training. And Jensen is obviously willing to be the “trainer of last resort.” As below-market compute contracts signed in 24/25 roll-off and reprice higher, cloud and neocloud revenue should continue to accelerate which will help address financing fears; I think we may be near peak external financing for this buildout. Noteworthy that using data from 3Fourteen Research, B200 clusters will generate a 37% IRR *if* their rental price declines 40% per year. Prices are still going up, but that IRR math is too ridiculous to print. 37% IRRs available at scale should get funded.

Re: this weekends controversy about MOUs, Anthropic’s $413 billion of real contractual commitments sounds scary. Reality is that it works out to roughly $31 billion of annual cloud spend, which is easily internally financed given their reported ARR, and $161 billion of spend at Broadcom which will be financed by Broadcom and private credit. Continue to believe fragmentation at the model layer is the best outcome for ordinary Americans and humans everywhere. As I have said before, do not trust any 5-20 individuals to make the best possible decisions about the direction of this technology. I want as many AIs as possible to maximize the odds that they encompass the diversity of human values across the world and richness of human variation. And I want an AI that shares my own personal values. Safety, freedom and the maximization of human agency through democratization, distribution and diffusion vs. centralization. History has spoken on the dangers of the latter path.

Finally, from my perspective anything that slows this buildout down is net positive over the long-term as the biggest risk and worst outcome is an *actual* overbuild. We are structurally short watts, wafers and increasingly capital. That is all good. God forbid we were still in ZIRP, TSM ramped capex to 3x their current plan and we weren’t short turbine blades: there would almost certainly be an overbuild and a bubble. That would be terrible. Slower and smoother for longer is better both for investors and for the world. A terrible cyber event is the most plausible near term bear case IMHO, but AI cyberdefense ramping quickly and I was happy to hear Sholto say that cyber probably becomes defense dominant in the near future.

Longer, threaded version follows:

It Ain't AGI Until H100s Go for $30 an Hour

Sparks of AGI

Greg Brockman claimed that GPT-6 Astra marks the start of the “AGI era.” We may be in the AGI era, but if AGI means an agent that can “outperform humans at most economically valuable work” it isn’t here yet. You’ll know it is when the price of AI compute jumps 10x from where it is now.

I’ve included a calculator below to visualize my assumptions of where the supply and demand of compute equalize based on different assumptions about the world. I’ll walk through them below.

Tokenomics with AGI Characteristics

In October 2026 you can rent an H100 for about $3.52 an hour. That’s up 18% on the year, which is surprising since old compute usually becomes less valuable over time. But it’s nothing compared to where prices are going as we start deploying real AGI.

Let’s do some basic math:

Supply: By AI 2040’s estimates, there’s enough compute today to run AGI agents that could replace about 95 million full-time workers.

Demand: The top 95 million earners in the world make at least $35 an hour. So an AGI that can really do one of these jobs must be worth at least $35 an hour. (Many white-collar jobs in today’s economy pay far more. That’s just the floor price!)

How much compute will one of these agents use? The most capable publicly available model today is Claude Opus 5.5, which tops the Artificial Analysis Intelligence Index. Run as a full-time agent, it costs about $8.21 an hour at unsubsidized API prices. The cost of reaching a given level of capability has been falling 5–10x a year, and we’re already close to the AGI threshold. So it’s reasonable to assume an AGI-complete agent will run on a hardware footprint no larger than Opus 5.5’s today.

Let’s further assume Anthropic spends half of that $8.21 on compute and is paying near-market prices. At $3.52 an hour, that’s 1.17 H100-hours for each hour of agent time. In other words, a production deployment can support about 0.86 agents running constantly for each H100-equivalent GPU in it.

If each of these agents becomes capable enough to truly replace a white-collar worker, but supply is still limited by the amount of available compute in the world, something has to give. We already know demand for these workers is high: at least $35 an hour. With supply fixed, either (a) compute gets rationed (frontier labs stop selling externally entirely), or (b) prices rise until supply and demand balance.

How far would the price need to rise? On the conservative assumptions above, to about $30 per H100-hour: 0.86 agents at $35 an hour. If we make less conservative assumptions, such as that Anthropic’s API margins are well above 50%, that AGI only requires the same hardware as a cheaper model like GPT-6.1 Sol, or that employers will pay what a worker actually costs them with taxes and benefits, you can make a compelling case that prices will rise as high as $75 per H100-hour.

Stratospheric GPU prices won’t be the weirdest thing we see on the way to AGI. But they may be one of the easiest to monitor and trade on.

Footnotes

1. Workers the compute could replace. AI 2040 estimates about 20M H100-equivalents of AI compute at the start of 2026, growing about 3.3x a year. That’s roughly 50M by October. Big AI companies spend roughly half their compute on R&D, leaving about 25M to serve customers. At 0.86 agents per H100, running around the clock (8,760 hours a year, against 2,000 for a full-time worker), that’s about 95M full-time workers.
2. The wage floor. US figures are measured: among people who worked at least 40 weeks in 2023, 27.9M earned $100,000 or more (about $50 an hour) and 75.8M earned $50,000 or more (about $25 an hour). I couldn’t find a global count. I estimate another 20–40M people outside the US earn $50 an hour or more, and assume the same shape below that. That puts the 95-millionth highest-paid worker at about $31–40 an hour. All earners stand in for white-collar workers here.
3. Cost per agent-hour. I assume an agent reads 1,000 input tokens and writes 80 output tokens a second, with 90% of input served from cache: 288,000 output and 3.6M input tokens an hour. At Opus 5.5 list prices ($20 per million output tokens, $0.20 cache reads, $5 cache writes), that’s $5.76 + $0.65 + $1.80 = $8.21.
4. Margins. This is a wildly conservative assumption. SemiAnalysis estimates Anthropic’s API gross margin is above 80%, and its margin across the whole business is in the mid-60s even including heavily subsidized subscription plans. Higher margins make each H100 more valuable. At 80% margins one H100 runs 2.1 agents. The fleet then covers about 240M workers, reaching down to a wage floor near $19 an hour, and an H100-hour is worth about $41.
5. Market prices. We don’t know Anthropic’s blended cost of compute, but based on its xAI deal it may in fact be paying above-market prices to secure large blocks of supply on short notice. That deal reportedly costs about $1.25B a month for Colossus 1, which by my rough math is $6–7 per H100-equivalent hour. If Anthropic is paying that much and still getting good inference margins, that makes my math even stronger.
6. “H100” is just a convenient price unit. It means compute that rents for $3.52 an hour today. Anthropic serves on TPUs, Trainium and newer Nvidia chips. A literal H100 may rise less than the newest chips.
7. The less conservative case. More agents per H100 means the fleet replaces more workers and reaches further down the wage ladder, so I lower the floor using the distribution in note 2. GPT-6.1 Sol ($2 per million input tokens, $0.10 cached, $10 output) costs $3.92 per agent-hour. At a 50% compute share that’s 1.8 agents per H100, a fleet of about 200M workers and a floor near $22 an hour. Employers’ full cost is about 1.43 times wages, since wages and salaries are 70% of private-industry compensation costs. Counting full employer cost, Opus at 80% margins gives about $58 per H100-hour, Sol at 50% about $57, and Sol at 80% about $75.

Ling-3.1-flash is a native hybrid reasoning model designed for complex, real-world tasks with 560 billion total parameters, 25 billion active parameters per token, and a one-million-token context window.

GTA VI is expected to be one of the biggest entertainment releases in history when it comes out on November 19. Its immediate predecessor is the best-selling nonmobile game of all time. The game's release is causing other entertainment companies to delay products until next year because they know GTA VI will suck up the planet's attention for most of November. It will almost certainly be the most expensive video game ever created. This article looks at how Rockstar Games built what is expected to be the biggest video game blockbuster of all time.

Most planetary scientists will tell you the heyday of Venus exploration was more than a generation ago. That is about to change.

Two NASA missions are on track to launch to Venus in the early 2030s. The first, known as the Deep Atmosphere Venus Investigation of Noble gases, Chemistry, and Imaging (DAVINCI), is now scheduled for launch in December 2030, a year earlier than previously planned. DAVINCI will send a parachute-assisted descent probe through the atmosphere of Venus, returning the first direct images of the planet’s surface since 1982. A second mission, Venus Emissivity, Radio science, InSAR, Topography, And Spectroscopy (VERITAS), will follow with a launch in November 2032 to map the Venusian surface by radar from orbit.

The European Space Agency is building another orbiter, Envision, for launch to Venus in 2032. The Venus armada might get started as soon as next year, when Rocket Lab aims to launch a privately funded probe to search for signs of organic material in the planet’s clouds.

That was the update last week at a meeting of the Venus Exploration Analysis Group (VEXAG), a confab of scientists whose research interests focus on the scorching world just sunward from Earth. Europe’s Envision orbiter will also launch to Venus in 2032. The Venus armada might get started as soon as late 2027, when Rocket Lab aims to launch a privately funded probe to search for signs of organic material in the planet’s high-altitude clouds.

None of this was guaranteed. Budget cuts imperiled one US mission to Venus and delayed the other, then forced NASA to pull out of a partnership with Europe to supply a key part for Envision. But scientists at last week’s VEXAG meeting struck an optimistic tone.

“We have some good news for Venus,” said Jim Garvin, a planetary scientist at NASA’s Goddard Space Flight Center and principal investigator for DAVINCI.

Rewinding the clock

It wasn’t easy for Venus to find its moment. Mars has hogged the spotlight for decades and remains the long-term lodestar for human spaceflight. The enigma of comets and asteroids lured the world’s space agencies into funding a series of robotic probes to study them. The icy moons of Jupiter and Saturn, some of which may hold the ingredients for life, are the latest hotspots for planetary exploration.

But Venus ventures closer to Earth than any other planet, making it the easiest world for a spacecraft to reach, if not land on. Venus is roughly the same size as Earth and may have once shared a similar climate, but something happened to turn it into an inhospitable world with an atmosphere made of nearly pure carbon dioxide, clouds of toxic sulfuric acid, and temperatures hot enough to melt lead, about 872° Fahrenheit (467° Celsius).

There’s still plenty to learn about Venus. The planet is less well mapped than Mars, and scientists aren’t exactly sure what triggered Venus’ runaway greenhouse effect. It is notoriously difficult to land on Venus, where the air pressure is 93 times that of sea level on Earth, equivalent to about 3,000 feet beneath the ocean’s surface.

NASA’s last mission to Venus was Magellan, which mapped the planet’s tortured surface with a radar instrument capable of peering through the atmosphere’s opaque acid clouds. Magellan launched in 1989 and ended its mission in 1994, revealing the broad strokes of Venus’ landscape for the first time, but with only enough resolution to resolve features the size of a football field or larger.

That left a lot to be desired. Japan and Europe launched orbiters to Venus in 2005 and 2010 with instruments tuned to study the planet’s atmosphere. But it took 30 years for missions to improve on Magellan’s map to win support from decision-makers at NASA and the European Space Agency.

The fortunes of Venus scientists changed in a big way in June 2021, when NASA and ESA selected three Venus missions for development in a span of just eight days. “The Venus community is absolutely elated and excited and wants to just get to work and see this happen,” said Ellen Stofan, a Venus scientist and under secretary for Science and Research at the Smithsonian Institution, in an interview with Ars at the time.

It turned out the fight for Venus wasn’t over. Less than two years later, NASA officials considered canceling one of the two NASA-led missions, known as VERITAS, amid budget pressures in the agency’s planetary science division. The money problems largely stemmed from delays and cost overruns with the Psyche mission, NASA’s first spacecraft to explore the metal asteroid.

The VERITAS project stayed alive, but just barely. At the same time, NASA’s Jet Propulsion Laboratory, the institution in charge of Psyche, VERITAS, and a host of other science missions, struggled through layoffs and the cancellation of NASA’s Mars Sample Return mission, which was set to become the most expensive project in the lab’s history.

The goal of VERITAS is to build on the legacy of the Magellan mission. The VERITAS orbiter, carrying a much better radar than Magellan, will vastly improve the resolution of maps of Venus, with a particular focus on regions with active volcanoes.

At the same time NASA slowed the development of VERITAS, it was committed to providing a radar instrument to the European Space Agency for the Envision mission, an orbiter with similar goals as VERITAS.

NASA’s contribution to Envision was put in question by the Trump administration last year, and Ars reported in September that ESA was moving ahead with a European-built radar for Envision after NASA pulled out of the agreement. Nick Lang, a program scientist in NASA’s planetary science division, confirmed this at last week’s VEXAG meeting, citing limitations in NASA’s budget. “That decision was not an easy one,” Lang said.

Acceleration shock

NASA’s planetary science budget remains under pressure. The Trump administration has twice proposed deep cuts in planetary science. Congress restored most of the funding for fiscal year 2026 and is expected to do the same in the 2027 budget.

Ending support for Europe’s Envision mission and canceling Mars Sample Return freed up enough funding for NASA to move forward with DAVINCI and VERITAS, and in some cases, buy back some of their past delays. DAVINCI is now targeting a launch in December 2030, a year earlier than the previous plan.

“Thanks to encouragement in early ’26 from our leaders at NASA Headquarters and the Planetary Science Division,” said Garvin, who leads the DAVINCI science team. “This is our new scenario for the most cost-effective and science-effective launch to Venus … What we’ve done now is accelerated launch by 54 weeks to a case where we have a very low delta-V trajectory that gets off the ground in late December 2030.”

Low delta-V is an important qualifier. This means the trajectory for DAVINCI will require relatively little energy, at least as far as interplanetary missions go. A flyby with Venus in early 2032 will set up DAVINCI to release its atmospheric entry probe for descent to the planet’s surface on October 3, 2032. The mission’s carrier spacecraft will continue zooming through the inner Solar System, and the revised trajectory should allow it to hold enough fuel in reserve to enter orbit around Venus when it comes back around again, something that wasn’t always assured, Garvin said.

The comeback of VERITAS is even more striking.

“VERITAS is now funded with a launch date of November 2032, so this is great news for us,” said Dan Nunes, the mission’s deputy project scientist at JPL. “We’re staffing up. We’re accelerating, not to the point of Jim’s acceleration [on DAVINCI], but we’re moving.”

VERITAS is targeted to launch the month after DAVINCI arrives at Venus. “It keeps building on momentum for Venus science, which is great,” Nunes said. “We’re going to arrive at Venus close to the middle of 2033, and we’re going to have a phase of aerobraking” before beginning routine science observations.

In recent months, engineers working on the DAVINCI mission completed experiments showing how the Venus probe’s cameras and instruments will capture data during its one-shot descent through the atmosphere. It will take about an hour for DAVINCI’s battery-powered titanium descent craft to travel from space to the surface of Venus, and all of its imagery and data must be relayed to Earth through a radio link with DAVINCI’s carrier spacecraft before it goes over the horizon.

“We have to optimize everything in the final eight to 10 minutes of descent as we come into the supercritical atmosphere to get this data back for all of us to work with,” Garvin said.

The DAVINCI team also recently subjected a prototype of the entry probe to the extreme temperatures it will see at Venus. The insulated sphere kept its internal components below 86° Fahrenheit (30° Celsius), meeting requirements. DAVINCI’s managers at Goddard reported results from these two tests to NASA Headquarters and Administrator Jared Isaacman, and to the White House, Garvin said.

The news is similarly positive for VERITAS, according to Nunes, who credited scientific advocacy groups for the mission’s survival.

“VERITAS is going,” Nunes said. “We would like to thank all the support from the community, from VEXAG, the Planetary Society, all those people that went out there and made the case for Venus exploration for keeping VERITAS alive and getting us a launch date, so we’re very thankful for that.”

The work that Silicon Valley's AI labs are doing could reshape entire economies, create vast fortunes, dominate future warfare, and potentially alter the geopolitical power balance. This makes the industry ripe for espionage. While the companies have been on guard against the threat, there's no such thing as a perfect defense. This article explores the different ways corporate espionage is happening in the AI industry.

OpenAI narrowed Anthropic's lead in business AI adoption in September. Today, more than half of businesses pay for AI, and many of them pay for multiple models. The growth in new AI customers has slowed down significantly. Growth in the market will now have to come from more intense adoption rather than more customers.

Cold-start natural-language search often lacks both real queries and relevance labels. Airbnb's CASTLE jointly generates synthetic queries and labels from de-identified booking sessions. Its queries matched real guest query patterns much better than baselines, and human reviewers agreed with its relevance labels >90% of the time. Carefully governed behavioral data can bootstrap search training before meaningful traffic exists.

GitHub built a star history API that returns daily counts without exposing users' identities, cutting React's full history from about 2,500 requests to 24. It reduces database work by filtering stars by date before counting them, rather than repeatedly processing every star.

This proof of concept shows how to add row-level security to Claude Dashboards using an MCP server to authenticate users and query DuckDB for only the data they can access. Access rules are enforced before data reaches Claude, allowing the same dashboard to show different results for each user.

Stop rebuilding what production already built. Change-aware SQL pipelines with all state in the warehouse.

SQLBuild is a SQL-first framework for building reliable warehouse pipelines. It points at your existing dbt project and makes your builds change-aware: it fingerprints your models, skips the ones that have not changed, and can reuse already-built tables from production instead of rebuilding them in dev. No SQLBuild models, no migration, no edits to your dbt files.

It is also a full standalone framework. All state is persisted as append-only tables in the warehouse alongside your data: no external state database, no manifest files, no paid add-on. It keeps a low, dbt-like floor for SQL models and adds ingestion, Python nodes, testing, and opt-in virtual environments as your project grows.

Key features

• Works with your existing dbt project. Point SQLBuild at a dbt project and get change-aware builds and production reuse with zero SQLBuild models. It reads the manifest and drives the dbt CLI as a subprocess; it never edits your dbt files.
• Reuse from production. Clone or copy already-built relations from another target, or from a production-shaped git branch, instead of rebuilding them. Zero compute for models that match.
• Change-aware builds by default. Models, seeds, functions, and Python nodes are fingerprinted, source freshness is tracked, and unchanged work (including audits that already passed) is skipped. Pass --force to run everything selected.
• Warehouse-native state. All change-tracking state lives in append-only tables (_sqlbuild_fingerprints, _sqlbuild_source_freshness, _sqlbuild_node_results) in your warehouse schemas. No external state machine, no corruption risk.
• Audits that block bad data. Audits run before data reaches the target table. Full table builds materialize into a staging table and only promote if audits pass; incremental models validate each batch before DML.
• SQL-first models, compile-time validation. Define models as SQL files with MODEL() headers. SQLBuild resolves references, validates SQL, infers columns, checks contracts, and computes column lineage before anything runs, all offline.
• Cursor-based incremental processing. Automatic gap detection and resume, with microbatch mode for large ranges. No external checkpoint to maintain.
• Ingestion and Python nodes. Load external data with Python @loader functions, and run @task, @asset, and @check nodes as first-class members of the same DAG as your SQL models.
• Testing. Chained SQL unit tests that resolve intermediate models automatically, plus end-to-end scenarios with local DuckDB replay for fast CI with no warehouse.
• Python you can read, Rust where it counts. The framework is Python. For SQL parsing, validation, column inference, lineage, and transpilation, SQLBuild uses Polyglot, a Rust reimplementation of SQLGlot's SQL analysis capabilities (MIT).

See the documentation for the full feature set, including providers, lifecycle hooks, Python macros, UDFs, custom materializations, data diffs, zero-copy cloning, and virtual environments.

Works with your existing dbt project

Point SQLBuild at a dbt project and run a sqb dbt command. The first time, it bootstraps a minimal twin project from your dbt_project.yml and profile (reusing your dbt connection), then builds your selection with state recorded:

sqb dbt build --select path:models/marts

Run it again and the models that have not changed are skipped:

dbt (3 selected resources)
  planned models: 0 run, 3 current, 0 blocked
  skipped: all planned dbt models are current

With reuse configured, branch builds clone unchanged tables from a production git branch instead of rebuilding them, so dbt only builds what your branch actually changed.

Quick start

pip install sqlbuild
# or
uv pip install sqlbuild

Create and run the included playground project:

sqb playground waffle-shop
cd waffle-shop
sqb plan
sqb build
sqb test

Example

A model is a SQL file with a MODEL() header and a SELECT. References use __ref() and __source(), and configuration, schema, and audits are declared inline:

MODEL (
  materialized table,
  columns (
    order_id (audits [not_null, unique]),
  ),
  tags [marts],
);

SELECT
  o.order_id,
  o.customer_id,
  p.amount_cents AS total_cents
FROM __ref("stg_orders") o
JOIN __ref("stg_payments") p USING (order_id)

A unit test mocks sources and asserts on the model, resolving every intermediate model automatically:

TEST();

WITH
__source__raw__orders AS (
  @mock_orders()
),
__source__raw__payments AS (
  SELECT 1 AS payment_id, 1 AS order_id, 1500 AS amount_cents, 'credit_card' AS method
),
__expected__fact_orders AS (
  SELECT 1 AS order_id, 100 AS customer_id, 1500 AS total_cents
)
SELECT 1

Supported adapters

ClickHouse, Redshift, Trino, Spark, and Athena are on the way.

Documentation

Full documentation is available at docs.sqlbuild.com.

Contributing

We welcome contributions. Please see CONTRIBUTING.md for guidelines.

License

SQLBuild is licensed under the Apache License 2.0.

KEY TAKEAWAYS

• Getty Images will go private after securing a restructuring deal.
• The Getty family and Koch Equity Development will invest $533 million.
• The restructuring involves a prepackaged Chapter 11 process to complete by year-end.

NEW YORK — Getty Images said on Monday it had secured support from major creditors and investors for a comprehensive restructuring that will cut debt, raise fresh capital and take the visual-content provider private.

The company said affiliates of the Getty family and Koch Equity Development committed to invest up to $533 million of new equity under the restructuring support agreement.

Holders of Getty's unsecured bonds agreed to retire $270 million of notes, while secured lenders agreed to reduce interest costs and extend debt maturities under the restructuring plan.

Upon completion of the restructuring, affiliates of the Getty family and Koch Equity Development will own 100% of the reorganized company, Getty said.

Getty plans to implement the transaction through an expedited prepackaged Chapter 11 bankruptcy process involving certain U.S. entities and expects the restructuring to be completed by year-end.

The announcement comes months after Getty abandoned its planned $3.7 billion merger with rival Shutterstock, after Britain's Competition and Markets Authority required Shutterstock to divest its editorial business as a condition for approving the transaction.

Emerging Patterns: Design Leadership in 2026.

The practice and profession of design has changed and grown dramatically over the past two decades. As design has become more deeply embedded in complex, technology-driven organizations, designers have stepped into increasingly senior positions of influence, often without a clear roadmap for what those roles require or where they might lead next.

During the spring of 2026, students in my Design Leadership seminar at the Sam Fox School of Design & Visual Arts at Washington University in St. Louis set out to better understand this evolving landscape. The course was part of the newly launched Master of Design for Human-Computer Interaction and Emerging Technology (MDes) graduate program. The students and I were interested in the experiences that have shaped today’s design leaders: how they found their way into the profession and into leadership, how they build influence across organizations, and how they are thinking about the forces reshaping their work and careers.

Our research included nearly 120 survey responses and 11 in-depth interviews with design leaders from a global range of organizations and backgrounds. What emerged was not a single portrait of design leadership, but a collection of patterns—around mentorship, career paths, changing ways of working, technology, and the enduring human qualities that enable leaders to thrive through change. Emerging Patterns: Design Leadership in 2026 brings together the quantitative findings and the voices of the leaders we interviewed to explore five of the themes we found most compelling.

This work would not have been possible without the generosity of the leaders who shared their experiences with us, or the curiosity and commitment of the students who shaped the research and made sense of what we learned. My hope is that this report offers a useful snapshot of design leadership today, and prompts new questions about what the profession, and its leaders, might become next.

The cohort for this Design Leadership seminar included students from a diverse range of academic backgrounds, spanning Journalism and Economics to Anthropology and Engineering. Naturally, we became curious about the “origin stories” of today’s design leaders. What first sparked their interest in design? What did they study in college? How did they enter the profession? The answers turned out to be one of the most surprising and revealing findings of our research.

Across our survey of 119 design leaders, we mapped 93 distinct pathways into the profession—suggesting that there is no single, predictable route to a successful career in design. Fewer than 30% of respondents had studied in a dedicated design program at university, while more than one in five had no formal design education at all. Nearly 40% began their careers in another profession or industry, and almost none reported entering the field through an internship. Rather than following a well-defined pipeline, today’s design leaders have arrived through an extraordinary diversity of experiences, disciplines, and opportunities.

Our qualitative interviews reinforced these survey findings and helped us connect the dots between these diverse beginnings and eventual design leadership roles.

"I got my bachelor's in English literature, and I had no idea I would be doing in-depth user research. But looking back, I can see, oh, there were a lot of skills that I gained and that I use every day now."

Some leaders realized their passion for design retrospectively after working in other fields:

"I realized that I was focusing more on presenting the project and not really on the calculus. Then I said, ‘Oh, maybe all the physics and math I've studied wasn't really what I really wanted to do.’"

These findings also reflect a profession at a formative and pivotal stage in the formation of design as we know it. Many of the leaders we surveyed entered fields like UX, service design, design strategy, and DesignOps before dedicated curricula, established career ladders, or even common job titles existed. Unlike long-established professions like law, medicine, or engineering, design is still defining itself. As a result, talented people found their way into the profession from journalism, psychology, engineering, architecture, business, and countless other disciplines. As design education and career pathways continue to mature, we may well see future generations follow more standardized routes into the profession. But our research captures a unique generation of leaders whose careers were shaped during a formative period in the evolution of design itself.

Our cohort was interested in examining the early career experiences and influences that have shaped this generation of design leaders, and we asked specifically about mentorship. We know from other professional disciplines that active early-career mentorship can have a dramatic effect on the career trajectory of future leaders by opening doors and providing access.

In contrast, the design leaders we surveyed told a much different story. Our research shows that only 21% of design leaders report having the support of a strong mentor in their early career, while more than 41% report having none at all. This suggests that many of today’s design leaders reached positions of influence largely through persistence, self-direction, and informal learning rather than structured guidance. While that resilience is admirable, it also exposes a significant weakness across the profession.

This data was echoed in many of the discussions we had with design leaders.

"I started alone. I learned the hard way. I learned by making many mistakes and improving myself."

"Some things helped, and some things I was kind of making it up."

An additional factor exacerbating this gap is the weakening of the professional communities that once helped fill it. Organizations such as AIGA and IxDA historically served as important bridges between emerging and established practitioners, creating opportunities for mentorship, networking, and informal career development that extended well beyond the workplace. Today, many of those communities have diminished in reach and influence.

"I was researching groups last night. There's not a ton. And so I'm just gonna build my own."

Likewise, the rise of remote and hybrid work, accelerated dramatically by the pandemic, has created new challenges for mentorship. Early-career development often happens in the informal moments between meetings: overhearing conversations, observing leaders in action, asking quick questions, or simply building trust through day-to-day proximity.

There is, however, an encouraging signal in our research. While relatively few of the leaders we surveyed credited mentorship as a defining part of their own early careers, 66% said they now mentor or coach other designers outside of their primary role. Through teaching, speaking, advising, portfolio reviews, community organizations, or informal coaching, many are choosing to invest in the next generation of designers.

"You build confidence when you build more people who are gonna be supporting you. You're good as the people that you're leading. So, the first thing you have to do is find the supporters."

Perhaps because they experienced the mentorship gap firsthand, today’s design leaders appear determined to ensure that others don't have to navigate their careers alone. If that momentum continues, mentorship may become one of the defining characteristics of the next generation of design leadership—not an occasional act of generosity, but an expected responsibility of the profession.

Of course, this is an especially uncertain moment to be starting a career in design. After a decade-plus of expansion, the industry is now experiencing a prolonged contraction, with layoffs affecting many established enterprise design teams and programs. AI is accelerating changes to the practice itself, forcing organizations to rethink everything from methodologies to workflows, and from team structures to hiring priorities. It is against this volatile backdrop that we began asking a simple question: What does a successful career in design actually look like anymore?

One of the clearest signals emerging from our research is a shift away from the traditional, linear career path toward a more diversified and self-directed model. When asked about their intended career trajectory over the next five years, fewer than half of respondents (49.5%) indicated they plan to continue on their current path. That alone suggests a meaningful break from the established assumption that design careers progress in a steady upward arc—from individual contributor to manager to executive. Instead, many designers are actively reconsidering not just where they work, but how they work.

What’s taking shape in its place is a more fluid, diversified approach to career building, what many describe as a “portfolio career.” Designers are exploring a mix of roles and modes of engagement: frequently operating as digital nomads, taking on fractional leadership roles, founding startups, stepping into advisory positions, teaching, or building independent consulting practices. Rather than anchoring their professional identity, and livelihood, to a single organization, designers are distributing their time, expertise, and income across multiple contexts. The “job” is no longer the unit of measure—the “practice” is.

"I'm moving away from pursuing executive positions and instead exploring independent roles in the startup world."

"I wanted to be that chief design officer. But I'm over that. Like I got to live for me… 'cause it was costing me a lot — my health, my sanity — and I'm off that now."

"When I see how the world of work is shifting, the ladder is gone. I’m building a portfolio career because like a good investor, it’s smart to diversify. If my time, skills and learnings are my valuable currency, I want to spend them wisely on things that matter."

In this context, diversification becomes a form of professional resilience for design leaders. By cultivating multiple income streams and professional identities, designers are reducing dependency on any single employer or role, while also expanding their surface area for opportunity and growth. The goal is to design a career that can flex, shift, and endure over time.

"I believe heavily in not staying tied to design to do design work. I think some of the best design work happens when you're not under a design title and you're not doing design work."

It was impossible for our cohort to ignore the pace at which AI and other emerging technologies are reshaping the design profession. Unsurprisingly, this became one of the central topics in our research.

We asked design leaders how much they expected their design tool stack to change over the next five years. An overwhelming 94% said they expect to see dramatic changes. This powerful response signals something much larger than a shift in software. It suggests that continuous change has become a core characteristic of design, and that leading through that change is becoming a defining responsibility of design leadership. As tools, practices, organizations, and business models evolve, designers must constantly rethink how they create value.

The implication is clear: the challenge is no longer simply learning the next generation of tools. It's learning how to lead in a profession where continual change has become the norm.

"It becomes less about the tactical use of AI, and more about asking strategically ‘how do we see this unfolding?’ This requires more judgment and more discernment."

"I do think in the dawn of these vibe coding mechanisms, where everyone can be a designer, everyone could push a product into code."

In other words, the challenge is shifting from technical proficiency alone to the ability to navigate ambiguity, evaluate emerging technologies, and make thoughtful decisions about how they should shape the practice of design.

While this level of change can feel destabilizing, it also creates opportunities for design leaders. Periods of technological disruption have historically rewarded those willing to experiment, codify what they learn, and help others navigate uncertainty. As new methods, workflows, and organizational models emerge, there will be growing demand for leaders who can develop practical frameworks, publish and teach new ways of working, build tools and systems, and help organizations translate technological possibility into business value.

As we synthesized our research, one theme surfaced again and again: today’s design leaders have built their careers by leading through change. New disciplines have emerged, technologies and tools have come and gone, organizations have scaled and evolved, and the boundaries of design itself have continually shifted. But the more closely we examined this pattern, the more we realized that adaptability alone doesn’t fully explain it. Instead, our research points to a cluster of related qualities—intuition, navigation, adaptability, resilience, and stewardship—that have long distinguished successful design leaders and may be even more essential in the years ahead.

01 · Intuition: Sensing what isn't yet obvious

For many of the leaders in our study, intuition was present long before they became leaders—or even designers. More than 70% did not begin on a design-specific academic path, instead coming to the profession from the arts and humanities, engineering, business, social sciences, architecture, and a wide range of other fields. Their journeys suggest an ability to recognize possibilities that weren't always obvious, or even fully formed yet, and to follow those instincts toward a profession that was itself still taking shape.

02 · Navigation: Finding a way forward without a clear map

If intuition helped many of these leaders recognize possibilities, the core quality of navigation helped them find their way toward them. Across the 119 leaders in our study, we identified an extraordinary 93 distinct career paths. Nearly 40% began their careers outside of design, and many entered the profession before clearly defined roles, career ladders, or leadership pathways existed. Rather than following an established route, they learned to continually orient themselves by recognizing opportunities, making course corrections, and charting a path forward even when the destination wasn’t entirely clear.

"Not because I earned some sort of title, it was just a necessity."

03 · Adaptability: Evolving as the conditions change

Navigation has helped design leaders find their way through uncertainty, but adaptability has allowed them to evolve as the conditions around them change. Throughout their careers, the leaders in our study have experienced the emergence of entirely new disciplines, technologies, organizational models, and ways of working.

"I've always kind of hired on potential, but I think adaptability — and people who are more generalists, people who do a lot of different things — is where I'd want to spend to see."

04 · Resilience: Sustaining momentum through uncertainty and disruption

The career journeys captured in our research were rarely smooth or predictable. Nearly 60% of respondents cited a lack of growth opportunities as a major career setback, while nearly half (48.7%) had experienced layoffs or industry contraction. An equal number identified burnout as a significant challenge in their careers. And when stepping into their current roles, respondents rated whether they were sufficiently trained or onboarded at an average of 2.8 out of 5 — fewer than one-third felt they had been sufficiently prepared.

"It feels very similar to AI right now."

05 · Stewardship: Shaping what comes next

Perhaps the most encouraging signal in our research is what design leaders are doing with the experience they've accumulated. More than two-thirds of respondents report actively investing in their own career aspirations, while more than half are actively developing successors and next-generation leaders within their organizations. And 66% reported mentoring or coaching other designers outside their primary role. These leaders are not simply drawing on intuition, navigation, adaptability, and resilience to sustain their own careers, they are using those qualities to create opportunities for others and help shape the next generation of the profession.

"How can we set everyone up for success and get everyone what they need to do really good work here?"

"Nobody teaches you how to manage a team, and how to grow and develop people."

"I think a lot of people kind of plateau when they're only ever seen as the doers or the makers, right? And then you're always kind of production, or you're always somebody's helping hands — and you're not leading."

Looking across our research, we were struck by an unexpected conclusion. The leaders we surveyed did not succeed because they followed the "right" career path, mastered a particular methodology, or built expertise in a specific tool. They succeeded by developing a set of enduring human capabilities: the intuition to recognize emerging possibilities, the navigation skills to find a way forward without a clear map, the adaptability to evolve as conditions changed, and the resilience to recover from setbacks and keep moving forward. And increasingly, they are putting those accumulated experiences to work through stewardship by investing in themselves and others to build a stronger foundation for the next generation of design leaders.

Full article content is not available for inline reading.

Read the original article →

Toolcraft is an open-source starter kit and UI library for building custom design apps with AI. Use it to create branded asset generators, explore new visual styles, and turn repetitive design tasks into reusable tools.

Experience planes big 4: the who, what, where and why of experiences

Recently, I was talking to a client and the issue of customer experience came up. They said “we do need to fix the customer experience.” Others nodded. And from that moment on, the conversation was about people, the customer: their frustrations, their expectations, their satisfaction scores. A useful conversation, no doubt about that.

But what struck me once more was what the client didn’t bring forward initially. For example, where these customers were when the frustration happened and what they were trying to get done. And finally linking it back to why the organization was offering the experience in the first place.

This wasn’t the first time. When people talk about experiences, they typically pick one view, one lens: the customer experience, the shopping experience, the brand experience, the home experience. And that initial choice of how you focus on an experience, you can think of them as experience lenses, is often made without anyone noticing. It does quietly shape the whole conversation that follows. It sets the broad direction of a potential design solution before it’s even started.

The Experience Lenses

Let’s look at a few examples of an experience lens:

• the Customer experience: we talk about the human element and who the customer is, what they feel, and if they are satisfied.
• Shopping experience: we talk about the store, the e-commerce website, product placement, the checkout experience.
• Home experience: we talk about the family, quality time with friends, the interior design of the living room, placement of things in the kitchen.
• Brand experience: we talk about the brand promise, value proposition and how the organization engages with customers, employees and partners.

All these conversations, these lenses, are useful. None of them are wrong. But each one is part of a bigger picture. And on top of that, then there is this: the person on the other side doesn’t necessarily experience things through your lens. They experience for example a different lens first, in a different sequence, or over a longer period of time.

So instead of choosing one specific lens, I’d like us to consider four Experience planes every time we consider the experience of a product or service.

The Four Experience Planes

In this framework, I propose three major planes for every type of experience lens: people, activities, and places. An easy way to think of this is as a PERSON, doing SOMETHING, SOMEWHERE. And the fourth plane: the organization’s reason for being there, their mandate, the value proposition, brand promise, and business model, and you get the four planes containing the experience lenses:

The customer lens lives on Who, the shopping lens on What and Where, the home lens on Where and Who, and the brand lens on Why. The point isn’t to throw away your lens. It’s a “yes, and.” Keep it, and then ask what’s on the other three planes?

On each plane, one or more experience lens guide our thinking. What we place on each plane in term of design solutions – that’s when the four experience quadrants come into play:

• Brand: the promise to your audience
• Content: the information, language, data, and wisdom shared
• Product: the thing that has to work (digital or physical)
• Service: the end-to-end design including all of the above – following a red thread

Example 1: Renewing my driver’s licence (government service)

• Who: A current driver. This is similar to a customer; it’s the person we need to engage in to trigger the renewal. Usually that is through a reminder post card or email. They quickly become a user trying to finish a task. Within that role there’s variety: the commuter who renews in five minutes, the senior with a new medical condition, the newcomer who isn’t sure what’s different.
• What: Renewing their licence, but also the smaller tasks and journeys inside it: checking eligibility, finding the reminder, filling out the forms, paying, sometimes getting a new photo.
• Where: The kitchen table with a laptop, or a service centre in the municipality with a queue. Two very different places, one goal, different experiences.
• Why: Government mandate: road safety and verified identity, delivered efficiently.
• Experience quadrants:
  • Brand sets the expectation of a government that is well run and respects your time.
  • Content is a huge part of the experience: whether people understand requirements.
  • Product is the online form, the payment, the temporary license you receive in the service center.
  • Service includes the connectors between online and in person.

Example 2: Renewing my home mortgage (financial service)

• Who: Not one person but heads of household, often two partners and sometimes a broker.
• What: Comparing, deciding fixed or variable, re-qualifying, signing. Underneath sits a bigger activity: re-planning the household budget.
• Where: For one, the home, very literally. The asset being financed is the place where the decision gets made, often late in the evening at the dining table, as well as at the bank branch, through the mobile app, and/or a phone call.
• Why: The bank’s promise of trust and partnership.
• Experience quadrants:
  • Brand is whether “we’re with you for the long term” survives past the renewal letter?
  • Content is providing plain-language payment scenarios and explanations.
  • Product is the comparison online tool, the office to meet the broker or financial advisor, the stack of papers to sign and increasingly the e-signing tools.
  • Service connects the branch, the advisor, and the app.

Example 3: Ordering food from home (shopping mobile app)

• Who: The hungry person holding the phone, but is often not alone.
• What: Deciding (the hardest part), ordering, paying, waiting, receiving, eating together, and later reordering or reviewing.
• Where: The couch, not “on the go.” Mobile may be defined as “on the go,” but what if the app is most frequently used on the couch?
• Why: The brand promise is about convenience and a good evening, not about food logistics.
• Experience quadrants:
  • Brand sets the tone: Friday-night treat, or functional fuel?
  • Content covers mobile app descriptions, menus, photos, dietary information.
  • Product is the user experience or app including group ordering and the tracking screen showing progress.
  • Service connects app, restaurant, and courier experiences into one red thread.

Example 4: Youth mental health support addressing isolation and loneliness in a city

• Who: A young person, say 19, recently moved to the city for school or work.
• What: Not “accessing services.” The real activities are recognizing not feeling right, searching privately, taking a tiny first step, showing up somewhere, and coming back a second time.
• Where: The bedroom with a phone at 1 a.m. The commute. The campus. And the places the city offers, or doesn’t.
• Why: The mandate can be wellbeing and belonging, measured by connection.
• Experience quadrants:
  • Brand has to feel like an invitation, a gentle nudge, not a diagnosis.
  • Content speaks the young person’s language.
  • Product is the low-barrier digital entry point.
  • Service is the connector from screen to place to people.

Example 5: Unboxing and setting up a new phone (physical/digital product experience)

• Who: The customer at purchase, who then becomes a user within minutes and loyal client over time.
• What: Buying, Unboxing, setup, transferring data, getting email working again.
• Where: The store counter, then the kitchen table or living room at home.
• Why: The brand promise that drove the purchase.

Expand your Conversation

So what changes in practice? Not your lens. Keep talking about the customer, the shopping, the home, the brand. Just don’t stop there. Next time you’re in a design conversation and someone says “the customer experience,” try asking four short questions:

1. Who else is in this experience, and what role are they in right now?
2. What are they really trying to get done, before and after “our” part?
3. Where does it happen, and how many places does it cross?
4. Why do we offer this, and does our promise hold up on the other three planes?

Then, for each plane, ask which experience quadrant it needs the most: brand, content, product, service.

It feels messier at first. But experiences don’t come in lenses. People live them on all four planes at once. If we want intentional, connected experiences, our conversations need to cover all four too.

How two simple dots gave Pictoplasma a fresh personality for 2026

andstudio gives Berlin's character design festival a brand new look, built entirely from type.

From childhood, most of us will be familiar with seeing a man in the moon. When you analyse it logically, there's nothing in our nearest astronomical neighbour that looks remotely like a face. But our brains are wired to see faces in everything, from a power socket to the front grille of a car to a house with two windows and a door.

It's a phenomenon known as pareidolia, and it came in handy recently at andstudio, a creative agency based in Vilnius, Lithuania, which has developed a new identity for Pictoplasma, the Berlin festival of character design.

If you've never heard of Pictoplasma, we heartily recommend attending, and you can learn more by reading our special report from this year's edition. It actually began life in 1999 as an online archive of character design, before founders Peter Thaler and Lars Denicke launched a conference in 2004, which later expanded to New York and London.

Pictoplasma is about the artists and their characters, so it's always kept its own branding firmly in the background. andstudio's challenge in 2026, then, was to breathe new life into the identity, without taking focus away from the work it aims to showcase. A tricky balance to strike, and no mistake.

A meeting of minds

The two teams met at a conference for conference organisers, which that year happened to be in Vilnius. A dinner led to further chats, and before long a working relationship.

The studio was initially only planning to work on strategy rather than identity. And there was good reason to start there: with cultural budgets in Europe getting smaller, and festivals losing funding every year, Pictoplasma wanted to reach more people without losing the playful, offbeat spirit its community loves.

To kick off their research, the andstudio team travelled to Berlin to experience the festival first-hand. The thing that struck them most was the strong sense of community. At one talk, for instance, a speaker was so nervous he could barely stand on stage. But every couple of minutes the audience cheered to keep him going. For the visiting team, this moment summed up exactly what the festival is about.

After days of discussing character design, project manager Rugilė Jasiulionytė came back with another discovery: "I understood that characters are everywhere: in brands, in AI, in tech, in everyday life." She realised, in other words, that what she'd thought of as a niche festival was actually something more universal.

Strategic approach

Back in Vilnius, the team pored over visitor surveys and feedback, and found that the community was built on location, curation and content, with design playing almost no part. Strategy alone wasn't enough: Pictoplasma needed a new identity.

The strategy itself was built directly on Rugilė's insight. If characters are everywhere, the thinking went, Pictoplasma should be able to speak to far more people. So, working with the organisers, the studio set two goals. The first was to attract creatives from a wider range of fields, including motion designers, filmmakers, studios and agencies. The second was to extend Pictoplasma's geographical reach, and be recognised as a global festival across Europe, the US and Asia Pacific.

Internally, the team defined the event's role as making the idea of character accessible to more people. And for a festival that had kept its own face hidden for two decades, the positioning line wrote itself: "Never faceless".

Visual identity

"Never faceless" set the brief for the visual identity. But the team also respected why the brand had stayed neutral for so long. In this light, they felt a full character to the identity would be going too far in the opposite direction.

Instead, they went for a typography-led approach. The team chose Melun by Lift Type and worked with the foundry to adapt it. The typeface's rounded, heavy forms bring personality without competing with the artists' work on show. The custom version, which they call Pictoplasma Melun, also gains a "ct" ligature to smooth out the event's name and – here's the most important bit – adds a glyph for a pair of eyes.

The letters act as a body and the two dots as eyes, which means the entire identity can simply be typed. Nothing's drawn on top, so it never tips over into becoming a fully detailed character. It's pareidolia, in the form of type. And the system works across every city: colours and layouts change for each edition, while the type and the dots stay constant.

Motion design

Another thing absent in the previous identity was motion. Introducing it into the new branding, the team was again concerned not to overpower the artists' work. Motion designer Greta Griniūtė took that as a challenge: creating a sharp, energetic motion identity that plays with the type and two dots only.

Finally, andstudio worked with Dutch space designer Tony Michiels to find simple, budget-friendly ways to bring the identity into the event itself. For print materials, the rule is simple: put two dots on any object, and it becomes a character. Two holes cut into the festival tents, meanwhile, create a playful exchange between people outside and visitors inside. Elsewhere, a flash tattoo booth offers just one design (two dots), and dots stuck on objects around the city become a guerrilla campaign.

Ultimately, then, Pictoplasma is still about the artists and their characters. It's just that now, the festival has a face too.

The new identity launched with the 2026 Berlin festival, with tickets sold out six weeks before the event - much earlier than usual. For a festival working with less funding, that means more time to plan the next edition – a tangible result from a well-thought-through rebrand.

NASA's Perseverance rover has now driven 28.1 miles on Mars' surface, surpassing the distance that Opportunity covered from 2004 to 2018. Perseverance landed on Mars in February 2021 and has been traveling in and around the Jezero Crater since. It was set to surpass the driving record back in August, but it came across some interesting rocks, so it dawdled for two months. Perseverance most likely still has many miles to go in the coming years. This article contains some of the highlights from its drive so far.

The bulk of the cuts will be to Netflix's product, tech, legal, and human resources teams.

Google has been in discussions with publishers in recent months about developing an audiobook feature that will likely be bundled with its ad-free YouTube Premium service.

Apple has removed its redesigned Health app from iOS 27.2 beta 4, restoring the existing interface after allowing testers to try the overhaul in earlier betas. The redesign introduced features including Insights, Longevity, Health Age, and readiness metrics, but Apple had already indicated that it would arrive separately later in 2026. Data collected while testing the new version remains available through Settings, suggesting the redesign has been delayed rather than abandoned.

André Britz's Gone with the Clouds collects intuitive landscapes drawn entirely from memory in a 64-page publication whose loose pages and pink elastic binding deliberately remain changeable, mirroring how memories blur, shift, and rearrange themselves over time.