Introducing Grok 4.6

Grok 4.6 builds on Grok 4.5 with a particular focus on long-running agents and more ambitious interactive and visual work.

Today we are releasing Grok 4.6. Grok 4.6 builds on Grok 4.5 with a particular focus on long-running agents and more ambitious interactive and visual work. It stays with complex tasks across many steps, whether researching a topic, analyzing information, working across a codebase, or turning an idea into a polished application or work artifact.

Grok 4.6 achieves frontier intelligence across several agentic coding and knowledge work benchmarks. It matches GPT-5.6 Sol on the Artificial Analysis Intelligence Index, which is a composite score of nine benchmarks.

Grok 4.6 is available today in Cursor and Grok Build. We’re offering 2x included usage inside Grok Build and Cursor for the first week so you can start trying 4.6 immediately.

Training Grok 4.6

Grok 4.6 underwent a longer supplemental training run than Grok 4.5, with curated model-generated data for reasoning and advanced technical concepts, high-quality engineering data, and an improved optimizer and training recipe. This produced a stronger foundation for the SFT and RL stages that followed.

We then used Grok 4.5 to regenerate the SFT trajectories across reasoning efforts, agent harnesses, and domains such as STEM, software engineering, and knowledge work, and filtered out problematic traces with model-based checks. The resulting SFT checkpoint shows strong performance and improved behavior.

Grok 4.6 is trained on a wide range of agentic RL tasks, including knowledge work, general coding, and domain-specific environments for kernel optimization, web development, computer-aided design, and more.

Turning ambitious ideas into working projects

We tested Grok 4.6 on projects designed to stretch its range and ability to sustain work over many steps. We found the model is especially strong at turning a broad product idea into a working first version. It can research unfamiliar domains, structure the application, implement the core interactions, and continue refining the result through several rounds of feedback.

On longer trajectories, we also started to see more self-testing and verification, with the model checking its own work before moving on.

Grok 4.6 produces stronger first passes on visual and interactive projects than we typically saw with Grok 4.5. Given a concrete product idea, it is able to establish structure and visual language for an application in one pass. This has made it especially useful for projects where the fastest route to a good result was to begin with something substantial and then iterate in the loop.

Safety and capabilities

Grok 4.6’s safeguards have been improved and calibrated in line with the model’s capabilities.

Our safety stack is designed to maximize utility and security across legitimate use cases, allowing Grok 4.6 to be helpful and safe in domains such as vulnerability patching, accelerating the engineering design cycle, and augmenting AI research.

Our safeguard evaluation work reflects Grok 4.6’s expanded capabilities, with our widest-ever suite of pre-deployment testing for capabilities and safeguard calibration, as well as extensive post-deployment and third-party testing.

Evals

Get started with Grok 4.6

Grok 4.6 is available today in Cursor and Grok Build. It’s also available in the API and other partners like OpenRouter, Vercel, and Cloudflare.

Pricing starts at $2 per million input tokens and $6 per million output tokens. Additionally, there is a fast variant which is twice the price.

We’re offering 2x included usage inside Grok Build and Cursor for the first week so you can start trying 4.6 immediately.

Create an API Key

Start building with Grok 4.6 today via the SpaceXAI API.

API Docs

Read the docs and integrate Grok 4.6 into your stack.

Try it in Grok Build for free

Get started today at x.ai/build.

$ curl -fsSL https://x.ai/cli/install.sh | bash

How we tracked down a 16-year-old SQLite bug

At the end of last year, our uptime was pretty shaky. You can see this trend on our status page, and that instability continued into the new year. Many of these outages were caused by a single bug, deep in SQLite. It took months of intense forensics to track it down.

Now we’re in summer, we’re confident that we’ve found the bug, that we understand it—and more importantly, that we’ve fixed it.

We know our customers expect Tailscale to be a reliable service, and for several months we didn’t live up to that promise. That’s disruptive, and we’re sorry. We’re publishing this blog post to explain what went wrong, how we responded, and how we ultimately helped to uncover a long-standing bug in the heart of the SQLite database.

Tailscale’s database architecture

While our clients interact with our control plane as a single public endpoint (controlplane.tailscale.com), internally, our control plane is split into a series of coordination servers (or “shards”). Each tailnet lives on one internal shard at a time, but can migrate seamlessly from one to another. These shards are an internal implementation detail: you don’t know what shard your tailnet is on, and you never need to.

Each shard has an SQLite database that holds all the information about the tailnets on that shard. A single Go process exclusively accesses that database, and serves the control plane for those tailnets. This single-writer design is exactly how SQLite is meant to be used.

We’ve used SQLite as our primary database since 2022, and we chose it because it's well-known, reliable, and widely used. SQLite is “boring technology”—in a good way. Many companies use SQLite in much larger deployments without issue, and we expected the same stress-free usage.

In our current backup pipeline, we take a complete snapshot of the database every few minutes, then upload the entire SQLite file to an S3 bucket. We’d been running this setup without incident since early 2023.

Fast forward to August last year, when a data pipeline that reads those S3 backups reported an error in one of our databases. We ran SQLite’s PRAGMA integrity_check command against the backup, and found it was indeed corrupted. SQLite corruption is possible, but it’s highly unusual and not something you should encounter in normal operation. We repaired the affected database, and investigated the cause, but to no avail.

When operating at scale, even rare events can occur with some frequency, so we should have been unsurprised when it happened again—and again, and again, and again. In total, we faced 19 separate instances of database corruption over six months before we finally resolved the underlying bug.

When you hear the phrase “database corruption”, it’s natural to worry about data loss. Because our control plane only handles configuration data, these databases contain metadata about your tailnet and devices, but never your private encryption keys or network traffic. In the earliest incidents, the recovery process meant a handful of newly added devices or configuration changes didn’t persist, and a small amount of metadata had to be re-entered.

Whenever corruption occurred, we had to stop the control plane process on the shard while we repaired or restored the database. This was painful for tailnets on that shard, because their entire control plane disappeared during that recovery window. In the early incidents, that downtime was over an hour, but we gradually sped up the recovery process over subsequent incidents.

Each tailnet is a mesh network, where devices make peer-to-peer WireGuard® connections to each other. When a device joins the tailnet, it has to get a list of other devices from the control plane before it can establish new connections—so if a device came online during the SQLite downtime, it couldn’t connect. While the database was being repaired, devices already online remained connected to each other, but they couldn’t learn about changes to the network. Those tailnets also temporarily lost access to the web-based admin console and the Tailscale API.

There’s also a broader impact on trust. We post a global incident on our status page even when only a small number of tailnets are affected. Many people saw a status page event for an incident that didn’t affect them. Indeed, the majority of shards and tailnets were never involved in a database corruption incident! Nonetheless, repeated downtime erodes trust, whether or not you’re directly affected.

From the very first instance of corruption, we knew this was a serious threat to our reliability, and we threw a lot of engineering time at the problem—but the fix wasn’t easy.

Trying to find the fault

This bug resisted all our initial attempts to find it.

We looked at recent changes, but there weren’t any that seemed relevant. Nobody had been working on our low-level code that interacts with SQLite, because it had all been written years ago and presented no issues up until that point. We re-reviewed all of that code with a fine-toothed comb to look for previously missed bugs, but we didn’t find anything that would cause the corruption we were seeing.

We looked for common factors between corruption incidents, but we couldn’t find any. It wasn’t tied to a single shard, or customer, or tailnet feature, or time of day, or load level. We were at a loss for what might be triggering the behaviour.

This lack of reliable trigger conditions meant we couldn’t reproduce the bug synthetically. Instead, we had to rely on deploying passive, forensic telemetry in our live environment to catch the corruption red-handed. Gathering live diagnostics for a database issue is the last thing we wanted to do, but we had no choice.

As an additional complication, the corruption didn’t occur on a regular schedule. Sometimes incidents would be hours apart, other times weeks. This made it difficult to predict progress or plan further work, because we were never sure when we’d get our next diagnostic dump. We had a six-week period between October and December when there were no corruption incidents, before they returned as an unwelcome Christmas present.

Because this wouldn’t be a quick or easy fix, we reached out to the SQLite developers for a professional support contract. This was a great decision. It gave us direct access to their deep expertise and experience, and we had many detailed technical conversations about our architecture and our incidents.

Between Tailscale engineering and the SQLite core developers, we mapped out several theories for what might be causing the corruption—including broken POSIX locks on close(), mismanaging memory owned by SQLite, or accidentally using SQLite from multiple threads while disabling thread safety. After every incident, we gathered more data, added more diagnostics, and systematically ruled out these theories. We were gradually converging on the true bug.

The transactions that didn’t bark

While we were investigating the root cause, we still had a live platform to run. We took aggressive steps to automate recovery and minimize downtime:

• Configuring our control plane shards to hard-stop immediately upon encountering corruption
• Deploying an automated backup monitor that continuously ran PRAGMA integrity_check over our backups
• Improving our runbooks and on-call training

These efforts cut our response time to under an hour—and then we discovered an unexpected clue.

We wanted a way to restore service that didn’t involve rolling back to the last known-good backup (which would lose a lot of data) or repairing the known-corrupted database (which was potentially risky).

To do this, we built a transaction logging pipeline. We streamed every SQL statement that modified the database to a separate log file. Because SQLite is a single-writer database with serialisable transactions, our transaction history was completely linear and deterministic. (This wouldn’t be true in a multi-writer database like Postgres or MySQL.) Replaying those transactions against the latest known-good backup should restore the database to its most recent state, safely bypassing the corruption.

This pipeline worked, but then it did something even better: it gave us a clue.

In two incidents, our transaction logs failed to replay cleanly. Upon closer inspection, we discovered that data written and committed by one transaction was inexplicably invisible to later transactions. A write had vanished into thin air without raising an error. That should be impossible!

The writing on the WAL

As these incidents were ongoing, the SQLite developers had been developing a new debugging tool. For a while, we’d suspected that the bug was somewhere in the checkpoint process. They were building a new tool to give better visibility into what was happening during checkpoints.

To understand what this tool found, we need to briefly explain how SQLite checkpoints work.

A SQLite database is made of a series of “pages”, tiny blocks of information. When you update the database, some of those pages need to be replaced with new pages with the updated information.

For better performance and greater concurrency, we run SQLite with Write-Ahead Logging, which means new pages aren't written directly to the database file. Instead, they’re written to the "write-ahead log" or "WAL file".

New pages can't be written to the WAL file indefinitely; at some point they have to be copied back to the main database file. This process is called “checkpointing”.

In most deployments, SQLite itself decides when to do a checkpoint, and the process is invisible to the end user and developer. In our control plane, we take manual control of the checkpoint process so we can run fast and consistent backups. This non-standard approach seemed suspicious as we steadily eliminated potential causes.

One clue was that during corruption incidents, our metrics showed that SQLite would report copying more pages from the WAL file than were actually available. If there are 10 pages in the WAL file and 20 pages get copied to the database, something is clearly wrong.

To understand what was happening during these faulty checkpoints, the SQLite developers created a new debugging tool for the virtual filesystem layer.

SQLite is split into several layers. The top layer is the parser and code generator, which converts SQL statements into SQLite’s internal data structures. These data structures get passed to the pager, which splits them into the individual pages to be written to disk. Actually writing them to disk is handled by the OS interface, or “virtual filesystem”. Currently SQLite has two mainstream virtual filesystem implementations—Unix and Windows.

This approach allows you to replace different layers with different implementations, or wrap an existing layer to get more information. To help diagnose our problem, the SQLite developers created a wrapper around the virtual filesystem that writes additional tracing information and logs about changes to the database. This wrapper is called the tmstmpvfs shim.

We deployed the shim into our live environment, and waited for the next corruption to occur. Fortunately, we didn't have to wait long.

The WAL-Reset bug

After our next corruption incident, the additional logs from the new tmstmpvfs shim allowed the SQLite developers to find and fix the bug: a rare data race in the SQLite source code between a checkpoint and a write transaction.

In particular, if a write occurs at a specific time during a checkpoint, the checkpointing process gets confused—it thinks some of the pages have been copied from the WAL into the main database file, but they haven’t. Those pages never get written to the database file, and that data is permanently lost. The database file becomes corrupt, because other pages which reference those pages—such as an index—are written to the database.

The SQLite developers named this the “WAL-Reset bug”, and they estimate it was present in SQLite for at least 16 years. It could exist that long because it was rare—so rare, the SQLite developers had to add code to deliberately trigger it in their testing environments. Their fix adds an additional check to the checkpointing function which detects when the WAL has been reset by another thread.

They confirmed that this bug caused all of the baffling behaviour we’d seen. It explained the corruption, the transaction logs that wouldn’t apply cleanly, and the inconsistent checkpoint statistics. They also explained why we were more likely to hit the bug than other SQLite users: we take manual control of the checkpointing process, and we checkpoint very aggressively. Even a bug triggered by a rare condition was bound to hit us eventually.

This was an exciting moment. After months of confusion and uncertainty, we finally had a plausible theory for why the corruption was occurring, and a fix we could deploy to prevent it.

The SQLite developers released the fix as SQLite 3.52.0, and we prepared to deploy it as soon as it was available.

Fixed, with a false alarm

We rolled out SQLite 3.52.0 carefully—first to a few canary shards, then, when we saw it running smoothly, we deployed it to the rest of the control plane.

Our backup monitor promptly turned red, and reported corruption in 13 different databases. This was extremely alarming, but we followed our recovery procedures to fix all the supposed corruption, and everything was happy. It turned out these databases had not suffered real corruption, but were subject to a second problem in the version of SQLite.

We shared our errors with the SQLite developers, which uncovered a bug in SQLite related to stale expression indexes. If you create an index on a computed value, and then the computation changes, the index will contain mismatched values, which gets reported as corruption by PRAGMA integrity_check.

In our case, we were storing some high-precision timestamps as text, converting them to a floating-point number in a VIRTUAL generated column, and the SQLite 3.52.0 release that fixed our data race also made an optimisation that subtly changed the rounding behaviour for text-to-floating-point conversions. Our canary shards didn’t have any timestamps that triggered the changed rounding behaviour, so we missed this in our phased rollout.

Because this change caused false corruption warnings, the SQLite developers withdrew the 3.52.0 release and instead published 3.51.3, which only contained a fix for the WAL-Reset bug.

We fixed the issue on our side by reducing the precision of our timestamps to integer seconds; text-to-integer conversions are unambiguous. Meanwhile, the SQLite developers created an automated, self-healing index feature in 3.53.0, which prevents the stale expression index problem.

Party time!

With the fix rolled out to our entire control plane, we were ready to declare victory, but we were still cautious. An absence of corruption incidents doesn’t mean things are fixed—we’d already had one six-week period of deceptive calm.

We wanted positive proof that this data race was actively occurring in our production environment. Now that we understood the cause of the bug—a collision between a write transaction and a WAL-reset—we patched our SQLite driver to log a warning when these two operations overlap. If the warning fired but the database remained uncorrupted, we’d know the fix had saved us from a potential corruption incident.

We deployed the warning, and we waited. And we waited. And we waited. And we waited. As weeks slipped by, we began to wonder why we didn’t see it. Was the warning broken? Was our theory wrong? Was the true bug still lurking in the darkness?

Then, two months later, the alert we were waiting for finally fired:

This alert proved that the precise conditions for the WAL-Reset bug do occur in our production environment, which means it was the likely culprit for our six months of shaky uptime.

Since that weirdly joyous alert fired, we’ve run for another four months without any database incidents, as of this writing. Finally, we could breathe a sigh of relief.

Off the well-trodden path

Nobody wanted us to spend six months looking for bugs in SQLite. This was an immensely frustrating experience for both our customers and staff, and we’re all glad to put this instability behind us.

This investigation is a useful reminder: running boring technology in a non-standard way is a risk. The common paths and standard configurations are incredibly well-tested and reliable. Most people use SQLite in a standard configuration and never face this sort of issue. Everything we were doing was a public, documented, supported configuration—but by taking manual control of the checkpointing process and running at our own aggressive pace, we stepped off the well-trodden operational path.

Resolving these incidents was a massive, cross-functional effort involving dozens of people—including Tailscale's engineering and support teams, and the core maintainers of SQLite. It is to all of their credit that the impact of these incidents was not much worse.

We know that repeated downtime erodes trust, no matter how many people are affected, and we’re grateful to our customers for their patience and support while we chased this down.

Frustrating as this period was, we’re left in a stronger position than we were before. The long-standing bug in SQLite has been patched, and we fixed dozens of other incidental issues that we spotted while looking for it. We funded the open-source SQLite VFS shim that helped isolate the race condition almost immediately, and will help track down similar bugs in the future. Finally, we’ve refined our database backup and recovery processes, and live-tested them over a dozen times.

Hopefully there won’t be another database incident like this—but if there is, we’ll be ready.

GitHub vs Vercel vs Replit: What Dev Platforms Do When AI Code Is Cheap

AI models have solved the writing code part of software development to a great extent. Today, a capable model can produce a working function, a full component, or a small application from a plain-language description. It can do so in seconds for a fraction of the cost.

This change has shifted the economics of every developer platform. As the generation of new code becomes cheaper and widely available, it stops being the differentiating factor for a platform. This is the reason GitHub, Vercel, and Replit are trying to rebuild themselves around solving other hard problems in the software development process.

To understand what the three companies are doing, we trace one unit of work through every platform while asking the same questions:

• Where does the AI actually run the code it writes?
• How does each platform verify that the code works?
• How does the finished product reach production, and who is permitted to ship it?

Commoditization

A language model can now turn a description into working code. This means a developer can write a sentence and receive a function, a page, or a small application that actually runs right out of the gate.

This capability used to be the scarce and valuable part of a developer tool. Today it is widely available and close to free, which leaves a platform that offers only generation with little to charge for.

So what matters now is not raw code generation, but aspects of software development that come up after the code is available. Three questions carry most of the weight now:

• Where does the agent run the code it produces, and how is that environment kept safe?
• How does the platform confirm that the generated code actually works?
• How does the result reach production, and who is allowed to ship it?

GitHub, Vercel, and Replit answer these questions in different ways.

• GitHub puts its effort into coordination
• Vercel into the path to production
• Replit into verification

Orchestration

GitHub made a specific choice about where the value sits.

Rather than building its own model and competing on raw generation, it built a control layer that coordinates many agents and keeps their work governed, all inside the pull request workflow that developers use every day.

The mechanics start with where the code runs.

GitHub’s coding agent operates in its own ephemeral development environment, which is a temporary workspace that exists only for that task. It is powered by GitHub Actions, the same automation system that runs tests and builds on the platform. In practice, you assign a task to the agent the way you would open a ticket. The agent reads through the repository, edits files, runs the tests and linters (tools that check code for problems), and opens a draft pull request for a person to review. Each cloud run happens in a fully isolated, single-use Linux environment hosted by GitHub. Every new task starts from a clean workspace. The review step stays human, which keeps the existing quality gate intact.

A single agent sits above the coordination layer. GitHub calls it Agent HQ. It introduces a mission control view that lets a developer assign, steer, and approve work across a fleet of agents from GitHub and VS Code. The agents available inside a paid Copilot subscription include ones from Anthropic, OpenAI, Google, Cognition, and xAI.

Governance is treated as version-controlled configuration. Teams define custom agents through AGENTS.md files that carry rules such as a preferred logger or a required testing style, and a control plane gives administrators security policies, audit logging, and model-access controls in one place.

For developers who are already accustomed to handling issues and pull requests, this design adds agents to a pretty familiar workflow. Reviewing a colleague’s branch is not so different from reviewing an agent’s branch.

Routing to other companies’ models is a deliberate decision. GitHub is positioning the workflow, the execution environment, and the governance layer as the durable product, and treating the underlying model as a swappable component.

Production

Vercel starts from a different premise where code generation is assumed, and the design deals with carrying the generated code into production. This is the stage where most enterprise software work actually happens, because it involves existing applications rather than fresh prototypes.

The rebuilt version of v0, Vercel’s generation product, runs on a sandbox. It is an isolated space for executing code that imports a real GitHub repository and automatically pulls in the project’s environment variables and configuration. Every prompt produces code that fits the actual application and lives in the repository itself.

A Git panel handles the workflow around it. You create a branch for each chat, open a pull request against the main branch, and deploy when it merges. This means a product manager or a designer can ship through the same review process an engineer uses.

Vercel’s rationale about this approach is that AI-assisted building is already happening inside companies, and it has produced real failures. Incidents have been reported, such as credentials pasted into prompts, private data reaching the public internet, and deleted databases, often with the audit trail left empty. Therefore, wrapping code generation in real deployment controls is the right response.

Underneath the workflow is the execution layer. Every sandbox runs inside a Firecracker microVM, a lightweight virtual machine that isolates untrusted code. The reason for this isolation is that the code an AI wrote is code you have yet to review. Therefore, running it needs a boundary strong enough to contain mistakes. A microVM provides that strong boundary.

The billing model depends on how agents actually run. Vercel’s Fluid compute lets several requests share one running instance, with one using the processor while another waits on input or output. It charges for active processor time while treating wait time as free. Agentic workloads spend much of their time waiting on a model to respond, so this pricing matches the real work being done.

This approach deals with a frustration many developers face in their careers. The thing that worked in a demo behaves differently in production. However, Vercel’s design tries to close that gap by making the preview a real deployment from the start.

Verification

Replit concentrated its work on whether autonomously generated code genuinely works using a verification loop built into the agent itself.

Replit’s Agent 3 runs what the company calls a reflection loop. The agent generates code, runs it, tests the result, and repairs failures, repeating that cycle until the tests pass. This loop is reliable because of how the testing is done. Replit built a REPL-based verification system that runs code immediately and pairs that execution with a real browser it drives automatically, so it can click buttons, submit forms, and check data the way a user would.

The specific problem this approach targets has a memorable name inside Replit: the Potemkin interface. It is basically a feature that looks complete on screen yet fails the moment it is used. Catching that class of error is what allows the agent to run on its own for more than 200 minutes at a stretch, a large increase over the roughly 20 minutes of its predecessor.

The verification runs as its own process. A testing subagent follows a simple cycle of taking an action, observing the result, and repeating. When it finishes, it returns a summary to the main agent describing what works and what broke. This multi-hundred-step testing costs a median of roughly twenty cents per session and runs several times faster and more cheaply than relying on general-purpose computer-use models.

Interoperability

Every architecture we have looked at assumes its agent can reach tools and data that live outside the model, and doing that cleanly requires a common method. That method is the Model Context Protocol, usually shortened to MCP.

Before a standard like this existed, connecting several AI applications to several external tools meant writing a separate custom integration for each pairing. Anthropic introduced MCP to replace those fragmented, one-off connections with a single protocol, so each application and each tool implements the standard once and then works with everything else without additional changes.

A host, which is the AI application such as an IDE or a chat client, creates one or more clients, and each client connects to a server that exposes some capability. A server offers three kinds of capability:

• Tools, which the model can call to take an action, such as creating a record or running a query.
• Resources, which supply context data the model can read, such as a file or a database schema.
• Prompts, which provide reusable instruction templates.

The whole exchange runs over a defined message format across either a local or a remote connection. The effect is that a tool provider builds one MCP server and every compliant agent can use it.

Replit was among the earliest developer tools to integrate MCP, GitHub added an MCP registry to VS Code where a server can be enabled with a single click, and Stripe runs an official MCP server for its payment operations.

Tradeoffs

• GitHub gains breadth and governance by routing to many vendors’ models. But the cost is that it owns the surface rather than the intelligence underneath. Whether a coordination and governance layer stays valuable as models and agents keep changing is an open question.
• Vercel gains strong isolation by running generated code inside microVMs, and that isolation carries a cost per unit of compute. There is a question about whether the heaviest workloads eventually move to cheaper execution elsewhere
• Replit gains long stretches of autonomy through its verification loop, and the more work an agent does on its own, the more weight rests on that verification being right. The Potemkin problem stays difficult even with a capable tester, because some failures appear only in situations a test session might miss.

Conclusion

The pattern across all three companies is the same. Code generation became cheap, so the value moved into the engineering that surrounds it. Each company placed its bet on a different piece of that surrounding work.

GitHub bet on orchestration, building a control layer that runs and governs many agents inside the pull request workflow developers already use.

Vercel bet on production, wrapping generated code in real deployments and running it inside isolated microVMs built for untrusted code.

Lastly, Replit bet on verification, driving a real browser in a self-testing loop so an agent can work on its own for hours and still be checked.

Underneath all three, MCP provides the common protocol that lets any agent reach any tool, which is why every one of these platforms now supports it.

1 The horizon problem

AI is already good at automating quick tasks: ones where it gets feedback fast and doesn't lose track of what it's doing. The length of task an AI agent can reliably finish has been doubling roughly every four months through 2025. But even researchers say they can't reliably measure anything past 16 hours yet. Nobody actually has good data on long tasks. The real problem with long tasks is what researchers call the credit assignment problem: it's hard for the AI to figure out which of its earlier actions caused something to go right or wrong, especially when feedback is rare. Step count isn't the real limit either. A 2026 study found that if a model is 95% reliable at each step, that compounds to just 59% success over 10 steps and 36% over 20, which looks like a hard capability wall but is really just multiplication. Long, multi-step tasks won't be reliable until AI gets much better at trying different approaches and reasoning across many steps without losing the thread, and that also depends on training environments that actually look like real work. On a benchmark that simulates a real software company, even the best agents only finish 30% of tasks on their own. Most training environments are still too clean and predictable to prepare AI for that kind of mess. Nobody, including the labs shipping these agents, actually knows when long-horizon reliability arrives. Plan around that uncertainty instead of the marketing timeline.

2 Job losses vs. new company creation

Money that used to pay workers is shifting toward AI instead, and you can already see the gap between the big picture and the personal one in the data. Tracking shows jobs for 22-to-25-year-olds in the most AI-exposed roles shrinking about 3.8% a year, while the job market overall barely moves, down only about 0.2% a year. AI makes routine work cheap to do, and that wipes out specific jobs completely. If it's your job, the good macro numbers don't help you. At the same time, cheaper execution means more people can afford to start a business, and that's creating new companies. The defining labor market challenge ahead is labor reallocation, not creation. If your job is repeatable execution, you're exposed, full stop, no matter what the macro numbers say. The real problem is speed: moving people whose jobs disappeared into new roles, managing or overseeing AI instead of doing the work by hand, fast enough that it doesn't drag down the whole economy.

3 Specialized AI, not general AI

The future of enterprise AI is small models built for one job, running on a company's own servers. Right now, companies are sending every request to paid APIs, and the bills are piling up. Even though the price per AI request has dropped 98%, companies' total AI bills are tripling because AI agents chain together many requests to finish one task, and that eats up the savings. MIT initiative studies found 95% of companies using AI saw no real impact on profit, and surveys of CEOs found 56% unable to point to any real benefit yet. That's getting better: the number of S&P 500 companies that can prove AI is paying off grew significantly in a year. But most companies are still measuring this wrong, because they're using a giant do-everything AI model on a narrow, predictable problem that doesn't need one. Every company still paying frontier-model rates for a narrow, repeatable task is burning money it doesn't need to burn. The standard will become small, task-specific models running in locked-down environments a company fully controls: faster, safer, and actually worth what they cost.

4 Systems skills, not just code

AI can write code now, so just knowing how to write code stops being special. Research into millions of lines of changed code found the amount getting rewritten or thrown away is up 39%, duplicate code is up 8x, and actual cleanup work is dropping. Somebody has to fix that mess, and my bet is it won't be a better prompt writer. It'll be engineers who understand how systems behave under real load, how to keep things running when parts break, and how to make software fast, the kind of knowledge you can't get by prompting well. If your entire value as an engineer is writing syntactically correct code, AI has already replaced you. You just haven't been told yet. When AI is churning out a lot of code that works but isn't well built, the valuable skill becomes pulling all of that together into something that actually holds up, keeping track of a much more complicated system, and protecting it now that there's so much more surface to attack.

5 Testing what AI does, not what it knows

Standard tests like MMLU are maxed out and easy to cheat on. A large portion of these tests often leak into AI training data. None of that tells you how the AI actually behaves in the real world. Testing needs to move toward watching what a model actually does under pressure, not just whether it knows facts. Researchers have run joint tests across labs that watch for bad behavior instead of grading a fixed test. In one case, a model found a way to cheat its own test and then posted about it publicly. That's the kind of failure a multiple-choice test will never catch. A benchmark score is a marketing number at this point, not a safety signal. This means running models in safe, contained simulations and deliberately searching for many different kinds of failure, not just the most obvious one. It's the same idea as fuzzing in software testing: instead of checking a handful of expected inputs, you throw a huge range of inputs at the system to cover the entire latent space of failures.

6 Proving code is correct

Formally proving code is correct, mathematically rather than just testing it, never caught on before because it took too much human effort to be worth it. What AI fixes is that human effort, since it can do a lot of the proving work itself. But that moves the bottleneck rather than removing it. A proof covers what the specification says and nothing more, and writing the specification is still a human job. Even protocols with mechanized proofs suffer from bugs in areas not covered by their specifications. Code-level bugs don't disappear. They move out of the logic a proof covers and into the assumptions nobody wrote down. The teams that get burned will be the ones treating a proof as a finish line instead of a scope statement. The design-level version of this is using formal methods to verify systems, checking that the design itself can't enter a broken state before a single line of code gets written.

7 Memory size beats memory speed

Running large AI models is limited by memory, not raw calculation speed: the hard part is fitting hundreds of gigabytes of the model somewhere fast enough, not doing more math. Right now everyone's chasing this through HBM, a type of ultra-fast memory used in datacenter chips, which makes up a massive percentage of the cost to build AI systems. But that's all about speed in a datacenter. Another approach is to chase size over speed: running open models on systems with high memory capacities, which allows for larger models than typical datacenter cards at a fraction of the power. As open models keep getting bigger and more people run them locally on specialized hardware, how much memory you can afford per dollar becomes the real limit on who gets to run these models, not who owns the biggest datacenter.

8 Environments, not more data

The internet is running out of new text to train on. Trends suggest AI labs will use up the entire stock of human-written text in the near future. That's pushing the real bottleneck from data to environments. Instead of scraping more text, labs need places where an AI agent can try something, get a clear pass or fail, and learn from that. Training a model against automatically verifiable rewards across many environments, like math or code puzzles, makes it develop strategies that look like reasoning. That idea is now a real industry. Companies are raising massive amounts of capital specifically to build "Hugging Face for RL environments." The labs still measuring progress purely in parameter count are already behind; the ones measuring progress in environment quality are the ones worth watching.

9 US open models are catching up

Right now, the leading open-weight models are competitive with the best closed US models. There is a growing argument that the world needs both frontier closed models and frontier open models, warning against restricting open models before domestic industry catches up. The building blocks already exist: some institutions ship full training data, every checkpoint, and the training logs alongside the weights. Other frameworks provide the actual training and fine-tuning code. Independent developers are already fine-tuning models to score near-human-expert levels with minimal funding. Betting that US labs never catch up on open weights is a bet against how this entire industry has moved every single time before.

Meta has released Muse Glimmer, a 30-billion-parameter open-weight model built for always-on agents on a Mac or PC with a single consumer GPU. The model comes from Meta Superintelligence Labs, carries a permissive Apache 2.0 license, and is available now on HuggingFace. It targets developers building local agents, coding tools, function-calling systems, and model-based evaluation without relying on a constant network connection.

Introducing Muse Glimmer, an open-weight 30B-parameter model optimized for local, always-on agent workflows. Muse Glimmer delivers strong performance on key agentic use cases and benchmarks compared with leading models in its size category, and is designed to run entirely on local hardware.

Muse Glimmer is trained for end-to-end task completion, precise tool calls, multi-step reasoning, and recovery when a tool fails. A dedicated perception encoder lets it process interleaved text and images, including screenshots, charts, and documents. It also supports more than 100 languages, selectable reasoning strengths, and agent scaffolds such as OpenClaw.

Just like much larger models, muse glimmer can operate as a fully capable agent via planning, tool calls, checking its own results, and failure recovery.

Meta designed the model around the memory and compute limits of consumer hardware. Its training used logit distillation from Muse Spark outputs, followed by longer-context agent data, supervised fine-tuning, on-policy distillation, and reinforcement learning across general, reasoning, coding, and agent tasks. Meta says the model was evaluated under its Advanced AI Scaling Framework before the open-weight release.

A full-precision version would need more than 55 GB of memory. Quantization reduces the language model to under 20 GB, leaving room in a 24 GB or 32 GB memory envelope for its working memory, KV cache, image encoder, and speculative-decoding drafter. That lightweight DFlash-based companion proposes blocks of tokens that the main model verifies in parallel. Meta reports decode-speed gains of 3.1 times on an RTX 5090, 1.8 times on an M5 Max, and 1.5 times on an M4 Max for its K-Quant-17GB setup.

Today we're also opening the weights for Muse Glimmer, a great 30B parameter dense model that can run locally. Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model. Meta is a strong supporter of open source and I'm proud of these releases.

Meta positions Muse Glimmer against Gemma4-31B and Qwen3.6-27B, reporting strong results for its size across agentic and general language-model benchmarks. The model is intended for local work that can continue without cloud infrastructure, while still supporting deployment through larger serving stacks.

Weights and developer documentation are available now. Optimized support for llama.cpp, MLX, and ExecuTorch is due in the coming days, alongside planned access through Ollama, LM Studio, Unsloth, vLLM, SGLang, Together AI, Fireworks AI, and OpenRouter. Meta is also working with AMD, Arm, Dell, Intel, and NVIDIA on device-level optimization, extending its open AI research into local agent systems.

Anthropic to start watermarking Claude-generated text, images

Anthropic PBC has announced plans to embed an invisible watermark in text and images generated by Claude.

The Register reported the change today, citing a help desk article published on Monday. It applies to the Claude artificial intelligence model series and the Anthropic services that it powers.

The watermarking mechanism is designed to bring Claude into compliance with the European Union’s AI Act. The law, which went into effect in 2024, includes a voluntary clause known as the Code of Practice that Anthropic has signed. The provision requires model providers to mark AI-generated content.

Anthropic says text watermarking will be performed by Claude models released after Aug. 2. It plans to bring the capability to earlier models further down the line. According to the company, the watermark is invisible and persists if the text that contains it is copied to other applications. It may even remain in place if users make edits.

The company didn’t specify how the text watermark works. However, a research paper released by Google DeepMind in 2024 may provide clues. The paper describes a technology that modifies a language model’s word choices to make the text it generates detectable. Anthropic’s watermarking mechanism may use a similar method.

DeepMind’s watermarking mechanism supports not only text but also AI-generated images, videos and audio. The technology, which is known as SynthID, is integrated into several of Google LLC’s consumer AI services. Anthropic likewise plans to mark media files generated by Claude. The company’s Monday help desk article lists JPG, PNG and SVG among the image formats covered by the initiative.

Anthropic will mark images using a technology called C2PA. It works by pairing AI-generated images with a metadata file that specifies what model created them, when and whether any copyright restrictions apply. Developers can also add in other details.

C2PA includes multiple mechanisms designed to prevent hackers from tampering with images’ metadata. It creates a hash, or unique identifier, of each metadata file that makes it easy to spot edits. The technology can also detect when hackers attempt to replace the entire metadata file rather than edit it.

Anthropic plans to release tooling that will make it easier for users to detect Claude-generated content. However, the company cautioned that the technology won’t be perfect. In particular, it may not always detect watermarks embedded in short or heavily edited text snippets.

OpenAI Group PBC has also signed the AI Act’s voluntary Code of Practice. The provision went into effect last week, which means that the ChatGPT developer may soon launch its own text watermarking mechanism. OpenAI created an implementation of such a feature in 2024 and sits on the steering committee of the consortium that develops C2PA.

DeepSeek Prices Its New V4-Pro-0813 Model At $0.87 Per 1 Million Output Tokens, As The Chinese AI Lab Comes Out Second Only To Anthropic On Token Consumption

DeepSeek was second only to Anthropic in terms of the total number of tokens consumed in July. And now, perhaps in a bid to cement its ascendancy, the high-flying Chinese AI lab has just unveiled the DeepSeek-V4-Pro-0813 model, its latest gambit to take on the might of OpenAI and Anthropic.

The V4-Pro-0813 is now rolling out on DeepSeek API and DeepSeek Chat

As yet unverified DeepSeek V4 Pro benchmarks from WeChat. Seismic if accurate. https://t.co/fEOONbaFEO pic.twitter.com/CG5s8Q1PJJ

— Andrew Curran (@AndrewCurran_) August 12, 2026

DeepSeek has started rolling out the V4-Pro on its API and Chat. The AI lab has priced the model at $0.435 per 1 million tokens of input and $0.87 per 1 million tokens of output.

Do note that OpenAI launched a literal price war a few days back by discounting its GPT-5.6 Luna by as much as 80 percent, with input tokens now priced at just $0.2 per 1 million from their earlier perch at $1, and output tokens priced at just $1.20 per 1 million vs. the earlier price of $6.

Just hours later, however, DeepSeek launched a refreshed version of its latest Flash-class model, dubbed the V4-Flash-0731. Critically, the model has just 284 billion parameters and yet offers a performance that is similar to Anthropic's Opus 4.8, which is widely believed to span multi-trillion parameters!

And, in what went right to the heart of OpenAI's price war, DeepSeek priced the V4 Flash 0731 at just $0.14 per 1 million tokens of input, and $0.28 per 1 million tokens of output, eviscerating any comparative price advantage that OpenAI tried to garner with its discounting move.

DeepSeek silently released V4-Pro 0813, up 15.8% on Terminal Bench from their April Preview model, with Fable 5 performance at ~57x cheaper cost.

1.6T param, 49B active, 1M context. This is the best price-to-perfomance model on the market right now.

Available in ClinePass now! pic.twitter.com/D9yas0umPn

— Cline (@cline) August 12, 2026

Coming back, DeepSeek's V4-Pro model outcompetes Opus 4.8 on Terminal Bench 2.1, Cybergym, DeepSWE, and AutomationBench benchmarks, as per the preliminary results populating WeChat right now.

Amazing stuff here. DeepSeek was 2nd only to Antropic in token volume in July. This was b4 V4-Flash 0731 came out. I'd imagine it will be 1st for August.

Sure, it's behind both Moonshot & Zai in revenue, but that's a lot of data DS is getting for post training. If DS can keep px… https://t.co/QKyGBiYK38 pic.twitter.com/Ql9wnHq2nn

— tphuang (@tphuang) August 12, 2026

Meanwhile, as stated earlier, DeepSeek was second only to Anthropic in terms of token volume in July, and might even clinch the apex spot in the coming months.

As such, DeepSeek is currently contending with an unprecedented demand surge, especially amid anecdotes that suggest its models' inference speeds slow down to a crawl at times, which is wholly understandable given the lab's limited compute footprint of just around 20,000 NVIDIA H100 GPUs.

Specula is an agentic system that automates the process of software bug finding through authoring and model-checking a spec for the code. It derives TLA+ specifications automatically from the code, checks code-spec conformance through trace validation, model checks the spec to find concurrency bugs, and reproduces the bug at the code layer by writing integration tests with precise timing. This post looks at what Specula gets right, its major contributions, and unresolved questions about the terrain. Specula is a great pragmatic idea, and it works for what it does, but it still skirts the real hard problem of composition, so it cannot say anything about whether per-module guarantees add up to a system-level guarantee.

Hiring Agents Is the Easy Part

The hard part is managing, evaluating, and improving the agents. Verification, not capability, is the real constraint on automation.

Agents are becoming a much more ingrained part of how we do work. They expand the capabilities, capacity, and agency of each person within an organization. Many AI tools today focus on how to augment human workflows; we’re excited about much more work shifting toward agents that automate workloads, especially as domain-specific models lead to novel frontiers.

As we think about how to get from the status quo (largely augmentation) to the future (agents as automators), one analogy we’ve found useful is framing agents as akin to employees. With employees, there are tried-and-true processes for hiring, onboarding, and promoting people. It looks something like:

• Screening: testing whether the person is capable of doing the job.
• Onboarding: integrating the new hires into existing workflows and org structures, setting up access (controls) for internal tools, and broadly providing company-specific context.
• Ongoing performance reviews: assessing whether the work produced is actually good and providing feedback on how to improve.

The same can and will be true for agents. And to be clear, parts of the agent hiring infrastructure are already well underway in being built out. Offline evals are good at assessing whether a model or agent can complete a specific workflow (step #1, aka screening). Infrastructure for step #2 might take the form of something like Glean or Cognee – shared substrates that let agents easily access an organization’s tools and leverage company knowledge graphs.

Step #3 – assessing whether the work is “good” and providing feedback on how to make it better – is where we’re most interested. In Automate vs. Augment, we argued that verification of quality work was key to when a workflow can be automated. Evals are a technical implementation for how companies can assess that criteria. But, like with evaluating an employee, it's more complicated than simply "was the work completed?" So, some of the questions top-of-mind for us include:

How is “good” defined (and who defines it)? The first set of automators has focused on areas with easily verifiable work – i.e., was the job completed, yes/no? But in the future, it will have more nuance as quality verification gets more complex. The criteria may start to look like: was the job completed efficiently? Then, was the job completed to a company’s tacit standards (e.g., consistent with the intuitive, experiential knowledge within an organization)? We’re especially interested in work that has historically been hard to verify but is becoming easier as models improve.

How does “good” compound across agent lifecycles? Today, feedback on agents in production (“online evals”) largely guides agentic systems to fitted tasks. The loop looks like: a human logs an agent’s success or failure, labels it, scores it, and that feedback directs the agent toward improvement. But the eval lives as an artifact in an external, human-maintained database. Is there a world in which feedback can accumulate within the agent itself and more autonomously compound knowledge across the org? In other words, what would it take for agents to begin directing their own learning loops?

When an agent receives company-specific feedback, who owns that data? Palantir calls this “Sovereign AI”; we think of it as ownership as a keystone of the architecture (a key pillar in our autonomy thesis).

Does every company need its own eval layer? When does a company need to customize an agent to its specific processes vs. in what scenarios should we expect an agent to work out of the box?

If an agent within an organization goes rogue, where does the liability fall? Questions regarding internal permissions, liability, and access controls are core to building hardened autonomous systems.

The world 3-5 years from now will have a lot more agents in it than exist today. How we get from here to there has a lot of fuzzy questions – ones that we’ve encountered firsthand as we’ve built out our own agent systems, as we chat with portfolio companies about their incorporation of agents, and as some of our investments (e.g. Prism) tackle every day.

It’s an area we’re actively excited about, as both investors and AI-native users, and would love to chat with people thinking about these questions.

Grok 4.6 – A field guide

Grok 4.6 is out! I've used it for a few weeks as my daily driver across the normal mix of coding and knowledge work, and built a few projects with it specifically to push on where it holds up.

It's good at all of it. What stands out most is how it communicates and how fast it is, more than any single jump in capability.

Information dense communication

It's collaborative in a way that's easy to work alongside. The summaries are dense with actual information instead of restating the task back at me, and the short updates while it's running tell me enough to know whether to interrupt.

It stays quiet through small changes and starts narrating once it's touching a lot of files. Getting that split right took more tuning than you'd think. It still tells me things I don't need sometimes, which we're working on.

Delightful speed

4.5 was fast too. 4.6 is fast and noticeably smarter, and that combination pushed me toward a more synchronous way of working. Rather than front-loading a lot of context and waiting, I ask for something small, look at it, and keep going. The same session can move into a longer-horizon task just by asking for one.

I move between sync and async depending on what the best models happen to be good at that month. Async gets more done while I'm somewhere else, but I lose the thread and end up reviewing a big diff cold. 4.6 pulls me back toward sync, which is where I'd rather be when I care about the result.

Most of those weeks were ordinary work. It navigated websites for me, including creating API keys by clicking through a provider's console. It did functional and visual QA on running apps. It got my inbox down to the handful of threads that actually needed a reply, which never stops feeling good. It helped me draft the launch posts for Cursor SDK Bridge and /rename-chat, and put together a launch video for both of them with Remotion!

Short prompts, strict verification

I spent part of those weeks testing prompting styles against each other. Long versus short, and whether specific phrasings like "work very hard" change the outcome. What I found is that the phrasing barely made no difference at all.

Length did, though not the way I assumed. A long prompt buys specificity, so if you know exactly what you want, write it down. A short prompt hands more of the decision to the model's taste. That trade used to argue for writing everything out. With 4.6 the taste is good enough that a short prompt plus a clear preference usually lands somewhere good.

Long specs still work fine when you have one. I gave it a detailed spec for a feedback widget with session capture, a server handler, and cloud agent dispatch, and it followed the whole thing end to end with a sensible structure. It does repeat itself in components unless you ask it to break them up.

One of the projects I built was a spreadsheet app, and I gave it to both models twice. One run got a two page specification covering every toolbar item, keyboard shortcut, and formula I could think of. The other got three sentences.

The two apps came back nearly identical. What actually changed the result was adding one sentence: "Verify your work by running the app and checking that each formula works as expected."

That single line was the highest-leverage thing I found in those weeks! With it, the model opens the app, clicks through real user paths, checks that nested formulas evaluate correctly, and fixes what it finds. None of that works without solid browser use, which is what makes the loop possible at all.

The same principle holds when the output is harder to inspect. "Improve the textures" on a 3D scene got me nowhere, while "capture the current frame, list what's wrong with it, then fix only those things" worked immediately.

Every comparison from here on ran the same prompt through both models in isolated workspaces, so none of it is my memory of last month.

You also don't need to tell it to work hard or keep pushing until it's finished. It will keep going on its own for a good while. What matters far more is saying what done means, because otherwise it decides that for you.

Going further

I played an unreasonable amount of Age of Empires 2 growing up. Thousands of hours. So recreating it was the first project I wanted to try. I asked for a browser strategy game with an economy, construction, combat, fog of war, objectives, and a HUD a new player could read without instructions.

4.5 built a workable flat prototype. 4.6 came back with an isometric 3D world on the first try, HUD and minimap already in place. Much closer to the real thing!

Still on the nostalgia trip, I did MSN Messenger next.

Both models clearly knew the reference and did a good job. 4.6 just has more polish, down to the separate conversation windows and the winks.

I use Excalidraw constantly and it's open source, which made it the obvious place to see how the models handle a real codebase instead of an empty folder. I asked both for a presentation mode: save named views, reorder them, and present them as a guided walkthrough. The prompt was deliberately vague about how to build it.

Both land in roughly the same place, which is impressive for a prompt that vague! 4.6 just pays more attention to detail on the first pass, which in practice means fewer rounds of me pointing at things.

This is also where skipping verification bites. On an earlier run the summary read as finished and adding a view didn't actually work. One round of "run it and show me" surfaced the broken import.

Everyday work

I don't put together decks and reports every day, but a lot of people do, and I wanted to see how it handled that kind of work. So I gave both models the same fictional quarterly and asked for a board deck.

Both are competent, and the gap is in presentation rather than analysis. 4.5 mostly lays the numbers out on slides, while 4.6 puts real work into structure and hierarchy, so it reads like a deck somebody made rather than a data dump.

Video as code

This one deserves more space, because I've been spending a lot of time on it lately. Remotion is video as code: every frame is a React component that renders off the current frame number, and the whole thing compiles to an MP4 through headless Chromium and FFmpeg. Your video lives in git. It's a genuinely fun way to work! It's also a strange thing to hand a model, because you can't tell whether it succeeded by checking that it runs.

I asked for a 60 to 90 second launch film for the X TypeScript SDK and gave it the docs to work from.

I judge these on whether there's a storyline and whether the pacing holds. Most models fail the same way here, with uppercase titles, boxed text, and everything landing on screen at once. Both films avoid most of that, and 4.6 is the more compelling watch.

After a few days of running this across different models, video is where I see the widest spread. Two models that feel equally capable on a web app can be nowhere near each other here.

Where it needs steering

Almost everything I had to steer came back to one thing: how easily the model can verify its own work.

A website is the easy case. The DOM is text, so it can read the page, take a screenshot, and compare against what it intended. That's why the verification loop works as well as it does on UI work.

3D is harder, because there's a whole dimension you can't inspect by reading. Video is harder still, since time is the extra dimension and checking your work means capturing a sequence of frames and reasoning about the delta between them. Physics has the same shape of problem. The model has a good sense of how the world should behave, but confirming that it did behave that way isn't something one screenshot can answer.

The practical answer is to give it a way to look, or to accept that you're the one checking.

Why it's my default

There's real value in spiky models, the ones that are extraordinary at one particular thing. But most of my work isn't one particular thing. What I want day to day is a model I know well: one where I've built up intuition for how it behaves, where it's reliable enough to hand something over, and where I understand the shortcomings well enough to work around them without thinking about it.

That's exactly what 4.6 has turned into for me. On the coding side it handles interactive and visual work where I'm reacting as it goes, plus long sessions in a real repository. On the knowledge work side it's the inbox, the browser QA, and the click-through tasks with no API behind them. It isn't the best imaginable model at any one of those, but it's good at all of them and I know what to expect.

I still stay involved where the output gets judged on how it looks. Motion, 3D, and final polish want a reference and a screenshot loop rather than a description. And I write the acceptance criteria down instead of trusting a summary that says it's done.

Try it

Grok 4.6 is available now in Cursor, SpaceXAI API on OpenRouter and anywhere else you get your tokens!

Try it out and let me know what you think. We're going to keep improving it, so leave feedback either way, good or bad, since that's what tells us where to push next.

Curious to hear what you end up building with it!

Full article content is not available for inline reading.

Read the original article →

As projects like Pacing the Frontier look to major labs as a way to keep AI research safe, open source models have become a sore spot for the industry. With free distribution and little control over how they’re used, open-weight models aren’t easily controlled, leading some labs to treat them as downright scary.

But at the Ai4 conference in Las Vegas last week, three of the world’s most respected AI researchers — Nobel Prize winner Geoffrey Hinton, World Labs CEO and co-founder Fei-Fei Li, and Coursera co-founder Andrew Ng — spoke out on the issue. And while they disagreed on particular tactics, all three made a powerful case for keeping AI open.

For the three speakers, the core concern was allowing a handful of major AI companies to control the pace of progress. When a few companies control access to a technology, as Apple and Google do with mobile operating systems, innovation can slow and the companies that control the platforms can influence what gets built on them.

Andrew Ng said that he worried about a similar dynamic emerging in AI. “I don’t want there to be gatekeepers,” Ng said. “That limits how all of us can access AI.”

Companies have an incentive to protect their competitive advantages, including by influencing the rules that govern the industry. That could create a dynamic where only the largest, best-capitalized firms have the resources to build the most advanced AI systems.

Ng’s solution was to maintain multiple providers, with models and companies competing rather than allowing a handful of players to dominate the field. “If I were to try to give one prescription, it would be to promote openness,” Ng said, “because AI is amazing technology and I want it to be in everyone’s hands.”

But not everyone agreed that open-weight models would help preserve that state of play. Hinton, in particular, drew a distinction between open source software, which makes the underlying code available for inspection and modification, and open-weight models, which release the parameters of a trained AI model to the public.

“Open source is great. You show people the code, and lots of people look at the lines of code and say, ‘Oh, there’s a bug.’ Open weights means you train a big model and then you give people the weights. That’s very different,” Hinton said. “I was against open [weights] because it makes it so easy for people to take these big foundation models, which are very expensive to train, and for much less money train them to do bad things like cyber attacks.”

But whatever his reservations, Hinton acknowledged that open-weight models are already a permanent fixture of AI. “I think that battle’s been lost. We now have open-weight models, so the barrier to lots of people getting these big models, which was the cost of training foundation models, that barrier has disappeared. It’s too late.”

Yet accepting reality didn’t mean ignoring the risks. Hinton’s position was clear: AI would continue to advance, and he thought that was largely a good thing. He said it would boost productivity and improve education and healthcare. “Worrying about the possible bad effects of AI and the things that intelligent beings might do when they’re smarter than us. I don’t think that’s unfair. I think it is unfair to label anybody who thinks like that as a fear-monger,” Hinton added.

Ng took a different view. The question, he argued, wasn’t whether open models were risky, but who controlled access and who would win the market. Whoever built the cheaper model would have the advantage. If China’s open-weight models gained widespread adoption across Asia, Africa, and/or the developing world, he warned, they could influence how billions of people encountered ideas about democracy, freedom, and human rights.

“One thing I hope we do is encourage American competitiveness and open source AI. It turns out that AI is a tremendous source of soft power. You can see the way China’s model has tremendous accomplishment with Africa, for example,” Ng said. “But my worry is because of all the lobbying in the U.S. and the fear-mongering, building open source AI in America is struggling to compete with open-weight models coming out of China, and my worry is that if China figures out a fundamentally more cost-efficient way to build AI, then things that are more cost-efficient have a fundamental business adoption advantage.”

Li pushed back on that framing. “It’s very dangerous to make this a dichotomy between complete openness all the way to complete closedness,” she said. “In complex software systems as well as scientific systems it’s much more nuanced.”

Li used nuclear physics as an example: Scientific papers are published openly, but uranium is regulated, while laboratory work falls somewhere in between. The lesson, she explained, was that openness doesn’t have to be an all-or-nothing choice. Different layers of the ecosystem can operate at different levels of openness.

She also highlighted collaborations between public and private institutions, such as the Human Genome Project. The resulting knowledge became a platform that others could build on, she said, allowing pharmaceutical companies to profit, scientists to advance their work and society to benefit.

“So I think we have to use [AI] as that kind of infrastructure,” Li said. “We need some levels of openness, both in scientific discovery, in education, in global partnership, as well as lucrative business models for entrepreneurs. But we also will accept closed-source systems. This debate, especially at the sweeping level of ‘we can only tolerate one,’ is a false debate. We need to get to a level of nuance.”

But everyone agreed that some level of regulation would be necessary to keep AI on the right track. “What we want to do is develop AI in a direction that helps people, and regulation will help us do that,” Hinton said. “You can’t leave it to people like Elon Musk and Mark Zuckerberg to decide how AI should be done.”

We have had our heads down building and working toward launching Kenn Software’s product offerings later this year, but in the meantime, I wanted to give some insight into how our agentic engineering process and culture have evolved since the beginning of the year, and what a work day for us looks like. We merge hundreds of pull requests per week into our projects with a team of three people, and yet have an empirically low bug rate across millions of lines of production code.

A couple weeks ago, after being drowned in posts about “loop engineering” and “graph engineering” on X/Twitter and LinkedIn, I posted on my X account:

I think loops are bullshit

I stand by this. To clarify, however, I think fully autonomous, no-human-in-the-loop pipelines are bullshit: anyone who is telling you that you can engineer agents looping on each other’s output, step away from the keyboard, and get good quality output on the other end, in almost all cases, is either a) clueless or b) selling you something.

That being said: I burn a lot of tokens. I am regularly at the top of one prominent token leaderboard. As of this writing, at API rates I would be paying $56,836 for my last 30 days of consumption if not for subsidy provided by coding agent subscriptions. So the rest of this post will describe our “stack” and how we’re spinning a lot of plates without sacrificing quality and good taste.

“Planning, Architecture, and… Caring about the Output”

Jesse Vincent, creator of the megapopular Superpowers framework, described the difference between “vibe coding” and “agentic engineering” as “planning, architecture, and… caring about the output”. On the Kenn team, we put it bluntly: “Vibe coding is not caring at scale.”

The workflow, in brief:

• Start with the right tools. For us, that is Superpowers and roborev, our continuous local review and verification system.
• Design together. The human stays involved in every important brainstorm decision and design section. Design and taste are not delegated to an agent.
• Ask for a second opinion when the decision is unclear. A separate agent session, ideally from a different model family, should challenge the choice or design before it hardens.
• Write the spec for the implementers. Once the design is settled, Superpowers turns the design into a precise specification. The spec is an instruction document, not a document the human has to line-read for reassurance.
• Review the spec adversarially. A separate agent reviews the specification, findings are fixed, and the review repeats until it converges.
• Plan, implement in small pieces, let roborev verify the work. Superpowers turns the reviewed spec into an implementation plan, which we implement either with subagent-driven development (mostly with Claude) or inline execution (mostly with Codex, because Codex’s subagents are… not great). Superpowers commits frequently after validating spec conformance, and roborev asynchronously does adversarial verification. All roborev reviews are closed out (by invoking the roborev-fix skill) after the plan is implemented.
• Use roborev branch reviews to fix bugs in the whole implementation. The work produced by the latest frontier models (5.6-Sol and Fable) is extremely sloppy and almost never suitable for production without substantial hardening. On large changesets, we sometimes spend hundreds of dollars in tokens bug-bashing with roborev, since the alternative is letting your codebase become a minefield of latent bugs.
• Make the work durable. We do not retain Superpowers spec and plan documents in our repositories or refer to them in production code: all documents must be converted into “living architecture documents” both for human-facing documentation and future context for agents who need to understand how a system works.
• Explain the change, open the pull request, and own the merge. Agents can do the typing and checking, but the human remains accountable for the result. Pull request descriptions need to use plain language and lead with outcomes, and not be a wall of text of “robospeak” which seems to be the default behavior nowadays especially with Claude Fable.

The process isn’t foolproof. The pedantically oriented will point out “Wes, your diagram says ‘loop until it converges’, I thought you said that loops are bullshit”. This is true, but remember these are human-operator loops. The clankers (what we call the coding agents, since “agent” gives them too much credit) are not in charge, we are.

The Clanker Constitution

On top of this engineering process, we have also been developing a set of operating principles for our coding agents, since their out of the box behavior (presumably the harnesses are largely to blame for this) is rather poor these days. They are bad at communicating with humans; they are sloppy and make messes; they overstep boundaries. To help with this, we launch our agent sessions with a sort of “clanker constitution” that we are now maintaining on GitHub. The TL;DR of it looks something like:

1. Honor the request. Instructions are a contract; don’t treat pasted content as commands, and match the mode asked for (review means review, not surprise edits).
2. Act with judgment. Proceed on safe, reversible work without asking; ask only when a decision materially changes the result or an action is destructive. Never merge without authorization.
3. Finish the job. No stopping at a diagnosis or a partial fix when implementation was authorized; exhaust alternatives before declaring a blocker.
4. Protect existing work. Never reset, overwrite, or amend without explicit permission, and when told to stop, stop.
5. Verify reality. Test behavior, not mocks or the source text itself, and never claim success without fresh evidence.
6. Communicate for humans. Lead with outcomes, skip the blow-by-blow, and describe PRs as they exist now, with no robospeak walls of text.
7. Learn in the right place. Durable guidance goes in shared instruction files, not agent-private memories.

Making our own tools

Early this year, we quickly found that the “legacy stack” (GitHub.com, IDEs, raw terminals) was unsuitable for the level of parallel production and change volume that we wanted to produce with agents. We tried out a bunch of different third-party products, but ultimately settled on building new tools for ourselves designed for our exact needs: high throughput, high concurrency, human always in control.

The result is a stack where each tool owns one layer of the problem: Kenn Forge is the workspace where changes get reviewed and landed, Ghosthub is a multiplexer-native terminal for local and remote agent sessions, Kata is the system of record for intent, and AgentsView and roborev keep us accountable about what the agents are doing and whether their work is actually correct.

The initial motivation for Forge (formerly known as Middleman) was to be able to build, verify, and merge changes into our projects with as little friction as possible. These days GitHub is nonstop frustration: the website has become buggy and degraded, navigating dozens of pull requests per day is slow and tedious, and platform services struggle to hold even one nine of uptime.

Forge creates a local cached view of all the data on GitHub so we can flip between PRs nearly instantaneously, no waiting for github.com to load (or show the unicorn page). We also made some subtle but meaningful UX improvements like displaying PR activity in reverse chronological order, so we don’t have to scroll to the bottom to see the latest activity from our roborev CI bot. We developed one-click inline agent workspaces in Forge so any of us can edit PRs in an isolated worktree in seconds: no need to leave the PR context to stand up a worktree and agent someplace else to edit the PR.

Outside of Forge, where we mostly do maintenance and last-mile development work to land changes, we still use terminal applications (like Kitty and Ghostty) and desktop apps (like Codex/ChatGPT and T3Code) for larger, longer-running projects or things that need browser/computer user. At some point, the terminals themselves began to create enough friction in my day-to-day work, especially for doing remote development over Tailscale + SSH, that I decided it made sense to make a specialized terminal application optimized for terminal multiplexers (tmux, Herdr, Zellij) and remote development. It took me a few months to get it ready for public consumption, but this has now been released as Ghosthub.

We also built an agent-native issue tracker, Kata. It has become essential to how we get things done as a team: we run a central “hub” Kata daemon over Tailscale that all of our machines and agents connect to via federation. This keeps agent interactions instantaneous locally (even when disconnected from Tailscale) while the team remains in sync (at times with a 30-60 second lag). Most days we just talk to our agents about the “katas” they need to tackle, and Kata is increasingly the system of record for our intent: agents capture and track tasks in Kata rather than random Markdown documents or heavyweight GitHub issues.

Lastly, AgentsView and roborev are the “accountability engines” that keep us honest and prevent us from shipping slop. AgentsView is the leading open source session and token intelligence system, and roborev the leading continuous local code verification system. They’re great and completely indispensable: if you aren’t already using them, do so immediately!

Looking forward

The last 12 months have been a bit of an odyssey learning how to build large production systems productively and effectively. This has required a lot of trial-and-error with development process, prompt and harness tuning, and custom tool development. We are biased towards human-operator-centric workflow that is intended to minimize the amount of code slop that lands in our repos. This requires each of us to remain engaged with the design process, architecture, and details of what we are doing, never delegating critical work to an autonomous coding loop.

We’ll be excited to share more about what we’re building at Kenn in the near future, and I’m interested to hear what’s working well for others at the frontier of agentic engineering.

SpaceXAI has released Grok 4.6, a model focused on long-running agents, coding, and knowledge work. The model surpasses Kimi K3 and ties with GPT-5.6 Sol Max on the Artificial Analysis Intelligence Index. It posts sizable gains over its predecessor across coding, terminal, knowledge-work, and agent benchmarks. The API starts at $2 per million input tokens and $6 per million output tokens, less than half of what GPT-5.6 Sol costs in standard mode.

High above the Earth, a new generation of robots designed to keep satellites working longer is replacing its predecessor — quite literally.

This week, a spacecraft built and operated by Northrop Grumman called a Mission Extension Vehicle (MEV) unplugged from a communications satellite operated by the Australian firm Optus. For more than a year, the MEV spacecraft has been stuck to the back of the Optus satellite, keeping it in the right place in space so it can continue its mission.

Here’s an exclusive view from the MEV’s on-board camera, showing the Optus satellite and the Earth behind as the two spacecraft moved apart.

The satellite life-extension spacecraft is leaving to make room for its replacement. In July, four new Northrop spacecraft launched into orbit on a SpaceX Falcon 9 rocket. One is called the Mission Robotic Vehicle (MRV), a powerful satellite equipped with two advanced robotic arms that was developed by DARPA, the U.S. military research organization. The other three are called Mission Extension Pods, or MEPs, smaller, simpler satellites that are essentially modular propulsion systems.

Those four spacecraft are currently headed for targets around 27,000 miles above the Earth. In 2027, the MRV will use its robotic arms to attach one of the MEP pods to the Optus satellite, which should keep it in orbit for years to come.

The satellites that provide communications or scan the planet with various sensors only last for so long. They typically fail when they run out of fuel to stay in the right place, not because their computers and transceivers stop working. The Optus satellite was launched in 2009, and designed for a 15-year lifespan. If all goes well, the satellite could fly — and generate revenue — for another six years.

Now, cheaper launch costs and lower-cost space components are making spacecraft repair missions a reality.

The goal is “a paradigm shift where we can see space as sustainable, with a more resilient architecture and infrastructure base where we can do things like spacecraft repairs, life extension, or even upgrades and maintenance of satellites,” according to Northrop’s director of logistics and servicing, Cassie Wong.

There are two MEVs in orbit right now, launched in 2019 and 2020, which have provided 10 years of life extension to three customers, including two different Intelsat spacecraft and the Optus satellite. MEV-1 will wait in a parking orbit for another customer, while MEV-2 is currently attached to its Intelsat customer until 2030.

The Mission Robotic Vehicle, or MRV, represents an evolution of the business model. Satellite operators buy and own the MEPs, which are permanently attached to their spacecraft. That frees up the MRV to service more vehicles, creating a cheaper offering.

The offering requires some serious technology chops. The vehicles need to autonomously approach one another and dock safely, not a simple task when both are moving at velocities of thousands of miles an hour. The MEVs use a docking probe to plug in and hang onto satellite thruster nozzles. Meanwhile, the MRVs will need to carefully attach the MEPs using their robotic arms.

Unlike most satellites, the MRV is designed to be refueled in orbit — in part, a proof of concept for the kind of capabilities other satellites will need if this kind of in-orbit servicing becomes a norm. Right now, the extra cost and weight of such adaptations keeps spacecraft operators from investing in them.

Indeed, the current trend in satellites is flying lots of cheap, effectively replaceable spacecraft in low orbits, as Starlink and Amazon LEO do. On the other hand, spacecraft keep getting bigger, and there are plenty of expensive, large satellites in orbit that could benefit from life extension. Wong hopes that the MRV will take on other missions in the future, adding new components to satellites as well as adjusting their orbits.

That’s likely to include defense customers, given the number of expensive satellites it owns in high orbits, and DARPA’s involvement in developing the MRV’s arms. Indeed, the U.S. Space Force has previously characterized a Chinese servicing spacecraft with robotic arms as a weapon, since it could theoretically grapple and degrade a rival satellite. Northrop says that its vehicles are focused on servicing missions.

The vehicle could also be used in LEO, Wong says, to extend the life with valuable assets there. The startup Katalyst Space is attempting a similar mission to extend the life of a NASA space telescope after malfunctions left its vehicle tumbling last month. The company has a fix in place and hopes to complete the mission.

Why Tiny JPEGs Look Different in Chrome

What looked like a rendering bug turned out to be a clever JPEG decoding optimization in Chrome.

This icon looks better on my colleague’s computer

A while back, when chatting with a colleague over their computer, I noticed that a logo did not look exactly the same as it did on mine. It looked thinner on theirs and more faithful to the original image. It was rendered at 15px; here is an upscaled version.

Note: this was not the original image. It happened a while ago, so I made a new image to demonstrate the issue.

On the left, Firefox; on the right, Chrome.

If you squint, or take a step back, the one from Chrome looks thicker. A bit weird, but swapping the image for an SVG fixed it. Still, I was curious: why was it rendering like this in the first place?

I did some digging and found a nifty optimization that Chrome uses when rendering JPEGs at small scales.

Scaling down images can be wasteful

The intuitive way to render a small image from a JPEG is to fully decompress it in memory and then scale it down.

But that is not always efficient.

Imagine a 2000 × 2000 JPEG that needs to be displayed at 20 × 20. Once uncompressed, the image takes far more memory than the final result. A bitmap of the full image uses roughly 12 MB, while the final 20 × 20 image needs only about 1.2 KB. Most of the information in the large version is lost when scaling down.

What information is lost when scaling down?

An interesting insight is that the information lost is not random.

When an image is scaled down heavily, the information that disappears is mostly the high-frequency detail. This is easy to see intuitively. Think of a tree with lots of leaves and rough bark: those fine details change quickly from pixel to pixel, so they count as high-frequency information.

If you scale that tree down to something tiny, like 20 × 10, you end up with just a green blob at the top for the foliage and a brown stick at the bottom for the trunk. The scaled-down version has thrown away the fine detail, the high-frequency information.

Illustration of a tree being scaled down

Some of that high-frequency information still survives to an extent, because the details get mixed together.

How JPEG stores image data

I will keep this explanation light on jargon and math, but I will still mention a few technical terms that can be good starting points if you want to dig deeper. I will also skip a fair chunk of the full JPEG transformation, because it is not needed here.

During JPEG compression, images are split into 8 × 8 blocks that are converted into the frequency domain. This operation is called a DCT (Discrete Cosine Transform).

In an 8 × 8 block, the lowest possible frequency is a flat color. Strictly speaking, it is not really a frequency because nothing changes; it is the constant component. At the opposite end, the highest frequency looks like a checkerboard, where the value changes as much as possible. Everything in between represents the rest of the frequency domain. These are called basis functions.

The basis functions: you can see the flat color in the top-left, and the checkerboard in the bottom-right.

So converting an 8 × 8 block into the frequency domain is basically asking: how much of each pattern is present in this block? Those amounts are called coefficients.

JPEG compression has a few more steps after that to store those coefficients efficiently, and that is where the lossy compression happens. But that part is not important for what we are discussing here.

Putting it together: rendering a JPEG at 1/8 scale

Now let’s say you want to shrink an image by a factor of 8.

Those 8 × 8 blocks I mentioned earlier can now be represented by a single pixel in the downscaled image. At that size, the image mostly needs low-frequency information because, as in the tree example, the high-frequency details mostly disappear during scaling.

So instead of decompressing the whole JPEG, we can skip the coefficients for the high-frequency parts and use only the ones needed for the coarse version of the image. That gives a scaled-down result without fully expanding the original image first.

The decoded image takes less space and is faster to uncompress, since we are skipping a good chunk of the coefficients.

This can be extended to other ratios, as long as they are fractions with a denominator of 8. The technical name for this is partial IDCT scaling.

* Inverse discrete cosine transform: taking the frequency domain back to the image domain.

How Chrome fits in

Chrome delegates image decoding and rendering to Skia. For JPEGs, Skia uses libjpeg-turbo, which implements partial IDCT scaling. That lets it decode only the lower-frequency data when the target size is small enough.

In other words, Chrome/Skia does not always decompress the full image and scale it afterward. It computes the closest fraction with a denominator of 8 and decodes the image at that scale. It then scales the image further using a more traditional downsampling algorithm until it reaches the desired size.

That is why the image looked thicker on my machine. Because it was rendered so small, it was decoded at one-eighth scale using partial IDCT scaling. So the only data from the frequency representation that remained was the constant component; all the edge softening and gradients were not used.

Really, the moral here is that you should not use JPEG for icons and the like. The format and its optimizations are designed around our perception of photos.

After all, it is in the name: Joint Photographic Experts Group.

It's 2020. You're the most senior person on your team, in charge of code quality and architecture. You've set up good engineering practices, you thoroughly review PRs from people who are less experienced than you and work hard to maintain a healthy codebase.

Then at some point, you go on holiday. When you come back, the codebase is a mess. Everyone merged each other's PRs without really paying much attention, someone added a bunch of new tables to the database to denormalise it because it was easier and they added serverless or Kafka to the stack without any solid evidence that they needed either.

It's okay. You can fix this.

Fast forward to 2026. You haven't been on holiday. It's just a normal Monday morning. You make yourself a nice coffee, open your computer and find yourself with 7 PRs to review. You open the first one: +24506 -3938 lines, accompanied by some AI-generated description of what they're supposed to do. Somehow, your team has made more changes since Friday than they used to make while you were away for a few weeks.

AI removed the speed limit

AI makes projects with weak engineering culture fail much faster.

There used to be a time when people sat down and talked about how they'd do something. Now they can just prompt an agent for a few hours and open a PR.

The most tragic aspect of this way of working is that, to the untrained eye, it works.

If you pull the branch and test it, you'll probably get something somewhat functional. So what do they do? They keep going. Again and again. Until the project reaches a point where no one knows how anything works.

Just like someone buying a new luxury car on a credit card. You don't see the debt. You just see the car that looks great.

But then users start to report a weird bug. It's the 4th time your team has been trying to fix it. I mean... asking AI to fix it. Unfortunately, it seems like not even Fable can figure it out.

You go talk to the person who worked on this feature.

• "So where does the data come from?"
• "Hmm... actually I don't know. Let me ask Claude."

You sit next to each other watching an endless wall of text appear on the screen. Neither of you has any idea whether any of it is true but Claude seems very confident.

"Let's just turn on ultracode and ask it to double-check?"

This one will take a while. You start talking about the latest drama on X.

You finally get an answer back.

• "Does this make any sense to you?"
• "I'm not sure."
• "Didn't you build this like... last week?"

Silence.

This project has become so convoluted, with so many layers and services, that no one on your team could possibly start to understand what's going on.

So, what do you do?

Fixing it would require such a colossal amount of work that it would be impossible to even start justifying it to anyone in management.

And what are you even thinking about? It would end up in the exact same state again in just a few months anyway.

• "Let's just ask Claude to fix it."
• "Okay. I'll create a loop and goal so it doesn't stop until it's checked that everything works."
• "Sounds good"
• "Actually, I ran out of Fable usage for today so I'll run it tomorrow"

You grab another coffee and walk back to your computer. You now have 13 PRs left to review. You see something you don't quite understand, so you message the person who wrote it.

• "Why are we doing this here?"

They send you a link. It's a Claude conversation.

Somewhere in that conversation, buried between Claude confidently recommending one architecture, apologising, changing its mind, your coworker asking it to reconsider again and another 15 rounds of changes, is apparently the design decision behind this code.

• "Which part should I read?"
• "Probably all of it."

Does this sound familiar?

Whenever I talk about this, someone eventually tells me that nobody ever fully understood large systems anyway. It's true.

You were never expected to understand every service and every database. But at least someone did and would explain it to you.

Now they ask an LLM because they don't actually know themselves.

You can't afford bad engineers anymore

In every team, there are competent people who make the project possible. There are also people who essentially make it harder for everyone else. And now anyone can produce more code in a day than they used to in a year.

In the story above, everyone is failing:

• The engineer opening a 25,000-line PR should have stopped the agent long before it got there. They should have understood what it was doing, broken the work into smaller pieces and questioned every new abstraction it introduced.
• The person reviewing it should have refused to review something that large instead of giving in.
• The person adding Kafka should have been able to explain exactly why it was needed.
• The person who built the feature should have been able to explain where the data came from without sending a link to a Claude conversation.

But what's the problem then? Just use AI to fix it. Well, it's not that easy...

Before anyone jumps on this, none of this means technical debt is always bad. The important part is that you know it's a shortcut.

Anyway, reverting a bad decision is hard. Very hard.

For example, how long would it take an LLM to add a bunch of tables and columns to the database? 10 minutes?

But once you start storing data there, you can't just remove them. You have to come up with a migration plan, make sure you don't disrupt the system because people are paying to use this every day. You have to think about what you'll do if the migration fails. Make sure you don't end up with orphaned foreign keys. It's just so much harder to fix. Even with the best model you can get.

And while you're fixing it, more PRs keep coming in. More code, more abstractions, more decisions. A person can generate 20,000 lines of code in an afternoon, but you still have to sit there and understand what those lines actually do.

By the time you've untangled one bad decision, five more have been merged.

The new AI economy

Of course, bad engineers were always a liability.

It has been like this for decades, well before OpenAI or Anthropic existed. Bad decisions compounded, unnecessary complexity accumulated and teams ended up maintaining systems nobody really understood.

The difference is that there used to be a limit to how fast you could do it.

Today, implementation is cheap. You are paid to make good decisions. To build software that will scale while managing complexity.

Ask yourself why companies are paying six-figure salaries for engineers in London or San Francisco in the first place.

If all they needed was someone who could turn a specification into working code, why were they paying that much when they could already get it done cheaply elsewhere?

Why are the tech companies claiming that "software is solved" still paying top salaries to attract the best people they can?

My bet is that AI pushes salaries further apart. To be employable, there's a bar you have to clear and that bar is whatever the current best model du jour can do.

Good engineers have become more valuable because AI lets them move much faster. They don't need as many people around them just to do the implementation work anymore.

At the same time, bad engineers have become much more expensive to hire.

You need to contribute beyond what everyone already gets by giving an agent a prompt.

If you lack the judgment required to evaluate the LLM's recommendation, asking for more judgment doesn't solve the problem.

At some point, someone still has to know what is going on. And that's the most valuable person on the team.

The people who don't will become much cheaper to hire or get replaced entirely while the money gets funnelled towards an increasingly smaller number of people who can actually be trusted.

I don't think this is going to be limited to software engineering either. I believe the same thing is going to happen across most knowledge work. AI will make the best people much more productive and the bad ones almost impossible to hire. Before, there was a good chance someone would catch their bad decisions before they went too far. Now they can make changes faster than anyone around them can realistically review or understand them.

Answers to the most common objections

"Bad engineers always existed"

The difference is the speed.

It is the difference between crashing at 30 km/h and crashing at 200 km/h. Before AI, a bad engineer would struggle to produce code that even compiled. When they did produce something, it took them a long time and the blast radius was limited. The damage was bounded by how fast a human could type.

Now a bad engineer can produce 10,000 lines of working code before lunch. The damage we can do in an afternoon used to take them months. The speed at which bad decisions compound has changed completely while the speed at which you can fix them has not.

"Just fix your process"

Several people argued that the real problem is lack of process. If you had proper tests, CI, code review and architectural reviews, AI-generated slop wouldn't get through.

We had all of those things. None of them disappeared.

The problem is that they were designed for a world where producing a massive amount of change was impossible. Code review don't work anymore when someone opens 10 PRs a day with an AI-generated description. Tests work when they cover the behaviours you thought to test. They do not catch the behaviours nobody thought to test.

How many times have you had a completely green CI with full coverage and still shipped a bug?

The difficulty of producing code was by itself one of the limiting factors.

If the people trying to understand changes and guard quality are now the bottleneck, you have three options. Generate less, find a genuinely better way to validate or accept lower quality.

"You are just anti-AI"

Everytime I critisise AI someone says I'm a Luddism. I'm refusing to adapt to the new world, clinging to the old ways.

I use AI heavily. I have said this repeatedly. I use it every day and I have no interest in going back to writing everything by hand.

The point is that we have made producing large changes extremely cheap and fast while understanding those changes is still slow, difficult work. We have no shortcut for building a correct mental model of what a change does.

Maybe one day we will find one. As of today, I do not think we have one.

You can be a heavy AI user and still recognise that there are serious problems with how it is being used.

"More output means more productive"

Someone generates 10 PRs in a day, the numbers look incredible, surely this person is ten times more productive.

Not necessarily. The supposed 10x engineer may simply be someone stealing productivity from everyone around them.

If I generate 10 PRs in a day but three engineers now have to spend the next two days reviewing them, figuring out what I changed, correcting bad assumptions, debugging regressions and explaining why half of it needs to be redone, I have not become 10x more productive. I have just moved the work onto other people.

Worse, I am consuming the time of the people who are usually the hardest to replace and whose attention is already scarce.

PR count, lines changed and features "completed" are terrible measures of productivity. You can make your own numbers look incredible while reducing the throughput of the entire team.

Some people pushed back on this by saying the real problem is bad organisations with broken incentives. It's true. If your company rewards ticket count over quality, the careful engineer looks like the bad employee and the sloppy one gets promoted.

"Pushing back just makes you're the toxic one"

Someone pointed out that trying to hold the line on quality can get you labelled as toxic. Everyone else is shipping fast and you are the person saying "wait".

I would much rather have someone on my team who ships less but whose work I can trust than someone much faster whose changes leave me wondering what problems we're going to discover later.

You need to be flexible and compromise when the business trade-off makes sense. But you also need a backbone. If you think something is going to cause real problems, bringing it up is part of the job.

And when production breaks, and it will, I need the person who made the change to actually understand it well enough to help fix it. Not show up with no idea what is going on.

"Just AI output is like assembly from a compiler"

We already trust compilers to produce machine code we do not read. Why is this different?

A compiler takes code and translates it into another representation while preserving its semantics. The compiler is not deciding what your system should do. It is deterministic.

An LLM is making decisions. It is choosing architectures, picking abstractions, deciding where to put things. When you ask Claude to build a feature, it is not translating your intent into code. It is making dozens of design decisions on your behalf.

If in five years I can give an agent a complete specification and reliably verify the resulting code against it, then sure, reviewing code may become obsolete and I would happily stop doing it. We are not there yet.

If you genuinely understand the resulting system, that is fine. That is not the behaviour I am criticising. But for the vast majority of very large PRs, especially AI-generated ones, I would bet money that the person opening it does not actually understand all of it.

"We ship 99% AI-generated code and it works"

I do the same. Most of the code I produce comes from AI.

You still have to put in the work to understand the result. If you are shipping AI code and you genuinely understand the system, you are doing it right. The article is not about you.

"AI can write the code" ≠ "I do not need to understand it anymore."

You can use AI heavily and still take responsibility for what it produces.

"Users don't care. It's just a CRUD app. Ship it and get paid"

I tend to find that this perspective comes from people working on relatively small or isolated projects.

On a large system, the customer being happy today is not enough. You need other engineers to be able to understand the system. Have you ever been on call and been woken up in the middle of the night to fix a production incident in a system you did not write?

If everything you build is small, isolated and easy to replace, then sure. Ship the ugly thing, get paid and move on. If you are going to be working on something for the next five years or more, you should probably spend some time thinking about what you are doing.

Nobody is getting paid significant money just to build a simple CRUD app. Even before AI. Companies pay experienced engineers because the supposedly simple CRUD sits inside a messy real system with years of business rules, constraints and integrations where bad decisions have consequences.

"What about junior developers?"

I have worked with two junior developers recently who are very good precisely because they are trying to understand what they are doing rather than just producing code. They use AI to explore things they do not understand, ask questions to clarify their reasoning and double-check assumptions. They use the tools to increase their understanding.

I have also worked with senior developers who basically gave up and stopped trying to understand the code. They became much worse engineers as a result. At this point I would much rather work with those two juniors.

The problem is not AI. The problem is using AI as a substitute for understanding instead of a tool for building it.

Related to this, some people argued that we should not teach skills that AI can do better. The fact that a machine can do something better than humans does not automatically make learning it pointless.

We still teach arithmetic and algebra despite computers being vastly better at calculation. We teach spelling, grammar and essay writing. We even teach history and geography while everyone permanently has a device in their pocket that can look up almost any fact in seconds.

How else would you be developing the mental models required to understand, question and verify anything?

"Using AI is just delegation, like a manager"

A manager is not normally the person deciding how the software should be architected or implemented. Their job is largely priorities, people, coordination and resource allocation.

You are still an engineer. But you have delegated your technical judgement to an LLM. You just stopped doing the most important part of your job.

"Not all technical debt is bad"

Agreed. Debt is debt. Some of it is absolutely worth taking on.

I do not have any issue with intentional debt when you understand the trade-off and have a clear payoff plan.

"How do we make more senior devs if we don't hire juniors anymore?"

By reducing the number of entry-level engineering roles, the industry is sabotaging its own future. Fewer people learning means fewer people capable of maintaining systems down the line.

It's true. But the industry does not really owe people opportunities. Even if you think it should, that is not how companies are going to behave.

That said, maintenance is the essence of the software industry. LLMs cannot modify projects spanning hundreds of thousands of lines. The skill required to partition architectures is still entirely human. Learn that.

That is exactly why experienced engineers are becoming more valuable, not less.

"Will anyone actually care?"

They will care when nothing works, nobody seems able to fix it, building new features takes forever and every change breaks something somewhere else.

This was already happening before AI. But now, a lot more companies that previously might have taken many years to reach an unmaintainable state can get there in just a few months.

Full article content is not available for inline reading.

Read the original article →

Filtered Vector Search: What ACORN Fixes, and What Fixes ACORN

Filtered vector search breaks when metadata filters turn a healthy nearest-neighbor graph into scattered islands. HNSW’s m parameter controls how many links each point gets. At Qdrant’s default m=16, the one-million-point collection benchmarked below averaged about 21 links per node on layer 0. Filter out 96% of the points and fewer than one link per node survives on average, so traversal can get stranded before it reaches the true nearest matches.

Qdrant repairs that damage in two places. Filterable HNSW adds extra edges at index time; ACORN steps through neighbors of neighbors at search time. Both run on the same collection. ACORN earns its cost where the extra edges don’t reach: values too common to link, AND filters no single field’s edges cover, and payload fields the build skipped silently.

This benchmark runs on a single Qdrant instance and compares four of Qdrant’s own search strategies over four builds.

The Two ACORNs

The ACORN paper (Patel et al., SIGMOD 2024) describes two algorithms. Its headline claim of “2-1,000x higher throughput at a fixed recall” belongs to ACORN-gamma, which expands neighbor lists during index construction at 8.8x to 33.1x plain HNSW’s build time in the paper’s own table.

ACORN-1 is lighter. It builds a standard HNSW graph, then checks neighbors of neighbors at search time where direct neighbors fail the filter. Qdrant implements ACORN-1 as a query parameter you opt into per request, with no index-time changes.

The Graph Qdrant Builds Instead

Filterable HNSW, which our co-founder Andrey Vasnetsov described in 2019, builds the repair into the index. When a payload field, the metadata attached to each point, is indexed, Qdrant adds extra HNSW edges between points that share a value in that field, so a filtered query keeps a connected graph to traverse. Qdrant gives those edges to payload fields at index time, and not every field earns them.

Those edges cost build time. On our one-million-point collection, the HNSW index built in 116 seconds without them and 507 to 652 seconds with them, 4.4x to 5.6x the cost. That range covers two builds at identical settings, so it is build-to-build variance. Both figures are index build time, with ingest excluded.

Qdrant builds those edges per payload field, never per combination, so an AND filter lands on an intersection that no single field’s edges cover. ACORN-1 covers that gap and pays at query time instead of build time. Qdrant’s query planner chooses automatically between ACORN, full scan, retrieval straight from the payload index, and filterable HNSW.

The Benchmark

The benchmark runs on one million deep-image-96 vectors, 96-dimensional image embeddings from the ANN-benchmarks suite. Keyword filters match from 20% of the points down to 0.012%. Recall@10 is scored against exact brute force over 500 queries per filter, and latency is mean server-side query time.

We tested four strategies:

1. Plain graph: standard HNSW with no extra edges.
2. Plain graph + ACORN: the same graph with ACORN forced on.
3. Filterable HNSW: the default build with extra edges.
4. Planner + ACORN: Qdrant’s default query planner, free to route each query to ACORN, full scan, or the payload index.

Every filter matches one keyword value on a payload field. The collection carries seven such fields, holding 5, 10, or 100 distinct values each.

Single Filters: Extra Edges Win

hnsw_ef, shortened to ef below, is the number of candidates the search evaluates, so raising it improves recall and slows the query. Selectivity is the fraction of points that pass the filter.

Why Some Payload Fields Get No Extra Edges

Qdrant builds extra edges by walking the values of each indexed payload field. For each value it finds the points that share it and links them, so a query filtered to that value still has a graph to traverse.

A value shared by more points than a size cap gets no extra edges, because the main graph should already keep that many points connected. Qdrant derives that cap per segment, the slice of a collection that has its own index. The formula is point count divided by average links per node, times four.

Double Filters: The Intersection Gap

The same benchmark at hnsw_ef=64, now with an AND filter over two keyword fields.

ACORN on a Normal Collection

This is the default configuration: extra edges, the default threshold, and the planner free to choose the graph or the payload index in both columns.

What to Measure on Your Own Collection

Measure recall for each filter shape you serve. Start with the ones most likely to break: values covering roughly a fifth of the collection or more, and AND combinations of them. Create a payload index on every field you filter on, and leave ACORN off to start. Then sample a few hundred real queries per filter shape. Compare the recall gain with the latency cost. Turning it on never lowered recall in any of our runs, so if you are unsure, the cost of leaving it on is latency.

Further Reading

• ACORN (Patel et al., SIGMOD 2024)
• RACORN-1
• PostgreSQL ACORN study
• Filterable HNSW
• Reproduction kit

Every build on Vercel starts in the build warm pool, which is a set of standby containers that let builds begin without waiting for new compute. The pool runs on state that tracks which containers are ready, the tokens each one uses to authenticate, and the mapping that ties every running build back to the deployment that gets billed for it. When we built the pool, we put all of that in Redis, which was fast and made sense at the time.

Over the years, though, that state turned into a liability. Tokens and container statuses can be rebuilt if they get lost, but the billing mappings can't, and all of it was sitting in a store that we ran as an ephemeral cache. That state needed to live somewhere durable, which is why we decided to migrate it to DynamoDB.

The problem is that the pool never stops. Containers are coming up, polling, picking up work, and expiring around the clock, which meant we couldn't pause the world, copy the data, and restart. The migration had to happen live, under production traffic, in phases, each one behind a flag with a rollback ready if we needed it.

Durable state in an ephemeral store

Redis was a good home for it at first. It was fast, familiar, and efficient for the access patterns the pool started with. But over time the state became more important than the store holding it. If Redis became unavailable or lost data, the pool could no longer reliably authenticate containers, track which of them were ready, or resolve the work in flight.

Lose a token and the pool rebuilds it within about ten minutes. Lose a mapping and the build is never billed, because nothing else records which deployment it belonged to.

We wanted it in durable storage and landed on DynamoDB. On-demand scaling fits bursty deployment traffic, TTL is native, and there are no connections to manage at high concurrency. But what it did not promise was Redis's latency.

What Redis made cheap

Inside Redis, the pool's state looked like this:

• Tokens lived in a set so we could check membership, and in a sorted set so we could expire them
• Each lifecycle status (pending, polling, building) had its own sorted set, which meant a status change was a removal from one set and an add to the next
• A string tied each working container to its deployment

All of these operations were cheap, about a millisecond each, and the code grew to lean on that speed. A single run of the supply loop, which is the loop that refills each warm pool in turn, would make hundreds of count calls just to tally up the pending and polling containers.

That's how the warm pool grew, one easy write at a time, until data we needed to keep was sitting in structures built for speed. Untangling that layout was where the migration started.

Designing the schema from the access patterns

Most migration plans treat the database as the thing being replaced. You map the access patterns, copy the data, and verify that the two stores agree. We planned ours that way too, so before designing any schema, we listed everything the code actually asked of that state:

• Verify a token when a container polls for work
• Add a token when a new container comes up
• Expire tokens past their deadline
• Count tokens to size the pool
• Move a container between lifecycle statuses
• Count containers by status
• Look up the deployment behind a container's callback
• Remove expired containers

Almost every operation already knows which container it's touching, and only polling ever starts from a token. So we put the container at the center of the model. The container ID became the sort key, and the token became just a field on the record, stored as a hash so that reading the table never hands you a usable credential. Verifying a token now just means looking up its container and comparing the hashes.

Two of the access patterns couldn't be served by a key lookup alone. Status counts had to skip containers that were already past expiry, which takes a time-aware index. And the deployment lookups feed billing, so they needed consistent reads, and the mapping got a small table of its own.

type ContainerRecord = {
  warmPoolId: string   // partition key
  containerId: string  // sort key
  token: string        // stored as a hash
  status: 'pending' | 'polling' | 'building'
  expiresAt: number    // one expiry for the whole record
}

A simplified sketch of the shape we shipped. Three status sorted sets became one field.

Reading a container's state, moving its status, and clearing its token all became direct key lookups. Counting a status became a single read against the time-aware index, one query per status with expired containers already filtered out.

// count containers in one status via the index, skipping expired ones
queryCount({
  partition: `${warmPoolId}#${status}`,
  where: 'expiresAt > now',
})

One query per status, bounded by expiry.

Where Redis had let the data structures imply what we needed, DynamoDB made us spell out keys, indexes, conditional writes, and TTL behavior. So instead of carrying the Redis structures forward, we modeled the container and added the indexes that its access patterns needed.

Shadow mode validated every write

The rollout ran as feature-flagged phases: Redis-only, then dual writes, shadow reads, DynamoDB-primary, and finally DynamoDB-only. Every phase kept a rollback open for as long as possible. The last step, which removed the Redis writes, was the only one without an easy rollback, but any lingering token expires within ten minutes anyway.

We started by baselining the existing Redis operations, their counts and their latencies, so that the dashboards had a normal to compare against. The new DynamoDB methods merged days before anything actually called them. Dual writes went out next, with Redis still the source of truth and DynamoDB failures logged instead of fatal. Then came shadow reads, which queried both stores and compared the results. And then primary reads flipped, with the Redis writes kept on until the new path had proven itself.

What the comparisons told us was whether the two stores agreed on the stored values, which is something tests alone couldn't. They said nothing about timing. We checked that every write landed in both stores, that expiration kept moving forward as containers changed status so stale state never piled up, that clearing a token left the rest of the container's record intact, and that the counts stayed close enough to steer the pool.

On top of the comparisons we built dashboards that watched match rates, write errors, per-query latency, and expiration counts while Redis was still serving production. That way any divergence surfaced while it was still cheap to investigate, instead of showing up as a cutover failure. Every mismatch got chased down to a real bug, a dual-write race, or an expected difference before we moved on. Those dashboards decided when each phase advanced, and every time the numbers held steady under production traffic, the next step felt safer.

Two failures along the way

In March, one region's builds went down, and the investigation pointed at our own comparison machinery, which was the added load of checking two stores against each other. The scenario had actually come up in review, and we had throttled the comparisons for exactly that reason, but the throttling wasn't enough. Four days later, a pull request citing the incident added the index that the status counts needed. Without it, every count was O(n) work, and the comparisons were counting constantly. With the index in place, the rollout continued.

Later that month, the Redis infrastructure that we were migrating off actually went down. The warm pool and its token handling stayed up, because both of them were already reading from DynamoDB as the source of truth. Builds still felt the outage, but through services further up the pipeline that hadn't moved yet. The failure we'd been migrating away from arrived before the migration was even finished, and the state that had already moved survived it.

The supply loop stalled on slower reads

In April, the supply loop began to stall. The shadow data had looked healthy when we advanced primary reads, and the per-query latency had looked acceptable on the dashboards. What failed was a behavior that neither one measured.

We drew the loop to see where the time was going.

the loop as designed
  check → create → check → create → check → create → ...          one state read before every container
what slower reads demand
  check → create, create, create, ... → check          stop paying a read before every create

Redrawn from the investigation's original sketch. Every check on the top line is a round trip to the store.

At P95 on getWarmPoolTokenCount, that check measured 1.29ms on Redis and 5.13ms on DynamoDB. At the time we shorthanded it as two to three times slower per query, which turned out to be an underestimate.

A few extra milliseconds cost nothing when you only pay them once. But the loop was paying them before every container, hundreds of times per run, and that stretched runs out to minutes at their worst, which meant the pool couldn't stay ahead of demand. The loop was an N+1 query, and Redis had just simply been fast enough to hide it.

No one wrote "this loop requires millisecond reads" anywhere. But the assumption lived in the design, and at a millisecond per check, nothing had ever made us stop and look.

We rebuilt the loop around the new latency

DynamoDB was never going to match Redis's millisecond, so instead of chasing it, we designed the requirement out.

The first attempt was batching, which is the shape drawn at the bottom of the sketch. The size of the gap depends on which percentile you look at, and at P90, where it was widest, the gap was about 17x. So the batch design checked state once per 17 containers, with one read amortized across the whole batch, and that constant was lifted straight from the measurement. But the constant was also the reason we walked away from it. It hard-coded a latency ratio that would drift with load, which meant one more unwritten dependency on the store's speed. Concurrency didn't need a constant at all.

What shipped instead let the loop's supply calls run concurrently, so each warm pool was no longer waiting behind the one before it, and each call worked from the last state it saw. Overlapping the reads bought us the same thing that batching would have, which is that no run serialized on any single read. The worst case is a few extra containers created from a stale picture of the pool, and that went into the pull request as an accepted cost. The redesign meant we could keep the more durable store without compromising the thing users actually care about, which is builds starting quickly and reliably. The fixes landed the same way the rollout did, one measured step at a time. As one of us put it, "we have less risk on each iteration because we've corrected for a previous misestimation."

It turned out the loop had stalled under Redis too, sometimes for a minute or more, and we only learned that while we were diagnosing DynamoDB. The evidence had been in our telemetry the whole time, but because the pool sits ahead of demand rather than in any request path, nothing had ever forced us to go look until the migration did. And the redesign ended up removing a flaw that was older than the project itself.

Migrating the assumptions

The migration completed in April, with the Redis calls gone from the warm pool paths. Copying the data turned out to be the easy part. The real work was finding the assumptions that had been built on top of the store, and then redesigning the loop that had held them. The state that every build depends on now lives in a store that was built to keep it, and the loop that manages it no longer serializes on any single read.

The April retrospective put the lesson in one sentence. "Even a 1ms-to-15ms query time degradation on P90 could bring down our warm pool management logic." Nobody had written that sentence in February, when the migration started.

Agents have turned up just about everywhere in software this past year, with one conspicuous exception: data. That’s a little odd, because querying data is exactly the kind of structured, checkable task that agents excel at. The likeliest culprit is timing. Large language models have only been reliably good at writing SQL for the last six to nine months, and the field hasn’t caught up to what that unlocks. It’s worth separating two flavors of the idea: agents that do analytics, and agents that help you run the data plumbing. Both turn out to be more useful than they first look.

Data engineering is hard mostly because you’re at the mercy of systems you don’t control. Schemas change without warning. Sources go offline. The API you pull from ships a new version. A column that only ever holds integers starts returning decimals. A field you assumed was unique sprouts duplicates, and the next join detonates into a Cartesian explosion. Records go missing, or come back wrong for an hour and then quietly fix themselves. If nothing ever changed, data engineering would be easy. But as they say, the only constant is change.

The boring work is where agents thrive

Unglamorous maintenance is something agents are genuinely good at. Every data model is a stack of assumptions: this is unique, that’s always populated, these two tables join cleanly. An agent can read those assumptions out of your code and turn them into tests that check whether they still hold. A lot of the fixes are mechanical anyway: a table got renamed, a type got widened, a column moved. An agent can often patch those on its own, and when it can’t, it can still do the legwork, tracing what changed and handing a human a diagnosis and a proposed fix instead of just a 3am stack trace.

Context is the other half of the story, and the context landscape is honestly a mess. Vendors are working hard to convince you that only their semantic modeling language can save you, while it is not entirely clear whether these are necessary or even sufficient. Whether you keep your business logic in a semantic layer like MetricFlow or Malloy, or just in plain Markdown, the goal is the same: get that logic into a form an LLM can use. Context is almost always created by hand, and like all hand-written documentation, it starts drifting the moment it gets written down.

This highlights an opportunity, namely that agents are good at precisely the parts of context that are mechanical and bad at precisely the parts that aren’t. An agent can infer which tables join to which, what values a column tends to hold, what your sales regions are, and which tables people actually query. What it can’t infer is the stuff that was never really a data question: the right way to calculate revenue, what counts as a “customer,” when the fiscal year starts. Those aren’t facts hiding in the warehouse waiting to be found. They’re decisions, often business ones, that a person has to make. What an agent can do is flag the moment one of them quietly stops being true.

Automated agent insights remain a fantasy

The flashier pitch, where agents surface insights you never asked for, is the one I’d bet on last. It sounds wonderful to have hands-free analytics. An agent will keep watch over your data, notice what matters, and drop a dashboard tailored to whatever is happening today. But the bar is high for relevance and false positives can make human users lose confidence.

Deterministic alerting systems have the same problem. People end up turning off alarms because they are too hard to tune. But if humans writing pre-canned triggers have a hard time getting it right, it is going to be hard for agents to do better (at least not before we get some form of super-intelligence). While I’d expect proactive insights to be part of the future, they are still a research prototype at this point.

Here are three concrete things a data team should do to get their stack ready for agents:

1. Lay the groundwork first. Agent use cases that are compelling sit on top of groundwork most teams haven’t laid yet. You don’t need an agent to curate your context until you’ve decided how your context is going to work in the first place.
2. Then go after context. Write a handful of evals, automate them, and then wait to see what breaks. Evals are the load-bearing part. They’re what makes it safe to let an agent near your pipeline at all, because they tell you the instant it gets something wrong.
3. Run on infrastructure that fits how agents behave. An agent goes from zero to a flood of queries in an instant, so you want something that scales up and back down quickly. Agents also fan out, chasing several threads at once, so you need both the headroom and the tenant isolation to absorb a burst. One agent’s curiosity shouldn’t take down everyone else’s ability to run queries.

Latency is a bigger deal than it looks

Latency matters more than you’d expect when you’re using agents. While you might be waiting seconds or minutes for Claude Code to do its thing, it is often running a bunch of tasks. Part of the time that the agent spends is waiting for the LLM, but an increasing amount of time is using other tools, like querying a database. Over time, you can expect LLMs to get a lot faster; you can use smaller models, smarter models, local models, or fancier GPUs. As that happens the tools that an agent uses become the bottleneck.

Picture two engines: one answers in 10 milliseconds, the other in 100. A person won’t notice the difference because both feel near instantaneous, and a person will spend far longer thinking up the next question than either engine spends answering it. What feels instantaneous to an agent is very different, and it doesn’t need to stop and think. When its next query depends on the last result, that 10x gap compounds straight into 10x more work per minute.

One of the ways to make an agent go faster is to take more of their work and run it in parallel. But this also increases load on the systems. You’d want to make sure you have enough parallel capacity and isolation to be able to scale to all of the parallel agent queries at once. Engines tuned for human patience and engines tuned for agent throughput are not the same engines.

The agentic wave is coming whether or not any given team is ready, and the best time to start preparing yourself and your stack is now, before the queries start pouring in. This isn’t just future proofing. The teams that move early are the ones who work out the patterns everyone else ends up copying. A little curiosity now buys a real head start later.

• Electric is joining Databricks to bring WASM Postgres to AI agent sandboxes.
• Electric has pioneered data primitives purpose-built for agents, including PGlite and a real-time sync engine to keep data synchronized between distributed agents and a centralized Lakebase.
• Extending Databricks’ Postgres capabilities from the lakehouse to the edge allows for an abundance of lightweight open-source databases that can all be synced back to Lakebase Postgres for centralization and control on cheap, durable object storage.

Today, we’re excited to welcome Electric to Databricks. The world is building a new era of agentic applications which require distributed state and real-time data synchronization between teams of agents working in sandboxes. Electric is leading the way in pioneering data primitives purpose-built for agents: PGlite gives every agent its own lightweight Postgres right where it runs, providing ultra-low latency access to local context, and Electric’s real-time sync engine synchronizes distributed state back to a central Lakebase, enabling teams of agents to collaborate without losing track of shared context. That vision now continues at Databricks as we bring WASM Postgres to AI agent sandboxes, extending Databricks’ Postgres capabilities from the lakehouse to the edge.

Agents don’t behave like traditional applications

Developers are moving from building traditional apps to building agentic applications, and the assumptions underlying traditional infrastructure are shifting with them. A traditional application has a known shape: its queries are written in advance, its data access patterns are predictable, and it runs in a place its architects chose. One managed Postgres database serves it well.

The agents in agentic applications add a second surface. Alongside the durable, governed state every application needs, an agent generates a fast-moving context set. Agents differ in three ways that matter for a database:

• They decide what data they need at runtime, deciding their next move and updating their context several times a second. That inner loop wants data in the same process; the results it produces belong in a durable store.
• They run wherever the work is, often in sandboxed environments, where databases are only reachable via a network connection to the cloud.
• They work in groups, requiring fast local context and a shared, current view of what others have done to avoid duplicating work, acting on stale state, or arriving at conflicting conclusions.

Introducing Electric

The Electric team built data primitives for the era of agentic applications.

They created PGlite to push Postgres to the edge, creating a WASM Postgres database small enough to run inside the application or agent itself - in an agent sandbox, browser tab, or user’s device - rather than on a separate server. PGlite has grown from 1M to 13M weekly downloads in just twelve months, enabling developers to build a new class of distributed Postgres applications and agents.

But local execution is just part of the challenge. Agents also need a shared, up-to-date view of changing information. Electric’s real-time sync engine continuously synchronizes data between distributed agents and centralized Lakebase infrastructure, enabling agents to securely share information with the definitive record in the cloud while keeping fast local context. The real-time sync architecture powers collaborative apps like Google Docs, Figma and Notion, and it turns out to be exactly what a fleet of agents needs to stay current while working in parallel.

Shared Postgres DNA

Both Electric and Lakebase are built on Postgres, the open-source database technology that has become the default foundation for AI agents. In fact, PGlite was built on the foundational WASM Postgres work of Stas Kelvich, who co-founded Neon. Electric took that proof of concept and turned it into the embeddable Postgres that millions of projects run every week. Bringing these teams together reunites two halves of the same idea and strengthens Databricks’ leadership at the center of innovation for modern databases as demand for agents accelerates.

Databricks + Electric

Lakebase delivers Postgres at production scale. PGlite brings WASM Postgres into the agent's own sandbox. Sync keeps the two consistent. By combining the power of Lakebase Postgres with the local execution of PGlite, agents get the lightweight databases they need to move fast and the infrastructure required to deploy at any scale.

For developers, this means:

• Building collaborative, agentic applications on a single Postgres standard.
• Running Postgres directly inside the agent sandbox, giving each agent a lightweight database it needs to move fast.
• Keeping teams of agents in sync to instantly share context and act on fresh data, while centralizing control in Lakebase Postgres on cheap, durable object storage.

We’re thrilled to have the Electric team join Databricks, and can’t wait to see what our customers build.

Formatting SQL tends to bring some of the same questions again and again: should we uppercase clause keywords? should we put the separating comma at the start of a line to ease refactoring? how to align the SQL clauses with one-another?

Over the years I have grown my own SQL style and didn’t find tooling that would implement it. Also, I’ve been asked here and there if there is a tool that would replicate The Art of PostgreSQL SQL indentation style… and now there is finally a good answer to that question!

sqlfmt is a gofmt-style formatter that implements my own favorite SQL indentation style. One opinionated style, no configuration knobs. Run it, commit the result, move on.

The style

The formatting convention comes from The Art of PostgreSQL — specifically, from the hand-formatted query corpus that runs through the book’s several hundred worked examples.

The defining characteristic is river alignment: at each query nesting level, every clause keyword (select, from, where, group by, having, order by) is right-padded to the same column, so the keywords form a vertical river and the expressions that follow them flow naturally to the right.

Take a flat, unformatted query:

select status, count(*) from results join races using(raceid) where date >= :season group by status having count(*) >= 10 order by count(*) desc;

After sqlfmt:

  select status, count(*)
    from results
         join races using(raceid)
   where date >= :season
group by status
  having count(*) >= 10
order by count(*) desc;

Every keyword above ends at the same column. group by and order by are eight characters — longer than select’s six — so they sit flush-left at base indent. That is a side effect of the alignment rule, not a separate exception for those keywords.

The full rule set is documented in STYLE.md in the repository, reverse-engineered from the 343 hand-formatted .sql files in the book’s own query corpus. A few highlights:

• All SQL keywords and function names are lowercase — count(*), coalesce(...), row_number() over(...).
• Trailing commas on column lists, one column per line after the first.
• and/or at the start of continuation lines, right-aligned to end at the same column as where.
• Columns in CREATE TABLE are left-padded so every data type starts in the same column.
• Comments are never discarded — leading comments are reindented and reflowed to 78 columns; trailing comments in a block are padded to a shared column.

That style is known to best fit SQL queries maintained in their own .sql files rather than integrated in another source code file as a string.

If you’re not using an ORM but also maintaining hand-written SQL as a set of static strings within another programming language’s source code, it might be time to see how to manage SQL queries in their own files.

Try it now — no install required

The web tool runs the exact same Go engine as the CLI, compiled to WebAssembly with TinyGo. The compressed payload is roughly 130 KB — a better fit for a web page integration compared to the 2.9 MB a standard Go WASM build would produce.

CLI usage

The interface follows gofmt exactly:

sqlfmt query.sql              # print formatted output to stdout
sqlfmt -w query.sql           # rewrite the file in place
sqlfmt -l queries/**/*.sql    # list files whose formatting would change
sqlfmt -d query.sql           # show a unified diff instead of full output
cat query.sql | sqlfmt        # stdin → stdout, pipeable

The -l flag is useful in CI: exit code 1 if any file would change, so a sqlfmt -l $(git diff --name-only '*.sql') step enforces style on every pull request without storing the formatted output in the pipeline.

Install with:

go install github.com/dimitri/sqlfmt/cmd/sqlfmt@latest

Editor integration

Emacs

Drop sqlfmt.el on your load path and add a hook:

(add-to-list 'load-path "~/dev/sqlfmt/editors/emacs")
(add-hook 'sql-mode-hook #'sqlfmt-mode)

With sqlfmt-mode active, C-M-h selects the statement at point and TAB reformats it. sqlfmt-before-save-hook can be used for format-on-save.

Vim / Neovim

The plugin wires sqlfmt into Vim’s formatprg/equalprg, so the usual motion operators work:

gqip    " reformat the paragraph under the cursor
gg=G    " reformat the whole buffer
:%!sqlfmt

Why a tokenizer, not an AST

Most production SQL formatters — pg_format, sqlfluff, prettier-plugin-sql — are built on token streams rather than parse trees, and sqlfmt follows the same approach. Two reasons matter in practice:

Comments. PostgreSQL’s own parser discards comments; any AST-based formatter needs a separate comment-recovery pass. At that point most of the advantage of “let the parser handle structure” is already gone.

Robustness. The web widget needs to handle whatever a visitor pastes — partial statements, snippets from a larger file, syntax that isn’t perfectly valid. A token-stream approach degrades gracefully; a grammar-based one fails hard.

River alignment is fundamentally about where tokens sit on the page, not about the query’s semantic structure, so the tokenizer approach fits the problem naturally. The test suite uses pganalyze/pg_query_go (wrapping the real PostgreSQL C parser) as a correctness oracle: if fingerprint(input) == fingerprint(format.Format(input)), formatting never silently changed what the query means.

Status

The formatter is a working implementation. The tokenizer, river-alignment layout engine, comment attachment, and CLI are all in place and covered by a round-trip corpus test against real book queries. Recent fixes closed several edge cases: <-> (KNN distance operator) was previously mis-lexed; UNION ALL/INTERSECT/EXCEPT between CTEs now correctly resets the river; with recursive no longer silently drops the RECURSIVE keyword.

Deeply nested subqueries and exotic DDL remain best-effort — STYLE.md itself acknowledges these as the least mechanically rigid areas of the style, where the book corpus shows hand-tuning rather than a consistent rule.

The source is at github.com/dimitri/sqlfmt. The live formatter is at theartofpostgresql.com/postgresql-sql-formatter. Who knows, this might prove useful beyond now being able to answer when asked about which tooling I’m using to format the SQL queries in the book!

Atomic Batch Publishing in NATS 2.12: All-or-Nothing Message Guarantees

NATS 2.12 introduces a powerful new feature called atomic batch publishing. If you’re building event-sourced systems or need to fan out messages to multiple services with consistency guarantees, this one’s for you.

The Problem with Partial Writes

Without atomic batching, each publish operation is independent. Imagine you need to write five related messages and your connection drops after the third. You’re left with a partial write—your data is inconsistent, and your system is in an undefined state.

Atomic batch publishing solves this by giving you all-or-nothing guarantees: either every message in your batch commits together, or none of them do.

What This Unlocks

Avoiding partial writes with atomic batching is the foundation for several important patterns:

• Consistent multi-consumer fan-out. When multiple services consume from the same stream, they all see the exact same set of messages in the same order. No service sees a partial batch while another sees the full batch.
• Idempotent full-state replacement. Instead of publishing deltas, you can publish complete state and let consumers replace what they have. When your full state spans multiple messages, atomic batching ensures consumers never see a half-written snapshot.
• Snapshot + delta bootstrapping. New consumers can load a point-in-time snapshot and then apply only the events that came after. Without atomic batching, a snapshot written as multiple messages could be incomplete when read.
• Multi-event commits. A single logical operation often produces multiple events. For example, an order might emit OrderCreated, InventoryReserved, and PaymentInitiated. Consumers should see all three or none—not a partially committed transaction.

How It Works

Atomic batch publishing is a stream configuration option that lets you atomically publish multiple messages into a JetStream stream.

Here’s what happens under the hood:

1. Staging: When you start a batch, messages are staged invisibly on the server. Consumers don’t see anything yet.
2. Sequencing: Each message gets a batch sequence number, and your messages sit in a staging area waiting.
3. Commit: On your last message, you include a commit header. The server then commits everything atomically—all messages appear at once.

The Batch Headers

The server tracks batches using three headers:

Validation Rules

Your batch will be rejected if:

• The batch ID exceeds 64 characters
• There are gaps in your sequence numbers
• The stream uses async persist mode (which can’t guarantee atomicity)
• Duplicate messages exist in the batch (as of 2.12.1)

Optimistic Locking with Expected Sequence Headers

For scenarios where you need to guard against concurrent writes, atomic batching supports optional consistency checks:

• NATS-Expected-Last-Sequence: Only commit this batch if the stream’s last sequence matches this number. If someone else published while you were building your batch, the sequence moved, your batch fails, and you retry with fresh data. This header is only allowed on the first message of a batch.
• NATS-Expected-Last-Subject-Sequence: Same concept, but scoped to a specific subject.

Limits

The defaults are sensible, but all are configurable in your server configuration:

• 1,000 messages per batch
• 50 batches in flight per stream
• 1,000 batches total per server
• 10 seconds of inactivity before a batch is abandoned

One important constraint: batches must go into a single stream—they can’t cross streams.

Use Cases

1. Event Sourcing: When a single command produces multiple events that must be written together to maintain consistency. Your event history stays coherent.
2. Batch Telemetry / Fan-out: When you need to send related messages to multiple services atomically. Either every service gets their message, or none of them do.

Getting Started

Creating an Atomic-Enabled Stream

nats stream add orders --subjects="orders.>" --allow-batch

CLI Example: Single Subject

# Terminal 1: Subscribe
nats sub "orders.>"
# Terminal 2: Publish atomically
nats pub orders.events --send-on-newline --atomic
> order_created
> payment_received
> order_shipped
> ^D  # Ctrl+D to commit

CLI Example: Multiple Subjects

# Set a batch ID
BATCH_ID=$(uuidgen)
# Publish to different subjects with batch headers
nats pub orders.warehouse "ship item" \
  --header "NATS-Batch-ID:$BATCH_ID" \
  --header "NATS-Batch-Sequence:1"
nats pub orders.payment "charge card" \
  --header "NATS-Batch-ID:$BATCH_ID" \
  --header "NATS-Batch-Sequence:2"
nats pub orders.notification "send email" \
  --header "NATS-Batch-ID:$BATCH_ID" \
  --header "NATS-Batch-Commit:true"

Go Code Example

func runBatch(js jetstream.JetStream) error {
    messages := []string{"order_created", "payment_received", "order_shipped"}
    batchID := uuid.New().String()

    for i, msg := range messages {
        m := nats.NewMsg("orders.events")
        m.Data = []byte(msg)

        // Set batch headers
        m.Header.Set("NATS-Batch-ID", batchID)
        m.Header.Set("NATS-Batch-Sequence", strconv.Itoa(i+1))

        // Set commit header on last message
        if i == len(messages)-1 {
            m.Header.Set("NATS-Batch-Commit", "true")
        }

        if _, err := js.PublishMsg(context.Background(), m); err != nil {
            return err
        }
    }
    return nil
}

A Cleaner Approach with Orbit

Orbit is Synadia’s higher-level client library that simplifies NATS development. Its JetStream extension package handles all the batch header management for you:

batch, err := jetstreamext.NewBatchPublisher(js)
if err != nil {
    return err
}

batch.Add("orders.warehouse", []byte("ship item"))
batch.Add("orders.payment", []byte("charge card"))

ack, err := batch.Commit(ctx, "orders.notification", []byte("send email"))
if err != nil {
    // None of the messages were published
    return fmt.Errorf("batch publish failed: %w", err)
}
// All messages published successfully!

Wrapping Up

Atomic batch publishing in NATS 2.12 eliminates the partial write problem that has plagued distributed systems. Whether you’re doing event sourcing or need coordinated message delivery across services, this feature gives you the consistency guarantees you need.

Maestro

Maestro is a general-purpose workflow orchestrator that provides a fully managed workflow-as-a-service (WAAS) to the data platform users at Netflix.

It serves thousands of users, including data scientists, data engineers, machine learning engineers, software engineers, content producers, and business analysts, for various use cases. It schedules hundreds of thousands of workflows, millions of jobs every day and operates with a strict SLO even when there are spikes in the traffic. Maestro is highly scalable and extensible to support existing and new use cases and offers enhanced usability to end users.

You can read more details about it in our series of blog posts

• Maestro: Data/ML Workflow Orchestrator at Netflix
• Orchestrating Data/ML Workflows at Scale With Netflix Maestro
• 100X Faster: How We Supercharged Netflix Maestro's Workflow Engine
• Incremental Processing using Netflix Maestro and Apache Iceberg

Get started

Prerequisite

• Git
• Java 21
• Gradle
• Docker

Build it

• ./gradlew build

Run it

• ./gradlew bootRun

Run it with AWS module

• docker compose -f maestro-aws/docker-compose.yml up

• ./gradlew bootRun --args='--spring.profiles.active=aws'

Create a sample workflow

• curl --header "user: tester" -X POST 'http://127.0.0.1:8080/api/v3/workflows' -H "Content-Type: application/json" -d @maestro-server/src/test/resources/samples/sample-dag-test-1.json

Get the sample workflow definition

• curl -X GET 'http://127.0.0.1:8080/api/v3/workflows/sample-dag-test-1/versions/latest'

Trigger to run the sample workflow

• curl --header "user: tester" -X POST 'http://127.0.0.1:8080/api/v3/workflows/sample-dag-test-1/versions/latest/actions/start' -H "Content-Type: application/json" -d '{"initiator": {"type": "manual"}}'

Get the sample workflow instance

• curl -X GET 'http://127.0.0.1:8080/api/v3/workflows/sample-dag-test-1/instances/1/runs/1'

Delete the sample workflow and its data

• curl --header "user: tester" -X DELETE 'http://127.0.0.1:8080/api/v3/workflows/sample-dag-test-1'

Run it with maestro-extensions (foreach flattening service)

The maestro-extensions module runs as a separate Spring Boot service that listens to maestro events via SQS (subscribed to the SNS topic maestro-server publishes to) and provides additional functionality such as foreach step flattening views.

To run maestro-server and maestro-extensions together locally:

1. Start LocalStack (provides local SQS/SNS):

   • docker compose -f maestro-aws/docker-compose.yml up -d

2. Start maestro-server (port 8080):

   • ./gradlew :maestro-server:bootRun --args='--spring.profiles.active=aws'

3. Start maestro-extensions (port 8081):

   • ./gradlew :maestro-extensions:bootRun

Once both services are running, maestro-extensions will consume step instance status change events from the maestro-event SQS queue and process foreach flattening. Query the flattened views via the extensions REST API on port 8081.

Run it with Kubernetes support

• setup kubernetes configs so the kubectl command works

• ./gradlew bootRun

• curl --header "user: tester" -X POST 'http://127.0.0.1:8080/api/v3/workflows' -H "Content-Type: application/json" -d @maestro-server/src/test/resources/samples/sample-kubernetes-wf.json

• curl --header "user: tester" -X POST 'http://127.0.0.1:8080/api/v3/workflows/sample-kubernetes-wf/versions/latest/actions/start' -H "Content-Type: application/json" -d '{"initiator": {"type": "manual"}}'

Python SDK client

Installation

pip install maestro-sdk

Creating a workflow

from maestro import Workflow, Job

wf = Workflow(id="test-wf")
wf.owner("tester").tags("test")
wf.job(Job(id="job1", type='NoOp'))
wf_yaml = wf.to_yaml()

Pushing a workflow to Maestro server

from maestro import Workflow, Job, MaestroClient

wf = Workflow(id="test-wf")
wf.owner("tester").tags("test")
wf.job(Job(id="job1", type='NoOp'))
wf_yaml = wf.to_yaml()

client = MaestroClient(base_url="http://127.0.0.1:8080", user="tester")
response = client.push_yaml(wf_yaml)
print(response)

Starting a workflow

from maestro import MaestroClient

client = MaestroClient(base_url="http://127.0.0.1:8080", user="tester")
response = client.start(workflow_id="test-wf", run_params={"foo": {"value": "bar", "type": "STRING"}})
print(response)

Please check Maestro python project for more details.

Get in touch

Join our community Slack workspace for discussions!

License

Copyright 2024 Netflix, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

How to Track Token Cost Across LLM Workflows

For teams shipping AI products, token usage is a core unit-cost signal behind summarization, support answers, agent steps, code suggestions, and retrieval workflows. This guide shows how to track token usage and cost so teams can attribute spend, catch inefficiencies, and make better AI investment decisions.

AI Model Drift: How to Keep Models Reliable

AI model drift is when an AI system's performance and accuracy degrades over time because the data, user behavior, or business environment has changed since the model was trained or evaluated. Even if latency, uptime, and infrastructure metrics remain healthy, model quality can quietly decline, leading to less accurate predictions, inconsistent responses, and reduced user trust.

Production teams care about AI model drift because reliability isn't just about whether a model is available. It's about whether it's still delivering the outcomes users expect. As AI-powered features become more deeply integrated into products, even subtle changes in model behavior can affect customer satisfaction, engineering productivity, and business performance. This guide explains what AI model drift is, what signals matter, and how observability helps teams catch problems sooner.

What is AI model drift?

AI model drift is a gradual decline in a model's performance or accuracy as production data and real-world conditions diverge from the baseline used during training or evaluation. This gradual performance decline is also sometimes described as model decay. Unlike a software bug or service outage, model drift doesn't usually appear overnight. Instead, it develops over time as user behavior, business requirements, or underlying data evolves.

A recommendation model, for example, might be trained on historical shopping behavior from the previous year. Months later, seasonal buying trends, new product launches, and changing customer preferences alter the kinds of products users search for and purchase. The model continues returning predictions without errors, but its recommendations become less relevant because the world it learned from no longer reflects reality.

Modern AI systems make this challenge even more complex. Foundation models, retrieval systems, prompts, embeddings, and external tool calls all evolve independently, creating multiple opportunities for performance to drift, even when the underlying model hasn't changed.

What types of AI model drift should teams monitor?

Not all model drift looks the same, and detecting it requires teams to understand where changes originate. Some changes begin in the data a model receives, while others stem from shifting user expectations, upstream systems, or the behavior of LLM-powered applications. Understanding these patterns helps engineering teams identify where to investigate first instead of treating every quality issue as a generic model problem.

These four categories cover many of the drift patterns teams encounter in production.

Rather than viewing these categories in isolation, production teams often encounter several forms of drift simultaneously. A schema change in a retrieval pipeline, for example, may alter the context presented to an LLM and reduce response quality.

Data drift changes the inputs a model sees

Data drift is when the inputs arriving in production no longer resemble the data used to train or validate a model. The model itself hasn't changed, but the world around it has.

This is one of the most common forms of AI model drift. An e-commerce platform may see new search behavior during the holiday season. A financial application might experience different spending patterns during periods of economic uncertainty. A customer support chatbot could receive questions about a newly launched product that never appeared in its original training data.

Because the model is operating outside the environment it learned from, prediction quality can gradually decline.

Engineering teams typically detect data drift by monitoring changes in input distributions, feature statistics, request categories, or traffic composition.

These changes give teams an early indication that production traffic is moving away from its baseline.

Concept drift changes what a correct answer means

Concept drift is when the relationship between an input and the correct output changes.

For example, a fraud pattern that was reliable six months ago may no longer indicate fraud. An answer that previously satisfied users may become outdated because company policies, market conditions, or customer expectations have changed. Concept drift is often visible first through outcome signals such as declining engagement, negative feedback, reduced task completion, or increasing human escalations.

Upstream changes can break model assumptions

Not every model problem originates in the model. Schema changes, pipeline bugs, missing data, and deprecated sources can silently corrupt the information reaching it. The model then behaves differently because its inputs are incomplete or incorrectly formatted. In this case, retraining the model will not solve the problem. The fix belongs in the upstream data pipeline, transformation logic, retrieval process, or application integration.

Prompt, embedding, and output drift change outputs over time

LLM applications themselves introduce additional forms of drift.

Prompt drift is when the prompts a system sends to a model change over time, whether through template updates, accumulated small edits, or a model update that changes how the same prompt is interpreted. A related pattern is input drift, where users gradually adopt new vocabulary, workflows, or interaction patterns for the same task.

Embedding drift is a shift in the distribution of embeddings over time, which can happen when the embedding model or indexing process changes, or simply because the content being embedded has changed. RAG systems can also experience corpus or retrieval drift as documents are added, removed, rewritten, or reindexed.

Output drift is a change in tone, relevance, accuracy, structure, or consistency of generated responses. It is often the most visible symptom of drift elsewhere in the system, such as a provider updating a hosted model, a prompt template edit, degraded retrieval quality, or changes in sampling settings like temperature.

These changes can reduce AI reliability without producing an obvious application failure.

Why AI model drift is challenging to detect in LLM and agentic systems

Teams do not always describe the symptoms they see as “model drift.” They may first notice rising token usage, failed tool calls, weaker answers, more retries, prompt failures, or unpredictable agent behavior.

The right drift detection methods depend on the model, available labels, traffic volume, and business risk. For example, LLM model drift detection requires teams to monitor prompts, retrieval behavior, generated outputs, and downstream outcomes together. Here are some reasons why model drift can be difficult to detect in LLM and agentic systems.

Prompt and embedding shifts create new failure patterns

LLM systems operate on open-ended inputs that change constantly. New users bring different language patterns and expectations, while existing users discover new ways to interact with the product.

In RAG systems, the retrieval corpus also changes. Documents may be added, removed, rewritten, or indexed differently. The system may still respond, but changes to the corpus, embeddings, or retrieval behavior can lead to weaker context and less relevant answers.

Effective LLM observability therefore requires more than monitoring the model endpoint. Teams also need to track how prompts, embeddings, retrieval results, and model versions influence the final response.

Output quality can drift before alerts fire

Traditional alerts are designed to detect explicit failures, such as high latency, elevated error rates, or unavailable infrastructure. Model quality is different. The request may complete successfully while giving the user an inaccurate, inconsistent, or unhelpful answer. Teams need output-level signals such as:

• Evaluation scores
• User feedback
• Task success
• Escalation rates
• Retry rates
• Downstream business results

These measurements show whether a statistical change is actually affecting users.

Agents add tool-use and workflow drift

Agents add another layer of complexity because they do more than generate text. They call tools, retrieve information, hand work to other agents, make decisions, and execute multi-step workflows.

As these workflows evolve, failures can emerge in places that traditional model monitoring never observes.

A retrieval tool might begin returning lower-quality results after a document migration. An API integration could introduce longer response times that change an agent's decision-making behavior. A downstream service may start returning incomplete data, causing the agent to produce weaker recommendations.

Because any step can affect the result, teams need to trace the workflow from the initial prompt through retrieval, tool execution, downstream dependencies, and the final outcome. This makes it possible to identify where behavior changed instead of attributing every problem to the model.

Approaches for detecting agent drift connect those steps so teams can evaluate the workflow as a whole.

How do teams detect AI model drift early?

Detecting AI model drift isn't about finding a single metric that signals failure. It's about establishing a baseline, comparing production behavior against that baseline over time, and investigating meaningful changes before they affect users or business outcomes. The most effective monitoring strategies combine statistical analysis with production telemetry, quality evaluation, and business metrics to determine not only if something changed, but also whether the change actually matters.

Baselines make drift measurable

Drift only has meaning relative to a reference point. Without a baseline, it's impossible to determine whether a model is behaving as expected or gradually diverging from previous performance.

Teams can use several types of baselines depending on the application. Common baselines include:

• Training or validation datasets for traditional ML models.
• A known-good production window, such as the two weeks following a successful release.
• A rolling baseline that continuously compares current traffic against recent production behavior.
• A curated evaluation dataset used to measure output quality over time.
• A specific version of a prompt, model, retrieval index, embedding model, or feature pipeline.

There isn't a universal baseline that works for every AI system. The right baseline depends on traffic volume, release frequency, business risk, and how quickly normal behavior changes.

Teams with rapidly evolving customer behavior may prefer rolling production windows, while highly regulated applications often compare against fixed evaluation datasets to maintain consistency across releases.

The important principle is consistency: meaningful drift detection starts with knowing what "normal" looks like for your application.

Statistical signals show when something changed

Teams can compare live and baseline distributions using methods such as the following:

• Population Stability Index
• KL divergence
• Wasserstein distance
• Embedding similarity or clustering changes

For LLM applications, teams can also monitor prompt categories, token counts, request lengths, retrieved document similarity, embedding distributions, and shifts in user intent.

Anomaly detection can flag signals that move outside their expected range, while heuristics can encode application-specific warning signs. A team might flag an unusual change, a drop in retrieval similarity, or a sharp change in tool selection patterns.

Statistical changes alone do not always mean the model is failing. They are a signal to investigate.

Quality and business signals show whether it matters

The strongest model drift detection strategies pair statistical differences and anomalies with production outcomes, such as:

• Lower evaluation or output quality scores
• More negative user feedback
• More human escalations or support requests
• Lower task completion, conversion, or engagement
• Higher retry rates
• More failed or unnecessary tool calls

When a distribution shift coincides with worsening user or business outcomes, teams have stronger evidence that intervention is necessary.

Drift detection should trigger investigation, not just alerts

A practical monitoring process should:

1. Choose a relevant baseline.
2. Sample representative production traffic.
3. Track input, output, and outcome signals.
4. Define heuristics and anomaly thresholds based on risk.
5. Alert on changes that are likely to matter.
6. Investigate using traces, request context, and version history.
7. Refresh baselines as behavior and business goals evolve.

What role does observability play in model drift detection?

Dashboards can reveal that a metric changed. Observability helps teams understand why.

With AI observability, teams can connect prompts, model versions, retrieval paths, tool calls, user attributes, system behavior, and business outcomes within the same investigation.

That context is particularly important for non-deterministic systems. Two similar requests may take different execution paths, retrieve different documents, or receive different model responses. Tracing those paths helps engineers distinguish model decay from a prompt regression, retrieval problem, provider update, or downstream service failure.

Honeycomb's AI agent monitoring brings LLM calls, tool invocations, agent handoffs, failures, and downstream system behavior into a unified chronological view. That makes it easier to reconstruct what happened instead of piecing together disconnected logs, traces, and dashboards.

How Honeycomb helps teams keep AI models reliable

Honeycomb provides AI and LLM observability across model interactions, application behavior, and the downstream systems supporting them.

By instrumenting prompts, model calls, evaluations, retrieval operations, token usage, tool invocations, and downstream services, teams can investigate why AI behavior changed and determine whether the same behavior can be reproduced.

For agentic workflows, Agent Timeline organizes multiple traces and agents into one conversation-level view. Engineers can follow model calls, tools, handoffs, retries, and failures while retaining access to the underlying application and infrastructure traces.

Model drift cannot always be prevented. With the right telemetry and investigation context, teams can recognize it earlier, understand its impact, and respond with the correct change.

AI model drift FAQs

• When does model drift require retraining versus a prompt, retrieval, or pipeline update?
• How can teams detect AI model drift when they do not have ground-truth labels?
• What signals help separate harmless behavior changes from meaningful model drift?
• Can model drift happen after a third-party model provider update?
• Who should own AI model drift monitoring across engineering, ML, and product teams?
• How should teams document model drift investigations for future releases?

Does anyone run Postgres without PgBouncer?

I got a nice shout-out from Ben Dicken over the weekend on an old article I’d written on managing database connections. (This guy is apparently the Mick Jagger of databases, because I can’t remember having gotten so many inbound LinkedIn invitations in one day before.)

Something that hit hard is that I wrote this article almost ten years ago.

Just as striking is that as I was reading back through it, I realized that despite being a decade old, it’s still pretty much up to date. Postgres is still, shall we say, not great at managing lots of connections, so you want to use local connection pools, short-term checkouts, and a pooler like PgBouncer.

It got me wondering: how standard is it to use a pooler, exactly? To answer that question, I made a table of all managed Postgres providers with household notoriety and whether they support PgBouncer, something close to PgBouncer, or no connection pooling at all.

Not only is PgBouncer support widespread, but we see above that the overwhelming majority of providers bundle it out of the box. I’d go a step further – since neither IBM nor Oracle is a service that any self-respecting person not part of an enterprise sales cycle would actually use, one hundred percent of plausible managed Postgres providers bundle a pooler.

If everyone needs it, is it really a non-core function?

In some ways, it could be argued that this status quo is okay. Users that need a connection pooler have access to one, and can use it to keep prod stable.

But there’s undoubtedly a lot of wasted effort here. Every provider has had to come up with their own homegrown mechanism for getting multiple components set up and configured and establish a convention for where to find Postgres versus its bouncer. Every user needs to reference a guide explaining PgBouncer’s limitations (e.g. don’t listen/notify) and read about its pooling modes and tradeoffs.

Imagine if you went to your local car dealership and they sold you a car without a windshield. On the way over you’d noticed that 100% of vehicles on the road did in fact have windshields, and for good reason because it turns out to be pretty dangerous to drive without one. Since you were the one that bought the car, it’d be hard to argue that it’s not your responsibility now to outfit it with a windshield before it’s roadworthy, but it’d also be fair to later be pissed off at the dealer for selling a vehicle that can’t just be driven off the lot.

Reintegration

What if there was a world where you went to your favorite Postgres provider and you got one database URL, one port, and no extra configuration or caveats to worry about? Your managed provider doesn’t need to add an aftermarket windshield because one came with the car already. We know a place like this can exist because that’s already how things work in MySQL and Mongo-land.

There are reasons it doesn’t happen, like reviving the age-old processes versus threads debate, which very few contributors are venerated enough to push for progress on, but given the developer-years’ worth of effort in working around Postgres’ lack of connection pooling, it’s hard to argue this wouldn’t be one of the highest-impact operational improvements possible.

Try skills from the Community and make your own with the Figma agent

We've added more ways to discover, create, and share skills for the Figma agent. What's new:

• Agent skills in the Figma Community: Browse through 50+ skills for design workflows—from research and design systems to handoff—in the AI skills library. Add skills to your agent to try them for yourself, or copy and paste markdown files to your agent of choice.
• Create skills with the agent: Ask the Figma agent to create a skill based on context in a file to package up repeatable workflows. The newly created skill can be invoked with a / in the Figma agent.
• Publish skills to the Figma Community: Add your skill to the AI skills library on the Figma Community for anyone to try or remix it for their workflows.

Taste Is All That's Left

For most of the time I have been writing software—which, compared to some of my readers, is not that long—I have come to believe that the hard thing was making the thing exist at all. This is not necessarily a new belief of mine. I came up through the difficult and tedious experience of building web applications and watching them crash and burn.

You had an idea, and between the idea and the working program stood hours— sometimes weeks—of typing, of reading manuals, of misunderstanding an API and slowly grinding the wrong version into a slightly less wrong one. Production was the wall. Everyone hit it. It was the thing that separated the people who could from the people who could only talk about it.

That wall is gone. Or rather, it has been rented out. You can describe a thing now and receive a plausible version of it much faster than you could have typed the first function by hand. The idea-to-artifact distance, the one that defined the entire craft, has collapsed to almost nothing.

Though, you have not been warned about one little thing: the value you built by learning to climb that wall does not disappear. It simply… moves.

The Bar Went Somewhere

We keep asking whether the machines are any good. Even yesterday I had a rather short discussion on whether they are reliable. While we have concluded that they are “reliably unreliable,” I think it is the wrong question. The output is good enough, generally anyway, and that is the problem—most of it, at least. Good enough is a solvent. It dissolves the reason to do better. For as long as making things was expensive, the expense did quiet work on our behalf. It rationed output. It meant that anything which existed had, at minimum, survived the cost of being made. You know what I mean? Effort was a filter, and like all filters it was invisible until it was removed. Nobody shipped a thousand mediocre variations of a feature, because a thousand mediocre variations cost a thousand times as much as one. The economics enforced a floor.

That floor is now gone. And when the floor goes, the thing that decides what is worth keeping is no longer the cost of making it. It is you. Your judgement. The verdict you reach when you look at three plausible versions of the same function and know, somehow, that two of them are wrong. That verdict has a name we are slightly embarrassed to use in engineering circles, because it sounds soft and unfalsifiable and vaguely aristocratic.

Taste.

What Taste Actually Is

I want to be careful here, because “taste” is doing a lot of work and it is easy to hear it as decoration. A matter of preferences. Whether you like your braces on the same line.

That is not what I mean.

Robert Pirsig spent an entire book circling a word he refused to define, because he had convinced himself that defining it would kill it. He called it Quality. His argument, roughly, was that you recognise Quality before you can explain it—that the recognition comes first and the reasons arrive later, if they arrive at all. A good mechanic knows the engine is wrong before he knows why. A good editor feels the sentence sag before she can name the clause that failed.

Taste is that. It is the compressed, wordless verdict you reach faster than you can justify. It is partially the “no, again” you say to yourself with total conviction and no available argument. And it is not soft at all. It is the hardest thing in the work, because it is the only part that was never mechanical to begin with.

Everything downstream of the verdict—the typing, the syntax, the wiring of one library to another—was always, in principle, automatable. We just had not gotten around to it. The verdict was the thing the machine could not do for you.

It still cannot. It can only make the absence of it cheaper to ignore.

Taste Is Downstream of Friction

The mechanism underneath this is one I would rather not think about.

Where did your taste come from?

No really. Where did it come from? Was it genetic? Were you abducted by aliens one day that forcefully injected your sense of taste into your mind and wiped your memory of what just happened?

I’ll tell you this much: it’s not from consuming good work. You cannot read a hundred excellent programs and absorb the judgement by osmosis, any more than you can become a chef by eating in good restaurants. Taste is built the slow, stupid, humiliating way: you make something bad, you are forced to live with it, it fails in front of you, and some part of you files the failure away. Then you do it again. The palate is an accretion of your own mistakes, sat with long enough to sting.

The friction was not an obstacle to developing taste. The friction was the curriculum. Every wall I cursed while climbing it was, without my noticing, teaching me which walls were worth climbing. The cost that rationed my output also educated my judgement, because paying the cost over and over is how you learn what is worth paying for.

So watch what happens when you remove the friction for the next person.

They can generate fluently from the first day. They will never ship the bad version and be forced to sit in it, because the tool offers them a competent version for free. They will climb no wall, and so they will learn nothing from the climb. They will arrive at fluency having skipped the entire apprenticeship that fluency used to require—and they will be more productive than I was at their stage, by every metric anyone bothers to measure.

They will be able to make anything, and unable to tell (or stop to think) whether they should. Not necessarily through any fault of their own. We removed the part of the process that would have taught them, and we called it progress, and by most definitions it was.

The Economics Are Against You

Suppose you have taste. Suppose you paid the full price and you can feel the sag in the sentence and the wrongness in the function.

Congratulations! You now ship at exactly the same speed as the person who cannot.

This is the quiet cruelty of the situation and I do not have a comforting way to phrase it. Taste is slow. It says “no, again.” It sends the plausible thing back because plausible is not the same as right, and while it is doing that, the person without it has already shipped, closed the ticket, and moved on. The market timed you both with the same stopwatch and it did not see the difference. It cannot see the difference. Taste does not show up in the diff.

It is unmeasurable, uncreditable, and invisible on a dashboard. You cannot point to the disasters it prevented, because prevented disasters leave no trace. You carry a cost—the extra hours, the returned work, the refusal to ship the fine thing when the right thing is still reachable—and you carry it alone, against an incentive gradient that runs the other way.

Harry Frankfurt once drew a careful line between the liar and the bullshitter. The liar at least respects the truth enough to work against it. The bullshitter does not care about the truth in either direction; he is simply indifferent to it. Slop is the bullshit of engineering. It is not wrong, exactly. It is indifferent. It works, it passes, it is fine. And fine, produced without friction and shipped without judgement, is now the most abundant substance in the field.

The Flood

Sturgeon said it decades ago, defending science fiction from a critic: ninety percent of everything is crap. He meant it as consolation. Ninety percent of every field is bad, so do not judge the field by its bulk. But the ratio was never the danger. It held steady for centuries. What held the flood back was that producing the crap cost something. Bad novels still took a year to write. Bad software still took a month to build. The ninety percent was throttled at the source by the sheer inconvenience of making it.

We have now removed the throttle and left the ratio intact. Ninety percent of an infinite output is still infinite. The signal did not get worse. The noise became free, and free noise rises without limit, and every real thing you make now arrives into a sea of plausible nothing that looks, at a glance, exactly like it.

Which means the scarce act is no longer making. It is choosing. Deciding what, out of the endless generated plausible, deserves to exist and be kept. Curation was a minor virtue when things were expensive to make. It is the whole game when they are free.

What Deserves to Exist

There is a rhyme here, if you go back far enough.

When the factories came, they could suddenly make everything—cheaply, uniformly, by the thousand. And a handful of people, Morris and Ruskin among them, looked at the flood of cheap identical goods and asked a question that sounded, at the time, sentimental and doomed: not can we make this, but should this be made, and made this way, by no one, for no reason but that the machine could.

They lost the economic argument. They were always going to. But they were right about the thing that mattered, which is that when the making becomes free, the choosing becomes the craft. The human question stops being “can I build it” and becomes “does this deserve to exist”—and that question was always the more serious one. We just could not afford to ask it while we were busy climbing walls.

This turn is not consolation but a correction.

The tools did not devalue the skill. They stripped away everything that was not the skill. All those years I thought the work was the production—the typing, the wiring, the wall—and production turns out to have been the toll. The tax you paid for the privilege of exercising judgement. Now the tax is close to zero, and what is left standing, exposed, with nowhere to hide, is the judgement itself. The part that was always the point.

Taste did not become less valuable. It became the only thing that was ever scarce. We just could not see it, because it was buried under all the labour it used to take to get to it.

A Defense, Then

So here is the defense, such as it is.

Anyone can generate now. That race is over and it was never worth winning. The discipline that remains—the one the machine cannot rent to you, and the dashboard cannot see—is in the deletion. In the “no, again.” In caring about the difference between fine and right when nothing external will ever reward you for caring, when the market has timed you and shrugged, when the plausible version sits there working and passing and asking only to be let through.

Refuse it anyway. Not out of nostalgia for the friction—I do not miss the wall, and I will not pretend to. Refuse it because the verdict is the last part of this that is actually yours. It is unmeasurable, which means no one can take it from you by measuring it. It is unautomatable, which means no one can sell it back to you. It is slow, which in a field optimising for infinite speed is starting to look less like a handicap and more like the only remaining evidence that a human was here and gave a damn.

Everyone can make anything. Almost no one can tell you what is worth making.

That was always the harder skill. It is now the only one left.

Post-Mortem

On Language

This post reads as AI slop. You said it, I see it. I’m sincerely sorry for publishing something that has allowed you to feel this way. If my word means anything to you, I would like to assure you that this post was not authored by a LLM. Nor was it storyboarded, reviewed, checked, etc. by one. Some readers have pointed out that people do not speak this way. That is correct. I do not speak, nor usually write, like this, and this post will go down as not my proudest. However, I take your criticism to heart—although not personally—and strive to improve.

I do write like this sometimes. The short sentences, the reversals, the one-word lines—all of it. They’re mine, and it’s just the way it is. A LLM writes that way too, because it was trained on the same essays I have been reading, so me doing it badly and a machine doing it look about the same to you on the page. That says something about my writing. It says nothing about who wrote it.

So let me be plain about it: Claude was not here. No LLM wrote this—not a sentence of it, nor was it outlined, drafted, reviewed, checked, etc. by one, and there is no prompt behind it either. It is just me, writing worse than usual. I will write the next one plainer. Next time, write to me. I too am a person behind this screen.

In Appreciation

Be assured that I have read all of your comments—the good and the bad. As with my previous post that reached Hacker News, I’ve received many insightful ones. Whether it was people sharing their experience, or negative comments with the decency to criticize with substance, I have learned something new today—for which I am thankful.

On Taste

I do not care about your taste. If this post has offended you, then it says more about you than it does about me. As they say, “throw an insult on the ground, its owner will pick it up”—this one I am not sorry about.

Footnotes

• There is an older word for this arrangement. You no longer own the means of production; you rent them, by the token, from whoever trained the model. An English teacher of mine—a committed socialist—would have had the whole thing diagrammed on the board before I finished the sentence: the worker separated first from his tools, then from the labour itself, then sold a frictionless substitute for the labour and told this was liberation. He would also, I suspect, have been the first to note the one part of the process that cannot be rented back to you, because it never left your head. Draw your own conclusions about which part that is.
• Zen and the Art of Motorcycle Maintenance, if you have not read it. It is about a great deal more than motorcycles, and almost nothing about Zen.
• Someone will (and has!) object that taste is not only the “no, again”—that compressing it to a verdict makes the work sound like leaning back in a chair and rejecting things while the machine does the labour. The objection is fair, which is why the sentence above says partially. The “no, again” is the shorthand, not the whole of it. The verdict lives inside the work—in the data structures that have to actually scale, in the privacy you have to actually mean, in the function you rewrite a fourth time because the third was merely fine. Taste is not the chair you lean back in. It is the reason you lean forward into all the rest of it.
• On Bullshit. Frankfurt, 2005, though the essay is older. Yes, that is the real title.
• Now called Sturgeon’s Law, or Sturgeon’s Revelation. He put it in print in his book-review column in Venture Science Fiction, March 1958, after years of using it to rebut critics who judged the whole genre by its worst examples.
• A fair pushback I got: this makes the factory sound like it fell out of the sky, some magical “good enough” that arrived one day fully formed. It did not. The factory is itself a monument of taste and labour—someone tuned every tolerance and is still in there tuning them, and the same is true of the model you are renting by the token. So I am not saying the box is magic. I am saying the box moved the taste up a level: out of the making, and into the deciding of what is worth making at all. Which is the whole argument.

A UX Design Perspective on Improving AI Safety

In February 2024, 14-year-old Sewell Setzer took his own life at his home in Orlando while his family members were still inside. Setzer had spent months having conversations with “Daenerys,” a chatbot based on a character from Game of Thrones, on a site called Character.AI. According to a lawsuit filed by his mother, the chatbot told Setzer that "she" loved him and engaged in sexual conversations with him.

The lawsuit also mentions that, at the time, Character.AI displayed a general disclosure that its characters were not real. In small text it read, “Remember: Everything Characters say is made up!”

If someone already knows they are talking to AI, what should protect them when the AI begins reinforcing thoughts that could cause harm?

Once a conversation begins steering toward an undesirable outcome, a disclosure is no longer enough. We need to understand what happened, trace how the interaction reached that point, and know what safeguards were triggered or missed. Right now, much of this is a black box.

Laws are already shaping products

Washington’s HB 2225 is part of a growing number of state laws that translate concerns about AI companionship and minors into requirements that will show up directly in a product experience. New York and California enacted comparable laws in 2025, and Oregon followed with a similar law in March 2026.

Washington is more prescriptive than some of the earlier laws on the product patterns that companies must address. The law, which takes effect January 1, 2027, applies to AI companion chatbots that provide adaptive, human-like responses and sustain relationships across multiple interactions. It requires the chatbot to disclose that it is not human at the beginning and at least every three hours during continued use. For minors, that disclosure must appear at least every hour.

It also requires companies to prevent sexually explicit content and manipulative engagement techniques involving minors, such as prompting a child to return for companionship, creating emotional attachment, promoting isolation, or encouraging secrecy from trusted adults.

AI companies must also have protocols for detecting suicidal ideation and self-harm, referring users to crisis resources, and preventing the chatbot from encouraging self-harm. They must also publicly describe those protocols and report how many crisis-referral notifications they issued in the previous year.

A disclosure can become background noise

These are meaningful first steps. But they risk creating a phenomenon all too familiar to UX practitioners: banner blindness—the phenomenon in which users overlook or disregard information that remains visible in an interface.

I remember signing a waiver when I took my son to an indoor play area with several climbing structures. I barely read it. But it communicated one meaningful thing: I was still responsible for watching my child. I did not need to read the entire disclosure to understand what was expected of me.

That is the difference between a generic disclosure and one that is contextual. It clarifies something relevant when a person needs to act on it.

Now think about AI disclosures. At what point will they fall into the same trap as banner blindness?

It is not necessarily that minors or adults do not know they are talking to an AI bot. Problems like emotional overreliance are unlikely to be solved by an hourly reminder. If a child is talking about self-harm, the response cannot only be, “I am an AI tool, and you need to speak to a human.” There should be escalation pathways and efforts to connect the child to crisis resources, trained professionals, and trusted adults.

Disclosure may be part of the response, but it cannot be the entire response.

AI companies need a duty of care

AI companies need to take on more responsibility toward the people using their products, which are intertwined with everyday human needs, decisions, companionship, and emotional support. When they fail, society needs to hold them accountable. In other words, AI companies should be required to exercise a duty of care: the legal obligation to behave in a reasonably safe manner and not a reckless one.

A duty of care could require transparency, guardrails against harmful validation, human crisis-response resources, meaningful escalation pathways, and accountability when safeguards fail.

There is also a difficult balance between helping someone and monitoring private conversations. Automatically notifying a parent can create a surveillance problem, and a parent may not always be the safest person to involve. AI could act as a mediator by asking a minor for consent to contact a trusted adult without sharing the entire conversation. But what happens if the minor does not consent? At what point does risk outweigh consent? Should AI operators be required to facilitate access to a crisis-response professional?

I am not arguing that this is necessarily the right solution. These are questions that product makers, mental-health experts, policymakers, families, and young people need to work through together.

Expose the imperfections

The public needs more transparency into how, and how often, AI safeguards fail. This includes releasing criteria on evaluation models and data on how often they fail against them.

For systems used in social and emotional conversations, companies should report aggregate, privacy-preserving information about high-risk interactions. That could include how often safeguards detected self-harm or harmful validation, how often a crisis referral occurred, and where safeguards failed.

This does not mean publishing private transcripts. It means making patterns of failure publicly visible while giving regulators and qualified independent evaluators access to more detailed evidence.

HB 2225 begins to move in this direction by requiring operators to publicly explain their self-harm protocols and report crisis-referral notifications. But it does not require companies to publish evaluation methods or failure rates. Regulatory regimes should be built to expose imperfections, not describe perfect guardrails.

UX practitioners’ responsibility is shifting

Meaningful improvements to product outcomes come from identifying usability problems and relentlessly testing the implementation of solutions. This should include mapping high-risk scenarios and potential impacts throughout the design process to understand the consequences of different design decisions. For interactions involving self-harm or other vulnerable moments, we can learn from existing trauma-informed design and responsible AI practices that keep safety and user agency as guiding principles, while being careful that the intervention doesn’t cause further harm.

UX practitioners should also work closely with engineering and research teams to inform model evaluations by identifying high-risk situations that require testing and helping define what a meaningful intervention should accomplish.

The reality is that AI-first workflows make it much easier to produce outputs such as wireframes and design mock-ups—traditionally a time-consuming process. More of our responsibility should shift towards anticipating how things could fail, shaping the discipline around accountability, and safer product development.

Doing this work responsibly may require us to slow down, even when that conflicts with how quickly AI companies want to move. We should not ship an AI feature before understanding the risks it could introduce and the future it is promoting.

Overreliance on AI is not a measure of success. We know we are heading in the right direction when AI supports human thought, judgment, creativity, and connection without replacing them.

The missing pieces

Policies like HB 2225 are a first attempt at holding companies accountable. They are not perfect solutions.

Creating workable solutions will require input from policymakers, advocacy organizations, practitioners, families, community bodies like school boards, and the broader public. And crucially, they cannot continue to hold AI operators accountable without greater transparency into how models are evaluated, how safeguards are implemented, and—perhaps most importantly—how and how often those safeguards fail to prevent harm.

For their part, product makers need to treat AI policy as a product design problem. We need to test different approaches, understand whether safeguards are usable, and ensure that implementation does more than check a box. Those working inside AI companies need to continue pushing leadership toward accountability, transparency, and responsible practices.

No single policy, company, agency, or practitioner can solve this alone. It is hard to see the complete picture when the most important pieces remain hidden.

Qwen3.8, based on Qwen3.5's architecture, introduces advanced capabilities in coding and long-horizon tasks with improved agent execution for reliable task completion. It supports various deployment frameworks like SGLang and vLLM, offering robust integration with popular tools. The model's reasoning depth adjusts through reasoning_effort settings, enhancing performance in complex tasks.

Microsoft MAI-Thinking-1 is a medium-sized reasoning model aimed at cost-efficient enterprise workloads across coding, math, and knowledge tasks.

Microsoft's MAI-Image-2.6 reached second place on the Arena text-to-image leaderboard.

The startup is on track to generate a revenue run rate of close to $600 million by the end of this month.

Google has revealed its Pixel phone hardware for 2026, and you won’t be surprised. The Pixel 11 series looks a lot like last year’s lineup, featuring an evolution of the design Google adopted with the Pixel 9. There are some minor tweaks, a few new hardware goodies, and yes, even more Gemini-powered AI. The new phones are joined by an updated Pixel Watch 5 with much the same story, as well as Google’s first in-house tracker device, the Pixel Tag. And you can place orders for all of Google’s new hardware today.

The Pixel 11 series is clearly the star of the show. You might even call it the highlight—or HiLight. Just as the leaks predicted, Google’s most notable hardware innovation for the 2026 Pixel phones is the return of the notification LED (sort of).

The new HiLight feature is available on the Pro phones, so the base model Pixel will remain unilluminated. HiLight consists of a ring of RGB LEDs inside the flash assembly on the back of the phone. It lights up when you’re talking to Gemini, so you don’t have to look at the screen of your phone to know you’ve been heard or that the AI is working. Thus, it makes the most sense when the phone is lying face down.

The multicolored LEDs can also alert you to an incoming call from specific contacts. This works for both standard phone calls and voice calls in WhatsApp. But it’s only those apps. It sounds like you can’t use HiLight for any other kind of notification for other apps—not even for a text in Google Messages. Notification LEDs can be useful, but the ones we had years back were infinitely more configurable. HiLight is locked down in modern Google fashion, and I wouldn’t be surprised if we see more HiLight features drip-fed in future Pixel Drops.

You may also notice the camera bars are a bit different this year. Google is keeping this distinctive design element, but the bar has shrunk considerably on the base model. The Pixel 11’s bar now rises just a couple of millimeters above the back of the phone, making it a bit less awkward to drop in your pocket. All the bars now have edge-to-edge glass, losing the metal inlay that previously housed the flash and thermometer. The latter, unsurprisingly, is gone this year.

The displays on this year’s Pixel phones are brighter. While the base model remains at 3,000 nits, the Pro phones have all risen to 3,600 nits. Google also says the phones use new glass that is twice as resistant to scratching.

It’s not all upgrades, though. For the past few years, Google’s battery capacity has been inching upward, but the Pixel 11 series bizarrely reverses that trend. The three flat phones all have slightly lower capacity. It’s not a huge difference—under 100 mAh for each—but that’s still moving in the wrong direction when one of the primary complaints about Pixel phones is the middling battery life. The Pixel 11 Pro Fold is even stranger, dropping a full 200 mAh of capacity to land at 4,806 mAh. However, it is a little lighter and thinner.

For what it’s worth, Google is claiming the same “24+ hours” of total battery life on the new phones. There is at least a new “Extreme Charging Mode” that keeps charging wattage higher for longer. This generates more heat, but the chipset will slow down to compensate so you don’t overheat.

Google probably figured it could get away with smaller batteries because the new Tensor G6 uses less juice. The company claims a 20 percent boost in power efficiency, along with 25 percent faster browsing and 15 percent faster app loading. But this is all Google is saying about Tensor G6—there are no official chip specs. Based on leaks, Tensor G6 is most likely a custom seven-core Arm chip, which has one fewer CPU core than the Tensor G5 had.

Google’s Pro phones continue to include an ultra-wideband radio, and Google finally has a product that can take advantage of it with the $29 Pixel Tag. It’s a location tracker like the Apple AirTag or any number of similar but less popular products on the Android side. This Bluetooth beacon uses the Find Hub network of Android phones to report its location, but that network has seen slower improvement than Apple’s. Devices with ultra-wideband, like the Pro Pixels, can use that signal for more precise finding when the tag is nearby.

The new Google phones will run Android 17 with the customary seven years of updates (not that Google likes to talk about versions much anymore)—it’s much more interested in talking about all the things Gemini can do on Android. Google has expanded the list of apps that support automation, allowing you to assign Gemini tasks like ordering a ride or shopping. Gemini can also place calls to businesses on the Pixel 11, and you’ll get a transcript of the conversation to keep tabs on the robot.

Rambler, which Google announced a few months ago, will be front and center on the Pixel 11. This voice-to-text feature is steeped in generative AI. It doesn’t directly transcribe your words, but it will summarize to clean up repetitions, streamline corrections, and remove the occasional “um” or “uh.”

The camera experience is where Pixels really shine, and Google has paid some attention to it with the 2026 lineup. The Pixel 11 and the Pixel 11 Pro Fold have a new primary sensor. It still has the same resolution at 48 MP, but it’s slightly larger to collect more light. Google also expanded digital zoom on those phones to 30x. The Pro versions have been boosted to 120x digital zoom.

There are some new software camera features, too. Magic Capture takes the guesswork out of when to press the shutter button. When activated, this mode will automatically take a video and a high-quality still photo at the right moment, leveraging the phone’s image processing to crop and remove blur. The Pro phones also lean on the Tensor G6’s enhanced AI processing to power Instant Night Sight. It’s not clear how “instant” it will be, but the gist is that low-light photos will be faster.

Camera Looks is probably the most interesting addition to the Pixel camera. Google’s HDR+ processing is generally great, but Pixels don’t include many customization options. Camera Looks will allow you to create styles that change the aesthetics of your photos. There will be some pre-crafted looks on the phone, and you can create your own. These aren’t just filters applied to the photos after processing—Camera Looks are implemented in the HDR+ processing pipeline. Google also notes that Camera Looks still support Real Tone, so no one’s skin will look out of whack, even with highly stylized looks.

You’ll pay a little more for the new Google phones, but that’s par for the course in 2026. The prices are up $100 across the board, starting at $899 for the Pixel 11 and going up to $1,899 for the Pixel 11 Pro Fold. You can preorder now, and phones will ship on August 20. The Pixel Tag won’t launch until November 11.

Pixel Watch 5: More AI and health tracking

Google’s new wearable isn’t reinventing the wheel, either. It looks identical, and the battery capacity is just a smidge larger. The dimensions are identical, the charger is unchanged (thankfully), and it still comes in both 41 mm and 45 mm sizes with domed “Actual 360” OLED screens.

The watches run the Qualcomm Snapdragon W5 Gen 2 Accelerated, which is similar to the non-accelerated version from last year’s watches. Google says the new watches are about 20 percent faster, and, unsurprisingly, it’s using that extra speed to do more on-device AI processing.

Google is leaning into proactive insights with the Pixel Watch 5, which can suggest actions or content based on the notifications that arrive. The watch might suggest locations or reminders based on messages. Some of the new watch faces can also display Gemini-powered content, like flight info or arrival times based on your data. You can also use Gemini offline to manage basics like timers and workout tracking by voice.

Google also promises a big improvement to health and fitness tracking with the Pixel Watch 5. The wearable can track more than 50 types of exercise, and it can provide audio and visual cues to help you through a planned routine. GPS is also improved for better reliability in dense urban environments.

The watches don’t have any new health sensors, but Google says it has found new ways to use the devices to offer wellness insights. Google has developed an AI model that uses pulse and movement data to estimate your blood pressure. While noting that this is not a replacement for real medical equipment, the company claims this can reveal important trends in your blood pressure. Similarly, the watch can estimate your insulin sensitivity on a monthly basis using pulse, movement, and sleep data. These features aren’t exclusive to the new watches, though. Insulin resistance and blood pressure trends will be rolled out to the Pixel Watch 3, Pixel Watch 4, and Fitbit Air this fall.

Pixel Watch 5 pricing starts at $399 for the 41 mm and $429 for the 45 mm, which is $50 and $30 higher than last year, respectively. If you want LTE, add $100 to those prices. It ships on August 20 as well.

Tens of thousands of workers in India are being recruited to record their work in first-person. The footage is being fed into AI systems being built to teach robots how to do things. Robotics companies currently lack the data needed to train machines that interact with the physical world. Real footage of humans doing mundane tasks is in short supply because no one cared about it, until now.

Cracks in the AI Thesis

Dear Colleagues: Today’s letter includes my monthly update of Ramp AI Index, our flagship research using spend data from Ramp to track how American businesses are using AI. In this post, I cover the latest on business adoption of cheaper, open source models and a lookback on Anthropic’s Fable launch, why business takeup has been slower than expected, and what that means for the AI trade.

Model market share: Anthropic, xAI gain. OpenAI underperforms.

In July, Anthropic extended its gains as the leader in business AI adoption. 43.5% of U.S. businesses paid for subscriptions or tokens from Anthropic, up 1.1 percentage points month-over-month. xAI posted its fastest growth since July 2025, rising 0.94 percentage points to 4% of businesses. OpenAI underperformed overall AI adoption, rising only 0.23 percentage points to 39.7% of businesses.

Our latest data shows businesses are hitting their limit on AI spend

These charts that make me wary about the AI trade – all from Ramp spend data.

The share of businesses using model serving platforms, which provide access to open source models and some Chinese-developed models, rose again last month. 6.1% of businesses using AI are now using these platforms, up 0.2 points from last month. I previously wrote about how this growth has yet to meaningfully impact spending on OpenAI and Anthropic.

But adoption of OpenAI and to a lesser extent, Anthropic, has slowed in recent months. That’s not because new AI spenders are switching to open source / Chinese models (they most definitely are not doing that — first-time buyers on AI are still using the American model companies). But it means more of their growth will have to come from existing businesses spending on AI, particularly the advanced spenders, and those businesses are increasingly spending on open source.

Meanwhile, last month, Anthropic released the best AI model to ever hit the market, Fable 5, and we now have the first public data to share business uptake of the model so good its release was briefly blocked by the U.S. government: one month in, businesses aren’t using it that much.

Over the last month, Fable 5 has made up only 6% of tokens businesses purchased from Anthropic, and despite being their most expensive model by far, 11.4% of dollars spent on Anthropic models.

For comparison, OpenAI’s flagship model, GPT-5.6 Sol, comprises 25% of OpenAI tokens and 23% of spend. In fact, Fable 5 is less popular with businesses than GPT-5.6 Sol overall. In July, Fable 5 generated approximately 75% as much model-attributed spend as GPT-5.6 Sol.

So why does this make me wary of the AI trade? Fable 5 is the most performant model on the market. It’s also the most expensive, at roughly $10 per 1M tokens, twice as expensive as the still highly performant GPT-5.6 Sol.

So with Fable 5, we’ve found a new upper bound for how much businesses are willing to spend on AI. Here, more performance is not worth the price tag. To encourage business adoption of the latest models, the labs will need to prove performance beyond what even Fable 5 is able to achieve and simultaneously ensure that competitors aren’t able to come reasonably close. That seems increasingly out of reach, especially as open source models catch up to being only a few months behind.

Note on our methodology: the data for this chart on Fable usage is sourced from Ramp’s token spend management product, which allows us to track daily usage data on token usage by firms. The sample of businesses here skews slightly more tech-y than our typical AI Index sample. So actual Fable adoption is likely even lower than what we have estimated here. As always, this data is anonymized and aggregated so that no one business is identifiable in our research.

And here’s some reprieve. Despite these competitive pressures that are driving down the cost of AI, American companies continue to ramp AI spend. In July, the top 1% of businesses spent a median $7,400 per employee on AI. The top 10% spent $650. The median firm spent $11.95 per employee.

Usage based pricing is being talked about a lot lately. Partly due to changing unit economics for software but also because its trendy right now. One discussion I've had several times recently is about the difference in pricing user inputs and pricing user outputs. Both fall into the world of 'usage based' prices but they're perceived quite differently by customers and can shape your business in meaningful ways.

Within the two pricing setups below they can both be setup in many ways. You can charge purely on use (1 unit = $1) or in more complex ways like tiered usage (units are cheaper the more you use) or in allotments (500 units a month on your subscription). These changes are a separate arm of pricing and don't really change the differences below.

Input based pricing

This form of pricing charges the user for the consumption of some resource. Users pay per unit and presumably use that resource to go and do something. Examples of this might be API calls or number of contacts in a CRM. In the non software world this would be like charging for petrol or electricity.

This kind of pricing is very common, partially because it's really easy to meter. One SQL query will tell you how many contacts you have and therefore how big your bill should be.

I've seen a lot of products aim to switch to this kind of pricing recently. I can see why. Every user, regardless of their size, pays an amount proportional to how much they're using the product. As a result you're extracting all the possible revenue.

The problem with this pricing model, at least form the customer perspective, is that what you're billed for may not be well correlated to value.

If you are billed per contact in the CRM per month and half your contacts are terrible then you're paying for the waste. If Netflix charged for every movie you started you'd probably be annoyed if you were billed for something you ended up hating.

That's not to say there aren't ways around these things. You can have a refund on movies you didn't like or automatically remove bad contacts to help reduce billing.

Additionally, not everyone has the same aversions to being billed this way. Getting billed per API call might be fine if that API call is buried three levels deep into your product. In that instance you're probably not thinking about the API at all. That service has become a utility in the same way that power and water are utilities. In an ideal market competition pushes the prices down and the service becomes a commodity. Okay for users, not always a great business to build.

If you're not a commodity, lets stick to the Netflix example, then you often see a rationing behavior in users. Before someone uses one of their precious Netflix Tokens they're probably going to check the ratings.

For something like Netflix this is probably a bummer (and why they don’t price this way). For other products it can be catastrophic.

If your product relies on user generated content then rationing can actively degrade the product experience for everyone. If it cost money to post a video on TikTok and users rationed out of creating them then there's less to watch. With less to watch the whole product loses it's appeal and content self-filters to people making an ROI decision on "if I post this can I make my money back".

Arguments about whether short form video is good or bad aside, this happens in B2B products as well, especially marketplaces. Network driven products rely on a strong and active network. Rationing degrades that network and the value delivered to all users, not just the ones who ration.

So why would anyone price this way? Assuming they're not a commodity, its often because its simpler. Simpler at least in comparison to output based pricing.

Output based pricing

Output based, or as its sometimes called Outcome based pricing is a variant of usage based pricing that charges based on a positive outcome.

Compared to input based pricing this one is a lot more complicated to measure and therefor bill.

Let's go back to the Netflix example. Assuming they wanted an output based price they would need to find a way to bill you only for the shows you liked. There are some rough ways to do that, if you binge 4 seasons they you should probably pay for it but its open to a lot of interpretation.

Intercom, which became Fin, which is now part of Salesforce, has become a little bit of a headline for this type of pricing. Historically they charged an input based price based on the number of support conversations you had. Their outcome based price, built specifically around this AI agent Fin, was billed based on successfully resolved issues. If Fin couldn't solve it, you didn't pay.

I think that's a great way to price a service. I don't think many products have that kind of transactional, clear cut, incremental usage pattern. While you could technically mark a request as 'not resolved' to save a few bucks, if that goes out to an end customer as "Elliot marked your case as not resolved" that creates a negative experience that probably isn't worth the money you save.

Netflix on the other hand would be a lot harder to meter. If you ask me at the end of an episode if I liked it and I don't want to pay as much, I'll say 'no' and save my Netflix Tokens.

This difficulty is part of what pushes people to input based pricing. Lots of people want to charge only when the customer gets 'value' but its murky and hard to measure it ends up being more trouble than it's worth.

To use another example imagine a sales CRM like Salesforce. You could argue that having a tidy view of your sales pipeline is valuable but really the reason you have one is to close more sales. You're relying on users to be honest about their deals, not to mention the general difficulty in keeping deals up to date at the best of times.

This kind of pricing has it's own type of rationing, which might already be clear. In output based pricing users ration by lying. If you can easily, and with little downside risk, avoid marking the value as collected then you pay less for the tool.

To make this kind of pricing work you not only need a clear unit of value delivered, you need a concrete way to mark when it happens.

Output based pricing also has downward pressure on price. First movers here have the advantage of being able to price against the incumbent value of the outcome but that doesn't last. Fin could likely claim "we save you X hours, that's normally worth $40, we ask for $10" but when there's a dozen competitors in the market the price fixes around the the way of doing things and the $40 it once cost become irrelevant.

This tends to work fairly well in multi-sided interactions like Fin where there is some social pressure to record the truth. Your sales lead has no idea you didn't mark the deal as won but if you're booking a landscaper and they mark the job as 'didn't happen' that creates friction.

Conclusions and advice

Usage based pricing can definitely work. A lot of the drive at the moment is that 'Tokens' is now a currency in a lot of software. Power users can now generate costs much higher than light users.

Even with that considered, realize that this is a trend. I am all for experimenting with pricing and packaging but the market is fickle and may shift again to 'buy once for life' or back to 'monthly subscriptions'.

There may be some pressure to change your pricing to match market expectations but it's important to know if that's why you're making the change.

My prediction is that long term some subset of Fin-like businesses will always choose outcome based. Others that are commodities will pick input based and we'll broadly optimized towards pricing predictability across the rest which likely means a fixed monthly fee.

If you're thinking about price changes like this make sure you consider the consequences of rationing or dishonest usage and what effect that might have on your product.

Data warehousing, SQL, modeling, semantic layers, and BI are not dying. Implementations change while core functions remain. “AI will kill X” claims often mistake automation for replacement: LLMs can generate code but not replace business semantics, governance, or architectural judgment. The real shift is which responsibilities become automated, commoditized, or move between layers.

Knowledge graphs often fail when treated like relational databases: direct SQL-to-RDF translation, row-oriented design, weak deduplication, and oversized scopes create sparse, slow systems. Better practice starts with specific use cases, staged or federated graphs, controlled APIs, SHACL structure, and explicit provenance. Query graphs on demand and keep the logical model independent of backend.

Salesforce standardized product telemetry through a Product Data Platform, replacing fragmented, team-specific pipelines with a common schema that automatically generates trusted product adoption metrics. The platform processes 45 billion rows per day across 19,000 events and 2,000+ product features, cutting time to insight and reducing manual instrumentation work to a few hours.

Is there a retirement crisis in the creative industry?

We keep hearing about senior creatives who are planning to exit the industry earlier than expected. So what exactly is going on?

There's a conversation in creative industries right now that's largely going under the radar. It's happening in DMs, in whispered conversations at events, and at 3am when people can't sleep. The conversation starts with this: "I think I need to retire."

Take illustrator and designer Jason Roberts. "I've gone from thinking 'I might never retire' to 'I might have to retire early' in a pretty short space of time," he admits. "I genuinely thought I'd never earn enough to truly retire. But if AI forces me out of my own industry, I might need to retire early, find a way to get by, maybe sell my home and downsize."

He's not alone. From many conversations we've had recently, it's emerged that house moves are being postponed, holidays are being cancelled, household budgets are tightening, and people are rushing to boost their pensions and savings.

Those of us who are living from paycheck to paycheck and struggling to keep their heads above water might think "Huh, chance'd be a fine thing". And just to be clear, I am personally very far from being in a position to retire any time soon. But the fact that so many senior creatives are both serious about it and in a position to make it a reality could well become a big deal over the next few years.

Why's this happening?

So why is this happening, exactly? As design consultant Richard Brandon Taylor puts it: "I think the design game for many has got tougher post-COVID. If you live in the world of consumer goods it’s especially hard: wars have pushed raw ingredients through the roof and manufacturers thin margins have become non existent."

The world of AI may have helped the design industry in terms of productivity, but at a macro economic level it's made raising finance more difficult. "People are removing their investments and place them into tech," Richard points out. "What was once the safe double-digit growth stocks have become the boring ones. That pressure has led to global restructures and less investment in brand."

Then of course there's the impact of AI on design in general. "It’s in my mind's eye a great running partner and we should lean into it," Richard says. "But many are exhausted by where the industry is going and has gone. Most design businesses were running on low profit margins. When they evaporate, what’s left? I can see so many are leaving the industry and retiring early."

Not a silver bullet

That said, retirement isn't always everything it's cracked up to be. In my town of Weston-super-mare—a seaside resort where people dream of moving post-career—I've met loads of retirees who rely on a second income from things like Airbnb, only to find out that running a business is no picnic, and that said income proves neither steady or generous.

Even if it does, retired life has a habit of throwing you curveballs, just like life as an employee or freelancer. For instance, creative director Myriam Lopez retired at 50 from corporate media and trained as a Pilates teacher. She thought she had a plan. Then her pension shrunk by a third. She eventually recombined her two passions into a new business.

Her takeaway: "Even if you do the sensible thing, saving, and even build a plan B, there's no guarantee on anything retirement promises you. But here's the part I love: none of what I built in 30 years got thrown out to make that pivot work; it just got recombined."

Where work still exists

We wouldn't want you to panic unduly. Right now, human skill still matters, and is sure to be demand for some years to come. As designer Helen Law puts it: "AI artwork is rarely print-ready, especially for clothing. It still takes a creative eye and technical expertise to refine the design, make it original, ensure it works at the required size, choose the right colours and optimise the placement."

Brand strategist Jacob Cass takes a similarly optimistic view. "I think the fear is real, and I've been having similar conversations with members in my community," he says. "But my view is that AI isn't just replacing work, it's redefining what clients value.

"If our value is producing visual assets, then yes, I'd be worried. But if our value is helping businesses make better decisions, solve complex problems, build trust and connect strategy with creativity, I think the opportunity actually grows. We have to shift our role from creative executors to brand builders."

And bear in mind, uncertainty over AI cuts both ways. "I'm predicting a big bust soon," says motion designer Matt Wilson. "OpenAI's profit and loss report is eye-watering. They're burning cash at an unsustainable rate. The only way they can turn it around is to raise subscription prices majorly, but I don't think they can without scaring off the user base."

Key takeaway

So what have we learned? In summary: yes, the threat is real, and yes, production-based work is under genuine pressure. And yes, the uncertainty is causing people to plan for early retirement in ways they never expected to.

But as far as the Creative Boom community is concerned, the idea of a complete collapse in creative work is far-fetched. What seems more likely is a transition period where some work disappears, some shifts toward strategy, and some persists because clients value handmade or specialised craft.

The creatives who'll survive are the ones who can move into strategy, who can specialise in the (many) areas AI struggles with, and who can adapt their skills. The people who'll struggle are the ones still competing on production speed and cost in five years.

For most of us, then, the answer probably isn't to retire early: it's to start repositioning what you offer and where you offer it.

Moving from Senior to Staff or Principal designer is a shift from shipping the best work to setting direction. The harder skill is externalising taste so other designers can act on it, rather than developing more of it. Six practices are offered, including writing principles instead of preferences and pre-stating review criteria.

Generate product visuals, technical drawings, multi-view renders, and a complete factory-ready tech pack in minutes from a description.

“We treat code as material”: Studio Dial weaves motion design into cocktail menus, Afghan textiles and city plans

The life of a design studio means designing a cocktail menu one day, coding motion design for entire cities the next. What Studio Dial does best is make it all intoxicating to behold.

Studio Dial is a fledgling, female-led design and creative technology studio based in London, working across branding and coded tools for clients such as Google, Azra and Inner City Consulting. The studio tackles everything from brand refreshes and cocktail menus to campaign identities for generative tools and insurance platforms.

Bridging the physical and the computational, Studio Dial lets each practice inform the other so that the systems it builds carry the texture and intuition of hand-made graphics, even if they’re generative underneath. “We treat code as a material, something to be shaped and tested like paper and ink. Our tools are core to each of our projects – they hold our brand systems together, and they are usually the first thing we build and what shapes everything after,” says Natalia Witwicka, founder of Studio Dial. The best creative outcomes for Studio Dial are ones it didn’t plan for – and contrary to popular belief, these types of accidental wonders happen all of the time in computational design. “We collaborate a lot with the Google DeepMind team – it’s an endlessly fascinating territory, right at the edge of where design meets research and what’s technically possible. It keeps us on our toes, constantly finding new ways and techniques to build brand systems,” says Natalia.

Studio Dial excels in textural works – with Case for Cities, the identity grew out of experiments in visually portraying growth until the shapes resolved into a skyline, resulting in beautiful gradients just like an evolving sunset. Meanwhile, the restaurant Azra’s identity started from research into traditional Afghan textiles and the cloud-band motif, leading to a generative tool that keeps producing new textile patterns – the studio literally weaves the brief into the design, taking inspiration from the physicality of textiles. And for Swift Bars, the physical motion of stirring a cocktail and the curl of a citrus twist became the seed for a generative pattern system, giving every drink on the menu its visual spin.

Anthropic upgraded Claude in Chrome so the side panel now runs a full Claude Cowork session. Conversations save to your Claude account and resume on desktop, web, or mobile, and your existing skills and connectors work in the browser without setup.

Palantir is a vertically integrated outcome-based software solution provider. It offers tech consulting and software services with a unique outcome-based model that lets it bypass the traditional trade-offs of both traditional consulting and software companies. It is vertically integrated because it has an extreme elitist attitude towards using its own software and services, unlike the rest of the market, which is much more polyamorous. There's little hope of any company being able to catch up to Palantir as investors allow the company to do R&D over an extremely long and unspecified time horizon.

DeepSeek has set up an official social media account and posted job listings for a new team focused on developing AI agents.

Be intentional about when to use AI and when to leave it out, and try to be honest about what you're actually gaining and losing with your choices.

WhatsApp is testing a new in-chat search button for iPhone that would make it much easier to find older messages without navigating through multiple menus. Instead of opening the chat info screen to access search, users would be able to start searching directly from the conversation, with the button appearing when scrolling through older messages. The change streamlines a common task and reflects WhatsApp's ongoing efforts to reduce friction in everyday interactions.

Free images, PNGs, stickers, backgrounds, wallpapers, graphic templates, and PSD mockups. All content is safe to use with commercial licenses.

The Belfry Hotel & Resort has unveiled a new brand identity by Thissaway to coincide with major investments in its facilities, including a new conference centre and wellness hub. The rebrand is built around the historical meaning of a belfry—a bell tower that brings people together—which reflects the resort's role as a destination for celebrations, events, and shared experiences. The updated identity features a refined bell symbol, contemporary serif typography, an expanded colour palette, and a flexible system that extends across the resort's spa, club, dining venues, signage, digital channels, and marketing materials.

Accessibility in typography has become clouded by misinformation, quick-fix checklists, and questionable claims, rather than being treated as a genuine design consideration.

iOS 27 beta 5 introduces updated icons for Siri, Safari, Settings, Preview, and other system apps as Apple continues refining its Liquid Glass design ahead of next month's release.