Previewing the Model Hardware Standard

We’re opening a research preview of the Model Hardware Standard (MHS), a shared specification for AI agents to safely operate physical devices, to a first group of scientific research labs and advanced manufacturers. MHS enables AI agents to operate multiple lab and manufacturing instruments, such as microscopes, liquid handlers, and robotic arms, in parallel, and perform intricate tasks ranging from routine drug discovery experiments to laser calibration on a quantum computer. The development of MHS began as a collaboration between Anthropic and HHMI Janelia Research Campus.

It typically takes a lab or manufacturing facility weeks, if not months, to set up and integrate their hardware. Most devices don’t communicate with each other, instead requiring specialists to build bespoke integrations. MHS reduces this integration work to hours or minutes. And by incorporating AI into these tools, MHS also helps researchers and engineers more readily orchestrate autonomous, round-the-clock experiments and workflows, with agents able to reason through each step in an experiment, update parameters in real time, and, in some cases, recover from hardware errors without intervention.

We’re sharing an early version of MHS with partners across science, robotics, electronics, and manufacturing so we can collaborate to build safety evaluations and develop best practices for AI systems operating physical equipment, ahead of making the standard open source. MHS works with any device that has a programmable interface. It is also model-agnostic, and any agent harness can access it using standard protocols, such as the Model Context Protocol. To apply for access to the research preview, head here.

How MHS works

Getting multiple devices in a lab or on a factory floor to communicate with one another can be challenging, even setting aside the added difficulty of integrating AI into the setup. Each device tends to have its own programming interface, and so far there has been no standardized way to integrate them. And once the devices are connected, there is no common way for them to share data with an AI agent, nor to let the agent operate them safely.

MHS addresses these challenges by introducing a standardized driver: software that translates between a computer’s operating system and a hardware device. The MHS driver uses a simple set of primitives—commands like “read” (for example, “get temperature”) or “write” (for example, “set temperature”)—that any hardware device can understand and act on. And it makes each device discoverable in a standard format, so that devices and agents can find each other and communicate across networks without needing a bespoke “translator” program in between.

The MHS driver also helps an AI agent understand how to use a device it has never seen before, giving it information about machine characteristics that may not be discernable from code alone (for example, the weight of a robot arm, which is important for knowing how to manipulate it safely). To date, much of this information has been stored in paper manuals, on a user’s computer, or as tacit knowledge. But the MHS driver contains tags that let the user write this information directly in natural language (users can either do this themselves, or by chatting to an agent that interviews them about their hardware setup). With the information from these tags, the MHS driver then automatically produces a reference file with information about a device’s general characteristics, such as what it can measure, what can be adjusted, and what safety limits will be enforced. This file gives the agent everything it needs to know to operate the device.

After the devices are connected and the agent knows how to use each one, the agent needs a way to control the hardware. For MHS, there are three such mechanisms: MCP, the command line interface, and code files (APIs). These work together to enable orchestration across multiple devices via a single line of code.

Once the agent can control the devices, it’s able to receive operating data from each one and supervise and direct the work at a high level. The agent can sequence steps across instruments, monitor results, and adjust parameters as conditions change in real time. When the agent needs to execute long-running tasks or operate devices faster than its online reasoning would allow, it can chain together driver commands from one or more devices in code files. This allows the devices to carry out operations themselves, without the agent needing to reason at every step.

As we’ve tested MHS, we’ve found that Claude interacts with experiments and hardware in an exploratory manner, much as a scientist would. For example, we observed Claude make an adjustment to a laser, observe the results through a camera to assess how its adjustment moved the laser beam, and repeat the process, seeking to understand the sequence of events. Claude then packaged what it learned into code files, writing a deterministic script that let it align the laser without having to reason at each step, so the whole process could run as a single command.

Early examples from MHS

We are only just beginning to see what people can do with frontier models and MHS, but our hope is that the standard can be of use to researchers, engineers, and other practitioners in speeding up the process of discovery and experimentation in any domain that uses devices with a programmable interface.

As we developed MHS, we shared it with a handful of labs and hardware manufacturers in biotech, robotics, quantum computing, and other fields. Across these early projects, we saw MHS reduce the time it took to integrate devices, make it possible to iterate faster in a variety of experimental settings, and assist with the live operation of machines and real-time fault detection. Below, our partners share the details of some of their early projects involving MHS.

Genentech: Implementing MHS for lab automation

Researchers at Genentech implemented and tested MHS as a proof-of-concept for automating the BCA protein assay, a standard procedure to measure total protein concentration in a sample, which requires coordinating across a liquid handler, a robotic arm, and a plate reader.

University of Washington Baker and Pinglay labs: Bringing AI agents to the bench

Zihao Song, a PhD student in the University of Washington Baker and Pinglay labs, used MHS to build a dashboard to remotely monitor his instruments; an AI agent-supervised qPCR (which copies a target DNA sequence through repeated cycles of heating and cooling) that watches amplification curves and halts the procedure at the right moment; and an integration between a robotic arm and a liquid handler for collision-free plate handoffs.

Carnegie Mellon University: Determining dose-response curves through rapid automation

Researchers at Carnegie Mellon University used MHS to run serial dilution dose-response experiments about three times faster than before, with an AI agent orchestrating a liquid handler, a plate reader, a robotic arm, and monitoring cameras spread across three computers with fundamentally incompatible interfaces.

HHMI Janelia: Using MHS to accelerate microscopy research

At HHMI Janelia Research Campus, researchers are using MHS to speed up a range of microscopy-related projects. Here, Virginie Ruetten, a scientist in the Ahrens lab who studies how sleep helps the body recover from stress, shares how she used MHS to unify and orchestrate a rig that previously involved seven different vendor programs without a shared interface.

QuEra Computing: Using MHS in quantum laser stabilization

QuEra, a company that builds quantum computers using neutral atoms, used MHS to give an AI agent control over parts of the laser system inside its quantum machines. The agent developed a controller that recovers the laser’s “lock”—the ultra-precise frequency the lasers must hold to interact with the atoms—99.3% of the time without human intervention.

Tetsuwan Scientific: Using MHS to run qPCRs to profile local pollution

Researchers at Tetsuwan integrated MHS with its automated biology lab platform, ResearchOS. MHS helped orchestrate a qPCR workflow to contribute to citizen science efforts to characterize pollution in California’s San Pedro Creek.

Hardware vendors and the software companies that support them are also building MHS support into their equipment so AI agents can more easily discover and operate their devices. For example:

• Amazon Web Services will support MHS through Strands Robots, the library for connecting AI agents to physical devices. AWS will provide participants a private, pre-release version of the Strands Robots package for the duration of the MHS research preview.
• Automata is adding MHS support to LINQ, their lab automation platform, to perform intelligent error handling of instruments in autonomous labs.
• Danaher and Anthropic are actively exploring how MHS-supported capabilities could enable its smart instruments and autonomous laboratories to scale biomedical research and development.
• Doosan Robotics is testing MHS with their robotic arms, including to perform automated quality assurance and coordinate tasks across multiple robots.
• MBF Bioscience is building an MHS driver for ScanImage, the software that runs laser-scanning microscopes in hundreds of neuroscience labs worldwide, to integrate AI agents into real-time data analysis and experiments.
• QIAGEN is experimenting with MHS through a working proof-of-concept on its nucleic acid purification platform, QIAsymphony Connect, showing how AI agents could help laboratories troubleshoot instrument issues faster, guide operators through recovery, and improve instrument uptime while reducing risk to biological samples.
• Tecan is adding MHS support for their Fluent liquid handling platforms so AI agents can discover and operate them directly.
• Universal Robots has had early access to MHS and plans to add support to its robotics platform.

Joining the research preview

These early results from our partners are encouraging, but we have more work to do on the standard before we open-source it. As a large language model, Claude learns about the physical world through text and images, meaning its spatial and physical reasoning have limitations that still require expert oversight. When working with protein samples, for example, Genentech researchers had to guide Claude to recognize that errors caused by foaming in samples were physical failures, not software bugs, that could only be mitigated through the appropriate physical corrections.

MHS also doesn’t yet work with hardware that lacks a programming interface, so we’re working with the manufacturers of such devices to build in MHS drivers. Many developers already use Claude Code to work with individual pieces of physical equipment; for the next phase of MHS, we hope to expand the standard to cover more of the devices developers build on. Early adopters include Hugging Face, who are adding MHS support in LeRobot, their robotics library, and Raspberry Pi, who are enabling MHS integration across a number of their products following successful tests using their Camera MHS Driver.

We will also use the research preview to build additional safety evaluations with our launch partners and strengthen protections for the use of AI in the physical world. We are developing a physical safety roadmap to further bolster our safeguards policy and enforcement coverage against the risk of misuse. When we open-source MHS, we will release findings from the research preview as part of our guidance for deploying the standard safely.

We’re inviting stakeholders across industries to join the waitlist for our research preview of MHS. If you’d like to participate, submit your interest here.

Acknowledgments

MHS began as a collaboration between Alek Kemeny on Anthropic’s Beneficial Deployments team and Arco Bast, a postdoctoral scientist at HHMI Janelia Research Campus. Bast was running complex brain-imaging experiments on a rig that combined lasers, motorized focusers, and specialized cameras from different vendors with no common interface. To speed up his experiments, he developed a shared memory dictionary that enabled the instruments to communicate with one another at memory speed. Kemeny and Bast worked together to integrate AI models into that interface.

We thank everyone who has contributed to this work so far, including, but not limited to, Aaron Boswell, Ben Arthur, Boaz Mohar, Gagan Bhat, Mark Kittisopikul, Nadine Yasser, Nick Purcell, Takashi Kawase, and Virginie Ruetten. We look forward to moving MHS forward with our industry partners and, soon, with the open-source community.

Many industries rely on relational databases that are queried with SQL. Most SQL is machine-written, but humans alone likely write billions of custom SQL queries each month prompted by business questions. They are quite good at it — humans score 92.96% on BIRD, a realistic benchmark for translating natural-language questions into SQL.

However, AI performance on text-to-SQL has lagged behind. LLM scores on the BIRD leaderboard improved from just below 70% in 2024 to 82% today. Frontier models like GPT-5.6 Sol Ultra and Claude Fable 5 can score in the mid-80s, albeit at a cost that is prohibitive for high-volume applications. This isn’t for lack of training data: SQL is widely represented in the internet content used in LLM pretraining. The challenge for AI is in navigating the ambiguous questions and highly-contextual schema that characterize real-world examples.

A common approach for improving AI performance on tasks people understand well is building agentic scaffolding. Systems such as OpenHands, AI co-scientist, and MetaGPT decompose a task into stages, each with its own prompt or model call. Text-to-SQL scaffolds follow the same pattern. A schema-linking stage narrows thousands of columns down to a candidate set. Real-world, enterprise data systems (including databases, data warehouses, and data lakehouses) contain up to millions of columns. Answering business questions often involves understanding which columns to use. Academic benchmarks are simpler. A generation stage samples queries. A self-correction stage repairs execution errors. A selection stage votes among the survivors. Every component is a separate call, and the orchestration is typically tuned for a given benchmark.

Scaffolding is an attempt to get over the limitations of model reasoning by making it adhere to a sequence of steps that mirrors how a human would approach the task. And yet, the best scaffolded models still lag 11 points behind humans on SQL. Human professionals acquire their skills with repeated experience, not by being handed a list of instructions — the same should be true of LLMs. Experience with the task should be used to train better reasoning about queries and databases into the model, instead of simply updating the prompts it receives from the scaffold.

In this blog post we describe fine-tuning a model that achieves human-level accuracy on text-to-SQL without scaffolding, using reinforcement learning with verifiable rewards (RLVR) on Tinker.

Extracting the correct answer with SQL is a verifiable task that can be trained in a straightforward manner. However, the performance gap of SQL models suggested that the standard recipe can be improved. Our approach has two crucial improvements: an expert-verified training set purged of the label errors that could poison RLVR, and a reward-shaping technique that targets two common failure modes of RLVR in this domain.

The trained model, ReViSQL-K2.6, exceeds the human mark of 92.96% when picking from 16 samples (SC-16) at a cost of $0.56 per task. It is more accurate than Fable 5 and GPT-5.6 Sol Ultra at 12–15% of their cost, and much more accurate than any scaffolded model on the leaderboard.

Curating high-quality training data

RLVR is effective at improving domain-specific reasoning, but it is sensitive to data with incorrect labels. In RLVR, the scalar reward is the entire learning signal for the training step. Mislabeled instances reverse the signal, degrading learning significantly. Our research found that algorithmic tweaks cannot compensate for this loss — cleaning up the data is crucial for RLVR to work well.

We found existing text-to-SQL data to be extremely noisy. Our analysis has shown that many widely known benchmarks contain large amounts of noise. We sampled 2.5k instances from BIRD Train, a training dataset for text-to-SQL. Our audit found errors in every component of the dataset: the questions, the external knowledge supplied, and in more than half of the “golden SQL queries” that the model’s answer is compared against.

We cleaned up the training set in a multi-stage process. First, an LLM (OpenAI’s o3) and a human expert reviewed each instance and flagged errors. The expert review found that the LLM auditor was precise in catching annotation mistakes (90.6% precision) but only caught 24.5% of the errors flagged by humans. The errors and proposed fixes from this first stage were sent to a different expert for verification. Where the verifier disagreed with the initial auditor, the sample was sent back for additional loops of conflict resolution.

We released the cleaned-up training set to the community as BIRD-Platinum.

We suspected that the evaluation dataset, BIRD Mini-Dev, likewise contained annotation errors. A first cleanup pass of BIRD Mini-Dev was done by Arcwise and corrected errors in 32.3% of instances. We did a second pass that confirmed the vast majority of Arcwise’s flags and found many more, bringing the total detected error rate in BIRD Mini-Dev to 52.8%. The evaluation set with the gold query errors cleaned up was released as Arcwise-Plat-SQL.

BIRD-Platinum lifts RLVR above prior best models

We fine-tuned Kimi-K2.6 with RLVR on BIRD-Platinum to produce ReViSQL-K2.6. Training on verified data alone lifted ReViSQL-K2.6 well above both frontier generalist LLMs and the leading open-weight fine-tuned text-to-SQL models on Arcwise-Plat-SQL, with an accuracy score of 88.55%. This shows that the annotation errors in standard training data were the binding constraint on RLVR for text-to-SQL.

To demonstrate that this approach generalizes to other models and eval sets we didn’t touch, we fine-tuned Qwen3-235B-A22B with RLVR on BIRD-Platinum and the original BIRD Train. We tested the model on two new text-to-SQL benchmarks, widely considered more difficult than BIRD:

• Spider2-SQLite: a variant of the Spider2 benchmark that contains complex queries with 5.2× as many tokens on average as Arcwise-Plat-SQL.
• Spider2-Snow: a variant of Spider2 that uses the Snowflake SQL dialect.

Training on the more carefully curated BIRD-Platinum improves the model’s accuracy by 16% on Arcwise-Plat-SQL, 12% on Spider2-SQLite, and 14% on Spider2-Snow compared to BIRD Train. This indicates that our verified data produces a more transferable learning signal across benchmarks and SQL dialects.

Accurate reward signal for text-to-SQL RLVR

Training on clean data brought the fine-tuned model closer to human parity, but a gap of over 4% remained. We looked at the cases where the model failed to identify patterns, which led us to examine the reward function used in training.

Standard text-to-SQL RLVR assigns a reward of 1 when the generated query returns the same result as the gold query on the benchmark database. This mirrors the scoring used in evaluation, but doesn’t fully capture the general behavior we want the model to learn. We focused on two ways result-based reward can diverge from the intended behavior and amended the reward function to address them.

Divergence 1: execution match is not semantic equivalence

The standard result-based reward checks the generated query’s output on a single database instance, but this doesn’t guarantee that the result would hold for a different one. A wrong join key or a dropped predicate can be missed when the particular database instance does not expose the error. Only queries that are semantically equivalent are guaranteed to produce the same result on any database instance.

We test for the semantic equivalence of SQL queries with VeriEQL, a solver that uses bounded verification and incurs negligible CPU costs relative to the total training costs (less than 0.1%). In a pilot training run, we found that 32.8% of positive result-based rewards were given to queries that weren’t fully equivalent to the correct one. That means that nearly one time in three, the reward reinforced the wrong query.

We updated the reward signal by downweighting it in cases where the query was accepted by execution matching but wasn’t equivalent according to VeriEQL. With an additional source of verification, the reward pushes training towards the correct query that generalizes to different database instances.

Divergence 2: outcome rewards are blind to provided knowledge

BIRD-style problems provide external knowledge alongside the question in a prompt. A result-based reward conditions only on the final result, which means it cannot distinguish a model that read the provided information from one that guessed correctly based on its pretraining priors. For example, because “sodium = 0” and “sodium < 5” produce the same result set, a result-based reward cannot distinguish a model that correctly uses external knowledge to choose “sodium = 0” from one that memorizes or hallucinates “sodium < 5.” In the absence of a gradient pushing models to incorporate external knowledge, they tend to default to the prior. In a pilot analysis on a validation set, 24.2% of failures were traced to the model ignoring the necessary information that was supplied.

We address this with rule-based process rewards. The model must emit a requirement block that translates each external-knowledge entry into an explicit query constraint, and a verification block that audits the generated query against those constraints, with penalties for non-compliance. Our process reward encourages the model to ground its reasoning and generation in the provided knowledge rather than blindly relying on its pre-trained prior. The rewards are rule-based rather than model-graded, making the scoring cheap and free from contamination by a judge model’s priors.

Training recipe

Results

We present ReViSQL-K2.6, the model fine-tuned on Tinker with verified data and both reward modifications. Under greedy decoding (single sample, temperature = 0) our model achieves an accuracy of 91.37% on Arcwise-Plat-SQL at a cost of $0.035 per task. This is an 8.4 point improvement over OpenSearch, the strongest prior open-source pipeline, at a 37% lower cost. The cost advantage is a direct consequence of removing auxiliary scaffolding around the model.

If ReViSQL-K2.6 votes among 16 candidates generated with temperature = 1, accuracy rises to 92.97% at a cost of $0.56 per task. This is the first time a text-to-SQL AI system has exceeded the human benchmark to our knowledge.

Conclusion

When AI shows poor performance in a domain-specific task, the common response is to add scaffolding around the task execution. This approach improved performance for text-to-SQL models somewhat, but it ultimately hits the ceiling imposed by the base model’s capability. This doesn’t mean that scaffolding is worthless. Rather, it suggests that the task knowledge contained in the scaffold belongs in the training signal, the same training that would uplift the model’s overall capacity.

The common thread with our previous work on AI trained for financial judgment is that custom models can outperform the frontier on a wide range of tasks involving expert taste and judgment, often at a fraction of the cost. This requires expert judgment to be part of the training process: in identifying how AI fails, labeling the training data, and aligning training to the intended behavior.

In the case of text-to-SQL we saw major improvements from both a rigorous cleanup of the data and the shaping of the reward function to teach the model the correct skill. This took more effort upfront, but the resulting model achieves better performance at lower cost than both humans and scaffolded models. Putting task expertise into task-specific training is ultimately what scales.

Citation

Please cite this work as:

Zhu, Yuxuan et al., "Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering", arXiv:2603.20004 (2026).

@article{zhu2026revisql,
  title = {Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering},
  author = {Zhu, Yuxuan and Jin, Tengjun and Choi, Yoojin and Kang, Daniel},
  journal = {arXiv preprint arXiv:2603.20004},
  year = {2026}
}

Breaking Claude Code Opus 5 Auto Mode

Anthropic are putting a great deal of faith in Claude Code's auto mode for protecting their coding agent users against prompt injection attacks. They recently made that the default and have made bold claims about its effectiveness.

Johann Rehberger is one of the most credible prompt injection researchers active today. He found an attack against auto mode which he claims works 80% of the time, by tricking Claude Code into downloading and uncompressing a zip archive, then executing code that imports base64 without noticing that this will import and execute a local struct.py file extracted from the archive.

In a few cases auto mode directly prevented the agent from preventing harmful code from continuing to execute!

In a few runs Claude tried to terminate the malware process once it noticed the compromise, but Auto Mode denied the cleanup command.

Claude detects the compromise, but Auto Mode blocks its cleanup command

The safety mechanism itself can become part of the failure. The classifier allowed the creation of the malware process, but then it blocked the command intended to stop it!

I agree with Johann's conclusion here: the only safe way to run agents if there's any risk of attracting the attention of an adversarial attack is with a sandbox:

• Run unattended coding agents in a container, VM or OS sandbox.
• Restrict network egress.
• Monitor your agents.
• Do not expose home directories, SSH keys, cloud credentials,… to the agent runtime.

Big Pineapple, the platform behind 1.1.1.1, Gateway DNS, DNS Firewall, AS112, and several other Cloudflare DNS services, stores over 250 billion DNS cache entries at any given time. At that scale, wasting a single byte per entry costs more than 250 gigabytes of memory across our fleet.

Five successive changes to how cache entries are stored in memory cut the per-entry footprint by over 50%. Across our fleet, these changes freed up roughly 100 terabytes of memory, equivalent to the amount of RAM in 130 of our Gen 13 servers. The cache also got faster. Insert throughput rose 43% and lookup latency dropped 19%, as fewer allocations and better memory locality meant we did not trade speed for space.

What we cache

On cold start, Big Pineapple starts out with an empty cache. As DNS queries arrive, the cache fills until it hits its maximum entry count, at which point we evict older or less popular items to make room.

The exact cache size varies by data center. When EDNS Client Subnet (ECS) is in use, authoritative servers return different answers depending on the client's network, so we cache multiple versions of the same query. This increases both the number of entries and the memory each one consumes, making the optimizations in this post especially impactful for ECS-heavy locations.

Each item in the cache is a key-value pair. The key identifies what was queried:

pub struct CacheKey {
    qname: Name,
    qtype: Rtype,
    authenticated: bool,
    tag: Vec<u8>,
}

The value stores the DNS response itself: the answer, authority, and additional record sections, along with metadata like the creation time, a hit counter, and the Time-to-Live (TTL).

pub struct CacheEntry {
    timestamp: UnixTimeStamp,
    pub inception: Instant,
    pub ttl: Ttl,
    pub hits: u32,
    pub answers: Vec<Record>,
    pub authority: Vec<Record>,
    pub additional: Vec<Record>,
    pub errors: Vec<ExtendedError>,
    ...
}

Both structs have room for improvement. Several fields use types that carry overhead we don't need once the entry is stored.

Benchmarking memory usage

To measure the impact of each change, we benchmark by filling the cache with randomly generated entries that roughly match the traffic distribution we see in production: 56% A records, 25% AAAA, and 19% TXT. Each entry contains between one and four records.

TXT records serve as a stand-in for all non-A/AAAA record types in the benchmark. Their size is randomized between 64 and 224 bytes, close to the average response size we see for variable-length record types.

We track memory usage using a custom allocator that wraps Rust’s System allocator and records the number and size of allocations per cache entry. Alongside memory, we measure insert throughput and lookup latency across the full cache flow to make sure memory savings don’t come at the cost of performance.

These inputs approximate production rather than reproduce it exactly. Process memory also depends on traffic mix, cache occupancy, allocator state, and memory used outside the cache. We therefore measured resident memory across production instances during the rollout.

The cost of capacity

Vec<T> stores three fields: a pointer to heap-allocated data, the current length, and the total capacity. When you push an item, Vec checks whether the length exceeds the capacity and reallocates if needed. If there’s room, it just appends the item and increments the length.

Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec. The over-allocated heap space is wasted as well, as a Vec with capacity for eight items but only five stored leaves three slots unused on the heap.

Using Box<[T]> solves both problems. It can’t grow after creation, so it doesn’t need a capacity field or reserve space for future elements. The same applies to String, which also carries a capacity field. Box<str> drops it.

Each cache entry stores 8 Vec and String fields. Replacing them with Box<[T]> and Box<str> saves 8 bytes per field, 64 bytes per entry. It also eliminates the excess heap memory that Vec reserves for future growth. The combined savings add up to over 15 terabytes with over 250 billion cache entries.

Fewer lists, fewer pointers

Rather than storing the answer, authority, and additional sections in separate lists, we can store a single list with offsets to the start of each section. Since DNS record counts per section fit in a u16, we can use a u16 (2 bytes) for each offset, compared to the 8-byte pointer and 8-byte length that each separate Box<[T]> requires.

This removes two lists, each with an 8-byte pointer and 8-byte length, and replaces them with two 2-byte offsets, saving 28 bytes per entry.

These savings do not always map directly to the number of bytes removed from individual fields. Rust inserts padding to satisfy alignment requirements and rounds a struct’s size up to a multiple of its alignment. Removing a small field can therefore eliminate additional padding. For example, we also packed several boolean fields into a single bitflag. This reduced the surrounding padding, causing the struct to shrink by more than the size of the individual booleans.

Dropping the owner

Each DNS record has an owner, the domain the record belongs to. In many cases, this owner is identical to the domain being queried.

The DNS wire format handles repeated owners using name compression, as defined in RFC 1035. Rather than encoding the same domain twice, subsequent occurrences store a 2-byte pointer to the first occurrence.

This works well on the wire, but in our cache we store the full owner name alongside each record. Following compression pointers during cache lookups is expensive on the hot path, so we trade memory for speed.

Most records, however, have an owner identical to the queried domain. For those, we can drop the owner entirely and infer it at read time. When the owner differs, such as the A records behind a CNAME, we store the full name.

pub struct Record {
    owner: Option<Box<Name>>,
    class: Class,
    ttl: Ttl,
    rtype: Rtype,
    data: RecordData,
}

When owner is None, response construction restores the queried domain from the cache key, avoiding a heap allocation. This means the record is no longer self-contained, but the cache key is already available during every lookup. When the owner differs, Some stores a pointer to the full name on the heap.

In practice, most cached records have an owner identical to the queried domain, so the majority require no heap allocation for the owner field.

Enum sizing

Rust enums are sum types: each variant can carry different data, but the enum is always the size of its largest variant.

For record data, it seems natural to store each DNS record type as an enum variant. But the enum is always as large as its largest variant. In our case, that’s NAPTR at 136 bytes. It stores three variable-length text fields, a domain name, and two integers. As a result, the full enum, including the variant tag and padding, becomes 144 bytes.

An A record only needs 4 bytes, and an AAAA record needs 16 bytes. A and AAAA make up over 80% of our traffic, so most records waste over 120 bytes on padding. Since a single cache entry can store many records this quickly adds up.

Boxing the variants

To solve this problem, we can box the larger variants of the enum, moving them to a separate heap allocation. The enum then stores an 8-byte pointer to the heap, where the data takes up only the size it actually requires.

For A and AAAA records, this saves 120 bytes per record. Smaller variant types like TXT and CNAME also benefit. They still occupy the 24-byte enum, but their heap allocation is sized to their actual data rather than padded to 144 bytes.

The costs of boxing

Boxing has two costs. The first is allocator overhead. Each boxed variant becomes a separate heap allocation, and allocators round up to the nearest size class. Big Pineapple uses jemalloc, an allocator designed for multithreaded, allocation-heavy workloads.

The second cost is poor memory locality. Without boxing, the record enum values for a cache entry sit in a single contiguous allocation. With boxing, data for each boxed variant lives in a separate heap region. Reading it requires following a pointer, and when that pointer lands far from the rest of the entry, the CPU has to fetch a new cache line.

Storing records in wire format

As a middle ground, we store just the record data as raw bytes, while keeping the rest of the cache entry as structured fields. Instead of a list of parsed enum variants, we store the records as a single Box<[u8]> containing each record encoded as a 2-byte length prefix followed by its raw bytes.

This eliminates the per-variant enum overhead and the boxed heap allocations from the previous optimization. The data also becomes packed contiguously, which improves CPU cache locality.

When building a DNS response from cached records, most record types can be copied directly from the buffer into the outgoing message. Previously, each parsed record had to be serialized field by field back into DNS wire format. The new layout skips that work for A, AAAA, TXT, and all DNSSEC record types by copying their encoded bytes directly.

The results

Per-instance memory usage dropped across all percentiles. At p99, memory dropped from 9.3 GB to 5.3 GB, a 43% reduction in resident memory. At p90, memory dropped from 6.5 GB to 3.8 GB, a 42% reduction. Instances with fuller caches saw the largest absolute savings.

In our benchmarks, these five optimizations reduced the per-entry memory footprint from 953 bytes to 420 bytes, a 56% reduction. Per-entry allocations dropped from 1.1 KB to 461 bytes. The reductions measured in production are smaller because resident memory includes the cache alongside all other process data. After the rollouts settled, aggregate working-set memory across the fleet was roughly 100 terabytes lower.

Performance also improved. Cache insert throughput increased by 43%, while lookup latency dropped by 19%.

We plan to reinvest the freed memory into increasing cache capacity without increasing our memory usage, which improves cache hit rates and reduces upstream query volume. We're also exploring further optimizations to the cache itself.

Full article content is not available for inline reading.

Read the original article →

Tailcat

"Tailscale without Tailscale, by Tailscale"

Tailcat is a remix of Tailscale open source pieces to act like netcat, but over Tailscale's data plane, without Tailscale's control plane. Tailscale's data plane (magicsock, internally) gives you point-to-point WireGuard®-encrypted tunnels between two machines with DERP as the NAT-hole-punching communication side channel and the ultimate relay-of-last-resort if NAT traversal fails. Instead of using the Tailscale control plane, all tailcat connection metadata is exchanged out of band, however you want.

The tailcat CLI (in cmd/tailcat) is built on the tailcat Go library (importable as github.com/tailscale/tailcat).

Whether you use tailcat as a CLI tool or library, one side runs a tailcat server (listener) and gets back a short connection token. The other side passes that token to tailcat's client side to connect. All traffic between the two is encrypted end-to-end with WireGuard. The initial connection bootstraps through a DERP server, and then magicsock performs NAT traversal to upgrade to a direct peer-to-peer UDP connection when possible (usually!).

You don't need a Tailscale account, root/admin access on the machine (it doesn't alter your machine's routing tables, DNS, etc.). It's just a userspace library and CLI tool.

And it's all open source.

You can use our free rate-limited DERP relays (the default DERP map is https://tailcat.dev/derpmap.json) or you can run your own.

There's also an experimental in-browser web demo (tailcat compiled to WebAssembly) at https://tailscale.github.io/tailcat/ that can send and receive files or text, interoperating with the CLI. Browser traffic is relayed over DERP only, with no direct connections until WebRTC support.

Install

$ go install github.com/tailscale/tailcat/cmd/tailcat@latest

Or with Nix flakes, run it directly or install it:

$ nix run github:tailscale/tailcat
$ nix profile install github:tailscale/tailcat

Usage

Pipe stdin/stdout between two machines

Server starts, printing out its ephemeral address:

$ tailcat
# Selected bootstrap relay region 302, San Francisco
# 🐈 Server listening with new address: tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
(hangs, waiting...)

And then the client can:

$ echo hello | tailcat tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
$

Then the server unblocks:

$ tailcat
# Selected bootstrap relay region 302, San Francisco
# 🐈 Server listening with new address: tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
hello
$

Expose local ports through the tunnel

Or you can serve a local TCP port, forwarded to localhost:

$ tailcat --serve=8080,8443 # or --serve=all
# 🐈 Server listening with new address: tcXXXXXXXXX

And then the client:

$ tailcat tcXXXXXXXXX 8080
GET / HTTP/1.1
Host: foo

HTTP/1.1 200 OK
....

Auth-free SSH server

On Linux and macOS, you can run an SSH server too with no auth. (If you want auth, you can just tailcat --serve=22 and proxy to your system SSH server)

$ tailcat --serve=no-auth-ssh
# 🐈 Server listening with new address: tcXXXXXXXXX

And on the client side:

$ tailcat ssh tcXXXXXXXXX
$ tailcat ssh tcXXXXXXXXX ls -la

Misc commands

Ping to test connectivity; each pong reports whether it arrived via a DERP relay or a direct path. --until-direct keeps pinging (up to --timeout, default 10s) until a direct path works, exiting non-zero if one doesn't:

$ tailcat ping --until-direct <token>
pong in 42.1ms via DERP(sfo)
pong in 1.2ms via 203.0.113.7:41641

Run a command through a SOCKS5 proxy routed over the tunnel:

$ tailcat socks <token> curl http://server.tailcat:8081/

Tokens also work directly as URL hostnames: the SOCKS proxy recognizes and dials them, so the token argument is optional. (Tokens are case-sensitive; this works with curl and most CLI tools, but not with browsers, which lowercase hostnames.)

$ tailcat socks curl http://<token>:8081/

Act as an exit node so the client can reach the server's network:

$ tailcat --serve=exit-node

Parse a connection token and print its contents (the server's WireGuard public key and DERP info) as JSON, without connecting to anything:

$ tailcat parse tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
{
    "ServerPublic": "nodekey:9c8d2e6728da80a1dd37e275a82595b42d9a838610bc53f74a7670d1610f2e34",
    "RegionID": 302
}

Resolve a short token (which references a DERP region by ID, requiring clients to fetch the DERP map) into a longer self-contained one with the DERP server info embedded, letting clients connect more quickly:

$ tailcat resolve tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFygaFhToGjYWhudGMzMDJhLmlwbi5kZXZhNG0yMDguMTExLjM5LjM4YTZzMjYwNzpmNzQwOjA6M2Y6OjcyMA

Parsing that resolved token shows the embedded DERP info:

$ tailcat parse tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFygaFhToGjYWhudGMzMDJhLmlwbi5kZXZhNG0yMDguMTExLjM5LjM4YTZzMjYwNzpmNzQwOjA6M2Y6OjcyMA
{
    "ServerPublic": "nodekey:9c8d2e6728da80a1dd37e275a82595b42d9a838610bc53f74a7670d1610f2e34",
    "Region": [
        {
            "Nodes": [
                {
                    "HostName": "tc302a.ipn.dev",
                    "IPv4": "208.111.39.38",
                    "IPv6": "2607:f740:0:3f::720"
                }
            ]
        }
    ]
}

A server can print the long self-contained form directly with the --full-address flag.

Key Management

A server's address (connection token) is derived from its WireGuard key, so the key you use determines who can reach you:

• Ephemeral keys (the default): each server run generates a fresh key in memory and prints an address nobody has ever seen. When the process exits, the key is discarded and the address is dead forever. This is the safe default: sharing that address only ever refers to that one run.
• Saved keys: tailcat genkey generates a key saved to disk so the address stays stable across restarts. The flip side: anyone you've ever shared that address with can connect to any future server using that key, unless you restrict clients with --allow (see tailcat genkey --client).

The CLI says at startup which kind it's using, so you know whether you're starting a fresh single-use server or re-listening on an address you may have shared in the past.

$ tailcat genkey --region=nyc
# prints the token; key saved to ~/.config/tailcat/keys/default.private.json

# later; the key named "default" is used automatically once it exists:
$ tailcat --serve=8080
# 🐈 Server listening with saved key "default": tcXXXXXXXXX

# ... unless you force a one-off ephemeral key:
$ tailcat --serve=8080 --key=new
# 🐈 Server listening with new address: tcXXXXXXXXX

That is, default is a magic key name: once it exists, plain tailcat silently uses it instead of generating an ephemeral key, and the startup line above is what tells you which happened. Use --key=new to get an ephemeral key anyway, --key=<name> to use a different saved key, or tailcat genkey --delete --key=default to remove the saved default key. tailcat genkey --list lists your saved keys.

Tokens can also be published as DNS TXT records and looked up by name; a DNS name works anywhere the CLI takes a token:

# If example.com has a TXT record "tailcat=tc..."
$ tailcat example.com 8080
$ tailcat ssh example.com
$ tailcat ping example.com

Examples

Protected SSH server over DNS

Who needs port forwarding or port knocking? This runs an SSH server reachable from anywhere by name, with no open inbound ports on the server, where WireGuard authenticates the client before the SSH server ever sees a packet.

On the client machine, generate a client identity keypair. It prints the public key, which is all the server needs to know:

client$ tailcat genkey --client
# wrote file to ~/.config/tailcat/keys/client-default.private.json
nodekey:cfb6bfa77a0654d7450947fd6acef17d2cd848da1d30b2540b13dac272ddfd16

On the server, generate a server keypair pinned to its nearest DERP region (see why below), then serve SSH to only that client:

server$ tailcat genkey --fixed-region
# wrote file to ~/.config/tailcat/keys/default.private.json
tcXXXXXXXXX

server$ tailcat --serve=22 --allow=nodekey:cfb6bf...ddfd16
# 🐈 Server listening with saved key "default": tcXXXXXXXXX

Publish the token in DNS as a TXT record:

my-server.example.com. 300 IN TXT "tailcat=tcXXXXXXXXX"

And then the client side is just:

client$ tailcat ssh my-server.example.com

Client modes automatically use the saved client-default key when it exists, so no extra flags are needed to present the allowed identity. Anyone else's handshake is silently ignored: they can't reach the SSH server, or even learn that one is running.

Bring your own DERP relay

Nothing requires Tailscale's relays: run your own DERP server (it needs a hostname with a TLS certificate, which derper can get itself via Let's Encrypt), then generate a server key that uses it by passing its hostname (or several, comma-separated) as the region:

server$ tailcat genkey --region=derp.example.com
tcomFwWCCAIsKOqPUux6ClG2RM4A_vOq4VBzGgHGGjq9OsJuFKSWFygaFhToGhYWhwZGVycC5leGFtcGxlLmNvbQ

server$ tailcat --serve=22

Go library

A minimal server that answers any TCP port through the tunnel and prints its token.

package main

import (
	"fmt"
	"log"
	"net"

	"github.com/tailscale/tailcat"
)

func main() {
	s := &tailcat.Server{
		OnTCP: func(port uint16) func(net.Conn) {
			return func(c net.Conn) {
				fmt.Fprintf(c, "hello from port %v\n", port)
				c.Close()
			}
		},
	}
	if err := s.Start(); err != nil {
		log.Fatal(err)
	}
	fmt.Println(s.ConnBlob())
	select {}
}

How it works

Connection tokens

A Tailcat server is identified by a connection token (called a ConnBlob internally). It looks like tcXYZ... and is a "tc" prefix followed by base64-encoded CBOR containing the server's WireGuard public key and DERP info.

Network stack

• WireGuard -- a userspace WireGuard implementation for encrypting all tunnel traffic.
• magicsock -- Tailscale's transport layer that multiplexes traffic over direct UDP and DERP relays.
• Netstack (gVisor) -- a userspace TCP/IP stack that terminates TCP connections inside the process.
• DERP relay -- Tailscale's encrypted relay protocol, used as a rendezvous channel and as a fallback data path when direct connectivity isn't possible.

Connection flow

1. Server starts. It generates (or loads) a WireGuard keypair, connects to a DERP relay, and prints its connection token to stderr.
2. Client parses the token to learn the server's public key and DERP region.
3. Discovery handshake. The client sends a "Meow" ping message to the server through the DERP relay.
4. WireGuard tunnel. With both sides configured as WireGuard peers, the standard WireGuard handshake proceeds.
5. NAT traversal. In parallel, each side advertises its UDP endpoints to the other in disco messages over DERP, attempting UDP hole-punching.
6. Data transfer. The client dials a TCP port on the server through the tunnel.

Stability

Tailcat is free to use, but it comes with no API or CLI stability promises: the Go API, the CLI flags and output, and the wire format may all change. The public rate-limited Tailcat DERP relays have no uptime SLAs or throughput targets.

History

Tailcat began life in September 2023. It was open sourced August 2026 at the TailscaleUp conference.

An AI factory is not just a model or a cluster. It is a pool of GPUs that many teams draw from at once: one team fine-tuning, another serving inference, a third running evaluations, all on the same accelerators. NVIDIA frames it as “infrastructure for the full AI lifecycle, from data preparation through training, fine-tuning, and high-volume inference”. In an enterprise that means one fleet, many teams, and different quotas, policies, and trust boundaries layered on top. The hard question is no longer how to train a model. It is how to give every team safe, isolated access to the same expensive hardware without anyone stepping on anyone else.

Two years ago every platform team was building a developer platform. Kubernetes already had mature primitives for containers, RBAC, autoscaling, and policy. What it did not have was a clean answer for accelerators, or for keeping tenants apart on the same nodes. That is the gap an AI factory has to close, and the cloud native ecosystem now supplies most of the parts to close it.

The bottleneck is utilization, not model serving

Accelerators are the dominant capital expense in the building, and the metric that decides whether that spend pays off is utilization, not a peak tokens-per-second number from a single run. The market grades GPU clouds the same way. ClusterMAX scores providers on security, networking, storage, reliability, and support rather than raw throughput, and its security criteria reward hard per-tenant isolation, down to per-tenant Kubernetes clusters and DPU-based isolation, while flagging weak boundaries like putting many tenants on one cluster. The wrapper around the GPUs is what gets judged.

Two things keep utilization low. First, the resource model: in the traditional device-plugin model a pod asks for nvidia.com/gpu: 1 and pins a whole accelerator even at ten percent use. Dynamic Resource Allocation (DRA), GA in Kubernetes 1.34, lets the scheduler treat accelerators as rich devices with attributes, memory, and topology, though it does not by itself carve a GPU into fractions; density comes from the device layer underneath. Second, the isolation model: to keep teams apart, platforms default to a dedicated cluster or a dedicated set of GPUs per team, which is the safe choice when trust is strict and wastes most of the hardware.

The same pattern recurs in the field: operators managing tenants with a bare metal provisioner and manual workarounds, or handing each customer a dedicated block of GPUs and turning away demand they cannot isolate cleanly. The fix is not a new model server. It is a stack that allocates accelerators so capacity is neither stranded nor unsafe, and isolates tenants so packing them together holds up.

The stack, layer by layer

An AI factory is an assembly problem. Most layers are Kubernetes native or CNCF projects, with a few OSS tools such as NVIDIA’s MIG, vCluster and Dynamo.

From bare metal to validated capacity

Everything starts at the rack.

Take a typical modern AI supercomputing platform as an example. Before any GPU can run a workload, something has to turn raw servers into a usable pool. That is the provisioning layer, often a proprietary hardware manager that ships with the system.

It works in steps. First it discovers each node, taking inventory: which GPUs and how many, whether memory is healthy (ECC state, meaning error-correction is on and not logging faults), and the identities of the network cards (InfiniBand GUIDs and NIC MACs, the permanent hardware IDs used to wire up and boot the node). Next it network boots the node and installs an OS image with the GPU driver and the CUDA and NCCL libraries baked in, so it can compute the moment it comes up. It then applies BIOS settings that match the node’s goal: baseline, performance, or confidential-compute.

Before a node joins the pool, it is tested. This burn-in runs the node under load to catch early failures, and an NCCL test confirms the GPUs actually talk to each other at full bandwidth. The result is written to a source of truth like NetBox, which also tracks IP address assignments (IPAM). Retiring a node runs the flow in reverse: wipe the disks, reset the remote-management login (eg the BMC), and return the clean node to the pool.

GPU allocation: the layer that makes the economics work

DRA gives a richer device-claim model, but fractional density comes from the device implementation. HAMi, a CNCF Incubating project, enforces per-pod memory and compute limits in software so several pods run on one card with guardrails between them, and it spans multiple accelerator vendors.

Operators heading toward confidential computing do not place untrusted tenants on the same physical GPU; they give each tenant a whole GPU and reserve partitioning for workloads inside a single trust domain. MIG does isolate memory and faults in hardware, but its use as a boundary between hostile tenants is contested, so the conservative default is whole-GPU per tenant. The layer has two jobs: whole-GPU allocation for tenant isolation, and partitioning for density within a tenant. Scheduling is separate: KAI Scheduler and Volcano handle gang and topology-aware placement, and Kueue handles queueing, admission, and quota.

The workload layers: serving, Slurm, and VMs

Above allocation sit the things teams actually run. For inference, vLLM is a common engine and KServe, a CNCF incubating project, wraps it with autoscaling and standard endpoints, while NVIDIA Dynamo and llm-d push disaggregated inference for larger deployments. In front, Gateway API handles routing and LiteLLM adds an OpenAI-compatible gateway so dozens of specialized models speak one API.

Training customers usually live in Slurm, and the pattern has converged on running it on Kubernetes through SchedMD’s Slinky, which represents the Slurm daemons as CRDs and integrates with the GPU Operator and DRA for topology-aware scheduling, with pyxis and enroot, GPUDirect RDMA at full NCCL bandwidth, and prolog and epilog health checks. And some tenants want plain virtual machines rather than pods; KubeVirt runs VMs as Kubernetes workloads, so one platform hands out both containers and VMs from the same pooled fleet under the same RBAC and quotas.

Networking, storage, and observability

Training and disaggregated inference are bandwidth-bound, so the network is part of the design. Cilium handles the primary CNI and network policy; for the fast path, Multus and SR-IOV expose the NIC directly and RDMA over RoCEv2 or InfiniBand carries inter-node GPU traffic, with the isolation layer kept off that data path.

A real cloud also gives tenants the cloud-edge services they expect: elastic IPs, NAT, and L4 load balancing from a gateway in front of the fabric. Storage needs per-tenant persistence, usually CSI with Rook and Ceph or a parallel filesystem, governed by per-tenant StorageClasses and quotas.

For observability, OpenTelemetry is the neutral collection layer that keeps backends swappable, with Prometheus for metrics and VictoriaLogs for logs; the DCGM exporter publishes GPU telemetry that becomes per-tenant only with labels and a cost pipeline, and OpenCost turns GPU-seconds into chargeback.

Reliability and security

At fleet scale GPUs fail constantly: ECC errors, cards that fall off the bus, NVLink and thermal faults. The operator’s job is to catch these before a tenant does, which makes health a first-class layer rather than a dashboard afterthought.

Active and passive checks on DCGM watch for degradation, Node Problem Detector turns hardware signals into node conditions, and a remediation loop cordons and drains a suspect node before new work lands on it. This is one of the categories the rating systems weigh most, because reliability, not peak throughput, is what a customer feels first. Identity and policy round it out: Keycloak over OIDC, OpenBao with the External Secrets Operator, Kyverno or OPA for guardrails, and Falco and Trivy for runtime and supply chain, with audit logs exported to the observability stack and traffic encrypted in transit.

Isolating tenants

Every layer above assumes one thing: that you can safely run more than one team on the same hardware. That is the tenant-isolation problem, and it has two halves worth keeping separate.

The first is the control plane. The tenant-cluster pattern gives each team a virtual control plane: a full Kubernetes API server with its own CRDs, admission webhooks, versions, and RBAC, running as a workload on a single underlying cluster, with no view into another tenant. Several CNCF and open source projects implement this pattern, like vCluster. Because each tenant cluster is conformant Kubernetes, plain kubectl, Helm, and Argo CD with no proprietary extensions, the model gives tenants a clean exit path rather than lock-in.

In practice operators run two tiers. High-trust or enterprise tenants get a dedicated cluster, sometimes dedicated hardware, where the boundary is physical; smaller or cost-sensitive tenants get a tenant cluster on pooled capacity. The same control plane drives both. Reliability follows from the same design: because a tenant control plane runs as pods, Kubernetes reschedules it on failure, and the open question is blast radius, so operators cap how many tenants ride one underlying cluster.

The second half is the data plane, which a tenant cluster does not solve on its own. You still need network isolation, storage isolation, quotas, Pod Security, and a runtime boundary. Network isolation usually comes from the fabric rather than from Kubernetes: a control plane carves per-tenant VPCs with VXLAN and EVPN on the Ethernet side and partition keys on InfiniBand. Increasingly that enforcement is pushed into hardware, where DPUs (Data Processing Units) such as NVIDIA BlueField or AMD Pensando move isolation and encryption off the host CPU, which is also how operators reach a confidential computing posture.

For the runtime boundary on shared nodes, the options range from dedicated nodes to sandboxed runtimes such as vNode. The bar for a real cloud is hardware-enforced isolation, not namespaces and good intentions.

What makes it a cloud, not just infrastructure

The line between a pile of GPUs and a cloud is that a customer can provision it themselves and get a bill that makes sense. Both are cloud native problems. Self-service means API-first with no UI-only paths: a tenant creates and deletes clusters through an API, a Terraform provider, or GitOps, with resources expressed as declarative CRDs reconciled by Flux or Argo CD, and access scoped by RBAC through OIDC.

The bill comes from the metering layer: DCGM-driven GPU-seconds and OpenCost allocation, exported per tenant. None of this is glamorous, and it is usually the widest gap between a lab and a product. It is also, more than raw performance, what customers experience day to day.

From demo to production

Put the stack together and the demo is simple: two teams, two tenant clusters, two model endpoints, one physical GPU partitioned by MIG or software limits, each with its own RBAC, network policy, metrics, and cost line, neither aware of the other. This has been shown live on stage at KubeCon + CloudNativeCon with a single modern GPU serving two models at once.

Two things turn it into production. The first is conformance: tooling like NVIDIA’s AI Cluster Runtime validates cluster configurations against the hardware you actually have and emits reproducible Helm or GitOps artifacts, and the Kubernetes AI Conformance program, introduced in the 1.35 release, pushes the same idea at the platform level.

The second is scale: the design has to hold at hundreds of GPU nodes and several data centers, not the handful you prove it on, which is the real reason the foundation is GitOps, declarative tenants, and a single source of truth. There is a strategic choice here too, because the hardware vendor is moving into this layer with an integrated suite, NVIDIA’s DSX OS, so an operator decides layer by layer whether to adopt it, assemble the equivalent from cloud native projects, or compose the two.

The Takeaway

An AI factory is not another AI platform or model serving product. It is an operating model for running GPU infrastructure at scale on Kubernetes. Just as Kubernetes became the operating system for cloud native applications, it is becoming the foundation for AI infrastructure, making GPUs schedulable resources, providing isolated environments for tenants, and enabling on demand compute. The challenge is not deploying technologies like MIG, DRA, HAMi, or vLLM, but combining them into a platform that balances utilization, isolation, and cost while allowing multiple teams to safely share expensive GPU infrastructure without compromising performance or security.

Software is only half of it. The hardware layer is just as hard, often harder. Topology decides performance: which GPUs share an NVLink or NVSwitch domain, how each node attaches to a rail-optimized InfiniBand or RoCE fabric, whether the GPU, NIC, and CPU sit on the same NUMA node, and whether GPUDirect RDMA has a clean path. Schedule work without accounting for any of it and collective operations stall on the slowest hop, no matter how healthy the platform looks on paper. The stack has to be topology-aware, not just resource-aware.

The hard part is not naming the tools. It is making density, isolation, and chargeback work together, with hardware-enforced boundaries where the trust model demands them, without hiding the GPU data path behind an abstraction.

Announcing H3 Max, a post-trained version of MiniMax H3 developed by fal Research and optimized for maximum speed by fal's inference team.

In our human preference evaluations, H3 Max ranks #1 across overall quality, prompt understanding, and aesthetics against leading video models. It does so while generating a 5-second video in under 3 seconds, which is roughly 35x the throughput of the official MiniMax H3 endpoint, and on average 15x faster than anything with comparable quality.

These results challenge a common tradeoff in generative video: that higher quality has to come at the cost of slower inference.

H3 Max is the result of working on both sides of that problem at once. We optimized the model for stronger real-world performance while co-designing the inference system around it to make sure faster than real-time is still possible. This required two capabilities that rarely sit under the same roof: frontier model research and deep inference optimization/kernel work.

Post-training H3 Max

We started with the open-weights MiniMax H3 model and introduced substantial new data during post-training, with a particular focus on prompt adherence and visual quality. We aimed for a better model at much faster speed and after our work, the base model's core capabilities are intact with extremely low latency.

The MiniMax H3 team shared their perspective on H3 Max:

H3 Max combines SOTA video quality with a step-change in generation speed, making high-quality video generation practical across a much broader range of real-world applications. We’ve worked closely with fal since day one, and their expertise in generative AI infrastructure and bringing frontier models into production quickly and reliably makes them a natural partner for H3 Max.

Throughout post-training, we continuously evaluated checkpoints through head-to-head preference studies across three dimensions: overall quality, prompt understanding, and aesthetics. Evaluating these dimensions independently gave us a much clearer signal than optimizing against a single aggregate score.

The result is a model that meaningfully improves on the original H3 across the qualities people actually notice when generating video: understanding what you asked for and producing something you want to use.

Co-designing the inference engine

With H3 Max, the research and inference work was deeply connected. Our inference team has spent the past four years optimizing diffusion and generative media workloads. For H3 Max, we applied those systems techniques while the model itself was still being developed, allowing decisions on the training and inference stack to inform each other. H3 Max was trained and served entirely on NVIDIA GB200 NVL72 systems. On a per-chip basis, GB200s deliver up to 2x the performance of the previous-generation accelerators we used to train and serve our models.

The objective wasn't just a faster model, our team has also worked on maximizing the throughput while preserving the quality gains from post-training. That distinction shaped the entire optimization process.

There are ways to make a video model faster: reduce precision, remove sampling steps, or approximate expensive operations. Some produce impressive speedups while degrading the output. For H3 Max, an optimization only survived if the resulting model continued to hold its position in our internal quality evaluations.

Measuring quality

Evaluating generative video is inherently difficult. Automated metrics capture only part of what makes one generation better than another, so our primary evaluation uses head-to-head human preferences.

We benchmarked H3 Max against twelve leading video models, including the official MiniMax H3 endpoint, Gemini Omni Flash, Wan 3.0, Seedance 2.5, Kling 3, and Veo 3.1.

Evaluators compared generations across three dimensions:

• Overall preference: which video they preferred as a whole
• Prompt understanding: which generation more faithfully followed the instruction
• Aesthetics: which generation was visually stronger

We aggregated these comparisons using Bayesian Elo ratings with 95% confidence intervals.

H3 Max ranks #1 across all three dimensions and wins the majority of head-to-head matchups against every model we tested, including the original H3.

The results hold up outside our own evaluations. In independent benchmarks from Artificial Analysis and Design Arena, H3 Max also ranks #1 against other video models.

On H3 Max, Design Arena noted:

MiniMax H3 Max by fal delivers the quality of MiniMax H3 at more than 50× the speed, according to Design Arena’s independent benchmarking, establishing a new speed–preference Pareto frontier.

Quality without the latency tradeoff

The more interesting result is when we look at quality and speed together.

H3 Max generates a 5-second video in approximately 3 seconds. That's roughly 35x the throughput of the official H3 endpoint and faster than every other model in our comparison.

Typically, the frontier forces a choice: faster models occupy one end of the curve and higher-quality models occupy the other.

H3 Max moves the frontier.

It achieves the highest human-preference score in our evaluation while simultaneously delivering the highest throughput.

This is the advantage of treating model research and inference optimization as the same problem. Post-training gives us control over the quality of the model. Systems work gives us control over how efficiently that model executes. Co-designing the two means improvements don't have to come at each other's expense.

For generative media, this will increasingly matter. Models are getting larger and more computationally demanding at the same time that video generation is moving into interactive and high-volume production workloads.

A model isn't useful in production because it tops the benchmark in isolation. What matters is the frontier across quality, latency, and cost.

Our goal at fal is to push all three.

Try H3 Max

H3 Max is available today on fal. Try it in the Playground, via fal Agent, or call it from the API.

For the first week, we're offering H3 Max at 50% off.

Try it now:

• Text to Video
• Image to Video

H3 Max was trained on fal Serverless, the same infrastructure available to developers and teams building their own models. Learn more about fal Serverless.

Key takeaways

• Best-in-class value: outperforms leading document parsers and hyperscaler services while remaining cost-effective at enterprise scale.
• Beyond OCR: understands tables, forms, diagrams, and images to extract richer semantic context across key global commercial languages.
• Enterprise-first: trained to handle business documents in major industries and domains, such as finance, insurance, and scientific work.
• Spatially aware: returns bounding boxes for visual elements, preserving document structure for retrieval, grounding, and automation.
• Available in Compass: use Parse today as part of Cohere’s Compass search and retrieval stack - alongside Embed, and Rerank.
• Secure deployments: run in any private cloud or on-premises environment to match your organization's security and compliance profile.

Cohere Parse is a cost-effective vision language model for processing large volumes of enterprise documents. It converts complex, multimodal files into structured, machine-readable data that can power enterprise knowledge use cases–including document indexing, RAG, and agentic retrieval.

More than text recognition, Parse detects and understands key visual elements - such as tables and embedded images - and returns clean Markdown files for downstream processing and application. Use Parse to process your documents and images across nine major world languages.

Parse is designed to preserve parsing quality and keep inference costs predictable as workloads scale. It supports the high-throughput needed in production environments. Customers can access Parse through the Cohere API for just $1.50 per 1,000 pages, or deploy in Model Vault for secure, single-tenant inference and even further cost savings per page. Teams in regulated industries can deploy Parse securely on their own infrastructure with a minimal serving footprint.

Want to try it first? See how Cohere Parse handles your documents for free using our Space.

Performance

Cohere Parse delivers the strongest price–performance tradeoff among the models we evaluated. It is a highly competitive model that outperforms leading specialized document parsing solutions while maintaining a price point suitable for high-volume workloads spanning hundreds of thousands to millions of pages.

On ParseBench - which measures agent-suitable parsing performance - Parse scores 79.2 across three evaluation dimensions compared with 74.5 for Mistral OCR 4, 72.4 for Databricks AI Parse, and 78.3 for LlamaParse’s Cost Effective offering.

This performance gap is even larger compared with hyperscaler document intelligence solutions, with an over 20-point improvement on both AWS Textract and Google Document AI. In our evaluation set, Parse is only bettered by the frontier LLMs (GPT-5.5, Opus 4.8 and Gemini 3.5 Flash) - each general purpose and significantly larger than Parse.

ParseBench scores by capability dimension. Tables tests for accurate structural extraction of data grids and cells. Content Faithfulness tests for text omissions, hallucinations, and broken reading order. Semantic Formatting 1 measures a model’s ability to capture styles that change data meaning, such as strike-throughs or italics. This evaluation did not test for Charts or Visual Grounding 2. (Zhang et al., 2026)

In terms of throughput, Cohere Parse processes 4.5 pages per second (36 pages per second or 2160 pages per minute on an 8 H100 GPU node) - approximately 1.4x the throughput of RedNote's dots.mocr and 2.2x that of Chandra OCR 2 on the same GPU configuration.

Parse is available through both the Cohere API and Model Vault, Cohere's secure, single-tenant platform for managed inference. For sustained, high-volume production workloads, we recommend Model Vault, which delivers significant cost savings as utilization grows. At 50% GPU utilization, Model Vault reduces inference costs by 23% compared with the Cohere API. At full hourly utilization, those savings can grow to 61%.

Consider a large enterprise accounts payable workflow processing approximately 13 million document pages per month. At this scale, deploying Cohere Parse through Model Vault instead of the Cohere API would reduce inference costs by approximately $12,000 per month, or $144,000 annually. Compared with a hyperscaler offering priced at $10 per 1,000 pages, annual savings would be approximately $1.47 million for this single workflow.

What you can build with Parse

Parse provides the foundation for the full document intelligence stack. Use Parse for:

Automated document processing – Extract structured data from high-volume documents such as claims, contracts, and invoices without manual review or data entry.

Semantic search and RAG – Build higher-quality retrieval systems with representations optimized for chunking, indexing, and citation.

Multimodal agents – Equip AI agents with context they need for autonomous workflows and action-taking.

The Cohere Search ecosystem

Parse is also available as part of Compass, alongside Embed and Rerank.

All our models are designed to work both independently and together. Adopt the components you need, or deploy the full managed platform for an integrated document-to-answer pipeline.

This is great for users who want:

• A single, easy-to-configure interface for document ingestion, parsing, chunking, embedding, indexing, hybrid search, and two-stage retrieval.
• Support for a broader range of document formats (including .xlsx, .docx, and .html) without building and maintaining custom preprocessing pipelines.
• Smart parsing that automatically routes documents through text or vision pathways to optimize latency and token usage.
• Fully managed indexes, including seamless embedding model upgrades without re-ingesting your data.
• Multi-tenant deployments with document-level access controls for sensitive workloads.
• Out-of-the-box connectors for your cloud storage systems such as SharePoint and Google Drive.

Getting started

Parse is now generally available via the Cohere API, Model Vault, Microsoft Foundry, and AWS SageMaker.

import base64
import os
import requests

API_KEY = os.environ["CO_API_KEY"]
IMAGE_PATH = "YOUR_IMAGE.png"

with open(IMAGE_PATH, "rb") as f:
    image = base64.b64encode(f.read()).decode("utf-8")

response = requests.post(
    "https://api.cohere.com/v2/parse",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "model": "parse-v5.0",
        "document": {
            "type": "image_url",
            "image_url": f"data:image/png;base64,{image}",
        },
        "output_format": "markdown",
    },
)

response.raise_for_status()
result = response.json()

markdown = "\n\n".join(
    page["markdown"]["content"] for page in result["pages"]
)

with open("parsed_output.md", "w", encoding="utf-8") as f:
    f.write(markdown)

print("Saved to parsed_output.md")

Resources

• Microsoft Foundry
• Amazon SageMaker
• Model documentation
• API reference
• Quickstart
• Model Vault documentation

Footnotes

1

All ParseBench scores reported here use the latest evaluation rules as of Aug 2026, which includes a fix to bold/heading detection that previously inflated Semantic Formatting scores. For a fair comparison, we re-scored all competitor models using their inference outputs against the updated rules.

2

We exclude the Layout and Chart dimensions from our ParseBench comparison because they measure capabilities outside our current product scope, not model quality deficiencies:

• Layout scores element-level spatial detection (bbox + class label matching). Our model is designed to produce reading-order markdown — it does not emit per-element bounding boxes for text, only for tables or images. Scoring low on this dimension reflects an intentional output-format choice, not a transcription failure.
• Chart scores extraction of numerical data series from chart images into structured tables. Our model treats charts as visual elements with descriptive metadata, not as data-extraction targets. This is a product scoping decision — chart data extraction is planned for the next Parser version.

On the three dimensions that align with our product goals — Tables, Text Content, and Text Formatting — we report full results. These cover the structured transcription and semantic fidelity that our model is trained and optimized for.

MiniMax-H3 on 8×H200: 1.95× Lossless, Up to 6.24× at 0.76–0.91 SSIM

TL;DR

We benchmarked MiniMax-H3 video generation on 8× NVIDIA H200 with SGLang Diffusion, holding prompts, seeds, resolution, frame rate, and denoising steps fixed across six workloads.

• SGLang's dense, lossless path is 1.85–1.95× faster than Diffusers with no approximation: the same denoising work, on a faster runtime.
• Stacking step reuse and sparse attention reaches up to 6.24×, at 0.76–0.91 mean SSIM. The fastest tested profile, SubBlock 0.80 + Cache-DiT stride, delivers 5.06×/5.72× on 5 s/10 s T2VA and 5.86×/6.24× on FL2VA. The cost is not uniform: FL2VA holds 0.85–0.91 SSIM there, while T2VA drops to 0.76–0.78.
• For a quality-first default, use Cache-DiT alone (up to 2.99×, mean SSIM 0.90–0.92). For a balanced trade-off, SubBlock 0.75 + Cache-DiT stride gives 4.90–5.93× at SSIM 0.79–0.90.
• The gains come from three layers that compose: fused kernels, step reuse (Cache-DiT skips redundant denoising steps), and SubBlock sparse attention (NVIDIA's block-sparse forward, which cuts the cost of the steps that do run).

Background

While SGLang Diffusion already delivers a fast lossless path for MiniMax-H3, faster lossy generation of high-quality video has long been sought after by the community. Video diffusion is dominated by two costs: the denoising loop runs the same transformer dozens of times, and each step spends most of its budget in attention over a very long token sequence.

Three accelerations attack that from different directions, and they compose:

• Fused kernels cut the fixed cost of every step without changing its math.
• Cache-DiT reuses results between denoising steps, so some steps never run.
• SubBlock sparse attention reduces the cost of the steps that do run, by skipping attention blocks whose contribution is below a threshold.

Detailed Results

T2VA

FL2VA

Where the Speedup Comes From

Fused kernels reduce the cost of each step that still runs. The H3 path fuses indexed AdaLN updates, gated residuals, SwiGLU activation, and QK RMSNorm with 3D RoPE.

Cache-DiT attaches one DBCache context to MiniMax-H3's shared DiT block stack. After the warmup steps, it evaluates the configured boundary blocks and compares the normalized residual change with the previous cached state. All cache modes use Fn=1, Bn=0, and four warmup steps.

SubBlock sparse attention reduces the KV blocks read on computed steps. It uses n_k=n_q=4; the first ten denoising steps use dense attention, and SubBlock is enabled afterward. The minimum sequence length is 4096.

The Kernel Layer

How SubBlock Sparse Attention Works

SubBlock is a training-free router for block-sparse attention. It divides the sequence into 64-token query and key blocks, then splits each block into four 16-token sub-blocks on both sides. A lightweight pooling and log-sum-exp score estimates each key block's unnormalized softmax mass for each query block and head. The router keeps the highest-scoring key blocks and passes their indices to the block-sparse attention kernel.

Acknowledgement

• SGLang Diffusion Team — wrote the first version of this post, drives the SGLang kernel work these results build on, and provides the diffusion runtime, the fused kernels, and the parallelism measured here.
• Ji Huang (@IPostYellow), Ant Group — ran the H200 benchmark, brought SubBlock sparse attention into SGLang Diffusion, and revised this post.
• Cache-DiT Team — @DefTruth and the vipshop.com team, for Cache-DiT and for support integrating its cache profiles into SGLang Diffusion.
• MiniMax — for open-sourcing MiniMax-H3, the model every measurement here runs on.
• NVIDIA — for the underlying SubBlock sparse attention support, including the block-sparse attention forward these results depend on.

Sopro V2 Turbo

Today we are presenting a new family of TTS models called Sopro V2, and open-sourcing our fastest one: sopro-v2-turbo. Sopro V2 Turbo is a 120M-parameter voice-cloning text-to-speech model that streams, runs comfortably on a laptop CPU or in the browser, and is multilingual: English, German, French, and, more importantly to us, it is to our knowledge the first open TTS model natively targeting European Portuguese.

It is also genuinely fast. On an Apple M3 CPU, sopro-v2-turbo generates offline at 0.24 real-time factor (RTF, generation time divided by audio duration) and streams with a time-to-first-audio of about 300 ms at 0.21 RTF. On a single H100 it reaches 0.07 RTF offline and about 200 ms to first audio when streaming. All numbers are single-stream PyTorch with the default settings, no batching.

Sopro aims to be as simple as possible to use. One command spins up a local demo:

uvx --from sopro soprotts serve

There is also a fully in-browser ONNX demo that runs without any server; note that on mobile the model is quantized, so results can be slightly below the demo above, and devices with low memory may crash.

For everything else, the repository README has the details.

We want people to communicate as fast as they used to.

A bit of backstory

Sopro started as a personal project in December 2025, built over two weeks of vacation from the startup I co-founded — Halo NeuroAI. Halo is a software and hardware company whose main goal is to give a voice back to people who lost theirs, to conditions like ALS or post-stroke aphasia. We offered (and still offer) several voice-cloning providers: OpenAI, Cartesia, ElevenLabs. The recurring problem was that European Portuguese didn't sound right on any of them, mostly due to Brazilian Portuguese data bias, and the workarounds we built to compensate increased latency substantially, especially on our best-quality provider at the time, OpenAI, where generating a sentence took 4 to 6 seconds. For us that is a deal breaker: we want people to communicate as fast as they used to. ElevenLabs has improved since, and Cartesia is the fastest of the three because streaming cuts time-to-first-audio substantially, but its European Portuguese pronunciation still isn't there. And that is before we even talk about privacy or price.

So I had an itch to explore existing models and train my own in my free time. Budget constraints more or less forced me into a very small model, which in retrospect was a good thing: it made me realize there was a gap, one that has been closing since, for good, local, fast TTS. Sopro V1 made some headlines and reached #2 on Hacker News, and a few influential people got in touch. But V1 had real problems: it was unstable, cloning quality was inconsistent across voices, and it was not built with rigorous ablations and evaluations; it was trained for $250, after all. It was also English-only, so it didn't even address the problem that started all of this. It was, however, enough to gather attention and eventually compute. We are still pre-funding, so compute is not something we could buy. Our partners at FCCN-FCT stepped in promptly, first on Portugal's supercomputer Deucalion, and later on MareNostrum 5. We are very thankful to them: they made the research and the results below possible.

The model

Sopro V2 evolved organically from the ablations we ran over time. The starting point was Sopro V1, whose architecture was close to Sesame's CSM at the time: Mimi as the codec, an autoregressive convolutional model predicting semantic codes (Mimi's codebook 0), and a non-autoregressive head predicting the remaining acoustic codebooks in stages, with cross-attention to the reference codes and FiLM-injected speaker features. Over V2's development, every one of those components was replaced or improved.

Text tokenizer

The first component we replaced was the Llama text tokenizer and its 128k-entry vocabulary. At a 384-dimensional embedding table, 128k entries are ~49M parameters, so Sopro V1 was really an ~85M-parameter model, not the headline number. For V2 we trained an 8,192-token SentencePiece unigram tokenizer: we keep some compression without paying for a huge vocabulary.

AR model

The second change was replacing the convolutional base model with a transformer decoder. The conv model was too myopic and led to repetitions, and at the sentence lengths we work with, attention is cheap with a KV cache. We kept it simple: a decoder with RoPE, RMSNorm, and grouped-query attention.

Unlike CosyVoice-style models, we don't require a transcription of the reference audio: the prompt to the semantic LM is just the reference's semantic tokens plus some style tokens. That removes an ASR dependency at inference, which matters on the edge, with no audible difference in pronunciation or similarity.

Acoustic flow-matching head

The third change was the acoustic head, where we explored several directions. If you tried Sopro V1 you may have noticed speech often came out clean but failed to carry the things that make a voice sound like itself: microphone character, room tone, and so on. Part of that (not all: recent models like Qwen3-TTS show discrete heads can work) comes from the discrete nature of acoustic codebooks. Recent work has been moving to continuous acoustic heads (F5-TTS, CosyVoice, PocketTTS), so we did too. We first tried keeping Mimi's semantic branch and swapping the acoustic branch for a continuous VAE, similar to CALM and PocketTTS. It partly worked, but it was hard to keep the latent space well-behaved enough for downstream modeling, even with KL regularization, and equally hard to guarantee a clean semantic/acoustic separation between branches. In the end we adopted the F5/CosyVoice approach and use mel spectrograms as the acoustic frontend, which is inherently well-structured. Differently from those models, we add a conditioning mask so the model knows which region is the prompt, and we mean/std-normalize the flow-matching mels.

Semantic speech tokenizer

The last change was the speech tokenizer itself. We originally kept Mimi's encoder and its WavLM-distilled semantic branch, but an intermediate model trained on it was unstable and had poor intelligibility, around 7% WER on LibriSpeech test-clean and 14% on Seed-TTS test-en. WavLM features inherently carry more than semantics and don't guarantee alignment with text. So we trained an ASR-aligned tokenizer for English, French, German, and Portuguese by warm-starting from the Whisper large-v3 encoder and inserting an FSQ bottleneck. This massively improved intelligibility. We then distilled that tokenizer into an equivalent 20M-parameter one. It runs at 23.4375 Hz instead of Mimi's 12.5 Hz, so one token corresponds to exactly 4 mel frames of our 93.75 Hz vocoder frontend and less upsampling is needed.

Vocoder

We fine-tuned a slightly deeper Vocos on our data for the offline path, and fine-tuned that further into a causal variant with 3 frames of lookahead for streaming.

Training

We trained Sopro V2 on a mix of open-source and publicly available data, Emilia YODAS and FalAR among them, with the main focus on English and European Portuguese, while keeping French and German supported. Training was split into four stages: pre-training, preference tuning, distillation, and reflow.

We first pre-trained a 0.5B base model with CFG-aware dropout; at inference the base model runs with CFG 3.0 and 32 acoustic solver steps. It trained for 400k steps at an effective batch size of 72 across 4 H100s. To support both offline and streaming generation, half the samples trained with full acoustic attention context and the other half sampled a future chunk size uniformly from 32, 64, 128, and 256 frames. That buys versatility at inference: batch offline generation when a GPU is available, or streaming with a smaller chunk for lower time-to-first-audio (at some real-time-factor cost) and a larger chunk for the reverse trade. Training segments are capped at 30 seconds, but generation length is not: longer inputs are split into segments, and each segment is generated with the previous generation as context to the semantic LM, so the model can speak indefinitely.

We then experimented with GRPO using WER, similarity, and duration rewards, but found it destabilized the model. DPO over the same kinds of preference pairs worked much better: it improved the failure tail while keeping the base model stable. We ran three rounds of it on the teacher.

Next we distilled the 0.5B teacher into a 120M model, mixing the ground-truth data the teacher saw with curated teacher rollouts. The student ended up even more stable than the teacher, trailing only slightly in similarity.

The final stage cut solver steps from 32 to 2, a 16× speedup of the acoustic head, via self-distillation with reflow, reaching near-parity with the 32-step model with no measurable loss in quality, similarity, or intelligibility. That model is sopro-v2-turbo.

Evaluation

We evaluate on three benchmarks: Seed-TTS-eval test-en, LibriSpeech test-clean under the F5-TTS protocol, and the MiniMax multilingual test set. Throughout: WER is computed with Whisper large-v3 and similarity with a WavLM speaker-verification model, following each benchmark's official harness. Baseline numbers come from the papers cited in each table, and bold marks the best value in each column (reference rows excluded).

Seed-TTS test-en

On Seed-TTS test-en, Sopro V2 Turbo reaches SOTA-level intelligibility and competitive similarity against models 3-14× larger, some running 16× more solver steps. We could probably squeeze further with more post-training, but as the table shows we are already below ground-truth WER and close to resynthesized ground-truth similarity; past this point we would mostly be feeding Goodhart's law. We use the official evaluation harness, which applies only light text normalization: if Whisper transcribes "fifty" as "50", WER is taxed aggressively, and optimizing that away means optimizing the model's enunciation for Whisper rather than for people.

LibriSpeech test-clean (F5 protocol)

We use the F5-TTS cross-sentence protocol (1,127 samples from LibriSpeech-PC test-clean). Baseline rows come from the F5-TTS paper.

MiniMax multilingual eval

On the MiniMax-Speech multilingual test set, Sopro V2 Turbo holds up well against closed systems across English, French, German, and Portuguese. On Portuguese the comparison is not entirely fair to us: the reference speakers in the benchmark's Portuguese portion are Brazilian, while Sopro's Portuguese targets the European variant. European pronunciation itself also inflates WER: unstressed vowels get reduced, so words like "telefone", "esperança", or "desenvolvimento" come out clipped compared to their Brazilian reading, and the ASR is more likely to mis-transcribe them even when the European pronunciation is correct.

European Portuguese

European Portuguese is the reason Sopro exists, so here are a few voices from our internal EP test set, built from European Portuguese Common Voice speakers.

Limitations and disclaimers

We did not add watermarking: with an open-source inference pipeline it would be trivial to remove, so it would only provide a false sense of safety. Please use the model for good: do not impersonate people.

We deliberately keep the text frontend minimal, so some abbreviations, numbers, and symbols may not be pronounced correctly. Prefer words: 1 + 2 should be written one plus two. That said, Sopro generally reads common abbreviations like "CPU" or "TTS" fine, and you can put a language-specific normalizer in front of it.

Mixed-language text is another weak spot: words from one language inside a sentence of another (an English product name in a Portuguese sentence, for example) can be mispronounced.

We are not planning to release the training code in the near future due to its complexity.

Compute for this project was funded by FCCN-FCT and Fábrica de IA / Barcelona Supercomputing Center, under project reference eporaif07.

Conclusion

Once again, thank you to FCCN-FCT for the compute, and to the open-source community whose work was a constant inspiration for Sopro V2.

Sopro V2 delivers SOTA-level text-to-speech that is local and fast, and it is the foundation for the models coming next from Halo Research. We are excited to add more languages, emotion control, and to keep improving from here.

Gemini Omni 1.1 Flash lets you build with more control

Omni now delivers studio-quality video production, including the ability to extend a scene, first and last frame interpolation, crisp 4K upscaling, faster prototyping, and more.

Today, we’re introducing Gemini Omni 1.1 Flash, a new suite of creative controls and generative video capabilities to support developers. Gemini Omni brought real-world reasoning to generative creation, and today’s updates make Omni 1.1 production-ready for professional use via the Gemini API in Google AI Studio.

Whether you’re building generative video workflows, creative tools, or media editing software, these updates make generative video more controllable, faster to iterate on, and polished for real-world deployment. Here’s a look at what’s new:

Extend scenes for longer storytelling

Scene extension allows you to take an existing video and continue generating footage seamlessly from where it left off.

With Omni 1.1, the model can now analyze up to 10 seconds of prior context — a leap from previous models that only referenced the final second. The result is improved visual consistency and narrative adherence, letting you build longer stories or branch into new creative directions. You can extend videos in 10-second increments up to a total cumulative length of 40 seconds.

from google import genai

client = genai.Client()

interaction = client.interactions.create(
    model="gemini-omni-1.1-flash",
    previous_interaction_id=previous_video_interaction.id,
    input=[
        {"type": "text", "text": "Continue the scene."}
    ],
    response_format={
        "resolution": "360p",
    },
)

Specify first and last frames

Achieve smooth transitions and camera movements by specifying the starting and ending frames of a shot. Omni 1.1 generates continuous video between two keyframes, making it ideal for complex camera orbits, zoom transitions, or seamless looping clips.

Draft videos more efficiently in 360p

Generate lightweight previews in 360p resolution up to 60% faster and at a third of the cost compared to Omni 1.1’s standard 720p resolution. This is helpful for rapid prototyping, storyboard iteration, and quick rendering in developer platforms.

Upscale up to 4K resolution

Generate polished, high-resolution 1080p or 4K outputs that are ready for professional production with Omni 1.1.

Add video references in your multimodal input

Reference up to three seconds of video when crafting your scene, allowing you to maintain visual context and character consistency based on video references.

Inspiring concepts for what you can build

Here are a few ideas showing how developers can put these new capabilities into action across custom tools and creative workflows.

See how customers are putting Omni Flash in production

Our customers are already driving real-world production with Gemini Omni Flash via the Agent Platform API.

“Gemini Omni Flash is one of the strongest video models available in Figma Weave, where the canvas helps creative teams build on every generation — attaching references, branching different versions, and shaping something unique. With extensions, richer reference material, and 4K resolution, Gemini Omni Flash takes teams beyond generating videos to truly directing them.” - Itay Schiff, Creative Director, Figma Weave.

“At GMI Cloud, we give creators centralized access to the world’s most capable models. What stands out about Gemini Omni Flash is its accuracy: the details hold up under scrutiny. For customers creating educational and explanatory content, where getting things right is essential, that reliability matters more than any single feature. Omni has made AI video viable for a segment that previously couldn’t rely on it.” - Louisa Guo, VP of Marketing, GMI Cloud.

“Omni Flash fits naturally into how people already use Runway: start with a prompt, an image or a video, then generate or edit from there. It's another way for our users to move quickly between ideas.” – Jamie Umpherson, Chief Creative Officer, Runway.

Build with Gemini Omni 1.1 Flash Today

Omni 1.1 is rolling out across the Google developer ecosystem:

• Start building in Google AI Studio: Try out Omni 1.1 directly in Google AI Studio.
• Build on Gemini Enterprise Agent Platform: Enterprises can build with Omni 1.1 directly via Agent Platform API.
• Explore the developer documentation: Check out the official documentation, the cookbook and prompting guides to learn how to integrate scene extensions, video references, and upscaling into your applications.

Omni 1.1 is also available to all Google AI Plus, Pro and Ultra subscribers globally in Google Flow, starting today. Scene extension is available to all Google AI Plus, Pro and Ultra subscribers globally in the Gemini app.

Terminal-Bench-Science 0.1

Terminal-Bench-Science evaluates AI agents on workflows from researchers' own work. Scientists, not model developers or data vendors, set the bar for scientific capability in AI.

Terminal-Bench-Science is a benchmark led by researchers at Stanford University and built by the team behind Terminal-Bench in collaboration with domain experts from a range of scientific disciplines and research institutions around the world. It measures the AI agent capabilities through a diverse set of challenging, expert-curated workflows drawn from scientific research.

Terminal-Bench-Science is a continuous benchmark that evolves alongside frontier AI, creating a feedback loop between scientific needs and AI development. Our first release includes 70 tasks from the life, physical, Earth, mathematical, and engineering sciences. The strongest model evaluated, Claude Opus 5, achieves a 30% resolution rate on Terminal-Bench-Science 0.1.

Overview

While Terminal-Bench has driven progress in AI agents for software engineering, Terminal-Bench-Science brings the same ambition to science. Our goal is to drive the development of agents with scientific capabilities that make them useful research assistants. These agents should execute technically demanding and time-consuming workflows, freeing scientists to focus more of their time on the parts of science where human judgment matters most: defining research questions, forming hypotheses, interpreting and validating results, and communicating findings. In this role, AI agents can extend what researchers accomplish and help accelerate scientific discovery.

Achieving this requires benchmarks that reflect real scientific practice, provide verifiable evidence of capability, and evolve alongside the AI frontier.

We need benchmarks drawn from real scientific workflows. Scientific capability should be evaluated on real research practice, not textbook questions or standardized exercises, contributed by practicing scientists themselves. Terminal-Bench-Science gives scientists across domains a direct voice and a shared platform to set the bar for AI progress on the problems they care about. The stakes in science are too high, and its benchmarks must reflect the scientific community's priorities rather than outside interests.

We need verifiable evidence of scientific capability. Without reliable evaluation, we cannot tell whether agent capabilities are improving or where their limitations remain. Terminal-Bench-Science evaluates agents in realistic environments and grades concrete artifacts such as analyses, simulations, proofs, code, and data products with reproducible, task-specific tests.

We need a benchmark that keeps pace with the frontier. Too often, scientific benchmarks are treated as papers to publish rather than mechanisms for driving progress. They are released once and then abandoned as models advance and known limitations persist. Terminal-Bench-Science is a continuous benchmark that evolves alongside the AI frontier. Through regular releases, scientists can contribute new workflows, improve existing tasks, and create a feedback loop between scientific needs and AI development.

Tasks

Terminal-Bench-Science 0.1 includes 70 tasks across the life, physical, Earth, mathematical, and engineering sciences. Tasks span scientific data analysis, statistical inference, simulation, optimization, theorem proving, image reconstruction, signal processing, inverse problems, sensor calibration, model fitting, classification, and scientific machine learning.

Tasks are contributed by researchers through an open process on GitHub, with discussion and feedback in the #tb-science channel on Discord. Contributions begin as proposals, where reviewers discuss each idea, leave feedback, and approve those that look like a strong fit: scientifically grounded workflows worth measuring in the benchmark. Approved proposals are implemented as pull requests, where reviewers confirm that each task is objectively verifiable, genuinely challenging for AI agents, and not something today's frontier systems already solve easily. To merge, domain reviewers assess scientific validity and realism, technical reviewers inspect task construction and verification, and a bar raiser performs a final quality check. Of 920 proposals, 464 were approved for implementation and 386 pull requests were opened, but only 70 tasks made it into Terminal-Bench-Science 0.1. That selectivity reflects how difficult it is to create tasks that are scientifically interesting, challenging for frontier agents, and sufficiently well specified for rigorous evaluation. Progress across proposals, pull requests, and reviews is tracked on the public task dashboard.

Results

Terminal-Bench-Science 0.1 leaves substantial room for progress on AI agents for scientific research. Each evaluated model ran three independent trials per task across all 70 tasks. Claude Opus 5 with Claude Code achieves the highest resolution rate at 30.0%, followed by GPT-5.6 Sol with Codex at 22.4% and Claude Fable 5 with Claude Code at 21.4%. Claude Opus 4.8 sits in the middle at 10.5%. GPT-5.6 Terra, Kimi K3, and Grok 4.6 all resolve less than 10% of tasks. GLM 5.3 is the strongest open model at 8.1%, and GPT-5.6 Luna is last at 3.3%.

Terminal-Bench-Science distinguishes between systems about as well as Terminal-Bench 3.0 while pushing resolution rates down by more than 10 percentage points for every model evaluated on both. That gap is deliberate: during review, tasks were calibrated to challenge the newest frontier models.

Performance is only one dimension of progress. The cost-resolution plot shows total evaluation cost across all 70 tasks against resolution rate. GPT-5.6 Luna, Kimi K3, and GPT-5.6 Terra occupy the low-cost end of the frontier. GPT-5.6 Sol and Claude Opus 5 reach the highest resolution rates at greater cost, with Opus 5 at $7.0k. GPT-5.6 Sol matches Claude Fable 5's performance at less than a third of the cost ($4.2k vs $14.2k). Token usage shows a different frontier. Claude Fable 5 matches GPT-5.6 Sol's performance while using about a quarter fewer tokens (6.4B vs 8.4B). Kimi K3 anchors the low-token end and Claude Opus 5 the high-resolution end. Only Kimi K3 and Claude Opus 5 appear on both Pareto frontiers.

Resolution rates also vary by scientific domain. Anthropic and OpenAI models take the top two spots in every domain except the engineering sciences, where Grok 4.6 ties GPT-5.6 Sol for second place (14.8%) at lower cost and token usage. Claude Opus 5 leads both GPT-5.6 Sol and Claude Fable 5 in every domain except the mathematical sciences, where Claude Fable 5 (33.3%) and GPT-5.6 Sol (31.4%) take the top two spots.

Conclusion and Roadmap

Terminal-Bench-Science 0.1 is a community effort by researchers across the life, physical, Earth, mathematical, and engineering sciences, together with the Terminal-Bench and Harbor team. It is the most rigorous benchmark of scientific agent capabilities we could build in the open, and we are only getting started: Terminal-Bench-Science 0.1 is the first release of a continuous benchmark.

Regular releases will add tasks, broaden coverage across the five scientific domains, retire tasks that agents saturate or that review reveals to be underspecified, and keep the leaderboard current as new frontier models are released. Each release is calibrated against the frontier at the time. For Terminal-Bench-Science 0.1, this was Claude Opus 5 and GPT-5.6 Sol, and as stronger models emerge we will use them to evaluate and calibrate new and improved tasks. Tasks are versioned so that trials can be re-used, re-graded, or re-run with a single Harbor command, which keeps the cost of updating results low. Progress is tracked in the open on the task dashboard, and every release is tagged on GitHub and Harbor Hub.

Work on Terminal-Bench-Science 0.2 is already underway, with a pull request deadline of October 5, 2026. If you are a researcher with a workflow that frontier agents should be able to do but cannot yet, we want it in the benchmark. The contribution flow is Propose → Build → Review: propose your task through the task proposal form, build it following the contributing guide, and it will go through automated checks, parallel domain and technical review, and final bar-raiser approval before merge.

Join the effort in #tb-science on Discord and on GitHub, and drop into our weekly meetings and office hours via the project calendar. Let's let scientists define what scientific capability in AI looks like, and measure it rigorously, together.

Citation

If you find this work useful, please cite it. You can use the "Cite this repository" button on GitHub or cite manually using the information below.

@software{Terminal-Bench-Science_Team_Terminal-Bench-Science_Evaluating_AI_2026,
  author = {{Terminal-Bench-Science Team}},
  doi = {10.5281/zenodo.22110254},
  license = {Apache-2.0},
  month = aug,
  title = {{Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains}},
  url = {https://github.com/harbor-framework/terminal-bench-science},
  version = {v0.1.0},
  year = {2026}
}

Acknowledgements

Thank you to all of the task contributors, reviewers, and advisors behind Terminal-Bench-Science.

Special thanks to our project lead advisors Ludwig Schmidt and Sanmi Koyejo; our senior reviewers Allen Hart, Ivan Bercovich, Joseph Janssen, Jiaming Hu, Steffen Bollmann, and Sergey Aganezov; our AI research advisors Ryan Marten, Alex Shaw, Lin Shi, Benjamin Feuer, Mike A. Merrill, Alex Dimakis, Jenia Jitsev, Bodhisattwa Majumder, Peter Clark, Thomas Wolf, Braden Hancock, and Andy Konwinski; and our scientific advisors Sara Beery, Jo Dunkley, J. Nathan Kutz, Ching-Yao Lai, Scott Linderman, Emma Lundberg, Russ Poldrack, Aviv Regev, and Risa Wechsler.

Terminal-Bench-Science is an open academic collaboration hosted by Stanford University and the Laude Institute, in partnership with the Stanford AI Lab (SAIL), the Stanford Institute for Human-Centered Artificial Intelligence (HAI), Stanford AI Measurement Science (AIMS), the NSF AI Institute for Foundations of Machine Learning (IFML), the Allen Institute, and the Allen Institute for AI (Ai2). As part of the Terminal-Bench franchise, it is built by the Terminal-Bench and Harbor team together with a community of scientific contributors.

We thank the Laude Institute for support through the Slingshots program, Snorkel AI for support through the Open Benchmarks Grants program, the 2077AI Open Source Foundation for PP API credits supporting task review and curation, and UniPat AI and Modal for their support of Terminal-Bench-Science. We thank Bespoke Labs, Anthropic, Google, Moonshot AI, SpaceXAI, and Z.ai for API credits supporting leaderboard evaluations.

Written by: Steven Dillmann

Piloting the world's first double-blind AI evaluations

Building trust in proprietary model benchmarks using cryptographically secure environments

Imagine a student is set to take a high-stakes exam. If they accidentally peek at the test questions in advance, achieving a perfect score is influenced by this knowledge, making it a meaningless accomplishment. To truly measure what they know, they must have no visibility of the test questions until it's time to take the exam. That is the exact challenge the industry faces when evaluating advanced AI models. If a model has already seen the test questions - a problem known as benchmark contamination - the results can only be trusted to an extent.

Today, we’re introducing the world’s first double-blind evaluation of a proprietary, frontier class AI model, which keeps external evaluations confined to a cryptographic “box” where they can’t be used by models later to optimize performance ahead of testing. We're partnering with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons, to test a Gemini Flash Lite model against confidential benchmarks in a privacy-preserving environment, increasing evaluation integrity.

At Google, we assess our AI systems using a broad spectrum of evaluations throughout model development and deployment, but we don’t rely on internal testing alone. To identify potential blindspots, we work with a diverse group of external partners, including specialized research labs, civil society and national AI Safety and Security Institutes (AISIs), using their unique expertise to stress-test our models.

As AI models become more capable, ensuring the model has not seen the test questions or prompts in advance is critical, as this can skew the results. Policymakers, researchers, and enterprises need to trust that AI benchmarks accurately reflect a model's true capabilities and safety, but if models are able to “peek” at the evaluation questions in advance, it can artificially inflate scores and undermine this trust.

Although zero-logging protocols and rigorous contractual safeguards have long kept external test prompts confidential, incorporating technical and cryptographic safeguards marks a major step forward in secure model evaluation.

How double-blind evaluations work

Historically, high-stakes external evaluations required a tradeoff. Either evaluators handed over their testing prompts (risking the model provider seeing the test questions in advance), or the model provider handed over their model weights (risking their intellectual property).

Double-blind evaluations eliminate this compromise. By using Confidential Space within Google Cloud’s Confidential Computing portfolio, we can cryptographically verify that both the external evaluation data and the proprietary model remain private to their respective owners. The evaluator cannot see the Gemini model weights, and Google cannot see the evaluator’s test prompts.

A novel approach to building trust in model evaluations

This cryptographic evidence helps prevent benchmark contamination and protects sensitive data. As models become more capable this becomes particularly important for highly sensitive evaluations, such as those used for cybersecurity or by government bodies. Double-blind evaluations unlock the ability for independent organizations to rigorously test advanced models without compromising data sovereignty or security.

We hope this pilot establishes a new frontier for model oversight, helping the broader industry build safer, more reliable, and widely trusted AI systems. To learn more about our methodology and findings, read our technical report.

For all the interest in and uptake of agentic AI systems over the past year or so, the world of automated AI has thus far been primarily limited to text, images, code, and other data and actions that take place inside a computer. Anthropic is now aiming to change that somewhat with what it’s calling the Model Hardware Standard (MHS), a set of standardized drivers designed to let AI agents easily interface with and control arbitrary devices.

For now, the “research preview” of the MHS effort is being sold mainly as a way to help scientists streamline the arduous process of creating the custom software integrations that are often needed to get disparate components of an experiment working in concert. MHS can provide a common interface and common format for data sharing between these devices, Anthropic says, allowing them to talk to each other across a network “without needing a bespoke ‘translator’ program in between.” The standardized system could reduce weeks or months of exacting experimental setup down to “hours or minutes,” Anthropic writes.

In a video posted alongside the announcement, Anthropic Technical Staffer Alek Kemeny says the MHS effort was inspired by observing neuroscientist Arco Bast work through an experiment on memory formation in the brain at the HHMI Janelia Research Campus in Ashburn, Virginia. Kemeny said Bast had worked out an interface to get the rotating laser beams, microscopes, cameras, and myriad other components of the experiment to coordinate through a common interface. “This idea could be used to have AI run any science experiment in the world,” Kemeny recalls thinking at the time.

Your new robotic lab assistant?

There’s nothing about a common machine interface language that requires the use of AI models, of course. And Anthropic says MHS devices can be controlled directly in real time via command-line prompts and API code files. But integrating an MHS system with an AI model through the Model Context Protocol lets scientists interact with devices using natural language and lets models “reason through each step in an experiment, update parameters in real time, and, in some cases, recover from hardware errors without intervention,” Anthropic writes.

Anthropic gave the example of a model like Claude adjusting a laser, checking the results via a separate camera, then repeating the process to automatically calibrate the whole system. MHS could also allow an AI model to focus a microscope, analyze the results, decide what part needs more observation, then automatically move the microscope to the relevant section to continue the experiment.

In a video, Anthropic also showed Claude reasoning how to get a robotic arm to pick up an aluminum can even though it had not been specifically trained on the required steps. And rather than reasoning through each step each time, Anthropic says MHS-enabled models can sequence steps across instruments by writing API scripts and adjusting them as conditions require.

Anthropic says MHS also includes a standardized tagging system to describe hardware’s real-world constraints for models that may have been trained more in the virtual world. That includes encoded information about the hardware’s physical characteristics (e.g., the weight and range of a robot arm) as well as its adjustable parameters, measurement options, and enforced safety limits. These tags can then be integrated into a reference file that can quickly provide an AI model with crucial information about a device it has no previous training experience with.

For now, Anthropic says it is working with “a first group of scientific research labs and advanced manufacturers” during an MHS preview period, including Amazon Web Services (Strands Robots), Hugging Face (LeRobot), Raspberry Pi, Automata, and Universal Robots. These partners will help Anthropic “build safety evaluations and develop best practices for AI systems operating physical equipment,” the company writes. After that, the plan is for MHS to eventually become an open source and “agent agnostic” standard for integrating AI and physical systems.

In early testing with scientific partners over the past year, Anthropic says it “saw MHS reduce the time it took to integrate devices, mak[ing] it possible to iterate faster in a variety of experimental settings.”

“If you can test hypotheses faster, you could create general technologies faster,” Kemeny said in a promo video alongside the announcement. “This is how a century of progress can condense into a decade.”

OpenAI put an impressive-sounding case study about how they helped Asana save $5.9M with a single migration.

“Asana cleared 5 years of engineering work in 2 weeks with Codex. Using OpenAI Codex, Asana replaced an outdated testing system in two weeks for about $12K.
For this project, Codex helped Asana’s engineers remove Enzyme, an outdated testing system that had made the company’s code harder to upgrade. Model and infrastructure costs came to about $12,000, compared with Asana’s roughly $6 million estimate for the previous staffing plan.
After 1.5 weeks of engineering effort spread across two calendar weeks, Enzyme was fully removed. Model and infrastructure costs totaled about $12K. For comparison: the previous plan was expected to take at least five years and estimated to cost roughly $6M. The experience changed which long-running software projects the company believes are practical to take on.”

For context, Asana migrated from Enzyme to React Testing Library, which indeed would not be a simple migration. However, their estimate of four engineers (each on circa $300K/year, according to OpenAI’s arithmetic), spending five years on the project, had me like: But thinking about this for longer raised the question: what does a migration from Enzyme to React Testing Library even look like?

Enzyme to React Testing Library migration

Let’s take a simple test and see how it looks in various testing libraries. For our test, we want to verify that a button increments a counter. Here’s our button in React:

Now, the test to verify this, in Enzyme:

And let’s rewrite this test in React Testing Library:

The two tests do the same, but they have completely different syntax! The main reason for this difference is that the two frameworks use a fundamentally different approach to testing. Whereas Enzyme is oriented towards component testing (notice how the test operates on a component instance), the React Testing Library operates on the rendered Document Object Model (a data structure representing the HTML shipped to the client) so the test sees the whole rendered page, not just the component its written for. That’s why the testing approach will differ radically between the two frameworks, especially when testing complex user journeys.

It took Airbnb 6 weeks to migrate 3,500 tests with AI

Last year, Airbnb revealed how they migrated their Enzyme test suite of 3,500 component test files within six weeks with LLMs. The estimate of doing this by hand was 1.5 engineering years. Airbnb did the LLM-aided migration in a multi-phase process.

Airbnb’s team had to build loops to keep retrying migrations; once they did, 75% of files were migrated in just four hours, and the migrations were straightforward. They then built a more sophisticated refactor pipeline for the remaining 25% of tests; after building the pipeline, the new loop migrated most of the remaining tests (97%) in total, after running over 4 days. The remaining 3% was done with LLM input, with engineers finishing it in a week.

AI makes impractical migrations doable

On the basis that it took Airbnb six weeks, I find it credible that it took Asana two weeks to migrate what is probably a similarly complex test infrastructure from Enzyme to RTL, a year later.

The time and $6M cost as quoted by OpenAI feels inflated. I assume the numbers were based on an estimate that a fulltime engineer could do a maximum of X tests migrated per day, where X was between 5 and 10. Then, calculate the number of engineering years this takes (perhaps 20 engineering years), and multiply by the cost of an engineer. You estimate a project like this when it’s an undesirable project you really don’t want to do as an engineer!

So, looking at it from this point of view: does it matter if the estimate was 1.5 years (Airbnb) or half a decade (Asana, hypothetically)? Or if the estimated cost was $1M or $5M? It would still be an impractical migration and a foolish endeavor: too long and distracting, at least, in the “traditional” way!

Pre-AI, years-long migrations were rare. In 2021, Sentry took 1.5 years to convert their frontend codebase from JavaScript to TypeScript. That was a migration of 1,100 files and 95,000 lines of code! Around 10 engineers worked on the migration, so if we assume a $300K-per-engineer cost, that’s a $2–4M cost for circa 95,000 lines of code.

Indeed, Asana’s reported $12,000 migration cost could actually cost even less if there was ruthless focus on cost optimization. What about using a model that’s 10x cheaper than OpenAI, like an open model running on inference providers? After all, why use the most expensive model, especially if a company already owns GPUs, running models, making inference practically free (except for power costs, that is.)

A few more details from inside Asana

I managed to catch up with Dan Ubilla at Asana, who leads the Developer Productivity group at the company. He helpfully clarified a few things about the Enzyme migration post on the Asana site:

• Asana started the Enzyme migration in 2024. The team had 4,000+ Enzyme files to migrate, and they took a first stab at the migration with LLMs, at the time. Beyond migrating existing tests, much of this phase focused on getting the codebase into a state where future tests could be written more idiomatically.
• The remainder of the migration became low priority. Asana prioritized migrations as critical (“must do soon”), important (“important to do soon”), and opportunistic (“nice to have”). The company has a good number of critical and important migrations, and the remainder of the Enzyme migration sat as one of the many opportunistic.
• Five years was about when the migration was projected to finish. The effort was not about “a team working for five years, nonstop on the project” – not at all! It was about the timeframe that the opportunistic Enzyme migration would have been completed, given its priority, and done, realistically. Given this migration was going on for two years, the five-year estimate was a rational one.
• The team wanted to prove that LLMs drastically speed up migrations, and demonstrated it with an opportunistic, low-priority one. The Enzyme migration was not all that important to do quickly, and would have sat around for years. So it was the perfect candidate to prove that with LLMs, long migrations can be sped up!

And finally, one addition from my end:

The $6M cost was a back-of-the envelope estimation. I asked Dan how this estimate of the migration costing $6M came together. Dan confirmed that the estimation was done the same way as most of us do these estimations:

• Estimate how long it takes an engineer to manually migrate a single Enzyme file (including rewrite + validation). Be generous with this estimation
• Multiply it by the outstanding files to migrate (the 3,000+ ones)
• Add in the time to remove any traces of the framework, monitoring, static analysis tools
• Multiply by the hourly rate of a typical engineer
• … and the number came out to $6M

Expect long-avoided migrations to finally happen

A few months ago, Uber shared that they executed a massive JUnit migration in four months with two engineers and AI: moving 600,000 unit tests spanning 15 million lines of code by moving from the unsupported JUnit 4 to JUnit 5. This type of migration used to be impractical; with AI tools, it took eight engineering months of effort, plus AI costs.

The shared characteristic of all of the above migrations is that engineers needed to plan for it, design verification loops, and be involved throughout. I always dreaded migrations, so I see it as good news that we have a new tool to use for “grunt work” like this.

Chroma's Foundation is a memory layer that operates through a swarm of agents modifying shared state, ingesting coding agent traces and company data to build a durable record and index.

Harness Engineering

Harness engineering is the practice of surrounding AI-assisted code generation with deterministic tooling, agent-based review, and periodic entropy checks so that AI-generated code stays correct and coherent over time. This document explains where the idea came from, what it consists of, and how this plugin implements it.

The Origin

The term comes from Birgitta Boeckeler's article on martinfowler.com, written in the ThoughtWorks context of teams shipping real software with AI coding assistants. Boeckeler observed something that many teams had noticed independently: AI assistants produce plausible-looking code, but left unconstrained they drift. They forget conventions, repeat mistakes, and slowly erode the internal consistency of a codebase. The code continues to compile and pass tests. The degradation is quiet.

Boeckeler's insight was that this problem already has a solved analogue in software engineering: the test harness. Tests do not make code correct by construction. They detect when code stops being correct. A test harness is not a constraint on what code you write; it is a mechanism that continuously checks whether what you wrote meets a standard. The harness does not trust the programmer. It verifies.

The same logic applies to AI-assisted development, with one crucial difference. Test harnesses check functional correctness: does the program do what it is supposed to do? A harness for AI coding needs to check something broader: does the codebase still embody the architectural decisions, naming conventions, security constraints, and structural rules that the team has agreed on? Functional tests are necessary but not sufficient for this. You need a different kind of harness.

That is what harness engineering provides.

The Three Components

Boeckeler describes three categories of concern that a harness must address.

Context Engineering

An AI coding assistant can only work within what it knows. If it does not know that your project uses a particular logging library, it will invent its own approach. If it does not know that you never use mutable global state, it will use it when convenient. If it does not know that all database writes must go through a specific abstraction layer, it will bypass that layer.

Context engineering is the discipline of making sure the AI knows what it needs to know. In practice, this means maintaining a document — HARNESS.md in this plugin's conventions — that captures the stack, the architectural decisions, the naming conventions, the constraints, and the rationale behind each of them. This document is not a README for humans. It is a knowledge base for the AI. It needs to be accurate, specific, and kept current.

The distinction matters: a README explains what the project does. A context document tells an AI agent what it must and must not do, and why. These are different documents with different audiences and different update rhythms.

Architectural Constraints

Knowing the rules and enforcing the rules are separate problems. You can write every constraint into HARNESS.md and the AI will still violate them, because the AI is a probabilistic system optimising for plausibility, not a rule-following machine. Context engineering reduces violations. It does not eliminate them.

Architectural constraints are the mechanisms that catch violations. Boeckeler calls the enforcement points "verification slots" — defined moments in the development workflow where a check runs and either passes or blocks progress. The key design decision for each verification slot is whether it uses a deterministic tool or an agent-based review.

A deterministic tool is a linter, a script, a regex check, a file-structure assertion — anything that produces a pass/fail result without judgment. These are preferable when the constraint can be expressed precisely. They are fast, cheap, and completely reliable within their specification.

An agent-based review is a language model looking at code against a constraint description and making a judgment. This is necessary when the constraint involves intent, semantics, or patterns that are difficult to express as a mechanical rule. Agents are more expensive and less deterministic, but they can catch things that no script can catch.

Both types of verification slots belong in a harness. The goal over time is to migrate constraints from agent-based to deterministic as your understanding of the constraint sharpens enough to specify it precisely. This is the progressive hardening principle, described below.

Garbage Collection

A codebase is a living system. Even with good context engineering and strict architectural constraints, entropy accumulates. Dead code grows. TODO comments persist for months. Dependencies go stale. Abstractions that made sense at one stage of the project become obstacles at a later stage. Conventions established early get quietly abandoned when they become inconvenient.

Garbage collection is the periodic process of fighting this entropy. Unlike the other two components, which operate at the moment of code generation or review, GC operates on a schedule. It is not triggered by a specific coding event. It runs because time has passed.

In a harness engineering framework, GC rules are explicit declarations of what "clean" looks like, paired with scheduled agents or scripts that check whether the codebase still meets those standards. The output is not a list of errors to block a PR; it is a report that draws attention to accumulating problems before they become serious.

The Living Harness

The most important property of a well-maintained harness is that it is not static. A harness that was written once and never updated reflects the understanding of the team at one point in time. The codebase continues to evolve. New patterns emerge. Old constraints become irrelevant. New categories of AI-generated mistake appear that the original authors did not anticipate.

HARNESS.md is designed as a self-referential document. It does not only describe what constraints are in force; it tracks the status of each constraint: whether it is currently unverified, under agent review, or enforced deterministically. The document declares what should be true. Agents, hooks, and CI checks verify whether it is true. The harness auditor — a scheduled agent in this plugin — reads the results of those checks and updates the status entries in HARNESS.md to reflect reality.

This creates a feedback loop. The document is both a specification and a health record. Reading HARNESS.md at any point in time tells you not just what the team has agreed should be true about the codebase, but how well those agreements are actually being maintained.

The self-referential property is what distinguishes a living harness from a document that gets outdated and ignored. Because the harness itself is a target of enforcement — the harness-audit agent checks whether HARNESS.md accurately reflects the current state of verification — neglecting the harness becomes visible rather than invisible. The everyday entry to this self-check is /harness-sync, which runs the audit's detection logic and presents a unified drift table; users see misalignment between the declared harness and reality without having to remember to invoke a separate diagnostic.

Progressive Hardening

Not all constraints are equal, and not all constraints are ready to be enforced deterministically from the start. Progressive hardening is the promotion ladder that describes how constraints mature.

The ladder is one axis. Reach — whether a constraint is required on every PR or complete-if-present — is a second, and the Enforcement field does not record it.

Unverified is the starting state. You have declared a constraint in HARNESS.md. You believe it is important. You do not yet have a mechanism to check it. This state is not a failure; it is honest accounting. An unverified constraint is a commitment to build enforcement, not a claim that enforcement already exists.

Agent is the second state. You have written an agent prompt that checks the constraint as part of PR review or a scheduled inspection. The constraint is being enforced, but by a language model making a judgment, not by a deterministic rule. Agent enforcement catches most violations most of the time. It is not perfectly reliable, and it requires human review of the agent's output.

Deterministic is the final state. You have expressed the constraint precisely enough to encode it as a script, a linter rule, or a structural check. It runs in CI. It either passes or it blocks the merge. There is no judgment involved, and no possibility of the check being confused or misled.

The direction of movement is always toward deterministic. When an agent repeatedly catches the same class of violation, that repetition is a signal: the pattern is now understood well enough to automate. Write the script, retire the agent check for that specific constraint, and move the entry in HARNESS.md to deterministic status.

Progressive hardening is important because it prevents two failure modes. The first failure mode is trying to enforce everything deterministically from the start, which is impossible for novel or semantically complex constraints. The second failure mode is accepting agent-based enforcement as a permanent state, which is expensive and unreliable. The ladder gives you a path between them.

How This Plugin Implements It

This plugin structures the verification slots into three enforcement loops, operating at different timescales and with different tolerances for false positives.

The inner loop is advisory and runs at edit time. When you save a file or complete a coding session, lightweight checks run and surface potential issues as suggestions rather than blocks. The inner loop is optimised for low friction. It should not interrupt flow. Its job is to make problems visible early, not to stop work.

The middle loop is strict and runs at PR time. When you open a pull request, a full suite of agent-based and deterministic checks runs. This loop has the authority to block a merge. It is the main enforcement point for architectural constraints. Failures here must be addressed before the code lands.

The outer loop is investigative and runs on a schedule. Garbage collection rules, fitness functions, and harness audits run periodically — daily, weekly, or on whatever cadence makes sense for the rule. The outer loop produces reports rather than blocks. Its findings feed back into the harness as potential new constraints or updates to existing ones.

The three loops correspond roughly to the three components: inner loop serves context engineering (keeping the AI informed in the moment), middle loop serves architectural constraints (enforcing agreed standards at integration time), outer loop serves garbage collection (detecting slow entropy between integration events).

Agents in this plugin operate with bounded trust. No agent has unilateral authority to modify production code or merge changes. Agents review, suggest, report, and flag. Humans decide. This is a deliberate design choice: the harness amplifies human judgment; it does not replace it.

The Self-Improving Dimension

The original Boeckeler framework describes a harness as something a team builds and maintains. This plugin adds a layer: the harness can learn from its own operation.

After each coding session, the /reflect command captures what went well, what failed, what conventions were violated, and what new patterns emerged. These reflections accumulate in a learnings log. The harness agents read from this log when making decisions, so patterns of past mistakes inform current review.

Regression detection works in the same direction. When the harness-audit agent runs, it does not only check whether current constraints are being met. It looks at the history of constraint violations to identify patterns: are the same constraints being violated repeatedly? If so, that is a signal that the constraint needs a stronger enforcement mechanism, or that the context document does not explain the rationale clearly enough, or that the constraint itself is wrong and needs to be reconsidered.

This closes a loop that the original framework leaves open. A static harness gets better only when humans notice failures and update it manually. A self-improving harness treats its own operational history as input data and generates proposals for its own improvement. Humans still decide which proposals to accept, but the work of pattern recognition — reading the violation log and noticing that the same mistake keeps recurring — is delegated to the agents.

The auto-harness additions in this plugin extend this further: the harness-init process itself reads existing code to infer constraints that are already present in the codebase but not yet declared. Rather than requiring the team to specify everything from scratch, the agent bootstraps a candidate HARNESS.md from observed patterns and asks the developer to confirm, reject, or refine each entry. The human remains the authority, but the initial cost of building the harness is substantially reduced.

Harness-init also supports incremental adoption. Teams choose which features to configure — context engineering, constraints, garbage collection, CI, and observability — and can re-run the command later to add more. Existing configuration is preserved across runs. This means a team can start with just context and constraints, prove the value, and add garbage collection and CI enforcement when they are ready. The harness grows with the team's maturity rather than demanding full commitment upfront.

Further Reading

The conceptual foundation for this plugin rests on Birgitta Boeckeler's article on martinfowler.com. That article is the primary reference for the three-component model and the verification slot framing. The Diataxis framing also draws on Addy Osmani's Agent Harness Engineering, which sharpens the model-plus-harness distinction and the "every line earned" discipline. The Diataxis framework (diataxis.fr) informs the documentation structure. The three-enforcement-loops model, progressive hardening ladder, and self-improving dimension described here are extensions developed in the context of this plugin.

For the document itself — what HARNESS.md is, how it is operated, and how it compares to AGENTS.md / CI / hooks — see HARNESS.md, the Document.

See the other pages in this section for deeper treatment of each component.

Find Resilience Risks Automatically, Then Confirm Them | Harness Blog

RT Agents detect resilience risk in your CD pipelines and Kubernetes workloads, then generate and run chaos experiments or load tests to confirm it.

Resilience Testing (RT) Agents scan your CD pipelines and Kubernetes workloads for resilience risk before it reaches production, then generate and run the chaos experiments or load tests needed to confirm it.

What We're Announcing

We're announcing a major update to our Resilience Testing called RT Agents. Instead of handing you a testing tool and a blank page, our agents now do the first part of the work, often the hardest part, for you. They continuously analyze your deployment pipelines and infrastructure to predict where resilience risks are hiding, tell you exactly what kind of testing is needed to prove it, and recommend next steps. From there, you decide where to take action. When you do, the agents help you get there: generating the chaos experiments and load tests, running them, and reading the results, so you don't have to start from scratch.

Resilience Testing no longer has to start from zero. Harness finds the risk and helps you act on it, directly in your existing pipeline.

Key Features

Passive risk detection in your Harness Continuous Delivery (CD) pipelines

If you're a Harness CD user, this shows up as Agentic Resilience inside the pipeline you already run. RT Agents continuously read your deployment configuration, manifests, and pipeline history to flag resilience risk as it's introduced, before it reaches production. There's nothing to instrument and nothing to run. Detection happens automatically as part of your existing CD workflow, so risk gets caught at the moment it's created instead of months later during an incident.

Simplified onboarding for Kubernetes apps

Getting resilience insight into your Kubernetes workloads no longer requires a lengthy chaos experiment or load test setup process. Onboard your applications through a streamlined flow that connects to your existing cluster and infrastructure metadata, and the agents start surfacing the resilience risks specific to your environment right away. No faults injected, nothing touched in production.

In-product dashboards for Resilience Insights

Every risk the agents detect rolls up into dashboards built for tracking resilience posture over time, not just point-in-time results. You get a consolidated view of risk across services and pipelines, so teams and leadership can see whether resilience is trending in the right direction.

Resilience Scores for your Load Tests

Load testing results now come with a Resilience Score: a single, trackable number that reflects how your services hold up under load, beyond raw throughput and latency numbers. It's built to make load test outcomes easier to compare over time and easier to explain to stakeholders who don't work with performance data day to day.

Java, JavaScript, and Python support for Load Test scripting

You can now write Load Test scripts in the language your team already uses. That removes a common barrier to adoption: teams don't need to learn a new scripting language just to write a load test, so load testing fits more naturally into how your engineers already work.

Free Plan: Try It Out

You don't need to take our word for it. Harness offers a free plan for resilience testing, so you can put it to work in your own environment before committing to anything. It includes a hosted experience with the core capabilities you'd expect: an extensive fault library, a centralized control plane, native integration with Harness pipelines, and enterprise features like RBAC and hosted logging, all at no cost. It's a way to validate the resilience of your own services without the limits of a typical trial account.

If you're new to Resilience Testing, this is the easiest way to see what RT Agents find in your own pipelines. Sign up, connect your infrastructure, and let the agents show you where your risk actually is.

FAQ

Does RT AI Agents touch production or inject faults during detection?

No. Detection is passive: the agents read your existing deployment configuration, manifests, and pipeline history. Nothing is instrumented and nothing runs against your services until you choose to generate and run a chaos experiment or load test.

Do I need to set up a chaos experiment or load test before RT Agents can find risk?

No. The agents analyze your CD pipelines and Kubernetes workloads on their own and tell you what kind of test would confirm the risk. You only run a test once you decide to act on what they find.

What's included in the free plan for Resilience Testing?

A hosted experience with the full fault library, a centralized control plane, native Harness pipeline integration, and enterprise features like RBAC and hosted logging, all at no cost.

Which languages can I use to write load test scripts?

Java, JavaScript, and Python, alongside Harness's existing load testing options. Use whichever your team already works in instead of learning a new scripting language.

What is a Resilience Score?

A single, trackable number attached to each load test that reflects how a service holds up under load, beyond raw throughput and latency numbers, so teams can compare results over time and report them to stakeholders.

Empowering autonomous agents with advanced security governance

AI agents are the ultimate insiders. We grant them permission to read emails, query databases, and trigger API calls. They don’t just retrieve information, they take action.

Agents offer incredible potential for increased productivity and better customer experiences, but they also come with new security concerns. In our new State of AI infrastructure report, 79% of tech leaders cite security, governance, or operations as their most significant challenge to scaling inference.

While there’s still a crucial role for traditional security tools, the threat model has fundamentally changed. Autonomous workflows have redefined enterprise risk, so it's crucial that we give agents the access they need without compromising security.

The agentic paradox

The path to success starts with viewing governance as a driver for innovation. To be useful and secure, an agent needs access — and also guardrails. Yet 35% of senior IT decision makers cite insufficient security for multi-system access as a primary issue preventing agentic deployment.

Agents expand the surface area that defenders need to protect, and can introduce new threats, including tool poisoning and indirect prompt injection, where an attacker can hijack an agent’s logic through the data it processes. Managing the dynamic permissions that agents need to succeed at their tasks can also be a significant challenge, particularly as legacy security wasn’t designed for today’s automated threats.

Securing the chain of thought

Along with securing more identity and access issues, it’s important for defenders to secure both the network layer and the model.

Security leaders are increasingly shifting their focus from preventing breaches to verifying provenance to guard against misuse, including indirect prompt injection.

From an infrastructure perspective, what are your top security concerns related to AI?

From blocking to managing

We’ve looked at the new security challenges posed by agentic AI. You can’t solve them by simply locking down the system, as that defeats the purpose of autonomous agents.

Many organizations are turning to integrated, full-stack cloud platforms to give them greater oversight. 69% of surveyed executives now rate a full-stack platform as a critical requirement, and 80% say data compliance is the primary factor dictating that choice.

By adopting frameworks like the Secure AI Framework (SAIF) and moving to a central control plane, purpose-built platforms such as Gemini Enterprise Agent Platform, organizations can manage risk in three main areas:

• Secure-by-default design: Embedding security directly into the AI development process to proactively guard against threats including prompt injection.
• Agent governance and oversight: Adopting purpose-built permission and identity management for agents — giving greater control over agent interactions, exposing blind spots and limiting risks tools.
• Human-in-the-loop control: Enforcing clear rules that automatically flag when an agent requires human approval before moving forward with a critical action.

Governance will guide you to success

The true value of a modern security foundation is its ability to encourage innovation. By embedding robust governance directly into a unified foundation, organizations can deploy agents with confidence across their most sensitive, business-critical workloads.

The leaders of the agentic era are re-architecting their stack to use security as a launchpad — empowering them to innovate securely and scale faster than their competition.

Find out more about how enterprise leaders are rethinking security for the agentic era in the State of AI infrastructure report.

Archify

Turn a codebase or system description into a polished, interactive system map — directly in chat.

Archify is a Node.js rendering and validation system for Cursor, Claude Code, Codex CLI, and OpenCode. Agents produce typed JSON IR; Archify deterministically compiles it into HTML/SVG.

• Open it and present — five diagram types, four presets, dark/light themes, built-in brand marks, and finite motion
• Review architecture changes before merge — compare two validated snapshots as Before / Delta / After, with exact added, removed, changed, moved, and rerouted facts
• Every interaction stays grounded — search nodes, optionally open revision-verified source, trace upstream/downstream authored reach and exact routes, compare roles, and play guided stories without inventing topology
• One file, ready to trust and share — typed JSON IR and deterministic checks produce self-contained HTML plus PNG, SVG, WebM, and 1200×630 share cards

Current development version: v2.16.0-dev.0.

Using Cursor? Open the agent-aware quick start for exact global and project commands.

Then ask your agent: Use archify to map this repository's runtime architecture.

❤️ Sponsors

See Archify in action

These are generated Archify artifacts, not product mockups. Click a frame to open its live, shareable state.

Three real generated artifacts. Signal Flow · Blueprint · Classic

The Proof Lab contains all 11 checked-in scenarios, their JSON sources, named views, and validation receipts.

A real repository, mapped from source

Archify traced mco-org/mco at 9f1a1cf and produced this checked map.

Preview

Same diagram, two themes, one click to switch.

The Export menu copies PNG to the clipboard and downloads static or motion formats.

Use Copy Share Card when you want a canonical 1200×630 image for a README, release, or social post.

After tracing a route, Export → Route Share Card downloads that authored path as a 1200×630 PNG with the full diagram retained for context.

After tracing authored Upstream or Downstream reach, Export → Reach Share Card captures that exact reading without claiming runtime impact.

Quick start

1. Install

npx skills add tt-a1i/archify -g

For an explicit, non-interactive Cursor install:

npx -y skills add tt-a1i/archify --skill archify --agent cursor --global --copy --yes

To try without installing:

npx skills use tt-a1i/archify@archify --agent codex

2. Ask for one bounded view

Analyze this repository, then use archify to create a high-level runtime architecture diagram.
Show 8–12 core components, one primary path, external dependencies, and trust boundaries.
Put supporting detail in cards instead of adding more edges.

3. Refine in chat

Continue with focused requests such as add Redis, move auth to the left, or highlight the rollback path. Archify keeps the typed source available for targeted iteration.

Choose the right diagram

For a production deployment review, Architecture can optionally enable the deployment-ownership engineering profile. It fails closed when owners, single-region placement, private database scope, or named boundary crossings are missing. It is never enabled silently and validates authored facts—not live infrastructure.

Why Archify

• Layout judgment over generic auto-layout — the agent chooses hierarchy, spacing, routes, and emphasis; shared automatic endpoints spread deterministically instead of piling arrows on one midpoint.
• Typed JSON IR — every renderer-backed mode has a schema and reproducible source.
• Atomic validation before delivery — schema, layout, HTML/SVG, route, and label-to-route clearance checks must all pass before a showcase artifact replaces the last known good output.
• Failures come with a repair receipt — validate --json and deliver --json return stable rule codes, the exact subject, measured evidence, and only supported repair controls instead of a Node stack or an unstructured retry guess.
• Last-good live preview — an optional desktop loop watches one JSON file, refreshes only after the latest candidate passes every gate, and keeps the previous verified diagram visible when a save is incomplete or invalid.
• Truthful interaction — focus, upstream/downstream reach, exact routes, role comparison, and stories reuse authored nodes and relationships instead of inventing topology or claiming runtime impact.
• Source evidence, only when requested — Evidence-backed Architecture nodes mark themselves SRC n and open Git-verified files and line ranges pinned to one public commit; ordinary artifacts stay source-free.
• Portable by default — the result is one HTML file; exports remain full-diagram and free of temporary viewer state.

How it works

Explore and share the output

Installation options

License

MIT — free to use, modify, and distribute.

Contributing

Issues, pull requests, and real-world diagrams are welcome.

Full article content is not available for inline reading.

Read the original article →

Agents are already calling your self-service APIs, querying your catalog, and independently provisioning resources around the clock. According to Gartner’s 2026 Hype Cycle for Agentic AI report, agents have had the most aggressive adoption curve of any emerging technology. Gartner even considers agents to be a formal user persona, referring to the agent experience (AX) throughout the report. The question that platform teams are asking has shifted from “Should agents use the platform?” to “How should the platform evolve now that they do?”

For teams using Golden Paths to encode supported development workflows, this shift introduces a new challenge. Paths originally designed to guide human developers must adapt to guide autonomous agents.

In this post, we’ll discuss how to adapt Golden Paths so that agents can use your platform effectively and within constraints. Specifically, we’ll cover the following:

• How to match Golden Paths to the agent’s workload requirements and control boundaries
• How to make the platform capabilities used by Golden Paths machine-consumable and enforceable
• How to turn each Golden Path into a controlled platform workflow

How should you design Golden Paths for AI agents?

An agent-facing Golden Path should be designed around an execution pattern and define which steps must remain under deterministic control.

Match AI agent execution patterns to workload requirements

The term execution pattern describes how and where an agent’s process runs. Platform teams should select an execution pattern based on the workload’s latency, durability, isolation, and API requirements. A Golden Path designed for one execution pattern may suit some workloads but not others.

We recommend that you start by choosing an execution pattern that addresses the workload’s primary constraint. Use a synchronous, low-latency pattern for real-time interactions and a durable pattern for asynchronous or resumable tasks. You can supplement the execution pattern with supporting capabilities as needed, such as an isolated sandbox for development tasks.

You may recognize the underlying principle driving our recommendation. For more than two decades, methods like Attribute-Driven Design, developed by the Carnegie Mellon Software Engineering Institute, have taught engineers to select an execution pattern based on workload requirements. The unique challenge when working with agents is the workload profile, which includes more variable execution time, dynamically selected actions, and potentially greater requirements for durability or isolation.

Real-time conversational AI agents need a low-latency execution pattern

A real-time conversational agent operates within a tight latency budget that typically ranges from seconds to a few minutes. These agents need Golden Paths that use a synchronous, low-latency execution pattern because the agent receives a request, invokes models and tools, and returns or streams a response within a single interaction.

The Golden Path must account for time-to-first-output and total completion time. Streaming reduces the wait for initial output but doesn’t guarantee that the agent will complete its work within the overall latency budget. Low latency and a responsive user experience have always been priorities for synchronous services.

To keep this work within the latency budget, the Golden Path should set an end-to-end deadline, apply timeouts to individual calls, and define what happens if the work cannot finish before the deadline.

Long-running, proactive AI agents need a durable execution pattern

Long-running agents operate asynchronously on tasks that continue for hours or days. Because their work can outlive a single request or application process, the Golden Path should use a durable, resumable execution pattern.

With durable execution, workflow state must persist independently of model context and in-memory agent state to keep the record of the workflow’s progress intact. The record should include completed steps, intermediate outputs, and pending approvals so that work can resume after an application restart or infrastructure interruption.

Add isolated execution when AI agents need to run code

Agents that operate in a development environment inherit that environment’s permissions. Model-generated commands and repository code run with the same access to files, credentials, and network endpoints as the agent process. The Golden Path should isolate these operations from the primary agent service.

When possible, keep the agent harness in a standard, observable service and expose an isolated sandbox as a remote tool. Restrict the sandbox’s filesystem and network access, resource usage, and execution time to fit what the operation requires. The Golden Path should also specify the sandbox’s lifetime and cleanup behavior to prevent state from persisting between runs.

Separate probabilistic agent work from deterministic control

Golden Paths can include probabilistic and deterministic steps. Probabilistic steps allow the agent to perform work that requires interpretation or judgment. Deterministic steps can act as controls by using CI workflows, policy checks, authorization, approvals, and deployment rules to determine whether work can proceed.

When designing an agent-facing Golden Path, identify where agent judgment produces an output that could lead to a consequential action. To reduce the risk of unverified results being executed, place deterministic controls between a probabilistic output and the resulting action.

How can I help AI agents use my platform capabilities?

Golden Paths will tell the agent what to call and when, but they won’t make your platform’s capabilities usable by agents. Each capability’s interface must be legible so that your agent can reason about it independently, and appropriate security measures need to be enforceable at the platform level.

Platform capabilities exposed to agents should have reusable, machine-consumable contracts that define how agents can use them safely. For each workflow, the Golden Path can impose stricter permissions, limits, or approval requirements as necessary. A Golden Path should use authoritative service metadata to determine which resources the agent can act on and which requirements apply. Since guardrails at the orchestration level do not transfer to the capability itself, the platform—not the agent—must enforce both the capability-level contracts and the Golden Path-specific constraints.

Expose capabilities through structured interfaces

Every action exposed to an agent should have a machine-consumable contract that contains the following:

• A clear description
• Typed inputs and outputs
• Stable and versioned schemas
• Machine-readable limits and error categories
• Declared side effects, preconditions, and approval requirements

Rate limits, concurrency limits, execution timeouts, spending limits, and compute or memory boundaries should be enforced at the interface, gateway, or workflow orchestrator. Errors should be machine-readable and differentiate between invalid input, authorization failure, rate limiting, temporary unavailability, and non-retryable policy rejections.

Because agents might repeat a request even after the first attempt succeeds, state-changing operations should be idempotent where possible. When inherent idempotency is not an option, the platform can accept an idempotency key or unique operation identifier so that repeated requests return the original result instead of performing the action again.

Keep the service catalog authoritative for AI agent decisions

Agents can consult wikis and runbooks, but they cannot reliably reconcile information that is incomplete, outdated, or contradictory. An agent-facing Golden Path should be able to direct agents to an authoritative catalog for ownership, dependencies, service criticality, environments, and supported actions.

The Golden Path should verify that required catalog metadata is present and trustworthy before allowing a consequential action to proceed. If that metadata is missing, stale, or contradictory, the path should restrict the agent to read-only operations, require human review, or block the action according to its potential impact.

How can I control autonomous AI agent work?

An agent-facing Golden Path should define the dispatch and workflow controls that govern the majority of autonomous work, including why a run may begin, what the agent may access, where it may execute, and how its results must be recorded.

Make dispatch an explicit step in agent-facing Golden Paths

Dispatch is the platform step that converts a signal into a bounded, authorized agent run. A signal can come from an alert, schedule, code change, human request, or other sources. The signal identifies potential work but doesn’t authorize the agent to perform that work.

Formalize this transition within the Golden Path as: signal → curated context → task identity → allowed capabilities → execution target → recorded output. At the curated-context step, the platform combines authoritative catalog metadata with evidence from the triggering signal. It then assigns a unique task identity that connects the run to its source and enables attribution, deduplication, and audit.

Build checkpoints and retry budgets into AI agent workflows

An autonomous workflow needs explicit points where the platform evaluates progress and determines whether the agent can continue. Build approval states, dry-run or preview modes, and precondition checks into the Golden Path before actions that modify production systems or create other consequential effects.

Bound each correction loop by its number of attempts, elapsed time, cost, or a combination of these factors. Without these limits, an agent can repeatedly consume model, tool, and infrastructure capacity without achieving a successful outcome.

Preserve visibility and accountability in AI agent workflows

An agent-facing Golden Path should define the execution telemetry data and audit evidence that every run must produce. Capture model calls, tool selection, retrieved context, latency, errors, retries, evaluations, and cost.

A trace explains how the agent run unfolded, while an audit event records an action taken against a platform resource. For each external action, record the task identity, credential or workload identity used, and the affected resource, requested operation, approval state, timestamp, and outcome.

Key takeaways: Evolving Golden Paths for AI agents

Golden Paths do not disappear when platform users become agents. They multiply because conversational, long-running, and code-executing agents have different requirements for latency, durability, and isolation. Supporting their execution patterns safely requires machine-consumable capability contracts, authoritative service context, controls enforced outside the agent, and evidence that includes the details of each run. Begin with one bounded path, verify that the agent selects appropriate tasks and operates within its constraints, and then expand its authority incrementally.

Adobe says that the best creative software lets users focus more on their creative ideas and less on the actual tools required to bring them to life. The company believes Photoshop’s latest update builds upon this vision and gives photographers more flexibility and choice in how they create.

These latest Photoshop innovations promise more efficiency and precision in how they edit images inside of Photoshop. Photoshop is getting a Light Adjustment Layer, a Firefly-powered masking and editing tools, and markup tools. The longtime image editing app is also getting a full-fledged AI Assisted Editor, albeit just in beta at first.

The new Light Adjustment Layer promises professional-grade lighting controls, including over Exposure, Contrast, Highlights, Shadows, Whites, and Blacks. This non-destructive adjustment layer takes some of Adobe Camera Raw’s most powerful photo editing tools and puts them directly inside Photoshop. The more that photographers can do directly inside Photoshop, the better.

Powered by Firefly Image 5, the new Instruct Edit with Masks function promises to understand the entire context of a photograph. Users essentially describe what they want to change in the image, such as opening a person’s closed eyes or changing a model’s clothing, and Instruct Edit with Masks will automatically mask and change the relevant area, leaving the rest of the photo untouched.

Many photographers who work with clients or editors use Photoshop. Dedicated AI Markup tools are now available in Photoshop, letting photographers (or clients) communicate desired edits directly on the photo. Users can select areas to recolor, sketch arrows to indicate where to make changes, or draw rough sketches to suggest more significant changes to an image. Firefly then does its best to show what the markup refers to.

Although not quite as exciting for photographers as the new Light Adjustment Layer, Photoshop’s Dynamic Text can now work alongside any custom shape or path. There’s also improved Adobe Stock Panel Integration that brings more than 900 million Adobe Stock assets into Photoshop on a dockable panel, so users no longer need to leave Photoshop to discover, preview, or use Adobe Stock assets in their work.

Arriving in Photoshop today as a beta, the new AI Assisted Editor introduces a dedicated interface with a natural language bar that lets people make edits through simple prompts. Depending on the prompt, Photoshop will leverage its wide array of tools to try and achieve the desired results.

The AI Assisted Editor can work alongside AI-powered features like Remove Background, Generative Expand, Generative Fill, and Generative Upscale without requiring the user to actually do anything more than type out a few words.

“Whether you prefer prompts, precision tools, or a combination of both, Photoshop offers the flexibility to support the way you create,” Adobe says.

The new Adobe Photoshop updates are available now.

Design system pattern names like Overview and PageHeader are shorthand for decisions a team should not have to reopen every time. Learning that vocabulary takes a person months, but an agent can read the repository and retrieve the code, guidance, and examples behind each name. That lets more people describe a screen in the system's own language and judge the result.

Perfection is not over-engineering

"We don't want to do perfect." "We don't want to build the perfect solution." I've heard versions of that line more times than I can count, delivered as if "perfect" were a dirty word. And I understand the caution — over-engineering burns teams, and people have learned to treat anything that smells like perfection as the same risk.

It isn't. The industry has quietly conflated the two.

Over-engineering is solving the wrong problem. That's the whole definition. Not "caring too much." Not "making it too good." Solving the wrong problem. Often with good intentions, and almost always with a growing pile of incidental complexity.

There is a perfect solution

I believe a perfect solution exists. With one big caveat: you need a very clear set of requirements. Every constraint on the table. Tighten those enough and something interesting happens, you end up with only one possible solution. And that solution is, somewhat ironically, the perfect one. It's perfect because it's the only one that fits.

Start a new project. Every language, every tool, every hosting model available. You pick serverless. Python is a strong choice: no compilation step, upload your files to Lambda, ship. For someone else it's the wrong choice, they don't know Python, or they need to optimize for a different set of requirements, such as performance. Different constraints, different answer. Same problem space, different "perfect."

Or you chose Python and you're building a web app. Django or Flask? You can reach similar results with both. They're still different tools with completely different philosophies. Which one wins? It depends. Set clearer requirements, set stricter constraints, and the solution follows. That solution is the perfect one for you, for that case.

Systems are products

When a system is over-engineered, the cause is almost always requirements. And I mean requirements in the product sense, not just the technical one.

A library, an API, an internal tool... we like to pretend these are "purely technical," that they somehow sit outside the idea of a product. They don't. You have users. Those users have needs. You need to understand those needs well enough to address them properly.

Maybe what they need is a service. Or maybe it's better served by a library than by an HTTP call. Instead of handing them an API, maybe you hand them a package. The shape of the solution only becomes obvious once you treat the system as a product and define the requirements honestly. Then the solution follows.

How you can tell

The clearest tell that something is over-engineered: you start asking why are things built the way they are? and the answers don't hold.

Classic example. A team of three people maintains five microservices. The services share data among each other. Is it over-engineered? Figure out which problem they were trying to solve. Most likely you'll conclude they were solving the wrong ones (or several of them at once).

Look at what the split actually costs. What used to be a hard reference in a database, a foreign key the engine enforces for you, is now a loose string id sitting in a field. Data integrity is gone. One service can delete a record and the other has no idea; it just keeps a dangling reference and finds out later, the hard way. Why all this ceremony between services when it's all part of the same domain? Why give up those integrity checks?

What did you gain in exchange? Usually: not as much as you lost. Independent deploys, sure, but was that a problem you actually had? Three people, one domain. You solved for a scaling and ownership problem that wasn't on the table, and paid for it with distributed inconsistency, operational overhead, and a system that solves multiple problems partially, none of them completely, while introducing a bunch of problems you wouldn't have had otherwise.

That's the signature. Not elegance. Not thoroughness. And it's not that these solutions are bad, usually they're the correct answer to the problems that were proposed. The problem is that those were problems you never had.

Gather the right requirements

So the diagnosis is simple, even if the work isn't. Over-engineering is a failure of requirements gathering. Call it product engineering if you want. It's the consequence of collecting the wrong requirements, and then engineering diligently against them.

Perfection was never the enemy. Ambiguous requirements were. Get those right, get every constraint on the table, and the perfect solution stops being a fantasy. It becomes the only thing left standing.

GPT 5.6 Discounts & Jevons Paradox

Highlights

• Tokens spiked: Daily Terra token usage rose 5.6x during the discount window, while daily Luna token usage jumped 13.8x. The Sol model, which remained a list price, saw only a gentle bump of 1.1x over the same period.
• Share was reconfigured: Most of the share gained by the OpenAI discounts came from other labs as opposed to cannibalization within the OpenAI family of models.
• Users stuck around: Nearly a third of users who tried a discounted OpenAI model during the discount window kept using it after the discounts expired.

Discount impact on token usage

The moment GPT 5.6 discounts kicked in, the token volume exploded.

If you compare the in-program token usage to the pre-period daily averages, the effect is crystal clear: Terra tokens rose 5.6x and Luna 13.8x. The Sol model, which remained un-discounted, saw a minor 1.11x bump while other OpenAI models actually fell slightly in tokens used. Outside the OpenAI family, the other models rose gently.

Competitive displacement

Terra/Luna went from 0.7% to 7.8% of all OpenRouter tokens between the pre-period and the program, a gain of 7.1 share points. All competitors are pooled into the light-grey segment, and the label above each color stack is the combined OpenAI-family share (Terra + Luna + Sol + other OpenAI).

Competitors gave up 5.3 points and other OpenAI models gave up 1.9 points over the same comparison, so roughly three quarters of the gain came from outside OpenAI. Across the whole OpenAI family, token share grew from 7.1% to 12.4% and occasionally crested over 15% in specific days during the discount period.

Tokens mostly rose across key model authors, though Anthropic did not in this timeframe. OpenAI token volume nearly doubled on average and has maintained that higher level after the discount program ended.

Users retention

Clearly the discounts drove increased token usage during the program period, but did those users stick around after the discounts had ended?

Of the 100K+ customers with Terra/Luna usage during the program, about 32% retained some usage in the subsequent days and 18% ran at or above their program pace. To be clear, this is a count of customers, not weighted by tokens. Obviously the post-program period is quite a bit shorter than the full discount timeframe (6 days so far vs a 19-day program), so the story may change as more data comes in.

If you look instead at the daily Terra/Luna token volume, the post-program period saw 1.38x the tokens of the discount period on average, which implies the retained accounts are far larger than the median program user.

Sol averaged 79.1B tokens/day during the Terra/Luna program against 71.2B/day pre-period, an effectively flat control group.

However, the shaded region in the chart above from Aug 17 onward is Sol’s own 50% discount. Sol jumps immediately, reproducing the Terra/Luna pattern.

Methodology & Notes

Data Details

• Pre: Jul 8 to Jul 26 | Program: Jul 27 to Aug 14 | Post: Aug 15 to Aug 20
• Jul 30: OpenAI cut its own list prices (Luna 80%, Terra 20%) on top of the 50% discount through OpenRouter. As such, effective discounts from Jul 30 were Luna 90% and Terra 60%.
• Notes: Terra, Luna and Sol all launched Jul 9. Sol is a useful control up until it got its own 50% discount on Aug 17.

Methodology

• Exclusions: banned, deleted, admin, data-deletion-requested, internal and churned accounts are dropped.
• Aug 20 is the last day in the window and may be partial.
• Sol is a valid control only through Aug 16. The post-period is 6 days.

Anthropic and OpenAI show impressive revenue growth, with expectations for continued rapid expansion. Forecasts indicate moderated growth for semiconductor stocks but an uptick for software stocks, with AI's impact on the software sector remaining a key consideration.

AI and the city

More businesses are forming outside major metros, but big cities still matter for frontier tech.

In a previous post, we argued that AI appears to be behind a significant increase in entrepreneurship, particularly among solopreneurs. Part of the reason is that AI fills the capability gaps that used to require other people with complementary skills. One possible implication of this is that cities—where people go to meet other people—could become less valuable in the age of AI.

This debate has played out before. During the COVID-19 pandemic, work from home provoked predictions about the end of cities. So too did the rise of email and instant messaging, and before that, the telephone, and earlier still, the telegraph.

There are many reasons to believe that technology could weaken agglomeration effects, but history shows that the substitution of technology for in-person interaction is not straightforward. Cities benefit from both centripetal forces that draw firms and individuals in, such as shared infrastructure, amenities, and deep labor markets; and from centrifugal forces that push people out, such as congestion, high housing costs, and crime (Krugman 1996). New technological waves often act in both directions, and it is difficult to predict ex ante what the net effect will be.

Overall urbanization has been rising for over a century, but specific cities and neighborhoods within them rise and fall relative to each other. Jobs move to new places for two reasons: existing companies grow, shrink or relocate, and new companies get founded elsewhere. Stripe’s data allows us to observe the latter particularly well.

We find that AI-era businesses (which we define as companies that were started from 2024 and on, and which includes both AI companies and non-AI companies) are far more distributed than before. Close to 40% of new businesses on Stripe this year are in metros with fewer than one million people, up 10% relative to four years ago. This year, twice as many new businesses were formed per person in Cheyenne, Wyoming as in New York City; in 2022, the rates of business formation in both cities were equal. And even for businesses still forming within major metros, they are increasingly forming in the outer suburbs rather than in city centers and high-density areas: the share of businesses forming in outer suburbs is up by as much as three to four percentage points in metros such as Houston, Austin, Tampa, Orlando, and Dallas in the last few years.

There are two caveats to the AI-era story. First, there have been other factors influencing the spread of business formation, particularly the post-COVID-19 rise of remote work, and it might be challenging to entirely disentangle these. Second, not all AI-era businesses are more distributed geographically. In fact, the companies that are most responsible for AI development itself—AI labs and AI product companies—continue to cluster in major metro areas.

New businesses in the AI era are less concentrated in major cities than in the past

At first glance, AI-era business formation appears to be broadly distributed across the United States. While larger metros do on average see higher sign-up density than smaller metros and micropolitan areas, the distribution of sign-ups is not especially concentrated. We find smaller metro areas like Hinesville, Georgia (population 90,000), Cheyenne, Wyoming (100,000), and Fayetteville, North Carolina (390,000) among the densest hubs for new business formation. Meanwhile, major metro areas that one would expect to be at the core of this surge in new businesses—like New York, Los Angeles, or San Francisco—are seeing below-average business formation relative to their sizes.

To see how this compares to historical trends, we map Stripe sign-ups relative to population across US metro areas larger than 200,000 people from 2019 (which we use as our pre-COVID-19 baseline) to 2026. Micropolitan areas are excluded from this map for legibility but included in our analysis. Here we see that the current AI-era distributed formation is actually a continuation of a broader shift in entrepreneurial activity away from major metropolitan areas (larger bubbles getting lighter) and toward secondary metros (smaller bubbles getting darker).

Business formation is up across the country, so to understand relative winners and losers, we need to look at how the share of businesses forming in different locations is changing. This confirms what the map suggests, which is a gradual move away from larger metro areas (top 10 and metros with more than one million people) and toward smaller metros and micropolitan areas.

The biggest jump happened during COVID-19 as a consequence of work from home, but the general trajectory has continued in recent years and appears to be reaccelerating since 2024, closely tracking the AI era. The shift we’re seeing now is also reflected in Census data and is roughly one third as large as the COVID-era move away from major cities.

Within major cities, AI-era businesses are also less clustered in city centers

Within major cities, we also see that Stripe sign-ups are widely distributed outside of city centers. If agglomeration effects were key to AI-era business formation, you might expect to see the share of sign-ups in city centers and high-density areas vastly outweighing population share. Instead, sign-ups track population distribution within cities closely, and close to 80% of sign-ups in major metropolitan areas are outside of the city center and high-density inner ring.

What’s more, this distribution marks a change from previous waves of business formation. Much like the trend away from major metros, the shift in business formation away from city centers and toward outer suburbs has reaccelerated recently, relative to the post-COVID-19 plateau. Houston saw 37% of new businesses forming in the outer suburbs this year, up four percentage points relative to 2023. While this is a particularly stark example, the general trend holds across most major metros. The COVID-19-era trend is well documented in the literature (Ramani and Bloom 2021, Decker and Haltiwanger 2023), though to our knowledge, the recent reacceleration in this trend has not been covered elsewhere.

Disentangling AI from work from home is complicated

The data on AI-era business formation would thus appear to be consistent with AI dampening the value of agglomeration in major metros. But the extent to which AI is independently causing this change is hard to untangle. One confounding factor is the interaction between AI and work from home. For example, AI might be unlocking latent entrepreneurial potential among individuals that relocated with work from home policies—a continuation of the entrepreneurial activity spawning from remote work documented in the literature (Kwan et al. 2024).

If this were true, we might expect to see businesses disproportionately forming in areas with high levels of work from home. Our data does not currently support this. Business formation appears uncorrelated with work from home intensity, and that holds whether we look across metro areas or at the distribution of businesses within them. Still, this is not enough to conclude that AI-era business formation would be as widely distributed absent work from home.

AI work itself is immune to this effect

That AI-era business formation is broadly distributed does not mean that agglomeration does not matter with Al. In general, knowledge at the technological frontier is highly tacit, and frontier firms cluster in major cities to benefit from knowledge spillovers in their domain. The field of AI development itself is no different. If anything, the speed at which innovation in AI is moving would suggest that the proximity of cities should be even more valuable.

Stripe data allows us to test this empirically. We can categorize AI-era businesses in three groups, according to their distance from the technological frontier. First are the AI labs themselves. Then there are businesses in which AI is part of the product. Finally, there are businesses that are AI-enabled, but that are building and selling non-AI products.

Aligned with our hypothesis, these different types of businesses show different spatial distribution, with those closest to the technological frontier clustering most tightly in big cities. Almost 80% of AI businesses closest to the technological frontier (the labs) are in San Francisco alone. Among AI product companies, which are also innovating close to the frontier, 50% are in San Francisco or top 10 metro areas. Finally, it’s the AI-enabled businesses that are most widely distributed across the country.

Conclusions

• US business formation in the AI era appears to be more decentralized than previous waves: a higher share of firms are forming outside of major cities; and even among firms forming in major cities, a growing share are forming outside the city centers.
• We saw this trend during COVID-19 with the rise of work from home, but it has reaccelerated in the last few years, coinciding with the advent of AI.
• While it’s hard to fully disentangle this shift from changes driven by work from home, the distribution of AI-era businesses appears uncorrelated with work from home intensity at the city level.
• The subset of new firms closest to the technological frontier—labs and AI product companies—continue to cluster more tightly in San Francisco and other major metro areas, presumably because of the continued benefits of agglomeration for innovation.

A year ago, the average sell-side consensus estimates for Nvidia’s FY 2028 (which is essentially CY 2027) revenue was ~$310 Billion. But yesterday, Nvidia guided for ~70% revenue growth in FY’28 which translates to approaching ~$700 Billion revenue next year! That means Nvidia’s FY’28 revenue estimates was wrong by ~$400 Billion just a year ago! Even before the call, analysts were estimating ~$574 Billion revenue next year. So even though analysts kept updating the estimates throughout last year, they were still short by ~$125 Billion! If that wasn’t enough, Nvidia repeatedly mentioned during the call that this growth outlook is supply constrained and they could probably double the revenue next year if that weren’t the case. Given their tone during the call, I think it’s fair to say ~70% revenue growth may be a floor for next year. It’s a cliche to say it these days, but I have never seen anything like it!

Perhaps equally perplexingly, just when Nvidia was making mockery of analyst estimates over the last year, the stock has barely outperformed S&P 500 and actually underperformed both QQQ and semiconductor ETF. In case you needed another reminder why investing is hard, I’m happy to help.

Such relatively muted stock performance compared to Nvidia’s operating performance is a confirmation that buy-side likely have a materially different estimates compared to sell-side. Even though sell-side estimates show Nvidia will keep growing revenue with operating margins at mid to high 60s, Nvidia’s current earnings multiple likely implies that buy-side is far more skeptical about that. This doesn’t mean buy-side is necessarily right, but an indication that Nvidia will have to climb such wall of worries in the coming years.

Nvidia’s revenue mix also shows what they’re trying to execute to climb the wall of worries. While revenue from hyperscalers doubled YoY, revenue from AI Clouds, Industrial, & Enterprise (AICE) segment grew by ~140% YoY! Nvidia mentioned they now expect neoclouds will exit the year with 8 GW of installed capacity in 2026, up from only ~3 GW capacity in 2025.

To contextualize the neoclouds capacity, let me mention that AWS, the largest hyperscaler in the world, only had an estimated ~15 GW capacity in 2025. Since Amazon mentioned that they expect to double their capacity from 2025 to 2027, let’s assume they’ll add 7-8 GW capacity this year which implies neoclouds in aggregate will add ~60-70% of AWS incremental capacity this year. Since the neoclouds are almost all exclusively buying Nvidia GPUs, the rise of neoclouds is a massive boon for Nvidia. Of course, the rise of neoclouds is also very much direct consequence of Nvidia’s deep desire to commoditize the hyperscalers while hyperscalers are busy trying to commoditize the chip layer. At the same time, let me repeat what I mentioned yesterday, who exactly are in the backlog of these neoclouds? It’s the same hyperscalers! So the biggest question here is what will happen to these neoclouds if hyperscalers manage to bring capacity closer to the demand curve over time. Nikesh Arora, CEO of Palo Alto Network, captures the skepticism many investors hold about most neoclouds’ future:

Beyond the hyperscaler vs neoclouds debate, the other vector of concern is whether much of the incremental compute demand is going to be dominated by just a couple of frontier labs such as OpenAI and Anthropic. Both the labs have made it abundantly clear so far that they intend to have heterogenous compute fleet and do not want to exclusively use Nvidia’s GPUs (I guess they can change their mind if Nvidia provides them the funding to buy the GPUs). Nvidia highlighted yesterday that OpenAI’s existing and planned commitments represent 12 GW of Nvidia compute. Interestingly, OpenAI already signed a 6 GW compute deal with AMD, 10 GW with Broadcom, and 2 GW with Amazon. As a result, Nvidia’s share in OpenAI’s commitments is only ~40%. However, much of these commitments are likely to be frontloaded in Nvidia’s chips but that is expected to flip to non-Nvidia chips over the medium to long term.

Anthropic is on the opposite spectrum. Their compute commitment so far is roughly half of OpenAI’s: 5 GW with Amazon, 2 GW with AMD, ~1 GW with Google Cloud, ~3.5 GW with Broadcom+ Google, and likely 3 GW with Nvidia. So, Nvidia has only ~20% share in Anthropic’s commitments. Nvidia probably would love to grow their share in Anthropic in the future, especially when they may start losing share in OpenAI’s compute footprint, but you can imagine Amazon and Alphabet will be ready to sweeten the deal to sell incremental chips to Anthropic. You may argue at the rate Anthropic is growing, they may not need such circular funding sweeteners but in that case, who do you think hold the negotiating leverage if four different companies are trying to cajole one company to buy their silicon?

This is why having any concentrated compute demand will likely prove to be quite destabilizing for the long-term economics for both the silicon layer and hyperscalers. Such dynamic is perhaps why buy-side is less comfortable in underwriting higher multiple in Nvidia’s earnings power despite eye-watering growth.

To be clear, Jensen Huang is very, very clear-eyed about the risks here. Even when he’s providing backstop or financing to AI labs, he does not want to see massive concentration in compute demand in the medium to long term. That explains why Nvidia is going to acquire Hugging Face. As “Modest Proposal” wittily put it, Nvidia wants to commoditize both its competitors and customers so that Nvidia remains the King! Indeed, AI is increasingly the game of Kings and while everyone appears to be friendly with each other, they are also very much wary of defending their thrones! It’s hard to know how the battle lines will be ultimately drawn. Given that context, perhaps we should wonder less about Nvidia’s anemic earnings multiple and question more why other AI beneficiaries deserve lofty multiples.

DeepSeek is seeking to raise $7.4 billion to bankroll research and development and the buildout of computing infrastructure.

This week, artificial intelligence giant Anthropic began advertising a new job to lead its “National Security Sales” team, according to a public posting reviewed by the Prospect. The job opening, which hasn’t been previously reported, signals that the end may be in sight for Anthropic’s extended spat with the Department of Defense and the Trump administration, which earlier this year blacklisted the company from doing business with the federal government or any federal contractors, forever.

Per the job opening’s description, Anthropic’s new Head of National Security Sales will play a leadership role in rejuvenating the company’s contracting relationship with military and intelligence agencies. The new hire will “lead and scale our national security sales organization to drive the adoption of safe, frontier AI across the Department of War and the Intelligence Community.” The job pays well, up to $700,000 a year.

The pivot for Anthropic, which has taken pains to market itself as the most ethical AI company, comes as military and intelligence agencies push aggressively to adopt AI for a host of applications, like sophisticated surveillance and autonomous weapons systems. Human rights and civil liberties advocates have condemned the fast-accelerating and error-prone adoption of those technologies for increasing the risk of harm to civilians, limiting accountability mechanisms for the use of lethal force, and invading privacy without oversight. In Ukraine, the danger of AI-driven weapons became newly apparent this month after a fully autonomous Russian drone killed three civilians as it descended on a gas station.

“Automation writ large and AI more specifically have increased the pace and tempo of warfare, and the ability to identify hundreds, if not thousands, of target recommendations and take action against these targets indicates that the scale of harm is going to be greatly amplified,” said Amos Toh, who serves as senior counsel at the Brennan Center’s Liberty and National Security Program. AI models, meanwhile, may nurture the worst impulses of the fog of war, producing recommendations that “can sound extremely convincing” while being thoroughly off base, Toh explained.

This summer, AI agents showed how much damage they can do off the battlefield as well, by going rogue and attacking cyber infrastructure, as was the case with OpenAI’s hack of a competitor’s data systems, which went unnoticed for more than a week. That raises further national security concerns, Toh said, introducing a scenario where “an agent might also inadvertently leak classified information without the knowledge of the military or the developer.” “There is a huge national security risk to integrate these systems into military systems without sufficient guardrails and due diligence,” he said.

The Department of Defense (DOD) and Anthropic have been duking it out for much of the last year, after the DOD designated the AI giant a national security “supply chain risk,” which summarily cut it off from federal contracting. Anthropic has been contesting the designation for months, arguing it was a form of politically motivated retaliation over red lines the AI lab said it drew over the use of its technology for “mass domestic surveillance or fully autonomous weapons.” That came shortly after it was revealed that Anthropic’s Claude model had been used in the military’s raid to capture Venezuelan President Nicolás Maduro, and just before Anthropic’s tech was implicated in a U.S. drone strike on an Iranian girls’ school.

The White House responded in kind, calling Anthropic a “radical left, woke company” run by “leftwing nut jobs.” Hours later, the DOD signed an agreement with Anthropic rival OpenAI instead. Sam Altman’s company made assurances that its “services will not be used by Department of War intelligence agencies like the NSA” (the National Security Agency) or “intentionally used for domestic surveillance of U.S. persons and nationals.” Despite those nice words, the deal wasn’t convincing to all, leading OpenAI’s head of robotics to quit. “Surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got,” Caitlin Kalinowski said at the time.

A federal appeals panel is set to rule imminently on whether the DOD broke the law by blacklisting Anthropic, which may set the stage for future defense contracting opportunities. Anthropic is clearly preparing for that possibility. Anthropic did not respond to questions about how it intends to pursue defense contracts going forward, or how it plans to ensure its often-publicized red lines about the use of its technology will be respected by the Defense Department and intelligence agencies. It isn’t immediately clear how the company will have to approach mending the broken relationship, or if that could involve making concessions to military and intelligence community demands. The DOD did not respond to a request for comment.

OpenAI is also beefing up its national security contracting team, targeting further deals with the Defense Department and the intelligence community. Even when it struck its original deal with the DOD, OpenAI left open the possibility of signing “new agreement[s]” with agencies like the NSA at a later date. Now, it’s hiring more sales reps with top secret clearance to focus on selling OpenAI services to national security and defense agencies, helping them along “through their AI adoption journey, from consideration to successful deployment.”

OpenAI did not immediately respond to a request for comment.

Many people have called Nvidia's AI investments circular financing, but the company sees it differently. Frontier AI labs are growing faster than what their balance sheets and credit profiles can support. Growth can only continue as long as they can secure ever-greater amounts of computing power, which they would have trouble doing on their own as they lack the ability to borrow huge sums of money at competitive rates. Nvidia aims to power this flywheel until AI labs are able to support themselves.

Hugging Face unveiled the Microduck on Thursday, a cute little duck-like robot that sells for $399 — and ships before Christmas.

Clem Delangue, CEO of Hugging Face, said the Microduck is an “open-source robot you can teach new tricks with reinforcement learning.” The 25-centimeter-tall duck can waddle, pick things up with its beak, get back up when it falls, crouch, and even roller skate.

“Welcome to the era of open-source affordable robots to democratize physical AI and world models!” Delangue said.

Hugging Face is best known as a platform and community where developers go for open model weights. However, the company acquired French startup Pollen Robotics in April 2025 to build affordable, open source AI hardware. A couple of months later, the two launched Reachy Mini, a small desktop robot. Today, they sell the $499 Reachy Mini, which is powered by a Raspberry Pi computer, and the $399 Reachy Mini Lite, powered by a Mac or PC.

The new Microduck perceives the world with a camera, lidar sensors, and two IMUs (inertial measurement units that measure and report an object’s movement).

Pollen Robotics noted that the Microduck’s behaviors can be trained in simulation and directly deployed on the robot. Developers can then fine-tune the bot, retrain it, and redeploy it. The SDK (software development kit), simulation, and full RL training stack are available on GitHub.

For those who might find it concerning to have a robot with a camera in their bedrooms and private spaces, Delangue previously told TechCrunch that bots run by open source models are much better from a privacy standpoint than “a black box system” controlled by a few organizations, “especially if these organizations’ CEO is not the most stable person in the world.”

While open source gives developers auditability and control, it doesn’t provide a guarantee that sensitive data remains private after consumers start installing software applications on top of the model. Those apps can access the bot’s cameras and microphones and, depending on how they’re built, send that data to external services.

The launch of the Microduck comes as Hugging Face is reportedly set to be acquired by Nvidia at a $13 billion valuation. Nvidia and Hugging Face have been partners for years, with Nvidia providing the startup’s infrastructure since at least 2023. Both companies have also been publicly aligned in promoting open source AI.

Hugging Face was recently in the headlines over a cybersecurity incident that occurred after OpenAI’s systems breached its sandbox during safety testing and hacked into the platform’s servers.

TechCrunch has reached out to Hugging Face and Pollen Robotics for more information.

Correction: A previous version of this article misstated how much weight the Microduck could pick up with its beak.

Transfyr is a startup working to uncover the hidden factors that make some experiments succeed while others fail. Coming out of stealth mode this week with $25 million in seed funding, the startup is building a system that can absorb enormous amounts of lab data in the form of video, audio, and sensor logs from lab equipment. The approach will hopefully reveal some secrets about why experiments succeed or fail. Its analysis has so far revealed that lab workers perform the same experiments in many different ways despite following a protocol, demonstrating how important these variations are.

Small Models Have Arrived

For the past few weeks, I've been playing with gpt-5.6-luna. It is shockingly capable, fast, and smart. I regularly see it do ~100 tps, and rip around my codebase, email, and knowledge base.

Of course, the biggest thing with luna is the cost. I've tried running some fairly complicated research threads, and it's pretty tough to run up a large bill. Even having it search across thousands of emails, I end up with an API cost in the tens of cents.

With GLM 5.3, we even have a new option at the Pareto frontier.

When doing coding work, I almost always reach for the most expensive and capable models (Fable 5, 5.6 Sol). So it's been easy to miss the progress the small fast models have made.

One thing a few investors I've talked with have mentioned: "It's weird we're not seeing more consumer AI companies. Why is that?"

There's a straightforward answer: token costs.

In the times before AI, the playbook for big consumer apps looked like this...

• create some sort of compelling website which is fairly cheap to run
• attract a bunch of users (typically with some virality)
• raise money, scale to more users
• create an ads marketplace

This roughly describes most of the big consumer companies (Google, Facebook, Snapchat, etc.).

But what if you want to add AI to your product? Well, now you have some real inference costs on every request! Suddenly the amount of capital required increases dramatically.

A pet eval of mine is to build a daily news site, personalized to me:

research @calvinfo on the internet. figure out what news they might like. build a micro-site with today's top stories, personalized for them. search hn, reddit, twitter, etc.

With the previous generation of models (Sonnet class), you'd spend ~$1 to get anywhere. Charging $30/mo is untenable for a consumer app. There's obviously a lot we can optimize here, but if you're charging what the WSJ or The Economist charges, you'd better be delivering similar value.

But looking at luna, the results are pretty decent, and the average cost is ~$0.10. Now we're talking!

Where I think this gets even more interesting is in the world of business.

My Segment co-founder Peter and I were recently comparing notes on a hike. Across his various startups, Peter has seen two kinds of work:

1. the "IQ 180" work. some mad scientist genius type comes up with some crazy solution you've never thought of.
2. the "token spewer" work. being ultra responsive, pushing the ball forward across dozens of different fronts.

Peter runs multiple companies. Beyond Segment, he's raised $100m+ for Charm Industrial, and just recently closed a Series A for Revoy. He's incredibly organized and efficient with his time.

And yet, Peter mentioned that ~95% of the work he does falls into bucket 2. It's hopping on calls. Nudging people. Blocking and tackling.

To be clear, Peter says his companies would be dead-in-the-water today without an IQ 180 technical mind solving the deep problems. Just that most of his work falls in bucket 2.

I think demand for "frontier-level" models is going to keep compounding. Especially for fields that require novel breakthroughs or discovery (engineering, hard science, model training).

But I also think the demand for "fast/cheap/good-enough" models is just about to take off.

Think of the people you interact with on a daily basis: coworkers, vendors, and customers. Nine times out of ten, you want someone who is super responsive, and just handles things for you. Most of the "human tokens" at companies today are spent this way — hiring skews heavily toward the fast/cheap/good-enough archetype.

There's a lot of work that needs to happen to make fast/cheap/good-enough models a reality for business. New harnesses, prompt injection safety, roles, and permissions. But I'm confident we'll figure that out.

If you're also experimenting with making small models useful, please drop me a line.

Footnotes

1. Amazon and Netflix are the notable exceptions
2. Peter is also being modest here. He's sharp as a tack.

Mark Zuckerberg recently indirectly took aim at rival Anthropic in an essay, saying that leading AI labs were trying to consolidate power while painting the future as filled with doom. The 6,500-word essay claims the balance of power will favor larger institutions over individuals if those labs lead. Meta is a heavy user of Anthropic's AI products. It is internally projected to spend as much as $10 billion annually on Anthropic's services, making it one of the AI lab's largest customers.

OpenRouter allows Stripe to be the bank that every AI company is currently forced to build for itself.

Grafana 13.2 expands shared saved queries with search, RBAC, command palette access, and Terraform provisioning, while the new View panel sidebar improves exploration without edit permissions. Additional updates include Git Sync enhancements, a refreshed homepage, secure cloud data source authentication, improved variable editing, and dashboard organization.

AI is making code abundant and shifting software development's central constraint from implementation to trust. As agents increasingly handle the lifecycle, enterprises need durable architecture for context, verification, governance, evidence, and organizational ownership across people, agents, and models.

Presentation startup Gamma has acquired Accel-backed design startup Lica in a bid to build out its own design research lab, TechCrunch has learned exclusively. Lica’s co-founders will lead the effort.

Lica started off as an app that could turn screenshots and recordings into project presentations and videos. Founded by Priyaa Kalyanaraman and Purvanshi Mehta in 2023, the company a year later raised $4 million from investors including Accel, South Park Commons, and Village Global. Over the past couple of years, it has focused on working with e-commerce sites to create marketing videos that adhere to brand guidelines.

Kalyanaraman said Lica wanted to build AI models around communication methods, including video and design, and the founders were familiar with Gamma’s CEO, Grant Lee, as both companies are backed by Accel and South Park Commons. Those conversations eventually led to the acquisition.

“We have a very similar North Star vision of wanting to make visual communication easy. We focused a lot on frontier research and making sure that became accessible to users. But that’s just one part of the solution. Gamma had focused a lot on building distribution, taking the application to 100 million-plus users. Given how fast things are moving, we decided to join forces,” Kalyanaraman told TechCrunch.

Gamma will continue to focus on helping people build presentations, but like other design companies, it’s also started offering image generation. The company says it now wants to explore different formats of communication with the new research division.

“Presentations for us are today a core use case, but how do we expand what presentations look and feel like, and how do people engage with them in the future? What visuals are possible? How interactive are they? These [questions] all require us to think deeply about the core responsibility of communication. This is why we’re excited to invest in this area, because we don’t think anybody else is thinking at that level,” Lee said.

Lica’s Mehta said the new research division would work on customizing the presentation and communication styles of AI models for different audiences and tastes.

“The point of the presentation is to communicate a large goal. We are thinking about building a platform where these two goals can be personalized to such an extent that you just get a really good output, and then you’re able to edit for different audiences,” she said.

Gamma didn’t reveal what products it plans to build as a result of this acquisition, but it did say that it wants to create fluid, multimodal presentations and other communication modalities.

AI presentation startups have received tons of investor attention and money over the last few years. Prezent raised $50 million in two rounds; Gamma raised $68 million at a $2.1 billion valuation; and Presentations.ai was also backed by Accel. We’re even starting to see some consolidation in this category: Earlier this month, OpenAI acquired NextSlide.

AI is collapsing the traditional boundaries between product design, product management, and engineering, making the old handoff-based workflow increasingly obsolete. As AI tools can now turn ideas, requirements, and design systems into working prototypes and code, designers are being pushed toward a broader role the author calls a product architect—someone who combines product thinking, systems design, technical understanding, and creative judgment. Rather than focusing on screens, specs, or deliverables, the most valuable skill becomes deciding what should be built, how it should work, and whether it solves a real problem. This can be an opportunity rather than a threat: design isn't disappearing, but evolving into a more strategic, multidisciplinary role where taste, judgment, and systems thinking matter more than execution.

The Creativity We Outsource — AI, Creativity and the Lost Value of Doing Things Yourself

I ask myself ... What happens to our ability to create when we no longer go through the process of creating things ourselves? Well ... I’m 60 years old. I belong to a generation that learned to make things ourselves — slowly, sometimes awkwardly, and with a surprising number of mistakes. And I do have my perspective in today’s discussion about artificial intelligence. We didn’t have generative AI that could produce a design, a piece of text, an image, or an idea in seconds. We had to start — Then fail — Then start again.

Creativity Takes Time

The British comedian and writer John Cleese has spent a lot of time thinking about creativity. One of his key ideas is that creative people need an open, playful space where they can experiment. Playing doesn’t mean being unproductive. It means trying things without immediately knowing whether they will work. Allowing an absurd idea. Following a bad idea. Trying something that probably won’t work. And SURPRISE! ... sometimes discovering that this is exactly where something interesting begins.

Cleese describes this as an open mode of thinking. It is different from the closed, goal-oriented mode we use when we need to make decisions and get things done. We need both. But if we move into solution mode too quickly, we may lose something important: the time to explore. And this is where AI becomes really interesting. AI is incredibly good at giving us a solution before we have had enough time to explore the problem ourselves.

What Photoshop Already Changed

I remember how digital tools changed design. Photoshop was a revolution. And of course, a great one. Suddenly, we could correct things, move them, copy them, undo them, and change them over and over again. Something that used to be destroyed could simply be restored. But even back then ... at some point I started wondering: What are we no longer learning because we no longer have to do it ourselves? When you draw something by hand, you learn to see differently. When you have to start a design over and over again, you develop a different understanding of form, proportion, and decision-making. When something doesn’t work and you can’t simply hit “Undo,” you have to figure out why it doesn’t work. The detour becomes part of the learning. Photoshop didn’t make that process disappear. But it started to remove some of the friction. Generative AI takes this much further.

And Now Comes AI

Photoshop, Figma u name it ... helped us design faster. AI can now take over parts of the design process itself.

• It writes text.
• It creates images.
• It develops concepts.
• It generates variations.
• It formulates ideas.

And it can even explain why those ideas are supposedly good. That is fascinating. And scary. And fascinating.

But it is also why I think we should be careful when we talk about AI mainly in terms of productivity. I think ... The real question is not: “How much work can AI take away from us?” It is: “What work should we perhaps not give away at all?”

The Value of the Detour

A young designer who gets a task today and can generate ten variations with AI within seconds clearly has a huge advantage. But what about the hours earlier generations might have needed to develop those ten variations themselves? Those hours were not simply working hours.

• They were learning.
• They were practice.
• They were observation.
• They were frustration.
• They were accidents.
• They were mistakes.

And I keep saying it — because it is true: We often learn more from our mistakes than from our successes. So what happens if we remove the learning, the practice, the observation, the frustration, the unexpected discoveries, and the mistakes? Maybe we are also removing one of the most important parts of the creative of the learning process. We should not confuse working time with learning time.

That distinction becomes especially important for children, teenagers, students — and young designers. Imagine a student who is asked to write an essay and ChatGPT produces an amazingly good text within seconds. The result may be much better than what the student could have written alone.

• But did the student learn how to develop an idea?
• Did they learn how to build a structure?
• Did they learn how to write a bad first paragraph and then make it better?
• Did they learn how to develop their own voice?
• Or did they simply learn how to give the machine the right instructions?

Those are very different skills.

The Dangerous Moment Is Not the Final Result

Maybe this is one of the biggest challenges for education and design. We usually judge the result.

• A beautiful image.
• A good piece of writing.
• A convincing presentation.
• A working website.

But creativity doesn’t only happen in the final result. It happens on the way there. And we are seeing less and less of that journey. If a student uses AI to create an excellent essay, you may not be able to tell from the finished document that some of the actual thinking never happened. The result can be better than the learning process. And that is where I see the danger.

I’m Not Saying We Should Ban AI - not at all, Ai is fascinating and I love

I’m not arguing that we should ban AI from schools, universities, or design studios. That would probably be just as unrealistic as trying to remove Photoshop from design. AI is here to stay. And it will be an incredibly powerful tool. But perhaps we need new rules for how we use it. Not everything that can be automated should be automated.

• Maybe a design student should first develop an idea without AI.
• Maybe a student should write a text themselves before asking a machine to improve it.
• Maybe a sketch should first be drawn by hand.
• Maybe there should be assignments where mistakes are not only allowed, but actually encouraged.

Not because the old methods are better. But because through these methods, we learn something a machine cannot teach us ... how to come up with an idea ourselves.

For me, this is also a UX question. We constantly talk about User Experience in technology. But maybe we need to start thinking about something else: Learning Experience. When we design products that make more and more decisions for us, we should ask ourselves what experiences the user no longer gets to have. A system that takes a young designer straight to a perfect result may offer an excellent User Experience. But it might provide a poor Learning Experience. Maybe we need to separate these two things much more carefully in the future. Because a system can make our lives easier while also making us less capable. And perhaps this is one of the most interesting challenges for UX: How do we design AI systems that help people without taking away the experiences through which people become better at what they do?

The Most Important Job of AI

John Cleese’s ideas about creativity lead to a surprisingly relevant question. If creativity needs space to play, maybe we will need to deliberately create spaces where AI doesn’t immediately step in.

Spaces for bad ideas. For detours. For boredom. For making things by hand. For trial and error. For the five minutes when you simply don’t have an answer yet.

Maybe we even need to teach young people again that it is perfectly okay not to be good at something right away. Because that is exactly where learning begins. AI can take an incredible amount of work off our shoulders. It can inspire us. It can challenge us. It can expand what we are capable of.

But there is one thing we should not hand over too easily: the journey that makes us creative ourselves. Because maybe the greatest value of a bad first idea is not the idea itself. Maybe it is simply the fact that we came up with it ourselves.

And maybe this is where Monty Python can help us one last time.

John Cleese once said, “If you want to be creative, allow yourself to be stupid.” Maybe that is exactly what we need to protect in the age of AI: the freedom to be stupid for a while.

To have a bad idea. To make something that doesn’t work. To ask a completely ridiculous question.

Before asking AI to give us the perfect answer. Because sometimes the stupid idea is where the interesting one begins. And if nothing else works: Nobody expects the Spanish Inquisition! 😄

So maybe the future of creativity needs a little less perfection. A little more playing. A few more bad ideas. And occasionally, a completely pointless five-minute detour. John Cleese would probably approve. Or perhaps he would simply say: “And now for something completely different.” And honestly — that might be exactly what creativity needs. 😄

The real value of AI now comes from integrating it into core business operations rather than running isolated experiments. Design teams benefit most when AI supports entire workflows from concept creation to production. AI's biggest near-term impact is augmenting human work rather than replacing it, making skills like judgment, creativity, and critical thinking even more important. Automation is reducing traditional entry-level design tasks, creating a risk to the future talent pipeline unless companies redesign junior roles around AI-assisted workflows while also addressing growing legal and copyright concerns around AI-generated work.

When I heard about a visibly low-budget cartoon in China becoming a box office smash, it immediately brought me back to my childhood in 1976, watching the Sex Pistols on TV snarling "Pretty Vacant", with the emphasis, shockingly, on the last syllable (in their Cockney accent, it sounded like another word entirely). If you can't see the connection, then let me explain. It does make sense, honestly…

Short history lesson: In the first part of the 1970s, new technology meant rock music had become highly produced, which meant it sounded beautifully clean and pure. Unfortunately, it also started to sound a little boring and formulaic, straying very far from its wild and controversial rock 'n' roll roots.

Well, every action has a reaction, as the saying goes. And in the late 1970s, that reaction arrived with a bang in the form of punk rock bands like The Sex Pistols, The Damned and The Clash. Yes, their outfits, lyrics, music and attitude might seem tame, almost cosy now. But back then people – especially my parents – were seriously shocked and offended. Which just made us kids love them more.

50 years on, I've long suspected a similar dynamic will play out with AI. In both cases, new technology has taken over the production of creative works (although with AI, it's not just music but a wide range of cultural products, from art to visual design and now even filmmaking.) So where will the reaction come from? Well, it's probably going to come from some unexpected places, and this Chinese movie looks like being one of them.

What's going on?

For the uninitiated, Niu Lai is an animated film made on an almost zero budget, almost entirely by a mother and son using second-hand computers. I'm not being mean, but the animation is... not high quality. Yet in their native China, it's been an absolute smash. In a world where a $300 million blockbuster like Supergirl struggles to get anyone interested, audiences are flocking to see jerky movements, crude 3D models, weird sound effects, and a plot that doesn't make a lot of sense. Why? Because, the consensus seems to say, it was so obviously, painfully, made by human hands; something that's been long missing from the kind of geometrically-precise CG animation that dominates today.

The story of how it was made reads like the plot of a movie itself. Xin Yumeng is a former interior designer with no film training, based in Dalian, a coastal city in Liaoning province, north-east China. He spent five years building Niu Lai in his spare time, handling every job himself, from modelling and animation to lighting and editing.

His mum, Sun Lifang, taught herself screenwriting and voiced the female characters. The film launched with no trailer, no publicity and a single ink-wash poster. It premiered in 245 cinemas on 5 August and took just 7,169 yuan (around $1000 / £786) in its first nine days.

Then, social media noticed it. People started sharing clips online, initially to take the mickey out of the low-rent animation. But then, as sometimes happens, "so bad" turned to "so bad it's good". People started to become fascinated with this plucky, out-of-the-ordinary movie.

Its cumulative box office has since grossed over ¥25.67 million (about $160K / £120K), putting it among the top 10 highest-grossing animated films in China this year; a staggering return on its minimal investment. It was such a hit that cinemas without official posters got their staff to draw their own by hand.

Reporting on this story last week, fellow Creative Bloq writer Joe Foley took a negative view of this whole phenomenon. But I'm afraid I have to differ. To me, it all sounds like a lot of fun. And I'd rather watch something fun than something technically perfect, but soulless and empty.

More broadly, many of the naysayers here sound eerily like the old men who once dismissed my beloved punk bands as a lot of unlistenable noise. (Well, to be fair a lot of them were. A few of the doom metal bands I saw at Bloodstock last weekend were pretty unlistenable to my ears too. But a lot of people love them, and who am I to say they're wrong?)

More broadly, the love for Niu Lai in China chimes neatly with everything I'm hearing from everyday people in my own country: they're sick of AI visuals, AI music and AI videos, and they're desperately looking for authentic alternatives. It seems that in 2026 the less digital polish there is on things, the more people are likely to like it.

That's another cultural revolution I can get behind, and this time around, there's no need for spitting or swear words. Now, where did I put that Ramones album?

State attorneys general are arguing in a federal trial that Meta's endless feed was engineered to break human willpower.

Codex has added 'persistent' to the reasoning-effort protocol and TypeScript SDK types. When a user targets a custom Responses-compatible provider whose model-defined effort is literally persistent, this now deserializes to the new Persistent variant and is unconditionally rewritten to disabled. Previously, it remained Custom("persistent") and was forwarded unchanged. Existing configurations and resumed sessions can therefore send a different value and change behavior or be rejected, so this alias should be limited to providers that define the translation.

OpenAI and Anthropic are seeing unprecedented growth, with revenues surpassing $100 billion combined, fueled by rapid adoption and technical advancements. This swift rise challenges expectations, as typical tech growth slows at such scales, potentially reshaping the economy if sustained.

Coding agents' configurations have a half-life. Models improve, harnesses add capabilities, codebases change, and tasks evolve. Research shows that people are getting inconsistent value from personalized skills. Run Claude's /doctor every few weeks, review memory separately, and ask each instruction to earn its place again.

Taste is about making choices, and whether those choices are good depends on who they are made for.

DigitalOcean has launched v5 Droplets, a new generation of virtual machines built on 5th Gen AMD EPYC processors.

The smartphone industry appears to be coalescing around the book-style foldable design, where a phone unfolds into a small tablet, rather than the flip-phone format that folds a standard phone in half. Samsung's latest Galaxy Z Fold, Apple's rumored iPhone Ultra, and leaked images of Xiaomi's upcoming 18 Fold all point to growing momentum behind larger-screen foldables, suggesting that this form factor is emerging as the preferred direction for premium folding phones.

A free stock photography collection led by disabled creatives to help marketers, advertisers, and creative teams tell more authentic stories about disability.

Every top AI image, video, and audio model on one infinite canvas. Storyboard, iterate, and publish templates — or open a focused generator for a single shot.

Creative confusion stalls work, while clarity comes from breaking the process into phases and knowing which one is current.